HackFest Series: Tcp Treason Uncloaked

Wed Mar 18 12:44:22 MST 2009

A common kernel level TCP whine seen in dmesg is:

TCP: Treason uncloaked! Peer 38.108.180.106:14059/80 shrinks window 1536549741:1536551189. Repaired.

Treason?  Give me liberty or give me death?  
Treason?  As far as I know my peers are all loyal?
Shrinks window?  This is not a Windows box? 

What could it mean?

TCP Treason messages indicate a failure of the TCP header in a 3 way handshake 
ACK in TCP/IP protocol stack and can be either a bug or an exploit.

Specifically, they indicate that someone or something has shrunk the WINDOW size, a header parameter in the network header, or stack.

Good news!  Understanding and repairing TCP kernel level issues, equates to the cornerstone of Linux Systems Administration called Performance Tuning!

TCP TREASON is due to:
1) Broken (old) or unpatched 2.6 kernel

An "old" (read "broken") TCP stack on the receiver.  The original RFC (793 IIRC) allowed this behavior -- RFC 1122 corrected the issue by prohibiting the RECEIVER from doing this.
See 2.6 kernel patch analysis below.

2) Some mobile HTTP clients which, in order to get only the first portion of an HTTP document, purposely(!!!) set a small initial window size and don't allow it to move (until/if the user requests more of the document, at which time they send ACK, thus reopening the window).

3) Zero Windows hacks, SSL/DoS attacks; or DoS phishing attacks coordinated with DNS cache poisoning for high traffic sites present as TCP Treason.  I.E. if I can steal 1/10th of your hits I get $n google ads hits or pay to click cashola.  

Understanding Kernel and Windows hacks

A "tar-pit" system is  designed to keep sockets in a bad state and run you out of kernel memory as a DoS.  Many scripts exist - a simple nmap or netcat can trivially be crafted to provide window changes (see insecure.org link below).

The mechanisms provided allow TCP to advertise a large window and to subsequently advertise a much smaller window without having accepted that much data. This exploit called "shrinking the window," interferes with the robustness principle that dictates that TCPs will not shrink the window themselves, but will be prepared for such behavior on the part of other TCPs.

The sending TCP must be prepared to accept from the user and send at least one octet of new data even if the send window is zero.  The sending TCP must regularly retransmit to the receiving TCP even when the window is zero.  Two minutes is recommended for the retransmission interval when the window is zero.  This retransmission is essential to guarantee that when either TCP has a zero window the re-opening of the window will be reliably reported to the other.

When the receiving TCP has a zero window and a segment arrives it must still send an acknowledgment showing its next expected sequence number and current window (zero).4) Kernel level debugging is enabled.  Believe it or not, some administrators simply turn tcp debugging off and ignore these, however the messages are often a good indicator of why a high volume Ecommerce site is dog slow.

Patch for 2.6 kernel

A patch was released that explains the process in the protocol sequence under the Linux 2.6 kernel:
http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=2ad41065d9fe518759b695fc2640cf9c07261dd2

This excerpt explains the SYN/ACK well:

The effect of this bug ranges from harmless "Treason uncloaked messages to hung/aborted TCP connections.  The details of the bug
and fix is as follows.

When snd_wnd is updated, we only update pred_flags if
tcp_fast_path_check succeeds.  When it fails (for example,
when our rcvbuf is used up), we will leave pred_flags with
an out-of-date snd_wnd value.

When the out-of-date pred_flags happens to match the next incoming
packet we will again hit the fast path and use the current snd_wnd
which will be wrong.

In the case of the treason messages, it just happens that the snd_wnd
cached in pred_flags is zero while tp->snd_wnd is non-zero.  Therefore
when a zero-window packet comes in we incorrectly conclude that the
window is non-zero.

In fact if the peer continues to send us zero-window pure ACKs we
will continue making the same mistake.  It's only when the peer
transmits a zero-window packet with data attached that we get a
chance to snap out of it.  This is what triggers the treason
message at the next retransmit timeout.

-end excerpt-

How do we defend against Zero Window and TCP hacks?

1) Kernel level stack tuning (after verifying we don't need the patch).

First we need to fully understand that this is an exploit of the inherent protocol.  

Next we need to enable the BASIC kernel level security features from Linux:

Reference: http://cr.yp.to/syncookies.html

/etc/sysctl.conf

# Enable Spoof protection (reverse-path filter)
net.ipv4.conf.default.rp_filter=1

# Syncookies
net.ipv4.tcp_syncookies=1

#Advanced kernel stack tuning
net.ipv4.tcp_synack_retries = 2
net.ipv4.tcp_abort_on_overflow = 1

Why not just TCP deny all the source IP's?

Spoofing these addresses is trivial.

TCP window scaling and broken "routers"
http://lwn.net/Articles/92727/

Vista is vulnerable also
http://www.tech-recipes.com/rx/1744/vista_tcp_cannot_communicate_primary_dns_server

Outbound connection issues

While not a part of this discussion, can be a difficulty in portal or other distibuted systems:
    echo 0 > /proc/sys/net/ipv4/tcp_default_win_scale 
or by adding to /etc/sysctl.conf  

    net.ipv4.tcp_default_win_scale = 0

or

With kernel 2.6.17.13 or higher, you can also do:

THEIR_IP=1.2.3.4
MY_GATEWAY=5.6.7.8

ip route add $THEIR_IP/32 via $MY_GATEWAY window 65535

which only limits window scaling for that destination without interfering with your other connections.or custom tuning:

decrease the TCP outgoing buffer:

from

net.ipv4.tcp_rmem = 4096 87380 4194304

to this

net.ipv4.tcp_rmem = 4096 87380 207520

How to offend with TCP
http://insecure.org/stf/tcp-dos-attack-explained.html

OpenSSL and TCP Windows
openssl-too-open is a remote exploit for the KEY_ARG overflow in OpenSSL 0.9.6d  and older

See also:  http://www.openssl.org/news/changelog.html

How to offend via Debian Linux with SSH Bruteforce
http://www.astalavista.com/index.php?section=exploits&cmd=details&id=5722

"Women who seek to be equal to men lack ambition".  -Timothy Leary

Obnosis | (503)754-4452
PLUG Linux Security Labs 2nd Saturday Each Month at Noon - 3PM

_________________________________________________________________
Windows Live™ Contacts: Organize your contact list. 
http://windowslive.com/connect/post/marcusatmicrosoft.spaces.live.com-Blog-cns!503D1D86EBB2B53C!2285.entry?ocid=TXT_TAGLM_WL_UGC_Contacts_032009
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.PLUG.phoenix.az.us/pipermail/plug-discuss/attachments/20090318/f16a5bd2/attachment.htm 