TCP Connection Setup

TCP connection setup requires a three-way handshake between the two parties. Initially, one party is passively waiting for the establishment of a connection. This passive party is said to be “listening.” The passive party is typically a server. The other party actively opens the TCP connection by sending the first IP datagram. The active party is typically a client. The definition of client and server is separate from active and passive parties during the setup phase. This discussion refers to the parties as client and server merely to be more suggestive of the typical roles they will play later.

The client starts things off by sending a TCP header with the SYN flag set. SYN stands for “synchronize” and refers to the synchronization of initial sequence numbers. The TCP protocol assigns each data byte sent on a connection its own sequence number. Every TCP header contains a sequence number field corresponding to the sequence number in the first data byte of the field. Initial sequence numbers should be random rather than merely arbitrary. Randomness of initial sequence number is important for handling the situation when a connection is established, the machine on one side crashes, and then attempts to reestablish a connection. The other machine needs to be able to detect wild out-of-range sequence and acknowledgment numbers to close its side of the connection to the program that is no longer running. TCP only sets the SYN flag when the connection is started.

The server replies to the SYN header with a header containing both a SYN and an ACK flag set. ACK stands for “acknowledgment.” The SYN lets the client know its initial sequence number—TCP connections are bi-directional. The ACK flag lets the client know that it received the initial sequence number. Whenever the acknowledgment number field is valid, corresponding to the sequence number of the next data byte expected, the TCP sets ACK flag.

To complete the connection, the client responds back to the server with a TCP header that has the ACK flag set. The acknowledgment lets the server know that it is now ready to begin receiving data. Understanding the sequence of events with SYN and ACK flags during the establishment of a connection is also important when configuring firewalls (see Chapter 7, “How to Build a Firewall,” for more information).

---

Note:  See CERT Advisory CA-96.21 about a denial of service attack using a flood of TCP SYN packets to overload a host with “half-open” TCP sessions. Internet Service Providers, their customers, and vendors should pay particular attention to the guidelines therein.

---

TCP Data Exchange

During normal TCP data exchange, one party will send one or more TCP/IP datagrams. The other party will occasionally send back a TCP/IP datagram with the TCP header having the ACK flag set to let the sender know that the data arrived. During establishment of the connection both parties also inform the other of how much room they have in their receive buffers. TCP transmits the amount of available room in the window field of the TCP header in each datagram sent to inform the sender how much more data may be sent before the receive buffer fills. As the program on the receiving side empties the receive buffer, the number in the window field increases. The acknowledgment number specifies the lowest sequence number of a data byte that it expects to receive. The acknowledgment number plus the number in the window field specifies the highest sequence number of a data byte that will be placed in the input buffer when received.

Occasionally, IP datagrams will arrive out of order. When a datagram arrives earlier than expected, the early datagram goes into the receiver’s input buffer but the receiver does not immediately acknowledge it. When the expected datagram arrives, the receiver may acknowledge both sets of TCP data at once. At this point, however, the receiving program will be able to read both sets of data without waiting for any more action from the sender.

Forged TCP/IP Datagrams

To successfully forge a datagram that will be accepted as part of an existing connection, an attacker only needs to estimate the sequence number to be assigned to the next data byte to be sent by the legitimate sender. Consider the three cases of being exact, being a bit too low with the estimate, and being a bit too high with the estimate.

If the attacker knows or successfully guesses the exact value of the next sequence number of the next byte being sent, the attacker can forge a TCP/IP datagram containing data that will be placed in the receiver’s input buffer in the next available position. If the forged datagram arrives after the legitimate datagram, the receiver may completely discard the forged datagram if it contains less data than the legitimate one. However, if the forged datagram contains more data, the receiver will discard only the first part. The receiver will place into its input buffer the part of the forged datagram with data bytes having larger sequence numbers than those received in the earlier legitimate datagram.

On the other hand, if the forged datagram arrives before the legitimate datagram, the legitimate datagram will be discarded by the receiver (at least partially).

If the attacker’s guess of the sequence number is a bit too low, it will definitely not get the first part of the data in the forged TCP/IP datagram placed in the receiver’s input buffer. However, if the forged datagram contains enough data, the receiver may place the last part of the forged data in its input buffer.

If the attacker’s guess of the sequence number is a bit too high, the receiver will consider it to be data that simply arrived out of order and put it into its input buffer. Some of the data bytes at the end of the forged datagram may have sequence numbers that do not fit in the current window, so the receiver will discard these. Later, the legitimate datagram arrives to fill in the gap between the next expected sequence number and the sequence number of the first forged data byte. Then, the whole forged datagram is available to the receiving program.