Analysis of tcp, UDP and IP Protocols


In the early days of interconnection, NCP protocol was used for interconnection between hosts. The protocol itself has many defects, such as: it can not interconnect different hosts, different operating systems, and no error correction function. In order to improve this disadvantage, Daniel came up with the TCP/IP protocol. Now almost all operating systems implement TCP/IP stack.

TCP/IP stack is divided into four layers: application layer, transport layer, network layer and data link layer. Each layer has corresponding protocols, as shown below.

The so-called protocol is a format for data transmission between two parties. There are many protocols used throughout the network. Fortunately, each protocol has an RFC document. In this paper, we only analyze IP, TCP and UDP protocol headers.

Let’s first look at the format of an Ethernet packet in the network:

In Linux operating system, when we want to send data, we only need to prepare the data in the upper layer and submit it it to the kernel protocol stack, which automatically adds the corresponding protocol header. Now let’s look at the specific content of the protocol header added to each layer.

I. TCP Protocol

TCP protocol is a connection-oriented transport layer protocol that guarantees high reliability (no data loss, no data disorder, no data error, no data duplicate arrival).

1. TCP Header Analysis

Firstly, the format of TCP header and the meaning of each field are analyzed.

(1) Port number [16bit]

We know that the network implements inter-process communication between different hosts. In an operating system, there are many processes. When data arrives, which process should be submitted for processing? This requires the use of end-to-end slogans. In the TCP header, the active port number (Source Port) and the target port number (Destination Port). The source port number identifies the process of the sending host and the target port number identifies the process of the receiving host.

(2) serial number [32bit]

The serial numbers are divided into Sequence Number and Acknowledgment Number.

Send Sequence Number: Used to identify the data byte stream sent from the TCP source to the TCP destination, which represents the sequence number of the first data byte in this message segment. If the byte stream is viewed as a one-way flow between two applications, TCP counts each byte with a sequence number. The serial number is an unsigned number of 32 bits, and it begins at 0 when it reaches 2 32-1. When a new connection is established, the SYN flag changes to 1, and the sequence number field contains the initial sequence number ISN (Initial Sequence Number) of the connection selected by the host.

Confirmation Sequence Number: Contains the next sequence number expected to be received by the sending end of the confirmation. Therefore, the confirmation sequence number should be the last successful data byte sequence number plus 1. Only when the ACK flag is 1 can the serial number field be valid. TCP provides full duplex service for application layer, which means that data can be transmitted independently in two directions. Therefore, each end of the connection must maintain the serial number of data transmitted in each direction.

(3) Migration [4bit]

The offset here actually refers to the length of the TCP header, which is used to indicate the number of 32 bit words in the TCP header, through which it can know where the user data of a TCP packet begins. This field takes up 4 bits, if the value of 4 bits is 0101, then the TCP first length is 5 * 4 = 20 bytes. So the maximum TCP header is 15 * 4 = 60 bytes. However, there are no optional fields, and the normal length is 20 bytes.

(4)Reserved [6bit]

It is not currently in use, and its values are all 0.

(5) Sign [6bit]

There are six flag bits in the TCP header. Many of them can be set to 1 at the same time.

URG. Urgent pointer is valid

ACK) Confirm that the serial number is valid

PSH directs the recipient to hand over the message segment to the application layer as soon as possible without waiting for the buffer to fill up
RST generally means disconnecting a connection

For example, a TCP client initiates a connection to a server without a listening port. Wirshark grabs the following packages

You can see that host: initiates a connection request to host:, but host: is not on the server side listening for the corresponding port, at this time.

Host: TCP packets with one RST bit are disconnected.

SYN Synchronization Number is used to initiate a connection

FIN) The sender completes the sending task (i.e., disconnects)

(6) Window size [16bit]

The size of the window indicates the maximum number of bytes that the source method can accept.

(7) Checksum [16bit]

Check and cover the entire TCP segment: TCP header and TCP data. This is a mandatory field that must be computed and stored by the originator and verified by the recipient.

(8) Emergency pointer [16bit]

The emergency pointer is valid only when the URG flag is set to 1. The emergency pointer is a positive offset and adds the value in the ordinal field to indicate the ordinal number of the last byte of the emergency data. The emergency mode of TCP is a way for the sender to send emergency data to the other end.

(9) TCP option

It’s optional. When we grab the bag in the back, we’re looking at it.

2. Emphasis Detailed

(1) Three handshakes to establish connections

A. The requester (usually referred to as the client) sends a SYN segment indicating the port of the server the client intends to connect to, as well as the initial serial number (ISN, in this case 141553521). This SYN segment is message segment 1.
B. The server sends back the SYN message segment (message segment 2) containing the initial serial number of the server as a response. At the same time, the confirmation number is set to ISN plus 1 to confirm the customer’s SYN message segment. A SYN will occupy a serial number
C. Clients must set the confirmation number to the server’s ISN plus 1 to confirm the server’s SYN message segment (message segment 3)
These three message segments complete the establishment of the connection. This process is also known as the three-way handshake.

Use wirshark to grab the package as follows:

You can see that three handshakes determine the serial number of the packets between the two parties, the maximum size of the received data (window) and the MSS (Maximum Segment Size).
MSS = MTU – IP header – TCP header, MTU represents the largest transmission unit, we will talk about when analyzing IP header, it is generally 1500 bytes. Both IP headers and TCP headers have 20 bytes with optional options. In this case, MSS = 1500 – 20 – 20 = 1460.

MSS limits the size of data carried by TCP packets. It means that when the application layer submits data to the transport layer for transmission through the TCP protocol, if the application layer data > MSS must be segmented, divided into several segments, one by one.

For example, the application layer submits 4096 bytes of data to the transport layer at one time. At this time, the effect of grabbing packets through wirshark is as follows:

The first three times are the process of shaking hands three times, and the last three times are the process of transmitting data. Because the data size is 4096 bytes, three times are used to transfer (1448 + 1448 + 1200). Careful people ask why the maximum data size is not 1460 bytes per transmission, because TCP here carries optional TCP header length = 20 + 12 (optional size) = 32 bytes. The maximum data that can be transmitted in this way is 1500 – 20 – 32 = 1448 bytes.

(2) Four waves to disconnect

A. Nowadays, the network communication is based on socket. When the client closes its socket, the kernel protocol stack sends a FIN-bit package to the server automatically, requesting disconnection. We call the party that initiates the disconnection request the active disconnector.
B. When the server receives the FIN disconnection request from the client, the kernel protocol stack immediately sends an ACK package in response to the request from the client.
C. The server shuts down its socket after running for a period of time. At this point, the kernel protocol stack sends a FIN-bit package to the client, requesting disconnection.
D. When the client receives the FIN disconnection request from the server, it sends an ACK to reply that it has received the request from the server.

Wirshar’s grabbing package was used to analyze the following:

(3) Guarantee of TCP Reliability

TCP uses a technology called “positive acknowledge with delivery with retransmit function” as the basis for providing reliable data transmission services. This technology requires the receiver to send back the confirmation information ACK to the source station after receiving the data. The sender keeps a record of each packet sent and waits for confirmation before sending the next packet. The sender also starts a timer while sending out the packet, and resends the packet just sent when the timer expires and the confirmation message has not arrived. Figure 3-5 shows the transmission of data by affirmative acknowledgment protocol with retransmit function, and Figure 3-6 shows the time-out and retransmit caused by packet loss. In order to avoid late confirmation and duplicate confirmation due to network delay, the protocol stipulates that a packet number should be added to the confirmation information so that the receiver can correctly associate the packet with the confirmation.

As can be seen from Figure 3-5, although the network has the capability of two-way communication at the same time, the simple affirmation protocol wastes a lot of valuable network bandwidth because the transmission of the next packet must be delayed before receiving the confirmation information of the previous packet. To this end, TCP uses sliding window mechanism to improve network throughput, while resolving end-to-end traffic control.

(4) Sliding window technology

Sliding window technology is a more complex variant of a simple affirmative acknowledgment mechanism with retransmissions, which allows senders to send multiple packets before waiting for an acknowledgment message. As shown in Figure 3-7, the sender sends a packet sequence. The sliding window protocol places a fixed-length window in the packet sequence, and then sends out all the packets in the window. When the sender receives the confirmation information for the first packet in the window, it can slide backward and send the next packet. With the arrival of confirmation, the window is also continuous. Slide backwards.

UDP Protocol

UDP protocol is also a transport layer protocol. It is connectionless and does not guarantee reliable transport layer protocol. Its protocol header is relatively simple, as follows:

The port number here does not explain, and TCP port number is the same meaning.

Length takes two bytes to identify the length of the UDP header. Checksum: Checksum, which contains UDP headers and data sections.

III. IP Protocol

I P is the core protocol in the T C P/I P protocol family. All T C P, U D P, I C M P and I G M P data are transmitted in I P datagram format. Its characteristics are as follows:
Unreliable (u n r e l i a B e) means that it does not guarantee that the I P datagram can reach its destination successfully. I P only provides the best transmission service. If an error occurs, such as a router running out of buffer temporarily, I P has a simple error handling algorithm: discard the datagram, and then send the I C M P message to the source. Reliability of any requirement must be provided by the upper level (e.g. T C P).

The term c o n n e c t i o n l e s s (c o n e c t i o n e s) means that I P does n o t maintain any status information about subsequent datagrams. The processing of each datagram is independent of each other. This also shows that IP datagrams can be received out of order. If a source sends two consecutive datagrams (first A, then B) to the same sink, each datagram is routed independently and may choose different routes, so B may arrive before A arrives.

1. IP Header Format

(1) Version occupies 4 bits, referring to the version of IP protocol. The versions of IP protocols used by both sides of communication must be consistent. At present, the widely used version number of IP protocol is 4 (IPv4). With regard to IPv6, it is still in the draft stage.  

(2) The first length is 4 bits, and the maximum decimal value is 15. Note that the unit of number represented by this field is 32-bit word length (one 32-bit word length is 4 bytes). Therefore, when the IP first length is 1111 (that is, 15 decimal), the first length is 60 bytes. When the first length of an IP packet is not an integer multiple of 4 bytes, it must be filled with the last filling field. Therefore, the data part always starts at an integer multiple of 4 bytes, which makes it more convenient to implement IP protocol. The disadvantage of limiting the first length to 60 bytes is that sometimes it may not be enough. But in doing so, users are expected to minimize overhead. The most commonly used first length is 20 bytes (that is, the first length is 0101), at which point no option is used.

(3) Differentiated services account for 8 places to obtain better services. This field is called service type in the old standard, but it has never been used in practice. In 1998, IETF renamed this field Differentiated Services. This field only works when differentiated services are used.  

(4) The total length refers to the length of the sum of the head and data in bytes. The total length field is 16 bits, so the maximum length of the datagram is 216-1 = 65535 bytes.

Each data link layer under the IP layer has its own frame format, including the maximum length of data fields in the frame format, which is called Maximum Transfer Unit. When a datagram is encapsulated as a frame in the link layer, the total length of the datagram (that is, the header plus the data part) must not exceed the MTU value of the data link layer below.

(5) Identification accounted for 16 places. IP software maintains a counter in memory, and every time a datagram is generated, the counter is added 1, and this value is assigned to the identification field. But this “ID” is not a serial number, because IP is a connectionless service and there is no problem of orderly receiving datagrams. When a datagram has to be fragmented because its length exceeds the MTU of the network, the value of this identification field is copied to the identification field of all datagrams. The same value of the identifying field enables the fragmented datagrams to be reloaded correctly into the original datagrams.  

(6) Flag occupies 3 places, but at present only 2 places are meaningful.

The lowest bit in the flag field is MF (More Fragment). MF = 1 means that there are “fragmented” datagrams behind. MF = 0 means this is the last of several datagrams

The middle of the marker field is denoted as DF (Don’t Fragment), which means “cannot be fragmented”. Fragmentation is allowed only when DF = 0.  

(7) The slice migration accounts for 13 places. The slice offset indicates the relative position of a slice in the original group after the longer group is sliced. That is to say, where does the slice start relative to the starting point of the user data field? Slice migration takes 8 bytes as the unit of migration. That is to say, the length of each fragment must be an integral multiple of 8 bytes (64 bits).  

(8) Time to Live is the most commonly used abbreviation in the field of time to live, which indicates the lifetime of a data report in the network. This field is set by the source point from which the datagram is sent. Its purpose is to prevent the undeliverable datagrams from circulating indefinitely in the Internet, thus consuming network resources in vain. The original design was to use seconds as TTL units. Every time a router passes by, TTL is subtracted from the time consumed by the router by the datagram. If the data packet consumes less than 1 second on the router, the TTL value is reduced by 1. When the TTL value is 0, the report is discarded.

(9) The protocol occupies 8 bits. The protocol field indicates which protocol is used for the data carried by the datagram, so that the IP layer of the destination host knows which process the data part should be handed over to.  

(10) The first test accounted for 16 places. This field only checks the header of the datagram, but does not include the data section. This is because every time a datagram passes through a router, the router recalculates the header checksum (some fields, such as lifetime, flag, chip offset, etc., may change). Not checking the data part can reduce the workload of calculation.

(11) Source IP addresses account for 32 bits.  

(12) The destination IP address occupies 32 bits.

2. Piecewise Interpretation

Fragmentation refers to the need to transmit data larger than the maximum transmission unit (MTU), it needs to be divided into multiple packets, and then sent to each other one by one. When we talk about TCP, when we talk about MSS, many people can’t distinguish them. From the following figure, I think we can distinguish them completely.

Personally, if the data is transmitted through TCP protocol, when it comes to IP layer, there is no need for fragmentation. Partitioning is required only when large data is transmitted through UDP protocol.
For example, 10240 bytes of data are transmitted by UDP protocol

As you can see, when the data is submitted to the network layer, it is fragmented because the data exceeds the maximum transmission unit. Divide into multiple packets and send each other through IP protocol. The largest byte per packet is MTU – IP header = 1500 – 20 = 1480.

IV. Ethernet Header

The protocol used in source MAC Address | destination MAC Address | is composed of three parts.

So in ethernet, there are several formats of data packets:

ARP protocol obtains the corresponding MAC address through IP address, which is called address resolution protocol.

RARP protocol obtains the corresponding IP address through MAC address, which is called reverse address resolution protocol.

The above is the whole content of this article. I hope it will be helpful to everyone’s study, and I hope you will support developpaer more.