Learning notes of Py network programming

Time:2019-11-3

I. exception handling

An exception is a signal that an error occurs when a program is running (an exception will be generated when an error occurs in the program. If the program does not handle it, the exception will be thrown and the operation of the program will be terminated). In Python, the exception triggered by an error is as follows

There are two kinds of mistakes:

1. Syntax error

Syntax error is sb error

2. Logic error

Logical errors include data type errors and value errors, which are caused by their own logical confusion.

 

Different exceptions in Python can be identified by different types (classes and types are unified in Python, i.e. types). An exception identifies an error

Attributeerror attempts to access a tree without an object, such as foo. X, but foo has no attribute x
 Ioerror I / O exception; basically cannot open file
 Importerror failed to import module or package; basically a path or name error
 Indexerror syntax error (subclass of); code not properly aligned
 Indexerror index index is beyond the sequence boundary. For example, when x has only three elements, it attempts to access x [5]
 Keyerror trying to access a key that does not exist in the dictionary
 Keyboardinterrupt Ctrl + C pressed
 NameError uses a variable that has not been assigned to an object
 Syntax error Python code is illegal, code cannot be compiled (I think it's a syntax error, wrong writing)
 Typeerror the type of the incoming object does not match the expected
 Unboundlocalerror attempts to access a local variable that has not been set, basically because there is another global variable with the same name,
 That makes you think you're visiting it
 Valueerror passes in a value that the caller does not expect, even if the value type is correct

 

Therefore, in order to ensure the robustness and fault tolerance of the program, that is, the program will not crash when encountering an error, we need to deal with the exception, and the conditions of error occurrence are divided into two situations

Predictability:

Can be handled with if: prevention before an error occurs

AGE=10
 while True:
     age=input('>>: ').strip()
     If age. Isdigit(): ා only when age is an integer in the form of a string, the following code will not make an error. This condition is predictable
         age=int(age)
         if age == AGE:
             print('you got it')
             break

 

Unpredictable:

When the condition of error occurrence is unpredictable, try… Except: process after error occurrence

#The basic syntax is
try:
    Detected code block
Exception type:
    Once an exception is detected in try, the logic at this location is executed
A case study
try:
    f=open('a.txt')
    g=(line.strip() for line in f)
    print(next(g))
    print(next(g))
    print(next(g))
    print(next(g))
    print(next(g))
except StopIteration:
    f.close()

 

#1 exception class can only be used to handle the specified exception. If the exception is not specified, it cannot be handled.
s1 = 'hello'
try:
    int(s1)
Except indexerror as e: if no exception is caught, the program will directly report an error
    print e

#2 multiple branches
s1 = 'hello'
try:
    int(s1)
except IndexError as e:
    print(e)
except KeyError as e:
    print(e)
except ValueError as e:
    print(e)

#3 universal exception
s1 = 'hello'
try:
    int(s1)
except Exception as e:
    print(e)

#4 multi branch anomaly and universal anomaly
#4.1 if the effect you want is to discard all exceptions or use the same code logic to deal with them, it's enough to have only one exception.
#4.2 if the effect you want is that we need to customize different processing logic for different exceptions, then we need to use multiple branches.

#You can also use an exception after multiple branches
s1 = 'hello'
try:
    int(s1)
except IndexError as e:
    print(e)
except KeyError as e:
    print(e)
except ValueError as e:
    print(e)
except Exception as e:
    print(e)

#6 other abnormal institutions
s1 = 'hello'
try:
    int(s1)
except IndexError as e:
    print(e)
except KeyError as e:
    print(e)
except ValueError as e:
    print(e)
#except Exception as e:
#    print(e)
else:
    Print ('execute me if there is no exception in the code block in 'try')
finally:
    Print ('the module will execute no matter whether it is abnormal or not, usually for cleaning work ')

#7 active trigger exception
try:
    Raise typeerror ('type error ')
except Exception as e:
    print(e)

#8 custom exception
class EgonException(BaseException):
    def __init__(self,msg):
        self.msg=msg
    def __str__(self):
        return self.msg

try:
    Raise egonexception ('wrong type ')
except EgonException as e:
    print(e)

#9 assertion: assert condition
assert 1 == 1
assert 1 == 2

#10 summary try.. except

1: separate error handling from real work
2: the code is easier to organize, clearer, and the complex tasks are easier to implement;
3: there is no doubt that it is safer, and it will not cause the program to crash accidentally due to some minor negligence;

Frequently used anomalies

 

Try… Except should be added only when the condition of the error is unpredictable, otherwise the readability of the code will be poor.

 

2. Socket programming

Skip the network foundation

Note:

Socket is the intermediate software abstraction layer of communication between application layer and TCP / IP protocol family. It is a group of interfaces. In the design mode, socket is actually a facade mode. It hides the complex TCP / IP protocol family behind the socket interface. For users, a group of simple interfaces is all. Let socket organize data to meet the specified protocol. Therefore, we don’t need to understand the TCP / UDP protocol in depth. Socket has been encapsulated for us. We just need to follow the rules of socket to program, and the written program naturally follows the TCP / UDP standard.

Socket workflow:

Socket() module function usage

import socket
socket.socket(socket_family,socket_type,protocal=0)
Socket family can be AF UNIX or AF INET. Socket "type can be sock" stream "or sock" Dgram. Generally, protocol is not filled in, and the default value is 0.

Get TCP / IP socket
tcpSock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

Get UDP / IP socket
udpSock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)

Because there are too many properties in the socket module. We use the 'from module import *' statement here as an exception. By using 'from socket import *', we have brought all the properties in the socket module into our namespace, which can greatly reduce our code.
For example, tcpsock = socket (AF? INET, sock? Stream)

 

Server socket function
S.bind() binding (host, port number) to socket
S.listen() starts TCP listening
S.accept() passively accepts the TCP client's connection, (blocking) waits for the connection to arrive

Client socket function
S.connect() initiatively initializes TCP server connection
The extended version of the s.connect_ex() connect() function returns the error code when an error occurs, rather than throwing an exception

Socket functions for public use
S.recv() receives TCP data
S.send() sends TCP data (when the amount of data to be sent is greater than the remaining space of the end cache, the data is lost and will not be sent out)
S.sendall() sends complete TCP data (in essence, it is a circular call to send. Sendall does not lose data when the amount of data to be sent is greater than the remaining space of its own cache area, and calls send until the end of sending)
S.recvfrom() receives UDP data
S.sendto() send UDP data
S.getpeername() address of the remote connection to the current socket
S.getsockname() address of the current socket
S.getsockopt() returns the parameters of the specified socket
S.setsockopt() sets the parameters of the specified socket
S.close() close socket

Lock oriented socket method
S.setblocking() sets the blocking and non blocking modes of sockets
S.settimeout() sets the timeout for blocking socket operations
S.gettimeout() gets the timeout of blocking socket operation

Function of file oriented socket
File descriptor for s.fileno() socket
S.makefile() creates a file related to the socket

 

#_*_coding:utf-8_*_
import socket
IP ﹐ port = ('127.0.0.1 ', 8081) ﹐ phone card
BUFSIZE=1024
S = socket. Socket (socket. AF? INET, socket. Sock? Stream)
S.bind (IP port) mobile card
S.listen (5) standby


While true: receive link loop, can answer calls continuously
    Conn, addr = s.accept() ා answer the phone
    # print(conn)
    # print(addr)
    Print ('received call from% s'% addr [0])
    While true: ා new communication cycle, continuous communication, sending and receiving messages
        MSG = conn.recv (bufsize) listen to the message

        #If len (MSG) = = 0: break ා if it is not added, then the client being linked suddenly disconnects, and recv will no longer block, and a dead cycle will occur

        print(msg,type(msg))

        Conn.send (MSG. Upper()) (send message, speak)

    Conn.close() (hang up)

S.close() (cell phone off)

server

 

#_*_coding:utf-8_*_
import socket
ip_port=('127.0.0.1',8081)
BUFSIZE=1024
s=socket.socket(socket.AF_INET,socket.SOCK_STREAM)

S. connect? Ex (IP? Port)? Dial the phone

While true: the client can send and receive messages continuously
    msg=input('>>: ').strip()
    if len(msg) == 0:continue
    S.send (MSG. Encode ('utf-8 ')): send message, speak (only byte type can be sent)

    Feedback = s.recv (bufsize) accept the message and listen
    print(feedback.decode('utf-8'))

S.close() (hang up)

client

When restarting the server, you may encounter address already in use, because when restarting the server, the server is still in the time “wait” state of four waves of TCP. There will be a lot of optimization methods of time ﹣ wait state in the case of high concurrency of servers

resolvent:

1. Add a socket configuration and reuse IP and port

phone=socket(AF_INET,SOCK_STREAM)
Phone. Setsockopt (SOL? Socket, so? Reuseaddr, 1) reuse IP and port before bind
phone.bind(('127.0.0.1',8080))

2. Adjust Linux kernel parameters

vi /etc/sysctl.conf

Edit the file and add the following:
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_fin_timeout = 30

Then execute / SBIN / sysctl - P for the parameter to take effect.

Net.ipv4.tcp_syncookies = 1 means syn cookies are enabled. When the syn waiting queue overflows, enable Cookies to handle, which can prevent a small number of syn attacks. The default value is 0, which means it is closed;

Net. IPv4. TCP? TW? Reuse = 1 means reuse is enabled. Allows time-wait sockets to be reused for new TCP connections. The default value is 0, which means it is closed;

Net.ipv4.tcp "TW" recycle = 1 means to enable the quick recycle of time-wait sockets in the TCP connection. The default value is 0, which means to close.

Net.ipv4.tcp'fin'timeout modify the default timeout of the system
UDP based socket

Since UDP is unlinked, no error will be reported on either end of the startup

####################server
import socket
ip_port=('127.0.0.1',9000)
BUFSIZE=1024
udp_server_client=socket.socket(socket.AF_INET,socket.SOCK_DGRAM)

udp_server_client.bind(ip_port)

while True:
    msg,addr=udp_server_client.recvfrom(BUFSIZE)
    print(msg,addr)

    udp_server_client.sendto(msg.upper(),addr)

####################client
import socket
ip_port=('127.0.0.1',9000)
BUFSIZE=1024
udp_server_client=socket.socket(socket.AF_INET,socket.SOCK_DGRAM)#tcp socket.SOCK_STREAM

while True:
    msg=input('>>: ').strip()
    if not msg:continue

    udp_server_client.sendto(msg.encode('utf-8'),ip_port)

    back_msg,addr=udp_server_client.recvfrom(BUFSIZE)
    print(back_msg.decode('utf-8'),addr)

UDP implementation NTP server

##############################server
from socket import *
from time import strftime

ip_port=('127.0.0.1',9000)
bufsize=1024

tcp_server=socket(AF_INET,SOCK_DGRAM)
tcp_server.bind(ip_port)

while True:
    msg,addr=tcp_server.recvfrom(bufsize)
    print('===>',msg)

    if not msg:
        time_fmt='%Y-%m-%d %X'
    else:
        time_fmt=msg.decode('utf-8')
    back_msg=strftime(time_fmt)

    tcp_server.sendto(back_msg.encode('utf-8'),addr)

tcp_server.close()

###############################client
from socket import *
ip_port=('127.0.0.1',9000)
bufsize=1024

tcp_client=socket(AF_INET,SOCK_DGRAM)

while True:
    MSG = input ('Please input the time format (for example% Y% m% d) >: '). Strip()
    tcp_client.sendto(msg.encode('utf-8'),ip_port)

    data=tcp_client.recv(bufsize)

    print(data.decode('utf-8'))

tcp_client.close()

 

Sticking phenomenon

The Conduit:

res=subprocess.Popen(cmd.decode(‘utf-8’),
shell=True,
stderr=subprocess.PIPE,
stdout=subprocess.PIPE)

The coding of the result is based on the current system. If it is windows, thenRes.stdout.read() reads GBK encodedAt the receiving endTo decode with GBK,And the results can only be read once from the pipeline

Based on TCP, we first make a program to execute command remotely (1: execute error command 2: execute LS 3: execute ifconfig)

from socket import *
import subprocess

ip_port=('127.0.0.1',8080)
BUFSIZE=1024

tcp_socket_server=socket(AF_INET,SOCK_STREAM)
tcp_socket_server.bind(ip_port)
tcp_socket_server.listen(5)

while True:
    conn,addr=tcp_socket_server.accept()
    Print ('Client ', addr)

    while True:
        cmd=conn.recv(BUFSIZE)
        if len(cmd) == 0:break

        res=subprocess.Popen(cmd.decode('utf-8'),shell=True,
                         stdout=subprocess.PIPE,
                         stdin=subprocess.PIPE,
                         stderr=subprocess.PIPE)

        stderr=act_res.stderr.read()
        stdout=act_res.stdout.read()
        conn.send(stderr)
        conn.send(stdout)

Server side

import socket
BUFSIZE=1024
ip_port=('127.0.0.1',8080)

s=socket.socket(socket.AF_INET,socket.SOCK_STREAM)
res=s.connect_ex(ip_port)

while True:
    msg=input('>>: ').strip()
    if len(msg) == 0:continue
    if msg == 'quit':break

    s.send(msg.encode('utf-8'))
    act_res=s.recv(BUFSIZE)

    print(act_res.decode('utf-8'),end='')

Client

 

Based on TCP socket, packet sticking occurs at runtime

UDP based socket will never stick at runtime

Only TCP has packet sticking, UDP will never stick packets

 

why:

The principle of socket sending and receiving messages:

The sender can send data one K one K, while the receiver’s application can carry data two K two K, of course, it is also possible to carry 3K or 6K data at a time, or only carry a few bytes of data at a time, that is to say, the data seen by the application is a whole, or a stream. How many bytes of a message are invisible to the application Therefore, TCP is a flow oriented protocol, which is also the cause of packet sticking. UDP is a message oriented protocol. Every UDP segment is a message. The application program must extract data in message units, not any bytes at a time. This is very different from TCP. How to define a message? It can be considered that the data written / sent by the other party at one time is a message. It should be understood that when the other party sends a message, no matter how the underlying layer is segmented, the TCP protocol layer will sort the data segments that make up the whole message before presenting them in the kernel buffer.

For example, when a socket client based on TCP uploads a file to the server, the content of the file is sent according to a segment of byte stream. When the receiver sees it, he doesn’t know where the byte stream of the file starts or ends

The so-called packet sticking problem is mainly caused by the receiver not knowing the boundary between messages and how many bytes of data are extracted at one time.

In addition, the packet sticking caused by the sender is caused by the TCP protocol itself. In order to improve the transmission efficiency, the sender often needs to collect enough data before sending a TCP segment. If there is little data to send for several times in a row, usually TCP will synthesize these data into a TCP segment according to the Nagle optimization algorithm and send them out once, so the receiver receives the packet sticking data.

  1. TCP (Transport Control Protocol) is connection oriented, flow oriented and provides high reliability services. There must be one-to-one pairs of sockets at both ends of the transceiver (client and server). Therefore, in order to send multiple packets to the receiver more effectively, the sender uses the optimization method (Nagle algorithm) to merge the data with small interval and small amount of data into a large data block, and then packet. In this way, it is difficult to distinguish the receiver, so a scientific unpacking mechanism must be provided That is, flow oriented communication has no message protection boundary.
  2. UDP (User Datagram Protocol) is connectionless, message oriented and provides efficient services. The block merging optimization algorithm will not be used. Because UDP supports one to many mode, the skbuff (socket buffer) at the receiving end uses the chain structure to record every UDP packet arriving. In each UDP packet, there is a message header (message source address, port and other information). Thus, for the receiving end, it is easy to distinguish.That is to say, message oriented communication has message protection boundary.
  3. TCP is based on data flow, so the received and sent messages cannot be empty. This requires adding a processing mechanism of empty messages to both the client and the server to prevent the program from getting stuck. UDP is based on data messages, even if you input empty content (direct carriage return), it is not empty message. UDP protocol will help you encapsulate the message header,

The recvfrom of UDP is blocked. A recvfrom (x) must be sent to the only one (y). After receiving x bytes of data, it will be completed. If Y > x data, it will be lost. This means that UDP will not stick to packets at all, but it will lose data, which is unreliable

The TCP protocol data will not be lost and the packet will not be completely received. The next time it is received, it will continue to receive the last time. The client will always clear the buffer content when it receives the ACK. The data is reliable, but it sticks.

 

StickyTwo situations:

1. The sender needs to wait for the buffer to be full before sending out, resulting in packet sticking (the time interval of sending data is very short, the data is very small, and the packets are stuck when they are combined)

2. The receiver does not receive the packets in the buffer in time, resulting in multiple packets receiving (the client sends a piece of data, the server only receives a small part, and the next time the server receives the data left over from the buffer, it will generate sticky packets)

Note:

When the buffer length of the sender is greater than the maximum data transmission length MTU of the network card, TCP will split the data sent into several packets and send them out.

Supplement 1:
When TCP transmits data, the sender first sends the data to its own cache, and then the protocol controls to send the data in the cache to the opposite end. The opposite end returns an ACK = 1, the sender cleans up the data in the cache, and the opposite end returns ack = 0, and then sends the data again. Therefore, TCP is reliable

When UDP sends data, the peer will not return confirmation information, so it is unreliable

Supplement two:
Send (byte stream) and recv (1024) and sendall,
The 1024 specified in recv means to take out 1024 bytes of data from the cache at a time

The byte stream of send is first put into the end-to-end cache, and then sent to the opposite end by protocol control. If the size of the byte stream to be sent is larger than the remaining space of the cache, then the data is lost. Send will be called circularly with sendall, and the data will not be lost

 

Solution to sticking

The root of the problem lies in that the receiver does not know the length of the byte stream to be transmitted by the sender, so the solution to packet sticking is to let the receiver know the total size of the byte stream to be sent by the sender before sending the data, and then the receiver comes to a dead cycle to receive all the data.

1. Before sending data, judge the data size, tell the client how much data to receive, and add an interaction through the network

#######################server
import socket,subprocess
ip_port=('127.0.0.1',8080)
s=socket.socket(socket.AF_INET,socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)

s.bind(ip_port)
s.listen(5)

while True:
    conn,addr=s.accept()
    Print ('Client ', addr)
    while True:
        msg=conn.recv(1024)
        if not msg:break
        res=subprocess.Popen(msg.decode('utf-8'),shell=True,\
                            stdin=subprocess.PIPE,\
                         stderr=subprocess.PIPE,\
                         stdout=subprocess.PIPE)
        err=res.stderr.read()
        if err:
            ret=err
        else:
            ret=res.stdout.read()
        data_length=len(ret)
        conn.send(str(data_length).encode('utf-8'))
        data=conn.recv(1024).decode('utf-8')
        if data == 'recv_ready':
            conn.sendall(ret)
    conn.close()

###################################client

import socket,time
s=socket.socket(socket.AF_INET,socket.SOCK_STREAM)
res=s.connect_ex(('127.0.0.1',8080))

while True:
    msg=input('>>: ').strip()
    if len(msg) == 0:continue
    if msg == 'quit':break

    s.send(msg.encode('utf-8'))
    length=int(s.recv(1024).decode('utf-8'))
    s.send('recv_ready'.encode('utf-8'))
    send_size=0
    recv_size=0
    data=b''
    while recv_size < length:
        data+=s.recv(1024)
        recv_size+=len(data)


    print(data.decode('utf-8'))

code

Disadvantages: the running speed of the program is much faster than the network transmission speed, so before sending a byte, send to send the byte stream length first, which will enlarge the performance loss caused by the network delay

2. Add a custom fixed length header for the byte stream. The header contains the length of the byte stream, and then send it to the peer once. When the peer receives it, first take the fixed length header from the cache, and then take the real data

Struct module

This module can convert a type, such as a number, into fixed length bytes

import json,struct
#Suppose to upload the file a.txt of 1t: 1073741824000 through the client

#To avoid sticking, you must customize the header
Header = {'file_size': 1073741824000, 'file_name': '/ A / B / C / D / E / a.txt', 'MD5': '8f6fbf8347faa4924a76856701edb0f3'} 1t data, file path and MD5 value

#In order for the header to be transmitted, it needs to be serialized and converted to bytes
Head#bytes = bytes (JSON. Dumps (header), encoding ='utf-8 ') ාserialize and convert to bytes for transmission

#In order to let the client know the length of the header, the number of header length is converted to a fixed length by using the struct: 4 bytes
Head ﹐ len ﹐ bytes = struct. Pack ('I ', len (head ﹐ bytes)) ﹐ these four bytes contain only one number, which is the length of the header

#Client starts sending
Conn.send (head ﹐ len ﹐ bytes) ﹐ length of the first header, 4 bytes
Conn.send (head × u bytes) × byte format of resend header
Conn.sendall (file content) and send the byte format of the real content

#Server starts receiving
Head ﹐ len ﹐ bytes = s.recv (4) ﹐ first receive 4 bytes of the header to get the byte format of the header length
X = struct.unpack ('I ', head ﹐ len ﹐ bytes) [0] ﹐ length of extracted header

Head#u bytes = s.recv (x) ා񖓿ාaccording to the header length x, receive the bytes format of the header
Header = JSON. Loads (JSON. Dumps (header)) ා extract header

#Finally, extract the real data according to the content of the header, such as
real_data_len=s.recv(header['file_size'])
s.recv(real_data_len)

code

 

We can make the header into a dictionary, which contains the details of the real data to be sent, and then we can serialize it with JSON, and then we can use struct to package the serialized data into four bytes

When sending:

Initial header length

Re code the header content and send it

Send the real content at the end

When receiving:

First hand header length, take it out with struct

The header content is collected according to the extracted length, then decoded and deserialized

Take the details of the data to be fetched from the result of deserialization, and then get the real data content

import socket,struct,json
import subprocess
phone=socket.socket(socket.AF_INET,socket.SOCK_STREAM)
Phone. Setsockopt (socket. Sol? Socket, socket. So? Reuseaddr, 1)

phone.bind(('127.0.0.1',8080))

phone.listen(5)

while True:
    conn,addr=phone.accept()
    while True:
        cmd=conn.recv(1024)
        if not cmd:break
        print('cmd: %s' %cmd)

        res=subprocess.Popen(cmd.decode('utf-8'),
                             shell=True,
                             stdout=subprocess.PIPE,
                             stderr=subprocess.PIPE)
        err=res.stderr.read()
        print(err)
        if err:
            back_msg=err
        else:
            back_msg=res.stdout.read()

        headers={'data_size':len(back_msg)}
        head_json=json.dumps(headers)
        head_json_bytes=bytes(head_json,encoding='utf-8')

        Conn.send (struct.pack ('I ', len (head ABCD JSON ABCD bytes))) ා length of the first header
        Conn.send (head ABCD bytes) × resend header
        Conn.sendall (back × MSG) × sending real content

    conn.close()

 Server side: customizing a slightly more complex header

code

from socket import *
import struct,json

ip_port=('127.0.0.1',8080)
client=socket(AF_INET,SOCK_STREAM)
client.connect(ip_port)

while True:
    cmd=input('>>: ')
    if not cmd:continue
    client.send(bytes(cmd,encoding='utf-8'))

    head=client.recv(4)
    head_json_len=struct.unpack('i',head)[0]
    head_json=json.loads(client.recv(head_json_len).decode('utf-8'))
    data_len=head_json['data_size']

    recv_size=0
    recv_data=b''
    while recv_size < data_len:
        recv_data+=client.recv(1024)
        recv_size+=len(recv_data)

    print(recv_data.decode('utf-8'))
    #Print (recv? Data. Decode ('gbk '))? Windows default GBK encoding

client

Link validity of authentication client

 

from socket import *
import hmac,os

secret_key=b'linhaifeng bang bang bang'
def conn_auth(conn):
    '''
    Authentication client link
    :param conn:
    :return:
    '''
    Print ('Start validation of new links')
    msg=os.urandom(32)
    conn.sendall(msg)
    h=hmac.new(secret_key,msg)
    digest=h.digest()
    respone=conn.recv(len(digest))
    return hmac.compare_digest(respone,digest)

def data_handler(conn,bufsize=1024):
    if not conn_auth(conn):
        Print ('the link is illegal, closed ')
        conn.close()
        return
    Print ('link legal, start communication ')
    while True:
        data=conn.recv(bufsize)
        if not data:break
        conn.sendall(data.upper())

def server_handler(ip_port,bufsize,backlog=5):
    '''
    Process links only
    :param ip_port:
    :return:
    '''
    tcp_socket_server=socket(AF_INET,SOCK_STREAM)
    tcp_socket_server.bind(ip_port)
    tcp_socket_server.listen(backlog)
    while True:
        conn,addr=tcp_socket_server.accept()
        Print ('New connection: [% s:% s] '% (addr [0], addr [1]))
        data_handler(conn,bufsize)

if __name__ == '__main__':
    ip_port=('127.0.0.1',9999)
    bufsize=1024
    server_handler(ip_port,bufsize)

server

from socket import *
import hmac,os

secret_key=b'linhaifeng bang bang bang'
def conn_auth(conn):
    '''
    Verify client to server links
    :param conn:
    :return:
    '''
    msg=conn.recv(32)
    h=hmac.new(secret_key,msg)
    digest=h.digest()
    conn.sendall(digest)

def client_handler(ip_port,bufsize=1024):
    tcp_socket_client=socket(AF_INET,SOCK_STREAM)
    tcp_socket_client.connect(ip_port)

    conn_auth(tcp_socket_client)

    while True:
        data=input('>>: ').strip()
        if not data:continue
        if data == 'quit':break

        tcp_socket_client.sendall(data.encode('utf-8'))
        respone=tcp_socket_client.recv(bufsize)
        print(respone.decode('utf-8'))
    tcp_socket_client.close()

if __name__ == '__main__':
    ip_port=('127.0.0.1',9999)
    bufsize=1024
    client_handler(ip_port,bufsize)

client

 

3. Socket server realizes concurrency

The key of socket based on TCP is two loops, one link loop and one communication loop

Socket server module is divided into two categories: server (to solve the link problem) and request (to solve the communication problem)

 

Class server

 

Class request

Inheritance relationship:

 

 

Take the following code as an example to analyze the socketserver source code:

ftpserver=socketserver.ThreadingTCPServer(('127.0.0.1',8080),FtpServer)ftpserver.serve_forever()

Order to find attributes: threadingtcpserver – > threadingmixin – > tcpserver – > baseserver

  1. Instantiate to get ftpserver. First find the “init” of threadingtcpserver, find it in tcpserver, and then execute “server bind, server active”
  2. Find the server “forever” under the ftpserver, find it in the baseserver, and then execute self. 65123; handle ﹣ request ﹣ noblock(), which is also in the baseserver
  3. Execute self. ﹣ handle ﹣ request ﹣ noblock() and then execute request, client ﹣ address = self. Get ﹣ request() (that is, self. Socket. Accept()) in tcpserver), and then execute self. Process ﹣ request (request, client ﹣ address)
  4. Find process “request” in threadingmixin, enable multithreading to deal with concurrency, and then execute process “request” thread, and execute self.finish “request (request, client” address)
  5. The above four parts complete the link cycle. This part begins to enter the communication processing part. Find the finish ﹣ request in the baseserver, trigger the instantiation of the class defined by ourselves, and find the ﹣ init ﹣ method. If the class defined by ourselves does not have this method, go to its parent class, baserequesthandler to find

Source code analysis summary:

Socket server based on TCP

  1. Self.server is the socket object
  2. Self.request is a link
  3. Self.client “address is the client address

Socket server based on UDP

  1. Self.request is a tuple (the first element is the data sent by the client, and the second part is the UDP socket object of the server), such as (b’adsf ‘,)
  2. Self.client “address is the client address

 

import  subprocess
import  struct
import socketserver

class Myserver(socketserver.BaseRequestHandler):
    def handle(self):
        print('coon is',self.request)  #con
        print('addr is',self.client_address) #addr
        buffer_size = 1024

        while True:
            try:
                Collect information
                data=self.request.recv(buffer_size)
                if not data:break
                Print ('the message received by the client is, 'data)
                #Execute the command and get the result of the command
                res = subprocess.Popen(cmd.decode('utf-8'), shell=True,
                                       stderr=subprocess.PIPE,
                                       stdout=subprocess.PIPE,
                                       stdin=subprocess.PIPE)
                err = res.stderr.read()
                if err:
                    cmd_res = err
                else:
                    cmd_res = res.stdout.read()

                if not cmd_res:
                    Cmd_res = 'execution succeeded'. Encode ('utf-8 ')

                length = len(cmd_res)
                data_length = struct.pack('i', length)
                self.request.send(data_length)
                self.request.send(cmd_res)

                #Low version solution to package sticking
                # conn.send(str(length).encode('utf-8'))
                # client_ready=conn.recv(buffer_size)
                #
                # if client_ready==b'ready':
                #     conn.send(cmd_res)

            except Exception as e:
                print(e)
                break

if __name__=='__main__':
    s=socketserver.ThreadingTCPServer(('127.0.0.1',8080),Myserver)
    s.serve_forever()
import socketserver

class Myserver(socketserver.BaseRequestHandler):
    def handle(self):
        print('coon is',self.request)  #connect
        print('addr is',self.client_address) #addr

        while True:
            try:
                Collect information
                data=self.request.recv(1024)
                if not data:break
                Print ('the message received by the client is, 'data)

                Send messages
                self.request.sendall(data.upper())
            except Exception as e:
                print(e)
                break


if __name__=='__main__':
    s=socketserver.ThreadingTCPServer(('127.0.0.1',8080),Myserver)
    s.serve_forever()