After the server is attacked, this troubleshooting process does not back pot!

Time:2020-11-25

After the server is attacked, this troubleshooting process does not back pot!

Original text: https://www.toutiao.com/a6688…

Security is always relative, and even a secure server may be attacked. As a security operation and maintenance personnel, the principle to grasp is: do a good job in the system security protection, repair all known dangerous behavior, at the same time, after the system is attacked, can quickly and effectively deal with the attack behavior, minimize the impact of the attack on the system.

1、 General idea of dealing with server attack

System attack is not terrible, terrible is the face of attack helpless, the following is a detailed description of the general processing ideas after the server is attacked.

1. Cut off the network

All attacks come from the network. Therefore, after knowing that the system is being attacked by hackers, the first thing to do is to disconnect the network connection of the server. In this way, in addition to cutting off the source of attack, it can also protect other hosts in the network where the server is located.

2. Find the source of the attack

You can view suspicious information by analyzing the system log or login log file. At the same time, you should also check which ports the system has opened and which processes are running, and analyze which suspicious programs through these processes. This process should be traced and analyzed according to experience and comprehensive judgment ability. The following sections will detail the processing ideas of this process.

3. Analyze the reasons and ways of invasion

Since the system has been intruded, there are many reasons, which may be system vulnerabilities or program vulnerabilities. It is necessary to find out which reason is caused, and also to find out the attack path and find the attack source, because only by knowing the cause and path of the attack can we delete the attack source and repair the vulnerability at the same time.

4. Backup user data

After the server is attacked, the user data on the server needs to be backed up immediately, and at the same time, whether the attack source is hidden in the data should be checked. If the attack source is in the user data, it must be completely deleted, and then the user data will be backed up to a safe place.

5. Re install the system

Never think that you can completely remove the source of the attack, because no one can understand the attack program better than the hacker. After the server is attacked, the safest and simplest way is to re install the system. Because most of the attack programs will be attached to the system files or the kernel, so re installing the system can completely remove the attack source.

6. Fix program or system vulnerability

After the system vulnerability or application vulnerability is found, the first thing to do is to repair the system vulnerability or change the program bug, because only after the program vulnerability is repaired can it be formally run on the server.

7. Restore data and connect to the network

Copy the backup data to the newly installed server, and then start the service. Finally, open the network connection of the server to provide external services.

2、 Check and lock suspicious users

When the server is found to be under attack, the network connection must be cut off first. However, in some cases, such as when the network connection cannot be cut off immediately, you must log in to the system to check whether there are suspicious users. If a suspicious user logs in to the system, the user needs to be locked immediately, and then the remote connection of the user is interrupted.

1. Log in to the system to check suspicious users

Log in through the root user, and then execute the “W” command to list all the users who have logged in to the system, as shown in the figure below.

After the server is attacked, this troubleshooting process does not back pot!

With this output, you can check whether there are suspicious or unfamiliar users logging in. At the same time, you can judge whether they are illegal users based on the user name, the source address of the user login and the process they are running.

2. Lock suspicious users

Once a suspicious user is found, it must be locked immediately. For example, after executing the “W” command above, it is found that the nobody user should be a suspicious user (because nobody has no login permission by default), so lock the user first and perform the following operations:

[[email protected] ~]# passwd -l nobody

After locking, it is possible that the user is still in the login state, so the user will be kicked off the line. According to the output of the “W” command above, the PID value of this user login can be obtained. The operation is as follows:

[[email protected] ~]# ps -ef|grep @pts/3
531 6051 6049 0 19:23 ? 00:00:00 sshd: [email protected]/3
[[email protected] ~]# kill -9 6051

This will kick the suspicious user nobody off the line. If this user tries to log in again, it will no longer be able to log in.

3. View the user login events through the last command

The last command records the log of all users logging in to the system, which can be used to find the login events of unauthorized users. The output of last command comes from the / var / log / wtmp file. Some experienced intruders will delete / var / log / wtmp to clear their tracks, but there are still traces in this file.

3、 View system log

Viewing the system log is the best way to find the attack source. The system logs that can be checked include / var / log / messages, / var / log / secure, etc. these two log files can record the running status of the software and the login status of remote users, as well as view the. Bash in each user directory_ History files, especially. Bash in the / root directory_ All the user history files are recorded.

4、 Check and shut down suspicious system processes

There are many commands to check suspicious processes, such as PS, top, etc., but sometimes you only know the name of the process, but you can’t know the path. At this time, you can check it with the following command:

Firstly, the pidof command can be used to find the PID of the running process. For example, to find the PID of the sshd process, execute the following command:

After the server is attacked, this troubleshooting process does not back pot!

Then enter the memory directory to view the information of the EXE file under the corresponding PID directory

After the server is attacked, this troubleshooting process does not back pot!

In this way, the full execution path corresponding to the process is found. If you have a handle to view the file, you can view the following directory:

[[email protected] ~]# ls -al /proc/13276/fd

In this way, the complete execution information of any process can be found. In addition, there are many similar commands that can help system operation and maintenance personnel to find suspicious processes. For example, you can find the process PID by specifying the port or TCP or UDP protocol, and then find the related process

After the server is attacked, this troubleshooting process does not back pot!

In some cases, the attacker’s program is very hidden, such as rootkits backdoor program. In this case, PS, top, netstat and other commands may have been replaced. If the system’s own commands are used to check suspicious processes, it will become totally untrustworthy. At this time, it is necessary to check suspicious system programs with the help of third-party tools, such as chkrootkit and rkhunt R and other tools, through these tools can be very convenient to find the system has been replaced or tampered with the program.

5、 Check the integrity of the file system

Checking whether the file attributes have changed is the simplest and most direct method to verify the integrity of the file system. For example, you can check whether the size of the / bin / LS file on the invaded server is the same as that of the file on the normal system to verify whether the file has been replaced, but this method is relatively low-level. At this point, you can use the RPM tool under Linux to complete the verification. The operations are as follows:

After the server is attacked, this troubleshooting process does not back pot!

The meaning of each tag in the output is described as follows:

  • S indicates that the file length has changed
  • M means that the access rights or file type of the file have changed
  • 5 indicates that the MD5 check sum has changed
  • D indicates that the attributes of the device node have changed
  • L indicates that the symbolic link of the file has changed
  • U indicates that the owner of the file / subdirectory / device node has changed
  • G indicates that the group of the file / subdirectory / device node has changed
  • T indicates that the last modification time of the file has changed

If the “m” mark appears in the output result, the corresponding file may have been tampered with or replaced. At this time, you can clear the attacked file by uninstalling the RPM package and re installing it.

However, this command has a limitation, that is, it can only check all files installed through the RPM package, and can’t do anything about the files installed through the non RPM package mode. At the same time, if the RPM tool is also replaced, this method cannot be used. At this time, you can copy an RPM tool from a normal system for testing.

The file system can also be checked through chkrootkit and rkhunter. The use of chkrootkit and rkhunter will be introduced next time.