Sample code of using shell script to segment nginx logs


This article introduces the example code of using shell script to segment nginx logs, which is shared to you as follows:

1. Log format

Default log format: Main

log_format main '$remote_addr - $remote_user [$time_local] "$request" '
          $status $body_bytes_sent "$http_referer" '
         '"$http_user_agent" "$http_x_forwarded_for"';

For example, the default main log format records these items

Remote IP remote user / user time request method (such as get / post) request body length referer source information

HTTP user agent user agent / spider, the original IP of the forwarded request

HTTP “X” forwarded “for: when passing through an agent, the agent adds your original IP address to the header information and transmits your original IP address

2. Implementation ideas

Shell + timed task + nginx signal management, complete log storage by date

log_path=$(date -d yesterday +"%Y%m")
day=$(date -d yesterday +"%d")
mkdir -p $base_path/$log_path
mv $base_path/access.log $base_path/$log_path/access_$day.log
kill -USR1 `cat /usr/local/nginx/logs/`

3. Scheduled tasks

Crontab edit scheduled task

01 00 * * / xxx / path / 0:01 every day (it is recommended to be between 02-04, with small system load)

The above is the whole content of this article. I hope it will help you in your study, and I hope you can support developepaer more.