Sample code of using shell script to segment nginx logs

Time:2020-1-17

This article introduces the example code of using shell script to segment nginx logs, which is shared to you as follows:

1. Log format

Default log format: Main


log_format main '$remote_addr - $remote_user [$time_local] "$request" '
          $status $body_bytes_sent "$http_referer" '
         '"$http_user_agent" "$http_x_forwarded_for"';

For example, the default main log format records these items

Remote IP remote user / user time request method (such as get / post) request body length referer source information

HTTP user agent user agent / spider, the original IP of the forwarded request

HTTP “X” forwarded “for: when passing through an agent, the agent adds your original IP address to the header information and transmits your original IP address

2. Implementation ideas

Shell + timed task + nginx signal management, complete log storage by date


#!/bin/bash
base_path='/usr/local/nginx/logs'
log_path=$(date -d yesterday +"%Y%m")
day=$(date -d yesterday +"%d")
mkdir -p $base_path/$log_path
mv $base_path/access.log $base_path/$log_path/access_$day.log
kill -USR1 `cat /usr/local/nginx/logs/nginx.pid`

3. Scheduled tasks

Crontab edit scheduled task

01 00 * * / xxx / path / b.sh 0:01 every day (it is recommended to be between 02-04, with small system load)

The above is the whole content of this article. I hope it will help you in your study, and I hope you can support developepaer more.