Log file migration to OSS

Time:2021-2-22

A server is using alicloud ECS because it is poor and has limited disk space. The server log files are growing every day. It needs to be cleaned up regularly to release disk space. Several solutions are proposed

  1. The write task is backed up to the local server regularly
  2. Download directly to local mobile hard disk for backup
  3. Backup to alicloud OSS object storage

The first two schemes need to occupy the external network bandwidth of ECs, and can only be backed up in the low peak period. The accumulated log files are tens of gigabytes, and the small pipe bandwidth backup takes at least half a day.

After checking the OSS object storage, you can use the intranet to transfer files between ECs and OSS of the same operator, so you don’t have to worry about bandwidth and speed.

OSS has the following storage types:

Because it’s a log file, you won’t read it often when there’s no problem, so you choose the archive storage, which is suitable for you and the price is beautiful.

Specific operation
Download tools

wget http://gosspublic.alicdn.com/ossutil/1.6.0/ossutil64

Grant executable rights

chmod 755 ossutil64

Set to Chinese (optional with mood)

./ossutil64 config -L ch

Set the configuration file and fill in the information as required

./ossutil64 config

If you need to use intranet transmission, you need to fill in the intranet address in the figure below.


Common commands:

upload

./ossutil64 cp yourFile oss://yourBucket/path

Upload folder

./ossutil64 cp -r youDir oss://yourBucket/path

Incremental upload

./ossutil64 cp -r -u yourDir oss://yourBucket/path

Backup log can write incremental upload command to script

#!/usr/bin/bash
your-install-path/ossutil64 cp -r -u yourDir oss://yourBucket/path

Start timing task

crontab -e

At 4 o’clock every day

SHELL=/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=root
HOME=/
00 04 * * * sh yourBash.sh

In this way, the log incremental backup operation is simple

Ossutil reference: https://help.aliyun.com/document_ detail/50452.html

Recommended Today

Background management system menu management module

1 menu management page design 1.1 business design Menu management, also known as resource management, is the external manifestation of system resources. This module is mainly to add, modify, query and delete the menu. CREATE TABLE `sys_menus` ( `id` int(11) NOT NULL AUTO_INCREMENT, `Name ` varchar (50) default null comment ‘resource name’, `URL ` varchar […]