Log Management with LogRotate CentOS 7

If after a long-time work of a server there was no file rotation, compression, or regular deletion, they can occupy all the available memory space.

Logrotate is installed in the default settings on a server. The system is designed to process the history of all the packages and apps installed.

To check the Logrotate version:

logrotate --version

Command output will be like this:

[root@kvmde54-19861 ~]# logrotate --version

logrotate 3.8.6

The standard Logrotate configuration is stored by the two roots:

The main configuration file - /etc/logrotate.conf.
To create certain logs settings, you should use the directory /etc/logrotate.d

Let’s look at the Logrotate configuration file /etc/logrotate.d:

nano /etc/logrotate.conf

Command output will be like this:

# see "man logrotate" for details

# rotate log files weekly

weekly

# keep 4 weeks worth of backlogs

rotate 4

# create new (empty) log files after rotating old ones

create

# use date as a suffix of the rotated file

dateext

# uncomment this if you want your log files compressed

#compress

# RPM packages drop log rotation information into this directory

include /etc/logrotate.d

# no packages own wtmp and btmp -- we'll rotate them here

/var/log/wtmp {

monthly

create 0664 root utmp

minsize 1M

rotate 1

}

/var/log/btmp {

missingok

monthly

create 0600 root utmp

rotate 1

}

# system-specific logs may be also be configured here.

include /etc/logrotate.d/web

The file contains configuration blocks for two different files of a catalog history. The two blocks have the same functions. Any features not set in the configuration blocks accept the default settings and the ones from the file /etc/logrotate.conf.

The main directives for managing and processing logs:

monthly – a rotation one time per month. There are other options which include daily, weekly, monthly, size;
notifempty – no rotation for an empty log file;
rotate – how many old logs you need to store; the number is presented is parameters;
create – means that you should create an empty log file after you replace the old one;
dateext – the directive adds the rotation date before the old log name;
compress – means that you should compress the log;
delaycompress – do not compress the last log and the one before it;
extension – save the original log file after rotation if it has a set extension;
mail – send an email after rotation is finished;
maxage – rotate logs if they are older than necessary;
missingok – do not show an error if the log file is absent;
olddir – replace old logs to a stated folder;
postrotate/endscript – random commands after rotation;
start – the number from which the old logs start their numeration;
size – the log size after it has been replaced;

To manage log files for apps, you can use the two options:

Create a new configuration file Logrotate and move it to the catalog /etc/logrotate.d/. The file will work every day as a root user. It will also include other standard LogRotate tasks.
Create a new configuration file and launch it using the LogRotate default settings in Ubuntu.

How to create configuration in /etc/logrotate.d/

To make an example, let’s set updates for the server which creates logs to files access.log и error.log stored in the catalog /var/log/example-app/.

To add the configuration to the catalog /etc/logrotate.d/, open a new file:

sudo nano /etc/logrotate.d/example-app

/var/log/example-app/*.log {

daily

missingok

rotate 14

compress

notifempty

create 0640 www-data www-data

sharedscripts

postrotate

systemctl reload example-app

endscript

}

Description of directives:

create 0640 www-data www-data – the command creates a new empty file of the log after rotation using the set extensions (0640), owner (www-data), and group (www-data);
sharedscripts – this option means that any scripts that you add to the configuration are processed only once during launching after compression. They are not processed for separate updated files. As our configuration will correspond to the two log files (access.log и error.log), the script mentioned it postrotate will be launched only once;
postrotate to endscript – the script used in this block will be launched after the log file gets updated. In the example, the app is reset.

After you have set the configuration using the requirements, you can test it with the command:

sudo logrotate /etc/logrotate.conf --debug

[root@kvmde54-19861 ~]# sudo logrotate /etc/logrotate.conf --debug

reading config file /etc/logrotate.conf

including /etc/logrotate.d

Ignoring nginx.rpmnew, because of .rpmnew ending

reading config file bootlog

reading config file chrony

reading config file example-app

error: example-app:7 unknown user 'www-data'

error: found error in /var/log/example-app/*.log , skipping

removing last 1 log configs

reading config file exim

reading config file httpd

reading config file mariadb

reading config file nginx

reading config file ppp

reading config file proftpd

reading config file roundcubemail

reading config file syslog

Ignoring web because it's not a regular file.

reading config file wpa_supplicant

reading config file yum

including /etc/logrotate.d/web

Allocating hash table for state file, size 15360 B

Handling 13 logs

rotating pattern: /var/log/boot.log

after 1 days (7 rotations)

empty log files are not rotated, old logs are removed

considering log /var/log/boot.log

log does not need rotating (log has been rotated at 2020-5-19 15:23, that is not day ago yet)

rotating pattern: /var/log/chrony/*.log weekly (4 rotations)

empty log files are rotated, old logs are removed

considering log /var/log/chrony/*.log

log /var/log/chrony/*.log does not exist -- skipping

not running postrotate script, since no logs were rotated

rotating pattern: /var/log/exim/*log weekly (4 rotations)

empty log files are not rotated, old logs are removed

considering log /var/log/exim/main.log

log does not need rotating (log has been rotated at 2020-5-19 15:23, that is not week ago yet)

considering log /var/log/exim/reject.log

log does not need rotating (log has been rotated at 2020-5-19 15:23, that is not week ago yet)

rotating pattern: /var/log/httpd/*log after 1 days (3 rotations)

empty log files are not rotated, old logs are removed

considering log /var/log/httpd/access_log

log does not need rotating (log has been rotated at 2020-5-19 15:23, that is not day ago yet)

considering log /var/log/httpd/error_log

log does not need rotating (log has been rotated at 2020-5-19 15:23, that is not day ago yet)

not running postrotate script, since no logs were rotated

rotating pattern: /var/log/nginx/*.log after 1 days (3 rotations)

empty log files are not rotated, old logs are removed

considering log /var/log/nginx/access.log

log does not need rotating (log has been rotated at 2020-5-19 15:23, that is not day ago yet)

considering log /var/log/nginx/error.log

log does not need rotating (log has been rotated at 2020-5-19 15:23, that is not day ago yet)

not running postrotate script, since no logs were rotated

rotating pattern: /var/log/ppp/connect-errors after 1 days (5 rotations)

empty log files are not rotated, old logs are removed

considering log /var/log/ppp/connect-errors

log /var/log/ppp/connect-errors does not exist -- skipping

rotating pattern: /var/log/proftpd/*.log /var/log/xferlog weekly (4 rotations)

empty log files are not rotated, old logs are removed

considering log /var/log/proftpd/controls.log

log does not need rotating (log has been rotated at 2020-5-19 15:23, that is not week ago yet)

considering log /var/log/xferlog

log /var/log/xferlog does not exist -- skipping

not running postrotate script, since no logs were rotated

rotating pattern: /var/log/roundcubemail/*.log 30720 bytes (4 rotations)

empty log files are not rotated, old logs are removed

switching euid to 0 and egid to 48

considering log /var/log/roundcubemail/*.log

log /var/log/roundcubemail/*.log does not exist -- skipping

switching euid to 0 and egid to 0

rotating pattern: /var/log/cron

/var/log/maillog

/var/log/messages

/var/log/secure

/var/log/spooler

weekly (4 rotations)

empty log files are rotated, old logs are removed

considering log /var/log/cron

log does not need rotating (log has been rotated at 2020-5-19 15:23, that is not week ago yet)

considering log /var/log/maillog

log does not need rotating (log has been rotated at 2020-5-19 15:23, that is not week ago yet)

considering log /var/log/messages

log does not need rotating (log has been rotated at 2020-5-19 15:23, that is not week ago yet)

considering log /var/log/secure

log does not need rotating (log has been rotated at 2020-5-19 15:23, that is not week ago yet)

considering log /var/log/spooler

log does not need rotating (log has been rotated at 2020-5-19 15:23, that is not week ago yet)

not running postrotate script, since no logs were rotated

rotating pattern: /var/log/wpa_supplicant.log 30720 bytes (4 rotations)

empty log files are not rotated, old logs are removed

considering log /var/log/wpa_supplicant.log

log does not need rotating (log size is below the 'size' threshold)

rotating pattern: /var/log/yum.log yearly (4 rotations)

empty log files are not rotated, log files >= 30720 are rotated earlier, old logs are removed

considering log /var/log/yum.log

log does not need rotating (log has been rotated at 2020-5-19 15:23, that is not year ago yet)

rotating pattern: /var/log/wtmp monthly (1 rotations)

empty log files are rotated, only log files >= 1048576 bytes are rotated, old logs are removed

considering log /var/log/wtmp

log does not need rotating (log has been rotated at 2020-5-19 15:23, that is not month ago yet)

log does not need rotating ('misinze' directive is used and the log size is smaller than the minsize value

rotating pattern: /var/log/btmp monthly (1 rotations)

empty log files are rotated, old logs are removed

considering log /var/log/btmp

log does not need rotating (log has been rotated at 2020-5-19 15:23, that is not month ago yet)

As a result, you will get the logrotate utility and state the standard configuration file in debug mode.

The information on the logrotate files processed at the moment will get output in the console. The standard Logrotate setting will be processed once a day together with the new configuration.

After this, you can check what you have created:

[root@kvmde54-19861 ~]# logrotate -d /etc/logrotate.d/example-app

reading config file /etc/logrotate.d/example-app

removing last 1 log configs

Allocating hash table for state file, size 15360 B

Handling 0 logs

How to create the LogRotate configuration

In this example, we use an app controlled by user testing, which is the generation of logs stored in the catalog /home/testing/logs/. We need to rotate the logs hourly. That is why we should install it beyond the structure /etc/logrotate.d that you can see in Ubuntu.

Let’s create a configuration file in our catalog using the text editor.

nano /home/testing/logrotate.conf

Then you should paste the configuration below:

/home/testing/logrotate.conf

/home/testing/logs/*.log {

hourly

missingok

rotate 24

compress

create

}

Then save and close the file.

Such a configuration will rotate files hourly. It will compress them and save twenty-four old logs at the same time saving the new log file to replace the previous one.

Then you should set the configuration based on your app.

Let’s create a log file to check how it works:

cd ~

mkdir logs

touch logs/access.log

As the logs belong to the testing user, you don’t need to use sudo. Yet, you should state the status file. The file records the facts that logrotate saw or did last time. For this reason, the status file knows what operations it needs to do at the next launch.

Let’s ask Logrotate to place the status file directly to the home catalog for this example. We can state anything available and convenient:

logrotate /home/testing/logrotate.conf --state /home/testing/logrotate-state --verbose --force

Output

[testing@kvmde54-19861 ~]$ logrotate /home/testing/logrotate.conf --state /home/testing/logrotate-state --verbose --force

reading config file /home/testing/logrotate.conf

Allocating hash table for state file, size 15360 B

Handling 1 logs

rotating pattern: /home/testing/logrotate.conf

/home/testing/logs/*.log forced from command line (24 rotations)

empty log files are rotated, old logs are removed

considering log /home/testing/logrotate.conf

log needs rotating

considering log /home/testing/logs/access.log

log needs rotating

rotating log /home/testing/logrotate.conf, log->rotateCount is 24

dateext suffix '-2020051916'

glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'

renaming /home/testing/logrotate.conf.24.gz to /home/testing/logrotate.conf.25.gz (rotatecount 24, logstart 1, i 24),

old log /home/testing/logrotate.conf.24.gz does not exist

renaming /home/testing/logrotate.conf.23.gz to /home/testing/logrotate.conf.24.gz (rotatecount 24, logstart 1, i 23),

–verbose prints the detailed information on Logrotate actions. It is the first time when LogRotate has seen this log file. As far as we know, at that moment the file is zero hours. So, we should not apply rotation to it.

If we look at the status file, we can see that Logrotate recorded information on the launch:

cat /home/testing/logrotate-state

Output

[testing@kvmde54-19861 ~]$ cat /home/testing/logrotate-state

logrotate state -- version 2

"/home/testing/logs/access.log" 2020-5-19-16:39:6

"/home/testing/logrotate.conf" 2020-5-19-16:39:6

Logrotate noticed that it had seen logs and when it had watched their rotation for the last time. If we launch the same command one hour later, the log will be rotated.

If you want to make LogRotate rotate the log file, you need to use the flag –force:

logrotate /home/testing/logrotate.conf --state /home/testing/logrotate-state --verbose --force

Then you should set the task cron to launce Logrotate hourly. Open crontab of the user:

crontab -e

You will see a text file opened. The file may have got such comments explaining the supposed main syntax.

Replace the cursor to the new empty line at the end of the file, and add the following:

14 * * * * /usr/sbin/logrotate /home/testing/logrotate.conf --state /home/testing/logrotate-state

The task will be processed every fourteen minutes of each hour daily.

Blog

Subscribe to our Newsletter