Log Management with LogRotate CentOS 7

If a server runs for a long time without log rotation, compression or cleanup, log files can fill up all the available disk space. Logrotate solves this problem: it rotates, compresses and deletes old logs automatically.

Logrotate is installed by default and rotates the logs of installed packages and applications. The examples below were made on CentOS 7, but logrotate works the same way on AlmaLinux, Rocky Linux, Ubuntu and Debian.

Check the logrotate version:

logrotate --version

logrotate 3.8.6

Logrotate Configuration

The configuration is stored in two places:

the main configuration file: /etc/logrotate.conf;
the directory /etc/logrotate.d/ with separate files for applications.

Let us look at the main configuration file:

nano /etc/logrotate.conf

The file looks like this:

# see "man logrotate" for details
# rotate log files weekly
weekly

# keep 4 weeks worth of backlogs
rotate 4

# create new (empty) log files after rotating old ones
create

# use date as a suffix of the rotated file
dateext

# uncomment this if you want your log files compressed
#compress

# RPM packages drop log rotation information into this directory
include /etc/logrotate.d

# no packages own wtmp and btmp -- we'll rotate them here
/var/log/wtmp {
    monthly
    create 0664 root utmp
    minsize 1M
    rotate 1
}

/var/log/btmp {
    missingok
    monthly
    create 0600 root utmp
    rotate 1
}

The global settings at the top apply to all logs. The two blocks at the end set rules for two specific log files; any directive not set in a block is taken from the global settings.

Main Directives

daily, weekly, monthly, yearly: how often to rotate the log;
size: rotate the log when it grows larger than the given size;
rotate: how many old logs to keep;
create: create a new empty log file after rotation;
dateext: add the rotation date as a suffix to the old log name;
compress: compress old logs;
delaycompress: postpone compression of the most recent rotated log until the next rotation;
extension: keep the given file extension after the rotation suffix;
mail: send the rotated log by email;
maxage: remove rotated logs older than the given number of days;
missingok: do not show an error if the log file does not exist;
notifempty: do not rotate empty logs;
olddir: move rotated logs to the given directory;
postrotate/endscript: commands to run after rotation;
start: the number from which old logs are numbered.

There are two ways to manage the logs of your applications:

create a configuration file in /etc/logrotate.d/. It will run every day as root, together with the other standard logrotate tasks;
create a separate configuration file and run logrotate with it from your own cron job.

How to Create a Configuration in /etc/logrotate.d/

Let us set up rotation for an application that writes the logs access.log and error.log to the /var/log/example-app/ directory. Create a new file:

sudo nano /etc/logrotate.d/example-app

Paste the configuration:

/var/log/example-app/*.log {
    daily
    missingok
    rotate 14
    compress
    notifempty
    create 0640 nginx nginx
    sharedscripts
    postrotate
        systemctl reload example-app
    endscript
}

The directives used here:

create 0640 nginx nginx: after rotation, create a new empty log with permissions 0640, owned by the nginx user and group. Use a user that exists on your system (for example, www-data on Ubuntu and Debian), otherwise logrotate skips this configuration with the error unknown user;
sharedscripts: run the postrotate script only once, not for each log file. Our configuration matches two files (access.log and error.log), and the script runs once;
postrotate ... endscript: the commands to run after rotation. In this example, the application is reloaded so that it reopens its log files.

Test the configuration in debug mode. Logrotate shows what it would do, but does not change any files:

sudo logrotate -d /etc/logrotate.d/example-app

reading config file /etc/logrotate.d/example-app
Allocating hash table for state file, size 15360 B

Handling 1 logs

rotating pattern: /var/log/example-app/*.log  after 1 days (14 rotations)
empty log files are not rotated, old logs are removed
considering log /var/log/example-app/access.log
  log does not need rotating
considering log /var/log/example-app/error.log
  log does not need rotating

The standard daily logrotate run will now process this configuration together with the others.

How to Run Logrotate with Your Own Configuration

In this example, an application runs as the user testing and writes logs to /home/testing/logs/. We need to rotate them every hour. The standard logrotate run happens only once a day, so we create a separate configuration and run it from cron.

Create the configuration file:

nano /home/testing/logrotate.conf

Paste:

/home/testing/logs/*.log {
    hourly
    missingok
    rotate 24
    compress
    create
}

Save and close the file. This configuration rotates the logs every hour, compresses them, keeps 24 old logs and creates a new empty log after each rotation.

Create a test log file:

cd ~
mkdir logs
touch logs/access.log

The logs belong to the testing user, so you do not need sudo. But you need to specify a state file: logrotate records there what it saw and did during the last run, so it knows what to do next time. Let us keep the state file in the home directory:

logrotate /home/testing/logrotate.conf --state /home/testing/logrotate-state --verbose

reading config file /home/testing/logrotate.conf
Handling 1 logs
rotating pattern: /home/testing/logs/*.log  hourly (24 rotations)
empty log files are rotated, old logs are removed
considering log /home/testing/logs/access.log
  log does not need rotating

The --verbose option prints details about what logrotate does. This is the first time logrotate sees this log file, and the file is less than an hour old, so it is not rotated yet. The state file now contains the record of this run:

cat /home/testing/logrotate-state

logrotate state -- version 2
"/home/testing/logs/access.log" 2020-5-19-16:0:0

If you run the same command an hour later, the log will be rotated. To rotate it right away, add the --force option:

logrotate /home/testing/logrotate.conf --state /home/testing/logrotate-state --verbose --force

Finally, set up a cron job to run logrotate every hour. Open the crontab of the user:

crontab -e

Move the cursor to a new line at the end of the file and add:

14 * * * * /usr/sbin/logrotate /home/testing/logrotate.conf --state /home/testing/logrotate-state

The job runs at minute 14 of every hour. If you need a server for your applications, take a look at our Linux VPS plans.

Blog

Subscribe to our Newsletter