How to Setup and Manage Log Rotation Using Logrotate in Linux

One of the most interesting (and perhaps one of the most important as well) directories in a Linux system is /var/log. According to the Filesystem Hierarchy Standard, the activity of most services running in the system are written to a file inside this directory or one of its subdirectories.

Such files are known as logs and are the key to examining how the system is operating (and how it has behaved in the past). Logs are also the first source of information where administrators and engineers look while troubleshooting.

If we look at the contents of /var/log on a CentOS/RHEL/Fedora and Debian/Ubuntu (for variety) we will see the following log files and subdirectories.

Please note that the result may be somewhat different in your case depending on the services running on your system(s) and the time they have been running.

In RHEL/CentOS and Fedora

# ls /var/log
Log Files and Directories under CentOS 7
Log Files and Directories under CentOS 7

In Debian and Ubuntu

# ls /var/log
Log Files and Directories in Debian 8
Log Files and Directories in Debian 8

In both cases, we can observe that some of the log names end as expected in “log”, while others are either renamed using a date (for example, maillog-20160822 on CentOS) or compressed (consider auth.log.2.gz and mysql.log.1.gz on Debian).

This is not a default behavior based on the chosen distribution but can be changed at will using directives in the configuration files, as we will see in this article.

If logs were kept forever, they would eventually end up filling the filesystem where /var/log resides. In order to prevent that, the system administrator can use a nice utility called logrotate to clean up the logs on a periodic basis.

In a few words, logrotate will rename or compress the main log when a condition is met (more about that in a minute) so that the next event is recorded on an empty file.

In addition, it will remove “old” log files and will keep the most recent ones. Of course, we get to decide what “old” means and how often we want logrotate to clean up the logs for us.

Installing Logrotate in Linux

To install logrotate, just use your package manager:

---------- On Debian and Ubuntu ---------- 
# aptitude update && aptitude install logrotate 

---------- On CentOS, RHEL and Fedora ---------- 
# yum update && yum install logrotate

It is worth and well to note that the configuration file (/etc/logrotate.conf) may indicate that other, more specific settings may be placed on individual .conf files inside /etc/logrotate.d.

This will be the case if and only if the following line exists and is not commented out:

include /etc/logrotate.d

We will stick with this approach, as it will help us to keep things in order, and use the Debian box for the following examples.

Configure Logrotate in Linux

Being a very versatile tool, logrotate provides plenty of directives to help us configure when and how the logs will be rotated, and what should happen right afterward.

Let’s insert the following contents in /etc/logrotate.d/apache2.conf (note that most likely you will have to create that file) and examine each line to indicate its purpose:

apache2.conf
/var/log/apache2/* {
    weekly
    rotate 3
    size 10M
    compress
    delaycompress
}

The first line indicates that the directives inside the block apply to all logs inside /var/log/apache2:

  • weekly means that the tool will attempt to rotate the logs on a weekly basis. Other possible values are daily and monthly.
  • rotate 3 indicates that only 3 rotated logs should be kept. Thus, the oldest file will be removed on the fourth subsequent run.
  • size=10M sets the minimum size for the rotation to take place to 10M. In other words, each log will not be rotated until it reaches 10MB.
  • compress and delaycompress are used to tell that all rotated logs, with the exception of the most recent one, should be compressed.

Let’s execute a dry-run to see what logrotate would do if it was actually executed now. Use the -d option followed by the configuration file (you can actually run logrotate by omitting this option):

# logrotate -d /etc/logrotate.d/apache2.conf

The results are shown below:

Rotate Apache Logs with Logrotate
Rotate Apache Logs with Logrotate

Instead of compressing the logs, we could rename them after the date when they were rotated. To do that, we will use the dateext directive. If our date format is other than the default yyyymmdd, we can specify it using dateformat.

Note that we can even prevent the rotation from happening if the log is empty with notifempty. In addition, let’s tell logrotate to mail the rotated log to the system administrator ([email protected] in this case) for his / her reference (this will require a mail server to be set up, which is out of the scope of this article).

If you want to get emails about logrotate, you can setup Postfix mail server as shown here: Install Postfix Mail Server

This time we will use /etc/logrotate.d/squid.conf to only rotate /var/log/squid/access.log:

squid.conf
/var/log/squid/access.log {
    monthly
    create 0644 root root
    rotate 5
    size=1M
    dateext
    dateformat -%d%m%Y
    notifempty
    mail [email protected]
}

As we can see in the image below, this log did not need to be rotated. However, when the size condition is met (size=1M), the rotated log will be renamed access.log-25082020 (if the log was rotated on August 25, 2020) and the main log (access.log) will be re-created with access permissions set to 0644 and with root as owner and group owner.

Finally, when the number of logs finally reaches 6, the oldest log will be mailed to [email protected].

Rotate Squid Logs with Logrotate
Rotate Squid Logs with Logrotate

Now let’s suppose you want to run a custom command when the rotation takes place. To do that, place the line with such command between the postrotate and endscript directives.

For example, let’s suppose we want to send an email to root when any of the logs inside /var/log/myservice gets rotated. Let’s add the lines in red to /etc/logrotate.d/squid.conf:

squid.conf
/var/log/myservice/* {
	monthly
	create 0644 root root
	rotate 5
	size=1M
    	postrotate
   		echo "A rotation just took place." | mail root
    	endscript
}

Last, but not least, it is important to note that options present in /etc/logrotate.d/*.conf override those in the main configuration file in case of conflicts.

Logrotate and Cron

By default, the installation of logrotate creates a crontab file inside /etc/cron.daily named logrotate. As it is the case with the other crontab files inside this directory, it will be executed daily starting at 6:25 am if anacron is not installed.

Otherwise, the execution will begin around 7:35 am. To verify, watch for the line containing cron.daily in either /etc/crontab or /etc/anacrontab.

Summary

In a system that generates several logs, the administration of such files can be greatly simplified using logrotate. As we have explained in this article, it will automatically rotate, compress, remove, and mail logs on a periodic basis or when the file reaches a given size.

Just make sure it is set to run as a cron job and logrotate will make things much easier for you. For more details, refer to the man page.

Do you have any questions or suggestions about this article? Feel free to let us know using the comment form below.

Hey TecMint readers,

Exciting news! Every month, our top blog commenters will have the chance to win fantastic rewards, like free Linux eBooks such as RHCE, RHCSA, LFCS, Learn Linux, and Awk, each worth $20!

Learn more about the contest and stand a chance to win by sharing your thoughts below!

Gabriel Cánepa
Gabriel Cánepa is a GNU/Linux sysadmin and web developer from Villa Mercedes, San Luis, Argentina. He works for a worldwide leading consumer product company and takes great pleasure in using FOSS tools to increase productivity in all areas of his daily work.

Each tutorial at TecMint is created by a team of experienced Linux system administrators so that it meets our high-quality standards.

Join the TecMint Weekly Newsletter (More Than 156,129 Linux Enthusiasts Have Subscribed)
Was this article helpful? Please add a comment or buy me a coffee to show your appreciation.

11 Comments

Leave a Reply
  1. Thanks, worked well for me! I just wanted to point out in this line from the article:

    /var/log/apache2/* {

    ..if you just use an asterisk * it seems like logrotate will rotate any files, even previously rotated files. To correct this, I had to use *.log.

    Reply
  2. Hi log rotate is not working, please help me.

    /opt/var/foo.log {
        copytruncate
        daily
        dateext
        rotate 3
        compress
        missingok
        size 10M
    }
    

    It is appending data on same file, now the file size reached 6GB how can I fix it

    Reply
  3. Hello Gabriel A. Cánepa,

    Are we allowed to add our own log path of apache2 here ? with new file on /etc/logrotate.d/mylog with my own log path ?

    Reply
  4. So i wasn’t sure, if i already have logrotate and it seems like it takes care of logs in /var/log/ folder, all of them seem to be numbered, dated, and stuff, so i am assuming it’s on and working, but what about manually specified folders, for example, all of my websites have their logs in their special folders, and not in /var/log/. How do i go about this? will simple directive on holding those logs in /var/log take care of log-rotation as well? i mean, if i set my website to keep its log in /var/log, do i need to worry about rotation, or will everything in that folder auto-rotate if logrotate is installed?

    a bit confusing, but basically trying to go the way of least resistance… =)

    Reply
  5. Hi, I am new to Linux and recently built a syslog server but the log file is not rotating. I have the entry below for the logrotate.conf file and the error I get when I try to force it to run using:

    # logrotate -vdf /etc/logrotate.conf
    

    Can you tell me what I need to change for the rotate to run properly.

    /var/log/Firewalls/firewall.log {
    
    daily
    rotate 30
    dateyesterday
    missingok
    compress
    postrotate
    systemctl restart rsyslog
    endscript
    }
    

    (It’s world writable or writable by group which is not “root”) Set “su” directive in config file to tell logrotate which user/group should be used for rotation.

    Reply
  6. Hi all,

    What about cases where the outdoor is never close. I had that case, it meant that the file ran full, i.e. the file system ran full, the file however showed only a small.size. A real problem when you don’t know. The file system says it’s full, but the sum of all file sizes is much smaller than the file system.
    Regards

    Reply
    • @Ravikumar,
      As explained in this very article, you can rotate the logs by using the same command as in the dry-run – just omit the -d option. Assuming you want to process the /var/log/squid/access.log file, do:
      logrotate /etc/logrotate.d/squid.conf
      Best,
      Gabriel

      Reply

Got Something to Say? Join the Discussion...

Thank you for taking the time to share your thoughts with us. We appreciate your decision to leave a comment and value your contribution to the discussion. It's important to note that we moderate all comments in accordance with our comment policy to ensure a respectful and constructive conversation.

Rest assured that your email address will remain private and will not be published or shared with anyone. We prioritize the privacy and security of our users.