How to Setup and Manage Log Rotation Using Logrotate in Linux

One of the most interesting (and perhaps one of the most important as well) directories in a Linux system is /var/log. According to the Filesystem Hierarchy Standard, the activity of most services running in the system are written to a file inside this directory or one of its subdirectories.

Such files are known as logs and are the key to examining how the system is operating (and how it has behaved in the past). Logs are also the first source of information where administrators and engineers look while troubleshooting.

If we look at the contents of /var/log on a CentOS/RHEL/Fedora and Debian/Ubuntu (for variety) we will see the following log files and subdirectories.

Please note that the result may be somewhat different in your case depending on the services running on your system(s) and the time they have been running.

In RHEL/CentOS and Fedora

# ls /var/log
Log Files and Directories under CentOS 7

Log Files and Directories under CentOS 7

In Debian and Ubuntu

# ls /var/log
Log Files and Directories in Debian 8

Log Files and Directories in Debian 8

On both cases, we can observe that some of the log names end as expected in “log”, while other are either renamed using a date (for example, maillog-20160822 on CentOS) or compressed (consider auth.log.2.gz and mysql.log.1.gz on Debian).

This is not a default behavior based on the chosen distribution, but can be changed at will using directives in the configuration files, as we will see in this article.

If logs were kept forever, they would eventually end up filling the filesystem where /var/log resides. In order to prevent that, the system administrator can use a nice utility called logrotate to clean up the logs on a periodic basis.

In few words, logrotate will rename or compress the main log when a condition is met (more about that in a minute) so that the next event is recorded on an empty file.

In addition, it will remove “old” log files and will keep the most recent ones. Of course, we get to decide what “old” means and how often we want logrotate to clean up the logs for us.

Installing Logrotate in Linux

To install logrotate, just use your package manager:

---------- On Debian and Ubuntu ---------- 
# aptitude update && aptitude install logrotate 

---------- On CentOS, RHEL and Fedora ---------- 
# yum update && yum install logrotate

It is worth and well to note that the configuration file (/etc/logrotate.conf) may indicate that other, more specific settings may be placed on individual .conf files inside /etc/logrotate.d.

Suggested Read: Manage System Logs (Configure, Rotate and Import Into Database) Using Logrotate

This will be the case if and only if the following line exists and is not commented out:

include /etc/logrotate.d

We will stick with this approach, as it will help us to keep things in order, and use the Debian box for the following examples.

Options

Being a very versatile tool, logrotate provides plenty of directives to help us configure when and how the logs will be rotated, and what should happen right afterwards.

Let’s insert the following contents in /etc/logrotate.d/apache2.conf (note that most likely you will have to create that file) and examine each line to indicate its purpose:

apache2.conf
/var/log/apache2/* {
    weekly
    rotate 3
    size 10M
    compress
    delaycompress
}

The first line indicates that the directives inside the block apply to all logs inside /var/log/apache2:

  1. weekly means that the tool will attempt to rotate the logs on a weekly basis. Other possible values are daily and monthly.
  2. rotate 3 indicates that only 3 rotated logs should be kept. Thus, the oldest file will be removed on the fourth subsequent run.
  3. size=10M sets the minimum size for the rotation to take place to 10M. In other words, each log will not be rotated until it reaches 10MB.
  4. compress and delaycompress are used to tell that all rotated logs, with the exception of the most recent one, should be compressed.

Let’s execute a dry-run to see what logrotate would do if it was actually executed now. Use the -d option followed by the configuration file (you can actually run logrotate by omitting this option):

# logrotate -d /etc/logrotate.d/apache2.conf

The results are shown below:

Rotate Apache Logs with Logrotate

Rotate Apache Logs with Logrotate

Instead of compressing the logs, we could rename them after the date when they were rotated. To do that, we will use the dateext directive. If our date format is other than the default yyyymmdd, we can specify it using dateformat.

Suggested Read: Install ‘atop’ to Monitor Logging Activity of Linux System Processes

Note that we can even prevent the rotation from happening if the log is empty with notifempty. In addition, let’s tell logrotate to mail the rotated log to the system administrator ([email protected] in this case) for his / her reference (this will require a mail server to be set up, which is out of the scope of this article).

If you want to get mails about logrotate, you can setup Postfix mail server as show here: Install Postfix Mail Server

This time we will use /etc/logrotate.d/squid.conf to only rotate /var/log/squid/access.log:

squid.conf
/var/log/squid/access.log {
    monthly
    create 0644 root root
    rotate 5
    size=1M
    dateext
    dateformat -%d%m%Y
    notifempty
    mail [email protected]
}

As we can see in the image below, this log did not need to be rotated. However, when the size condition is met (size=1M), the rotated log will be renamed access.log-25082016 (if the log was rotated on August 25, 2016) and the main log (access.log) will be re-created with access permissions set to 0644 and with root as owner and group owner.

Finally, when the number of logs finally reaches 6, the oldest log will be mailed to [email protected].

Rotate Squid Logs with Logrotate

Rotate Squid Logs with Logrotate

Now let’s suppose you want to run a custom command when the rotation takes place. To do that, place the line with such command between the postrotate and endscript directives.

For example, let’s suppose we want to send an email to root when any of the logs inside /var/log/myservice gets rotated. Let’s add the lines in red to /etc/logrotate.d/squid.conf:

squid.conf
/var/log/myservice/* {
	monthly
	create 0644 root root
	rotate 5
	size=1M
    	postrotate
   		echo "A rotation just took place." | mail root
    	endscript
}

Last, but not least, it is important to note that options present in /etc/logrotate.d/*.conf override those in the main configuration file in case of conflicts.

Logrotate and Cron

By default, the installation of logrotate creates a crontab file inside /etc/cron.daily named logrotate. As it is the case with the other crontab files inside this directory, it will be executed daily starting at 6:25 am if anacron is not installed.

Suggested Read: 11 Cron Scheduling Task Examples in Linux

Otherwise, the execution will begin around 7:35 am. To verify, watch for the line containing cron.daily in either /etc/crontab or /etc/anacrontab.

Summary

In a system that generates several logs, the administration of such files can be greatly simplified using logrotate. As we have explained in this article, it will automatically rotate, compress, remove, and mail logs on a periodic basis or when the file reaches a given size.

Just make sure it is set to run as a cron job and logrotate will make things much easier for you. For more details, refer to the man page.

Do you have any questions or suggestions about this article? Feel free to let us know using the comment form below.

Best Affordable Linux and WordPress Services For Your Business
Outsource Your Linux and WordPress Project and Get it Promptly Completed Remotely and Delivered Online.

If You Appreciate What We Do Here On TecMint, You Should Consider:

  1. Stay Connected to: Twitter | Facebook | Google Plus
  2. Subscribe to our email updates: Sign Up Now
  3. Get your own self-hosted blog with a Free Domain at ($3.45/month).
  4. Become a Supporter - Make a contribution via PayPal
  5. Support us by purchasing our premium books in PDF format.
  6. Support us by taking our online Linux courses

We are thankful for your never ending support.

Gabriel Cánepa

Gabriel Cánepa is a GNU/Linux sysadmin and web developer from Villa Mercedes, San Luis, Argentina. He works for a worldwide leading consumer product company and takes great pleasure in using FOSS tools to increase productivity in all areas of his daily work.

Your name can also be listed here. Got a tip? Submit it here to become an TecMint author.

RedHat RHCE and RHCSA Certification Book
Linux Foundation LFCS and LFCE Certification Preparation Guide

You may also like...

10 Responses

  1. Sarjit Singh says:

    How can I setup logrotate to rotate the logs hourly?
    There is no option and minimum frequency is daily.

  2. Siva says:

    Hi log rotate is not working, please help me.

    /opt/var/foo.log {
        copytruncate
        daily
        dateext
        rotate 3
        compress
        missingok
        size 10M
    }
    

    It is appending data on same file, now the file size reached 6GB how can I fix it

  3. Kumar P says:

    Hello Gabriel A. Cánepa,

    Are we allowed to add our own log path of apache2 here ? with new file on /etc/logrotate.d/mylog with my own log path ?

  4. youtube to mp3 says:

    So i wasn’t sure, if i already have logrotate and it seems like it takes care of logs in /var/log/ folder, all of them seem to be numbered, dated, and stuff, so i am assuming it’s on and working, but what about manually specified folders, for example, all of my websites have their logs in their special folders, and not in /var/log/. How do i go about this? will simple directive on holding those logs in /var/log take care of log-rotation as well? i mean, if i set my website to keep its log in /var/log, do i need to worry about rotation, or will everything in that folder auto-rotate if logrotate is installed?

    a bit confusing, but basically trying to go the way of least resistance… =)

  5. K Sinclair says:

    Hi, I am new to Linux and recently built a syslog server but the log file is not rotating. I have the entry below for the logrotate.conf file and the error I get when I try to force it to run using:

    # logrotate -vdf /etc/logrotate.conf
    

    Can you tell me what I need to change for the rotate to run properly.

    /var/log/Firewalls/firewall.log {
    
    daily
    rotate 30
    dateyesterday
    missingok
    compress
    postrotate
    systemctl restart rsyslog
    endscript
    }
    

    (It’s world writable or writable by group which is not “root”) Set “su” directive in config file to tell logrotate which user/group should be used for rotation.

  6. fatboy92 says:

    Hi all,

    What about cases where the outdoor is never close. I had that case, it meant that the file ran full, i.e. the file system ran full, the file however showed only a small.size. A real problem when you don’t know. The file system says it’s full, but the sum of all file sizes is much smaller than the file system.
    Regards

  7. Ravikumar says:

    How to rotate logs manually. If I want to rotate the log files now, what to do?

    • Gabriel A. Cánepa says:

      @Ravikumar,
      As explained in this very article, you can rotate the logs by using the same command as in the dry-run – just omit the -d option. Assuming you want to process the /var/log/squid/access.log file, do:
      logrotate /etc/logrotate.d/squid.conf
      Best,
      Gabriel

    • gasmyr says:

      logrotate -f myConfig_file, have a look in man page of logrotate(man logrotate)

Leave a Reply to Sarjit Singh Cancel reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.