agaskar.com

How to logrotate your Drupal cron logs.

The ‘man logrotate’ command should actually give you a very accessible overview (unlike many other man pages) about how to use logrotate. However, if you’re like me and search google before looking at manpages, read on for detailed instructions. First off, I’m logging my drupal cron jobs using the following commands in crontab (‘crontab -l’ lists, ‘crontab -e’ edits with your default system editor — likely vim):

* * * * * /usr/bin/wget -O- http://agaskar.com/cron.php >> /var/log/drupal-agaskar-cron.log 2>&1

Why ‘wget’? Well, it’s simple and straightforward and seems to be how most people do drupal crons. It also is very clear about what site you’re executing a cron on. I believe that executing cron.php using PHP in command line mode *should* work just as well, in most cases, but I’m not doing it here. The

>> /var/log/drupal-agaskar-cron.log

causes all output from cron.php to be placed in a file named /var/log/drupal-agaskar-cron.log. There’s a possible gotcha here — if the command isn’t run as root, it’s unlikely that it will be able to write to /var/log. That means you should’ve ‘su’-ed or logged in as root (the latter is considered bad practice if you’re remoting) before editing the crontab file with ‘crontab -e’. The following

2>&1

means take the standard error (any errors from running the command, or STDERR, here represented by 2) and redirect (>&) it to the standard output (1) which we just set to be ‘» /var/log/drupal-agaskar-cron.log’. (You may be wondering — as I did, if it’s necessary to use ‘2»&1’ in order to make sure it appends — this does not appear to be the case, ‘>&’ is a handle-redirect, and not at all used like the ‘>’ (create) or ‘»’ append angle bracket operators. Since we already told STDOUT to append with ‘» /var/log/drupal-agaskar-cron.log’ we should be fine.). Read more about ‘2>&1’ here. Ok, so now we’re running cron every minute, and redirecting everything it outputs to a logfile. As you might imagine, this logfile is going to get pretty big. Here’s where logrotate comes in. logrotate configuration is handled in (for CentOS at least) the /etc/logrotate.conf file. Mine looks something like this:

# see "man logrotate" for details
# rotate log files weekly
weekly

# keep 4 weeks worth of backlogs
rotate 4

# create new (empty) log files after rotating old ones
create

# uncomment this if you want your log files compressed
compress

# RPM packages drop log rotation information into this directory
include /etc/logrotate.d

# no packages own wtmp -- we'll rotate them here
/var/log/wtmp {
    monthly
    create 0664 root utmp
    rotate 1
}

# system-specific logs may be also be configured here.

I found this surprisingly well commented and easy to read compared! Take the advice of the first comment and take a moment to look at ‘man logrotate’. You should see a number of examples that really clarify things. Essentially what this file says is:

# see "man logrotate" for details
# rotate log files weekly
weekly

By default, rotate all log files weekly (you can override this later)

# keep 4 weeks worth of backlogs
rotate 4

Again, by default, keep 4 rotate timespans — remember, we set the rotate timespan to default to weeks — of back logs. Thus, by default. we’ll have 5 weeks total of logs (in separate files). Logs older than 5 weeks will be deleted from the system.

# create new (empty) log files after rotating old ones
create

Makes a new file after rotation, instead of (presumedly) just waiting until the process that created the logfile creates a new one.

# uncomment this if you want your log files compressed
compress

This tells logrotate to gzip up all your ‘rotated’ log files — everything older than the current log file will be compressed. Obviously there is some server overhead in this compression (one day I’ll tell you about the time I ‘grep -R’-ed a massive web directory at the same time this process was occurring) but hey, it saves space and in our case — with a weekly rotate — it’s not happening all that often.

# RPM packages drop log rotation information into this directory
include /etc/logrotate.d

This tells logrotate, ‘there’s more logrotate configuration files in the directory /etc/logrotate.d’, which happens to be the default location most packages will install their custom logrotate commands. We *could* put *our* commands in there, but then they might be hard to find for whoever comes next — it won’t necessarily be clear if they’re part of a package or if they’re hand-written.

# no packages own wtmp -- we'll rotate them here
/var/log/wtmp {
    monthly
    create 0664 root utmp
    rotate 1
}

This tells logrotate to rotate the /var/log/wtmp file and includes some additional instructions:

monthly

Rotate /var/log/wtmp monthly instead of our default (weekly).

create 0664 root utmp

After rotating, create a new empty logfile (like default) BUT give it permission bit 0664, set the owner to root, and the group to utmp.

rotate 1

Only keep 1 rotate timespan (in this case, a month) of back logs. Since we had compress on by default, the single monthly backlog of wtmp will be gzipped.

# system-specific logs may be also be configured here.

Alright! That means it’s our time to shine. Let’s add in our Drupal log commands:

# rotate drupal logs
/var/log/drupal*log {
    daily
    missingok
}

Again, line-by-line

/var/log/drupal*log {

Here’s a set of instructions for any file that matches ‘/var/log/drupal*log’ — the wildcard means all our future logs that start with ‘drupal’ and end with ‘log’ will be included in this; with a separate set of backlogs for each file.

daily

Rotate the Drupal cron logs daily instead of weekly. (I’m really not sure I need a month of Drupal cron logs. Maybe you do — in that case, you can leave this command out).

missingok

If you don’t find any drupal*log files in /var/log — don’t worry! just keep going and do not register an error. I’m not sure we need this here, but this should hopefully prevent any errors from gumming anything else up in case we made some mistakes with our crontab commands or if someone deletes our file. } End the set of additional instructions for Drupal cron logs. That’s it! I’d recommend reading through the logrotate manpage so you understand exactly is happening before making any changes to *your* logrotate, especially since logrotate deletes files after a particular age — you wouldn’t want to make a mistake and delete all of your httpd logs along with your Drupal cron logs.