Pre-processor for pflogsumm or awstats

Introduction

The preprocessor script could be of help if you use:

perl is a prerequisite (lastest version was tested with perl 5.8.8)

The script sanitizes mail log files that are to be passed to pflogsumm or awstats disregarding the lines relating to the re-injection of messages into postfix. I use it with amavisd-new configured as a postfix after queue content filter which does anti spam checks through spamassassin and antivirus checks through a third party virus checker. If your postfix setup requires additional content filters resulting in 3 or more passages of the same message through postfix, this script will not work for you.

I developed it for my own use, so testing is limited to my own server and with small volumes. It should be considered a beta version. If you use it and have feedback, let me know how you got on. I'd be particularly interested to know if it scales to larger installations and whether the logic behind it holds up in general. You might want to use it on a test server before installing in a production environment.

AN ALTERNATIVE WAY TO SOLVE THE DOUBLE COUNTING PROBLEM is to use multiple postfix instances each with a different syslog name. In that way you can process only the log entries from one of the instances via pflogsumm. Although it may take a bit more work to set up, it is a more robust method since it does not rely on updates to prepflog if log file formats change and you are not parsing the log file twice.

Usage

Use prepflog.pl -help for the options.

Typical usage with pflogsumm:

prepflog.pl -d yesterday /var/log/mail | pflogsumm

Typical usage with awstats:

In awstats configuration file set

LogFile="/usr/local/bin/prepflog.pl < /var/log/mail | perl /usr/local/awstats/tools/maillogconvert.pl standard |"

Note for Debian users (courtesy of Corné van Eeden)

Debian users: make sure the libdate-calc-perl package is installed.

Usage on debian with default amavis and postfix installation (Tested on Sarge):

LogFile= "/usr/local/bin/prepflog.pl < /var/log/mail.info | perl /usr/share/doc/awstats/examples/maillogconvert.pl standard |"

Updated version for pflogsumm 1.1.3

prepflog version 0.4 is the update for the ISO date parsing in pflogsumm 1.1.3

Updated version for postfix 2.3

prepflog version 0.3 is the update for the changed logging format of postfix 2.3 and contains a bug fix for 0.2 which did not work correctly in parsing the new log format. Pflogsumm versions earlier than 1.1.1 require a small patch to take into account the DSN logging and delays logging of postfix 2.3. patch to pflogsumm 1.1.0

Obtaining and installing

Download

Latest version

prepflog-0.4.tgz
GPG signature for prepflog-0.4.tgz
John Fawcett public key

Previous version

prepflog-0.3.tgz
GPG signature for prepflog-0.3.tgz
John Fawcett public key

Installation

Contact

john at gufonero.com

Last modified Tuesday, 20-Apr-2010 08:10:19 CEST