Never Ending Security

It starts all here

Postscreen-Stats – A simple script to parse Postscreen logs

Postscreen Statistics Parser

Simple script to compute some statistics on Postfix/Postscreen Activity Run it against your postfix syslogs

Published under GPL v2

    parses postfix logs to compute statistics on postscreen activity

usage: -f mail.log

  -a|--action=   action filter with operators | and &
                      ex. 'PREGREET&DNSBL|HANGUP' = ((PREGREET and DNSBL) or HANGUP)
                      ex. 'HANGUP&DNSBL|PREGREET&DNSBL' 
                          = ((HANGUP and DNSBL) or (PREGREET and DNSBL)

  -f|--file=     log file to parse (default to /var/log/maillog)

  -g            /!\ slow ! ip geoloc against (default disabled)

  --geofile=    path to a maxmind geolitecity.dat. if specified, with the -g switch
                the script uses the maxmind data instead of (faster)

  -G            when using --geofile, use the pygeoip module instead of the GeoIP module

  -i|--ip=      filters the results on a specific IP

  --mapdest=    path to a destination HTML file that will display a Google Map of the result
                  /!\ Require the geolocation, preferably with --geofile

  --map-min-conn=   When creating a map, only map the IPs that have connected X number of times

  -r|--report=  report mode {short|full|ip|none} (default to short)

  -y|--year=    select the year of the logs (default to current year)

  --rfc3339     to set the timestamp type to "2012-04-13T08:53:00+02:00" instead of the regular syslog format "Oct 23 04:02:17"

example: $ ./ -f maillog.3 -r short -y 2011 --geofile=../geoip/GeoIPCity.dat -G --mapdest=postscreen_report_2012-01-15.html

Julien Vehent (!j) -

Basic usage

Generate a report form a syslog postfix log file. If you are parsing logs from a year that is not the current year, use the -y option to specify the year of the logs.

$ python -f maillog.1 -r short -y 2011
=== unique clients/total postscreen actions ===
2131/11010 CONNECT
463/536 DNSBL
305/503 HANGUP
1884/2258 NOQUEUE 450 deep protocol test reconnection
1/42 NOQUEUE too many connections
1577/1600 PASS NEW
866/8391 PASS OLD
181/239 PREGREET

=== clients statistics ===
4 avg. dnsbl rank
505 blocked clients
2131 clients
840 reconnections
32245.4285714 seconds avg. reco. delay

=== First reconnection delay (graylist) ===
delay| <10s   |>10to30s| >30to1m| >1to5m | >5to30m|>30mto2h| >2hto5h|>5hto12h|>12to24h| >24h   |
count|12      |21      |21      |196     |261     |88      |40      |29      |53      |119     |
   % |1.4     |2.5     |2.5     |23      |31      |10      |4.8     |3.5     |6.3     |14      |

Get the statistics for a specific IP only

$ python -f maillog.1 -r ip -i
Filtering results to match:
    connections count: 2
    first seen on 2011-10-22 09:37:54
    last seen on 2011-10-22 09:38:00
    DNSBL count: 1
    DNSBL ranks: ['6']
    HANGUP count: 2

Geo Localisation of blocked IPs

There are 3 GeoIP modes:

  1. Use online geoip service. This is free but slow and not very accurate
  2. Use Maxmind’s GeoIP database. You can use either the free version of the DB from their website, or get a paid version.

To use, just set the -g option. To use maxmind, set the --geofile to point to your Maxmind DB (ie. --geofile=/path/to/GeoIPCity.dat) By default, geofile use the GeoIP python module, but if you prefer to use pygeoip instead, set the -G option as well.

$ ./ -r short --geofile=../geoip/GeoIPCity.dat -G -f maillog.3 -y 2011


=== Top 20 Countries of Blocked Clients ===
 167 (33.00%) United States
  59 (12.00%) India
  33 ( 6.50%) Russian Federation
  26 ( 5.10%) Indonesia
  23 ( 4.60%) Pakistan
  21 ( 4.20%) Vietnam
  20 ( 4.00%) China
  13 ( 2.60%) Brazil
  11 ( 2.20%) Korea, Republic of
   9 ( 1.80%) Belarus
   8 ( 1.60%) Turkey
   7 ( 1.40%) Iran, Islamic Republic of
   7 ( 1.40%) Ukraine
   6 ( 1.20%) Kazakstan
   6 ( 1.20%) Chile
   5 ( 0.99%) Italy
   5 ( 0.99%) Romania
   4 ( 0.79%) Poland
   4 ( 0.79%) Spain
   3 ( 0.59%) Afghanistan

Geo IP database installation

Using the MaxMind free database at 1. Download the database and extract GeoLiteCity.dat at the location of your choice 2. install the GeoIP maxmind package # aptitude install python-geoip 3. launch postscreen_stats with --geofile="/path/to/geolistcity.dat"

Google Map of the blocked IPs

You can use the --mapdest option to create an HTML file with a map of the blocked IPs.

$ ./ -f maillog.3 -r none -y 2011 --geofile=../geoip/GeoIPCity.dat -G --mapdest=postscreen_report_2012-01-15.html

Google map will be generated at postscreen_report_2012-01-15.html
using MaxMind GeoIP database from ../geoip/GeoIPCity.dat
Creating HTML map at postscreen_report_2012-01-15.html

If you have a lot of IPs to map, you can use --map-min-conn to only map IPs that connected X+ number of times.

./ -f maillog.3 -y 2011 -g --geofile=../geoip/GeoIPCity.dat -G --mapdest=testmap.html --map-min-conn=5

More information can be found on:


Leave a Reply

Please log in using one of these methods to post your comment: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s