Extract bandwidth information from lighttpd log files

I was looking for a simple way to get some bandwidth statistics for websites that I host. Interested in historical data my only option was to look back over my webserver log files.

My webserver of choice on linux systems is currently lighttpd. Here’s a quick Bash script to get the bandwidth statistics out of the default lighttpd log files:

#!/bin/bash
cat access.log | awk '{
month=substr($4,5,3)
year= substr($4,9,4)
timstamp=year" "month
bytes[timstamp] += $10
} END {
for (date in bytes)
printf("%s %20d MB\n", date, bytes[date]/(1024*1024))
}' | sort -k1n -k2M

That will give you a table containing the stats based on the bytes sent for the body of the pages:

2011 Jan                  662 MB
2011 Feb                12090 MB
2011 Mar                13645 MB
2011 Apr                12274 MB
2011 May                12279 MB
2011 Jun                 9551 MB

This could be easily adapted to work out the number of hits or other information contained in the log files.