News for hackers. Stuff that matters.
Re:Ask HN: what software do you use for analyzing apache logs? by sam on Tuesday April 08, 2008 @01:25AM (#157599)
apachelog is a nice python module for parsing log lines from apache. it works as a great base for doing your own analysis. http://code.google.com/p/apachelog/

It's based on this perl module: http://cpan.uwinnipeg.ca/~peterhi/Apache-LogRegex

To manage log files, use cronolog

edit: I use cronolog to break up the logfiles daily and then I run a python script (which uses apachelog to handle the nasty parsing) to create a summary dictionary of parameters from that day. For example:

{num_unique_ips:140, num_pageviews:532, ...}

I pickle that dictionary and save it as a file. So every day has a raw log file and a "summary dictionary" file. To make plots I go through the summary files and unpickle them to extract the quantities of interest.

There are no comments on this comment yet.