[Gllug] Apache log files

[william pink]

On Wed, Apr 8, 2009 at 3:02 PM, Peter Corlett <abuse at cabal.org.uk> wrote:

> On Wed, Apr 08, 2009 at 02:49:15PM +0100, John Hearns wrote:
> [...]
> > That's more like it. But I can still make out access.log - so I can
> figure
> > out it does something with access logs. More compression! More confusion!
>
> Nah, that's a terrible idea. The original, non-obfuscated version I slapped
> together in a few minutes is this one, which should be more obvious:
>
> #!/usr/bin/env perl
> use warnings;
> use strict;
>
> my($fh, $curpath);
>
> while(<>) {
>  my($day, $month, $year) = (m~\[(..)/(...)/(....):~);
>  die "Regex match failed" unless defined $1; # regex sanity check
>  my $path = "$year-$month-$day.access.log";
>
>  unless($fh && $curpath eq $path) {
>    $curpath = $path;
>    open $fh, '>', $path or die "Can't create $path: $!";
>  }
>  print $fh $_;
>
>  # uncomment if you want progress reports
>  #print STDERR "[$.] $curpath\r" unless $. % 1e4;
> }
>
> __END__
>
> (And this is a proper, solid, efficient script, unlike some of the crap
> that
> has been posted so far.)
>
> Change the $path assignment and maybe the date-extracting regex to taste. I
> ran it like so to test:
>
> zcat -f apache2/*access* | ./logsplit.pl
>
> ... and it ripped through 80MB of test logs in three seconds. So 40GB will
> probably take of the order of an hour. (The main constraint is going to be
> disk performance.)
>
> --
> Gllug mailing list  -  Gllug at gllug.org.uk
> http://lists.gllug.org.uk/mailman/listinfo/gllug
>

That looks great but I am using the timestamps in the actual logfile (which
from what I can work out that script uses the date in the log file name but
that maybe a error of judgment?

I have just ordered Perl for system administration also as I think Perl is
another thing I should really learn.

Thanks,
Will
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.lug.org.uk/pipermail/gllug/attachments/20090408/9eaca503/attachment.html>
-------------- next part --------------
-- 
Gllug mailing list  -  Gllug at gllug.org.uk
http://lists.gllug.org.uk/mailman/listinfo/gllug