[Gllug] Apache log files

william pink will.pink at gmail.com
Wed Apr 8 13:47:38 UTC 2009


On Wed, Apr 8, 2009 at 2:05 PM, Tim Schofield <tim at scoffer.net> wrote:

> On Tue, 2009-04-07 at 11:11 +0100, william pink wrote:
> > Hi,
> >
> > I have the rather horrible task of splitting up lots (40Gb's worth) of
> > Apache log files by date, the last time I did this I found the line
> > number I then tailed the file and outputted it into a new file which
> > was a long arduous task. I imagine this can be done in a few minutes
> > with some Regex/Sed/AwkBash trickery but I wouldn't know where to
> > start can anyone give me any pointers to get started?
> >
> > Thanks,
> > Will
>
> Hi
>
> time for i in {jan,feb,mar,apr,may,jun,jul,aug,sep,oct,nov,dec} ; do
> zcat access.*.gz | grep -i $i\/2008 | sort -k 4.9b,4.12bn -k 4.5b,4.7bM
> -k 4.2b,4.3bn -k 4.14b,4.15bn -k 4.17b,4.18bn -k 4.20b,4.21bn  >
> ${i}_split.txt; done
>
> This bash oneliner works for me please note the hardcoded 2008 in the
> grep statement. It also splits the data by month and produces a bunch of
> files based on  month_split.txt. Also note the zcat statement this has
> to produce a concat of your input files.  Not sure if this will be able
> to handle 40 gigs worth of data though
>
> Tim
>
> --
> Gllug mailing list  -  Gllug at gllug.org.uk
> http://lists.gllug.org.uk/mailman/listinfo/gllug
>


Hi,

Thanks but I decided to go through them manually with

grep -G "01\/Mar" access.log* >> access_march.log

I tried to adjust and use Tethys script but was only getting output files
with no content and just got far to frustrated trying to get it to work.

I am now the proud owner of classic shell scripting now so Tet can sleep
easier at night knowing I won't be posting anymore questions to the list
regarding shell scripting.

Regards,
Will
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.lug.org.uk/pipermail/gllug/attachments/20090408/0f82adc2/attachment.html>
-------------- next part --------------
-- 
Gllug mailing list  -  Gllug at gllug.org.uk
http://lists.gllug.org.uk/mailman/listinfo/gllug


More information about the GLLUG mailing list