[Sussex] SkyPickle stuff...

[Geoffrey Teale]

Alan F wrote:
> I notice the absense of perl, it really seems to be a language people
> love or absolutely hate to use. :-)

I fall into the hate it category I'm afraid.  I learned it because I 
went to a couple of job interviews where the interviewer thought that 
Perl was synonymous with "Unix Programmer", having learnt it I decided 
that any job that forced me to do Perl programming was a job I'd rather 
not have ;-)

Having said that I do use `perl -p -i -e "s/foo/bar/g" ` type commands 
fairly regularly, mainly because my team leader at work was a Perl guy 
for 5 years and I've picked up a few of his habits, but I using Perl as 
a sed substitute really doesn't count ;-)

> I used to use this hideous script to summarise SA tests, you see why I
> ended up writing something more pleasing...
> 
> grep SPAM-TAG /var/log/messages | tr ' ' '\012' | grep _ | grep -v
> tagged_above | grep -v \<.*\> | tr -d 'tests=' | tr -d ',' | sort |
> uniq -c | sort -n

Yeah.. and you have to be so careful with uniq, you've got to do the 
sort beforehand or you get somewhat fruity behaviour.

> Good reasons methinks. perl is the only language I'm reasonably
> competent with, so I don't have a great deal of choice, even though I
> do like how simple and logical (IMO) it is to use.

One of the best pieces of advice I ever read (I think it was in Dr. 
Dobbs Journal) was to learn at least one new programming language every 
year.  I try to keep to that rule, sometimes I learn more than one, but 
always I try to seek out a language that will make me think about things 
in a different way.  There's always another way to do any job.  Whoops, 
sounding dangerously close to a Perl guru there ;-)

> I would like to learn C sometime, attempting to do so when I have
> exams in January would not be in my best interests I suspect. :)

Actually C is really easy to learn - and if you're programming on 
Unix/Linux already then everything works as you'd already expect it to. 
  C++ on the other hand is an infinite abyss of new things to learn.

> I was thinking more about the incremental approach creating less
> overhead with the parsing process, but true that a full featured
> database like postgreSQL isn't the most efficient way of storing data
> and then processing it at a later date. I always have postgreSQL
> running on my server box, for many tasks, so that's not a big issue,
> but I can see it might be for others.

Well, part of what I was saying is that most people would have no issue 
with running a database service.

> The nice thing about perl's DBI is that with a simple modification of
> the DSN, it'd work with SQLite, which is a nippy library for keeping a
> database in a file (no server) and will perform most basic SQL queries.
> With some modification a Berkely flatfile DB would work, but I'm going
> in favour of flexibility rather than performance since the later isn't
> a huge issue in the case of this tool.

I'm familiar with SQLite.  I'd favour using something like Berkely DB 
directly rather than querying anything using SQL if performance were the 
main concern.

All of that comes with the caveat that the circumstances where 
efficiency is _that_ important are very rare.  The ease of maintenance 
and general familiarity of a SQL based solution are major factors in 
their favour.

--
Geoff Teale
Free Software Foundation <tealeg at member.fsf.org>