[Sussex] SkyPickle stuff...

[Alan F]

On Wed, Dec 29, 2004 at 10:22:10PM +0000, Geoffrey Teale wrote:
> I can see why you ask the question.  To do a job like this normally I 
> would use Python, Ruby or even Bash.  Those tools make jobs like this 
> very easy to write and maintain.
 
I notice the absense of perl, it really seems to be a language people
love or absolutely hate to use. :-)

> A Bash script ends up just being very ugly and needing a lot of 
> parameters to cover even the basic set of options that Grumbleweed 
> supports.

I used to use this hideous script to summarise SA tests, you see why I
ended up writing something more pleasing...

grep SPAM-TAG /var/log/messages | tr ' ' '\012' | grep _ | grep -v
tagged_above | grep -v \<.*\> | tr -d 'tests=' | tr -d ',' | sort |
uniq -c | sort -n

> - C and C++ are at once my most treasured (and hard earned) skills and 
> the ones I have least opportunity to practice in the day to day.
> 
> - I find C a pleasant language to write in (something aesthetic that I 
> can't explain).
> 
> - The GNU "Languages of Choice" are C, C++, LISP (GUILE).

Good reasons methinks. perl is the only language I'm reasonably
competent with, so I don't have a great deal of choice, even though I
do like how simple and logical (IMO) it is to use.

I would like to learn C sometime, attempting to do so when I have
exams in January would not be in my best interests I suspect. :)

> 
> That approach is very flexible I don't agree about the low overhead 
> aspect of it though.  A database like postgreSQL is a hell of a thing to 
> be running on a system, and while they are very useful and flexible 
> relational databases are no where near the most efficient way or writing 
> a retrieving data.  With more configurable data gathering your system 
> would be fine for 99% of systems I can think of and much more useful as 
> a general purpose reporting system than Grumbleweed.
> 
I was thinking more about the incremental approach creating less
overhead with the parsing process, but true that a full featured
database like postgreSQL isn't the most efficient way of storing data
and then processing it at a later date. I always have postgreSQL
running on my server box, for many tasks, so that's not a big issue,
but I can see it might be for others.

The nice thing about perl's DBI is that with a simple modification of
the DSN, it'd work with SQLite, which is a nippy library for keeping a
database in a file (no server) and will perform most basic SQL queries.
With some modification a Berkely flatfile DB would work, but I'm going
in favour of flexibility rather than performance since the later isn't
a huge issue in the case of this tool.

Alan