[dundee] Noob Programming Question

[Robert McWilliam]

On Mon, 15 Mar 2010 15:37 +0000, "Gary Short" <gary at garyshort.org>
wrote:
> Hi Guys,
> 
> I was wondering if anyone here can give me some help. What I'm trying to
> do
> is to run curl and pipe the output to file1, then after a time t, stop
> curl
> and restart it piping it's output to file2 and so on to fileN, so that I
> can
> use another script to process the files, something like this...
> 

I'd use python for this. It has pretty good support for creating and
managing subprocesses, throw in time.sleep() for the waiting and the
whole thing is pretty straight forward. I could probably knock together
a script pretty quickly (I already have scripts doing similar things
where so let me know if you want that. 

> I also need a script to monitor this first one so that if it dies it can
> be
> restarted, something like...
> 
> If script_is_not_running:
>    Arg = digits_at_end_of_newest_file
>    Run script arg

Such a script is trivial in most languages. I'd use python again because
I find regexps to be a nightmare in bash. 

If I were writing this I'd actually give up on the whole idea of saving
and initialising the counter though and write a function to create the
next file by looking at which ones are there already (pretty much the
implementation of "Arg = digits_at_end_of_newest_file" from your
monitoring script) as that means that you don't have any state in the
script any more and the monitoring becomes a trivial infinite loop
launching the script again if it exits. It will mean more processing in
the script to go and calculate the counter each time but if the time
that curl is running is reasonably big, and you don't have a filesystem
with terrifying slow reads, then I wouldn't think it'll be an issue. 

You can use the daemon managing software to do the monitoring side of
things, it should have conventions to make it easy for you to have the
script report errors if it needs to be respawned. I think upstart was in
use by 8.04 so would the the Right Way to look after a daemon but if not
init scripts do work.  


Robert
________________________________________________________
Robert McWilliam     rmcw at allmail.net    www.ormiret.com

The opinions expressed herin are not necessarily those of my 
employer, not necessarily mine, and probably not necessary.