[Gllug] Advanced versions of cron for scheduling

[Peter Grandi]

>>> On Sat, 16 Jul 2005 09:56:53 +0100, Richard Jones
>>> <rich at annexia.org> said:

[ ... ]

rich> (1) Run one job straight after another/some others have
rich>     finished. [ ... ]
rich> (2) Run a job continually - after it finishes, schedule it to
rich>     run [ ... ]
rich> (3) Set a time limit on a job. [ ... ]
rich> (4) Running jobs regularly, but at odd intervals. [ ... ]
rich> (5) Keep the crontab file under CVS control. [ ... ]
rich> (6) Ensure only one job is running at once. [ ... ]

[ ... ]

rich> These jobs are user programs which must be run at
rich> particular times. The programs are used mainly to fetch
rich> stats from remote SOAP services and change bidding prices
rich> based on these stats. Since the remote services are
rich> failure-prone, we need to do things like handling jobs if
rich> they get "stuck" accessing the services, running bidding
rich> jobs only if stats jobs all succeed, not trying to run two
rich> bidding programs at the same time, and so on.

[ ... ]

rich> You've described the sorts of things we're doing at the
rich> moment.  It involves 10s of shell scripts and is ugly and
rich> unmaintainable. Much of it is basically there to work
rich> around limitations in cron.

>From this discussion it is not clear whether you should be using
an extended version of 'cron', of which there are many, or a job
scheduling system, of which there are also many or use both.

There are links to some popular job scheduling engines in the
SlashDot article you mention, and perhaps those should be looked
at again, incouding the one mentioned in this thread, GridEngine.

As to things like dependencies etc., before using shell scripts
I would look at 'make' or on of its alternative like 'icmake'.

Also, looking at your problem, perhaps reconsidering the problem
and/or the solution slightly might help, e.g. restructure things
so they are (as they can be if I understand your ultimate
requirements) a bit more ''idempotent''.

Ultimately you may have to think carefully about your problem
and perhaps write a custom ''driver'' for your transactions.

-- 
Gllug mailing list  -  Gllug at gllug.org.uk
http://lists.gllug.org.uk/mailman/listinfo/gllug