[Gllug] Imposing a delay in a pipeline

[Walter Stanish]

> Does anyone know of a utility to impose a delay in a pipeline? I want to do:
> 	process_a | delay -5 | process_b
> 
> such that process_b receives data 5 seconds after it was emitted by
> process_a. Any suggestions?

Overall, the use of a pipe to execute multiple transactions in sequence
is something of a monolithic approach and will present limitations.

Whilst it *may* be the right thing to do, it quite possibly isn't the
most appropriate approach.

I would suggest taking a step back and reconsidering the design, starting
with questions like these:
 - How many transactions (max) in a queue?  If always small, maybe pipe
   aint right.
 - Is order critical?
 - Is your input always going to come from a single source, or is it coming
   from multiple sources? (eg: disparate network clients)
 - What happens if one or more of the transactions fail?
 - Is there the requirement for feedback (success/failure, or more detailed)
   on a per-transaction basis?
 - Is input or output likely to vary, either over time or on a 
   per-transaction basis? (Test transactions, detailed diagnostics, etc.)
 - How long does a transaction take to execute?  Will this vary?  What 
   happens at the end of the day when the peer or market shuts down?
 - Are you going to forward-estimate execution time vs. market time 
   remaining?

Thus, depending on the precise nature of the input and output, how much
you can control or modify each, and the overall environment expectations,
you may find that a pipe is simply not the right tool for the job.

If execution time will vary significantly (it probably won't) then you
could for example check out the smstools, hylafax or asterisk callfile
queuing code.  From memory all have excellent filesystem-based approaches
(you can sit this on a ramdisk, DRBD + DLM-capable filesystem on a ramdisk
for redundancy + speed, or SSD easy enough)...

- Walter
-------------- next part --------------
A non-text attachment was scrubbed...
Name: winmail.dat
Type: application/ms-tnef
Size: 3679 bytes
Desc: not available
URL: <http://mailman.lug.org.uk/pipermail/gllug/attachments/20100413/22b82a56/attachment.bin>
-------------- next part --------------
-- 
Gllug mailing list  -  Gllug at gllug.org.uk
http://lists.gllug.org.uk/mailman/listinfo/gllug