[Gllug] Imposing a delay in a pipeline
Kostas Georgiou
k.georgiou at atreides.org.uk
Wed Apr 14 15:56:18 UTC 2010
On Tue, Apr 13, 2010 at 05:33:40PM +0100, - Tethys wrote:
> On Tue, Apr 13, 2010 at 5:34 PM, John G Walker
> <johngeoffreywalker at yahoo.co.uk> wrote:
>
> > If you explained a bit more what you want to use this for then maybe
> > someone could use a bit of lateral thinking to solve your problem,
>
> It's a feed of time-sensitive price information for high frequency
> trading. That information is obviously extremely valuable, and becomes
> progressively less valuable, the more out of date it is. So I want to
> be able to configure a delay, the magnitude of the delay depending on
> how much the consumer of the feed is paying.
>
> The data is in the form of lines of ASCII text. At busy times, we're
> talking about thousands of lines per second. The incoming data is
> fairly continuous, but I might be able to tolerate quantizing the data
> for output so we output a bunch of lines once per second, for example.
Something like this more or less then?
from sys import stdin
from time import time, sleep
from Queue import Queue
from threading import Thread
class delaywriter(Thread):
def __init__(self, delay):
Thread.__init__(self)
self.delay = delay
def run(self):
while True:
data = q.get()
while(data[0] + self.delay > time()):
sleep(time() - data[0] + self.delay)
print data[1],
q.task_done()
if __name__ == '__main__':
q = Queue(1000)
t = delaywriter(5)
t.setDaemon(True)
t.start()
for line in stdin:
q.put((time(),line))
q.join()
--
Gllug mailing list - Gllug at gllug.org.uk
http://lists.gllug.org.uk/mailman/listinfo/gllug
More information about the GLLUG
mailing list