[Gllug] Imposing a delay in a pipeline

Kostas Georgiou k.georgiou at atreides.org.uk
Wed Apr 14 15:56:18 UTC 2010


On Tue, Apr 13, 2010 at 05:33:40PM +0100, - Tethys wrote:

> On Tue, Apr 13, 2010 at 5:34 PM, John G Walker
> <johngeoffreywalker at yahoo.co.uk> wrote:
> 
> > If you explained a bit more what you want to use this for then maybe
> > someone could use a bit of lateral thinking to solve your problem,
> 
> It's a feed of time-sensitive price information for high frequency
> trading. That information is obviously extremely valuable, and becomes
> progressively less valuable, the more out of date it is. So I want to
> be able to configure a delay, the magnitude of the delay depending on
> how much the consumer of the feed is paying.
> 
> The data is in the form of lines of ASCII text. At busy times, we're
> talking about thousands of lines per second. The incoming data is
> fairly continuous, but I might be able to tolerate quantizing the data
> for output so we output a bunch of lines once per second, for example.

Something like this more or less then?

from sys import stdin                                                                                          
from time import time, sleep                                                                                   
from Queue import Queue                                                                                        
from threading import Thread                                                                                   
                                                                                                               
class delaywriter(Thread):                                                                                     
  def __init__(self, delay):                                                                                   
    Thread.__init__(self)                                                                                      
    self.delay = delay                                                                                         
                                                                                                               
  def run(self):                                                                                               
    while True:                                                                                                
      data = q.get()                                                                                           
      while(data[0] + self.delay > time()):                                                                    
        sleep(time() - data[0] + self.delay)                                                                   
      print data[1],                                                                                           
      q.task_done()                                                                                            
                                                                                                               
if __name__ == '__main__':                                                                                     
                                                                                                               
  q = Queue(1000)                                                                                              
  t = delaywriter(5)                                                                                           
  t.setDaemon(True)                                                                                            
  t.start()
  for line in stdin:                                                                                           
    q.put((time(),line))                                                                                       
  q.join()
-- 
Gllug mailing list  -  Gllug at gllug.org.uk
http://lists.gllug.org.uk/mailman/listinfo/gllug




More information about the GLLUG mailing list