[Nottingham] Eager Queues and GNU Hurd.

[Jason Cozens]

On Sat, 2007-01-20 at 17:19 +0000, Martin wrote:
> Jason Cozens wrote:

> > Solution
> > ========
> > 
> > A protocol that uses UDP is defined called the Eager Queue Protocol.
> > This protocol works in a similar way to a job exchange. Workers enter
> [...]
> > The protocol uses UDP a well established and widely available protocol.
> > The main benefit of the protocol will be to move away from a centralised
>  more powerful and expensive processors.
> > 
> > From initial investigations it look as though EQP will be able to
> > combine with the translators of GNU Hurd.
> 
> YEAH!!
> 
> My first thought is that there is an awful lot of _broadcasting_ going
> on with that lot that could then quickly swamp a network as you add more
> processors / processing elements. You also get more overhead with all
> the broadcast noise that everyone must listen to. A bit like controlling
> the job market with spam rather than filtering via recruiting agencies...?
> 
There is a lot of _broadcasting_ going on but this is intentional.

Firstly the idea would be to have scheduling on a private network
reserved exclusively for scheduling. To test this I'd like to get
some computers, put in an extra network card and write the scheduling
software.

The channel is very noisy so that processors can join quickly.
i.e. when the scheduling is idling a processor can join within
2 - 3 broadcast units of time. It listens for an update and joins
immediately following the update. I'm thinking that a processor
should be able to join the scheduling by immediately making a job
request.

Although there are a lot of broadcasts there won't be too many
collisions. The purpose of continually broadcasting the state of
the scheduling is that every scheduling processor knows which
processor is expected to broadcast next. This not only avoids
collisions but helps to isolate faulty processors.

Another point to take into account is where this would be used.
The intention is to use it where scheduling is required but
once a job has been scheduled the job's lifetime is quite long
compared to the time to schedule it. The scheduling should work
best where there are short bursts of scheduling. Since it is
intended to use this in a multiprocessor system scheduling is
only responsible for linking processors together and there is
no time slicing or context switching to be done.

> 
> More seriously:
> 
> I've long been uneasy about continuing to rely on one (big) monolithic
> kernel and then having to bolt on mutexes/spin-locks and various other
> mechanisms to mitigate/block races and confusion between multiple
> processors/processes all running on and fighting for the same one lump
> of hardware.
> 
> Meanwhile, the idea of the transputer came and went... And others and
> more esoteric such as various "active memory" and "processor element
> arrays" massively parallel systems...
> 
> And then there is the GNU Hurd...
> 
> 
> Jason, as mentioned Thursday, care to give a talk? (Please :-) )
> 
> For the non-techies: This is an interestingly different way to arrange
> what your CPU/hardware works on next. It's easy really!
> 
> Cheers,
> Martin

I'd like to give a talk on this. It would be best if it was towards the
end of Feb or beginning of March as I'd like to flesh the ideas out a
bit more and maybe get some code written.

Cheers,
Jason.

PS: Does anyone know a good reference on C and UDP?
I've only used UDP with C# so far and don't think a managed environment
is what I need here.