[Gllug] File synchronisation

[pauln at truemesh.com]

On Thu, May 09, 2002 at 02:57:20PM +0000, Bruce Richardson wrote:
 
> Unless you can guarantee that no two machines will write to the same file 
> between syncs, you are heading for a horrible mess.  As long as you can 

That's what I was fearing, I'm working within the current set up of how
the application works (not my design...)

Trying to fix a current single point of failure for the data source.  

> Even then, creating a system of 
> synchronisation across multiple boxes that doesn't either silt up with 
> obsolete files or erroneously delete live ones is extremely ticklish.

unison has some quite nice merge stuff which would allow me to say most
recent wins.  The split node thing is horrible with each node needing
n-1 connections.  I think most ppl would add a db tier here, but I'm
discouraged from suggesting that. 

> You might find that your determination to avoid a filesystem causes more 
> problems than it solves.  In which case you might benefit from looking at 
> WebDAV, a virtual filesystem that runs over http. 

I was thinking of either dav or Berkeley DB.  How would you deal with
writes here - each one writing to a servername based dir and sharing via
dav, or with a master.  

I guess with dav you'd have locking and versioning, but I'm unclear on
how you do that in a cluster. How do you share the dav info across your
cluster. I'd rather not have the application probing all the servers for
live servers.  You also get the problem if one node can recieve external
data but not talk to other nodes (eg internal interfaces blown), it
shouldn't write, etc.

DB does transactions, client->master promotion via elections, etc but
would require code changes, etc and unsure about storing xml feeds, etc.

Zope has some good use cases about this stuff.

http://dev.zope.org/Wikis/DevSite/Projects/ZEOReplicatedStorage/UseCases

Yes, I know it's messy and non-trivial, dav would be great
(could eventually play with http://rproxy.sf.net and
http://www.ietf.org/rfc/rfc3229.txt), but can't see how to do either
master takeover or distributed server. 

Paul
 

-- 
Gllug mailing list  -  Gllug at linux.co.uk
http://list.ftech.net/mailman/listinfo/gllug