[dundee] OpenWRT on a Linksys WAG354G

Mon Sep 29 17:08:06 UTC 2008

2008/9/29 Philipp Geyer <nistur at googlemail.com>:
> In terms of the version control and syncronisation...
> Yes, I do only have 2 peers (at the moment) but Hazel's wanting a
> similar solution too. Also, using my NAS drive on it's own would
> restrict us to not being able to have version control without having one
> of our PCs on.

Sorry, just to be clear (as I don't think I was), my point about peers
was meerly to indicate that it seems like you have peers, rather than
having some kinda master/slave scenario.  i.e. changes may occur on
either the laptop or the pc (or any of Hazels machines if you're
planning on sharing files).  rsync is more suited to master/slave
scenarios.

> What I've decided I'm going to do is set up the laptop with ssh, rsync,
> cvs and git (maybe some other random versioning systems) but primarily I
> will be using cvs while I get a hang of git and then I will probably
> replace the cvs repositories with git-cvsserver for the PS2s. For the
> syncronisation, I will probably stick with rsync (although I will be
> looking more into unison and having a fiddle) but in general I will only
> have the one changed copy as I very rarely have both machines on at the
> same time I will also be scripting backups, possibly daily, of the
> repositories, just in case I make a mess of things :)

Seems like a reasonably idea....  But you'll need to be *VERY* careful
with rsync...  And I'd urge you to think about this in terms of
traditional computer science concurrency and be VERY diligent about
how you edit and manage your files; otherwise you risk race
conditions, and loss of data.

There's a few strategies I could suggest:

1) Be 100% sure you NEVER EVER EVER make a change to a file in two
places.  In practice this means you need to remember when you edited
and last sync'd.  Also be careful about how you manage renames and
deletes.

2) Give up on the automatic bidirectional synchronisation of data
between peers, and do syncs in the other direction manually.  Instead
just use rsync to backup your PC and laptop automatically to your NAS
drive....  Each into a seperate directory.  If you want to share
working documents (i.e. ones that change) between peers then maintain
a seperate copy for each peer.  And manually manage the change sets.
The constraint here is that writes are only occuring in one direction.

3) Adopt solution 1, and realise that there are different kinds of
data, that need to be managed differently.  e.g. you might never edit
photographs or your music collection, but you want them to be sync'd
between all peers.  In these cases you're safe to do bidirectional
rsyncing (providing your data is immutable).  Be aware that if you
ever want to rename/delete these files then you'll effectively need to
reset everything.  Note, that all other (mutable) data types don't get
this privilige though!

4) If you require access to the data mirrors on all peers, acknowledge
that you can only safely get away with having read only mirrors of
other machines on other peers.  e.g. your laptop might pull your PC's
backup into a /pc directory.  NEVER push your laptops /pc directory
back up to the NAS (effectively treating it as a read only copy).

5) If 1, 2, 3 & 4 aren't good enough, then use Unison... and
occaisionally put up with it telling you when you need to merge
changes :-)

Anyway, all the best getting it setup, and if you have any questions
about getting started with git (everyone does) post them to the list
and I can probably help.

R.

> Rick Moynihan wrote:
>> 2008/9/28 Philipp Geyer <nistur at googlemail.com>:
>>
>>> Ok, sorry if it was a little unclear. The whole thing was because I'm
>>> trying to organise some sort of synchronisation of documents between my
>>> PC and my laptop, both running multiple OSs. If it isn't done
>>> automagically then it's unlikely to last all too long before I forget to
>>> use it and then gets abandonned.
>>>
>>
>> I know the problem, but without explicitly taking care to NEVER modify
>> the same or in the case of code related files on both peers, then I'm
>> not sure how realistic it is to expect this to happen automagically
>> without risking the loss of changes to your data.  I think you simply
>> need to pop up a unison GUI everytime either of your peers connect to
>> your network.  (see below)
>>
>>
>>> I would also like to have source control for any programming I do, the
>>> university has a CVS server on their "kydd" machine that we use for our
>>> ps2 work (the only version control the PS2s have on them) so I'm
>>> familiar with that, although we can't use it outside of the university
>>> very easily. I have also had SVN on our ubuntu router and had a little
>>> play around with that, although I normally used the CVS that was also on
>>> it for the sole reason that I could have all my work in one place, and
>>> access it from any machine I'm planning on using (including the
>>> university PS2s)
>>>
>>> I haven't looked into distrubuted version control much yet mainly
>>> because for any purposes I'm planning on using it for it seems a little
>>> complex.
>>>
>>
>> The perceived complexity of distributed versioning systems is a common
>> misconception.  The reason?  They don't enforce a particular topology
>> on you, so in terms of use cases they scale much better than
>> traditional client server ones.  You can literally run them in any
>> model, ranging from having no server, and simply maintaining
>> versions/branches/tags on a handful of files locally, to running them
>> in a client/server model like CVS/SVN, and then if you want there are
>> a variety of distributed models you can choose from too....
>>
>> Anyway I've no experience with git on Windows, so it may or may not be
>> suitable, and I don't know the status of the various ports...  What I
>> will say though, is that git repositories can pretend to be legacy cvs
>> servers with git-cvsserver.  This might be handy as if you were to go
>> distributed with your repo's you could still use the CVS client on the
>> PS2's to pull/push to the repo.  I hear mercurial/hg and bzr both have
>> pretty good windows support, so if you're interested in using
>> something better than CVS/SVN these might also be an option.... but
>> IMHO git is the daddy here =)
>>
>>
>>> Back to the synchronisation. I considered using CVS but constantly
>>> adding new files, deleting them and committing seemed like a lot of
>>> hassle. I looked at unison but it seemed to again not be precisely what
>>> I want. It seemed to be more peer to peer whereas I wanted a more
>>> definate client/server so I wouldn't have to have my PC on to have the
>>> newest documents on my laptop. Unison may be able to do this but from
>>> what I saw it looked more complicated to do something like this than it
>>> would using rsync.
>>>
>>
>> You're right that Unison is more p2p than rsync, but the problem is
>> that it sounds like you essentially want to have two peers.  As you
>> probably want to change documents on both the laptop and desktop and
>> sync files between them via the NAS device.  I'm sure this is possible
>> with rsync, but to me this seems more up Unison's street, as it can at
>> least provide a GUI or interface to let you manually resolve potential
>> conflicts....  i.e. what do you want to happen if you make two
>> different changes to the same file on both the laptop and the PC?  You
>> can obviously get around this issue by deciding on one to take
>> priority so you can automate it - but are you sure this'll be the
>> right behaviour for you, in every instance?
>>
>>
>>> Which leads me to the linksys router.
>>> As it's (currently) coping under our workload, we've indefinately
>>> retired the ubuntu box as it uses far more power than the tiny linksys,
>>> however it doesn't have any services we could use. Basically, I decided
>>> that what would work best was to get rsync running on a samba share on
>>> the router, if that would work. Possibly also some sort of version
>>> control to the same place, depending on whether the router can take it.
>>> However, I know next to nothing about the workings of the router and how
>>> one would go about setting the thing up, the link I sent had photos and
>>> diagrams of taking it apart and attaching a serial console to it...
>>> which I'm fairly sure would cause me to turn it into a odd shaped brick.
>>>
>>> I hadn't seen gibak (as I haven't yet looked at git much) but that's
>>> definately an option, although I don't really want to install cygwin to
>>> get the same thing working under windows... there appears to be a
>>> msys/mingw version...I'll have a look
>>>
>>> So yeah, I've babbled far too much again... meh, summary?
>>> 1) Would like to know if it's "safe" to use rsync(/git) on OpenWRT.
>>> 2) same as above but with CVS (still using the PS2s this year)
>>> 3) Are there any better suggestions than possibly burning out the router?
>>> 4) Can anyone lend a hand with the OpenWRT at all?
>>>
>>> Thanks again
>>>
>>
>>
>> I have no experience of OpenWRT, so can't really offer much advice
>> regarding it... My assumption has been that it can run Linux and GNU
>> tools...  So I'd assume that rsync, unison and git/svn/cvs etc...
>> would be able to work.  Unison shouldn't require samba, and I'd
>> imagine any rsync client for windows wouldn't need it either, though
>> you might want it simply to have easy access to the drive and it's
>> backups.
>>
>> Oh, and if you settle on using a versioning system alongside some
>> kinda rsync/unison strategy, then make sure that the repositories are
>> ignored by it, as the potential for conflict and serious damage to the
>> repo's internal integrity is too great.
>>
>> Let us know what route you go down and how you get on...
>>
>> R.
>>
>> _______________________________________________
>> dundee GNU/Linux Users Group mailing list
>> dundee at lists.lug.org.uk  http://dundee.lug.org.uk
>> https://mailman.lug.org.uk/mailman/listinfo/dundee
>> Chat on IRC, #tlug on dundee.lug.org.uk
>>
>
>
> --
> ------------------------------------
> Philipp Geyer (AKA Nistur)
> http://nistur.chaosnet.org
>
>
> _______________________________________________
> dundee GNU/Linux Users Group mailing list
> dundee at lists.lug.org.uk  http://dundee.lug.org.uk
> https://mailman.lug.org.uk/mailman/listinfo/dundee
> Chat on IRC, #tlug on dundee.lug.org.uk
>