[Gllug] Lots of computing power required - is Amazon EC2 an answer?

[Andrew Farnsworth]

Interesting, assuming linear acceleration across parallel computing, lets do
some math...

1 dual core AMD CPU (assume ~2.4 Ghz) = 20 days
1 Quad core AMD CPU 2.4 Ghz = 10 days
2 Quad core AMD CPU 2.4 Ghz = 5 days
4 Quad core AMD CPU 2.4 Ghz = 2.5 days
8 Quad core AMD CPU 2.4 Ghz = 1.25 days

This means that you should be able to run the entire data set over a weekend
using 8 Quad Core machines, or 16 dual core machines. I
would definitely contact your school about a condor pool. Even a small one
would make this very manageable.

Also, many of the systems shipped by HP and other server companies are now
shipping quad socket six core CPUs.  With one of these beasts you could also
run it over a weekend pretty easily.  If you have any contacts in industry,
you might ask about and see if someone has a server (or servers) being
burned in you could use for a weekend.

Andy



On Thu, Jun 30, 2011 at 6:05 PM, Adrian McMenamin
<adrian at mcmen.demon.co.uk>wrote:

> I have struggled to find an algorithmic way out of this - but cannot see
> one.
>
> I have some code which is very parallelisable but requires huge computing
> resources - on a AMD 64 dual box I have here I reckon it will take about
> 20 days to run - but if I had access to about 600 CPUs it might take 20 -
> 30 minutes (it's run against a massive XML file - 800 MB or so - with
> different parameters)
>
> I am also doing this for a student project so have no big budget to spend
> - but would be willing to spend some money.
>
> Is Amazon EC2 a possible answer - it seems less than clear to me if this
> is about web serving or computing - so I thought I'd ask where people will
> know the answer straight away :)
>
> --
> Gllug mailing list  -  Gllug at gllug.org.uk
> http://lists.gllug.org.uk/mailman/listinfo/gllug
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.lug.org.uk/pipermail/gllug/attachments/20110701/5dd11833/attachment.html>
-------------- next part --------------
--
Gllug mailing list  -  Gllug at gllug.org.uk
http://lists.gllug.org.uk/mailman/listinfo/gllug