[Nottingham] Downloading multiple images from a website

[Michael Quaintance]

Robert Hart said:
> On most shells (e.g. bash) "echo" is a "builtin", which means the shell
> bypasses running the "echo" program you have in /bin, and does it
> directly.

That makes sense. Does the builtin nature also stop the commandline
getting too long? 1000 URLs of approx 30 chars each plus a bit of overhead
for the initial wget options is more than I expect I could legally type as
a commandline.

> Also running wget (I think... curl certainly does this) with a list of
> urls from the same site means it will use a single TCP/IP connection,
> and download multiple files in one go.

I know from experiments I did last night, wget will reuse the connection
if it can.

> Note you could use
>
> $ curl http://blah/[0-999].jpeg -O --limit-rate 3k
>
> Which would do it all in one go, but there would be no delay, instead
> the connection is throttled to 3kb/s

This was almost what I was trying to get wget to do but it doesn't support
globbing for http transfers. As long as I rate-limit it, that seems fair
to the website owner. I have curl installed on my WinNT :-( box here so I
have the man page equivalent easily available but it is not on the machine
at home I want to use for the transfers. I should have thought of that
instead of being blinkered into only looking at wget.

Thanks. The curl solution is almost exactly what I wanted to do in the
first place.

-Penfold.