[Gllug] lots of little files

Minty mintywalker at gmail.com
Sat Oct 15 00:46:40 UTC 2005


I have a little script, the job of which is to create a lot of very
small files (~1 million files, typically ~50-100bytes each).

I'm running on a 500Mhz box PII box with a new 7,200rpm ATA disk.
Debian Stable / Sarge. Ext3.  It's a old compaq desktop.

I'm massively limited by the disk IO.

So...

without spending loadsa cash on shiny raid / sata / scsi disks,
without reformatting the disk and loosing all the nice other stuff on there
without re-installing a different kernel etc

is there anything sensible I can do to tweak the disk IO a bit?

It's a bit of a one-off (or twice, maybe) script, and currently due to
finish in about 15 hours, hence why I don't want to spend too much
effort on rebuilding the box.  Would rather take the chance to maybe
learn something useful about tuning...

The operations are basically

10 stat path to check if dir exists
20 mkdir if not
30 write a bunch of little files to the dir
40 goto 10

ps. got samba working, thanks for peoples help : just needed to
convince the wireless router not to block the various ports.
-- 
Gllug mailing list  -  Gllug at gllug.org.uk
http://lists.gllug.org.uk/mailman/listinfo/gllug




More information about the GLLUG mailing list