[Gllug] lots of little files

[Minty]

On 10/15/05, Richard Jones <rich at annexia.org> wrote:
> On Sat, Oct 15, 2005 at 01:46:40AM +0100, Minty wrote:
> > I have a little script, the job of which is to create a lot of very
> > small files (~1 million files, typically ~50-100bytes each).
> I too would be seriously tempted to change to using a database.

The reason I didn't pick a database was that whilst I'll only be
generating them once or twice, they will get read a lot.  I grant a db
is probably going to be faster to generate, but could it be faster
than directly accessing a known path of a text file?

Hence optimise for the larger use.

> I hope the files aren't all in the same directory, or else ext3 is
> going to behave very badly.  Limit yourself to max 10,000 files /
> directory.

Nope, all nicely and evenly spread out.  No more than ~50 files to a directory.
-- 
Gllug mailing list  -  Gllug at gllug.org.uk
http://lists.gllug.org.uk/mailman/listinfo/gllug