[Gllug] Difference between "buffers" and "cached"?

[Daniel P. Berrange]

On Fri, Jan 27, 2006 at 06:41:46AM +0000, John Winters wrote:
> I recently put an extra gig of RAM in my server bringing it to 1.5G, for
> no better reason than that the motherboard has three memory slots and
> 512M DIMMs seem to come free with the cornflakes these days.
> 
> I tested the new memory with memtest before I booted up and all seems to
> be fine.  Very quickly after the system started work I noticed that most
> of the memory had been swallowed up as "cached" when viewed in top, and
> that's how it seems to stay most of the time.  However, first thing in
> the morning I find that more of the memory (say half a gig) is taken up
> as "buffers" and only about 350M as "cached".  Then during the day
> "cached" drifts up and "buffers" drifts down.
> 
> Can anyone explain the difference between "cached" and "buffers" please?

Buffers basically refers to the kernel's caching of writes to block
devices. Buffering writes has many performance advantages - the application
writing is only blocked in kernel space for as long as is required to
copy the data to the buffer, it can then carry on while the kernel 
asynchronoously writes the data out to disk[1]. It also lets the kernel
do more 'intelligent' disk updates, by clustering separate write requests
together. So if the application calls 'write' 4 times in succession, 
instead of doing 4 writes to the block device, the kernel will cluster
them into a single write. 

You can get a reasonable indication of how much the buffering helps, by
mounting your filesystems with the 'sync'  flag, which tells the tell
kernel to send writes to disk straight away - hint: you won't want to 
keep the sync flag on because performance sucks ass.

Buffers are in the block device layer of the kernel; The cache meanwhile
is higher up in the filesystem (VFS) layer, and refers to caching of the
contents & metadata of files in RAM, so that read requests can be satisfying 
immediately rather than having to pull the data in from disk every time. 

Buffers can be discarded by the kernel once the write is complete, although
it'll typically not release the memory immediately since its stupid to 
repeatedly alloc & free memory if there's a reasonable chance more writes
will be coming along soon. The cache data will be kept around pretty much
forever, until an application runs which wants memory for its own use 
at which point the kernel will release some of the cache. Basically under
normal operation you should expect the kerenl to use pretty much all your
free RAM for buffers & cache, only releasing it when an application needs
it for 'real work'

There are all sorts of thresholds & tunables which control how aggresively
the kernel allocs & releases caches, to change the balance between good
throughput and good interactivity. If you  have a 2.6 kernel (and particularly
if its RHEL-4), then Neil Horman has written a very good document talking
about how the VM works & how you can tune it 

http://people.redhat.com/nhorman/papers/papers.html

Regards,
Dan.

[1] If the app writes huge amounts of data, it can eventually get blocked
    during writes when the kernel runs out of room for buffers and has to
    actually write some out immediatley.
-- 
|=-            GPG key: http://www.berrange.com/~dan/gpgkey.txt       -=|
|=-       Perl modules: http://search.cpan.org/~danberr/              -=|
|=-           Projects: http://freshmeat.net/~danielpb/               -=|
|=-   berrange at redhat.com  -  Daniel Berrange  -  dan at berrange.com    -=|
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 196 bytes
Desc: Digital signature
URL: <http://mailman.lug.org.uk/pipermail/gllug/attachments/20060127/32cf0d44/attachment.pgp>
-------------- next part --------------
-- 
Gllug mailing list  -  Gllug at gllug.org.uk
http://lists.gllug.org.uk/mailman/listinfo/gllug