[Gllug] HugePages

[Nix]

On 28 Oct 2010, James Courtier-Dutton told this:

> On 28 October 2010 19:54, Richard Jones <rich at annexia.org> wrote:
>> On Thu, Oct 28, 2010 at 07:06:45PM +0100, James Courtier-Dutton wrote:
>>> My understanding of hugepages is that you only use them if you
>>> really have to.  The reason to have hugepages is that the x86/x64
>>> CPU has a fixed size page table. I.e. How many pages it can refer
>>> to.  If you have more physical memory than the CPU can reference
>>> with normal sized pages, one has to then use hugepages.
[...]
> And your post is about as useful as a wet fish.

However, he's right: your post *was* a load of drivel. The advantage of
hugepages is nothing to do with the size of the page table, which is
expandable *and* multi-level.

What hugepages gives you is a reduction in the pressure on your
translation lookaside buffer, which *is* fixed-size, and not terribly
large (even the Nehalem's only has 64/128 entries in it, and earlier
generations have much smaller TLBs). Huge pages are good firstly because
you need many fewer TLBs ('cos the hugepages are 500 times the size of
conventional pages) and secondly because there's a small chunk of the
TLB set aside *only* for huge pages, which is hardly being used right
now. If the TLB isn't populated the machine has to dash out to L2/L3
cache or even main memory to reread the page tables all the time, which
for programs like GCC that allocate tens of thousands of pages and stamp
all over them pretty much at random, blowing the TLB to hell and back,
is a major cause of slowdowns. (You still need to read the page
directories with hugepages, but they're much smaller so more likely to
be in the L2 cache). Hugepages aren't that huge, unfortunately: 4Mb
isn't going to eliminate pagetable reads entirely, but AMD and (soon)
Intel will soon be providing 1Gb hugepages (needs a new CPU,
natch). With *those*, even a big GCC run will usually fit in one or two
pages, eliminating pagetable reads as a cause of slowdowns. Oh, it also
means you can often fit whole virtual machines in many fewer pages:
since more people run VMs than do massive compilations, this was the
real impetus behind making hugepages useful to things other than Oracle
at long last.

See e.g. <http://www.linux-kvm.org/wiki/images/9/9e/2010-forum-thp.pdf>.
-- 
Gllug mailing list  -  Gllug at gllug.org.uk
http://lists.gllug.org.uk/mailman/listinfo/gllug