[Gllug] HugePages

[Steve Parker]

On 29/10/10 00:28, James Courtier-Dutton wrote:
>
>> Does anybody have much experience with hugepages? This is RHEL5.3, on 2
>> x quad-core hyperthreaded Intel Xeon 5570s
>>
>>      
> It sounds to me that you are suffering from memory pressure.
> Some things to look for.
> 1) Apart from the JVMs, what other functions are using memory?
Not much, there's a little bit of Java using the remaining 12Gb of 
normal memory, but those, plus the OS etc, leave a good 3Gb of free 
memory; there's a small amount swapped out (no swapping happening) and 
small amounts (<1Gb) of cache/buffers
> 3) Memory fragmentation.
> This is one of the worst problems and it is a very difficult problem to fix.
>    
Indeed. I've learned a lot from /proc/buddyinfo, and wrote a small tool 
to graph it (which I must get around to putting online, it could be 
generally useful). I am seeing lots of 4Kb, 8Kb, 16Kb blocks free, but 
very few 1024Kb blocks. So memory is highly fragmented. However, 
/proc/buddyinfo only seems to tell us about non-huge pages.
> So, by using hugepages, you have just turned 8K of RAM resource hog
> into a 4M of RAM resource hog.
>
> So, this is one of the reasons why using too many hugepages is a bad idea.
>    
Can you elaborate on this? I haven't been able to find much about 
fragmentation in HugePages space, though I have been told that in RHEL53 
(2.6.18 plus various patches) days, hugepage support was much worse than 
it is now; I am inclined to test RHEL53 plus Oracle's new 2.6.32 kernel 
(http://www.oracle.com/us/technologies/linux/ubreakable-enterprise-kernel-linux-173350.html) 
with the same workload. I haven't been able to find anything concrete 
about hugepage fragmentation, or what can cause/cure it
> Java does in fact help you here, because although a native x86
> application cannot get out of the memory fragmentation problem
> described, the Java Memory manager can because it can move memory from
> one page to another without the Java application having to know. This
> is one of the advantages of a software VM over a hardware implemented
> VM.
>    
Indeed; and this is the first time I have dealt with JRockit,
> You will have to investigate where your memory pressure problem
> actually lies, and hopefully it is not due to (3) because fixing that
> generally means reworking the application.
>    
That does seem to be the most likely.
-- 
Gllug mailing list  -  Gllug at gllug.org.uk
http://lists.gllug.org.uk/mailman/listinfo/gllug