[Gllug] 4G Memory Restriction

[Daniel P. Berrange]

On Mon, Apr 10, 2006 at 11:00:01AM +0100, Steve Nelson wrote:
> Hi all,
> 
> I wanted to check my understanding on a matter before I write some
> documentation on Linux memory management.
> 
> I've currently stated:
> 
> """4GB is the maximum any OS can address in 32-bits.
> 
> However, Linux (and other OS) have strategies to cope with >4GB memory.
> 
> Current Linux 2.4 & 2.6 series, RedHat compiled, kernels on IA-32
> hardware support >4GB memory by default.
> 
> Current Linux 2.4 & 2.6 series, RedHat compiled, "huge memory" kernels
> (kernel-hugemem) on IA-32 support >16GB memory.
> 
> Earlier versions of RedHat's OS had the option of "big memory" kernels
> ( 4GB< kernel-bigmem <16GB) this is no longer the case as the required
> patches are included in the default kernels (both uni &
> multi-processor flavours)."""
> 
> Any inaccuracies here, or is this pretty spot on?  Anything I've missed out?

Its close to correct, but it could be clarified a little more.

 - The Intel 32-bit x86 cpus can normally address upto 4 GB of 
   physical RAM

 - To allow addressing > 4GB of physical RAM, Intel introduced 
   PAE mode - Physical Address Extension, which IIRC increased
   the address space to 40-bits.

 - PAE allow the OS to deal with >4 GB RAM, but at any time, only
   4 GB can actually be mapped

 - Normally the Linux address space is split 1 GB for kernel, and 
   3 GB for userspace 

Now, wrt to the various RHEL kernels..

  - RHEL-2.1 / 3  UP are not PAE enabled, and thus limit to 4 GB.
 
  - RHEL-2.1 SMP was not PAE enabled, and thus limited to 4 GB

  - RHEL-2.1 Enterprise kernel was PAE enabled, however, due to
    the 3/1 GB split, you can only usefully support upto 16 GB
    of memory. If you installed > 16 GB, the page tables were
    soo large, that the kernel will frequently OOM due to its
    1 GB space limit. Even 16 GB is pushing it, if you have an
    driver that needs alot of LowMem.
  
  - RHEL-3 SMP was PAE enabled, but still had the 3/1 GB split.
    So RHEL-3 SMP was more or less same model as RHEL-2.1 Enterprise

  - RHEL-3 Hugemem, was PAE enabled, but also switched to a 4 GB/ 4 GB
    kernel / userspace memory model split. Now remember even with 
    PAE, x86 can only have 4 GB mapped at any time. So with the 4-4
    split, every time you do a system call the pagetables must be
    switched. Since the kernel now has its own 4 GB to play with, 
    you can install > 16 GB of RAM and still operate without risk
    of OOM

So, in summary once with PAE mode, you have an effective 40 bit address
space so any kernel supporting PAE can *theoretically* support > 4 GB.
Due to the Liunux address space model, however, you can be *practically*
limited to a smaller amount, predicated on your ability to maintain the
page table structure. You can find more tech details on the LKML message
where Ingo posted the original 4/4 GB patches

  http://lwn.net/Articles/39283/

Regards,
Dan.
-- 
|=-            GPG key: http://www.berrange.com/~dan/gpgkey.txt       -=|
|=-       Perl modules: http://search.cpan.org/~danberr/              -=|
|=-           Projects: http://freshmeat.net/~danielpb/               -=|
|=-   berrange at redhat.com  -  Daniel Berrange  -  dan at berrange.com    -=|
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 196 bytes
Desc: Digital signature
URL: <http://mailman.lug.org.uk/pipermail/gllug/attachments/20060410/a7681b93/attachment.pgp>
-------------- next part --------------
-- 
Gllug mailing list  -  Gllug at gllug.org.uk
http://lists.gllug.org.uk/mailman/listinfo/gllug