[Gllug] Random freezes

Tue Feb 8 15:43:21 UTC 2005

On Sun, 6 Feb 2005, Nix wrote:

>>>(Note that while several different MMUs are used on SPARC-class boxes,
>>>the UltraSPARC MMUs are recognisably related to (some of) the SPARC
>>>ones, and many of the same weird features are still needed (IIRC, the
>>>biggest being software-invalidated caches and the lovely instructions
>>>that let you look at address spaces `as if' you were running in some
>>>other protection domain; but it's several years since I looked at this
>>>last...)
>>
>> Crikey, software invalidated cache is truly the spawn of satan.
>
>I dunno: you can do really wicked cool evil tricks with it...
>
>... however, nobody ever does, so everyone just gets the costs without
>the (questionable) benefits.
>
>Quite like register windows really. (You *could* have an infinitely
>large stack with it, thanks to the software window underflow/overflow
>traps... but nobody ever does it.)

Yeah, register windows are causing problems with the new threading
implementation on NetBSD 2.0 (typical, you advocate a platform, which then
procedes to bite you in the ass:)

Yet, despite the scalability problems SPARC has witnessed, largely because
of register windows, it amazes me that Itanic also uses register windows
on an even bigger scale than SPARC. Register windows are apparently really
difficult to implement in hardware in a fast way.

>
>>>So what's NetBSD got that Linux hasn't in this area? Where's Linux's MM
>>>`fundamentally weak'? (And which MM do you mean? 2.0, 2.2, 2.4 <2.4.10,
>>>2.4.x >2.4.10, and 2.6 are really quite different from each other in
>>>a number of fundamental ways.)
>>
>> The fact that you were asking for qualification on which version of the
>> Linux VM has the problem shows that it is not a particularly stable
>> abstraction.
>
>Actually, the VM itself stayed quite stable, modulo a rewrite in 2.4.10;
>it's the parameters that control things like memory balancing that have
>been tweaked like mad. (That's also where its weaknesses were: the
>inability pre-rmap to go from virtual to physical address really caused
>problems with things like swapping.)

rmap certainly makes a difference. It is amazing that so many people were
against rmap because of it's effect on micro-optimisations.

>
>>                                     It works alright in TLB based MMUs,
>> such as MIPS and SPARC V9, but doesn't really fit for inverted page table
>> based architectures.
>
>Well, it can work on them. It's just trickier. (Quite a bit trickier).

And for pmap, it's not an issue, as the mapping is handled behind the
scenes.

>
>> Page tables are largely a hardware notion.
>
>Linux page tables are divorced from the hardware: we just use the same
>terms as do the hardware people. s/page table/way of tracking a page-sized
>block of memory/ if you prefer.

Hmm, I was under the impression that they were same, hence the ability
of levels being folded down to match the hardware (such as PMD being an
identity level on x86.)

In that case, if there are seperate page tables, this is an even less
efficient use of memory as there is effectively duplicate information.

>
>>                                            Linux page tables are
>> overloaded with swap information, for example, making them impossible to
>> page out or discard.
>
>The inability to swap out page tables is definitely a failing, but it's
>hard to see how it could be fixed while retaining the ability to examine
>memory globally to search for things to swap out. I guess you'd have to
>impose a rule that says that page tables are swapped out when the pages
>they control have all been swapped out, and use something like swap
>clustering...

Who said swap out? Page tables should be discardable. There should really
be no information in there that can't be reconstructed from other
information.

>
>... but none of that requires moving the page flags out of `struct
>page'. Doing that would be hugely disruptive, actually...

Per-page flags are generally a function of the object mapping they are
part of. When you mmap a region, you generally do it in a uniform across
the whole region. Yes, there are exceptions, but these can be handled
cleanly by simply splitting up an existing region.

>
>>                      Not an issue for memory laden modern desktop, but an
>> issue for embedded or small machines.
>>                                       And page table reuse could even be a
>> win on memory rich machines, due to increased cache utilisation, though I
>> guess it would not be a big win.
>
>It's a *huge* win. The `struct page' uses up ~4% of all memory; that's a
>significant amount no matter how much memory you've got.
>
>> Mach/BSD pmap and SVR4 hat are much better MMU abstractions. Simple APIs
>> which hides *all* the details of the hardware MMU, on top of which truly
>> platform independent VM can be built.
>
>Downside: if you hide *all* the details then you can't build a VM that
>takes full advantage of the hardware.

MMUs provide a simple abstraction, that are represented quite well by a
simple API. They may virtual to physical addresses and implement security
policy. Details such as cache architecture can quite easily be hidden
behind such an API.

Details such as NUMA are difficult to abstract behind a MMU API, but it
should be easy to also provide a generalised NUMA API, which can be used
to direct the use of the generalised MMU API in a platform independent
manner.

Other optimisation beyond the above are probably a moot point, along the
lines of all the 'optimisations' available in a CISC instruction set (and
ignored by almost all compilers.)

>
>>                                       NetBSD had a completely new VM model
>> between 1.4 and 1.5 across all 12 CPU families then supported, possible
>> only because the VM model was completely MMU agnostic.
>
>That's hyperbole.
>
>Being completely VM agnostic is of course completely impractical: you
>still acknowledge the existence of swap, L2 cache, or at least staggered
>levels of storage of different speeds: that's the whole *point* of MM.

UVM is *MMU* agnostic. NetBSD isn't VM agnostic.

The virtual memory sub-system in NetBSD (and OpenBSD now) is UVM,
basically a simplified version of the origional Mach VM. But the MMU
agnostic pmap stayed the same. Hence, MachVM/UVM is MMU agnostic.

UVM details can be found here:
http://ccrc.wustl.edu/pub/chuck/tech/uvm

Christian

-- 
    /"\
    \ /    ASCII RIBBON CAMPAIGN - AGAINST HTML MAIL
     X                           - AGAINST MS ATTACHMENTS
    / \
-- 
Gllug mailing list  -  Gllug at gllug.org.uk
http://lists.gllug.org.uk/mailman/listinfo/gllug