[Gllug] Re: Memory usage
Nix
nix at esperi.org.uk
Thu Oct 13 11:51:37 UTC 2005
On Wed, 12 Oct 2005, Peter Grandi stipulated:
> Also, my KDE performance improvements notes:
>
> http://WWW.sabi.co.UK/Notes/linuxKDE.html#performance
... contain some... debatable statements.
> The KDE developers have also managed by and large to restrain themselves
> from making a whole mess like GNOME, and to maintain some degree of
> consistency and discipline in the implementation.
Strongly agreed! The number of things shared between GNOME apps seems
to be going down over time, not up: GNOME makes almost no use of Bonobo
anymore, for instance, so there's almost no embeddable stuff around,
and lots of things have invented their own plugin systems.
> * The extensive use of shared code means most processes running KDE
> applications
An incisive point missing a clause. :)
> * The widget set is Qt, which unfortunately is written in C++, which
> is a terribly complicated language with terribly complicated
> implementation issues, in particular relating to the ABI and
> linking.
`Relink everything when libstdc++ is upgraded' is not very difficult
advice to follow. We change libstdc++'s soname precisely to force people
to do this. (It would be nice if libstdc++'s ABI didn't break so often,
but until support for separate compilation exists in G++, which isn't
going to happen soon, we're stuck with it.)
> By and large these issues have been minimized by not
> maintaining binary compatibility between major versions of KDE, and
That has nothing whatsoever to do with the implementation language; just
to do with the Qt API being intentionally broken by Trolltech every so
often (pretty rarely, actually). (Glib/Gtk's problem is the converse:
they have horrible design flaws which they're marking WONTFIX simply
because fixing them would involve ABI breakage.)
> making this feasible by having only a few major libraries.
That doesn't have anything to do with the implementation language
either. (In fact, using larger libraries probably *amplifies*
unavoidable link-time relocation delays, even in prelinked binaries,
because of the two non-prelinkable relocations per C++ virtual method
table in the cross-platform C++ ABI.)
> C++
> complexity however has been a significant drain on both development
> complexity and performance.
My understanding was that it had made things *easier*. You don't *have*
to use features in C++ that you don't like: you can use as many as you
need and no more.
> * The system at runtime not only relies on C++, but also extensively on
> shared libraries. Unfortunately existing shared library implementations
> are fairly terrible and not designed for lots of large shared libraries,
> C++ has very poor support for shared library based programming, and the
> performance implications of shared libraries are usually bad and even
> worse unless considerable care is taken.
This assertion has no basis in reality, or is based on *very* old
(pre-2000).
ld-linux.so.2's handling of systems with lots of shared libraries is
excellent (witness the number of libraries used by, say, GnuCash or
Mozilla). In fact the limiting factor here was the kernel until 2.4.10
or thereabouts: nobody imagined that you'd need thousands of VMAs so
nobody had optimized for that case.
As for large shared libraries, well, yes, if you don't prelink something
with a lot of symbols and those symbol names are long (as C++'s
generally are), you will see slow statup times. This is an unfixable
consequence of ELF. You're not recommending we shift away from ELF,
surely? 'Cos everyone will laugh at you if you do. Since prelinking
makes this `problem' completely vanish, and you can run prelink
from cron, I'd call this a problem that isn't.
The only other negative performance implications of shared libraries are
the need to apply PLT relocations at runtime (a few dozen clock ticks
the first time a method is called is lost in the noise, and you can turn
off the laziness if need be), and the need to produce PIC code, which
eats a register. Alas, not only is this unfixable but it's true of *all*
shared libraries, and even of executables now that PIE is coming in;
everyone agrees that the benefits of massive memory reduction and of
easy upgrades militate *strongly* in favour of shared libraries. The
real solution to the PIC speed problem is to move to a system with a
bigger register file, like x86-64.
In practice, we see a 5%-10% reduction in speed of CPU-intensive code in
shared libraries on x86, almost entirely due to the PIC-register-file
problem and GCC's crap register allocator. That's about the speed you
gain in a couple of months, given Moore's Law. i.e., not significant.
> * Many developers do not really understand the UNIX philosophy and as
> a result the system and its implementation have an increasingly
> Microsoft style feel, down to the use of .INI file syntax.
The problem here, I think, is that the Unix philosophy has never really
been properly applied to graphical user interfaces; we have no graphical
analogue of the pipe, for instance.
In my experience the KDE guys have a hell of a lot more respect for the
Unix philosophy than have the GNOME guys, who actively go around
recommending that we write code like Microsoft. (No thanks. I'll leave
the massive security holes, ridiculously entangled and API-bloated code
and insane rates of API churn to the Windows people.)
> * Many developers seem to have gotten nice jobs related to the KDE, and
> I suppose this made them keener to make it popular as opposed to being
> done well. The major way to get something popular is to parrot Microsoft
> Windows, and as a result much of the KDE development seems to aim to
> develop cheap, cool demo-like special effects more than refactoring or
> documenting at least the basics.
This cheap slam against aseigo almost doesn't deserve rebuttal, but, um,
you complain in one place about people not rethinking things but copying
Microsoft, and now you complain that people are rethinking things. The
whole idea of the Appeal project is to try something *different*, not
least since X is now allowing things like efficient transparency, and
may soon allow efficient use of OpenGL as well. If the `something
different' turns out to suck, well, it was an experiment...
As for refactoring, what do you think is *happening* with KDE 4?
Refactoring. Lots of it. (You slammed *that* earlier. Not very
consistent, are you?)
Oh, and lack of docs is not really a problem with KDE. Too many docs for
any human being to read in a lifetime almost might be :)
> * Several details are ill thought out, mostly I think because not all
> the developers can be bothered to think things through. For example the
> font selection dialogue does not allow specifying many important
> attributes of which font one wants to select, most importantly which
> font system the font should be rendered with.
That's because core fonts are considered obsolete by the X developers
(as well as the KDE and GNOME teams). (Proper internationalization
also effectively requires use of fontconfig.)
I'd agree that it would be very nice to be able to specify a complete
font name rather than disparate pieces. This is something to bug
the fontconfig people about, probably.
> * Konqueror not growing endlessly due to extremly poor memory
> allocation practices.
I've seen no sign of this. If you're using an old G++, you might see
this, as pre-G++-3.4, the pool allocators preferered speed over size and
had a couple of unfortunate bugs causing rather a lot more fragmentation
than should be the case. G++-3.4 fixes these bugs.
(And exactly *what* `extremely poor memory allocation practices'? It
lets you control the amount of caching it does itself, and sensibly
assumes that Qt/libstdc++/malloc are reasonably efficient. Yes, it
allocates a lot of small objects. So do X, Emacs, GNOME, the kernel, and
virtually every other program of any size.)
> * Konqueror not giving free rein to embedded media and to scripts and
> thus making browsing everything hudeously slow while they are running.
You can turn them all off on a site-by-site basis or across all sites.
I find myself wondering if you've even looked in the Konq prefs
dialog...
> * Being able to specify in the GUI where to put the Internet cache directory.
Personally I don't use it, but agreed. (Does any web browser allow this?)
> * Fix whatever makes Konsole take up several dozen megabytes of resident memory.
Amortized across multiple processes, it doesn't. You're forgetting
sharable mappings, and nearly all the memory consumption of konsole is
sharable mappings.
PID RSS VSZ
11570 16640 31376
11582 15868 31188
11589 17980 34856
11609 15372 32120
11631 18404 35080
11639 14384 31164
11647 15464 32580
11654 15256 32124
11796 14612 31448
822 14764 31268
26732 17504 32336
Let's pick on my largest konsole, PID 11631 (which incidentally has about
thirty tabs open, each with 500 lines of scrollback: running thirty xterms
would eat *far* more memory).
11631: konsole -session 10db616465000112250294900000015330024_1128641379_400723
08048000 4K r-x-- /usr/packages.bin/kdebase/3.4-050913/bin/konsole
08049000 28K rw--- /usr/packages.bin/kdebase/3.4-050913/bin/konsole
08050000 6900K rw--- [ anon ]
41000000 32K r-x-- /lib/libgcc_s.so.1
41008000 4K rw--- /lib/libgcc_s.so.1
4100b000 96K r-x-- /usr/packages.bin/kdelibs/3.4-050911/lib/libkdesu.so.4.2.0
41023000 4K rw--- /usr/packages.bin/kdelibs/3.4-050911/lib/libkdesu.so.4.2.0
41030000 68K r-x-- /usr/packages.bin/zlib/1.2.3/lib/libz.so.1.2.3
41041000 4K rw--- /usr/packages.bin/zlib/1.2.3/lib/libz.so.1.2.3
41075000 796K r-x-- /usr/packages.bin/gcc/3.4.4/lib/libstdc++.so.6.0.3
4113c000 20K rw--- /usr/packages.bin/gcc/3.4.4/lib/libstdc++.so.6.0.3
41141000 20K rw--- [ anon ]
41148000 436K r-x-- /usr/packages.bin/freetype/2.1.9/lib/libfreetype.so.6.3.7
411b5000 28K rw--- /usr/packages.bin/freetype/2.1.9/lib/libfreetype.so.6.3.7
411be000 68K r-x-- /usr/X11R6/lib/libXft.so.2.1.2
411cf000 4K rw--- /usr/X11R6/lib/libXft.so.2.1.2
411d2000 144K r-x-- /usr/packages.bin/libpng/1.2.8/lib/libpng12.so.0.1.2.8
411f6000 4K rw--- /usr/packages.bin/libpng/1.2.8/lib/libpng12.so.0.1.2.8
411f9000 184K r-x-- /usr/packages.bin/libidn/0.5.18/lib/libidn.so.11.5.11
41227000 4K rw--- /usr/packages.bin/libidn/0.5.18/lib/libidn.so.11.5.11
4122a000 204K r-x-- /usr/packages.bin/kdelibs/3.4-050911/lib/libDCOP.so.4.2.0
4125d000 8K rw--- /usr/packages.bin/kdelibs/3.4-050911/lib/libDCOP.so.4.2.0
4125f000 4K rw--- [ anon ]
41262000 2252K r-x-- /usr/packages.bin/kdelibs/3.4-050911/lib/libkdecore.so.4.2.0
41495000 68K rw--- /usr/packages.bin/kdelibs/3.4-050911/lib/libkdecore.so.4.2.0
414a6000 8K rw--- [ anon ]
414aa000 168K r-x-- /usr/packages.bin/kdelibs/3.4-050911/lib/libkdefx.so.4.2.0
414d4000 4K rw--- /usr/packages.bin/kdelibs/3.4-050911/lib/libkdefx.so.4.2.0
414d5000 4K rw--- [ anon ]
414d8000 64K r-x-- /usr/packages.bin/kdelibs/3.4-050911/lib/libkwalletclient.so.1.0.1
414e8000 4K rw--- /usr/packages.bin/kdelibs/3.4-050911/lib/libkwalletclient.so.1.0.1
41534000 832K r-x-- /usr/packages.bin/kdelibs/3.4-050911/lib/libkdeprint.so.4.2.0
41604000 36K rw--- /usr/packages.bin/kdelibs/3.4-050911/lib/libkdeprint.so.4.2.0
4160f000 580K r-x-- /usr/packages.bin/kdebase/3.4-050913/lib/libkdeinit_konsole.so
416a0000 32K rw--- /usr/packages.bin/kdebase/3.4-050913/lib/libkdeinit_konsole.so
419f6000 32K r-x-- /usr/packages.bin/fam/2.7.0/lib/libfam.so.0.0.0
419fe000 4K rw--- /usr/packages.bin/fam/2.7.0/lib/libfam.so.0.0.0
41f48000 2860K r-x-- /usr/packages.bin/kdelibs/3.4-050911/lib/libkdeui.so.4.2.0
42213000 172K rw--- /usr/packages.bin/kdelibs/3.4-050911/lib/libkdeui.so.4.2.0
4223e000 4K rw--- [ anon ]
42813000 8424K r-x-- /usr/packages.bin/Qt/3.3.5/lib/libqt-mt.so.3.3.5
4304d000 256K rw--- /usr/packages.bin/Qt/3.3.5/lib/libqt-mt.so.3.3.5
4308d000 16K rw--- [ anon ]
43093000 3272K r-x-- /usr/packages.bin/kdelibs/3.4-050911/lib/libkio.so.4.2.0
433c5000 120K rw--- /usr/packages.bin/kdelibs/3.4-050911/lib/libkio.so.4.2.0
433e3000 8K rw--- [ anon ]
47d78000 84K r-x-- /lib/ld-2.3.5.so
47d8d000 4K r---- /lib/ld-2.3.5.so
47d8e000 4K rw--- /lib/ld-2.3.5.so
47d91000 1128K r-x-- /lib/tls/libc-2.3.5.so
47eab000 8K r---- /lib/tls/libc-2.3.5.so
47ead000 8K rw--- /lib/tls/libc-2.3.5.so
47eaf000 8K rw--- [ anon ]
47eb3000 140K r-x-- /lib/tls/libm-2.3.5.so
47ed6000 4K r---- /lib/tls/libm-2.3.5.so
47ed7000 4K rw--- /lib/tls/libm-2.3.5.so
47eda000 8K r-x-- /lib/tls/libdl-2.3.5.so
47edc000 4K r---- /lib/tls/libdl-2.3.5.so
47edd000 4K rw--- /lib/tls/libdl-2.3.5.so
47ee0000 804K r-x-- /usr/X11R6/lib/libX11.so.6.2
47fa9000 16K rw--- /usr/X11R6/lib/libX11.so.6.2
47faf000 56K r-x-- /usr/X11R6/lib/libXext.so.6.4
47fbd000 4K rw--- /usr/X11R6/lib/libXext.so.6.4
47fd3000 56K r-x-- /lib/tls/libpthread-2.3.5.so
47fe1000 4K r---- /lib/tls/libpthread-2.3.5.so
47fe2000 4K rw--- /lib/tls/libpthread-2.3.5.so
47fe3000 8K rw--- [ anon ]
47fe7000 84K r-x-- /usr/X11R6/lib/libICE.so.6.3
47ffc000 4K rw--- /usr/X11R6/lib/libICE.so.6.3
47ffd000 8K rw--- [ anon ]
48001000 32K r-x-- /usr/X11R6/lib/libSM.so.6.0
48009000 4K rw--- /usr/X11R6/lib/libSM.so.6.0
48082000 120K r-x-- /usr/packages.bin/expat/1.95.8/lib/libexpat.so.0.5.0
480a0000 8K rw--- /usr/packages.bin/expat/1.95.8/lib/libexpat.so.0.5.0
480a4000 324K r-x-- /usr/X11R6/lib/libXt.so.6.0
480f5000 16K rw--- /usr/X11R6/lib/libXt.so.6.0
480f9000 4K rw--- [ anon ]
4812c000 28K r-x-- /usr/X11R6/lib/libXrender.so.1.2.2
48133000 4K rw--- /usr/X11R6/lib/libXrender.so.1.2.2
48136000 8K r-x-- /usr/X11R6/lib/libXinerama.so.1.0
48138000 4K rw--- /usr/X11R6/lib/libXinerama.so.1.0
4814f000 32K r-x-- /usr/X11R6/lib/libXcursor.so.1.0.2
48157000 4K rw--- /usr/X11R6/lib/libXcursor.so.1.0.2
4815a000 12K r-x-- /usr/X11R6/lib/libXrandr.so.2.0
4815d000 4K rw--- /usr/X11R6/lib/libXrandr.so.2.0
4823f000 116K r-x-- /usr/packages.bin/libjpeg/6b/lib/libjpeg.so.62.0.0
4825c000 4K rw--- /usr/packages.bin/libjpeg/6b/lib/libjpeg.so.62.0.0
48365000 84K r-x-- /usr/X11R6/lib/libaudio.so.2.3
4837a000 4K rw--- /usr/X11R6/lib/libaudio.so.2.3
48ee4000 164K r-x-- /usr/packages.bin/libcms/1.14/lib/liblcms.so.1.0.14
48f0d000 8K rw--- /usr/packages.bin/libcms/1.14/lib/liblcms.so.1.0.14
48f0f000 8K rw--- [ anon ]
4f953000 84K r-x-- /usr/packages.bin/libart_lgpl/2.3.17/lib/libart_lgpl_2.so.2.3.17
4f968000 4K rw--- /usr/packages.bin/libart_lgpl/2.3.17/lib/libart_lgpl_2.so.2.3.17
b7ba6000 32K r-x-- /lib/tls/libnss_files-2.3.5.so
b7bae000 4K r---- /lib/tls/libnss_files-2.3.5.so
b7baf000 4K rw--- /lib/tls/libnss_files-2.3.5.so
b7bbc000 60K r---- /usr/X11R6/lib/X11/fonts/TrueType/VeraSeBd.ttf
b7bcb000 60K r---- /usr/X11R6/lib/X11/fonts/TrueType/VeraBd.ttf
b7bda000 60K r---- /usr/X11R6/lib/X11/fonts/TrueType/VeraSe.ttf
b7be9000 380K r-x-- /usr/packages.bin/libmng/1.0.9/lib/libmng.so.1.0.0
b7c48000 12K rw--- /usr/packages.bin/libmng/1.0.9/lib/libmng.so.1.0.0
b7c4b000 36K r-x-- /usr/X11R6/lib/X11/locale/lib/common/xomGeneric.so.2
b7c54000 4K rw--- /usr/X11R6/lib/X11/locale/lib/common/xomGeneric.so.2
b7c55000 68K r---- /usr/X11R6/lib/X11/fonts/TrueType/Vera.ttf
b7d67000 204K r-x-- /usr/packages.bin/kdelibs/3.4-050911/lib/kde3/plugins/styles/keramik.so
b7d9a000 8K rw--- /usr/packages.bin/kdelibs/3.4-050911/lib/kde3/plugins/styles/keramik.so
b7d9c000 108K r-x-- /usr/X11R6/lib/X11/locale/lib/common/ximcp.so.2
b7db7000 8K rw--- /usr/X11R6/lib/X11/locale/lib/common/ximcp.so.2
b7db9000 1824K r---- /usr/share/locale/locale-archive
b7f81000 12K rw--- [ anon ]
b7f84000 160K r-x-- /usr/packages.bin/fontconfig/2.3.2/lib/libfontconfig.so.1.0.4
b7fac000 20K rw--- /usr/packages.bin/fontconfig/2.3.2/lib/libfontconfig.so.1.0.4
b7fb1000 28K rw--- [ anon ]
b7fbf000 28K r---- /usr/X11R6/lib/X11/fonts/misc/6x10.pcf.gz
b7fc6000 16K r-x-- /usr/packages.bin/Qt/3.3.5/lib/Qt/plugins/imageformats/libqmng.so
b7fca000 4K rw--- /usr/packages.bin/Qt/3.3.5/lib/Qt/plugins/imageformats/libqmng.so
b7fcb000 8K r-x-- /usr/lib/gconv/ISO8859-1.so
b7fcd000 4K r---- /usr/lib/gconv/ISO8859-1.so
b7fce000 4K rw--- /usr/lib/gconv/ISO8859-1.so
b7fcf000 4K r---- /usr/share/locale/en_GB/LC_MESSAGES/libc.mo
b7fd0000 8K r-x-- /usr/X11R6/lib/X11/locale/lib/common/xlcDef.so.2
b7fd2000 4K rw--- /usr/X11R6/lib/X11/locale/lib/common/xlcDef.so.2
bf9bd000 88K rw--- [ stack ]
ffffe000 4K ----- [ anon ]
Adding up the stack and anonymous areas (mmap()ed heap and brk() areas) gives us:
08050000 6900K rw--- [ anon ]
41141000 20K rw--- [ anon ]
4125f000 4K rw--- [ anon ]
414a6000 8K rw--- [ anon ]
414d5000 4K rw--- [ anon ]
4223e000 4K rw--- [ anon ]
4308d000 16K rw--- [ anon ]
433e3000 8K rw--- [ anon ]
47eaf000 8K rw--- [ anon ]
47fe3000 8K rw--- [ anon ]
47ffd000 8K rw--- [ anon ]
480f9000 4K rw--- [ anon ]
48f0f000 8K rw--- [ anon ]
b7f81000 12K rw--- [ anon ]
b7fb1000 28K rw--- [ anon ]
bf9bd000 88K rw--- [ stack ]
ffffe000 4K ----- [ anon ]
That's 7132Kb not shared between konsoles, and this konsole has been
running for weeks with dozens of tabs and heavy activity so is about as
bloated as you'll ever see. I don't exactly consider this a bloated
application. Most of those mappings are shared not just across all
konsoles but across *every KDE app you're running*. So this is not
a hog at all.
xterm actually allocates substantially more *nonsharable* state than
konsole does.
> * Fix whatever makes Konsole consume 40% of CPU time on a 2GHz CPU
> when displaying the output of programs that generate is very quickly.
The major causes of this are probably transaprent backgrounds (not
efficiently implementable on X until X.org 6.9.0 / Qt 4) and the
non-optimized state of the RENDER extension (partially fixed
in X.org 6.9.0).
> * Many core applications are styled as special ones, not applications;
> for example they don't have obvious options to quit, for example the
> Kicker and KDesktop. This is very un-UNIXy.
The kicker has a huge blatant obvious quit button (well, it's labelled
`log out' by default but you can change that; it's assumed that newbies
will likely not be running kicker on its own but as part of a complete
desktop via startkde).
The desktop doesn't have a quit option because it doesn't make much
sense to run it *except* via startkde, whereupon it will die when X is
shut down.
> * Most applications lack man-pages, or options at all.
The lack of manpages sucks (they should take them from Debian and I
think are doing so). The lack of options is a vague generalization
without backing in reality.
--
`Next: FEMA neglects to take into account the possibility of
fire in Old Balsawood Town (currently in its fifth year of drought
and home of the General Grant Home for Compulsive Arsonists).'
--- James Nicoll
--
Gllug mailing list - Gllug at gllug.org.uk
http://lists.gllug.org.uk/mailman/listinfo/gllug
More information about the GLLUG
mailing list