[Gllug] Debian 32/64 bitness

[Nix]

On Thu, 13 Oct 2005, Peter Grandi said:
> In theory, in theory indeed, one can install a many-platform
> system, using somthing like: 
> 
>   /usr/share			platform independent
>   /usr/fedora-linux-i386/lib	platform dependent
>   /usr/debian-netbad-amd64/lib	platform dependent
>   /usr/apple-darwin-ppc/lib	platform dependent
>   /usr/ibm-aix-power5/lib	platform dependent

There's also the hwcap system in glibc (responsible for searching things
like /lib/tls and /usr/lib/sparcv9 and such things), which can be
used to solve a similar but not identical problem.

> and similarly for '/lib' and so on, and the rudiments of such a
> logic are in the filesystem shape standard and in the GNU
> autoconf tools (check 'ls /usr') even if the order of the
> ''vendor'', ''API'' and ''ABI'' fields is arguably wrong.

The what fields? The fields in the config.guess-generated architecture
triplet are called CPU, VENDOR and OS. A few systems have OS fields
elaborated in various ways, but the format of those fields is not
consistent. (Linux is one of them, with the elaborations `gnu' and
`gnulibc1', but it's a fairly unusual case).

> The issue then: can you do a multi platform install? That is one
> for example where there are _two_ 'libc.so', one for 32 bit and
> one for 64 bit, installed at the same time. Yes, of course, if
> your package and dependency managers both support it.

Indeed; you just need to make sure that GCC gets passed the right
--dynamic-linker switches; there should be dynamic linker of the right
bitness in that location (which is standardized by the FHS). This is
generally done for you by the specs files, normally by two different
compilers but in some cases (i386/x86-64, SPARC32/SPARC64) optionally by
a single compiler that can generate code for both platforms.

>>> For example, ask yourself: why is the major version of many
>>> '.deb's repeated as a suffix to the package name? Why not
>>> add the name of the architecture as another suffix? :-)
> 
> martin> You've still lost me, could you explain your point in
> martin> simpler terms, please?
> 
> The problem here is: suppose you have programs that want to use
> two different API/ABI versions of the same library, let's say
> the Berkeley DB library.
> 
> Easy! Just install both the relevant packages, and because each
> API/ABI versions has a different set of numbers after ".so"
> everything is all right.

This works except in the case where library A depends on libraries B and
C, and library B *also* depends on library C, but a different version of
it, and A and B pass around objects obtained from library C, and the
format of those objects changed...

... the most common example of this failure mode is probably libstdc++.

Fixing it without quadratically slowing down the dynamic linker is
*hard* (which is why it hasn't been done yet).

>   This is desirable in order to have the ability to boot the
>   same disc on different architectures (something that did
>   happen sometimes when one had rmeovable disc packs), or to
>   share the same installed tree over the network with clients
>   with different architectures. An AMD64 machine is a special
>   case of the former situation, as in effect it is both an
>   'amd64' and a 'i386' CPU at the same time.

Likewise an UltraSPARC, which is sort of an x86-64 turned upside
down, in that most programs on an ultralinux box will be 32-bit,
while on an x86-64 most will probably be 64-bit.

>   The reason why multiplatform installs haven't been well
>   targeted is lazyness, induced by the x86 monoculture that has
>   infected so many environments,

Yeah right. That explains perfectly well why Debian's having trouble,
since with its 11 environments it's *such* a monoculture.

It's not been fixed in package managers for much the same reason as the
reason why cross-compilation support is so bad: firstly, it's a hard
problem to solve correctly, and secondly most people don't need it. Even
if you have multiple machines of different architectures, it is uncommon
to share binaries between them, and you need to fake out the package
manager to do that even in a network containing maches all of the same
model.

The only reason most people need it is x86-64 (and to a lesser extent
UltraSPARC and MIPS).

>                                  and secondarily by cheap disc
>   space.

*That* is significant. Non-debugging binaries are *tiny* compared to the
size of modern disks: adding complexity to the package manager to allow
NFS-sharing of package-managed state is just not worth it. It won't get
tested and will rot. Best to leave it out.

> Note, the point here is that the packages are called:
> 
>   libdb4.1-4.1.25-18_i386.deb
>   ...
>   libdb4.3-4.3.28-3_i386.deb
> 
> and so on. Why aren't they called instead:
> 
>   libdb-4.1.25-18_i386.deb
>   ...
>   libdb-4.3.28-3_i386.deb
> 
> and so on, and then install different versions of a package with
> the same base name? Well, because you cannot if you use DPKG/APT.

*sigh* this has been answered over and over again by the dpkg hackers.
Either you are trolling or you aren't doing any research; this is a
question thirty seconds with Google could answer.

Your proposed solution brings *nothing* and actually adds complexity and
reduces transparency.

Think. With the current rule (only one version of a given package
installed at once), it is trivial for both humans and the package
manager to tell whether two packages can coexist: if they have
different names and do not conflict, they can coexist: if not,
they cannot. The rule is absolute and very simple.

If you allow the installation of multiple versions of a package at once,
how do you answer that question at all? How can you tell that libdb
4.3.28 can coexist with libdb 4.1.25? After all, kdelibs 3.2.1 and
kdelibs 3.3.2 *cannot* coexist, so you can't just use the version number
as a clue.

And if you can't answer the question of whether two versions of a
package can coexist or not, the entire packaging system falls apart.

> Similary, if you have both
> 
>   libdb-4.3.28-3_i386.deb
>   libdb-4.3.28-3_amd64.deb
> 
> you would want to be able to install them all, as after all they
> would live in different directories. Well, you cannot if you use
> DPKG/APT.

Well, actually, no, you'd need a special version of the amd64 package
anyway. Both these package contain files with the same names (in
/usr/share/doc if nowhere else).

> Now admittedly it is not quite as simple as ''install them all''

It's quite a lot more complex. The packaging system I use here (of my
own invention and not robust enough for public release yet) has about a
thousand lines of code specifically to handle the case of multi-arch
installation, and more to handle the case of packages that must be
installed repeatedly after building with distinct flags (a superset of
the biarch problem).

It is *not* a simple problem.

> but after all the Linux kernel has support for multiple ELF
> executable types, and from there it is not that hard to have
> multiple loaders etc., and with some hack or another other
> distros handle it.

Yes. Debian dislikes hacks, has to allow smooth upgrades, and
more importantly has to allow an upgrade to version N using
the dpkg from version N-1. So even if support for all this were
added to dpkg tomorrow, you'd have to wait before it could be
safely used.

> The basic problem is that the DPKG/APT combination only looks at
> the package base name, not at all three/four parts of the full
> name,

Utter nonsense. It looks at all parts of the name *and this is
documented in Debian Policy*.

> For this reason Debian package names have the major version
> appended as a suffix to the base package name, and they even
> have, as in this case:
> 
> i 427kB 1020kB 4.3.28-3 4.3.28-3 libdb4.3++c2
> 
> the ABI version of the C++ library they depend upon appended to
> the base name; sooner or later we shall see the architecture
> type appended too, and why not? :-(

Yes, indeed. This is unavoidable. Note that the ELF soname system has no
way to represent the case that `these libraries have transitive
dependencies on libraries which must not appear together in a process's
address space'. (libgcc is another one of those, which is why the
world will dissolve in fire before we bump its soname).

> And> as 'ia32-libs' is just a compilation of the contents of existing
> '_x86.deb' packages, with a different, architecture dependent name.

That's the only sane way to do it. That or add a `major number' field
which is, in every way that matters, *part of the package name*.

only people installing libraries explicitly will ever see these names
outside of ignorable gibberish flying past in an aptitude run, though,
and the people installing libraries really *need* to think about which
version and architecture the library they want to install is for.

-- 
`Next: FEMA neglects to take into account the possibility of
fire in Old Balsawood Town (currently in its fifth year of drought
and home of the General Grant Home for Compulsive Arsonists).'
            --- James Nicoll
-- 
Gllug mailing list  -  Gllug at gllug.org.uk
http://lists.gllug.org.uk/mailman/listinfo/gllug