This bootstrap aspect got me and I discussed this with a number of
people and did some research.
Given the feedback, I am convinced that changing PT_INTERP is a stupidMe too, I was never persuaded.
idea regardless of whether it is technically feasible. There must be a
better way. Let's step back a bit.
4. Change the bootstrap protocol. In essence, this has been attemptedI think that this is being dismissed too easily, mostly because the
in debootstrap by creating these symlinks prior to unpack, but no
consensus has evolved around this approach yet. The category is
wider though and generally requires changes to all bootstrapping
tools.
Moving on to category 4 feels rather obvious, especially because workBut the code is trivial. It is currently more complex than it is needed
has been done there in debootstrap. The approach in debootstrap however
is one that I see as a dead end, because it causes us to maintain this
code multiple times. It's the number of derivatives times the number of bootstrap tools and that doesn't scale.
https://wiki.debian.org/Teams/Dpkg/Spec/InstallBootstrap. In particular, making architecture bootstrap become chrootless would likely be aWhy not?
generic solution to this and other problems. However, any change needs
to propagate to a stable release in all bootstrapping tools. Therefore
we cannot reasonably finish the transition before forky. This makes
So what's left is category 1. I looked into what the minimum set ofInteresting approach. If it is so much simple, why are you not persuaded
files to be retained could be. To do that end, I moved everything and
then reverted as much as was needed to make bootstrapping work.
* /lib64/ld-linux-x86-64.so.2 (hopefully obvious)
* /lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 (link target)
* /bin/sh (hopefully obvious)
* /bin/dash (link target)
* /bin/bash (usrmerge runs ldd, which is a #!/bin/bash script)These can be easily fixed, maybe the ldd issue too.
* /bin/more (update-alternatives doesn't like its absence)
* /bin/cp (unless usrmerge stops hard coding its path)
The other major takeaway is that a significantAgreed.
chunk of the problems mentioned in this mail cannot be fixed by
modifying dpkg only.
"Helmut" == Helmut Grohne <[email protected]> writes:
"Helmut" == Helmut Grohne <[email protected]> writes:
Hi,
This bootstrap aspect got me and I discussed this with a number of
people and did some research.
On Sun, May 07, 2023 at 12:51:21PM +0100, Luca Boccassi wrote:
I don't think this is true? At least not in the broader sense: if you compile something on Debian, it will obviously get linked against
libraries and dependencies as they are in Debian.
Perhaps what you mean is that, given an entire separate sysroot-like
tree, passing the appropriate compiler and linker flags and
environment variables, you can use the local compiler we ship to build 'foreign' programs. That is true, but again it requires to set up the environment appropriately, including linker flags. And the caller
needs to ensure the environment, including linker flags, is
appropriate for the target environment (I guess 'host' environment, in
GNU parlance). Therefore, I don't think it would be unreasonable to
require that if the target environment is split-usr, then the caller
also needs to specify an appropriate '-Wl,--dynamic-linker=/lib/ld-whatever' option.
Given the feedback, I am convinced that changing PT_INTERP is a stupid
idea regardless of whether it is technically feasible. There must be a
better way. Let's step back a bit.
The underlying problem here is performing the initial filesystem
bootstrap. The semantics of this are a bit vague as they are not spelled
out in policy, so we will have to derive them from implementations.
I think the major players are (in descending popularity):
* debootstrap
* mmdebstrap
* cdebootstrap
* multistrap
multistrap predates mmdebstrap and when there was no mmdebstrap, I used
it a lot. When attempting to test it, I totally couldn't convince it to bootstrap from an unsigned or locally signed repository. The patch in #908451 didn't cut it. I also note that it creates a /lib64 -> /lib
symbolic link which feels quite incompatible with merged-/usr. For
these reasons, I am dropping multistrap from the tools under
consideration and recommend removing it from the archive. If you happen
to use multistrap, now would be a good moment to tell me. Personally,
all of my use cases of multistrap have been converted to mmdebstrap and
that made a lot of things simpler.
cdebootstrap vaguely works though unsigned operation seems dysfunctional
as it runs apt-get update during cdebootstrap-helper-apt.postinst and
that fails. I happen to not have figured out why and treat this failure
as a success.
So the most popular implementations quite evidently are debootstrap and mmdebstrap and both "just work". I note though that they work quite differently:
* debootstrap (depending on flags including --variant) pre-merges its
chroot while mmdebstrap relies on packages doing it.
I think that the question whether a distribution is merged is a
property of the distribution and not the bootstrap tool, so I
strongly recommend following mmdebstrap's view on this. The
debootstrap way means that we have to include patches for every
derivative, which is a process that does not scale well.
* mmdebstrap operates in two phases. It first unpacks and configures a
rather minimal set of packages and then proceeds to adding packages
passed to --include in a second phase once essential is fully
configured while debootstrap immediately unpacks everything.
I think the debootstrap approach is slightly worse here, because it
means that preinst scripts of non-essential packages cannot rely on
essential packages having been configured.
In any case, we have to deal with both behaviours.
After this little excursion into bootstrap technology, let's go back to
the /usr-merge and its effects.
I think at this point, we have quite universal consensus about the goal
of moving files to their canonical location (i.e. from / to /usr) as a solution to the aliasing problems while we do not have consensus on
precisely how to do this (i.e. with changing dpkg or without). If you
believe that this is not consensus, please speak up.
So in a distant future our packages will not contain any files in /bin
or /lib. In particular, this affects /bin/sh and the dynamic loader,
both of which are required to run maintainer scripts, which are
currently required for creating the symbolic links. Boom.
Hi,
On 5/18/23 18:08, Luca Boccassi wrote:
Without it, leaving them in place makes no difference for usrmerged
systems, and allows derived distributions that don't need usrmerge to
continue using our packages.
Not quite. Having packages only ship files under /usr (and possibly
/etc) is very much a goal in itself for a lot of us.
My point is: that is not an end goal, because it provides no real
tangible benefit on its own.
It does make sense in the context of building immutable images, which is *also* a bootstrapping problem, and probably worth being supported with proper tooling.
- there is no guarantee that usrmerge will be permanent or the last
transition of this kind
It is permanent, there are several upstream projects that will drop
support for legacy layouts very soon, and it will not be re-added
back.
You are currently building a "legacy" system, it will just take a bit of
time to reach that status. The less you anticipate future needs, the
faster this will be.
I understand that you are also a member of one of these upstream
projects, and that you are taking the interests of this project to
"pretty much the last relevant holdout" here. Has it occurred to you
that you are also wearing a Debian hat, and you could be taking the
interests of the Debian project to said upstream project?
- it also solves the bootstrap problem
It also is the least likely to succeed, and the most likely to cause significant "social" upheavals.
The only social problem I see is that you are trying to create a
situation in which other people are compelled to do your work for you if
they want it to be done properly.
So far, you have been throwing out "solutions", and left the analysis of
the feasibility of those to other people, then, after three iterations,
you demanded a full write-up of all existing use cases and blanket
permission to ignore anything not brought up in this list.
The thing is: I see more enthusiasm and self-directed problem solving
skills from the interns at the company where I work, and at the same
time you are one of the top contributors of the upstream project whose
ideas of a "supported" configuration we are supposed to follow.
Hi,
On 5/18/23 02:15, Sam Hartman wrote:
Helmut> I think at this point, we have quite universal consensus
Helmut> about the goal of moving files to their canonical location
Helmut> (i.e. from / to /usr) as a solution to the aliasing problems
Helmut> while we do not have consensus on precisely how to do this
Helmut> (i.e. with changing dpkg or without). If you believe that
Helmut> this is not consensus, please speak up.
I agree we have strong consensus that we want to move files to their canonical locations.
I'm not entirely sure I'd agree that we have consensus that's our
solution to the aliasing problem.
It's the other way around: moving the files as a solution to the
aliasing problem is the strongest argument in favour of moving the files inside the packages.
Without it, leaving them in place makes no difference for usrmerged
systems, and allows derived distributions that don't need usrmerge to continue using our packages.
If for example we accomplish the move to canonical locations by changing dpkg, we might well get some form of aliasing support in dpkg.
IMO, that is still the preferred solution:
- it is actually safe, because dpkg knows what is going on and can
reject conflicting changes
- there is no guarantee that usrmerge will be permanent or the last transition of this kind
- it also solves the bootstrap problem
Helmut> I think at this point, we have quite universal consensus
Helmut> about the goal of moving files to their canonical location
Helmut> (i.e. from / to /usr) as a solution to the aliasing problems
Helmut> while we do not have consensus on precisely how to do this
Helmut> (i.e. with changing dpkg or without). If you believe that
Helmut> this is not consensus, please speak up.
I agree we have strong consensus that we want to move files to their canonical locations.
I'm not entirely sure I'd agree that we have consensus that's our
solution to the aliasing problem.
If for example we accomplish the move to canonical locations by changing dpkg, we might well get some form of aliasing support in dpkg.
Without it, leaving them in place makes no difference for usrmerged
systems, and allows derived distributions that don't need usrmerge to
continue using our packages.
Not quite. Having packages only ship files under /usr (and possibly
/etc) is very much a goal in itself for a lot of us.
- there is no guarantee that usrmerge will be permanent or the last
transition of this kind
It is permanent, there are several upstream projects that will drop
support for legacy layouts very soon, and it will not be re-added
back.
- it also solves the bootstrap problem
It also is the least likely to succeed, and the most likely to cause significant "social" upheavals.
Hi,
On Wed, 17 May 2023, Helmut Grohne wrote:
For completeness sake, there is one more entry in category 3: We can run the dynamic loader from its canonical location explicitly, so we'd
modify maintainer scripts to start with:
#!/usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 /usr/bin/sh
This would only affect preinst scripts participating in bootstrap and
we'd not bother with changing PT_INTERP in the toolchain nor in
packages. Unfortunately, this completely breaks the DPKG_ROOT work and
with that, I see no viable entries in category 3 for moving forward.
Moving on to category 4 feels rather obvious, especially because work
has been done there in debootstrap. The approach in debootstrap however
is one that I see as a dead end, because it causes us to maintain this
code multiple times. It's the number of derivatives times the number of bootstrap tools and that doesn't scale.
Category 4 is wider though and we also have other prior art at https://wiki.debian.org/Teams/Dpkg/Spec/InstallBootstrap. In particular, making architecture bootstrap become chrootless would likely be a
generic solution to this and other problems. However, any change needs
to propagate to a stable release in all bootstrapping tools. Therefore
we cannot reasonably finish the transition before forky. This makes category 4 rather unattractive in a short term, but still worth pursuing
in a long term.
In the same spirit, I'd like to throw an idea... could we decide that base-files is the first package to be configured as part of the bootstrap protocol and change base-files maintainer's scripts into statically linked executables so that they can work even if we don't have the library loader
on the ABI-compliant path?
And creating the required symlinks would be done by those (standalone) maintainer scripts...
I don't know if we already have some rule/invariant in the configuration order of the unpacked packages, but I doubt so.
Having ruled out categories 3 and 4 maybe category 2 would be good? We could just ship those symlinks in base-files and be done, right? Unfortunately, we pass -k to tar in debootstrap, so when it extracts base-files and tries to unpack the bin -> usr/bin symlink, it sees that
oh no, there already is a symlink at bin (as debootstrap placed it
there) and thus fails. So in order to make this work, we also have to modify debootstrap (and thus are in a combination of category 3 and 4).
So when we will have fixed this, and waited for a release cycle, we can
get rid of the statically compiled maintainer scripts and simply ship the symlinks in base-files.
For completeness sake, there is one more entry in category 3: We can run
the dynamic loader from its canonical location explicitly, so we'd
modify maintainer scripts to start with:
#!/usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 /usr/bin/sh
This would only affect preinst scripts participating in bootstrap and
we'd not bother with changing PT_INTERP in the toolchain nor in
packages. Unfortunately, this completely breaks the DPKG_ROOT work and
with that, I see no viable entries in category 3 for moving forward.
Moving on to category 4 feels rather obvious, especially because work
has been done there in debootstrap. The approach in debootstrap however
is one that I see as a dead end, because it causes us to maintain this
code multiple times. It's the number of derivatives times the number of bootstrap tools and that doesn't scale.
Category 4 is wider though and we also have other prior art at https://wiki.debian.org/Teams/Dpkg/Spec/InstallBootstrap. In particular, making architecture bootstrap become chrootless would likely be a
generic solution to this and other problems. However, any change needs
to propagate to a stable release in all bootstrapping tools. Therefore
we cannot reasonably finish the transition before forky. This makes
category 4 rather unattractive in a short term, but still worth pursuing
in a long term.
Having ruled out categories 3 and 4 maybe category 2 would be good? We
could just ship those symlinks in base-files and be done, right? Unfortunately, we pass -k to tar in debootstrap, so when it extracts base-files and tries to unpack the bin -> usr/bin symlink, it sees that
oh no, there already is a symlink at bin (as debootstrap placed it
there) and thus fails. So in order to make this work, we also have to
modify debootstrap (and thus are in a combination of category 3 and 4).
In the same spirit, I'd like to throw an idea... could we decide that base-files is the first package to be configured as part of the bootstrap protocol and change base-files maintainer's scripts into statically linked executables so that they can work even if we don't have the library loaderIt could be even easier: base-files could be unpacked once without
on the ABI-compliant path?
And creating the required symlinks would be done by those (standalone) maintainer scripts...Indeed, this would be very simple and it has already been proposed.
I don't know if we already have some rule/invariant in the configuration order of the unpacked packages, but I doubt so.
On Fri, 09 Jun 2023, Marco d'Itri wrote:
On Jun 08, Raphael Hertzog <[email protected]> wrote:
In the same spirit, I'd like to throw an idea... could we decide that base-files is the first package to be configured as part of the bootstrap protocol and change base-files maintainer's scripts into statically linkedIt could be even easier: base-files could be unpacked once without
executables so that they can work even if we don't have the library loader
on the ABI-compliant path?
running the maintainer scripts and then "reinstalled" again later as
usual.
I think you are missing the point here, that only works if the package is shipping the symlinks. And the idea is to not do this immediately because
it breaks debootstrap: if I understood correctly unpacking base-files
with the symlinks would fail if debootstrap had already pre-created those symlinks (due to a -k option that we should get rid of in /usr/share/debootstrap/scripts/debian-common).
Hence the special maintainer script to create the required symlinks
without relying on /bin/sh or any dynamically linked executable.
On Jun 08, Raphael Hertzog <[email protected]> wrote:
In the same spirit, I'd like to throw an idea... could we decide that base-files is the first package to be configured as part of the bootstrap protocol and change base-files maintainer's scripts into statically linked executables so that they can work even if we don't have the library loader on the ABI-compliant path?It could be even easier: base-files could be unpacked once without
running the maintainer scripts and then "reinstalled" again later as
usual.
And creating the required symlinks would be done by those (standalone) maintainer scripts...
I don't know if we already have some rule/invariant in the configuration order of the unpacked packages, but I doubt so.
Indeed, this would be very simple and it has already been proposed.
But somebody then complained that special-casing a package would violate
the design contraints he self-imposed to his own image building tool,
and as we all know every Debian maintainer can veto any systemic changes that they do not like.
In the same spirit, I'd like to throw an idea... could we decide that base-files is the first package to be configured as part of the bootstrap protocol and change base-files maintainer's scripts into statically linked executables so that they can work even if we don't have the library loader
on the ABI-compliant path?
On Jun 08, Raphael Hertzog <[email protected]> wrote:
And creating the required symlinks would be done by those (standalone) maintainer scripts...
I don't know if we already have some rule/invariant in the configuration order of the unpacked packages, but I doubt so.Indeed, this would be very simple and it has already been proposed.
But somebody then complained that special-casing a package would violate
the design contraints he self-imposed to his own image building tool,
and as we all know every Debian maintainer can veto any systemic changes that they do not like.
Add a new package usrmerge-support (or whatever). It is a bit similar to multiarch-support: It must not have any dependencies or pre-dependencies. It will not have files, but maintainer scripts. Those scripts set up protective diversions on behalf of base-files for the symbolic links that cause aliasing. Then base-files will issue a Pre-Depends on usrmerge-support (but not yet ship symlinks). I initially thought, this could be part of usr-is-merged, but then base-files would pull that and standard mmdebstrap would no longer pull usrmerge and break. So it really needs to be a separate package. Anyway, once we have protective diversions, we can move files without risking that dpkg deletes the symbolic links.
Then we can actually perform that move of files to their canonical
locations except for a small set of locations including dash, bash,
libc6, and util-linux (maybe not exhaustive). [There is a lot of missing detail about non-bootstrap aspects here.]
Once all essential packages (but the exceptions) have no files left in aliased locations, we can upload base-files adding the symlinks together
with the packages previously kept unmodified in one dinstall. Before
that dinstall, things will continue to work normally. The protective diversions will not affect unpacking, because dpkg only performs exact matches on diversions. After that dinstall, base-files will create the symlinks and things will hopefully work (because the patched debootstrap only creates them after the initial unpack).
if I understand that plan correctly, the usrmerge-support package setting up diversions is only necessary because you want to avoid having to do the move to
/usr of *all* affected packages in the essential set in a single dinstall? Is that correct?
If yes, how many source packages are we have to be modified part from base-files, dash, bash, libc6, and util-linux?
Would it be too much to prepare patches for all of these, test that everything
works with some QA setup and then NMU all 22 source packages with pre-approved
patches in a single dinstall? Would that avoid having to temporarily go via a usrmerge-support package?
Hi Johannes,
On Fri, Jun 09, 2023 at 05:47:56PM +0200, Johannes Schauer Marin Rodrigues wrote:
if I understand that plan correctly, the usrmerge-support package
setting up diversions is only necessary because you want to avoid
having to do the move to /usr of *all* affected packages in the
essential set in a single dinstall? Is that correct?
This is not correct. In bookworm -> trixie upgrade scenario, we intend
to move all the files from / to /usr. Now we look into how this
happens focus on one particular symlink, without loss of generality
choose /bin. Since /bin is no longer in the dpkg database at the end
of the upgrade, some package must be the last one to contain /bin.
When upgrading (or removing that package), dpkg will attempt to remove
/bin (which in its opinion is an empty directory and the last consumer
is releasing it). However, since dpkg has no clue about file types, it doesn't actually know that this is a directory and takes care of the
/bin -> /usr/bin symlink using unlink(). And this is where /bin
vanishes. Oops.
So the idea here is to add a protective diversion for /bin such that
removing /bin instead removes some path we don't care about. [...]
Did you consider just having one package keep one dummy file in /bin?
While this isn't elegant it sounds much less complex than diversions and tricky pre-depend loops, etc.
I might be very well missing something here (for example maybe it's
really essential that no files remain in /bin, even not a dummy file).
But in the other branch of this thread you welcomed "dumb" questions, so
here you go ;]
* mmdebstrap operates in two phases. It first unpacks and configures a
rather minimal set of packages and then proceeds to adding packages
passed to --include in a second phase once essential is fully
configured while debootstrap immediately unpacks everything.
I think the debootstrap approach is slightly worse here, because it
means that preinst scripts of non-essential packages cannot rely on
essential packages having been configured.
| Sysop: | Keyop |
|---|---|
| Location: | Huddersfield, West Yorkshire, UK |
| Users: | 715 |
| Nodes: | 16 (2 / 14) |
| Uptime: | 03:48:12 |
| Calls: | 12,099 |
| Calls today: | 7 |
| Files: | 15,003 |
| Messages: | 6,517,881 |