Hi Luca,
On Sat, Apr 22, 2023 at 01:06:18PM +0100, Luca Boccassi wrote:
On Sat, 22 Apr 2023 at 11:50, Helmut Grohne <[email protected]> wrote:
On Fri, Apr 21, 2023 at 03:29:33PM +0100, Luca Boccassi wrote:
After Bookworm ships I plan to propose a policy change to the CTTE and policy maintainers to forbid shipping files in the legacy directories altogether, followed by a debhelper change to adjust any stragglers automatically at build time and a mass rebuild, plus MBF for the small % that does not use dh and a piuparts test to stop migration for anything that is uploaded and doesn't comply. That should bring the matter to an end, without needing to modify dpkg.
I agree with the goal of removing aliases by moving files to their canonical locations. However, I do not quite see us getting there in the way you see it, but maybe I am missing something. As long as dpkg does not understand the effects of aliasing, we cannot safely move those
files and thus the file move moratorium will have to be kept in place. And while moving the files would bring the matter to an end, we cannot
do so without either modifying dpkg or rolling back the transition and starting over. I hope that we all agree that rolling back would be too insane to even consider, but I fail to see how you safely move files without dpkg being changed. Can you elaborate on that aspect?
Moving files within _the same_ package is actually fine as far as I
know. It's moving between location _and_ packages within the same
upgrade that is problematic. The piuparts test I added is overzealous,
but it doesn't need to be.
You got me interested to dig deeper. I looked into that piuparts
check[1]. From what I understand, it does something differently from
what you suggest here. It detects files moved between / and /usr (which
is what is going to happen according to your plan) and it does not
detect files being moved between packages (which would actually be problematic here). It also does not produce an error (merely a warning),
so it doesn't halt testing migration and in particular, it doesn't
detect the problematic situation at all. That's kinda disappointing.
You also side stepped the question of how to handle the situation where
we've moved files from / to /usr and then need to move files between
packages in a safe way, though your other response to Simon McVittie
suggests you have an idea there.
On Sat, 22 Apr 2023 13:03:01 +0100, Luca Boccassi wrote:
We already have piuparts tests detecting files moving, it should be
easy enough to extend that to check that the appropriate
Breaks/Replaces have been added. Correct me if I'm wrong, but I
believe it's already against policy to do this without
Breaks/Replaces, so it's not a use case that we need to support, no?
If someone does that by mistake, the package will not migrate to
testing.
Yeah, we agree that you need Breaks+Replaces. The issue here is that due
to dpkg not knowing about the aliasing, Breaks+Replaces is insufficient.
Due to the insufficiency the CTTE enacted the moratorium.
My impression is that you believe that with bookworm, the moratorium is
being lifted and thus we can start moving files. Unfortunately, the underlying problem does not go away just because we've released
bookworm. It's a problem that is unique to merged installations and
those are not going to go away in bookworm.
So yeah, we all want these files moved to their canonical locations and
I kinda like the simplicity of your approach, but thus far my
understanding is that it is plain broken and doesn't work. Well, yeah it
does work in the sense that we break user installations during upgrade
and notice so late in the freeze without any good options to fix.
On Sat, Apr 22, 2023 at 01:06:18PM +0100, Luca Boccassi wrote:
On Sat, 22 Apr 2023 at 11:50, Helmut Grohne <[email protected]> wrote:
On Fri, Apr 21, 2023 at 03:29:33PM +0100, Luca Boccassi wrote:
After Bookworm ships I plan to propose a policy change to the CTTE and policy maintainers to forbid shipping files in the legacy directories altogether, followed by a debhelper change to adjust any stragglers automatically at build time and a mass rebuild, plus MBF for the small
% that does not use dh and a piuparts test to stop migration for
anything that is uploaded and doesn't comply. That should bring the matter to an end, without needing to modify dpkg.
I agree with the goal of removing aliases by moving files to their canonical locations. However, I do not quite see us getting there in the way you see it, but maybe I am missing something. As long as dpkg does
not understand the effects of aliasing, we cannot safely move those
files and thus the file move moratorium will have to be kept in place.
And while moving the files would bring the matter to an end, we cannot
do so without either modifying dpkg or rolling back the transition and starting over. I hope that we all agree that rolling back would be too insane to even consider, but I fail to see how you safely move files without dpkg being changed. Can you elaborate on that aspect?
Moving files within _the same_ package is actually fine as far as I
know. It's moving between location _and_ packages within the same
upgrade that is problematic. The piuparts test I added is overzealous,
but it doesn't need to be.
We already have piuparts tests detecting files moving, it should be
easy enough to extend that to check that the appropriate
Breaks/Replaces have been added. Correct me if I'm wrong, but I
believe it's already against policy to do this without
Breaks/Replaces, so it's not a use case that we need to support, no?
If someone does that by mistake, the package will not migrate to
testing.
I'd also be interested on how you plan to move important files in
essential packages. This is an aspect raised by Simon Richter and where
I do not see an obvious answer yet.
Do you have a pointer? Not sure I follow what "important" files means
here, doesn't ring a bell.
Dpkg already has defined behaviour for directory vs symlink: the directory wins. In principle a future version of dpkg could change that, but /lib/ld-linux.so.2 is just too special, we'd never want to have a package that
actually moves it.
Hi Luca,
On Sat, Apr 22, 2023 at 01:06:18PM +0100, Luca Boccassi wrote:
On Sat, 22 Apr 2023 at 11:50, Helmut Grohne <[email protected]> wrote:
On Fri, Apr 21, 2023 at 03:29:33PM +0100, Luca Boccassi wrote:
After Bookworm ships I plan to propose a policy change to the CTTE and policy maintainers to forbid shipping files in the legacy directories altogether, followed by a debhelper change to adjust any stragglers automatically at build time and a mass rebuild, plus MBF for the small % that does not use dh and a piuparts test to stop migration for anything that is uploaded and doesn't comply. That should bring the matter to an end, without needing to modify dpkg.
I agree with the goal of removing aliases by moving files to their canonical locations. However, I do not quite see us getting there in the way you see it, but maybe I am missing something. As long as dpkg does not understand the effects of aliasing, we cannot safely move those
files and thus the file move moratorium will have to be kept in place. And while moving the files would bring the matter to an end, we cannot
do so without either modifying dpkg or rolling back the transition and starting over. I hope that we all agree that rolling back would be too insane to even consider, but I fail to see how you safely move files without dpkg being changed. Can you elaborate on that aspect?
Moving files within _the same_ package is actually fine as far as I
know. It's moving between location _and_ packages within the same
upgrade that is problematic. The piuparts test I added is overzealous,
but it doesn't need to be.
You got me interested to dig deeper. I looked into that piuparts
check[1]. From what I understand, it does something differently from
what you suggest here. It detects files moved between / and /usr (which
is what is going to happen according to your plan) and it does not
detect files being moved between packages (which would actually be problematic here). It also does not produce an error (merely a warning),
so it doesn't halt testing migration and in particular, it doesn't
detect the problematic situation at all. That's kinda disappointing.
You also side stepped the question of how to handle the situation where
we've moved files from / to /usr and then need to move files between
packages in a safe way, though your other response to Simon McVittie
suggests you have an idea there.
On Sat, 22 Apr 2023 13:03:01 +0100, Luca Boccassi wrote:
We already have piuparts tests detecting files moving, it should be
easy enough to extend that to check that the appropriate
Breaks/Replaces have been added. Correct me if I'm wrong, but I
believe it's already against policy to do this without
Breaks/Replaces, so it's not a use case that we need to support, no?
If someone does that by mistake, the package will not migrate to
testing.
Yeah, we agree that you need Breaks+Replaces. The issue here is that due
to dpkg not knowing about the aliasing, Breaks+Replaces is insufficient.
Due to the insufficiency the CTTE enacted the moratorium.
My impression is that you believe that with bookworm, the moratorium is
being lifted and thus we can start moving files. Unfortunately, the underlying problem does not go away just because we've released
bookworm. It's a problem that is unique to merged installations and
those are not going to go away in bookworm.
So yeah, we all want these files moved to their canonical locations and
I kinda like the simplicity of your approach, but thus far my
understanding is that it is plain broken and doesn't work. Well, yeah it
does work in the sense that we break user installations during upgrade
and notice so late in the freeze without any good options to fix.
On Sat, Apr 22, 2023 at 01:06:18PM +0100, Luca Boccassi wrote:
I'd also be interested on how you plan to move important files in essential packages. This is an aspect raised by Simon Richter and where
I do not see an obvious answer yet.
Do you have a pointer? Not sure I follow what "important" files means
here, doesn't ring a bell.
In <[email protected]>, Simon Richter
said:
Dpkg already has defined behaviour for directory vs symlink: the directory wins. In principle a future version of dpkg could change that, but /lib/ld-linux.so.2 is just too special, we'd never want to have a package that
actually moves it.
This and /bin/sh is the kind of files I'd consider important. And then
upon thinking further it became more and more difficult for me to make
sense of the objection. On a merged system, we can just move that file
to its canonical location without having any trouble even with an
unmodified dpkg. So from my pov, the question about important files can
be disregarded. I hope Simon Richter agrees.
Let us circle back to your "broken" approach. It sure is simple (just
Ok, let's move on. I've proposed diversions as a cure, but in reality diversions are a problem themselves. Consider that
cryptsetup-nuke-password diverts /lib/cryptsetup/askpass, which is
usually owned by cryptsetup. If cryptsetup were to move that file to
/usr, the diversion would not cover it anymore and the actual content of askpass would depend on the unpack order. That's very bad and none of
what I proposed earlier is going to fix this.
And of course, this is not some special example, it's a pattern:
* /lib/udev/rules.d/60-cdrom_id.rules: udev -> amazon-ec2-utils
* /sbin/dhclient: isc-dhcp-client -> isc-dhcp-client-ddns
* /bin/systemd-sysusers: systemd -> opensysusers
* ...
So how do we fix diversions? Let's have a look into the dpkg toolbox
again. I've got an idea. Diversions. What you say? How do you fix
diversions with diversions? Quite obviously, you divert
/usr/bin/dpkg-divert! And whenever dpkg-divert is instructed to add a diversion for a non-canonical path, you forward that call to the real dpkg-divert, but also call it with a canonicalized version such that
both locations are covered. When initially deploying the diversion of /usr/bin/dpkg-divert, we also need to transform existing diversions.
Other than that, things should work after doubling down on diversions.
Sorry, I don't have a test case for this yet.
On Thu, Apr 27, 2023 at 12:34:06AM +0200, Helmut Grohne wrote:
Ok, let's move on. I've proposed diversions as a cure, but in reality diversions are a problem themselves. Consider that
cryptsetup-nuke-password diverts /lib/cryptsetup/askpass, which is
usually owned by cryptsetup. If cryptsetup were to move that file to
/usr, the diversion would not cover it anymore and the actual content of askpass would depend on the unpack order. That's very bad and none of
what I proposed earlier is going to fix this.
And of course, this is not some special example, it's a pattern:
* /lib/udev/rules.d/60-cdrom_id.rules: udev -> amazon-ec2-utils
* /sbin/dhclient: isc-dhcp-client -> isc-dhcp-client-ddns
* /bin/systemd-sysusers: systemd -> opensysusers
* ...
So how do we fix diversions? Let's have a look into the dpkg toolbox
again. I've got an idea. Diversions. What you say? How do you fix diversions with diversions? Quite obviously, you divert /usr/bin/dpkg-divert! And whenever dpkg-divert is instructed to add a diversion for a non-canonical path, you forward that call to the real dpkg-divert, but also call it with a canonicalized version such that
both locations are covered. When initially deploying the diversion of /usr/bin/dpkg-divert, we also need to transform existing diversions.
Other than that, things should work after doubling down on diversions. Sorry, I don't have a test case for this yet.
I still don't have a test case, but I have data. Using
binarycontrol.d.n, I identified packages setting up diversions in
preinst (this seems most common, but dash for instance sets up a
diversion in postinst instead, so there are some false negatives). And
while I initially tried to parse those preinst scripts, solving the
halting problem seemed just too hard, so I opted for just running them.
I'm attaching the relevant scripts and showing the affected diversions:
diversion of /lib/udev/rules.d/60-cdrom_id.rules to /lib/udev/rules.d/60-cdrom_id.rules.disabled by amazon-ec2-utils in stable, testing, unstable
diversion of /sbin/coldreboot to /lib/container/divert/coldreboot.orig by bfh-container in testing, unstable
diversion of /sbin/halt to /lib/container/divert/halt.orig by bfh-container in testing, unstable
diversion of /sbin/poweroff to /lib/container/divert/poweroff.orig by bfh-container in testing, unstable
diversion of /sbin/reboot to /lib/container/divert/reboot.orig by bfh-container in testing, unstable
diversion of /sbin/shutdown to /lib/container/divert/shutdown.orig by bfh-container in testing, unstable
diversion of /lib/cryptsetup/askpass to /lib/cryptsetup/askpass.cryptsetup by cryptsetup-nuke-password in testing, unstable
diversion of /sbin/dhclient to /sbin/dhclient-noddns by isc-dhcp-client-ddns in stable, testing, unstable
diversion of /sbin/coldreboot to /lib/molly-guard/coldreboot by molly-guard in stable, testing, unstable
diversion of /sbin/halt to /lib/molly-guard/halt by molly-guard in stable, testing, unstable
diversion of /sbin/poweroff to /lib/molly-guard/poweroff by molly-guard in stable, testing, unstable
diversion of /sbin/reboot to /lib/molly-guard/reboot by molly-guard in stable, testing, unstable
diversion of /sbin/shutdown to /lib/molly-guard/shutdown by molly-guard in stable, testing, unstable
diversion of /bin/systemd-sysusers to /bin/systemd-sysusers.real by opensysusers in stable, testing, unstable
diversion of /sbin/coldreboot to /lib/open-infrastructure/container/divert/coldreboot.orig by progress-linux-container in stable, testing, unstable
diversion of /sbin/halt to /lib/open-infrastructure/container/divert/halt.orig by progress-linux-container in stable, testing, unstable
diversion of /sbin/poweroff to /lib/open-infrastructure/container/divert/poweroff.orig by progress-linux-container in stable, testing, unstable
diversion of /sbin/reboot to /lib/open-infrastructure/container/divert/reboot.orig by progress-linux-container in stable, testing, unstable
diversion of /sbin/shutdown to /lib/open-infrastructure/container/divert/shutdown.orig by progress-linux-container in stable, testing, unstable
diversion of /bin/zcat to /bin/zcat.gzip by zutils in stable, testing, unstable
diversion of /bin/zcmp to /bin/zcmp.gzip by zutils in stable, testing, unstable
diversion of /bin/zdiff to /bin/zdiff.gzip by zutils in stable, testing, unstable
diversion of /bin/zegrep to /bin/zegrep.gzip by zutils in stable, testing, unstable
I have a bad feeling about this. I think some dpkg maintainer warned us
that diversions would break. Let's peek at his list again. He also said update-alternatives would be broken. I admit not having dug into this
yet, but my gut feeling already is that update-alternatives will become "funny" as well though I guess we cannot fix update-alternatives by
adding alternatives.
On Tue, 02 May 2023, Helmut Grohne wrote:
I think there is a caveat (whose severity I am unsure about): In order
to rely on this (and on DEP 17), we will likely have versioned
Pre-Depends on dpkg. Can we reasonably rule out the case where and old
dpkg is running, unpacking a fixed dpkg, configuring the fixed dpkg and then unpacking an affected package still running the unfixed dpkg
process?
I don't know APT well enough to answer that question but from my point of view it's perfectly acceptable to document in the release notes that you
need to upgrade dpkg first.
Hi Luca,
On Sat, May 06, 2023 at 04:52:30PM +0100, Luca Boccassi wrote:
To have a working system you need several more steps that are
performed by the instantiator/image builder, such as providing working
and populated proc/sys/dev, writable tmp/var, possibly etc. And it
needs to be instantiated with user/password/ssh certs/locale/timezone.
And if it needs to be bootable on baremetal/vm, it needs an ESP. And
then if you have an ESP and want to run in a VM with SB, you'll need self-enrolling certs on first use or ensuring the 3rd party CA is provisioned. And then...
You paint it this way, but it really used to just work until we got the /usr-merge. Indeed, debvm creates virtual machine images effectively by bootstrapping a filesystem from packages and turning the resulting tree
into a file system image.
* /proc, /sys, /dev are mounted by systemd. All you need to do here is
create the directories and base-files does so.
* /tmp is shipped by base-files.
* user and password creation is not handled yet, but can be handled by
something similar to systemd-firstboot.
* Not sure what you mean with certs, locale and timezone. You can just
install ca-certificates, locales and tzdata as part of the bootstrap.
* The bootloader part for baremetal is kinda out of scope for
bootstrap, which is why debvm side-steps this. You can also skip it
for containers and build chroots. So it is one out of multiple use
cases that needs extra work here.
In a good chunk of situations, you can get just by without messing
around. Well that is until we broke it via usr-is-merged. I concur with
Simon Richter, that restoring this property is a primary concern.
You get the point. Going from a bunch of packages to a running system necessarily has many steps in between, some that are already done and
taken for granted, for example when you say "works as a container" I'm pretty sure the "container" engine is taking care of at the very least proc/dev/sys for you, and it's just expected to work. bin -> usr/bin,
sbin -> usr/sbin and lib -> usr/lib should get the same treatment: if
they are not there, the invoked engine should prepare them. systemd
and nspawn have been able to do this for a while now.
No, this misses the point. You can configure essential in a very limited environment. However, you cannot do so without the lib or lib64 symlink (depending on the architecture) and the bin symlink. This is so
critical, that it cannot be deferred to some external entity. It must be
part of the bootstrap protocol. There are some suggested ways to fix
this (such as adding separate bootstrap scripts next to maintainer
scripts), but nothing implemented.
Not having those hard coded means that the use case of / on a tmpfs
with the rest instantiated on the fly, assembled with the vendor's
/usr and /etc trees, becomes possible, which is neat. And said trees
can pass the checksum/full integrity muster.
It's neat that you can solve your use case by breaking other people's
use cases. This is not constructive interaction however. This kind of behaviour is precisely what caused so much conflict around the
/usr-merge. What if I gave a shit for your use case? Denying the
/usr-merge and just continuing unmerged as long as possible (as merging
would break my use case) would be my strategy of choice. You can make a difference here by starting to recognize other people's use cases and proposing solutions in that merged world. And no, it's not "add duct
tape to every bootstrap tool".
So I really want to see a solution for the bootstrap protocol before
moving the dynamic linker and /bin/sh to its canonical location. The
current bootstrap protocol is kept on life-support by installing the
usrmerge package by default. Dropping usrmerge from the
init-system-helpers dependency as first alternative or moving the
dynamic linker would break it. If I had a solution in mind, I'd
definitely post it right here, but unfortunately I have not.
Helmut
On Sat, 6 May 2023 at 19:51, Helmut Grohne <[email protected]> wrote:
- the moratorium on moving files from bin/ sbin/ lib/ _and_ to other
packages at the same time is maintained from bookworm till trixie, and
will lifted after trixie ships, and applies implicitly to all the
~2000 binary pkgs that are affected by the above
While the CTTE placed the moratorium until the release of bookworm, the
release team extended it beyond, see
https://lists.debian.org/[email protected]. So you
need explicit agreement from the release team on your plan. Failing
that, any package that has been forcefully moved is immediately
rc-buggy due to failing a release team requirement.
Of course the release team needs to be on board, no questions about
that. But given the idea is to maintain their decision exactly as it
stands I wouldn't imagine it would be an issue? Once again, the
moratorium is explicitly about moving between locations _and_
packages, in combination, not either/or. From that same email you
linked:
"Files moving their canonical location between / and /usr (details in
[1]) *and* from one binary package to another binary package within
one release cycle remain an RC issue unless dpkg supports it."
I'm proposing to keep this in place as a general rule, with the new
escape hatch that you devised as the only addition.
Hi Luca,
I fear you are still missing too many relevant details. What sounds like
a simple plan is severely broken if carried out in this simple form. It disappoints me that you continue to present it as this simple given the earlier evidence.
On Fri, May 05, 2023 at 11:11:54PM +0100, Luca Boccassi wrote:
- every package is forcefully canonicalized as soon as trixie is open
for business
I gave you scripts to prototype this (via binary patching) and
demonstrated that this immediately makes a fair number of packages rc
buggy due to causing unpack errors. I used zutils as an example. It
really is not that simple and it absolutely cannot be done forcefully,
but must be done in an opt-in way. I won't repeat the rationale here.
- the moratorium on moving files from bin/ sbin/ lib/ _and_ to other packages at the same time is maintained from bookworm till trixie, and
will lifted after trixie ships, and applies implicitly to all the
~2000 binary pkgs that are affected by the above
While the CTTE placed the moratorium until the release of bookworm, the release team extended it beyond, see https://lists.debian.org/[email protected]. So you
need explicit agreement from the release team on your plan. Failing
that, any package that has been forcefully moved is immediately
rc-buggy due to failing a release team requirement.
Again, let me try to fix up your plan:
1. Implement a service that reliably notices missing Breaks + Replaces
even in the presence of aliasing. Augment it to notice
canonicalization changes and have it report that Conflicts (or other
measures) are needed in such cases. This should operate for upgrades
from stable to testing, stable to unstable and testing to unstable.
2. Add a usr-merge-helper script to usr-is-merged that helps with
instating temporary diversions for the purpose of avoiding file loss.
3. Add a dpkg-divert wrapper for duplicating diversions according to
aliasing to usr-is-merged. Add postinst scripts that duplicate
existing diversions.
4. Add canonicalization support to debhelper as an (internal) addon.
Enable this addon in the next compat level. This will again populate
${misc:Pre-Depends} with "usr-is-merged (>= fixed version)" as needed.
Note that usrmerge's provides is unversioned and doesn't satisfy such
a dependency.
Sure, there are some things that need special handling, as you have
pointed out. What I meant is that I don't think we need special
handling for _all_ affected packages. AFAIK nothing is using
diversions for unit files or udev rules, for example (I mean if any
package is, please point it out, because I would like a word...). I
very strongly suspect this will be a small minority out of the total
of ~2k would need this treatment, and the vast majority would not. Do
you disagree with this gut feeling?
Of course, it goes without saying, we should check this before going
forward in any direction.
Of course the release team needs to be on board, no questions about
that. But given the idea is to maintain their decision exactly as it
stands I wouldn't imagine it would be an issue? Once again, the
moratorium is explicitly about moving between locations _and_
packages, in combination, not either/or. From that same email you
linked:
There are already distro-wide upgrade piuparts checks run occasionally
IIRC, at least I've seen a bug from one being reported this week, so
we should be most of the way there already?
To be clear, this would be very nice and welcome to have obviously,
but I don't think it needs to be a blocker. We don't have such checks
for vast parts of Policy, including moving files without
Breaks+Replaces as evidenced by the recent MBF, and yet we managed to
survive thus far. I don't think it's fair that this workstream should
be held to higher standards than the rest of the project.
4. Add canonicalization support to debhelper as an (internal) addon.
Enable this addon in the next compat level. This will again populate
${misc:Pre-Depends} with "usr-is-merged (>= fixed version)" as needed.
Note that usrmerge's provides is unversioned and doesn't satisfy such
a dependency.
As already mentioned, I do not believe this is necessary for _all_
cases. It is necessary for a certain number (that we should ascertain beforehand!) of cases, and we need the machinery implemented for them,
but I don't think we should impose this workflow with pre-depends and diversions for all affected packages. I think it should be mandatory
for problematic packages such as those you already pointed out, _and_
for cases where the maintainer wants to move files also between binary packages.
Curve ball question: is there anything blocking us from canonicalizing PT_INTERP at the source in gcc so that the essential set (and
everything else, really) can work without the lib -> usr/lib symlink?
Would anything obvious break?
Hello,
On Sat 06 May 2023 at 09:47PM +01, Luca Boccassi wrote:
On Sat, 6 May 2023 at 19:51, Helmut Grohne <[email protected]> wrote:
- the moratorium on moving files from bin/ sbin/ lib/ _and_ to other
packages at the same time is maintained from bookworm till trixie, and >> > will lifted after trixie ships, and applies implicitly to all the
~2000 binary pkgs that are affected by the above
While the CTTE placed the moratorium until the release of bookworm, the
release team extended it beyond, see
https://lists.debian.org/[email protected]. So you
need explicit agreement from the release team on your plan. Failing
that, any package that has been forcefully moved is immediately
rc-buggy due to failing a release team requirement.
Of course the release team needs to be on board, no questions about
that. But given the idea is to maintain their decision exactly as it
stands I wouldn't imagine it would be an issue? Once again, the
moratorium is explicitly about moving between locations _and_
packages, in combination, not either/or. From that same email you
linked:
"Files moving their canonical location between / and /usr (details in
[1]) *and* from one binary package to another binary package within
one release cycle remain an RC issue unless dpkg supports it."
I'm proposing to keep this in place as a general rule, with the new
escape hatch that you devised as the only addition.
Actually, the morotorium is not explicitly only about moving between
both locations and packages. Firstly, the TC's morotorium does not have
the qualification, restricting *any* movements in data.tar.*:
The Technical Committee recommends that during the Debian 12
development cycle, the maintainers of individual packages should not
proactively move files from the root filesystem to the corresponding
locations in /usr in the data.tar.* of packages. Files that were in
/usr in the Debian 11 release should remain in /usr, while files
that were in /bin, /lib* or /sbin in the Debian 11 release should
remain in those directories. If files were moved from /bin, /lib*
or /sbin into /usr since the Debian 11 release, they should be moved
back to their Debian 11 locations.
Then secondly, the RT's message is ambiguous, because it says both that
they want the /TC's/ morotorium to remain in place, and also that files moving between both locations and packages is an RC issue.
Until the RT's position is clarified, I think we should treat the
broader prohibition as what they require. The TC are going to discuss
this issue at our meeting on Tuesday, and one possible outcome is that
we reissue our version of the broader prohibition.
Stepping back:
I am far from being an expert on the details of merged-/usr. But one
thing I've noticed is that among the people who have spent the most time looking into it, the majority think that simple fixes are not going to
be sufficient. Only a few people who have spent a lot of time on it
still think that the fixes that are required are relatively simple ones.
If the former group are wrong then the transition takes longer than it
needs to, but we don't lose any confidence in the state of our users' installations.
If the latter group are wrong then we'll probably end up with a longer transition anyway, and a worse situation for both our maintainers and
our users. And it's within one of the areas of Debian that we're most
proud of -- completely smooth upgrades between stable releases.
So, I think we should assume the people who are more worried are right,
for the time being. We lose little in doing so.
Hi Luca,
On Sat, May 06, 2023 at 09:47:15PM +0100, Luca Boccassi wrote:
Sure, there are some things that need special handling, as you have
pointed out. What I meant is that I don't think we need special
handling for _all_ affected packages. AFAIK nothing is using
diversions for unit files or udev rules, for example (I mean if any
package is, please point it out, because I would like a word...). I
I've posted a list in https://lists.debian.org/[email protected] and indeed,
udev rules are being diverted in one case.
But then, you only capture diversions inside Debian's ecosystem and miss
out on other kinds of diversions such as local diversions. We currently support imposing local diversions on pretty much arbitrary files
including unit files. And while we've occasionally moved files between /
and /usr before the transition, doing it for 2000 packages significantly grows the risk of it affecting someone. So really, we want all such diversions duplicated before unpacking a package the has moved its
files. The way to achieve that is Pre-Depending on usr-is-merged. To me,
this sounds like we really want some special handling for all affected packages.
I also caution that we've started from a very simple approach and tried fixing it up to address the problems that we recognized in analyzing it.
My impression is that we are not finished with our discovery here and
won't be for quite some time. This uncertainty of what else we might
break is the most significant downside I see with your approach.
very strongly suspect this will be a small minority out of the total
of ~2k would need this treatment, and the vast majority would not. Do
you disagree with this gut feeling?
I do disagree indeed. While the special handling may be mostly
mechanical for the majority of packages, it still see it as required.
Worse, we also need to discuss how this affects backporting of packages.
Any package enabling the addon needs to have the addon removed for a
backport to undo the move. Worse, when backporting debhelper, any
package that uses the new compat level must explicitly disable the
addon. And then we may need to fix upgrade paths from backports to
stable.
Curve ball question: is there anything blocking us from canonicalizing PT_INTERP at the source in gcc so that the essential set (and
everything else, really) can work without the lib -> usr/lib symlink?
Would anything obvious break?
This would be in trixie, so even when unpacked on bookworm for the
upgrade case, the loader is guaranteed to be accessible from the
canonical path. Heck, we could even consider canonicalizing shebangs
in scripts shipped in essential packages? I'd imagine /bin/sh would
have the same issue as the loader?
Heh! As much as this initially sounds insanely crazy,
I was unable to
spot a reason for why this cannot work at all. It definitely sounds like fixing the bootstrap protocol to me. On the flip side, I'd assume the
loader location (for the purpose of examining ELF objects) to be hard
coded in quite some places. Changing gcc seems out of question, because
we also expect binaries built on Debian to run on non-Debian systems.
The local/external aspect is already covered in Ansgar's reply and subthread.
Sure, but adding changes that are (seemingly) unnecessary for a large percentage of affected packages also brings uncertainty. Every
software has bugs, thus it follows that injecting more software in the
way of a package being installed will likely also inject bugs. Which
doesn't mean we shouldn't consider it, however, it should be weighted appropriately.
Packages that need special handling will need special handling for backporting too. This is nothing new, there was never a project-wide guarantee that a package uploaded to testing can apply 1:1 to
backports, it is common enough to require changes/reverts/adjustments,
and if it's fine to require that in other cases, it's fine for this
case too.
If the majority of packages are simply converted, with no manual
handling and no diversion, then it should be simple to handle: the
debhelper in stable will not perform the conversion by definition as
the logic won't be present, and any dh upload to backports will have
such logic disabled, so that other packages that get uploaded to
backports and built with either the stable or the backports debhelper
won't have any change performed on them.
Or to put it in another way: I think our defaults should prioritize
the Debian native use case. Given we ship our loader in /usr/lib/ld*
now, it makes sense to me that the default in GCC is to point to /usr/lib/ld*. Callers can override that as needed for third-party/external/foreign use cases.
How about the long-term vision of this? Elsewhere you indicated that
you'd like the aliasing symlinks to not be shipped by any data.tar. Does that imply that we'd keep patching the interpreter and using /usr/bin/sh forever in the essential set? If adding the links to base-files, it
would be of temporary nature only.
If adding the symlinks to base-files, how about /lib64? Would we ship it for all architectures or just for those that need it (e.g. amd64,
loong64, mips64el, ppc64, ppc64el)? https://wiki.debian.org/ArchitectureSpecificsMemo has a list of dynamic loaders we also need /libx32 for x32 at least. If making this architecture-dependent, would base-files become Multi-Arch: same?
...
I think we should leave the long term vision for another day, and
focus on your requirements for the essential set unpacking right now.
Hi Luca,
On Sun, May 07, 2023 at 12:51:21PM +0100, Luca Boccassi wrote:
The local/external aspect is already covered in Ansgar's reply and subthread.
I hope that we can at least agree that we don't have consensus on this
view. And the more I think about it, the more it becomes clear to me
that this non-consensus is part of the larger disagreement we have about
this whole transition. Do you see any way towards getting to common
ground here?
Sure, but adding changes that are (seemingly) unnecessary for a large percentage of affected packages also brings uncertainty. Every
software has bugs, thus it follows that injecting more software in the
way of a package being installed will likely also inject bugs. Which doesn't mean we shouldn't consider it, however, it should be weighted appropriately.
Let me put this into perspective. In this scenario, we will have a few packages with versioned Pre-Depends on usr-is-merged. The seemingly unnecessary change here is adding more Pre-Depends of the same kind to
many more packages. It seems very likely to me that one of the few Pre-Depends will cause usr-is-merged to be upgraded early and thus those possibly unnecessary Pre-Dependencies will be harmless. Do you actually
have some scenario in mind that would warrant judging this as risky
beyond suspicion? (Which is not to say that there is no risk as the
whole affair bears quite some risk.)
Packages that need special handling will need special handling for backporting too. This is nothing new, there was never a project-wide guarantee that a package uploaded to testing can apply 1:1 to
backports, it is common enough to require changes/reverts/adjustments,
and if it's fine to require that in other cases, it's fine for this
case too.
It seems that you missed my argument and it likely wasn't spelled out explicitly enough, so let me retry. Yes, you may need to adapt packages
that are being backported. We don't disagree about that (and hope people
get it right, which they won't, but so be it). The really bad thing here
is that a backports upload may require changes to the package in
unstable!
Say we packaged foo version 1 in stable and it puts everything in /bin.
Then we update foo to version 2 in unstable and foo gains a new
/bin/bar. Due to the debhelper addon, this is actually shipped as /usr/bin/bar. Great. Then we backport foo version 2 to stable. Given
that debhelper no longer moves, it'll be /bin/bar. Then we notice that
foo is not laid out nicely and we split a bar package from it in version
3 and move /usr/bin/bar into bar. Now a user may install stable, install
foo version 1, install the foo version 2 backport and then update to nextstable. In that stable upgrade, bar version 3 may be unpacked before
foo version 3 and as a result /usr/bin/bar goes missing when the
backported foo version 2 gets upgraded to the regular foo version 3 as
this deletes /bin/bar.
So when we backport a package, the unstable package may need to be
modified to avoid such unpack file loss scenarios. In a simple case, we
may be able to just add Conflicts, but the takeaway is that backporting
a package may now break upgrades to nextstable in a way that requires
fixes in nextstable to accommodate for such upgrades.
If the majority of packages are simply converted, with no manual
I can see we don't agree on this matter, of course, that is clear. And
I hope we can find common ground. But let me provocatively ask this
first: is the same rule going to be enforced for all other changes
that happen in the project that might affect external packages? If
anybody points out past changes, recent or less recent, that caused
issues for third party packages, will the TC ask for those changes to
be reverted or otherwise modified accordingly? Will a change to Policy
be proposed that spells out that third party packages cannot ever be
broken, no matter what they do, and must always work?
The more pre-depends, the more constraints we put on apt. I do not
have a specific scenario in mind as we don't even have a full set of
changes to look at, but it seems clear to me it will have _some_
effect, no?
Sure that's a legitimate concern, however, wouldn't it fall into the
"needs special handling" bucket? It is a case where the file is moving
both in location and package, so it is covered by the blank statement
"either don't do that or implement the required workaround via diversion/conflict/etc". What am I missing?
But the more I think about it, the more I am convinced that the
default option working best for Debian is the one that matches the
project's choice of a filesystem layout. After all, this is
configurable in the toolchain for a reason.
And the vast majority of the rest of the world has long since finished
this transition, so I struggle to think where software built with this default wouldn't work. Bullseye will be oldoldstable at that point,
and even that was default merged for new installations, and really old
ones (oldoldoldoldstable at that point? I lost count) will be long
EOL. I suppose they could still be around unmaintained, but who uses a toolchain from 8 years in the future to build software for an EOL distribution 8 years in the past? Normally it's the other way around,
as even glibc adds new symbols and is not forward compatible.
On the ELF interpreter, as long as we can reasonably ensure it works,
I do believe we should switch it, regardless of what we do with the
symlinks, how we ship/add/build/package/create/manage them, as a
desired final state. Again, we should make the default in Debian work
for Debian. And given the default for Debian from Bookworm onward is
that the loader is in /usr/lib/, it seems perfectly reasonable to me
that it software built for Debian and shipped in Debian should look
there for it.
On Mon, May 08, 2023 at 02:07:08AM +0100, Luca Boccassi wrote:
I can see we don't agree on this matter, of course, that is clear. And
I hope we can find common ground. But let me provocatively ask this
first: is the same rule going to be enforced for all other changes
that happen in the project that might affect external packages? If
anybody points out past changes, recent or less recent, that caused
issues for third party packages, will the TC ask for those changes to
be reverted or otherwise modified accordingly? Will a change to Policy
be proposed that spells out that third party packages cannot ever be broken, no matter what they do, and must always work?
I'm not sure about the TC's role in this. For the record, I am doing all
of the analysis (and design work) in this thread without a TC hat. I
also cannot comment on what the TC is going to rule this matter. Can we
leave that aside or formally file it there if you see a need?
I agree that what we support is vague at best and we can readily see
from earlier conflicts that this is a recurring matter. We still
disagree over how much maintainers should support sysvinit. I've also
quite recently failed at properly preparing a transition (non-essential adduser) and while we could write about it in release-notes, what is
going to happen is that we'll revert it for bookworm and then I can
retry properly.
You may also have noticed that my analysis of possible problems in this thread very much reasons about packages shipped in Debian releases. I
would actually like to call external packages and local diversions unsupported, but I was rightfully criticised that this is falling short.
So no, I cannot tell you where the boundary of our support is. I
initially assumed it to be closer to where you paint it and am now
trying to adapt to meet the expectations of others.
For instance, I've also reached out to DSA and inquired on their use.
While I haven't found local diversions or local statoverrides in dsa-puppet.git, it seems that a number of external packages ship files
in /sbin or /lib (including udev rules and systemd units).
The more pre-depends, the more constraints we put on apt. I do not
have a specific scenario in mind as we don't even have a full set of changes to look at, but it seems clear to me it will have _some_
effect, no?
We've been there with multiarch-support and my experience with that
suggests that the primary effect is increasing the size of Packages
files. Though given that you are obviously worried here, I suppose more research is warranted.
Sure that's a legitimate concern, however, wouldn't it fall into the
"needs special handling" bucket? It is a case where the file is moving
both in location and package, so it is covered by the blank statement "either don't do that or implement the required workaround via diversion/conflict/etc". What am I missing?
You are missing the distribution of responsibility. Quite commonly,
backports are performed by someone else than the package maintainer.
Yet, an uncoordinated backport can now render the package in unstable rc-buggy.
| Sysop: | Keyop |
|---|---|
| Location: | Huddersfield, West Yorkshire, UK |
| Users: | 715 |
| Nodes: | 16 (2 / 14) |
| Uptime: | 08:30:58 |
| Calls: | 12,100 |
| Files: | 15,003 |
| Messages: | 6,517,955 |