Package: dpkg
Version: 1.22.4
Severity: important
xz-utils 5.6.0 has been uploaded to unstable. A changed behaviour of
`xz' is now that mutlti threaded compress/ decompression is now enabled
by default. This in turn leads to warnings if the requested amount of
memory exceeds the available amount. A snippet from
https://ci.debian.net/data/autopkgtest/testing/armel/d/dpkg/43341232/log.gz
| 88s /tmp/autopkgtest-lxc.dumkcbm0/downtmp/build.4CO/src/src/at/deb-format.at:518:
| 88s # Extract the base members
| 88s xz -c control.tar >control.tar.xz
| 88s xz -c data.tar >data.tar.xz
| 88s
| 89s --- /dev/null 2024-02-26 09:29:33.669234548 +0000
| 89s +++ /tmp/autopkgtest-lxc.dumkcbm0/downtmp/autopkgtest_tmp/src/at/testsuite.dir/at-groups/4/stderr 2024-02-26 09:30:58.601386838 +0000
| 89s @@ -0,0 +1,2 @@
| 89s +xz: Reduced the number of threads from 16 to 8 to not exceed the memory usage limit of 1400 MiB
| 89s +xz: Reduced the number of threads from 16 to 8 to not exceed the memory usage limit of 1400 MiB
| 89s 4. deb-format.at:511: FAILED (deb-format.at:518)
Allowing output on stderr would be a possible tix.
Hi!Hi Guillem,
| 89s +xz: Reduced the number of threads from 16 to 8 to not exceed the memory usage limit of 1400 MiB
| 89s +xz: Reduced the number of threads from 16 to 8 to not exceed the memory usage limit of 1400 MiB
| 89s 4. deb-format.at:511: FAILED (deb-format.at:518)
Allowing output on stderr would be a possible tix.
Hmm, that's warning is unfortunate, and a quick check at the xz code
didn't reveal a way to only suppress it. :/
For dpkg, I'd like to either not get the warning or being able to tell xz that even if the threads are reduced, that's ok and it should not warn
about that (but that would then require depending on a new enough
version supporting that, perhaps that could be suppresses instead via
an envvar) so that?
Ignoring stderr could be a workaround, but I'd need to do something as
well for the libdpkg code and the perl code calling xz, which will get
very annoying.
This is also going to get in the way of migrating both xz and dpkg
(which seems like would need to be uploaded today for the time64
transition).
Or perhaps that warning could be disabled for now in Debian until things
are sorted out with upstream?
(Had not seen this test suite failure yet, as I've got xz on hold due
to the breaks on pristine-tar, but would probably stumble over it
during the release process anyway I guess, so thanks for the heads up!)
Thanks,
Guillem
On 2024-02-26 19:23:58 [+0100], Guillem Jover wrote:
| 89s +xz: Reduced the number of threads from 16 to 8 to not exceed the memory usage limit of 1400 MiB
| 89s +xz: Reduced the number of threads from 16 to 8 to not exceed the memory usage limit of 1400 MiB
| 89s 4. deb-format.at:511: FAILED (deb-format.at:518)
Allowing output on stderr would be a possible tix.
Hmm, that's warning is unfortunate, and a quick check at the xz code
didn't reveal a way to only suppress it. :/
For dpkg, I'd like to either not get the warning or being able to tell xz that even if the threads are reduced, that's ok and it should not warn about that (but that would then require depending on a new enough
version supporting that, perhaps that could be suppresses instead via
an envvar) so that?
Could poke upstream.
Ignoring stderr could be a workaround, but I'd need to do something as
well for the libdpkg code and the perl code calling xz, which will get
very annoying.
This is also going to get in the way of migrating both xz and dpkg
(which seems like would need to be uploaded today for the time64 transition).
Or perhaps that warning could be disabled for now in Debian until things are sorted out with upstream?
Falling back to -T1 in order to keep the previous is not an option?
Let me try to sell this "we move this warning to verbose" to upstream in
the meantime…
(Had not seen this test suite failure yet, as I've got xz on hold due
to the breaks on pristine-tar, but would probably stumble over it
during the release process anyway I guess, so thanks for the heads up!)
This poped up on xz debci only armel and armhf.
Ignoring stderr could be a workaround, but I'd need to do something as well for the libdpkg code and the perl code calling xz, which will get very annoying.
This is also going to get in the way of migrating both xz and dpkg
(which seems like would need to be uploaded today for the time64 transition).
Or perhaps that warning could be disabled for now in Debian until things are sorted out with upstream?
Falling back to -T1 in order to keep the previous is not an option?
I'm not sure I understand what you are saying. Do you mean dpkg would fallback to pass -T1 (maybe you mean -T+1, otherwise we might get unreproducible output due to switching to single-threaded)? And
fallback on what condition?
Ah, I think you mean to pass -T+1 to the xz invocations for the test
suite, right, that could workaround the issue there indeed. Thanks, I
think I'll do that for now.
Let me try to sell this "we move this warning to verbose" to upstream in the meantime…
That would actually be great!
(Had not seen this test suite failure yet, as I've got xz on hold due
to the breaks on pristine-tar, but would probably stumble over it
during the release process anyway I guess, so thanks for the heads up!)
This poped up on xz debci only armel and armhf.
Perhaps I'll not see it in my local tree then, but I think the dpkg
failure will at least block xz migration, but thinking about it now,
probably not dpkg's migration. So this might not be blocking for dpkg.
(Sorry if this sounded alarming, but didn't want to add new roadblocks
to the time64 transition from the dpkg side! :D)
Thanks,
Guillem
| Sysop: | Keyop |
|---|---|
| Location: | Huddersfield, West Yorkshire, UK |
| Users: | 715 |
| Nodes: | 16 (0 / 16) |
| Uptime: | 161:57:43 |
| Calls: | 12,094 |
| Calls today: | 2 |
| Files: | 15,000 |
| Messages: | 6,517,778 |