LTO significantly increase memory requirements for buildd machines. Do we have enough RAM and swap on each build server?

Link time optimizations are also at least turned on in other distros like Fedora, OpenSuse (two years) and Ubuntu (one year).

I know Ubuntu has builders with 8 GB RAM + 4 GB swap which is not enough in
all cases. https://answers.launchpad.net/launchpad/+question/694428
-----BEGIN PGP SIGNATURE-----

iHUEABYIAB0WIQQRm7llN8yxifaG60cF2qh9JI3wlQUCYqxMEgAKCRAF2qh9JI3w lZweAQCEPtqpkPXUr5MDmK3BpdrC4jFWlN4viZkOGdP+Nq7UfAEAhoqeVpYTaeEr RrzUKABGoPmDQaErA1CmJYOE8k9EyQE=
=n4lP
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Fri, Jun 17, 2022 at 10:18:43AM +0200, Matthias Klose wrote:

...
The proposal is to turn on LTO by default on most 64bit release architectures.
...

By what factor does -ffat-lto-objects increase disk space usage during
package builds?

Please coordinate with DSA to ensure that the buildds on these
architectures have sufficient diskspace.

amd64 buildds have/had(?) only 74 GB of diskspace, which has even
without LTO already forced some packages to do manual cleanup steps
during the build to stay within the limited disk space.

...
Link time
optimizations are also at least turned on in other distros like Fedora, OpenSuse (two years) and Ubuntu (one year).
...
The idea is to file wishlist bug reports for those 373 packages and then see how far we get, and if it's feasible to already turn on LTO for bookworm.
If not, it should be turned on by default for the following release.

I assume these 373 packages have already been fixed/workarounded in Ubuntu? Submitting 373 bugs with patches should settle the feasibility question.

A bigger worry is the schedule of such a change.
A major toolchain change shortly before the freeze means the vast
majority of packages will be shipped with non-LTO builds in the release,
with security updates or point release updates triggering a change to
an LTO built package.
This means few packages actually benefitting from LTO, but a higher
regression risk when fixing bugs in stable.
The best timing for such a change would be immediately after the release
of bookworm.

Matthias

cu
Adrian

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Fri, Jul 01, 2022 at 07:52:16PM +0300, Adrian Bunk wrote:

On Fri, Jun 17, 2022 at 10:18:43AM +0200, Matthias Klose wrote:

The proposal is to turn on LTO by default on most 64bit release architectures.

By what factor does -ffat-lto-objects increase disk space usage during package builds?

Please coordinate with DSA to ensure that the buildds on these
architectures have sufficient diskspace.

amd64 buildds have/had(?) only 74 GB of diskspace, which has even
without LTO already forced some packages to do manual cleanup steps
during the build to stay within the limited disk space.

As amd64 is a fat arch these days, there's no excuse to not simply throw
more hardware at this particular problems. We do have excess money and sponsors.

A bigger worry is the schedule of such a change.
A major toolchain change shortly before the freeze means the vast
majority of packages will be shipped with non-LTO builds in the release,

So let's do such changes NOW rather than in December.


Meow!
--
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ What kind of a drug are "base" and "red pill"? I think acid is
⢿⡄⠘⠷⠚⠋⠀ LSD, which would make base... ? Judging from the behaviour of
⠈⠳⣄⠀⠀⠀⠀ those "based and redpilled", something nasty.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Fri, Jun 17, 2022 at 12:40:34PM +0300, Nicholas Guriev wrote:

LTO significantly increase memory requirements for buildd machines. Do we have
enough RAM and swap on each build server?

Link time optimizations are also at least turned on in other distros like Fedora, OpenSuse (two years) and Ubuntu (one year).

I know Ubuntu has builders with 8 GB RAM + 4 GB swap which is not enough in all cases. https://answers.launchpad.net/launchpad/+question/694428

This is on 𝐚𝐦𝐝𝟔𝟒. You can have 24𝐓B RAM boxen these days. Assigning 8GB
to a buildd is a configuration error -- it's not about building package X
at home or porting it to a bitty box.

The proposal is to turn on LTO by default on most 64bit release architectures.

As doko CCed -ports:
With the hat of one of riscv64 porters on: hardware that is deemed
sufficient for buildds has 16 GB RAM and nvme, with very slow CPUs.
Thus, please include riscv64 among LTO archs, barring compiler issues
of course.


Meow!
--
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ What kind of a drug are "base" and "red pill"? I think acid is
⢿⡄⠘⠷⠚⠋⠀ LSD, which would make base... ? Judging from the behaviour of
⠈⠳⣄⠀⠀⠀⠀ those "based and redpilled", something nasty.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Adrian Bunk:

On Fri, Jun 17, 2022 at 10:18:43AM +0200, Matthias Klose wrote:

...

...
Link time
optimizations are also at least turned on in other distros like Fedora, OpenSuse (two years) and Ubuntu (one year).
...
The idea is to file wishlist bug reports for those 373 packages and then see
how far we get, and if it's feasible to already turn on LTO for bookworm. If not, it should be turned on by default for the following release.

I assume these 373 packages have already been fixed/workarounded in Ubuntu? Submitting 373 bugs with patches should settle the feasibility question.

For my software (bart), it triggers a compiler bug where the
compiler crashes.

Martin

A bigger worry is the schedule of such a change.
A major toolchain change shortly before the freeze means the vast
majority of packages will be shipped with non-LTO builds in the release,
with security updates or point release updates triggering a change to
an LTO built package.
This means few packages actually benefitting from LTO, but a higher regression risk when fixing bugs in stable.
The best timing for such a change would be immediately after the release
of bookworm.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)