• Re: Open Source does not mean easily re-compile-able

    From Richard Kettlewell@21:1/5 to Kenny McCormack on Fri Dec 27 14:56:28 2024
    [email protected] (Kenny McCormack) writes:
    Kalevi Kolttonen <[email protected]> wrote:
    ...
    Thunderbird is open source, maybe even free software.

    I am sure it is extremely easy to either remove the
    "reply" button or to bind it to the "followup" function.

    I doubt it is any simple matter to find it in the massive source code,
    much less to figure out how to fix it, much less figure out how to successfully re-build (and test!) it after making your changes.

    Generally, most complicated GUI software, even if technically Open
    Source, is difficult, if not impossible, for ordinary people using
    ordinary machines, to re-build from source. Among other things, you
    will generally end up spending many hours, if not days, getting all
    the dependencies installed.

    On Debian-derived platforms, that’s what apt-get build-dep is for.
    Source package rebuild is also standardized. It looks like the RH world
    has something pretty similar.

    So I don’t think the need to install build dependencies is likely to be
    a real obstacle to rebuilding Thunderbird.

    --
    https://www.greenend.org.uk/rjk/

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Kenny McCormack@21:1/5 to [email protected] on Fri Dec 27 16:14:20 2024
    In article <[email protected]>,
    Richard Kettlewell <[email protected]d> wrote:
    ...
    On Debian-derived platforms, thats what apt-get build-dep is for.
    Source package rebuild is also standardized. It looks like the RH world
    has something pretty similar.

    I know all that - and, in theory, it should "just work".

    But my experience is that theory and practice diverge.

    Now, I may not be the most capable person in the world, probably not even
    in the top 10 (or 100). But that's exactly my point. It's just not an
    easy task for ordinary people under ordinary circumstances.

    --
    The randomly chosen signature file that would have appeared here is more than 4 lines long. As such, it violates one or more Usenet RFCs. In order to remain in compliance with said RFCs, the actual sig can be found at the following URL:
    http://user.xmission.com/~gazelle/Sigs/Cancer

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Salvador Mirzo@21:1/5 to Kenny McCormack on Fri Dec 27 15:07:59 2024
    [email protected] (Kenny McCormack) writes:

    In article <[email protected]>,
    Richard Kettlewell <[email protected]d> wrote:
    ...
    On Debian-derived platforms, thats what apt-get build-dep is for.
    Source package rebuild is also standardized. It looks like the RH world >>has something pretty similar.

    I know all that - and, in theory, it should "just work".

    But my experience is that theory and practice diverge.

    Now, I may not be the most capable person in the world, probably not even
    in the top 10 (or 100). But that's exactly my point. It's just not an
    easy task for ordinary people under ordinary circumstances.

    If I were not full of tasks right now, I would set up a VM with Debian
    and try it out---build-dep for Thunderbird. Just to see if compiles successfully without much hacking involved. I am also skeptical of such things. It usually works on smaller projects; I'd be surprised and
    happy to find out that it works with no hacking involved.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Paul@21:1/5 to Kenny McCormack on Fri Dec 27 13:11:29 2024
    On Fri, 12/27/2024 11:14 AM, Kenny McCormack wrote:
    In article <[email protected]>,
    Richard Kettlewell <[email protected]d> wrote:
    ...
    On Debian-derived platforms, thats what apt-get build-dep is for.
    Source package rebuild is also standardized. It looks like the RH world
    has something pretty similar.

    I know all that - and, in theory, it should "just work".

    But my experience is that theory and practice diverge.

    Now, I may not be the most capable person in the world, probably not even
    in the top 10 (or 100). But that's exactly my point. It's just not an
    easy task for ordinary people under ordinary circumstances.


    One of the solutions, is to not put email and USENET news, on the same tool.

    I hit the reply by accident the other day here, and when I clicked
    Send, a dialog whined about "Could not send because..." I have no email
    account set up on the thing. Of course it cannot erroneously send
    to email, because there is no email. The Reply button then is neutered.
    I changed over to Followup, sent to newsgroup, and finished the job.

    IDK about capable, but Thunderbird is a huge package, and the
    installation may require Rust and some other materials to be
    loaded. The description for building, may have started with
    a Mercurial (Hg) clone, and then the build is based on that.
    The tarball off the site, would be insufficient for an immediate
    build. The recipe no longer describes starting with a tarball.

    As a consequence, the distro people might not have a "conventional"
    setup for Thunderbird. One of the distros, a .deb arrives from
    Mozilla in a cardboard box. As a result of the package
    being manufactured elsewhere, instead of in-house in the
    Buildmeister corral, the source option just might be missing.

    Still, even without doing the actual build, it would be fun
    to tease the distro and see what it has to offer, and see whether
    a patched source magically shows up. It would be good to know
    whether in "difficult cases", it actually arrives ready to be
    built for you.

    For some of these "big" projects, a machine with 32GB of RAM
    is recommended. Which helps with the linking phase. Some
    of the compiling, the RAM footprint isn't all that large.
    The procedure used to have a hideous linkage methodology
    at one time, but that got modified a bit. To make a 32-bit
    copy of the thing, it is best to build on a 64-bit OS!
    If your distro isn't 64-bit, it would be not-possible to finish.

    While a lot of distros are 64-bit only, you can *still* find 32-bit ones.
    But not for much longer. Building Thunderbird on the one with
    the arrow below, that could be a waste of your time. I don't know if
    some PAE thing would work or not in 32-bit mode. I'm using a
    mirror here, because the main site isn't as easy to get around on.

    https://mirror.csclub.uwaterloo.ca/linuxmint/debian/

    lmde-6-cinnamon-32bit.iso 22-Sep-2023 17:07 2G <=== lmde-6-cinnamon-64bit.iso 22-Sep-2023 16:26 3G
    sha256sum.txt 01-Feb-2024 16:09 552
    sha256sum.txt.gpg 01-Feb-2024 16:10 833

    Paul

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Lawrence D'Oliveiro@21:1/5 to Richard Kettlewell on Fri Dec 27 23:11:32 2024
    On Fri, 27 Dec 2024 14:56:28 +0000, Richard Kettlewell wrote:

    On Debian-derived platforms, that’s what apt-get build-dep is for.

    Even if you want your build to be different from the Debian package, apt-
    get build-dep will still likely get you 90% of the way to installing all
    the needed dependencies.

    I used this as a starting point for my Blender build. Now that is a
    package with some complex dependencies.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Lawrence D'Oliveiro@21:1/5 to Salvador Mirzo on Fri Dec 27 23:09:40 2024
    On Fri, 27 Dec 2024 15:07:59 -0300, Salvador Mirzo wrote:

    If I were not full of tasks right now, I would set up a VM with Debian
    and try it out---build-dep for Thunderbird. Just to see if compiles successfully without much hacking involved.

    It will certainly work for (re)building the Debian package. Remember,
    that’s how they build Debian in the first place.

    Also, you can try a container rather than a full VM.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Kalevi Kolttonen@21:1/5 to Salvador Mirzo on Fri Dec 27 23:22:17 2024
    Salvador Mirzo <[email protected]> wrote:
    If I were not full of tasks right now, I would set up a VM with Debian
    and try it out---build-dep for Thunderbird. Just to see if compiles successfully without much hacking involved. I am also skeptical of such things. It usually works on smaller projects; I'd be surprised and
    happy to find out that it works with no hacking involved.

    No need to be skeptical, we live in modern ages
    where things have been made quite convenient for us.
    Compiling Thunderbird should be very easy indeed
    when we use Linux distro's package management.

    I run Fedora Linux 41 xfce spin and I love it. If my
    memory serves right, so far I have performed the
    following steps:

    1) Download the Thunderbird source RPM
    dnf download --source thunderbird

    2) Install the source RPM
    rpm -Uvh thunderbird-128.5.2-1.fc41.src.rpm

    3) Bump release from 2 to 3
    vi ~/rpmbuild/SPECS/thunderbird.spec

    4) Extract the tar.xz
    tar xJf thunderbird-128.5.2esr.source.tar.xz

    5) Edit function MsgReplyMessage to contain just "return;"

    vi +/MsgReplyMessage thunderbird-128.5.2/comm/suite/mailnews/content/mailWindowOverlay.js

    6) Recreate the tar.xz
    tar cJf thunderbird-128.5.2esr.source.tar.xz thunderbird-128.5.2

    7) Install all RPM build dependencies, letting dnf do the heavy lifting
    dnf builddep ~/rpmbuild/SPECS/thunderbird.spec

    8) Build Thunderbird binary RPM:

    rpmbuild -bb ~/rpmbuild/SPECS/thunderbird.spec

    Since Thunderbird is pretty huge, I am guessing that
    the build will take some time to complete.

    In Finland it is now 01:17 o'clock in the middle of the
    night. Unfortunately I have to go to work tomorrow so
    I must go to sleep at 2:00. I have no idea when the
    build will be complete or whether my JavaScript hack
    works or not.

    br,
    KK

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Janis Papanagnou@21:1/5 to Kalevi Kolttonen on Sat Dec 28 00:44:10 2024
    On 28.12.2024 00:22, Kalevi Kolttonen wrote:
    [...]

    No need to be skeptical, we live in modern ages
    where things have been made quite convenient for us.

    LOL. :-)

    Compiling Thunderbird should be very easy indeed
    when we use Linux distro's package management.

    You expect _users_ of tools to use a _development_
    environment to fix *inherent* shortcomings of a tool?
    (Shortcomings that should not be there in the first
    place!)

    [ snip suggested 8-step development process ]

    Janis

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Kalevi Kolttonen@21:1/5 to Kalevi Kolttonen on Sat Dec 28 00:11:48 2024
    Kalevi Kolttonen <[email protected]> wrote:
    I got 16GM of RAM. Maybe the build parameters in the
    spec file are too aggressive for this modest laptop,
    but I did not find any "make -j" invocation.

    Now I have to sleep and maybe try it again tomorrow.

    Still awake for a while. I looked at the spec file
    and replaced:

    ./mach build -v

    with

    ./mach build -j6 -v

    Maybe in the morning we have a complete build.

    br,
    KK

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Kalevi Kolttonen@21:1/5 to Janis Papanagnou on Fri Dec 27 23:56:50 2024
    Janis Papanagnou <[email protected]> wrote:
    On 28.12.2024 00:22, Kalevi Kolttonen wrote:
    [...]

    No need to be skeptical, we live in modern ages
    where things have been made quite convenient for us.

    LOL. :-)

    My comment above was a reference to the bad old
    days when you had to manually download tar.gz packages
    and compile them to satisfy dependencies. Now the
    builds are super easy with the help of package management.

    Compiling Thunderbird should be very easy indeed
    when we use Linux distro's package management.

    You expect _users_ of tools to use a _development_
    environment to fix *inherent* shortcomings of a tool?
    (Shortcomings that should not be there in the first
    place!)

    Why would you think so? This is just one way to
    solve the problem. I would never ever use TB
    for anything. I have used a basic newsreader called
    tin for over 30 years now. It works fine as I have
    no need for any fancy features.


    Unfortunately I could not complete my experiment.
    The build process jammed my Lenovo Thinkpad so bad
    that it was completely stuck, probably because
    of build parallelism and memory hogging.

    $ grep ^proc /proc/cpuinfo
    processor : 0
    processor : 1
    processor : 2
    processor : 3
    processor : 4
    processor : 5
    processor : 6
    processor : 7
    processor : 8
    processor : 9
    processor : 10
    processor : 11

    $ grep ^vendor /proc/cpuinfo |head -1
    vendor_id : AuthenticAMD

    I got 16GM of RAM. Maybe the build parameters in the
    spec file are too aggressive for this modest laptop,
    but I did not find any "make -j" invocation.

    Now I have to sleep and maybe try it again tomorrow.

    br,
    KK

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Salvador Mirzo@21:1/5 to Kalevi Kolttonen on Fri Dec 27 21:22:03 2024
    [email protected] (Kalevi Kolttonen) writes:

    Kalevi Kolttonen <[email protected]> wrote:
    I got 16GM of RAM. Maybe the build parameters in the
    spec file are too aggressive for this modest laptop,
    but I did not find any "make -j" invocation.

    Now I have to sleep and maybe try it again tomorrow.

    Still awake for a while. I looked at the spec file
    and replaced:

    ./mach build -v

    with

    ./mach build -j6 -v

    Maybe in the morning we have a complete build.

    Thanks for doing the work and keeping us posted. I wish you a good
    night and that the compilation completes just fine.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Lawrence D'Oliveiro@21:1/5 to Janis Papanagnou on Sat Dec 28 02:07:44 2024
    On Sat, 28 Dec 2024 00:44:10 +0100, Janis Papanagnou wrote:

    On 28.12.2024 00:22, Kalevi Kolttonen wrote:

    Compiling Thunderbird should be very easy indeed when we use Linux
    distro's package management.

    You expect _users_ of tools to use a _development_ environment to fix *inherent* shortcomings of a tool?

    On Linux, there is no “development environment” versus “user environment”.
    The same packaging tools work with both source code and built binaries.

    Platforms like Microsoft and Apple try to build a wall between two
    separate modes of using the system; Linux doesn’t.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Janis Papanagnou@21:1/5 to Kalevi Kolttonen on Sat Dec 28 19:27:07 2024
    On 28.12.2024 00:56, Kalevi Kolttonen wrote:
    Janis Papanagnou <[email protected]> wrote:
    On 28.12.2024 00:22, Kalevi Kolttonen wrote:
    [...]

    No need to be skeptical, we live in modern ages
    where things have been made quite convenient for us.

    LOL. :-)

    My comment above was a reference to the bad old
    days when you had to manually download tar.gz packages
    and compile them to satisfy dependencies. Now the
    builds are super easy with the help of package management.

    I see; you were referring to the way the technical process
    works.

    Personally I don't think that package managers contribute
    a lot since for ordinary users it's the same whether the
    package managers install a binary package or a source that
    is compiled under the hood. The difference is that source
    package needs a development environment (compiler, etc.)
    that "ordinary users" might not have installed or may not
    want to get installed (just for that).


    Compiling Thunderbird should be very easy indeed
    when we use Linux distro's package management.

    You expect _users_ of tools to use a _development_
    environment to fix *inherent* shortcomings of a tool?
    (Shortcomings that should not be there in the first
    place!)

    Why would you think so? This is just one way to
    solve the problem. [...]

    For a specific type of users. - The description you gave
    was describing a development process; that's not something
    that ordinary users would typically do (or want to do).

    Your problem solving suggestion goes even farther with yet
    more inherent issues that users of package managers might
    not like (editing sources, bypassing standard installation
    of regular updates with an own [temporary] version/branch).

    Janis

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Janis Papanagnou@21:1/5 to Lawrence D'Oliveiro on Sat Dec 28 19:40:46 2024
    On 28.12.2024 03:07, Lawrence D'Oliveiro wrote:
    On Sat, 28 Dec 2024 00:44:10 +0100, Janis Papanagnou wrote:

    On 28.12.2024 00:22, Kalevi Kolttonen wrote:

    Compiling Thunderbird should be very easy indeed when we use Linux
    distro's package management.

    You expect _users_ of tools to use a _development_ environment to fix
    *inherent* shortcomings of a tool?

    On Linux, there is no “development environment” versus “user environment”.
    The same packaging tools work with both source code and built binaries.

    You think it's normal that on a Linux installation where, say, no 'cc'
    (as prominent example of a development tool) is installed the package
    manager would first install ALL the necessary compilers and scripting
    languages just to create a binary (as opposed to just installing the
    binary)? - I'm not sure I understood what you were saying or aiming at especially in context of what I had been saying.

    For the Unix systems I worked with (commercial Unixes, Cygwin, Linux)
    there was no development environment guaranteed to be in the default installation.

    Janis

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Kalevi Kolttonen@21:1/5 to Kalevi Kolttonen on Sat Dec 28 19:48:51 2024
    Kalevi Kolttonen <[email protected]> wrote:
    ./mach build -v

    with

    ./mach build -j6 -v

    Okay, I am finally back at home and it is 21:43
    o'clock.

    The change above did not help because earlier in the
    spec file there is this:

    # Require 4 GB of RAM per CPU core
    %constrain_build -m 4096

    I now have:

    %constrain_build -m 1024

    But I am still very baffled. I tried to build TB
    without any source code modifications to make sure
    that the building process with rpmbuild works okay.

    Instead of success, my build has terminated with:

    RPM build errors:
    No patch number 416
    No patch number 419
    Bad exit status from /var/tmp/rpm-tmp.N27puv (%build)

    I cannot understand this because all references to
    patches 416 and 419 are commented out in the Fedora
    41 spec file. I now completely removed them and will
    try again...

    br,
    KK

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From James Kuyper@21:1/5 to Janis Papanagnou on Sat Dec 28 14:26:22 2024
    On 12/27/24 18:44, Janis Papanagnou wrote:
    On 28.12.2024 00:22, Kalevi Kolttonen wrote:
    [...]

    No need to be skeptical, we live in modern ages
    where things have been made quite convenient for us.

    LOL. :-)

    Compiling Thunderbird should be very easy indeed
    when we use Linux distro's package management.

    You expect _users_ of tools to use a _development_
    environment to fix *inherent* shortcomings of a tool?
    (Shortcomings that should not be there in the first
    place!)

    IIRC, this is in reference to my difficulty when Thunderbird changed the
    Reply button to mean "Reply" rather than "Followup", and instead added a
    new button that is labelled "Followup". I have never complained about
    that change - it was an entirely sensible one. I'm just having trouble re-training myself to use the newer, more sensible interface in a few
    years after spending a couple of decades using the older, less sensible
    one. And I fully appreciate other people's irritation at my difficulty
    with re-training.
    I wouldn't mind if they reinstated the ability, which existed in older
    versions of Thunderbird, to rearrange the list of buttons that are
    displayed. I do complain about the removal of that customization
    ability. I don't want to go back to those older versions because that
    would mean undoing other improvements. I'm especially worried about
    undoing security bug fixes.

    I don't like the idea of creating my own personal version of Thunderbird
    by modifying their source code, because it means I would have to re-do
    the build every time they put out a new version. I want quick and easy
    upgrades to newer versions, especially security bug fixes, and that
    desire conflicts with the desire for customization.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Kalevi Kolttonen@21:1/5 to Kalevi Kolttonen on Sat Dec 28 20:30:07 2024
    Kalevi Kolttonen <[email protected]> wrote:
    I cannot understand this because all references to
    patches 416 and 419 are commented out in the Fedora
    41 spec file. I now completely removed them and will
    try again...

    I am having massive problems with having only 16GB of
    RAM.

    Using 'top', I was able to see that Rust compiler 'rustc'
    was hogging something like 11GB of memory, and then
    after a while OOM killer got rid of the Rust compiler
    process. I am also seeing swapping take place when I
    attempt the build.

    It is really painful but I guess I have to use
    just a single CPU:

    ./mach build -j1 -v

    Because of this, the build takes forever to complete.

    br,
    KK

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Kalevi Kolttonen@21:1/5 to Kalevi Kolttonen on Sat Dec 28 21:07:19 2024
    Kalevi Kolttonen <[email protected]> wrote:
    It is really painful but I guess I have to use
    just a single CPU:

    ./mach build -j1 -v

    Because of this, the build takes forever to complete.

    Despite that, rpmbuild was using three CPUS. Now trying
    to add this too:

    RPM_BUILD_NCPUS=1 rpmbuild -bb ~/rpmbuild/SPECS/thunderbird.spec


    The build is in progress again and things look better now:

    $ grep MOZ_MAKE_FLAGS /var/tmp/rpm-tmp.5ctL0k
    echo "mk_add_options MOZ_MAKE_FLAGS=\"-j1\"" >> .mozconfig

    Without explicit RPM_BUILD_NCPUS=1, rpmbuild defaulted to
    3.

    br,
    KK

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Lawrence D'Oliveiro@21:1/5 to Janis Papanagnou on Sat Dec 28 23:03:59 2024
    On Sat, 28 Dec 2024 19:27:07 +0100, Janis Papanagnou wrote:

    Personally I don't think that package managers contribute a lot since
    for ordinary users it's the same whether the package managers install a binary package or a source that is compiled under the hood.

    Package managers contribute a lot to both tasks. On Linux, there is no “development environment” versus “user environment”. The same packaging tools work with both source code and built binaries.

    The difference is that source package needs a development
    environment (compiler, etc.) that "ordinary users" might not have
    installed or may not want to get installed (just for that).

    Platforms like Microsoft and Apple try to build a wall between two
    separate “developer” versus “ordinary user” modes of using the system; Linux doesn’t.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Lawrence D'Oliveiro@21:1/5 to Janis Papanagnou on Sat Dec 28 23:00:32 2024
    On Sat, 28 Dec 2024 19:40:46 +0100, Janis Papanagnou wrote:

    You think it's normal that on a Linux installation where, say, no 'cc'
    (as prominent example of a development tool) is installed the package
    manager would first install ALL the necessary compilers and scripting languages just to create a binary (as opposed to just installing the
    binary)?

    The discussion has to do with creating your own version of the binary,
    rather than using the repo-provided version.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Kalevi Kolttonen@21:1/5 to Lawrence D'Oliveiro on Sat Dec 28 23:32:37 2024
    Lawrence D'Oliveiro <[email protected]d> wrote:
    On Sat, 28 Dec 2024 19:40:46 +0100, Janis Papanagnou wrote:

    You think it's normal that on a Linux installation where, say, no 'cc'
    (as prominent example of a development tool) is installed the package
    manager would first install ALL the necessary compilers and scripting
    languages just to create a binary (as opposed to just installing the
    binary)?

    The discussion has to do with creating your own version of the binary,
    rather than using the repo-provided version.

    Right.

    Anyway, to be honest, I never realized how bloated Thunderbird is.
    The source RPM thunderbird-128.5.2-1.fc41.src.rpm is 690MB and
    the main source directory unpacked is:

    ~/tmp/tb/thunderbird-128.5.2 $ du -sh
    4.2G .

    Building TB with the help of a pre-made spec file on Fedora is
    probably very much easier than doing 'git clone' and trying to
    build it from there. Using 'dnf', it was just one command to
    download all the dependencies. I suppose the size of the
    dependency packages was 260MB in total. It would be a nightmare
    having to download them manually and then building them.

    Packages are just so handy.

    Fedora and Red Hat have already done the hard work so it
    is wise to use their source RPM as a basis for your own
    modifications when you are on Fedora or Red Hat Enterprise
    Linux.

    My single CPU Thunderbird build has now lasted for over two
    and half hours and I have no clue when it will be ready.

    This codebase is absolutely massive! I am beginning to
    lose patience.

    br,
    KK

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Grant Taylor@21:1/5 to Kalevi Kolttonen on Sat Dec 28 19:02:48 2024
    On 12/28/24 17:32, Kalevi Kolttonen wrote:
    This codebase is absolutely massive! I am beginning to lose patience.

    I remember when compiling X11 was a good burn in text for a processor.

    Then it was the Linux kernel.

    Now it's Thunderbird / Firefox, followed by Chromium, followed by Rust.



    --
    Grant. . . .

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Paul@21:1/5 to Kalevi Kolttonen on Sat Dec 28 21:12:38 2024
    On Sat, 12/28/2024 6:32 PM, Kalevi Kolttonen wrote:
    Lawrence D'Oliveiro <[email protected]d> wrote:
    On Sat, 28 Dec 2024 19:40:46 +0100, Janis Papanagnou wrote:

    You think it's normal that on a Linux installation where, say, no 'cc'
    (as prominent example of a development tool) is installed the package
    manager would first install ALL the necessary compilers and scripting
    languages just to create a binary (as opposed to just installing the
    binary)?

    The discussion has to do with creating your own version of the binary,
    rather than using the repo-provided version.

    Right.

    Anyway, to be honest, I never realized how bloated Thunderbird is.
    The source RPM thunderbird-128.5.2-1.fc41.src.rpm is 690MB and
    the main source directory unpacked is:

    ~/tmp/tb/thunderbird-128.5.2 $ du -sh
    4.2G .

    Building TB with the help of a pre-made spec file on Fedora is
    probably very much easier than doing 'git clone' and trying to
    build it from there. Using 'dnf', it was just one command to
    download all the dependencies. I suppose the size of the
    dependency packages was 260MB in total. It would be a nightmare
    having to download them manually and then building them.

    Packages are just so handy.

    Fedora and Red Hat have already done the hard work so it
    is wise to use their source RPM as a basis for your own
    modifications when you are on Fedora or Red Hat Enterprise
    Linux.

    My single CPU Thunderbird build has now lasted for over two
    and half hours and I have no clue when it will be ready.

    This codebase is absolutely massive! I am beginning to
    lose patience.

    br,
    KK

    This is one of the bigger things that you could attempt to build.

    In the old days, there would have been almost zero chance
    of you finishing the linking phase (XUL linking). Today,
    they've fixed up how linking is done, so you might
    actually finish the build without incident.

    Keep a copy of "top" open, so you can watch the RAM consumption
    during linking.

    Paul

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From [email protected]@21:1/5 to All on Sun Dec 29 09:50:55 2024
    On Sat, 28 Dec 2024 20:30:07 -0000 (UTC)
    [email protected] (Kalevi Kolttonen) gabbled:
    Kalevi Kolttonen <[email protected]> wrote:
    I cannot understand this because all references to
    patches 416 and 419 are commented out in the Fedora
    41 spec file. I now completely removed them and will
    try again...

    I am having massive problems with having only 16GB of
    RAM.

    Using 'top', I was able to see that Rust compiler 'rustc'
    was hogging something like 11GB of memory, and then
    after a while OOM killer got rid of the Rust compiler
    process. I am also seeing swapping take place when I
    attempt the build.

    Why TF is the rust compiler involved in the process at all?

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From [email protected]@21:1/5 to All on Sun Dec 29 09:54:52 2024
    On Sat, 28 Dec 2024 23:32:37 -0000 (UTC)
    [email protected] (Kalevi Kolttonen) gabbled:
    Lawrence D'Oliveiro <[email protected]d> wrote:
    On Sat, 28 Dec 2024 19:40:46 +0100, Janis Papanagnou wrote:

    You think it's normal that on a Linux installation where, say, no 'cc'
    (as prominent example of a development tool) is installed the package
    manager would first install ALL the necessary compilers and scripting
    languages just to create a binary (as opposed to just installing the
    binary)?

    The discussion has to do with creating your own version of the binary,
    rather than using the repo-provided version.

    Right.

    Anyway, to be honest, I never realized how bloated Thunderbird is.
    The source RPM thunderbird-128.5.2-1.fc41.src.rpm is 690MB and
    the main source directory unpacked is:

    ~/tmp/tb/thunderbird-128.5.2 $ du -sh
    4.2G .

    Welcome to Lego brick style programming where the main application devs are incompetant halfwits unable to implement even simple things themselves beyond designing (if that applies to Thunderbird) a GUI so have to import 101 libraries
    to do everything for them.

    I've written my own newsreader system and while admittedly its command line only it requires only one 3rd party library which is OpenSSL.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Kalevi Kolttonen@21:1/5 to [email protected] on Sun Dec 29 13:07:35 2024
    [email protected] wrote:
    Why TF is the rust compiler involved in the process at all?

    TB codebase is a mix of C++, Rust and JavaScript.

    Without my hack, the build process took four hours to complete
    and it produced a working TB. However, with my tiny JavaScript
    modification, the build failed.

    Because these builds take four hours, I have to admit defeat.
    I simply do not have the time to make more modification
    attempts.

    What is more, James Kuyper said that he does not want to
    build his own TB so it was all in vain anyway.

    br,
    KK

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Paul@21:1/5 to [email protected] on Sun Dec 29 07:39:42 2024
    On Sun, 12/29/2024 4:54 AM, [email protected] wrote:
    On Sat, 28 Dec 2024 23:32:37 -0000 (UTC)
    [email protected] (Kalevi Kolttonen) gabbled:
    Lawrence D'Oliveiro <[email protected]d> wrote:
    On Sat, 28 Dec 2024 19:40:46 +0100, Janis Papanagnou wrote:

    You think it's normal that on a Linux installation where, say, no 'cc' >>>> (as prominent example of a development tool) is installed the package
    manager would first install ALL the necessary compilers and scripting
    languages just to create a binary (as opposed to just installing the
    binary)?

    The discussion has to do with creating your own version of the binary,
    rather than using the repo-provided version.

    Right.

    Anyway, to be honest, I never realized how bloated Thunderbird is.
    The source RPM thunderbird-128.5.2-1.fc41.src.rpm is 690MB and
    the main source directory unpacked is:

    ~/tmp/tb/thunderbird-128.5.2 $ du -sh
    4.2G .

    Welcome to Lego brick style programming where the main application devs are incompetant halfwits unable to implement even simple things themselves beyond designing (if that applies to Thunderbird) a GUI so have to import 101 libraries
    to do everything for them.

    I've written my own newsreader system and while admittedly its command line only it requires only one 3rd party library which is OpenSSL.

    You know that Thunderbird uses the source code of Firefox,
    to build XUL.so , which is the rendering engine for the interface.

    Thunderbird is the demo app for XUL. It's not really
    a product, it was partially a creation which was
    intended to show how XUL could provide a web render
    engine for another software package.

    As a result, a relatively small amount of code, implements
    News and Email functions. At least 90% of the vast volume
    of files in the tarball, is a copy of the Firefox source.

    The Thunderbird build tree, even has an option to "just build Firefox".
    This is a means of proving the Firefox portion of the tree was
    not damaged by staff during tree preparation.

    The Thunderbird program, tries to restrict just exactly
    how much of Firefox is used for "browsing". If there is a
    URL in an email message, Thunderbird would prefer to call
    the platform browser (whatever it is) to handle the URL.
    But if the Thunderbird staff want to put up an appeal for
    donations, using a Mozilla-hosted web page, that part uses
    the Firefox code inside Thunderbird, for rendering. This is a
    security consideration, an analysis and action on the attack
    surface available. Notice that Thunderbird does not do
    "Quantum" the way Firefox does, so it doesn't have exactly
    the same security precautions. And that's because the
    browser portion, is not really intended for "general browsing".

    The Thunderbird GUI is a three-pane view. If the render engine
    fails, the pane view turns yellow and there is a reference
    on the screen "to an XML file". It is that XML file, which
    draws the three pane view and populates it with decorations.
    When you see that yellow failure condition, that's your
    chance to verify exactly how the product works. The graphics
    are not drawn with Athena widgets. The graphics are a demo
    of what the XUL shared library can do for you.

    Is the whole thing obscene ? Yes. You won't find too many
    software creations, this distorted. Still, people are using it.
    Most people are not aware what is under the hood. It's
    a herd of elephants :-)

    Paul

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Kalevi Kolttonen@21:1/5 to Kalevi Kolttonen on Sun Dec 29 14:09:40 2024
    Kalevi Kolttonen <[email protected]> wrote:
    Because these builds take four hours, I have to admit defeat.
    I simply do not have the time to make more modification
    attempts.

    Well, I did give it one more go and this happened:

    Dec 29 16:03:10 14-5A-FC-31-E8-67 kernel: [ pid ] uid tgid total_vm rss rss_anon rss_file rss_shmem pgtables_bytes swapents oom_score_adj name
    Dec 29 16:03:10 14-5A-FC-31-E8-67 kernel: [ 1058] 998 1058 4009 225 32 193 0 73728 160 -900 systemd-oomd
    Dec 29 16:03:10 14-5A-FC-31-E8-67 kernel: oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/user.slice/user-1000.slice/session-2.scope,task=rustc,pid=244790,uid=1004
    Dec 29 16:03:10 14-5A-FC-31-E8-67 kernel: Out of memory: Killed process 244790 (rustc) total-vm:16524360kB, anon-rss:9185412kB, file-rss:448kB, shmem-rss:0kB, UID:1004 pgtables:30152kB oom_score_adj:0
    Dec 29 16:03:13 14-5A-FC-31-E8-67 kernel: oom_reaper: reaped process 244790 (rustc), now anon-rss:212kB, file-rss:448kB, shmem-rss:0kB


    That was a single CPU build but it still failed.
    My hardware just does not cut it with monstrous
    builds like TB.

    br,
    KK

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Kenny McCormack@21:1/5 to [email protected] on Sun Dec 29 14:32:18 2024
    In article <vkrfue$vl1b$[email protected]>, Paul <[email protected]d> wrote: ...
    Is the whole thing obscene ? Yes. You won't find too many
    software creations, this distorted. Still, people are using it.
    Most people are not aware what is under the hood. It's
    a herd of elephants :-)

    Just out curiosity, does all of this apply to the Windows version as well?

    I know this thread is mostly about the Linux version, and although I
    actually don't use TB at all, I know someone who uses the Windows version.

    --
    I've learned that people will forget what you said, people will forget
    what you did, but people will never forget how you made them feel.

    - Maya Angelou -

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Eric Pozharski@21:1/5 to Kalevi Kolttonen on Sun Dec 29 17:56:34 2024
    with <vkpkn3$ga7s$[email protected]> Kalevi Kolttonen wrote:
    Kalevi Kolttonen <[email protected]> wrote:
    ./mach build -v
    with
    ./mach build -j6 -v
    Okay, I am finally back at home and it is 21:43 o'clock.
    *SKIP* [ 14 lines 1 level deep]
    I cannot understand this because all references to patches 416 and 419
    are commented out in the Fedora 41 spec file. I now completely removed
    them and will try again...

    Yay! The joy of building redhat. Expect your build dependencies being inadequate, missing, or plainly wrong. Just saying.

    p.s. No, I'm not enjoing your pain. Just to make things clear -- about
    two decades ago I've upgraded RH5.1-modified to something RH6 (IIRC)
    then added RH7.3 to the mix. Slackware way (rpmbuild wasn't an option
    because look-ma-no-interwebs).

    p.p.s. Then in The Moment of Clarity I realised that I was making a
    mistake. And immediately made another.

    --
    Torvalds' goal for Linux is very simple: World Domination
    Stallman's goal for GNU is even simpler: Freedom

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Kalevi Kolttonen@21:1/5 to Eric Pozharski on Sun Dec 29 18:59:48 2024
    Eric Pozharski <[email protected]> wrote:
    Yay! The joy of building redhat. Expect your
    build dependencies being inadequate, missing,
    or plainly wrong. Just saying.

    After some minor spec file tweaking, I managed to do
    *one* successful TB build, but because Rust compiler can
    hog almost 16GB of memory, most of the time I just
    cannot build TB using my modest Lenovo laptop. OOM
    killer kicks in and destroys the build.

    I never could have believed that having 16GB of
    RAM and 8GB of swap is not enough for building TB!

    br,
    KK

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Janis Papanagnou@21:1/5 to James Kuyper on Sun Dec 29 21:10:30 2024
    On 28.12.2024 20:26, James Kuyper wrote:
    On 12/27/24 18:44, Janis Papanagnou wrote:
    On 28.12.2024 00:22, Kalevi Kolttonen wrote:

    Compiling Thunderbird should be very easy indeed
    when we use Linux distro's package management.

    You expect _users_ of tools to use a _development_
    environment to fix *inherent* shortcomings of a tool?
    (Shortcomings that should not be there in the first
    place!)

    IIRC, this is in reference to my difficulty when Thunderbird changed the Reply button to mean "Reply" rather than "Followup", and instead added a
    new button that is labelled "Followup". I have never complained about
    that change - it was an entirely sensible one. I'm just having trouble re-training myself to use the newer, more sensible interface in a few
    years after spending a couple of decades using the older, less sensible
    one. And I fully appreciate other people's irritation at my difficulty
    with re-training.
    I wouldn't mind if they reinstated the ability, which existed in older versions of Thunderbird, to rearrange the list of buttons that are
    displayed. I do complain about the removal of that customization
    ability. I don't want to go back to those older versions because that
    would mean undoing other improvements. I'm especially worried about
    undoing security bug fixes.

    The post didn't contain a reference to your case (but I also had it
    still in mind). My reply was based mainly on own experiences with
    TB (and with experience in software development, software ergonomy,
    and system environments in principle).

    I do understand the "re-training" aspect. - Been there. It was so
    annoying (to me) that I was desperately seeking a way to fix it on
    the user-interface level (and finally [somehow] succeeded in some
    [non-obvious] way).

    A feature to rearrange buttons (as being present in some former TB
    releases) is not something that I'd consider to be a sensible user
    interface for application software for several reasons.[*] - Here
    we might be disagreeing on what should be part of a user interface
    and what should be defined in a sensible way in a predefined form
    that matches the application case, and not polluting the interface.

    I understand well that you don't want to go back to older versions.
    Myself I also don't want to go forward if that means that I have to
    buy some change that results in inferior software behavior; but it
    happens, sadly.[**]


    I don't like the idea of creating my own personal version of Thunderbird
    by modifying their source code, because it means I would have to re-do
    the build every time they put out a new version. I want quick and easy upgrades to newer versions, especially security bug fixes, and that
    desire conflicts with the desire for customization.

    Exactly, that's one reason; against a system-wide replacement.[***]

    Janis

    [*] Given that I meanwhile see tons of followups on this thread and
    the [in this NG] well known effect that even small "BTW-statements"
    are leading to bandworm-threads with much heat and little substance
    I'll not extend on that here; with minimum software experience and
    an open-minded thinking it should be obvious anyway.

    [**] That's why I was amused by the other posters "[...] modern ages
    where things have been made quite convenient for us."

    [***] You could of course create a separate version in /usr/local and
    define your PATH appropriately, but you still would have to keep track
    of newer changes, e.g. those security fixes that you are concerned
    about.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Janis Papanagnou@21:1/5 to Paul on Sun Dec 29 21:55:04 2024
    On 29.12.2024 13:39, Paul wrote:
    On Sun, 12/29/2024 4:54 AM, [email protected] wrote:
    [...]

    I've written my own newsreader system and while admittedly its command line >> only it requires only one 3rd party library which is OpenSSL.

    That's a fine property. - Too bad one has to write one's own piece
    of software to get a more sensibly defined product.

    [ snip explanations of some TB internals ]

    Very interesting.


    Is the whole thing obscene ? Yes. You won't find too many
    software creations, this distorted. Still, people are using it.
    Most people are not aware what is under the hood. It's
    a herd of elephants :-)

    You get some impression of what's under the hood if you try to fix
    that [supposedly] primitive issue with the Reply-button. But it's
    (software design wise) actually even worse than I'd have imagined.

    Janis

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Janis Papanagnou@21:1/5 to Kenny McCormack on Sun Dec 29 22:03:19 2024
    On 29.12.2024 15:32, Kenny McCormack wrote:
    In article <vkrfue$vl1b$[email protected]>, Paul <[email protected]d> wrote:
    ...
    Is the whole thing obscene ? Yes. You won't find too many
    software creations, this distorted. Still, people are using it.
    Most people are not aware what is under the hood. It's
    a herd of elephants :-)

    Just out curiosity, does all of this apply to the Windows version as well?

    I know this thread is mostly about the Linux version, and although I
    actually don't use TB at all, I know someone who uses the Windows version.

    I've used the Windows version quite some time ago. I can only say from
    a user's perspective that it was similar to use; maybe the menus were
    organized a bit differently (memories are faint). (Can't say anything
    about the/any "under the hood obscenities" on that platform.)

    Janis

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Janis Papanagnou@21:1/5 to Janis Papanagnou on Sun Dec 29 22:07:03 2024
    On 29.12.2024 21:55, Janis Papanagnou wrote:
    On Sun, 12/29/2024 4:54 AM, [email protected] wrote:
    [...]

    I've written my own newsreader system and while admittedly its command line >>> only it requires only one 3rd party library which is OpenSSL.

    That's a fine property. - Too bad one has to write one's own piece
    of software to get a more sensibly defined product.

    I forgot to ask; wasn't it an option to use any existing text oriented newsreader (nn, rtin, ...)?

    (I recall I liked 'nn' a lot back these days.)

    Janis

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From James Kuyper@21:1/5 to Kalevi Kolttonen on Sun Dec 29 16:41:01 2024
    On 12/29/24 08:07, Kalevi Kolttonen wrote:
    ...
    Without my hack, the build process took four hours to complete
    and it produced a working TB. However, with my tiny JavaScript
    modification, the build failed.

    Because these builds take four hours, I have to admit defeat.
    I simply do not have the time to make more modification
    attempts.

    What is more, James Kuyper said that he does not want to
    build his own TB so it was all in vain anyway.

    Your efforts were not entirely wasted - your (lack of) results make me
    even less willing to build my own. :-}

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Paul@21:1/5 to Kenny McCormack on Sun Dec 29 19:49:10 2024
    On Sun, 12/29/2024 9:32 AM, Kenny McCormack wrote:
    In article <vkrfue$vl1b$[email protected]>, Paul <[email protected]d> wrote:
    ...
    Is the whole thing obscene ? Yes. You won't find too many
    software creations, this distorted. Still, people are using it.
    Most people are not aware what is under the hood. It's
    a herd of elephants :-)

    Just out curiosity, does all of this apply to the Windows version as well?

    I know this thread is mostly about the Linux version, and although I
    actually don't use TB at all, I know someone who uses the Windows version.


    It's a FOSS software that compiles on multiple platforms.

    Just as Firefox (which is most of the code inside after all),
    is FOSS software that compiles on multiple platforms.
    There's even a Firefox.dmg for example, for a Mac computer.
    I don't keep track of how many platforms it supports.

    One way to do this, is to, say, use OpenGL for graphics, as
    OpenGL was available in lots of places. But, they don't do
    that, not exactly. On Windows, the Google ANGLE driver is
    used, which converts something like Direct3D, into an emulation
    of OpenGL. And later, Google may have added WebGL or something.
    The Firefox graphics runs at 20% speed on Windows, compared to
    Linux, and it has something to do with the different means
    of getting a working WebGL. There could have been support provided
    by graphics card drivers, a more direct path, but they didn't use that.

    In fact, the Mozilla graphics designer, is more than a bit annoyed
    about just how many graphics standards and APIs that ended up supported.
    Any notion of Keeping It Simple, went out the window long ago.
    I'm impressed it works as well as it does.

    Like the design of the iceburg, the news and email code is
    the 10% that floats above the water line. While the huge mass
    of cross-platform-ready code underneath for Firefox, does the
    rendering.

    If you have ever examined the tarball for a copy of Firefox
    or Thunderbird, you will develop new respect for it. In the
    sense that, somehow, a team of people corralled 400,000 files
    of various types and made something that sorta works out of it.
    How many projects do you know of, that have 400,000 files in the
    tree ? Many of the files are test benches, for detecting
    regressions when minor code changes are made.

    One day, I was sick of line ending problems, so I made a little
    project out of converting (400K files) to something common I could use.
    Before doing this, I did a scan with the Linux "file" command first,
    to get a declaration of the couple text file formats I was expecting.
    When I sorted all the declarations found, there were *100 text file formats*
    in the tree. For one particular file, if you change the line endings
    in any way, it triggers a bug in the compiler, and you don't get
    your build. And that's what I mean by the herd of elephants thing,
    there are extensive amounts of excrement down there, and don't
    step in it. It's real easy to think you can kick the tree around, when
    it doesn't actually accept abuse as a tree.

    One day, I used Visual Studio, and a debug build, to single-step
    Firefox through a Print routine. As IDE windows opened and closed,
    I noticed I had traversed three source files, source files which
    modified some common print settings, but not the exact same set of
    common settings. It seemed there were three routines running
    sequentially, and presumably the last one executing, was the "latest version". The two moribund versions of code, having never been removed.

    And that's how you manage 400,000 files in a tree. Careful
    where you step!

    Paul

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Salvador Mirzo@21:1/5 to Kalevi Kolttonen on Sun Dec 29 22:19:32 2024
    [email protected] (Kalevi Kolttonen) writes:

    Eric Pozharski <[email protected]> wrote:
    Yay! The joy of building redhat. Expect your
    build dependencies being inadequate, missing,
    or plainly wrong. Just saying.

    After some minor spec file tweaking, I managed to do
    *one* successful TB build, but because Rust compiler can
    hog almost 16GB of memory, most of the time I just
    cannot build TB using my modest Lenovo laptop. OOM
    killer kicks in and destroys the build.

    I never could have believed that having 16GB of
    RAM and 8GB of swap is not enough for building TB!

    You did it. Thanks for sharing the experience.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Kalevi Kolttonen@21:1/5 to Salvador Mirzo on Mon Dec 30 19:31:58 2024
    Salvador Mirzo <[email protected]> wrote:
    [email protected] (Kalevi Kolttonen) writes:

    Eric Pozharski <[email protected]> wrote:
    Yay! The joy of building redhat. Expect your
    build dependencies being inadequate, missing,
    or plainly wrong. Just saying.

    After some minor spec file tweaking, I managed to do
    *one* successful TB build, but because Rust compiler can
    hog almost 16GB of memory, most of the time I just
    cannot build TB using my modest Lenovo laptop. OOM
    killer kicks in and destroys the build.

    I never could have believed that having 16GB of
    RAM and 8GB of swap is not enough for building TB!

    You did it. Thanks for sharing the experience.

    With some incredible luck, it worked out *once*. :-)

    br,
    KK

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Salvador Mirzo@21:1/5 to Kalevi Kolttonen on Mon Dec 30 18:10:22 2024
    [email protected] (Kalevi Kolttonen) writes:

    Salvador Mirzo <[email protected]> wrote:
    [email protected] (Kalevi Kolttonen) writes:

    Eric Pozharski <[email protected]> wrote:
    Yay! The joy of building redhat. Expect your
    build dependencies being inadequate, missing,
    or plainly wrong. Just saying.

    After some minor spec file tweaking, I managed to do
    *one* successful TB build, but because Rust compiler can
    hog almost 16GB of memory, most of the time I just
    cannot build TB using my modest Lenovo laptop. OOM
    killer kicks in and destroys the build.

    I never could have believed that having 16GB of
    RAM and 8GB of swap is not enough for building TB!

    You did it. Thanks for sharing the experience.

    With some incredible luck, it worked out *once*. :-)

    That also explains why some people were skeptical here. Even with a sophisticated system to make the compilation succeed, it's still not
    without a thrill.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Kalevi Kolttonen@21:1/5 to Salvador Mirzo on Mon Dec 30 23:11:40 2024
    Salvador Mirzo <[email protected]> wrote:
    That also explains why some people were skeptical here. Even with a sophisticated system to make the compilation succeed, it's still not
    without a thrill.

    The modifications I made to the spec file were in fact
    quite trivial. The compilation would have succeeded with
    32GB of RAM. Skepticism was not really warranted because
    anyone with little Fedora/Red Hat experience could have
    done what I did.

    Others here have access to more powerful machines than
    I do, so they can finish this task if they want to.

    I am a bit curious whether my JavaScript hack works
    or not.

    br,
    KK

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Paul@21:1/5 to Kalevi Kolttonen on Thu Jan 2 03:40:25 2025
    On Sun, 12/29/2024 1:59 PM, Kalevi Kolttonen wrote:
    Eric Pozharski <[email protected]> wrote:
    Yay! The joy of building redhat. Expect your
    build dependencies being inadequate, missing,
    or plainly wrong. Just saying.

    After some minor spec file tweaking, I managed to do
    *one* successful TB build, but because Rust compiler can
    hog almost 16GB of memory, most of the time I just
    cannot build TB using my modest Lenovo laptop. OOM
    killer kicks in and destroys the build.

    I never could have believed that having 16GB of
    RAM and 8GB of swap is not enough for building TB!

    br,
    KK


    Try a chain saw next time :-)

    It's one of the first practical tests the machine got.
    The RPMBuild phase was awfully slow (it spoiled the fun).
    But the compiles and linking behaved well. RPM compression
    seems to run on one core.

    [Picture] During compile phase...

    https://i.postimg.cc/44qRrgxb/Thunderbird-Fedora41-Build-From-Source-via-Mock.gif

    I think it's possible the build slows down, the longer it runs.
    Like "something" is fragmenting.

    Even on this machine, the process does not encourage
    interactive operation. It takes too long. Adjusting the
    command a bit so it just compiles and links, would be
    better, if that's possible.

    mock --resultdir=/tmp/results --rootdir=/tmp/mock --rebuild thunderbird-128.5.2-1.fc41.src.rpm | tee /tmp/build_out.txt

    I picked Fedora for the job, because it only takes two commands
    in a terminal, to do it. In simplified terms...

    dnf download --source packagename # Downloading source doesn't need root. mock --rebuild packagename # User account belongs to "mock" group, doesn't build as root

    But I need to do something else to that Mock command,
    to get what I want (a "portable" copy of Thunderbird,
    there should be a dir created with that sitting in it).

    Summary: No question, a bit of RAM helps. Some of the RAM
    accounting in Linux is just weird (process resident
    seen rising, graph in system monitor remains flat).
    I was expecting to see a "hump" while linking, but
    the graph was relatively flat and featureless.

    Paul

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Scott Lurndal@21:1/5 to Paul on Thu Jan 2 16:29:48 2025
    Paul <[email protected]d> writes:
    On Sun, 12/29/2024 1:59 PM, Kalevi Kolttonen wrote:
    Eric Pozharski <[email protected]> wrote:
    Yay! The joy of building redhat. Expect your
    build dependencies being inadequate, missing,
    or plainly wrong. Just saying.

    After some minor spec file tweaking, I managed to do
    *one* successful TB build, but because Rust compiler can
    hog almost 16GB of memory, most of the time I just
    cannot build TB using my modest Lenovo laptop. OOM
    killer kicks in and destroys the build.

    I never could have believed that having 16GB of
    RAM and 8GB of swap is not enough for building TB!

    br,
    KK


    Try a chain saw next time :-)

    It's one of the first practical tests the machine got.
    The RPMBuild phase was awfully slow (it spoiled the fun).
    But the compiles and linking behaved well. RPM compression
    seems to run on one core.

    [Picture] During compile phase...

    https://i.postimg.cc/44qRrgxb/Thunderbird-Fedora41-Build-From-Source-via-Mock.gif

    I think it's possible the build slows down, the longer it runs.
    Like "something" is fragmenting.

    Even on this machine, the process does not encourage
    interactive operation. It takes too long. Adjusting the
    command a bit so it just compiles and links, would be
    better, if that's possible.

    mock --resultdir=/tmp/results --rootdir=/tmp/mock --rebuild thunderbird-128.5.2-1.fc41.src.rpm | tee /tmp/build_out.txt

    I picked Fedora for the job, because it only takes two commands
    in a terminal, to do it. In simplified terms...

    dnf download --source packagename # Downloading source doesn't need root. >mock --rebuild packagename # User account belongs to "mock" group, doesn't build as root

    But I need to do something else to that Mock command,
    to get what I want (a "portable" copy of Thunderbird,
    there should be a dir created with that sitting in it).

    Summary: No question, a bit of RAM helps. Some of the RAM
    accounting in Linux is just weird (process resident
    seen rising, graph in system monitor remains flat).
    I was expecting to see a "hump" while linking, but
    the graph was relatively flat and featureless.

    Paul

    Why would you expect the link step to require a lot of
    memory? The linker builds an elf executable from the contents
    of ELF object files, one ELF section at a time. It doesn't
    construct the entire ELF executable in memory before writing it out.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Paul@21:1/5 to Scott Lurndal on Thu Jan 2 19:36:50 2025
    On Thu, 1/2/2025 11:29 AM, Scott Lurndal wrote:

    Why would you expect the link step to require a lot of
    memory? The linker builds an elf executable from the contents
    of ELF object files, one ELF section at a time. It doesn't
    construct the entire ELF executable in memory before writing it out.


    It's based on experience, not imagination.

    I've built Thunderbird on both Windows and Linux.
    It was the Windows build that left a bad taste.
    Once you repeatedly have build failures during linking,
    you are always looking for it.

    I've built Thunderbird multiple times over the years.
    At one time, there was a nasty "ramp" in memory consumption
    visible while I was building in Windows XP, and using
    the Visual Studio compiler and linker, as instructed
    by the Mozilla build page for Thunderbird. I just follow
    the recipe when doing these.

    The builds today take a lot more RAM than back then.

    [Picture]

    https://i.postimg.cc/85bRBYpX/buckets-of-ram.gif

    Paul

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Lawrence D'Oliveiro@21:1/5 to Paul on Fri Jan 3 02:55:03 2025
    On Thu, 2 Jan 2025 19:36:50 -0500, Paul wrote:

    I've built Thunderbird on both Windows and Linux.
    It was the Windows build that left a bad taste.
    Once you repeatedly have build failures during linking, you are always looking for it.

    Yours is not the only experience. I recall a blog post from the
    LibreOffice folks, soon after they forked off from OpenOffice, that they
    tended to suffer intermittent unexplainable build failures on Windows,
    too.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Scott Lurndal@21:1/5 to Paul on Fri Jan 3 18:15:25 2025
    Paul <[email protected]d> writes:
    On Thu, 1/2/2025 11:29 AM, Scott Lurndal wrote:

    Why would you expect the link step to require a lot of
    memory? The linker builds an elf executable from the contents
    of ELF object files, one ELF section at a time. It doesn't
    construct the entire ELF executable in memory before writing it out.


    It's based on experience, not imagination.

    I've built Thunderbird on both Windows and Linux.
    It was the Windows build that left a bad taste.
    Once you repeatedly have build failures during linking,
    you are always looking for it.


    Ah, well windows. You need not elaborate.

    I've been fortunate to have never built software in a
    microsoft environment (aside an optical jukebox driver
    for NT3.51 once on a contract job - even then I did
    all the editing on unix and just compiled and tested
    on the windows box).

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From [email protected]@21:1/5 to All on Sat Jan 4 10:12:51 2025
    On Fri, 03 Jan 2025 18:15:25 GMT
    [email protected] (Scott Lurndal) gabbled:
    Paul <[email protected]d> writes:
    On Thu, 1/2/2025 11:29 AM, Scott Lurndal wrote:

    Why would you expect the link step to require a lot of
    memory? The linker builds an elf executable from the contents
    of ELF object files, one ELF section at a time. It doesn't
    construct the entire ELF executable in memory before writing it out.


    It's based on experience, not imagination.

    I've built Thunderbird on both Windows and Linux.
    It was the Windows build that left a bad taste.
    Once you repeatedly have build failures during linking,
    you are always looking for it.


    Ah, well windows. You need not elaborate.

    I've been fortunate to have never built software in a
    microsoft environment (aside an optical jukebox driver
    for NT3.51 once on a contract job - even then I did
    all the editing on unix and just compiled and tested
    on the windows box).

    I did a Windows C++ job for a year. I still can't believe how complicated Visual Studio (2017 IIRC) made the most basic things such as setting library and include paths which were buried 2 or 3 levels down in some sub menu not
    to mention all the "project" BS which forced a certain structure on to your code filesystem layout which I didn't particularly want. Also the fact that console and GUI apps require a totally different project setup and boiler plate code from the start is just mind boggling.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)