Hi team,

While preparing ghc 9.4.5 in Debian experimental, it fails to build with
OOM on mipsel [1].

The command that fails is:

$ "inplace/bin/ghc-stage1" -hisuf p_hi -osuf p_o -hcsuf p_hc -static -prof -H32m -O -lffi -optl-pthread -optc--param -optcggc-min-expand=5 -optc--param -optcggc-min-heapsize=4096 -Wall -this-unit-id ghc-9.4.5 -hide-all-packages -package-env - -i -
icompiler/. -icompiler/stage2/build -Icompiler/stage2/build -icompiler/stage2/build/./autogen -Icompiler/stage2/build/./autogen -Icompiler/. -Icompiler/stage2/build/. -optP-DHAVE_INTERNAL_INTERPRETER -optP-DCAN_LOAD_DLL -optP-include -optPcompiler/stage2/
build/./autogen/cabal_macros.h -package-id array-0.5.4.0 -package-id base-4.17.1.0 -package-id binary-0.8.9.1 -package-id bytestring-0.11.4.0 -package-id containers-0.6.7 -package-id deepseq-1.4.8.0 -package-id directory-1.3.7.1 -package-id exceptions-0.
10.5 -package-id filepath-1.4.2.2 -package-id ghc-boot-9.4.5 -package-id ghc-heap-9.4.5 -package-id ghci-9.4.5 -package-id hpc-0.6.1.0 -package-id process-1.6.16.0 -package-id stm-2.5.1.0 -package-id template-haskell-2.19.0.0 -package-id terminfo-0.4.1.5
-package-id time-1.12.2 -package-id transformers-0.5.6.2 -package-id unix-2.7.3 -Wall -Wno-name-shadowing -Wnoncanonical-monad-instances -Wnoncanonical-monoid-instances -this-unit-id ghc -XHaskell2010 -XNoImplicitPrelude -XBangPatterns -
XScopedTypeVariables -XMonoLocalBinds -XTypeOperators -Rghc-timing -Wcpp-undef -Wincomplete-uni-patterns -Wincomplete-record-updates -no-user-package-db -rtsopts -O0 -Wnoncanonical-monad-instances -outputdir compiler/stage2/build -c compiler/./
GHC/Hs/Instances.hs -o compiler/stage2/build/GHC/Hs/Instances.p_o -dyno compiler/stage2/build/GHC/Hs/Instances.dyn_o

virtual memory exhausted: Cannot allocate memory
`gcc' failed in phase `C Compiler'. (Exit code: 1)

Building an unregisterised ghc compiler on 32-bit targets has trouble
handling the Instances.hs file. This has been reported upstream here[2].
Until now, we have managed to workaround this by compiling this file
alone with O0 (as shown in the above command), but since version 9.4.5
even that doesn't work.

Is there anything else we can try to compile ghc 9.4.5 in mipsel, or
should we give up in supporting mipsel at all?

[1] https://buildd.debian.org/status/package.php?p=ghc&suite=experimental
[2] https://gitlab.haskell.org/ghc/ghc/-/issues/17048

PS please CC me as I am not subscribed to this list.

Best,

--
Ilias

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hi Ilias,

On Wed, Jul 26, 2023 at 12:33 PM Ilias Tsitsimpis <[email protected]> wrote:

Hi team,

While preparing ghc 9.4.5 in Debian experimental, it fails to build with
OOM on mipsel [1].

The command that fails is:

$ "inplace/bin/ghc-stage1" -hisuf p_hi -osuf p_o -hcsuf p_hc -static -prof -H32m -O -lffi -optl-pthread -optc--param -optcggc-min-expand=5 -optc--param -optcggc-min-heapsize=4096 -Wall -this-unit-id ghc-9.4.5 -hide-all-packages -package-env - -i -

icompiler/. -icompiler/stage2/build -Icompiler/stage2/build -icompiler/stage2/build/./autogen -Icompiler/stage2/build/./autogen -Icompiler/. -Icompiler/stage2/build/. -optP-DHAVE_INTERNAL_INTERPRETER -optP-DCAN_LOAD_DLL -optP-include -optPcompiler/stage2/
build/./autogen/cabal_macros.h -package-id array-0.5.4.0 -package-id base-4.17.1.0 -package-id binary-0.8.9.1 -package-id bytestring-0.11.4.0 -package-id containers-0.6.7 -package-id deepseq-1.4.8.0 -package-id directory-1.3.7.1 -package-id exceptions-0.
10.5 -package-id filepath-1.4.2.2 -package-id ghc-boot-9.4.5 -package-id ghc-heap-9.4.5 -package-id ghci-9.4.5 -package-id hpc-0.6.1.0 -package-id process-1.6.16.0 -package-id stm-2.5.1.0 -package-id template-haskell-2.19.0.0 -package-id terminfo-0.4.1.5
-package-id time-1.12.2 -package-id transformers-0.5.6.2 -package-id unix-2.7.3 -Wall -Wno-name-shadowing -Wnoncanonical-monad-instances -Wnoncanonical-monoid-instances -this-unit-id ghc -XHaskell2010 -XNoImplicitPrelude -XBangPatterns -
XScopedTypeVariables -XMonoLocalBinds -XTypeOperators -Rghc-timing -Wcpp-undef -Wincomplete-uni-patterns -Wincomplete-record-updates -no-user-package-db -rtsopts -O0 -Wnoncanonical-monad-instances -outputdir compiler/stage2/build -c compiler/./
GHC/Hs/Instances.hs -o compiler/stage2/build/GHC/Hs/Instances.p_o -dyno compiler/stage2/build/GHC/Hs/Instances.dyn_o

virtual memory exhausted: Cannot allocate memory
`gcc' failed in phase `C Compiler'. (Exit code: 1)

Building an unregisterised ghc compiler on 32-bit targets has trouble handling the Instances.hs file. This has been reported upstream here[2]. Until now, we have managed to workaround this by compiling this file
alone with O0 (as shown in the above command), but since version 9.4.5
even that doesn't work.

Is there anything else we can try to compile ghc 9.4.5 in mipsel, or
should we give up in supporting mipsel at all?

[1] https://buildd.debian.org/status/package.php?p=ghc&suite=experimental
[2] https://gitlab.haskell.org/ghc/ghc/-/issues/17048

Speaking as someone who helps maintain C/C++ libraries; not someone
who packages for Debian or administers the mipsel gear...

I used to run into this alot with the Crypto++ library on ARM gadgets. "Gadgets" are what I call the dev boards and IoT boards. The dev
boards were low resource, and often had 512MB or 1GB of RAM with no
swap file because of a SDcard. One particular file used to cause an
OOM kill more frequently than I care to remember.

I found two things helped or fixed the problem. First, I would build
with only one make job. That is, 'make -j 1'. 'make -j 3' or 'make -j
5' was sure to break the build.

Second, I would add a swap partition that provided enough space to run
the compiler. For a dev board with 512MB of RAM, I would setup a 2GB
swap partition. I would also set swappiness to 1 or 2 to keep things
in memory. zram would not help because even compressed, the compiler
needed a fair amount of space.

And I do burn through a lot of SDcards. But a new SDcard once or twice
a year is a better outcome than a DoS as the OOM Killer keeps killing
my jobs.

Jeff

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hi Jeffrey,

On Wed, Jul 26, 2023 at 01:02PM, Jeffrey Walton wrote:

[...]
I found two things helped or fixed the problem. First, I would build
with only one make job. That is, 'make -j 1'. 'make -j 3' or 'make -j
5' was sure to break the build.

Second, I would add a swap partition that provided enough space to run
the compiler. For a dev board with 512MB of RAM, I would setup a 2GB
swap partition. I would also set swappiness to 1 or 2 to keep things
in memory. zram would not help because even compressed, the compiler
needed a fair amount of space.

Thanks for the hints. Unfortunately none of them have worked so far. I
am trying to compile just this file and nothing else, and I am using a
host with 8G of RAM and 8G of Swap. The problem seems to be the fact
that mipsel is limited to only 2G RAM space for user processes.

Unfortunately I cannot see a way forward here, other than removing ghc
(and every Haskell program) from mipsel.

--
Ilias

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hi Ilias,

On Fri, Jul 28, 2023 at 4:27 PM Ilias Tsitsimpis <[email protected]> wrote:

On Wed, Jul 26, 2023 at 01:02PM, Jeffrey Walton wrote:

[...]
I found two things helped or fixed the problem. First, I would build
with only one make job. That is, 'make -j 1'. 'make -j 3' or 'make -j
5' was sure to break the build.

Second, I would add a swap partition that provided enough space to run
the compiler. For a dev board with 512MB of RAM, I would setup a 2GB
swap partition. I would also set swappiness to 1 or 2 to keep things
in memory. zram would not help because even compressed, the compiler
needed a fair amount of space.

Thanks for the hints. Unfortunately none of them have worked so far. I
am trying to compile just this file and nothing else, and I am using a
host with 8G of RAM and 8G of Swap. The problem seems to be the fact
that mipsel is limited to only 2G RAM space for user processes.

Oh, that's an interesting twist. But I see it could make sense since
this is a 32-bit platform.

Unfortunately I cannot see a way forward here, other than removing ghc
(and every Haskell program) from mipsel.

I'm spit-balling here, since I don't recall doing this in C/C++ for
this problem... Based on:

virtual memory exhausted: Cannot allocate memory
`gcc' failed in phase `C Compiler'. (Exit code: 1)

Break that source file up into two or three more manageable pieces.

If I am parsing the command line correctly, the source file is
Instances.hs and the output file is Instances.p_o. I would try to
create Instances_1.hs, Instances_2.hs and Instances_3.hs, and then let
the linker combine their object files later.

What I don't know is, is it possible to do that in Haskell. Some
languages are picky about things like that. C# and Java come to mind.

Jeff

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hi Jeff,

On Fri, Jul 28, 2023 at 05:09PM, Jeffrey Walton wrote:

Break that source file up into two or three more manageable pieces.

If I am parsing the command line correctly, the source file is
Instances.hs and the output file is Instances.p_o. I would try to
create Instances_1.hs, Instances_2.hs and Instances_3.hs, and then let
the linker combine their object files later.

What I don't know is, is it possible to do that in Haskell. Some
languages are picky about things like that. C# and Java come to mind.

This was suggested as a solution by upstream as well (see [1]) but they
found it was too difficult to do and ended up using to -O0 instead.

[1] https://gitlab.haskell.org/ghc/ghc/-/issues/18256

--
Ilias

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Sat, Jul 29, 2023 at 2:44 AM Ilias Tsitsimpis <[email protected]> wrote:

On Fri, Jul 28, 2023 at 05:09PM, Jeffrey Walton wrote:

Break that source file up into two or three more manageable pieces.

If I am parsing the command line correctly, the source file is
Instances.hs and the output file is Instances.p_o. I would try to
create Instances_1.hs, Instances_2.hs and Instances_3.hs, and then let
the linker combine their object files later.

What I don't know is, is it possible to do that in Haskell. Some
languages are picky about things like that. C# and Java come to mind.

This was suggested as a solution by upstream as well (see [1]) but they
found it was too difficult to do and ended up using to -O0 instead.

[1] https://gitlab.haskell.org/ghc/ghc/-/issues/18256

The 18256 issue looks like it is a different complaint. 18256 tries to
speed up compile time. An OOM kill is a different problem.

Looking at the original command, the project is using -O, which is
-O2. Maybe you can drop it to -O1 -fno-inline.

From Crypto++, I've found there's no material difference between -O1,

-O2 or -O3. Crypto++ provides SSE/AVX/NEON/ASIMD/Altivec acceleration
where it matters, so compiler optimizations make almost no difference
at runtime according to benchmarks.

I see Debian has https://wiki.debian.org/ReduceBuildMemoryOverhead,
but it does not look too helpful. (I updated with your discussion).

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hi Ilias ,

On Sat, Jul 29, 2023 at 2:44 AM Ilias Tsitsimpis <[email protected]> wrote:

On Fri, Jul 28, 2023 at 05:09PM, Jeffrey Walton wrote:

Break that source file up into two or three more manageable pieces.

If I am parsing the command line correctly, the source file is
Instances.hs and the output file is Instances.p_o. I would try to
create Instances_1.hs, Instances_2.hs and Instances_3.hs, and then let
the linker combine their object files later.

What I don't know is, is it possible to do that in Haskell. Some
languages are picky about things like that. C# and Java come to mind.

This was suggested as a solution by upstream as well (see [1]) but they
found it was too difficult to do and ended up using to -O0 instead.

[1] https://gitlab.haskell.org/ghc/ghc/-/issues/18256

Since the compile is failing in preprocessing, I took a look at GCC preprocessor options.[1]

These look like they could help:

* -ftrack-macro-expansion=0
- Don't track macro expansion. Saves memory
* -no-integrated-cpp
- invoke cc1 twice, once for preprocessing and
once for actual compilation of the preprocessed input
* -fmax-include-depth=20
- Normally 200, don't allow deep nesting

And if it is an option, then:

* -fno-large-source-files

-flarge-source-files is an option, but I am not sure about the -fno-*
variant in this case.

Finally, this may help, but I don't understand it. I have never used it:

* -Mno-modules
- Disable dependency generation for compiled module interfaces.
Jeff

[1] https://gcc.gnu.org/onlinedocs/gcc/Preprocessor-Options.html

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Sat, Jul 29, 2023 at 01:35PM, Jeffrey Walton wrote:

The 18256 issue looks like it is a different complaint. 18256 tries to
speed up compile time. An OOM kill is a different problem.

I agree. I linked to it because I wanted to point out that splitting the GHC.Hs.Instances module is not an option, as noted in the issue.

Looking at the original command, the project is using -O, which is
-O2. Maybe you can drop it to -O1 -fno-inline.

If you notice towards the end of the command, we pass -O0, which
overrides the first -O. This used to work (and it's still useful for the
other 32-bit architectures) but doesn't work any more on mipsel.

--
Ilias

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)