Hello everyone,

I got interested in the problem of bootstrapping a modern versions of
Gradle, mainly due to the old version available in Debian. As far as I
can tell, efforts have been mostly focused on using older versions to
bootstrap an up-to-date version. I wanted to explore the feasibility and
effort required to bootstrap with Maven instead. I now have a working
purely Maven-based build that generates a distribution capable of
building Gradle itself. The code is available here <https://github.com/MorMundHS-MA/gradle-maven-bootstrap>.

While it was fun, I am not sure how much this actually helps as it still requires a pre-build Kotlin compiler (at least 1.8 I think, even though

2.0 is used). Additionally some of the other dependencies are also

built with Gradle and would have to be bootstrapped as well. But I
wanted to share my progress so far non the less.

Best,
Moritz

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hi Moritz,

Le 2025-03-16 11:10, Moritz a écrit :

I got interested in the problem of bootstrapping a modern versions of
Gradle, mainly due to the old version available in Debian. As far as I
can tell, efforts have been mostly focused on using older versions to bootstrap an up-to-date version.

Most but not all. My own branch also uses the same version of Gradle to
rebuild itself.

I wanted to explore the feasibility and effort required to bootstrap
with Maven instead. I now have a working purely Maven-based build that generates a distribution capable of building Gradle itself. The code is available here
<https://github.com/MorMundHS-MA/gradle-maven-bootstrap>.

That's an interesting approach. Did you use a generator for the pom.xml
files, and if so how much manual adjustment was required afterwards?

On my side I've started to write a Makefile generator that uses Gradle
itself to generate the recipes. It's not finished (and not functional)
yet, but you can get an idea of what the result would look like there
[1]. The generator is the MakefileMaker class at the bottom of [2]. I'm planning to add custom logic to generate the additional code and
properties at (re)build time. The idea is to have something reasonably
simple that could be used to bootstrap a future version with minimal or
no changes (ideally) to the generation logic.

While it was fun, I am not sure how much this actually helps as it
still requires a pre-build Kotlin compiler (at least 1.8 I think, even
though >2.0 is used). Additionally some of the other dependencies are
also built with Gradle and would have to be bootstrapped as well. But I wanted to share my progress so far non the less.

They still use Kotlin 2.0.21 currently, setting the source compatibility
to 1.9 (or 1.8 maybe?) to force the use of the legacy K1 compiler, and
adding another legacy dependency as they use APIs that are now
deprecated and even removed for some. It's possible to compile the
Kotlin code using older compilers (with a few minor adjustments
eventually) but the Kotlin scripting integration (DSL) in Gradle is much
less likely to be buildable or work with older versions of Kotlin
compiler libraries without some extensive changes.

So far the only other dependencies I found that will need to be
bootstrapped in the same batch (with gradle and kotlin) are:
- kotlinx.serialization
- kotlinx-metadata-jvm 0.5.0 (Kotlin 1.8.10 is the last release with
that version).

Thank you for sharing this. I will add a link to your repository on my
own wip/README.

Cheers,


[1]: https://salsa.debian.org/jpd/gradle/-/blob/478abafcd391cc28549f9b369d7b42a3febb3197/debian/gradle.Makefile
[2]: https://salsa.debian.org/jpd/gradle/-/blob/478abafcd391cc28549f9b369d7b42a3febb3197/debian/init.d/gradlebuild/DebianGradleBuildPlugin.gradle#L638

--
Julien Plissonneau Duquène

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hi Julien,

thanks for referencing my repository, hope it helps.

That's an interesting approach. Did you use a generator for the
pom.xml files, and if so how much manual adjustment was required
afterwards?

Yes, I built a crude generator for this based on the output from
`./gradlew [module]:dependencies --configuration compileClasspath` to
get the module dependencies with each other. I think most poms are
unchanged from the generated code. I've uploaded the code as a PR here <https://github.com/MorMundHS-MA/gradle-maven-bootstrap/pull/2>

On my side I've started to write a Makefile generator that uses Gradle
itself to generate the recipes. It's not finished (and not functional)
yet, but you can get an idea of what the result would look like there [1].

Using a Gradle plugin for this is smart. I wasn't familiar enough with
the ecosystem to attempt that, but with Gradle/Maven plugins you can
probably get to a fully automated/reproducible conversion.

Not sure if Makefiles will be a good choice, as the project is large
with a lot of dependencies between the modules. The generated poms are
easily reviewable by a human, I think this will be much harder with
Makefiles. But maybe I'm overthinking the supply chain trust issues with bootstrapping build tools.

The generator is the MakefileMaker class at the bottom of [2]. I'm
planning to add custom logic to generate the additional code and
properties at (re)build time. The idea is to have something reasonably
simple that could be used to bootstrap a future version with minimal
or no changes (ideally) to the generation logic.

I was hoping that bootstrapping one modern version of Gradle/Kotlin
would be enough to build future versions using the tooling itself. But I
don't know how commonly they still introduce new functionality/breaking
changes that would prevent building with older versions.

They still use Kotlin 2.0.21 currently, setting the source
compatibility to 1.9 (or 1.8 maybe?) to force the use of the legacy K1 compiler, and adding another legacy dependency as they use APIs that
are now deprecated and even removed for some. It's possible to compile
the Kotlin code using older compilers (with a few minor adjustments eventually) but the Kotlin scripting integration (DSL) in Gradle is
much less likely to be buildable or work with older versions of Kotlin compiler libraries without some extensive changes.

Which version of Kotlin has been bootstrapped? I think Debian had 1.3 at
some point, but I'm not sure if that was completely built from source.
At least it isn't available in testing anymore.

So far the only other dependencies I found that will need to be
bootstrapped in the same batch (with gradle and kotlin) are:

Gradle also references these dependencies that are built with Gradle
(list won't be complete, just the obvious ones):

* kotlin-stdlib
* kotlin-reflect
* kotlinx-serialization-core
* kotlinx-serialization-json
* kotlin-compiler-embeddable (dependency of
'gradle-declarative-dsl-core', probably the biggest one together with
stdlib as it requires a modern bootstrapped kotlin compiler)

* org.jetbrains.annotations
* org.jetbrains.intellij.deps.trove4j

There is also one additional Gradle repository, which I cannot find
right now. But I had to downgrade its version manually as the Gradle
devs hadn't uploaded their newest version to a public registry yet when
I was working on it. But it was also built with Gradle.

Best,
Moritz

On 17.03.2025 10:42, Julien Plissonneau Duquène wrote:

Hi Moritz,

Le 2025-03-16 11:10, Moritz a écrit :

I got interested in the problem of bootstrapping a modern versions of
Gradle, mainly due to the old version available in Debian. As far as
I can tell, efforts have been mostly focused on using older versions
to bootstrap an up-to-date version.

Most but not all. My own branch also uses the same version of Gradle
to rebuild itself.

I wanted to explore the feasibility and effort required to bootstrap
with Maven instead. I now have a working purely Maven-based build
that generates a distribution capable of building Gradle itself. The
code is available here
<https://github.com/MorMundHS-MA/gradle-maven-bootstrap>.

That's an interesting approach. Did you use a generator for the
pom.xml files, and if so how much manual adjustment was required
afterwards?

On my side I've started to write a Makefile generator that uses Gradle
itself to generate the recipes. It's not finished (and not functional)
yet, but you can get an idea of what the result would look like there
[1]. The generator is the MakefileMaker class at the bottom of [2].
I'm planning to add custom logic to generate the additional code and properties at (re)build time. The idea is to have something reasonably
simple that could be used to bootstrap a future version with minimal
or no changes (ideally) to the generation logic.

While it was fun, I am not sure how much this actually helps as it
still requires a pre-build Kotlin compiler (at least 1.8 I think,
even though >2.0 is used). Additionally some of the other
dependencies are also built with Gradle and would have to be
bootstrapped as well. But I wanted to share my progress so far non
the less.

They still use Kotlin 2.0.21 currently, setting the source
compatibility to 1.9 (or 1.8 maybe?) to force the use of the legacy K1 compiler, and adding another legacy dependency as they use APIs that
are now deprecated and even removed for some. It's possible to compile
the Kotlin code using older compilers (with a few minor adjustments eventually) but the Kotlin scripting integration (DSL) in Gradle is
much less likely to be buildable or work with older versions of Kotlin compiler libraries without some extensive changes.

So far the only other dependencies I found that will need to be
bootstrapped in the same batch (with gradle and kotlin) are:
- kotlinx.serialization
- kotlinx-metadata-jvm 0.5.0 (Kotlin 1.8.10 is the last release with
that version).

Thank you for sharing this. I will add a link to your repository on my
own wip/README.

Cheers,

[1]: https://salsa.debian.org/jpd/gradle/-/blob/478abafcd391cc28549f9b369d7b42a3febb3197/debian/gradle.Makefile
[2]: https://salsa.debian.org/jpd/gradle/-/blob/478abafcd391cc28549f9b369d7b42a3febb3197/debian/init.d/gradlebuild/DebianGradleBuildPlugin.gradle#L638

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Le 2025-03-17 21:44, Moritz a écrit :

Yes, I built a crude generator for this based on the output from
`./gradlew [module]:dependencies --configuration compileClasspath` to
get the module dependencies with each other. I think most poms are
unchanged from the generated code. I've uploaded the code as a PR here <https://github.com/MorMundHS-MA/gradle-maven-bootstrap/pull/2>

If you made that into a custom Gradle task like I did for
MakefileMakerTask you could get rid of the parser and the hardcoding in SettingsParser, and possibly find ways to minimize the amount of manual tweaking afterwards.

Not sure if Makefiles will be a good choice, as the project is large
with a lot of dependencies between the modules. The generated poms are
easily reviewable by a human, I think this will be much harder with Makefiles. But maybe I'm overthinking the supply chain trust issues
with bootstrapping build tools.

Having looked at both I think they are both similarly inconvenient to
review ;) mostly because this is a large project.

The generated PoC Makefile for Gradle 8 is more than 7 MiB and 77k
lines, but on the other hand it lists exhaustively every single source
file, jar dependency and intermediate generated file input for every
task. This is IMO interesting for auditing source code as you can then
use that as an input to identify unused source files or graph which
tasks are run for a given source file (I add comments with the original
gradle task name and dependencies into that Makefile). There is
certainly some Graphviz potential in there.

Other technical reasons that made me choose a Makefile over other
possibilities were:
- it's a single file, and it's trivial to keep it out of the upstream
source tree and relocate it if needed (which is much more convenient for packaging)
- it's future-proof: the generated syntax has worked for decades, and is extremely unlikely to stop working with future releases of GNU Make
(which removes a whole source of possible future issues with the
generator code)
- dependencies are resolved at generation time by Gradle (tweaked by a
custom plugin and Debian helper libraries) which makes the build fairly reproducible; having the dependencies resolved again at build time by a different tool (in theory the logic is the same, but the devil is in the details) may introduce issues.

I was hoping that bootstrapping one modern version of Gradle/Kotlin
would be enough to build future versions using the tooling itself. But
I don't know how commonly they still introduce new
functionality/breaking changes that would prevent building with older versions.

Unfortunately with their development model that can't be ensured,
especially on the Gradle side. They don't even try to make a given
release of Gradle buildable by a previously released version. New major releases are commonly built by rc releases, which may be built by
milestone releases or even nightly builds. I'm pretty sure that Gradle 9
for example is going to introduce breaking changes that will make it
impossible to build it with any release of Gradle 8 without downgrading
or rewriting significant parts of the build scripts, which is not
trivial. I'm expecting that bootstrapping it again is going to be
easier.

Which version of Kotlin has been bootstrapped? I think Debian had 1.3
at some point, but I'm not sure if that was completely built from
source. At least it isn't available in testing anymore.

That's 1.3.31 currently. Thank you for reminding me that it's still not
into testing, I've just closed two resolved FTBFS bugs to see if that
helps with the migration.

Gradle also references these dependencies that are built with Gradle
(list won't be complete, just the obvious ones):

* kotlin-stdlib
* kotlin-reflect
* kotlin-compiler-embeddable (dependency of
'gradle-declarative-dsl-core', probably the biggest one together with
stdlib as it requires a modern bootstrapped kotlin compiler)

These are packaged with kotlin,

* kotlinx-serialization-core
* kotlinx-serialization-json

these will be with kotlinx-serialization,

* org.jetbrains.annotations
* org.jetbrains.intellij.deps.trove4j

and these are already packaged (libjetbrains-annotations-java and libtrove-intellij-java) and appear to be usable to rebuild gradle.

There is also one additional Gradle repository, which I cannot find
right now. But I had to downgrade its version manually as the Gradle
devs hadn't uploaded their newest version to a public registry yet when
I was working on it. But it was also built with Gradle.

native-platform and/or fileevents maybe? That happens sometimes, or they
forget to push release tags. Messaging them on their Slack usually gets
the issue quickly resolved.

Cheers,

--
Julien Plissonneau Duquène

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)