Dear Debian gurus,

Lazy me would love to get automation going akin to the one in
conda-forge (see e.g. [1]) -- whenever a new upstream is released
(as could be detected with a run of uscan) the CI would

- would run uscan

- get new source {tarball}
- get new {uversion} (from uscan output/tarball)

- would run

gbp import-orig -- --pristine-tar --debian-branch=new-{uversion}/$(gbp config buildpackage.debian-branch) ...

i.e. import sources and prepare all 3 branches with dedicated prefix
new-{uversion}/

- might do some other automated/decided upon actions such as e.g.
"drop all debian/patches/00*" which were cherry-picked from upstream,
and were applied etc.

- push those branches into salsa repo and submit a merge request
against the debian-branch (while providing URLs for other two
branches).

- employ salsa-ci-team/pipeline (or alike) to build package from
those 3 branches

- maintainer then reviews/tunes up the MR and if merges, CI would
merge/push all the new-{uversion}/ branches into corresponding
one and delete them from CI.

If MR is closed, CI would just remove all those new-{uversion}
branches.


Is there any package/tool which already does something like that on CI?
Or may be you see some issues with such a procedure?

Thanks in advance!

[1] https://github.com/conda-forge/datalad-feedstock/pull/95

--
Yaroslav O. Halchenko
Center for Open Neuroscience http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
WWW: http://www.linkedin.com/in/yarik


-----BEGIN PGP SIGNATURE-----

iQIzBAABCAAdFiEExbkF8OjZ/ZZo/zZvot4jUGLaM/oFAmPKutQACgkQot4jUGLa M/rwsw//THL8dXc19SZjixMxCSahuJWCN6bmqMcrb0UPN97HhVF4AmBWD2Wp3C35 5JIu16DzySAt/sa+dqcW6tGfJiGOK8VNsI2zdriDQk3ZUdmtHJvxoTqa4MxdI7jU P91rSRw+gvWDmjXcY4lQGJHOuZjuDWcGUtm3THZjivCyTkN11aVbKjvDK9UGzkAl GzbvjhWiRfxRS+3iV0YlOQzW7wkU0N854RCvNnNZj+370P2fRnML/9ePaZ8yKPQ4 ktL5jF+VPBlXkjhPMF+zKAVLmHCPFGSKsn0KkwTDdvVgsXRKvO7kYsniDPlQr4oa TvE/VEYRXxuYBLKQ7moUzPH7Ra9ZU7qtRf9MUTGthMyU30YhXX0SS5miNUpzs4wp 8TBHU1visIOulkn62Krc/BGWiNiNzDhcWHokKYyxyQS8syCmz/goqzf70FTCw1V+ rCRfFN5Z/PrijmjLPX8IlPHrkFAAqK6jQVTYwQLkV0a7A/PEb+cXYSCLPM8yRTuS dNrXTmpTSO1KHk9c1MbiblJDnsIHx/U7kAW1N1M8D7rjeMVbqGRriwgt1mRqZU7D jRiDbEA9DC15Ck+9LmR2xT1aKez3wr5eA7H6o8GHhu1uLsDyWhITyBP4xvSCGCTN eSdQkExJaXiEvmj3HY3AgikoJ9iWWbP8paIJjHxnr0

Lazy me would love to get automation going akin to the one in
conda-forge (see e.g. [1]) -- whenever a new upstream is released
(as could be detected with a run of uscan) the CI would

It's not exactly what you describe (because we're rebasing), but the
Mobian team has been doing something like this for a while to update our
kernel packages. The code is at https://salsa.debian.org/Mobian-team/packaging-tools/-/blob/master/rebase-ci.yml

You'd need to replace the git work in the first few lines with uscan,
but otherwise:

- gbp pq should deal with upstreamed patches (and list them)

- The package will be pushed to a new branch and an MR opened. This
requires a bot account or access token to be created.

- After the push/MR, normal salsa jobs will be run. (This one only runs
on a schedule with a defined variable, see https://salsa.debian.org/Mobian-team/packaging-tools/-/blob/master/rebase-ci.yml#L61)

- If the patches fail to apply an issue is opened with a link to the
failed CI job.


Only real issue we've had with it so far is getting a heap of "failed to
rebase on 6.1.x" issues raised, but the kernel has a near weekly release
cycle, so it's probably not as bad elsewhere.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Fri, 2023-01-20 at 11:01 -0500, Yaroslav Halchenko wrote:

Is there any package/tool which already does something like that on CI?
Or may be you see some issues with such a procedure?

The Debian Janitor is already doing some of that for all packages,
for both new upstreams and new upstream commits.

https://janitor.debian.net/fresh-releases/ https://janitor.debian.net/fresh-snapshots/

--
bye,
pabs

https://wiki.debian.org/PaulWise

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEYQsotVz8/kXqG1Y7MRa6Xp/6aaMFAmPLJoQACgkQMRa6Xp/6 aaPv1A/8DGVqFc1xOLnOgfpLt1AbjJkLV4aJ3Jmdozpx0AEZ1LMLjykyzTl5UHcI L0qwYNEo6xmagXQyF5KYo1N3DNyQPjkNMZmYdTnxDq6SaPauTiLbHpSxxt3x6L63 23ZPcVlOyXoPv0l+SZvvLQ+w25pvtzk8iY5zuOh+z5KspIa8Sh1pYK4KqeD4c/qF U+h5mZ9rmslyYlXhgmNixvYViHzCeptl3Le+1YM7j2sv6TArNRNnk70ETjJ7Yk0u wiDvrYujFQTdTRzd7E1SnEHqAMYKXaoTkyvHLzdDTnq4J7GFhwu8kNIvxtQnFpOK c+BLAbC7QhHiMgxDQYG7ROZUtvt15zrVWVOKwzXrVwFYFDTBKZvsSxSBYcbP1Jup Z7J8JkeDrn0RQshudQJKd+/Js2X6nNHrOkDbgC4K81HhgNWE5kUp/O3rO/pZaJHz a24qFx/Y9+zegAeGuZd/rUcDHJkEFa8oJoMDnhWwdTvt7XPVS7jprawk68h2AS1S /6HDJzu2lpkv8SppNJAGSgGnBB3RJrveCcCozb18brsYVZ4JLrsJ/j9x9Ot4PUMk uuYB5RvxutbmiBX0l6xa3Jox9gjKyjz1w7bNhDC2WOILWoIYkWcDbA6ETRJVqdvb IIkehSvSbTpb5BhD0kOZjbi1Pf6Btbb2JMmCs7ZhKRzGaedoqWU=
=b8v+
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hi,

On Fri, 20 Jan 2023, at 17:01, Yaroslav Halchenko wrote:

Dear Debian gurus,

Lazy me would love to get automation going akin to the one in
conda-forge (see e.g. [1]) -- whenever a new upstream is released
(as could be detected with a run of uscan) the CI would

<...>

Is there any package/tool which already does something like that on CI?
Or may be you see some issues with such a procedure?

Not quite the same, but we have machinery to update packages from Debian in Apertis:
https://gitlab.apertis.org/infrastructure/ci-package-builder/

Related scripts live here: https://gitlab.apertis.org/infrastructure/apertis-docker-images/-/tree/apertis/v2024dev1/package-source-builder/overlay/usr/bin
https://gitlab.apertis.org/pkg/apertis-dev-tools/-/tree/apertis/v2024dev1/tools

--
Cheers,
Andrej

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Dear Andrej, Paul and undef,

I just wanted to say Thank you for all the pointers! Overall, janitor
looks great and the most advanced and seems already covering various
aspects but might be a bit "too big to tune for our own use-case", so we
might indeed pick up on ideas/implementations... we will see.

Cheers,

On Sat, 21 Jan 2023, Andrej Shadura wrote:

Hi,

On Fri, 20 Jan 2023, at 17:01, Yaroslav Halchenko wrote:

Dear Debian gurus,

Lazy me would love to get automation going akin to the one in
conda-forge (see e.g. [1]) -- whenever a new upstream is released
(as could be detected with a run of uscan) the CI would

<...>

Is there any package/tool which already does something like that on CI?
Or may be you see some issues with such a procedure?

Not quite the same, but we have machinery to update packages from Debian in Apertis:
https://gitlab.apertis.org/infrastructure/ci-package-builder/

Related scripts live here: https://gitlab.apertis.org/infrastructure/apertis-docker-images/-/tree/apertis/v2024dev1/package-source-builder/overlay/usr/bin
https://gitlab.apertis.org/pkg/apertis-dev-tools/-/tree/apertis/v2024dev1/tools

--
Yaroslav O. Halchenko
Center for Open Neuroscience http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
WWW: http://www.linkedin.com/in/yarik


-----BEGIN PGP SIGNATURE-----

iQIzBAABCAAdFiEExbkF8OjZ/ZZo/zZvot4jUGLaM/oFAmPRRRwACgkQot4jUGLa M/pJNA/9GuFsxmrknPN07u/tK6VqRL3+OLA1lGBG6hcgTek3izq7GySCHR5T+Lkf 3BgAjsLR+1Ld5myBdm8mVxFF1pvVPkwyBum3JMGXLBzP3DBUlWhlkYX6/sp4Wzqo UNfQxYiykFgAxYA9uyrDfUG95IAYSQ5K+IjHwl6AnGTc3bINEtrNTiKt18uOQWf/ H5SUy/Y2c5r5AzqXLoXjNSRDAf26dgF5LlkRiSXVGSh2XT+CGUe2iK4WvhiB7nIb hFeTnDFvoaayr7kZW0+XIQJo7wBBDwAyQyWtz5LuuGJYbHaDaUAt5XrxUMu7Gnrv c3yYjTS7mh1HNwAnNrTXVS2f6C8r39oGBLwiGRhNYXwHPFDE/AO9wwTPeVuQjVXX d9filYlO6pUJRLpz6Tw26g5oWxuwxpDm4A1HnhX2mtIK08Wnzy2fu0bLRYszSA8V 4/fzyc2ito/PeGbSPd0+k8nAL2MdQwfBA9Q6u2zGo8yK41GTxJ5JNHrUUcZ5ld3i nb4GYC7VluMdWzLxpCExpUj1WG1KywkKFbUd4LPg9KqeqzjjM/Y8fgNasiwJIMro LY1M6YaMJGL7zdon9nj7aGfI9IRE3PuU2P2MGT+Ihzxm2yUdQ83eiomlrdMegEt5 0PO8BPXBWJouJD0iUKI7C4GRF3A9PIW9oohUno/dS/

Hi Yaroslav,

On Wed, Jan 25, 2023 at 10:05:03AM -0500, Yaroslav Halchenko wrote:

I just wanted to say Thank you for all the pointers! Overall, janitor
looks great and the most advanced and seems already covering various
aspects but might be a bit "too big to tune for our own use-case", so we might indeed pick up on ideas/implementations... we will see.

The janitor runs the "deb-new-upstream" command under the hood to make the actual changes. You should be able to run that from e.g. CI. It tries to
do the right thing as much as possible without flags, but I'm happy to
add flags to allow tweaking its behaviour.

Alternatively, if the package is in Debian you could simply merge
the changes the Janitor has already published at a predictable
location (https://janitor.debian.net/git/<source-pkg> -b fresh-releases)

Cheers,

Jelmer

On Sat, 21 Jan 2023, Andrej Shadura wrote:

Hi,

On Fri, 20 Jan 2023, at 17:01, Yaroslav Halchenko wrote:

Dear Debian gurus,

Lazy me would love to get automation going akin to the one in
conda-forge (see e.g. [1]) -- whenever a new upstream is released
(as could be detected with a run of uscan) the CI would

<...>

Is there any package/tool which already does something like that on CI? Or may be you see some issues with such a procedure?

Not quite the same, but we have machinery to update packages from Debian in Apertis:
https://gitlab.apertis.org/infrastructure/ci-package-builder/

Related scripts live here: https://gitlab.apertis.org/infrastructure/apertis-docker-images/-/tree/apertis/v2024dev1/package-source-builder/overlay/usr/bin
https://gitlab.apertis.org/pkg/apertis-dev-tools/-/tree/apertis/v2024dev1/tools

--
Yaroslav O. Halchenko
Center for Open Neuroscience http://centerforopenneuroscience.org Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
WWW: http://www.linkedin.com/in/yarik

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hi Yaroslav,

Am Wed, Jan 25, 2023 at 10:05:03AM -0500 schrieb Yaroslav Halchenko:

I just wanted to say Thank you for all the pointers! Overall, janitor
looks great and the most advanced and seems already covering various
aspects but might be a bit "too big to tune for our own use-case", so we might indeed pick up on ideas/implementations... we will see.

May be a bit late, but may be

apt install routine-update

is your friend. It also calls scripts that are used by Janitor.

Kind regards
Andreas.

--
http://fam-tille.de

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Fri, Feb 17, 2023 at 09:26:17PM +0100, Andreas Tille wrote:

Hi Yaroslav,

Am Wed, Jan 25, 2023 at 10:05:03AM -0500 schrieb Yaroslav Halchenko:

I just wanted to say Thank you for all the pointers! Overall, janitor looks great and the most advanced and seems already covering various aspects but might be a bit "too big to tune for our own use-case", so we might indeed pick up on ideas/implementations... we will see.

May be a bit late, but may be

apt install routine-update

is your friend. It also calls scripts that are used by Janitor.

Does it call deb-new-upstream as well? That's the one that mainly seems relevant here.

Jelmer

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Am Fri, Feb 17, 2023 at 10:32:06PM +0000 schrieb Jelmer Vernooij:

is your friend. It also calls scripts that are used by Janitor.

Does it call deb-new-upstream as well? That's the one that mainly seems relevant here.

I just learned about deb-new-upstream. Its parsing uscan manually but its not very robust and I should probably switch to use deb-new-upstream.

Kind regards
Andreas.

--
http://fam-tille.de

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Am Sat, Feb 18, 2023 at 11:25:11AM +0100 schrieb Andreas Tille:

Does it call deb-new-upstream as well? That's the one that mainly seems relevant here.

I just learned about deb-new-upstream. Its parsing uscan manually but its not
very robust and I should probably switch to use deb-new-upstream.

I just learned that routine-update is currently broken due to a changed
output by uscan (need to check this later when I have more time). This
gave me sufficient motivation to check out deb-new-upstream. I admit
that bug #1031913 is some kind of show stopper for me currently to
consider it as a replacement for the current strategy in routine-update
to simply use uscan.

How to you work around this in janitor?

Kind regards
Andreas.

--
http://fam-tille.de

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)