Gerardo Ballabio writes ("What is the source code (was: [RFC] General Resolution to deploy tag2upload)"):

Paul R. Tagliamonte wrote:

I wonder if we have a good idea of what the project believes to be the case between #1 and #2:

1) Is the source of a package the debian source distribution?
2) Is the source of a package the VCS where the source is held?

Let me rewrite that in a different way:

1) is the source of a package the current version of the code? [*]
2) is the source of a package the complete history of the project? [**]

Yes, this ks the key question. I agree with you that the
representation doesn't matter for this purpose. [1]

My answer is that it depends on the development practices (mostly, the development practices of upstream), but for most modern software the
answer is 2.

I should be clear that I thiknk there is room for reasonable
disagreement on this, particularly since the question is so
context-dependent.

Speaking for myself, I believe the source is "the set of files that
are required in order to build the package",

Well, *that's* certainly not sufficient. Because intermediate build
products like minified js or whatever, might be sufficient to build
the package.

We want to be able to *study* and *modify* and *share* the package.
Including sharing changes we make with other people working on other
distros or directly with upstream.

This is why the GPL defines "source code" as the "preferred form for modification".

When upstream maintains the code in git, they (and their users)
typicaly sometimes *use* that git history to decide whether and how to
modify the program. That, after all, is one of the reasons the
history is useful.

In this sense, the history is like comments. You wouldn't think it
was still the source code if all the comments had been stripped out.

that is, the current version, and only that.

So, I think, often, I would disagree with you about that.

The history of the project may be useful information as it documents
how the code was developed, but it is not necessary in order to build
the package AND it is not necessary in order to develop a modified
version. One could argue that the "preferred form for modification",
as per the GPL, includes anything that might provide useful
information to a developer.

I think it includes the information that upstream have, and which
upstream use as part of the representation of the project.

One way of looking at this is that the point of this rule is that the
upstream is not supposed to be in a better position than one of our
users - at least, not because they lack some of the computer data that
upstream has and uses.

I hope this explains my view. I won't be particularly surprised if
you're not convinced, TBH - as I say, there's definitely room for
disagreement. But I thought it worth setting out my reasoning.

Thanks,
Ian.

[1] If the cover messages contained in files in debian/patches are
part of "the source", they're still part of "the source" if they are transformed into git commit messages and the actual patch files
deleted.

Conversely, if git commit messages and diffs aren't "source code",
then when they get converted into files in debian/patches they're
still not "source code" and you wouldn't be breaching any principles
if you converted the package to "3.0 (native)", retaining only the
unpacked state, and threw away the patch files.

--
Ian Jackson <[email protected]> These opinions are my own.

Pronouns: they/he. If I emailed you from @fyvzl.net or @evade.org.uk,
that is a private address which bypasses my fierce spamfilter.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

"Ian" == Ian Jackson <[email protected]> writes:

Ian> Gerardo Ballabio writes ("What is the source code (was: [RFC]
Ian> General Resolution to deploy tag2upload)"):
>> Paul R. Tagliamonte wrote:
>> > I wonder if we have a good idea of what the project believes to
>> be the case between #1 and #2:
>> >
>> > 1) Is the source of a package the debian source distribution?
>> > 2) Is the source of a package the VCS where the source is held?
>>
>> Let me rewrite that in a different way:
>>
>> 1) is the source of a package the current version of the code?
>> [*] 2) is the source of a package the complete history of the
>> project? [**]

Ian> Yes, this ks the key question. I agree with you that the
Ian> representation doesn't matter for this purpose. [1]

Ian> My answer is that it depends on the development practices
Ian> (mostly, the development practices of upstream), but for most
Ian> modern software the answer is 2.

Ian> I should be clear that I thiknk there is room for reasonable
Ian> disagreement on this, particularly since the question is so
Ian> context-dependent.

I agree with Ian's answer.

I happened to do some digging through mail archives about our internal discussions about preferred form of modification over the years
recently.

My conclusions are:

* There is no consensus here. We've been arguing about related issues
since 2003 and while the argument has advanced, we're no more near
consensus now than then.

* Simon Mcvittie talks about "a preferred form of modification" rather
than "the preferred form of modification."
I think that is a useful think to keep in mind.

Even for a GPLed work, I do not think we would consider either a release tarball or a git repository to fail to comply with the GPL's requirement
to distribute the preferred form of modification for a work.
(Obviously, other factors might impact things. For example, if the work included minified javascript without corresponding source, a lot of us
would argue that failed to be the preferred form of modification. Note I
don't even think there is project consensus on that point; it doesn't
matter because there is consensus in the FTP team that minified
javascript is not enough without the source somewhere in the archive.)

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

In this sense, the history is like comments. You wouldn't think it
was still the source code if all the comments had been stripped out.

But if by mistake one upstream adds a proprietary file in git and then removes it… we can't distribute the whole .git directory as is.

--
Salvo Tomaselli

"Io non mi sento obbligato a credere che lo stesso Dio che ci ha dotato di senso, ragione ed intelletto intendesse che noi ne facessimo a meno."
-- Galileo Galilei

https://ltworf.codeberg.page/

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

"Salvo" == Salvo Tomaselli <[email protected]> writes:

>> In this sense, the history is like comments. You wouldn't think
>> it was still the source code if all the comments had been
>> stripped out.

Salvo> But if by mistake one upstream adds a proprietary file in git
Salvo> and then removes it… we can't distribute the whole .git
Salvo> directory as is.


Can we dig into this a little?
I'd like to compare along two directions:
First, what do you mean by proprietary.
Let's assume you mean that upstream includes a file that they don't have
rights to distribute at all.
For example, someone accidentally checks in confidential passwords to an internal CI/CD environment, or checks in something that is a trade
secret.

Second, let's compare this to shipping a tar ball.

This same thing can happen with a tar file.
We could discover that we are distributing some file that we do not have
rights to distribute in a stable release--burned onto DVDs and in our
signed images.

We could get a takedown notice for such a stable release years later.
And depending on the circumstances, we might well have to comply. If we actually found ourselves distributing someone's trade secret, we
probably would have to comply.
It would be a huge mess, but legally we would have to find a way to do
it.

But in a lot of cases, we probably could get away with removing it from
future releases, and probably would not even have to stop shipping the
old release.

If for example we found ourselves distributing something that we legally
could distribute, but that did not end up following the DFSG, that's the approach we would (and have regularly) taken.

It's likely we would have similar options for a git repository.
In the most extreme cases, we would need to rewrite history.
There are tools to do this relatively painlessly.
It absolutely would break the signatures on tags. We could resign the
tags as part of the processes.
In some cases, depending on the legal reason we could not redistribute,
being part of a signed tag might give us legal leverage for not removing
the information.

In other cases, we probably could get away with keeping the information
in tags, but rewriting current branches, and removing the information
from future releases.


we face all these challenges with dgit and salsa today.
We also face these challenges with tar files today and
archive.debian.org and snapshot.debian.org.

--=-=-Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iHUEARYIAB0WIQSj2jRwbAdKzGY/4uAsbEw8qDeGdAUCZnNdJgAKCRAsbEw8qDeG dDtzAP9PlWKHWcH0fj2TTQVgdq+2BRmS2eLchgBorQe2XjmTLQD/Vd1ab24zRRL0 H4IYXNIFiUzX6DGndG1LN0xzHTgkuQg=iBOK
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Wed, Jun 19, 2024 at 04:35:18PM -0600, Sam Hartman wrote:

we face all these challenges with dgit and salsa today.

Yeah, the argument against shipping git repos in the archive predates
salsa and alioth wasn't used as widely. I always felt like salsa is a counter-argument to it.



--
WBR, wRAR

-----BEGIN PGP SIGNATURE-----

iQJhBAABCgBLFiEEolIP6gqGcKZh3YxVM2L3AxpJkuEFAmZz91stFIAAAAAAFQAP cGthLWFkZHJlc3NAZ251cGcub3Jnd3JhckBkZWJpYW4ub3JnAAoJEDNi9wMaSZLh /JYP/1YUBVc6JsbwKqnALPvkz35eh67Ga3KSyZ/6DYecf/eo45XaS5dAeLv1S7ZC v84jYzArpcNM7ipMJpZWguug++euzv0MxGS7xMsrnLnabeJXJyZTlX8UcBP+cgXW PGz5x21NtGS+gEX67RA+sDVj4GXwiiiPCUqFRvMTQTIG05tOfI+sZUXrleYhQWP+ +lUib2AK+orQ8JP85dUcGs/prk7aCrAyrFd64qWEOtw343XC5IRw5caglfKLuQPx Qe8nryBJNl5Qg4VH2rE8nT4sp2Sae/qEgv12+HFeU1jmciUJVnBpENJIZOdYogK8 TUFHWINYY8cpieK/hdMF/o57Vi/MB+rUsQK5WjDUzrrMVwbYFQtzY9Cis2ywG9rv 7PDFmSK6QY4Ec/dhOxOMkKgpRONahFA8N5edDGnUmBuzIK5HEaHlhvmNa4YR2NFa Z1KUET9yT3+QYfpFpy2fxxu4ndUz/kuaNdbaTKFjNd69onkeJS0qGeZ3L729pGpG ddTbkEZuGgP7OqvAfqZW5zguJ+atQIOPIPxJ2foflkSpgqdDo0GDKQiVfp7waxCO sBEJQ0GXklZGJ1cuuIKrFxa/lwZCrdIImjBxue0A8cFsJabnAzB2e7kl6e4iD1jV Ex3uwXPp8CJz94u6uU3wqD3Ow4KynTqDB0vJfFp3nl7cysal
=fh91
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Wed, 19 Jun 2024 at 12:57, Gerardo Ballabio
<[email protected]> wrote:

Paul R. Tagliamonte wrote:

I wonder if we have a good idea of what the project believes to be the case between #1 and #2:

1) Is the source of a package the debian source distribution?
2) Is the source of a package the VCS where the source is held?

Let me rewrite that in a different way:

1) is the source of a package the current version of the code? [*]
2) is the source of a package the complete history of the project? [**]

That is a very different set of questions and that is based on a false premise.

There is a very significant difference between the files in the latest
debian source distribution and the
files in the latest checkout of the git repo that the Debian developer
is using to maintain that package.

The latest checkout does not even actually have all the information
that is required to generate
the contents of the produced Debian source distribution.

For example, in the git-debrabase workflow that Russ described on this
list recently the latest checkout
would not have any of the debian/patches files in it. The patches in
this case are represented as separate
commits in the recent git history that are rebased on top of the
latest upstream release version commit.
Conversions from git repo to Debian source *generates* those patch
files. It is all still *just* the source for
the current release version, but it is not coming from latest
checkout, but from recent commit history.

The minimal set of source data would then be the contents of the
files/folders in the latest upstream release,
plus contents and metadata of all rebased patches on top of that
latest release commit. And that is the source
as that is the preferred way to modify this software.

History before the upstream release commit does not really matter
anymore in the source code context. If one
would shallow clone the git repo with depth to at least the last
upstream merge commit, they will get the same
results in generating the Debian source package and in all following steps.

Using git for maintenance does not require keeping or shipping the
*entire* history of the project. Just the latest
iteration is enough. But that one latest iteration may involve more
than one file/folder tree and more than one commit.

The questions that Paul wrote are very different: is the specific
format that Debian chose to package the source
code for consumption inside Debian toolchain considered to be *the*
source for software, or is it just an
intermediate technical artefact of the process and the *actual* source
of the software is whatever VCS or workflow
that the Debian and/or upstream developer is using to actually manage
and modify the software.

It has nothing to do with history. Unless you want to do deep dives
and do git blame research. Something that is
not possible with Debian source code packages beyond the uploader/maintainer/developer boundary.

--
Best regards,
Aigars Mahinovs mailto:[email protected]
#--------------------------------------------------------------#
| .''`. Debian GNU/Linux (http://www.debian.org) |
| : :' : Latvian Open Source Assoc. (http://www.laka.lv) |
| `. `' Linux Administration and Free Software Consulting |
| `- (http://www.aiteki.com) |
#--------------------------------------------------------------#

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Thu, 20 Jun 2024 at 13:19, Ian Jackson
<[email protected]> wrote:

Aigars Mahinovs writes ("Re: What is the source code (was: [RFC] General Resolution to deploy tag2upload)"):

On Wed, 19 Jun 2024 at 12:57, Gerardo Ballabio
<[email protected]> wrote:

1) is the source of a package the current version of the code? [*]
2) is the source of a package the complete history of the project? [**]

That is a very different set of questions and that is based on a
false premise.

I think you're talking somewhat at cross purposes with Gerardo and
Paul. This subthread is about a wider, more philosophical question:
is the git history, in the general case, a necessary part of "the
source code".

I was more objecting to the conflation of "necessary" and "desirable"
in the context of
"all history" and "just the latest history".

I think it is desirable to have as much of a historical view as
possible on the source
code and its development. Even to the point of upstream switching
being represented
in the historical view as overwriting merges from a different branch.

Is it, strictly speaking, *necessary* for the purposes of source code distribution, preservation
and enabling of derived works? I don't think so. a shallow clone with
just enough history
to be able to reproduce the source package build is what would be the
minimum necessity to
publish such source code. And in certain cases (like epoch changes or
history being
contaminated by non-redistributable files) the source distribution can
be rolled back to this bare
minimum to preserve technical sanity.

It has nothing to do with history. Unless you want to do deep dives
and do git blame research. Something that is not possible with
Debian source code packages beyond the uploader/maintainer/developer boundary.

History diving is an important part of the maintenance and development
of much software, nowadays. Which is why I'm in your "unless":

I think the git history is often an essential part of the source code.
And yes, that means, that for many packages, I think what is published
in the Debian archive as the "source code" is *not* the source code.
It's an intermediate build product.

I do agree with that sentiment. Just did not want to push it as
forcefully as to state that
as a requirement which would then bring up heavy technical and
possibly legal issues with it.
--
Best regards,
Aigars Mahinovs mailto:[email protected]
#--------------------------------------------------------------#
| .''`. Debian GNU/Linux (http://www.debian.org) |
| : :' : Latvian Open Source Assoc. (http://www.laka.lv) |
| `. `' Linux Administration and Free Software Consulting |
| `- (http://www.aiteki.com) |
#--------------------------------------------------------------#

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Aigars Mahinovs writes ("Re: What is the source code (was: [RFC] General Resolution to deploy tag2upload)"):

On Wed, 19 Jun 2024 at 12:57, Gerardo Ballabio
<[email protected]> wrote:

1) is the source of a package the current version of the code? [*]
2) is the source of a package the complete history of the project? [**]

That is a very different set of questions and that is based on a
false premise.

I think you're talking somewhat at cross purposes with Gerardo and
Paul. This subthread is about a wider, more philosophical question:
is the git history, in the general case, a necessary part of "the
source code". I think this isn't a simple question. Gerardo is
pointing out, correctly, that this doesn't depend on the
*representation* of the previous version information.

This subthread isn't about the source package construction question
(which is the real core of the disagreement we have with ftpmster).

On a factual point, though:

For example, in the git-debrabase workflow that Russ described on
this list recently the latest checkout would not have any of the debian/patches files in it. The patches in this case are represented
as separate commits in the recent git history that are rebased on
top of the latest upstream release version commit.

I should point out that with git-debrebase, although the *metadata*
and *unpatched version* information is (in the maintainer's git) only
in the form of commits, the final patches applied version *is*
in the maintainer's git tip - that's what the tree is.

Like I said in my other mail, no matter what your views are on the "is
history part of the source code" question, whether you think the
contents of debian/patches is a necessary part of the source can't
depend on the representation.

Either (for a particular package) the commentary etc. in
debian/patches is not "part of the source code" even though it's
present in the "3.0 (quilt)" package; or (for that package) it *is*
part of the source code even though in the maintainer's git tree it's
only in the history, not in the tree.

I think most users of git-rebrebase think the patch stack, which they manipulate as a rebasing git branch, is very much part of the source
code. You wouldn't want to try to update to a new upstream version by
just merging the trees, or trying to apply and resolve one mega-diff.

There will be other packages where things are different. One of the
things that's so striking about this area, that we have learned while developing dgit, is how radically different different people's
workflows and opinions are. Debian contributors who have mostly
worked on their own packages won't have seen this diversity are likely
to (quite understandably) wildly underestimate the diversity and
complexity.

It has nothing to do with history. Unless you want to do deep dives
and do git blame research. Something that is not possible with
Debian source code packages beyond the uploader/maintainer/developer boundary.

History diving is an important part of the maintenance and development
of much software, nowadays. Which is why I'm in your "unless":

I think the git history is often an essential part of the source code.
And yes, that means, that for many packages, I think what is published
in the Debian archive as the "source code" is *not* the source code.
It's an intermediate build product.

Ian.

--
Ian Jackson <[email protected]> These opinions are my own.

Pronouns: they/he. If I emailed you from @fyvzl.net or @evade.org.uk,
that is a private address which bypasses my fierce spamfilter.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

I'm going to try to ramp down my involvement in this subthread about
the philosophical question of whether and when the git history is a
necessary part of "the source code". Just one more message...

Gerardo Ballabio writes ("Re: What is the source code (was: [RFC] General Resolution to deploy tag2upload)"):

Il giorno mer 19 giu 2024 alle ore 19:03 Ian Jackson <[email protected]> ha scritto:

[the rest of your message]

Overall, you make some very valid points. But if the concept that "the history is the source" were really upheld and taken to its
consequences, it might disrupt the whole free software ecosystem.

I agree that there are implications for the way we share software.
(I prefer to think of this from an ethical and ideological point of
view, rather than a legal one.)

Because, then, every upstream of a GPL project who develops on a
private VCS (or without using a VCS) and only releases tarballs would
be violating the GPL, and nobody could distribute binary packages
because they couldn't provide the "complete corresponding source
code".

I think a downstream who is using git, and distributing binaries,
should indeed publish their git history. (In the cases where git is
the source, but I think this is common.)

I don't know if it'sa still true, but a high-profile Linux distro were
in the habit of providing their Linux kernel sources to their users
only as an upstream source tarball plus a single giant diff - but,
internally they're using git. Keeping the git to themselves in this
situation seems like a fairly blatant trick to stop people from
exercising their (moral and legal) rights.

And since Debian currently distributes only the current version in its
source packages, Debian would be in violation too.

For these packages, I don't view the source packages as discharging
our (moral and legal) obligations, indeed. We *do* sort-of discharge
those obligations because typically the maintainer publishes the git
history on salsa, somewhere, in some form or other.

If you're a Debian expert, salsa is often where you get the source
code to a program if you want to seriously work on it. (It seems to
me that this is another strong signal that we *do*, in practice,
consider the salsa git repos to be the preferred form for
modification.)

And tag2upload isn't a step towards fixing that, because (as I
understand, correct me if I'm wrong) it too builds packages that only
contain the current version.

Yes, tag2upload doesn't make source packages significantly better at discharging our (moral and legal) obligations. It may have integrity
and convenience benefits, and fix a few anomalies, but it doesn't
improve the actual source packages very much.

But tag2upload *does* arrange the publication of the git history, in a standardised and official and machine-useable place - the dgit-repos
git server.

So I think it does address this problem.

Furthermore, there's another issue regarding the freeness of the
package: if the source must include the history, then you aren't
actually completely free to modify the source, because you can't
delete the history.

I'm not sure if you mean a barrier that is practical, or legal/moral.
You certainly can rewrite the history. Whether you *should* is
another matter. (Sometimes you might be legally *required* to, as has
been discussed in elsewhere in this megathread.) I think this is
related to, for example the GPL's clause requiring notice of
modification (5(a)). Those notices are a bit like an invariant
section, too. I don't see the problem here that you do.

Ian.

--
Ian Jackson <[email protected]> These opinions are my own.

Pronouns: they/he. If I emailed you from @fyvzl.net or @evade.org.uk,
that is a private address which bypasses my fierce spamfilter.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Gerardo Ballabio writes ("Re: What is the source code (was: [RFC] General Resolution to deploy tag2upload)"):

And if you relax the requirement and allow for history to be "edited",
where do you draw the line?

Well, I'm afraid I just have to say that I think there *is* a line.
It's not just not hard and fast, nor susceptible to automatic
calculation.

This is a moral and ethical and legal question. Moral, ethical, and
(yes!) legal questions often have exceptions and grey areas.
This is one.

Or to put it another way, when I say "the git history is usually part
of the source code", I don't mean "always the one and only unaltered
history that is never rewritten". Like the question of whether the
git history is part of the source, the question of whether that
history can or should be rewritten is context-dependent.

I think *usually* the git history is part of the source, but sometimes
it isn't (and sometimes it's even harmful and must be discarded).
Usually you wouldn't (and shouldn't) rewrite the history, but
sometimes you should (and sometimes you must).

Ian.

--
Ian Jackson <[email protected]> These opinions are my own.

Pronouns: they/he. If I emailed you from @fyvzl.net or @evade.org.uk,
that is a private address which bypasses my fierce spamfilter.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --------------pHISE0HukkQDG7l6vZpVDlmT
Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: base64

T24gMjAuMDYuMjQgMTU6NTIsIEdlcmFyZG8gQmFsbGFiaW8gd3JvdGU6DQo+IEFuZCB0YWcy dXBsb2FkIGlzbid0IGEgc3RlcCB0b3dhcmRzIGZpeGluZyB0aGF0LCBiZWNhdXNlIChhcyBJ DQo+IHVuZGVyc3RhbmQsIGNvcnJlY3QgbWUgaWYgSSdtIHdyb25nKSBpdCB0b28gYnVpbGRz IHBhY2thZ2VzIHRoYXQgb25seQ0KPiBjb250YWluIHRoZSBjdXJyZW50IHZlcnNpb24uDQoN Ck9uIHRoZSBvdGhlciBoYW5kLCBkZ2l0IGRvZXMgc2VuZCB0aGUgZ2l0IHRhZyB0byBhbiBh cHBlbmQtb25seSBhcmNoaXZlLg0KDQpDb25jZWl2YWJseSB3ZSBjb3VsZCBwb2ludCBwZW9w bGUgdG8gdGhhdCBzZXJ2aWNlIGFuZCBzYXkgImhlcmUncyB0aGUgDQpjb21wbGV0ZSBzb3Vy Y2UiIGFuZCBkcm9wIG91ciBtaXJyb3JlZCBzb3VyY2UgYXJjaGl2ZSBlbnRpcmVseS4NCg0K PiBGdXJ0aGVybW9yZSwgdGhlcmUncyBhbm90aGVyIGlzc3VlIHJlZ2FyZGluZyB0aGUgZnJl ZW5lc3Mgb2YgdGhlDQo+IHBhY2thZ2U6IGlmIHRoZSBzb3VyY2UgbXVzdCBpbmNsdWRlIHRo ZSBoaXN0b3J5LCB0aGVuIHlvdSBhcmVuJ3QNCj4gYWN0dWFsbHkgY29tcGxldGVseSBmcmVl IHRvIG1vZGlmeSB0aGUgc291cmNlLCBiZWNhdXNlIHlvdSBjYW4ndA0KPiBkZWxldGUgdGhl IGhpc3RvcnkuDQoNClRoZSBjb25jZXB0IG9mICJzb3VyY2UgY29kZSIgaXNuJ3QgYXMgY2xl YXIgY3V0IGFzIG9uZSBtaWdodCB3aXNoLiBUaGUgDQpHUEwgb25seSBzdGF0ZXMgdGhhdCBz b3VyY2UgaXMgd2hhdGV2ZXIgeW91IGFjdHVhbGx5IHRvdWNoIHdoZW4geW91IHRvIA0KbW9k aWZ5IHRoZSBwcm9ncmFtLiBUaGF0IGRlc2NyaWJlcyBhIGRpcmVjdG9yeSB0cmVlIHdpdGgg YSBidW5jaCBvZiB0ZXh0IA0KZmlsZXMgd2hpY2ggeW91IHJ1biAibWFrZSIgb3IgImRlYnVp bGQgLWIiIG9yIOKApiBpbi4gTm8gVkNTIG1lbnRpb25lZCwgDQp0aHVzIG5vdCByZXF1aXJl ZC4gTGljZW5zZXMgYXJlIGZ1bm55IHRoYXQgd2F5Lg0KDQpUaGUgaGlzdG9yeSBpcyBvYnZp b3VzbHkgaXJyZWxldmFudCBmb3IgZWRpdGluZyBwZXIgc2UuIEl0J3Mgb25lIHNvdXJjZSAN CmZvciBkZWNpZGluZyB3aGF0IHRvIGNoYW5nZSAoYW5kIGhvdyksIGFuZCBpdCdzIHVzZWQg dG8gb3JnYW5pemUgYW5kIA0KZGlzdHJpYnV0ZSB0aGVzZSBjaGFuZ2VzIGFmdGVyd2FyZHMu IEJ1dCB0aGF0IHBhcnQgaXNuJ3QgY292ZXJlZCBieSBhbnkgDQpsaWNlbnNlOyB5b3Ugd2Fu dCB0byBiZSB1bmhlbHBmdWwsIHdlbGwsIHB1c2ggYSAiMy4wIChuYXRpdmUpIiB0cmVlIHRv IA0KdGhlIGFyY2hpdmUgYW5kIGxldCB3aG9ldmVyIHdhbnRzIHRvIHRyYWNrIHlvdXIgY2hh bmdlcyBwdXp6bGUgdGhlbSBvdXQsIA0Kbm90IGEgcHJvYmxlbSBhcyBmYXIgYXMgdGhlIEdQ TCBpcyBjb25jZXJuZWQuDQoNCi0tIA0KLS0gcmVnYXJkcw0KLS0gDQotLSBNYXR0aGlhcyBV cmxpY2hzDQoNCg==

--------------pHISE0HukkQDG7l6vZpVDlmT--

-----BEGIN PGP SIGNATURE-----

wsF5BAABCAAjFiEEr9eXgvO67AILKKGfcs+OXiW0wpMFAmZ0PZIFAwAAAAAACgkQcs+OXiW0wpN/ qg//f+DXI/gTCP8arpjaSkGQS3yScCBt88q1xjR7ckMzwk8FUBnlXi4UQ1XTKEbmGPsgIOqdBMHG anDtXKdaX09pYsCO+ibUEvysJGoBME4iTJ5DijQX0y+VdLq4eA0pBgbfWMsp8YXRGN+6V0dH7hIq /bwXua3cV/Uu+ilc1Yz5+bE9azteulEJdyNNqfqVYVoDPvZfIY2gYgoMLfgeY62/M0JLVlLHfbRo gnehCWJVNL4wtHyfdvBZHwlXAEvsy/p4zLV3OOS9dXWSlrrXv8Y25Plr/c8WccF9mAukQdkLYQl7 qbKnF1xc+dwhd901MchLNCOXJuzFrsObdFiV4OQZHV9Wxkd++TV+WOxT3iqKACpama8RSHIJr7UW jQYC57DUxbxnmjviX3ec1BY7qxXKay4XeHY7EoR4MYtXs8uwpUF6FgWQ6N3k3Pa6DngQbsHRAGTp +Wn3PG63mAS7JindTsMg/KdAUvVMfi2ogGurl+JA0ECcRs5k4uchR1JKZezsUD8LE8llgS1cXfj6 wJVDEhG+UtSK5G2QfkyneiXq6nlMM5DhNX65iogDJxyA5fYWv+2F7+l6SSQ7XkIbXSrG2pcAyxHy RE7ztt2DjmtprxoN+7FI1vEz/eGdMetjO5euinxnujKPAfCzZCh19G/oMh79K8raTVXdsPjAIjKD nlw=
=pmtH
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Thu, Jun 20, 2024 at 11:17:42AM -0400, Paul R. Tagliamonte wrote:

Is uploading an NMU by dgetting the .dsc, modifying it, and dputing the changed source polite these days? Is it something we want to encourage? Discourage? Should it happen in Git?

As long as there is no way to properly contribute to someone else's repo (because there is no way to find what workflow you need to use and also learning a new workflow for every repo is too time-consuming and also not
in all repos HEAD corresponds to the last unstable upload, shifting in
either direction) basing NMUs on .dsc from the archive is the only
scalable way.

We want^W^WIt would be better to discourage this, of course, by
standartizing a single worklow or two, but people will resign. And I don't
have a proposed solution to "the repo has changes on top of the last
unstable upload".


--
WBR, wRAR

-----BEGIN PGP SIGNATURE-----

iQJhBAABCgBLFiEEolIP6gqGcKZh3YxVM2L3AxpJkuEFAmZ0UWMtFIAAAAAAFQAP cGthLWFkZHJlc3NAZ251cGcub3Jnd3JhckBkZWJpYW4ub3JnAAoJEDNi9wMaSZLh qbkP/2BtiXq/56Jr1Hc167B8Qp+FJmsML57M0+H/lGb/+nX+ptL3T9GZu0FdV5j1 R9V7u+UxAt69s/duwr/mKjGoOYG/dUQkbLH/xvorA4Zkzu20UhZtyomevFcVM9oj kmzHjt1ImcuFkMhA/M9tg6VPk+umwLtDJIhRKmgTMDs8yHa3QD+ufxtZc+hYpN35 0RUSIkx9sHXhRYk5RnIkslEEQgIchMkWxJvdSdfUefn3AdfFfLZn5E+HmTrwApLT g+THdGptTRRYgmxrHlNhbuwBeiQpW7ORKXp9wvLTcGKVrS3BdNrXVBDRFCL5cMTj WPvY4dW1bHjx5aX2DtT+3BMNX6lfvKXbeXHXWnykhJ5IK4waNjwuzHMF1Ayyw3wg aH6+qr1+KU03WyVxFsojL1uMUeo2beNZboGKW8DbzhfXdyS9WUW3JPn+CN+GbWqM N7VLk6DhPlU2QmmS6U+cZ41DPKOCpNn1qHHtTqI5VtbujN39M6omcqlzC3JBn/Kl PmZ68Q2hiUxqewz8V43S3WAr4qF8mXYKy/bZF3GKketx7qRrHFZggQGXKYK1LdO9 UDDrP54Coe6JX+WR4xe1RwBDXUhIro7tLnld6fRxKX4oYUJMpPFCWKQSkPi9ciKg fAxoYiFQt3c/VH2CsM9XXlkhAHzTJvFSf+zXdZHcSegKgU6A
=+JLL
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hi Ian,

On 6/20/24 23:27, Ian Jackson wrote:

I think *usually* the git history is part of the source, but sometimes
it isn't (and sometimes it's even harmful and must be discarded).
Usually you wouldn't (and shouldn't) rewrite the history, but
sometimes you should (and sometimes you must).

Over time I'd expect that to come up quite a few times, so we need to be organizationally prepared for that -- either have a history redaction
team or expect maintainers to be able to do that.

I still have a hard time picturing how this is going to go into
practical use over years, especially with rising numbers of users of
such a system.

The technical aspect of doing uploads from git is the easy part, but we
also need to bring people with various levels of existing git knowledge
up to speed on packaging in a git-like environment with several
different repository layouts that have subtle differences and pitfalls.

From a didactic point of view, if I had the time to be a mentor, I'd
teach "traditional" packaging first, and extend that with git recipes
later on, once the basics are there, but obviously that is a non-starter
for having people join team-maintained packages.

I don't see a good structured approach to teaching git-based packaging approaches first though, because these would require both marking parts
of their existing git knowledge as inapplicable (because it would create
a repository structure that doesn't fit the expected layout), and also
teaching magic incantations that they have no mental model for yet, to interface with "legacy" systems.

That's why I think having a conceptual split between collaborative
maintenance and the public archive makes sense, and abbreviating history
in the archive limits the impact of redaction actions: the archive is
redacted by ftpmaster, who are generally expected to know what they are
doing, and getting the collaborative maintenance repository into a state
that allows releases to be made without reuploading the offending blobs
should be reasonably easy as well, because if the blob is not part of
the current state, it will not be part of the upload.

Also, I think the usefulness of our collaborative maintenance history is overrated -- in those packages where I actually use salsa and git to collaborate, we also have a lot of "fix problem", "actually fix
problem", "disable feature for release", "release", "revert 'disable
feature for release'" commits that are an accurate audit trail, but less
useful as documentation than a squashed history.

Simon

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --------------IpGhlg0IgzH5yRn8vr7f9oUk
Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: base64

T24gMjAuMDYuMjQgMTc6NTcsIEFuZHJleSBSYWtobWF0dWxsaW4gd3JvdGU6DQo+IEFzIGxv bmcgYXMgdGhlcmUgaXMgbm8gd2F5IHRvIHByb3Blcmx5IGNvbnRyaWJ1dGUgdG8gc29tZW9u ZSBlbHNlJ3MgcmVwbw0KPiAoYmVjYXVzZSB0aGVyZSBpcyBubyB3YXkgdG8gZmluZCB3aGF0 IHdvcmtmbG93IHlvdSBuZWVkIHRvIHVzZQ0KDQpJZiB0aGUgcGFja2FnZSBoYXMgYmVlbiB1 cGxvYWRlZCB1c2luZyB0MnUgdGhlIHdvcmtmbG93IHVzZWQgZG9lc24ndCANCm1hdHRlciBi ZWNhdXNlIGRnaXQgdW5kZXJzdGFuZHMgaXQgYW5kIHlvdSBvbmx5IG5lZWQgdG8gcHVzaCBh IG5ldyB0MnUgDQp0YWcgdG8gbWFrZSBpdCBoYXBwZW4uDQoNCk9mIGNvdXJzZSB5b3UgZmly c3QgbmVlZCB0byBmaWd1cmUgb3V0IGhvdyB0byBmaXQgeW91ciBwYXRjaCBpbnRvIHRoZSAN CmV4aXN0aW5nIHJlcG9zaXRvcnkgc3RydWN0dXJlLCBidXQgcHJlc3VtYWJseSB0aGF0J3Mg Y29tcGFyYXRpdmVseSBlYXN5IA0KZ2l2ZW4gdGhhdCB5b3VyIGNoYW5nZSBpcyB1bmxpa2Vs eSB0byBiZSB0aGUgZmlyc3Qgb25lLg0KDQotLSANCi0tIHJlZ2FyZHMNCi0tIA0KLS0gTWF0 dGhpYXMgVXJsaWNocw0KDQo=

--------------IpGhlg0IgzH5yRn8vr7f9oUk--

-----BEGIN PGP SIGNATURE-----

wsF5BAABCAAjFiEEr9eXgvO67AILKKGfcs+OXiW0wpMFAmZ0ciYFAwAAAAAACgkQcs+OXiW0wpPS LxAAn4AtfwfUTCpfF8jkQckXFOD4nYLoRV4oC9T3PjdSLF4w7G5kvMvL2KsT3ITSHgmAnhm/lHdb KzWBPxxnJusA6FjzvdKtGN5eIpteX7j5ngO8CQPaIWp90odxbJ81X8aSEFjTqwpKg2Vyq21Q2ids AfaxTXS4WMdYnkqbwJclls/qWj9j1MUxWtkIfbI+3nxIuaa76yDUIScs5pt7c2YvMSEp9LKOGTKv AM6BY0+DB7jSLG284rjApHhh27sxZcdYbFDRaoiSY5OGYBqINFIR2FeTAKrx8+Ai5zRd3A80f70p mFepvPTipmZPQvZZuzKK7nskEWlHMWw0ybb+jGAk6ytpq59bB8mHHbr0fDpiu0GU59yth2sYwxWR M/Wf31p1o5CwF+4gU5+D1ofuOiNUqJ2v+ehOn2u7OAkcgQpkmUQu9EDJkcdfN3Fmtl+VsSji7JE3 eoVpe6M7S0efiW3cxKNFr3RnHR/42fOSxVmq3yrZJes0iYNJ/Hw/08Lrm1pPqrb9G9T86PopYpEE T4CTwn5gBz3LHsskamu0FRU+gl1FXU+6BTb4bQL/sPUYb97ldbz8YPPrQo0nSfOtlmmZIVbanWhE b/3vZI/nxq9S38eohk1B7GSaMF3B5nE1CeXwk4qBa3UpiR6Cfj2fJv7GkIQXNaRvJjcomvfRwhqK yQQ=
=r+nI
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

If you're a Debian expert, salsa is often where you get the source
code to a program if you want to seriously work on it.

I use apt source. I don't care about the unreleased untested changes that are likely to be present on salsa.

At work I made a whole system to add patches to packages, and of course it
uses apt source, because using salsa is not possible to automate anything. And we want what's in the archive + our patch, nothing else.

And we have an internal mirror of the archive. Salsa is slow.

If you want to contribute upstream, you follow the upstream procedure, and salsa won't be involved.

So at least in my case, I never touch salsa if it's a non-debian activity.

--
Salvo Tomaselli

"Io non mi sento obbligato a credere che lo stesso Dio che ci ha dotato di senso, ragione ed intelletto intendesse che noi ne facessimo a meno."
-- Galileo Galilei

https://ltworf.codeberg.page/

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Paul R. Tagliamonte writes ("Re: What is the source code (was: [RFC] General Resolution to deploy tag2upload)"):

If we (as a project) truly regard a .dsc/source distribution as an unfortunate intermediate build artifact that we wish to offload to a
source buildd network, I have to wonder why we keep them around.

This is an illuminating question. I have two answers:

Firstly, not everything is maintained in git (upstream, or in Debian),
or is compatible with dgit or tag2upload. So there are packages for
which the tarballs-and-patches really is primary. I think they're not
the usual case, any more, but such packages do exist.

The other is that we have a vast ecosystem of tooling, culture, and
personal learning, surrounding source packages. We can bodge the not-really-maintained-in-git packages into git by importing tarballs,
but getting rid of source packages completely would be hugely
disruptive - both technically and socially.

One of the nice things that tag2upload does is decouple the form in
which a user consumes a package, from the form in which the maintainer
uploaded it: you upload using git, and the infrastructure generates
*both* the source package and a canonicalised git format.

I'd like to have the same decoupling for packages that aren't natively
in git, too. `dgit clone` sort of does that, but it does it on the
client. Eventually I'd like to see the infrastructure automtically
import every non-git-based .dsc into git. Then, it will be possible
to obtain the source code of *any* package, in a predictable and
buildable form, with a simple git clone.

When we have that, we will support *both* git-based and non-git-based workflows, end-to-end (from the original upstream through to our
users), through all our workflows and infrastructure. And we'll have a project-maintained canonical gateway between the two views of the
Debian source code. Everyone can have their favourite breed of pony.

Ian.

--
Ian Jackson <[email protected]> These opinions are my own.

Pronouns: they/he. If I emailed you from @fyvzl.net or @evade.org.uk,
that is a private address which bypasses my fierce spamfilter.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Andrey Rakhmatullin writes ("Re: What is the source code (was: [RFC] General Resolution to deploy tag2upload)"):

On Thu, Jun 20, 2024 at 11:17:42AM -0400, Paul R. Tagliamonte wrote:

Is uploading an NMU by dgetting the .dsc, modifying it, and dputing the changed source polite these days? Is it something we want to encourage? Discourage? Should it happen in Git?

As long as there is no way to properly contribute to someone else's
repo (because there is no way to find what workflow you need to use
[...] basing NMUs on .dsc from the archive is the only scalable way.

Yes, this is true.

tag2upload improves this situation, too. [0]

Firstly, as an NMUer, you can work in git, *without knowing the
maintainer's workflow* [1]. This is because tag2upload generates not
only a coherent source package, but also a canonicalised git view,
which is stored in a canonical location.

That canonicalised git view contains all of the maintainer's git
history, as an ancestor, and (in the usual case) represents the patch
queue as commits. So, for example, git log and git blame on an
upstream file which is modified by patches, correctly shows the
patches in that file's git history.

Secondly, because we now have a taxonomy of maintainer git branch
formats, it is possible to write a tool that would turn an NMU made
with tag2upload into an MR on salsa in the maintainer's branch
format. I think this could be done purely in git. [2]

Ian.

[0] Much of what I write here is all already possible if everyone is
using dgit - but only a minority of maintainers are using
`dgit push-source`, so it's not as good as it could be.

[1] This isn't true for new upstream versions. Right now if you want
to do that a as an NMUer you typically need the maintainer's
permission. And the maintainer will insist you update their git, on
salsa, so you end up having to use their workflow.

[2] Even, I think, a "new upstream version" NMU made *using the
NMUer's preferred git workflow* (assuming that the NMUer's workflow
preserves the patch stack integrity - so for example gbp pq, or
git-debrebase). This conversion is not *easy*, but it becomes
possible.

--
Ian Jackson <[email protected]> These opinions are my own.

Pronouns: they/he. If I emailed you from @fyvzl.net or @evade.org.uk,
that is a private address which bypasses my fierce spamfilter.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --------------aQAio1gpyT5FEehSlTobFMLR
Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: base64

T24gMTkuMDYuMjQgMjM6NDksIFNhbHZvIFRvbWFzZWxsaSB3cm90ZToNCj4gQnV0IGlmIGJ5 IG1pc3Rha2Ugb25lIHVwc3RyZWFtIGFkZHMgYSBwcm9wcmlldGFyeSBmaWxlIGluIGdpdCBh bmQgdGhlbiByZW1vdmVzDQo+IGl04oCmIHdlIGNhbid0IGRpc3RyaWJ1dGUgdGhlIHdob2xl IC5naXQgZGlyZWN0b3J5IGFzIGlzLg0KDQpUcnVlIOKAkyBidXQgd2hhdCB3ZSAqY2FuKiBk byBpcyB0byBhZGQgYSAiZGViaWFuL3NvdXJjZS9zdHJpcC1ibG9icyIgDQpmaWxlLCB0aGVu IHJ1biAiZ2l0IGZpbHRlci1yZXBvIC0tc3RyaXAtYmxvYnMtd2l0aC1pZHMgDQpkZWJpYW4v c291cmNlL3N0cmlwLWJsb2JzIiBvbiBhIGNsb25lIG9mIHRoZSB1cHN0cmVhbSByZXBvc2l0 b3J5IGJlZm9yZSANCm1lcmdpbmcgZnJvbSBpdC4NCg0KVGhpcyBzdGVwIGlzIGRldGVybWlu aXN0aWMsIGlkZW1wb3RlbnQsIGFuZCBlYXNpbHkgYXV0b21hdGVkIGlmIG5lY2Vzc2FyeS4N Cg0KLS0gDQotLSByZWdhcmRzDQotLSANCi0tIE1hdHRoaWFzIFVybGljaHMNCg0K

--------------aQAio1gpyT5FEehSlTobFMLR--

-----BEGIN PGP SIGNATURE-----

wsF5BAABCAAjFiEEr9eXgvO67AILKKGfcs+OXiW0wpMFAmZ3CIkFAwAAAAAACgkQcs+OXiW0wpMh iw//QjbDmCMcrPaod/w4CIX5QeP7LkPAnTdFrOJQCNGSq3EW/11xZ3YboUCkOFFMYe8ZL7a8mpo6 ZuMD8bGwSZGQ9JcTy1TSJSnD31mxKVyZ/cbPXbA+P5AWFgnJ/+NMuVyt1yHv+vvHpuWSgqcN4Ybr jt+W/Jh+tgGquh2CPEhF8X/OqLcmqbZn8o6JzpKdYfeS3k969iUThG48R62TBR8GCbPPlaC64rFm fF0KT1MfLxy+bbH5ZlMZWXCpLFnmIZBZJRe3hEixe2ziTlPUGeLBi+LHC0z4b7VavDQCI99hKiV3 sPiwBPPUO75oyRwmxhkFcSBrNk4atE4opEch75VBjAyOxqjRSK3+Xr5Fk9Hb+txfZbmE26VNvFct wa351/LGDZa7U8Rl1K1DqXusEumhuYx6Lh/uqb3AA1Elw4aFCmIT4/5cVVo3HNguBG+4bZePrKH7 7tf/Bhxca2ocxFbON//BMPBicsGfGFcZGYCBrorrritqpAq1S2XV4bCGf4bn40dQwJvH6XbrRDD/ jisZdWCHM/AYbvBqxUyeSqA3ai1Jftleg/1MItEi2AZMbp+18VPyOxrxmaTaUZcnWD/fwXabiN/o rxweZMEj4SZ26sfCxs4QtDCu8HAHrFccFcVXhHYSJbOk49gqKQQOWGpUYYXmniw4GodFSU6Z3N3P BjU=
=7n52
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)