I'm sorry Ian, but I couldn't second such a GR. I know you've worked on this for 6+ years and it's finished baking and ready for prime time, and that you're frustrated that it's not deployed yet, but I can't second something that puts more pressure or rush on the ftpmaster team. They have a (really freaking huge) responsibility to keep uploads / publishing of packages safe, if anything, I would want them to be as thoughtful and methodological as humanly possible when it comes to making changes, and I while I don't expect perfection from anyone, I would want them at the very least to be somewhat confident of what is implemented and how it works before it goes live.
Also, we are 11 days away from soft freeze starting, and while the NEW queue has gotten shorter (which is great), imho that should really get priority over new implementations so that DDs and other contributors can get a chance to finish their goals for the Trixie release.
REJECTED ALTERNATIVE - ADDING THE KEY TO THE DD KEYRING
One approach that would let tag2upload function correctly, would be
adding the tag2upload service key to debian-keyring.gpg, as if it were
a human uploading DD.
On 2025/04/04 11:08, Ian Jackson wrote:
ftpmaster don't want to see tag2upload in use, and so they are
choosing not to respond to our requests to install the key.
Therefore, we need to temporarily delegate someone else to do it.
Wait, what? I thought that things were going well and that ftpmaster supported the project and that things were mostly falling in place?
Just to clarify, I do believe the Gregorian year is now 2025.
Are the dates / times you describe here accurate? It seems that you are accusing DSA of ignoring your messages since the beginning of April
2025, which we're 4 days in right now. Even 14th of March was just about
3 weeks ago?
Also, we are 11 days away from soft freeze starting, and while the NEW
queue has gotten shorter (which is great), imho that should really get priority over new implementations so that DDs and other contributors can
get a chance to finish their goals for the Trixie release.
How about adding the tag2upload keys to a NEW keyring instead? https://salsa.debian.org/debian-keyring/keyring/
Better is to have dak accept two keyrings with identical authority: debian-keyring.gpg, and a service keyring debian-tag2upload.gpg
containing the tag2upload key.
By including it in the official debian keyring package, we get some
historic accountability of which keys were used. You also get a way
to phase in new keys and phase out old keys.
Hi Ian
On 2025/04/04 11:08, Ian Jackson wrote:
Hi everyone.
tl;dr
We need to make an exceptional, short term delegation
authorising
the installation of tag2upload's signing key on ftp-master.
We need two volunteers to take on this responsibility; please
volunteer in response to this message if you can do it.
Oh, that sounds like a quick and simple procedural admin task, sure,
I'd be happy to help!
INTRODUCTION
Thanks to a lot of help from DSA, tag2upload is deployed, all
according to spec! We are very excited about this.
However, we can't start our closed beta because the ftpmaster team
will
not authorise tag2upload's signing key to upload packages.
We asked them to install the key on 2025-03-14. After some
resistance, they told us they could make their changes and install the
key by the end of March. We've offered technical information, and
asked how it's been going, and in particular when April arrived we
asked about agreeing a new timeline. However, they have been ignoring
our messages for weeks, and there has been no update, and no sign of
any activity.
Are the dates / times you describe here accurate? It seems that you
are accusing DSA of ignoring [snip]
Here is a detailed timeline of the messages in the lasst month:
2025-03-14 We formally provide the production key
and ask for it to be isntalled.
2025-03-15 Ansgar sends us an obnoxious reply.
2025-03-15 Ian forwards Ansgar's reply to the Community Team.
2025-03-15 Joerg tells us "no".
2025-03-16 Sean reminds Joerg of our agreement and reconfirms
that everything is as agreed, and clarifies a few things. 2025-03-16 Joerg complains (2 emails) that we didn't email them earlier
about the work they said they wanted to do.
2025-03-17 Sean graciously apologises for not emailing ftpmaster earlier,
and asks for a time estimate.
2025-03-17 Ian asks for immediate installation since ftpmaster not
doing work that only they want done, should not be a blocker. 2025-03-19 Sean pings Joerg asking for a response.
2025-03-19 Joerg replies offering to do the work by the end of March. 2025-03-20 Sean agrees to this as a compromise, and restates technical info. 2025-03-20 Ian provides additional technical info and offers of support. 2025-03-26 Sean pings, asking "how is it going".
2025-03-26 Community Team apologises for not replying yet.
2025-04-01 Sean pings, asking for a new timeline.
2025-04-01: ansgar asks me to install dgit on fasolo to facilitate the integration. While complicated by the fact that fasolo is not upgraded to stable yet, it's installed the same morning (but not instantly).
Standing on the other side of the fence of another team that nearly
avoided an escalation: Given the amount of tickets in the DSA queue, I understand the feeling that people think their wishes are not acted upon quickly enough by their fellow volunteers. Which feels like gatekeeping,
but is also owed to some degree to the interests and time availability
of a small set of people - where for various reasons the work is not
easily shardable to the project.
A PR for dak would probably have gone a long way. I understand the complexities with that, but to be fair, that was basically the point ftp-master was making in response.
(...)
As you write:
By including it in the official debian keyring package, we get some
historic accountability of which keys were used. You also get a way
to phase in new keys and phase out old keys.
keyring-maint, would you welcome an MR for this?
Ian Jackson <[email protected]> writes:
REJECTED ALTERNATIVE - ADDING THE KEY TO THE DD KEYRING
One approach that would let tag2upload function correctly, would be
adding the tag2upload service key to debian-keyring.gpg, as if it were
a human uploading DD.
How about adding the tag2upload keys to a NEW keyring instead?
https://salsa.debian.org/debian-keyring/keyring/
I think tag2upload is a different enough mechanism that it could warrant
an entire new class of keyring. By including it in the official debian keyring package, we get some historic accountability of which keys were
used. You also get a way to phase in new keys and phase out old keys.
I suggest not putting it in the "debian-role-keys" but instead a new "debian-tag2upload-keys" keyring. It seems like a sensible
implementation approach to accept any key in that keyring as being a "tag2upload" key, rather than hard-coding particular key fingerprints in various configuration files or scripts.
/Simon
We see ourselves as an operational team, but not as a decision-making team, except when it comes to determining i.e. a given category of keys is no longer trustable (as we did back in 2014). Thus, we will be happy to add
what would amount to a role key, or a fourth active keyring, following the instructions given by the relevant delegates (that would most certainly be the DSA and/or ftpmaster teams).
We see ourselves as an operational team, but not as a decision-making team, except when it comes to determining i.e. a given category of keys is no longer trustable (as we did back in 2014). Thus, we will be happy to add
what would amount to a role key, or a fourth active keyring, following the instructions given by the relevant delegates ...
Hi everyone.
tl;dr
We need to make an exceptional, short term delegation authorising
the installation of tag2upload's signing key on ftp-master.
We need two volunteers to take on this responsibility; please
volunteer in response to this message if you can do it.
INTRODUCTION
Thanks to a lot of help from DSA, tag2upload is deployed, all
according to spec! We are very excited about this.
However, we can't start our closed beta because the ftpmaster team will
not authorise tag2upload's signing key to upload packages.
We asked them to install the key on 2025-03-14. After some
resistance, they told us they could make their changes and install the
key by the end of March. We've offered technical information, and
asked how it's been going, and in particular when April arrived we
asked about agreeing a new timeline. However, they have been ignoring
our messages for weeks, and there has been no update, and no sign of
any activity.
The tag2upload developers have a DPL delegation which explicitly
grants us a signing key with certain upload privileges. Therefore,
the ftpmaster team does not have any authority to block us, in just
the same way that they can't tell the Release Team that they can't
make a point release. It's just not for them to say.
They could, of course, refuse to offer the Release Team any assistance
in making a point release actually happen. This is Constitution
2.1(1), the volunteer principle.
ftpmaster don't want to see tag2upload in use, and so they are
choosing not to respond to our requests to install the key.
Therefore, we need to temporarily delegate someone else to do it.
TEMPORARILY DELEGATE?
We are all used to delegations as ongoing and not time-limited, but
this is not their only purpose in our Constitution.
Just as an NMU is how we fix RC bugs or implement TC decisions when
the maintainer won't do it, time-limited delegations are the
equivalent mechanism the Constitution provides for similar situations
outside of the archive itself.
WHAT ABOUT THE DPL?
The DPL has been CC'd on all of the private messages between the
tag2upload developers and the FTP team. We have seen *no response whatsoever* from him.
Given this, and other interactions we have had or are aware of, we do
not have confidence in the DPL's willingness to take the necessary
action.
THE 2024 AGREEMENT WITH FTPMASTER
Q. Didn't you come to an agreement with ftpmaster last year?
We did. It was explicitly signed off by both sides, here:
https://browse.dgit.debian.org/dgit.git/commit/?id=e5512e874ddd755e2168b34d1b95f5f3ee487e71
https://lists.debian.org/debian-vote/2024/07/msg00024.html
That agreement involved us doing substantial additional work to
support additional checks by dak (that no other core team thought worthwhile). It also envisaged ftpmaster making changes to dak, to
perform those additional checks.
We have spent the past eight months implementing our side of the
arrangement. They have done nothing, and are still doing nothing.
In other words, they are in breach of our agreement.
FTPMASTER'S POSITION
We don't know what ftpmaster think should happen next since they
haven't said. There is no indication that ftpmaster will start the implementation work, nor that they are prepared to proceed without it.
They are stonewalling us.
DRAFT GENERAL RESOLUTION
Once we have some volunteers:
We exercise the power of the DPL (via Debian constitution 4.1(3)),
to make a Delegation (8.2). We hereby delegate the
tag2upload Key Installation Task
to the following Task Delegates:
- Name
- Name
Task description
----------------
1. Install the tag2upload package signing key on fasolo.
(i) in such a way that dak will treat it identically
to a key belonging to a normal uploading DD;
(ii) or some other similar authority or abilities as
is consistent with the tag2upload service's needs,
and seems convenient to the Task Delegates;
(iii) keeping all changes as minimal as possible.
2. Collaborate with the tag2upload Delegates.
3. Collaborate with the FTP Master Delegates, if they express
an interest, but without introducing any significant delay.
Final decisions lie with the Task Delegates.
4. Confirm with the tag2upload Delegates that things are working.
Resolve any problems (with the key, or with other aspects of dak).
5. Document what was done by email to the FTP Master Delegates,
and/or in git, as the Task Delegates consider reasonable.
6. When completed, or if significant obstacles are encountered,
report to the debian-project mailing list.
The Task Delegates should be granted by DSA whatever permissions are
necessary to accomplish the task.
This is a new delegation. It is limited to the duration of the
task, or until withdrawn by present or future Project Leaders.
REJECTED ALTERNATIVE - ADDING THE KEY TO THE DD KEYRING
One approach that would let tag2upload function correctly, would be
adding the tag2upload service key to debian-keyring.gpg, as if it were
a human uploading DD.
This is an alternative way to work around ftpmaster's unwillingness.
One of them even mentioned it in one message, though it wasn't clear
how serious they were.
But this is a really unpleasant workaround. We would need to audit
elections to check that the tag2upload key hadn't "voted". We would
need to remember to subtract one every time we were calculating
quorum. The keyring maintainers don't like the idea, and we don't
either.
Better is to have dak accept two keyrings with identical authority: debian-keyring.gpg, and a service keyring debian-tag2upload.gpg
containing the tag2upload key. This seems to us like it would be easy
(and lowest risk, given that this task may be performed by someone
with limited knowledge of the codebase). So that is what we propose.
Q. LET'S JUST GIVE FTPMASTER SOME MORE TIME
ftpmaster have already had 8 months since our agreement last year, and
did nothing during that time. When their inaction became the final
blocker, they initially refused, then pleaded for more time (which we accepted), then missed their own deadline, then went back to
stonewalling when we asked for a new timeline.
We do not expect that any new deadline would be met. Setting a new
deadline would simply delay matters, and then leave us back where we
started. More generally, it is not the tag2upload developers' job to project-manage ftpmaster's implementation work.
Ian
for the tag2upload Delegates
--
Ian Jackson <[email protected]> These opinions are my own.
Pronouns: they/he. If I emailed you from @fyvzl.net or @evade.org.uk,
that is a private address which bypasses my fierce spamfilter.
We all know Debian is [dying], right?
[dying]: https://salsa.debian.org/rafael/debian-contrib-years
We will prepare an MR
Gunnar Wolf writes ("Re: Call for volunteers and GR draft: tag2upload key installation [and 1 more messages]"):
We see ourselves as an operational team, but not as a decision-making team, >> except when it comes to determining i.e. a given category of keys is no
longer trustable (as we did back in 2014). Thus, we will be happy to add
what would amount to a role key, or a fourth active keyring, following the >> instructions given by the relevant delegates ...
Right.
Management of this key is currently shared between DSA and the
tag2upload team. I was the person who instructed the hardware token
to generate it, so the key bears my signature. (See Sean's reply.)
In any case it doesn't seem to be controversial that this key ought to
be properly published in the debian-keyring package.
I think it's clear that it ought to be its own keyring file.
Automated systems need to verify with it, so if it were in with the
other role keys there would have to be some kind of separate
name-based or fingerprint-based access control as well, which would be needless complication and opportunity for error.
As it happens we (the tag2upload team) have a need for this public key
on another system - the dgit-repos git server. Right now we've done
that ad-hoc, but I think doing it via debian-keyring is much better.
I think Sean will agree.
I think debian-keyring would probably also be a convenient way for dak
to get this public key, but of course that is up to the ftpmasters.
We will prepare an MR, with more details about the key's provenance
etc. in the MR discussion comment. If ftpmaster have an opinion about
this aspect, I think it would be OK to ask them to make it known
there.
Thanks for your reply.
Wouldn't the relevant delegates be Ian and I? Our delegation says:
Maintaining and managing the tag2upload system design and security
policy, including for the tag2upload services's archive signing key.
(This key has equivalent permissions for source-only uploads to the
Debian archive as does an uploading Debian Developer's upload
signing key.)
[much that I agree with snipped]
# FTP & tag2upload
Ian. The project is bleeding and you're telling me ... we've had the
bandaid ... in hand ... and could have applied it ... four years ago!?
Ian, you have my full support in doing what is necessary to stop the
bleeding and I'll be happy to volunteer to help with the dak hack job if
that still makes sense with Simon's keyring idea on the table.
the five (at least, right?) different flavors of git packaging repo
layouts
The reason there are as many as four there is that we're maintaining a downstream patch queue, and maintaining and updating and sharing a
such a downstream patch queue, with git, is an open research problem.
Again, I'm not giving a team opinion, but just ⅓ of it: I don't think we should take any action until the infrastructural issue of how this will be handled in the archive side is settled.
The months before a stable release pose a really high pressure on
ftpmasters workload,
nobody wants to break dak when the freeze is approaching
I know after a lot of development, watching the result of the job
not being live due to missing work on somebody else hands is...
sad...
I only joined Debian in 2024 and since then one thing has become abundantly clear to me: it's dying because potential contributors are put off by our archaic, fragmented and most importantly not natively git-based workflow(s!).
What they will do, very readily and with much gusto is hop on GH to file an upstream bug or shoot a patch — no problem.
Not just of t2u, but the incremental improvements it enables. Their
inaction has cost us four years of new, young, motivated people joining the project and actually improving things alongside us.
We aren't waiting because ftpmaster haven't done the work agreed
in 2024, that only they want. That work is *not necessary*.
We are waiting because the DPL won't authorise an "NMU", despite
- the team involved being well-known as dysfunctional for many years
- the team's explicit position being that they don't want tag2upload
- the team having failed to deliver on its promises
- a complete lack of communication from that team
We are waiting because the DPL won't authorise an "NMU", despite
- the team involved being well-known as dysfunctional for many years
- the team's explicit position being that they don't want tag2upload
- the team having failed to deliver on its promises
- a complete lack of communication from that team
The second point is a lie, and you know it. [...]
Git is not a packaging tool. There is no way to have a "native"
git-based packaging workflow.
2. use 'apt build-dep x' to install the build dependencies locally
3. build the package with 'dpkg-buildpackage -b -us -uc' to see that it
works
4. use 'dch -i' to create a new changelog entry
5. make local changes
Even "how you create a changelog entry" is not uniform, much less
"how to register a patch" or "how to rebase patches for a new
upstream version."
The git-based workflows are a great collaboration tool, but to use
them you already need to understand the Debian packaging tools and
git, then on top there is also the policy on how branches inside the repository are used.
You and your behaviours effects are worse than those from many of the
people we had to expell in the past.
We need to make an exceptional, short term delegation authorising
the installation of tag2upload's signing key on ftp-master.
Am 7. April 2025 16:17:27 GMT+05:30 schrieb Ian Jackson <[email protected]>:
But I have my doubts that Debian Developers will find the technical
wording of the draft GR digestible.
It still isn't better digestable to me.
1. tag2upload allows DDs and DMs to upload simply by using the
git-debpush(1) script to push a signed git tag.
2. tag2upload has been fully implemented and deployed, and ready
for immediate operation, since the 15th of March 2025.
3. However, the Debian FTP Archive does not trust its signing key,
so tag2upload cannot be put into service.
In my eyes 2. can only be true if 3. is true. So, what version of
dak contains the code changes to trust tag2upload's signing key?
When did it get backported to oldstable for use on
fasolo.debian.org? I haven't seen it in the oldbackports-new queue
yet.
I was under the impression a (NMU for a) dak code changes doesn't
need that. Or, what am I missing?
Hi Ian, Hi all,
We all know Debian is [dying], right?
Frankly, which other recourse does a DD have, in your opinion, if they think a team is obstructing what they see as a valuable contribution to Debian and the DPL (who arguably does have the authority to kick the whole FTP team out and appoint somebody else) doesn't want to put his foot down?
We are waiting because the DPL won't authorise an "NMU", despite
[the TC] seems] to be way more collaborative to me, than - once
again - immediatly escalating this to -vote and threatening a GR.
The Technical Committee may:
Decide on any matter of technical policy.
Decide any technical matter where Developers' jurisdictions overlap.
Make a decision when asked to do so.
Overrule a Developer (requires a 3:1 majority).
Offer advice.
They all seem to be way more collaborative to me, than - once again - immediatly escalating this to -vote and threatening a GR.
*Also*, because some of us would like to focus on getting this trixie
release out.
Hi Ian, Hi all,
We all know Debian is [dying], right?
[dying]: https://salsa.debian.org/rafael/debian-contrib-years
Observation 1:
Young developers don't see Debian as a place where work gets done.
Observation 2:
Young developers are not lacking motivation to work on FLOSS.
Observation 3:
Young developers can't access the help they need to be effective.
You and your behaviours effects are worse than those from many of the
people we had to expell in the past.
Joerg,
Since you are (also) DAM, this constitutes a direct threat which is a violation of the CoC.
Cheers,
On 09.04.25 03:44, Sean Whitton wrote:
But the DPL didn't respond to a single message in the long private
thread, despite being CCed on every message.
The question is, did you *ask* him to? Explicitly?
Hi Pierre-Elliott,
At 2025-04-09T11:00:37+0200, Pierre-Elliott Bécue wrote:
Bill Allombert <[email protected]> wrote on 07/04/2025 at 10:02:01+0200: >> >> You and your behaviours effects are worse than those from many of
the people we had to expell in the past.
Joerg,
Since you are (also) DAM, this constitutes a direct threat which is
a violation of the CoC.
It seems like some people's passion is to find the biggest can of oil
and spread it on fire.
It's not clear to me whether you're addressing Joerg here, Bill, or
both. Unfortunately even in a universe with only two elements, the word "some" does not enable us to construct a well-defined set.
"These", for example, would have been unambiguous.
Please be specific when criticizing multiple people's emails in the
future.
Someone writes:
[the TC] seems] to be way more collaborative to me, than - once
again - immediatly escalating this to -vote and threatening a GR.
Dear Community Team:
I am getting very fed up of the repeated accusations, by multiple
people, that we "immediately escalated" this to -vote.
As we stated clearly in our thread starter message, and have now
explicitly documented in detail, we tried very hard in private emails.
We tried asking nicely. We tried negotiating. We tried the DPL.
Of course the previous attempts were, yes, private emails.
Escalating to an unstructured moan on a public list would not have
been sensible. The draft GR proposal is our last resort, but *must*
be public. So that is why the first *public* thing is the GR
proposal.
Therefore these allegations are a gross misrepresentation of the
facts. Indeed they are made in disregard of the facts.
That these kind of things are being said by multiple people is no
excuse. Indeed, it makes the situation worse.
People are entitled to disagree with us about timescales,
but hyperbolic accusations like "immediately" are unacceptable.
Hello,
On Tue 08 Apr 2025 at 01:09pm GMT, Holger Levsen wrote:
The Technical Committee may:
Decide on any matter of technical policy.
Decide any technical matter where Developers' jurisdictions overlap.
Make a decision when asked to do so.
Overrule a Developer (requires a 3:1 majority).
Offer advice.
They all seem to be way more collaborative to me, than - once again - immediatly escalating this to -vote and threatening a GR.
This is not fair.
Firstly, the TC cannot overrule delegates. This is well established.
During my recently-completed TC term, there was no doubt within the
committee that we could only deal with disputes among package maintainers/contributors, not anything involving DPL delegates.
Le Wed, Apr 09, 2025 at 09:44:31AM +0800, Sean Whitton a �crit :
Firstly, the TC cannot overrule delegates. [...]
Without contesting this point, I suggest that the TC could still provide
a non-binding neutral opinion on a technical issue when requested.
When did it get backported to oldstable for use on fasolo.debian.org?
I haven't seen it in the oldbackports-new queue yet.
On Mon 07 Apr 2025 at 05:23pm +0530, Micha Lenk wrote:
When did it get backported to oldstable for use on fasolo.debian.org?
I haven't seen it in the oldbackports-new queue yet.
src:debian-tag2upload-keyring was unblocked by the release team,
migrated to trixie today, and I just uploaded it to backports-NEW.
Sean Whitton writes ("Re: Call for volunteers and GR draft:
tag2upload key installation"):
On Mon 07 Apr 2025 at 05:23pm +0530, Micha Lenk wrote:
When did it get backported to oldstable for use on
fasolo.debian.org?
I haven't seen it in the oldbackports-new queue yet.
src:debian-tag2upload-keyring was unblocked by the release
team,
migrated to trixie today, and I just uploaded it to
backports-NEW.
I wasn't aware that the intent was to provide this key to dak
via this
.deb. Our intent with the package was to provide it to programs
like
dscverify on end-user systems. But, OK.
Note that debian-tag2upload-keyring_1.1_all.deb from trixie is
directly installable on buster. So if DSA is able to organise
it,
that might be a simpler approach than going via oldbackports.
[Ian Jackson writes:]...
[Micha Lenk wrote:]
When did it get backported to oldstable for use on
fasolo.debian.org?
I haven't seen it in the oldbackports-new queue yet.
I wasn't aware that the intent was to provide this key to dak via
this .deb. Our intent with the package was to provide it to
programs like dscverify on end-user systems. But, OK.
No, not ok.
We already have a keyring distribution mechanism for...
keyrings that live on keyring.d.o. Please use that one.
I believe the setup for that syncing is managed by keyring-maint,
but I could be wrong about that.
Installing packages from backports from the wrong distribution
(relative to what the host is running) will not fill us with joy.
There's no way to get a package into oldstable-backports now.
| Sysop: | Keyop |
|---|---|
| Location: | Huddersfield, West Yorkshire, UK |
| Users: | 715 |
| Nodes: | 16 (2 / 14) |
| Uptime: | 24:03:25 |
| Calls: | 12,105 |
| Calls today: | 5 |
| Files: | 15,006 |
| Messages: | 6,518,155 |