• Bug#1109423: want dgit push-source --facilitate-xz-attack

    From Ian Jackson@21:1/5 to All on Thu Jul 17 15:00:02 2025
    Package: dgit
    Version: 13.5

    It has become clear to me in many corridor conversations that,
    workflows involving pristine-tar and upstream origs are really very
    common. Where upstream origs are not treesame to git (which is
    basically, whenever they were not made by git-archive):

    1. Existing non-git-first workflows (git-buildpackage) do not report
    the discrepancy. They treat the tarball as more authoritative.

    2. dgit push-source will fail. tag2upload will fail, even if
    we implement pristine-tar support.

    Of course no-one should be using these workflows, but we need to think
    whether we would rather somehow dismantle this barrier to dgit/t2u
    adoption. In practice I think the folks with the vulnerable workflow
    are going to keep with their vulnerable workflow anyway.

    The obvious way would be to reify the xz attack diff as an extra patch
    in d/patches, during git canonicalisation (as we do for .gitignore).

    Ian.

    --
    Ian Jackson <[email protected]> These opinions are my own.

    Pronouns: they/he. If I emailed you from @fyvzl.net or @evade.org.uk,
    that is a private address which bypasses my fierce spamfilter.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Sean Whitton@21:1/5 to Ian Jackson on Fri Jul 18 11:10:01 2025
    Hello,

    On Thu 17 Jul 2025 at 01:49pm +01, Ian Jackson wrote:

    Package: dgit
    Version: 13.5

    It has become clear to me in many corridor conversations that,
    workflows involving pristine-tar and upstream origs are really very
    common. Where upstream origs are not treesame to git (which is
    basically, whenever they were not made by git-archive):

    1. Existing non-git-first workflows (git-buildpackage) do not report
    the discrepancy. They treat the tarball as more authoritative.

    2. dgit push-source will fail. tag2upload will fail, even if
    we implement pristine-tar support.

    Of course no-one should be using these workflows, but we need to think whether we would rather somehow dismantle this barrier to dgit/t2u
    adoption. In practice I think the folks with the vulnerable workflow
    are going to keep with their vulnerable workflow anyway.

    The obvious way would be to reify the xz attack diff as an extra patch
    in d/patches, during git canonicalisation (as we do for .gitignore).

    Interesting. Doing it as an extra patch sounds good; it's easy to
    inspect what was done. The UI for turning this on for tag2upload seems
    tricky unless we only add one additional quilt mode which is like --gbp
    plus this. Otherwise we need another [dgit ...] element.

    Could it be on by default if the user is already using the foreseen pristine-tar support? I.e. we defer to their pristine-tar data.

    --
    Sean Whitton

    --=-=-Content-Type: application/pgp-signature; name="signature.asc"

    -----BEGIN PGP SIGNATURE-----

    iQJNBAEBCgA3FiEEm5FwB64DDjbk/CSLaVt65L8GYkAFAmh6DmoZHHNwd2hpdHRv bkBzcHdoaXR0b24ubmFtZQAKCRBpW3rkvwZiQP2oD/9l3TwWt7YXMtyxFZzGLMRT dc/hCPmRr/Wo/aGN66qYLi9LiNNAv7RhG0WwRwlgwUFugAc23lX32n8Mw0DSSiYs dhRUfrYHmMkSCoKWhOrMdTzuE90NZvF4+Zeno3cf0eRAU4NfxLAZzwDfgA8pZhao G6NZAw+0/4N+w2+3lWTYLlbHh7Ui1gtCRj2VjyNMjZNhjGfxg9uxazAzovgcROIj nApCL6KxjYQNj0yt7lvA4+f1l7nob6bM7Yfk0dumqS8AFFy9Kkoy6nf23r0W/mNh 48MRyab2EOlg5k08q3sIqu/OQ86ASh31AEoeGF8iFijIKaufkjCygLtlPQSiIsdF K/GdfYa5f9LI0feyC5ktoCEe9xYexlMkRkbGN0LnNBAsicWaPj9AzsZFYa5eM63n zx3fL81ftShSQWTGtBSj690qk9eUMMUWtR6UZx2IHbu+SDpyw3qR66BN7enr7gUJ /iZDOqlJKKxlT92oF+7bf1qVdtdPDH+CIsGnV8oKHn0cKiL8rrdmG/P9n+iA2vau q/W2AronvYmldsBEEFZzN7sNhqU+LlTjtje14trwWGB+gVBcTF6alh7bWPDLmvFJ wlBA1tuiEPyrLDBSIl8gt0ZEL+s0LV6ulPlQVAusxY/DRoK55amGCBir6raKB7om tN4p6MOB/UXyP74y8NvK7g==xxar
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Us
  • From Colin Watson@21:1/5 to All on Sun Jul 20 16:00:02 2025
    I made some comments about this in a tag2upload contributors BoF at
    DebConf which contained details new to Ian, and they asked me to write
    those details down here.

    While it's possible to use pristine-tar to commit deltas for tarballs
    that are not represented by any git commit, the typical workflows that I believe most Debian developers use take care to avoid that situation.
    The ones I'm familiar with are git-buildpackage and git-dpm. In both
    cases, the typical practice is to use "gbp import-orig" or "git-dpm import-new-upstream" respectively to import a new upstream tarball into
    git and merge it into the packaging branch. Among other things, these
    tools construct a commit on the upstream branch (which may be called
    "upstream" or some minor variation of that) with a tree whose contents
    are identical to those of the tarball. That commit may or may not have
    the corresponding upstream commit as an additional parent (via "gbp
    import-orig --upstream-vcs-tag" or "git-dpm import-new-upstream
    --parent". The commit corresponding to the upstream tarball is then
    merged onto the packaging branch in some way.

    My layperson understanding of pristine-tar is that "pristine-tar commit"
    (which may be called directly, or via "gbp import-orig --pristine-tar"
    or "git-dpm import-new-upstream --pristine-tar-commit") constructs a
    binary delta expressing the differences between its canonicalized
    compression of "git archive" and the target tarball, and commits that
    delta to a branch called "pristine-tar". If its input parameters
    include an upstream commit that doesn't correspond exactly to the target tarball, then it's true that you might end up with tree contents that
    doesn't really live anywhere else other than the delta. However, modulo
    bugs or weird edge cases (perhaps involving .gitattributes), this only
    happens if somebody has called "pristine-tar commit" directly on a
    mismatching commit; these higher-level tools won't do it.

    https://salsa.debian.org/auth-team/libfido2 and https://salsa.debian.org/debian/libpipeline are examples I'm familiar
    with from each of those tools. In each case you'll find an "upstream"
    branch that should be identical to the corresponding unpacked upstream
    tarball. In the libfido2 case, there's no true upstream git history and
    the "upstream" branch is just a sequence of tarball imports. In the libpipeline case, the "upstream" branch has the corresponding upstream
    commits as additional parents, so you can see everything clearly in git history: e.g. "git diff upstream^2 upstream" shows you the differences
    between upstream git and the tarball, while "git diff upstream~
    upstream" shows you the differences between successive upstream
    tarballs. While these packages use git-buildpackage and git-dpm
    respectively, you can use both modes (with or without an additional
    parent) with either tool.

    I hope this is helpful. Let me know if you need any extra help getting
    your heads around pristine-tar and the associated workflows; I'm not
    really an implementation expert, but I'm a proficient user and can
    probably help to bridge any remaining gaps.

    --
    Colin Watson (he/him) [[email protected]]

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Ian Jackson@21:1/5 to Colin Watson on Thu Jul 24 23:00:01 2025
    Control: severity -1 wishlist

    Colin Watson writes ("Bug#1109423: want dgit push-source --facilitate-xz-attack"):
    I made some comments about this in a tag2upload contributors BoF at
    DebConf which contained details new to Ian, and they asked me to write
    those details down here.

    Thanks. This is very helpful.

    While it's possible to use pristine-tar to commit deltas for tarballs
    that are not represented by any git commit, the typical workflows that I believe most Debian developers use take care to avoid that situation.

    I think that means that we can relax about this issue, and press on
    with pristine-tar support.

    Let's leave this bug open at wishlist, for now at least.

    Ian.

    --
    Ian Jackson <[email protected]> These opinions are my own.

    Pronouns: they/he. If I emailed you from @fyvzl.net or @evade.org.uk,
    that is a private address which bypasses my fierce spamfilter.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)