• Strange difference between bullseye and bookworm.

    From Tim Woodall@21:1/5 to All on Tue May 28 19:40:02 2024
    I start a new user namespace as follows:
    (The special bashrc is just because there are some things in my default
    one that (expectedly) don't work in the lxc user namespace)

    lxc-usernsexec -m b:0:689824:65536 -- /bin/bash --rcfile ~/.bashrc.lxc

    Inside there I mount a squash fs image that includes the normal tools
    for building packages

    squashfuse bookworm.amd64.build-deb.sqfs lower

    I then mount an overlayfs on top of that:
    fuse-overlayfs -o lowerdir=lower,upperdir=overlay,workdir=work mount

    I bind mount /dev/null into there
    cd mount
    touch dev/null
    mount -o bind /dev/null dev/null

    and then I chroot:
    /sbin/chroot .

    This all appears to be working perfectly on both bookworm and bullseye
    hosts.

    But in bookworm, apt-get update fails in a weird way:

    root@dirac:/# apt-get update
    Get:1 http://aptmirror.home.woodall.me.uk/local bookworm InRelease [18.9 kB] Get:2 http://deb.debian.org/debian bookworm InRelease [151 kB]
    Err:1 http://aptmirror.home.woodall.me.uk/local bookworm InRelease
    Couldn't execute /usr/bin/apt-key to check /var/lib/apt/lists/partial/aptmirror.home.woodall.me.uk_local_dists_bookworm_InRelease
    Get:3 http://deb.debian.org/debian-security bookworm-security InRelease [48.0 kB]
    Err:2 http://deb.debian.org/debian bookworm InRelease
    Couldn't execute /usr/bin/apt-key to check /var/lib/apt/lists/partial/deb.debian.org_debian_dists_bookworm_InRelease
    Err:3 http://deb.debian.org/debian-security bookworm-security InRelease
    Couldn't execute /usr/bin/apt-key to check /var/lib/apt/lists/partial/deb.debian.org_debian-security_dists_bookworm-security_InRelease

    Notice that "Couldn't execute /usr/bin/apt-key"

    Running exactly the same on a bullseye host and this "just works"

    Running:
    strace -f apt-get update |& less

    [pid 6619] execve("/usr/bin/apt-key", ["/usr/bin/apt-key", "--quiet", "--readonly", "verify", "--status-fd", "3", "/tmp/apt.sig.xWh7oI", "/tmp/apt.data.JpfP2n"], 0x5566c9baafc0 /* 48 vars */) = -1 EOPNOTSUPP (Operation not supported)

    This is my problem!

    If I unpack the squashfs image but otherwise follow the same steps (i.e.
    lower is a normal directory) then this works.

    When I compare other things I see this in the working one:
    [pid 6701] openat(AT_FDCWD, "/proc/self/fd", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = -1 ENOENT (No such file or directory)

    while I see this in the non-working one:
    [pid 6701] openat(AT_FDCWD, "/proc/self/fd", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = -1 EOPNOTSUPP (Operation not supported)

    ENOENT is expected as I don't have /proc mounted in the namespace.

    execve works for other tasks:
    [pid 6693] execve("/usr/bin/dpkg", ["/usr/bin/dpkg", "--print-foreign-architectures"], 0x7ffe96e5ffa0 /* 42 vars */) = 0
    works on both the bullseye and bookworm hosts, there's something special
    about apt-key. Weirdly, copying dpkg over apt-key and I still get this EOPNOTSUPP error. But deleting it completely and I get ENOENT from the
    execve call.

    Does anyone have any ideas what might be be wrong here, what I could try
    to get this working again?

    Tim.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Tim Woodall@21:1/5 to Tim Woodall on Tue May 28 20:30:02 2024
    On Tue, 28 May 2024, Tim Woodall wrote:

    I start a new user namespace as follows:
    (The special bashrc is just because there are some things in my default
    one that (expectedly) don't work in the lxc user namespace)

    <snip>
    I then mount an overlayfs on top of that:
    fuse-overlayfs -o lowerdir=lower,upperdir=overlay,workdir=work mount

    <snip>

    And it appears that fuse-overlayfs is the problem. Downgrading to the
    version from bullseye fixes:

    root@bookworm19:~# apt-get install fuse-overlayfs=1.4.0-1
    Reading package lists... Done
    Building dependency tree... Done
    Reading state information... Done
    The following packages will be DOWNGRADED:
    fuse-overlayfs


    The problem only seems to trigger when lower is a squashfs mount...

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Tim Woodall@21:1/5 to Tim Woodall on Wed May 29 19:50:01 2024
    On Tue, 28 May 2024, Tim Woodall wrote:

    On Tue, 28 May 2024, Tim Woodall wrote:

    I start a new user namespace as follows:
    (The special bashrc is just because there are some things in my default
    one that (expectedly) don't work in the lxc user namespace)

    <snip>
    I then mount an overlayfs on top of that:
    fuse-overlayfs -o lowerdir=lower,upperdir=overlay,workdir=work mount

    <snip>

    And it appears that fuse-overlayfs is the problem. Downgrading to the
    version from bullseye fixes:

    root@bookworm19:~# apt-get install fuse-overlayfs=1.4.0-1
    Reading package lists... Done
    Building dependency tree... Done
    Reading state information... Done
    The following packages will be DOWNGRADED:
    fuse-overlayfs


    The problem only seems to trigger when lower is a squashfs mount...


    And upgrading to the version in trixie (doesn't even need to be
    backported, the package installs as is) also fixes.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)