• Re: Having ten thousands of mount bind causes various processes to go i

    From Julien Petit@21:1/5 to All on Fri Jun 14 12:00:02 2024
    Best question probably is: what exactly are you needing 14.000 mounts for? Even snaps shouldn't be that ridiculous. So what's your use case? Maybe there's a better solution to what you are doing. If it's just about having a place that is rw only
    without execution permissions, just crate a separate partition, mount it somewhere - e.g. /home/test/mounts and tell mount/fstab to use the option noexec. No need for for your script. Or if it's a more advanced file system like btrfs you may be able to
    simply create a subvolume with the same capabilities, no need to tinker around with partitions.

    We use the mounts to share an initial folder with either rw or ro
    wrights in a user directory. The user directory is then accessible
    through a web interface, sftp, webdav and rsync. There is probably
    better ways to do that now but that's a legacy app (2009) that we'd
    rather leave alone :)

    It's true this issue should be looked into, but it doesn't look urgent as long as there are alternatives.

    Yes, not urgent and very specific. I'm going to try to set the mounts
    to private as Max suggested and see how it goes.
    Thanks for your help.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Julien Petit@21:1/5 to All on Wed Jun 19 11:10:02 2024
    Just to learn about it.
    What about using acl rather than bind mounts? What should be the
    problem in this solution?

    As i said to Richard, rights are not the challenge here. It's to be
    able to share a directory across multiple users. For instance you
    would have : /users/bob/accounting shared with Alice and accessible in
    her home directory /users/alice/accounting

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Julien Petit@21:1/5 to All on Wed Jun 19 11:50:01 2024
    Does it have some logic to avoid descending into bind mounts? Maybe I am wrong with my expectation that it does not use anything besides st_dev from stat result. It may be promising case to demonstrate the issue in a way independent of systemd and
    sandboxing. You can obtain command line arguments. Attach to its mount namespace and inspect content of its /proc/<PID>/mounts or mountinfo. The next step would be to profile or at least to trace a process.

    I'm not sure i understand you there.

    I have not figured out from your description what problem you solved by using bind mounts, but bublewrap (so flatpak and snap) and firejail relies on bind mounts as well. Perhaps you have some unique factors.

    Mounts are used as a way of sharing folders in different users' home directories.
    For instance you would have : /users/bob/accounting shared with Alice
    and accessible in her home directory /users/alice/accounting

    Thanks for your help :)

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Julien Petit@21:1/5 to All on Thu Jun 20 00:30:01 2024
    For this, probably the easiest is to set up a common directory/a few common directories, set up proper permissions through use of groups and worst case create some symlinks from the user's home directories, if these directories really need to be
    accessible from within their home directories. That's pretty much how shared directories are always done. As this would be a one time effort, it would be doable.

    You're thinking of a traditional file server in a business. Our
    solution is a cloud platform. We don't know ahead how our customers
    are going to manage their files and shares. And we don't need to.
    As i said to Eduardo, it doesn't really matter where folders/mounts
    are. Users can share any directory (and subdirectories) in their home
    directory with any other user. The shared folder is mounted in the
    special directory "Shared with me" of the recipient home directory.
    I.e: John/Sales/Invoices is mounted in Alice/Shared with me/Invoices.
    The shares can be read/write or read-only.

    But at this point, you should really think about paying some company with deep knowledge of Linux that can come up with a sustainable plan. Because obviously, your way of doing things isn't anything that could (or on that note should) be a long-term
    solution. And maybe think about rewriting the ancient software that causes this setup in the first place.

    We've come up with that solution in 2009 and it's been working until
    now (and still is but eating away cpus). So i guess it makes it a
    proven long-term solution ;) Is there a better way to do it now?
    Maybe. But not as easy as setting up a few symlinks and some
    permissions.

    Anyway, that's out of subject. My request is simpler than knowing how
    our solution works. Mounting many thousands folders wasn't an issue
    before and it is now (i haven't nailed on which Debian update it broke
    yet). As i said in my request, i know that this usage is an heavy
    usage of mounts but it worked perfectly for many years. I'm just
    trying to understand the cause behind it.

    Desperately trying to cling to something that has been out of support for decades is just not sustainable, not on any OS.

    We're the maintainers of our software so it's not out of support :)
    I'm here because we'd like to save a few trees reducing that cpu usage
    down :D
    Thanks again for your time!

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Eduardo M KALINOWSKI@21:1/5 to Julien Petit on Thu Jun 20 00:40:02 2024
    On 19/06/2024 19:06, Julien Petit wrote:
    It doesn't really matter where folders/mounts are. Users can share any directory (and subdirectories) in their home directory with any other
    user. The shared folder is mounted in the special directory "Shared
    with me" of the recipient home directory. I.e: John/Sales/Invoices is
    mounted in Alice/Shared with me/Invoices.

    Can be done with symlinks.

    I imagine there's some process that creates these bind mounts, so the
    process could create symlinks.

    Symlinks are no good since the user sharing his directory can decide
    to share it read/write to one user but read only to another

    This can be solved with ACLs. Instead of creating a bind mount, this
    process that allows the user to share the directory can set an ACL and
    create a symlink.

    PS: It would be better if you used a mailer that correctly sets mail
    headers References and/or In-Reply-To so that your replies are properly threaded.


    --
    Go placidly amid the noise and waste, and remember what value there may
    be in owning a piece thereof.
    -- National Lampoon, "Deteriorata"

    Eduardo M KALINOWSKI
    [email protected]

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Julien Petit@21:1/5 to All on Fri Jun 21 03:40:02 2024
    PS: if you maintain your own software and aren't able to find a way for your user to do shares - especially while systems that most likely have such functionality built-in out of the box surely exist, think Nextcloud etc - that is covered by how Linux
    is supposed to be used, by definition it's pretty much out of support.

    Nextcloud doesn't offer sftp or rsync access to users that i know of.
    The specifications are much simpler because they only deal with web
    access (the web interface and the webdav server written in PHP).

    How Linux is supposed to be used? That's why i'm here. There wasn't
    until kernel 4.19 an official limit to the number of mounts in the documentation. Even though we use mounts a lot, we're still far from
    the official limit. Did we get lucky for 15 years and we should change
    the way we do things or is it a bug ? I will now take this to the
    kernel team and see what they have to say about it.

    Especially if you keep insisting on using a way that was never officially supported, just because you got away with it for 15 years.

    That's the very question i guess! How much mount is too much mount ;)

    Thanks again for your help.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From [email protected]@21:1/5 to Julien Petit on Fri Jun 21 12:30:01 2024
    Julien Petit <[email protected]> wrote:
    How Linux is supposed to be used? That's why i'm here. There wasn't
    until kernel 4.19 an official limit to the number of mounts in the documentation. Even though we use mounts a lot, we're still far from
    the official limit. Did we get lucky for 15 years and we should change
    the way we do things or is it a bug ? I will now take this to the
    kernel team and see what they have to say about it.

    I take it you have read
    https://docs.kernel.org/filesystems/sharedsubtree.html which says "A
    shared mount can be replicated to as many mountpoints and all the
    replicas continue to be exactly same" and seems to be trying to say
    your use case is valid.

    I'd be interested to follow your discussion with the kernel devs if you
    could post a link.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)