• Bug#267142: debian-policy: Sections 10.4 and 6.1 are inconsistent (Posi

    From Wouter Verhelst@1:229/2 to Thomas Bushnell BSG on Sat Aug 21 02:00:22 2004
    XPost: linux.debian.policy
    From: [email protected]

    On Fri, Aug 20, 2004 at 01:28:00PM -0700, Thomas Bushnell BSG wrote:
    I believe there are four reasonable sorts of policy amendment that
    would solve this problem:

    OPTION 1: Change 10.4 to require that scripts never use /bin/sh.

    OPTION 2: Restrict /bin/sh to a specified list of shells, rather
    than "any POSIX compatible shell", and require that shell scripts
    run correctly on that list.

    OPTION 3: Extend 10.4 to require the use only of Posix features not
    only for the shell, but for a specified list of other commands; that
    list would be based upon which commands are in fact being built-in
    by shells like dash in ways which are inconsistent with the binaries
    in /usr/bin.

    OPTION 4: Change section 10.4 to require the use of full pathnames
    for everything which is not ordered by Posix to be builtin to the
    shell; remove the "no pathnames on commands" direction in section
    6.1.

    There's also,

    OPTION 5: Change section 10.4 to require the use of full pathnames when non-Posix extensions are being used in commands; remove the "no
    pathnames on commands" direction in section 6.1, but do discourage its
    use whenever possible.

    I strongly prefer option 2. Options 1 and 3 seem ok to me, with a
    slight preference for option 1 because it is more maintainable.
    Option 4 is horrible, and you don't need me to say why.

    Indeed. However, I think option 5 is, at least, worth considering, even
    if it's hard to implement. Your thought?

    --
    EARTH
    smog | bricks
    AIR -- mud -- FIRE
    soda water | tequila
    WATER
    -- with thanks to fortune


    --
    To UNSUBSCRIBE, email to [email protected]
    with a subject of "unsubscribe". Trouble? Contact [email protected]

    --- SoupGate-Win32 v1.05
    * Origin: you cannot sedate... all the things you hate (1:229/2)
  • From Wouter Verhelst@1:229/2 to Thomas Bushnell BSG on Sat Aug 21 02:30:13 2004
    XPost: linux.debian.policy
    From: [email protected]

    On Fri, Aug 20, 2004 at 04:41:18PM -0700, Thomas Bushnell BSG wrote:
    Wouter Verhelst <[email protected]> writes:

    There's also,

    OPTION 5: Change section 10.4 to require the use of full pathnames when non-Posix extensions are being used in commands; remove the "no
    pathnames on commands" direction in section 6.1, but do discourage its
    use whenever possible.

    Under one interpretation, this option doesn't work; under the other,
    it is as bad as option 4.

    First, if you mean "when non-Posix extensions are being used in Posix-specified commands" (like "test"), then it doesn't work, because
    a shell might well override "debconf" with a builtin, as I described.

    Well, yeah, theoretically a shell could do that. However, in this
    specific example, I would consider such a shell to be horribly broken. 'debconf' is a name which is clearly chosen to avoid namespace conflicts
    such as the ones you're suggesting; and seen the fact that debconf is
    quite well known right now, I consider it extremely unlikely for that to
    be implemented in such a way.

    Of course, that's not what you meant; but I'd think that other examples
    would need to be examined on a case-by-case basis, and dealt with as
    they occur, using a fair dosis of common sense.

    The point being that, yeah, this is a possibility, but one that isn't
    too likely. Given a) that Policy is just a guideline on how to perform
    computer implementations, not one of those implementations itself; and
    b) the absence of any proof to the contrary, I don't think these
    problems are so likely that we need to worry about them right now. Also,
    even if it's allowed, I don't think it's likely for a shell to implement
    a builtin which is not specified by Posix (but I don't know all shells
    out there and haven't read a Posix standard, ever, so I could be wrong)

    Worst case, we could use wording which would make it clear that in the
    case such a conflict ever appears, scripts breaking because of that
    conflict should prepend a fixed path to the problematic command (or stop
    using /bin/sh).

    In other words, I suggest to deal with the things that are more likely
    to happen, and to just ignore (but of course not dismiss) the less
    likely problems.

    But if you mean "when non-Posix extension are being used, or non-Posix commands" then (because almost everything is non-Posix) we end up with
    all the brittleness that section 6.1 is trying to avoid,

    No, that's the wrong interpretation ;-)

    and we lose all the performance benefits of shell builtins (when those builtins are well-behaved).

    Indeed; however, if performance is an issue, then using /bin/sh is not
    a real option anyway.

    --
    EARTH
    smog | bricks
    AIR -- mud -- FIRE
    soda water | tequila
    WATER
    -- with thanks to fortune


    --
    To UNSUBSCRIBE, email to [email protected]
    with a subject of "unsubscribe". Trouble? Contact [email protected]

    --- SoupGate-Win32 v1.05
    * Origin: you cannot sedate... all the things you hate (1:229/2)
  • From Thomas Bushnell BSG@1:229/2 to Wouter Verhelst on Sat Aug 21 02:40:08 2004
    XPost: linux.debian.policy
    From: [email protected]

    Wouter Verhelst <[email protected]> writes:

    Well, yeah, theoretically a shell could do that. However, in this
    specific example, I would consider such a shell to be horribly broken. 'debconf' is a name which is clearly chosen to avoid namespace conflicts
    such as the ones you're suggesting; and seen the fact that debconf is
    quite well known right now, I consider it extremely unlikely for that to
    be implemented in such a way.

    But this is exactly the problem. A shell might well do what dash
    does, and override a non-Posix program, but in a minimalizing way
    (perhaps, in the assumption that people only need the minimum thing).

    Of course, that's not what you meant; but I'd think that other examples
    would need to be examined on a case-by-case basis, and dealt with as
    they occur, using a fair dosis of common sense.

    Why not just fix the problem entirely? That's what I meant by saying
    that this option 5 doesn't work: it doesn't actually fix the problem.

    We could always add a new thing that said "stick to Posix test, too",
    and that would solve *this* case, but do we really want to have to do
    this piecemeal?

    By contrast, listing a bunch of shells directly means that policy
    doesn't have to get into the nitty gritty--which can only be a good
    thing!



    --
    To UNSUBSCRIBE, email to [email protected]
    with a subject of "unsubscribe". Trouble? Contact [email protected]

    --- SoupGate-Win32 v1.05
    * Origin: you cannot sedate... all the things you hate (1:229/2)
  • From Thomas Bushnell BSG@1:229/2 to All on Fri Aug 20 23:20:15 2004
    XPost: linux.debian.policy
    From: [email protected]

    Package: debian-policy
    Version: 3.6.1.1
    Severity: normal

    This arose out of David Weinehall's search for the use of non-Posix
    features in package scripts. See bug #260092 for an example, and
    #264176 for some of his discussion with me.


    Policy 10.4 reminds that "The standard shell interpreter /bin/sh can
    be a symbolic link to any POSIX compatible shell....Thus, shell
    scripts specifying /bin/sh as interpreter should only use POSIX
    features."

    And we will note section 6.1, which says: "Programs called from
    maintainer scripts should not normally have a path prepended to them."

    If you look at bugs #260092 and #264176, you'll see that the complaint
    is that the script is using -o and -a as arguments to "test". It is
    true that -o and -a are not specified by Posix "test". But--and this
    is absolutely crucial--Posix "test" is not part of the Posix shell specification. As far as Posix is concerned, "test" is simply one
    more program.

    Because of policy 6.1, we are not supposed to write /usr/bin/test.

    What is going on here is that Posix allows a conformant shell to
    implement programs as builtins. ANY program. ANY program WHATSOEVER!
    Ain't that a kicker? The assumption of Posix is that the shell and
    the utilities are being written and distributed by the same people,
    and that the shell, when it builds something in, will build it in in a
    way which matches the behavior of the utility. But this assumption is
    (oddly!) nowhere specified by Posix.

    If you install "dash" as /bin/sh on a Debian system, then you break
    this assumption.

    From the Posix perspective, there are two kinds of utilities: those
    specified to exist by Posix, and those which are extra. A
    Posix-conformant shell is allowed to buildin *either* of those. (And
    shells liberally take advantage of this, of course.) Posix assumes,
    but does not require, that you won't make a buildin which is
    inconsistent with a regular program.

    For a Posix-specified utility, it is obvious that a Posix-conformant
    shell (if it builds it in) must build in a Posix-conformant
    implementation of the utility. And dash does do this. But it also
    builds in one which is inconsistent with /usr/bin/test on Debian, and
    that's a problem.

    But I hasten to note that a Posix-conformant shell is allowed to build
    in *anything*, and if it's not a Posix-specified utility that's being
    built in, then it can have *any behavior whatsoever*. For example, a Posix-compatible shell may build in a shell command named "debconf",
    such that when you run it, it tries deletes everything in /etc. Since
    my maintainer scripts are require to work with any Posix-compatible
    shell whatsoever, I cannot call debconf from the PATH.

    Given Policy as it currently exists, the only way to reliably conform
    to section 10.4 is either:
    * Never use /bin/sh, ever, as a shell interpereter; or
    * Call all programs with fully specified paths to avoid unexpected
    builtins.

    The second of those solutions conflicts with section 6.1's requirement
    not to normally prepend a path. The former seems suboptimal, but does
    work. However, Policy as currently worded makes it sound like it is
    ever safe to use /bin/sh, and it is not, ever.

    David Weinehall proposed one possible fix, to extend 10.4 to restrict
    the use not only of non-Posix shell features, but also non-Posix
    features in other commands, if those commands are "historically built
    in". This is unworkably vague, but can be toned up to something that
    does work; see the third option below.

    I believe there are four reasonable sorts of policy amendment that
    would solve this problem:

    OPTION 1: Change 10.4 to require that scripts never use /bin/sh.

    OPTION 2: Restrict /bin/sh to a specified list of shells, rather
    than "any POSIX compatible shell", and require that shell scripts
    run correctly on that list.

    OPTION 3: Extend 10.4 to require the use only of Posix features not
    only for the shell, but for a specified list of other commands; that
    list would be based upon which commands are in fact being built-in
    by shells like dash in ways which are inconsistent with the binaries
    in /usr/bin.

    OPTION 4: Change section 10.4 to require the use of full pathnames
    for everything which is not ordered by Posix to be builtin to the
    shell; remove the "no pathnames on commands" direction in section
    6.1.


    I strongly prefer option 2. Options 1 and 3 seem ok to me, with a
    slight preference for option 1 because it is more maintainable.
    Option 4 is horrible, and you don't need me to say why.


    -- System Information:
    Debian Release: 3.1
    APT prefers unstable
    APT policy: (500, 'unstable'), (500, 'testing')
    Architecture: powerpc (ppc)
    Kernel: Linux 2.4.20-ben10
    Locale: LANG=en_US, LC_CTYPE=en_US

    -- no debconf information



    --
    To UNSUBSCRIBE, email to [email protected]
    with a subject of "unsubscribe". Trouble? Contact [email protected]

    --- SoupGate-Win32 v1.05
    * Origin: you cannot sedate... all the things you hate (1:229/2)
  • From Wouter Verhelst@1:229/2 to Thomas Bushnell BSG on Sat Aug 21 04:10:07 2004
    XPost: linux.debian.policy
    From: [email protected]

    On Fri, Aug 20, 2004 at 05:16:35PM -0700, Thomas Bushnell BSG wrote:
    By contrast, listing a bunch of shells directly means that policy
    doesn't have to get into the nitty gritty--which can only be a good
    thing!

    Sorry; but I find a list of shells far more ugly than using hard-coded
    paths.

    --
    EARTH
    smog | bricks
    AIR -- mud -- FIRE
    soda water | tequila
    WATER
    -- with thanks to fortune


    --
    To UNSUBSCRIBE, email to [email protected]
    with a subject of "unsubscribe". Trouble? Contact [email protected]

    --- SoupGate-Win32 v1.05
    * Origin: you cannot sedate... all the things you hate (1:229/2)
  • From Thomas Bushnell BSG@1:229/2 to Wouter Verhelst on Sat Aug 21 10:10:11 2004
    XPost: linux.debian.policy
    From: [email protected]

    Wouter Verhelst <[email protected]> writes:

    On Fri, Aug 20, 2004 at 05:16:35PM -0700, Thomas Bushnell BSG wrote:
    By contrast, listing a bunch of shells directly means that policy
    doesn't have to get into the nitty gritty--which can only be a good
    thing!

    Sorry; but I find a list of shells far more ugly than using hard-coded
    paths.

    That's ok; I think we've done a decent job of narrowing the
    possibilities down, and I did miss that fifth possible resolution, so
    thanks.

    If you don't mind, I'll write up a description of both of our
    favorites, giving the pros and cons as I understand them, hopefully
    which we would both agree to, and then post it for the comments of
    others?



    --
    To UNSUBSCRIBE, email to [email protected]
    with a subject of "unsubscribe". Trouble? Contact [email protected]

    --- SoupGate-Win32 v1.05
    * Origin: you cannot sedate... all the things you hate (1:229/2)
  • From Wouter Verhelst@1:229/2 to Thomas Bushnell BSG on Sat Aug 21 14:00:12 2004
    XPost: linux.debian.policy
    From: [email protected]

    On Sat, Aug 21, 2004 at 01:02:18AM -0700, Thomas Bushnell BSG wrote:
    Wouter Verhelst <[email protected]> writes:

    On Fri, Aug 20, 2004 at 05:16:35PM -0700, Thomas Bushnell BSG wrote:
    By contrast, listing a bunch of shells directly means that policy
    doesn't have to get into the nitty gritty--which can only be a good thing!

    Sorry; but I find a list of shells far more ugly than using hard-coded paths.

    That's ok; I think we've done a decent job of narrowing the
    possibilities down, and I did miss that fifth possible resolution, so
    thanks.

    If you don't mind, I'll write up a description of both of our
    favorites, giving the pros and cons as I understand them, hopefully
    which we would both agree to, and then post it for the comments of
    others?

    Good plan.

    --
    EARTH
    smog | bricks
    AIR -- mud -- FIRE
    soda water | tequila
    WATER
    -- with thanks to fortune


    --
    To UNSUBSCRIBE, email to [email protected]
    with a subject of "unsubscribe". Trouble? Contact [email protected]

    --- SoupGate-Win32 v1.05
    * Origin: you cannot sedate... all the things you hate (1:229/2)