• Bug#924401: base-files fails postinst when base-passwd is unpacked

    From Helmut Grohne@21:1/5 to All on Tue Mar 12 16:30:01 2019
    XPost: linux.debian.bugs.dist

    Package: base-passwd,base-files,debian-policy

    Debian policy section 3.8 says:

    | Essential is defined as the minimal set of functionality that must be
    | available and usable on the system at all times, even when packages
    | are in the “Unpacked” state.

    When unpacking (but not configuring) a buster or unstable essential
    package set, nothing creates /etc/passwd. Creation of that file is
    performed by base-passwd.postinst. base-files.postinst relies on a
    working /etc/passwd by using e.g. "chown root:root".

    Now we can make a choice:
    A. /etc/passwd is part of base-passwd's interface and base-files is
    right in relying on it working at all times. Then base-passwd is rc
    buggy for violating a policy must. Fixing this violation is
    technically impossible.
    B. /etc/passwd is not part of base-passwd's interface and base-files
    wrongly relies on its presence rendering base-files rc buggy.
    C. Guillem Jover hinted that policy expects every essential package to
    be configured at least once. The current text does not make this
    assumption clear. If it holds, policy would simply say nothing about
    how to bootstrap an essential system, which may be fine. Given that
    we have debootstrap, cdebootstrap, multistrap, and mmdebstrap, it
    seems like specifying the bootstrap interface would be a good idea.
    Unfortunately, I don't exactly understand the bootstrap interface at
    present. In practise, you cannot run postinsts of essential packages
    in arbitrary order.

    I argue that something is buggy. I'm not sure what. I gave three
    options. Can we gather consensus on one of these?

    Helmut

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Simon McVittie@21:1/5 to Helmut Grohne on Tue Mar 12 17:40:02 2019
    XPost: linux.debian.bugs.dist

    It would probably be good for the overall robustness of the system if
    we try to solve this from multiple angles.

    On Tue, 12 Mar 2019 at 16:17:10 +0100, Helmut Grohne wrote:
    A. /etc/passwd is part of base-passwd's interface and base-files is
    right in relying on it working at all times. Then base-passwd is rc
    buggy for violating a policy must. Fixing this violation is
    technically impossible.

    If it isn't implementable then it's probably the wrong design.

    Strictly speaking, I think /etc/passwd *is* part (or maybe all?) of base-passwd's Essential interface, but then the Policy requirement that
    it provide this interface even when unconfigured is unimplementable, and
    we can't do unimplementable things.

    I think it would be reasonable to say that Essential packages are *not* entitled to assume that base-passwd provides /etc/passwd, even though non-Essential packages are entitled to assume it.

    The lateral-thinking approach to solving this would be to have an
    Essential or transitively-Essential fallback NSS module that provides
    uids and gids 0 to 99 and 65534 without needing /etc/passwd. After all,
    they're part of the ABI of a Debian system, so deleting them is not a
    supported action (although reconfiguring their shells, etc. in /etc/passwd
    is supported, especially for uid 0, so nsswitch.conf would need to have
    either files or compat before this module). However, that's probably
    a bigger hammer than people really want, and is annoying for multiarch
    (we'd want it installed for each foreign architecture).

    libnss-systemd partially solves this: it guarantees to provide uid and
    gid 0 and 65534, but doesn't provide the 1-99 range, isn't Essential,
    isn't portable to non-Linux, depends on systemd, and would cause political objections.

    B. /etc/passwd is not part of base-passwd's interface and base-files
    wrongly relies on its presence rendering base-files rc buggy.

    Perhaps base-files should use chown 0:0, etc.? That would be more robust.

    In addition to the root user and group, it would have to hard-code
    the numeric group IDs of staff, mail and utmp, AFAICS. That doesn't
    seem awful? They're all in the statically-allocated range.

    The postinst is already a built file (a .in file with sed substitutions),
    so it could take the numeric staff, mail and utmp gids at build time
    and hard-code them in the binary package without also having to hard-code
    them in the source package, if preferred.

    Other possibilities in this direction include:

    - complete the staff-group-for-usr-local transition and stop using the
    staff group
    - use systemd-tmpfiles (or something compatible with tmpfiles.d(5)
    on non-systemd systems) to create /var/log/*tmp, /var/log/lastlog,
    /run/utmp, /var/mail and any other /var files that are needed during boot
    (in fact systemd already has tmpfiles snippets for all the files I named,
    except /var/mail)

    Given that
    we have debootstrap, cdebootstrap, multistrap, and mmdebstrap, it
    seems like specifying the bootstrap interface would be a good idea.
    Unfortunately, I don't exactly understand the bootstrap interface at
    present. In practise, you cannot run postinsts of essential packages
    in arbitrary order.

    This is certainly more fragile than I'd hope: I've seen debootstrap fail
    in Open Build Service chroots when presented with a modified Essential
    set (in a Debian derivative targeting containers that are not multi-user systems and never run on bare metal, which doesn't need everything that a "real" Debian system does).

    If we rely on bootstrap implementations having out-of-band knowledge
    of the right order to configure the Essential set, the risk is that
    they need to have different out-of-band knowledge for different target distributions, leading to the bootstrap implementation becoming relatively tightly coupled to the target distribution.

    Maybe the rule should be to retry configuration of each unconfigured
    package until either they all succeed, or forward progress stops being
    made? Pythonesque pseudocode:

    to_configure = set(["base-files", "base-passwd", ...])

    while to_configure is non-empty:
    failed = set()

    for p in to_configure: # can be in arbitrary order
    if run_postinst(p) fails:
    failed.add(p)

    if len(failed) == len(to_configure):
    # Nothing succeeded since the last iteration of the outer loop.
    # Assume that trying again isn't going to help.
    fatal("Could not configure Essential packages:", failed)
    else:
    # Progress has been made, so retry any failures in the hope that
    # the progress we made has unblocked them, or terminate the loop
    # if there were none
    to_configure = failed

    assert(to_configure is empty)
    log success

    Regards,
    smcv

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Santiago Vila@21:1/5 to Helmut Grohne on Tue Mar 12 16:40:02 2019
    XPost: linux.debian.bugs.dist

    On Tue, Mar 12, 2019 at 04:17:10PM +0100, Helmut Grohne wrote:
    Package: base-passwd,base-files,debian-policy

    Debian policy section 3.8 says:

    | Essential is defined as the minimal set of functionality that must be
    | available and usable on the system at all times, even when packages
    | are in the “Unpacked” state.

    When unpacking (but not configuring) a buster or unstable essential
    package set, nothing creates /etc/passwd. Creation of that file is
    performed by base-passwd.postinst. base-files.postinst relies on a
    working /etc/passwd by using e.g. "chown root:root".

    Now we can make a choice:
    A. /etc/passwd is part of base-passwd's interface and base-files is
    right in relying on it working at all times. Then base-passwd is rc
    buggy for violating a policy must. Fixing this violation is
    technically impossible.
    B. /etc/passwd is not part of base-passwd's interface and base-files
    wrongly relies on its presence rendering base-files rc buggy.
    C. Guillem Jover hinted that policy expects every essential package to
    be configured at least once. The current text does not make this
    assumption clear. If it holds, policy would simply say nothing about
    how to bootstrap an essential system, which may be fine. Given that
    we have debootstrap, cdebootstrap, multistrap, and mmdebstrap, it
    seems like specifying the bootstrap interface would be a good idea.
    Unfortunately, I don't exactly understand the bootstrap interface at
    present. In practise, you cannot run postinsts of essential packages
    in arbitrary order.

    I think you actually can as soon as the system has been properly
    bootstrapped, which is precisely the work of bootstrapping tools.

    Can we please reassign this to whatever package is not bootstrapping
    the system properly in your case?

    Thanks.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)