• Re: dpkg currently warning about merged-usr systems (revisited)

    From Simon Richter@21:1/5 to Luca Boccassi on Fri May 12 14:20:01 2023
    Hi Luca,

    [dropping the tech-ctte bug Cc, because most of this mail is irrelevant
    to that discussion, I'll send a separate mail for that one paragraph]

    On 5/12/23 02:51, Luca Boccassi wrote:

    The crux of the issue is that we are hearing how negatively affecting derivatives in any way, even purely theoretically, is a big no-no, in
    this very same thread and topic. Reaching out and asking for directions/help/whatever is not enough in that context. So it follows
    that it cannot be enough in this context either, and it must be fixed instead.
    There are different levels of expectations here: one for things that
    were explicitly promised, and one for things that are only incidentally
    the way they are.

    For example, whether a binary is in /bin or /usr/bin is inconsequential
    except for the two-stage boot process used by sysvinit, and whether a
    library is in /lib or /usr/lib is inconsequential except for whether a
    binary in /bin uses it as part of the first sysvinit boot stage.

    The systemd boot process has a different early/late split and uses an
    initramfs as the first stage, so it has no use for the old separation policies[1], and they were only ever promised as part of sysvinit
    policy, which has been demoted to a status between "probably still
    useful to someone" and "annoying that it still exists", depending on who
    you ask.

    For that reason I think we do have a consensus that moving the files is allowed, and derived distributions may not expect us to follow the
    superseded policy anymore -- so Devuan will definitely need to provide
    their own packages for all the early boot stuff, and that is fine,
    because the split policy has been dropped with the introduction of
    systemd, even if communication on that has been poor because the people
    driving it could not even be bothered to update Policy.

    On the other hand, we need to either keep the interfaces that have been documented and that people still rely on, or deprecate them, document
    that they have been deprecated, give users time to adapt, and ideally
    explain the reasoning and alternatives.

    Anything that is not an interface can be changed, and anyone relying on
    an implementation detail is on their own. You can get the same
    information from reading /var/lib/dpkg/diversions or from calling
    "dpkg-divert --list", but it is immediately obvious which of those is
    the interface and which is incidental.

    This discussion is mostly about interfaces that have been broken by the transition: are we supposed to fix them, can we break them more since
    the sky hasn't fallen, and whose responsibility is it to fix problems
    when it is disputed whether an interface has been broken. Replacing a
    directory controlled by dpkg with a symlink is explicitly allowed, but
    whether the symlink may point to a directory also controlled by dpkg is
    unclear -- the normative language is silent on this, and the use case in
    the design document does not anticipate it.

    If the file system interface on the bottom of dpkg was broken by
    usrmerge, then it would have been the responsibility of the usrmerge
    team to coordinate that with the dpkg team. If it was not, then it's the
    dpkg team's responsibility to fix their data model.

    Right now, all we have is after-the-fact documentation from the dpkg
    team that some of the interfaces do not work in the context of usrmerge.
    That is unfortunate, but not easy to change.

    #1035904 does not even attempt that, it just aims to remove
    documentation, that is definitely the wrong approach even in a world
    where after-the-fact notification instead of pre-transition coordination
    is the norm.

    The usrmerge transition so far has been an elaborate game of "I'm not
    touching you" around existing interfaces to get where it is today[2],
    but continuing requires changing interfaces. That is a completely valid approach especially in the free software world, but requires a bit more stakeholder management than moving a few files around while keeping them accessible through their old location.

    You may also have noticed that responsibility for completing it has
    shifted to different people now that it has hit this wall.

    We recognize that you are holding the bag here and support you in the
    search for a minimally invasive solution, but there is also a good
    chance that none exists and the scope of this change is too large for a
    bunch of hobbyists without a process.

    If that is the case, we need to take a step back and remember that we're
    *also* professionals, so we have the tools and knowledge to get this
    done, even if it feels a lot like our day job. :P

    Simon

    [1] or at least they think so
    [2] stuck

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Luca Boccassi@21:1/5 to Simon Richter on Fri May 12 17:10:01 2023
    On Fri, 12 May 2023 at 13:18, Simon Richter <sjr@debian.org> wrote:

    Hi Luca,

    [dropping the tech-ctte bug Cc, because most of this mail is irrelevant
    to that discussion, I'll send a separate mail for that one paragraph]

    On 5/12/23 02:51, Luca Boccassi wrote:

    The crux of the issue is that we are hearing how negatively affecting derivatives in any way, even purely theoretically, is a big no-no, in
    this very same thread and topic. Reaching out and asking for directions/help/whatever is not enough in that context. So it follows
    that it cannot be enough in this context either, and it must be fixed instead.
    There are different levels of expectations here: one for things that
    were explicitly promised, and one for things that are only incidentally
    the way they are.

    For example, whether a binary is in /bin or /usr/bin is inconsequential except for the two-stage boot process used by sysvinit, and whether a
    library is in /lib or /usr/lib is inconsequential except for whether a
    binary in /bin uses it as part of the first sysvinit boot stage.

    The systemd boot process has a different early/late split and uses an initramfs as the first stage, so it has no use for the old separation policies[1], and they were only ever promised as part of sysvinit
    policy, which has been demoted to a status between "probably still
    useful to someone" and "annoying that it still exists", depending on who
    you ask.

    For that reason I think we do have a consensus that moving the files is allowed, and derived distributions may not expect us to follow the
    superseded policy anymore -- so Devuan will definitely need to provide
    their own packages for all the early boot stuff, and that is fine,
    because the split policy has been dropped with the introduction of
    systemd, even if communication on that has been poor because the people driving it could not even be bothered to update Policy.

    Is that really the case though? AFAIK the kernel/initramfs side of
    things dropped support for split-usr booting long ago, and long before
    any of this.

    On the other hand, we need to either keep the interfaces that have been documented and that people still rely on, or deprecate them, document
    that they have been deprecated, give users time to adapt, and ideally
    explain the reasoning and alternatives.

    Anything that is not an interface can be changed, and anyone relying on
    an implementation detail is on their own. You can get the same
    information from reading /var/lib/dpkg/diversions or from calling "dpkg-divert --list", but it is immediately obvious which of those is
    the interface and which is incidental.

    This discussion is mostly about interfaces that have been broken by the transition: are we supposed to fix them, can we break them more since
    the sky hasn't fallen, and whose responsibility is it to fix problems
    when it is disputed whether an interface has been broken. Replacing a directory controlled by dpkg with a symlink is explicitly allowed, but whether the symlink may point to a directory also controlled by dpkg is unclear -- the normative language is silent on this, and the use case in
    the design document does not anticipate it.

    If the file system interface on the bottom of dpkg was broken by
    usrmerge, then it would have been the responsibility of the usrmerge
    team to coordinate that with the dpkg team. If it was not, then it's the
    dpkg team's responsibility to fix their data model.

    Right now, all we have is after-the-fact documentation from the dpkg
    team that some of the interfaces do not work in the context of usrmerge.
    That is unfortunate, but not easy to change.

    #1035904 does not even attempt that, it just aims to remove
    documentation, that is definitely the wrong approach even in a world
    where after-the-fact notification instead of pre-transition coordination
    is the norm.

    It attempts to remove harmful documentation, because it was expressed
    clearly that even unintentionally harming downstreams is bad and needs
    to be avoided. And in case it's fine to leave that standing as it
    turns out we don't need to worry about such things, that would also be
    good to clarify.

    The usrmerge transition so far has been an elaborate game of "I'm not touching you" around existing interfaces to get where it is today[2],
    but continuing requires changing interfaces. That is a completely valid approach especially in the free software world, but requires a bit more stakeholder management than moving a few files around while keeping them accessible through their old location.

    You may also have noticed that responsibility for completing it has
    shifted to different people now that it has hit this wall.

    We recognize that you are holding the bag here and support you in the
    search for a minimally invasive solution, but there is also a good
    chance that none exists and the scope of this change is too large for a
    bunch of hobbyists without a process.

    If that is the case, we need to take a step back and remember that we're *also* professionals, so we have the tools and knowledge to get this
    done, even if it feels a lot like our day job. :P

    I don't think that's the case, there are at least several viable paths
    ahead that Helmut is exploring.

    Kind regards,
    Luca Boccassi

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)