• Wrestling with debconf

    From Kip Warner@21:1/5 to All on Fri Nov 26 05:20:01 2021
    Hey list,

    I'm having an issue with my myfoo-server maintainer scripts for my
    package myfoo-server. In a nutshell, I am trying to get the user's
    requested configuration to be handled elegantly.

    I've read the debconf specification and debconf-devel(7). I see that my
    issue is discussed:

    https://manpages.debian.org/jessie/debconf-doc/debconf-devel.7.en.html#HACKS

    If the user installs the myfoo-server package they will be prompted by
    debconf in myfoo-server.config to select whether to enable zeroconf,
    upnp, etc. and various other settings for the server. Those selected
    settings, whatever their values, should then be written out during the subsequent run of myfoo-server.postinst to /etc/myfoo/server.conf.

    If myfoo-server.config detects that the user already had an /etc/myfoo/server.conf, such as when they are re-installing the
    package, myfoo-server.conf should read those values in-place from disk
    and then seed debconf via my SeedFromUserConfiguration POSIX shell
    function before any prompts might be displayed (depends on priority, of course).

    That is what I had intended. I think this behaviour is reasonable.

    So I start by installing the myfoo-server package locally, e.g.

    $ sudo apt install ./myfoo-server(...).deb
    ...
    Preconfiguring packages ...
    ...

    I then see debconf prompts, as expected, from within my myfoo-
    server.config. I select true to enable upnp and zeroconf (false and
    true by default in their templates respectively).

    For the sake of debugging, I checked the myfoo-server/zeroconf debconf
    field as you see on myfoo-server.config:192 after it is read back from
    debconf and it is indeed whatever value the user had set it to when
    prompted (and same for all other settings).

    Great. That much works as expected.

    But after that session of myfoo-server.config is run, it appears to be
    run _again_ after the package is unpacked, but _before_ myfoo-
    server.postinst is run:

    ...
    Setting up myfoo-server (...)
    ...

    The problem here is this re-running of the myfoo-server.config happens
    before the myfoo-server.postinst. This is bad because the latter is
    supposed to update the values in /etc/myfoo/server.conf to whatever the
    user just entered via debconf prompts.

    Because myfoo-server.config's second invocation sees the newly unpacked /etc/myfoo/server.conf, it unintentionally seeds debconf with the
    values contained therein.

    The myfoo-server.postinst script is then run, writing out to /etc/myfoo/server.conf the clobbered debconf values.

    What am I doing wrong and what do you recommend?

    These are my myfoo-server.config and myfoo-server.postinst:

    https://pastebin.com/hyWyH5id

    https://pastebin.com/xW1andmW

    --
    Kip Warner -- Senior Software Engineer
    OpenPGP signed/encrypted mail preferred
    https://www.thevertigo.com

    -----BEGIN PGP SIGNATURE-----

    iQIzBAABCgAdFiEEaUi0dj/qpTzsKe4xRLRlxF7U4wQFAmGgXTcACgkQRLRlxF7U 4wQxWRAAgeTZWhFJEtdXv7JVSWkGcFVPjJukg1kpepTh1A9HzIFOYRaXkcopU/OG PYEk/JKr3wZORQFFBTA/EcU9qGIYfrGdgor5iXtXC/yFaBMH2MSrgFF3B5WM4606 55MVpYCPssEjQEprg3rfmIFtDV4mMp33201g/Zyi3MxcfE1QJf0ptCTsGd5E4ovK V/zHuXFMoEsVpbuyjkbe3eIlJ5UGlBhgRsqx3pda7Ap5E9iuSkacNCsmu46EURwO 55gjgkPDk4u+XTsEL/Eq6UaMxFxY62G8ztVPvUMe/K3BdEWe483nicgIAs08Nntj gUqUt/YpNjYZOwSavvw+pg0J3aAHQ7s4APp25ckZ0yVyKlxomm7OIw7BWiGd2PKi pmnklsAtMmoo1O0Q/afQNUXGXxhaLVWjxyKcGyJbm1N3xAVEawmVubdpX1dPKztC sYlPzR5qSp+hMaG6xGSyZmXYmLlrUt15/DmEsPr1rJMIbRGjp+v9xZgZ55VB19sg 6I009+zbnqlqjGC7/v6GvqfMW0M9erS4eC3nJYkOS4mPALAhkuhiNyl05cPxoMQ1 Iui4ltUTyP4LMT/aKY6Fc9L6nMVzpbkrgiz4DOaN8U3RkFOg0zrnXKSxtUL8g3Ko 0pq9phX56VqDWXRsdM+aXB36MgS6q9TzX9lkWKVbUkvQaUrDwWc=
    =wFL0
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gatew
  • From Niels Thykier@21:1/5 to All on Tue Jan 4 12:40:02 2022
    Kip Warner:
    Hey list,

    [...]
    The problem here is this re-running of the myfoo-server.config happens
    before the myfoo-server.postinst. This is bad because the latter is
    supposed to update the values in /etc/myfoo/server.conf to whatever the
    user just entered via debconf prompts.

    Because myfoo-server.config's second invocation sees the newly unpacked /etc/myfoo/server.conf, it unintentionally seeds debconf with the
    values contained therein.

    [...]
    Hi,

    (Having seen your enquiry on IRC, I presume this issue was still relevant)

    I read the "newly unpacked /etc/myfoo/server.conf" as you shipping "/etc/myfoo/server.conf" directly inside the package in that path. If
    that is correctly understood, then I think that is the source of your woes.

    As I recall, when you manage a file via debconf, you should *not* ship
    it directly in the package. You can ship a template in a different
    location (e.g., /usr/share/myfoo/server.conf.template) and then use that combined with the debconf answers to generate the initial /etc/myfoo/server.conf.

    Perhaps have a look at openssh-server (postinst + config + file listing)
    as an example, which does something similar. It does use "ucf" for
    handling the merge on updates, which is a different approach than yours
    for creating/updating the configuration file.
    I can recommend that from a consistency PoV, so your package would
    behave the same as other Debian packages if the user were to change the
    file directly. However, I do not think it would be strictly necessary
    to migrate to ucf in order to fix your immediate issue.

    I hope that helps.

    Thanks,
    ~Niels

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Kip Warner@21:1/5 to Niels Thykier on Wed Jan 5 08:10:01 2022
    On Tue, 2022-01-04 at 12:34 +0100, Niels Thykier wrote:
    (Having seen your enquiry on IRC, I presume this issue was still
    relevant)

    Hey Niels. Yes, indeed. Still trying to solve this problem.

    I read the "newly unpacked /etc/myfoo/server.conf" as you shipping "/etc/myfoo/server.conf" directly inside the package in that path.  If
    that is correctly understood, then I think that is the source of your
    woes.

    Yes, that is what I was trying to do.

    As I recall, when you manage a file via debconf, you should *not* ship
    it directly in the package.  You can ship a template in a different
    location (e.g., /usr/share/myfoo/server.conf.template) and then use
    that combined with the debconf answers to generate the initial /etc/myfoo/server.conf.

    Ok, I see that now buried in the last paragraph in DPM s 5.1:

    "Note that a package should not modify a dpkg-handled conffile in its
    maintainer scripts. Doing this will lead to dpkg giving the user
    confusing and possibly dangerous options for conffile update when the
    package is upgraded."

    Perhaps have a look at openssh-server (postinst + config + file
    listing) as an example, which does something similar.  It does use
    "ucf" for handling the merge on updates, which is a different approach
    than yours for creating/updating the configuration file.
      I can recommend that from a consistency PoV, so your package would
    behave the same as other Debian packages if the user were to change the
    file directly.  However, I do not think it would be strictly
    necessary to migrate to ucf in order to fix your immediate issue.

    I've taken a look at openssh-server.{config,postinst} and the contents
    of the package in what's shipped to /etc (nothing). The way they are
    doing it is a bit complicated and I am surprised there isn't an elegant solution yet with debconf. Although it could be I just don't understand
    what they were doing, in which case it's probably even more
    complicated.

    I also still can't figure out why the .config is run twice during
    installation.

    Anyways, what I am trying to do now is ship server.conf as
    server.conf.template in /usr/share/myserver/ and then have .postinst
    attempt the following:

    new_config="$(mktemp)"
    cp -a /usr/share/myserver/server.conf.template "$new_config"
    mkdir -p /etc/myserver/
    ucf --three-way --debconf-ok \
    "$new_config" /etc/myserver/server.conf
    ucfr myserver-server /etc/myserver/server.conf

    But it's not really clear to be what needs to be done within .config.
    Should the script check for /etc/myserver/server.conf early, and if
    it's not available, exit 0?

    --
    Kip Warner -- Senior Software Engineer
    OpenPGP signed/encrypted mail preferred
    https://www.thevertigo.com

    -----BEGIN PGP SIGNATURE-----

    iQIzBAABCgAdFiEEaUi0dj/qpTzsKe4xRLRlxF7U4wQFAmHVPxMACgkQRLRlxF7U 4wQSEhAAgtinUO+qqkkHI7n7v3CJZsMjBIMts1MvkidvquOlzS4rHwyakL0tAB0E p6Hh/+vUuMDgx+YLfzRPhZ4yNq+H5rC0FDMjLuy4GCAXkg1kKus4NFVwJet86lxx J+YjRCto2JvDNl5WHKYIGCZtoIm8P3KbEx00scaFBnvzBFwF0ZBOo7r5xM/sTPmd qMWGuCjVsZ+Rde6xT73PWizFEJOVXNEOnr1o4XaWeMFM41v/Yq3kBBN1tmtfxCpn A2MJQc1uOB66Vi+bMEkpPmP4mf8J4qeZNTIEGqzF7RBD6Rr3eIJCXKlZy1TCfktc fYThSZrdesBiKjzR3jsIgTrR5VfrjLEuKIBS45Iu6DDZXnGMFvmIv0MRU9eyvu8O zuEi4KvmuFewCtWZSzquTvOt49CGLnkRf6tyCeBTrwJ4li/bY+UEJXP8ZeTTtRLP psQjV4jXyOco62HkZTanyit2miybwRH9MfnF3ggl6rAH7Jd4a6StLS/ka96uqdnA Q/arWe9V4fg8SkBg4TkXampDxQHoDbQAugJRmQ380+X9/MQeirq0KVnGmkAy65b1 nzzfp1Maq/DDkQjt0dx80WSGGBspOIaF6hbNb60aD6ONX3sNY9OKAyjWVC8AMGFt b5Hc+N5MZFq0go1U+XUWu74ISHy9urhRPUf3o++VWjT5yvOhOb8=
    =mt+V
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gatew
  • From Kip Warner@21:1/5 to Niels Thykier on Fri Jan 7 08:00:01 2022
    On Tue, 2022-01-04 at 12:34 +0100, Niels Thykier wrote:
    As I recall, when you manage a file via debconf, you should *not* ship
    it directly in the package.  You can ship a template in a different
    location (e.g., /usr/share/myfoo/server.conf.template) and then use
    that combined with the debconf answers to generate the initial /etc/myfoo/server.conf.

    Hey Niels,

    Thanks a lot for your help. Maintainer scripts are often difficult to
    debug because the execution environment is fairly sophisticated with
    many moving pieces. I gave it another go this evening and it appears to
    work. 

    But before I get into how I (hopefully) solved my problem, I should
    mention that the DPM and the debconf manual need to be reconciled on
    the issue of using debconf to try and manage config files that ship
    with the package. In DPM § 5.1, it says don't do this in the last
    paragraph:

    https://www.debian.org/doc/debian-policy/ap-pkg-conffiles.html#automatic-handling-of-configuration-files-by-dpkg

    In the debconf manual, it implies that it can be properly done (even if
    most of the ways people try are not):

    https://manpages.debian.org/jessie/debconf-doc/debconf-devel.7.en.html#Config_file_handling

    Anyways, what I did for myfoo-server.config was have it seed debconf
    from the /usr/share/myfoo/server.conf.template if
    /etc/myfoo/server.conf isn't available. The first time the script is
    run neither file will be available during an installation because
    nothing has been unpacked yet, so it exits quietly. During it's second
    run just before postinst, if the template exists and the user's
    configuration does not, it will seed debconf from the template:

    https://pastebin.com/hyWyH5id

    In my postinst I studied more carefully again the openssh-
    server.postinst logic and it seems to make more sense now. Although, admittedly, it is still a bit of a pain that this pattern doesn't have
    an automated solution yet. This is what I came up with:

    https://pastebin.com/xW1andmW

    In my postrm, if the user requested a purge, I call ucf(1) with --purge
    on the /etc/myfoo/server.conf before rm(1)'ing it:

    https://pastebin.com/3SjBAJRL

    Does this look sane now?

    --
    Kip Warner -- Senior Software Engineer
    OpenPGP signed/encrypted mail preferred
    https://www.thevertigo.com

    -----BEGIN PGP SIGNATURE-----

    iQIzBAABCgAdFiEEaUi0dj/qpTzsKe4xRLRlxF7U4wQFAmHX4rYACgkQRLRlxF7U 4wQo0Q//TSrZqHZfiUDCY+ssMqhu62l6MV5LlUSLuyh3L9ZIjbT5EISlbSGTseI/ p/+MfAyM7IA50umpX341DxqHTv2rNCQng9tFVsDMDgUUns8RY3LF0HblU0Du1fWF r/D1EPOvzW8ZhhY83S9Z1djLet0JvZYb7tDqG7Tq3a6MlcURwnyVcKJcQlTsa6t3 kPJrxoncjtiWd24sDCbgHlUb3MgROCQbXYvZx7B/HKLlM9LHB1Gx+wLLu3E4EFu0 qnZiemNextQPIATUTdI6IX6JAptv92uH9kUsF0WeKRHuIgEWNyZfvhsSgzDBmllB K3UuAAN7O3JzFn0mBTYkxjerfk6o2o9i4eRoEGB0lQJGqaE6HQJG7iFEc7z+OtXT Z7lwmH/kQMmVorMJGsGIYecz1W42zejGgSvZ2kXk3F8ts9qQWs/0PZdI/BpIi7ds MF543UrL5CZCuI4l9V39bxAkMlbDvZZVobnQvS57e+4pke6qlEebtUj8KUzhVKWJ bPBo7sIOc+pYYreVtSXoKYE90nB1tNhSYGymJ1z0fC6wSW9jCz6amH/g2KiSqmDp 3q9XsL/9aZZpMkYfs/yD0kfjS9H+aNVx/i0C9qsZ0OQHsfhGeeBswe1zoSyboo+M 9EuDROWPWQKlkB3D1WwvYV58AJUIhSRW9rL0fDUxClNW2CfUK1k=
    =riQ+
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gatew