• d-i user questions, web proxies, automated installation

    From Marc Haber@21:1/5 to All on Sun Mar 27 15:50:01 2022
    Hi,

    first let me thank you for helping me with my adduser hat on last week
    by giving helpful and quick answers to my question regarding debconf. I
    really appreciate that.

    Now I am coming to you as a User of Debian-Installer and am fully
    prepared to be sent to some other place with my questions.

    In a small side project I am currently helping a company to design and implement a deployment process for embedded Debian to a headless
    multimedia controller based on amd64 (sic!). Those machines do have HDMI
    and USB, but hidden away inside the box so you need to open the case to
    access those ports. I would like to avoid that at least for the bulk of installations.

    Unfortunately, I find the Installation Guide a bit terse on the topic of preseeded, headless installation, but am prepared to help improving the document. I would probably need some guidance in finding the "right" way
    to do things before writing them down (or, if weirdness turns out to be
    a bug, filing the appropriate bugs) and am wondering whether this list
    might be the correct medium to answer installation questions regarding preseeding. I am afraid that technical questions regarding preseeding
    will get drowned on debian-user, and probably not find in-depth answers
    on debian-user-german.

    After a preseeded installation, a common task is to hand over the
    freshly installed system to some kind of configuration management like
    puppet. I have seen more or less ugly methods do to that, including a multi-hundred character long shell string as d-i preseed/late_command
    with advanced multi level quoting hell, having an localinit.deb which is installed along with the base system and which does that initialization
    and enrollment after first reboot into the fresly installed system,
    staying around for the entire life of the box, etc. Is there any
    documentation / opinion about doing this more elegantly?

    And then: Can I have a single pre-seed file that would allow me to
    configure the Installer and Apt to choose the first web proxy from a
    list of proxies defined in some pre-seed data field, choosing the first
    one that happens to be available and responding, falling back to direct connection if none of the proxies is there? That would allow me to use
    the same Installer image for installations in a variety of places with
    various differnt proxy setups, avoiding the work to have an Installer
    image per site (giving the users the possibility to fail by just
    choosing the wrong USB stick).

    Imagine that I have a remote system in a network that I don't manage,
    for example a hosting network. I have a box running some kind of
    unnamed linux, a locally provided rescue system, or grml. I am currently installing in such environments by debootstrapping manually, chrooting
    into that system and using my existing configuration management to "personalize" the installation. Is there a possibility to start d-i
    proper in such a system to get all of d-i's magic, probably skipping partitioning and filesystem creation (that might already been done)?

    And last question: How seriously pluggable is the Installer? Can I, for example, build my own udeb for apt configuration or my own, much less
    flexible partitioner and just throw those udebs into an installer tree
    and directly use it? Or do I need to delve in-depth into the build
    process of the entire Installer to do that?

    Thanks for helping. No need to Cc me any more, I am subscribed now.

    Greetings
    Marc

    -- ----------------------------------------------------------------------------- Marc Haber | "I don't trust Computers. They | Mailadresse im Header Leimen, Germany | lose things." Winona Ryder | Fon: *49 6224 1600402 Nordisch by Nature | How to make an American Quilt | Fax: *49 6224 1600421

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Roland Clobus@21:1/5 to All on Sun Mar 27 18:10:01 2022
    This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --------------E60CIQ7MmSqODoyWIe4DPgUp
    Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: base64

    SGVsbG8gTWFyYywNCg0KT24gMjcvMDMvMjAyMiAxNTo0MywgTWFyYyBIYWJlciB3cm90ZToN Ci4uLg0KPiBBbmQgdGhlbjogQ2FuIEkgaGF2ZSBhIHNpbmdsZSBwcmUtc2VlZCBmaWxlIHRo YXQgd291bGQgYWxsb3cgbWUgdG8NCj4gY29uZmlndXJlIHRoZSBJbnN0YWxsZXIgYW5kIEFw dCB0byBjaG9vc2UgdGhlIGZpcnN0IHdlYiBwcm94eSBmcm9tIGENCj4gbGlzdCBvZiBwcm94 aWVzIGRlZmluZWQgaW4gc29tZSBwcmUtc2VlZCBkYXRhIGZpZWxkLCANCi4uLg0KDQpUaGlz IGxvb2tzIGxpa2UgYSB1c2UtY2FzZSBmb3IgRkFJIChGdWxseSBBdXRvbWF0aWMgSW5zdGFs bGF0aW9uKTogDQpodHRwczovL3dpa2kuZGViaWFuLm9yZy9GQUkNCkRpZCB5b3UgbG9vayBh bHJlYWR5IHdoZXRoZXIgdGhhdCB3b3VsZCBmaXQgeW91ciBwdXJwb3NlPw0KDQpXaXRoIGtp bmQgcmVnYXJkcywNClJvbGFuZCBDbG9idXMNCg==

    --------------E60CIQ7MmSqODoyWIe4DPgUp--

    -----BEGIN PGP SIGNATURE-----

    iQIzBAEBCgAdFiEEUFVLM5Bdj7GSJEb+YsV8aqYUlb0FAmJAihgACgkQYsV8aqYU lb3rwA/+N/i0jlXL2JKHUpYmmHkReM5H3JapDdor56gGC7UcfE4o+YSaxrAb/v1N wowu6xZMeomGdQrIyYF3Qn6BubrLaN8snPHds3j3UCg5jbok2EkR+CFuSFz3agrE cQ6NCU5yT5GzkhoWEavHaWCsQ+7Yvvcbz6KvkLuA8Qg9e2xP2ictlfMV13erPPIC oBcD5wXX0xufwVAQXedYLulvgqna+KbQpRnZomEPJQjnso+psLrU5FHbqC4N0J5+ U5msr4Zi1c6cjVgrZw0O+j7oY0QmjbjvYSUnAY5D+JQY+8klpJ3bmaeHodvRBTYv x0uW1rwMCoSu4dtUD/dQziEuHsBpXsc2WEJVZflR/ESMwwci10YiG5/55zbL00iR Yq8dl4ct0X1RzOvbCF9cQnmmkxu9IXrd5EgnuadyrK984lYjCC8qQeSSk+7xp3Hc anBQyRYo7pmltbhSJi0LfEnIdIVmZWSGORuwtYYm5Rni7z6H0bqMIAwwt7a7TgdU RCxS0+oBMjJBZucmVOAPWAWGAzmXj3tNZmbtSfBcwVqj4zRi8KDWYA90p7cPOm5C OjZ0y8ZbqUs5WmS/JtRxSzISj0LUYBujK8M5Y4z8lK7+Sgbu+ScxfoHCf/FhC1BO 3D9jK1hG6PvqPin1RnxX1pYN6iBtO2tB5K3OETB+G55SrI5Eld8=
    =QMSY
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Geert Stappers@21:1/5 to Marc Haber on Mon Mar 28 09:10:01 2022
    On Sun, Mar 27, 2022 at 03:43:47PM +0200, Marc Haber wrote:
    Hi,

    ... intro ...

    Welcome

    In a small side project I am currently helping a company to design and implement a deployment process for embedded Debian to a headless
    multimedia controller based on amd64 (sic!). Those machines do have HDMI
    and USB, but hidden away inside the box so you need to open the case to access those ports. I would like to avoid that at least for the bulk of installations.

    Unfortunately, I find the Installation Guide a bit terse on the topic of preseeded, headless installation, but am prepared to help improving the document. I would probably need some guidance in finding the "right" way
    to do things before writing them down (or, if weirdness turns out to be
    a bug, filing the appropriate bugs) and am wondering whether this list
    might be the correct medium to answer installation questions regarding preseeding. I am afraid that technical questions regarding preseeding
    will get drowned on debian-user, and probably not find in-depth answers
    on debian-user-german.

    After a preseeded installation, a common task is to hand over the
    freshly installed system to some kind of configuration management like puppet. I have seen more or less ugly methods do to that, including a multi-hundred character long shell string as d-i preseed/late_command
    with advanced multi level quoting hell, having an localinit.deb which is installed along with the base system and which does that initialization
    and enrollment after first reboot into the fresly installed system,
    staying around for the entire life of the box, etc. Is there any documentation / opinion about doing this more elegantly?

    And then: Can I have a single pre-seed file that would allow me to
    configure the Installer and Apt to choose the first web proxy from a
    list of proxies defined in some pre-seed data field, choosing the first
    one that happens to be available and responding, falling back to direct connection if none of the proxies is there? That would allow me to use
    the same Installer image for installations in a variety of places with various differnt proxy setups, avoiding the work to have an Installer
    image per site (giving the users the possibility to fail by just
    choosing the wrong USB stick).

    A few days (a week?) ago wrote Phill something like
    There is https://hands.com/d-i/
    and would like revisited together with you
    for recent release
    (Yeah, I staid in my email program, didn't visit the archive of
    the mailinglist for the exact text)

    Imagine that I have a remote system in a network that I don't manage,
    for example a hosting network. I have a box running some kind of
    unnamed linux, a locally provided rescue system, or grml. I am currently installing in such environments by debootstrapping manually, chrooting
    into that system and using my existing configuration management to "personalize" the installation. Is there a possibility to start d-i
    proper in such a system to get all of d-i's magic, probably skipping partitioning and filesystem creation (that might already been done)?

    url=the.remote.system
    as boot parameter.


    And last question: How seriously pluggable is the Installer? Can I, for example, build my own udeb for apt configuration or my own, much less flexible partitioner and just throw those udebs into an installer tree
    and directly use it? Or do I need to delve in-depth into the build
    process of the entire Installer to do that?

    Begin.
    Start with what is already available.
    That way you can find "your missing piece".
    When "found" start wondering about how to optimize development
    of the new piece.


    Thanks for helping. No need to Cc me any more, I am subscribed now.

    Please consider to have one subject in a thread.


    Greetings
    Marc


    Groeten
    Geert Stappers
    --
    Silence is hard to parse

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Philip Hands@21:1/5 to Geert Stappers on Mon Mar 28 14:30:02 2022
    Geert Stappers <stappers@stappers.nl> writes:

    On Sun, Mar 27, 2022 at 03:43:47PM +0200, Marc Haber wrote:
    Hi,

    ... intro ...

    Welcome

    In a small side project I am currently helping a company to design and
    implement a deployment process for embedded Debian to a headless
    multimedia controller based on amd64 (sic!). Those machines do have HDMI
    and USB, but hidden away inside the box so you need to open the case to
    access those ports. I would like to avoid that at least for the bulk of
    installations.

    Unfortunately, I find the Installation Guide a bit terse on the topic of
    preseeded, headless installation, but am prepared to help improving the
    document. I would probably need some guidance in finding the "right" way
    to do things before writing them down (or, if weirdness turns out to be
    a bug, filing the appropriate bugs) and am wondering whether this list
    might be the correct medium to answer installation questions regarding
    preseeding. I am afraid that technical questions regarding preseeding
    will get drowned on debian-user, and probably not find in-depth answers
    on debian-user-german.

    After a preseeded installation, a common task is to hand over the
    freshly installed system to some kind of configuration management like
    puppet. I have seen more or less ugly methods do to that, including a
    multi-hundred character long shell string as d-i preseed/late_command
    with advanced multi level quoting hell, having an localinit.deb which is
    installed along with the base system and which does that initialization
    and enrollment after first reboot into the fresly installed system,
    staying around for the entire life of the box, etc. Is there any
    documentation / opinion about doing this more elegantly?

    And then: Can I have a single pre-seed file that would allow me to
    configure the Installer and Apt to choose the first web proxy from a
    list of proxies defined in some pre-seed data field, choosing the first
    one that happens to be available and responding, falling back to direct
    connection if none of the proxies is there? That would allow me to use
    the same Installer image for installations in a variety of places with
    various differnt proxy setups, avoiding the work to have an Installer
    image per site (giving the users the possibility to fail by just
    choosing the wrong USB stick).

    A few days (a week?) ago wrote Phill something like
    There is https://hands.com/d-i/
    and would like revisited together with you
    for recent release
    (Yeah, I staid in my email program, didn't visit the archive of
    the mailinglist for the exact text)

    I did a bit of testing of that at the time and got it working with
    bookworm, so pushed the new layout up to https://hands.com/d-i/ (which
    I've not yet tested from there, so it might still have some rough edges)

    At the very least, the documentation needs to be edited to talk about
    bookworm rather than jessie.

    Imagine that I have a remote system in a network that I don't manage,
    for example a hosting network. I have a box running some kind of
    unnamed linux, a locally provided rescue system, or grml. I am currently
    installing in such environments by debootstrapping manually, chrooting
    into that system and using my existing configuration management to
    "personalize" the installation. Is there a possibility to start d-i
    proper in such a system to get all of d-i's magic, probably skipping
    partitioning and filesystem creation (that might already been done)?

    url=the.remote.system
    as boot parameter.


    And last question: How seriously pluggable is the Installer? Can I, for
    example, build my own udeb for apt configuration or my own, much less
    flexible partitioner and just throw those udebs into an installer tree
    and directly use it? Or do I need to delve in-depth into the build
    process of the entire Installer to do that?

    Begin.
    Start with what is already available.
    That way you can find "your missing piece".
    When "found" start wondering about how to optimize development
    of the new piece.

    The whole process is the result of udebs being pulled in by their
    dependencies, so you can replace parts or all of the process by
    satisfying the relevant dependencies with your own udebs. Of course,
    it's not possible to replace the bits that have already run by the time preseeding happens, unless you rebuild the media with your bits.

    You can also do pretty-much anything directly via preseeding if you
    don't mind being a little evil -- here's an example of some in-flight
    brain surgery: replacing bits of a udeb as it is unpacked to change its behaviour (which is OK for testing, but I'd generally recommend doing it properly in a udeb):

    https://hands.com/d-i/bug/846002/

    BTW When it comes to testing new udebs, I have some stuff on salsa that
    should allow you to concentrate on building your udeb, and then have
    salsa generate a mini-ISO to test it.

    It uses this:

    https://salsa.debian.org/installer-team/branch2repo

    which kicks this off:

    https://salsa.debian.org/installer-team/debian-installer/-/blob/branch2repo/debian/salsa-ci.yml

    Which is a bit arcane, so feel free to ask about it if you want to give
    it a try, and I'll try to remember how it works ;-)

    HTH

    Cheers, Phil.
    --
    |)| Philip Hands [+44 (0)20 8530 9560] HANDS.COM Ltd.
    |-| http://www.hands.com/ http://ftp.uk.debian.org/
    |(| Hugo-Klemm-Strasse 34, 21075 Hamburg, GERMANY

    --=-=-Content-Type: application/pgp-signature; name="signature.asc"

    -----BEGIN PGP SIGNATURE-----

    iQIzBAEBCgAdFiEE3/FBWs4yJ/zyBwfW0EujoAEl1cAFAmJBqI8ACgkQ0EujoAEl 1cA4uxAAi4+UnG56NAHdkREigZ1ewqFYIOMPw9qNBOiJOEYobTbFPDhN5+i3XqLX J/L6JtlkzfNA22LIY106hDOStbHzEoXNSyaGI70uf0ElSF3vW1+bDDLlJ1QGG2jV /z++EFX0my0l/1Mq5XIyVDms+3Bus3fvUmDEo45bgCfI2QmR4n4cumxLNJ3vt//z OxPrH2E9FlTZZTXdbPdkwkMJKqyr17t5Va58xjg598nXByTIi245j0Ykdq8Dg5Rr r3UVSw/ZwYC/dKsOCeR5UjeHeHeB/ofyaPLOkEQINkdfvLp4m0Fm3xGkIjZr2C55 +5duuSP5c+NMSWV+YW9+bw05VWGryewWfUQj1t/yd9pkznbxOWnelra0XTB3ohcm TvkyvqZcc2ru6ws7uh0u6RMcuauwkPfqlhpXpdP3GRJXflP8yGrhWHI7XwH1/Oex uSpXXlSBOg0alXo/2yOjlE24T7JgzNFHW3NnWqwWHYu0Bf+nihsUlBEE7/rA11FP Ta4MCYi+S8mVYFJZMjfkRBQE0AAlnx6apONxbc2NvLaKu7fS3b/RJmg7j7bNSuJb 1dC+0IiYl84OlPuRIbIOTDDlOXCQhorTSxpaoI5rlx5Jz11tRqtG01GSXfiaa49A XEpnIF4QV4PUaQ0