• [gentoo-user] systemd mount - what on earth is it doing ...

    From antlists@21:1/5 to All on Fri Sep 24 21:30:02 2021
    This is weird ...

    My /home directory is set up somewhat strangely, as in I've got two raw partitions, I've put dm-integrity on them, raided them together, then
    put lvm on top of that.

    Which got me into a bind with fstab. I've created a systemd service,
    which fires up dm-integrity on those two partitions. But I get the
    impression it doesn't run until fstab completes. Catch-22 - fstab tries
    to mount /home, but it can't until dm-integrity has made the volume appear!

    So I created a systemd mount unit for /home, which only runs after dm-integrity. Great - I enabled it and it appeared to work!

    Only problem, startplasma-wayland now dumped me at a blank screen.

    Now for the weirdo. I disabled it, thinking I would have to log in as
    root, mount /home, and go from there. Except that, when I logged in,
    /home was mounted and startplasma-wayland worked!

    systemctl tells me mount.home is disabled, but also tells me that it ran
    and mounted the drive.

    So what on earth is going on, and more importantly, what am I
    misunderstanding or doing wrong. I would very much like to know why it's working, when I think it shouldn't be!

    Cheers,
    Wol

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From =?UTF-8?B?Q2FuZWsgUGVsw6FleiBWYWxkw@21:1/5 to antlists@youngman.org.uk on Fri Sep 24 22:40:01 2021
    On Fri, Sep 24, 2021 at 2:25 PM antlists <antlists@youngman.org.uk> wrote:

    This is weird ...

    My /home directory is set up somewhat strangely, as in I've got two raw partitions, I've put dm-integrity on them, raided them together, then
    put lvm on top of that.


    Which got me into a bind with fstab. I've created a systemd service,
    which fires up dm-integrity on those two partitions. But I get the
    impression it doesn't run until fstab completes. Catch-22 - fstab tries
    to mount /home, but it can't until dm-integrity has made the volume appear!


    Have you tried using Before=local-fs.target in the service? Or even Before=local-fs-pre.target?

    So I created a systemd mount unit for /home, which only runs after
    dm-integrity. Great - I enabled it and it appeared to work!


    You removed the /home entry from fstab, right?


    Only problem, startplasma-wayland now dumped me at a blank screen.

    Now for the weirdo. I disabled it, thinking I would have to log in as
    root, mount /home, and go from there. Except that, when I logged in,
    /home was mounted and startplasma-wayland worked!


    That sounds like /home was still on fstab...


    systemctl tells me mount.home is disabled, but also tells me that it ran
    and mounted the drive.


    That should be home.mount. What does systemctl status home.mount says?


    So what on earth is going on, and more importantly, what am I misunderstanding or doing wrong. I would very much like to know why it's working, when I think it shouldn't be!


    Regards.
    --
    Dr. Canek Peláez Valdés
    Profesor de Carrera Asociado C
    Departamento de Matemáticas
    Facultad de Ciencias
    Universidad Nacional Autónoma de México

    <div dir="ltr"><div dir="ltr">On Fri, Sep 24, 2021 at 2:25 PM antlists &lt;<a href="mailto:antlists@youngman.org.uk">antlists@youngman.org.uk</a>&gt; wrote:<br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;
    border-left:1px solid rgb(204,204,204);padding-left:1ex">This is weird ...<br>

    My /home directory is set up somewhat strangely, as in I&#39;ve got two raw <br>
    partitions, I&#39;ve put dm-integrity on them, raided them together, then <br> put lvm on top of that.<br></blockquote><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><br>
    Which got me into a bind with fstab. I&#39;ve created a systemd service, <br> which fires up dm-integrity on those two partitions. But I get the <br> impression it doesn&#39;t run until fst
  • From antlists@21:1/5 to All on Sat Sep 25 12:50:02 2021
    On 24/09/2021 21:30, Canek Peláez Valdés wrote:
    On Fri, Sep 24, 2021 at 2:25 PM antlists <antlists@youngman.org.uk <mailto:antlists@youngman.org.uk>> wrote:

    This is weird ...

    My /home directory is set up somewhat strangely, as in I've got two raw
    partitions, I've put dm-integrity on them, raided them together, then
    put lvm on top of that.


    Which got me into a bind with fstab. I've created a systemd service,
    which fires up dm-integrity on those two partitions. But I get the
    impression it doesn't run until fstab completes. Catch-22 - fstab tries
    to mount /home, but it can't until dm-integrity has made the volume
    appear!


    Have you tried using Before=local-fs.target in the service? Or even Before=local-fs-pre.target?

    You mean in my integritysetup.service file? No, because in scrabbling
    around to try and find out what magic incantation was required, I didn't
    find it.

    There's a lot of info about how to write a service file, but if you need
    to run before or after system service files, it's nowhere near so easy
    to find out Before=WHAT or After=WHAT :-)

    So I created a systemd mount unit for /home, which only runs after
    dm-integrity. Great - I enabled it and it appeared to work!


    You removed the /home entry from fstab, right?

    mad it noauto, yes ...

    Only problem, startplasma-wayland now dumped me at a blank screen.

    Now for the weirdo. I disabled it, thinking I would have to log in as
    root, mount /home, and go from there. Except that, when I logged in,
    /home was mounted and startplasma-wayland worked!


    That sounds like /home was still on fstab...

    #LABEL=boot /boot ext4 noauto,noatime 1 2 /dev/disk/by-uuid/F436-354A /boot vfat noauto,noatime 1 2 #UUID=58e72203-57d1-4497-81ad-97655bd56494 / ext4 noatime 0 1
    #LABEL=swap none swap sw 0 0 #/dev/cdrom /mnt/cdrom auto noauto,ro 0 0 /dev/vg-root/lv-gentoo / ext4 rw 0 0
    /dev/vg-home/lv-data /home ext4 noauto,rw 0 0 /dev/disk/by-uuid/e2b75f09-8cb4-4e20-9cfd-21c0aadfa540 none swap
    sw,pri=1 0 0
    /dev/disk/by-uuid/e03459f5-1f57-485b-92e4-b09cf50534c1 none swap
    sw,pri=1 0 0

    systemctl tells me mount.home is disabled, but also tells me that it
    ran
    and mounted the drive.


    That should be home.mount. What does systemctl status home.mount says?

    anthony@thewolery ~ $ systemctl status home.mount
    ● home.mount - Mount Home
    Loaded: loaded (/etc/systemd/system/home.mount; disabled; vendor
    preset: disabled)
    Active: active (mounted) since Fri 2021-09-24 20:13:49 BST; 2min
    51s ago
    Where: /home
    What: /dev/mapper/vg--home-lv--data
    CPU: 13ms
    CGroup: /system.slice/home.mount

    Sep 24 20:13:48 thewolery systemd[1]: Mounting Mount Home...
    Sep 24 20:13:49 thewolery systemd[1]: Mounted Mount Home.
    anthony@thewolery ~ $ cd Scans/HP-M477/

    So what on earth is going on, and more importantly, what am I
    misunderstanding or doing wrong. I would very much like to know why
    it's
    working, when I think it shouldn't be!

    So fstab says noauto, home.mount is disabled, and /home is mounted on
    boot...

    Cheers,
    Wol

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Rich Freeman@21:1/5 to antlists@youngman.org.uk on Sat Sep 25 13:30:02 2021
    On Sat, Sep 25, 2021 at 6:49 AM antlists <antlists@youngman.org.uk> wrote:

    On 24/09/2021 21:30, Canek Peláez Valdés wrote:

    There's a lot of info about how to write a service file, but if you need
    to run before or after system service files, it's nowhere near so easy
    to find out Before=WHAT or After=WHAT :-)

    So, I have a similar challenge because I use lizardfs and it can take
    a while to initialize, and it isn't something systemd understands out
    of the box. (Plus it depends on a container running on the same host,
    and those tend to run late, and other containers depend on the mount.)

    I solved my issues with drop-ins and dependencies.

    I did not create a mount unit for the mount. I'm using fstab and its
    generator to manage that. However, I did use a drop-in so that the automatically-created unit obtains the additional dependencies and
    loads in the right order. This requires using the right unit name as
    created by the generator.

    I set up dependencies for the mount unit so that it would run after
    things that it needed. I also set dependencies on services that use
    the mount, so that they load later. This wasn't something like home,
    so I didn't apply those dependencies to a target. However, if you
    want to tie this into a standard target I suggest checking man
    systemd.special or ask around.


    So I created a systemd mount unit for /home, which only runs after
    dm-integrity. Great - I enabled it and it appeared to work!


    You removed the /home entry from fstab, right?

    mad it noauto, yes ...

    That won't prevent the system from creating a unit for it - it just
    won't be a dependency of local-fs.target.

    I would suggest using only ONE of the following two approaches:1.
    Entry in fstab plus a drop-in to modify its behavior with the same
    name.
    2. Custom mount unit with no entry in fstab at all (commented out is
    fine, noauto is not).

    It is possible you have more than one unit for the same mountpoint.
    I'm not sure how exactly that behaves. I'd just run "systemctl | grep
    mount" and see what comes up - see if you have two apparent entries
    for home. If so one is probably from fstab and the other is probably
    from your custom unit. Unless they have the exact same name your unit
    won't override the system one, and I'm actually not quite sure how
    generators interact with unit overrides. They seem to work fine with
    drop-ins which is why I used this approach.

    That should be home.mount. What does systemctl status home.mount says?

    anthony@thewolery ~ $ systemctl status home.mount
    ● home.mount - Mount Home
    Loaded: loaded (/etc/systemd/system/home.mount; disabled; vendor preset: disabled)

    So what on earth is going on, and more importantly, what am I
    misunderstanding or doing wrong. I would very much like to know why
    it's
    working, when I think it shouldn't be!

    So fstab says noauto, home.mount is disabled, and /home is mounted on
    boot...


    Using noauto only prevents it from being a dep for local-fs.target.
    It doesn't prevent it from being mounted for some other reason.

    I'd look at:
    systemctl list-dependencies --reverse home.mount

    Another command you might find useful is systemd-analyze dot. The
    direct output is actually readable, though intended to be converted
    into a chart with graphviz using the hinted command it outputs.

    If anything explicitly wants home.mount then it is going to get
    mounted even if set to noauto.

    Systemd is one of those things that is really capable, but you have to
    grok a lot of concepts to make it dance for you, otherwise you'll feel
    like you're the one being made to dance. The internet is full of
    one-liners now that it is so popular, but these often obfuscate how it
    actually works. It operates using a series of basic rules and once
    you get how they're being strung together to make things work then it
    is a lot easier to make it do what you expect. Lennart's original
    blog posts about systemd are very useful for this - maybe there are
    some other good resources out there these days.

    --
    Rich

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From antlists@21:1/5 to All on Sat Sep 25 20:50:02 2021
    On 24/09/2021 21:30, Canek Peláez Valdés wrote:
    On Fri, Sep 24, 2021 at 2:25 PM antlists <antlists@youngman.org.uk <mailto:antlists@youngman.org.uk>> wrote:

    This is weird ...

    My /home directory is set up somewhat strangely, as in I've got two raw
    partitions, I've put dm-integrity on them, raided them together, then
    put lvm on top of that.


    Which got me into a bind with fstab. I've created a systemd service,
    which fires up dm-integrity on those two partitions. But I get the
    impression it doesn't run until fstab completes. Catch-22 - fstab tries
    to mount /home, but it can't until dm-integrity has made the volume
    appear!


    Have you tried using Before=local-fs.target in the service? Or even Before=local-fs-pre.target?

    Bear in mind, integritysetup is not in my initrd, so ...

    It looks to me like grub switches to the real root partition read-only
    before firing stuff off, so if I tell my integritysetup.service to run
    before local-fs.target, does that mean everything should be working
    before systemd runs fstab?

    Bear in mind dm-integrity is on my raw partitions, and I'm running lvm
    and md-raid, I'm guessing they're part of local-fs-pre.target? So I
    really need to run dm-integrity before that? Or preferably add
    dm-integrity TO that - how do I do that?

    The problem is this isn't all that discoverable - even digging around /etc/systemd, it's hard to find clues ...

    Cheers,
    Wol

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From antlists@21:1/5 to All on Sat Sep 25 23:10:01 2021
    On 25/09/2021 21:30, Canek Peláez Valdés wrote:
    On Sat, Sep 25, 2021 at 1:48 PM antlists <antlists@youngman.org.uk <mailto:antlists@youngman.org.uk>> wrote:
    [...]

    Bear in mind, integritysetup is not in my initrd, so ...


    I think you would need to put it in your initrd.

    If I've got my root on it (which is the plan, but that means learning
    how to cut my own stuff into the initrd ...)

    It looks to me like grub switches to the real root partition read-only
    before firing stuff off, so if I tell my integritysetup.service to run
    before local-fs.target, does that mean everything should be working
    before systemd runs fstab?

    Bear in mind dm-integrity is on my raw partitions, and I'm running lvm
    and md-raid, I'm guessing they're part of local-fs-pre.target? So I
    really need to run dm-integrity before that? Or preferably add
    dm-integrity TO that - how do I do that?

    The problem is this isn't all that discoverable - even digging around
    /etc/systemd, it's hard to find clues ...


    As Rich already mentioned it, noauto != remove mount point from fstab.
    That's the first thing.

    Well, it's working, which is the important thing. I just want to
    understand WHY it's working - especially since when it was working the
    other way it broke Wayland ... which was why it was weird ...

    Then you need to specify that home.mount should run *after* md/LVM,
    which in turn runs after your integrity setup service. The problem is:
    you have root (/) on md/LVM too; I know they are different
    RAIDs/volumes, but there is one *service* for RAID and LVM, at least out-of-the-box, in systemd. You could try to split different instances
    using templates (the units with @ in their names), but the simplest
    solution is to put your integrity logic in the initrd (with the added
    benefit that you can mount it in emergencies if something goes wrong
    with root).

    At the moment, with root on lvm but not anything else, gentoo takes care
    of all that for me. I'm not ready for the next learning curve of doing
    all that myself in the initrd ...

    If you have the integrity logic in early boot, then your integrity setup service should have:

    Before=mdmonitor.service lvm2-lvmetad.service

    And I think that should be enough? The home.mount unit depends on the underlying device being available, and that will happen only after md
    and LVM have done it's thing.

    I'll try that. lvm should act on the root (sda3) device directly, but
    integrity needs to run before md will join sda4 and sdc4, and then lvm
    will run on the raid to make home available for fstab.

    I'll play with that and get rid of the home.mount, and hope that all
    that doesn't break Wayland. What I don't understand was that when
    home.mount was enabled, it worked but broke Wayland, now I've disabled
    it it still appears to be working, but Wayland is working too! That's
    what I'd like to understand.

    Although if fixing it how you suggest fixes everything, then I'll have
    an understandable, working system, and I'll be happy.

    Cheers,
    Wol

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From =?UTF-8?B?Q2FuZWsgUGVsw6FleiBWYWxkw@21:1/5 to antlists@youngman.org.uk on Sat Sep 25 22:40:02 2021
    On Sat, Sep 25, 2021 at 1:48 PM antlists <antlists@youngman.org.uk> wrote: [...]

    Bear in mind, integritysetup is not in my initrd, so ...


    I think you would need to put it in your initrd.


    It looks to me like grub switches to the real root partition read-only
    before firing stuff off, so if I tell my integritysetup.service to run
    before local-fs.target, does that mean everything should be working
    before systemd runs fstab?

    Bear in mind dm-integrity is on my raw partitions, and I'm running lvm
    and md-raid, I'm guessing they're part of local-fs-pre.target? So I
    really need to run dm-integrity before that? Or preferably add
    dm-integrity TO that - how do I do that?

    The problem is this isn't all that discoverable - even digging around /etc/systemd, it's hard to find clues ...


    As Rich already mentioned it, noauto != remove mount point from fstab.
    That's the first thing.

    Then you need to specify that home.mount should run *after* md/LVM, which
    in turn runs after your integrity setup service. The problem is: you have
    root (/) on md/LVM too; I know they are different RAIDs/volumes, but there
    is one *service* for RAID and LVM, at least out-of-the-box, in systemd. You could try to split different instances using templates (the units with @ in their names), but the simplest solution is to put your integrity logic in
    the initrd (with the added benefit that you can mount it in emergencies if something goes wrong with root).

    If you have the integrity logic in early boot, then your integrity setup service should have:

    Before=mdmonitor.service lvm2-lvmetad.service

    And I think that should be enough? The home.mount unit depends on the underlying device being available, and that will happen only after md and
    LVM have done it's thing.

    Regards.
    --
    Dr. Canek Peláez Valdés
    Profesor de Carrera Asociado C
    Departamento de Matemáticas
    Facultad de Ciencias
    Universidad Nacional Autónoma de México

    <div dir="ltr"><div dir="ltr">On Sat, Sep 25, 2021 at 1:48 PM antlists &lt;<a href="mailto:antlists@youngman.org.uk">antlists@youngman.org.uk</a>&gt; wrote:<br></div><div dir="ltr">[...]</div><div class="gmail_quote"><blockquote class="gmail_quote" style=
    "margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
    Bear in mind, integritysetup is not in my initrd, so ...<br></blockquote><div><br></div><div>I think you would need to put it in your initrd.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,
    204,204);padding-left:1ex">
    It looks to me like grub switches to the real root partition read-only <br> before firing stuff off, so if I tell my integritysetup.service to run <br> before local-fs.target, does that mean everything should be working <br>
    before systemd runs fstab?<br>

    Bear in mind dm-integrity is on my raw partitions, and I&#39;m running lvm <br> and md-raid, I&#39;m guessing they&#39;re part of local-fs-pre.target? So I <br>
    really need to run dm-integrity before that? Or preferably add <br> dm-integrity TO that - how do I do that?<br>

    The problem is this isn&#39;t all that discoverable - even digging around <br> /etc/systemd, it&#39;s hard to find clues ...<br></blockquote><div><br></div><div>As Rich already mentioned it, noauto != remove mount point from fstab. That&#39;s the first thing.</div><div><br></div><div>Then you need to specify that home.mount should
    run *after* md/LVM, which in turn runs after your integrity setup service. The problem is: you have root (/) on md/LVM too; I know they are different RAIDs/volumes, but there is one *service* for RAID and LVM, at least out-of-the-box, in systemd. You
    could try to split different instances using templates (the units with @ in their names), but the simplest solution is to put your integrity logic in the initrd (with the added benefit that you can mount it in emergencies if something goes wrong with
    root).</div><div><br></div><div>If you have the integrity logic in early boot, then your integrity setup service should have:</div></div><div><br></div><div>Before=mdmonitor.service lvm2-lvmetad.service</div><div><br></div><div>And I think that should
    be enough? The home.mount unit depends on the underlying device being available, and that will happen only after md and LVM have done it&#39;s thing.</div><div><br></div><div>Regards.</div><div>--<br></div><div dir="ltr" class="gmail_signature"><div dir="
    ltr"><div><div dir="ltr"><div><font face="arial, helvetica, sans-serif">Dr. Canek Peláez Valdés<br>Profesor de Carrera Asociado C<br>Departamento de Matemáticas<br>Facultad de Ciencias<br>Universidad Nacional Autónoma de México</font></div></div></
    </div></div></div>

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Rich Freeman@21:1/5 to caneko@gmail.com on Sat Sep 25 23:10:02 2021
    On Sat, Sep 25, 2021 at 4:30 PM Canek Peláez Valdés <caneko@gmail.com> wrote:

    the simplest solution is to put your integrity logic in the initrd (with the added benefit that you can mount it in emergencies if something goes wrong with root).


    If you want to try this, consider this article I wrote ages ago. I no
    longer use this but it is a quick recipe for how to create your own
    dracut module: https://rich0gentoo.wordpress.com/2012/01/21/a-quick-dracut-module/

    It is really easy to tweak dracut, and it has a lot of automagic
    functionality which means you mostly only need to worry about whatever
    is particular with your setup. You can also have your module grab any
    config files and binaries/libraries needed off of your filesystem so
    that they are available when it runs.

    --
    Rich

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)