Howdy,

I thought I had done this before but it appears I have not.  I want to
replace a 8TB drive with a 10TB drive and then remove the 8TB drive when
done but these drives are encrypted with cryptsetup on top of LVM.  I
looked through my how tos and can't find one for encrypted drives.  I
did find them for regular LVM and have done that in the past.  While I
have fresh backups, I don't want to test them if not needed.  Does this
work just like with LVM or does the encryption change how this is done
or even if it can be done?

I googled and can't really find a good how to.  Does anyone have a link
to a 'how to' to do this with encrypted stuff, have notes with the
sequence of commands that I can go by or is this not doable when it is encrypted? 

Thanks.

Dale

:-)  :-) 

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Dale wrote:

Howdy,

I thought I had done this before but it appears I have not.  I want to replace a 8TB drive with a 10TB drive and then remove the 8TB drive when
done but these drives are encrypted with cryptsetup on top of LVM.  I
looked through my how tos and can't find one for encrypted drives.  I
did find them for regular LVM and have done that in the past.  While I
have fresh backups, I don't want to test them if not needed.  Does this
work just like with LVM or does the encryption change how this is done
or even if it can be done?

I googled and can't really find a good how to.  Does anyone have a link
to a 'how to' to do this with encrypted stuff, have notes with the
sequence of commands that I can go by or is this not doable when it is encrypted? 

Thanks.

Dale

:-)  :-) 

Since my previous post, I've continued to look for a way to do this on
google, youtube and any place else I can think of.  So far, I have yet
to find a way.  Given no one has replied with info, I'm assuming that no
one here knows or can find a way either.  Since this involves LVM which
I know has this ability, it appears that the encryption part complicates matters but it should be doable, I have two drives in the current setup
so I at least added a drive once before.  So, I updated my backups,
stopped any changes to the affected LVM and set out to test my theory. 
This is a list of commands in sequence that are from my notes, you may
recall my memory isn't great.  lol 

To replace drive on encrypted LVM. 
pvcreate /dev/sd<*new* drive or partition>
vgextend <group name> /dev/sd<*new* pvdevice above>
pvmove /dev/sd<drive being *removed*>  This takes a while. ;-)

vgreduce crypt /dev/sd<drive being *removed*>
pvremove /dev/sd<drive being *removed*>

To add extra/remaining space on replacement drive if any
vgdisplay <name of vg> and get unused space or PE available if using
--extents option or use 100% option
lvresize -l +100%FREE /dev/<pv name>/vg name> EXAMPLE
lvresize -l +100%FREE /dev/crypt/crypt

If LUKS, resize LUKS with
cryptsetup resize <name of encrypted>
or close and reopen.

resize2fs -p /dev/mapper/<name of lv which should be same as mount
point>  EXAMPLE
resize2fs -p /dev/mapper/crypt

Once this was done, I have a previous 8TB and a larger 10TB drive which
gives me 17TBs of space now.  I plan to buy a 12 or 14TB drive to
replace existing 8TB.  I should be done for a while. 

Now to replace my /home drive which is also close to full.  It's not
encrypted tho. The biggest difference in this and plain LVM, resizing
with cryptsetup or close and reopen.  Keep in mind, while I did all
this, LUKS, cryptsetup, whatever was open.  I'm not sure about this
being done while closed.  I did find one mention that it had to be open
at certain points. 

By the way, if someone wants to take this and make a how to out of it somewhere, I think it would be great.  This is doable.  I even rebooted
and even tho the drives changed SATA ports, it worked fine.  So, I guess
I did it right, even if no one thought it could be done.  lol 

I really hope this helps someone else.  My brain hurts a little.  :/

Dale

:-)  :-) 

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Wed, Sep 14, 2022 at 1:40 AM Dale <rdalek1967@gmail.com> wrote:

Now to replace my /home drive which is also close to full. It's not encrypted tho. The biggest difference in this and plain LVM, resizing
with cryptsetup or close and reopen. Keep in mind, while I did all
this, LUKS, cryptsetup, whatever was open. I'm not sure about this
being done while closed. I did find one mention that it had to be open
at certain points.

If you want to do operations on a DM volume, then it needs to exist,
which in the cryptsetup world means "open."

Most of the details of how to do an operation like moving around
encrypted volumes depends on the layers you're using (LUKS, LVM,
mdadm, whatever), and how they're stacked. If you're running LVM on
top of LUKS then you create new LUKS volumes and add them to LVM and
move stuff around. If you're running LUKS on top of LVM then you add unencrypted volumes to LVM, extend the LV underlying LUKS, and then
resize the LUKS volume. Last comes resizing whatever filesystems are
on top if desired.

When you want to get advice around these kinds of operations it is
usually necessary to detail exactly how things are set up, because to
Linux they're just layers and you can stack them however you want. A
script that adds a hard drive to your setup will look different if
you're running mdadm on the bottom vs LUKS vs LVM and so on.

Plus if you do things in the wrong order there is actually a chance
that it will work and leave you with half your filesystem on an
encrypted drive and half of it on an unencrypted one, or whatever. DM
doesn't care - it just shuffles around blocks and transforms them as
told, and as long as you stack things up the same way every time it
should still work. It just won't necessarily be doing what you want
it to...

--
Rich

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Am Wed, Sep 14, 2022 at 12:40:21AM -0500 schrieb Dale:

Now to replace my /home drive which is also close to full.  It's not encrypted tho. The biggest difference in this and plain LVM, resizing
with cryptsetup or close and reopen.  Keep in mind, while I did all
this, LUKS, cryptsetup, whatever was open.

I recently learnt that you can encrypt an unencrypted devices on-the-fly.
Kinda neat. It is also resilient to hard interruptions like power-offs.
See man cryptsetup-reencrypt.

But if you want to replace the drive anyways, it may be faster to set up a
new device and rsync the files in the last step. I always use these kinds of opportunities to weed out old cruft on my filesystem.

--
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

Just because I wrote a word wrong doesn’t mean that I am imcomentent.

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEVbE9o2D2lE5fhoVsizG+tUDUMMoFAmMhzhkACgkQizG+tUDU MMp6qQ/+NBuPnmNnvPH+wIbHrCuV9JwK2P64D/SzFtvFWhJTzQoC0ha7XPhUs3Yp 86PQn+DvC30dQ7grirzS11oLaQRxu6XLMjWlmw8x+YqjvP4eNQvGY7LEtbA8BONw lreRB1tNZ82KxiN8OUTulgmbOKthTXxNQWB5yfGquA0F7BW+4OEFDxrztFnxiKkC YvltP+tkWX6BM5p4B27h4bdCdTMIXieWM8ankS8xdIxPSnszG7LhcjRdR9ICIhk/ rZzxtBTmFRHEWn39+sle8MSvQ37IIqEkHORzpDOFW37nMw/vNPS+GuEiPjeBU28H sF/Oe0WY+X30Yg3Awfi365gPgpIEYnFQgekCWeIvtKGBbvdUlbUwREq0aAexgJ6u UqB/KtNWQsxXvOHpgh1hSF7JHxbzOw3NQ42tJc5X6wJaWOwTlXbjPaKJsmO2w4gj xe8N92StCjF2aAEBzpJCGM2tPeR1KMeQ3lLLZLiUNV5zCW+VTLMuHBscfA4nL9nT aMKR/1PwIm/s4KCk1BK3UV3G8g85EHr5BQfzPO7LcbiYUj2zIzOePr8DL11vj4rb fXmVtXMVrPw5TkjlqtAcnH84jUz+gbE5lXfS972rjfnYvXMS/PqK8r+7yLLUxlne 5sa63DO9xxQAnbXI34OFio34HogVrFTkZ0dQ1KdnNMzV/8Ts2bs=
=K6J0
-----E

Rich Freeman wrote:

On Wed, Sep 14, 2022 at 1:40 AM Dale <rdalek1967@gmail.com> wrote:

Now to replace my /home drive which is also close to full. It's not
encrypted tho. The biggest difference in this and plain LVM, resizing
with cryptsetup or close and reopen. Keep in mind, while I did all
this, LUKS, cryptsetup, whatever was open. I'm not sure about this
being done while closed. I did find one mention that it had to be open
at certain points.

If you want to do operations on a DM volume, then it needs to exist,
which in the cryptsetup world means "open."

Most of the details of how to do an operation like moving around
encrypted volumes depends on the layers you're using (LUKS, LVM,
mdadm, whatever), and how they're stacked. If you're running LVM on
top of LUKS then you create new LUKS volumes and add them to LVM and
move stuff around. If you're running LUKS on top of LVM then you add unencrypted volumes to LVM, extend the LV underlying LUKS, and then
resize the LUKS volume. Last comes resizing whatever filesystems are
on top if desired.

When you want to get advice around these kinds of operations it is
usually necessary to detail exactly how things are set up, because to
Linux they're just layers and you can stack them however you want. A
script that adds a hard drive to your setup will look different if
you're running mdadm on the bottom vs LUKS vs LVM and so on.

Plus if you do things in the wrong order there is actually a chance
that it will work and leave you with half your filesystem on an
encrypted drive and half of it on an unencrypted one, or whatever. DM doesn't care - it just shuffles around blocks and transforms them as
told, and as long as you stack things up the same way every time it
should still work. It just won't necessarily be doing what you want
it to...

I see the point but wasn't aware there was more than one way to do it
with cryptsetup.  It seems there is several options for this.  I was
pretty sure LVM was on bottom and mentioned it in my original post. 
After reading your post, I got to wondering, did I do this the right
way?  So, I started looking to see how to tell for sure.  I used several
LVM type commands but didn't see anything that I recognized anyway. 
Keep in mind, I'm not real sure what I'm looking for either. Then I ran
lsblk -f and found a clue that I've never noticed before. 


sdd                                                                                                    

└─sdd1              LVM2_member LVM2 001          pVnP2i-sj48-3co9-nJpa-9tQr-08pa-9JqASR               
  └─crypt-crypt     crypto_LUKS 2                 6e884aae-9377-49ef-a602-e13cba89a377                 
    └─crypt         ext4        1.0      crypt    76653316-329f-4747-8fed-fc9b1723bd14      3.5T    79% /home/dale/Desktop/Crypt


I know that is going to be line wrapped and mess up things but the part
I noticed was the drive partition "sdd1" and "LVM2 member".  On top of
that is crypto.  So, LVM is on bottom.  If that is the case, my pvmove command should be moving what I think you call "raw data", doesn't
matter if it is encrypted or not, right?  Either way, other than
resizing the files system, I was dealing with LVM layers.  Just in case
it matters, could I have done everything but the file system resize
while it was closed?  It seems it is basically encrypted on the layer
just below the file system to me. 

Your post helped me to have a better idea of what I got here.  By all
means, point out if I got anything wrong.  Biggest thing, it seems to
have worked.  Still, I need more drive space.  :/

Thanks.

Dale

:-)  :-) 

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Am Wed, Sep 14, 2022 at 08:55:26AM -0500 schrieb Dale:

I see the point but wasn't aware there was more than one way to do it
with cryptsetup.  It seems there is several options for this.  I was
pretty sure LVM was on bottom and mentioned it in my original post. 

Indeed you did and it confused me at first. Then I gave it some thought and concluded: why not?

You do it like so:
Block device --,
Block device --+-- LVM --- LUKS --- File system
Block device --'

After reading your post, I got to wondering, did I do this the right
way?

Your advantage: only one LUKS header to take care of. That means no extra
crypt management when adding or removing disks, except for resizing the
crypt volume. And there is only a single place of storage for your keys (in case you ever need to change them).

I’m not sure whether it’s the right™ way. It is *one* way. Perhaps there are
drawbacks that I can’t think of right now.


I would typically have done:
Block device --- LUKS --,
Block device --- LUKS --+-- LVM --- File system
Block device --- LUKS --'

That’s how my NAS works at the moment (with ZFS instead of LVM + filesystem). But that’s because ZFS didn’t have built-in encryption when I set it up some
years ago. These days I would do:

Block device --,
Block device --+-- ZFS
Block device --'

That’s it. :D Encryption, disk arrays and file system all in one shop.

So, I started looking to see how to tell for sure.  I used several
LVM type commands but didn't see anything that I recognized anyway. 
Keep in mind, I'm not real sure what I'm looking for either. Then I ran
lsblk -f and found a clue that I've never noticed before. 

sdd                                                                                                    

└─sdd1              LVM2_member LVM2 001          pVnP2i-sj48-3co9-nJpa-9tQr-08pa-9JqASR               
  └─crypt-crypt     crypto_LUKS 2                 6e884aae-9377-49ef-a602-e13cba89a377                 
    └─crypt         ext4        1.0      crypt   
76653316-329f-4747-8fed-fc9b1723bd14      3.5T    79% /home/dale/Desktop/Crypt

I know that is going to be line wrapped and mess up things

You could have redacted the long UUIDs which aren’t relavant anyways. I write my mail in mutt and vim, thus I can rewrap paragraphs individually and at will. That way I can paint ASCII art, paste over-long console output or write one-line paragraphs
like this one. ;-)

but the part I noticed was the drive partition "sdd1" and "LVM2 member". 
On top of that is crypto.  So, LVM is on bottom.  If that is the case, my pvmove command should be moving what I think you call "raw data", doesn't matter if it is encrypted or not, right?

Yup. This kind of layering is one of the big beauty of Linux for me. It’s
all interchangable and layer X doesn’t care what layer X+1 is doing and vice versa.

Just in case it matters, could I have done everything but the file system resize while it was closed?  It seems it is basically encrypted on the
layer just below the file system to me. 

I think so, yes.


PS.: All your LVM threads made me embrace LVM on my PC when I recently
switched it from SATA to NVMe. And because after many years of ignorance, I finally had an actual use case: my laptop’s root partition became too small and I had to give it some space from the data partition. In my early Gentoo years I didn’t use an initrd and didn’t want to, so LVM was never an option.
But when I set up the (then brand-new) laptop, I used Sakaki’s howto for full-disk encryption, which used an initrd + LVM anyways. This saved the
SSD from a full reformat and rewrite.

--
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

The longer it rains, the better the prospect of nicer weather.

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEVbE9o2D2lE5fhoVsizG+tUDUMMoFAmMh7TEACgkQizG+tUDU MMrLQA/6A9tsOKbr9bcqsAffjXn8T5W1htKADPFn+GE8nq7h5lRWPHDokuXkzc6I 7uvRpwbUyoUpTxz5/nWfy8qhNzuUk9clj4h5iHGUTcUIIVmanvUTLEnC4w31B4aB NdllBehxoeTgSnbBgKlNQcQvLTK9uMEYtzfkGHY4iack1cJpf6sescrkbf711kzd bxz8qoCnOV+4nmDNLp8oGc2hpB6EMzyNXpKhxRmNErj/HXLi+YCHRAW6JGGE84lL jfZpV6leik372gQpBN80y8B7l/3n0FwlVsNiifY3pzfKDKIfi7NEjosXMs0u35o5 MVUYE9AYplPeOfk0W9z4EGfxO7HmZigYBu8TZDcXpjyhg6HXshZlTjmR404TVmJC yPtldWKXApGZPcma/JLBXHYzexPGxCEblrwt2dwCIYLaZFjbGl591QDa6qzlZMFC 1kZeHb6R53dEdMOaobYENR8uI3TPJnfANaq6vBjiNdZhMg627uHIgrAKwN/orOfw fAlZr+/O5uFuoRNo3mcmn/+VUC7hJmimYpy+Qz21uU9GYlxK2SEvyq3EwAve/c4h 54wk0eOw1YdansQb1ZWQPvAT4+cqZBIQ8rafcO5ywrDKHzSLChWZeXrbJCMLmRGm lDAfZvgu71EKMW+7F0nRroEMojySJozNCMBjneZkssWWjAEgERs=
=j8Tr
-----END PGP SI

Frank Steinmetzger wrote:

Am Wed, Sep 14, 2022 at 08:55:26AM -0500 schrieb Dale:

I see the point but wasn't aware there was more than one way to do it
with cryptsetup.  It seems there is several options for this.  I was
pretty sure LVM was on bottom and mentioned it in my original post. 

Indeed you did and it confused me at first. Then I gave it some thought and concluded: why not?

You do it like so:
Block device --,
Block device --+-- LVM --- LUKS --- File system
Block device --'

After reading your post, I got to wondering, did I do this the right
way?

Your advantage: only one LUKS header to take care of. That means no extra crypt management when adding or removing disks, except for resizing the
crypt volume. And there is only a single place of storage for your keys (in case you ever need to change them).

I’m not sure whether it’s the right™ way. It is *one* way. Perhaps there are
drawbacks that I can’t think of right now.

By right way, I was talking about the way I moved things around. It
seems to be working so I guess I did it the right way, for this setup
anyway.  I might have broke things if it was set up some other way. 

I would typically have done:
Block device --- LUKS --,
Block device --- LUKS --+-- LVM --- File system
Block device --- LUKS --'

That’s how my NAS works at the moment (with ZFS instead of LVM + filesystem).
But that’s because ZFS didn’t have built-in encryption when I set it up some
years ago. These days I would do:

Block device --,
Block device --+-- ZFS
Block device --'

That’s it. :D Encryption, disk arrays and file system all in one shop.

So, I started looking to see how to tell for sure.  I used several
LVM type commands but didn't see anything that I recognized anyway. 
Keep in mind, I'm not real sure what I'm looking for either. Then I ran
lsblk -f and found a clue that I've never noticed before. 


sdd                                                                                                    

└─sdd1              LVM2_member LVM2 001          >> pVnP2i-sj48-3co9-nJpa-9tQr-08pa-9JqASR               
  └─crypt-crypt     crypto_LUKS 2                 >> 6e884aae-9377-49ef-a602-e13cba89a377                 
    └─crypt         ext4        1.0      crypt   
76653316-329f-4747-8fed-fc9b1723bd14      3.5T    79%
/home/dale/Desktop/Crypt


I know that is going to be line wrapped and mess up things

You could have redacted the long UUIDs which aren’t relavant anyways. I write my mail in mutt and vim, thus I can rewrap paragraphs individually and at will. That way I can paint ASCII art, paste over-long console output or write one-line paragraphs

like this one. ;-)

I started to but thought it would cause yet another issue.  ROFL 

but the part I noticed was the drive partition "sdd1" and "LVM2 member".  >> On top of that is crypto.  So, LVM is on bottom.  If that is the case, my >> pvmove command should be moving what I think you call "raw data", doesn't
matter if it is encrypted or not, right?

Yup. This kind of layering is one of the big beauty of Linux for me. It’s all interchangable and layer X doesn’t care what layer X+1 is doing and vice
versa.

Just in case it matters, could I have done everything but the file system
resize while it was closed?  It seems it is basically encrypted on the
layer just below the file system to me. 

I think so, yes.

PS.: All your LVM threads made me embrace LVM on my PC when I recently switched it from SATA to NVMe. And because after many years of ignorance, I finally had an actual use case: my laptop’s root partition became too small and I had to give it some space from the data partition. In my early Gentoo years I didn’t use an initrd and didn’t want to, so LVM was never an option.
But when I set up the (then brand-new) laptop, I used Sakaki’s howto for full-disk encryption, which used an initrd + LVM anyways. This saved the
SSD from a full reformat and rewrite.

I have to say, before I started using LVM, it was very hard to change
storage up.  As it is, /boot and / are on a traditional partition, no
LVM.  Everything else is on LVM.  At the time, I was avoiding the init thingy.  Since I have /usr separate, and /var, I ended up having to use
one anyway but dracut came along.  Only gave me a problem once so
overall, pretty good. 

I'm not sure who got me started on LVM.  Could be Alan who isn't on this
list anymore, Neil or someone else.  It's been a long long time ago.  I
tell you what tho, it has made things a LOT easier.  I've read about btrfs(sp?).  It could be the next big thing but I'll likely stick with
LVM as long as it works for me.  ZFS seems to be getting popular too.
I'm not sure who is winning the race between them tho.  I think they all
each have their use case. 

My biggest thing, I couldn't find hardly anything to do that with it
encrypted and what I did find was not very clear.  I couldn't imagine it
not being doable tho.  After all, this is Linux.  :-) 

I'm just glad it worked out and if anyone runs up on this thread, this additional info will help someone else.  Now, people can see what the
setup is.  See if it matches what they have or not. 

Dale

:-)  :-) 

P. S.  My second move is almost 90% done with pvmove. 

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 14/09/2022 14:55, Dale wrote:

I see the point but wasn't aware there was more than one way to do it
with cryptsetup.  It seems there is several options for this.  I was
pretty sure LVM was on bottom and mentioned it in my original post.
After reading your post, I got to wondering, did I do this the right
way?  So, I started looking to see how to tell for sure.  I used several LVM type commands but didn't see anything that I recognized anyway.
Keep in mind, I'm not real sure what I'm looking for either. Then I ran
lsblk -f and found a clue that I've never noticed before.

https://raid.wiki.kernel.org/index.php/Asking_for_help#lsdrv

Try running this utility. Note that it's written in Python 2, and
unfortunately doesn't work with Python 3 ...

It'll tell you EXACTLY what you've got, provided it knows about the sort
of layers. (It certainly knows about raid and LVM, probably LUKS too ...)

Cheers,
Wol

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Laurence Perkins wrote:

By the way, if someone wants to take this and make a how to out of it
somewhere, I think it would be great. This is doable. I even rebooted
and even tho the drives changed SATA ports, it worked fine. So, I guess
I did it right, even if no one thought it could be done. lol

I really hope this helps someone else. My brain hurts a little. :/

Dale

:-) :-)

If you want, you can slap it into your personal page on the wiki and then it will show up in searches when people go looking for how to do things with LVM and/or cryptsetup.

LMP

I don't have a personal page.  Not that I know of anyway.  Plus, I
haven't wrote a how to in ages. 

Dale

:-)  :-) 

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Laurence Perkins wrote:

By the way, if someone wants to take this and make a how to out of it
somewhere, I think it would be great. This is doable. I even rebooted
and even tho the drives changed SATA ports, it worked fine. So, I guess
I did it right, even if no one thought it could be done. lol

I really hope this helps someone else. My brain hurts a little. :/

Dale

:-) :-)

If you want, you can slap it into your personal page on the wiki and then it will show up in searches when people go looking for how to do things with LVM and/or cryptsetup.

LMP

Well, I know now.  For future reference tho, you can also tell from the
output of lsblk -f which I think is installed as part of @system.  It
shows the layers.  Funny tho, I've used lsblk to get UUIDs for a long
while and never once noticed it shows that.  It was after Rich posted
that I noticed it.  I was looking for UUIDs and that was all I saw I
guess.  lol

Always good to have more than one tool to do something tho.  ;-)

Dale

:-)  :-) 

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)