Below steps are validated on Red Hat Enterprise Linux 7
Also in the end of the article a quick example to migrate all my logical volumes to new partition (disk)
- LVM Mirroring
- LVM pvmove command
Method 1: LVM Mirroring
First of all let's check the mapped devices with the logical volumes, here as you see my logical volume lv1 is residing on /dev/sdb1
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert Devices
lv1 rhel -wi-a----- 1.00g /dev/sdb1(0)
root rhel -wi-ao---- <13.10g /dev/sda2(401)
swap rhel -wi-ao---- <1.57g /dev/sda2(0)
This logical volume is mounted on /lv1
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/rhel-root 14G 7.0G 6.2G 54% /
devtmpfs 671M 0 671M 0% /dev
tmpfs 686M 0 686M 0% /dev/shm
tmpfs 686M 8.7M 678M 2% /run
tmpfs 686M 0 686M 0% /sys/fs/cgroup
/dev/sda1 1014M 160M 855M 16% /boot
tmpfs 138M 0 138M 0% /run/user/0
/dev/mapper/rhel-lv1 976M 2.6M 907M 1% /lv1
On this partition I have a single file which we will use to monitor our migration
5dd39cab1c53c2c77cd352983f9641e1 /lv1/myfile
[root@golinuxhub-server ~]# cat /lv1/myfile
This is a test file
Now introduce a new PV where we would like to move our new logical volume. Hence I added a new virtual disk /dev/sdc
[0:0:0:0] cd/dvd VBOX CD-ROM 1.0 /dev/sr0
[1:0:0:0] cd/dvd VBOX CD-ROM 1.0 /dev/sr1
[2:0:0:0] disk ATA VBOX HARDDISK 1.0 /dev/sda
[3:0:0:0] disk ATA VBOX HARDDISK 1.0 /dev/sdb
[4:0:0:0] disk ATA VBOX HARDDISK 1.0 /dev/sdc
So lets create a partition /dev/sdc1 on my newly added disk with partition type as "Linux LVM"
Welcome to fdisk (util-linux 2.23.2).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.
Device does not contain a recognized partition table
Building a new DOS disklabel with disk identifier 0x046a1def.
Command (m for help): n
Partition type:
p primary (0 primary, 0 extended, 4 free)
e extended
Select (default p): p
Partition number (1-4, default 1):
First sector (2048-4194303, default 2048):
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-4194303, default 4194303):
Using default value 4194303
Partition 1 of type Linux and of size 2 GiB is set
Command (m for help): t
Selected partition 1
Hex code (type L to list all codes): 8e
Changed type of partition 'Linux' to 'Linux LVM'
Command (m for help): p
Disk /dev/sdc: 2147 MB, 2147483648 bytes, 4194304 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x046a1def
Device Boot Start End Blocks Id System
/dev/sdc1 2048 4194303 2096128 8e Linux LVM
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
Create a new physical volume using the new partition
Physical volume "/dev/sdc1" successfully created.
Validate the newly created physical volume
PV VG Fmt Attr PSize PFree
/dev/sda2 rhel lvm2 a-- <14.67g 4.00m
/dev/sdb1 rhel lvm2 a-- <1.94g 960.00m
/dev/sdc1 lvm2 --- <2.00g <2.00g
Next extend the "rhel" volume group with the new physical volume
Volume group "rhel" successfully extended
Use "-v" to enable the verbose option and see the list of partitions used for the "rhel" volume group
--- Volume group ---
VG Name rhel
System ID
Format lvm2
Metadata Areas 3
Metadata Sequence No 6
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 3
Open LV 3
Max PV 0
Cur PV 3
Act PV 3
VG Size 18.60 GiB
PE Size 4.00 MiB
Total PE 4762
Alloc PE / Size 4010 / 15.66 GiB
Free PE / Size 752 / <2.94 GiB
VG UUID W9RBxy-be7G-7Mai-unE9-CU1P-os6O-1IrAwg
--- Logical volume ---
LV Path /dev/rhel/swap
LV Name swap
VG Name rhel
LV UUID 5y06cM-RBdD-bP9o-XyTn-vODc-OkdS-1DCCj4
LV Write Access read/write
LV Creation host, time localhost, 2017-08-20 12:35:44 +0530
LV Status available
# open 2
LV Size <1.57 GiB
Current LE 401
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 8192
Block device 253:1
--- Logical volume ---
LV Path /dev/rhel/root
LV Name root
VG Name rhel
LV UUID 8XkPVc-spib-oNu8-3D5E-f1vT-6RpW-ivZvjL
LV Write Access read/write
LV Creation host, time localhost, 2017-08-20 12:35:44 +0530
LV Status available
# open 1
LV Size <13.10 GiB
Current LE 3353
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 8192
Block device 253:0
--- Logical volume ---
LV Path /dev/rhel/lv1
LV Name lv1
VG Name rhel
LV UUID 1B3itY-r46q-LMrz-Cby0-YqCt-XV1G-TmsqMx
LV Write Access read/write
LV Creation host, time golinuxhub-server.example, 2018-04-07 21:22:45 +0530
LV Status available
# open 1
LV Size 1.00 GiB
Current LE 256
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 8192
Block device 253:2
--- Physical volumes ---
PV Name /dev/sda2
PV UUID Qreqy9-GHr6-mlDN-D3ki-NU5k-3VUb-o2g34D
PV Status allocatable
Total PE / Free PE 3755 / 1
PV Name /dev/sdb1
PV UUID VqVvQi-I0BM-epGZ-lhpf-n48v-LM0u-yGGjgK
PV Status allocatable
Total PE / Free PE 496 / 240
PV Name /dev/sdc1
PV UUID 6Rzgfb-NOeK-MT5F-TrQ5-cM6i-si8v-192uWO
PV Status allocatable
Total PE / Free PE 511 / 511
So everything looks correct, next again before starting with our migration last time lets again validate the partition used by our lv1 which is /dev/sdb1
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert Devices
lv1 rhel -wi-ao---- 1.00g /dev/sdb1(0)
root rhel -wi-ao---- <13.10g /dev/sda2(401)
swap rhel -wi-ao---- <1.57g /dev/sda2(0)
The same can be validated using below command
1 dependencies : (8, 17)
As you see we have only single dependency and it is mapped to 8,17 which you see below is for /dev/sdb1
brw-rw---- 1 root disk 8, 0 Apr 7 21:25 sda
brw-rw---- 1 root disk 8, 1 Apr 7 21:25 sda1
brw-rw---- 1 root disk 8, 2 Apr 7 21:31 sda2
brw-rw---- 1 root disk 8, 16 Apr 7 21:25 sdb
brw-rw---- 1 root disk 8, 17 Apr 7 21:31 sdb1
brw-rw---- 1 root disk 8, 32 Apr 7 21:30 sdc
brw-rw---- 1 root disk 8, 33 Apr 7 21:31 sdc1
Let's start with the migration
We will create a single mirror using the below command
Are you sure you want to convert linear LV rhel/lv1 to raid1 with 2 images enhancing resilience? [y/n]: y
Logical volume rhel/lv1 successfully converted.
Let's see if our file is still available
This is a test file
If we see the list of devices, it is a bit different because we are spanning two underlying devices as we are in a mirror formation
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert Devices
lv1 rhel rwi-aor--- 1.00g 100.00 lv1_rimage_0(0),lv1_rimage_1(0)
root rhel -wi-ao---- <13.10g /dev/sda2(401)
swap rhel -wi-ao---- <1.57g /dev/sda2(0)
The dmsetup should also show similar output as expected
4 dependencies : (253, 6) (253, 5) (253, 4) (253, 3)
Now we can break the mirror and get rid of old device which we wanted to remove
Are you sure you want to convert raid1 LV rhel/lv1 to type linear losing all resilience? [y/n]: y
Logical volume rhel/lv1 successfully converted.
So the command executed sucessfully
Here you can see our logical volume lv1 is now residing on /dev/sdc1
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert Devices
lv1 rhel -wi-ao---- 1.00g /dev/sdc1(1)
root rhel -wi-ao---- <13.10g /dev/sda2(401)
swap rhel -wi-ao---- <1.57g /dev/sda2(0)
Same can be validated using dmsetup
1 dependencies : (8, 33)
brw-rw---- 1 root disk 8, 0 Apr 7 21:25 sda
brw-rw---- 1 root disk 8, 1 Apr 7 21:25 sda1
brw-rw---- 1 root disk 8, 2 Apr 7 21:38 sda2
brw-rw---- 1 root disk 8, 16 Apr 7 21:25 sdb
brw-rw---- 1 root disk 8, 17 Apr 7 21:38 sdb1
brw-rw---- 1 root disk 8, 32 Apr 7 21:30 sdc
brw-rw---- 1 root disk 8, 33 Apr 7 21:38 sdc1
The final test is to validate our file and its content which looks same as it was before migration
This is a test file
[root@golinuxhub-server ~]# md5sum /lv1/myfile
5dd39cab1c53c2c77cd352983f9641e1 /lv1/myfile
Now since everything is done we don't need /dev/sdb1 anymore and can be safely removed from "rhel" volume group
Removed "/dev/sdb1" from volume group "rhel"
Validate the same, as you see we no more have /dev/sdb1
VG #PV #LV #SN Attr VSize VFree Devices
rhel 2 3 0 wz--n- 16.66g 1.00g /dev/sda2(0)
rhel 2 3 0 wz--n- 16.66g 1.00g /dev/sda2(401)
rhel 2 3 0 wz--n- 16.66g 1.00g /dev/sdc1(1)
Method 2: Using pvmove
Here I will migrate our logical volume "lv1" from /dev/sdc1 to /dev/sdb1
Let us again extend our volume group with /dev/sdb1
Volume group "rhel" successfully extended
Next monitor the device id using dmsetup and below command
brw-rw---- 1 root disk 8, 0 Apr 7 21:25 sda
brw-rw---- 1 root disk 8, 1 Apr 7 21:25 sda1
brw-rw---- 1 root disk 8, 2 Apr 7 21:40 sda2
brw-rw---- 1 root disk 8, 16 Apr 7 21:25 sdb
brw-rw---- 1 root disk 8, 17 Apr 7 21:40 sdb1
brw-rw---- 1 root disk 8, 32 Apr 7 21:30 sdc
brw-rw---- 1 root disk 8, 33 Apr 7 21:40 sdc1
1 dependencies : (8, 33)
Now time to migrate our logical volume from /dev/sdc1 to /dev/sdb1, the below command may take some time
/dev/sdc1: Moved: 1.56%
/dev/sdc1: Moved: 100.00%
So the migration is completed, validate this using dmsetup
1 dependencies : (8, 17)
[root@golinuxhub-server ~]# ls -l /dev | grep sd
brw-rw---- 1 root disk 8, 0 Apr 7 21:25 sda
brw-rw---- 1 root disk 8, 1 Apr 7 21:25 sda1
brw-rw---- 1 root disk 8, 2 Apr 7 21:40 sda2
brw-rw---- 1 root disk 8, 16 Apr 7 21:25 sdb
brw-rw---- 1 root disk 8, 17 Apr 7 21:40 sdb1
brw-rw---- 1 root disk 8, 32 Apr 7 21:30 sdc
brw-rw---- 1 root disk 8, 33 Apr 7 21:40 sdc1
[root@golinuxhub-server ~]# lvs -o+devices
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert Devices
lv1 rhel -wi-ao---- 1.00g /dev/sdb1(0)
root rhel -wi-ao---- <13.10g /dev/sda2(401)
swap rhel -wi-ao---- <1.57g /dev/sda2(0)
Everything looks perfect..
Lastly again re-validate your file content
This is a test file
[root@golinuxhub-server ~]# md5sum /lv1/myfile
5dd39cab1c53c2c77cd352983f9641e1 /lv1/myfile
A quick example migrating all my lvms to a new device /dev/sdd
Currently as you see my root and swap partition reside on /dev/sda which I intend to move to /dev/sdd
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 15.7G 0 disk
├─sda1 8:1 0 1G 0 part /boot
└─sda2 8:2 0 14.7G 0 part
├─rhel-root 253:0 0 13.1G 0 lvm /
└─rhel-swap 253:1 0 1.6G 0 lvm [SWAP]
sdd 8:48 0 20G 0 disk
sr0 11:0 1 1024M 0 rom
sr1 11:1 1 1024M 0 rom
My volume group currently resides on /dev/sda2
VG #PV #LV #SN Attr VSize VFree Devices
rhel 2 3 0 wz--n- <16.61g 964.00m /dev/sda2(0)
rhel 2 3 0 wz--n- <16.61g 964.00m /dev/sda2(401)
Below are my two logical volumes which I intend to migrate
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
root rhel -wi-ao---- <13.10g
swap rhel -wi-ao---- <1.57g
Extend our "rhel" volume group with the new disk /dev/sdd
Volume group "rhel" successfully extended
[root@golinuxhub-server ~]# pvs
PV VG Fmt Attr PSize PFree
/dev/sda2 rhel lvm2 a-- <14.67g 4.00m
/dev/sdb1 rhel lvm2 a-- <1.94g 960.00m
/dev/sdd rhel lvm2 a-- <20.00g <20.00g
[root@golinuxhub-server ~]# pvmove -n root /dev/sda2 /dev/sdd
/dev/sda2: Moved: 0.03%
/dev/sda2: Moved: 2.15%
/dev/sda2: Moved: 4.18%
/dev/sda2: Moved: 6.38%
/dev/sda2: Moved: 8.89%
/dev/sda2: Moved: 25.80%
/dev/sda2: Moved: 28.48%
/dev/sda2: Moved: 30.72%
/dev/sda2: Moved: 33.28%
/dev/sda2: Moved: 36.00%
/dev/sda2: Moved: 38.74%
/dev/sda2: Moved: 41.40%
/dev/sda2: Moved: 44.32%
/dev/sda2: Moved: 51.95%
/dev/sda2: Moved: 54.61%
/dev/sda2: Moved: 57.20%
/dev/sda2: Moved: 59.89%
/dev/sda2: Moved: 62.75%
/dev/sda2: Moved: 65.64%
/dev/sda2: Moved: 68.39%
/dev/sda2: Moved: 75.57%
/dev/sda2: Moved: 78.53%
/dev/sda2: Moved: 100.00%
[root@golinuxhub-server ~]# pvmove -n swap /dev/sda2 /dev/sdd
/dev/sda2: Moved: 0.00%
/dev/sda2: Moved: 100.00%
As you see my root and swap logical volume is now residing on /dev/sdd
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert Devices
root rhel -wi-ao---- <13.10g /dev/sdd(0)
swap rhel -wi-ao---- <1.57g /dev/sdd(3353)
Same for my volume group which now resides on /dev/sdd
VG #PV #LV #SN Attr VSize VFree Devices
rhel 4 3 0 wz--n- <38.60g 22.93g /dev/sdd(3353)
rhel 4 3 0 wz--n- <38.60g 22.93g /dev/sdd(0)
So both my root and swap partition are now migrated to /dev/sdd
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 15.7G 0 disk
├─sda1 8:1 0 1G 0 part /boot
└─sda2 8:2 0 14.7G 0 part
sdb 8:16 0 2G 0 disk
└─sdb1 8:17 0 2G 0 part
sdd 8:48 0 20G 0 disk
├─rhel-root 253:0 0 13.1G 0 lvm /
└─rhel-swap 253:1 0 1.6G 0 lvm [SWAP]
sr0 11:0 1 1024M 0 rom
sr1 11:1 1 1024M 0 rom
I hope the article was useful.
Hi,
nice to read this article.
I’m missing 2 things that are important for me.
1. There is no explanation on the difference between mirroring and moving. Pros and cons?
2. How to mirror or move a logical volume that is spread over 3 different harddisks to 1 new (bigger) harddisk?
I hope you can clarify on these 2 subjects.
Thanks,
Ron