How to migrate (move) logical volumes and volume groups from one disk to another disk online without reboot in Linux

In this article I will show you two methods to migrate a logical volume /dev/rhel/lv1 from one disk to another without reboot assuming you have the extra disk already available on your system.

Below steps are validated on Red Hat Enterprise Linux 7

Also in the end of the article a quick example to migrate all my logical volumes to new partition (disk)

  • LVM Mirroring
  • LVM pvmove command

Method 1: LVM Mirroring

First of all let's check the mapped devices with the logical volumes, here as you see my logical volume lv1 is residing on /dev/sdb1

[root@golinuxhub-server ~]# lvs -o+devices
  LV   VG   Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert Devices
  lv1  rhel -wi-a-----   1.00g                                                     /dev/sdb1(0)
  root rhel -wi-ao---- <13.10g                                                     /dev/sda2(401)
  swap rhel -wi-ao----  <1.57g                                                     /dev/sda2(0)

This logical volume is mounted on /lv1

[root@golinuxhub-server ~]# df -h
Filesystem             Size  Used Avail Use% Mounted on
/dev/mapper/rhel-root   14G  7.0G  6.2G  54% /
devtmpfs               671M     0  671M   0% /dev
tmpfs                  686M     0  686M   0% /dev/shm
tmpfs                  686M  8.7M  678M   2% /run
tmpfs                  686M     0  686M   0% /sys/fs/cgroup
/dev/sda1             1014M  160M  855M  16% /boot
tmpfs                  138M     0  138M   0% /run/user/0
/dev/mapper/rhel-lv1   976M  2.6M  907M   1% /lv1

On this partition I have a single file which we will use to monitor our migration

[root@golinuxhub-server ~]# md5sum /lv1/myfile
5dd39cab1c53c2c77cd352983f9641e1  /lv1/myfile

[root@golinuxhub-server ~]# cat /lv1/myfile
This is a test file

Now introduce a new PV where we would like to move our new logical volume. Hence I added a new virtual disk /dev/sdc

[root@golinuxhub-server ~]# lsscsi
[0:0:0:0]    cd/dvd  VBOX     CD-ROM           1.0   /dev/sr0
[1:0:0:0]    cd/dvd  VBOX     CD-ROM           1.0   /dev/sr1
[2:0:0:0]    disk    ATA      VBOX HARDDISK    1.0   /dev/sda
[3:0:0:0]    disk    ATA      VBOX HARDDISK    1.0   /dev/sdb
[4:0:0:0]    disk    ATA      VBOX HARDDISK    1.0   /dev/sdc

So lets create a partition /dev/sdc1 on my newly added disk with partition type as "Linux LVM"

[root@golinuxhub-server ~]# fdisk /dev/sdc
Welcome to fdisk (util-linux 2.23.2).

Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table
Building a new DOS disklabel with disk identifier 0x046a1def.

Command (m for help): n
Partition type:
   p   primary (0 primary, 0 extended, 4 free)
   e   extended
Select (default p): p
Partition number (1-4, default 1):
First sector (2048-4194303, default 2048):
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-4194303, default 4194303):
Using default value 4194303
Partition 1 of type Linux and of size 2 GiB is set

Command (m for help): t
Selected partition 1
Hex code (type L to list all codes): 8e
Changed type of partition 'Linux' to 'Linux LVM'

Command (m for help): p

Disk /dev/sdc: 2147 MB, 2147483648 bytes, 4194304 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x046a1def

   Device Boot      Start         End      Blocks   Id  System
/dev/sdc1            2048     4194303     2096128   8e  Linux LVM

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

Create a new physical volume using the new partition

[root@golinuxhub-server ~]# pvcreate /dev/sdc1
  Physical volume "/dev/sdc1" successfully created.

Validate the newly created physical volume

[root@golinuxhub-server ~]# pvs
  PV         VG   Fmt  Attr PSize   PFree
  /dev/sda2  rhel lvm2 a--  <14.67g   4.00m
  /dev/sdb1  rhel lvm2 a--   <1.94g 960.00m
  /dev/sdc1       lvm2 ---   <2.00g  <2.00g

Next extend the "rhel" volume group with the new physical volume

[root@golinuxhub-server ~]# vgextend rhel /dev/sdc1
  Volume group "rhel" successfully extended

Use "-v" to enable the verbose option and see the list of partitions used for the "rhel" volume group

[root@golinuxhub-server ~]# vgdisplay rhel -v
  --- Volume group ---
  VG Name               rhel
  System ID
  Format                lvm2
  Metadata Areas        3
  Metadata Sequence No  6
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                3
  Open LV               3
  Max PV                0
  Cur PV                3
  Act PV                3
  VG Size               18.60 GiB
  PE Size               4.00 MiB
  Total PE              4762
  Alloc PE / Size       4010 / 15.66 GiB
  Free  PE / Size       752 / <2.94 GiB
  VG UUID               W9RBxy-be7G-7Mai-unE9-CU1P-os6O-1IrAwg

  --- Logical volume ---
  LV Path                /dev/rhel/swap
  LV Name                swap
  VG Name                rhel
  LV UUID                5y06cM-RBdD-bP9o-XyTn-vODc-OkdS-1DCCj4
  LV Write Access        read/write
  LV Creation host, time localhost, 2017-08-20 12:35:44 +0530
  LV Status              available
  # open                 2
  LV Size                <1.57 GiB
  Current LE             401
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     8192
  Block device           253:1

  --- Logical volume ---
  LV Path                /dev/rhel/root
  LV Name                root
  VG Name                rhel
  LV UUID                8XkPVc-spib-oNu8-3D5E-f1vT-6RpW-ivZvjL
  LV Write Access        read/write
  LV Creation host, time localhost, 2017-08-20 12:35:44 +0530
  LV Status              available
  # open                 1
  LV Size                <13.10 GiB
  Current LE             3353
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     8192
  Block device           253:0

  --- Logical volume ---
  LV Path                /dev/rhel/lv1
  LV Name                lv1
  VG Name                rhel
  LV UUID                1B3itY-r46q-LMrz-Cby0-YqCt-XV1G-TmsqMx
  LV Write Access        read/write
  LV Creation host, time golinuxhub-server.example, 2018-04-07 21:22:45 +0530
  LV Status              available
  # open                 1
  LV Size                1.00 GiB
  Current LE             256
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     8192
  Block device           253:2

  --- Physical volumes ---
  PV Name               /dev/sda2
  PV UUID               Qreqy9-GHr6-mlDN-D3ki-NU5k-3VUb-o2g34D
  PV Status             allocatable
  Total PE / Free PE    3755 / 1

  PV Name               /dev/sdb1
  PV UUID               VqVvQi-I0BM-epGZ-lhpf-n48v-LM0u-yGGjgK
  PV Status             allocatable
  Total PE / Free PE    496 / 240

  PV Name               /dev/sdc1
  PV UUID               6Rzgfb-NOeK-MT5F-TrQ5-cM6i-si8v-192uWO
  PV Status             allocatable
  Total PE / Free PE    511 / 511

So everything looks correct, next again before starting with our migration last time lets again validate the partition used by our lv1 which is /dev/sdb1

[root@golinuxhub-server ~]# lvs -o+devices
  LV   VG   Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert Devices
  lv1  rhel -wi-ao----   1.00g                                                     /dev/sdb1(0)
  root rhel -wi-ao---- <13.10g                                                     /dev/sda2(401)
  swap rhel -wi-ao----  <1.57g                                                     /dev/sda2(0)

The same can be validated using below command

[root@golinuxhub-server ~]# dmsetup deps /dev/rhel/lv1
1 dependencies  : (8, 17)

As you see we have only single dependency and it is mapped to 8,17 which you see below is for /dev/sdb1

[root@golinuxhub-server ~]# ls -l /dev/ | grep sd
brw-rw---- 1 root disk      8,   0 Apr  7 21:25 sda
brw-rw---- 1 root disk      8,   1 Apr  7 21:25 sda1
brw-rw---- 1 root disk      8,   2 Apr  7 21:31 sda2
brw-rw---- 1 root disk      8,  16 Apr  7 21:25 sdb
brw-rw---- 1 root disk      8,  17 Apr  7 21:31 sdb1
brw-rw---- 1 root disk      8,  32 Apr  7 21:30 sdc
brw-rw---- 1 root disk      8,  33 Apr  7 21:31 sdc1

Let's start with the migration

We will create a single mirror using the below command

[root@golinuxhub-server ~]# lvconvert -m 1 rhel/lv1 /dev/sdc1
Are you sure you want to convert linear LV rhel/lv1 to raid1 with 2 images enhancing resilience? [y/n]: y
  Logical volume rhel/lv1 successfully converted.

Let's see if our file is still available

[root@golinuxhub-server ~]# cat /lv1/myfile
This is a test file

If we see the list of devices, it is a bit different because we are spanning two underlying devices as we are in a mirror formation

[root@golinuxhub-server ~]# lvs -o+devices
  LV   VG   Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert Devices
  lv1  rhel rwi-aor---   1.00g                                    100.00           lv1_rimage_0(0),lv1_rimage_1(0)
  root rhel -wi-ao---- <13.10g                                                     /dev/sda2(401)
  swap rhel -wi-ao----  <1.57g                                                     /dev/sda2(0)

The dmsetup should also show similar output as expected

[root@golinuxhub-server ~]# dmsetup deps /dev/rhel/lv1
4 dependencies  : (253, 6) (253, 5) (253, 4) (253, 3)

Now we can break the mirror and get rid of old device which we wanted to remove

[root@golinuxhub-server ~]# lvconvert -m 0 rhel/lv1 /dev/sdb1
Are you sure you want to convert raid1 LV rhel/lv1 to type linear losing all resilience? [y/n]: y
  Logical volume rhel/lv1 successfully converted.

So the command executed sucessfully

Here you can see our logical volume lv1 is now residing on /dev/sdc1

[root@golinuxhub-server ~]# lvs -o+devices
  LV   VG   Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert Devices
  lv1  rhel -wi-ao----   1.00g                                                     /dev/sdc1(1)
  root rhel -wi-ao---- <13.10g                                                     /dev/sda2(401)
  swap rhel -wi-ao----  <1.57g                                                     /dev/sda2(0)

Same can be validated using dmsetup

[root@golinuxhub-server ~]# dmsetup deps /dev/rhel/lv1
1 dependencies  : (8, 33)

[root@golinuxhub-server ~]# ls -l /dev/ | grep sd
brw-rw---- 1 root disk      8,   0 Apr  7 21:25 sda
brw-rw---- 1 root disk      8,   1 Apr  7 21:25 sda1
brw-rw---- 1 root disk      8,   2 Apr  7 21:38 sda2
brw-rw---- 1 root disk      8,  16 Apr  7 21:25 sdb
brw-rw---- 1 root disk      8,  17 Apr  7 21:38 sdb1
brw-rw---- 1 root disk      8,  32 Apr  7 21:30 sdc
brw-rw---- 1 root disk      8,  33 Apr  7 21:38 sdc1

The final test is to validate our file and its content which looks same as it was before migration

[root@golinuxhub-server ~]# cat /lv1/myfile
This is a test file

[root@golinuxhub-server ~]# md5sum /lv1/myfile
5dd39cab1c53c2c77cd352983f9641e1  /lv1/myfile

Now since everything is done we don't need /dev/sdb1 anymore and can be safely removed from "rhel" volume group

[root@golinuxhub-server ~]# vgreduce rhel /dev/sdb1
  Removed "/dev/sdb1" from volume group "rhel"

Validate the same, as you see we no more have /dev/sdb1

[root@golinuxhub-server ~]# vgs -o+devices
  VG   #PV #LV #SN Attr   VSize  VFree Devices
  rhel   2   3   0 wz--n- 16.66g 1.00g /dev/sda2(0)
  rhel   2   3   0 wz--n- 16.66g 1.00g /dev/sda2(401)
  rhel   2   3   0 wz--n- 16.66g 1.00g /dev/sdc1(1)

Method 2: Using pvmove

Here I will migrate our logical volume "lv1" from /dev/sdc1 to /dev/sdb1

Let us again extend our volume group with /dev/sdb1

[root@golinuxhub-server ~]# vgextend rhel /dev/sdb1
  Volume group "rhel" successfully extended

Next monitor the device id using dmsetup and below command

[root@golinuxhub-server ~]# ls -l /dev | grep sd
brw-rw---- 1 root disk      8,   0 Apr  7 21:25 sda
brw-rw---- 1 root disk      8,   1 Apr  7 21:25 sda1
brw-rw---- 1 root disk      8,   2 Apr  7 21:40 sda2
brw-rw---- 1 root disk      8,  16 Apr  7 21:25 sdb
brw-rw---- 1 root disk      8,  17 Apr  7 21:40 sdb1
brw-rw---- 1 root disk      8,  32 Apr  7 21:30 sdc
brw-rw---- 1 root disk      8,  33 Apr  7 21:40 sdc1

[root@golinuxhub-server ~]# dmsetup deps /dev/rhel/lv1
1 dependencies  : (8, 33)

Now time to migrate our logical volume from /dev/sdc1 to /dev/sdb1, the below command may take some time

[root@golinuxhub-server ~]# pvmove -n lv1 /dev/sdc1 /dev/sdb1
  /dev/sdc1: Moved: 1.56%
  /dev/sdc1: Moved: 100.00%

So the migration is completed, validate this using dmsetup

[root@golinuxhub-server ~]# dmsetup deps /dev/rhel/lv1
1 dependencies  : (8, 17)

[root@golinuxhub-server ~]# ls -l /dev | grep sd
brw-rw---- 1 root disk      8,   0 Apr  7 21:25 sda
brw-rw---- 1 root disk      8,   1 Apr  7 21:25 sda1
brw-rw---- 1 root disk      8,   2 Apr  7 21:40 sda2
brw-rw---- 1 root disk      8,  16 Apr  7 21:25 sdb
brw-rw---- 1 root disk      8,  17 Apr  7 21:40 sdb1
brw-rw---- 1 root disk      8,  32 Apr  7 21:30 sdc
brw-rw---- 1 root disk      8,  33 Apr  7 21:40 sdc1

[root@golinuxhub-server ~]# lvs -o+devices
  LV   VG   Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert Devices
  lv1  rhel -wi-ao----   1.00g                                                     /dev/sdb1(0)
  root rhel -wi-ao---- <13.10g                                                     /dev/sda2(401)
  swap rhel -wi-ao----  <1.57g                                                     /dev/sda2(0)

Everything looks perfect..

Lastly again re-validate your file content

[root@golinuxhub-server ~]# cat /lv1/myfile
This is a test file

[root@golinuxhub-server ~]# md5sum /lv1/myfile
5dd39cab1c53c2c77cd352983f9641e1  /lv1/myfile

A quick example migrating all my lvms to a new device /dev/sdd

Currently as you see my root and swap partition reside on /dev/sda which I intend to move to /dev/sdd

[root@golinuxhub-server ~]# lsblk
NAME          MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda             8:0    0 15.7G  0 disk
├─sda1          8:1    0    1G  0 part /boot
└─sda2          8:2    0 14.7G  0 part
  ├─rhel-root 253:0    0 13.1G  0 lvm  /
  └─rhel-swap 253:1    0  1.6G  0 lvm  [SWAP]
sdd             8:48   0   20G  0 disk
sr0            11:0    1 1024M  0 rom
sr1            11:1    1 1024M  0 rom

My volume group currently resides on /dev/sda2

[root@golinuxhub-server ~]# vgs -o+devices
  VG   #PV #LV #SN Attr   VSize   VFree   Devices
  rhel   2   3   0 wz--n- <16.61g 964.00m /dev/sda2(0)
  rhel   2   3   0 wz--n- <16.61g 964.00m /dev/sda2(401)

Below are my two logical volumes which I intend to migrate

[root@golinuxhub-server ~]# lvs
  LV   VG   Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  root rhel -wi-ao---- <13.10g
  swap rhel -wi-ao----  <1.57g

Extend our "rhel" volume group with the new disk /dev/sdd

[root@golinuxhub-server ~]# vgextend rhel /dev/sdd
  Volume group "rhel" successfully extended

[root@golinuxhub-server ~]# pvs
  PV         VG   Fmt  Attr PSize   PFree
  /dev/sda2  rhel lvm2 a--  <14.67g   4.00m
  /dev/sdb1  rhel lvm2 a--   <1.94g 960.00m
  /dev/sdd   rhel lvm2 a--  <20.00g <20.00g

[root@golinuxhub-server ~]# pvmove -n root /dev/sda2 /dev/sdd
  /dev/sda2: Moved: 0.03%
  /dev/sda2: Moved: 2.15%
  /dev/sda2: Moved: 4.18%
  /dev/sda2: Moved: 6.38%
  /dev/sda2: Moved: 8.89%
  /dev/sda2: Moved: 25.80%
  /dev/sda2: Moved: 28.48%
  /dev/sda2: Moved: 30.72%
  /dev/sda2: Moved: 33.28%
  /dev/sda2: Moved: 36.00%
  /dev/sda2: Moved: 38.74%
  /dev/sda2: Moved: 41.40%
  /dev/sda2: Moved: 44.32%
  /dev/sda2: Moved: 51.95%
  /dev/sda2: Moved: 54.61%
  /dev/sda2: Moved: 57.20%
  /dev/sda2: Moved: 59.89%
  /dev/sda2: Moved: 62.75%
  /dev/sda2: Moved: 65.64%
  /dev/sda2: Moved: 68.39%
  /dev/sda2: Moved: 75.57%
  /dev/sda2: Moved: 78.53%
  /dev/sda2: Moved: 100.00%

[root@golinuxhub-server ~]# pvmove -n swap /dev/sda2 /dev/sdd
  /dev/sda2: Moved: 0.00%
  /dev/sda2: Moved: 100.00%

As you see my root and swap logical volume is now residing on /dev/sdd

[root@golinuxhub-server ~]# lvs -o+devices
  LV   VG   Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert Devices
  root rhel -wi-ao---- <13.10g                                                     /dev/sdd(0)
  swap rhel -wi-ao----  <1.57g                                                     /dev/sdd(3353)

Same for my volume group which now resides on /dev/sdd

[root@golinuxhub-server ~]# vgs -o+devices
  VG   #PV #LV #SN Attr   VSize   VFree  Devices
  rhel   4   3   0 wz--n- <38.60g 22.93g /dev/sdd(3353)
  rhel   4   3   0 wz--n- <38.60g 22.93g /dev/sdd(0)

So both my root and swap partition are now migrated to /dev/sdd

[root@golinuxhub-server ~]# lsblk
NAME         MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda            8:0    0 15.7G  0 disk
├─sda1         8:1    0    1G  0 part /boot
└─sda2         8:2    0 14.7G  0 part
sdb            8:16   0    2G  0 disk
└─sdb1         8:17   0    2G  0 part
sdd            8:48   0   20G  0 disk
├─rhel-root  253:0    0 13.1G  0 lvm  /
└─rhel-swap  253:1    0  1.6G  0 lvm  [SWAP]
sr0           11:0    1 1024M  0 rom
sr1           11:1    1 1024M  0 rom

I hope the article was useful.

1 thought on “How to migrate (move) logical volumes and volume groups from one disk to another disk online without reboot in Linux”

  1. Hi,
    nice to read this article.
    I’m missing 2 things that are important for me.
    1. There is no explanation on the difference between mirroring and moving. Pros and cons?
    2. How to mirror or move a logical volume that is spread over 3 different harddisks to 1 new (bigger) harddisk?
    I hope you can clarify on these 2 subjects.
    Thanks,
    Ron

    Reply

Leave a Comment