# LVM

LVM 磁碟管理

# Learning

##### 單個磁碟擴充未分割磁區

- [How to Resize LVM Partition Inside an Extended Partition](https://linuxhandbook.com/resize-lvm-partition/)

##### LVM Snapshot

- [Use LVM Snapshot To Backup Your Data In Linux - OSTechNix](https://ostechnix.com/how-to-use-lvm-snapshot-to-backup-your-data-in-linux/)

# 常用指令

#### Logical Volume - LV

```shell
# 顯示所有 LV 使用狀況
lvdisplay -C

# 新增 LV
lvcreate -L 10G -n LV_name VG_name

# 刪除 LV
lvremove /dev/VG_name>/LV_name

# 更名 LV
lvrename /dev/VG-name/old-LV-name /dev/VG-name/new-LV-name

# 顯示 LV 狀況
lvs
lvs -a -o name,copy_percent,devices
lvs -a -o name,copy_percent,devices <vg-name>
lvs -a --segments -o +devices

# 擴充 LV 使用空間
lvextend -L +2G /dev/vg/lv
lvextend -l +100%FREE /dev/vg_db2v9/lv_root  將剩餘空間都加上

ext2online /dev/vg/lv (RHEL v4)
resize2fs /dev/vg/lv (RHEL v5,6) 
xfs_growfs /dev/vg/lv (RHEL v7 with XFS filesystem)
btrfs filesystem resize max <mount-point>  (Fedora 40)

# 縮小 LV 空間 (RHEL v4)
#說明：445GB -> 2GB
umount /worktmp
e2fsck -f /dev/rootVG/worktmpLV
resize2fs /dev/rootVG/worktmpLV 1843M
lvreduce -L 2GB /dev/rootVG/worktmpLV
resize2fs /dev/rootVG/worktmpLV
mount /worktmp 
#註：1843MB 大約是要 2GB 的 90%，這裡的指令也能使用單位 GB

# Shrinking LV (RHEL 5/6)
#100 GB -> 5GB
umount /opt/oracle/arclog
lvreduce -L 5G /dev/VolGroup00/arclogLV
resize2fs /dev/VolGroup00/arclogLV 5G
e2fsck -f /dev/VolGroup00/arclogLV
mount /opt/oracle/arclog
```

#### Volume Group - VG

```shell
# 顯示 VG 使用狀況
vgdisplay -C

# 啟動/關閉 VG
vgchange -a y VG_name
vgchange -a n VG_name
#註：關閉後，執行 vgdisplay 仍會顯示，但 /dev/*** 已不存在。

# 新建 VG
vgcreate VG_name /dev/pv1 /dev/pv2 

# 更名 VG
vgrename vg_esxa01db01 vg_root

# 刪除 VG
vgremove VG_name

# 增加 PV 到 VG
vgextend VG_name /dev/pv3 

# 從 VG 中移除 PV
#NOTE: PV 必須沒有任何資料(檢查指令 pvs -o+pv_used)
vgreduce VG_name /dev/mypv 
```

#### Physical Volume - PV

```shell
# 顯示 PV 使用狀況
pvdisplay -C

# 檢查所有 PV 的空間使用狀況
pvs -o+pv_used

# 新增 PV
pvcreate /dev/hdd1 

# 刪除 PV
pvremove /dev/hdd1 

# 如果整個 disk 都要作為 PV，該磁碟不能建立任何磁區，要清除舊磁區可以執行
dd if=/dev/zero of=/dev/sdd bs=512 count=1 

# 既有的 LUN Disk 增加磁碟空間後，還需要執行
pvresize /dev/sd[X]
```

##### LVM 磁區的磁碟加大

> 在 RHEL 8 下，移除 LVM 磁區時，可能出現提示:
> 
> Do you want to remove the signature? \[Y\]es/\[N\]o:
> 
> 注意: 這裡必須是 N，否則原有的磁區會被徹底清除。
> 
> 技巧: 要確認 LVM 磁區是有完成延伸，或者不小心被移除，在 reboot 前執行
> 
> pvs
> 
> 如果有顯示原有的 PV disk，表示磁區延伸成功。

Extend PV Disk: /dev/sda2 50GB to 100GB

```
[root@my-db2v11fp7 ~]# lsblk
NAME               MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda                  8:0    0  100G  0 disk
├─sda1               8:1    0    1G  0 part /boot
└─sda2               8:2    0   49G  0 part
  ├─rootvg-root    253:0    0   15G  0 lvm  /
  ├─rootvg-swap    253:1    0    4G  0 lvm  [SWAP]
  └─rootvg-worktmp 253:2    0  512M  0 lvm  /worktmp
sr0                 11:0    1 1024M  0 rom

[root@my-db2v11fp7 ~]# pvs
  PV         VG     Fmt  Attr PSize   PFree
  /dev/sda2  rootvg lvm2 a--  <49.00g <29.50g
  
[root@my-db2v11fp7 ~]# fdisk -ul /dev/sda
Command (m for help): p

Disk /dev/sda: 107.4 GB, 107374182400 bytes, 209715200 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x000def6b

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048     2099199     1048576   83  Linux
/dev/sda2         2099200   104857599    51379200   8e  Linux LVM

Command (m for help): d
Partition number (1,2, default 2): 2
Partition 2 is deleted

Command (m for help): p

Disk /dev/sda: 107.4 GB, 107374182400 bytes, 209715200 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x000def6b

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048     2099199     1048576   83  Linux

Command (m for help): n
Partition type:
   p   primary (1 primary, 0 extended, 3 free)
   e   extended
Select (default p): p
Partition number (2-4, default 2): 2
First sector (2099200-209715199, default 2099200):
Using default value 2099200
Last sector, +sectors or +size{K,M,G} (2099200-209715199, default 209715199):
Using default value 209715199
Partition 2 of type Linux and of size 99 GiB is set

Command (m for help): t
Partition number (1,2, default 2): 2
Hex code (type L to list all codes): 8e
Changed type of partition 'Linux' to 'Linux LVM'

Command (m for help): p

Disk /dev/sda: 107.4 GB, 107374182400 bytes, 209715200 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x000def6b

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048     2099199     1048576   83  Linux
/dev/sda2         2099200   209715199   103808000   8e  Linux LVM

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.

WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table. The new table will be used at
the next reboot or after you run partprobe(8) or kpartx(8)
Syncing disks.


[root@my-db2v11fp7 ~]# reboot


[root@my-db2v11fp7 ~]# lsblk
NAME               MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda                  8:0    0  100G  0 disk
├─sda1               8:1    0    1G  0 part /boot
└─sda2               8:2    0   99G  0 part
  ├─rootvg-root    253:0    0   15G  0 lvm  /
  ├─rootvg-swap    253:1    0    4G  0 lvm  [SWAP]
  └─rootvg-worktmp 253:2    0  512M  0 lvm  /worktmp
sr0                 11:0    1 1024M  0 rom

[root@my-db2v11fp7 ~]# pvresize /dev/sda2
  Physical volume "/dev/sda2" changed
  1 physical volume(s) resized or updated / 0 physical volume(s) not resized
[root@my-db2v11fp7 ~]# pvs
  PV         VG     Fmt  Attr PSize   PFree
  /dev/sda2  rootvg lvm2 a--  <99.00g <79.50g

```

##### 指令速查表

LVM

[![linux_LVM_small.jpg](https://osslab.tw/uploads/images/gallery/2023-10/scaled-1680-/linux-lvm-small.jpg)](https://osslab.tw/uploads/images/gallery/2023-10/linux-lvm-small.jpg)

# FAQ

##### 無法新增 PV

> Device /dev/sdj not found (or ignored by filtering).

Solution: 可能該磁碟有包含其他磁區

```
# fdisk -l /dev/sdj

WARNING: GPT (GUID Partition Table) detected on '/dev/sdj'! The util fdisk doesn't support GPT. Use GNU Parted.


Disk /dev/sdj: 3848.3 GB, 3848290697216 bytes
255 heads, 63 sectors/track, 467861 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sdj1               1      267350  2147483647+  ee  GPT
Partition 1 does not start on physical sector boundary.
```

移除這磁碟裡的所有磁區後再試一次。

##### 移除已損壞的 PV

無法移除已經損壞 PV 的 VG 與 LV

> 有一個 PV - /dev/sdc 發生損毀，所屬 VG 與 LV 要移除時遇到錯誤：
> 
>  Couldn't find device with uuid APgmaN-yCZG-WfQS-L5zm-vd6g-oEJc-WjzNrT.  
>  Can't remove locked LV isths\_DMS
> 
>  Couldn't find device with uuid APgmaN-yCZG-WfQS-L5zm-vd6g-oEJc-WjzNrT.  
>  WARNING: Partial LV isths\_home needs to be repaired or removed.  
>  WARNING: Partial LV isths\_DMS needs to be repaired or removed.  
>  WARNING: Partial LV pvmove0 needs to be repaired or removed.  
>  There are still partial LVs in VG vg\_db.  
>  To remove them unconditionally use: vgreduce --removemissing --force.  
>  Proceeding to remove empty missing PVs.

Solution: VG Name: vg\_db

```
# vgcfgbackup vg_db
  Couldn't find device with uuid APgmaN-yCZG-WfQS-L5zm-vd6g-oEJc-WjzNrT.
  Volume group "vg_db" successfully backed up.
```

/etc/lvm/backup/vg\_db:

```
...
logical_volumes {
...
...
}
... 
```

> 將段落 logical\_volumes 整個移除。

執行 vgcfgrestore

```
# vgcfgrestore vg_db
  Restored volume group vg_db
```

重新掃描再重新執行移除 LV 與 VG

```
pvs
vgs
lvs
vgremove vg_db
```

##### USB 掛載包含有 LVM 的硬碟，出現錯誤訊息

> \# mount -t ext3 /dev/VolGroup00/LogVol00 /mnt/lvm00  
>  mount: wrong fs type, bad option, bad superblock on /dev/mapper/VolGroup00-LogVol00,  
>  missing codepage or helper program, or other error  
>  In some cases useful info is found in syslog - try  
>  dmesg | tail or so

Solution:

```shell
vgchange -an VolGroup00
vgchange -ay VolGroup00
mount -t ext3 /dev/VolGroup00/LogVol00 /mnt/lvm00
```

##### 無法新增 LV

系統剛安裝完，執行 pvdisplay -C 出現錯誤，並且無法新增 LV

> Couldn't find device with uuid ZrneWS-8KHd-UiXV-rQ4q-YU0o-r9Ga-0ZVBPW

Solution: 這個硬碟可能曾經做過 LVM 格式化，這些是之前遺留的裝置名稱，要清除這些不存在的舊設定。

```
# vgreduce -removemissing <vg-name>
```

> TIPs：
> 
>  要查出這些不存在的 device 的所屬 VG，可以執行 pvdisplay 。

##### 建立 GPT 磁區

With parted

```
# parted /dev/vdb
GNU Parted 3.4
Using /dev/vdb
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) mklabel GPT                                                      
(parted) mkpart primary 2048s 100%                                        
(parted) q                                                                
Information: You may need to update /etc/fstab.
```

# 進階應用

#### VG Import/Export

用途：將 VG 的磁碟遷移至不同的主機上。

> 大致方法是在遷移磁碟前，先將 VG 做 Exort；遷移後做 VG Import。

- [How To Move LVM Volume Group To Another Machine In Linux](https://ostechnix.com/move-lvm-volume-group-to-another-machine/)

#### 移除 PV Disk

從一個現有 LV 中移除正在使用的一個 PV Disk (/dev/sdb1)

> NOTE: 這個 LV 的所屬 VG 必須要有足夠的可用空間 [https://access.redhat.com/site/documentation/en-US/Red\_Hat\_Enterprise\_Linux/6/html-single/Logical\_Volume\_Manager\_Administration/index.html#disk\_remove\_ex](https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/6/html-single/Logical_Volume_Manager_Administration/index.html#disk_remove_ex)
> 
> [http://www.tecmint.com/lvm-storage-migration/](http://www.tecmint.com/lvm-storage-migration/)

```
# pvs -o+pv_used
  PV         VG   Fmt  Attr PSize  PFree  Used
  /dev/sda1  myvg lvm2 a-   17.15G 12.15G  5.00G
  /dev/sdb1  myvg lvm2 a-   17.15G 12.15G  5.00G
  /dev/sdc1  myvg lvm2 a-   17.15G 12.15G  5.00G
  /dev/sdd1  myvg lvm2 a-   17.15G  2.15G 15.00G
```

```
# pvremove /dev/sdb1
  /dev/sdb1: Moved: 2.0%
 ...
  /dev/sdb1: Moved: 79.2%
 ...
  /dev/sdb1: Moved: 100.0%
```

```
# pvs -o+pv_used
  PV         VG   Fmt  Attr PSize  PFree  Used
  /dev/sda1  myvg lvm2 a-   17.15G  7.15G 10.00G
  /dev/sdb1  myvg lvm2 a-   17.15G 17.15G     0
  /dev/sdc1  myvg lvm2 a-   17.15G 12.15G  5.00G
  /dev/sdd1  myvg lvm2 a-   17.15G  2.15G 15.00G
```

```
# vgreduce myvg /dev/sdb1
  Removed "/dev/sdb1" from volume group "myvg"
[root@tng3-1 ~]# pvs
  PV         VG   Fmt  Attr PSize  PFree
  /dev/sda1  myvg lvm2 a-   17.15G  7.15G
  /dev/sdb1       lvm2 --   17.15G 17.15G
  /dev/sdc1  myvg lvm2 a-   17.15G 12.15G
  /dev/sdd1  myvg lvm2 a-   17.15G  2.15G
```

#### 搬移 PV Disk

搬移現有 PV Disk (/dev/sdb1) 內的所有資料至一個新的 PV Disk (/dev/sdd1)

方法一：使用 Mirror 方式 \*建議\*

參考連結:

- [https://www.thegeekdiary.com/centos-rhel-7-how-to-create-and-remove-the-lvm-mirrors-using-lvconvert/](https://www.thegeekdiary.com/centos-rhel-7-how-to-create-and-remove-the-lvm-mirrors-using-lvconvert/)
- [https://linux.m2osw.com/creating-drive-mirror-lvm-including-exact-partition-cloning](https://linux.m2osw.com/creating-drive-mirror-lvm-including-exact-partition-cloning)

說明:

- VG: vg\_data
- LV: db\_worktmp, lv\_mydev, lv\_worktmp
- PV: from /dev/sdb1 to /dev/sdc

1\. 確認新硬碟的容量需求

```
#> lvs -a -o name,copy_percent,devices,lv_size vg_data

  LV         Cpy%Sync Devices         LSize
  db_worktmp          /dev/sdb1(0)      20.00g
  lv_mydev            /dev/sdb1(5121)   60.00g
  lv_mydev            /dev/sdb1(10242)  60.00g
  lv_worktmp          /dev/sdb1(20482) 500.00m 
```

2\. 將新硬碟加入到同一個 VG 內

```
#> vgextend vg_data /dev/sdc
```

3\. 逐一將所有 LV 作完 Mirror

```
#> lvconvert -m1 vg_data/db_worktmp
#> lvconvert -m1 vg_data/lv_mydev
#> lvconvert -m1 vg_data/lv_worktmp 

檢查 Mirror 時的進度
#> lvs -a -o name,copy_percent,devices,lv_size vg_data 
```

4\. 逐一將所有 LV 移除 Mirror，需指定舊硬碟代號

```
#> lvconvert -m0 vg_data/db_worktmp /dev/sdb1
#> lvconvert -m0 vg_data/lv_mydev /dev/sdb1
#> lvconvert -m0 vg_data/lv_worktmp /dev/sdb1
```

5\. 從 VG 裡移除 PV /dev/sdb1, 再移除整個 PV

```
#> vgreduce vg_data /dev/sdb1
#> pvremove /dev/sdb1 
```

方法二：使用 PV 搬移指令 pvmove

NOTE: PV 必須在同一個 VG 才能互相搬移

```
# pvs -o+pv_used
  PV         VG   Fmt  Attr PSize  PFree  Used
  /dev/sda1  myvg lvm2 a-   17.15G  7.15G 10.00G
  /dev/sdb1  myvg lvm2 a-   17.15G 15.15G  2.00G
  /dev/sdc1  myvg lvm2 a-   17.15G 15.15G  2.00G
```

```
# pvcreate /dev/sdd1
  Physical volume "/dev/sdd1" successfully created
```

```
# vgextend myvg /dev/sdd1
  Volume group "myvg" successfully extended
[root@tng3-1]# pvs -o+pv_used
  PV         VG   Fmt  Attr PSize  PFree  Used
  /dev/sda1   myvg lvm2 a-   17.15G  7.15G 10.00G
  /dev/sdb1   myvg lvm2 a-   17.15G 15.15G  2.00G
  /dev/sdc1   myvg lvm2 a-   17.15G 15.15G  2.00G
  /dev/sdd1   myvg lvm2 a-   17.15G 17.15G     0
```

```
# pvmove /dev/sdb1 /dev/sdd1
  /dev/sdb1: Moved: 10.0%
...
  /dev/sdb1: Moved: 79.7%
...
  /dev/sdb1: Moved: 100.0%

[root@tng3-1]# pvs -o+pv_used
  PV          VG   Fmt  Attr PSize  PFree  Used
  /dev/sda1   myvg lvm2 a-   17.15G  7.15G 10.00G
  /dev/sdb1   myvg lvm2 a-   17.15G 17.15G     0
  /dev/sdc1   myvg lvm2 a-   17.15G 15.15G  2.00G
  /dev/sdd1   myvg lvm2 a-   17.15G 15.15G  2.00G 
```

```
# vgreduce myvg /dev/sdb1
  Removed "/dev/sdb1" from volume group "myvg"
```

# Filesystems

Linux Filesystem Comparison

[![linux_filesystem.jpg](https://osslab.tw/uploads/images/gallery/2023-10/scaled-1680-/linux-filesystem.jpg)](https://osslab.tw/uploads/images/gallery/2023-10/linux-filesystem.jpg)