This turned out to be easier than expected, so I’m posting this short how-to. The restriction to ext3 is due to the man page for resize2fs claiming this is the only fs that can be resized without unmounting.
The scenario: you have an ext3 partition LogVol00 mounted on / and a swap partition LogVol01 in a logical volume group VolGroup00 (the default install), but you’re running out of space in /, there’s no room left in the volume group but you have a brand new disk drive with lots of free space. How to add some space to the / partition?
The usual problem is that / can’t be unmounted, there are guides out there which involve using a livecd or dual booting into another os that recognizes logical volumes, but in this case you can do it ‘on-the-fly’ in Fedora.
WARNING: If you have important data in any directory under / then backup before attempting this.
I’ll assume the spare disk is called /dev/sdb
1. Create a new partition of appropriate size using fdisk
Code:
fdisk /dev/sdb
n (create new partition, select start and end cylinders, all free space is selected by default)
w (save partition table and exit)
At this point you may be told that a reboot is necessary, I found it was unnecessary, but for safety you should probably reboot. Type fdisk -l again to check the new partition, I’ll assume /dev/sdb1 was created ~10G in size
2. Create an ext3 filesystem in the partition (this step is optional, just a quick test that all is well)
Code:
mkfs.ext3 /dev/sdb1
3. Initialise the partition for use as a physical volume in lvm and add it to VolGroup00
Code:
pvcreate /dev/sdb1
vgextend VolGroup00 /dev/sdb1
4. Extend LogVol00 (use ‘df /’ to check that / is mounted on LogVol00) as required (by 10G here)
Code:
lvextend -L +10G /dev/VolGroup00/LogVol00
If you receive errors about not enough physical extents, then reduce the size of the extension a little until it fits.
5. Finally, resize the filesystem (this part normally would require unmounting /, but for ext3 and 2.6 Kernels it works while the partition is mounted)
Code:
resize2fs /dev/VolGroup00/LogVol00
Type ‘df -h /’, to check it worked, there should be an extra 10G available 
(you can use vgdisplay, pvdisplay and lvdisplay (as root) to see detailed info, pvscan for a summary)
Note.
I’ve seen horror stories with lvm manipulation whereby an ‘unknown uuid’ error occurs for a partition, and the VolGroup can no longer be found, which prevents booting. This can be due to using e2label, mkfs etc on the partition after it has been initialised as a pv and added to the group. The error is fixable if you haven’t rebooted: you need to edit /etc/lvm/backup/VolGroup00 and delete the lines referring to the unrecognized partition (they’ll be a few between braces {}, the unknown uuid will be at the beginning, something like:
Code:
pv1 {
id = "gAMCGW-jdaH-VXse-HR2j-PA24-AIbb-aHMjKl"
device = "/dev/sdb1" # Hint only
status = ["ALLOCATABLE"]
dev_size = 20482812 # 9.76697 Gigabytes
pe_start = 384
pe_count = 312 # 9.75 Gigabytes
}
Remove all these lines, save the file, then run
Code:
vgcfgrestore -f /etc/lvm/backup/VolGroup00 VolGroup00
vgscan
vgscan should report all is ok (you can now try reinitialising the ‘bad’ partition using pvcreate, and then add it back to the VolGroup with vgextend)
Introduction
LVM is a Logical Volume Manager for the Linux kernel. With LVM you can abstract your storage space and have “virtual partitions” which are easier to modify. The basic building blocks of LVM are:
- Physical volume (PV): Partition on hard disk (or even hard disk itself or loopback file) on which you can have volume groups. It has a special header and is divided into physical extents. Think of physical volumes as big building blocks which can be used to build your hard drive.
- Volume group (VG): Group of physical volumes that are used as storage volume (as one disk). They contain logical volumes. Think of volume groups as hard drives.
- Logical volume(LV): A “virtual/logical partition” that resides in a volume group and is composed of physical extents. Think of logical volumes as normal partitions.
- Physical extent (PE): A small part of a disk (usually 4MB) that can be assigned to a logical Volume. Think of physical extents as parts of disks that can be allocated to any partition.
With LVM you can more easily handle your partitions (logical volumes) than normal hard drive partitions. For example, you can:
- Use any number of disks as one big disk(VG)
- Have partitions(LV) streched over several disks (they can be as big as all of your disk storage together)
- Resize/create/delete partitions(LV) and disks(VG) as you like (it doesn’t depend on position of the logical volumes within volume groups as with normal partitions)
- Resize/create/delete partitions(LV) and disks(VG) online (filesystems on them still need to be resized, but some support online resizing)
- Name your disks(VG) and partitions(LV) as you like
- Create small partitions(LV) and resize them “dynamically” as they get more filled (growing must be still done by hand, but you can do it online with some filesystems)
- …
Example:
Physical disks
Disk1 (/dev/sda):
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
|Partition1 50GB (Physical volume) |Partition2 80GB (Physical volume) |
|/dev/sda1 |/dev/sda2 |
|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ |
Disk2 (/dev/sdb):
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
|Partition1 120GB (Physical volume) |
|/dev/sdb1 |
| _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _|
LVM logical volumes
Volume Group1 (/dev/MyStorage/ = /dev/sda1 + /dev/sda2 + /dev/sdb1):
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
|Logical volume1 15GB |Logical volume2 35GB |Logical volume3 200GB |
|/dev/MyStorage/rootvol|/dev/MyStorage/homevol |/dev/MyStorage/mediavol |
|_ _ _ _ _ _ _ _ _ _ _ |_ _ _ _ _ _ _ _ _ _ _ _ _ |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ |
To sum it all up: With LVM you can use all your storage space as one big disk (volume group) and have more flexibility over your partitions (logical volumes).
Installation
Before doing anything we need to load the appropriate module:
# modprobe dm-mod
If you already have Arch Linux installed and just you want to add/try a partition with LVM jump to partition disks.
Installing Arch Linux on LVM
Prior to running Arch Linux install scripts (/arch/setup) to install Arch Linux, you need to partition your disk with cfdisk (or any other tool of your liking). Because grub legacy (grub with version less than 1.0) can’t boot from LVM logical volumes you can’t have /boot in LVM, so you need to create a boot partition. 100MB should be enough. The other solution would be to use lilo or grub with version 1.95 or newer.
Partition disks
Next you need to create a partition for LVM. Its filesystem type should be ‘Linux LVM’, so use a partition id 0x8e (filesystem type: 8e). You need to create only one LVM partition on each disk you want to use with LVM. Your logical volumes will reside inside these partitions so size them accordingly. If you will use only LVM and no other external partitions, use all of free space on each disk.
Attention: /boot can’t reside in LVM partition because grub (version<1.95) can’t boot from it.
Information: All LVM partitions on all disks can be set to be seen as one big disk.
Create Physical volumes
Now you need to initialize these partitions so they can be used by LVM. Use ‘fdisk -l’ to find out which partitions have filesystem type ‘Linux LVM’ and create a physical volume on them:
# pvcreate /dev/sda2
Substitute /dev/sda2 with all your partitions to create physical volumes on all of them. This command creates a header on each partition so it can be used for LVM. You can track created physical volumes with:
# pvdisplay
Create Volume group(s)
Next step is to create a volume group on this physical volumes. First you need to create a volume group on one of the new partitions and then add to it all other physical volumes you want to have in it:
# vgcreate VolGroup00 /dev/sda2
# vgextend VolGroup00 /dev/sdb1
Also you can use any other name you like instead of VolGroup00 for a volume group when creating it. You can track how your volume group grows with:
# vgdisplay
Information: You can create more than one volume group if you need to, but then you wont have all your storage presented as one disk.
Create Logical Volumes
Now we need to create logical volumes on this volume group. You create them with the next command by giving them the name of a new logical volume, its size and volume group they will live on:
# lvcreate -L 10G VolGroup00 -n lvolhome
This will create a logical volume that you can access later with /dev/mapper/Volgroup00-lvolhome or /dev/VolGroup00/lvolhome. Same as with the volume groups you can use any name you want for your logical volume when creating it.
Note: You may need to load the device-mapper kernel module for the above command to succeed:
# modprobe dm-mod
You can track created logical volumes with:
# lvdisplay
Attention: When creating swap on logical volume use the next command instead of previous one:
# lvcreate -C y -L 10G VolGroup00 -n lvolswap
The ‘-C y’ is used to create a contiguous partition, that means that your swap space doesn’t get partitioned over one or more disks nor over non-contiguous physical extents.
Information: you can create logical volumes smaller if you don’t know how much data will end on them and leave some free space in volume group. You can resize the logical volumes later if needed.
Hint: If you want to fill all the free space left on a volume group use the next command:
# lvcreate -l +100%FREE VolGroup00 -n lvolmedia
Create filesystem and mount logical volumes
Your logical volumes should now be located in /dev/mapper/ and /dev/YourVolumeGroupName. If you can’t find them use the next commands to bring up the module for creating device nodes and to make volume groups availabile:
# modprobe dm-mod
# vgchange -ay
Now you can create filesystems on logical volumes and mount them as normal partitions (if you are installing Arch linux, skip this step):
# mkfs.ext3 /dev/mapper/VolGroup00-lvolhome
# mount /dev/mapper/VolGroup00-lvolhome /home
If you are installing Arch linux, start /arch/setup, go to Prepare Hard Drive directly to step 3 Set Filesystem Mountpoints and read the Important section below before proceeding with installation!
Important
There are just a few things you need to be careful while using/installing Arch Linux with LVM (in brackets are the corresponding menus during installation):
- When choosing mountpoints, just select your newly created logical volumes (use: /dev/mapper/Volgroup00-lvolhome).
Do NOT select the actual partitions on which logical volumes were created (don’t use: /dev/sda2). (Set Filesystem Mountpoints)
- Make sure you change USELVM=”no” to USELVM=”yes” in /etc/rc.conf (Configure System)
- Make sure that lvm2 is in the HOOKS section of /etc/mkinitcpio.conf just before the filesystems so that your kernel will find LVM volumes at boot time. (Configure System)
- Make sure /boot/grub/menu.lst uses the right volumes for root. It should look something like this: (Install Bootloader)
...
# (0) Arch Linux
title Arch Linux
root (hd0,0)
kernel /vmlinuz26 root=/dev/mapper/VolGroup00-lvolroot resume=/dev/mapper/VolGroup00-lvolswap ro
initrd /kernel26.img
...
If you are using LILO check /etc/lilo.conf:
image=/boot/vmlinuz26
label=arch
append="root=/dev/mapper/VolGroup00-lvolroot resume=/dev/mapper/VolGroup00-lvolswap ro"
initrd=/boot/kernel26.img
Configuration
Grow logical volume
To grow a logical volume you first need to grow the logical volume and then the filesystem to use the newly created free space. Let’s say we have a logical volume of 15GB with ext3 on it and we want to grow it to 20G. We need to do the following steps:
# lvextend -L 20G VolGroup00/lvolhome (or lvresize -L +5G VolGroup00/lvolhome)
# resize2fs /dev/VolGroup00/lvolhome
You may use lvresize insted of lvextend.
Attention: Not all filesystem support growing without loss of data and/or growing online.
Information: If you don’t resize your filesystem, you will still have a volume with the same size as before (volume will be bigger but partly unused).
Hint: If you want to fill all the free space on a volume group use the next command:
# lvextend -l +100%FREE VolGroup00/lvolhome
Shrink logical volume
Because your filesystem is probably as big as the logical volume it resides on, you need to shrink the filesystem first and then shrink the logical volume. Depending on your filesystem, you may need to unmount it first. Let’s say we have a logical volume of 15GB with ext3 on it and we want to shrink it to 10G. We need to do the following steps:
# resize2fs /dev/VolGroup00/lvolhome 9G
# lvreduce -L 10G VolGroup00/lvolhome (or lvresize -L -5G VolGroup00/lvolhome)
# resize2fs /dev/VolGroup00/lvolhome
Here we shrunk the filesystem more than needed so that when we shrunk the logical volume we didn’t accidentally cut of the end of the filesystem. After that we normally grow the filesystem to fill all free space left on logical volume. You may use lvresize insted of lvreduce.
Attention: Don’t reduce the filesystem size to less than it is used by data on it or you can lose your data.
Attention: Not all filesystems support shrinking without loss of data and/or shrinking online.
Attention: It is better to reduce the filesystem to lower size than the logical volume, so that after a resizing logical volume, we don’t accidentally cut off some data from the end of the filesystem.
Add partition to a volume group
To add a partition to your volume group you must first make its type ‘Linux LVM’ (for example with cfdisk). Then you need to create a physical volume on it and extend the volume group over it:
# pvcreate /dev/sdb1
# vgextend VolGroup00 /dev/sdb1
Now you have free space in your volume group that can be used by logical volumes in this group.
Information: You can add partitions from any disks to volume groups.
Remove partition from a volume group
All of the data on that partition needs to be moved to another partition. Fortunately, lvm makes this easy:
# pvmove /dev/mapper/myvg-mypv
If you want to have the data on a specific physical volume, specify that as the second argument to pvmove.
Then the physical volume needs to be removed from the volume group:
# vgreduce myVg /dev/mapper/myvg-mypv
Or remove all empty physical volumes:
# vgreduce --all vg0
And lastly, if you want to use the partition for something else, and want to avoid lvm thinking that the partition is a physical volume:
# pvremove /dev/mapper/myvg-removedpv
Snapshots
Introduction
LVM allows you to take a snapshot of your system in a much more efficient way than a traditional backup. It does this efficiently by using a COW (copy-on-write) policy. The initial snapshot you take simply contains hard-links to the inodes of your actual data. So long as your data remains unchanged, the snapshot merely contains its inode pointers and not the data itself. Whenever you modify a file or directory that the snapshot points to, LVM automatically clones the data, the old copy referenced by the snapshot, and the new copy referenced by your active system. Thus, you can snapshot a system with 35GB of data using just 2GB of free space so long as you modify less than 2GB (on both the original and snapshot).
Configuration
You create snapshot logical volumes just like normal ones.
# lvcreate --size 100M --snapshot --name snap01 /dev/mapper/vg0-pv
With that volume, you may modify less than 100M of data, before the snapshot volume fills up.
Todo: scripts to automate snapshots of root before updates, to rollback… updating menu.lst to boot snapshots (separate article?)
snapshots are primarily used to provide a frozen copy of a filesystem to make backups; a backup taking two hours provides a more consistent image of the filesystem than directly backing up the partition.
Troubleshooting
LVM commands don’t work
try preceeding commands with lvm like this:
# lvm pvdisplay
Set Filesystem Mountpoints page doesn’t show logical volumes
If you are installing on a system where there is an existing volume group, you may find that even after doing “modprobe dm-mod” you don’t see the list of logical volumes.
In this case, you may also need to do:
# vgchange -ay <volgroup>
in order to activate the volume group and make the logical volumes available.
