Safe KVM Snaphots With Valine

KVM (Kernel-based Virtual Machine) is a full virtualization solution for Linux on AMD64 and Intel 64 hardware that is built into recent Linux kernel. It can run multiple, unmodified Windows and Linux guest operating systems. The KVM hypervisor in RedHat/CentOS Linux is managed with the libvirt API and tools built for libvirt (such as virsh, virt-install, virt-clone or virt-manager ). Virtual Machines (VMs) are executed and run as multi-threaded Linux processes controlled by these tools.

If you have a KVM host configured which is running several VMs (aka KVM guests) over Logical Volume Manager (LVM), and unless you don’t administrate it, you’ll probably at some point perform sensitive operations within the VM that might break its configuration (Ex: kernel update and compilation of a new GFPS/Lustre client against it). To avoid having to reverse-engineering the performed operation to return to a stable config, it is advisable to test the sensible operation on top of a LVM snapshot of the disk image of the VM.

You can of course create them manually, yet easier the management of these snapshot, I discovered a very useful script named valine from Ryan Sawhill Aroha (ryan on Github) you will want to deploy on all of your KVM hosts.

Installation

Install valine as follows:

1
2
3
4

$> git clone https://github.com/ryran/valine /usr/local/src/valine
$> ln -s /usr/local/src/valine/valine /usr/local/bin/valine    # Create the symlink
$> ln -s /usr/local/src/valine/valine-bash-completion.bash /etc/bash_completion.d/valine
$> mkdir /var/lib/valine

Now you can test the installation by running the valine command line:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

$> valine      # See current situation and KVM guests defined on this host
─────────────────┬───┬─────────┬───────────────────────────────────────────
 DOMAIN          │ ⚡ │ STORAGE │ SNAPSHOTS
─────────────────┼───┼─────────┼───────────────────────────────────────────
 dns1            │ ◉ │ lvm     │ /dev/vms/dns1 ←
 lm1             │ ◉ │ lvm     │ /dev/vms/lm1 ←
 ldap1           │ ◉ │ lvm     │ /dev/vms/ldap1 ←
 brightmanager1  │ ◉ │ lvm     │ /dev/vms/brightmanager1 ←
 slurm1          │ ◉ │ lvm     │ /dev/vms/slurm1 ←
 ganglia         │ ◉ │ lvm     │ /dev/vms/ganglia ←
 puppet1         │ ◉ │ lvm     │ /dev/vms/puppet1 ←
─────────────────┴───┴─────────┴───────────────────────────────────────────

Key:
 • Each domain's current storage is marked with a green arrow: ←
 • Storage which is revertable & Deletable is displayed in cyan

KVM Snapshot creation

The workflow to setup a new screenshot would be as follows:

1
2
3

(host)$> valine   # Check current status
(host)$> valine <guest> new-snap [sname]   # Create a new snaphot <sname> for the KVM guest <guest>
# By default, <guest>-snap0 is created, and guest <guest> is rebooted on top of it

Now you can connect to the guest VM <guest> to try all you sensitive operations (kernel upgrade, puppet receipts application etc.),

Revert back to the original image

Once you reach a procedure that seems to work and is ready for the real deployment, you can revert to the default LVM disk image as follows:

1
2
3

(host)$> valine
(host)$> valine <guest> revert-snap <guest>    # Revert to the normal disk
(host)$> valine <guest> Delete-snap <sname>    # Delete the snapshot