When you compare Linux-KVM to Hyper-V or VMWare your initial results will indicate that Linux-KVM is lacking when it comes to management tools, and basic functionality. You would be correct, however you would also be incorrect. You see with Linux-KVM we can leverage the underlying power of the Linux userland, and with this frankly all things are possible. Here is one of the basic bits of management functionality which can be attained with a little bit of bash scripting knowledge. I started by writing a VM backup script, then a VM export script, and of course a VM import script. Eventually I ended up with a full-blown end-to-end VM migration script.
My environment is based on Ubuntu 11.04 amd64 with the latest patches as of August 1, 2011. This script should work elsewhere unless file locations are different (VG Name) or if certain utilities are not included by default.
Features:
Requirements:
Known Issues:
Data Flow Example
Figure 1 – This diagram illustrates the data movers without compression.
Figure 2 – This diagram illustrates the data movers with compression.
When I first started this script I assumed that compression was a must, however as I started doing some test scripts I noticed that compression really did not impact the speed of the migration, now of course using compression will drastically reduce the amount of data which is transferred over the network. However when factoring in the processing time on either end for compression and decompression it just really didn’t matter. That said that was on 1Gb networking, so as long as your moves are local then it probably won’t make sense to use compression. It might make sense to use compression if your networking is less than 1Gb or if you are running it over a WAN connection. Additionally if you have very large volumes with a large amount of compressible space then you could gain a benefit from the compression option.
Please keep in mind this script has been tested thoroughly in my environments, and other logical environments that I can replicate. However I did not test this in your environment, that responsibility will lie with you. So please do not use this in a production environment until you (1) have tested it and (2) understand what it is doing.
Now to get to the code…
Name : vmmigrate.sh
Version : 0.9.3
MD5 : ff842f77db7225478c3b048af43b00c7
SHA256 : 141174ea2dbfb3c8b7c2753c92f1557c34fba2fd7576582676b3fd06d4e58a7e
URL : http://code.allanglesit.net/bash/vmmigrate.sh
[sourcecode gutter=”true” collapse=”true” language=”bash” firstline=”1″ title=”Expand vmmigrate.sh”]#!/bin/bash
# chkconfig:
# description:
#
#: Script Name : vmmigrate.sh
#: Version : 0.9.3
#: Author : Matthew Mattoon – http://blog.allanglesit.com
#: Date Created : August 1, 2011
#: Date Updated : February 20, 2013
#: Description : Offline Migration script for KVM Virtual Machines
#: Examples : vmmigrate.sh -n VMNAME -r REMOTEHOST -v REMOTEVGNAME -c COMPRESSIONLEVEL[0-9] -t -s
#: : vmmigrate.sh -n testvm -r testhost -v testhost -c 0 -t -s
usage()
{
cat << EOF
usage: $0 options
This script will allow you to perform an offline migration of an LVM-backed KVM domain from one host to another, using ssh, to monitor progress of the data moves you must have pv installed.
OPTIONS:
-h Show this message.
-n Name of the KVM domain to be migrated (required).
-r Remote KVM server name or IP address (required).
-v Volume Group on remote server on which to create the LV for the migrated VM (required).
-c Compression (gzip) level 0-9. 0 is no encryption, 1 is fastest (lowest) compression, 9 is slowest (highest) compression.
-t Use temporary ssh keys (recommended). script will create and deploy keys on the source and destination server, to eliminate the need for ssh password. Upon completion these keys will be destroyed.
-s Start migrated VM on completion of migration.
EOF
}
while getopts “hn:r:v:c:ts” OPTION
do
case $OPTION in
h) usage; exit 1;;
n) vmname=$OPTARG;;
r) remotehost=$OPTARG;;
v) remotevgname=$OPTARG;;
c) compressionlevel=$OPTARG;;
t) tempkey=1;;
s) startoncomplete=1;;
?) usage; exit;;
esac
done
if [[ -z "$vmname" ]] || [[ -z "$remotehost" ]] || [[ -z "$remotevgname" ]] || [[ -z "$compressionlevel" ]] || [ "$compressionlevel" -gt "9" ]
then
usage
exit 1
fi
shutoffcheck=`virsh list --all | grep "$vmname " | sed 's/shut off/shutoff/g' | tr -s [:space:] | cut -d " " -f 4`
if [ "$shutoffcheck" != "shutoff" ]
then
echo "${vmname} is not shut off. This script requires that virtual machines be shut off prior to beginning the migration process."
exit
fi
disklist=`virsh dumpxml ${vmname} | sed -n "/
if [[ -n “$disklist” ]]
then
echo “Script has detected non-LVM disks in use. This script cannot safely process non-LVM disks.”
exit
fi
cdlist=`virsh dumpxml ${vmname} | sed -n “/
if [[ -n “$cdlist” ]]
then
echo “The safety checks have detected an ISO file attached to this virtual machine. Migration will strip the ISO image from the configuration of the virtual machine. If you would like to resolve this in a different way please do not proceed with the migration when given the opportunity to proceed.”
fi
user=”root”
basesize=”1024″
if [[ -n “$tempkey” ]]
then
key=”/root/.ssh/vmmigrate_id_${vmname}”
publickey=”/root/.ssh/vmmigrate_id_${vmname}.pub”
authkeys=”/root/.ssh/authorized_keys”
ssh-keygen -t rsa -q -f ${key} -N ”
chmod 600 ${key}
chmod 600 ${publickey}
pubkeystring=`cat ${publickey} | sed -e ‘s/\//.*/g’`
ssh-copy-id -i ${publickey} root@${remotehost} 2>&1 1>/dev/null
sshstring=”ssh -i ${key} ${user}@${remotehost}”
else
sshstring=”ssh ${user}@${remotehost}”
fi
xmlfile=”/tmp/${vmname}.xml”
lvlist=`virsh dumpxml ${vmname} | sed -n “/
localvgname=`virsh dumpxml ${vmname} | grep “_boot” | cut -d “‘” -f 2 | cut -d “/” -f 3`
localvg=`vgs ${localvgname} -o vg_name,vg_extent_size,vg_free_count | grep ${localvgname}`
remotevg=`${sshstring} “vgs ${remotevgname} -o vg_name,vg_extent_size,vg_free_count” | grep ${remotevgname}`
localvgextentsize=`echo ${localvg} | tr -s [:space:] | cut -d ” ” -f 2 | cut -d “.” -f 1`
remotevgextentsize=`echo ${remotevg} | tr -s [:space:] | cut -d ” ” -f 2 | cut -d “.” -f 1`
remotevgfreeextents=`echo ${remotevg} | tr -s [:space:] | cut -d ” ” -f 3`
remotevgcheck=`expr $remotevgextentsize \* $remotevgfreeextents`
remotevgcheck=`expr $remotevgcheck \/ $basesize`
for lv in ${lvlist}
do
lvextents=`lvdisplay ${lv} -c | cut -d “:” -f 8`
lvsize=`expr $lvextents \* $localvgextentsize`
lvsizetotal=`expr $lvsizetotal + $lvsize`
done
lvsizetotal=`expr $lvsizetotal \/ $basesize`
if [ “$remotevgcheck” -lt “$lvsizetotal” ]
then
echo “The remote Volume Group does not contain the free space necessary for the creation of all Logical Volumes for this migration, script must exit.”
echo “Cleaning up temporary ssh keys…”
${sshstring} “cat ${authkeys} | sed -e ‘/$pubkeystring/d’ > ${authkeys}”
rm ${key} ${publickey}
exit
fi
read -p “Would you like to begin the migration of ${vmname} to ${remotehost}? (y/n) ”
if [ “$REPLY” != “y” ]
then
echo “User chose to abort migration.”
if [[ -n “$tempkey” ]]
then
echo “Cleaning up temporary ssh keys…”
${sshstring} “cat ${authkeys} | sed -e ‘/$pubkeystring/d’ > ${authkeys}”
rm ${key} ${publickey}
fi
echo “Exiting…”
exit
fi
echo “=============== BEGIN DATA MOVES ===============”
echo “”
for lv in ${lvlist}
do
lvname=`basename ${lv}`
lvextents=`lvdisplay ${lv} -c | cut -d “:” -f 8`
lvsize=`expr $lvextents \* $localvgextentsize`
remotelvcheck=`${sshstring} “lvs /dev/${remotevgname}/${lvname} 2>&1″`
if [ “$remotelvcheck” == ” One or more specified logical volume(s) not found.” ]
then
echo “Creating Logical Volume ${lvname} on ${remotehost}.”
${sshstring} “lvcreate -L${lvsize}M ${remotevgname} -n ${lvname} 2>&1 > /dev/null”
else
echo “Logical Volume (${lvname}) already exists on the remote host, for the safety of your data and system, this script will not overwrite a Logical Volume which it did not create, script must exit.”
exit
fi
localpvtest=`hash pv 2>&- || { echo >&2 “pv not installed”; } 2>&1`
remotepvtest=`${sshstring} “hash pv 2>&- || { echo >&2 “pv not installed”; } 2>&1″`
if [[ -n “$localpvtest” ]] || [[ -n “$remotepvtest” ]]
then
message=”Transferring ${lvname} to ${remotehost}.”
message2=”Progress is not available, please install pv for progress…”
if [ “$compressionlevel” -eq “0” ]
then
echo “${message} With no compression. ${message2}”
dd if=${lv} 2>/dev/null | ${sshstring} “dd of=/dev/${remotevgname}/${lvname} 2>/dev/null”
else
echo “${message} With compression. ${message2}”
dd if=${lv} 2>/dev/null | gzip -${compressionlevel} | ${sshstring} “gunzip | dd of=/dev/${remotevgname}/${lvname} 2>/dev/null”
fi
else
message=”Transferring ${lvname} to ${remotehost}.”
if [ “$compressionlevel” -eq “0” ]
then
echo “${message} With no compression…”
dd if=${lv} 2>/dev/null | pv -tpreb -N ${lvname} -s ${lvsize}M | ${sshstring} “dd of=/dev/${remotevgname}/${lvname} 2>/dev/null”
else
echo “${message} With compression…”
dd if=${lv} 2>/dev/null | pv -tpreb -N ${lvname} -s ${lvsize}M | gzip -${compressionlevel} | ${sshstring} “gunzip | dd of=/dev/${remotevgname}/${lvname} 2>/dev/null”
fi
fi
done
echo “”
echo “================ END DATA MOVES ================”
virsh dumpxml ${vmname} | sed “/
\|vnet\|echo “Transferring configuration file.”
if [[ -n “$tempkey” ]]
then
scp -i ${key} ${xmlfile} ${user}@${remotehost}:${xmlfile} #2&>1 > /dev/null
else
scp ${xmlfile} ${user}@${remotehost}:${xmlfile} #2&>1 > /dev/null
fi
rm ${xmlfile}
echo “Defining domain from configuration.”
${sshstring} “virsh define ${xmlfile}; rm ${xmlfile}” #2&>1 > /dev/null
if [[ -n “$startoncomplete” ]]
then
${sshstring} “virsh start ${vmname}” #2&>1 > /dev/null
fi
if [[ -n “$tempkey” ]]
then
echo “Cleaning up temporary ssh keys…”
${sshstring} “cat ${authkeys} | sed -e ‘/$pubkeystring/d’ > ${authkeys}”
rm ${key} ${publickey}
fi[/sourcecode]
UPDATE Feb 2, 2013
I have migrated my scripts to a new location.