KVM and Graceful Shutdown on Ubuntu
For quite some time, I have been trying to figure out, how to gracefully shutdown the KVM-based virtual machines running on a Ubuntu 10.04 LTS (Lucid Lynx) host system. This problem consists of two parts: First you have to make the virtual machines support the shutdown event from libvirt and second you have to call the shutdown action for each virtual machine on system shutdown.
The first part is very easy for Linux VMs and also not too hard for Windows VMs. I described the necessary steps in my wiki
The seconds part is harder to accomplish: On Ubuntu 8.04 LTS (Hardy Heron) I just modified the /etc/init.d/libvirt-bin script to call a Python script in the stop action. This solution was not perfect, as it meant that the virtual machines were also shutdown, when libvirtd was just restarted, however it was a quick and easy solution.
For Ubuntu 10.04, the init script has been converted to an Upstart job. So the easiest way was to create a upstart job that is starting on the stopping libvirt-bin event. However, this did not solve the problem, because the system powered off or rebooted before the shutdown of the virtual machines was finished. As it turns out, Ubuntu 10.04 uses an odd combination of Upstart jobs and traditional init scripts. This leads to a situation, where /etc/init.d/halt or /etc/init.d/reboot are called, before all upstart jobs have stopped, when one of the upstart jobs needs a significant amount of time to stop. This can be solved by adding an init script, than runs before the halt or reboot scripts and waits for the respective Upstart job to finish. In fact, it is best to run this script before the sendsigs script to avoid processes started by one of the upstart jobs to receive a SIGKILL.
I added the complete scripts and configuration files needed for this feature to my wiki. In fact, this solution also ensures, that the virtual machines are only shutdown if libvirt is stopped because of a runlevel change. Thus, the libvirt-bin package can now be upgraded without resulting in a restart of the VMs.
For me, automatically shutting down the virtual machines is very important. The KVM hosts I manage are connected to an uninterruptible power supply with limited battery time. Although in the past years I remember only a single time, the host systems were shutdown because the battery was nearly empty (most power interrupts are very short), I want to make sure that all virtual machines are in a safe, consistent state, when the power finally goes off. So I hope that the scripts in the wiki are also helpful to other people, having the same problem.