HowTo: Hot Bundle a running Citrix Xen Server Virtual Machine Instance

This advanced guide will help you migrate your existing infrastructure to AWS EC2 in situ - that is, while the VM instance is running in your datacenter. There are a few requirements:

Requirements:

  1. AWS account
  2. rsync package on the running server
  3. A running EC2 instance in your preferred region
  4. Patience

In this HowTo I will show you how to rsync from a running instance to a volume attached to an EC2 instance, which we then bundle into an AMI for use on EC2.

First you want to have a running instance on the proper availability region on AWS. Create a volume of the proper size to hold your root (/) and boot (/boot) filesystem on the cloud. Attach this device to your running EC2 instance and us fdisk to partition the /boot and / partitions (usually +64M for /boot.) Use mkfs (or whichever tools you’re going to use) to format the two partitions, and then mount the root, create the boot directory, then mount the boot:

This rsync command is run from the VM instance that you want to export the VM Disk Image from. We used the long command switches so you can see what is going on. The switches do this, in order: be verbose, copy every directory recursively, preserve symlinks, preserve permissions, preserve filesystem times, only copy from one file system (Do not go into mounted systems, regardless of where they are mounted), exclude OS folders, make output human readable, and show progress. Finally, we tell rsync to sync the root filesytem (/*) to the rsync server located on the EC2 instance, as the user ec2-user, and to place the output in the folder /mnt. This can take a while:

Console - root@hostname ~ #

1
2
3
4
5
rsync --verbose --recursive --links --perms --times --one-file-system \
--exclude=/dev --exclude=/sys --exclude=/proc --exclude=/mnt \
--human-readable --progress --archive --compress \
-e "ssh -i EC2-private-key-file.pem" --rsync-path="sudo rsync" /* \
ec2-user@ec2-184-72-19-199.us-west-1.compute.amazonaws.com:/mnt

make the directory /mnt/boot/boot and copy the stuff from /mnt/boot into it:

Console - root@hostname ~ #

1
2
mkdir /mnt/boot/boot/
mv /mnt/boot/* /mnt/boot/boot/ 2> /dev/null > /dev/null

Open /mnt/boot/boot/grub/menu.lst for editing and change the references to /kernelimage to /boot/kernelimage:

Save your changes and open device.map and change /dev/xvd(a) to /dev/sd(a):

The next stage is conversion from local Xen hosts to the Amazon Web Services hosts. Depending on your setup, you may have to change a few files, or hundreds. Automation may be key. For example, DHCP needs to be turned on for eth0 and hostnames need to be changed/erased/commented out. You also may have internal package repositories, and you will need to update your /etc/ config files for all things like this. Also, this may go without saying, but NFS shares and the like will need to be mirrored on S3 or another means in order for your servers to have access to the shares. Finally, you need to edit /boot to reflect the pvgrub stage of amazon. Some common edits you may have to make depending on your datacenter environment:

  1. /etc/hosts
  2. /etc/resolv.conf
  3. /etc/sysconfig/network
  4. /etc/sysconfig/network-scripts/ifcfg-eth0 and /etc/sysconfig/network-scripts/ifcfg-eth1 if present
  5. Any other proprietary internal stuff (will update if I find more issues)
  6. Relink /etc/grub.conf to match /boot/boot/grub/grub.conf (chroot /mnt first, then relink it!)
  7. Edit /etc/fstab to change the LVM info to sda
  8. Edit /etc/mtab to reflect changes to /etc/fstab

Save your changes and move on to the final steps, ensuring that all of the config files in /etc are set up and ready to go. After you’ve done that you need to create a snapshot of this volume on EC2:

Go into the snapshot section of the management console, and wait for the snapshot to be completed. Once it has, create an image from this snapshot, and change /dev/sda1 to /dev/sda. You can set the kernel AKI-id in here (you have to use the pv-grub-hd00 AKI for your region and arch) but we’ve found that occasionally the management console will not save this setting so it is up to you:

The AMI will appear in the AMI list, which you can then use to spin up your instance for testing.