Additional EC2 Tips

Contents

• Converting PV to HVM
• Shrinking AMI Images

Here's a small collection of some AWS EC2 tips and tricks.

Converting PV to HVM

Many of the EC2 instances now use HVM images rather than the PV images we used for our experiments. Converting between the formats is fairly straightforward: the idea is to copy the /boot/ from an existing HVM image to the PV image to convert the latter into an HVM image.

The kernel versions of the two images must match. We ensure this by updating to the latest kernel, so you may wish to change this if it is undesirable.

1. Start a t1.micro instance with the desired PV image. Run:

sudo apt-get update
sudo apt-get -y install grub-pc grub-pc-bin grub-legacy-ec2 grub-gfxpayload-lists
sudo apt-get -y install linux-virtual linux-image-virtual   # update kernel
sudo apt-get -y autoremove   # remove old kernels
sudo update-grub2

2. Start a t2.micro instance with an HVM image. Use, e.g., ami-09470539 or anything that is 12.04-amd64-server (more generally, choose something that matches the OS version of your PV image). Then run:

sudo apt-get update
sudo apt-get install -y linux-virtual linux-image-virtual
sudo apt-get -y autoremove
sudo update-grub2

3. Stop the t1.micro instance. Detach the t1.micro's volume and attach it to /dev/sdf of the t2.micro instance.

4. On the t2.micro instance, run:

sudo su
mkdir -p /mnt/xvdf && mount /dev/xvdf /mnt/xvdf
rsync -avzXA /boot/ /mnt/xvdf/boot/
mount -o bind /dev /mnt/xvdf/dev && mount -o bind /dev/pts /mnt/xvdf/dev/pts && mount -o bind /proc /mnt/xvdf/proc && mount -o bind /sys /mnt/xvdf/sys
chroot /mnt/xvdf
grub-install --no-floppy --recheck --force /dev/xvdf
update-grub2

5. Stop the t2.micro instance and detach all volumes.

6. Attach the t1.micro's original volume to t2.micro at /dev/sda1.

7. Create an AMI image of the t2.micro instance (no need to start it). You now have an HVM image!

Note: /dev/sda1 is always the root/boot device of any instance, regardless of PV or HVM.

Shrinking AMI Images

To shrink an AMI image, you will need to copy all the contents of an existing image to a smaller EBS volume and create a new image from the volume. Here's how to do this.

1. Create a new instance (t1/t2.micro for PV/HVM, resp.) using the image and use df -h to see how small the volume can be.

2. Create a new EBS volume of that size under Volumes (make sure the availability zone matches the instance's).

3. Attach the new volume to the instance at /dev/sdf.

4. On the instance, execute the following (we highly recommended running rsync in a "screen"!):

sudo su
mkfs.ext4 /dev/xvdf

# this sets both LABEL and UUID (grub will use UUID if update-grub2 is ran)
# HVM appears to use /dev/xvda, while PV uses /dev/xvda1
e2label /dev/xvdf $(e2label /dev/xvda)
tune2fs /dev/xvdf -U $(blkid /dev/xvda | sed -e 's/.*UUID="//g' -e 's/".*//g')

mkdir /mnt/new
mount -t ext4 /dev/xvdf /mnt/new

# to be safe, run rsync in a screen with sudo! copying can take a while
rsync -aAX --exclude={"/dev/*","/proc/*","/sys/*","/tmp/*","/run/*","/mnt/*","/media/*","/lost+found"} /* /mnt/new

# do this after rsync is complete
mount -o bind /dev /mnt/new/dev && mount -o bind /dev/pts /mnt/new/dev/pts && mount -o bind /proc /mnt/new/proc && mount -o bind /sys /mnt/new/sys
chroot /mnt/new
grub-install --no-floppy --recheck --force /dev/xvdf
update-grub2

5. Stop the instance. Detach both volumes and reattach the smaller volume to /dev/sda1.

6. Create an image from the instance (no need to start it).

Note 1: If there are issues booting the instance (e.g., "ALERT! /dev/disk/by-label/.... does not exist" or the instance just shuts down), try mounting the smaller volume again as /dev/sdf/ on a bootable instance and double check that LABEL and UUID are both set correctly (via, e.g., blkid). Also do another update-grub2 via chroot (step 4).

Note 2: UUID comes into use only when update-grub2 is used (should be a bug but they won't fix); grub will be using LABEL.