VirtualBox on Headless Gentoo

Being an adherent of Continuous Integration, I need a build machine that runs round the clock even when my workstation is turned off. As I’m running a small home server, this wouldn’t be an issue — if it weren’t for the fact that my home server runs Linux and 99% of my development happens in Windows. So I use virtualization to run a small Windows system on top of my home server.

In the past I used VMware for this job. VMware worked well for me and performance was quite good, but now that I’ve switched to a fully headless system, I noticed that the vmware-server package pulls in most of the X11 libraries – which I’m not particularly keen on having on my system due to their compile times.

So I went shopping for some alternatives. KVM sounded interesting (and was the leanest virtualization solution I could find), but the Gentoo Wiki stated that Windows didn’t work in qemu with recent kernels, so I went looking on – and found VirtualBox. This article explains how to set up VirtualBox on a headless Gentoo system.

1. Install VirtualBox

VirtualBox can be trimmed down so it builds without a graphical client, thereby avoiding any X11 dependencies. Here are the USE flags I set in my /etc/portage/package.use to get rid of most of VirtualBox’ dependencies:

###############################################################################
# virtualbox-ose
#
#   headless  Don't build graphical client (avoid X serve dependency)
#   -chm      Don't build CHM viewer for online help
#   -hal      Don't use Hardware Access Layer on host OS
#   -opengl   Disable OpenGL emulation
#   -qt4      Do not use qt4
#
app-emulation/virtualbox-ose headless -chm -hal -opengl -qt4

After that is taken care of, emerge VirtualBox on your system:

emerge virtualbox-ose

2. Create a Virtual Machine

Let VirtualBox create a new VM. VirtualBox puts the VM and later its hard drives in the home directory of the user that invoked VBoxManage, so make sure you’re logged in as the user you later want to run your VMs with. The Gentoo ebuild for VirtualBox’ commercial distribution currently neglects to set up symlinks, so you might have to run the commands (eg. VBoxManage) by specifying their full path (eg. /opt/VirtualBox/VBoxManage instead)

VBoxManage createvm \
  --name "Builder1" \
  --register

Next step: configure it. I’m using 512 MB of RAM and built-in NAT networking because it’s way easier to configure than bridged networking and you might not want to expose additional IP addresses through your network adapter in a server environment. VirtualBox’ built-in NAT supports port forwarding, so there’s really no need for bridging.

VBoxManage modifyvm "Builder1" \
  --memory 512 \
  --acpi on \
  --boot1 dvd \
  --nic1 nat \
  --ostype WindowsXP

You can omit the ostype setting if you wish. This is a hint to VirtualBox about which operating system you plan to install as the guest OS, allowing for some OS-specific optimizations to be performed. I’ve had BSODs booting my VM if I used Windows 7 with a wrong ostype, so it’s probably a good idea to specify this. You can get a list of valid OS identifiers with VBoxManage list ostypes.

Next, create a hard drive image to install the guest operating system on

VBoxManage createhd \
  --filename "Builder1.vdi" \
  --size 16384 \
  --register

Then assign this image to the VM’s virtual hard drive:

VBoxManage modifyvm "Builder1" \
  --hda "Builder1.vdi"

3. Install OS on Virtual Machine

I normally do a base install (OS without development tools) and archive it somewhere so I can later restore if I want to do a clean upgrade – eg. if I switch to Visual Studio 2010, I easily start fresh instead of uninstalling Visual Studio 2008 and ending up with a build machine carrying around remains from the past.

If you’re running the commercial edition of VirtualBox, you can mount a CD/DVD image of your operating system’s install disk and perform the install via VirtualBox’ built-in RDP server:

VBoxManage registerimage dvd /var/storage/ISOs/Windows\ XP\ SP3.iso

VBoxManage modifyvm "Builder1" \
  --dvd /var/storage/ISOs/Windows\ XP\ SP3.iso

Don’t forget to enable RDP after you started the virtual machine:

VBoxManage startvm Builder1 \
  --type headless

VBoxManage controlvm Builder1 \
  vrdp on

If you’re using the Open Source Edition of VirtualBox, you should install one of the full VirtualBox releases on your workstation to prepare a hard drive image with the guest OS that you can then copy to your headless server.

4. Install Guest Additions

The first thing I recommend doing is to install the Guest Additions for VirtualBox. These provide better mouse cursor integration and an optimized graphics driver.

There are VirtualBox Guest Additions available for the commercial edition of VirtualBox directly from sun, but if you, like me, decided to run the open source edition, you can find an Open Source reimplementation of the Guest Additions for Windows systems here: VirtualBox Windows Guest Additions Installer.

5. Enable SATA Interface

You can further improve the performance of your virtual machine by using a virtual SATA hard drive instead of an IDE one:

1. Shut down your guest OS so you can modify the virtual machine’s hardware

2. Enable the SATA controller for the virtual machine

   VBoxManage modifyvm Builder1 \
     --sata on \
     --sataportcount 2
   

3. Start the virtual machine up again and install the Intel Rapid Storage Technology Driver.

4. Shut down your guest OS again.

5. Remove the hard drive from the IDE controller

   VBoxManage modifyvm Builder1 --hda none
   

6. Assign the hard drive image to the SATA controller

   VBoxManage modifyvm Builder1 --sataport1 Builder1.vdi
   

6. Back Up your Base Image

This would be a good time to let your OS download its updates and to create a backup of your base image that you can use to set up future virtual machines.

To minimize the size of the image, I usually follow these steps:

1. Run Disk Cleanup (Start Menu -> All Programs -> Accessories -> System Tools -> Disk Cleanup) and first, select the option to remove all but the latest system restore point (on the second tab), then (on the first tab) check all options except for “Compress Old Files” and click Ok.

2. Defragment your virtual hard drive a few times until the free space has been sufficiently consolidated.

3. Wipe the unused hard drive areas with zero-bytes. I use the Virtual Disk Precompactor from Microsoft Virtual PC for this purpose. It has a file named “filler.txt” that contains space characters, but which can be easily replaced by one containing zero-bytes.

4. Shut down the guest OS

5. Compact the virtual hard drive by running:

   VBoxManage modifyvdi Builder1.vdi compact
   

7. Daemonize VirtualBox

There’s a pretty good explanation complete with well-written init script available from the Gentoo wiki: Gentoo Linux Wiki – VirtualBox – Service.

Here’s a copy of their init script in case the wiki changes or goes down:

/etc/conf.d/virtualbox.example

# Username to start vbox as, must be part of vboxusers group.
VM_USER="nobody"

# Virtual Machine Name
VM_NAME="Windows Server 2003"

# Shutdown Method: pause|resume|reset|poweroff|savestate|acpipowerbutton|acpisleepbutton
VM_SHUTDOWN="savestate"

# Nice Priority: -20 (most favorable scheduling) to 19 (least favorable)
VM_NICE=0

/etc/init.d/virtualbox

#!/sbin/runscript

# Not sure why but gentoo forgot to add /opt/bin to the path.
VBOXPATH="/usr/bin:/opt/bin"
VBOXNAME="${SVCNAME#*.}"

depend() {
	need net
	
	if [ "${SVCNAME}" != "virtualbox" ] ; then 
		need virtualbox
	fi
}


checkconfig() {
	if [ ! -r /etc/conf.d/$SVCNAME ] ; then
		eerror "Please create /etc/conf.d/$SVCNAME"
		eerror "Sample conf: /etc/conf.d/virtualbox.example"
		return 1
	fi

	return 0
}

checkpath() {
	local r=0

	if ! su $VM_USER -c "PATH=$VBOXPATH command -v VBoxHeadless &>/dev/null" ; then
		eerror "Could not locate VBoxHeadless"
		r=1
	fi

	if ! su $VM_USER -c "PATH=$VBOXPATH command -v VBoxManage &>/dev/null" ; then
		eerror "Could not locate VBoxManage"
		r=1
	fi

	if [ $r -gt 0 ] ; then 
		eerror "Please verify the vm users path."
	fi

	return $r
}

isloaded() {
	lsmod | grep -q "$1[^_-]"
}

isvm() {
	[ $SVCNAME != "virtualbox" ]
}

loadmodules() {
	if ! isloaded vboxdrv ; then
		if ! modprobe vboxdrv > /dev/null 2>&1 ; then
			eerror "modprobe vboxdrv failed."
			return 1
		fi
	fi 

	if ! isloaded vboxnetflt ; then
		if ! modprobe vboxnetflt > /dev/null 2>&1 ; then
			eerror "modprobe vboxnetflt failed."
			return 1
		fi
	fi
	
	return 0
}

unloadmodules() {
	if isloaded vboxnetflt ; then
		if ! rmmod vboxnetflt > /dev/null 2>&1 ; then
			eerror "rmmod vboxnetflt failed."
			return 1
		fi
	fi

	if isloaded vboxdrv ; then
		if ! rmmod vboxdrv > /dev/null 2>&1 ; then
			eerror "rmmod vboxdrv failed."
			return 1
		fi
	fi 

	return 0
}

start() {
	# If we are the original virtualbox script [ $SVCNAME = "virtualbox" ]
	if ! isvm ; then
		ebegin "Starting Virtualbox"
		loadmodules
		eend $?
	else
		checkconfig || return $?
		checkpath   || return $?

		ebegin "Starting Virtualbox: $VBOXNAME"
		su $VM_USER \
			-c "PATH=$VBOXPATH nice -n $VM_NICE \
			VBoxHeadless -startvm \"$VM_NAME\" &>/dev/null" &
		pid=$!
		sleep 1
		
		kill -CHLD $pid &>/dev/null
		eend $?
	fi
}

stop() {
	# If we are the original virtualbox script [ $SVCNAME = "virtualbox" ]
	if ! isvm ; then
		ebegin "Stopping Virtualbox"
		unloadmodules
		eend $?
	else
		checkconfig || return $?
		checkpath   || return $?

		ebegin "Stopping Virtualbox: $VBOXNAME"
		su ${VM_USER} \
			-c "PATH=$VBOXPATH \
			VBoxManage controlvm \"$VM_NAME\" $VM_SHUTDOWN &>/dev/null"

		while [ \
			"$(su ${VM_USER} \
			-c "PATH=$VBOXPATH \
			VBoxManage showvminfo \"$VM_NAME\" | grep State | grep runn")" != "" \
		]
		do
			echo -n "."
			sleep 1
		done

		sleep 1
		echo

		eend $?
	fi
}

Above script has one master init script (/etc/init.d/virtualbox) and requires you to create symlinks to it for each virtual machine you want to run as a daemon:

cp /etc/conf.d/virtualbox.example /etc/conf.d/virtualbox.NEW_VM
cd /etc/init.d
ln -s virtualbox virtualbox.NEW_VM

8. Configure Port Forwarding

If you’re running any services on the virtual machine that require a port to be accessible from the outside, you can configure port forwarding for VirtualBox’ NAT like this:

VBoxManage setextradata Builder1 \
  "VBoxInternal/Devices/pcnet/0/LUN#0/Config/TeamCity/HostPort" 19090
VBoxManage setextradata Builder1 \
  "VBoxInternal/Devices/pcnet/0/LUN#0/Config/TeamCity/GuestPort" 19090
VBoxManage setextradata Builder1 \
  "VBoxInternal/Devices/pcnet/0/LUN#0/Config/TeamCity/Protocol" TCP

The VirtualBox NAT uses a kernel module to intercept and inject network packets. This approach requires zero configuration and works extremely well. I’m running OpenVPN on the host and the guest system accesses a remote server through the host’s OpenVPN connection and vice versa without problems.