BenV's notes

Another day, another time for fun!

Since we got Xen up and running yesterday, it’s now time for actually having some fun with it.
The goals are:

1. Getting xend started automagically when booting without destroying my network connection
2. Getting a domU up and running with a network connection
3. Getting an internal network between the domUs and dom0, shielded from the big bad internet.

First things first, yesterday we got xen installed and all, but that’s about as far as we got before the bed was required. Xend almost got it right when it put a startup script in /etc/rc.d/init.d called ‘xend’, but alas, slackware will ignore that file. Personally I simply add the thing to /etc/rc.d/rc.local:

echo "Starting xend..."
/etc/init.d/xend start

This fixes issue #1, but only the first part.

Before you reboot you might want to alter some xend settings. The file to molest for this is /etc/xend/xend-config.sxp. Mine looks a bit like this:

root@xenbro:~# grep -ve '#' -e '^$' /etc/xen/xend-config.sxp
(logfile /var/log/xen/xend.log)
(loglevel DEBUG)
(xen-api-server ((unix)))
(xend-unix-xmlrpc-server yes)
(xend-relocation-server yes)
(xend-unix-path /var/lib/xend/xend-socket)
(xend-address localhost)
(xend-relocation-hosts-allow '^localhost$ ^localhost\\.localdomain$')
(network-script 'bridge-wrapper-benv netdev=eth0 bridge=xenbr0')
(vif-script vif-bridge)
(dom0-min-mem 196)
(enable-dom0-ballooning yes)
(dom0-cpus 0)
(vncpasswd '')

Almost everything is set to the default value. However, you will want to pay attention to the network-script part at the very least. As you can see I changed this to a little wrapper script.
I won’t go into detail about all the possible Xen networking stuff, see the mailing list and the Xen docs for that, but what we want is the external network bridged to all domU’s, so they can have their own network space there and their external IP address and whatever they want to do without having to mess around in the dom0. To get this working you could stick to the default (network-script network-bridge). Back in the Xen 3.0.3 days this molested my default gateway because their bridge script sucked (it failed to parse some stuff if I remember correctly), but when I tested it today it simply worked.

We want to have an internal network though, so we have to create our own bridge wrapper script.The only thing this script wrapper does is
1. call the network bridge script as the default would do.
2. create a second xen bridge.
Here we go:

#!/bin/bash
/etc/xen/scripts/network-bridge "$@"

ifconfig ${bridge} up 10.0.0.1 netmask 255.255.255.0
fi

As you can see I also add an internal IP address to the bridge. You can make this bridge setup script as complex or as neat as you want, but I can’t be bothered to spend more time on it. It works for me ™ 😉

Now that all that is in place, let’s try out the creation of a domU. For testing purposes I steal my images from what used to be jailtime.org, these days it’s called Stacklet. Too bad they don’t have a 64 bit slackware domU yet, but I guess I’ll create that myself a little later. Also, to keep stuff in a place where I can track it, I usually create a /xen directory where I store my host configurations and images.

root@xenbro:~# mkdir -p /xen/hosts/slackware12.2
root@xenbro:~# cd /xen/hosts/slackware12.2
root@xenbro:/xen/hosts/slackware12.2# wget http://stacklet.com/sites/default/files/slackware/slackware.12-2.x86.20090509.img.tar.bz2
[ ... ]
root@xenbro:/xen/hosts/slackware12.2# tar jxvf slackware.12-2.x86.20090509.img.tar.bz2
[ *kaboom* ]

Edit the slackware.12-2.x86.xen3.cfg file and fix the location of the image file and the kernel. I simply used the same kernel that I use to boot my dom0 for now. Now let’s try it! Try xm create -c slackware.12-2.x86.xen3.cfg and it should enter the console of the now booting domU. To get out of the console hit CTRL-[.

Over here I got a bunch of kernel warnings like these:

[ 0.000000] ------------[ cut here ]------------
[ 0.000000] WARNING: at drivers/firmware/dmi_scan.c:425 dmi_matches+0x81/0x90()
[ 0.000000] dmi check: not initialized yet.
[ 0.000000] Modules linked in:
[ 0.000000] Pid: 0, comm: swapper Not tainted 2.6.29.6-BenV #3
[ 0.000000] Call Trace:
[ 0.000000] [] warn_slowpath+0xea/0x160
[ 0.000000] [] printk+0x4e/0x56
[ 0.000000] [] phys_pud_init+0x150/0x42b
[ 0.000000] [] reserve_early+0x13/0x28
[ 0.000000] [] init_memory_mapping+0x559/0xd40
[ 0.000000] [] dmi_matches+0x81/0x90
[ 0.000000] [] dmi_check_system+0x20/0x60
[ 0.000000] [] setup_arch+0x2aa/0xf19
[ 0.000000] [] start_kernel+0x81/0x483
[ 0.000000] [] x86_64_start_kernel+0xa9/0xbf
[ 0.000000] ---[ end trace 4eaa2a86a8e2da22 ]---

While these are a nuisance in dmesg, they seem to be fairly harmless. Somewhere on the Xen mailing list someone claims that you can disable CONFIG_DMI in the kernel to get rid of these, which I tend to believe but can’t be bothered. If I’m going to build a new kernel for my domU, I will take the latest vanilla kernel and add Xen domU support in that. It’s a mainline kernel option these days, you can see my post about it here: Xen and booting domU using a vanilla kernel
Login on the domU using root/password, note that your networking actually works, but you still have to enter an IP for eth1 if you want the internal network.

root@darkstar:~# ifconfig eth1 10.0.0.2/24
root@darkstar:~# ping 10.0.0.1
PING 10.0.0.1 (10.0.0.1) 56(84) bytes of data.
64 bytes from 10.0.0.1: icmp_seq=1 ttl=64 time=0.770 ms
64 bytes from 10.0.0.1: icmp_seq=2 ttl=64 time=0.192 ms
64 bytes from 10.0.0.1: icmp_seq=3 ttl=64 time=0.187 ms

Woohoo, it works! 🙂

For an overview of the currently running domains you can use

root@xenbro:~# xm list
Name ID Mem VCPUs State Time(s)
Domain-0 0 7447 4 r----- 16.0
test1 1 512 1 -b---- 5.9

The xm command has a ton of useful things, so check out the manpage/help for it.

For my real domU I will be using LVM partitions and install slackware 64 on it, but that’s fairly trivial to figure out once you’re this far. So have fun with your Xen install ^^