Since we got Xen up and running yesterday, it’s now time for actually having some fun with it.
The goals are:

1. Getting xend started automagically when booting without destroying my network connection
2. Getting a domU up and running with a network connection
3. Getting an internal network between the domUs and dom0, shielded from the big bad internet.

First things first, yesterday we got xen installed and all, but that’s about as far as we got before the bed was required. Xend almost got it right when it put a startup script in /etc/rc.d/init.d called ‘xend’, but alas, slackware will ignore that file. Personally I simply add the thing to /etc/rc.d/rc.local:

This fixes issue #1, but only the first part.

Before you reboot you might want to alter some xend settings. The file to molest for this is /etc/xend/xend-config.sxp. Mine looks a bit like this:

Almost everything is set to the default value. However, you will want to pay attention to the network-script part at the very least. As you can see I changed this to a little wrapper script.
I won’t go into detail about all the possible Xen networking stuff, see the mailing list and the Xen docs for that, but what we want is the external network bridged to all domU’s, so they can have their own network space there and their external IP address and whatever they want to do without having to mess around in the dom0. To get this working you could stick to the default (network-script network-bridge). Back in the Xen 3.0.3 days this molested my default gateway because their bridge script sucked (it failed to parse some stuff if I remember correctly), but when I tested it today it simply worked.

We want to have an internal network though, so we have to create our own bridge wrapper script.The only thing this script wrapper does is
1. call the network bridge script as the default would do.
2. create a second xen bridge.
Here we go:

As you can see I also add an internal IP address to the bridge. You can make this bridge setup script as complex or as neat as you want, but I can’t be bothered to spend more time on it. It works for me ™

Now that all that is in place, let’s try out the creation of a domU. For testing purposes I steal my images from what used to be jailtime.org, these days it’s called Stacklet. Too bad they don’t have a 64 bit slackware domU yet, but I guess I’ll create that myself a little later. Also, to keep stuff in a place where I can track it, I usually create a /xen directory where I store my host configurations and images.

Edit the slackware.12-2.x86.xen3.cfg file and fix the location of the image file and the kernel. I simply used the same kernel that I use to boot my dom0 for now. Now let’s try it! Try

and it should enter the console of the now booting domU. To get out of the console hit CTRL-[.

Over here I got a bunch of kernel warnings like these:

While these are a nuisance in dmesg, they seem to be fairly harmless. Somewhere on the Xen mailing list someone claims that you can disable CONFIG_DMI in the kernel to get rid of these, which I tend to believe but can’t be bothered. If I’m going to build a new kernel for my domU, I will take the latest vanilla kernel and add Xen domU support in that. It’s a mainline kernel option these days, you can see my post about it here: Xen and booting domU using a vanilla kernel
Login on the domU using root/password, note that your networking actually works, but you still have to enter an IP for eth1 if you want the internal network.

Woohoo, it works!

For an overview of the currently running domains you can use

The xm command has a ton of useful things, so check out the manpage/help for it.

For my real domU I will be using LVM partitions and install slackware 64 on it, but that’s fairly trivial to figure out once you’re this far. So have fun with your Xen install ^^

So after the latest local root exploit thing, and testing it on some machines during har2009, I figured it might be nice to get the patch into my kernels. Running a webserver usually means you’re at risk after all, especially when it’s running wordpress

So I grabbed the latest kernel that had that patch and went for the usual make menuconfig ordeal. After half an hour of selecting “yes”, “maybe”, “I guess so”, and
some XEN options (CONFIG_PARAVIRT_GUEST=y, CONFIG_XEN=y, CONFIG_HVC_DRIVER=y, CONFIG_XEN_BLKDEV_FRONTEND=y, CONFIG_XEN_NETDEV_FRONTEND=y, CONFIG_XEN_KBDDEV_FRONTEND=y, CONFIG_HVC_XEN=y, CONFIG_XEN_BALLOON=y, CONFIG_XEN_SCRUB_PAGES=y, CONFIG_XENFS=y, CONFIG_XEN_COMPAT_XENFS=y) I built the thing.
As I usually build bzImages I also did so here. make bzImage modules modules_install.
Half an hour later I had a shiny bzImage. Let’s try it on a test domain!

I copied the bzImage to /boot, changed the kernel = "/boot/vmlinuz-xen-old" option to the new place, and GO!
This is what it told me:

Huh, where’s my console?
Checking xm list it seemed to be there… but paused… and with constantly increasing ids. Waaait a minute!
Checking the log files (xend.log in this case) revealed a secret:

Note the ‘WARNING’ line. Crashed?!
Gee, that’s … interesting. Why? Took me 5 minutes to find the other log file, but xend-debug.log had a magic line:
ERROR Invalid kernel: xc_dom_find_loader: no loader found
… great. Another why.
In case you’re wondering, the increasing status in ‘xm list’ was caused by the on_crash = 'reboot' line in the xen host config.
I quickly destroyed the doman and changed it to a oneshot try: on_crash = 'destroy'

After some searching another hint presented itself. Aren’t they nice?
This hint was: “Xen is a retarded piece of cancer and can’t decypher the bzImage format, try vmlinux instead”. Aha!
Back to the kernel, make vmlinux, copy, and another try to boot it. Obviously this was way too simple. It still crashed.
Checking my old kernel that works for both dom0 and domU with file I noticed that gz should at least work.

Some more details about the crashing domains can be found using xm dmesg
It gave me something like:

As you can see, that’s really useful…..

A useful page with some info can be found here. It confirms that a gz kernel should work (but bz probably doesn’t) since I’m still running Xen 3.2. However, I can’t find a good reason for why it won’t work.

Google to the rescue! Hard to find a useful keyword, since most xen kernel issues seem to be similar, but eventually I found this tidbit:
tiny kernel patch.
Weird that this should be needed, one would hope that basic shit like this would work after 8 kernel versions, but obviously they’re all retards when it comes to this.
Thanks Jeremy Fitzhardinge for the patch!
Just for quick reference, this is the actual patch:

So much for ‘vanilla’.

However, it boots now:

and a lot more yadieyada until it hits a new wall: root device.
Well, that sounds solvable.

In fact, here’s a solution. What you say?
“AAAAH, I GET NOTHING, IT WON’T EVEN CRASH ANYMORE, JUST NO OUTPUT???!”
Ah yeah, I forgot to mention: they changed the console device as well as the block device. So here’s a solution for both issues:
Edit your xen host config file and make it so:

(xvda1 is what used to be sda1, you can figure out the rest).

And after all this: HAHA! Success is mine!
Except of course for the undeniable fact that my domU has a fucked up fstab and all, but no issues there
Now go mess up your own system! Oh, I see… you already did. Good luck fixing it

Update

I just tested a testing kernel — 2.6.31-rc8, and it seems like they fixed it. Probably has been fixed in the testing branch for quite a while now, but still not in stable 2.6.30
Let’s hope they release 2.6.31 soon.