I’m using Ubuntu servers with qemu-kvm as a virtualization solution. The software included in 10.04LTS includes a variety of annoyances, such as broken PXE, odd bridge behavior, and “general weirdness.” Although 10.10 is not supported in the long term, I decided to try it.
The good news is, the 10.10 virtualization stack works much better. The bad news is, 10.10 didn’t want to run on my diskless hardware. Boot attempts all died with many lines of:
ipconfig: no devices to configure
and a message about killing init. The server was quite explicit that it was dead, and how it was dying, but didn’t leave any clues as to what had killed it. I’m sure that the console showed useful error messages, but they had scrolled off the top of the screen.
The manual says that if you hit shift-PageUp, Ubuntu should page up through the console messages. That should be amended to read “unless init is dead and your keyboard LEDs are blinking slowly but steadily.”
The only way to resolve this problem is to see the error messages that say why the machine crashed. So, a serial console. I want PXE messages, initrd messages, and kernel boot messages sent to serial console. These are all controlled by the /tftpboot/pxelinux.cfg/machine
file. The actual file name is the MAC address of the booting NIC.
If you want to get messages from the pxe and initrd boot stages, the pxelinux.cfg file’s first line must include the SERIAL
statement. If you want to get console messages from the booting kernel and/or log into the running system over the serial console, you must append a serial
statement to the kernel boot command. The end result for a serial console looks like this:
SERIAL 0 115200
LABEL linux
KERNEL vmlinuz-2.6.35-27-server
APPEND root=/dev/nfs initrd=initrd.img-2.6.35-27-server-pxe nfsroot=192.0.2.1:/nfsroot ip=dhcp rw console=tty0 console=ttyS0,115200n8
TIMEOUT 0
The Web site will probably wrap the APPEND statement around, but that line and everything beneath it down to TIMEOUT is a single line.
If you want a serial login in multiuser mode, you need to create a script to activate the terminal. Here’s the Ubuntu default terminal script:
/etc/init/ttyS0.conf
# ttyS0 - getty
#
# This service maintains a getty on ttyS0 from the point the system is
# started until it is shut down again.
start on stopped rc RUNLEVEL=[2345]
stop on runlevel [!2345]
respawn
exec /sbin/getty -L 115200 ttyS0 vt102
The next time you reboot your diskless box, you should have a full serial console.
Some time soon, more on the actual error and how I fixed it.
Hi Michael,
Great blog, and pxelinux.cfg’s pickiness with regard to top-of-file comments that you note in a follow-up to this post is definitely good to know. I was wondering if you’d be able to elaborate on your statement that:
> The software included in 10.04LTS includes a variety of
> annoyances, such as broken PXE, odd bridge behavior, and
> “general weirdness.”
We have PXE booting of Ubuntu 10.04 working fine in our environment (the current PXE server is a Solaris box, not a Ubuntu box, so I can’t speak to PXE booting from an Ubuntu server), but we’re about to start testing KVM on 10.04 as a replacement for our current Xen boxes, and I would love to have a heads up on any issues for which I should keep my eyes open (especially the issues which you considered sufficient deal breakers to move you to 10.10). Not that I’m adverse to learning about software bugs through my own blood sweat and tears, I just appreciate short cuts where I can find them. 🙂
As an aside: Did you consider KVM on CentOS or another RHEL rebuild (eg: Scientific Linux)? If so, what made you decide to go with Ubuntu? (I know folks who have moved from KVM on Ubuntu to KVM RHEL/CentOS/Fedora, but none who have done the reverse.)
Best,
-Chris Allison
s/blood sweat and tears/blood, sweat, and tears/
Unfortunately, I didn’t track all of the problems I had. Some of them are documented elsewhere, e.g., problems with ldap thanks to the shift from OpenSSL to GNUTLS. Now and then virtual machines crashed, and my research showed that it was KVM bugs fixed in later versions that were not back-ported to 10.04. The syntax for KVM changed during updates to 10.04, so I could no longer use OpenNebula or virt-manager to PXE-boot machines.
Why Ubuntu? Favoritism. The company owner is a Ubuntu fan.
One of our NFS servers is OpenSolaris. It’s fine, although I prefer BSD… easier to get lots of disk into commodity hardware.