Posted on by Michael Lucas

Wherein I learn about initrd

Post summary: will someone PLEASE port a recent KVM to any BSD? There’s beer in it for you.

I’ve been attempting to upgrade my diskless virtualization cluster to Ubuntu 10.10. Diskless boot worked fine in the ESXi test area, but real hardware would not boot. This same hardware booted fine with Ubuntu 10.04 and 9.whatever. When I looked at the console, I saw:

ipconfig: no devices to configure
ipconfig: no devices to configure
ipconfig: no devices to configure
ipconfig: no devices to configure
/init: .: line 3: can't open '/tmp/net-*.conf'
[ 2.300079] Kernel panic - not syncing: Attempted to kill init!
[ 2.306052] Pid: 1, comm: init Not tainted 2.6.35-27-server #48-Ubuntu
[ 2.312653] Call Trace:
[ 2.315161] [] panic+0x90/0x113
[ 2.320025] [] forget_original_parent+0x33d/0x350
[ 2.326433] [] ? put_files_struct+0xc4/0xf0
[ 2.332339] [] exit_notify+0x1b/0x190
[ 2.337699] [] do_exit+0x1d5/0x400
[ 2.342817] [] ? do_page_fault+0x159/0x350
[ 2.348609] [] do_group_exit+0x55/0xd0
[ 2.354076] [] sys_exit_group+0x17/0x20
[ 2.359617] [] system_call_fastpath+0x16/0x1b

The useful messages are obviously further up, but the scrollback buffer is fubar. (Apparently when an Ubuntu box dies, it dies really really hard.) A serial console let me scroll back through the boot messages.

...
[ 2.004954] Uniform CD-ROM driver Revision: 3.20
[ 2.009944] sr 0:0:1:0: Attached scsi generic sg0 type 5
[ 2.015651] Freeing unused kernel memory: 836k freed
[ 2.021283] Write protecting the kernel read-only data: 10240k
[ 2.027551] Freeing unused kernel memory: 320k freed
[ 2.033118] Freeing unused kernel memory: 1620k freed
Loading, please wait...
[ 2.063067] udev[81]: starting version 163
Begin: Loading essential drivers ... done.
Begin: Running /scripts/init-premount ... done.
Begin: Mounting root file system ... Begin: Running /scripts/nfs-top ... done.
FATAL: Could not load /lib/modules/2.6.35-27-server/modules.dep: No such file or directory
FATAL: Could not load /lib/modules/2.6.35-27-server/modules.dep: No such file or directory
ipconfig: no devices to configure
ipconfig: no devices to configure
ipconfig: no devices to configure

The machine cannot find its modules directory? Odd. A packet sniffer found that the diskless client didn’t send an NFS request. It was just giving up after running initrd. I carefully reviewed the serial console output and compared it to the test Ubuntu systems, and found that the initial ramdisk wasn’t attaching a device driver to the Ethernet interface.

Initrd is an “initial ramdisk.” It loads a kernel and device drivers, for the purpose of finding the root file system and loading the real kernel and actual device drivers. If you install a machine in one environment, the initial ramdisk includes only the device drivers for that environment.

Checking /etc/udev/rules.d/70-persistent-net.rules of the older system revealed:

# PCI device 0x14e4:0x1659 (tg3)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:17:31:d8:42:52", ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1"

The Ethernet cards on my physical servers use Linux’s tg3 driver.

To add this driver to initrd, I went to the Ubuntu 10.10 server on my ESXi test box and created the file /usr/share/initramfs-tools/modules.d/tg3. That file contained a single word, tg3. I then created the new initrd with:

# update-initramfs -u -k all
# mkinitramfs -o /home/mwlucas/initrd.img-2.6.35-27-server-pxe

Copy that image to my TFTP server, reboot the hardware, and everything boots.

4 Replies to “Wherein I learn about initrd”

  1. That Linux still uses such archaic booting mechanism as initrd in 2011 always makes me laugh 🙂 Unfortunately, most of the distro’s higher level boot processes (rc.d equivalents) are also laughable 🙁
    Linux has its moments, but booting & low-level configuration (no boot dependency tracking, persistent ethX devices in Ubuntu… WTF?) is certainly not one of those.

  2. >It loads a kernel and device drivers, for the purpose of finding the root file system and loading the real kernel and actual device drivers.

    Not true. The kernel is vmlinuz/bzImage/whatever, not initrd, and it *is* the real kernel (no other kernel is used). Initrd is a compressed fs image containing modules and some scripts.

  3. I did the same procedure in order to fix “/init: .: line 3: can’t open ‘/tmp/net-*.conf'”. However after a reboot, I get:
    Gave up waiting for root device. Common problems:
    – Boot args (cat /proc/cmdline)
    – Check rootdelay= (did the system wait long enough?)
    – Check root =(did the system wait for right device?)
    – Missing modules (cat /proc/modules; ls /dev)
    ALERT! /dev/nfs does not exist. Dropping to a shell!!

    What is next?

  4. exelent tutorial, i had the same problem with Ubuntu 12.04.2 LTS , partially the probelm was solved because of this, i did also used a serial console as per your manual.

    vey good tutorial ,,

    Konstantin

Comments are closed.