[ec2] karmic on ec2: udev doesn't start

[Eric Hammond]

Andreas:

Thanks for finding and reporting this.  I can't imagine how long it
would have taken me to figure that out the next time I built a Karmic
AMI.  I hate debugging EC2 network boot problems :-/

Though (as we discussed on IRC) the ideal solution would be for
Canonical to publish 2.6.25+ (Karmic) kernels on EC2, I found a couple
workarounds until that happens:

1. If you start with a Karmic AMI that currently has an older udev like
 ami-19a34270 (alestic/ubuntu-9.10-karmic-base-20090623) then you can
prevent udev from being upgraded using a command like:

  echo udev hold | sudo dpkg --set-selections

then
  apt-get update && apt-get dist-upgrade -y

The main problem with this is that newer versions of the Karmic AMI are
probably not going to have the older udev installed and Karmic is not
much use if we aren't going to be able to publish new images.

2. Robert Coup shared that he drops udev on his VMs and creates
/var/run/network in /etc/init.d/networking on startup.  I found that it
was also necessary to create /dev/urandom so sshd would start, and ended
up with the following recipe after upgrading Karmic and before rebooting:

  sudo apt-get update && sudo apt-get dist-upgrade -y

  sudo apt-get purge -y udev

  cat <<EOM | sudo tee /etc/rcS.d/S39no-udev
#!/bin/sh
mkdir -p /var/run/network
cd /dev
/sbin/MAKEDEV urandom
/sbin/MAKEDEV random
EOM
  sudo chmod 755 /etc/rcS.d/S39no-udev

This worked on the Alestic.com image using the 2.6.21fc8 kernel, but I
was not able to get it to work after upgrading the Canonical Intrepid
AMI through Jaunty to Karmic.  On the Canonical image, not only did
network not come back up after a reboot, but EC2 was not returning any
console output for the instance, so it will be difficult to debug.

I'm not sure how much folks might need udev on EC2, but I'm tempted to
use this second approach until newer kernels are available or udev loses
the dependency on them.

And, for other folks' reference, Andreas submitted the following bug:

  https://bugs.launchpad.net/ubuntu-on-ec2/+bug/397187

--
Eric Hammond
ehammond at thinksome.com



Andreas Hasenack wrote:
> Just a heads up, if not known already.
> 
> udev in karmic is using signalfd(2), which doesn't exist in kernel < 2.6.25.
> 
> On EC2, both alestic images and the official one upgraded from jaunty fail with:
> 
> root at ip-10-244-15-193:~# udevd
> error getting signalfd
> 
> strace shows:
> 8706  SYS_327(0xffffffff, 0xbfc5238c, 0x8, 0, 0x2f4ff4) = -1 ENOSYS (Function
> not implemented)
> 8706  write(2, "error getting signalfd\n", 23) = 23
> 
> The kernel is (alestic image):
> 2.6.21.7-2.fc8xen
> 
>