[Bug 894279] Re: cloud-init fails to fetch metadata with OpenStack

[Scott Moser]

2 seconds might have been low, I agree, however the following works fine:
 $ for s in cnn.com brickies.net slashdot.org espn.com; do
    python -c 'import urllib2, sys; req=urllib2.Request(sys.argv[1]); \
     resp=urllib2.urlopen(req, timeout=float(sys.argv[2])); resp.read()'
     "http://$s" .1
   done

That does the same urllib2.urlopen with a timeout that cloud-init does.
Note the '.1' second timeout for all of those sites that are doing loads
more than serving a 10s of bytes of text (ok, brickies.net is not, but it
is also on shared hosting).  I had to set that to .08 to get failures on
any of the listed sites.  3 seconds (the default in the code) should be
plenty.

But... we can up it,and it probably was arbitrarily low.

we had to set a timeout otherwise there is none, and due to the fact
that cloud-init retries the MD again and again (giving it time to come
up due to bugs/bad-design in EC2 and Eucalyptus).  If the address
169.254.169.254 has been firewalled off, you potentially hang indefinitely
trying to connect (in the case where you're not running in a cloud).

I urge anyone interested in this to help to fix this bug in openstack.
There is a proposed solution at http://pad.lv/851159 although I have not
tested.  Additionally, it is suggested (by vish and others) that you can
reduce this slowness by running an API server on each of the nodes.  I
don't know how feasible that is.  It is simply absurd to not be able to
respond to requests to the metadata service in less than 3 seconds (or 5
or 10).

The issue we were trying to protect against really was "non-cloud"
installation of cloud-init.  Ie, in 10.04 or maybe maverick, install of
cloud-init on your laptop, and reboot would be blocked boot by cloud-init
for something like 1500 seconds. No fun.

Regarding your suggestion of 60 second timeout, and 5 retries, that
would also be insufficient.  There is a case where the metadata service
isn't ready yet (this occurred on Eucalyptus and we've also seen it on
EC2, although not recently on either).  In that case, the connections
would immeditely get "connection refused", so the 5 retries would occur
within 5 seconds, and never find the Metadata service.

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to cloud-init in Ubuntu.
https://bugs.launchpad.net/bugs/894279

Title:
  cloud-init fails to fetch metadata with OpenStack

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/894279/+subscriptions