JUJU BOOTSTRAP TO OPENSTACK FAILS

Heather Lanigan heather.lanigan at canonical.com
Wed Nov 29 21:51:07 UTC 2017


Navin,

Based on the trace we found for the 'No valid host' failure, trying looking
for Filter in the nova_scheduler.log. That might provide a hint as to which
filter is failing.

-Heather

The No valid host trace is as follows:

2017-11-28 22:13:10.710 44686 ERROR nova.conductor.manager Traceback (most
recent call last):
2017-11-28 22:13:10.710 44686 ERROR nova.conductor.manager   File
"/usr/lib/python2.7/dist-packages/nova/conductor/manager.py", line 1027, in
schedule_and_build_instances
2017-11-28 22:13:10.710 44686 ERROR nova.conductor.manager
 instance_uuids)
2017-11-28 22:13:10.710 44686 ERROR nova.conductor.manager   File
"/usr/lib/python2.7/dist-packages/nova/conductor/manager.py", line 626, in
_schedule_instances
2017-11-28 22:13:10.710 44686 ERROR nova.conductor.manager
 request_spec, instance_uuids)
2017-11-28 22:13:10.710 44686 ERROR nova.conductor.manager   File
"/usr/lib/python2.7/dist-packages/nova/scheduler/utils.py", line 586, in
wrapped
2017-11-28 22:13:10.710 44686 ERROR nova.conductor.manager     return
func(*args, **kwargs)
2017-11-28 22:13:10.710 44686 ERROR nova.conductor.manager   File
"/usr/lib/python2.7/dist-packages/nova/scheduler/client/__init__.py", line
52, in select_destinations
2017-11-28 22:13:10.710 44686 ERROR nova.conductor.manager
 instance_uuids)
2017-11-28 22:13:10.710 44686 ERROR nova.conductor.manager   File
"/usr/lib/python2.7/dist-packages/nova/scheduler/client/__init__.py", line
37, in __run_method
2017-11-28 22:13:10.710 44686 ERROR nova.conductor.manager     return
getattr(self.instance, __name)(*args, **kwargs)
2017-11-28 22:13:10.710 44686 ERROR nova.conductor.manager   File
"/usr/lib/python2.7/dist-packages/nova/scheduler/client/query.py", line 33,
in select_destinations
2017-11-28 22:13:10.710 44686 ERROR nova.conductor.manager
 instance_uuids)
2017-11-28 22:13:10.710 44686 ERROR nova.conductor.manager   File
"/usr/lib/python2.7/dist-packages/nova/scheduler/rpcapi.py", line 137, in
select_destinations
2017-11-28 22:13:10.710 44686 ERROR nova.conductor.manager     return
cctxt.call(ctxt, 'select_destinations', **msg_args)
2017-11-28 22:13:10.710 44686 ERROR nova.conductor.manager   File
"/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 169,
in call
2017-11-28 22:13:10.710 44686 ERROR nova.conductor.manager
 retry=self.retry)
2017-11-28 22:13:10.710 44686 ERROR nova.conductor.manager   File
"/usr/lib/python2.7/dist-packages/oslo_messaging/transport.py", line 123,
in _send
2017-11-28 22:13:10.710 44686 ERROR nova.conductor.manager
 timeout=timeout, retry=retry)
2017-11-28 22:13:10.710 44686 ERROR nova.conductor.manager   File
"/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py",
line 578, in send
2017-11-28 22:13:10.710 44686 ERROR nova.conductor.manager     retry=retry)
2017-11-28 22:13:10.710 44686 ERROR nova.conductor.manager   File
"/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py",
line 569, in _send
2017-11-28 22:13:10.710 44686 ERROR nova.conductor.manager     raise result
2017-11-28 22:13:10.710 44686 ERROR nova.conductor.manager
NoValidHost_Remote: No valid host was found.
2017-11-28 22:13:10.710 44686 ERROR nova.conductor.manager Traceback (most
recent call last):
2017-11-28 22:13:10.710 44686 ERROR nova.conductor.manager
2017-11-28 22:13:10.710 44686 ERROR nova.conductor.manager   File
"/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 232,
in inner
2017-11-28 22:13:10.710 44686 ERROR nova.conductor.manager     return
func(*args, **kwargs)
2017-11-28 22:13:10.710 44686 ERROR nova.conductor.manager
2017-11-28 22:13:10.710 44686 ERROR nova.conductor.manager   File
"/usr/lib/python2.7/dist-packages/nova/scheduler/manager.py", line 137, in
select_destinations
2017-11-28 22:13:10.710 44686 ERROR nova.conductor.manager     raise
exception.NoValidHost(reason="")
2017-11-28 22:13:10.710 44686 ERROR nova.conductor.manager
2017-11-28 22:13:10.710 44686 ERROR nova.conductor.manager NoValidHost: No
valid host was found.
2017-11-28 22:13:10.710 44686 ERROR nova.conductor.manager
2017-11-28 22:13:10.710 44686 ERROR nova.conductor.manager
2017-11-28 22:13:10.811 44686 WARNING nova.scheduler.utils
[req-e6a06e6f-5797-4cf5-81dd-3ec2aceb582e 4ac237f6708e4f0688589ce4efcae6af
850ecb27752344278c8816a39e26b6cd - 7565538147a74fc3add7dc0086b6ff1a
7565538147a74fc3add7dc0086b6ff1a] Failed to compute_task_build_instances:
No valid host was found.
Traceback (most recent call last):

  File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py",
line 232, in inner
    return func(*args, **kwargs)

  File "/usr/lib/python2.7/dist-packages/nova/scheduler/manager.py", line
137, in select_destinations
    raise exception.NoValidHost(reason="")

NoValidHost: No valid host was found.
: NoValidHost_Remote: No valid host was found.
Traceback (most recent call last):

  File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py",
line 232, in inner
    return func(*args, **kwargs)

  File "/usr/lib/python2.7/dist-packages/nova/scheduler/manager.py", line
137, in select_destinations
    raise exception.NoValidHost(reason="")

NoValidHost: No valid host was found.


On Mon, Nov 27, 2017 at 11:44 PM, Navin Sridharan <navinsridharan at ymail.com>
wrote:

> Hi Heather,
>
> Following is the CLI command I used to bootstrap JUJU in which I have
> specified the network UUID as private:
>
> *juju bootstrap openstack --debug --config
> image-metadata-url=$SWIFT_URL/simplestreams/images --config
> use-floating-ip=true --config network=private*
>
> Even though I specify the network, it still throws below error message:
>
> *caused by: request
> (http://192.168.122.190:8774/v2.1/850ecb27752344278c8816a39e26b6cd/servers
> <http://192.168.122.190:8774/v2.1/850ecb27752344278c8816a39e26b6cd/servers>)
> returned unexpected status: 409; error info: {"conflictingRequest":
> {"message": "Multiple possible networks found, use a Network ID to be more
> specific.", "code": 409}}}])*
>
> When I boot up the instance manually in Openstack cloud, I am able to
> successfully boot up and I mentioned the network as "private"( PFA
> screenshot) . Below given is the log from nova.compute log:
>
>
> 2017-11-28 02:19:00.839 278905 INFO oslo.privsep.daemon
> [req-242cfa20-8f75-4643-b2ff-447a5ed557ac 4ac237f6708e4f0688589ce4efcae6af
> 850ecb27752344278c8816a39e26b6cd - 7565538147a74fc3add7dc0086b6ff1a
> 7565538147a74fc3add7dc0086b6ff1a] Running privsep helper: ['sudo',
> 'nova-rootwrap', '/etc/nova/rootwrap.conf', 'privsep-helper',
> '--config-file', '/etc/nova/nova.conf', '--config-file',
> '/etc/nova/nova-compute.conf', '--privsep_context',
> 'vif_plug_ovs.privsep.vif_plug', '--privsep_sock_path',
> '/tmp/tmp0T_zTc/privsep.sock']
> 2017-11-28 02:19:01.734 278905 INFO oslo.privsep.daemon
> [req-242cfa20-8f75-4643-b2ff-447a5ed557ac 4ac237f6708e4f0688589ce4efcae6af
> 850ecb27752344278c8816a39e26b6cd - 7565538147a74fc3add7dc0086b6ff1a
> 7565538147a74fc3add7dc0086b6ff1a] Spawned new privsep daemon via rootwrap
> 2017-11-28 02:19:01.738 278905 WARNING oslo.privsep.daemon [-] privsep
> log: sudo: unable to resolve host rack-vir-m2
> 2017-11-28 02:19:01.738 278905 WARNING oslo.privsep.daemon [-] privsep
> log: Option "logdir" from group "DEFAULT" is deprecated. Use option
> "log-dir" from group "DEFAULT".
> 2017-11-28 02:19:01.627 2076418 INFO oslo.privsep.daemon [-] privsep
> daemon starting
> 2017-11-28 02:19:01.633 2076418 INFO oslo.privsep.daemon [-] privsep
> process running with uid/gid: 0/0
> 2017-11-28 02:19:01.638 2076418 INFO oslo.privsep.daemon [-] privsep
> process running with capabilities (eff/prm/inh):
> CAP_NET_ADMIN/CAP_NET_ADMIN/none
> 2017-11-28 02:19:01.638 2076418 INFO oslo.privsep.daemon [-] privsep
> daemon running as pid 2076418
> 2017-11-28 02:19:02.828 278905 INFO os_vif [req-242cfa20-8f75-4643-b2ff-447a5ed557ac
> 4ac237f6708e4f0688589ce4efcae6af 850ecb27752344278c8816a39e26b6cd -
> 7565538147a74fc3add7dc0086b6ff1a 7565538147a74fc3add7dc0086b6ff1a]
> Successfully plugged vif VIFBridge(active=False,address=fa:16:3e:73:ed:ff,
> bridge_name='qbre78939c1-66',has_traffic_filtering=True,id=
> e78939c1-66d4-4139-929b-7891295e60eb,network=Network(
> 383fd64b-4c4c-497d-809d-3bcf8ed72e1c),plugin='ovs',port_profile=
> VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_
> name='tape78939c1-66')
> 2017-11-28 02:19:11.995 278905 INFO nova.compute.manager [-] [instance:
> 7a732fc2-0a35-4127-9138-22f58e580b2f] VM Started (Lifecycle Event)
> 2017-11-28 02:19:12.015 278905 INFO nova.virt.libvirt.driver [-]
> [instance: 7a732fc2-0a35-4127-9138-22f58e580b2f] Instance spawned
> successfully.
> 2017-11-28 02:19:12.015 278905 INFO nova.compute.manager
> [req-242cfa20-8f75-4643-b2ff-447a5ed557ac 4ac237f6708e4f0688589ce4efcae6af
> 850ecb27752344278c8816a39e26b6cd - 7565538147a74fc3add7dc0086b6ff1a
> 7565538147a74fc3add7dc0086b6ff1a] [instance: 7a732fc2-0a35-4127-9138-22f58e580b2f]
> Took 11.56 seconds to spawn the instance on the hypervisor.
> 2017-11-28 02:19:12.508 278905 INFO nova.compute.manager
> [req-ffc7ee12-dbc2-407b-9c44-5ad70de22585 - - - - -] [instance:
> 7a732fc2-0a35-4127-9138-22f58e580b2f] VM Paused (Lifecycle Event)
> 2017-11-28 02:19:12.547 278905 INFO nova.compute.manager
> [req-242cfa20-8f75-4643-b2ff-447a5ed557ac 4ac237f6708e4f0688589ce4efcae6af
> 850ecb27752344278c8816a39e26b6cd - 7565538147a74fc3add7dc0086b6ff1a
> 7565538147a74fc3add7dc0086b6ff1a] [instance: 7a732fc2-0a35-4127-9138-22f58e580b2f]
> Took 100.40 seconds to build instance.
>
>
>
> [image: Inline image]
>
>
> Just to let you know that I am using "nosdn" scenario and not
> Opencontrail. Thanks for your continued support, much appreciated!!
>
> Regards,
> Navin S
>
>
>
>
> On Monday, 27 November 2017 7:46 PM, Heather Lanigan <
> heather.lanigan at canonical.com> wrote:
>
>
> Navin,
>
> The error thrown by juju is received from OpenStack when we tried to boot
> an instance.  OpenStack is requiring the name or uuid of the network to use
> because it doesn't know which network to choose.  The fix for that error is
> using --config network=<network> during bootstrap.
>
> Were there any hints in the nova log regarding details for the No Valid
> Hosts failure?  When you manually booted the OpenStack instance, did you
> need to specify `--nic net-id=net-uuid` or `--nic net-name=network-name`?
>
> Unfortunately I'm not familiar with Opencontrail SDN.
>
> -Heather
>
> On Mon, Nov 27, 2017 at 7:27 PM, Navin Sridharan <navinsridharan at ymail.com
> > wrote:
>
> Hi Heather,
>
> Thank you for the response. Earlier I used Opencontrail SDN while
> deploying JUJU, so I am guessing the issue was to do with the port binding
> as the script I was using to deploy was not compatible with the
> OpenContrail SDN. I tried re-deploying with "nosdn", but JUJU now throws
> the following error message:
>
> *nova.SecurityGroupName{Name:" juju-0fa027ff-0141-420f-8ec9-
> f27860520823-4325eb8d-2171- 427b-82cb-8beada255496-0"}}, Networks:[]nova.
> ServerNetworks{}, AvailabilityZone:"nova", Metadata:map[string]string{"
> juju-is-controller":"true", "juju-model-uuid":"4325eb8d-
> 2171-427b-82cb-8beada255496", "juju-controller-uuid":"
> 0fa027ff-0141-420f-8ec9- f27860520823"}, ConfigDrive:false}*
> *caused by: request (**http://192.168.122.190:8774/ v2.1/
> 850ecb27752344278c8816a39e26b6 cd/servers*
> <http://192.168.122.190:8774/v2.1/850ecb27752344278c8816a39e26b6cd/servers>*)
> returned unexpected status: 409; error info: {"conflictingRequest":
> {"message": "Multiple possible networks found, use a Network ID to be more
> specific.", "code": 409}}*
>
> Also, I am able to boot up an instance manually in the OpenStack cloud
> using m1.medium and the status of the instance reads "Active".  Any
> insights on the above error please?
>
> I see my error related to this post - https://bugs.launchpad.net/
> juju/+bug/1570219 <https://bugs.launchpad.net/juju/+bug/1570219>
>
> Regards,
> Navin S
>
>
>
>
> On Monday, 27 November 2017 6:55 PM, Heather Lanigan <
> heather.lanigan at canonical.com > wrote:
>
>
> Navin,
>
> From your email, it appears that juju is finding the network specified
> correctly.  Is there more information leading to believe that is the
> problem?
>
> Are you able to boot an instance within the OpenStack Cloud specifying the
> same network, using the m1.medium flavor?  I'd recommend checking the nova
> logs to see what more info can be found on the cause of the No Valid Host
> error.
>
> -Heather
>
> On Mon, Nov 27, 2017 at 5:22 PM, Navin Sridharan <navinsridharan at ymail.com
> > wrote:
>
> Hi all,
>
> I am fairly new to JUJU and am currently working on deploying OAI (Open
> Air Interface) on OPNFV using JOID installer.
>
> *H/W used *: HP- Proliant DL380G6 G6 bare metal box
> *OS*              : Ubuntu 16.04 Xenial
>
> I have spun up 3 VM's using MAAS (Metal as a Service) that's been deployed
> in the bare metal box. JUJU has been installed in the bare metal and I am
> trying to bootstrap it to Openstack cloud whose control and compute nodes
> are hosted on these VM's. Ultimate objective is to deploy OAI model using
> JUJU in the Openstack cloud.
>
> During bootsrapping using the below command, it throws an error "failed
> to bootstrap model: cannot start bootstrap instance: cannot run instance:
> No valid host was found."  I believe the issue is with the
> Openstack's network UUID that's passed on as a parameter while
> bootstrapping.
>
> *juju bootstrap openstack --debug --config image-metadata-url=$SWIFT_URL/
> simplestreams/images --config use-floating-ip=true --config network=private*
>
> *Below given is the snippet from the error log*:
>
>  10:36:16 DEBUG juju.provider.openstack provider.go:1022 using network id
> "383fd64b-4c4c-497d-809d- 3bcf8ed72e1c"
> 10:36:20 INFO  juju.provider.openstack provider.go:1146 trying to build
> instance in availability zone "nova"
> 10:36:33 INFO  juju.provider.openstack provider.go:1126 Instance
> "78df0280-a1f0-4993-8416- c0be69050a11" in ERROR state with fault "No valid
> host was found. "
> 10:36:33 INFO  juju.provider.openstack provider.go:1127 Deleting instance
> "78df0280-a1f0-4993-8416- c0be69050a11" in ERROR state
> 10:36:33 INFO  juju.provider.openstack provider.go:1155 failed to build
> instance in availability zone "nova"
> 10:36:33 ERROR juju.cmd.juju. commands bootstrap.go:496 failed to
> bootstrap model: cannot start bootstrap instance: cannot run instance: No
> valid host was found.
>
>
>
> From the error log I see that it uses *network id
> "383fd64b-4c4c-497d-809d- 3bcf8ed72e1c" *which is the network id of
> private subnet and I confirmed that from the Openstack GUI (PFA
> screenshot), but then fails to bootstrap. I am stuck at this point in time
> and any response would be considered more than a help. Thank you all!!
>
> [image: Inline image]
>
>
> Regards,
> Navin S
>
> --
> Juju mailing list
> Juju at lists.ubuntu.com
> Modify settings or unsubscribe at: https://lists.ubuntu.com/
> mailman/listinfo/juju <https://lists.ubuntu.com/mailman/listinfo/juju>
>
>
>
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/juju/attachments/20171129/c685ff2b/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: datauri-file.jpeg
Type: image/jpeg
Size: 141857 bytes
Desc: not available
URL: <https://lists.ubuntu.com/archives/juju/attachments/20171129/c685ff2b/attachment.jpeg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 98967 bytes
Desc: not available
URL: <https://lists.ubuntu.com/archives/juju/attachments/20171129/c685ff2b/attachment.png>


More information about the Juju mailing list