[Bug 1656010] [NEW] Incorrect notification to nova about ironic baremetall port (for nodes in 'cleaning' state)
George Shuklin
1656010 at bugs.launchpad.net
Thu Jan 12 15:52:41 UTC 2017
Public bug reported:
version: newton (2:9.0.0-0ubuntu1~cloud0)
When neutron trying to bind port for Ironic baremetall node, it sending
wrong notification to nova about port been ready. neutron send it with
'device_id' == ironic-node-id, and nova rejects it as 'not found' (there
is no nova instance with such id).
Log:
neutron.db.provisioning_blocks[22265]: DEBUG Provisioning for port db3766ad-f82b-437d-b8b2-4133a92b1b86 completed by entity DHCP. [req-49434e88-4952-4e9d-a1c4-41dbf6c0091a - - - - -] provisioning_complete /usr/lib/python2.7/dist-packages/neutron/db/provisioning_blocks.py:147
neutron.db.provisioning_blocks[22265]: DEBUG Provisioning complete for port db3766ad-f82b-437d-b8b2-4133a92b1b86 [req-49434e88-4952-4e9d-a1c4-41dbf6c0091a - - - - -] provisioning_complete /usr/lib/python2.7/dist-packages/neutron/db/provisioning_blocks.py:153
neutron.callbacks.manager[22265]: DEBUG Notify callbacks [('neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned--9223372036854150578', <bound method Ml2Plugin._port_provisioned of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7fc005834550>>)] for port, provisioning_complete [req-49434e88-4952-4e9d-a1c4-41dbf6c0091a - - - - -] _notify_loop /usr/lib/python2.7/dist-packages/neutron/callbacks/manager.py:142
neutron.plugins.ml2.plugin[22265]: DEBUG Port db3766ad-f82b-437d-b8b2-4133a92b1b86 cannot update to ACTIVE because it is not bound. [req-49434e88-4952-4e9d-a1c4-41dbf6c0091a - - - - -] _port_provisioned /usr/lib/python2.7/dist-packages/neutron/plugins/ml2/plugin.py:224
oslo_messaging._drivers.amqpdriver[22265]: DEBUG sending reply msg_id: 254703530cd3440584c980d72ed93011 reply queue: reply_8b6e70ad5191401a9512147c4e94ca71 time elapsed: 0.0452275519492s [req-49434e88-4952-4e9d-a1c4-41dbf6c0091a - - - - -] _send_reply /usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py:73
neutron.notifiers.nova[22263]: DEBUG Sending events: [{'name': 'network-changed', 'server_uuid': u'd02c7361-5e3a-4fdf-89b5-f29b3901f0fc'}] send_events /usr/lib/python2.7/dist-packages/neutron/notifiers/nova.py:257
novaclient.v2.client[22263]: DEBUG REQ: curl -g -i --insecure -X POST http://nova-api.p.ironic-dal-1.servers.com:28774/v2/93c697ef6c2649eb9966900a8d6a73d8/os-server-external-events -H "User-Agent: python-novaclient" -H "Content-Type: application/json" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}592539c9fcd820d7e369ea58454ee17fe7084d5e" -d '{"events": [{"name": "network-changed", "server_uuid": "d02c7361-5e3a-4fdf-89b5-f29b3901f0fc"}]}' _http_log_request /usr/lib/python2.7/dist-packages/keystoneauth1/session.py:337
novaclient.v2.client[22263]: DEBUG RESP: [404] Content-Type: application/json; charset=UTF-8 Content-Length: 78 X-Compute-Request-Id: req-a029af9e-e460-476f-9993-4551f3b210d6 Date: Thu, 12 Jan 2017 15:43:37 GMT Connection: keep-alive
RESP BODY: {"itemNotFound": {"message": "No instances found for any event", "code": 404}}
_http_log_response /usr/lib/python2.7/dist-packages/keystoneauth1/session.py:366
novaclient.v2.client[22263]: DEBUG POST call to compute for http://nova-api.p.ironic-dal-1.servers.com:28774/v2/93c697ef6c2649eb9966900a8d6a73d8/os-server-external-events used request id req-a029af9e-e460-476f-9993-4551f3b210d6 _log_request_id /usr/lib/python2.7/dist-packages/novaclient/client.py:85
neutron.notifiers.nova[22263]: DEBUG Nova returned NotFound for event: [{'name': 'network-changed', 'server_uuid': u'd02c7361-5e3a-4fdf-89b5-f29b3901f0fc'}] send_events /usr/lib/python2.7/dist-packages/neutron/notifiers/nova.py:263
oslo_messaging._drivers.amqpdriver[22265]: DEBUG received message msg_id: 0bf04ac8fedd4234bd6cd6c04547beca reply to reply_8b6e70ad5191401a9512147c4e94ca71 __call__ /usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py:194
neutron.db.provisioning_blocks[22265]: DEBUG Provisioning complete for port db3766ad-f82b-437d-b8b2-4133a92b1b86 [req-47c505d7-4eb5-4c71-9656-9e0927408822 - - - - -] provisioning_complete /usr/lib/python2.7/dist-packages/neutron/db/provisioning_blocks.py:153
Port info:
+---------------------+---------------------------------------------------------------------------------------+
| Field | Value |
+---------------------+---------------------------------------------------------------------------------------+
| admin_state_up | True |
| binding:host_id | d02c7361-5e3a-4fdf-89b5-f29b3901f0fc |
| binding:profile | {"local_link_information": [{"switch_info": "c426s1", "port_id": "1/1/21", |
| | "switch_id": "60:96:9f:69:b4:b4"}]} |
| binding:vif_details | {} |
| binding:vif_type | binding_failed |
| binding:vnic_type | baremetal |
| created_at | 2017-01-12T15:23:36Z |
| description | |
| device_id | d02c7361-5e3a-4fdf-89b5-f29b3901f0fc |
| device_owner | baremetal:none |
| extra_dhcp_opts | {"opt_value": "204.74.228.4", "ip_version": 4, "opt_name": "tftp-server"} |
| | {"opt_value": "204.74.228.4", "ip_version": 4, "opt_name": "server-ip-address"} |
| | {"opt_value": "pxelinux.0", "ip_version": 4, "opt_name": "bootfile-name"} |
| fixed_ips | {"subnet_id": "5402755a-0d8b-447d-9753-f3ba1ec39c22", "ip_address": "hidden"} |
| id | bc46cbdf-a82e-409d-9332-9eeb81aa0a94 |
| mac_address | 18:66:ee:aa:dd:cc |
| name | |
| network_id | 4b352ae7-141b-4c3f-a132-f5c006dc056c |
| project_id | 7d450ecf00d64399aeb93bc122cb6dae |
| revision_number | 8 |
| status | DOWN |
| tenant_id | 7d450ecf00d64399aeb93bc122cb6dae |
| updated_at | 2017-01-12T15:23:37Z |
+---------------------+---------------------------------------------------------------------------------
ironic node:
ironic node-list
/usr/lib/python2.7/dist-packages/urllib3/connection.py:303: SubjectAltNameWarning: Certificate for baremetal.ironic-dal-1.mgm.servers.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/shazow/urllib3/issues/497 for details.)
SubjectAltNameWarning
+--------------------------------------+-------+---------------+-------------+--------------------+-------------+
| UUID | Name | Instance UUID | Power State | Provisioning State | Maintenance |
+--------------------------------------+-------+---------------+-------------+--------------------+-------------+
| d02c7361-5e3a-4fdf-89b5-f29b3901f0fc | s8002 | None | power on | clean wait | True |
+--------------------------------------+-------+---------------+-------------+--------------------+-------------+
** Affects: ironic
Importance: Undecided
Status: New
** Affects: neutron
Importance: Undecided
Status: New
** Affects: ironic (Ubuntu)
Importance: Undecided
Status: New
** Affects: neutron (Ubuntu)
Importance: Undecided
Status: New
** Also affects: ironic
Importance: Undecided
Status: New
** Also affects: ironic (Ubuntu)
Importance: Undecided
Status: New
** Also affects: neutron (Ubuntu)
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to ironic in Ubuntu.
https://bugs.launchpad.net/bugs/1656010
Title:
Incorrect notification to nova about ironic baremetall port (for nodes
in 'cleaning' state)
Status in Ironic:
New
Status in neutron:
New
Status in ironic package in Ubuntu:
New
Status in neutron package in Ubuntu:
New
Bug description:
version: newton (2:9.0.0-0ubuntu1~cloud0)
When neutron trying to bind port for Ironic baremetall node, it
sending wrong notification to nova about port been ready. neutron send
it with 'device_id' == ironic-node-id, and nova rejects it as 'not
found' (there is no nova instance with such id).
Log:
neutron.db.provisioning_blocks[22265]: DEBUG Provisioning for port db3766ad-f82b-437d-b8b2-4133a92b1b86 completed by entity DHCP. [req-49434e88-4952-4e9d-a1c4-41dbf6c0091a - - - - -] provisioning_complete /usr/lib/python2.7/dist-packages/neutron/db/provisioning_blocks.py:147
neutron.db.provisioning_blocks[22265]: DEBUG Provisioning complete for port db3766ad-f82b-437d-b8b2-4133a92b1b86 [req-49434e88-4952-4e9d-a1c4-41dbf6c0091a - - - - -] provisioning_complete /usr/lib/python2.7/dist-packages/neutron/db/provisioning_blocks.py:153
neutron.callbacks.manager[22265]: DEBUG Notify callbacks [('neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned--9223372036854150578', <bound method Ml2Plugin._port_provisioned of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7fc005834550>>)] for port, provisioning_complete [req-49434e88-4952-4e9d-a1c4-41dbf6c0091a - - - - -] _notify_loop /usr/lib/python2.7/dist-packages/neutron/callbacks/manager.py:142
neutron.plugins.ml2.plugin[22265]: DEBUG Port db3766ad-f82b-437d-b8b2-4133a92b1b86 cannot update to ACTIVE because it is not bound. [req-49434e88-4952-4e9d-a1c4-41dbf6c0091a - - - - -] _port_provisioned /usr/lib/python2.7/dist-packages/neutron/plugins/ml2/plugin.py:224
oslo_messaging._drivers.amqpdriver[22265]: DEBUG sending reply msg_id: 254703530cd3440584c980d72ed93011 reply queue: reply_8b6e70ad5191401a9512147c4e94ca71 time elapsed: 0.0452275519492s [req-49434e88-4952-4e9d-a1c4-41dbf6c0091a - - - - -] _send_reply /usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py:73
neutron.notifiers.nova[22263]: DEBUG Sending events: [{'name': 'network-changed', 'server_uuid': u'd02c7361-5e3a-4fdf-89b5-f29b3901f0fc'}] send_events /usr/lib/python2.7/dist-packages/neutron/notifiers/nova.py:257
novaclient.v2.client[22263]: DEBUG REQ: curl -g -i --insecure -X POST http://nova-api.p.ironic-dal-1.servers.com:28774/v2/93c697ef6c2649eb9966900a8d6a73d8/os-server-external-events -H "User-Agent: python-novaclient" -H "Content-Type: application/json" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}592539c9fcd820d7e369ea58454ee17fe7084d5e" -d '{"events": [{"name": "network-changed", "server_uuid": "d02c7361-5e3a-4fdf-89b5-f29b3901f0fc"}]}' _http_log_request /usr/lib/python2.7/dist-packages/keystoneauth1/session.py:337
novaclient.v2.client[22263]: DEBUG RESP: [404] Content-Type: application/json; charset=UTF-8 Content-Length: 78 X-Compute-Request-Id: req-a029af9e-e460-476f-9993-4551f3b210d6 Date: Thu, 12 Jan 2017 15:43:37 GMT Connection: keep-alive
RESP BODY: {"itemNotFound": {"message": "No instances found for any event", "code": 404}}
_http_log_response /usr/lib/python2.7/dist-packages/keystoneauth1/session.py:366
novaclient.v2.client[22263]: DEBUG POST call to compute for http://nova-api.p.ironic-dal-1.servers.com:28774/v2/93c697ef6c2649eb9966900a8d6a73d8/os-server-external-events used request id req-a029af9e-e460-476f-9993-4551f3b210d6 _log_request_id /usr/lib/python2.7/dist-packages/novaclient/client.py:85
neutron.notifiers.nova[22263]: DEBUG Nova returned NotFound for event: [{'name': 'network-changed', 'server_uuid': u'd02c7361-5e3a-4fdf-89b5-f29b3901f0fc'}] send_events /usr/lib/python2.7/dist-packages/neutron/notifiers/nova.py:263
oslo_messaging._drivers.amqpdriver[22265]: DEBUG received message msg_id: 0bf04ac8fedd4234bd6cd6c04547beca reply to reply_8b6e70ad5191401a9512147c4e94ca71 __call__ /usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py:194
neutron.db.provisioning_blocks[22265]: DEBUG Provisioning complete for port db3766ad-f82b-437d-b8b2-4133a92b1b86 [req-47c505d7-4eb5-4c71-9656-9e0927408822 - - - - -] provisioning_complete /usr/lib/python2.7/dist-packages/neutron/db/provisioning_blocks.py:153
Port info:
+---------------------+---------------------------------------------------------------------------------------+
| Field | Value |
+---------------------+---------------------------------------------------------------------------------------+
| admin_state_up | True |
| binding:host_id | d02c7361-5e3a-4fdf-89b5-f29b3901f0fc |
| binding:profile | {"local_link_information": [{"switch_info": "c426s1", "port_id": "1/1/21", |
| | "switch_id": "60:96:9f:69:b4:b4"}]} |
| binding:vif_details | {} |
| binding:vif_type | binding_failed |
| binding:vnic_type | baremetal |
| created_at | 2017-01-12T15:23:36Z |
| description | |
| device_id | d02c7361-5e3a-4fdf-89b5-f29b3901f0fc |
| device_owner | baremetal:none |
| extra_dhcp_opts | {"opt_value": "204.74.228.4", "ip_version": 4, "opt_name": "tftp-server"} |
| | {"opt_value": "204.74.228.4", "ip_version": 4, "opt_name": "server-ip-address"} |
| | {"opt_value": "pxelinux.0", "ip_version": 4, "opt_name": "bootfile-name"} |
| fixed_ips | {"subnet_id": "5402755a-0d8b-447d-9753-f3ba1ec39c22", "ip_address": "hidden"} |
| id | bc46cbdf-a82e-409d-9332-9eeb81aa0a94 |
| mac_address | 18:66:ee:aa:dd:cc |
| name | |
| network_id | 4b352ae7-141b-4c3f-a132-f5c006dc056c |
| project_id | 7d450ecf00d64399aeb93bc122cb6dae |
| revision_number | 8 |
| status | DOWN |
| tenant_id | 7d450ecf00d64399aeb93bc122cb6dae |
| updated_at | 2017-01-12T15:23:37Z |
+---------------------+---------------------------------------------------------------------------------
ironic node:
ironic node-list
/usr/lib/python2.7/dist-packages/urllib3/connection.py:303: SubjectAltNameWarning: Certificate for baremetal.ironic-dal-1.mgm.servers.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/shazow/urllib3/issues/497 for details.)
SubjectAltNameWarning
+--------------------------------------+-------+---------------+-------------+--------------------+-------------+
| UUID | Name | Instance UUID | Power State | Provisioning State | Maintenance |
+--------------------------------------+-------+---------------+-------------+--------------------+-------------+
| d02c7361-5e3a-4fdf-89b5-f29b3901f0fc | s8002 | None | power on | clean wait | True |
+--------------------------------------+-------+---------------+-------------+--------------------+-------------+
To manage notifications about this bug go to:
https://bugs.launchpad.net/ironic/+bug/1656010/+subscriptions
More information about the Ubuntu-openstack-bugs
mailing list