[Bug 1834565] [python-eventlet/disco] verification still needed
Brian Murray
brian at ubuntu.com
Thu Oct 10 19:29:13 UTC 2019
The fix for this bug has been awaiting testing feedback in the -proposed
repository for disco for more than 90 days. Please test this fix and
update the bug appropriately with the results. In the event that the
fix for this bug is still not verified 15 days from now, the package
will be removed from the -proposed repository.
** Tags added: removal-candidate
--
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to python-eventlet in Ubuntu.
https://bugs.launchpad.net/bugs/1834565
Title:
Services not running that should be: cinder-scheduler, neutron-
lbaasv2-agent, neutron-dhcp-agent, neutron-metering-agent
Status in OpenStack cinder charm:
Invalid
Status in OpenStack neutron-gateway charm:
Invalid
Status in Ubuntu Cloud Archive:
Fix Committed
Status in Ubuntu Cloud Archive stein series:
Fix Committed
Status in Ubuntu Cloud Archive train series:
Fix Released
Status in python-eventlet package in Ubuntu:
Fix Released
Status in python-eventlet source package in Disco:
Fix Committed
Status in python-eventlet source package in Eoan:
Fix Released
Bug description:
[Impact]
Unable to enable TLS termination of any endpoints for OpenStack with Python 3.7.
[Test Case]
sudo apt install python3-eventlet
python3 << EOF
import eventlet
eventlet.monkey_patch()
import socket
import ssl
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
wrappedSocket = ssl.wrap_socket(sock)
EOF
Traceback (most recent call last):
File "<stdin>", line 8, in <module>
File "/usr/lib/python3/dist-packages/eventlet/green/ssl.py", line 403, in wrap_socket
return GreenSSLSocket(sock, *a, **kw)
File "/usr/lib/python3/dist-packages/eventlet/green/ssl.py", line 80, in __init__
*args, **kw
File "/usr/lib/python3.7/ssl.py", line 1222, in wrap_socket
suppress_ragged_eofs=suppress_ragged_eofs
File "/usr/lib/python3/dist-packages/eventlet/green/ssl.py", line 426, in wrap_socket
return GreenSSLSocket(sock, *a, _context=self, **kw)
File "/usr/lib/python3/dist-packages/eventlet/green/ssl.py", line 80, in __init__
*args, **kw
TypeError: wrap_socket() got an unexpected keyword argument '_context'
[Regression Potential]
Low; codepaths are all Python 3.7 specific due to internal changes in the SSL implementation.
[Original bug report]
In an SSL enabled OpenStack Stein deploy, deployed on Disco, the cinder-scheduler service is crashing with the below stacktrace:
2019-06-27 18:35:07.593 5427 WARNING cinder.scheduler.host_manager [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] volume service is down. (host: juju-20eff5-mojo-0-7 at LVM)
2019-06-27 18:35:07.594 5427 WARNING cinder.scheduler.host_manager [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] volume service is down. (host: cinder at cinder-ceph)
2019-06-27 18:35:07.610 5427 INFO cinder.rpc [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] Automatically selected cinder-volume objects version 1.37 as minimum service version.
2019-06-27 18:35:07.619 5427 INFO cinder.rpc [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] Automatically selected cinder-volume RPC version 3.16 as minimum service version.
2019-06-27 18:35:07.638 5427 INFO cinder.rpc [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] Automatically selected cinder-scheduler objects version 1.37 as minimum service version.
2019-06-27 18:35:07.647 5427 INFO cinder.rpc [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] Automatically selected cinder-scheduler RPC version 3.11 as minimum service version.
2019-06-27 18:35:07.702 5427 WARNING oslo_config.cfg [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] Deprecated: Option "auth_uri" from group "keystone_authtoken" is deprecated for removal (The auth_uri option is deprecated in favor of www_authenticate_uri and will be removed in the S release.). Its value may be silently ignored in the future.
2019-06-27 18:35:07.703 5427 WARNING oslo_config.cfg [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] Deprecated: Option "signing_dir" from group "keystone_authtoken" is deprecated for removal (PKI token format is no longer supported.). Its value may be silently ignored in the future.
2019-06-27 18:35:07.705 5427 WARNING oslo_config.cfg [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] Deprecated: Option "auth_uri" from group "keystone_authtoken" is deprecated. Use option "www_authenticate_uri" from group "keystone_authtoken".
2019-06-27 18:35:07.726 5427 INFO cinder.service [-] Starting cinder-scheduler node (version 14.0.0)
2019-06-27 18:35:07.736 5427 INFO cinder.manager [req-e208c3e5-83e5-4c5b-85a3-1f5e09c86da9 - - - - -] Initiating service 4 cleanup
2019-06-27 18:35:07.745 5427 INFO cinder.manager [req-e208c3e5-83e5-4c5b-85a3-1f5e09c86da9 - - - - -] Service 4 cleanup completed.
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service [req-e208c3e5-83e5-4c5b-85a3-1f5e09c86da9 - - - - -] Error starting thread.: TypeError: wrap_socket() got an unexpected keyword argument '_context'
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service Traceback (most recent call last):
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_service/service.py", line 796, in run_service
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service service.start()
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/cinder/service.py", line 236, in start
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self.rpcserver.start()
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/server.py", line 270, in wrapper
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service log_after, timeout_timer)
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/server.py", line 190, in run_once
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service post_fn = fn()
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/server.py", line 269, in <lambda>
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service states[state].run_once(lambda: fn(self, *args, **kwargs),
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/server.py", line 414, in start
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self.listener = self._create_listener()
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/rpc/server.py", line 151, in _create_listener
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service return self.transport._listen(self._target, 1, None)
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/transport.py", line 143, in _listen
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service batch_timeout)
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 652, in listen
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service conn = self._get_connection(rpc_common.PURPOSE_LISTEN)
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 559, in _get_connection
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service purpose=purpose)
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/common.py", line 407, in __init__
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self.connection = connection_pool.create(purpose)
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/pool.py", line 144, in create
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service return self.connection_cls(self.conf, self.url, purpose)
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/impl_rabbit.py", line 531, in __init__
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self.ensure_connection()
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/impl_rabbit.py", line 646, in ensure_connection
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self.connection.ensure_connection(errback=on_error)
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/kombu/connection.py", line 405, in ensure_connection
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service callback)
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/kombu/utils/functional.py", line 332, in retry_over_time
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service return fun(*args, **kwargs)
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/kombu/connection.py", line 261, in connect
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service return self.connection
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/kombu/connection.py", line 802, in connection
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self._connection = self._establish_connection()
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/kombu/connection.py", line 757, in _establish_connection
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service conn = self.transport.establish_connection()
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/kombu/transport/pyamqp.py", line 130, in establish_connection
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service conn.connect()
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/amqp/connection.py", line 307, in connect
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self.transport.connect()
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/amqp/transport.py", line 83, in connect
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self.socket_settings, self.read_timeout, self.write_timeout,
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/amqp/transport.py", line 187, in _init_socket
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self._setup_transport()
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/amqp/transport.py", line 302, in _setup_transport
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self.sock = self._wrap_socket(self.sock, **self.sslopts)
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/amqp/transport.py", line 309, in _wrap_socket
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service return self._wrap_socket_sni(sock, **sslopts)
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/amqp/transport.py", line 344, in _wrap_socket_sni
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service sock = ssl.wrap_socket(**opts)
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/eventlet/green/ssl.py", line 403, in wrap_socket
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service return GreenSSLSocket(sock, *a, **kw)
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/eventlet/green/ssl.py", line 80, in __init__
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service *args, **kw
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3.7/ssl.py", line 1222, in wrap_socket
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service suppress_ragged_eofs=suppress_ragged_eofs
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/eventlet/green/ssl.py", line 426, in wrap_socket
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service return GreenSSLSocket(sock, *a, _context=self, **kw)
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/eventlet/green/ssl.py", line 80, in __init__
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service *args, **kw
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service TypeError: wrap_socket() got an unexpected keyword argument '_context'
2019-06-27 18:35:07.776 5427 ERROR oslo_service.service
Additionally, charm-neutron-gateway has the same stacktrace in the
mentioned services.
To manage notifications about this bug go to:
https://bugs.launchpad.net/charm-cinder/+bug/1834565/+subscriptions
More information about the Ubuntu-openstack-bugs
mailing list