[Bug 1865754] Re: [SRU] tripleo-ci-centos-8-scenario001-standalone tempest-conf fails 500 PUT http://192.168.24.1:9292/v2/images/ RADOS invalid argument

Seth Tanner 1865754 at bugs.launchpad.net
Tue Oct 6 19:24:46 UTC 2020 

Corey,
My apologies for the previous comment please disregard, I forgot that I needed to run upgrade rather than install.

after running 
sudo apt-get upgrade os-brick-common/focal-proposed python3-os-brick/focal-proposed
Reading package lists... Done
Building dependency tree
Reading state information... Done
Selected version '3.0.1-0ubuntu1.3' (Ubuntu:20.04/focal-proposed [all]) for 'os-brick-common'
Selected version '3.0.1-0ubuntu1.3' (Ubuntu:20.04/focal-proposed [all]) for 'python3-os-brick'
Calculating upgrade... Done
The following packages will be upgraded:
  os-brick-common python3-os-brick
2 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Inst python3-os-brick [3.0.1-0ubuntu1.2] (3.0.1-0ubuntu1.3 Ubuntu:20.04/focal-proposed [all]) []
Inst os-brick-common [3.0.1-0ubuntu1.2] (3.0.1-0ubuntu1.3 Ubuntu:20.04/focal-proposed [all])
Conf python3-os-brick (3.0.1-0ubuntu1.3 Ubuntu:20.04/focal-proposed [all])
Conf os-brick-common (3.0.1-0ubuntu1.3 Ubuntu:20.04/focal-proposed [all])

and restarting cinder services
I am able to perform cinder backups and restore cinder backups, but it looks like there may be another issue with creating a snapshot and then creating a backup from the snapshot and then restoring that backup.

As a side note spawning a new instance from a snapshot, which creates a
new volume works just fine.

Here are the logs from a failed restore, when restoring a backup that
was created from a volume snapshot

2020-10-06 18:38:17.002 951 ERROR cinder.backup.manager [req-ff639558-c9ae-4dd3-923d-ec158b08f3dc 78af964f6db708aebd4103b6ee7fbdfe2d30f6e775a0a007ee04cd4633ccb87d aea00f20ec77483d963d2e8f54a1132a - e5bf6979cc0e42fa86ce5be4f12241f1 e5bf6979cc0e42fa86ce5be4f12241f1] Restoring backup 9574fcbf-fe7c-41f9-8d28-dc46e3643aae to volume a0fd2623-9ab0-402e-a9da-d7bcc4bd3247 failed.: rbd.ImageNotFound: [errno 2] RBD image not found (error opening image b'volume-a97b9aaa-bcce-4ac2-ab84-60a6f07f4be2.backup.base' at snapshot None)
2020-10-06 18:38:17.002 951 ERROR cinder.backup.manager Traceback (most recent call last):
2020-10-06 18:38:17.002 951 ERROR cinder.backup.manager   File "/usr/lib/python3/dist-packages/cinder/backup/manager.py", line 657, in _run_restore
2020-10-06 18:38:17.002 951 ERROR cinder.backup.manager     backup_service.restore(backup, volume.id,
2020-10-06 18:38:17.002 951 ERROR cinder.backup.manager   File "/usr/lib/python3/dist-packages/cinder/backup/drivers/ceph.py", line 1264, in restore
2020-10-06 18:38:17.002 951 ERROR cinder.backup.manager     self._restore_volume(backup, target_volume, volume_file)
2020-10-06 18:38:17.002 951 ERROR cinder.backup.manager   File "/usr/lib/python3/dist-packages/cinder/backup/drivers/ceph.py", line 1230, in _restore_volume
2020-10-06 18:38:17.002 951 ERROR cinder.backup.manager     self._full_restore(backup, volume_file, volume.name,
2020-10-06 18:38:17.002 951 ERROR cinder.backup.manager   File "/usr/lib/python3/dist-packages/cinder/backup/drivers/ceph.py", line 1023, in _full_restore
2020-10-06 18:38:17.002 951 ERROR cinder.backup.manager     src_rbd = eventlet.tpool.Proxy(self.rbd.Image(client.ioctx,
2020-10-06 18:38:17.002 951 ERROR cinder.backup.manager   File "rbd.pyx", line 3376, in rbd.Image.__init__
2020-10-06 18:38:17.002 951 ERROR cinder.backup.manager rbd.ImageNotFound: [errno 2] RBD image not found (error opening image b'volume-a97b9aaa-bcce-4ac2-ab84-60a6f07f4be2.backup.base' at snapshot None)
2020-10-06 18:38:17.002 951 ERROR cinder.backup.manager
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.rpc.server [req-ff639558-c9ae-4dd3-923d-ec158b08f3dc 78af964f6db708aebd4103b6ee7fbdfe2d30f6e775a0a007ee04cd4633ccb87d aea00f20ec77483d963d2e8f54a1132a - e5bf6979cc0e42fa86ce5be4f12241f1 e5bf6979cc0e42fa86ce5be4f12241f1] Exception during message handling: rbd.ImageNotFound: [errno 2] RBD image not found (error opening image b'volume-a97b9aaa-bcce-4ac2-ab84-60a6f07f4be2.backup.base' at snapshot None)
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3/dist-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3/dist-packages/oslo_messaging/rpc/dispatcher.py", line 276, in dispatch
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3/dist-packages/oslo_messaging/rpc/dispatcher.py", line 196, in _do_dispatch
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3/dist-packages/cinder/backup/manager.py", line 597, in restore_backup
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.rpc.server     backup.save()
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.rpc.server     self.force_reraise()
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3/dist-packages/six.py", line 703, in reraise
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.rpc.server     raise value
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3/dist-packages/cinder/backup/manager.py", line 586, in restore_backup
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.rpc.server     self._run_restore(context, backup, volume)
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3/dist-packages/cinder/backup/manager.py", line 657, in _run_restore
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.rpc.server     backup_service.restore(backup, volume.id,
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3/dist-packages/cinder/backup/drivers/ceph.py", line 1264, in restore
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.rpc.server     self._restore_volume(backup, target_volume, volume_file)
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3/dist-packages/cinder/backup/drivers/ceph.py", line 1230, in _restore_volume
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.rpc.server     self._full_restore(backup, volume_file, volume.name,
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3/dist-packages/cinder/backup/drivers/ceph.py", line 1023, in _full_restore
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.rpc.server     src_rbd = eventlet.tpool.Proxy(self.rbd.Image(client.ioctx,
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.rpc.server   File "rbd.pyx", line 3376, in rbd.Image.__init__
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.rpc.server rbd.ImageNotFound: [errno 2] RBD image not found (error opening image b'volume-a97b9aaa-bcce-4ac2-ab84-60a6f07f4be2.backup.base' at snapshot None)

** Tags removed: verification-ussuri-needed
** Tags added: verification-ussuri-done

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to Ubuntu Cloud Archive.
https://bugs.launchpad.net/bugs/1865754

Title:
  [SRU] tripleo-ci-centos-8-scenario001-standalone tempest-conf fails
  500 PUT http://192.168.24.1:9292/v2/images/ RADOS invalid argument

Status in charm-ceph-rbd-mirror:
  New
Status in Ubuntu Cloud Archive:
  Fix Released
Status in Ubuntu Cloud Archive ussuri series:
  Fix Committed
Status in Ubuntu Cloud Archive victoria series:
  Fix Released
Status in os-brick:
  Fix Released
Status in tripleo:
  Fix Released
Status in python-os-brick package in Ubuntu:
  Fix Released
Status in python-os-brick source package in Focal:
  Fix Committed
Status in python-os-brick source package in Groovy:
  Fix Released

Bug description:
  [Impact]

  Os-brick is unusable with Ceph Octopus because of a tightening of
  config formatting requirements in Ceph.

  [Test Case]

  Validate cinder-backup with Ceph on Ubuntu Focal, in addition to
  testing with Ceph on Xenial (OpenStack Mitaka) with Cinder (and
  cinder-backup) on Focal (OpenStack Ussuri)

  [Regression Potential]

  The risk should be fairly low as the new confiig line being generated
  is supported since Ceph Hammer, which shipped in 2015. I would expect
  any potential regressions to occur with the combination of a current
  os-brick with an older (Hammer timeframe) Ceph, as that is when this
  feature was introduced.

  the only potential regression I see in the previously mentioned
  timeframe would be with connecting to the older Ceph.

  -------------------------------------------------------------------------

  
  https://12b5c9445590716cbe81-2eb50734132c0e56282483bcdf57bf8a.ssl.cf2.rackcdn.com/710810/3/check/tripleo-ci-centos-8-scenario001-standalone/8a6d4a7/job-output.txt

  2020-03-02 18:07:56.142233 | primary | 2020-03-02 18:07:52.442 135023 WARNING urllib3.connectionpool [-] Retrying (Retry(total=8, connect=None, read=None, redirect=5, status=None)) after connection broken by 'ProtocolError('Connection aborted.', BrokenPipeError(32, 'Broken pipe'))': /v2/images/22582e2f-9ce2-43a6-90e2-fee0ddd283e3/file
  2020-03-02 18:07:56.142265 | primary | 2020-03-02 18:07:55.915 135023 INFO tempest.lib.common.rest_client [req-2c4e9f5f-5497-4eae-b26c-9203f8fbc85b ] Request (main): 500 PUT http://192.168.24.1:9292/v2/images/22582e2f-9ce2-43a6-90e2-fee0ddd283e3/file 14.582s
  2020-03-02 18:07:56.142297 | primary | 2020-03-02 18:07:55.917 135023 CRITICAL tempest [-] Unhandled error: tempest.lib.exceptions.ServerFault: Got server fault
  2020-03-02 18:07:56.142328 | primary | Details: b'500 Internal Server Error\n\nThe server has either erred or is incapable of performing the requested operation.\n\n   '
  2020-03-02 18:07:56.142377 | primary | 2020-03-02 18:07:55.917 135023 ERROR tempest Traceback (most recent call last):
  2020-03-02 18:07:56.142411 | primary | 2020-03-02 18:07:55.917 135023 ERROR tempest   File "/bin/discover-tempest-config", line 10, in <module>
  2020-03-02 18:07:56.142442 | primary | 2020-03-02 18:07:55.917 135023 ERROR tempest     sys.exit(main())
  2020-03-02 18:07:56.142473 | primary | 2020-03-02 18:07:55.917 135023 ERROR tempest   File "/usr/lib/python3.6/site-packages/config_tempest/main.py", line 605, in main
  2020-03-02 18:07:56.142505 | primary | 2020-03-02 18:07:55.917 135023 ERROR tempest     verbose=args.verbose
  2020-03-02 18:07:56.142549 | primary | 2020-03-02 18:07:55.917 135023 ERROR tempest   File "/usr/lib/python3.6/site-packages/config_tempest/main.py", line 545, in config_tempest
  2020-03-02 18:07:56.142582 | primary | 2020-03-02 18:07:55.917 135023 ERROR tempest     image.create_tempest_images(conf)
  2020-03-02 18:07:56.142612 | primary | 2020-03-02 18:07:55.917 135023 ERROR tempest   File "/usr/lib/python3.6/site-packages/config_tempest/services/image.py", line 113, in create_tempest_images
  2020-03-02 18:07:56.142644 | primary | 2020-03-02 18:07:55.917 135023 ERROR tempest     image_dest=img_path)
  2020-03-02 18:07:56.142675 | primary | 2020-03-02 18:07:55.917 135023 ERROR tempest   File "/usr/lib/python3.6/site-packages/config_tempest/services/image.py", line 161, in find_or_upload_image
  2020-03-02 18:07:56.142706 | primary | 2020-03-02 18:07:55.917 135023 ERROR tempest     image = self._upload_image(image_name, image_dest)
  2020-03-02 18:07:56.142737 | primary | 2020-03-02 18:07:55.917 135023 ERROR tempest   File "/usr/lib/python3.6/site-packages/config_tempest/services/image.py", line 212, in _upload_image
  2020-03-02 18:07:56.142769 | primary | 2020-03-02 18:07:55.917 135023 ERROR tempest     self.client.store_image_file(image['id'], data)
  2020-03-02 18:07:56.142801 | primary | 2020-03-02 18:07:55.917 135023 ERROR tempest   File "/usr/lib/python3.6/site-packages/tempest/lib/services/image/v2/images_client.py", line 151, in store_image_file
  2020-03-02 18:07:56.142832 | primary | 2020-03-02 18:07:55.917 135023 ERROR tempest     body=data, chunked=True)
  2020-03-02 18:07:56.142863 | primary | 2020-03-02 18:07:55.917 135023 ERROR tempest   File "/usr/lib/python3.6/site-packages/tempest/lib/common/rest_client.py", line 687, in request
  2020-03-02 18:07:56.142894 | primary | 2020-03-02 18:07:55.917 135023 ERROR tempest     self._error_checker(resp, resp_body)
  2020-03-02 18:07:56.142926 | primary | 2020-03-02 18:07:55.917 135023 ERROR tempest   File "/usr/lib/python3.6/site-packages/tempest/lib/common/rest_client.py", line 864, in _error_checker
  2020-03-02 18:07:56.142965 | primary | 2020-03-02 18:07:55.917 135023 ERROR tempest     message=message)
  2020-03-02 18:07:56.142996 | primary | 2020-03-02 18:07:55.917 135023 ERROR tempest tempest.lib.exceptions.ServerFault: Got server fault
  2020-03-02 18:07:56.143049 | primary | 2020-03-02 18:07:55.917 135023 ERROR tempest Details: b'500 Internal Server Error\n\nThe server has either erred or is incapable of performing the requested operation.\n\n   '
  2020-03-02 18:07:56.143083 | primary | 2020-03-02 18:07:55.917 135023 ERROR tempest 
  2020-03-02 18:07:56.143113 | primary |
  2020-03-02 18:07:56.143144 | primary |
  2020-03-02 18:07:56.143192 | primary | STDERR:
  2020-03-02 18:07:56.143225 | primary |
  2020-03-02 18:07:56.143256 | primary | /usr/lib/python3.6/site-packages/config_tempest/profile.py:45: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  2020-03-02 18:07:56.143286 | primary |   return yaml.load(stream)

  =====================================

  Actual error: is here:

  https://12b5c9445590716cbe81-2eb50734132c0e56282483bcdf57bf8a.ssl.cf2.rackcdn.com/710810/3/check
  /tripleo-ci-
  centos-8-scenario001-standalone/8a6d4a7/logs/undercloud/var/log/containers/glance/api.log

  InvalidArgumentError('RADOS invalid argument (error calling conf_read_file)',) trace_logging_wrapper /usr/lib/python3.6/site-packages/os_brick/utils.py:156
  2020-03-02 18:07:51.999 40 ERROR glance_store._drivers.cinder [req-5076b6b5-c8d6-4743-83dc-245492610376 b9f74f3fb12843818bc784bf2f81fca2 966194a384e641598d107cdf193a9a39 - default default] Exception while accessing to cinder volume 17a385a3-9331-4513-9a81-37a99200cefd.: rados.InvalidArgumentError: [errno 22] RADOS invalid argument (error calling conf_read_file)
  2020-03-02 18:07:51.999 40 ERROR glance_store._drivers.cinder Traceback (most recent call last):
  2020-03-02 18:07:51.999 40 ERROR glance_store._drivers.cinder   File "/usr/lib/python3.6/site-packages/glance_store/_drivers/cinder.py", line 554, in _open_cinder_volume
  2020-03-02 18:07:51.999 40 ERROR glance_store._drivers.cinder     device = conn.connect_volume(connection_info['data'])
  2020-03-02 18:07:51.999 40 ERROR glance_store._drivers.cinder   File "/usr/lib/python3.6/site-packages/os_brick/utils.py", line 150, in trace_logging_wrapper
  2020-03-02 18:07:51.999 40 ERROR glance_store._drivers.cinder     result = f(*args, **kwargs)
  2020-03-02 18:07:51.999 40 ERROR glance_store._drivers.cinder   File "/usr/lib/python3.6/site-packages/os_brick/initiator/connectors/rbd.py", line 203, in connect_volume
  2020-03-02 18:07:51.999 40 ERROR glance_store._drivers.cinder     rbd_handle = self._get_rbd_handle(connection_properties)
  2020-03-02 18:07:51.999 40 ERROR glance_store._drivers.cinder   File "/usr/lib/python3.6/site-packages/os_brick/initiator/connectors/rbd.py", line 122, in _get_rbd_handle
  2020-03-02 18:07:51.999 40 ERROR glance_store._drivers.cinder     rbd_cluster_name=str(cluster_name))
  2020-03-02 18:07:51.999 40 ERROR glance_store._drivers.cinder   File "/usr/lib/python3.6/site-packages/os_brick/initiator/linuxrbd.py", line 60, in __init__
  2020-03-02 18:07:51.999 40 ERROR glance_store._drivers.cinder     self.client, self.ioctx = self.connect()
  2020-03-02 18:07:51.999 40 ERROR glance_store._drivers.cinder   File "/usr/lib/python3.6/site-packages/os_brick/initiator/linuxrbd.py", line 73, in connect
  2020-03-02 18:07:51.999 40 ERROR glance_store._drivers.cinder     conffile=self.rbd_conf)
  2020-03-02 18:07:51.999 40 ERROR glance_store._drivers.cinder   File "rados.pyx", line 692, in rados.Rados.__init__
  2020-03-02 18:07:51.999 40 ERROR glance_store._drivers.cinder   File "rados.pyx", line 578, in rados.requires.wrapper.validate_func
  2020-03-02 18:07:51.999 40 ERROR glance_store._drivers.cinder   File "rados.pyx", line 743, in rados.Rados.__setup
  2020-03-02 18:07:51.999 40 ERROR glance_store._drivers.cinder   File "rados.pyx", line 578, in rados.requires.wrapper.validate_func
  2020-03-02 18:07:51.999 40 ERROR glance_store._drivers.cinder   File "rados.pyx", line 826, in rados.Rados.conf_read_file
  2020-03-02 18:07:51.999 40 ERROR glance_store._drivers.cinder rados.InvalidArgumentError: [errno 22] RADOS invalid argument (error calling conf_read_file)
  2020-03-02 18:07:51.999 40 ERROR glance_store._drivers.cinder
  2020-03-02 18:07:52.296 40 ERROR glance_store._drivers.cinder [req-5076b6b5-c8d6-4743-83dc-245492610376 b9f74f3fb12843818bc784bf2f81fca2 966194a384e641598d107cdf193a9a39 - default default] Failed to write to volume 17a385a3-9331-4513-9a81-37a99200cefd.: rados.InvalidArgumentError: [errno 22] RADOS invalid argument (error calling conf_read_file)
  2020-03-02 18:07:52.386 40 ERROR glance.api.v2.image_data [req-5076b6b5-c8d6-4743-83dc-245492610376 b9f74f3fb12843818bc784bf2f81fca2 966194a384e641598d107cdf193a9a39 - default default] Failed to upload image data due to internal error: rados.InvalidArgumentError: [errno 22] RADOS invalid argument (error calling conf_read_file)
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi [req-5076b6b5-c8d6-4743-83dc-245492610376 b9f74f3fb12843818bc784bf2f81fca2 966194a384e641598d107cdf193a9a39 - default default] Caught error: [errno 22] RADOS invalid argument (error calling conf_read_file): rados.InvalidArgumentError: [errno 22] RADOS invalid argument (error calling conf_read_file)
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi Traceback (most recent call last):
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi   File "/usr/lib/python3.6/site-packages/glance/common/wsgi.py", line 1347, in __call__
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi     request, **action_args)
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi   File "/usr/lib/python3.6/site-packages/glance/common/wsgi.py", line 1390, in dispatch
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi     return method(*args, **kwargs)
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi   File "/usr/lib/python3.6/site-packages/glance/common/utils.py", line 415, in wrapped
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi     return func(self, req, *args, **kwargs)
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi   File "/usr/lib/python3.6/site-packages/glance/api/v2/image-------------------------------------------------------------------------_data.py", line 299, in upload
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi     self._restore(image_repo, image)
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi     self.force_reraise()
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi     six.reraise(self.type_, self.value, self.tb)
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi   File "/usr/lib/python3.6/site-packages/six.py", line 693, in reraise
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi     raise value
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi   File "/usr/lib/python3.6/site-packages/glance/api/v2/image_data.py", line 164, in upload
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi     image.set_data(data, size, backend=backend)
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi   File "/usr/lib/python3.6/site-packages/glance/domain/proxy.py", line 198, in set_data
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi     self.base.set_data(data, size, backend=backend, set_active=set_active)
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi   File "/usr/lib/python3.6/site-packages/glance/notifier.py", line 501, in set_data
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi     _send_notification(notify_error, 'image.upload', msg)
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi     self.force_reraise()
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi     six.reraise(self.type_, self.value, self.tb)
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi   File "/usr/lib/python3.6/site-packages/six.py", line 693, in reraise
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi     raise value
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi   File "/usr/lib/python3.6/site-packages/glance/notifier.py", line 448, in set_data
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi     set_active=set_active)
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi   File "/usr/lib/python3.6/site-packages/glance/api/policy.py", line 204, in set_data
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi     return self.image.set_data(*args, **kwargs)
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi   File "/usr/lib/python3.6/site-packages/glance/quota/__init__.py", line 319, in set_data
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi     set_active=set_active)
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi   File "/usr/lib/python3.6/site-packages/glance/location.py", line 548, in set_data
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi     self._upload_to_store(data, verifier, backend, size)
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi   File "/usr/lib/python3.6/site-packages/glance/location.py", line 461, in _upload_to_store
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi     verifier=verifier)
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi   File "/usr/lib/python3.6/site-packages/glance_store/multi_backend.py", line 396, in add_with_multihash
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi     image_id, data, size, hashing_algo, store, context, verifier)
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi   File "/usr/lib/python3.6/site-packages/glance_store/multi_backend.py", line 478, in store_add_to_backend_with_multihash
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi     image_id, data, size, hashing_algo, context=context, verifier=verifier)
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi   File "/usr/lib/python3.6/site-packages/glance_store/driver.py", line 279, in add_adapter
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi     metadata_dict) = store_add_fun(*args, **kwargs)
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi   File "/usr/lib/python3.6/site-packages/glance_store/capabilities.py", line 176, in op_checker
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi     return store_op_fun(store, *args, **kwargs)
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi   File "/usr/lib/python3.6/site-packages/glance_store/_drivers/cinder.py", line 747, in add
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi     with self._open_cinder_volume(client, volume, 'wb') as f:
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi   File "/usr/lib64/python3.6/contextlib.py", line 81, in __enter__
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi     return next(self.gen)
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi   File "/usr/lib/python3.6/site-packages/glance_store/_drivers/cinder.py", line 554, in _open_cinder_volume
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi     device = conn.connect_volume(connection_info['data'])
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi   File "/usr/lib/python3.6/site-packages/os_brick/utils.py", line 150, in trace_logging_wrapper
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi     result = f(*args, **kwargs)
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi   File "/usr/lib/python3.6/site-packages/os_brick/initiator/connectors/rbd.py", line 203, in connect_volume
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi     rbd_handle = self._get_rbd_handle(connection_properties)
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi   File "/usr/lib/python3.6/site-packages/os_brick/initiator/connectors/rbd.py", line 122, in _get_rbd_handle
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi     rbd_cluster_name=str(cluster_name))
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi   File "/usr/lib/python3.6/site-packages/os_brick/initiator/linuxrbd.py", line 60, in __init__
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi     self.client, self.ioctx = self.connect()
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi   File "/usr/lib/python3.6/site-packages/os_brick/initiator/linuxrbd.py", line 73, in connect
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi     conffile=self.rbd_conf)
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi   File "rados.pyx", line 692, in rados.Rados.__init__
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi   File "rados.pyx", line 578, in rados.requires.wrapper.validate_func
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi   File "rados.pyx", line 743, in rados.Rados.__setup
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi   File "rados.pyx", line 578, in rados.requires.wrapper.validate_func
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi   File "rados.pyx", line 826, in rados.Rados.conf_read_file
  2020-03-02 18:07:52.438 40 ERROR glance.common.wsgi rados.InvalidArgumentError: [errno 22] RADOS invalid argument (error calling conf_read_file)

To manage notifications about this bug go to:
https://bugs.launchpad.net/charm-ceph-rbd-mirror/+bug/1865754/+subscriptions

More information about the Ubuntu-openstack-bugs mailing list