[Bug 1945500] Re: [SRU] It's not possible to upload a volume that was build from an image back to glance, if multistore (glance) is enabled.
James Page
1945500 at bugs.launchpad.net
Mon Jul 8 13:03:52 UTC 2024
This bug was fixed in the package cinder - 2:20.3.1-0ubuntu1.4~cloud0
---------------
cinder (2:20.3.1-0ubuntu1.4~cloud0) focal; urgency=medium
.
* SECURITY UPDATE for Ubuntu Cloud Archive. backport to focal.
.
cinder (2:20.3.1-0ubuntu1.4) jammy-security; urgency=medium
.
* SECURITY UPDATE: Arbitrary file access via custom QCOW2 external data
(LP: #2059809)
- debian/patches/CVE-2024-32498.patch: check for external qcow2 data
file.
- debian/control: added qemu-utils to Build-Depends so qemu-img is
available for new tests.
- CVE-2024-32498
.
cinder (2:20.3.1-0ubuntu1.2) jammy; urgency=medium
.
[ Jorge Merlino ]
* Increase size of volume image metadata values to 65535 bytes
(LP: #1988942)
.
[ Heather Lemon ]
* Start cinder-volume.service after tgt.service started (LP: #1987663)
- d/cinder-volume.service.conf: drop-in with 'After=' and 'Wants='
('Wants=' is not generated by pkgos-gen-systemd-unit currently).
- d/cinder-volume.install: ship the systemd service drop-in file.
.
[ Seyeong Kim ]
* HPE3PAR: Failing to clone a volume having children (LP: #1994521):
- d/p/0001-HPE-3PAR-Fix-umanaged-volumes-snapshots-missing.patch
- d/p/0002-3PAR-Error-out-if-vol-cannot-be-converted-to-base.patch
- api 4.0.17 is added as it is in the middle of the main patch
(4.0.18)
.
cinder (2:20.3.1-0ubuntu1.1) jammy; urgency=medium
.
* Revert driver assisted volume retype (LP: #2019190):
- d/p/0001-Revert-Driver-assisted-migration-on-retype-when-it-s.patch
.
cinder (2:20.3.1-0ubuntu1) jammy; urgency=medium
.
* New stable point release for OpenStack Yoga (LP: #2037332).
.
cinder (2:20.3.0-0ubuntu1) jammy; urgency=medium
.
* New stable point release for OpenStack Yoga (LP: #2025503).
* d/p/CVE-2023-2088.patch: Dropped. Fixed in point release.
.
cinder (2:20.2.0-0ubuntu1.1) jammy-security; urgency=medium
.
* SECURITY UPDATE: Unauthorized File Access (LP: #2021980)
- debian/patches/CVE-2023-2088.patch: Reject unsafe delete
attachment calls.
- CVE-2023-2088
.
cinder (2:20.2.0-0ubuntu1) jammy; urgency=medium
.
* New stable point release for OpenStack Yoga (LP: #2019759).
* d/p/lp1945500.patch: Dropped. Fixed in stable point release.
.
cinder (2:20.1.0-0ubuntu2.2) jammy-security; urgency=medium
.
* SECURITY REGRESSION: Regressions in other projects (LP: #2020111)
- debian/patches/series: Do not apply CVE-2023-2088.patch until
patches are ready for all upstream OpenStack projects.
- CVE-2023-2088
.
cinder (2:20.1.0-0ubuntu2.1) jammy-security; urgency=medium
.
* SECURITY UPDATE: Unauthorized File Access
- debian/patches/CVE-2023-2088.patch: Reject unsafe delete
attachment calls.
- CVE-2023-2088
.
cinder (2:20.1.0-0ubuntu2) jammy; urgency=medium
.
* d/p/lp1945500.patch: Filter reserved image properties (LP: #1945500).
.
cinder (2:20.1.0-0ubuntu1) jammy; urgency=medium
.
* New stable point release for OpenStack Yoga (LP: #2004030).
.
cinder (2:20.0.1-0ubuntu1) jammy; urgency=medium
.
* d/gbp.conf: Create stable/yoga branch.
* New stable point release for OpenStack Yoga (LP: #1985084).
.
cinder (2:20.0.0-0ubuntu1) jammy; urgency=medium
.
* d/watch: Scope to 20.x.
* New upstream release for OpenStack Yoga.
* d/control: Align (Build-)Depends with upstream.
.
cinder (2:19.0.0+git2022030310.b49fb59a6-0ubuntu2) jammy; urgency=medium
.
* d/p/fix-qos-computation.patch: Cherry-pick from upstream review to
fix TypeError exception when generating QOS feature name (LP: #1948507).
.
cinder (2:19.0.0+git2022030310.b49fb59a6-0ubuntu1) jammy; urgency=medium
.
* New upstream snapshot for OpenStack Yoga.
.
cinder (2:19.0.0+git2022011215.23494a6d6-0ubuntu1) jammy; urgency=medium
.
* New upstream snapshot for OpenStack Yoga.
* d/control, d/rules: Bump debhelper compat to 13.
.
cinder (2:19.0.0+git2021120811.e5ef39604-0ubuntu2) jammy; urgency=medium
.
* d/t/control: Add allow-stderr restriction to prevent autopkgtest failure
when SQLAlchemy issues a warning.
.
cinder (2:19.0.0+git2021120811.e5ef39604-0ubuntu1) jammy; urgency=medium
.
* New upstream snapshot for OpenStack Yoga.
* d/control: Align (Build-)Depends with upstream.
.
cinder (2:19.0.0-0ubuntu2) impish; urgency=medium
.
* d/py3dist-overrides: Add SQLAlchemy to ensure d/control is not overridden.
* d/control: Align (Build-)Depends with upstream.
.
cinder (2:19.0.0-0ubuntu1) impish; urgency=medium
.
* d/watch: Scope to 19.x.
* New upstream release for OpenStack Xena.
.
cinder (2:19.0.0~b1+git2021091409.768b8996b-0ubuntu1) impish; urgency=medium
.
* New upstream snapshot for OpenStack Xena.
.
cinder (2:18.0.0+git2021072116.81f2aaeea-0ubuntu1) impish; urgency=medium
.
* New upstream snapshot for OpenStack Xena.
* d/control: Align (Build-)Depends with upstream.
.
cinder (2:18.0.0+git2021061414.d5f0e5187-0ubuntu1) impish; urgency=medium
.
* New upstream snapshot for OpenStack Xena.
* d/control: Align (Build-)Depends with upstream.
.
cinder (2:18.0.0-0ubuntu3) hirsute; urgency=medium
.
* d/p/skip-victoria-failures.patch: Restored and rebased. This is still
necessary for Launchpad builds.
.
cinder (2:18.0.0-0ubuntu2) hirsute; urgency=medium
.
* d/p/skip-victoria-failures.patch: Dropped. Fixed upstream.
* d/p/add-mock-psutil-in-quobyte-tests.patch: Dropped. Fixed upstream.
.
cinder (2:18.0.0-0ubuntu1) hirsute; urgency=medium
.
* New upstream release for OpenStack Wallaby.
.
cinder (2:18.0.0~b1-0ubuntu2) hirsute; urgency=medium
.
* d/py3dist-overrides: Add boto3 which is a Suggests.
.
cinder (2:18.0.0~b1-0ubuntu1) hirsute; urgency=medium
.
* d/watch: Track 18.x series.
* New upstream milestone for OpenStack Wallaby.
* d/control: Align (Build-)Depends with upstream.
* d/p/skip-moto-tests.patch: Skip test dependency that is not yet
packaged in Ubuntu and was added late in cycle.
* d/p/patch-botocore-exceptions.patch: Account for changes to botocore
vendored exceptions.
.
cinder (2:17.0.1+git2021012507.d26092348-0ubuntu3) hirsute; urgency=medium
.
* d/*: Remove tgt in favor of targetcli-fb.
.
cinder (2:17.0.1+git2021012507.d26092348-0ubuntu2) hirsute; urgency=medium
.
* d/p/add-mock-psutil-in-quobyte-tests.patch: Add a mock of psutil
disk_partitions to fix failing unit test (LP: #1913607).
.
cinder (2:17.0.1+git2021012507.d26092348-0ubuntu1) hirsute; urgency=medium
.
* New upstream snapshot for OpenStack Wallaby.
.
cinder (2:17.0.1+git2021010614.a9c922ab7-0ubuntu1) hirsute; urgency=medium
.
* New upstream snapshot for OpenStack Wallaby.
* d/control: Align (Build-)Depends with upstream.
.
cinder (2:17.0.1+git2020120911.d3ffa90ba-0ubuntu1) hirsute; urgency=medium
.
* New upstream snapshot for OpenStack Wallaby.
* d/control: Align (Build-)Depends with upstream.
.
cinder (2:17.0.0-0ubuntu1) groovy; urgency=medium
.
* New upstream release for OpenStack Victoria.
.
cinder (2:17.0.0~rc2-0ubuntu1) groovy; urgency=medium
.
* d/control: Update VCS paths for move to lp:~ubuntu-openstack-dev.
* d/watch: Track 17.x series.
* New upstream release candidate for OpenStack Victoria.
* d/control: Align (Build-)Depends with upstream.
.
cinder (2:17.0.0~b3~git2020091007.afcaf0b9d-0ubuntu3) groovy; urgency=medium
.
* d/py3dist-overrides: Add python3-zstd to py3dist-overrides.
.
cinder (2:17.0.0~b3~git2020091007.afcaf0b9d-0ubuntu2) groovy; urgency=medium
.
* d/p/skip-victoria-failures.patch: Restored to skip failing unit tests.
.
cinder (2:17.0.0~b3~git2020091007.afcaf0b9d-0ubuntu1) groovy; urgency=medium
.
* d/control: Remove Breaks/Replaces that are older than Focal (LP: #1878419).
* New upstream snapshot for OpenStack Victoria.
* d/control: Align (Build-)Depends with upstream.
* d/p/*: Removed. Changes landed upstream and tests fixed.
* d/control: Add new python3-zstd package to depends.
.
cinder (2:17.0.0~b2~git2020073012.2124f39f9-0ubuntu1) groovy; urgency=medium
.
* New upstream snapshot for OpenStack Victoria.
* d/p/*: Refreshed.
.
cinder (2:17.0.0~b1~git2020062409.85fcf1057-0ubuntu1) groovy; urgency=medium
.
* SECURITY UPDATE: Dell EMC ScaleIO/VxFlex OS Backend Credentials Exposure
(LP: #1823200)
- Remove VxFlex OS credentials from connection_properties. Passwords are
now stored in separate file and are retrieved during each attach/detach
operation. Cinder is patched in 16.1.0 stable point release.
- d/control: Align (Build-)Depends with min version of python3-os-brick
required to fix credential exposure.
- CVE-2020-10755
* New upstream snapshot for OpenStack Victoria.
* d/control: Align (Build-)Depends with upstream.
* d/p/py38skip.patch: Dropped. No longer needed.
* d/p/skip-victoria-failures.patch: Rebased and updated with upstream bug.
--
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to cinder in Ubuntu.
https://bugs.launchpad.net/bugs/1945500
Title:
[SRU] It's not possible to upload a volume that was build from an
image back to glance, if multistore (glance) is enabled.
Status in Cinder:
Fix Released
Status in Ubuntu Cloud Archive:
Fix Released
Status in Ubuntu Cloud Archive yoga series:
Fix Released
Status in Ubuntu Cloud Archive zed series:
Fix Released
Status in cinder package in Ubuntu:
Fix Released
Status in cinder source package in Jammy:
Fix Released
Status in cinder source package in Kinetic:
Fix Released
Bug description:
* SRU TEMPLATE AT THE BOTTOM *
Brief description:
Cinder (including Wallaby release) is not able to upload a volume that was build from an image back to glance, if multistore (glance) is enabled.
Details:
After enabling glance multistore there will be two extra properties with every image. Those are `os_glance_failed_import`and `os_glance_importing_to_stores`.
If cinder creates a volume from an imageRef it will store all image_metadata with that volume, including `os_glance_failed_import`and `os_glance_importing_to_stores`.
Using the cinder action `volume_client.volumes.upload_to_image` cinder will try to upload that volume to glance including those two properties. But they are "reserved attributes" (in glance). The upload will fail with:
INFO cinder.volume.api [req-321adb05-189b-49f9-aa54-62432ee25907 917f4ddabf954b92b84babc613f0292d a69fdf3fa6654adcb8de23c803df6712 - f9f86f7344e942a5a66cec801aa60dbe f9f86f7344e942a5a66cec801aa60dbe] Volume info retrieved successfully.
INFO cinder.volume.api [req-321adb05-189b-49f9-aa54-62432ee25907 917f4ddabf954b92b84babc613f0292d a69fdf3fa6654adcb8de23c803df6712 - f9f86f7344e942a5a66cec801aa60dbe f9f86f7344e942a5a66cec801aa60dbe] Get volume image-metadata completed successfully.
DEBUG glanceclient.common.http [req-321adb05-189b-49f9-aa54-62432ee25907 917f4ddabf954b92b84babc613f0292d a69fdf3fa6654adcb8de23c803df6712 - f9f86f7344e942a5a66cec801aa60dbe f9f86f7344e942a5a66cec801aa60dbe] Request returned failure status 403. _handle_response /var/lib/kolla/venv/lib/python2.7/site-packages/glanceclient/common/http.py:125
ERROR cinder.volume.api [req-321adb05-189b-49f9-aa54-62432ee25907 917f4ddabf954b92b84babc613f0292d a69fdf3fa6654adcb8de23c803df6712 - f9f86f7344e942a5a66cec801aa60dbe f9f86f7344e942a5a66cec801aa60dbe] Error while doing something: HTTPForbidden: 403 Forbidden: Access was denied to this resource.: Attribute 'os_glance_importing_to_stores' is reserved. (HTTP 403)
ERROR cinder.volume.api Traceback (most recent call last):
ERROR cinder.volume.api File "/var/lib/kolla/venv/lib/python2.7/site-packages/cinder/volume/api.py", line 1322, in copy_volume_to_image
ERROR cinder.volume.api context, self.image_service._translate_to_glance(metadata))
ERROR cinder.volume.api File "/var/lib/kolla/venv/lib/python2.7/site-packages/cinder/image/glance.py", line 378, in create
ERROR cinder.volume.api **sent_service_image_meta)
ERROR cinder.volume.api File "/var/lib/kolla/venv/lib/python2.7/site-packages/cinder/image/glance.py", line 225, in call
ERROR cinder.volume.api return getattr(controller, method)(*args, **kwargs)
ERROR cinder.volume.api File "/var/lib/kolla/venv/lib/python2.7/site-packages/glanceclient/common/utils.py", line 598, in inner
ERROR cinder.volume.api return RequestIdProxy(wrapped(*args, **kwargs))
ERROR cinder.volume.api File "/var/lib/kolla/venv/lib/python2.7/site-packages/glanceclient/v2/images.py", line 361, in create
ERROR cinder.volume.api resp, body = self.http_client.post(url, headers=headers, data=image)
ERROR cinder.volume.api File "/var/lib/kolla/venv/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 334, in post
ERROR cinder.volume.api return self.request(url, 'POST', **kwargs)
ERROR cinder.volume.api File "/var/lib/kolla/venv/lib/python2.7/site-packages/glanceclient/common/http.py", line 377, in request
ERROR cinder.volume.api return self._handle_response(resp)
ERROR cinder.volume.api File "/var/lib/kolla/venv/lib/python2.7/site-packages/glanceclient/common/http.py", line 126, in _handle_response
ERROR cinder.volume.api raise exc.from_response(resp, resp.content)
ERROR cinder.volume.api HTTPForbidden: 403 Forbidden: Access was denied to this resource.: Attribute 'os_glance_importing_to_stores' is reserved. (HTTP 403)
ERROR cinder.volume.api
INFO cinder.api.openstack.wsgi [req-321adb05-189b-49f9-aa54-62432ee25907 917f4ddabf954b92b84babc613f0292d a69fdf3fa6654adcb8de23c803df6712 - f9f86f7344e942a5a66cec801aa60dbe f9f86f7344e942a5a66cec801aa60dbe] HTTP exception thrown: 403 Forbidden: Access was denied to this resource.: Attribute 'os_glance_importing_to_stores' is reserved. (HTTP 403)
INFO cinder.api.openstack.wsgi [req-321adb05-189b-49f9-aa54-62432ee25907 917f4ddabf954b92b84babc613f0292d a69fdf3fa6654adcb8de23c803df6712 - f9f86f7344e942a5a66cec801aa60dbe f9f86f7344e942a5a66cec801aa60dbe] http://cinder.service.stage.ewcs.ch/v2/a69fdf3fa6654adcb8de23c803df6712/volumes/16a09728-a1d9-4032-88a4-16a32536f481/action returned with HTTP 400
This issue is known by Nova and was fixed in Nova with:
https://github.com/openstack/nova/commit/dda179d3f901e4f23091f3095f1af58bc26e222e
It looks like the issue is still unknown in cinder?
Howto reproduce:
1. Install devstack stable/wallaby
2. Change glance to enable multistore:
#####################
*** /etc/glance/glance-api.conf.org 2021-09-29 16:39:41.813610795 +0200
--- /etc/glance/glance-api.conf 2021-09-29 16:40:23.397360914 +0200
*************** image_cache_dir = /opt/stack/data/glance
*** 11,16 ****
--- 11,20 ----
use_syslog = False
debug = True
+ enabled_backends = az1:file, az2:file
+ show_multiple_locations = True
+ show_image_direct_url = True
+
[database]
connection = mysql+pymysql://xxxxxxxxx:xyzxyz@127.0.0.1/glance?charset=utf8
*************** auth_type = password
*** 35,42 ****
--- 39,58 ----
[oslo_messaging_notifications]
driver = messagingv2
+ [os_glance_staging_store]
+ filesystem_store_datadir = /opt/stack/data/glance//os_glance_staging_store
+
+ [os_glance_tasks_store]
+ filesystem_store_datadir = /opt/stack/data/glance/os_glance_tasks_store
+
[glance_store]
+ default_backend = az1
+
+ [az1]
filesystem_store_datadir = /opt/stack/data/glance/images/
+ [az2]
+ filesystem_store_datadir = /opt/stack/data/glance/images2/
+
[cors]
allowed_origin = http://172.16.0.6
#######################
mkdir /opt/stack/data/glance/images2
mkdir /opt/stack/data/glance/os_glance_tasks_store
mkdir /opt/stack/data/glance/os_glance_staging_store
systemctl restart devstack at g-api.service
3. Copy image to second store:
glance image-import --stores az2 --import-method copy-image $(openstack image show cirros-0.5.2-x86_64-disk -c id -f value)
4. Create a volume from an image:
openstack volume create --size 1 --image cirros-0.5.2-x86_64-disk testvol
5. Try to upload that volume to glance:
openstack image create --volume testvol --disk-format raw image-from-vol-from-image --debug
This will result in a 403:
HTTP 403 Forbidden: Access was denied to this resource.: Attribute 'os_glance_failed_import' is reserved. (HTTP 400)
Fix: Best would be to not store os_glance* properties with a volume
but we should also remove those properties when a volume is uploaded
to glance.
============
SRU TEMPLATE
============
[Impact]
The issue impacts workflows when downloading an image from glance and
then uploading it back to glance, because of the extra metadata. The
easy workaround is to manually delete the metadata for every image
downloaded from glance prior to uploading the volumes. The fix changed
code only on the upload-volume-to-image API to not submit the metadata
back. Additionally, the behavior can be controlled through a config
option.
[TestCase]
1. Setting up env
1a. Deploy an environment with more than 1 store for glance, such as ceph + swift
1b. Upload cirros image to glance
1c. Add the cirros image to swift store
glance image-import <image-id> --stores swift --import-method copy-
image
1d. List images including store to confirm
glance image-list --include-store
1e. Create a volume using the image
openstack volume create --size 1 --image <image-id/name> testvol
1f. Confirm the "os_glance..." image metadata is in the volume
openstack volume show testvol
2. Reproducing the issue
openstack image create --volume testvol --disk-format raw image-from-
vol-from-image
Result should be:
HTTP 403 Forbidden: Access was denied to this resource.: Attribute
'os_glance_importing_to_stores' is reserved. (HTTP 400)
3. Cleanup not needed
4. Install package that contains the fixed code
5. Adjust cinder.conf as a workaround to issue in comment #21
glance_core_properties = checksum, container_format, disk_format,
image_name, image_id, min_disk, min_ram, name, size,
os_glance_failed_import,os_glance_importing_to_stores
6. Restart cinder services if needed
7. Repeat command in (2), result should now succeed.
[Regression Potential]
Fix has been tested in the Upstream Cinder CI (not specific scenario)
and through unit tests. Behavior is configurable through config
option. In case the code for the upload-volume-to-image API breaks,
then there is no other way to upload a volume to an image.
To manage notifications about this bug go to:
https://bugs.launchpad.net/cinder/+bug/1945500/+subscriptions
More information about the Ubuntu-openstack-bugs
mailing list