[Bug 2166537] Re: [SRU] python-glance-store 2024.1 fix Cinder Dell PowerFlex
Himawan Winarto
2166537 at bugs.launchpad.net
Wed Sep 9 16:58:55 UTC 2026
Woops wrong upstream link, should be
https://opendev.org/openstack/glance_store/commit/55cefedd169b84d713a8ff080d909483ea51c913
Edited. This was requested for Jammy-Carracal.
** Description changed:
[ Impact ]
* When Glance is configured to use Cinder as a store backend and the Cinder
volume backend uses the Dell PowerFlex (ScaleIO) driver, images can be
created/uploaded into the store, but every subsequent read of the image data
fails and returns no data.
* The ScaleIO brick connector in glance_store determines the device size by
seeking to the end of the attached volume device (os.fstat/stat report
st_size=0 for block devices exported by the ScaleIO/PowerFlex protocol,
so seek/tell is the only working method). This size probe is part of the
device-resize wait mechanism that runs on every volume attach -- including
image download -- and the file pointer is left at end-of-file afterwards, so
the download reads 0 bytes.
* This regression was introduced upstream with the generalized device-resize
wait mechanism in glance_store (reviews 826290, 843103) and is present since
glance_store 4.5.0 (2023.1 Antelope); both the Ubuntu archive (noble,
python-glance-store 4.7.0-0ubuntu1) and the UCA jammy-caracal pocket
(python-glance-store 4.7.0-0ubuntu1~cloud0) carry it.
* Backporting this fix restores correct image download by saving the device
file pointer before the size probe and restoring it afterwards, so reads
start at the correct offset. The uploaded fix comes from the upstream patch
that was already backported to glance_store stable/2024.1 and incorporated
in the 4.7.1 (Caracal) point release.
[ Test Plan ]
* [0] The following versions of Ubuntu-OpenStack will be tested:
- Ubuntu archive: noble
- UCA: jammy-caracal
* [1] Deploy an OpenStack environment with Glance configured to use the cinder
store (e.g. enabled_backends including cinder) and a Cinder volume backend
using the Dell PowerFlex (ScaleIO) driver (a volume type with
volume_backend_name pointing to the PowerFlex backend).
* [2] Upload an image into the cinder store:
openstack image create --file <img> --store cinder ...
this works before and after the patch.
* [3] Download the image data back from the cinder store:
openstack image save --file downloaded.img <image>
create a bootable volume from the image:
openstack volume create --size <n> --type powerflex --image <image> \
--bootable <vol>
* [4] Verify the behavior -- Before the patch, the downloaded image file is
empty/fails its checksum and the volume-create-from-image flow fails with
Corrupt image download. Hash was <hash>... expected <image
os_hash_value>
with volume goes to error. After the patch, openstack image save returns the
full image with matching SHA-512 and volumes created from the image become
available and bootable.
[ Where problems could occur ]
- * The change is confined to _get_device_size in the ScaleIO brick connector
- (glance_store/_drivers/cinder/scaleio.py), which is only called when
- glance's cinder store attaches a volume with driver_volume_type scaleio.
- Other glance store backends (file, http, swift, rbd) and cinder deployments
- using other drivers (NFS, RBD, iSCSI, etc.) are untouched.
- * The fix only saves and restores the file pointer around the pre-existing size
- probe; the device file object already had to support seek()/tell() for the
- wait mechanism, so no new capability is assumed of any backend.
+ * The change is confined to _get_device_size in the ScaleIO brick connector
+ (glance_store/_drivers/cinder/scaleio.py), which is only called when
+ glance's cinder store attaches a volume with driver_volume_type scaleio.
+ Other glance store backends (file, http, swift, rbd) and cinder deployments
+ using other drivers (NFS, RBD, iSCSI, etc.) are untouched.
+ * The fix only saves and restores the file pointer around the pre-existing size
+ probe; the device file object already had to support seek()/tell() for the
+ wait mechanism, so no new capability is assumed of any backend.
[ Other Info ]
- * Upstream fix to glance_store (master): https://review.opendev.org/c/openstack/glance_store/+/826290
+ * Upstream fix to glance_store (master): https://opendev.org/openstack/glance_store/commit/55cefedd169b84d713a8ff080d909483ea51c913
* Upstream commit to glance_store (2024.1):
https://opendev.org/openstack/glance_store/commit/8fbdf992296a7dad9fcf5fc7564e59f9c60da034
--
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to Ubuntu Cloud Archive.
https://bugs.launchpad.net/bugs/2166537
Title:
[SRU] python-glance-store 2024.1 fix Cinder Dell PowerFlex
Status in Ubuntu Cloud Archive:
In Progress
Status in Ubuntu Cloud Archive caracal series:
In Progress
Status in Ubuntu Cloud Archive dalmatian series:
Invalid
Status in python-glance-store package in Ubuntu:
In Progress
Status in python-glance-store source package in Noble:
In Progress
Bug description:
[ Impact ]
* When Glance is configured to use Cinder as a store backend and the Cinder
volume backend uses the Dell PowerFlex (ScaleIO) driver, images can be
created/uploaded into the store, but every subsequent read of the image data
fails and returns no data.
* The ScaleIO brick connector in glance_store determines the device size by
seeking to the end of the attached volume device (os.fstat/stat report
st_size=0 for block devices exported by the ScaleIO/PowerFlex protocol,
so seek/tell is the only working method). This size probe is part of the
device-resize wait mechanism that runs on every volume attach -- including
image download -- and the file pointer is left at end-of-file afterwards, so
the download reads 0 bytes.
* This regression was introduced upstream with the generalized device-resize
wait mechanism in glance_store (reviews 826290, 843103) and is present since
glance_store 4.5.0 (2023.1 Antelope); both the Ubuntu archive (noble,
python-glance-store 4.7.0-0ubuntu1) and the UCA jammy-caracal pocket
(python-glance-store 4.7.0-0ubuntu1~cloud0) carry it.
* Backporting this fix restores correct image download by saving the device
file pointer before the size probe and restoring it afterwards, so reads
start at the correct offset. The uploaded fix comes from the upstream patch
that was already backported to glance_store stable/2024.1 and incorporated
in the 4.7.1 (Caracal) point release.
[ Test Plan ]
* [0] The following versions of Ubuntu-OpenStack will be tested:
- Ubuntu archive: noble
- UCA: jammy-caracal
* [1] Deploy an OpenStack environment with Glance configured to use the cinder
store (e.g. enabled_backends including cinder) and a Cinder volume backend
using the Dell PowerFlex (ScaleIO) driver (a volume type with
volume_backend_name pointing to the PowerFlex backend).
* [2] Upload an image into the cinder store:
openstack image create --file <img> --store cinder ...
this works before and after the patch.
* [3] Download the image data back from the cinder store:
openstack image save --file downloaded.img <image>
create a bootable volume from the image:
openstack volume create --size <n> --type powerflex --image <image> \
--bootable <vol>
* [4] Verify the behavior -- Before the patch, the downloaded image file is
empty/fails its checksum and the volume-create-from-image flow fails with
Corrupt image download. Hash was <hash>... expected <image
os_hash_value>
with volume goes to error. After the patch, openstack image save returns the
full image with matching SHA-512 and volumes created from the image become
available and bootable.
[ Where problems could occur ]
* The change is confined to _get_device_size in the ScaleIO brick connector
(glance_store/_drivers/cinder/scaleio.py), which is only called when
glance's cinder store attaches a volume with driver_volume_type scaleio.
Other glance store backends (file, http, swift, rbd) and cinder deployments
using other drivers (NFS, RBD, iSCSI, etc.) are untouched.
* The fix only saves and restores the file pointer around the pre-existing size
probe; the device file object already had to support seek()/tell() for the
wait mechanism, so no new capability is assumed of any backend.
[ Other Info ]
* Upstream fix to glance_store (master): https://opendev.org/openstack/glance_store/commit/55cefedd169b84d713a8ff080d909483ea51c913
* Upstream commit to glance_store (2024.1):
https://opendev.org/openstack/glance_store/commit/8fbdf992296a7dad9fcf5fc7564e59f9c60da034
To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/2166537/+subscriptions
More information about the Ubuntu-openstack-bugs
mailing list