[Bug 1906266] Re: After upgrade: "libvirt.libvirtError: Requested operation is not valid: format of backing image %s of image %s was not specified"
Christian Ehrhardt
1906266 at bugs.launchpad.net
Tue Jun 29 07:26:35 UTC 2021
** Description changed:
+ [Impact]
+
+ * New libvirt got more strict in regard to file format specification.
+ While this is generally the right approach it causes some issues for
+ upgraders that have old image chains now failing.
+
+ * Upstream has added code to relax those checks under a set of conditions
+ which will allow to go forward with stricter conditions as planned but
+ at the same time not break/block upgrades.
+
+ [Test Plan]
+
+ * TBD
+
+ [Where problems could occur]
+
+ * Of the many things that qemu/libvirt do this changes only the format
+ probing. So issues (hopefully not) would be expected to appear mostly
+ around complex scenarios of image files.
+ We've had a look at image files and image file chains, and so far all
+ were good. But there are more obscure (and not supported) cases like
+ image backed by real-disk that might misbehave. But still it would
+ fix Focal to be the outlier as the past was ok (didn't care) and the
+ future (relaxed check) and only focal is left broken in between.
+
+ [Other Info]
+
+ * A lot has changes in that area, but instead of pulling in a vast set
+ of changes a smaller set was identified to suite the SRU needs. It
+ was so far found not found regressing anything and OTOH fixed the issue
+ (tested form PPA) for affected people.
+
+ ----
+
In a site upgraded to Ussuri we are getting faults starting instances
2020-11-30 13:41:40.586 232871 ERROR oslo_messaging.rpc.server
libvirt.libvirtError: Requested operation is not valid: format of
backing image '/var/lib/nova/instances/_base/xxx' of image
'/var/lib/nova/instances/xxx' was not specified in the image metadata
(See https://libvirt.org/kbase/backing_chains.html for troubleshooting)
Bug #1864020 reports similar symptoms, where due to an upstream change
in Libvirt v6.0.0+ images need the backing format specified.
The fix for Bug #1864020 handles the case for new instances. However,
for upgraded instances we're hitting the same problem, as those still
don't have backing format specified.
** Description changed:
[Impact]
- * New libvirt got more strict in regard to file format specification.
- While this is generally the right approach it causes some issues for
- upgraders that have old image chains now failing.
+ * New libvirt got more strict in regard to file format specification.
+ While this is generally the right approach it causes some issues for
+ upgraders that have old image chains now failing.
- * Upstream has added code to relax those checks under a set of conditions
- which will allow to go forward with stricter conditions as planned but
- at the same time not break/block upgrades.
+ * Upstream has added code to relax those checks under a set of conditions
+ which will allow to go forward with stricter conditions as planned but
+ at the same time not break/block upgrades.
[Test Plan]
- * TBD
+ * Thanks to Brett Milford for sharing his test steps for this
+
+ sudo apt-get update
+ sudo apt-get install libvirt-daemon-system cloud-image-utils virtinst -y
+
+ IMG="focal-server-cloudimg-amd64.img"
+ IMG_PATH="/var/lib/libvirt/images/base/$IMG"
+ INSTANCE_NAME=testinst
+ [ -f $IMG_PATH ] || {
+ sudo mkdir -p /var/lib/libvirt/images/base
+ sudo wget https://cloud-images.ubuntu.com/focal/current/focal-server-cloudimg-amd64.img \
+ -O $IMG_PATH
+ }
+ sudo mkdir -p /var/lib/libvirt/images/$INSTANCE_NAME
+ sudo qemu-img convert -O raw $IMG_PATH ${IMG_PATH%.*}
+ sudo qemu-img create -f qcow2 -o backing_file=${IMG_PATH%.*} /var/lib/libvirt/images/$INSTANCE_NAME/root.img
+ sudo qemu-img resize /var/lib/libvirt/images/$INSTANCE_NAME/root.img 5G
+
+ virt-install --connect qemu:///system --name $INSTANCE_NAME --cpu host
+ --os-type linux --os-variant generic --graphics vnc --console
+ pty,target_type=serial --disk
+ path=/var/lib/libvirt/images/$INSTANCE_NAME/root.img,bus=virtio,format=qcow2
+ --network default,model=virtio --noautoconsole --vcpus 1 --memory 1024
+ --import
+
[Where problems could occur]
- * Of the many things that qemu/libvirt do this changes only the format
- probing. So issues (hopefully not) would be expected to appear mostly
- around complex scenarios of image files.
- We've had a look at image files and image file chains, and so far all
- were good. But there are more obscure (and not supported) cases like
- image backed by real-disk that might misbehave. But still it would
- fix Focal to be the outlier as the past was ok (didn't care) and the
- future (relaxed check) and only focal is left broken in between.
+ * Of the many things that qemu/libvirt do this changes only the format
+ probing. So issues (hopefully not) would be expected to appear mostly
+ around complex scenarios of image files.
+ We've had a look at image files and image file chains, and so far all
+ were good. But there are more obscure (and not supported) cases like
+ image backed by real-disk that might misbehave. But still it would
+ fix Focal to be the outlier as the past was ok (didn't care) and the
+ future (relaxed check) and only focal is left broken in between.
[Other Info]
-
- * A lot has changes in that area, but instead of pulling in a vast set
- of changes a smaller set was identified to suite the SRU needs. It
- was so far found not found regressing anything and OTOH fixed the issue
- (tested form PPA) for affected people.
+
+ * A lot has changes in that area, but instead of pulling in a vast set
+ of changes a smaller set was identified to suite the SRU needs. It
+ was so far found not found regressing anything and OTOH fixed the issue
+ (tested form PPA) for affected people.
----
In a site upgraded to Ussuri we are getting faults starting instances
2020-11-30 13:41:40.586 232871 ERROR oslo_messaging.rpc.server
libvirt.libvirtError: Requested operation is not valid: format of
backing image '/var/lib/nova/instances/_base/xxx' of image
'/var/lib/nova/instances/xxx' was not specified in the image metadata
(See https://libvirt.org/kbase/backing_chains.html for troubleshooting)
Bug #1864020 reports similar symptoms, where due to an upstream change
in Libvirt v6.0.0+ images need the backing format specified.
The fix for Bug #1864020 handles the case for new instances. However,
for upgraded instances we're hitting the same problem, as those still
don't have backing format specified.
--
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to nova in Ubuntu.
https://bugs.launchpad.net/bugs/1906266
Title:
After upgrade: "libvirt.libvirtError: Requested operation is not
valid: format of backing image %s of image %s was not specified"
Status in OpenStack Compute (nova):
Won't Fix
Status in libvirt package in Ubuntu:
Fix Released
Status in nova package in Ubuntu:
New
Status in libvirt source package in Focal:
Triaged
Status in nova source package in Focal:
New
Status in libvirt source package in Groovy:
Invalid
Status in nova source package in Groovy:
New
Bug description:
[Impact]
* New libvirt got more strict in regard to file format specification.
While this is generally the right approach it causes some issues for
upgraders that have old image chains now failing.
* Upstream has added code to relax those checks under a set of conditions
which will allow to go forward with stricter conditions as planned but
at the same time not break/block upgrades.
[Test Plan]
* Thanks to Brett Milford for sharing his test steps for this
sudo apt-get update
sudo apt-get install libvirt-daemon-system cloud-image-utils virtinst -y
IMG="focal-server-cloudimg-amd64.img"
IMG_PATH="/var/lib/libvirt/images/base/$IMG"
INSTANCE_NAME=testinst
[ -f $IMG_PATH ] || {
sudo mkdir -p /var/lib/libvirt/images/base
sudo wget https://cloud-images.ubuntu.com/focal/current/focal-server-cloudimg-amd64.img \
-O $IMG_PATH
}
sudo mkdir -p /var/lib/libvirt/images/$INSTANCE_NAME
sudo qemu-img convert -O raw $IMG_PATH ${IMG_PATH%.*}
sudo qemu-img create -f qcow2 -o backing_file=${IMG_PATH%.*} /var/lib/libvirt/images/$INSTANCE_NAME/root.img
sudo qemu-img resize /var/lib/libvirt/images/$INSTANCE_NAME/root.img 5G
virt-install --connect qemu:///system --name $INSTANCE_NAME --cpu host
--os-type linux --os-variant generic --graphics vnc --console
pty,target_type=serial --disk
path=/var/lib/libvirt/images/$INSTANCE_NAME/root.img,bus=virtio,format=qcow2
--network default,model=virtio --noautoconsole --vcpus 1 --memory 1024
--import
[Where problems could occur]
* Of the many things that qemu/libvirt do this changes only the format
probing. So issues (hopefully not) would be expected to appear mostly
around complex scenarios of image files.
We've had a look at image files and image file chains, and so far all
were good. But there are more obscure (and not supported) cases like
image backed by real-disk that might misbehave. But still it would
fix Focal to be the outlier as the past was ok (didn't care) and the
future (relaxed check) and only focal is left broken in between.
[Other Info]
* A lot has changes in that area, but instead of pulling in a vast set
of changes a smaller set was identified to suite the SRU needs. It
was so far found not found regressing anything and OTOH fixed the issue
(tested form PPA) for affected people.
----
In a site upgraded to Ussuri we are getting faults starting instances
2020-11-30 13:41:40.586 232871 ERROR oslo_messaging.rpc.server
libvirt.libvirtError: Requested operation is not valid: format of
backing image '/var/lib/nova/instances/_base/xxx' of image
'/var/lib/nova/instances/xxx' was not specified in the image metadata
(See https://libvirt.org/kbase/backing_chains.html for
troubleshooting)
Bug #1864020 reports similar symptoms, where due to an upstream change
in Libvirt v6.0.0+ images need the backing format specified.
The fix for Bug #1864020 handles the case for new instances. However,
for upgraded instances we're hitting the same problem, as those still
don't have backing format specified.
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1906266/+subscriptions
More information about the Ubuntu-openstack-bugs
mailing list