[Bug 1625489] [NEW] Glance fails to retrieve image from rbd backend

[Dr. Jens Rosenboom]

Public bug reported:

We have a deployment with three controller nodes running glance version
2:12.0.0-0ubuntu2 with a Ceph backend. About 50% of the image download
requests from compute nodes are failing with an error like this:

2016-09-20 07:37:32.014 29606 WARNING glance.location [req-78e5191c-
e31b-428e-82c5-30ef87feef4e a73f3a58c6ad478a838c3ca9353704f7
7a17e2b80f274591aec96baec3f6bf68 - - -] Get image 41a6aadf-c655-49c8
-b77d-eedd356f5349 data failed: 'utf8' codec can't decode byte 0x82 in
position 24: invalid start byte.

Looking at the code, it looks like the error is occuring in
glance_store/_drivers/rbd.py while reading the image, but it seems very
strange that glance is trying to utf8-decode a raw image, which pretty
likely is bound to fail. But then it is unclear why only every other
request is failing.

The error can be reproduced by just doing a "glance image-download", so
it's nothing special happening from the Nova side.

** Affects: glance (Ubuntu)
     Importance: Undecided
         Status: New


** Tags: rbd

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to glance in Ubuntu.
https://bugs.launchpad.net/bugs/1625489

Title:
  Glance fails to retrieve image from rbd backend

Status in glance package in Ubuntu:
  New

Bug description:
  We have a deployment with three controller nodes running glance
  version 2:12.0.0-0ubuntu2 with a Ceph backend. About 50% of the image
  download requests from compute nodes are failing with an error like
  this:

  2016-09-20 07:37:32.014 29606 WARNING glance.location [req-78e5191c-
  e31b-428e-82c5-30ef87feef4e a73f3a58c6ad478a838c3ca9353704f7
  7a17e2b80f274591aec96baec3f6bf68 - - -] Get image 41a6aadf-c655-49c8
  -b77d-eedd356f5349 data failed: 'utf8' codec can't decode byte 0x82 in
  position 24: invalid start byte.

  Looking at the code, it looks like the error is occuring in
  glance_store/_drivers/rbd.py while reading the image, but it seems
  very strange that glance is trying to utf8-decode a raw image, which
  pretty likely is bound to fail. But then it is unclear why only every
  other request is failing.

  The error can be reproduced by just doing a "glance image-download",
  so it's nothing special happening from the Nova side.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/glance/+bug/1625489/+subscriptions