[Bug 1805332] Re: [Swift backend] Upload image hit error: Unicode-objects must be encoded before hashing

Liam Young liam.young at canonical.com
Wed May 22 07:47:48 UTC 2019


The package from rocky-proposed worked for me. Version info below:
python3-glance-store:
  Installed: 0.26.1-0ubuntu2.1~cloud0
  Candidate: 0.26.1-0ubuntu2.1~cloud0
  Version table:
 *** 0.26.1-0ubuntu2.1~cloud0 500
        500 http://ubuntu-cloud.archive.canonical.com/ubuntu bionic-proposed/rocky/main amd64 Packages
        100 /var/lib/dpkg/status
     0.26.1-0ubuntu2~cloud0 500
        500 http://ubuntu-cloud.archive.canonical.com/ubuntu bionic-updates/rocky/main amd64 Packages
     0.23.0-0ubuntu1 500
        500 http://nova.clouds.archive.ubuntu.com/ubuntu bionic/universe amd64 Packages

Test output:

$ openstack image create --public --file /home/ubuntu/images/bionic-server-cloudimg-amd64.img bionic-test
500 Internal Server Error: The server has either erred or is incapable of performing the requested operation. (HTTP 500)
                                  
$ juju run --unit glance/0 "apt-cache policy python3-glance-store"
python3-glance-store:
  Installed: 0.26.1-0ubuntu2~cloud0
  Candidate: 0.26.1-0ubuntu2~cloud0
  Version table:
 *** 0.26.1-0ubuntu2~cloud0 500
        500 http://ubuntu-cloud.archive.canonical.com/ubuntu bionic-updates/rocky/main amd64 Packages
        100 /var/lib/dpkg/status
     0.23.0-0ubuntu1 500
        500 http://nova.clouds.archive.ubuntu.com/ubuntu bionic/universe amd64 Packages


$ juju run --unit glance/0 "add-apt-repository cloud-archive:rocky-proposed --yes --update"                                                               
...
$ juju run --unit glance/0 "apt install --yes python3-glance-store; systemctl restart glance-api"
...
(clients) ubuntu at gnuoy-bastion2:~/branches/nova-compute$ juju run --unit glance/0 "apt-cache policy python3-glance-store"
python3-glance-store:
  Installed: 0.26.1-0ubuntu2.1~cloud0
  Candidate: 0.26.1-0ubuntu2.1~cloud0
  Version table:
 *** 0.26.1-0ubuntu2.1~cloud0 500
        500 http://ubuntu-cloud.archive.canonical.com/ubuntu bionic-proposed/rocky/main amd64 Packages
        100 /var/lib/dpkg/status
     0.26.1-0ubuntu2~cloud0 500
        500 http://ubuntu-cloud.archive.canonical.com/ubuntu bionic-updates/rocky/main amd64 Packages
     0.23.0-0ubuntu1 500
        500 http://nova.clouds.archive.ubuntu.com/ubuntu bionic/universe amd64 Packages

$ openstack image create --public --file /home/ubuntu/images/bionic-server-cloudimg-amd64.img bionic-test
+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field            | Value                                                                                                                                                                                      |
+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| checksum         | c8994590c7d61dc68922e461686ef936                                                                                                                                                           |
| container_format | bare                                                                                                                                                                                       |
| created_at       | 2019-05-22T07:41:28Z                                                                                                                                                                       |
| disk_format      | raw                                                                                                                                                                                        |
| file             | /v2/images/788db968-ea48-4b4f-8c91-4e15d23dbe4c/file                                                                                                                                       |
| id               | 788db968-ea48-4b4f-8c91-4e15d23dbe4c                                                                                                                                                       |
| min_disk         | 0                                                                                                                                                                                          |
| min_ram          | 0                                                                                                                                                                                          |
| name             | bionic-test                                                                                                                                                                                |
| owner            | 3d4ca9d5799546bd852db00ee6d5d4c0                                                                                                                                                           |
| properties       | os_hash_algo='sha512', os_hash_value='be4993640deb7eb99b07667213b1fe3a9145df2c0ed5c72cf786a621fe64e93fb543cbb3fafa9a130988b684da432d2a55493c50e77a9dfe336e7ed996be92d9', os_hidden='False' |
| protected        | False                                                                                                                                                                                      |
| schema           | /v2/schemas/image                                                                                                                                                                          |
| size             | 348127232                                                                                                                                                                                  |
| status           | active                                                                                                                                                                                     |
| tags             |                                                                                                                                                                                            |
| updated_at       | 2019-05-22T07:41:46Z                                                                                                                                                                       |
| virtual_size     | None                                                                                                                                                                                       |
| visibility       | public                                                                                                                                                                                     |
+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+


** Tags removed: verification-rocky-needed
** Tags added: verification-rocky-done

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to Ubuntu Cloud Archive.
https://bugs.launchpad.net/bugs/1805332

Title:
  [Swift backend] Upload image hit error: Unicode-objects must be
  encoded before hashing

Status in Ubuntu Cloud Archive:
  Fix Released
Status in Ubuntu Cloud Archive rocky series:
  Fix Committed
Status in Ubuntu Cloud Archive stein series:
  Fix Released
Status in glance_store:
  Triaged
Status in python-glance-store package in Ubuntu:
  Fix Released
Status in python-glance-store source package in Cosmic:
  Fix Committed
Status in python-glance-store source package in Disco:
  Fix Released
Status in python-glance-store source package in Eoan:
  Fix Released

Bug description:
  [Impact]
  If we upload a large image (larger than 1G), the glance_store will hit a Unicode error. To fix this a patch has been merged in upstream master and backported to stable rocky.

  [Test Case]
  Deploy glance related to swift-proxy using the object-store relation. Then attempt to upload a large image (not cirros)

  $ openstack image create --public --file /home/ubuntu/images/bionic-server-cloudimg-amd64.img bionic-test
  500 Internal Server Error: The server has either erred or is incapable of performing the requested operation. (HTTP 500)

  If the patch is manually applied and glance-api restarted then the
  above command succeeds.

  In order to avoid regression of existing consumers, the OpenStack team will
  run their continuous integration test against the packages that are in
  -proposed. A successful run of all available tests will be required before the
  proposed packages can be let into -updates.

  The OpenStack team will be in charge of attaching the output summary of the
  executed tests. The OpenStack team members will not mark ‘verification-done’ until
  this has happened.

  [Regression Potential]
  In order to mitigate the regression potential, the results of the
  aforementioned tests are attached to this bug.

  [Discussion]
  n/a

  [Original Description]

  env: master branch, Glance using swift backend.

  We hit a strange error, if we upload a large image (larger than 1G),
  the glance_store will hit a error:Unicode-objects must be encoded
  before hashing. But if the image is small enough, the error won't
  happen.

  error log:
  https://www.irccloud.com/pastebin/jP3DapNy/

  After dig into the code, it appears that when chunk reading the image
  data, the date piece may be non-byte, so the checksum.updating will
  raise the error.

  encoding the date piece to ensure it's byte can solve the problem.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/1805332/+subscriptions



More information about the Ubuntu-openstack-bugs mailing list