[Bug 1805332] Re: [Swift backend] Upload image hit error: Unicode-objects must be encoded before hashing
Liam Young
liam.young at canonical.com
Wed May 22 07:47:48 UTC 2019
The package from rocky-proposed worked for me. Version info below:
python3-glance-store:
Installed: 0.26.1-0ubuntu2.1~cloud0
Candidate: 0.26.1-0ubuntu2.1~cloud0
Version table:
*** 0.26.1-0ubuntu2.1~cloud0 500
500 http://ubuntu-cloud.archive.canonical.com/ubuntu bionic-proposed/rocky/main amd64 Packages
100 /var/lib/dpkg/status
0.26.1-0ubuntu2~cloud0 500
500 http://ubuntu-cloud.archive.canonical.com/ubuntu bionic-updates/rocky/main amd64 Packages
0.23.0-0ubuntu1 500
500 http://nova.clouds.archive.ubuntu.com/ubuntu bionic/universe amd64 Packages
Test output:
$ openstack image create --public --file /home/ubuntu/images/bionic-server-cloudimg-amd64.img bionic-test
500 Internal Server Error: The server has either erred or is incapable of performing the requested operation. (HTTP 500)
$ juju run --unit glance/0 "apt-cache policy python3-glance-store"
python3-glance-store:
Installed: 0.26.1-0ubuntu2~cloud0
Candidate: 0.26.1-0ubuntu2~cloud0
Version table:
*** 0.26.1-0ubuntu2~cloud0 500
500 http://ubuntu-cloud.archive.canonical.com/ubuntu bionic-updates/rocky/main amd64 Packages
100 /var/lib/dpkg/status
0.23.0-0ubuntu1 500
500 http://nova.clouds.archive.ubuntu.com/ubuntu bionic/universe amd64 Packages
$ juju run --unit glance/0 "add-apt-repository cloud-archive:rocky-proposed --yes --update"
...
$ juju run --unit glance/0 "apt install --yes python3-glance-store; systemctl restart glance-api"
...
(clients) ubuntu at gnuoy-bastion2:~/branches/nova-compute$ juju run --unit glance/0 "apt-cache policy python3-glance-store"
python3-glance-store:
Installed: 0.26.1-0ubuntu2.1~cloud0
Candidate: 0.26.1-0ubuntu2.1~cloud0
Version table:
*** 0.26.1-0ubuntu2.1~cloud0 500
500 http://ubuntu-cloud.archive.canonical.com/ubuntu bionic-proposed/rocky/main amd64 Packages
100 /var/lib/dpkg/status
0.26.1-0ubuntu2~cloud0 500
500 http://ubuntu-cloud.archive.canonical.com/ubuntu bionic-updates/rocky/main amd64 Packages
0.23.0-0ubuntu1 500
500 http://nova.clouds.archive.ubuntu.com/ubuntu bionic/universe amd64 Packages
$ openstack image create --public --file /home/ubuntu/images/bionic-server-cloudimg-amd64.img bionic-test
+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| checksum | c8994590c7d61dc68922e461686ef936 |
| container_format | bare |
| created_at | 2019-05-22T07:41:28Z |
| disk_format | raw |
| file | /v2/images/788db968-ea48-4b4f-8c91-4e15d23dbe4c/file |
| id | 788db968-ea48-4b4f-8c91-4e15d23dbe4c |
| min_disk | 0 |
| min_ram | 0 |
| name | bionic-test |
| owner | 3d4ca9d5799546bd852db00ee6d5d4c0 |
| properties | os_hash_algo='sha512', os_hash_value='be4993640deb7eb99b07667213b1fe3a9145df2c0ed5c72cf786a621fe64e93fb543cbb3fafa9a130988b684da432d2a55493c50e77a9dfe336e7ed996be92d9', os_hidden='False' |
| protected | False |
| schema | /v2/schemas/image |
| size | 348127232 |
| status | active |
| tags | |
| updated_at | 2019-05-22T07:41:46Z |
| virtual_size | None |
| visibility | public |
+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
** Tags removed: verification-rocky-needed
** Tags added: verification-rocky-done
--
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to Ubuntu Cloud Archive.
https://bugs.launchpad.net/bugs/1805332
Title:
[Swift backend] Upload image hit error: Unicode-objects must be
encoded before hashing
Status in Ubuntu Cloud Archive:
Fix Released
Status in Ubuntu Cloud Archive rocky series:
Fix Committed
Status in Ubuntu Cloud Archive stein series:
Fix Released
Status in glance_store:
Triaged
Status in python-glance-store package in Ubuntu:
Fix Released
Status in python-glance-store source package in Cosmic:
Fix Committed
Status in python-glance-store source package in Disco:
Fix Released
Status in python-glance-store source package in Eoan:
Fix Released
Bug description:
[Impact]
If we upload a large image (larger than 1G), the glance_store will hit a Unicode error. To fix this a patch has been merged in upstream master and backported to stable rocky.
[Test Case]
Deploy glance related to swift-proxy using the object-store relation. Then attempt to upload a large image (not cirros)
$ openstack image create --public --file /home/ubuntu/images/bionic-server-cloudimg-amd64.img bionic-test
500 Internal Server Error: The server has either erred or is incapable of performing the requested operation. (HTTP 500)
If the patch is manually applied and glance-api restarted then the
above command succeeds.
In order to avoid regression of existing consumers, the OpenStack team will
run their continuous integration test against the packages that are in
-proposed. A successful run of all available tests will be required before the
proposed packages can be let into -updates.
The OpenStack team will be in charge of attaching the output summary of the
executed tests. The OpenStack team members will not mark ‘verification-done’ until
this has happened.
[Regression Potential]
In order to mitigate the regression potential, the results of the
aforementioned tests are attached to this bug.
[Discussion]
n/a
[Original Description]
env: master branch, Glance using swift backend.
We hit a strange error， if we upload a large image (larger than 1G),
the glance_store will hit a error:Unicode-objects must be encoded
before hashing. But if the image is small enough, the error won't
happen.
error log:
https://www.irccloud.com/pastebin/jP3DapNy/
After dig into the code, it appears that when chunk reading the image
data, the date piece may be non-byte, so the checksum.updating will
raise the error.
encoding the date piece to ensure it's byte can solve the problem.
To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/1805332/+subscriptions
More information about the Ubuntu-openstack-bugs
mailing list