[Bug 948461] Re: apt-get hashsum/size mismatch because s3 mirrors don't support http pipelining correctly
Ben Howard
948461 at bugs.launchpad.net
Thu Mar 8 19:57:56 UTC 2012
The problem here is that S3 is broken regarding HTTP pipeline responses.
What is happening is that the client is that apt is requesting files A,
B, C, D in the same requests and S3 appears to be responding with files
B, C, D, A (or some simularly arbitrary order). The result is that apt
saves the data under the wrong file name.
Here is a summary of a packet trace (edited for readability). S3 uses
the "ETag" header to store the MD5 of the file. So using the the Etag,
you can conclude what file contents are.
GET /ubuntu/pool/main/libx/libxdamage/libxdamage1_1.1.3-2build1_amd64.deb HTTP/1.1
GET /ubuntu/pool/main/libx/libxfixes/libxfixes3_5.0-4ubuntu1_amd64.deb HTTP/1.1
GET /ubuntu/pool/main/libx/libxxf86vm/libxxf86vm1_1.1.1-2build1_amd64.deb HTTP/1.1
GET /ubuntu/pool/main/m/mesa/libgl1-mesa-glx_8.0.1-0ubuntu2_amd64.deb HTTP/1.1
GET /ubuntu/pool/main/g/gtk+2.0/libgtk2.0-common_2.24.10-0ubuntu5_all.deb HTTP/1.1
GET /ubuntu/pool/main/libx/libxcomposite/libxcomposite1_0.4.3-2build1_amd64.deb HTTP/1.1
GET /ubuntu/pool/main/libx/libxcursor/libxcursor1_1.1.12-1_amd64.deb HTT
GET /ubuntu/pool/main/libx/libxi/libxi6_1.5.99.3-0ubuntu2_amd64.deb HTTP/1.1
GET /ubuntu/pool/main/libx/libxinerama/libxinerama1_1.1.1-3build1_amd64.deb HTTP/1.1
6e773d7e05527fa35a201209660b517c - libxfixes3
x-amz-id-2: EgVy5vizxmUOYRxfyjFXJzt7k30EVGtn2gyZ1q0P/jN3ROZS+3KjUEwSzX3JFJKE
x-amz-request-id: 8CD6DF060D72CFDA
c06221dc919029cf8482198e724fa111 - libxxf86vm1
x-amz-id-2: xr56YqhmQzov1YP+Oev9lnpKQtOYX98DkrJD7aWkSwQq8Ke77+4PD4yHiRvcBo6w
x-amz-request-id: 67B263265E35A521
GET /ubuntu/pool/main/g/gtk+2.0/libgtk2.0-0_2.24.10-0ubuntu5_amd64.deb HTTP/1.1
a47eb17218d742a4ceae59cc08e80d8c - libgl1-mesa-glx
x-amz-id-2: Ou92TrRA7dfHyF97UEfO0TvQ9XI9vOxYj0LgWCR3GXFHMhAZ7nzQ71cGUqMQHI1X
x-amz-request-id: C9393E43C43C6471
90787d373defad09a6dd99db3c3e7614 - libgtk2.0-common
x-amz-id-2: ze4hg1y4hT8+ZXXzY7JHRclypybdYj+HIQEG6r2eulAt+twaZn/GG+y6ifi9MnL8
x-amz-request-id: 5DDC52A0301600FC
77440ea3c635a3243cc3f6bc98305f22 - libxcomposite1
x-amz-id-2: +hzkmvonwW+1nh4gcsQ/qXsmXUq2CUj4hAQ69EjqAEb6MVOpxHvPnh8l7Qh2Wc5Q
x-amz-request-id: 54FDE3289C1021F3
f5754520b029296a6f0a5a14a1dd4b5a - libxcursor1
x-amz-id-2: WvhZ9o1R3kMvsFEmERvI2V/eOY+0ksHnvs1suVCX1Es8aa/XxyU7drgvj0Bj9zHj
x-amz-request-id: 9F408ACCBFFA66E5
728ef8819d6f3ec4b827d887951a86ce - libxi6
x-amz-id-2: Tgqse2GgWGu+OEVzv/atXhjP6FdszvjVeFGic3gq5LxGUXDFVS5YdF+CGFtF7dkC
x-amz-request-id: C57783745F1192CE
90b84d263ad2b7f76868ce02967c2107 - libxinerama1
x-amz-id-2: 1Mz7B1r13S1NnNIingOxj2iOlQq8a5arP7K/Wt/zgd+NL78Q73/MRDzFhPf4kXFL
x-amz-request-id: DB666BE3004C6CD9
652e1f2550726fc72fa432413e6e287b - libxdamage1
x-amz-id-2: X5TQqCJey0xtne7jKZYlX+8c5AqRM2aTVD1cvEHWPIx4qcc8L9rvyqlG0FJcPT4g
x-amz-request-id: 74A2EB9093B4F00A
c8f474321079048af432d47d66a4afb4 - libgtk2.0-0
x-amz-id-2: Fgkgv/JY7WnSojk+pBeQ430gNCvjRvXG1KZzrbQfEG2j/jqPvvcFVRS07ZBMsFa9
x-amz-request-id: 57FF62A675D21685
GET /ubuntu/pool/main/g/gtk+2.0/libgtk2.0-bin_2.24.10-0ubuntu5_amd64.deb HTTP/1.1
e03a09b33f955e8da797a2f33fbf3423 - libgtk2.0-bin
x-amz-id-2: RaJe83jyFhTwQdvnp6ZjYJQMwKOIengoH2U//VnJZxgagvTPmGYMsAwKGkjGh+Mz
x-amz-request-id: B9D4754504E9CA3E
--
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to apt in Ubuntu.
https://bugs.launchpad.net/bugs/948461
Title:
apt-get hashsum/size mismatch because s3 mirrors don't support http
pipelining correctly
Status in “apt” package in Ubuntu:
Confirmed
Status in “apt” source package in Lucid:
New
Status in “apt” source package in Maverick:
New
Status in “apt” source package in Natty:
New
Status in “apt” source package in Oneiric:
New
Status in “apt” source package in Precise:
Confirmed
Status in “apt” source package in Hardy:
New
Bug description:
apt-get is appears to be mangling the local caching file names, which
is appearing as a hashsum or size mismatch.
Evidence below:
_______________________________________________________________________________________
Output of "apt-get -y install firefox":
Failed to fetch http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com/ubuntu/pool/main/p/python-defer/python-defer_1.0.2+bzr481-1_all.deb Size mismatch
Failed to fetch http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com/ubuntu/pool/main/a/aptdaemon/python-aptdaemon_0.43+bzr769-0ubuntu1_all.deb Size mismatch
Failed to fetch http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com/ubuntu/pool/main/a/aptdaemon/aptdaemon_0.43+bzr769-0ubuntu1_all.deb Size mismatch
Failed to fetch http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com/ubuntu/pool/main/u/ubufox/xul-ext-ubufox_2.0-0ubuntu1_all.deb Size mismatch
_______________________________________________________________________________________
apt-get -y install firefox with debug for HTTP turned on:
Get:63 http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com/ubuntu/ precise/main python-defer all 1.0.2+bzr481-1 [10.9 kB]
HTTP/1.1 200 OK^M
x-amz-id-2: EXccd6GVMfE6ly4SYdwy313VK42d/gI3ncyxmaotuVIMbBBi6FJkuDYWrzLw7vXE^M
x-amz-request-id: 7572BAB5E6FFFAC0^M
Date: Tue, 06 Mar 2012 20:06:13 GMT^M
Last-Modified: Fri, 03 Feb 2012 08:54:27 GMT^M
ETag: "6c07f8db615cc32657b1cc57450a1608"^M
Accept-Ranges: bytes^M
Content-Type: application/x-debian-package^M
Content-Length: 15142^M
Server: AmazonS3^M
^M
Get:64 http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com/ubuntu/ precise/main python-aptdaemon all 0.43+bzr769-0ubuntu1 [80.3 kB]
HTTP/1.1 200 OK^M
x-amz-id-2: IcNECY1OtzHOXnTnTW3fqoJGyfdODq+qRXXGYTDEOJxfw0CP9UH8EUSllVR3Gf3F^M
x-amz-request-id: 5DEEEE629E3BA012^M
Date: Tue, 06 Mar 2012 20:06:13 GMT^M
Last-Modified: Fri, 02 Mar 2012 20:55:40 GMT^M
ETag: "501373631a241847cf76d13aea9264b3"^M
Accept-Ranges: bytes^M
Content-Type: application/x-debian-package^M
Content-Length: 56824^M
Server: AmazonS3^M
^M
Get:65 http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com/ubuntu/ precise/main aptdaemon all 0.43+bzr769-0ubuntu1 [15.1 kB]
HTTP/1.1 200 OK^M
x-amz-id-2: HHNSOZKH74D4I1XCaOiwDOzIWxIVJ88ibfKxvIp/4NpkczzPmsOhMqxrTj2T02qO^M
x-amz-request-id: A6FD3143E6A0BA1A^M
Date: Tue, 06 Mar 2012 20:06:34 GMT^M
Last-Modified: Sat, 21 Jan 2012 00:48:55 GMT^M
ETag: "3653165af1f20a437a14632ce0a2e6c2"^M
Accept-Ranges: bytes^M
Content-Type: application/x-debian-package^M
Content-Length: 10902^M
Server: AmazonS3^M
_______________________________________________________________________________________
Evaluation of the MD5sums of downloaded failed items:
root at ip-10-6-85-206:/var/cache/apt/archives/partial# md5sum *
501373631a241847cf76d13aea9264b3 aptdaemon_0.43+bzr769-0ubuntu1_all.deb
6c07f8db615cc32657b1cc57450a1608 python-aptdaemon_0.43+bzr769-0ubuntu1_all.deb
566f7c92a9fcd0c5c40d08a977176291 python-defer_1.0.2+bzr481-1_all.deb
3653165af1f20a437a14632ce0a2e6c2 xul-ext-ubufox_2.0-0ubuntu1_all.deb
_______________________________________________________________________________________
Snipets from Packages.bz2
From Meta-data:
Package: aptdaemon
MD5sum: 6c07f8db615cc32657b1cc57450a1608
Package: xul-ext-ubufox
MD5sum: 501373631a241847cf76d13aea9264b3
Package: python-aptdaemon
MD5sum: 566f7c92a9fcd0c5c40d08a977176291
Package: python-defer
MD5sum: 3653165af1f20a437a14632ce0a2e6c2
_______________________________________________________________________________________
"dpkg -i" on pacakges downloaded
root at ip-10-6-85-206:/var/cache/apt/archives/partial# dpkg -I aptdaemon_0.43+bzr769-0ubuntu1_all.deb
new debian package, version 2.0.
size 56824 bytes: control archive= 1520 bytes.
23 bytes, 1 lines conffiles
1023 bytes, 29 lines control
1804 bytes, 21 lines md5sums
Package: xul-ext-ubufox
Source: ubufox
Version: 2.0-0ubuntu1
Architecture: all
Maintainer: Ubuntu Mozilla Team <ubuntu-mozillateam at lists.ubuntu.com>
Installed-Size: 383
Depends: aptdaemon, libglib2.0-0 (>= 2.26)
Recommends: firefox (>= 4.0~b6)
Enhances: firefox
Breaks: ubufox (<< 0.9~rc2-0ubuntu3)a
Replaces: ubufox (<< 0.9~rc2-0ubuntu3)
Provides: firefox-ubufox, ubufox
Section: web
Priority: optional
Homepage: https://launchpad.net/ubufox
Description: Ubuntu-specific configuration defaults and apt support for Firefox
Adds Ubuntu-specific modifications to Firefox.
.
Integrates the browser with Ubuntu to:
* Enable searching for missing plugins from Ubuntu software catalog
* Add the following options to the Help menu
- Get help on-line
- Help translating Firefox
- Ubuntu Release Notes
* Set homepage to Ubuntu Start Page
* Display a restart notification after upgrading Firefox
* Add ask.com to the search engines.
.
You can uninstall this if you prefer to use a pristine Firefox install.
root at ip-10-6-85-206:/var/cache/apt/archives/partial# dpkg -I python-aptdaemon_0.43+bzr769-0ubuntu1_all.deb
new debian package, version 2.0.
size 15142 bytes: control archive= 1508 bytes.
68 bytes, 2 lines conffiles
1390 bytes, 30 lines control
1088 bytes, 15 lines md5sums
Package: aptdaemon
Version: 0.43+bzr769-0ubuntu1
Architecture: all
Maintainer: Ubuntu Developers <ubuntu-devel-discuss at lists.ubuntu.com>
Installed-Size: 188
Depends: python, python-aptdaemon (= 0.43+bzr769-0ubuntu1), python-gi, gir1.2-glib-2.0
Breaks: software-center (<< 1.1.21)
Section: admin
Priority: extra
Homepage: https://launchpad.net/aptdaemon
Description: transaction based package management service
Aptdaemon allows normal users to perform package management tasks, e.g.
refreshing the cache, upgrading the system, installing or removing software
packages.
.
Currently it comes with the following main features:
.
- Programming language independent D-Bus interface, which allows one to
write clients in several languages
- Runs only if required (D-Bus activation)
- Fine grained privilege management using PolicyKit, e.g. allowing all
desktop user to query for updates without entering a password
- Support for media changes during installation from DVD/CDROM
- Support for debconf (Debian's package configuration system)
- Support for attaching a terminal to the underlying dpkg call
.
This package contains the aptd script and all the data files required to run
the daemon. Moreover it contains the aptdcon script, which is a command
line client for aptdaemon. The API is not stable yet.
Original-Maintainer: Julian Andres Klode <jak at debian.org>
ProblemType: Bug
DistroRelease: Ubuntu 12.04
Package: apt 0.8.16~exp12ubuntu4
ProcVersionSignature: Ubuntu 3.2.0-18.28-virtual 3.2.9
Uname: Linux 3.2.0-18-virtual x86_64
ApportVersion: 1.94-0ubuntu1
Architecture: amd64
Date: Tue Mar 6 21:21:17 2012
Ec2AMI: ami-0400dd6d
Ec2AMIManifest: (unknown)
Ec2AvailabilityZone: us-east-1c
Ec2InstanceType: m1.large
Ec2Kernel: aki-825ea7eb
Ec2Ramdisk: unavailable
SourcePackage: apt
UpgradeStatus: No upgrade log present (probably fresh install)
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/apt/+bug/948461/+subscriptions
More information about the foundations-bugs
mailing list