[Bug 948461] Re: apt-get hashsum/size mismatch due caused by swapped local file names
Scott Moser
smoser at ubuntu.com
Wed Mar 7 04:28:39 UTC 2012
Some more data, after discussing with slangasek.
The attached script will run seemingly forever (100 times run) as it is, however if we remove the flag:
-o=Acquire::http::Pipeline-Depth=0
to apt, then it will generally fail < 10 times.
(data collected on m1.large "right now", with mirrors set to http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com/ubuntu/).
It would seem to me that the Pipeline-Depth is the source of the issue.
See [1] for apt.conf description, and [2] for a quick google of 's3'
'http' 'pipeline' that gives some evidence that Pipeline and S3 aren't
friends.
--
[1] http://manpages.ubuntu.com/manpages/precise/en/man5/apt.conf.5.html
[2] http://stackoverflow.com/questions/7752802/does-s3-support-http-pipeling
** Attachment added: "apt-get download loop"
https://bugs.launchpad.net/ubuntu/+source/apt/+bug/948461/+attachment/2827247/+files/go-apt-loop
--
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to apt in Ubuntu.
https://bugs.launchpad.net/bugs/948461
Title:
apt-get hashsum/size mismatch due caused by swapped local file names
Status in “apt” package in Ubuntu:
Confirmed
Status in “apt” source package in Lucid:
New
Status in “apt” source package in Maverick:
New
Status in “apt” source package in Natty:
New
Status in “apt” source package in Oneiric:
New
Status in “apt” source package in Precise:
Confirmed
Status in “apt” source package in Hardy:
New
Bug description:
apt-get is appears to be mangling the local caching file names, which
is appearing as a hashsum or size mismatch.
Evidence below:
_______________________________________________________________________________________
Output of "apt-get -y install firefox":
Failed to fetch http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com/ubuntu/pool/main/p/python-defer/python-defer_1.0.2+bzr481-1_all.deb Size mismatch
Failed to fetch http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com/ubuntu/pool/main/a/aptdaemon/python-aptdaemon_0.43+bzr769-0ubuntu1_all.deb Size mismatch
Failed to fetch http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com/ubuntu/pool/main/a/aptdaemon/aptdaemon_0.43+bzr769-0ubuntu1_all.deb Size mismatch
Failed to fetch http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com/ubuntu/pool/main/u/ubufox/xul-ext-ubufox_2.0-0ubuntu1_all.deb Size mismatch
_______________________________________________________________________________________
apt-get -y install firefox with debug for HTTP turned on:
Get:63 http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com/ubuntu/ precise/main python-defer all 1.0.2+bzr481-1 [10.9 kB]
HTTP/1.1 200 OK^M
x-amz-id-2: EXccd6GVMfE6ly4SYdwy313VK42d/gI3ncyxmaotuVIMbBBi6FJkuDYWrzLw7vXE^M
x-amz-request-id: 7572BAB5E6FFFAC0^M
Date: Tue, 06 Mar 2012 20:06:13 GMT^M
Last-Modified: Fri, 03 Feb 2012 08:54:27 GMT^M
ETag: "6c07f8db615cc32657b1cc57450a1608"^M
Accept-Ranges: bytes^M
Content-Type: application/x-debian-package^M
Content-Length: 15142^M
Server: AmazonS3^M
^M
Get:64 http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com/ubuntu/ precise/main python-aptdaemon all 0.43+bzr769-0ubuntu1 [80.3 kB]
HTTP/1.1 200 OK^M
x-amz-id-2: IcNECY1OtzHOXnTnTW3fqoJGyfdODq+qRXXGYTDEOJxfw0CP9UH8EUSllVR3Gf3F^M
x-amz-request-id: 5DEEEE629E3BA012^M
Date: Tue, 06 Mar 2012 20:06:13 GMT^M
Last-Modified: Fri, 02 Mar 2012 20:55:40 GMT^M
ETag: "501373631a241847cf76d13aea9264b3"^M
Accept-Ranges: bytes^M
Content-Type: application/x-debian-package^M
Content-Length: 56824^M
Server: AmazonS3^M
^M
Get:65 http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com/ubuntu/ precise/main aptdaemon all 0.43+bzr769-0ubuntu1 [15.1 kB]
HTTP/1.1 200 OK^M
x-amz-id-2: HHNSOZKH74D4I1XCaOiwDOzIWxIVJ88ibfKxvIp/4NpkczzPmsOhMqxrTj2T02qO^M
x-amz-request-id: A6FD3143E6A0BA1A^M
Date: Tue, 06 Mar 2012 20:06:34 GMT^M
Last-Modified: Sat, 21 Jan 2012 00:48:55 GMT^M
ETag: "3653165af1f20a437a14632ce0a2e6c2"^M
Accept-Ranges: bytes^M
Content-Type: application/x-debian-package^M
Content-Length: 10902^M
Server: AmazonS3^M
_______________________________________________________________________________________
Evaluation of the MD5sums of downloaded failed items:
root at ip-10-6-85-206:/var/cache/apt/archives/partial# md5sum *
501373631a241847cf76d13aea9264b3 aptdaemon_0.43+bzr769-0ubuntu1_all.deb
6c07f8db615cc32657b1cc57450a1608 python-aptdaemon_0.43+bzr769-0ubuntu1_all.deb
566f7c92a9fcd0c5c40d08a977176291 python-defer_1.0.2+bzr481-1_all.deb
3653165af1f20a437a14632ce0a2e6c2 xul-ext-ubufox_2.0-0ubuntu1_all.deb
_______________________________________________________________________________________
Snipets from Packages.bz2
From Meta-data:
Package: aptdaemon
MD5sum: 6c07f8db615cc32657b1cc57450a1608
Package: xul-ext-ubufox
MD5sum: 501373631a241847cf76d13aea9264b3
Package: python-aptdaemon
MD5sum: 566f7c92a9fcd0c5c40d08a977176291
Package: python-defer
MD5sum: 3653165af1f20a437a14632ce0a2e6c2
_______________________________________________________________________________________
"dpkg -i" on pacakges downloaded
root at ip-10-6-85-206:/var/cache/apt/archives/partial# dpkg -I aptdaemon_0.43+bzr769-0ubuntu1_all.deb
new debian package, version 2.0.
size 56824 bytes: control archive= 1520 bytes.
23 bytes, 1 lines conffiles
1023 bytes, 29 lines control
1804 bytes, 21 lines md5sums
Package: xul-ext-ubufox
Source: ubufox
Version: 2.0-0ubuntu1
Architecture: all
Maintainer: Ubuntu Mozilla Team <ubuntu-mozillateam at lists.ubuntu.com>
Installed-Size: 383
Depends: aptdaemon, libglib2.0-0 (>= 2.26)
Recommends: firefox (>= 4.0~b6)
Enhances: firefox
Breaks: ubufox (<< 0.9~rc2-0ubuntu3)a
Replaces: ubufox (<< 0.9~rc2-0ubuntu3)
Provides: firefox-ubufox, ubufox
Section: web
Priority: optional
Homepage: https://launchpad.net/ubufox
Description: Ubuntu-specific configuration defaults and apt support for Firefox
Adds Ubuntu-specific modifications to Firefox.
.
Integrates the browser with Ubuntu to:
* Enable searching for missing plugins from Ubuntu software catalog
* Add the following options to the Help menu
- Get help on-line
- Help translating Firefox
- Ubuntu Release Notes
* Set homepage to Ubuntu Start Page
* Display a restart notification after upgrading Firefox
* Add ask.com to the search engines.
.
You can uninstall this if you prefer to use a pristine Firefox install.
root at ip-10-6-85-206:/var/cache/apt/archives/partial# dpkg -I python-aptdaemon_0.43+bzr769-0ubuntu1_all.deb
new debian package, version 2.0.
size 15142 bytes: control archive= 1508 bytes.
68 bytes, 2 lines conffiles
1390 bytes, 30 lines control
1088 bytes, 15 lines md5sums
Package: aptdaemon
Version: 0.43+bzr769-0ubuntu1
Architecture: all
Maintainer: Ubuntu Developers <ubuntu-devel-discuss at lists.ubuntu.com>
Installed-Size: 188
Depends: python, python-aptdaemon (= 0.43+bzr769-0ubuntu1), python-gi, gir1.2-glib-2.0
Breaks: software-center (<< 1.1.21)
Section: admin
Priority: extra
Homepage: https://launchpad.net/aptdaemon
Description: transaction based package management service
Aptdaemon allows normal users to perform package management tasks, e.g.
refreshing the cache, upgrading the system, installing or removing software
packages.
.
Currently it comes with the following main features:
.
- Programming language independent D-Bus interface, which allows one to
write clients in several languages
- Runs only if required (D-Bus activation)
- Fine grained privilege management using PolicyKit, e.g. allowing all
desktop user to query for updates without entering a password
- Support for media changes during installation from DVD/CDROM
- Support for debconf (Debian's package configuration system)
- Support for attaching a terminal to the underlying dpkg call
.
This package contains the aptd script and all the data files required to run
the daemon. Moreover it contains the aptdcon script, which is a command
line client for aptdaemon. The API is not stable yet.
Original-Maintainer: Julian Andres Klode <jak at debian.org>
ProblemType: Bug
DistroRelease: Ubuntu 12.04
Package: apt 0.8.16~exp12ubuntu4
ProcVersionSignature: Ubuntu 3.2.0-18.28-virtual 3.2.9
Uname: Linux 3.2.0-18-virtual x86_64
ApportVersion: 1.94-0ubuntu1
Architecture: amd64
Date: Tue Mar 6 21:21:17 2012
Ec2AMI: ami-0400dd6d
Ec2AMIManifest: (unknown)
Ec2AvailabilityZone: us-east-1c
Ec2InstanceType: m1.large
Ec2Kernel: aki-825ea7eb
Ec2Ramdisk: unavailable
SourcePackage: apt
UpgradeStatus: No upgrade log present (probably fresh install)
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/apt/+bug/948461/+subscriptions
More information about the foundations-bugs
mailing list