[Bug 1330621] [NEW] apt HTTPS connection reuse leading to 403 Forbidden against S3

[Phil Pennock]

Public bug reported:

Encountered with Trusty, apt package 1.0.1ubuntu2

This might be a consequence of
https://bugs.launchpad.net/ubuntu/+source/apt/+bug/1087543 enabling
HTTPS connection reuse.  This is not the same as
https://bugs.launchpad.net/ubuntu/+source/apt/+bug/1330619 which
pertains to + encoding in requests sent over HTTPS, seen at the same
time.

I saw failures with "apt-get update" against some repositories
configured as https, where those repositories are S3 backed.  I ran:

$ sudo apt-get -o Debug::Acquire::https=true update

I saw 403 Forbidden for some resources, but only when the connection was
being reused.  Please excuse the name mangling below; the repository is
open but intended for private use.

Get:9 https://censored.s3.amazonaws.com public/main Translation-en
72% [Waiting for headers] [9 Translation-en 0 B]* Found bundle for host censored.s3.amazonaws.com: 0x118c500
* Re-using existing connection! (#2) with host censored.s3.amazonaws.com
* Connected to censored.s3.amazonaws.com (176.32.101.8) port 443 (#2)
> GET /dists/public/main/i18n/Translation-en_US HTTP/1.1
User-Agent: Debian APT-CURL/1.0 (1.0.1ubuntu2)
Host: censored.s3.amazonaws.com
Cache-Control: max-age=0
Accept: text/*

Hit http://us.archive.ubuntu.com trusty-updates/main Translation-en
74% [Working]< HTTP/1.1 403 Forbidden
< x-amz-request-id: censored
< x-amz-id-2: censored
< Content-Type: application/xml
< Transfer-Encoding: chunked
< Date: Mon, 16 Jun 2014 18:51:03 GMT
* Server AmazonS3 is not blacklisted
< Server: AmazonS3

I do not see the same error with curl(1), so this appears to be
something specific to apt with the https acquire transport; took a while
to notice that the errors were all after connection reuse.  I could find
no tuning option to disable connection reuse.

** Affects: apt (Ubuntu)
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to apt in Ubuntu.
https://bugs.launchpad.net/bugs/1330621

Title:
  apt HTTPS connection reuse leading to 403 Forbidden against S3

Status in “apt” package in Ubuntu:
  New

Bug description:
  Encountered with Trusty, apt package 1.0.1ubuntu2

  This might be a consequence of
  https://bugs.launchpad.net/ubuntu/+source/apt/+bug/1087543 enabling
  HTTPS connection reuse.  This is not the same as
  https://bugs.launchpad.net/ubuntu/+source/apt/+bug/1330619 which
  pertains to + encoding in requests sent over HTTPS, seen at the same
  time.

  I saw failures with "apt-get update" against some repositories
  configured as https, where those repositories are S3 backed.  I ran:

  $ sudo apt-get -o Debug::Acquire::https=true update

  I saw 403 Forbidden for some resources, but only when the connection
  was being reused.  Please excuse the name mangling below; the
  repository is open but intended for private use.

  Get:9 https://censored.s3.amazonaws.com public/main Translation-en
  72% [Waiting for headers] [9 Translation-en 0 B]* Found bundle for host censored.s3.amazonaws.com: 0x118c500
  * Re-using existing connection! (#2) with host censored.s3.amazonaws.com
  * Connected to censored.s3.amazonaws.com (176.32.101.8) port 443 (#2)
  > GET /dists/public/main/i18n/Translation-en_US HTTP/1.1
  User-Agent: Debian APT-CURL/1.0 (1.0.1ubuntu2)
  Host: censored.s3.amazonaws.com
  Cache-Control: max-age=0
  Accept: text/*

  Hit http://us.archive.ubuntu.com trusty-updates/main Translation-en
  74% [Working]< HTTP/1.1 403 Forbidden
  < x-amz-request-id: censored
  < x-amz-id-2: censored
  < Content-Type: application/xml
  < Transfer-Encoding: chunked
  < Date: Mon, 16 Jun 2014 18:51:03 GMT
  * Server AmazonS3 is not blacklisted
  < Server: AmazonS3

  I do not see the same error with curl(1), so this appears to be
  something specific to apt with the https acquire transport; took a
  while to notice that the errors were all after connection reuse.  I
  could find no tuning option to disable connection reuse.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/apt/+bug/1330621/+subscriptions