[Bug 996151] Re: disable apt http pipelining in quantal

[Scott Moser]

David,
   Thank you for your well written comment above.

On Mon, 14 May 2012, David Kalnischkies wrote:

> As already said in the uds session (and after it) it's rather
> meaningless to perform a test on a single machine with a low-latency
> network connection. Obviously pipelining has only a benefit for the
> client if the connection is flaky/high-latency (like e.g. my phone). For

For some reason this point was lost on me.  You're right, the tests we've
done on EC2 to a lan-connected http server are not useful information.

I've tried a couple tests locally here (cable modem) with:
  sudo rm /var/lib/apt/lists/*;
  time sudo apt-get update -o Acquire::HTTP::Proxy=None \
     -o Acquire::http::Pipeline-Depth=5
and
  sudo rm /var/lib/apt/lists/*;
  time sudo apt-get update -o Acquire::HTTP::Proxy=None \
     -o Acquire::http::Pipeline-Depth=0

That didn't give me really good consistent data, though.
The most annoying issue was that apt would apparently hang on one
connection.  Ie, output would hang somewhere like:
  Get:101 http://us.archive.ubuntu.com precise-backports/universe Translation-en [696 B]
  100% [16 Packages 0 B/769 B 0%]

That single hang (apparent http get for a small-ish file, derails any
actual data).  I saw that both with both values of Pipeline-Depth.

> So we are back at square one: the web is a buggy mess. Lets just hope
> that Google will force the web once again (after they fixed there own
> repository to work with their own browser [reductio ad absurdum]) to be
> more standard conform and disable it until then by default as i don't
> have the energy to defend it like previous contributors did (which is
> the only real conclusion to be taken from the previously mentioned
> threads) and will just enable it on all my machines.

Do you have any thoughts on how we could collect enough data to show if it
is useful for our usecase?  I would think that my cable modem would
qualify as the target case for pipelineing (relatively high-bandwidth and
high-latency).


> (And now hands up, who imagined such an outcome after reading the
> previous four paragraphs? I just needed a reference to point people
> complaining about the new default to…)

I actually agree that it makes sense to have a safer default, and to allow
those interested to enable the more risky option.

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to apt in Ubuntu.
https://bugs.launchpad.net/bugs/996151

Title:
  disable apt http pipelining in quantal

Status in “apt” package in Ubuntu:
  Confirmed
Status in “apt” package in Debian:
  New

Bug description:
  Per UDS session on Apt improvements, it has been proposed to remove
  apt http pipelining

  The reasons:
  1. HTTP Pipelining has issue with certain proxy implementation
  2. Some new object stores, like S3, or Google's APT repositories have problems with HTTP Pipelining

  Running a test shows that disabling apt-pipelining has no perceptable
  diffferenvce, and disabling apt pipeling actually performed slightly
  better with an average of 31.899s versuses 32.456s. I tested an "apt-
  get -y update" with and without apt HTTP pipelining turned on.

  For more information on apt-pipelining, here are 2 threads to read:
   http://old.nabble.com/APT-do-not-work-with-Squid-as-a-proxy-because-of-pipelining-default-td28579596.html
   http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=565555

  
  Pipelining on (apt-get -y upgrade):
  33.92
  31.37
  31.64
  31.63
  33.29
  33.08
  32.92
  32.88
  31.73
  31.98
  32.01
  32.96
  31.51
  32.68
  33.25

  Pipelining off (apt-get -o Acquire::http::Pipeline-Depth="0" -y upgrade):
  31.66
  31.59
  31.24
  31.30
  31.29
  32.85
  32.75
  31.50
  31.18
  32.26
  31.43
  33.28
  31.67
  32.45
  32.04

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/apt/+bug/996151/+subscriptions