[Bug 1067934] Re: spends 10+ minutes deduplicating Package lists

[Stuart Longland]

Hi, Apologies for the delay.  I've been on other projects and only just
got back to this one.

We're using the files from
http://au.archive.ubuntu.com/pub/ubuntu/archive/dists/precise-
proposed/main/installer-amd64/current/images/netboot/netboot.tar.gz

I'm downloading the latest copy of that now, and will give it another
shot.  I'll let you know if this fixes the problem.

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to net-retriever in Ubuntu.
https://bugs.launchpad.net/bugs/1067934

Title:
  spends 10+ minutes deduplicating Package lists

Status in “net-retriever” package in Ubuntu:
  Fix Released
Status in “net-retriever” source package in Lucid:
  Fix Released
Status in “net-retriever” source package in Precise:
  Fix Released
Status in “net-retriever” source package in Quantal:
  Won't Fix
Status in “net-retriever” source package in Raring:
  Won't Fix
Status in “net-retriever” source package in Saucy:
  Fix Released

Bug description:
  [Impact] The netboot installer is unreasonably slow to deduplicate udeb Packages files retrieved from the network, scaling (at least) quadratically in the number of packages with a bad constant factor.
  [Test Case] Run a netboot installer built with the new net-retriever (we'll put one in -proposed at some point after this SRU is reviewed and accepted, and post the URLs to this bug).  It should work and should not take unreasonable time around the "Download installer components" stage.
  [Regression Potential] Cannot affect anything other than netboot installations.  I've compared the output against the old code for a sample Packages file provided by Michael and made sure it's identical; but if this goes wrong then the most likely result is a partitioning failure, or perhaps a complaint that no suitable kernel versions are available (although this can happen for other reasons, e.g. forgetting "apt-setup/proposed=true" for an image built against the kernel in -proposed).

  Original report follows;

  At some point after Lucid's initial release, the ubuntu alternate
  installer seems to stall for a very long time (10+ minutes) before the
  partitioning menu comes up. I don't think it was happening for the
  first year or so after Lucid's release, but I'm not real clear on when
  it started happening (it was only recently brought to my attention).

  I think it is happening in /usr/lib/debian-installer/net-retriever's deduplicate function...
  When it is stalled out, it keeps adding content to /tmp/net-retriever-NNNN-deduplicate/, eventually it moves on, but it takes forever. The speed of the install machine doesn't appear to matter much.
  There's nothing obvious in the log as to why it's stalling at that point...

  I thought it might be some issue with our preseeding, but it appears
  to happen on an install without preseeding as well.

  Newer versions of Ubuntu (at least Xubuntu 12.04) don't seem to
  experience this problem, as far as I can tell.

  We're using the netboot installer, apparently last updated on
  2011-04-29:

  2011-04-29 03:32:27-07:00 checking for updates: http://us.archive.ubuntu.com/ubuntu/dists/lucid/main/installer-i386/current/images/netboot/netboot.tar.gz
  2011-04-29 03:33:13-07:00 updated: /var/lib/tftpboot/ubuntu-installer/lucid.i386.netboot.tar.gz

  though as far as i can tell, that file hasn't been updated on the
  mirrors since 2010-04-25.

  live well,
    vagrant

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/net-retriever/+bug/1067934/+subscriptions