[Bug 1250181] Re: download update from 16 to 20 failed on mako

[Barry Warsaw]

I think I figured this out, but I'll need confirmation from Manuel.
Here's my theory.

I flashed my manta to r15 on trusty-proposed, then initiated an update
via -cli.  The SignatureError is completely reproducible.  I recreated
the environment locally (on amd64 desktop) by copying over the
client.ini and channel.ini and staged up the expected directories.  I
put a breakpoint right before the SignatureError and initiated an update
locally, reproduced the SignatureError and hitting the breakpoint.

>From the pdb prompt, I got a listing of the tar.xz and tar.xz.asc files
that were failing the signature test and md5 checksummed them in their
destination location.  Then I got the source path on the server and
wget'd them to a different location.  Indeed the md5 checksums do *not*
match.  So clearly udm is somehow corrupting the files when it
downloaded them.

There's more, and here's where it gets interesting.  At the breakpoint I
did a len(downloads) and got a count of 30.  For some reason I then did
len(set(downloads)) and got a count of 28.  Well *that's* strange, but
what it tells me is that there were 2 duplicate downloads in the udm
group download request.  Although I haven't verified (the paths in the
index.json are quite difficult to read), I bet there is some image in
the 15->20 upgrade path that names the same data file.

So my hypothesis is this: when udm gets a group download request which
names a destination (i.e. local path) twice, it will dutifully download
the second request right on top of the first.  udm *doesn't* raise the
duplicate-file exception because the file did not exist before the group
download request.

If my theory is correct, then it should be fairly simple to filter out
any duplicate downloads before the udm group download request.  There
may be some corner cases I need to worry about (e.g. what order do files
in a duplicate download get applied in, but I think that will naturally
play itself out).

I've added a udm bugtask so that Manuel can verify and add additional
safeguards in udm.

Testing now...

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to system-image in Ubuntu.
https://bugs.launchpad.net/bugs/1250181

Title:
  download update from 16 to 20 failed on mako

Status in Ubuntu Download Manager:
  Confirmed
Status in Ubuntu system image (server/client/updater):
  Confirmed
Status in “system-image” package in Ubuntu:
  Confirmed

Bug description:
  Trying to OTA update from 16 to 20 on mako. See screenshot.

  root at ubuntu-phablet:/var/log/system-image# system-image-cli -i
  current build number: 16
  device name: mako
  channel: trusty-proposed
  last update: 2013-11-09 13:52:54
  version version: 16
  version ubuntu: 20131109
  version device: 20131109

  From /var/log/system-image/client.log

  [systemimage] Nov 11 18:24:00 2013 (8816) Running group download reactor
  [systemimage] Nov 11 18:25:20 2013 (8816) Group download reactor done
  [systemimage] Nov 11 18:25:23 2013 (8816) uncaught exception in state machine
  Traceback (most recent call last):
    File "/usr/lib/python3/dist-packages/systemimage/state.py", line 138, in run_until
      step()
    File "/usr/lib/python3/dist-packages/systemimage/state.py", line 453, in _download_files
      raise SignatureError(dst)
  systemimage.gpg.SignatureError: /android/cache/recovery/device-bc45abbc8b271608f690cc12f7e3dd6e3b6527fdd818a22ea1d2d204350dc56e.delta-device-f67595b49ad4c8233c0ad4645695dd2154242dc3e29b4a60a3afc39058b126a6.tar.xz

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu-download-manager/+bug/1250181/+subscriptions