[Bug 1864623] [NEW] apt-get attempts to download Packages.xz which is not InRelease and does not exist

Tuure Laurinolli 1864623 at bugs.launchpad.net
Tue Feb 25 09:11:20 UTC 2020 

Public bug reported:

When trying to install the latest virtualbox-6.0 from virtualbox.org, we
got a failure:

Err:1 http://download.virtualbox.org/virtualbox/debian xenial/contrib amd64 virtualbox-6.0 amd64 6.0.16-135674~Ubuntu~xenial
  404  Not Found
E: Failed to fetch http://download.virtualbox.org/virtualbox/debian/pool/contrib/v/virtualbox-6.0/virtualbox-6.0_6.0.16-135674~Ubuntu~xenial_amd64.deb  404  Not Found


We expected that this was a case of `apt-get update` not having been run before `apt-get install`, but turns out it's a bit more interesting:

apt-get update fetched the following files:

    
ubuntu at ubuntu:~$ apt-get --print-uris update |grep virtualbox
'http://download.virtualbox.org/virtualbox/debian/dists/xenial/InRelease' download.virtualbox.org_virtualbox_debian_dists_xenial_InRelease 0 
'http://download.virtualbox.org/virtualbox/debian/dists/xenial/contrib/binary-amd64/Packages.xz' download.virtualbox.org_virtualbox_debian_dists_xenial_contrib_binary-amd64_Packages 0 
'http://download.virtualbox.org/virtualbox/debian/dists/xenial/contrib/binary-all/Packages.xz' download.virtualbox.org_virtualbox_debian_dists_xenial_contrib_binary-all_Packages 0 
'http://download.virtualbox.org/virtualbox/debian/dists/xenial/contrib/i18n/Translation-en.xz' download.virtualbox.org_virtualbox_debian_dists_xenial_contrib_i18n_Translation-en 0 

http://download.virtualbox.org/virtualbox/debian/dists/xenial/InRelease
returns 200 and lists a bunch of Packages lists. However, it does *not*
list contrib/binary-amd64/Packages.xz, only:

- contrib/binary-amd64/Packages
- contrib/binary-amd64/Packages.gz
- contrib/binary-amd64/Packages.bz2

For some reason, apt still attempts to download the .xz files, and does
not even report an error for them!

Trying to download the file locally, I get a 404, but with an HTML error
page as the content:

$ curl -v 'http://download.virtualbox.org/virtualbox/debian/dists/xenial/contrib/binary-amd64/Packages.xz' | head -n 20
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying 2.22.40.42...
* TCP_NODELAY set
* Connected to download.virtualbox.org (2.22.40.42) port 80 (#0)
> GET /virtualbox/debian/dists/xenial/contrib/binary-amd64/Packages.xz HTTP/1.1
> Host: download.virtualbox.org
> User-Agent: curl/7.58.0
> Accept: */*
> 
  0     0    0     0    0     0      0      0 --:--:--  0:00:02 --:--:--     0< HTTP/1.1 404 Not Found
< Accept-Ranges: bytes
< Content-Type: text/html
< ETag: "477d9862374177c810b0b7d73b4f1e53:1531792454.541174"
< Last-Modified: Wed, 31 Aug 2011 20:28:54 GMT
< Server: AkamaiNetStorage
< Content-Length: 7073
< Date: Tue, 25 Feb 2020 09:08:16 GMT
< Connection: keep-alive
< 
{ [7073 bytes data]
100  7<html>
<head>
<title>Error Page 404</title>
<base href="http://download.oracle.com/index.html" />
<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">
<link rel="stylesheet" type="text/css" href="/errors/us/assets/hp-styles.css" />
<link rel="stylesheet" type="text/css" href="/errors/us/assets/master-master.css" />


I think it's a bug that:
a) apt tries to download Packages files not listed in InRelease
b) apt does not report errors on 404 it gets when downloading them

It is of course possible that apt get a *different* InRelease and a
*different* response when trying to download the .xz files, but that
seems unlikely. If you want to, I can try to confirm the actual network
traffic.

** Affects: apt (Ubuntu)
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to apt in Ubuntu.
https://bugs.launchpad.net/bugs/1864623

Title:
  apt-get attempts to download Packages.xz which is not InRelease and
  does not exist

Status in apt package in Ubuntu:
  New

Bug description:
  When trying to install the latest virtualbox-6.0 from virtualbox.org,
  we got a failure:

  Err:1 http://download.virtualbox.org/virtualbox/debian xenial/contrib amd64 virtualbox-6.0 amd64 6.0.16-135674~Ubuntu~xenial
    404  Not Found
  E: Failed to fetch http://download.virtualbox.org/virtualbox/debian/pool/contrib/v/virtualbox-6.0/virtualbox-6.0_6.0.16-135674~Ubuntu~xenial_amd64.deb  404  Not Found

  
  We expected that this was a case of `apt-get update` not having been run before `apt-get install`, but turns out it's a bit more interesting:

  apt-get update fetched the following files:

      
  ubuntu at ubuntu:~$ apt-get --print-uris update |grep virtualbox
  'http://download.virtualbox.org/virtualbox/debian/dists/xenial/InRelease' download.virtualbox.org_virtualbox_debian_dists_xenial_InRelease 0 
  'http://download.virtualbox.org/virtualbox/debian/dists/xenial/contrib/binary-amd64/Packages.xz' download.virtualbox.org_virtualbox_debian_dists_xenial_contrib_binary-amd64_Packages 0 
  'http://download.virtualbox.org/virtualbox/debian/dists/xenial/contrib/binary-all/Packages.xz' download.virtualbox.org_virtualbox_debian_dists_xenial_contrib_binary-all_Packages 0 
  'http://download.virtualbox.org/virtualbox/debian/dists/xenial/contrib/i18n/Translation-en.xz' download.virtualbox.org_virtualbox_debian_dists_xenial_contrib_i18n_Translation-en 0 

  http://download.virtualbox.org/virtualbox/debian/dists/xenial/InRelease
  returns 200 and lists a bunch of Packages lists. However, it does
  *not* list contrib/binary-amd64/Packages.xz, only:

  - contrib/binary-amd64/Packages
  - contrib/binary-amd64/Packages.gz
  - contrib/binary-amd64/Packages.bz2

  For some reason, apt still attempts to download the .xz files, and
  does not even report an error for them!

  Trying to download the file locally, I get a 404, but with an HTML
  error page as the content:

  $ curl -v 'http://download.virtualbox.org/virtualbox/debian/dists/xenial/contrib/binary-amd64/Packages.xz' | head -n 20
    % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                   Dload  Upload   Total   Spent    Left  Speed
    0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying 2.22.40.42...
  * TCP_NODELAY set
  * Connected to download.virtualbox.org (2.22.40.42) port 80 (#0)
  > GET /virtualbox/debian/dists/xenial/contrib/binary-amd64/Packages.xz HTTP/1.1
  > Host: download.virtualbox.org
  > User-Agent: curl/7.58.0
  > Accept: */*
  > 
    0     0    0     0    0     0      0      0 --:--:--  0:00:02 --:--:--     0< HTTP/1.1 404 Not Found
  < Accept-Ranges: bytes
  < Content-Type: text/html
  < ETag: "477d9862374177c810b0b7d73b4f1e53:1531792454.541174"
  < Last-Modified: Wed, 31 Aug 2011 20:28:54 GMT
  < Server: AkamaiNetStorage
  < Content-Length: 7073
  < Date: Tue, 25 Feb 2020 09:08:16 GMT
  < Connection: keep-alive
  < 
  { [7073 bytes data]
  100  7<html>
  <head>
  <title>Error Page 404</title>
  <base href="http://download.oracle.com/index.html" />
  <META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">
  <link rel="stylesheet" type="text/css" href="/errors/us/assets/hp-styles.css" />
  <link rel="stylesheet" type="text/css" href="/errors/us/assets/master-master.css" />


  I think it's a bug that:
  a) apt tries to download Packages files not listed in InRelease
  b) apt does not report errors on 404 it gets when downloading them

  It is of course possible that apt get a *different* InRelease and a
  *different* response when trying to download the .xz files, but that
  seems unlikely. If you want to, I can try to confirm the actual
  network traffic.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/apt/+bug/1864623/+subscriptions

More information about the foundations-bugs mailing list