[Bug 1807023] Re: installer stock images fail to validate any HTTPS certificates (ca-certificates missing)

Fri Dec 7 15:30:12 UTC 2018

Testing :: Xenial
=================

original
--------

wget http://archive.ubuntu.com/ubuntu/dists/xenial-updates/main
/installer-amd64/20101020ubuntu451.26/images/netboot/ubuntu-
installer/amd64/{linux,initrd.gz}

~ # cat /etc/default-release 
xenial

~ # ls -d /usr/lib/ssl/certs
ls: /usr/lib/ssl/certs: No such file or directory

~ # wget http://github.com
...
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://github.com/ [following]
...
ERROR: cannot verify github.com's certificate, issued by 'CN=DigiCert SHA2 Extended Validation Server CA,OU=www.digicert.com,O=DigiCert Inc,C=US':
  Unable to locally verify the issuer's authority.
To connect to github.com insecurely, use `--no-check-certificate'.
~ # 

modified
--------

wget http://ppa.launchpad.net/mfo/sf205192/ubuntu/dists/xenial/main
/installer-amd64/20101020ubuntu451.27/images/netboot/ubuntu-
installer/amd64/{linux,initrd.gz}

~ # cat /etc/default-release
xenial

~ # ls -d /usr/lib/ssl/certs
/usr/lib/ssl/certs

~ # wget http://github.com
...
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://github.com/ [following]
...
Saving to: 'index.html'

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to debian-installer in Ubuntu.
https://bugs.launchpad.net/bugs/1807023

Title:
  installer stock images fail to validate any HTTPS certificates (ca-
  certificates missing)

Status in debian-installer:
  Unknown
Status in ca-certificates package in Ubuntu:
  In Progress
Status in debian-installer package in Ubuntu:
  In Progress
Status in ca-certificates source package in Trusty:
  New
Status in debian-installer source package in Trusty:
  New
Status in ca-certificates source package in Xenial:
  New
Status in debian-installer source package in Xenial:
  New
Status in ca-certificates source package in Bionic:
  In Progress
Status in debian-installer source package in Bionic:
  In Progress
Status in ca-certificates source package in Cosmic:
  In Progress
Status in debian-installer source package in Cosmic:
  In Progress
Status in ca-certificates source package in Disco:
  In Progress
Status in debian-installer source package in Disco:
  In Progress
Status in debian-installer package in Debian:
  Fix Released

Bug description:
  [Impact]

   * The installer stock images fail to validate any HTTPS
     certificates because ca-certificates is not available
     in the installer environment.

   * This causes wget/download errors for preseed files on
     HTTPS servers (or HTTP servers that redirect to HTTPS,
     which are increasingly common nowadays - e.g., GitHub)
     and theoretically any other files that are downloaded
     with d-i-utils/fetch-url/wget.

   * The fix is to ship ca-certificates-udeb in installer
     stock images.

   * Debian already ships ca-certificate-udeb in the stock
     installer images; the fix is applied since Jan 2017.
     (reference: Debian Bug #842040 / d-i commit 2f00c51a [1])

  [Test Case]

   * In the installer shell:

     ~ # wget http://github.com  # or https://github.com

     - FAIL if ca-certificates-udeb is missing:
       "ERROR: cannot verify github.com's certificate, <...>'

     - PASS if ca-certificates-udeb is available
       "Saving to: 'index.html'"

   * Test steps with virt-install and netboot images
     are provided in the comments, for each release.

  [Regression Potential]

   * Low. This just adds the ca-certificates files in
     /etc/ssl/certs and symlink in /usr/lib/ssl/certs,
     so only tools looking for that would be affected.

   * Apparently only wget checks for/uses those files,
     and the difference in behavior is download errors
     no longer occur.

  [Notes]

   * The ca-certificates-udeb is not currently present
     in the Ubuntu archive despite being available for
     download in Launchpad with a link for some reason
     (perhaps a problem during import from Debian/sid?)

   * So this fix includes a no-change-rebuild for the
     ca-certificates package, in order to publish the
     udeb in the archive.

   * The ca-certificates and debian-installer builds
     have been done in a PPA using all architectures,
     and testing has been done with the amd64 images.

   * This fix is request for Bionic, Cosmic, Disco.

     The older releases (Trusty, Xenial) are affected,
     but not requested for, and would need more work,
     as the udeb is not yet in the packaging but that
     is doable if required for the process.

  [1] https://salsa.debian.org/installer-team/debian-
  installer/commit/2f00c51a7ead982ae1cd71bee06c8416890196b6

  [Debugging]

  For debugging purposes, one can install strace-udeb in the installer
  to verify wget's stat() calls to /usr/lib/ssl/certs.

  ~ # anna-install strace-udeb

  ~ # strace -e stat wget -O- https://github.com >/dev/null
  ...
  Resolving github.com... stat("/etc/resolv.conf", {st_mode=S_IFREG|0644, st_size=20, ...}) = 0
  140.82.118.3, 140.82.118.4
  Connecting to github.com|140.82.118.3|:443... connected.
  stat("/usr/lib/ssl/certs/45bfefc3.0", 0x7ffdba51b570) = -1 ENOENT (No such file or directory)
  stat("/usr/lib/ssl/certs/244b5494.0", 0x7ffdba51b570) = -1 ENOENT (No such file or directory)
  stat("/usr/lib/ssl/certs/244b5494.0", 0x7ffdba51b570) = -1 ENOENT (No such file or directory)
  ERROR: cannot verify github.com's certificate, issued by 'CN=DigiCert SHA2 Extended Validation Server CA,OU=www.digicert.com,O=DigiCert Inc,C=US':
    Unable to locally verify the issuer's authority.
  To connect to github.com insecurely, use `--no-check-certificate'.
  +++ exited with 5 +++
  ~ #

  ~ # anna-install ca-certificates-udeb  # not in archive yet.
  unknown udeb ca-certificates-udeb

  ~ # wget --no-check-certificate
  https://launchpad.net/ubuntu/+archive/primary/+files/ca-certificates-
  udeb_20180409_all.udeb

  ~ # udpkg -i ca-certificates-udeb_20180409_all.udeb

  ~ # strace -e stat wget -O- https://github.com >/dev/null
  ...
  Resolving github.com... stat("/etc/resolv.conf", {st_mode=S_IFREG|0644, st_size=20, ...}) = 0
  140.82.118.3, 140.82.118.4
  Connecting to github.com|140.82.118.3|:443... connected.
  stat("/usr/lib/ssl/certs/45bfefc3.0", 0x7fffbb9431c0) = -1 ENOENT (No such file or directory)
  stat("/usr/lib/ssl/certs/244b5494.0", {st_mode=S_IFREG|0644, st_size=1367, ...}) = 0
  stat("/usr/lib/ssl/certs/244b5494.1", 0x7fffbb9431c0) = -1 ENOENT (No such file or directory)
  HTTP request sent, awaiting response... 200 OK
  stat("-", 0x7fffbb943558)               = -1 ENOENT (No such file or directory)
  Length: unspecified [text/html]
  Saving to: 'STDOUT'
  ...
  +++ exited with 0 +++

To manage notifications about this bug go to:
https://bugs.launchpad.net/debian-installer/+bug/1807023/+subscriptions