[Bug 1803385] Re: fetch-url does not use --no-check-certificate on HTTP to HTTPS redirects

[Dan Streetman]

this was fixed a different way, in d-i and ca-certificates.

bug 1807023

** Changed in: debian-installer-utils (Ubuntu)
     Assignee: Mauricio Faria de Oliveira (mfo) => (unassigned)

** Changed in: debian-installer-utils (Ubuntu Trusty)
     Assignee: Mauricio Faria de Oliveira (mfo) => (unassigned)

** Changed in: debian-installer-utils (Ubuntu Xenial)
     Assignee: Mauricio Faria de Oliveira (mfo) => (unassigned)

** Changed in: debian-installer-utils (Ubuntu Cosmic)
     Assignee: Mauricio Faria de Oliveira (mfo) => (unassigned)

** Changed in: debian-installer-utils (Ubuntu Bionic)
     Assignee: Mauricio Faria de Oliveira (mfo) => (unassigned)

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to debian-installer-utils in
Ubuntu.
https://bugs.launchpad.net/bugs/1803385

Title:
  fetch-url does not use --no-check-certificate on HTTP to HTTPS
  redirects

Status in debian-installer-utils package in Ubuntu:
  Invalid
Status in debian-installer-utils source package in Trusty:
  Invalid
Status in debian-installer-utils source package in Xenial:
  Invalid
Status in debian-installer-utils source package in Bionic:
  Invalid
Status in debian-installer-utils source package in Cosmic:
  Invalid
Status in debian-installer-utils package in Debian:
  New

Bug description:
  [Impact]

   * fetch-url fails to download files from URL with HTTP to HTTPS
     redirect if server has invalid/cannot be verified certificate.

   * Install fails in case a preseed/other files use an HTTP URL
     that redirects to an HTTPS URL with an invalid certificate.

   * Servers/URLs that started using HTTP to HTTPS redirect and
     have their URLs already spread over scripts/infrastructure
     start to cause install/deployment failures.

   * This fix checks for debian-installer/allow_unauthenticated_ssl
     in the HTTP protocol as well (to enable --no-check-certificate),
     which is OK as that option must be explicitly enabled by users,
     indicating awareness of the SSL/HTTPS context and certificates
     that may not be verified.

  [Test Case]

   * Setup web-server with HTTP to HTTPS redirect and an invalid/
     self-signed certificate, and put a file (eg, preseed) on it.

   * Boot with preseed option 'url=http://<server>/preseed' and
     the install will fail in the 'network-preseed' stage, with
     syslog errors about invalid/cannot be verified certificates,
     suggesting the 'wget --no-check-certificate' option.

   * Other files downloaded by the installer can hit same error,
     if using HTTP URLs from that server.

   * In the installer shell, run:
     ~ # fetch-url http://<server>/<file>

  [Regression Potential]

   * Low risk of regression, this only expands the check from HTTPS-only
     to HTTPS or HTTP, to *then* check for d-i/allow_unauthenticated_ssl.

   * The theoretical case is that a HTTP URL with no redirect to HTTPS
     may use --no-check-certificate, thus without actually needing it,
     (it should not cause problems at all, the option should be ignored)
     but anyway, since the user acknowledged that sort of behavior with
     the d-i/allow_unauthenticated_ssl, that should not be a concern.

  [Other Info]
   
   * Debian Bug #913740.

  [Problem Description]

  In fetch-url the --no-check-certificate option is conditioned to HTTPS.
  In case of HTTP to HTTPS redirect, that option is not enabled, and may
  cause fetch-url to fail if the certificate cannot be verified.

  Since that option/functionality must be explicitly requested with the
  debian-installer/allow_unauthenticated_ssl preseed option (i.e., user
  is aware of SSL/HTTPS context and agrees w/ non-verified certificates)
  we can just check for this in the HTTP protocol too, and assume HTTPS
  may potentially be used, as the user specified this option.

  An alternative/obvious solution in the _user_ side is to specify HTTPS
  URLs upfront, but there are cases when an user does not know for sure
  whether the server uses/supports that, or the server might change its
  behavior and start HTTP to HTTPS redirect after URLs have spread over
  (e.g., scripts and infrastructure) - thus a fix in the installer side
  is a simpler and more complete approach.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/debian-installer-utils/+bug/1803385/+subscriptions