[Bug 1803385] Re: fetch-url does not use --no-check-certificate on HTTP to HTTPS redirects
Mauricio Faria de Oliveira
mfo at canonical.com
Wed Nov 14 15:17:29 UTC 2018
Testing performed with Disco, Cosmic, Bionic, Xenial, and Trusty.
The output is similar if not identical, so pasting just one test, from
Disco.
(Web Server, HTTP/HTTPS redirect, setup not included, ping me if
interested.)
$ wget http://archive.ubuntu.com/ubuntu/dists/disco/main/installer-
amd64/current/images/netboot/ubuntu-installer/amd64/{linux,initrd.gz}
$ GUEST=disco
$ virt-install \
--name $GUEST \
--vcpus 2 \
--memory 1024 \
--disk $GUEST.qcow2,bus=virtio,format=qcow2,size=8 \
--network bridge=virbr0,model=virtio \
--graphics none \
--import \
--boot \
kernel=linux,\
initrd=initrd.gz,\
kernel_args='console=ttyS0 url=http://192.168.122.1/preseed debian-installer/allow_unauthenticated_ssl=true'
The installer hits an error when trying to get the preseed file:
┌──────────┤ [!!] Download debconf preconfiguration file ├──────────┐
│ │
│ Failed to retrieve the preconfiguration file │
│ The file needed for preconfiguration could not be retrieved from │
│ http://192.168.122.1/preseed. The installation will proceed in │
│ non-automated mode. │
│ │
│ <Continue> │
│ │
└───────────────────────────────────────────────────────────────────┘
The synthetic tests with fetch-url:
===
~ # cat /proc/cmdline
console=ttyS0 url=http://192.168.122.1/preseed debian-installer/allow_unauthenticated_ssl=true
~ # cat /etc/default-release
disco
Without patch:
---
~ # fetch-url http://192.168.122.1/preseed preseed
ERROR: cannot verify 192.168.122.1's certificate, ...
...
To connect to 192.168.122.1 insecurely, use `--no-check-certificate'.
~ # echo $?
1
With patch:
---
~ # wget --no-check-certificate http://192.168.122.1/di-utils_1.124ubuntu2_amd64.udeb
~ # udpkg -i di-utils_1.124ubuntu2_amd64.udeb
~ # fetch-url http://192.168.122.1/preseed preseed
WARNING: cannot verify 192.168.122.1's certificate, ...
...
2018-11-14 13:17:03 URL:https://192.168.122.1//preseed [11/11] -> "./_fetch-url_preseed.1467" [1]
~ # echo $?
0
With patch and Without d-i/allow_unauthenticated_ssl=true: No Change
---
~ # cat /proc/cmdline
console=ttyS0 url=http://192.168.122.1/preseed
~ # wget --no-check-certificate http://192.168.122.1/di-utils_1.124ubuntu2_amd64.udeb
~ # udpkg -i di-utils_1.124ubuntu2_amd64.udeb
~ # fetch-url http://192.168.122.1/preseed preseed
ERROR: cannot verify 192.168.122.1's certificate, ...
...
To connect to 192.168.122.1 insecurely, use `--no-check-certificate'.
~ # echo $?
1
** Bug watch added: Debian Bug tracker #913740
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=913740
** Also affects: debian-installer-utils (Debian) via
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=913740
Importance: Unknown
Status: Unknown
--
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to debian-installer-utils in
Ubuntu.
https://bugs.launchpad.net/bugs/1803385
Title:
fetch-url does not use --no-check-certificate on HTTP to HTTPS
redirects
Status in debian-installer-utils package in Ubuntu:
Confirmed
Status in debian-installer-utils package in Debian:
Unknown
Bug description:
[Impact]
* fetch-url fails to download files from URL with HTTP to HTTPS
redirect if server has invalid/cannot be verified certificate.
* Install fails in case a preseed/other files use an HTTP URL
that redirects to an HTTPS URL with an invalid certificate.
* Servers/URLs that started using HTTP to HTTPS redirect and
have their URLs already spread over scripts/infrastructure
start to cause install/deployment failures.
* This fix checks for debian-installer/allow_unauthenticated_ssl
in the HTTP protocol as well (to enable --no-check-certificate),
which is OK as that option must be explicitly enabled by users,
indicating awareness of the SSL/HTTPS context and certificates
that may not be verified.
[Test Case]
* Setup web-server with HTTP to HTTPS redirect and an invalid/
self-signed certificate, and put a file (eg, preseed) on it.
* Boot with preseed option 'url=http://<server>/preseed' and
the install will fail in the 'network-preseed' stage, with
syslog errors about invalid/cannot be verified certificates,
suggesting the 'wget --no-check-certificate' option.
* Other files downloaded by the installer can hit same error,
if using HTTP URLs from that server.
* In the installer shell, run:
~ # fetch-url http://<server>/<file>
[Regression Potential]
* Low risk of regression, this only expands the check from HTTPS-only
to HTTPS or HTTP, to *then* check for d-i/allow_unauthenticated_ssl.
* The theoretical case is that a HTTP URL with no redirect to HTTPS
may use --no-check-certificate, thus without actually needing it,
(it should not cause problems at all, the option should be ignored)
but anyway, since the user acknowledged that sort of behavior with
the d-i/allow_unauthenticated_ssl, that should not be a concern.
[Other Info]
* Debian Bug #913740.
[Problem Description]
In fetch-url the --no-check-certificate option is conditioned to HTTPS.
In case of HTTP to HTTPS redirect, that option is not enabled, and may
cause fetch-url to fail if the certificate cannot be verified.
Since that option/functionality must be explicitly requested with the
debian-installer/allow_unauthenticated_ssl preseed option (i.e., user
is aware of SSL/HTTPS context and agrees w/ non-verified certificates)
we can just check for this in the HTTP protocol too, and assume HTTPS
may potentially be used, as the user specified this option.
An alternative/obvious solution in the _user_ side is to specify HTTPS
URLs upfront, but there are cases when an user does not know for sure
whether the server uses/supports that, or the server might change its
behavior and start HTTP to HTTPS redirect after URLs have spread over
(e.g., scripts and infrastructure) - thus a fix in the installer side
is a simpler and more complete approach.
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/debian-installer-utils/+bug/1803385/+subscriptions
More information about the foundations-bugs
mailing list