OpenSSL 1.1.1 SRU into Bionic

[Dimitri John Ledkov]

18.04 LTS shipped with OpenSSL 1.1.0 as the default version of
OpenSSL. This version is not declared LTS by upstream and does not
have support for TLS v1.3. Given how long 18.04 will need to have
security support for, it is desirable to upgrade OpenSSL to 1.1.1 and
also gain TLS v1.3 functionality which will be increasingly desired.

A while back I have filed
https://bugs.launchpad.net/ubuntu/+source/openssl/+bug/1797386 SRU
bug, and started to prepare the necessary updates in a bileto PPA.

In addition to upgrading OpenSSL it also resolves a number of FTBFS
and test-suite / autopkgtest issues in a few related packages
(pythons, ruby, perl, R). This is to ensure there are no regressions
as part of landing OpenSSL and such that FTBFS are not introduced in
the archive by this update.

The bileto PPA will also be used as part of the upcoming bionic archive rebuild.

The debdiffs for all the packages involved are listed on the bileto
overview page at: https://bileto.ubuntu.com/#/ticket/3473

In addition to the required updates, I have also added no-change
rebuilds of python-defaults, python3-defaults and ruby-defaults, to
trigger autopkgtests runs with new pythons/ruby and the new openssl.
This is for information / regression spotting purposes only, and will
take a while to complete (more than 2k tests got triggered).

In general, the new OpenSSL is ABI and API compatible with the OpenSSL
shipped in bionic. There are only minor runtime differences involved
when TLS v1.3 is available (handshake, algos, sessions, SNI
enforcement are different) Majority of runtimes are unaffected by
these changes. There are small changes needed, for example setting
hostname for SNI verification (which used to be optional but now is
enforced). And sessions in TLSv1.3 are established asynchronously
post-handshake.

So far, no significant connectivity issues have been reported neither
with the proposed bileto ppa, or with the Cosmic release which has
shipped with OpenSSL 1.1.1.

These updates will not bring TLSv1.3 support in Apache2 nor make
openssh use libcrypto 1.1, however, both of these items are highly
requested as well, and will be part of future SRUs after OpenSSL 1.1.1
SRU lands.

Autopkgtest requests for these updates look good, however, there are a
number of false regressions identified. I hope to work with the SRU
team to hint those over, as false negatives. Please see
https://bileto.ubuntu.com/excuses/3473/bionic.html

I also hope that the diffs attached in the bileto PPA are good enough
for the SRU team to start reviewing them. If desired, some of these
can be split from this large SRU and uploaded individually (e.g.
things like python-boto, isync and similar)

This is a general call for testing of the proposed updates from this
PPA https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3473

--
Regards,

Dimitri.