[Bug 1906720] Re: Fix the disable_ssl_certificate_validation option

Wed Jan 20 07:57:22 UTC 2021

Please add a regression analysis as required by
https://wiki.ubuntu.com/StableReleaseUpdates#Procedure.

In particular, please take some steps here to make sure that we don't
accidentally disable certificate validation across the board - since
that would have severe consequences, we're messing with "should we check
the certificate" code, and the problem wouldn't be detected just by
checking this bug is fixed.

That's the most obvious possible issue to me, but please consider and
add anything else relevant.

** Changed in: python-httplib2 (Ubuntu Bionic)
       Status: In Progress => Incomplete

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to python-httplib2 in Ubuntu.
https://bugs.launchpad.net/bugs/1906720

Title:
  Fix the disable_ssl_certificate_validation option

Status in python-httplib2 package in Ubuntu:
  Fix Released
Status in python-httplib2 source package in Bionic:
  Incomplete
Status in python-httplib2 source package in Focal:
  Fix Released
Status in python-httplib2 source package in Groovy:
  Fix Released
Status in python-httplib2 source package in Hirsute:
  Fix Released

Bug description:
  [Environment]

  Bionic
  python3-httplib2 | 0.9.2+dfsg-1ubuntu0.2

  [Description]

  maas cli fails to work with apis over https with self-signed certificates due to the lack
  of disable_ssl_certificate_validation option with python 3.5.

  [Distribution/Release, Package versions, Platform]
  cat /etc/lsb-release; dpkg -l | grep maas
  DISTRIB_ID=Ubuntu
  DISTRIB_RELEASE=18.04
  DISTRIB_CODENAME=bionic
  DISTRIB_DESCRIPTION="Ubuntu 18.04.5 LTS"
  ii maas 2.8.2-8577-g.a3e674063-0ubuntu1~18.04.1 all "Metal as a Service" is a physical cloud and IPAM
  ii maas-cli 2.8.2-8577-g.a3e674063-0ubuntu1~18.04.1 all MAAS client and command-line interface
  ii maas-common 2.8.2-8577-g.a3e674063-0ubuntu1~18.04.1 all MAAS server common files
  ii maas-dhcp 2.8.2-8577-g.a3e674063-0ubuntu1~18.04.1 all MAAS DHCP server
  ii maas-proxy 2.8.2-8577-g.a3e674063-0ubuntu1~18.04.1 all MAAS Caching Proxy
  ii maas-rack-controller 2.8.2-8577-g.a3e674063-0ubuntu1~18.04.1 all Rack Controller for MAAS
  ii maas-region-api 2.8.2-8577-g.a3e674063-0ubuntu1~18.04.1 all Region controller API service for MAAS
  ii maas-region-controller 2.8.2-8577-g.a3e674063-0ubuntu1~18.04.1 all Region Controller for MAAS
  ii python3-django-maas 2.8.2-8577-g.a3e674063-0ubuntu1~18.04.1 all MAAS server Django web framework (Python 3)
  ii python3-maas-client 2.8.2-8577-g.a3e674063-0ubuntu1~18.04.1 all MAAS python API client (Python 3)
  ii python3-maas-provisioningserver 2.8.2-8577-g.a3e674063-0ubuntu1~18.04.1 all MAAS server provisioning libraries (Python 3)

  [Steps to Reproduce]

  - prepare a maas server(installed by packages for me and the customer). it doesn't have to be HA to reproduce
  - prepare a set of certificate, key and ca-bundle
  - place a new conf[2] in /etc/nginx/sites-enabled and `sudo systemctl restart nginx`
  - add the ca certificates to the host
  sudo mkdir /usr/share/ca-certificates/extra
  sudo cp -v ca-bundle.crt /usr/share/ca-certificates/extra/
  dpkg-reconfigure ca-certificates
  - login with a new profile over https url
  - when not added the ca-bundle to the trusted ca cert store, it fails to login and '--insecure' flag also doesn't work[3]

  [Known Workarounds]
  None

  [Test]
  # Note even though this change only affects Python3
  # I tested it with Python2 with no issues and was able to connect.
  Also please make note of the 2 packages. One is for Python2 the other Python3 

  Python2 ===> python-httplib2_0.9.2+dfsg-1ubuntu0.3_all.deb
  Python3 ===>  python3-httplib2_0.9.2+dfsg-1ubuntu0.3_all.deb

  helpful urls:
  https://maas.io/docs/deb/2.8/cli/installation
  https://maas.io/docs/deb/2.8/cli/configuration-journey
  https://maas.io/docs/deb/2.8/ui/configuration-journey

  # create bionic VM/lxc container
  lxc launch ubuntu:bionic lp1820083

  # get source code from repo
  pull-lp-source  python-httplib2 bionic

  # install maas-cli
  apt-get install maas-cli

  # install maas server
  apt-get install maas

  # init maas
  sudo maas init

  # answer questions

  # generate self signed cert and key
  openssl req -newkey rsa:4096 -x509 -sha256 -days 60 -nodes -out localhost.crt -keyout localhost.key

  # add certs
  sudo cp -v test.crt /usr/share/ca-certificates/extra/

  # add new cert to list
  sudo dpkg-reconfigure ca-certificates

  # select yes with spacebar
  # save

  # create api key files
  touch api_key
  touch api-key-file

  # remove any packages with this
  # or this python3-httplib2
  apt-cache search python-httplib2
  apt-get remove python-httplib2
  apt-get remove python3-httplib2

  # create 2 admin users
  sudo maas createadmin testadmin
  sudo maas createadmin secureadmin

  # generate maas api keys
  sudo maas apikey --username=testadmin > api_key
  sudo maas apikey --username=secureadmin > api-key-file

  # make sure you can login to maas-cli without TLS
  # by running this script
  # this is for the non-tls user
  # this goes into a script called maas-login.sh
  touch maas-login.sh
  sudo chmod +rwx maas-login.sh
  ----
  #!/bin/sh
  PROFILE=testadmin
  API_KEY_FILE=/home/ubuntu/api_key
  API_SERVER=127.0.0.1:5240

  MAAS_URL=http://$API_SERVER/MAAS

  maas login $PROFILE $MAAS_URL - < $API_KEY_FILE
  ----
  sudo chmod +rwx https-maas.sh
  # another script called https-maas.sh
  # for the tls user
  ----
  #!/bin/sh
  PROFILE=secureadmin
  API_KEY_FILE=/home/ubuntu/api-key-file
  API_SERVER=127.0.0.1

  MAAS_URL=https://$API_SERVER/MAAS

  maas login --insecure $PROFILE $MAAS_URL - < $API_KEY_FILE
  ----

  # try to login
  ./maas-login.sh

  cd /etc/nginx/sites-enabled
  sudo touch maas-https-default
  #example nginx config for maas https
  server {
   listen 443 ssl http2;

   server_name _;
   ssl_certificate /home/ubuntu/localhost.crt;
   ssl_certificate_key /home/ubuntu/localhost.key;

   location / {
    proxy_pass http://localhost:5240;
    include /etc/nginx/proxy_params;
   }

   location /MAAS/ws {
    proxy_pass http://127.0.0.1:5240/MAAS/ws;
                  proxy_http_version 1.1;
                  proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "Upgrade";
   }
  }

  # create link
  sudo ln -s /etc/nginx/sites-available/maas-https-default /etc/nginx/sites-enabled

  # look at errors
  cat /var/log/maas/regiond.log
  cat regiond.log | grep "Python-http"
  *i didn't see any 404's though

  2020-12-15 13:24:48 regiond: [info] 127.0.0.1 GET /MAAS/api/2.0/users/?op=whoami HTTP/1.1 --> 200 OK (referrer: -; agent: Python-httplib2/0.9.2 (gzip))
  2020-12-15 13:24:48 regiond: [info] 127.0.0.1 GET /MAAS/api/2.0/describe/ HTTP/1.1 --> 200 OK (referrer: -; agent: Python-httplib2/0.9.2 (gzip))
  2020-12-15 14:24:46 regiond: [info] 127.0.0.1 GET /MAAS/api/2.0/describe/ HTTP/1.0 --> 200 OK (referrer: -; agent: Python-httplib2/0.9.2 (gzip))

  # install fixed package
  sudo apt install ./python3-httplib2_0.9.2+dfsg-1ubuntu0.2.1_all.deb

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/python-httplib2/+bug/1906720/+subscriptions