[Bug 1408331] Re: libwww perl in ubuntu always enforces HTTPS server certificate

[sullr]

The option verify_hostname is like the name suggests only responsible
for verifying the host name against the certificate. It does not control
the verification of the certificate chain or any other certificate
validations, even if it can be used like this in some versions of
LWP::Protocol::https. But this is actually a bug, see https://github.com
/libwww-perl/lwp-protocol-https/pull/14 (very long discussion).

The only documentation of the option verify_hostname in LWP::UserAgent
says:

           "verify_hostname" => $bool
            When TRUE LWP will for secure protocol schemes ensure it connects to servers that have a valid certificate
            matching the expected hostname. 

Which confirms that this option cares about verifying the host name
only.

To disable any kind of certificate validation you have to use ssl_opts
to set SSL_verify_mode to 0 (i.e. SSL_VERIFY_NONE).

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to libwww-perl in Ubuntu.
https://bugs.launchpad.net/bugs/1408331

Title:
  libwww perl in ubuntu always enforces HTTPS server certificate

Status in libwww-perl package in Ubuntu:
  New

Bug description:
  Given this simple code:

      $ua = LWP::UserAgent->new;
      $ua->agent("netview");
      $ua->protocols_allowed( [ 'https' ] );
      $ua->ssl_opts( verify_hostname => 0 );
      push @{ $ua->requests_redirectable }, 'POST', 'GET';

      my $req = HTTP::Request->new( GET =>
                                    "https://$server/blc/api/routers/type/pe" );

      $req->content_type('application/json');
      $req->authorization_basic($apipw->{APIUSER}, $apipw->{APIPW});

      my $res = $ua->request($req);
      LOGDIE "Error getting PE routers via REST to $server: ".$res->status_line.
        "(".$res->content.")"
          if ! $res->is_success;

  I get this message:
  Error getting PE routers via REST to blc.serv.as2116.net: 500 Can't connect to blc.serv.as2116.net:443 (certificate verify failed)(Can't connect to blc.serv.as2116.net:443 (certificate verify failed)

  LWP::Protocol::https::Socket: SSL connect attempt failed with unknown
  error error:14090086:SSL
  routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed at
  /usr/share/perl5/LWP/Protocol/http.pm line 41.

  Strace shows that the code is looking for a CA file from the OpenSSL
  package.  blc.serv.as2116.net's sertificate is signed by a uncommon CA
  so this fails.

  BUT it should not be trying to verify this at all due to the
  verify_hostname setting.

  In HTTP::Protocol::https one finds a  _extra_sock_opts function that's
  different than the official LWP release.  I replaced it with the this
  LWP 6.04 code:

  sub _extra_sock_opts
  {
      my $self = shift;
      my %ssl_opts = %{$self->{ua}{ssl_opts} || {}};
      if (delete $ssl_opts{verify_hostname}) {
  	$ssl_opts{SSL_verify_mode} ||= 1;
  	$ssl_opts{SSL_verifycn_scheme} = 'www';
      }
      else {
  	$ssl_opts{SSL_verify_mode} = 0;
      }
      if ($ssl_opts{SSL_verify_mode}) {
  	unless (exists $ssl_opts{SSL_ca_file} || exists $ssl_opts{SSL_ca_path}) {
  	    eval {
  		require Mozilla::CA;
  	    };
  	    if ($@) {
  		if ($@ =! /^Can't locate Mozilla\/CA\.pm/) {
  		    $@ = <<'EOT';
  Can't verify SSL peers without knowing which Certificate Authorities to trust

  This problem can be fixed by either setting the PERL_LWP_SSL_CA_FILE
  envirionment variable or by installing the Mozilla::CA module.

  To disable verification of SSL peers set the PERL_LWP_SSL_VERIFY_HOSTNAME
  envirionment variable to 0.  If you do this you can't be sure that you
  communicate with the expected peer.
  EOT
  		}
  		die $@;
  	    }
  	    $ssl_opts{SSL_ca_file} = Mozilla::CA::SSL_ca_file();
  	}
      }
      $self->{ssl_opts} = \%ssl_opts;
      return (%ssl_opts, $self->SUPER::_extra_sock_opts);
  }

  Then I get this instead

  Error getting PE routers via REST to blc.serv.as2116.net: 401
  Unauthorized({"error":{"code":401,"message":"Unauthorized: Basic
  Authentication Required"}})

  which means that the SSL handshake was completed.

  ProblemType: Bug
  DistroRelease: Ubuntu 14.04
  Package: libwww-perl 6.05-2
  ProcVersionSignature: Ubuntu 3.13.0-43.72-generic 3.13.11.11
  Uname: Linux 3.13.0-43-generic x86_64
  ApportVersion: 2.14.1-0ubuntu3.6
  Architecture: amd64
  Date: Wed Jan  7 16:05:09 2015
  InstallationDate: Installed on 2014-12-19 (19 days ago)
  InstallationMedia: Ubuntu-Server 14.04 LTS "Trusty Tahr" - Release amd64 (20140416.2)
  PackageArchitecture: all
  SourcePackage: libwww-perl
  UpgradeStatus: No upgrade log present (probably fresh install)

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libwww-perl/+bug/1408331/+subscriptions