[Bug 1002434] [NEW] TLS interoperability issue in NSS based software

[Janne Snabb]

Public bug reported:

NSS (Netscape Security Services) module provides encryption services to
many applications, such as Thunderbird, Firefox and Chromium. NSS has a
hard coded maximum limit of 2236 bits for ephemeral Diffie-Hellman (DHE)
keys. If the TLS server (such as a web server, SMTP server, IMAP server,
etc) requests a bigger DHE key size, NSS based applications refuse to
interoperate. They just close the connection and display a confusing
error message (such as "Unknown error").

Recent versions of GnuTLS (as shipped by Ubuntu and other distributions)
include a new library API which recommends and automatically selects the
following key sizes:

Security level         key bits

LOW                         1248
LEGACY                   1776
NORMAL                 2432
HIGH                         3248

See the following for more information:
https://www.gnu.org/software/gnutls/manual/html_node/Selecting-
cryptographic-key-sizes.html

As can be seen, NSS's maximum limit of 2236 bits can only interoperate
with GnuTLS server which has been set at "LOW" or "LEGACY" security
level.

This bug was discovered when Exim's GnuTLS interface was revamped
recently. Thunderbird refused to complete TLS handshake with the Exim
SMTP server any more, because the new GnuTLS interface was following the
GnuTLS library's opinion on suitable key sizes.

Please patch the NSS library to accept reasonable key sizes: at the very
least 3248 bits should be accepted to allow interoperability with GnuTLS
at HIGH level. NSS is the only TLS library which has such a low hard
limit on DHE key size.

The only reason people are not hitting this bug frequently yet is that
most main stream server software still does not use GnuTLS library's new
API or recommendations but instead hard codes the DHE key size to 1024
or 2048 bits.

I am attaching a patch which points out the relevant #define in
blapit.h.

** Affects: nss
     Importance: Unknown
         Status: Unknown

** Affects: thunderbird
     Importance: Unknown
         Status: Unknown

** Affects: chromium (Ubuntu)
     Importance: Undecided
         Status: New

** Affects: firefox (Ubuntu)
     Importance: Undecided
         Status: New

** Affects: nss (Ubuntu)
     Importance: Undecided
         Status: New

** Affects: thunderbird (Ubuntu)
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Mozilla
Bugs, which is subscribed to thunderbird in Ubuntu.
https://bugs.launchpad.net/bugs/1002434

Title:
  TLS interoperability issue in NSS based software

To manage notifications about this bug go to:
https://bugs.launchpad.net/nss/+bug/1002434/+subscriptions