[Bug 1320094] Re: segfault from aes ccm encryption after RSA key generation and EVP_PKEY_assign_RSA()

[Adrien Nader]

Thanks for the report and for the reproducer. I haven't been able to
trigger a segfault despite numerous attempts. I'll therefore mark this
bug as Incomplete for now.

** Changed in: openssl (Ubuntu)
       Status: New => Incomplete

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to openssl in Ubuntu.
https://bugs.launchpad.net/bugs/1320094

Title:
  segfault from aes ccm encryption after RSA key generation and
  EVP_PKEY_assign_RSA()

Status in openssl package in Ubuntu:
  Incomplete

Bug description:
  1) lsb_release -rd
  Description:	Ubuntu 12.04.4 LTS
  Release:	12.04

  2) apt-cache policy libssl1.0.0
  libssl1.0.0:
    Installed: 1.0.1-4ubuntu5.13
    Candidate: 1.0.1-4ubuntu5.13
    Version table:
   *** 1.0.1-4ubuntu5.13 0
          500 http://be.archive.ubuntu.com/ubuntu/ precise-updates/main amd64 Packages
          500 http://security.ubuntu.com/ubuntu/ precise-security/main amd64 Packages
          100 /var/lib/dpkg/status
       1.0.1-4ubuntu3 0
          500 http://be.archive.ubuntu.com/ubuntu/ precise/main amd64 Packages

  For the sourcecode that triggers the segfault see the attachement.

  3/4)

  I was testing aes ccm encryption when I stumbled over a segmentation fault.
  I was able to reproduce this error using code from the openssl demos  at openssl.org.

  I started with demos/evp/aesccm.c and added rsa key generation as used in 'demos/tunala/cb.c' and convert this rsa key into an EVP_PKEY key as done in 'demos/selfsign.c'. Then I added this rsa key generation function in front of the aes ccm encryption and decryption.
  Finally, a for loop repeatedly performs the keygeneration, aes ccm encryption and aes ccm decryption. This eventually results in a segmentation fault during aes ccm encryption (see gdb output below) on a x64 Ubuntu 12.04 with latest openssl version as provided by ubuntu package system (1.0.1-4ubuntu5.13).

  Note that the segfault only occurs if the rsa key is assigned to an EVP_PKEY. Otherwise, if only the RSA key is generated, the segfault does not occur.
  Furthermore, the segfault does not occur if I use the standard openssl libraries from openssl.org.

  When encountering this error in my own code I could observe that the error occurred more often on a machine that only runs the standard processes and is accessed remotely by ssh, compared to a local workstation with running webbrowser, development IDE, etc., where the error occurred rather seldom. Hence, I have the feeling that this could be related to too little
  randomness for the RNG, but I do not have any idea how to debug this.

  
  === gdb backtrace ===

  (gdb) run
  Starting program: /home/hiller/openssl_bug/aesccm
  AES CCM Encrypt:
  Plaintext:
  0000 - c8 d2 75 f9 19 e1 7d 7f-e6 9c 2a 1f 58 93 9d fe   ..u...}...*.X...
  0010 - 4d 40 37 91 b5 df 13 10-                          M at 7.....
  Ciphertext:
  0000 - 8a 0f 3d 82 29 e4 8e 74-87 fd 95 a2 8a d3 92 c8   ..=.)..t........
  0010 - 0b 36 81 d4 fb c7 bb fd-                          .6......
  Tag:
  0000 - 2d d6 ef 1c 45 d4 cc b7-23 dc 07 44 14 db 50 6d   -...E...#..D..Pm
  AES CCM Derypt:
  Ciphertext:
  0000 - 8a 0f 3d 82 29 e4 8e 74-87 fd 95 a2 8a d3 92 c8   ..=.)..t........
  0010 - 0b 36 81 d4 fb c7 bb fd-                          .6......
  Plaintext:
  0000 - c8 d2 75 f9 19 e1 7d 7f-e6 9c 2a 1f 58 93 9d fe   ..u...}...*.X...
  0010 - 4d 40 37 91 b5 df 13 10-                          M at 7.....
  AES CCM Encrypt:
  [ the output above is repeated several times ]

  Program received signal SIGSEGV, Segmentation fault.
  0x0000000000000090 in ?? ()
  (gdb) backtrace
  #0  0x0000000000000090 in ?? ()
  #1  0x00007ffff7a948d4 in CRYPTO_ccm128_encrypt_ccm64 (ctx=0x604fd0,
  inp=0x401240
  "\310\322u\371\031\341}\177\346\234*\037X\223\235\376M at 7\221\265\337\023\020",

      out=0x7fffffffe0c0
  "\310\322u\371\031\341}\177\346\234*\037X\223\235\376M at 7\221\265\337\023\020",
  len=24, stream=<optimized out>) at ccm128.c:354
  #2  0x00007ffff7af1688 in aes_ccm_cipher (ctx=0x604e10,
  out=0x7fffffffe0c0
  "\310\322u\371\031\341}\177\346\234*\037X\223\235\376M at 7\221\265\337\023\020",

      in=0x401240
  "\310\322u\371\031\341}\177\346\234*\037X\223\235\376M at 7\221\265\337\023\020",
  len=24) at e_aes.c:1275
  #3  0x00007ffff7aedaa2 in EVP_EncryptUpdate (ctx=0x604e10,
  out=0x7fffffffe0c0
  "\310\322u\371\031\341}\177\346\234*\037X\223\235\376M at 7\221\265\337\023\020",
  outl=0x7fffffffe0bc,
      in=0x401240
  "\310\322u\371\031\341}\177\346\234*\037X\223\235\376M at 7\221\265\337\023\020",
  inl=<optimized out>) at evp_enc.c:314
  #4  0x0000000000400e37 in aes_ccm_encrypt () at aesccm.c:106
  #5  0x00000000004010ce in main (argc=1, argv=0x7fffffffe5e8) at aesccm.c:161

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/openssl/+bug/1320094/+subscriptions