[Bug 1989558] Re: [UBUNTU 22.10] opencryptoki C_GenerateKeyPair() fails after generating > 500 RSA keys with CEX7 and CEX8 crypto cards

Thu Sep 29 14:03:15 UTC 2022

** Description changed:

  ---Problem Description---
  opencryptoki C_GenerateKeyPair() fails after generating > 500 RSA keys with CEX7 crypto cards

  Running a program that generates RSA keys and tests
  encyrption/decryption on Crypto cards, is failing after about 500
  iterations. The program makes the following opencryptoki calls in a
  loop:

  C_GenerateKeyPair()
  C_EncryptInit()
  C_Encrypt()
  C_DecryptInit()
  C_Decrypt()
  C_SignInit()
  C_Sign()       (RSA Sign)
  C_VerifyInit()
  C_Verify()     (RSA Verify)
  C_SignInit()
  C_SignUpdate() (RSA Multipart sign)
  C_VerifyInit()
  C_VerifyUpdate() (RSA multi-part verify)
  C_VerifyFinal()
  C_DestroyObject() (Destroy RSA public key)
  C_DestroyObject() (Destroy RSA private key)

  When running single-threaded, it fails consistently on loop 504. When
  running multi-threaded it fails on about the 506th iteration across all
  threads (ie with 5 threads, each thread fails after about 100 loops).

  On the 504th loop, the C_GenerateKeyPair() call fails with return code
  6. Looking at the /var/log/opencryptoki/trace.* logfile, I found these
  error messages which seem to correspond to the failure:

  ```
  09/06/2022 15:37:09 6099 [usr/lib/api/api_interface.c:1106 api] DEVEL: fcn->ST_DestroyObject returned:0x0
  09/06/2022 15:37:09 6099 [usr/lib/api/api_interface.c:1996 api] INFO: C_GenerateKeyPair
  09/06/2022 15:37:09 6099 [usr/lib/api/api_interface.c:2019 api] INFO: Valid Session handle id: 2
  09/06/2022 15:37:09 6099 [usr/lib/ep11_stdll/ep11_specific.c:5977 ep11tok] INFO: rsa_ec_generate_keypair m_GenerateKeyPair rc=0x0 spki_len=0x1a6 privkey_blob_len=0x970 mech='CKM_RSA_PKCS_KEY_PAIR_GEN'
  09/06/2022 15:37:09 6099 [usr/lib/ep11_stdll/ep11_specific.c:6612 ep11tok] INFO: ep11tok_generate_key_pair rc=0x0 hpubkey=0x1 hprivkey=0x2 pub_name='' priv_name='' pub_obj=0x3ff7001d210 priv_obj=0x3ff70068460
  09/06/2022 15:37:09 6099 [usr/lib/common/loadsave.c:149 ep11tok] DEVEL: Unable to set permissions on file.
  09/06/2022 15:37:09 6099 [usr/lib/common/loadsave.c:149 ep11tok] DEVEL: Unable to set permissions on file.
  09/06/2022 15:37:09 6099 [usr/lib/common/loadsave.c:149 ep11tok] DEVEL: Unable to set permissions on file.
  09/06/2022 15:37:09 6099 [usr/lib/common/obj_mgr.c:613 ep11tok] DEVEL: Object created: handle: 2
  09/06/2022 15:37:09 6099 [usr/lib/common/loadsave.c:2409 ep11tok] ERROR: fopen(/var/lib/opencryptoki/ep11tok/TOK_OBJ/OBxIH3zb): Too many open files
  09/06/2022 15:37:09 6099 [usr/lib/ep11_stdll/ep11_specific.c:6749 ep11tok] DEVEL: ep11tok_generate_key_pair Object mgr create final failed
  09/06/2022 15:37:09 6099 [usr/lib/common/obj_mgr.c:991 ep11tok] DEVEL: Object found: handle: 2
  09/06/2022 15:37:09 6099 [usr/lib/common/loadsave.c:224 ep11tok] ERROR: fopen failed
  09/06/2022 15:37:09 6099 [usr/lib/ep11_stdll/new_host.c:3837 ep11tok] DEVEL: ep11tok_generate_key_pair() failed.
  09/06/2022 15:37:09 6099 [usr/lib/ep11_stdll/new_host.c:3840 ep11tok] INFO: C_GenerateKeyPair: rc = 0x00000006, sess = 2, mech = 0x0
  09/06/2022 15:37:09 6099 [usr/lib/api/api_interface.c:2041 api] DEVEL: fcn->ST_GenerateKeyPair returned:0x6
  09/06/2022 15:37:09 6099 [usr/lib/api/api_interface.c:1653 api] INFO: C_Finalize
  ```

  I tested it on both z15 and z16 (with different crypto cards) and got
  the same failure.  In all cases, I am running on a KVM guest and using
  CEX7 crypto cards. I have not tested yet on CEX8 crypto cards.
+ 
+ ---
+ External link: https://warthogs.atlassian.net/browse/PEI-29

** Tags added: pei-29

-- 
You received this bug notification because you are a member of Ubuntu
Sponsors Team, which is subscribed to the bug report.
https://bugs.launchpad.net/bugs/1989558

Title:
  [UBUNTU 22.10] opencryptoki C_GenerateKeyPair() fails after generating
  > 500 RSA keys with CEX7 and CEX8 crypto cards

Status in Ubuntu on IBM z Systems:
  In Progress
Status in opencryptoki package in Ubuntu:
  In Progress

Bug description:
  ---Problem Description---
  opencryptoki C_GenerateKeyPair() fails after generating > 500 RSA keys with CEX7 crypto cards

  Running a program that generates RSA keys and tests
  encyrption/decryption on Crypto cards, is failing after about 500
  iterations. The program makes the following opencryptoki calls in a
  loop:

  C_GenerateKeyPair()
  C_EncryptInit()
  C_Encrypt()
  C_DecryptInit()
  C_Decrypt()
  C_SignInit()
  C_Sign()       (RSA Sign)
  C_VerifyInit()
  C_Verify()     (RSA Verify)
  C_SignInit()
  C_SignUpdate() (RSA Multipart sign)
  C_VerifyInit()
  C_VerifyUpdate() (RSA multi-part verify)
  C_VerifyFinal()
  C_DestroyObject() (Destroy RSA public key)
  C_DestroyObject() (Destroy RSA private key)

  When running single-threaded, it fails consistently on loop 504. When
  running multi-threaded it fails on about the 506th iteration across
  all threads (ie with 5 threads, each thread fails after about 100
  loops).

  On the 504th loop, the C_GenerateKeyPair() call fails with return code
  6. Looking at the /var/log/opencryptoki/trace.* logfile, I found these
  error messages which seem to correspond to the failure:

  ```
  09/06/2022 15:37:09 6099 [usr/lib/api/api_interface.c:1106 api] DEVEL: fcn->ST_DestroyObject returned:0x0
  09/06/2022 15:37:09 6099 [usr/lib/api/api_interface.c:1996 api] INFO: C_GenerateKeyPair
  09/06/2022 15:37:09 6099 [usr/lib/api/api_interface.c:2019 api] INFO: Valid Session handle id: 2
  09/06/2022 15:37:09 6099 [usr/lib/ep11_stdll/ep11_specific.c:5977 ep11tok] INFO: rsa_ec_generate_keypair m_GenerateKeyPair rc=0x0 spki_len=0x1a6 privkey_blob_len=0x970 mech='CKM_RSA_PKCS_KEY_PAIR_GEN'
  09/06/2022 15:37:09 6099 [usr/lib/ep11_stdll/ep11_specific.c:6612 ep11tok] INFO: ep11tok_generate_key_pair rc=0x0 hpubkey=0x1 hprivkey=0x2 pub_name='' priv_name='' pub_obj=0x3ff7001d210 priv_obj=0x3ff70068460
  09/06/2022 15:37:09 6099 [usr/lib/common/loadsave.c:149 ep11tok] DEVEL: Unable to set permissions on file.
  09/06/2022 15:37:09 6099 [usr/lib/common/loadsave.c:149 ep11tok] DEVEL: Unable to set permissions on file.
  09/06/2022 15:37:09 6099 [usr/lib/common/loadsave.c:149 ep11tok] DEVEL: Unable to set permissions on file.
  09/06/2022 15:37:09 6099 [usr/lib/common/obj_mgr.c:613 ep11tok] DEVEL: Object created: handle: 2
  09/06/2022 15:37:09 6099 [usr/lib/common/loadsave.c:2409 ep11tok] ERROR: fopen(/var/lib/opencryptoki/ep11tok/TOK_OBJ/OBxIH3zb): Too many open files
  09/06/2022 15:37:09 6099 [usr/lib/ep11_stdll/ep11_specific.c:6749 ep11tok] DEVEL: ep11tok_generate_key_pair Object mgr create final failed
  09/06/2022 15:37:09 6099 [usr/lib/common/obj_mgr.c:991 ep11tok] DEVEL: Object found: handle: 2
  09/06/2022 15:37:09 6099 [usr/lib/common/loadsave.c:224 ep11tok] ERROR: fopen failed
  09/06/2022 15:37:09 6099 [usr/lib/ep11_stdll/new_host.c:3837 ep11tok] DEVEL: ep11tok_generate_key_pair() failed.
  09/06/2022 15:37:09 6099 [usr/lib/ep11_stdll/new_host.c:3840 ep11tok] INFO: C_GenerateKeyPair: rc = 0x00000006, sess = 2, mech = 0x0
  09/06/2022 15:37:09 6099 [usr/lib/api/api_interface.c:2041 api] DEVEL: fcn->ST_GenerateKeyPair returned:0x6
  09/06/2022 15:37:09 6099 [usr/lib/api/api_interface.c:1653 api] INFO: C_Finalize
  ```

  I tested it on both z15 and z16 (with different crypto cards) and got
  the same failure.  In all cases, I am running on a KVM guest and using
  CEX7 crypto cards. I have not tested yet on CEX8 crypto cards.

  ---
  External link: https://warthogs.atlassian.net/browse/PEI-29

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu-z-systems/+bug/1989558/+subscriptions