[SRU][J][I][F][PATCH 0/1] Null Pointer issue in nfs code running Ubuntu on IBM Z (LP: 1968096)

[frank.heimes at canonical.com]

BugLink: https://bugs.launchpad.net/bugs/1968096

SRU Justification:

[Impact]

* The kernel crashed under load with a null pointer issue in nfs code:
    [556585.270959] Krnl Code:#0000000000000000: 0000 illegal
                              >0000000000000002: 0000 illegal
                               0000000000000004: 0000 illegal
                               0000000000000006: 0000 illegal
                               0000000000000008: 0000 illegal
                               000000000000000a: 0000 illegal
                               000000000000000c: 0000 illegal
                               000000000000000e: 0000 illegal
    [556585.270967] Call Trace:
    [556585.270982] ([<000003ff80d6fb1a>] rpcauth_lookup_credcache+0x5a/0x300 [sunrpc])
    [556585.270993] [<000003ff80e1182c>] nfs_ctx_key_to_expire+0xec/0x130 [nfs]
    [556585.271004] [<000003ff80e1189c>] nfs_key_timeout_notify+0x2c/0x70 [nfs]
    [556585.271014] [<000003ff80dfdf7e>] nfs_file_write+0x3e/0x320 [nfs]
    [556585.271016] [<00000028165944a8>] new_sync_write+0x118/0x1b0
    [556585.271017] [<0000002816594ee0>] vfs_write+0xb0/0x1b0
    [556585.271019] [<0000002816596a1e>] ksys_pwrite64+0x7e/0xc0
    [556585.271021] [<0000002816bb26b2>] system_call+0x2a6/0x2c8

* Several dumps were generated and shared with Canonical.

* Analysis (done by kernel and SEG) point to refcount leaks fixed,
  that are already fixed in the following commit/fix:

[Fix]

* ca05cbae2a0468e5d78e9b4605936a8bf5da328b ca05cbae2a04 "NFS: Fix up nfs_ctx_key_to_expire()"

[Test Case]

* There is unfortunately no reproducer or trigger available for this issue.

* It just happens now and then under higher load.

* A patched kernel (focal 5.4 and bionic 5.4-hwe) were created and
  ran for more than a week in a special staging environment (at IBM)
  without further crashes.

* Hence the test and verification will be done by the IBM Z team.

[Where problems could occur]

* The inode handling can become broken, in case the changes
  on the pointers are erroneous.

* Problems with the authentication and/or the credentials could occur
  due to the modifications in put_rpccred, rpc_cred and rpc_auth.

* The expiration of the cached credentials could be harmed as well,
  due to the changes in nfs_ctx_key_to_expire.

* The different pointer artihmetic may cause further issues - wrong
  or null pointer references.

* Positive is that the original commit was brought upstream by nfs experts.

* A patched test kernel sustained day long runs under load in a staging
  and test environment.

* The author of the upstream commit/patch is well known in the NFS area.

[Other]

* The Salesforce Case Number 00334334 is associated with this bug.

* Commit ca05cbae2a04 was upstream accepted with 5.16-rc1.

* But commit ca05cbae2a04 was unfortunately not tagged as stable,
  hence it was not picked automatically.

* Since kinetic's (22.10) target kernel is 5.18,
  it will have the patch included,
  hence no dedicated PATCH request for kinetic.

Trond Myklebust (1):
  NFS: Fix up nfs_ctx_key_to_expire()

 fs/nfs/inode.c         |  4 ++--
 fs/nfs/write.c         | 41 ++++++++++++++++++++++++++++-------------
 include/linux/nfs_fs.h |  2 +-
 3 files changed, 31 insertions(+), 16 deletions(-)

-- 
2.25.1