[Bug 2093880] [NEW] libcephfs: flush the caps release in filesystem sync

[Kinson Chan]

Public bug reported:

Hi,

The bug is as mentioned in the Ceph upstream: https://tracker.ceph.com/issues/67221
and also Linux upstream: https://github.com/torvalds/linux/commit/ccda9910d8490f4fb067131598e4b2e986faa5a0

Under some situations, the libcephfs forgets the acquired capabilities
and thus gets evicted by the Ceph MDS.  I would like to see if a
backport will be available for Ubuntu 22.04 LTS (Jammy) Ceph Quincy
client (17.2.x).

Logs on the client:
  Dec 19 04:32:45 *** kernel: libceph: mds0 (1)***:6801 socket closed (con state OPEN)
  Dec 19 04:32:46 *** kernel: libceph: mds0 (1)***:6801 session reset
  Dec 19 04:32:46 *** kernel: ceph: mds0 closed our session
  Dec 19 04:32:46 *** kernel: ceph: mds0 reconnect start
  Dec 19 04:32:46 *** kernel: ceph: mds0 reconnect denied
  Dec 19 04:32:46 *** kernel: libceph: mds0 (1)***:6801 socket closed (con state V1_CONNECT_MSG)
  Dec 19 04:32:47 *** kernel: ceph: mds0 rejected session

Logs on the server: 
  Dec 19 04:30:17 *** ceph-mds[1408372]: log_channel(cluster) log [WRN] : client.911386 isn't responding to mclientcaps(revoke), ino 0x1004e6bede5 pending pAsLsXsFs issued pAsLsXsFs, sent 240.313055 seconds ago
  Dec 19 04:32:44 *** ceph-mds[1408372]: log_channel(cluster) log [INF] : Evicting (and blocklisting) client session 911386 (v1:***:0/362122962)

The Ubuntu client version:
  Description:	Ubuntu 22.04.5 LTS
  Release:	22.04

Package: 
  libcephfs2/jammy-updates,jammy-security,now 17.2.7-0ubuntu0.22.04.2 amd64 [installed,automatic]

What expected to happen:
* There shall be no 'client isn't responding to ... pending pAsLsXsFs ...' messages, and no eviction.

What happened instead:
* The error appeared and the client is evicted.  

Thanks,
Kinson

** Affects: ceph (Ubuntu)
     Importance: Undecided
         Status: New


** Tags: jammy

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to ceph in Ubuntu.
https://bugs.launchpad.net/bugs/2093880

Title:
  libcephfs: flush the caps release in filesystem sync

Status in ceph package in Ubuntu:
  New

Bug description:
  Hi,

  The bug is as mentioned in the Ceph upstream: https://tracker.ceph.com/issues/67221
  and also Linux upstream: https://github.com/torvalds/linux/commit/ccda9910d8490f4fb067131598e4b2e986faa5a0

  Under some situations, the libcephfs forgets the acquired capabilities
  and thus gets evicted by the Ceph MDS.  I would like to see if a
  backport will be available for Ubuntu 22.04 LTS (Jammy) Ceph Quincy
  client (17.2.x).

  Logs on the client:
    Dec 19 04:32:45 *** kernel: libceph: mds0 (1)***:6801 socket closed (con state OPEN)
    Dec 19 04:32:46 *** kernel: libceph: mds0 (1)***:6801 session reset
    Dec 19 04:32:46 *** kernel: ceph: mds0 closed our session
    Dec 19 04:32:46 *** kernel: ceph: mds0 reconnect start
    Dec 19 04:32:46 *** kernel: ceph: mds0 reconnect denied
    Dec 19 04:32:46 *** kernel: libceph: mds0 (1)***:6801 socket closed (con state V1_CONNECT_MSG)
    Dec 19 04:32:47 *** kernel: ceph: mds0 rejected session

  Logs on the server: 
    Dec 19 04:30:17 *** ceph-mds[1408372]: log_channel(cluster) log [WRN] : client.911386 isn't responding to mclientcaps(revoke), ino 0x1004e6bede5 pending pAsLsXsFs issued pAsLsXsFs, sent 240.313055 seconds ago
    Dec 19 04:32:44 *** ceph-mds[1408372]: log_channel(cluster) log [INF] : Evicting (and blocklisting) client session 911386 (v1:***:0/362122962)

  The Ubuntu client version:
    Description:	Ubuntu 22.04.5 LTS
    Release:	22.04

  Package: 
    libcephfs2/jammy-updates,jammy-security,now 17.2.7-0ubuntu0.22.04.2 amd64 [installed,automatic]

  What expected to happen:
  * There shall be no 'client isn't responding to ... pending pAsLsXsFs ...' messages, and no eviction.

  What happened instead:
  * The error appeared and the client is evicted.  

  Thanks,
  Kinson

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/ceph/+bug/2093880/+subscriptions