[PATCH][hirsute] UBUNTU: SAUCE: RDMA/core: Introduce peer memory interface

[Seth Forshee]

On Thu, Apr 08, 2021 at 03:47:31PM -0300, Guilherme Piccoli wrote:
> On Thu, Apr 8, 2021 at 11:38 AM dann frazier <dann.frazier at canonical.com> wrote:
> >
> > From: Jason Gunthorpe <jgg at nvidia.com>
> >
> > The peer_memory_client scheme allows a driver to register with the ib_umem
> > system that it has the ability to understand user virtual address ranges
> > that are not compatible with get_user_pages(). For instance VMAs created
> > with io_remap_pfn_range(), or other driver special VMA.
> >
> > For ranges the interface understands it can provide a DMA mapped sg_table
> > for use by the ib_umem, allowing user virtual ranges that cannot be
> > supported by get_user_pages() to be used as umems for RDMA.
> >
> > This is designed to preserve the kABI, no functions or structures are
> > changed, only new symbols are added:
> >
> >  ib_register_peer_memory_client
> >  ib_unregister_peer_memory_client
> >  ib_umem_activate_invalidation_notifier
> >  ib_umem_get_peer
> >
> > And a bitfield in struct ib_umem uses more bits.
> >
> > This interface is compatible with the two out of tree GPU drivers:
> >  https://github.com/RadeonOpenCompute/ROCK-Kernel-Driver/blob/master/drivers/gpu/drm/amd/amdkfd/kfd_peerdirect.c
> >  https://github.com/Mellanox/nv_peer_memory/blob/master/nv_peer_mem.c
> >
> > Signed-off-by: Yishai Hadas <yishaih at mellanox.com>
> > Signed-off-by: Feras Daoud <ferasda at mellanox.com>
> > Signed-off-by: Jason Gunthorpe <jgg at mellanox.com>
> > Signed-off-by: Leon Romanovsky <leonro at nvidia.com>
> > (cherry picked from commit a42989294cf39d6e829424734ab0e7ec48bebcef git://git.kernel.org/pub/scm/linux/kernel/git/leon/linux-rdma.git gpu-v5.11)
> > Signed-off-by: dann frazier <dann.frazier at canonical.com>
> 
> Hi Dann, thanks for the patch. It's a big one..was it discussed
> upstream? Do you have a Launchpad bug in which it is discussed?
> Usually there's a LP mentioned in the patch as "Buglink", you can
> check examples in our trees.
> 
> I suppose we need more information and preferably an upstream
> discussion link to gather basic information before accepting such a
> big change.

Hi Guilherme,

This patch has been the subject of considerable back-channel discussion
before Dann sent it. As for the size and out-of-tree status, those are
factors we're going to live with for the time being. I've reviewed the
patch, and absent any drivers which attempt to use the peer mem
interface it looks like it should have no fuctional impact. I also
didn't come across any issues which haven't already been addressed.

Additional review is very welcome though. Please let us know if you see
any issues in the code which ought to be addressed.

Thanks,
Seth