[Zesty][PATCH v4 03/10] KVM: PPC: VFIO: Add in-kernel acceleration for VFIO

[Kleber Souza]

On 08/07/17 17:44, Jose Ricardo Ziviani wrote:
> From: Alexey Kardashevskiy <aik at ozlabs.ru>
> 
> BugLink: http://bugs.launchpad.net/bugs/1686019
> 
> Removed WARN_ON(!kref_read(&stit->kref)); to avoid a dependency to
> commit 2c935bc, that adds sensible changes to other important kernel
> components.

Hi Ziviani,

Please include the backport comments bellow the original Sign-offs to
separate them from the original commit message. See example below.

> 
> This allows the host kernel to handle H_PUT_TCE, H_PUT_TCE_INDIRECT
> and H_STUFF_TCE requests targeted an IOMMU TCE table used for VFIO
> without passing them to user space which saves time on switching
> to user space and back.
> 
> This adds H_PUT_TCE/H_PUT_TCE_INDIRECT/H_STUFF_TCE handlers to KVM.
> KVM tries to handle a TCE request in the real mode, if failed
> it passes the request to the virtual mode to complete the operation.
> If it a virtual mode handler fails, the request is passed to
> the user space; this is not expected to happen though.
> 
> To avoid dealing with page use counters (which is tricky in real mode),
> this only accelerates SPAPR TCE IOMMU v2 clients which are required
> to pre-register the userspace memory. The very first TCE request will
> be handled in the VFIO SPAPR TCE driver anyway as the userspace view
> of the TCE table (iommu_table::it_userspace) is not allocated till
> the very first mapping happens and we cannot call vmalloc in real mode.
> 
> If we fail to update a hardware IOMMU table unexpected reason, we just
> clear it and move on as there is nothing really we can do about it -
> for example, if we hot plug a VFIO device to a guest, existing TCE tables
> will be mirrored automatically to the hardware and there is no interface
> to report to the guest about possible failures.
> 
> This adds new attribute - KVM_DEV_VFIO_GROUP_SET_SPAPR_TCE - to
> the VFIO KVM device. It takes a VFIO group fd and SPAPR TCE table fd
> and associates a physical IOMMU table with the SPAPR TCE table (which
> is a guest view of the hardware IOMMU table). The iommu_table object
> is cached and referenced so we do not have to look up for it in real mode.
> 
> This does not implement the UNSET counterpart as there is no use for it -
> once the acceleration is enabled, the existing userspace won't
> disable it unless a VFIO container is destroyed; this adds necessary
> cleanup to the KVM_DEV_VFIO_GROUP_DEL handler.
> 
> This advertises the new KVM_CAP_SPAPR_TCE_VFIO capability to the user
> space.
> 
> This adds real mode version of WARN_ON_ONCE() as the generic version
> causes problems with rcu_sched. Since we testing what vmalloc_to_phys()
> returns in the code, this also adds a check for already existing
> vmalloc_to_phys() call in kvmppc_rm_h_put_tce_indirect().
> 
> This finally makes use of vfio_external_user_iommu_id() which was
> introduced quite some time ago and was considered for removal.
> 
> Tests show that this patch increases transmission speed from 220MB/s
> to 750..1020MB/s on 10Gb network (Chelsea CXGB3 10Gb ethernet card).
> 
> Signed-off-by: Alexey Kardashevskiy <aik at ozlabs.ru>
> Acked-by: Alex Williamson <alex.williamson at redhat.com>
> Reviewed-by: David Gibson <david at gibson.dropbear.id.au>
> Signed-off-by: Paul Mackerras <paulus at ozlabs.org>
> (backported from commit 121f80ba68f1a5779a36d7b3247206e60e0a7418)

You can add the backport comments below the '(backported from ...)'
line, between square brackets and adding in identification of who is
including it. It would look something like this in this case:

[joserz: Removed WARN_ON(!kref_read(&stit->kref)); to avoid a dependency
to commit 2c935bc, that adds sensible changes to other important kernel
components.]

> Signed-off-by: Jose Ricardo Ziviani <joserz at linux.vnet.ibm.com>
> ---
>  Documentation/virtual/kvm/devices/vfio.txt |  18 +-
>  arch/powerpc/include/asm/kvm_host.h        |   8 +
>  arch/powerpc/include/asm/kvm_ppc.h         |   4 +
>  arch/powerpc/kvm/book3s_64_vio.c           | 305 ++++++++++++++++++++++++++++-
>  arch/powerpc/kvm/book3s_64_vio_hv.c        | 201 ++++++++++++++++++-
>  arch/powerpc/kvm/powerpc.c                 |   2 +
>  include/uapi/linux/kvm.h                   |   6 +
>  virt/kvm/vfio.c                            | 105 ++++++++++
>  8 files changed, 644 insertions(+), 5 deletions(-)
>