NACK(ish): [PULL][SRU Zesty] ARM GICv3 ITS save/restore

dann frazier dann.frazier at canonical.com
Wed Nov 15 21:57:44 UTC 2017


[Adding Christian since he's been helping on the QEMU side]

On Wed, Nov 15, 2017 at 9:31 AM, Stefan Bader
<stefan.bader at canonical.com> wrote:
> On 10.11.2017 18:49, dann frazier wrote:
>> BugLink: https://bugs.launchpad.net/bugs/1710019
>>
>> This series is necessary to enable KVM live migration for modern arm64
>> servers. All patches are clean cherry picks except for one trivial
>> offset fix (noted in the commit). This feature landed upstream back in
>> 4.12, but we've been waiting on a bug fix that just landed before
>> submitting (the last patch).
>>
>> Patches are needed for zesty's QEMU as well, which are under review by
>> the server team (see the bug) - however, even without the zesty QEMU
>> update, these changes are useful for those deploying OpenStack w/ the
>> hwe kernel + cloud archive.
>>
>> Tested on a Cavium ThunderX CRB and a Qualcomm Centriq board.
>>
>> The following changes since commit fe5e5001b417612723b174cbe3d1e9d5755b204f:
>>
>>   UBUNTU: Ubuntu-4.10.0-40.44 (2017-11-09 15:28:44 +0100)
>>
>> are available in the Git repository at:
>>
>>   git://git.launchpad.net/~dannf/ubuntu/+source/linux/+git/linux lp1710019-z
>>
>> for you to fetch changes up to 082d90155b3d8da957741c01a7c5ea1f2398fce4:
>>
>>   KVM: arm/arm64: vgic-its: Fix return value for device table restore
>> (2017-11-10 10:29:06 -0700)
>>
>> ----------------------------------------------------------------
>> Christoffer Dall (1):
>>       KVM: arm/arm64: Remove struct vgic_irq pending field
>>
>> Eric Auger (24):
>>       KVM: arm/arm64: Add ITS save/restore API documentation
>>       KVM: arm/arm64: Add GICV3 pending table save API documentation
>>       KVM: arm/arm64: vgic-its: rename itte into ite
>>       arm/arm64: vgic: turn vgic_find_mmio_region into public
>>       KVM: arm64: vgic-its: KVM_DEV_ARM_VGIC_GRP_ITS_REGS group
>>       KVM: arm/arm64: vgic: expose (un)lock_all_vcpus
>>       KVM: arm64: vgic-its: Implement vgic_its_has_attr_regs and
>> attr_regs_access
>>       KVM: arm64: vgic-its: Implement vgic_mmio_uaccess_write_its_creadr
>>       KVM: arm64: vgic-its: Introduce migration ABI infrastructure
>>       KVM: arm64: vgic-its: Implement vgic_mmio_uaccess_write_its_iidr
>>       KVM: arm64: vgic-its: Interpret MAPD Size field and check related errors
>>       KVM: arm64: vgic-its: Interpret MAPD ITT_addr field
>>       KVM: arm64: vgic-its: Check the device id matches TYPER DEVBITS range
>>       KVM: arm64: vgic-v3: vgic_v3_lpi_sync_pending_status
>>       KVM: arm64: vgic-its: Read config and pending bit in add_lpi()
>>       KVM: arm64: vgic-its: KVM_DEV_ARM_ITS_SAVE/RESTORE_TABLES
>>       KVM: arm64: vgic-its: vgic_its_alloc_ite/device
>>       KVM: arm64: vgic-its: Add infrastructure for table lookup
>>       KVM: arm64: vgic-its: Collection table save/restore
>>       KVM: arm64: vgic-its: vgic_its_check_id returns the entry's GPA
>>       KVM: arm64: vgic-its: Device table save/restore
>>       KVM: arm64: vgic-its: ITT save and restore
>>       KVM: arm64: vgic-its: Fix pending table sync
>>       KVM: arm64: vgic-v3: KVM_DEV_ARM_VGIC_SAVE_PENDING_TABLES
>>
>> Marc Zyngier (1):
>>       KVM: arm/arm64: vgic-v3: Don't pretend to support IRQ/FIQ bypass
>>
>> Vijaya Kumar K (7):
>>       KVM: arm/arm64: vgic: Implement support for userspace access
>>       KVM: arm/arm64: vgic: Add distributor and redistributor access
>>       KVM: arm/arm64: vgic: Introduce find_reg_by_id()
>>       KVM: arm/arm64: vgic: Introduce VENG0 and VENG1 fields to vmcr struct
>>       KVM: arm/arm64: vgic: Implement VGICv3 CPU interface access
>>       KVM: arm/arm64: vgic: Implement KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO ioctl
>>       KVM: arm/arm64: Documentation: Update arm-vgic-v3.txt
>>
>> wanghaibin (1):
>>       KVM: arm/arm64: vgic-its: Fix return value for device table restore
>>
>>  Documentation/virtual/kvm/devices/arm-vgic-its.txt |  120 ++
>>  Documentation/virtual/kvm/devices/arm-vgic-v3.txt  |   17 +-
>>  arch/arm/include/uapi/asm/kvm.h                    |   19 +-
>>  arch/arm/kvm/Makefile                              |    4 +-
>>  arch/arm/kvm/vgic-v3-coproc.c                      |   35 +
>>  arch/arm64/include/uapi/asm/kvm.h                  |   19 +-
>>  arch/arm64/kvm/Makefile                            |    3 +-
>>  arch/arm64/kvm/sys_regs.c                          |   27 +-
>>  arch/arm64/kvm/sys_regs.h                          |    4 +
>>  arch/arm64/kvm/vgic-sys-reg-v3.c                   |  346 ++++++
>>  include/kvm/arm_vgic.h                             |   16 +-
>>  include/linux/irqchip/arm-gic-v3.h                 |   18 +-
>>  virt/kvm/arm/vgic/vgic-its.c                       | 1176 +++++++++++++++++---
>>  virt/kvm/arm/vgic/vgic-kvm-device.c                |  255 ++++-
>>  virt/kvm/arm/vgic/vgic-mmio-v2.c                   |   87 +-
>>  virt/kvm/arm/vgic/vgic-mmio-v3.c                   |  203 +++-
>>  virt/kvm/arm/vgic/vgic-mmio.c                      |  178 ++-
>>  virt/kvm/arm/vgic/vgic-mmio.h                      |   34 +
>>  virt/kvm/arm/vgic/vgic-v2.c                        |   12 +-
>>  virt/kvm/arm/vgic/vgic-v3.c                        |  140 ++-
>>  virt/kvm/arm/vgic/vgic.c                           |   16 +-
>>  virt/kvm/arm/vgic/vgic.h                           |  108 +-
>>  22 files changed, 2531 insertions(+), 306 deletions(-)
>>  create mode 100644 arch/arm/kvm/vgic-v3-coproc.c
>>  create mode 100644 arch/arm64/kvm/vgic-sys-reg-v3.c
>>
> Less strongly in this case because the delta is restricted to architecture
> specific areas. So the impact of problems caused by the change is not that
> widespread. However, those are a lot of changes and are more in the enablement
> area.So gain I would rather defer this into the 4.13 hwe kernel. Even more since
> it will be only that last patch there. After all we got those rolling hwe
> kernels for that reason.
> But if there are strong reasons why this cannot wait I might get convinced in
> this case.

Stefan,
My goal here was to get this feature into Ubuntu Openstack as
"upstream" as possible, to accelerate the support and help simplify
maintenance.
As a side-effect of pushing the QEMU and kernel changes into zesty,
we'd get this feature ~3 months earlier in a supported hwe kernel, and
the QEMU changes would naturally propagate into the Ocata cloud
archive, resulting in live migration "just working" for arm64 on
Ubuntu OpenStack from Ocata onwards.

If rejected for zesty's kernel, we'll need to re-evaluate if it still
makes sense to push the QEMU changes into zesty, since the QEMU
changes alone don't add anything. If not, the next step will be to see
if we can get approval to carry those as a patch on top of zesty's
QEMU in the cloud archive (<- the maintenance issue).

Of course, all of that may fall into a "not your problem" bucket - I'm
just being transparent about my reasoning :)

  -dann




More information about the kernel-team mailing list