KVM support in Juju on ARM64

[Reed O'Brien]

On Mon, Jan 30, 2017 at 5:13 PM, Reed O'Brien <reed.obrien at canonical.com>
wrote:

> KVM as a juju deploy target now works on AMD64 and ARM64. There is an
> issue on ARM64 using trusty, so it only works with xenial on the hardware I
> could test on atm. There are a number of issues that prevent running juju
> on trusty on ARM64. Following is a support matrix, an outline of known
> issues, and scenarios where we might run into the issue of trusty not
> working as expected. Finally there are a couple thoughts about handling the
> issue. Any input on options/solutions for this issue are appreciated.
>
> ARM64 QEMU/KVM UEFI Support Matrix
>

I've put the support matrix here: http://pastebin.ubuntu.com/23907437/ since
the html table was stripped from the list archive.


>
>    - EFI guests require qemu-efi, which is only available in xenial.
>
>
>    - Trusty’s 3.13 kernel can’t boot on GICv3 systems, nor as a guest on
>    GICv3 systems.
>
>
>    - Trusty’s virt stack does not support GICv3 guests, but that support
>    is available in the cloud archive.
>
>
> The first issue is that there is a very limited set of hardware that was
> certified for trusty on ARM64. So that severely limits the number of
> systems where we would expect juju to be able to deploy to trusty. As noted
> in the next issue is possible using the hwe-x kernel, but that is not
> something juju can control.
>
> The second issue is that those systems include GICv2 (General Interrupt
> Controller v2), whereas newer systems include GICv3.  This would require
> trusty to boot with hwe-x kernel. AFAIU there are no official cloud images
> with that kernel pre-installed. Additionally, it currently seems it isn't
> possible to set hwe-x as the minimum kernel version in MAAS:
> https://bugs.launchpad.net/maas/+bug/1659694
>
> Newer versions of qemu/libvirt support setting the GIC version to "host"
> which allows the VM to use whatever the host supports as long as the guest
> supports. Trusty guests will not boot on a GICv3 host.  Also "host" as a
> GIC element’s version attribute isn't supported by the virt stack that
> ships with trusty.
>
> One final issue is that there is no qemu-efi package built for trusty. It
> is feasible that someone could build their own ppa for use.  Other options
> include:
>
> - adding qemu-efi to trusty-backports which is not easy to use,
> - push a stable release update back to trusty, which is a regression risk
> for x86 by updating   the source that also builds ovmf,
> - introduce qemu-efi in trusty which would require an exception from the
> security team.
>
>
> There are a few scenarios which involve deploying trusty.
>
> - juju bootstrap a-cloud arm64-controller --default-series trusty
>        This would a require a trusty cloud image with hwe-x kernel
> pre-installed.
> - juju add-machine --series trusty
>        This would require the underlying provider to boot with hwe-x kernel
>        minimum. This also seems like it is going to fail to deploy with the
>        wily kernel.
> - juju deploy wordpress --to kvm:0
>        This would require the machine-0 to be able to install qemu-efi and
> it
>        would also require a trusty cloud image with hwe-x kernel
> pre-installed.
>
>
>
> So given all the possible knobs one would need to turn to make it work,
> should we:
>
> - block it from working in juju and return an error noting that trusty on
> ARM64 isn’t supported?
>
> This in conjunction with the deploy scenario in the last section seems to
> be a real issue. One could decide to deploy only xenial hosts and guests on
> ARM64, but it would hobble the usefulness of quite a few charms that
> require trusty or bundles that require such charms. This seems draconian
> when selecting a particular minimum kernel in MAAS could make it work --
> that is if qemu-efi was available and there weren’t hardware version issues.
>
> - warn that it won't work without a custom setup and document it as a
> known issue pointing in to a wiki page in the warning/error that contains
> the information above?
> - something else/other ideas?
>
>
> Cheers,
> Reed
>
> --
Reed O'Brien
✉ reed.obrien at canonical.com
✆ 415-562-6797
💻 redir
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/juju-dev/attachments/20170201/36e5082b/attachment.html>