Apple M1 / Hypervisor.framework support

[flying-sausages]

What are you trying to do?
Run Multipass to launch instances on top of Apple Silicon devices

What's your proposed solution?
Restrict multipass on AS to use the Virtualisation API intrdouced in Big Sur only and ensure ARM builds of ubuntu are being installed

Additional context
I have 0 experience in any of the underlying technologies that are used here. I know 1.5.0 has Big Sur support on Intel through the API above, and so I'm wondering what are the extra steps that need to be taken care in order to allow users to have ARM-based VMs.

ED: Latest build

1.8.0 build 787

[Saviq]

Hi @flying-sausages, this is definitely something we want to get going, but it will take some time. The API you mention is quite low level, the likes of hyperkit and qemu are what we'd need to implement support for it first. Did you see evidence that the hardware even supports virtualization already? From what we've read the dev kits did not, and I've not seen anyone confirming they have a VM running under Apple Silicon.

[flying-sausages]

I've seen a success story here which I think could be helpful https://forums.macrumors.com/threads/ubuntu-linux-virtualized-on-m1-success.2270365/

[flying-sausages]

Another useful link from the same thread https://github.com/JacopoMangiavacchi/M1-Linux-SSH

[flying-sausages]

And another https://github.com/evansm7/vftool

[RyanMorash]

The M1 does have virtualization support. There have been a set of patches made to enable Hypervisor.framework (hvf) support to QEMU for aarch64: https://patchwork.kernel.org/project/qemu-devel/list/?series=391797

Also, there are now two different virtualization frameworks on macOS. There continues to be the lower-level Hypervisor.framework which is currently utilized by hyperkit via machyve/xhyve. Virtualization.framework is a new higher-level set of APIs which allows you to create VMs using the Virtio spec without having to deal with a separate hypervisor (such as xhyve)

[Saviq]

Yeah thanks @RyanMorash @flying-sausages for references, we're monitoring this space and will jump on it whenever things settle. One notorious source of problems for us is networking (NAT, bridging, DHCP, DNS), which hyperkit (through Hypervisor.framework) mostly solves and we'd love to continue using that. If, however, qemu gains support for all the bits'n'pieces we need, we may well go that route.

[flying-sausages]

https://gist.github.com/niw/e4313b9c14e968764a52375da41b4278#file-readme-md looks like there's QEMU support through some patches? Lively discussion happening underneath that.

[RyanMorash]

Docker is moving away from using HyperKit (which they originally created for Docker Desktop) and switching to Virtualization.framework: https://www.docker.com/blog/download-and-try-the-tech-preview-of-docker-desktop-for-m1/

[ChangheeOh]

I have tried to create VM with vmcli(https://github.com/gyf304/vmcli) and almost succeeded to run Ubuntu arm64 image.
vmcli is also based on MacOS Virtualization.framework and it also have several issues and not yet stable in my opinion.
I wish it could help multipass team to enhance it to support M1 Mac.

[nottrobin]

It seems docker have made good progress: https://docs.docker.com/docker-for-mac/apple-m1/

[Saviq]

@nottrobin yeah we know everything we need now, just need to roadmap it - hoping for early next cycle.

[andy-isoc]

Is there a good place (other than this issue!) to watch for updates on this and when it might be released?

[bassu]

There is also ACVM (https://github.com/KhaosT/ACVM) GUI qemu launcher (based on patches from Alexander Graf to support Hypervisor.framework syscalls on M1 macOS).

But I suppose if multipass uses Hyperkit then it might take some time!