Some notes from the FreeBSD virtualization devsummit. Very rough, but my understanding is very rough, so all is as it should be.
Bhyve moving to UEFI loader away from FreeBSD and grub2
• Fork of intel EDKII (BSD License), OVMF build target
• For bhyve instead of Qemu
• Includes CSM BIOS emulation for non-EFI aware OS’s
• Currently in-house, being moved to public git repo
• Buildable on FreeBSD (GCC 4.6 or later), needs to be a port – bhyve folks need port creation help
• Serial console only: working on VGA emulation with VNC client
Networking:
• Virtio doesn’t support modern networking features
• One NIC, e1000 ((multiq, jumbo frames, TSO) under way
• e1000e (82580) dev emulation in progress
• each has thousands of registers, still working on them
Considering:
• user mode using WANProxy/libuinet
• simple kernel eth switch
Storage
• zvol GEOM-avoidance in place (mav@) – prevent geom from sniffing ZFS partition tables, so host will never see VM filesystem
• virtio todo: asynch block writes, add virtio SCSI
• Wanted: BSD-licensed sparse image tools for working with vmdk, qcow2, vhd, etc. Would be nice to point bhyve at a VMDK file and so “go!”
Future
• AMD-SVM
• Windows guest support (requires UEFI)
• Illumos doesn’t need UEFI, needs a real BIOS – use BIOS compat in UEFI
• ARM(64) chips have virtualization support, get bhyve to work on it.
• Save state/restore/migrate
• configuration file, as the command line is unwieldy for hierarchical info – use UCL because the ports people also use UCL
• Regression suite – bhyve supports lots of different hardware and operating systems, so we need to have automated testing
Other virtualization
• Virtualbox – FreeBSD is tracking very closely, 4.3.10 came out 25 March, port updated on 28 March.
• HyperV – 10.0, amd64 and i386 guest support
o Recent Azure image announcement
o Nobody in the FreeBSD community tracks Hyper-V, it could use a nanny
Luigi Rizzo on performance with device drivers
• One option – e1000 emulation, performance will be poor, will be slow
• Some emulation drivers fake TSO, etc
• No good solutions outside paravirtualization
• High performance = modify guest device driver to be virtualization-aware
• Luigi got 17GB/s using netmap with bhyve
Roger Pau Monne’ on Xen
• Changes in FreeBSD 10
o Vector callback for injecting event channel interrupts
o PV timer
o PV IPIs
o Add Xen support into GENERIC – can now use freebsd-update
o Sponsored by SpectraLogic and Citrix
• PVH domU
o Supported guest mode since 4.4
o Builds atop of the PVHVM work introduced in FreeBSD 10
o Half-merged into –current, some work remains
o Same speed as PVHVM, main difference is way it boots
o Not as intrusive as a traditional PV port
• PVM Dom0
o Xen side patches almost fully merged
o Main difference between PVH DomU and Dom0 is that on Dom0 FreeBSD needs to manage the hardware
o Add support for PIRQ (physical interrupts routed atop event channels)
o ACPI tables parsed by Dom0, and Xen must be made aware of the underlying devices
o Xen user-space control devices needed by the toolstack:
Privcmd – allows issuing hypercalls into Xen and mapping foreign domain memory from userspace
Evtchn – allows registering and receiving interrupts by user-space applications
• Big items remaining
o Add multiboot support to the FreeBSD bootloader – right now, you must use pxelinux or grub
o Improve if.xn – doesn’t work correctly with a NetBSD dom0, doesn’t work properly between guests on the same host, paravirtualized interface does not perform well yet.
• Hoping to have Xen work for FreeBSD 11
VirtIO/VMWare guest drivers by Brian Venteicher
• Work done over last year
• VirtIO: new
o Unmapped IO – block and SCSI
o Network multiqueue
o Random (entropy) device
o Initial console driver –can do multi-consoles, hotplug is so-so
• VirtIO: remaining
o Support missing devices – MMIO
o Non-x86 architectures
o SCSI multiqueue
o VirtIO version 1 specification – very similar to existing virtio
• VMWare
o Vmxnet3
Vmware provided driver, messy
OpenBSD imported their own vmx driver May 2013
Ported to FreeBSD 10.0
• TSO/LRO offload
• Multiqueue
To do: PVSCSI & VMWare tools
Device emulation in bhyve
• Most emulated in userspace usr.sbin/bhyve
• Kernel ones in vmm/io/ (PICs and timers)
• ISA-LPC – uart, rtc
• PCI
o Virtio
Block – storage
Net – tap
Rng – random entropy from /dev/random
o Ahci
o Pass-through
• Go through how virtio device drivers work. Interesting, lots of diagrams he should post, but way above my head, so I didn’t take too many notes
• Virtio random number generator
o Usr.sbin/bhyve/pci_virtio_rnd.c
o Guest rng driver requests 32-bit number to replenish its random pool
o FreeBSD /dev/random is non-blocking, using Yarrow and (soonish) Fortuna