gistfile1.txt

Hello all

Miguel (in CC) and I have been working on adding macvtap support to Kubevirt. This allows setting up a macvtap interface for a VM that is directly attached to one of the node interfaces.

Why?

Macvtap interfaces are commonly used to simplify bridged virtualized networking. In the case of Kubevirt, the benefit is two-fold; it not only simplifies the networking at host level, but also at pod level, avoiding the extra bridge setup required there in a standard setup.

How?

The proposal encompasses changes in Kubevirt, as well as  a new macvtap device plugin for K8s and a macvtap cni.

The prerequisite to all this work is a relatively recent change in libvirt that adds support to use existing pre-configured macvtap interfaces [1]. Those interfaces are expressed in the libvirt's domain xml as interfaces of type 'ethernet' with non-managed target [2].

Libvirt relies on having 'rw' access to the tap character device associated to the macvtap interface. By updating the pod's cgroups specification, the macvtap device plugin [3] creates the interface on the node and ensures the tap device is accessible from the virtlauncher pod.

Libvirt also relies on having access to the macvtap interface itself. For this, and also for additional configuration of the interface (like MAc address, IPAM, renaming, etc) a macvtap cni is used [4]. 

Multus is used to associate a Network Attachment Definition to the resource allocated by the macvtap device plugin, which is the macvtap interface itself. It's name will be passed as DeviceID to the macvtap cni. From there, the macvtap cni can rename, configure and move the macvtap interface to the virtlauncher pod.

Finally, there are the required changes on Kubevirt to wire the macvtap interface to the domain xml definition [5].

Proposal

The proposal overall is both for adding support in Kubevirt for macvtap interfaces, as well as hosting within Kubevirt project the new macvtap device plugin and cni. The reasoning behind it is that they mostly make sense in the context of virtualization (because of tap devices) on top of K8s (because of the need for the device plugin).

Any feedback is welcome.