...
It has made great benefits in many areas, such as Hadoop AcceletationAcceleration, OpenSSL Integration, SDN and NFV Solutions Boost and so on.
...
With above support, QAT makes it easier for developers to integrate the accelerators in their designs and thus decrease the development time. And it can increases increase business flexibility by offering solutions that best fit the changing business requirements. It also frees up the valuable cycles on processors and allows it to perform value-added functionality.
What's more, QAT provides a uniform means of communication between accelerators, applications, and acceleration technologies. Due to this, the resources are managed more productively. Then It can boost application throughput, by reducing the demand on the platform and maximizes maximizing the CPU utilization.
Gaps
First of all, in a Kubernetes cluster, if we need utilize the QAT card and assign its vf to a container, we will compile the QAT driver on the host and deploy the QAT device plugin. After the QAT device plugin register successfully, a ListAndWatch function is for the Kubelet to Discover the devices and their properties as well as notify of any status change (devices become unhealthy). The list of devices is returned as an array of all devices description information (ID, health status) of the resource. Kubelet records this resource and its corresponding number of devices to node.status.capacity/allocable and updates it to apiserver.
...
- Add necessary information to swagger.json which is an add on for kubenetes API used in Kubevirt
... "qats": { ... "v1.QAT": { |
|---|
- Add Feature Gate to webhook validation service to config it in configmap
... if spec.Domain.Devices.QATs != nil && !config.QATPassthroughEnabled() { ... |
|---|
...
Non-privileged
To meet the Kubernetes and Kubevirt community specifications, the pod should be non-privileged. So we should mount the assigned QAT pci device to the VM through the interfaces Kubevirt provided.
| if util.IsQATVMI(vmi) { for _, qat := range vmi.Spec.Domain.Devices.QATs { requestResource(&resources, qat.DeviceName) } } |
|---|
This will call the Kubevirt to mount necessary resources for QAT into the VM, such as /sys/devices/.
...
Fundamentally, Kubevirt create a Kubernetes CRD to hold some resource configurations that fit a libvrit instance. So actually to assign a QAT vf into a VM is to create a libvirt and insert the specific device into hostdev block. (In libvirt, hostdev label means a plain host device assignment with all its limitations)
| <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x3d' slot='0x02' function='0x2'/> </source> <alias name='hostdev0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </hostdev> |
|---|
So when a QAT vf resource is required, I call the Kubevirt method to get the device id of the assigned QAT and integrate it into libvirt domain.
// Append HostDev to libvirt DomXML if QAT is required qatPCIAddresses := append([]string{}, c.QATDevices...) ... |
|---|
Example
Based on the Kubevirt official example VMI file, to make use of my QAT version, after building its docker images and uploading to docker, we should open the QAT feature gate through configmap.
| apiVersion: v1 kind: ConfigMap metadata: name: kubevirt-config namespace: kubevirt labels: kubevirt.io: "" data: feature-gates: "QAT" |
|---|
And the key point of QAT VMI:
Conclusion
The patch now has not been merged into the Kubevirt Project and is in review now. (PR: https://github.com/kubevirt/kubevirt/pull/2980)
...