This blueprint is part of the Kubernetes-Native Infrastructure for Edge family. All blueprints in this family follow the same installation guide, so please see the KNI family's User Documentation.
KNI PAE specific workloads
The KNI blueprints share same installation procedure, until the cluster has been successfully deployed. However, they have different workloads applied on it (knictl apply_workloads step). Following is a detail of the applied workloads and how to adapt for your site.
All-platforms workloads
- Kubevirt (https://kubevirt.io):
It allows to spin up VMs using Kubernetes platform.
To customize, you could patch the manifests from https://github.com/akraino-edge-stack/kni-blueprint-pae/tree/master/base/02_cluster-addons/00_kubevirt - CNI macvlan (https://intel.github.io/multus-cni/doc/how-to-use.html):
It adds a NetworkAttachmentDefinition configuration for Multus, creating a macvlan. It will allow pods to use the network interface created there.
To customize, you could patch the manifests from https://github.com/akraino-edge-stack/kni-blueprint-pae/tree/master/base/02_cluster-addons/01_cni-macvlan
You may need to update the configuration, replacing the full config string - CNI ipvlan:
Similar as previous one, but with ipvlan configuration. You can patch manifests on https://github.com/akraino-edge-stack/kni-blueprint-pae/tree/master/base/02_cluster-addons/02_cni-ipvlan - Node feature discovery (https://github.com/kubernetes-sigs/node-feature-discovery):
It adds NodeFeatureDiscovery component to the Kubernetes cluster. It performs a set of checks in nodes, and adds anotations with the information it can find. It will report hardware, software, network facts, etc...
To customize, you could patch the manifests from https://github.com/akraino-edge-stack/kni-blueprint-pae/tree/master/base/02_cluster-addons/03_nfd
Baremetal workloads
These workloads will only be applied when the site is using the baremetal profile:
- config-ran:
It adds the specific config to tune a CentOS node for realtime (assuming worker-0 by default). It has the following bits:
- https://github.com/akraino-edge-stack/kni-blueprint-pae/blob/master/profiles/production.baremetal/02_cluster-addons/00_config-ran/00_machine-config-pool-worker-rt.yaml : creates a worker-rt MachineConfigPool object. This will hold all the specific MachineConfig objects that need to be applied for realtime
- https://github.com/akraino-edge-stack/kni-blueprint-pae/blob/master/profiles/production.baremetal/02_cluster-addons/00_config-ran/00_machine-config-worker-rt-env-vars.yaml : creates a file that will hold environment vars needed to tune realtime. It's a data-url blob, that will contain the definitions for huge pages. Once decoded, it has the following format:hugepage_size_default=1G
hugepage_size=1G
hugepage_num=10
It can be overriden patching that manifest per site, the provided modified file needs to be data-url formatted.
- https://github.com/akraino-edge-stack/kni-blueprint-pae/blob/master/profiles/production.baremetal/02_cluster-addons/00_config-ran/01_machine-config-worker-rt.yaml: it provides the script used to enable RT kernel on worker-rt nodes. It adds a data-url encoded file, that can be overriden per site if it needs a different behaviour. It also creates a oneshot service, that will be executed once, and will install the realtime bits.
- https://github.com/akraino-edge-stack/kni-blueprint-pae/blob/master/profiles/production.baremetal/02_cluster-addons/00_config-ran/02_override_tuning_deployment.yaml, https://github.com/akraino-edge-stack/kni-blueprint-pae/blob/master/profiles/production.baremetal/02_cluster-addons/00_config-ran/03_disable_tuning_deployment.yaml, https://github.com/akraino-edge-stack/kni-blueprint-pae/blob/master/profiles/production.baremetal/02_cluster-addons/00_config-ran/04_add_tuned_affinity.yaml: manifest to disable the management of tuning operator, so it can be modified per our needs and disable it on worker-rt nodes
- https://github.com/akraino-edge-stack/kni-blueprint-pae/blob/master/profiles/production.baremetal/02_cluster-addons/00_config-ran/05_add_static_kubeletconfig.yaml: custom KubeletConfig that will enable Kubernetes static policy on all worker nodes: https://kubernetes.io/docs/tasks/administer-cluster/cpu-management-policies/
- https://github.com/akraino-edge-stack/kni-blueprint-pae/blob/master/profiles/production.baremetal/02_cluster-addons/00_config-ran/06_add_static_policy_label.yaml: add a label on MachineConfigPool for workers, to enable that custom KubeletConfig - sriov-network-operator:
It adds the SRIOV network operator, that will add support for managing SRIOV interfaces inside Kubernetes cluster (https://github.com/openshift/sriov-network-operator) . The following manifest can be patched at site level to reflect the settings needed for the environment: https://github.com/akraino-edge-stack/kni-blueprint-pae/blob/master/profiles/production.baremetal/02_cluster-addons/01_sriov-network-operator/03_sriovnetwork_v1_sriovnetworknodepolicy_crd.yaml ptp-daemonset:
It adds components to enable PTP (precision time protocol). It has the following components:
- https://github.com/akraino-edge-stack/kni-blueprint-pae/blob/master/profiles/production.baremetal/02_cluster-addons/02_ptp-daemonset/01_ptp-machineconfig.yaml : enables PTP kernel module on nodes labelled as worker-ran
- https://github.com/akraino-edge-stack/kni-blueprint-pae/blob/master/profiles/production.baremetal/02_cluster-addons/02_ptp-daemonset/05_configmap.yaml: configmap used to configure PTP. The configmap has two settings (PTP4.OPTIONS, PHC2CSYS.OPTIONS), that need to be configured properly per site. This manifest should be patched at site level, to change the NIC and the desired parameters.- storage:
Adds Ceph storage to the Kubernetes cluster. It deploys and configures the Rook Ceph operator (https://github.com/rook/rook/blob/master/Documentation/ceph-quickstart.md), relying on directories on nodes to setup the storage space. Following manifests can be patched:
- https://github.com/akraino-edge-stack/kni-blueprint-pae/blob/master/profiles/production.baremetal/02_cluster-addons/03_storage/02_ceph_cluster.yaml: adds specific settings for the ceph cluster
- https://github.com/akraino-edge-stack/kni-blueprint-pae/blob/master/profiles/production.baremetal/02_cluster-addons/03_storage/03_ceph_storage_class.yaml: it defines an CephBlockPool storage class to be used on pods.
- https://github.com/akraino-edge-stack/kni-blueprint-pae/blob/master/profiles/production.baremetal/02_cluster-addons/03_storage/04_ceph_storage_filesystem.yaml: it defines a CephFilesystem storage class to be used on pods.
- https://github.com/akraino-edge-stack/kni-blueprint-pae/blob/master/profiles/production.baremetal/02_cluster-addons/03_storage/05_ceph_image_registry_pvc.yaml: using the defined rook-filesystem class, creates a PersistenVolumeClaim, to be used as storage for Image Registry. - nodes:
It adds custom labels to specific worker nodes. Allows to give worker-rt, worker-ran and cpumanager-enabled labels at node level. It needs to be patched per site, as the node names will change.