...
These workloads will only be applied when the site is using the baremetal profile:
- Performance Profile
The PerformanceProfile CRD is the API of the openshift-performance-addon operator (https://github.com/openshift-kni/performance-addon-operators) that applies various performance tunings to cluster nodes to achieve lower latency.
The first step would be to install the operator. The operator manifest has the following bits -
Target Namespace - Namespace in which the operator will be installed.<insert-link>
Operator Group - Create an OperatorGroup CR in the target namespace - <insert-link>
Subscription - Create a subscription CR to subscribe the target namespace to the operator by tracking a channel, like so - <insert-link>The next step would be to create and apply the PerformanceProfile CRD.
An example can be found here <insert-link>.
This will automatically update the kernel by setting the kernel arguments as given in the yaml file with other parameters like the enablement of real time kernel, setting huge pages to 1G, reserving CPUs that will not be affected by any container workloads. - config-ran:
It adds the specific config to tune a CentOS node for realtime (assuming worker-0 by default). It has the following bits:- https://github.com/akraino-edge-stack/kni-blueprint-pae/blob/master/profiles/production.baremetal/02_cluster-addons/00_config-ran/00_machine-config-pool-worker-rt.yaml : creates a worker-rt MachineConfigPool object. This will hold all the specific MachineConfig objects that need to be applied for realtime
- https://github.com/akraino-edge-stack/kni-blueprint-pae/blob/master/profiles/production.baremetal/02_cluster-addons/00_config-ran/00_machine-config-worker-rt-env-vars.yaml : creates a file that will hold environment vars needed to tune realtime. It's a data-url blob, that will contain the definitions for huge pages. Once decoded, it has the following format:
hugepage_size_default=1G
hugepage_size=1G
hugepage_num=10
It can be overriden patching that manifest per site, the provided modified file needs to be data-url formatted. https://github.com/akraino-edge-stack/kni-blueprint-pae/blob/master/profiles/production.baremetal/02_cluster-addons/00_config-ran/01_machine-config-worker-rt.yaml: it provides the script used to enable RT kernel on worker-rt nodes. It adds a data-url encoded file, that can be overriden per site if it needs a different behaviour. It also creates a oneshot service, that will be executed once, and will install the realtime bits.
https://github.com/akraino-edge-stack/kni-blueprint-pae/blob/master/profiles/production.baremetal/02_cluster-addons/00_config-ran/02_override_tuning_deployment.yaml, https://github.com/akraino-edge-stack/kni-blueprint-pae/blob/master/profiles/production.baremetal/02_cluster-addons/00_config-ran/03_disable_tuning_deployment.yaml, https://github.com/akraino-edge-stack/kni-blueprint-pae/blob/master/profiles/production.baremetal/02_cluster-addons/00_config-ran/04_add_tuned_affinity.yaml: manifest to disable the management of tuning operator, so it can be modified per our needs and disable it on worker-rt nodes
https://github.com/akraino-edge-stack/kni-blueprint-pae/blob/master/profiles/production.baremetal/02_cluster-addons/00_config-ran/05_add_static_kubeletconfig.yaml: custom KubeletConfig that will enable Kubernetes static policy on all worker nodes: https://kubernetes.io/docs/tasks/administer-cluster/cpu-management-policies/
https://github.com/akraino-edge-stack/kni-blueprint-pae/blob/master/profiles/production.baremetal/02_cluster-addons/00_config-ran/06_add_static_policy_label.yaml: add a label on MachineConfigPool for workers, to enable that custom KubeletConfig
- https://github.com/akraino-edge-stack/kni-blueprint-pae/blob/master/profiles/production.baremetal/02_cluster-addons/00_config-ran/00_machine-config-pool-worker-rt.yaml : creates a worker-rt MachineConfigPool object. This will hold all the specific MachineConfig objects that need to be applied for realtime
- sriov-network-operator:
It adds the SRIOV network operator, that will add support for managing SRIOV interfaces inside Kubernetes cluster (https://github.com/openshift/sriov-network-operator) . The following manifest can be patched at site level to reflect the settings needed for the environment: https://github.com/akraino-edge-stack/kni-blueprint-pae/blob/master/profiles/production.baremetal/02_cluster-addons/01_sriov-network-operator/03_sriovnetwork_v1_sriovnetworknodepolicy_crd.yaml ptp-daemonset:
It adds components to enable PTP (precision time protocol). It has the following components:https://github.com/akraino-edge-stack/kni-blueprint-pae/blob/master/profiles/production.baremetal/02_cluster-addons/02_ptp-daemonset/01_ptp-machineconfig.yaml : enables PTP kernel module on nodes labelled as worker-ran
https://github.com/akraino-edge-stack/kni-blueprint-pae/blob/master/profiles/production.baremetal/02_cluster-addons/02_ptp-daemonset/05_configmap.yaml: configmap used to configure PTP. The configmap has two settings (PTP4.OPTIONS, PHC2CSYS.OPTIONS), that need to be configured properly per site. This manifest should be patched at site level, to change the NIC and the desired parameters.
- storage:
Adds Ceph storage to the Kubernetes cluster. It deploys and configures the Rook Ceph operator (https://github.com/rook/rook/blob/master/Documentation/ceph-quickstart.md), relying on directories on nodes to setup the storage space. Following manifests can be patched:- https://github.com/akraino-edge-stack/kni-blueprint-pae/blob/master/profiles/production.baremetal/02_cluster-addons/03_storage/02_ceph_cluster.yaml: adds specific settings for the ceph cluster
- https://github.com/akraino-edge-stack/kni-blueprint-pae/blob/master/profiles/production.baremetal/02_cluster-addons/03_storage/03_ceph_storage_class.yaml: it defines an CephBlockPool storage class to be used on pods.
- https://github.com/akraino-edge-stack/kni-blueprint-pae/blob/master/profiles/production.baremetal/02_cluster-addons/03_storage/04_ceph_storage_filesystem.yaml: it defines a CephFilesystem storage class to be used on pods.
- https://github.com/akraino-edge-stack/kni-blueprint-pae/blob/master/profiles/production.baremetal/02_cluster-addons/03_storage/05_ceph_image_registry_pvc.yaml: using the defined rook-filesystem class, creates a PersistenVolumeClaim, to be used as storage for Image Registry.
- https://github.com/akraino-edge-stack/kni-blueprint-pae/blob/master/profiles/production.baremetal/02_cluster-addons/03_storage/02_ceph_cluster.yaml: adds specific settings for the ceph cluster
- nodes:
It adds custom labels to specific worker nodes. Allows to give worker-rt, worker-ran and cpumanager-enabled labels at node level. It needs to be patched per site, as the node names will change.
Performance Profile:
The PerformanceProfile CRD is the API of the openshift-performance-addon operator (https://github.com/openshift-kni/performance-addon-operators) that applies various performance tunings to cluster nodes to achieve lower latency. An example for a PerformanceProfile CRD can be found here <insert-link>. This will automatically update the kernel by setting the kernel arguments as given in the yaml file with other parameters.