Page History

This blueprint is part of the Kubernetes-Native Infrastructure for Edge family. All blueprints in this family follow the same installation guide, so please see the KNI family's User Documentation.

...

These workloads will only be applied when the site is using the baremetal profile:

• Performance Profile
  config-ran:
  It adds the specific config to tune a CentOS node for realtime (assuming worker-0 by default). It has the following bits:

  The PerformanceProfile CRD is the API of the openshift-performance-addon operator (

  https://github.com/

  akraino

  openshift-

  edge-stack/

  kni

  -blueprint-pae/blob/master/profiles/production.baremetal/02_cluster-addons/00_config-ran/00_machine-config-pool-worker-rt.yaml : creates a worker-rt MachineConfigPool object. This will hold all the specific MachineConfig objects that need to be applied for realtime
  https://github.com/akraino-edge-stack/kni-blueprint-pae/blob/master/profiles/production.baremetal/02_cluster-addons/00_config-ran/00_machine-config-worker-rt-env-vars.yaml : creates a file that will hold environment vars needed to tune realtime. It's a data-url blob, that will contain the definitions for huge pages. Once decoded, it has the following format:
  hugepage_size_default=1G
  hugepage_size=1G
  hugepage_num=10
  It can be overriden patching that manifest per site, the provided modified file needs to be data-url formatted.
  

  /performance-addon-operators) that applies various performance tunings to cluster nodes to achieve lower latency.
  
  The first step would be to install the operator. The operator manifest has the following bits -
  Target Namespace - Namespace in which the operator will be installed - https://github.com/akraino-edge-stack/kni-blueprint-pae/blob/master/profiles/production.baremetal/02_cluster-addons/

  00

  05_

  config

  performance-

  ran

  operator/01_

  machine-config-worker-rt.yaml: it provides the script used to enable RT kernel on worker-rt nodes. It adds a data-url encoded file, that can be overriden per site if it needs a different behaviour. It also creates a oneshot service, that will be executed once, and will install the realtime bits.
  

  namespace.yaml
  
  Operator Group - Create an OperatorGroup CR in the target namespace - https://github.com/akraino-edge-stack/kni-blueprint-pae/blob/master/profiles/production.baremetal/02_cluster-addons/

  00

  05_

  config

  performance-

  ran

  operator/02_

  override_tuning_deployment.yaml,

  perf-operatorgroup.yaml
  
  Subscription - Create a subscription CR to subscribe the target namespace to the operator by tracking a channel, like so - https://github.com/akraino-edge-stack/kni-blueprint-pae/blob/master/profiles/production.baremetal/02_cluster-addons/

  00

  05_

  config

  performance-

  ran

  operator/03_

  disable_tuning_deployment.yaml

  perf-sub.yaml

  , https://github.com/akraino-edge-stack/kni-blueprint-pae/blob/master/profiles/production.baremetal/02_cluster-addons/00_config-ran/04_add_tuned_affinity.yaml: manifest to disable the management of tuning operator, so it can be modified per our needs and disable it on worker-rt nodes
  

  The next step would be to create and apply the PerformanceProfile CRD.
  An example can be found here - 

  https://github.com/akraino-edge-stack/kni-blueprint-pae/blob/master/profiles/production.baremetal/02_cluster-addons/

  00

  05_

  config

  performance-

  ran https://kubernetes.io/docs/tasks/administer-cluster/cpu-management-policies/
  https://github.com/akraino-edge-stack/kni-blueprint-pae/blob/master/profiles/production.baremetal/02_cluster-addons/00_config-ran/06_add_static_policy_label.yaml: add a label on MachineConfigPool for workers, to enable that custom KubeletConfig

  operator/

  05_add_static_kubeletconfig.yaml: custom KubeletConfig that will enable Kubernetes static policy on all worker nodes:

  04_perfprofile-conf.yaml. 
  This will automatically update the kernel by setting the kernel arguments as given in the yaml file with other parameters like the enablement of real time kernel, setting huge pages to 1G, reserving CPUs that will not be affected by any container workloads.

  
  
  

• sriov-network-operator:
  It adds the SRIOV network operator, that will add support for managing SRIOV interfaces inside Kubernetes cluster (https://github.com/openshift/sriov-network-operator) . The following manifest can be patched at site level to reflect the settings needed for the environment: https://github.com/akraino-edge-stack/kni-blueprint-pae/blob/master/profiles/production.baremetal/02_cluster-addons/01_sriov-network-operator/03_sriovnetwork_v1_sriovnetworknodepolicy_crd.yaml
  
  

• ptp-daemonset:
  It adds components to enable PTP (precision time protocol). It has the following components:

  • https://github.com/akraino-edge-stack/kni-blueprint-pae/blob/master/profiles/production.baremetal/02_cluster-addons/02_ptp-daemonset/01_ptp-machineconfig.yaml : enables PTP kernel module on nodes labelled as worker-ran
    
    

  • https://github.com/akraino-edge-stack/kni-blueprint-pae/blob/master/profiles/production.baremetal/02_cluster-addons/02_ptp-daemonset/05_configmap.yaml: configmap used to configure PTP. The configmap has two settings (PTP4.OPTIONS, PHC2CSYS.OPTIONS), that need to be configured properly per site. This manifest should be patched at site level, to change the NIC and the desired parameters.
    
    

• storage:
  Adds Ceph storage to the Kubernetes cluster. It deploys and configures the Rook Ceph operator (https://github.com/rook/rook/blob/master/Documentation/ceph-quickstart.md), relying on directories on nodes to setup the storage space. Following manifests can be patched:
  
  • https://github.com/akraino-edge-stack/kni-blueprint-pae/blob/master/profiles/production.baremetal/02_cluster-addons/03_storage/02_ceph_cluster.yaml: adds specific settings for the ceph cluster
    
    
  • https://github.com/akraino-edge-stack/kni-blueprint-pae/blob/master/profiles/production.baremetal/02_cluster-addons/03_storage/03_ceph_storage_class.yaml: it defines an CephBlockPool storage class to be used on pods.
    
    
  • https://github.com/akraino-edge-stack/kni-blueprint-pae/blob/master/profiles/production.baremetal/02_cluster-addons/03_storage/04_ceph_storage_filesystem.yaml: it defines a CephFilesystem storage class to be used on pods.
    
    
  • https://github.com/akraino-edge-stack/kni-blueprint-pae/blob/master/profiles/production.baremetal/02_cluster-addons/03_storage/05_ceph_image_registry_pvc.yaml: using the defined rook-filesystem class, creates a PersistenVolumeClaim, to be used as storage for Image Registry.
    
    
• nodes:
  It adds custom labels to specific worker nodes. Allows to give worker-rt, worker-ran and cpumanager-enabled labels at node level. It needs to be patched per site, as the node names will change.