Table of Contents
Introduction
KNI PAE is tested against 3 platforms: libvirt, aws and baremetal. A typical test consist on:
...
Overall Test Architecture
- Our tests are performed on Akraino Jenkins at https://jenkins.akraino.org/view/kni/ .
...
Following the different tests per platform are explained
kni-installer-verify-binary, kni-installer-verify-installer
...
Following there is the architecture *:
* Green=Akraino dependencies, Orange=system under test, Blue=components created
kni-blueprint-pae-verify-deploy-aws
...
Following there is the architecture *:
* Green=Akraino dependencies, Orange=system under test, Blue=components created
kni-blueprint-pae-verify-deploy-baremetal
...
Following there is the architecture *:
* Green=Akraino dependencies, Orange=system under test, Blue=components created
Test Framework
For the moment, the deployment tests are just consisting on deploying a cluster and running workloads on top of it, checking that they are successfully deployed.
In the future, validation tests will be executed: Testings in validation lab are executed manually for the moment
Automated test deploy
The validation of KNI-PAE blueprint is currently based on deploying our cluster and applying workloads on top of it, every time that there is a change on the blueprint repo .
Additionally test can be run manually to verify deployment at a certain stage.
Validation on the blueprint is done for AWS, GCP and virtual baremetal.
At the end of the tests, the deployed cluster is destroyed.
Sample tests can be seen at:
https://jenkins.akraino.org/view/kni/job/kni-blueprint-pae-verify-deploy-aws/
https://jenkins.akraino.org/view/kni/job/kni-blueprint-pae-verify-deploy-baremetal/
https://jenkins.akraino.org/view/kni/job/kni-blueprint-pae-verify-deploy-gcp/
Conformance Test
This is performed through Akraino Blueprint Validation project framework.
However the tests are currently launched manually after a cluster is deployed, integration with the CI is still pending, as we are hitting issues with the framework itself:
https://jira.akraino.org/projects/VAL/issues/VAL-108
https://jira.akraino.org/projects/VAL/issues/VAL-109
https://jirahttps://gerrit.akraino.org/rprojects/adminVAL/repos/validationissues/VAL-110
As we use OpenShift, we cannot use the standard k8s conformance tests, because they are aimed for upstream Kubernetes and not for OpenShift.
As an alternative, we can run the openshift test validation suite: https://github.com/openshift/origin/blob/master/test/extended/conformance-k8s.sh
This is similar as the sonobuoy one, launching a set of e2e tests to validate that the cluster is deployed and works at a functional level. Those are the collected results:
https://logs.akraino.org/redhat-kni/bluval_results/blueprint-pae/20200505-104443/out.log
Security Test
kube-hunter test was applied on the cluster:
https://logs.akraino.org/redhat-kni/bluval_results/blueprint-pae/20200423-071856/results/k8s/kube-hunter/Kube-Hunter.Kube-Hunter/cluster.log → In OpenShift we expose our version and we do not have control in configuration level for hiding it
https://logs.akraino.org/redhat-kni/bluval_results/blueprint-pae/20200423-071856/results/k8s/kube-hunter/Kube-Hunter.Kube-Hunter/pod.log → CAP_RAW is enabled by default in OpenShift, and same with the other Access Errors. It will need some advanced configuration to bypass these errors but by default the clusters will deploy with these security warnings.
Test Test Dashboards
https://jenkins.akraino.org/view/kni/