...
The validation of the IEC project in CI/CD consists in several steps:
- Automatically deploy the IEC platform and perform a basic platform test
Run the validation project K8S conformance tests
Info Enabling this step is still work in progress for Release2. Logs for this steps are not yet pushed to Nexus, but the results can be seen in Jenkins at https://jenkins.akraino.org/view/iec/job/validation-enea-daily-master
- Install SEBA use-case
...
At the end of each job the the environment is cleaned up by destroying the cluster and the networks that were created.
The IEC project tests can be divided into 2 parts. The first part is about IEC platform tests which is used for testing the basic function of Kubernetes and Container Networking interface (CNI).
The second part is Feature tests which is about IEC functional test for verifying IEC user cases (e.g. SEBA).
1. Automatic deploy
The deployment is done both using Fuel@OPNFV and Compass installers.
The Installer performs the following step:
- For virtual deploys it creates the cluster VMs (using KVM). For baremetal deploys it provisions the nodes via IPMI using PXE boot
- Will clone the IEC repo and it will install the IEC platform on then as described in the Installation guide
- Will perform the Nginx Deployment test described below
Platform tests
In IEC project, there are 2 cases in Platform test. One is about service deployment test for verifying the basic K8S network function and deployment function end to end. The other is the K8S smoke check for checking K8s environments.
Deployment case
This test verifies the basic function of K8S by deploying a simple Nginx server. After the Nginx server is deployed, the test will issue a request to the server and verify the reply got from Nginx. The test can also be started by shell scripts nginx.sh.
...
The second case is K8S healthcheck which is used for checking the Kubernetes environments. It creates a Guestbook application that contains redis server, 2 instances of redis slave, frontend application, frontend service and redis master service and redis slave service. The test will write an entry into the guestbook application which will store the entry into the backend redis database. Application flow must work as expected and the data written must be available to read. In the end, you can directly run it from iec/src/foundation/scripts/functest.sh scripts shell.
Feature Tests
The feature tests are still going on developing now.
Test Step:
All the test cases has been integrated into IEC projects. You can start those tests by simple scripts on K8s-master node just as follows.
# source src/foundation/scripts/functest.sh <master-ip> #Functest case
# source src/foundation/scripts/nginx.sh #Nginx case
2. Conformance Test
| Info |
|---|
Enabling this step is still work in progress for Release2. Logs for this steps are not yet pushed to Nexus, but the results can be seen in Jenkins at https://jenkins.akraino.org/view/iec/job/validation-enea-daily-master/) |
Test Dashboards
Single pane view of how the test score looks like for the Blue print.
...
Second Validation Lab
For supporting IEC R2, the second validation lab is found in Arm internal Lab. The tasks log was pushed into community Nexus servers each day. The corresponding file is located at logs/arm-china folder. Please refer to following websites:
https://nexus.akraino.org/#welcome
The IEC cluster is tested using the validation project by running the conformance tests from the k8s layer. Tests are passing 100%
| Info |
|---|
Enabling the validation testing in CI is still work in progress |
SEBA integration with IEC is tested using the upstream CORD tests. The bugs uncovered by testing are documented in IEC Jiras.
...
The K8S conformance tests are ran using the Akraino Blueprint Validation project framework.
The Test inputs
The inputs are given via the IEC Jenkins job parameters, plus validation specific parameters:
- Bluprint name (iec)
- Layer to test (k8s)
- Version of the validation docker images to use
- Flag to run optional tests
Test Procedure
The tests are ran using the Akraino Blueprint Validation project procedure, by calling the Bluval Jenkins job
Expected output
All tests pass
| Code Block |
|---|
Status: Downloaded newer image for akraino/validation:k8s-latest
==============================================================================
Conformance
==============================================================================
Conformance.Conformance :: Run k8s conformance test using sonobuoy
==============================================================================
Run Sonobuoy Conformance Test | PASS |
------------------------------------------------------------------------------
Conformance.Conformance :: Run k8s conformance test using sonobuoy | PASS |
1 critical test, 1 passed, 0 failed
1 test total, 1 passed, 0 failed
==============================================================================
Conformance | PASS |
1 critical test, 1 passed, 0 failed
1 test total, 1 passed, 0 failed
==============================================================================
Output: /opt/akraino/results/k8s/conformance/output.xml
Log: /opt/akraino/results/k8s/conformance/log.html
Report: /opt/akraino/results/k8s/conformance/report.html |
3. Install SEBA use-case
The installation of the use-case is done as recommended by the upstream CORD community.
The Test inputs
The inputs are given via the Jenkins job parameters.
Test Procedure
The script found here is used to install SEBA.
Expected output
Installation in Jenkins is successful
Test Dashboards
Single pane view of how the test score looks like for the Blue print.
| Total Tests | Test Executed | Pass | Fail | In Progress |
|---|---|---|---|---|
| 2 | 2 | 2 | 0 | 1 |