Introduction
This page is about work in progress regarding REC validation and testing, more information should be added in the near future.
The external validation labs in AT&T lab (Middletown, NJ) had successfully validating REC installation which identified by HP Gen 10 and Nokia OE clusters.
Post-install validation
These test cases is to identify hardware and network issues as well as identify areas for REC enhancements.
1.Verify that the deployment completed successfully.
Enter the following command:
tail -f /srv/deployment/log/bootstrap.log
In the command output the following text is present: Installation complete, Installation Succeeded.
2. Docker Version Test:
#docker --version
Expected Output: Docker version 18.09.2, build 6247962
3. Kubernetes Cluster – check Health/validation
# kubectl get pods --all-namespaces
Expected Output: status of all the pods should be running
4.Verify that CaaS-required system services are in active (running) state.
Enter the following commands:
systemctl status --no-pager docker.service status
systemctl status --no-pager kubelet.service status
Expected Output: both statuses should be shown as active
Step example:
● docker.service - Docker Application Container Engine
Loaded: loaded (/etc/systemd/system/docker.service; enabled; vendor preset: disabled)
Active: active (running)
5.Verify that CaaS nodes operate fine.
Enter the following commands:
kubectl get no --no-headers | grep -v Ready
The command output shows nothing.
kubectl get no --no-headers | wc -l
The command output shows the number of CaaS nodes.
6.Verify that CaaS components are running with no errors.
Enter the following command:
kubectl get po --no-headers --namespace=kube-system --field-selector status.phase!=Running
The command output shows nothing.
7.Verify that CaaS is capable to install applications via its package manager (Helm).
- Docker registry is running and images can be downloaded:
· image=$(docker images -f 'reference=*/rec/hypercube' --format="{{.Repository}}:{{.Tag}}"); docker rmi $image; docker pull $image
The command output is Status: Downloaded newer image for …
- Chart repository is up and running:
· curl -sS -XGET --cacert /etc/chart-repo/ssl/ca.pem --cert /etc/chart-repo/ssl/chart-repo?.pem --key /etc/chart-repo/ssl/chart-repo?-key.pem https://chart-repo.kubesystem.svc.rec.io:8088/charts/index.yaml
The command output is a yaml file.
- Helm is able to run a sample application:
· helm list
The command output is caas-infra.
Verifying Deployment
A post installation verification is required to ensure that all nodes and services were properly deployed.
You need to establish an ssh connection to the controller’s VIP address and login with administrative rights.
Example Needed
1. Verify Deployment Success.
Enter the following command:
tail /srv/deployment/log/bootstrap.log
You should see: Installation complete, Installation Succeeded.
2. Docker Version Test:
#docker --version
Expected Output: Docker version 18.09.2, build 6247962
3. Kubernetes Cluster – check Health/validation
# kubectl get pods --all-namespaces
Expected Output: status of all the pods should be running
4. Confirm active state of required services
Enter the following commands:
systemctl status --no-pager docker.service
systemctl status --no-pager kubelet.service
Example
systemctl status --no-pager docker.service* docker.service - Docker Application Container Engine
Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
Active: active (running)
5. Verify node functionality
Enter the following commands:
kubectl get no --no-headers | grep -v Ready
Output: The command output shows nothing.
kubectl get no --no-headers | wc -l
Output: The command output shows the number of CaaS nodes.
6. Verify Components
Enter the following command:
kubectl get po --no-headers --namespace=kube-system --field-selector status.phase!=Running
Output: The command output shows nothing.
7. Confirm Package Manager Status (Helm)
- Docker registry is running, and images can be downloaded:
image=$(docker images -f 'reference=*/rec/hypercube' --format="{{.Repository}}:{{.Tag}}"); docker rmi $image; docker pull $image
Output: Status: Downloaded newer image for …
- Chart repository is up and running: (The curl command below is really one line.)
curl -sS -XGET --cacert /etc/chart-repo/ssl/ca.pem --cert /etc/chart-repo/ssl/chart-repo?.pem
--key /etc/chart-repo/ssl/chart-repo?-key.pem https://chart-repo.kubesystem.svc.rec.io:8088/charts/index.yaml
Output: output is a yaml file.
- Helm is able to run a sample application:
helm list
Output: rec-infra.
Deployment Failures
Sometimes failures happen, usually do to misconfigurations or incorrect addresses entered.
To re-launch a failed deployment
There are two options for redeploying.
- /opt/nokia/cmframework/scripts/bootstrap.sh /opt/nokia/installer-ui/user_config/user_config.yaml (the user_config.yaml file is loaded from installer GUI)
- Openvt -s -w /opt/start-menu/start_menu.sh &/
Note: In some cases, modifications to the user_config.yaml may be necessary to resolve a failure.
If re-deployment is not possible, then the deployment will need to be started from booting to the REC.iso
Additional Testing
More testing cases will be added in the near future. For example,
1.Installing RIC on top of REC
2. RIC robot test
3. Nanobot