This document describes the use of Public Cloud Edge Interface (PCEI) implemented based on Edge Multi-Cluster Orchestrator (EMCO) for deployment of Public Cloud Edge (PCE) Apps from multiple clouds (Azure and AWS), deployment of a 3rd-Party Edge (3PE) App (an implementation of ETSI MEC Location API App), as well as the end-to-end operation of the deployed PCE Apps using simulated Low Power Wide Area (LPWA) IoT client.
The end-to-end validation environment and flow of steps is shown below:
Description of components of the end-to-end validation environmen:
Note that P1, P3, P5/P7 Reference Points are shown to illustrate alignment with general PCEI architecture.
The end-to-end PCEI validation steps are described below:
For performing Steps 1 and 2 please refer to the PCEI R4 Installation Guide.
Please refer to the PCEI R4 Installation Guide to deploy EMCO and Edge K8S Clusters.
Due to current limitations in EMCO functionality, it is recommended that VNC is used to access the EMCO UI via a Web Browser as shown in the diagram below:
In order to be able to access the EMCO cluster and the Edge K8S cluster via ssh directly from a laptop as shown in the diagram above, please copy the ssh key used in the PCEI Installation to you local machine:
### copy id_rsa key from deployment host to be able to ssh to eco and cluster vm’s sftp onaplab@10.121.1.12 get ~/.ssh/id_rsa pcei-emco ### ssh to VMs ssh -i pcei-emco onaplab@10.121.7.145 |
In order to access EMCO UI please install VNC Viewer on your local laptop and configure it to connect to the VNC server that was installed on the EMCO Host Server (refer to the PCEI R4 Installation Guide for VNC server installation instructions).
Connect to the VNC server, start the Browser and point it to the EMCO VM (amcop-vm-01) IP address and port 30480:
http://10.121.7.145:30480 |
You should be able to connect to EMCO UI as shown below:
The following Network Controllers must be provisioned in EMCO:
### Add controllers rsync: host : rsync port: 9041 type: < leave blank, not required.> priority: < leave blank, not required> ovnaction: host: ovnaction port: 9053 type: action priority: 1 |
To provision Network Controllers, please use the parameters shown above and EMCO UI "Controllers → Register Controllers" tab:
Fill in the parameters for the "rsync" and the "ovnaction" controllers:
To register the two Edge K8S Clusters created during PCEI installation (refer to PCEI R4 Installation Guide), the cluster configuration files need to be copied from the corresponding VMs to the machine that is used to run the VNC server to access EMCO UI. Please make sure that the ssh key used to access the VMs is present on the machine that runs the VNC server. The example below assumes that the VNC server is running on the Host Server used to deploy EMCO and Edge Cluster VMs
# Determine VM IP addresses: [onaplab@os12 ~]$ sudo virsh list --all Id Name State ---------------------------------------------------- 6 amcop-vm-01 running 9 edge_k8s-1 running 10 edge_k8s-2 running [onaplab@os12 ~]$ sudo virsh domifaddr edge_k8s-1 Name MAC address Protocol Address ------------------------------------------------------------------------------- vnet1 52:54:00:19:96:72 ipv4 10.121.7.152/27 [onaplab@os12 ~]$ sudo virsh domifaddr edge_k8s-2 Name MAC address Protocol Address ------------------------------------------------------------------------------- vnet2 52:54:00:c0:47:8b ipv4 10.121.7.146/27 sftp -i pcei-emco onaplab@10.121.7.146 cd .kube get config kube-config-edge-k8s-2 sftp -i pcei-emco onaplab@10.121.7.152 cd .kube get config kube-config-edge-k8s-1 |
Using EMCO UI "Cluster → Register Cluster Provider" tabs, provision Cluster Providers for EDGE-K8S-1 and EDGE-K8S-2 clusters:
Click on the Cluster Provider and click on "Add Cluster". Fill in the Cluster Name and add the config file downloaded earlier. Repeat for the two clusters EDGE-K8S-1 and EDGE-K8S-2:
You should see the following result:
At this point the edge clusters have been registered with the orchestrator and are redy for placing PCE and 3PE apps.
The deployment of Azure IoT Edge cloud native application as a PCE involves the following steps:
Login to your Azure Portal and add Subscription, IoT Hub and Container Registry Resources:
Provision IoT Hub:
Provision IoT Edge under IoT Hub:
Copy the "Primary Connection String" from the IoT Edge parameters. This string will be used later in the values.yaml file for the Azure IoT Edge Helm Charts.
Use this link for information on developing custom software modules for Azure IoT Edge:
The example below is optional. It shows how to build a custom module for Azure IoT Edge to read and decode Low Power IoT messages from a simulated LPWA IoT device. Follow the above link to:
The steps below show how to build custom IoT module for Azure IoT Edge using "LoRaEdgeSolution" code from PCEI repo:
Download PCEI repo to the machine that has VSC and Docker installed (per above instructions):
git clone "https://gerrit.akraino.org/r/pcei" cd pcei ls -l total 0 drwxr-xr-x 8 oberzin staff 256 Dec 24 15:44 LoRaEdgeSolution drwxr-xr-x 3 oberzin staff 96 Dec 24 15:44 iotclient drwxr-xr-x 5 oberzin staff 160 Dec 24 15:44 locationAPI |
Using VSC open the LoRaEdgeSolution folder that was downloaded from PCEI repo.
Add required credentials for Azure Container Registry (ACR) using .env file.
Build and push the solution to ACR as shown below. Righ-click on "deployment.template.json":
The docker image for the custom module should now be visible in Azure Cloud ACR:
Use this link to understand the architecture of Azure IoT Edge cloud native application:
https://microsoft.github.io/iotedge-k8s-doc/architecture.html
For the purposes of this document we use the Host Server on which EMCO has been deployed and on which we run the VNC server to package Helm charts for Azure IoT Edge.
Clone Azure IoT Edge from github:
git clone https://github.com/Azure/iotedge cd iotedge/kubernetes/charts/ mkdir azureiotedge1 cp -a edge-kubernetes/. azureiotedge1/ cd azureiotedge1 ls -al total 24 drwxrwxr-x. 3 onaplab onaplab 79 Dec 24 13:14 . drwxrwxr-x. 5 onaplab onaplab 77 Dec 24 13:14 .. -rw-rw-r--. 1 onaplab onaplab 137 Dec 24 13:02 Chart.yaml -rw-rw-r--. 1 onaplab onaplab 333 Dec 24 13:02 .helmignore drwxrwxr-x. 2 onaplab onaplab 220 Dec 24 13:02 templates -rw-rw-r--. 1 onaplab onaplab 14226 Dec 24 13:02 values.yaml |
Modify values.yaml file to specify the "Primary Connection String" from Azure Cloud generated during IoT Hub/IoT Edge provisioning:
vi values.yaml # Change the line below and save the file provisioning: source: "manual" deviceConnectionString: "PASTE PRIMARY CONNECTION STRING FROM AZURE IOT HUB / IOT EDGE SCREEN" #dynamicReprovisioning: false |
Create a tar file with Azure IoT Edge Helm Charts:
# Make sure to change to the "charts" directory cd .. pwd /home/onaplab/iotedge/kubernetes/charts # zip the "azureiotedge1" directory. Be sure to use "azureiotedge1.zip" file name. tar -czvf azureiotedge1.zip azureiotedge1/ azureiotedge1/ azureiotedge1/.helmignore azureiotedge1/Chart.yaml azureiotedge1/templates/ azureiotedge1/templates/NOTES.txt azureiotedge1/templates/_helpers.tpl azureiotedge1/templates/edge-rbac.yaml azureiotedge1/templates/iotedged-config-secret.yaml azureiotedge1/templates/iotedged-deployment.yaml azureiotedge1/templates/iotedged-proxy-config.yaml azureiotedge1/templates/iotedged-pvc.yaml azureiotedge1/templates/iotedged-service.yaml azureiotedge1/values.yaml [onaplab@os12 charts]$ ls -al total 12 drwxrwxr-x. 5 onaplab onaplab 102 Dec 24 13:23 . drwxrwxr-x. 4 onaplab onaplab 31 Dec 24 13:02 .. drwxrwxr-x. 3 onaplab onaplab 79 Dec 24 13:14 azureiotedge1 -rw-rw-r--. 1 onaplab onaplab 8790 Dec 24 13:23 azureiotedge1.zip drwxrwxr-x. 3 onaplab onaplab 79 Dec 24 13:14 edge-kubernetes drwxrwxr-x. 3 onaplab onaplab 60 Dec 24 13:02 edge-kubernetes-crd |
Due to limitations in the current EMCO implementation the following step must be performed manually:
SSH to EDGE-K8S-1 VM:
# Determine VMs IP [onaplab@os12 ~]$ sudo virsh domifaddr edge_k8s-1 Name MAC address Protocol Address ------------------------------------------------------------------------------- vnet1 52:54:00:19:96:72 ipv4 10.121.7.152/27 # ssh from your laptop ssh -i pcei-emco onaplab@10.121.7.152 |
Deploy Azure CRD:
helm install edge-crd --repo https://edgek8s.blob.core.windows.net/staging edge-kubernetes-crd NAME: edge-crd LAST DEPLOYED: Thu Dec 24 21:43:12 2020 NAMESPACE: default STATUS: deployed REVISION: 1 TEST SUITE: None kubectl get crd NAME CREATED AT edgedeployments.microsoft.azure.devices.edge 2020-12-24T21:43:13Z |
To define Azure IoT Edge Service in PCEI, connect to EMCO UI and select "Projects → Add Project":
Select the project you just created (PCEI-AZURE-IOT in the above example), select "Add Service", specify the Name and click on "Add App":
Specify the App Name.
IMPORTANT NOTE: Please ensure that the "App Name" string matches the name of the tar file (without the .tar extension" created earlier with the Helm charts for Azure IoT Edge ("azureiotedge1" in this example".
Click "Create".
You should see the screen below:
Click on the "azureiotedge1" Service and attach application definition files: