...
This document describes the use of Public Cloud Edge Interface (PCEI) implemented based on Edge Multi-Cluster Orchestrator (EMCO) for deployment of Public Cloud Edge (PCE) Apps from multiple clouds (Azure and AWS), deployment of a 3rd-Party Edge (3PE) App (an implementation of ETSI MEC Location API App), as well as the end-to-end operation of the deployed PCE Apps using simulated Low Power Wide Area (LPWA) IoT client.
Functional Roles
Developers
Individuals/entities that:
- Create the PCE/3PE Apps
- Provision services in Public Clouds (PCC)
- Note that valid Public Cloud subscriptions/accounts are required
- Package PCE/3PE Apps using Helm Charts
- Access PCEI/EMCO Orchestrator using UI or APIs to onboard PCE/3PE services/apps
- PCEI R4 supports UI access only
Operators
Entities that:
- Provide network functions and services such as:
- Access (mobile or fixed)
- Network connectivity
- Devices (e.g. LPWA IoT)
- Access PCEI/EMCO Orchestrator using UI or APIs to deploy PCE/3PE apps
- Connecto to Edge Providers/Clusters and Public Clouds using Public Internet or Private Interconnect
- PCEI R4 supports Public Internet connectivity only
Edge Providers
Entities that:
- Provide edge infrastructure, such as:
- Edge Data Centers
- Edge Compute/Network/Storage hardware
- Edge Virtualization layer/software, such as Kubernetes, Openstack
- PCEI R4 supports Kubernetes only
- Provide Public Peering and Private Interconnection
- Between Operators and Edge Providers
- Between Operators and Public Clouds
- Between Edge Providers and Public Clouds
Public Cloud Providers
Entities that:
- Provide Iaas/SaaS services
PCEI Enabler
- Orchestration Software and APIs
- May be provided by:
- Operators
- Edge Providers
- Public Cloud Providers
End-to-End Validation Environment
...
- EMCO - Edge Multi-Cloud Orchestrator. PCEI Enabler functions are based on EMCO implementation. EMCO is deployed in the K8S cluster.
- Edge K8S Clusters - Kubernetes clusters on which PCE Apps (Azure IoT Edge, AWS GGC), 3PE App (ETSI Location API Handler) are deployed.
- Public Cloud - IaaS/SaaS (Azure, AWS).
- PCE - Public Cloud Edge App (Azure IoT Edge, AWS GGC)
- 3PE - 3rd-Party Edge App (ETSI MEC PCEI Location API App)
- Private Interconnect / Internet - Networkin between IoT Device/Client and PCE/3PE as well as connectivity between PCE and PCC.
- IoT Device - Simulated Low Power Wide Area (LPWA) IoT Client.
- IP Network/vEPC/UPF) - Network providing connectivity between IoT Device and PCE/3PE. Note that in this validation the vEPC (virtual Evolved Packet Core) /UPF (User Plane Function) are not used.
- MNO DC - Mobile Network Operator Data Center. Not used in the validation.
- Edge DC - Edge Data Center. Equinix DC used in this validation.
- Core DC - Public Cloud.
- Developer - an individual/entity providing PCE/3PE App.
- Operator - an individual/entity operating PCEI functions.
...
For performing Steps 1 and 2 please refer to the PCEI R4 Installation Guide.
Provisioning PCEI Orchestration Infrastructure (EMCO)
...
To register the two Edge K8S Clusters created during PCEI installation (refer to PCEI R4 Installation Guide), the cluster configuration files need to be copied from the corresponding VMs to the machine that is used to run the VNC server to to access EMCO UI. Please make sure that the ssh key used to access the VMs is present on the machine that runs the VNC serverwhere the cluster config files are being copied to. The example below assumes that the VNC server is running on cluster config files are copied to the Host Server used to deploy EMCO and Edge Cluster VMs. Alternatively, the ssh key and the config files could be copied to a laptop.
Code Block | ||
---|---|---|
| ||
# Determine VM IP addresses: [onaplab@os12 ~]$ sudo virsh list --all Id Name State ---------------------------------------------------- 6 amcop-vm-01 running 9 edge_k8s-1 running 10 edge_k8s-2 running [onaplab@os12 ~]$ sudo virsh domifaddr edge_k8s-1 Name MAC address Protocol Address ------------------------------------------------------------------------------- vnet1 52:54:00:19:96:72 ipv4 10.121.7.152/27 [onaplab@os12 ~]$ sudo virsh domifaddr edge_k8s-2 Name MAC address Protocol Address ------------------------------------------------------------------------------- vnet2 52:54:00:c0:47:8b ipv4 10.121.7.146/27 sftp -i pcei-emco onaplab@10.121.7.146 cd .kube get config kube-config-edge-k8s-2 sftp -i pcei-emco onaplab@10.121.7.152 cd .kube get config kube-config-edge-k8s-1 |
...
https://microsoft.github.io/iotedge-k8s-doc/architecture.html
For the purposes of this document we use the Host Server on which EMCO has been deployed and on which we run the VNC server to package A local laptop can be used to package Helm charts for Azure IoT Edge. Alternatively the Host Server itself can be used to package the Helm charts for Azure IoT Edge.
Clone Azure IoT Edge from github:
NOTE: Due to Azure IoT Edge on K8S being in preview stage, please use the below URL to clone the Helm charts:
https://github.com/Azure/iotedge/tree/release/1.1-k8s-preview
Code Block | ||
---|---|---|
| ||
git clone https://github.com/Azure/iotedge cd iotedge/kubernetes/charts/ mkdir azureiotedge1 cp -a edge-kubernetes/. azureiotedge1/ cd azureiotedge1 ls -al total 24 drwxrwxr-x. 3 onaplab onaplab 79 Dec 24 13:14 . drwxrwxr-x. 5 onaplab onaplab 77 Dec 24 13:14 .. -rw-rw-r--. 1 onaplab onaplab 137 Dec 24 13:02 Chart.yaml -rw-rw-r--. 1 onaplab onaplab 333 Dec 24 13:02 .helmignore drwxrwxr-x. 2 onaplab onaplab 220 Dec 24 13:02 templates -rw-rw-r--. 1 onaplab onaplab 14226 Dec 24 13:02 values.yaml |
...
Code Block | ||
---|---|---|
| ||
vi values.yaml # Change the line below and save the file provisioning: source: "manual" deviceConnectionString: "PASTE PRIMARY CONNECTION STRING FROM AZURE IOT HUB / IOT EDGE SCREEN" #dynamicReprovisioningdynamicReprovisioning: false # Set LoadBalancer port mapping service: name: iotedged type: LoadBalancer edgeAgent: containerName: edgeagent image: repository: azureiotedge/azureiotedge-agent tag: 0.1.0-beta9 pullPolicy: Always hostname: "localhost" env: authScheme: 'sasToken' # Set this to one of "LoadBalancer", "NodePort", or "ClusterIP" to tell the # IoT Edge runtime how you want to expose mapped ports as Services. portMappingServiceType: 'LoadBalancer' |
...
Copy the infrastructure profile tar file to your home directory on from the Host Server. If using a local laptop, copy this file from the Host Server. This file is needed to define the Service and the App in EMCO:
...
Code Block | ||
---|---|---|
| ||
# Clone PCEI repo to your local directory git clone "https://gerrit.akraino.org/r/pcei" cd pcei mkdir awsggc1 cp -a aswggcawsggc/. awsggc1/ cd awsggc1 ls -al total 16 -rw-r--r-- 1 oberzin 108493823 116 Nov 23 10:55 Chart.yaml drwxr-xr-x 9 oberzin 108493823 306 Nov 24 23:10 templates -rw-r--r-- 1 oberzin 108493823 376 Nov 24 13:32 values.yaml # Modify template files: cd templates/ (base) USMBB6G8WL-3:templates oberzin$ ls -l total 56 -rw-r--r-- 1 oberzin 108493823 1042 Nov 23 10:55 _helpers.tpl -rw-r--r-- 1 oberzin 108493823 1443 Nov 24 22:37 awsggc-cert.yaml -rw-r--r-- 1 oberzin 108493823 1939 Nov 24 22:34 awsggc-privkey.yaml -rw-r--r-- 1 oberzin 108493823 1405 Nov 24 22:22 awsggc-rootca.yaml -rw-r--r-- 1 oberzin 108493823 1086 Nov 24 22:30 configmap.yaml -rw-r--r-- 1 oberzin 108493823 1701 Nov 24 23:10 deployment.yaml -rw-r--r-- 1 oberzin 108493823 407 Nov 23 10:55 service.yaml |
...
Code Block | ||
---|---|---|
| ||
# Modify config.yaml file. # Update the "thingArn", "iotHost" and "ggHost" valuses based on the config.json file from your GGC configuration. # LEAVE ALL OTHER LINES UNCHANGED. vi configconfigmap.yaml apiVersion: v1 kind: ConfigMap metadata: name: {{ include "awsggc.name" .}}-configmap data: config.json: |- { "coreThing" : { "caPath" : "root.ca.pem", "certPath" : "cert.pem", "keyPath" : "private.key", "thingArn" : "arn:aws:iot:us-west-2:XXXXX", "iotHost" : "XXXXX-ats.iot.us-west-2.amazonaws.com", "ggHost" : "greengrass-ats.iot.us-west-2.amazonaws.com", "keepAlive" : 600 }, "runtime" : { "cgroup" : { "useSystemd" : "yes" } }, "managedRespawn" : false, "crypto" : { "principals" : { "SecretsManager" : { "privateKeyPath" : "file:///greengrass/keys/private.key" }, "IoTCertificate" : { "privateKeyPath" : "file:///greengrass/keys/private.key", "certificatePath" : "file:///greengrass/certs/cert.pem" } }, "caPath" : "file:///greengrass/ca/root.ca.pem" } } |
...
Code Block | ||
---|---|---|
| ||
# Update awsggc-cert.yaml file # Paste the contents of "<awsggcid>-setup/cert/<awsggcid>-cert.pem" file # into the awsggc-cert.yaml file # Be sure to maintan indentation as shown in the example below: vi awsggc-cert.yaml apiVersion: v1 kind: ConfigMap metadata: # name: {{ include "awsggc.name" .}}-configmap name: awsggc-cert data: cert.pem: |- -----BEGIN CERTIFICATE----- MIIDWTCCAkGgAwIBAgIUDHCe12/lfgYnQPHdMGvGCI8PUHgwDQYJKoZIhvcNAQEL 7S8ygDYZ+LcDb7ZFWxckyL7kGSELDymzFDSvCgB69WI9svBdVRZivna4nrCBTbPA RVePPjw78b4GsIGzSOGTiBmwA1ZVNC2fZK0rVkwwKwRUS4YnsXQA2UNt15CgjZvW|- -----BEGIN CERTIFICATE----- n9EOKOuP/WEag/7/euYBzG69OzH9Gd2FlUjIOQKRstDAqK6dHMYyLooRZc5csNxP 2dXWq46VN472p6kZWxbyRSGd48/VoNBXaOcp4SLkKrRVbJY3GmSwtBpb3s/PPaste cert here -----END CERTIFICATE----- # Update awsggc-privkey.yaml file # Paste the contents of "<awsggcid>-setup/cert/<awsggcid>-private.key" file # into the awsggc-privkey.yaml file # Be sure to maintan indentation as shown in the example below: vi awsggc-privkey.yaml apiVersion: v1 kind: ConfigMap metadata: # name: {{ include "awsggc.name" .}}-configmap name: awsggc-privkey data: private.key: |- -----BEGIN RSA PRIVATE KEY----- MIIEpAIBAAKCAQEAtm3jmUgvIl7nh+2xl9IRoFCk15SJU2XFuM0GBFFaqjFiQL8P u6g0b7ycVa0d4LWDSgSvfP8udHfjdLAj27ZxcDZtHz+nHi/F/HSQatJw+GRrHtHI fx6HFFsSZ58yAdynjWlajmRDbT8MbncOomeP0d2n4EpxJCM2KmF1uwIKG1ow0pnj fCTy9xlPqSZJZS7JAPgrdi3y5wQVSX8a7SG8swlTHiQoLINkItfQy5WlklQhTrt7 YvOkiGC8ew/9/cYpsPYacjmipno4woTB2kUoQozqVKQxLdHmOq8NdnCczMZzIFXl rXexcHmrgun7FMIdQOapjrJuxFlf7W/U1jJS9GRcn4YZdb03gUaqPga4hwqcXrWY 83d+fwKBgQCuRED/fYH4EJT95NBMSzkWvYgV5I6OXj5vWAevPS8+2NLyQwWMu39C mdIPPA4ir63VhcucbBvg6tSw0LsEMKGR07YxFAOOj9uWgGDS/5LIlSlA1b9tWSuM qdwyBhVgXXIZOEIDXD7qxMqN7f0kyd name: {{ include "awsggc.name" .}}-configmap name: awsggc-privkey data: private.key: |- -----BEGIN RSA PRIVATE KEY----- Paste key here -----END RSA PRIVATE KEY----- # Update awsggc-rootca.yaml file # Paste the contents of "AmazonRootCA1.pem" file # into the awsggc-rootca.yaml file # Be sure to maintan indentation as shown in the example below: vi awsggc-rootca.yaml apiVersion: v1 kind: ConfigMap metadata: # name: {{ include "awsggc.name" .}}-configmap name: awsggc-rootca data: root.ca.pem: |- -----BEGIN CERTIFICATE----- MIIDQTCCAimgAwIBAgITBmyfz5m/jAo54vB4ikPmljZbyjANBgkqhkiG9w0BAQsF ADA5MQswCQYDVQQGEwJVUzEPMA0GA1UEChMGQW1hem9uMRkwFwYDVQQDExBBbWF6 b24gUm9vdCBDQSAxMB4XDTE1MDUyNjAwMDAwMFoXDTM4MDExNzAwMDAwMFowOTEL MAkGA1UEBhMCVVMxDzANBgNVBAoTBkFtYXpvbjEZMBcGA1UEAxMQQW1hem9uIFJv b3QgQ0EgMTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBALJ4gHHKeNXj ca9HgFB0fW7Y14h29Jlo91ghYPl0hAEvrAIthtOgQ3pOsqTQNroBvo3bSMgHFzZM 9O6II8c+6zf1tRn4SWiw3te5djgdYZ6k/oI2peVKVuRF4fn9tBb6dNqcmzU5L/qw IFAGbHrQgLKm+a/sRxmPUDgH3KKHOVj4utWp+UhnMJbulHheb4mjUcAwhmahRWa6 VOujw5H5SNz/0egwLX0tdHA114gk957EWW67c4cX8jJGKLhD+rcdqsq08p8kDi1L 93FcXmn/6pUCyziKrlA4b9v7LWIbxcceVOF34GfID5yHI9Y/QCB/IIDEgEw+OyQm jgSubJrIqg0CAwEAAaNCMEAwDwYDVR0TAQH/BAUwAwEB/zAOBgNVHQ8BAf8EBAMC AYYwHQYDVR0OBBYEFIQYzIU07LwMlJQuCFmcx7IQTgoIMA0GCSqGSIb3DQEBCwUA A4IBAQCY8jdaQZChGsV2USggNiMOruYou6r4lK5IpDB/G/wkjUu0yKGX9rbxenDI U5PMCCjjmCXPI6T53iHTfIUJrU6adTrCC2qJeHZERxhlbI1Bjjt/msv0tadQ1wUs N+gDS63pYaACbvXy8MWy7Vu33PqUXHeeE6V/Uq2V8viTO96LXFvKWlJbYK8U90vv o/ufQJVtMVT8QtPHRh8jrdkPSHCa2XV4cdFyQzR1bldZwgJcJmApzyMZFo6IQ6XU 5MsI+yMRQ+hDKXJioaldXgjUkK642M4UwtBV8ob2xJNDd2ZhwLnoQdeXeGADbkpy rqXRfboQnoZsG4q5WTP468SQvvG5 -----END CERTIFICATE----- |
...
Click on "+ Add App" and add the "awsggc1" app. Make sure to use the "awsggc1" name to match the app tar file name without the extension:
// Oleg Berzin, is this figure same as the previous figure? - Tina Tsou
Define the AWS GGC app by adding the App tar filr and the Profile tar file:
...
Code Block | ||
---|---|---|
| ||
ssh onaplab@10.121.7.146 # Clone the PCEI repo git clone "https://gerrit.akraino.org/r/pcei" cd pcei/locationAPI/nodejs # Build the Docker image /nodejs # Build the Docker image sudo docker build -f Dockerfile . -t pceilocapi:latest #Run local Docker repository sudo docker run build-d -fp Dockerfile . -t pceilocapi:latest5000:5000 --restart=always --name registry registry:2 # Push the PCEI Location API image to local Docker repository: sudo docker tag pceilocapi localhost:5000/pceilocapi sudo docker push localhost:5000/pceilocapi |
...
Code Block | ||
---|---|---|
| ||
git clone "https://gerrit.akraino.org/r/pcei" cd pcei mkdir pceilocapi1 cp -a pceilocapihelm/. pceilocapi1/ lscd -lpceilocapi1 ls -l total 16 -rw-r--r-- 1 oberzin staff 120 Dec 29 23:24 Chart.yaml drwxr-xr-x 6 oberzin staff 192 Dec 29 23:34 templates -rw-r--r-- 1 oberzin staff 376 Dec 29 23:34 values.yaml cd .. tar -czvf pceilocapi1.tar pceilocapi1/ a pceilocapi1 a pceilocapi1/Chart.yaml a pceilocapi1/.helmignore a pceilocapi1/templates a pceilocapi1/values.yaml a pceilocapi1/templates/deployment.yaml a pceilocapi1/templates/service.yaml a pceilocapi1/templates/configmap.yaml a pceilocapi1/templates/_helpers.tpl |
...
Code Block | ||
---|---|---|
| ||
# SSH to egde_k8s-1 VM: ssh onaplab@10.121.7.146 # Verify that PCEI Location API App pod is Running kubectl get pods NAME READY STATUS RESTARTS AGE v1-pceilocapi-64477bb5d8-dzjh7 1/1 Running 0 107s onaplab@localhost:~$ # Verify K8S service kubectl get service NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE pceilocapi-service NodePort 10.244.20.128 <none> 8081:30808/TCP 26m # Verify PCEI Location API logs kubectl logs v1-pceilocapi-64477bb5d8-dzjh7 > location-api@1.0.0 prestart /usr/src/app > npm install audited 147 packages in 1.235s 2 packages are looking for funding run `npm fund` for details found 8 vulnerabilities (2 low, 2 moderate, 4 high) run `npm audit fix` to fix them, or `npm audit` for details > location-api@1.0.0 start /usr/src/app > node index.js Your server is listening on port 8081 (http://localhost:8081) Swagger-ui is available on http://localhost:8081/docs # Verify PCEI Location API Docker container sudo docker ps |grep pcei 128842966f27 localhost:5000/pceilocapi "docker-entrypoint.s…" 4 minutes ago Up 4 minutes k8s_pceilocapi_v1-pceilocapi-64477bb5d8-dzjh7_default_7732dd43-4e61-48a7-bc2c-70770895fa6f_0 |
Verifying PCEI Location API App
Access the PCEI Location API App doc page:
http://<EDGE-K8S-1 VM IP Address>:<pceilocapi pod service port>/docs
Code Block | ||
---|---|---|
| ||
http://10.121.7.146:30808/docs |