Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

This document describes the use of Public Cloud Edge Interface (PCEI) implemented based on Edge Multi-Cluster Orchestrator (EMCO) for deployment of Public Cloud Edge (PCE) Apps from multiple clouds (Azure and AWS), deployment of a 3rd-Party Edge (3PE) App (an implementation of ETSI MEC Location API App), as well as the end-to-end operation of the deployed PCE Apps using simulated Low Power Wide Area (LPWA) IoT client.

Image Added

Functional Roles

Developers

Individuals/entities that:

  • Create the PCE/3PE Apps
  • Provision services in Public Clouds (PCC)
    • Note that valid Public Cloud subscriptions/accounts are required
  • Package PCE/3PE Apps using Helm Charts
  • Access PCEI/EMCO Orchestrator using UI or APIs to onboard PCE/3PE services/apps
    • PCEI R4 supports UI access only

Operators

Entities that:

  • Provide network functions and services such as:
    • Access (mobile or fixed)
    • Network connectivity
    • Devices (e.g. LPWA IoT)
  • Access PCEI/EMCO Orchestrator using UI or APIs to deploy PCE/3PE apps
  • Connecto to Edge Providers/Clusters and Public Clouds using Public Internet or Private Interconnect
    • PCEI R4 supports Public Internet connectivity only

Edge Providers

Entities that:

  • Provide edge infrastructure, such as:
    • Edge Data Centers
    • Edge Compute/Network/Storage hardware
    • Edge Virtualization layer/software, such as Kubernetes, Openstack
      • PCEI R4 supports Kubernetes only
  • Provide Public Peering and Private Interconnection
    • Between Operators and Edge Providers
    • Between Operators and Public Clouds
    • Between Edge Providers and Public Clouds

Public Cloud Providers

Entities that:

  • Provide Iaas/SaaS services

PCEI Enabler

  • Orchestration Software and APIs
  • May be provided by:
    • Operators
    • Edge Providers
    • Public Cloud Providers

End-to-End Validation Environment

...

  • EMCO - Edge Multi-Cloud Orchestrator. PCEI Enabler functions are based on EMCO implementation. EMCO is deployed in the K8S cluster.
  • Edge K8S Clusters - Kubernetes clusters on which PCE Apps (Azure IoT Edge, AWS GGC), 3PE App (ETSI Location API Handler) are deployed.
  • Public Cloud - IaaS/SaaS (Azure, AWS).
  • PCE - Public Cloud Edge App (Azure IoT Edge, AWS GGC)
  • 3PE - 3rd-Party Edge App (ETSI MEC PCEI Location API App)
  • Private Interconnect / Internet - Networkin between IoT Device/Client and PCE/3PE as well as connectivity between PCE and PCC.
  • IoT Device - Simulated Low Power Wide Area (LPWA) IoT Client.
  • IP Network/vEPC/UPF) - Network providing connectivity between IoT Device and PCE/3PE. Note that in this validation the vEPC (virtual Evolved Packet Core) /UPF (User Plane Function) are not used.
  • MNO DC - Mobile Network Operator Data Center. Not used in the validation.
  • Edge DC - Edge Data Center. Equinix DC used in this validation.
  • Core DC - Public Cloud.
  • Developer - an individual/entity providing PCE/3PE App.
  • Operator - an individual/entity operating PCEI functions.

...

For performing Steps 1 and 2 please refer to the PCEI R4 Installation Guide.

Provisioning PCEI Orchestration Infrastructure (EMCO)

...

To register the two Edge K8S Clusters created during PCEI installation (refer to PCEI R4 Installation Guide), the cluster configuration files need to be copied from the corresponding VMs to the machine that is used to run the VNC server to to  access EMCO UI. Please make sure that the ssh key used to access the VMs is present on the machine that runs the VNC serverwhere the cluster config files are being copied to. The example below assumes that the VNC server is running on cluster config files are copied to the Host Server used to deploy EMCO and Edge Cluster VMs. Alternatively, the ssh key and the config files could be copied to a laptop.

Code Block
languagebash
# Determine VM IP addresses:
[onaplab@os12 ~]$ sudo virsh list --all
 Id    Name                           State
----------------------------------------------------
 6     amcop-vm-01                    running
 9     edge_k8s-1                     running
 10    edge_k8s-2                     running

[onaplab@os12 ~]$ sudo virsh domifaddr edge_k8s-1
 Name       MAC address          Protocol     Address
-------------------------------------------------------------------------------
 vnet1      52:54:00:19:96:72    ipv4         10.121.7.152/27

[onaplab@os12 ~]$ sudo virsh domifaddr edge_k8s-2
 Name       MAC address          Protocol     Address
-------------------------------------------------------------------------------
 vnet2      52:54:00:c0:47:8b    ipv4         10.121.7.146/27

sftp -i pcei-emco onaplab@10.121.7.146
cd .kube
get config kube-config-edge-k8s-2

sftp -i pcei-emco onaplab@10.121.7.152
cd .kube
get config kube-config-edge-k8s-1

...

https://microsoft.github.io/iotedge-k8s-doc/architecture.html

For the purposes of this document we use the Host Server on which EMCO has been deployed and on which we run the VNC server to package A local laptop can be used to package Helm charts for Azure IoT Edge. Alternatively the Host Server itself can be used to package the Helm charts for Azure IoT Edge.

Clone Azure IoT Edge from github:

NOTE: Due to Azure IoT Edge on K8S being in preview stage, please use the below URL to clone the Helm charts:

https://github.com/Azure/iotedge/tree/release/1.1-k8s-preview

Code Block
languagebash
git clone https://github.com/Azure/iotedge

cd iotedge/kubernetes/charts/
mkdir azureiotedge1
cp -a edge-kubernetes/. azureiotedge1/
cd azureiotedge1
ls -al
total 24
drwxrwxr-x. 3 onaplab onaplab    79 Dec 24 13:14 .
drwxrwxr-x. 5 onaplab onaplab    77 Dec 24 13:14 ..
-rw-rw-r--. 1 onaplab onaplab   137 Dec 24 13:02 Chart.yaml
-rw-rw-r--. 1 onaplab onaplab   333 Dec 24 13:02 .helmignore
drwxrwxr-x. 2 onaplab onaplab   220 Dec 24 13:02 templates
-rw-rw-r--. 1 onaplab onaplab 14226 Dec 24 13:02 values.yaml

...

Code Block
languagebash
vi values.yaml
# Change the line below and save the file
provisioning:
  source: "manual"
  deviceConnectionString:  "PASTE PRIMARY CONNECTION STRING FROM AZURE IOT HUB / IOT EDGE SCREEN"
  #dynamicReprovisioningdynamicReprovisioning: false

# Set LoadBalancer port mapping

service:
    name: iotedged
    type: LoadBalancer

edgeAgent:
  containerName: edgeagent
  image:
    repository: azureiotedge/azureiotedge-agent
    tag: 0.1.0-beta9
    pullPolicy: Always
  hostname: "localhost"
  env:
    authScheme: 'sasToken'
    # Set this to one of "LoadBalancer", "NodePort", or "ClusterIP" to tell the
    # IoT Edge runtime how you want to expose mapped ports as Services.
    portMappingServiceType: 'LoadBalancer'

...

Copy the infrastructure profile tar file to your home directory on from the Host Server. If using a local laptop, copy this file from the Host Server. This file is needed to define the Service and the App in EMCO:

...

Code Block
languagebash
# Clone PCEI repo to your local directory

git clone "https://gerrit.akraino.org/r/pcei"
cd pcei
mkdir awsggc1
cp -a aswggcawsggc/. awsggc1/
cd awsggc1
ls -al
total 16
-rw-r--r--  1 oberzin  108493823  116 Nov 23 10:55 Chart.yaml
drwxr-xr-x  9 oberzin  108493823  306 Nov 24 23:10 templates
-rw-r--r--  1 oberzin  108493823  376 Nov 24 13:32 values.yaml

# Modify template files:
cd templates/
(base) USMBB6G8WL-3:templates oberzin$ ls -l
total 56
-rw-r--r--  1 oberzin  108493823  1042 Nov 23 10:55 _helpers.tpl
-rw-r--r--  1 oberzin  108493823  1443 Nov 24 22:37 awsggc-cert.yaml
-rw-r--r--  1 oberzin  108493823  1939 Nov 24 22:34 awsggc-privkey.yaml
-rw-r--r--  1 oberzin  108493823  1405 Nov 24 22:22 awsggc-rootca.yaml
-rw-r--r--  1 oberzin  108493823  1086 Nov 24 22:30 configmap.yaml
-rw-r--r--  1 oberzin  108493823  1701 Nov 24 23:10 deployment.yaml
-rw-r--r--  1 oberzin  108493823   407 Nov 23 10:55 service.yaml

...

Code Block
languageyml
# Modify config.yaml file.
# Update the "thingArn", "iotHost" and "ggHost" valuses based on the config.json file from your GGC configuration. 
# LEAVE ALL OTHER LINES UNCHANGED.
vi configconfigmap.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: {{ include "awsggc.name" .}}-configmap
data:
  config.json: |-
    {
      "coreThing" : {
        "caPath" : "root.ca.pem",
        "certPath" : "cert.pem",
        "keyPath" : "private.key",
        "thingArn" : "arn:aws:iot:us-west-2:XXXXX",
        "iotHost" : "XXXXX-ats.iot.us-west-2.amazonaws.com",
        "ggHost" : "greengrass-ats.iot.us-west-2.amazonaws.com",
        "keepAlive" : 600
      },
      "runtime" : {
        "cgroup" : {
          "useSystemd" : "yes"
        }
      },
      "managedRespawn" : false,
      "crypto" : {
        "principals" : {
          "SecretsManager" : {
            "privateKeyPath" : "file:///greengrass/keys/private.key"
          },
          "IoTCertificate" : {
            "privateKeyPath" : "file:///greengrass/keys/private.key",
            "certificatePath" : "file:///greengrass/certs/cert.pem"
          }
        },
        "caPath" : "file:///greengrass/ca/root.ca.pem"
      }
    }

...

Code Block
languageyml
# Update awsggc-cert.yaml file
# Paste the contents of "<awsggcid>-setup/cert/<awsggcid>-cert.pem" file
# into the awsggc-cert.yaml file
# Be sure to maintan indentation as shown in the example below:

vi awsggc-cert.yaml
apiVersion: v1
kind: ConfigMap
metadata:
#  name: {{ include "awsggc.name" .}}-configmap
  name: awsggc-cert
data:
  cert.pem: |-
    -----BEGIN CERTIFICATE-----
    MIIDWTCCAkGgAwIBAgIUDHCe12/lfgYnQPHdMGvGCI8PUHgwDQYJKoZIhvcNAQEL
    7S8ygDYZ+LcDb7ZFWxckyL7kGSELDymzFDSvCgB69WI9svBdVRZivna4nrCBTbPA
    RVePPjw78b4GsIGzSOGTiBmwA1ZVNC2fZK0rVkwwKwRUS4YnsXQA2UNt15CgjZvW|-
    -----BEGIN CERTIFICATE-----
    n9EOKOuP/WEag/7/euYBzG69OzH9Gd2FlUjIOQKRstDAqK6dHMYyLooRZc5csNxP
    2dXWq46VN472p6kZWxbyRSGd48/VoNBXaOcp4SLkKrRVbJY3GmSwtBpb3s/PPaste cert here
    -----END CERTIFICATE-----

# Update awsggc-privkey.yaml file
# Paste the contents of "<awsggcid>-setup/cert/<awsggcid>-private.key" file
# into the awsggc-privkey.yaml file
# Be sure to maintan indentation as shown in the example below:

vi awsggc-privkey.yaml
apiVersion: v1
kind: ConfigMap
metadata:
#  name: {{ include "awsggc.name" .}}-configmap
  name: awsggc-privkey
data:
  private.key: |-
    -----BEGIN RSA PRIVATE KEY-----
    MIIEpAIBAAKCAQEAtm3jmUgvIl7nh+2xl9IRoFCk15SJU2XFuM0GBFFaqjFiQL8P
    u6g0b7ycVa0d4LWDSgSvfP8udHfjdLAj27ZxcDZtHz+nHi/F/HSQatJw+GRrHtHI
    fx6HFFsSZ58yAdynjWlajmRDbT8MbncOomeP0d2n4EpxJCM2KmF1uwIKG1ow0pnj
    fCTy9xlPqSZJZS7JAPgrdi3y5wQVSX8a7SG8swlTHiQoLINkItfQy5WlklQhTrt7
    YvOkiGC8ew/9/cYpsPYacjmipno4woTB2kUoQozqVKQxLdHmOq8NdnCczMZzIFXl
    
    rXexcHmrgun7FMIdQOapjrJuxFlf7W/U1jJS9GRcn4YZdb03gUaqPga4hwqcXrWY
    83d+fwKBgQCuRED/fYH4EJT95NBMSzkWvYgV5I6OXj5vWAevPS8+2NLyQwWMu39C
    mdIPPA4ir63VhcucbBvg6tSw0LsEMKGR07YxFAOOj9uWgGDS/5LIlSlA1b9tWSuM
    qdwyBhVgXXIZOEIDXD7qxMqN7f0kyd  name: {{ include "awsggc.name" .}}-configmap
  name: awsggc-privkey
data:
  private.key: |-
    -----BEGIN RSA PRIVATE KEY-----
    Paste key here
    -----END RSA PRIVATE KEY-----

# Update awsggc-rootca.yaml file
# Paste the contents of "AmazonRootCA1.pem" file
# into the awsggc-rootca.yaml file
# Be sure to maintan indentation as shown in the example below:

vi awsggc-rootca.yaml

apiVersion: v1
kind: ConfigMap
metadata:
#  name: {{ include "awsggc.name" .}}-configmap
  name: awsggc-rootca
data:
  root.ca.pem: |-
    -----BEGIN CERTIFICATE-----
    MIIDQTCCAimgAwIBAgITBmyfz5m/jAo54vB4ikPmljZbyjANBgkqhkiG9w0BAQsF
    ADA5MQswCQYDVQQGEwJVUzEPMA0GA1UEChMGQW1hem9uMRkwFwYDVQQDExBBbWF6
    b24gUm9vdCBDQSAxMB4XDTE1MDUyNjAwMDAwMFoXDTM4MDExNzAwMDAwMFowOTEL
    MAkGA1UEBhMCVVMxDzANBgNVBAoTBkFtYXpvbjEZMBcGA1UEAxMQQW1hem9uIFJv
    b3QgQ0EgMTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBALJ4gHHKeNXj
    ca9HgFB0fW7Y14h29Jlo91ghYPl0hAEvrAIthtOgQ3pOsqTQNroBvo3bSMgHFzZM
    9O6II8c+6zf1tRn4SWiw3te5djgdYZ6k/oI2peVKVuRF4fn9tBb6dNqcmzU5L/qw
    IFAGbHrQgLKm+a/sRxmPUDgH3KKHOVj4utWp+UhnMJbulHheb4mjUcAwhmahRWa6
    VOujw5H5SNz/0egwLX0tdHA114gk957EWW67c4cX8jJGKLhD+rcdqsq08p8kDi1L
    93FcXmn/6pUCyziKrlA4b9v7LWIbxcceVOF34GfID5yHI9Y/QCB/IIDEgEw+OyQm
    jgSubJrIqg0CAwEAAaNCMEAwDwYDVR0TAQH/BAUwAwEB/zAOBgNVHQ8BAf8EBAMC
    AYYwHQYDVR0OBBYEFIQYzIU07LwMlJQuCFmcx7IQTgoIMA0GCSqGSIb3DQEBCwUA
    A4IBAQCY8jdaQZChGsV2USggNiMOruYou6r4lK5IpDB/G/wkjUu0yKGX9rbxenDI
    U5PMCCjjmCXPI6T53iHTfIUJrU6adTrCC2qJeHZERxhlbI1Bjjt/msv0tadQ1wUs
    N+gDS63pYaACbvXy8MWy7Vu33PqUXHeeE6V/Uq2V8viTO96LXFvKWlJbYK8U90vv
    o/ufQJVtMVT8QtPHRh8jrdkPSHCa2XV4cdFyQzR1bldZwgJcJmApzyMZFo6IQ6XU
    5MsI+yMRQ+hDKXJioaldXgjUkK642M4UwtBV8ob2xJNDd2ZhwLnoQdeXeGADbkpy
    rqXRfboQnoZsG4q5WTP468SQvvG5
    -----END CERTIFICATE-----

...

Click on "+ Add App" and add the "awsggc1" app. Make sure to use the "awsggc1" name to match the app tar file name without the extension:

// Oleg Berzin, is this figure same as the previous figure? - Tina Tsou


Define the AWS GGC app by adding the App tar filr and the Profile tar file:

...

Code Block
languagebash
ssh onaplab@10.121.7.146

# Clone the PCEI repo
git clone "https://gerrit.akraino.org/r/pcei"
cd pcei/locationAPI/nodejs

# Build the Docker image
/nodejs

# Build the Docker image

sudo docker build -f Dockerfile . -t pceilocapi:latest

#Run local Docker repository
sudo docker run build-d -fp Dockerfile . -t pceilocapi:latest5000:5000 --restart=always --name registry registry:2

# Push the PCEI Location API image to local Docker repository:
sudo docker tag pceilocapi localhost:5000/pceilocapi
sudo docker push localhost:5000/pceilocapi

...

Code Block
languagebash
git clone "https://gerrit.akraino.org/r/pcei"
cd pcei
mkdir pceilocapi1
cp -a pceilocapihelm/. pceilocapi1/
lscd -lpceilocapi1
ls -l
total 16
-rw-r--r--  1 oberzin  staff  120 Dec 29 23:24 Chart.yaml
drwxr-xr-x  6 oberzin  staff  192 Dec 29 23:34 templates
-rw-r--r--  1 oberzin  staff  376 Dec 29 23:34 values.yaml

cd ..
tar -czvf pceilocapi1.tar pceilocapi1/ 
a pceilocapi1
a pceilocapi1/Chart.yaml
a pceilocapi1/.helmignore
a pceilocapi1/templates
a pceilocapi1/values.yaml
a pceilocapi1/templates/deployment.yaml
a pceilocapi1/templates/service.yaml
a pceilocapi1/templates/configmap.yaml
a pceilocapi1/templates/_helpers.tpl

...

Code Block
languagebash
# SSH to egde_k8s-1 VM:

ssh onaplab@10.121.7.146

# Verify that PCEI Location API App pod is Running
kubectl get pods
NAME                             READY   STATUS    RESTARTS   AGE
v1-pceilocapi-64477bb5d8-dzjh7   1/1     Running   0          107s
onaplab@localhost:~$ 

# Verify K8S service
kubectl get service
NAME                 TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
pceilocapi-service   NodePort    10.244.20.128   <none>        8081:30808/TCP   26m

# Verify PCEI Location API logs
kubectl logs v1-pceilocapi-64477bb5d8-dzjh7

> location-api@1.0.0 prestart /usr/src/app
> npm install

audited 147 packages in 1.235s

2 packages are looking for funding
  run `npm fund` for details

found 8 vulnerabilities (2 low, 2 moderate, 4 high)
  run `npm audit fix` to fix them, or `npm audit` for details

> location-api@1.0.0 start /usr/src/app
> node index.js

Your server is listening on port 8081 (http://localhost:8081)
Swagger-ui is available on http://localhost:8081/docs

# Verify PCEI Location API Docker container
sudo docker ps |grep pcei
128842966f27        localhost:5000/pceilocapi                                         "docker-entrypoint.s…"   4 minutes ago       Up 4 minutes                                 k8s_pceilocapi_v1-pceilocapi-64477bb5d8-dzjh7_default_7732dd43-4e61-48a7-bc2c-70770895fa6f_0




Verifying PCEI Location API App

Access the PCEI Location API App doc page:

http://<EDGE-K8S-1 VM IP Address>:<pceilocapi pod service port>/docs

Code Block
languagebash
http://10.121.7.146:30808/docs

Image Added