Introduction

The guide covers the installation details which are related to

...

Enterprise Applications on Lightweight 5G Telco Edge (EALT) Blueprint.  

This

...

is the first release for this blueprint, the guide covers detailed information of the various types of deployments, detailed steps and what are the various components it will install. In addition the guide provides information on hardware requirements , prerequisite software and minimum hardware requirements. On successful deployment MECM and MEP nodes will be installed. The number of nodes in MECM cluster and MEP cluster is configurable. 

The MECM is a K8S Cluster and MEP is as K3S Cluster. 

The sections updated for Release 2 are Software Prerequisites, Installation High Level Overview, Virtual Deployment , Verifying the setup and Uninstall Guide, other sections remain same as Release 1.0 documentation.

For Release-2.0 the scope of the guide covers two ELIOT deployments one is using Kubernetes and the other one using Kubeedge (Lightweight Kubernetes).  

The ELIOT Manager can be any Cloud Virtual Machine or Bare Metal having Ubuntu OS 16.04 and should be able to ping and SSH to the ELIOT Edge Node - IoT Gateway Node.  The Edge Node will be a Virtual Machine having Ubuntu 0S 16.04.

For Kubernetes setup Kubeadm tool is being used and for Container Networking Interface Calico plugin is being used.

For KubeEdge the connection between the ELIOT Manager and ELIOT Edge Node is being established via Web Socket. Currently KubeEdge does not support Container Networking Interface.

The common features which are applicable for both the ELIOT blueprints (IoT Gateway and SD-WAN / WAN/ uCPE) include ELIOT Portal , ELIOT Command Line Interface to bootstrap the ELIOT Cluster Installation.  These features are developed to ease the use and provide value added features for deployment, usage and monitoring of ELIOT Cluster.

The common features installation are add on feature provided to manage the ELIOT Cluster. For detail usage and installation please refer to the link :- ELIOT Release 2 - ELIOT Operation & Management Features Installation Document (Optional)

How to use this document

The document includes details of prerequisites /pre-installation, installation and uninstalls steps.

...

The prerequisites and pre-installation software and hardware should be ready before executing the installation steps.

...

Two types of installation mechanisms are provided in this release 

1. Ansible-Playbook single command
2. Command Line Interface (CLI)

Deployment Architecture

...

The Deployment Architecture consists of

...

three nodes One Click Deployment Node,  MECM and a single

...

MEP Edge Node.

...

Note: For Development environment two nodes  is sufficient, where one node plays a dual role of One Click Deployment Node and MECM Node with other as MEP Edge Node.

Pre-Installation Requirements

Hardware Requirements

Number of Hardware requirements depends mainly on the Use Case Scenario and the enterprise scale. A use case can have one to multiple ELIOT MECM Cluster ,  with single or multiple MEP Edge Nodes

Note:  ELIOT Manager is acting a dual role Jump server (to execute the setup script) and as the Kubernetes Master. Also currently the below hardware capability are mentioned for the Virtual Machines.

ELIOT Manager - One VM - 8 vCPU - 32 GB RAM (can be hosted in any cloud provider) with Ubuntu OS having internet and network connectivity with ELIOT Edge Node.

...

cluster.

Minimum number of nodes required for a complete EALT Topology is three. (Bare-Metal or Virtual Machines) 

1) Deployment Node

2) MECM

3) MEP Edge Node

Note: The Hardware details provided are of Virtual Machine configurations.

Minimum Hardware Requirements

Recommended Hardware Requirements

Software Prerequisites

• Virtual Machines preinstalled with Ubuntu 16.04 or 18.04
• root user created in the

...

• Deployment Node, MEC Node and MEP Node.
• SSH Server running in

...

• all the Nodes.

...

• Ansible > 2.5 installed in One Click Deployment Node (Jump Host)
• git installed in Jump Host.
• Kubespray code is downloaded (https://github.com/kubernetes-sigs/kubespray.git)
• GO Lang (version greater than 1.12) installed in Jump Host, required for CLI

...

• .

Database Prerequisites

Schema scripts

N/A

Other Installation Requirements

Jump Host Requirements

N/A

Network Requirements

• Internet

...

• connectivity in Jump Host, MECM  and MEP Nodes.
• The MECM Master Node and MEP Master

...

• Node should be able to ping each other

...

• .

Bare Metal Node Requirements

N/A

Execution Requirements (Bare Metal Only)

N/A

Installation High-Level Overview

To setup the complete ELIOT IoT Gateway Environment need to execute one script which will bootstrap the installation and provide the complete ELIOT Cluster on which applications can be deployed.

In Release 2.0 installations steps are provided to setup following ELIOT IoT Gateway Environments : 

...

The blueprint provides single command and command line interface for installing the EALT blueprint components.

Bare Metal Deployment Guide

Install Bare Metal Jump Host

Currently

...

EALT Blueprint Deployment have been done in Huawei Cloud Virtual Machines

...

. The scripts will

...

run successfully in bare metal hardware too, if the hardware and software prerequisites are met.

...

Refer section

...

Virtual Deployment Guide

...

for deployment

...

.

ELIOT IoT Gateway Note : EALT blueprint installation is not done or tested in Bare-Metal Environment.

Creating a Node Inventory File

N/A

Creating the Settings Files

N/A

Running

N/A

Virtual Deployment Guide

For Virtual Deployment minimum

...

three Virtual machines

...

, following are the virtual machines and their usage

All the nodes should have internet connectivity , network interface and network connectivity between

...

the  VM's. 

Standard Deployment Overview

Deployment Type 1 : ELIOT IoT Gateway Deployment (Ubuntu & Cent OS)

Clone the ELIOT git Repo in ELIOT Manager node. It will create eliot folder, for executing the script go to $HOME/eliot/blueprint/iotgateway/scripts folder.

Edit the nodelist file with ELIOT Edge Node details.  Replace the below contents with the actual value with '|' separated. 

The below details are required in the setup.sh for sshpass command. The details are required by sshpass to login to ELIOT Edge Node from ELIOT Manager and execute the deployment scripts in the ELIOT Edge Node.

<eliotedgenodeusername>|<eliotedgenodeip>|<eliotedgenodepassword>

• eliotedgenodeusername – ELIOT Edge Node User Name. Example : akraino-slave-002
• eliotedgenodeip – ELIOT Edge Node IP. Example : 159.123.1.50
• eliotedgenodepassword - ELIOT Edge Node Password used in setup.sh script to login to ELIOT Edge node from ELIOT Manager to execute scripts in ELIOT Edge node.

The ELIOT Edge Node should be reachable from ELIOT Manager and ssh should work from ELIOT Manager to ELIOT Edge node as a prerequisite.

root@akraino-slave-001:#git clone "https://gerrit.akraino.org/r/eliot"
root@akraino-slave-001:#cd eliot/blueprints/iotgateway/scripts
root@akraino-slave-001:~/eliot/blueprints/iotgateway/scripts#vim nodelist

After editing the nodelist file , execute the setup.sh script.  The setup.sh script internally installs Docker and Kubernetes with Calico CNI plugin and RBAC for authorization.

root@akraino-slave-001:~/eliot/blueprints/iotgateway/scripts#./setup.sh

Once the execution is completed in console will see prompt "ELIOT IOT-Gateway Platform Deployment--------------------------SUCCESS".

Following packages will be installed in the cluster in the respective nodes:-

ELIOT Manager

...

Docker

...

ELIOT IoT Gateway Node

...

ELIOT Portal Installation (Optional) :

In release 2.0 ELIOT Portal is feature is released. Once the ELIOT Cluster setup is done ELIOT Portal can be installed. Refer ELIOT Portal Section in the document ELIOT Release 2 - ELIOT Operation & Management Features Installation Document (Optional)

Deployment 2 : ELIOT IoT Gateway Deployment (Tailored OS)

Clone the ELIOT git Repo in ELIOT Manager node. It will create eliot folder, for executing the script go to $HOME/eliot/blueprint/iotgateway/scripts folder.

The ELIOT Edge Node (IoT Gateway Node) is a tailored OS Virtual Machine. (Tailored OS used in the environment is created using steps mentioned in the link ELIOT Tailored OS Creation Document

Edit the nodelist file with ELIOT Edge Node details.  Replace the below contents with the actual value with '|' separated. 

The below details are required in the setup.sh for sshpass command. The details are required by sshpass to login to ELIOT Edge Node from ELIOT Manager and execute the deployment scripts in the ELIOT Edge Node.

<eliotedgenodeusername>|<eliotedgenodeip>|<eliotedgenodepassword>

• eliotedgenodeusername – ELIOT Edge Node User Name. Example : akraino-slave-002
• eliotedgenodeip – ELIOT Edge Node IP. Example : 159.123.1.50
• eliotedgenodepassword - ELIOT Edge Node Password used in setup.sh script to login to ELIOT Edge node from ELIOT Manager to execute scripts in ELIOT Edge node.

The ELIOT Edge Node should be reachable from ELIOT Manager and ssh should work from ELIOT Manager to ELIOT Edge node as a prerequisite.

...

In this release providing two ways to install the EALT environment. 

i)  EALT Deployment using Ansible-Playbook single command.

ii) EALT Deployment using CLI

Standard Deployment Overview

Jump Host Software Installations:

Login to the Jump Host and perform the below steps:

1. Install Ansible > 2.5.[ https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html]
2. Install git
3. Install GoLang > 1.12
4. Git clone the Kubespray repo. [https://github.com/kubernetes-sigs/kubespray.git]
5. Install python3 and pip3
6. cd kubespray && pip install -r requirements.txt

Jump Host Pre-Configurations for MECM Components Installation

Login to the Jump Host and perform the below configuration steps (Steps :

1. Generate public key : #ssh-keygen

2. Copy the ssh key to all the nodes in the MECM Cluster, using ssh-copy-id. (example : #ssh-copy-id root@159.178.17.16)

3. Kubespray configuration

      cd kubespray && cp -rfp inventory/sample inventory/mycluster

      Update ansible inventory file provided in kubesrpay repo with MECM Cluster node IP's 

      Example: 

       declare -a IPS=(158.176.15.55 158.176.15.56 158.176.15.57)

       CONFIG_FILE=inventory/mycluster/hosts.yaml python3 contrib/inventory_builder/inventory.py ${IPS[@]}

4. Review and Change Parameters under inventory/mycluster/group_vars

        cat inventory/mycluster/group_vars/all/all.yml
        cat inventory/mycluster/group_vars/k8s-cluster/k8s-cluster.yml

Installing Mode : EALT using Ansible-Playbooks

1. git clone the ealt repo, to download the software to install the EALT Environment.

root@akraino-mec-0002:~# git clone "https://gerrit.akraino.org/r/

...

ealt-edge"

2. go to the below directory 

root@akraino-

...

mec-

...

0002:~# cd ealt/infra/playbooks

3. Modify the Configuration File : ealt-inventory.ini with the details of MECM and MEP Nodes. 

root@akraino-mec-0002:~# vi ealt-inventory.ini

4. Execute the below command 

root@akraino-mec-0002:~# ansible-playbook ealt-all.yaml -i ealt-inventory.ini

After editing the nodelist file , execute the setup.sh script.  The setup.sh script internally installs Docker and Kubernetes with Calico CNI plugin and RBAC for authorization.

...

Once the execution is completed in console will see prompt 

...

"EALT Environment Installed  , Components Install MECM and MEP Nodes Successfully"

Other Options:

To Install only MECM Node

root@akraino-mec-0002:~# ansible-playbook ealt-mecm.yaml -i ealt-inventory.ini

To Install only MEP Node

root@akraino-mec-0002:~# ansible-playbook ealt-mep.yaml -i ealt-inventory.ini

Installing Mode : EALT using CLI

1. git clone the ealt repo, to download the software to install the EALT Environment.

root@akraino-mec-0002:~# git clone "https://gerrit.akraino.org/r/ealt-edge"

2. go to CLI directory 

root@akraino-mec-0002:~#cd ealt/infra/cli/

root@akraino-mec-0002:~#source build_cli.sh

3. Execute the below command to install EALT Environment 

root@akraino-mec-0002:~#ealt init all 

Above command will install the various component. Refer the below table for detail information. 

Other CLI Commands:

root@akraino-mec-0002:~#ealt init mecm

root@akraino-mec-0002:~#ealt init mep

Following packages will be installed in the cluster in the respective nodes:-

...

MECM

...

MEP

Deployment Type 3: ELIOT IoT Gateway Deployment using KubeEdge

Clone the ELIOT git repo in ELIOT Manager node. It will create eliot folder, for executing the script go to $HOME/eliot/blueprints/iotgateway/scripts/src folder.

Edit the config_kubeedge file with ELIOT Manager and ELIOT Edge Node

The format of config_kubeedge file is as follows :

EDGENODEUSR=""

EDGENODEIP=""

EDGENODEPASSWORD=""

MASTERNODEIP=""

HOME_EDGENODE=""

KUBEEDGE_ETC="/etc/kubeedge"

KUBEEDGE_VERSION_ETC="/etc/kubeedge-v1.1.0-linux-amd64/"

PATH_OF_KUBEEDGE=""

root@akraino-slave-001:#git clone "https://gerrit.akraino.org/r/eliot" 
root@akraino-slave-001:#cd eliot/blueprints/iotgateway/scripts/
root@akraino-slave-001:~/eliot/blueprints/iotgateway/scripts#./setup.sh 
root@akraino-slave-001:~/eliot/blueprints/iotgateway/scripts#vim config_kubeedge
root@akraino-slave-001:~/eliot/blueprints/iotgateway/scripts#cd kubeedge
root@akraino-slave-001:~/eliot/blueprints/iotgateway/scripts/kubeedge# ./kubeedge_setup.sh

Note:

We are running setup.sh without updating nodelist file.  This is required to setup k8s master node which is required as a prerequisite for KubeEdge setup

in KubeEdge_setup.sh → we are currently using V1.1.0 version of kubeedge ( latest stable version package )

PATH_OF_KUBEEDGE variable has to be set with path of the kubeedge folder.  This will be "eliot/blueprints/iotgateway/scripts/kubeedge"

KUBEEDGE_VERSION_ETC → variable has to be set with "/etc/kubeedge-v1.1.0-linux-amd64/" since we are using 1.1.0 version of kubeedge

Snapshot Deployment Overview

There are two images created for Manager and Node machines. These are .qcow2 files. After deploying the Virtual Machines using these images, your setup for ELIOT is ready in minutes. Following is the link to the document to use these images:

...

Snapshot Deployment Overview

Not Applicable

Special Requirements for Virtual Deployments

N/A

Install Jump Host

N/A

Verifying the Setup - VM's

N/A

Upstream Deployment Guide

Upstream Deployment Key Features

N/A

Special Requirements for Upstream Deployments

N/A

Scenarios and Deploy Settings for Upstream Deployments

N/A

Including Upstream Patches with Deployment

N/A

Running

N/A

Interacting with Containerized Overcloud

N/A

Verifying the Setup

Verifying

...

EALT Deployment

The setup.sh script will install the ELIOT Manager and one ELIOT Edge Node ( depending on the number of edge nodes configured in the nodelist configuration file).  The script deploys the Kubernetes cluster and also installs EdgexX platform,  NGINX, Prometheus and cAdvisor and verifies whether the cluster by checking if EdgeX platform , NGINX Server is deployed successfully or not. If the deployment is not successful then the ELIOT setup script will exit.

After script execution is completed the Kubernetes cluster setup can be checked by executing kubectl get nodes. It will show both the ELIOT Manager and the ELIOT Edge Node as ready.

...

Currently the verification is manually done. 

1. Login to the MECM Node and check whether K8S cluster is installed. 
2. Check grafana and AppLCM services are running as PODS. 
3. Login to MEP Node and check K3S is installed. 

Developer  Guide and Troubleshooting

Uninstall Guide

...

Following two files can be used to revert the changes made by ELIOT deployment through kubernetes.

...

...

...

Troubleshooting

...

1) Core-dns pod instability issue - CrashloopBack off / Error / Restarts while executing

$ kubectl get pods -n kube-system

Cause: /etc/resolv.conf is not configured properly to establish the network with coredns pod.

Solution:

check /etc/resolv.conf  → whether it is having the content as below

nameserver 8.8.8.8

2) Calico pods ( CNI ) instability issue - restarting the calico pods with unstable network connectivity

while executing $ kubectl get pods -n kube-system or getting BGP connection not estabilished in calico-node-<xyz> . Bird is not ready

while executing $ kubectl describe pods -n  kube-system calico-node-<xyz>

Cause: Error occurred due to network interface of the vm / machine which calico cni is using can't be able to connect to the internet.

Solution:

We have to execute below command in ELIOT Master node

$ kubectl set env daemonset/calico-node -n kube-system IP_AUTODETECTION_METHOD=can-reach=www.google.com    

...

$ kubectl set env daemonset/calico-node -n kube-system IP_AUTODETECTION_METHOD=interface=<interface-name>

<interface-name> must be the interface which the machine / VM can be able to connect with the internet

Kindly refer the below link to check detailed info

https://docs.projectcalico.org/networking/ip-autodetection

Error Message Guide

N/A

Maintenance

Blueprint Package Maintenance

Software maintenance

N/A

Hardware maintenance

N/A

Blueprint Deployment Maintenance

N/A

Frequently Asked Questions

N/A

License

Any software developed by the "Akraino

...

Enterprise Applications on Lightweight 5G Telco Edge Project is licensed under the
Apache License, Version 2.0 (the "License");
you may not use the content of this software bundle except in compliance with the License.
You may obtain a copy of the License at <https://www.apache.org/licenses/LICENSE-2.0>

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

References

Definitions, acronyms and abbreviations