Introduction

This guide provides instructions for installing and configuring the Smart Data Transaction for CPS blueprint, and also includes recommended hardware and software requirements for the blueprint. The guide describes a minimal installation of the blueprint consisting of a single "master" node and two "edge" nodes, with directions on how the number of nodes can be modified as needed.

How to Use This Document

This document assumes the reader is familiar with basic UNIX command line utilities and Kubernetes. Familiarity with Ansible and Docker may also be useful.

Start by reviewing the deployment architecture and requirements in the following sections, then follow the steps in the Installation section to set up the software and start it running. Confirm the services are functioning as expected by following the instructions in the Verifying the Setup section. The later sections in this document describe other tasks that can be performed on a running setup, alternate configuration options, and how to shut down and uninstall the software.

Deployment Architecture

The diagram below shows the major components and relationships in a deployment of this blueprint.

Deployment, as well as other tasks such as starting and stopping the cluster, is coordinated through a set of Ansible playbooks. (Ansible playbooks are a system used by the Ansible tool for describing the desired state of a system. In many ways they are similar to shell scripts. For more details see the Ansible documentation.) The playbooks are run exclusively by the deploy node, and they execute commands on the deploy node, the master node, and in some cases on the edge nodes. Once the nodes are set up, most activity is carried out by Kubernetes. Kubernetes is configured by the playbooks and told to start or stop services on the edge nodes. These services are run in containers, and the images for these containers are stored in a local Docker registry. There are containers for the Kubernetes components themselves, plus Flannel (a component which provides networking inside the Kubernetes cluster), EdgeX Foundry services, and two custom services (sync-app and device-lora) built using the EdgeX SDKs.

Note that the deploy node and the master node can be the same host or virtual machine.

The sensor nodes are not shown in the above diagram as they are not envisioned as being connected to the network, and are not configured by the playbooks from the deploy node. See the Sensor Nodes section of Installation for an example of how sensor nodes may be setup. 

Pre-Installation Requirements

Hardware Requirements

The table below shows the recommended minimum specifications for the hardware in the testing installation. It is possible that lower spec hardware could be used for many of the nodes. The sensor node hardware in particular is specific to the testing installation and could be swapped out with any number of other platforms as long as LoRa connectivity was possible using the hardware.

At a minimum one node is required for the master and deploy roles together, and at least one edge node and one sensor node. The testing installation uses two edge and sensor nodes.

Network Requirements

All nodes are expected to have IP connectivity to one another during installation and normal operation, with the exception of the sensor nodes. In the installation described here, all the nodes are connected to a private wired network operating at 100Mbps or better. However, there are no strict bandwidth or latency requirements.

During initial software installation all of the nodes will require access to the internet to download required software packages. Once the required software packages are installed and the docker registry is started, only the deploy node will need further access to the internet (unless, of course, software packages need to be changed or updated). The deploy node will need to access the internet when pulling upstream images to install in the docker registry, and when building docker images for custom services. Of course, if external tools are going to be used to access the collected data through the MQTT broker (Mosquitto), those tools will need network access to the master node.

When the edge node services are started, images will be downloaded from the docker registry on the master node to the edge nodes, so bandwidth may be a consideration if, for example, the edge nodes are accessed over a mobile network.

Software Prerequisites

The list below shows the required software for each node type prior to beginning the installation process.

• Deploy node
  • Ubuntu 20.04
  • Ansible 2.11.7
• Master node
  • Ubuntu 20.04
• Edge node
  • Ubuntu 20.04
• Sensor node
  • Rasbian 11.1

Note that Ansible 2.9.6 is installed from the regular Ubuntu repository on Ubuntu 20.04, but needs to be upgraded from the Ansible repository to support the kubernetes.core collection used by this blueprint. The setup_cicd.yml playbook can be run with Ansible 2.9.6 and will update Ansible to the required version.

Additional Installed Software Packages 

Note that the installation process will install several more software packages through Ansible playbooks. These are listed below for reference. Packages included by default in an install of Ubuntu 20.04 server are not included. The version numbers are those that are available/installed at the time of writing by the Ansible playbooks on Ubuntu 20.04.

• Deploy node
  • make 4.2.1, build-essential 12.8, python3-pip 20.0.2
  • Ansible collections community.docker, kubernetes.core, community.crypto
  • Docker (docker.io) 20.10.7
• Master node
  • Docker (docker.io) 20.10.7
  • python3-pip 20.0.2
  • Python packages cryptography
  • mosquitto 2.0.14, mosquitto-clients 2.0.14
  • Kubernetes (kubectl, kubelet, kubeadm) 1.22.6
  • Flannel 0.16.3, flannel-cni-plugin 1.0.1 (Note: These are containers installed via Kubernetes through a config file)
• Edge node
  • Docker (docker.io) 20.10.7
  • Kubernetes (kubelet, kubeadm) 1.22.6 (kubectl may be installed for debugging purposes)

Installation

Setting Up the Deploy Node

The deploy node will coordinate all other installation and operations, so it needs to be set up first. In the test installation, the deploy node is a VM running on a x86 PC, with Ubuntu Linux 20.04 installed. In addition, the Ansible tool must be installed. The Ansible tool provided in the Ubuntu software repository is a slightly older version which needs to be upgraded, but it is sufficient to execute the setup_deploy.yml playbook, which will install the newer version of Ansible and other tools required on the deploy node. But before running that playbook you need to configure a few things described in the section below.

The playbooks for use on the deploy node are stored in the deploy/playbook directory of the source repository. These playbooks refer to other files in the source code, so the entire directory tree should be copied onto the deploy node. The easiest way to do this is by cloning the git repository directly as shown below:

git clone <repository-url>

Note, using the --depth=1 option can save some disk space.

The git command will create a directory in the directory where it is run named after the repository. Inside the new directory will be the deploy/playbook directory. Unless noted otherwise, the commands below should be run in that directory.

Node and Cluster Configuration

Before running the setup_deploy.yml playbook

Set Up the Deploy Node

After setting the configuration variables in the master.yml and hosts files, you can run the setup_deploy.yml playbook using the command below.

ansible-playbook -i ./hosts setup_deploy.yml

Preparing the Master Node

Creating the Docker Registry

Preparing Edge Nodes

Starting the Cluster

Adding Edge Nodes to the Cluster

Starting EdgeX

Sensor Nodes

Verifying the Setup

as defined the Akraino validation feature project plus any additional testing specific to the blue print

Developer Guide and Troubleshooting

Changing Configuration

Enabling and Disabling Optional Services

Debugging Failures

Reporting a Bug

Uninstall Guide

Stopping EdgeX

Removing Edge Nodes

Stopping Kubernetes

Stopping the Docker Registry

Uninstalling Software Components

Removing Configuration and Temporary Data

Troubleshooting

Confirming Node and Service Status

Accessing Logs

Maintenance

Accessing EdgeX Service Configuration

Stopping and Restarting EdgeX Services

Stopping and Restarting the Kubernetes Cluster

Adding and Removing Edge Nodes

Updating the Software

Rebuilding Custom Services

License

The software provided as part of the Smart Data Transaction for CPS blueprint is licensed under the Apache License, Version 2.0 (the "License");

You may not use the content of this software bundle except in compliance with the License.

You may obtain a copy of the License at <https://www.apache.org/licenses/LICENSE-2.0>

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and limitations under the License.

The synchronization application and LoRa device service are linked with other Go packages/components when compiled, which are each covered by their own licenses, listed below. Other components downloaded and installed during the blueprint's installation process are covered by their own licenses.

Synchronization Application

The synchronization application is linked with the following packages when compiled:

LoRa Device Service

The LoRa device service is linked with the following packages when compiled:

References

Definitions, Acronyms and Abbreviations