...
Deployment, as well as other tasks such as starting and stopping the cluster, is coordinated through a set of Ansible playbooks. (Ansible playbooks are a system used by the Ansible tool for describing the desired state of a system. In many ways they are similar to shell scripts. For more details see the Ansible documentation.) The playbooks are run by the deploy node and build node, and they execute commands on the deploy node, the master node, the build node, and in some cases on the edge nodes. Once the nodes are set up, most activity is carried out by Kubernetes. Kubernetes is configured by the playbooks and told to start or stop services on the edge nodes. These services are run in containers, and the images for these containers are stored in a local Docker registry. There are containers for the Kubernetes components themselves, plus Flannel (a component which provides networking inside the Kubernetes cluster), EdgeX Foundry services, and four custom services (sync-app and , image-app and , device-lora and device-camera) built using the EdgeX SDKs.
...
- Build node
- make 4.2.1, build-essential 12.8, python3-pip 20.0.2, default-jre 2:1.11-72
- Robot Framework 6.0
- Docker (docker.io) 20.10.12
- Go 1.16.10
- Deploy node
- make 4.2.1, build-essential 12.8, python3-pip 20.0.2
- Ansible 2.12.95
- Ansible collections
community.docker,kubernetes.core,community.crypto
- Master node
- Docker (docker.io) 20.10.12
- python3-pip 20.0.2
- Python packages
cryptographyandkubernetes - mosquitto 2.0.15, mosquitto-clients 2.0.15
- Kubernetes (kubectl, kubelet, kubeadm) 1.22.9
- Flannel 0.17.0, flannel-cni-plugin 1.0.1 (Note: These are containers installed via Kubernetes through a config file)
- Edge node
- Docker (docker.io) 20.10.12
- Kubernetes (kubelet, kubeadm) 1.22.9 (kubectl may be installed for debugging purposes)
...
Modify the host names and ip addresses of master/, build/, cicd /and deploy nodes in the cicd/playbook/hosts file.
...
At this time, images for the four custom services, sync-app and , image-app and , device-lora and device-camera, need to be built from source and pushed to the private Docker registry. (In the future these images should be available on Docker Hub or another public registry.) Use the following playbooks from the cicd/playbook directory on the build deploy node to do so.
Note, limited by base image NVIDIA L4T CUDA which only supports arm architecture, so custom service image-app also only supports arm architecture. Other custom services support both arm64 and amd64 architecturearchitectures.
This command executed on deploy node will build local docker images of the custom microservices:
...
ansible-playbook -i ./hosts push_images.yml
At time of writing this step will also create some workaround images required to enable EdgeX security features in this blueprint's Kubernetes configuration. Hopefully, these images will no longer be needed once fixes have been made upstream.
Starting the Cluster
With the base software installed and configured on the master and edge nodes, the following command executed on the deploy node will start the cluster:
...
Note, during initialization of the services you may see some containers restart one or more times. This is part of the timeout and retry behavior of the services waiting for other services to complete initialization and does not indicate a problem.
Camera Nodes
Configuration of the Camera Nodes (TODO)
These readings Readings received from Camera nodes should appear in the core-data database and be possible to monitor using the edgex-events-nodename channel. For example, the following command run on the master node should show the readings arriving at an edge node named "jet03":
...
The custom services can be rebuilt by running the build_images.yml playbook in cicd/playbook. After successfully building a new version of a service, use push_images.yml to push the images to the private Docker registry. The source for the services is found in edgex/sync-app and , edgex/image-app, edgex/device-camera and edgex/device-lora.
License
...