...
This command should be run before stopping the cluster as described in the following section, in order to provide a clean shutdown. It is also possible to re-add the edge nodes using join_cluster.yml
, perhaps after editing the configuration in the hosts
file.
Stopping Kubernetes
Kubernetes can be stopped by running the following command. Do this after all edge nodes have been removed.
ansible-playbook -i ./hosts reset_cluster.yml --ask-become-pass
Stopping and Clearing the Docker Registry
Uninstalling Software Components
Removing Configuration and Temporary Data
Troubleshooting
Confirming Node and Service Status
The kubectl
command can be used to check the status of most cluster components. kubectl get node
will show the health of the master and edge nodes, and kubectl get pod
will show the overall status of the EdgeX services. The kubectl describe pod pod-name
command can be used to get a more detailed report on the status of a particular pod. The EdgeX configuration UI, described in the section EdgeX Service Configuration UI above, also shows the result of an internal health check of all EdgeX services on the node.
Accessing Logs
The main tool for accessing logs is kubectl logs, run on the master node. This command can be used to show the logs of a running container:
kubectl logs -c container-name pod-name
It can also be used to check the logs of a container which has crashed or stopped:
kubectl logs --previous -c container-name pod-name
And it can be used to stream the logs of a container to a terminal:
kubectl logs -c container-name pod-name -f
The container names can be found in the output of kubectl describe pod
or in the edgex/deployments/edgex.yml
file (the names of the entries in the containers
list).
For the rare cases when the Kubernetes log command does not work, it may be possible to use the docker log
command on the node you wish to debug.
Maintenance
Stopping and Restarting EdgeX Services
As described in the Uninstall Guide subsection Stopping EdgeX, the EdgeX services can be stopped and restarted using the edgex_stop.yml
and edgex_start.yml
playbooks.
Stopping and Restarting the Kubernetes Cluster
Similar to stopping and restarting the EdgeX services, the whole cluster can be stopped and restarted by stopping EdgeX, removing the edge nodes, stopping Kubernetes, starting Kubernetes, adding the edge nodes, and starting EdgeX again:
ansible-playbook -i ./hosts edgex_stop.yml
ansible-playbook -i ./hosts delete_from_cluster.yml
ansible-playbook -i ./hosts reset_cluster.yml --ask-become-pass
ansible-playbook -i ./hosts init_cluster.yml --ask-become-pass
ansible-playbook -i ./hosts join_cluster.yml
ansible-playbook -i ./hosts edgex_start.yml
Adding and Removing Edge Nodes
Updating the Software
...
If you need to stop the private Docker registry service for some reason, use the following command:
ansible-playbook -i ./hosts stop_registry.yml
With the registry stopped it is possible to remove the registry entirely. This will recover any disk space used by images stored in the registry, but means that pull_upsteam_images.yml, build_images.yml, and push_images.yml will need to be run again.
ansible-playbook -i ./hosts remove_registry.yml
Uninstalling Software Components
Installed software components can be removed with sudo apt remove package-name
. See the list of installed software components earlier in this document. Python packages (cryptography
and kubernetes
) can be removed with the pip uninstall
command.
Ansible components installed with ansible-galaxy (community.docker
, kubernetes.core
, community.crypto
) can be removed by deleting the directories under ~/.ansible/collections/ansible_collections
on the deploy node.
Removing Configuration and Temporary Data
This blueprint stores configuration and data in the following places. When uninstalling the software, these folders and files can also be removed, if present, on the master, deploy and edge nodes.
- Master node:
- ~/.lfedge
- /opt/lfedge
- /etc/mosquitto/conf.d/edge.conf
- /usr/share/keyrings/kubernetes-archive-keyring.gpg
- Edge node:
- /opt/lfedge
- /etc/docker/certs.d/master:5000/registry.crt
- /usr/local/share/ca-certificates/master.crt
- /etc/docker/daemon.json
- /usr/share/keyrings/kubernetes-archive-keyring.gpg
- Deploy node:
- /etc/profile.d/go.sh
- /usr/local/go
- ~/edgexfoundry
Troubleshooting
Confirming Node and Service Status
The kubectl
command can be used to check the status of most cluster components. kubectl get node
will show the health of the master and edge nodes, and kubectl get pod
will show the overall status of the EdgeX services. The kubectl describe pod pod-name
command can be used to get a more detailed report on the status of a particular pod. The EdgeX configuration UI, described in the section EdgeX Service Configuration UI above, also shows the result of an internal health check of all EdgeX services on the node.
Accessing Logs
The main tool for accessing logs is kubectl logs, run on the master node. This command can be used to show the logs of a running container:
kubectl logs -c container-name pod-name
It can also be used to check the logs of a container which has crashed or stopped:
kubectl logs --previous -c container-name pod-name
And it can be used to stream the logs of a container to a terminal:
kubectl logs -c container-name pod-name -f
The container names can be found in the output of kubectl describe pod
or in the edgex/deployments/edgex.yml
file (the names of the entries in the containers
list).
For the rare cases when the Kubernetes log command does not work, it may be possible to use the docker log
command on the node you wish to debug.
Maintenance
Stopping and Restarting EdgeX Services
As described in the Uninstall Guide subsection Stopping EdgeX, the EdgeX services can be stopped and restarted using the edgex_stop.yml
and edgex_start.yml
playbooks.
Stopping and Restarting the Kubernetes Cluster
Similar to stopping and restarting the EdgeX services, the whole cluster can be stopped and restarted by stopping EdgeX, removing the edge nodes, stopping Kubernetes, starting Kubernetes, adding the edge nodes, and starting EdgeX again:
ansible-playbook -i ./hosts edgex_stop.yml
ansible-playbook -i ./hosts delete_from_cluster.yml
ansible-playbook -i ./hosts reset_cluster.yml --ask-become-pass
ansible-playbook -i ./hosts init_cluster.yml --ask-become-pass
ansible-playbook -i ./hosts join_cluster.yml
ansible-playbook -i ./hosts edgex_start.yml
Adding and Removing Edge Nodes
Edge nodes can be added an removed by stopping the cluster and editing the deploy/playbook/hosts
file. The master_install.yml
and edge_install.yml
playbooks need to be run again to update /etc/hosts
and certificates on any added nodes.
Updating the Software
Running setup_deploy.yml, master_install.yml, and edge_install.yml playbooks can be used to update software packages if necessary. Note that Kubernetes is specified to use version 1.22 to avoid problems that might arise from version instability, but it should be possible to update if so desired.
Rebuilding Custom Services
The custom services can be rebuilt by running the build_images.yml
playbook in cicd/playbook
. After successfully building a new version of a service, use push_images.yml to push the images to the private Docker registry. The source for the services is found in edgex/sync-app
and edgex/device-lora
.
License
The software provided as part of the Smart Data Transaction for CPS blueprint is licensed under the Apache License, Version 2.0 (the "License");
...