What is a Blueprint?

In the object model that the Regional Controller provides a Blueprint is a formal description (in the form of a YAML file) of all the requirements for how an Akraino Blueprint can be installed and lifecycle managed on an Edgesite (a cluster of nodes). As such, a Blueprint provides:

An Example Blueprint

The bare bones of a Blueprint looks like:

# The Blueprint schema is required and is always 1.0.0
blueprint: 1.0.0

# The name and version are required, and should describe the Blueprint
name: Hello World Blueprint
version: 1.0.0

# The description is optional
description: This Blueprint demonstrates what is needed in a minimal ARC blueprint.

# The YAML is required and contains the workflow and hardware details
yaml:
  # The hardware_profile section describes what types of hardware this Blueprint may run on
  hardware_profile:
    or:
      - { uuid: c1dfa1ac-53e0-11e9-86c2-c313482f1fdb }
      - { name: 'HPE DL380.*' }

  # The workflow section lists all of the workflows
  workflow:
    # The create workflow is invoked when the POD is created
    create:
      # The Python script that is run to create the POD
      url: http://www.example.org/hello-world/create.py
      # input_schema describes what data is required to be POST-ed when the POD is created
      input_schema:
        name: { type: string }
        region: { type: string }
    # The delete workflow is invoked when the POD is DELETE-ed
    delete:
    # All other workflows are invoked when the POD is updated via PUT
    update_1:

How to Write Workflows

The Workflow Names

Workflows are pulled from the yaml.workflow stanza of the Blueprint associated with a POD. Every Blueprint should have a workflow named create for creating the POD, and one named delete, for use when the POD is torn down (deleted).

'update' workflows (which are invoked via a PUT on the POD’s URL) may have any name other than create or delete. They should probably start with update_, and there may be several update workflows (for different types of updates). Workflow names must be alphanumeric (_ is also allowed) and may not be longer than 36 characters long.

The Workflow Execution Procedure

When the RC prepares to start a workflow, it performs the following steps:

  1. Checks for the stanza in the Blueprint associated with the POD for a workflow of the type requested. For example, when first creating a POD, the create workflow is used, and so, the Blueprint is checked for a yaml.workflow.create stanza.

  2. It then builds a workspace directory for the workflow to operate in. This directory will contain all scripts, components, and other files needed by the workflow engine to perform its job. The directory is shared between the instances of Airflow and the API server.

  3. It then attempts to pull in the workflow script specified in the Blueprint, as well as any required component files (also specified in the Blueprint). These are placed into the workspace directory.

  4. It then creates a parent workflow from an internal template. This workflow consists of three tasks that are executed in succession.

    1. A preamble task that sets things up by appending the workspace directory to sys.path (Python) or $PATH (bash), and importing the user’s workflow file.
    2. The main task that just invokes the Blueprints’ workflow.
    3. A postamble task that cleans the workspace and performs some final bookkeeping (marks the POD as being ACTIVE. Note that if the the main task is a long running one, or never finishes, then the postamble may not run. However, the postamble must run before any other workflows may be scheduled for a particular POD.
  5. The parent workflow file is then moved into the Airship dags directory in order to cause Airship to register and start the workflow. While the workflow is running, no other operations may be performed upon the POD. Note: It can take Airflow up to 5 minutes before it notices that a new DAG has been placed in the dags directory; as a result, there is usually a lag between when a workflow is started and the workflow actually executes inside Airship.

Workflows may be either Python scripts or shell (bash) scripts. Python is recommended since it provides a bit more flexibility when it comes to working with DAGs and YAML files. The filename at the end of the URL specifying the workflow determines both the filename of the workflow file, and how it will be invoked. If it ends in .py, it is assumed to be a Python script; if it ends with .sh it is assumed to be a bash script. Any other ending is invalid. If the workflow is to have multiple tasks in a sub-DAG, then the script must be a Python script written according to the rules of Airflow.

Blueprint workflows may invoke other shell scripts or Python scripts. If specified in the Blueprint in the components section, then the RC will automatically download them into the workspace. Otherwise, it is up to the workflow itself to fetch any required files. This includes, for example, any Docker containers which the workflow may start.

Python Execution Environment

Workflows written in Python may assume the following is available:

The version of Python used inside Airflow is 3.6.

The Python script itself will be started via the main task (see above). It will call the start function in the Python workflow file which is required to have the following signature:

def start(ds, **kwargs):

Bash Execution Environment

Workflows written in bash may assume the following is available:

Note: any Docker containers run by the workflow should be run with the --rm option, in order to be automatically cleaned up on exit. In addition, these containers will run parallel to the Airflow containers in the RC. A such, they should use the --net=arc-net option, if they need to share the same Docker network with the rest of the RC.

The version of bash used inside Airflow is 4.4.12.