Page History

...

Storage Service for Local Controller

The requirement initial from RESTful API Binary Provisioning Agent which provide interface to operator to upload image objects.

...

1. Image provisioning (upload)
2. REST API (POST/PUT/PATCH/DELETE etc)
3. Support resumable upload

And backend storage is needed, the relationship as following chart.

Image Modified

Backend Storage

...

Solutions

There are Also we have several considerations for the backend storage solution:

1. Data reliability: some mechanism like replication to make sure the data can be recovered.
2. Cloud native: Support docker or Kubernetes deployment.
3. Amazon S3 API compatible: Support S3 API which is factual standard.
4. Multipart upload is better: which can speed up or resume when upload thread is break.

Solution 1: GrideFS

Image Modified

Solution 2: MinIO

Image Modified

Image Removed

Implementation

We choose MinIO as the backend storage solution which So the better solution is MinIO, which can provide Cloud Storage service Service for Local Controller.

MinIO Client plugin for RESTful API Agent

Image Removed

MinIO Server and storage service

MinIO server deploy by Kubernetes, which has 2 modes: Standalone and Distribute.

The volume of MinIO export also can be used for HTTP server.

The implementation for the module is to develop MinIO client plugin based on MinIO Golang API, and integrate with REST API agent, include interface for MinIO client plugin as:

1. Initialize(), which create new client and setup buckets
2. PutImage(), put the object file to MinIO server, support /PUT
3. PatchImage(), Put the object with offset and compose, support /PATCH
4. DeleteImage(), Delete the image
5. CleanupImages(), Cleanup and remove incomplete upload threads

Deployment

the reliable volume is provided by Ceph cluster with CSI,  and which used by MinIO standalone server, MinIO clinet plugin code interact with S3 API to provide object service for REST API agent.

Image Added

Optane Persistent Memory Plugin

The term persistent memory is used to describe technologies which allow programs to access data as memory, directly byte-addressable, while the contents are non-volatile, preserved across power cycles. It has aspects that are like memory, and aspects that are like storage.

Intel Optane DC Persistent Memory is an innovative technology that delivers a unique combination of affordable large memory capacity and persistence (non-volatility). The persistent memory technology can help boost the performance of data-intensive applications, such as in-memory analytics, databases, content delivery networks, and high performance computing (HPC), as well as deliver consistent service levels at scale with higher virtual machine and container density.

Image Added

To enable the Optane PM (Persistent Memory) on cloud native application, Intel has develop the PMEM-CSI driver for Kubernetes. Intel PMEM-CSI is a CSI (Container Storage Interface) storage driver for container orchestrators like Kubernetes. It makes local persistent memory (PMEM) available as a filesystem volume to container applications. It can currently utilize non-volatile memory devices that can be controlled via the libndctl utility library.

dImage Removed