This is a Golang application providing a RESTful API to interact with and upload image objects.
The API application source files are in the icn/cmd/bpa-restapi-agent directory.
While the database back-end is extensible, this initial release requires mongodb.
Motivation:
The ICN RESTful API intends to provide access to resources inside the system using a uniform interface through the provision of one logical URI. As a constriant, "Servers and clients may also be replaced and developed independently, as long as the interface between them is not altered."
Requirements:
This RESTful API service will expose the following resources to a user:
- Binary Images
- Container Images
- OS Images
Design:
- Use nouns to describe resources
- In the top URL, identify version, bare-metal cluster name, and resource
- GET - This will be used to list image resources
- POST - This will be used to create image resources
- PUT - This will be used to upload image resources
- DELETE - This will be used to delete image resources
Proposed Implementation:
Sample GET Implementation:
The sample API above was create using the swagger.yaml below.
swagger.yml
swagger: "2.0" error: |
---|
Testing:
Proposed sample command to GET all binary images
# Get all binary images
curl -i http:
//localhost
:9015
/v1/baremetalcluster/{clustername}/binary_images/
# Get one binary image
curl -i http:
//localhost
:9015
/v1/baremetalcluster/{clustername}/binary_images/{name}
Proposed sample command to GET all container images
# Get all container images
curl -i http://localhost:9015/v1/baremetalcluster/{clustername}/container_images/
# Get one container image
curl -i http://localhost:9015/v1/baremetalcluster/{clustername}/container_images/{name}
Proposed sample command to GET all OS images
# Get all container images
curl -i http://localhost:9015/v1/baremetalcluster/{clustername}/os_images/
# Get one container image
curl -i http://localhost:9015/v1/baremetalcluster/{clustername}/os_images/{name}
How To Use The BPA RESTful API Service
Install and start mongodb. For instructions: https://docs.mongodb.com/manual/installation/
git clone "https://gerrit.akraino.org/r/icn"
cd icn/cmd/bpa-restapi-agent
Run the application
go run main.go
Sample output without a config file:
2019/08/22 14:08:41 Error loading config file. Using defaults
2019/08/22 14:08:41 Starting Integrated Cloud Native API
# Cloud Storage with MinIO
Start MinIO server daemon with docker command before run REST API agent, default settings in config/config.go.
AccessKeyID: ICN-ACCESSKEYID
SecretAccessKey: ICN-SECRETACCESSKEY
MinIO Port: 9000
```
$ docker run -p 9000:9000 --name minio1 \
-v /mnt/data:/data \
-v /mnt/config:/root/.minio \
minio/minio server /data
```
MinIO Client will automatic initialize in main.go, and create 3 buckets: binary, container, operatingsystem.
The Upload image will "PUT" to corresponding buckets by HTTP PATCH request url.
You can also check by open browser: http://127.0.0.1:9000/
RESTful API usage examples
Sample POST request format
curl -i -F "metadata=<jsonfile;type=application/json" -F file=@/home/<username>/<dir>/jsonfile -X POST http://NODE_IP:9015//baremetalcluster/{owner}/{clustername}/<image_type>
#image type can be binary_image, container_image, or os_image
Example requests and responses:
Create image - POST
#Using a json file called sample.json
#image_length in sample.json can be determined with the command
ls -al <image_file>
Request
curl -i -F "metadata=<sample.json;type=application/json" -F file=@/home/enyi/workspace/icn/cmd/bpa-restapi-agent/sample.json -X POST http://localhost:9015/v1/baremetalcluster/alpha/beta/container_images
Response
HTTP/1.1 100 Continue
HTTP/1.1 201 Created
Content-Type: application/json
Date: Thu, 22 Aug 2019 22:56:16 GMT
Content-Length: 239
{"owner":"alpha","cluster_name":"beta","type":"container","image_name":"asdf246","image_offset":0,"image_length":29718177,"upload_complete":false,"description":{"image_records":[{"image_record_name":"iuysdi1234","repo":"icn","tag":"1"}]}}
#this creates a database entry for the image, and an empty file in the file system
List image - GET
curl -i -X GET http://localhost:9015/v1/baremetalcluster/{owner}/{clustername}/<image_type>/{imgname}
example:
#continuing with our container image from above
Request
curl -i -X GET http://localhost:9015/v1/baremetalcluster/alpha/beta/container_images/asdf246
Response
HTTP/1.1 200 OK
Content-Type: application/json
Date: Thu, 22 Aug 2019 22:57:10 GMT
Content-Length: 239
{"owner":"alpha","cluster_name":"beta","type":"container","image_name":"asdf246","image_offset":0,"image_length":29718177,"upload_complete":false,"description":{"image_records":[{"image_record_name":"iuysdi1234","repo":"icn","tag":"1"}]}}
Upload container image - PATCH
Request
curl --request PATCH --data-binary "@/home/enyi/workspace/icn/cmd/bpa-restapi-agent/sample_image" http://localhost:9015/v1/baremetalcluster/alpha/beta/container_images/asdf246 --header "Upload-Offset: 0" --header "Expect:" -i
Response
HTTP/1.1 204 No Content
Upload-Offset: 29718177
Date: Thu, 22 Aug 2019 23:19:44 GMT
Check uploaded image - GET
Request
curl -i -X GET http://localhost:9015/v1/baremetalcluster/alpha/beta/container_images/asdf246
Response
HTTP/1.1 200 OK
Content-Type: application/json
Date: Fri, 23 Aug 2019 17:12:07 GMT
Content-Length: 245
{"owner":"alpha","cluster_name":"beta","type":"container","image_name":"asdf246","image_offset":29718177,"image_length":29718177,"upload_complete":true,"description":{"image_records":[{"image_record_name":"iuysdi1234","repo":"icn","tag":"1"}]}}
#after the upload, the image_offset is now the same as image_length and upload_complete changed to true
#if upload was incomplete
Resumable upload instructions
Resumable upload -PATCH
#this is the current resumable upload mechanism
Request
curl --request PATCH --data-binary "@/home/enyi/workspace/icn/cmd/bpa-restapi-agent/sample_image" http://localhost:9015/v1/baremetalcluster/alpha/beta/container_images/asdf246 --header "Upload-Offset: 0" --header "Expect:" -i --limit-rate 200K
#the above request limits transfer for testing purposes
#'ctl c' out after a few seconds, to stop file transfer
#check image_offset with a GET
Check upload - GET
Request
curl -i -X GET http://localhost:9015/v1/baremetalcluster/alpha/beta/container_images/asdf246
Response
HTTP/1.1 200 OK
Content-Type: application/json
Date: Sat, 24 Aug 2019 00:30:00 GMT
Content-Length: 245
{"owner":"alpha","cluster_name":"beta","type":"container","image_name":"asdf246","image_offset":4079616,"image_length":29718177,"upload_complete":false,"description":{"image_records":[{"image_record_name":"iuysdi1234","repo":"icn","tag":"2"}]}}
#from our response you can see that image_offset is still less than image_length and #upload_complete is still false
#next we use the dd command (no limiting this time)
Request
dd if=/home/enyi/workspace/icn/cmd/bpa-restapi-agent/sample_image skip=4079616 bs=1 | curl --request PATCH --data-binary @- http://localhost:9015/v1/baremetalcluster/alpha/beta/container_images/asdf246 --header "Upload-Offset: 4079616" --header "Expect:" -i
#the request skips already uploaded 4079616 bytes of data
Response
25638561+0 records in
25638561+0 records out
25638561 bytes (26 MB, 24 MiB) copied, 207.954 s, 123 kB/s
HTTP/1.1 204 No Content
Upload-Offset: 29718177
Date: Sat, 24 Aug 2019 00:43:18 GMT
Update image description - PUT
# let's change the tag in description from 1 to latest
# once the change is made in sample.json (or your json file)
Request
curl -i -F "metadata=<sample.json;type=application/json" -F file=@/home/enyi/workspace/icn/cmd/bpa-restapi-agent/sample.json -X PUT http://localhost:9015/v1/baremetalcluster/alpha/beta/container_images/asdf246
Response
HTTP/1.1 100 Continue
HTTP/1.1 201 Created
Content-Type: application/json
Date: Fri, 23 Aug 2019 17:21:01 GMT
Content-Length: 239
{"owner":"alpha","cluster_name":"beta","type":"container","image_name":"asdf246","image_offset":0,"image_length":29718177,"upload_complete":false,"description":{"image_records":[{"image_record_name":"iuysdi1234","repo":"icn","tag":"2"}]}}
Delete an image - DELETE
Request
curl -i -X DELETE http://localhost:9015/v1/baremetalcluster/alpha/beta/container_images/asdf246
How To Write ICN Unit Tests in Go using Testify
Go version used is go1.12.9
To install Go, follow the instructions on the official site, (https://golang.org/doc/install).
Testify documentation can be found on Github, https://github.com/stretchr/testify
The first question is, "what to test?" In this case, I'll test a method called GetDirPath.
GetDirPath uses the Go core packages user and path to return a file path.
func (v *ImageClient) GetDirPath(imageName string) (string, string, error) { home, err := v.GetHomeDir() dirPath := path.Join(home, "images", v.storeName) filePath := path.Join(dirPath, imageName) return filePath, dirPath, err }
To Do - Explain
To test, I'll create the test file image_test.go. A test for the GetDirPath could look like this:
To Do - Explain
References: