Page History

Table of Contents

maxLevel	3

Introduction

This document describes the blueprint test environment for the Smart Data Transaction for CPS blueprint. The test results and logs are posted in the Akraino Nexus at the link below:Insert link here

https://nexus.akraino.org/content/sites/logs/fujitsu/job/sdt/r6/

Akarino Test Group Information

...

The test script will start the registry, pull upstream images and populate the registry, clean images left over from the pull process, stop the registry, and remove the registry. The robot command should report success for all test cases.

Test Results

Nexus URL: https://nexus.akraino.org/content/sites/logs/fujitsu/job/sdt/r6/lfedge-docker/1/

Pass (5/5 test cases)

CI/CD Regression Tests: Node Setup

...

The test scripts will initialize the master and edge nodes and verify the required software is installed. The robot command should report success for all test cases.

Test Results

Image Removed

Nexus URL: https://nexus.akraino.org/content/sites/logs/fujitsu/job/sdt/r6/lfedge-install/1/

Image Added

Pass (Pass (2/2 test cases)

CI/CD Regression Tests: Cluster Setup & Teardown

...

The test scripts will start the cluster, add all configured edge nodes, remove the edge nodes, and reset the cluster. The robot command should report success for all test cases.

Test Results

Nexus URL: https://nexus.akraino.org/content/sites/logs/fujitsu/job/sdt/r6/lfedge-cluster/7/

Pass (4/4 test cases)

CI/CD Regression Tests: EdgeX Services

...

The test scripts will start the EdgeX micro-services on all edge nodes, confirm that MQTT messages are being delivered from the edge nodes, and stop the EdgeX micro-services. The robot command should report success for all test cases.

Test Results

Image Removed

Nexus URL: https://nexus.akraino.org/content/sites/logs/fujitsu/job/sdt/r6/edgex-install/7/

Image Added

Pass (Pass (8/8 test cases)

CI/CD Regression Tests: LoRa Device Service

...

The Robot Framework should report success for all test cases.

Test Results

Nexus URL: https://nexus.akraino.org/content/sites/logs/fujitsu/job/sdt/r6/edgex-lora/3/

Pass (2/2 test cases)

Feature Project Tests

...

We use Ubuntu 20.04, so we ran Vuls test as follows:

Create directory

$ mkdir ~/vuls
$ cd ~/vuls
$ mkdir go-cve-dictionary-log goval-dictionary-log gost-log

Fetch NVD

$ docker run --rm -it \
    -v $PWD:/go-cve-dictionary \
    -v $PWD/go-cve-dictionary-log:/var/log/go-cve-dictionary \
    vuls/go-cve-dictionary fetch nvd

Fetch OVAL

$ docker run --rm -it \
     -v $PWD:/goval-dictionary \
     -v $PWD/goval-dictionary-log:/var/log/goval-dictionary \
     vuls/goval-dictionary fetch ubuntu 16 17 18 19 20

Fetch gost

$ docker run --rm -i \
     -v $PWD:/gost \
     -v $PWD/gost-log:/var/log/gost \
     vuls/gost fetch ubuntu

Create config.toml

[servers]

[servers.master]
host = "192.168.51.22"
port = "22"
user = "test-user"
sshConfigPathkeyPath   = "/root/.ssh/config"
keyPath = "/root/.ssh/id_id_rsa" # path to ssh private key in docker

Start vuls container to run tests

$ docker run --rm -it \
    -v ~/.ssh:/root/.ssh:ro \
    -v $PWD:/vuls \
    -v $PWD/vuls-log:/var/log/vuls \
    -v /etc/localtime:/etc/localtime:ro \
    -v /etc/timezone:/etc/timezone:ro \
    vuls/vuls scan \
    -config=./config.toml

Get the report

$ docker run --rm -it \
     -v ~/.ssh:/root/.ssh:ro \
     -v $PWD:/vuls \
     -v $PWD/vuls-log:/var/log/vuls \
     -v /etc/localtime:/etc/localtime:ro \
     vuls/vuls report \
     -format-list \
     -config=./config.toml

Lynis/Kube-Hunter

Create ~/validation/bluval/bluval-sdtfc.yaml to customize the Test

blueprint:
    name: sdtfc
    layers:
        - os
        - k8s

    os: &os
        -
            name: lynis
            what: lynis
            optional: "False"
    k8s: &k8s
        -
            name: kube-hunter
            what: kube-hunter
            optional: "False"

Update ~/validation/bluval/volumes.yaml file

volumes:
    # location of the ssh key to access the cluster
    ssh_key_dir:
        local: '/home/ubuntu/.ssh'
        target: '/root/.ssh'
    # location of the k8s access files (config file, certificates, keys)
    kube_config_dir:
        local: '/home/ubuntu/kube'
        target: '/root/.kube/'
    # location of the customized variables.yaml
    custom_variables_file:
        local: '/home/ubuntu/validation/tests/variables.yaml'
        target: '/opt/akraino/validation/tests/variables.yaml'
    # location of the bluval-<blueprint>.yaml file
    blueprint_dir:
        local: '/home/ubuntu/validation/bluval'
        target: '/opt/akraino/validation/bluval'
    # location on where to store the results on the local jumpserver
    results_dir:
        local: '/home/ubuntu/results'
        target: '/opt/akraino/results'
    # location on where to store openrc file
    openrc:
        local: ''
        target: '/root/openrc'

# parameters that will be passed to the container at each layer
layers:
    # volumes mounted at all layers; volumes specific for a different layer are below
    common:
        - custom_variables_file
        - blueprint_dir
        - results_dir
    hardware:
        - ssh_key_dir
    os:
        - ssh_key_dir
    networking:
        - ssh_key_dir
    docker:
        - ssh_key_dir
    k8s:
        - ssh_key_dir
        - kube_config_dir
    k8s_networking:
        - ssh_key_dir
        - kube_config_dir
    openstack:
        - openrc
    sds:
    sdn:
    vim:

Update ~/validation/tests/variables.yaml file

### Input variables cluster's master host
host: <IP Address>             # cluster's master host address
username: <username>            # login name to connect to cluster
password: <password>         # login password to connect to cluster
ssh_keyfile: /root/.ssh/id_rsa        # Identity file for authentication

Run Blucon

$ bash validation/bluval/blucon.sh sdtfc

Expected output

BluVal tests should report success for all test cases.

Test Results

Insert Results URL

Vuls

There are 17 CVEs with a CVSS score >= 9.0. These are exceptions requested here:

Release 5: Akraino CVE Vulnerability Exception Request

...

Vuls results (manual) Nexus URL: https://

...

nexus.

...

akraino.org/content/sites/logs/fujitsu/job/sdt/r6/sdt-vuls/2/

Lynis results (manual) Nexus URL: https://

...

nexus.akraino.org/content/sites/logs/fujitsu/job/sdt/r6/sdt-lynis/3/

Kube-Hunter results Nexus URL: https://nexus.akraino.org/content/sites/logs/fujitsu/job/sdt/r6/sdt-bluval/2/

Vuls

Nexus URL: https://nexus.akraino.org/content/sites/logs/fujitsu/job/sdt/r6/sdt-vuls/2/

There are 17 CVEs with a CVSS score >= 9.0. These are exceptions requested here:

Release 6: Akraino CVE and KHV Vulnerability Exception Request

CVE-ID	CVSS	NVD	Fix/Notes
CVE-2016-1585

nist.gov/vuln/detail/CVE-2021-20236No fix available (latest release of ZeroMQ for Ubuntu 20.04 is 4.3.2-2ubuntu1)CVE-2021-318709.8https://nvd.nist.gov/vuln/detail/CVE-2021-31870No fix available (latest release of klibc for Ubuntu 20.04 is 2.0.7-1ubuntu5)CVE-2021-318729.8https://nvd.nist.gov/vuln/detail/CVE-2021-31872No fix available (latest release of klibc for Ubuntu 20.04 is 2.0.7-1ubuntu5)CVE-2021-31873

9.8

https://nvd.nist.gov/vuln/detail/CVE-

2021

2016-

31873

1585	No fix available

(latest release of klibc for Ubuntu 20.04 is 2.0.7-1ubuntu5)

CVE-2021-

33574

20236

9.8

https://nvd.nist.gov/vuln/detail/CVE-2021-

33574Will not be fixed in Ubuntu stable releases

20236	No fix available (latest release of ZeroMQ for Ubuntu 20.04 is 4.3.2-2ubuntu1)
CVE-2021-

45951

31870

9.8

https://nvd.nist.gov/vuln/detail/CVE-2021-

45951

31870

No fix available (

vendor disputed

latest release of klibc for Ubuntu 20.04 is 2.0.7-1ubuntu5)

CVE-2021-

45952

31872

9.8

https://nvd.nist.gov/vuln/detail/CVE-2021-

45952

31872

No fix available (

vendor disputed

latest release of klibc for Ubuntu 20.04 is 2.0.7-1ubuntu5)

CVE-2021-

45953

31873

9.8

https://nvd.nist.gov/vuln/detail/CVE-2021-

45953

31873

No fix available (

vendor disputed)

latest release of klibc for Ubuntu 20.04 is 2.0.7-1ubuntu5)

CVE-2021-

45954

33574	9.8	https://nvd.nist.gov/vuln/detail/CVE-2021-33574	Will not be fixed in Ubuntu stable releases
CVE-2021-45951	9.8	https://nvd.nist.gov/vuln/detail/CVE-2021-

45954

45951	No fix available (vendor disputed)
CVE-2021-

45955

45952

9.8

https://nvd.nist.gov/vuln/detail/CVE-2021-

45955

45952	No fix available (vendor disputed)
CVE-2021-

45956

45953

9.8

https://nvd.nist.gov/vuln/detail/CVE-2021-

45956

45953	No fix available (vendor disputed)
CVE-2021-

45957

45954

9.8

https://nvd.nist.gov/vuln/detail/CVE-2021-

45957

45954	No fix available (vendor disputed)
CVE-

2022

2021-

23218

45955

9.8

https://nvd.nist.gov/vuln/detail/CVE-

2022

2021-

23218

45955

No fix available (vendor disputed)

Reported fixed in 2.31-0ubuntu9.7 (installed), but still reported by Vuls

CVE-

2022

2021-

23219

45956

9.8

https://nvd.nist.gov/vuln/detail/CVE-

2022

2021-

23219

45956

No fix available (vendor disputed)

Reported fixed in 2.31-0ubuntu9.7 (installed), but still reported by Vuls

CVE-

2016

2021-

9180

45957

9.

1

8	https://nvd.nist.gov/vuln/detail/CVE-

2016

2021-

9180

45957	No fix available (vendor disputed)
CVE-

2021

2022-

35942

23218

9.

1

8	https://nvd.nist.gov/vuln/detail/CVE-

2021

2022-

35942

23218

Reported fixed in 2.31-0ubuntu9.7 (installed), but still reported by Vuls

Lynis

Image Removed

The initial results compare with the Lynis Incubation: PASS/FAIL Criteria, v1.0 as follows.

The Lynis Program Update test MUST pass with no errors.

2022-03-04 15:33:28 Test: Checking for program update...
2022-03-04 15:33:31 Current installed version  : 301
2022-03-04 15:33:31 Latest stable version      : 307
2022-03-04 15:33:31 Minimum required version   : 297
2022-03-04 15:33:31 Result: newer Lynis release available!
2022-03-04 15:33:31 Suggestion: Version of Lynis outdated, consider upgrading to the latest version [test:LYNIS] [details:-] [solution:-]

Fix: Download and run the latest Lynis directly on SUT.

The following list of tests MUST complete as passing

CVE-2022-23219	9.8	https://nvd.nist.gov/vuln/detail/CVE-2022-23219	Reported fixed in 2.31-0ubuntu9.7 (installed), but still reported by Vuls
CVE-2016-9180	9.1	https://nvd.nist.gov/vuln/detail/CVE-2016-9180	No fix available
CVE-2021-35942	9.1	https://nvd.nist.gov/vuln/detail/CVE-2021-35942	Reported fixed in 2.31-0ubuntu9.7 (installed), but still reported by Vuls

Lynis

Nexus URL (run via Bluval, without fixes): https://nexus.akraino.org/content/sites/logs/fujitsu/job/sdt/r6/sdt-bluval/2/

Nexus URL (manual run, with fixes): https://nexus.akraino.org/content/sites/logs/fujitsu/job/sdt/r6/sdt-lynis/2/

Image Added

The initial results compare with the Lynis Incubation: PASS/FAIL Criteria, v1.0 as follows.

The Lynis Program Update test MUST pass with no errors.

2022-03-04 15:33:28 Test: Checking for program update...
2022-03-04 15:33:31 Current installed version  : 301
2022-03-04 15:33:31 Latest stable version      : 307
2022-03-04 15:33:31 Minimum required version   : 297
2022-03-04 15:33:31 Result: newer Lynis release available!
2022-03-04 15:33:31 Suggestion: Version of Lynis outdated, consider upgrading to the latest version [test:LYNIS] [details:-] [solution:-]

Fix: Download and run the latest Lynis directly on SUT.

Steps To Implement Security Scan Requirements#InstallandExecute

The following list of tests MUST complete as passing

No.	Test	Result	Fix
1	Test: Checking PASS_MAX_DAYS option in /etc/login.defs	Result: password minimum age is not configured Suggestion: Configure minimum password age in /etc/login.defs [test:AUTH-9286]	Set PASS_MAX_DAYS 180 in /etc/login.defs
2	Performing test ID AUTH-9328 (Default umask values)	Result: found umask 022, which could be improved Suggestion: Default umask in /etc/login.defs could be more strict like 027 [test:AUTH-9328]	Set UMASK 027 in /etc/login.defs
3	Performing test ID SSH-7440 (Check OpenSSH option: AllowUsers and AllowGroups)	Result: SSH has no specific user or group limitation. Most likely all valid users can SSH to this machine. Hardening: assigned partial number of hardening points (0 of 1).	Configure AllowUsers in /etc/ssh/sshd_config
4	Test: checking for file /etc/network/if-up.d/ntpdate	Test: checking for file /etc/network/if-up.d/ntpdate Result: file /etc/network/if-up.d/ntpdate does not exist ... Hardening: assigned maximum number of hardening points for this item (3).	OK
5	Performing test ID KRNL-6000 (Check sysctl key pairs in scan profile) : Following sub-tests required	N/A	N/A
5a	sysctl key fs.suid_dumpable contains equal expected and current value (0)	Result: sysctl key fs.suid_dumpable has a different value than expected in scan profile. Expected=0, Real=2	Set recommended value in /etc/sysctl.d/90-lynis-hardening.conf and disable apport in /etc/default/apport
5b	sysctl key kernel.dmesg_restrict contains equal expected and current value (1)	Result: sysctl key kernel.dmesg_restrict has a different value than expected in scan profile. Expected=1, Real=0	Set recommended value in /etc/sysctl.d/90-lynis-hardening.conf
5c	sysctl key net.ipv4.conf.default.accept_source_route contains equal expected and current value (0)	Result: sysctl key net.ipv4.conf.default.accept_source_route has a different value than expected in scan profile. Expected=0, Real=1	Set recommended value in /etc/sysctl.d/90-lynis-hardening.conf
6	Test: Check if one or more compilers can be found on the system	Result: found installed compiler. See top of logfile which compilers have been found or use /usr/bin/grep to filter on 'compiler' Hardening: assigned partial number of hardening points (1 of 3).	Uninstall gcc and remove /usr/bin/as (installed with binutils)

Results after the above fixes are as follows:

The Lynis Program Update test MUST pass with no errors.

2022-03-07 15:19:07 Test: Checking for program update...
2022-03-07 15:19:10 Current installed version : 308
2022-03-07 15:19:10 Latest stable version : 307
2022-03-07 15:19:10 No Lynis update available.

The following list of tests MUST complete as passing

No.	Test	Result
1	Test: Checking PASS_MAX_DAYS option in /etc/login.defs	Result: max password age is 180 days Hardening: assigned maximum number of hardening points for this item (3).
2	Performing test ID AUTH-9328 (Default umask values)	Result: umask is 027, which is fine Hardening: assigned maximum number of hardening points for this item (2).
3	Performing test ID SSH-7440 (Check OpenSSH option: AllowUsers and AllowGroups)	Result: SSH is limited to a specific set of users, which is good

No.	Test	Result	Fix
1	Test: Checking PASS_MAX_DAYS option in /etc/login.defs	Result: password minimum age is not configured Suggestion: Configure minimum password age in /etc/login.defs [test:AUTH-9286]	Set PASS_MAX_DAYS 180 in /etc/login.defs
2	Performing test ID AUTH-9328 (Default umask values)	Result: found umask 022, which could be improved Suggestion: Default umask in /etc/login.defs could be more strict like 027 [test:AUTH-9328]	Set UMASK 027 in /etc/login.defs
3	Performing test ID SSH-7440 (Check OpenSSH option: AllowUsers and AllowGroups)	Result: SSH has no specific user or group limitation. Most likely all valid users can SSH to this machine. Hardening: assigned partial number of hardening points (0 of 1).	Configure AllowUsers in /etc/ssh/sshd_config
4	Test: checking for file /etc/network/if-up.d/ntpdate	Test: checking for file /etc/network/if-up.d/ntpdate Result: file /etc/network/if-up.d/ntpdate does not exist ... Hardening: assigned maximum number of hardening points for this item (32).	OK	5	Performing test ID KRNL-6000 (Check sysctl key pairs in scan profile) : Following sub-tests required	N/A
N/A	5a	sysctl key fs.suid_dumpable contains equal expected and current value (0)	Result: sysctl key fs.suid_dumpable has a different value than expected in scan profile. Expected=0, Real=2Set recommended value in /etc/sysctl.d/90-lynis-hardening.confcontains equal expected and current value (0) Hardening: assigned maximum number of hardening points for this item (1).
5b	sysctl key kernel.dmesg_restrict contains equal expected and current value (1)	Result: sysctl key kernel.dmesg_restrict has a different value than expected in scan profile. Expected=1, Real=0	Set recommended value in /etc/sysctl.d/90-lynis-hardening.conf
5c	sysctl key net.ipv4.conf.default.accept_source_route contains equal expected and current value (0)	Result: key net.inet.ip.sourceroute does not exist on this machine ... Hardening: assigned maximum number of hardening points for this item (1)	OK
6	Test: Check if one or more compilers can be found on the system	Result: found installed compiler. See top of logfile which compilers have been found or use /usr/bin/grep to filter on 'compiler' Hardening: assigned partial number of hardening points (1 of 3).	Uninstall gcc

...

contains equal expected and current value (1)
Hardening: assigned maximum number of hardening points for this item (1).

5c

sysctl key net.ipv4.conf.default.accept_source_route contains equal expected and current value (0)

Result: sysctl key net.ipv4.conf.default.accept_source_route contains equal expected and current value (0)

Hardening: assigned maximum number of hardening points for this item (1).

6

Test: Check if one or more compilers can be found on the system

Result: no compilers found
Hardening: assigned maximum number of hardening points for this item (3).

The post-fix manual logs can be found at https://nexus.akraino.org/content/sites/logs/fujitsu/job/sdt/r6/sdt-lynis/3/.

Kube-Hunter

Nexus URL (with fixes): https://nexus.akraino.org/content/sites/logs/fujitsu/job/sdt/r6/sdt-bluval/2/

Image Modified

There are 5 Vulnerabilities.

KHV002
KHV005
KHV050
CAP_NET_RAW Enabled
Access to pod's secrets

Fix for KHV002

$ kubectl replace -f - <<EOF
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  annotations:
    rbac.authorization.kubernetes.io/autoupdate: "false"
  labels:
    kubernetes.io/bootstrapping: rbac-defaults
  name: system:public-info-viewer
rules:
- nonResourceURLs:
  - /healthz
  - /livez
  - /readyz
  verbs:
  - get
EOF

Fix for KHV005, KHV050, Access to pod's secrets

$ kubectl replace -f - <<EOF
apiVersion: v1
kind: ServiceAccount
metadata:
  name: default
  namespace: default
automountServiceAccountToken: false
EOF

The above fixes are implemented in the Ansible playbook deploy/playbook/init_cluster.yml and configuration file deploy/playbook/k8s/fix.yml

...

Create a PodSecurityPolicy with requiredDropCapabilities: NET_RAW. The policy is shown below. The complete fix is implemented in the Ansible playbook deploy/playbook/init_cluster.yml and configuration files deploy/playbook/k8s/default-psp.yml and deploy/playbook/k8s/system-psp.yml, plus enabling PodSecurityPolicy checking in deploy/playbook/k8s/config.yml.

apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: psp-baseline
spec:
  privileged: true
  allowPrivilegeEscalation: true
  allowedCapabilities:
  - IPC_LOCK
  - NET_ADMIN
  requiredDropCapabilities:
  - NET_RAW
  hostIPC: true
  hostNetwork: true
  hostPID: true
  hostPorts:
  - max: 65535
    min: 0
  readOnlyRootFilesystem: false
  fsGroup:
    rule: 'RunAsAny'
  runAsUser:
    rule: 'RunAsAny'
  seLinux:
    rule: 'RunAsAny'
  supplementalGroups:
    rule: 'RunAsAny'
  volumes:
  - '*'

Results after fixes are shown below:

...

*One Kube-Hunter failure is counted as a pass. See above. .

Vuls and Lynis test cases are failing, an exception request is filed for Vuls-detected vulnerabilities that cannot be fixed. The Lynis results have been confirmed to pass the Incubation criteria.

Additional Testing

None at this time.

...

Space shortcuts

Page tree

Versions Compared

Old Version 38

New Version Current

Key

Introduction

Akarino Test Group Information

Test Results

CI/CD Regression Tests: Node Setup

Test Results

CI/CD Regression Tests: Cluster Setup & Teardown

Test Results

CI/CD Regression Tests: EdgeX Services

Test Results

CI/CD Regression Tests: LoRa Device Service

Test Results

Feature Project Tests

Lynis/Kube-Hunter

Expected output

Test Results

Vuls

Vuls

Lynis

Image Removed

The Lynis Program Update test MUST pass with no errors.

The following list of tests MUST complete as passing

Lynis

Image Added

The Lynis Program Update test MUST pass with no errors.

The following list of tests MUST complete as passing

The Lynis Program Update test MUST pass with no errors.

The following list of tests MUST complete as passing

Kube-Hunter

Additional Testing