Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Note

Kubernetes 1.14 deprecates several legacy APIs and Kubernetes 1.16 disables them by default. For deployment of SEBA, it is necessary to manually enable these legacy Kubernetes APIs since they are not supported by Telco Appliance.  The deprecated APIs will be removed in Kubernetes 1.18.

  • Enable legacy APIs by

    updating 

    adding --runtime-config option to the command section of /etc/kubernetes/manifests/apiserver.yml on each node in the cluster.

    Code Block

...

  • collapsetrue
    ---
    apiVersion: v1
    kind: Pod
    metadata:
      name: kube-apiserver
      namespace: kube-system
    spec:
      hostNetwork: true
      containers:
        - name: kube-apiserver
          image: registry.kube-system.svc.rec.io:5555/caas/hyperkube:1.16.0-5
          securityContext:
            runAsUser: 144
          command:
            - "/kube-apiserver"
            - --admission-control=DefaultStorageClass,LimitRanger,MutatingAdmissionWebhook,NamespaceExists,NamespaceLifecycle,NodeRestriction,PodSecurityPolicy,ResourceQuota,ServiceAccount,ValidatingAdmissionWebhook
            - --advertise-address=192.168.12.51
            - --allow-privileged=true
            - --anonymous-auth=false
            - --apiserver-count=3
            - --audit-policy-file=/var/lib/caas/policies/audit-policy.yaml
            - --audit-log-format=json
            - --audit-log-maxsize=100
            - --audit-log-maxbackup=88
            - --audit-log-path=/var/log/audit/kube_apiserver/kube-apiserver-audit.log
            - --authorization-mode=Node,RBAC
            - --bind-address=192.168.12.51
            - --client-ca-file=/etc/openssl/ca.pem
            - --enable-bootstrap-token-auth=true
            - --etcd-cafile=/etc/etcd/ssl/ca.pem
            - --etcd-certfile=/etc/etcd/ssl/etcd1.pem
            - --etcd-keyfile=/etc/etcd/ssl/etcd1-key.pem
            - --etcd-servers=https://192.168.12.51:4111,https://192.168.12.52:4111,https://192.168.12.53:4111
            - --experimental-encryption-provider-config=/etc/kubernetes/ssl/secrets.conf
            - --feature-gates=SCTPSupport=True,CPUManager=False,TokenRequest=True,DevicePlugins=True
            - --insecure-port=0
            - --kubelet-certificate-authority=/etc/openssl/ca.pem
            - --kubelet-client-certificate=/etc/kubernetes/ssl/kubelet-server.pem
            - --kubelet-client-key=/etc/kubernetes/ssl/kubelet-server-key.pem
            - --kubelet-https=true
            - --max-requests-inflight=1000
            - --proxy-client-cert-file=/etc/kubernetes/ssl/metrics.crt
            - --proxy-client-key-file=/etc/kubernetes/ssl/metrics.key
            - --requestheader-client-ca-file=/etc/openssl/ca.pem
            - --requestheader-extra-headers-prefix=X-Remote-Extra-
            - --requestheader-group-headers=X-Remote-Group
            - --requestheader-username-headers=X-Remote-User
            - --secure-port=6443
            - --service-account-key-file=/etc/kubernetes/ssl/service-account.pem
            - --service-account-lookup=true
            - --service-cluster-ip-range=10.254.0.0/16
            - --tls-cert-file=/etc/kubernetes/ssl/tls-cert.pem
            - --tls-private-key-file=/etc/kubernetes/ssl/apiserver1-key.pem
            - --token-auth-file=/etc/kubernetes/ssl/tokens.csv
            - --runtime-config=apps/v1beta1=true,apps/v1beta2=true,extensions/v1beta1/daemonsets=true,extensions/v1beta1/deployments=true,extensions/v1beta1/replicasets=true,extensions/v1beta1/networkpolicies=true,extensions/v1beta1/podsecuritypolicies=true
    
          resources:
            requests:
              cpu: "50m"
          volumeMounts:
            - name: time-mount
              mountPath: /etc/localtime
              readOnly: true
            - name: secret-kubernetes
              mountPath: /etc/kubernetes/ssl
              readOnly: true
            - name: secret-root-ca
              mountPath: /etc/openssl/ca.pem
              readOnly: true
            - name: secret-etcd
              mountPath: /etc/etcd/ssl
              readOnly: true
            - name: audit-kube-apiserver
              mountPath: /var/log/audit/kube_apiserver/
              readOnly: false
            - name: audit-policy-dir
              mountPath: /var/lib/caas/policies
              readOnly: true
      volumes:
        - name: time-mount
          hostPath:
            path: /etc/localtime
        - name: secret-kubernetes
          hostPath:
            path: /etc/kubernetes/ssl
        - name: secret-root-ca
          hostPath:
            path: /etc/openssl/ca.pem
        - name: secret-etcd
          hostPath:
            path: /etc/etcd/ssl
        - name: audit-kube-apiserver
          hostPath:
            path: /var/log/audit/kube_apiserver/
        - name: audit-policy-dir
          hostPath:
            path: /var/lib/caas/policies

...


  • Delete the kube-apiserver pods and wait for the pods to be recreated.

  • Add cluster-admin rights to to the tiller service account.
  • Add the CORD repository and updated indexes.
  • Install the CORD platform.
  • Wait until all 3 etcd CRDs are present in Kubernetes
  • Install the SEBA profile.
  • Code Block
    kubectl delete pod -n kube-system kube-apiserver-192.168.12.51
    kubectl delete pod -n kube-system kube-apiserver-192.168.12.52
    kubectl delete pod -n kube-system kube-apiserver-192.168.12.53


  • Add cluster-admin rights to to the tiller service account.

    Code Block
    kubectl create clusterrolebinding tiller-cluster-admin --clusterrole=cluster-admin --serviceaccount=kube-system:tiller


  • Add the CORD repository and updated indexes.

    Code Block
    helm repo add cord https://charts.opencord.org
    helm repo update


  • Install the CORD platform.

    Code Block
    helm install -n cord-platform --version 6.1.0 cord/cord-platform


  • Wait until all 3 etcd CRDs are present in Kubernetes

    Code Block
    kubectl get crd | grep -i etcd | wc -l


  • Install the SEBA profile.

    Code Block
    helm install -n seba --version 1.0.0 cord/seba


  • Install the AT&T workflow

    Code Block
    helm install -n att-workflow --version 1.0.2 cord/att-
    Install the AT&T
    workflow


  • Wait for all pods to reach Completed or Running status.

...