Application scaling based on performance metrics
Basically there is one performance management solution in CaaS sub-system and it is exposed to the application via the HPA API provided by Kubernetes platform. Applications can use both core metrics and custom metrics to horizontally scale themselves. The first is based on CPU and memory usage and the other uses practically every metric that the developer provides to the API Aggregator via an HTTP server.
In the following there is a short overview of the components of Performance management and elasticity sub-system of CaaS.
- API Aggregator: The API Aggregator is part of the K8S (Kubernetes) API server. The aggregation layer allows Kubernetes to be extended with additional APIs, beyond what is offered by the core Kubernetes APIs. The aggregation layer enables installing additional Kubernetes-style APIs in K8S cluster. These can either be pre-built, existing 3rd party solutions or user-created APIs. The role of the API aggregator in scaling is to proxy the core, and custom metrics API requests to the core, and custom metrics API handler servers registered to serve the APIs. In CaaS these are the metrics server for core: link, and Prometheus for custom: link.
- API Server: The Kubernetes API server validates and configures data for the API objects which include pods, services, replication controllers, and others. The API Server services REST operations and provides the frontend to the cluster’s shared state through which all other components interact.
- cAdvisor: cAdvisor is an open source container resource usage and performance analysis agent. It is purpose-built for containers and supports Docker containers natively. In Kubernetes, cAdvisor is integrated into the Kubelet binary. cAdvisor auto-discovers all containers in the machine and collects CPU, memory, filesystem, and network usage statistics. cAdvisor also provides the overall machine usage by analyzing the ‘root’ container on the machine.
- Custom Metric adapter: Custom Metric adapter is an element to provide connection between Prometheus and API Aggregator.
- HPA (Horizontal Pod Autoscaler): The Horizontal Pod Autoscaler is implemented as a Kubernetes API resource and a controller. It automatically scales the number of pods in a replication controller, deployment or replica set based on core or custom metrics.
- Kubelet: The Kubelet acts as a bridge between the Kubernetes master and the nodes. It manages the pods and containers running on a machine. Kubelet translates each pod into its constituent containers and fetches individual container usage statistics from cAdvisor. It then exposes the aggregated pod resource usage statistics via a REST API.
- Prometheus: Prometheus is an open-source software project written in Go that is used to record real-time metrics in a time series database (allowing for high dimensionality) built using a HTTP pull model, with flexible queries and real-time alerting.
The key differences between core and custom metrics are that core metrics support scraping metrics only from CPU and memory whereas custom metrics can scrape practically every kind of metrics. In the first case Kubernetes offers the metrics out of box, but in the second case users have to implement the metrics provider HTTP server.
Note that the database behind the performance management system is not persistent but uses time-series database to store metric values in both solutions.
Check core metrics in the system
Metrics APIs provided by Kubernetes can be gotten by:
cAdvisor automatically scrapes metrics from every active pod in the system regarding to CPU and memory usage. The state of core metrics can be requested by the following:
With this command you can get core metrics from nodes respectively to CPU and memory usage.
The printout shows the names of nodes actually the IP addresses of nodes, the usage of CPUs in percentage and milli standard for 2 CPUs in the example furthermore memory usage in percentage and Mi (MiB) standard.
Console output shows pod names and their CPU and memory consumption in the same format.
Check custom metrics in the system
In case of the usage of Custom Metrics the developer has to provide the exposition of metrics in his application in Prometheus format. There are specific libraries that can be used for creating HTTP server and Prometheus client for this purpose in Golang, Python etc.
Basic example application in Python:
This application imports http_server and Histogram metrics from Prometheus client library and exposes metrics from the func() function. Prometheus can scrape these metrics from port 9100.
The command result lists the custom metrics in the system, each metrics can be requested one by one for more details:
This is a HTTP request with cURL, it shows the custom metrics exposed by an HTTP server of an application running in a Kubernetes pod.
Core metrics and Custom metrics examples
HPA manifest with core metrics scraping:
In this example HPA scrapes metrics from CPU consumption of php-apache-deployment. The initial pod number is one and the maximum replica counts are five. HPA initiates pod scaling when the CPU utilization is higher than 50%. If the utilization is less than 50% HPA starts scaling down the number of pods by one.
HPA manifest with custom metrics scraping:
In the second example HPA uses custom metrics to manage the performance. The podinfo application contains the implementation of an HTTP server which exposes the metrics in Prometheus format. The initial number of pods are two and the maximum are ten. The custom metric is the cardinality of the http requests on the HTTP server regarding to the metrics exposed.
The starter command for custom metric based HPA manifest is:
In case of starting core metrics HPA the command is the same.
After HPA start you can get information about the state of actual performance management of the system with:
Note that: HPA API supports scaling based on both core and custom metrics within the same HPA object.
Example Kubernetes manifests for testing with core metrics:
Example Kubernetes manifests for testing with custom metrics: