Versions Compared

Old Version 3

changes.mady.by.user Tapio Tallgren

Saved on

compared with

New Version Current

changes.mady.by.user Tapio Tallgren

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Links: https://docs.openstack.org/tempest/latest/overview.html

Kubernetes

The Kubernetes integration testing only uses the local machine. The integration tests are run using make test-integration. The Kubernetes integration tests are written using the normal golang testing package but expect to have a running etcd instance to connect to.

End-to-end (e2e) tests for Kubernetes provide a mechanism to test end-to-end behavior of the system, and is the last signal to ensure end user operations match developer specifications. Although unit and integration tests provide a good signal, in a distributed system like Kubernetes it is not uncommon that a minor change may pass all unit and integration tests, but cause unforeseen changes at the system level. The primary objectives of the e2e tests are to ensure a consistent and reliable behavior of the kubernetes code base, and to catch hard-to-test bugs before users do, when unit and integration tests are insufficient.

Kubernetes has a large set of end-to-end tests which can be classified as follows:

  • If a test has no labels, it is expected to run fast (under five minutes), be able to be run in parallel, and be consistent.

  • [Slow]: If a test takes more than five minutes to run (by itself or in parallel with many other tests), it is labeled [Slow]. This partition allows us to run almost all of our tests quickly in parallel, without waiting for the stragglers to finish.

  • [Serial]: If a test cannot be run in parallel with other tests (e.g. it takes too many resources or restarts nodes), it is labeled [Serial], and should be run in serial as part of a separate suite.

  • [Disruptive]: If a test restarts components that might cause other tests to fail or break the cluster completely, it is labeled [Disruptive]. Any [Disruptive] test is also assumed to qualify for the [Serial] label, but need not be labeled as both. These tests are not run against soak clusters to avoid restarting components.

  • [Conformance]: Designate that this test is included in the Conformance test suite for Conformance Testing. This test must meet a number of requirements to be eligible for this tag. This tag does not supersed any other labels.

  • [LinuxOnly]: If a test is known to be using Linux-specific features (e.g.: seLinuxOptions) or is unable to run on Windows nodes, it is labeled [LinuxOnly]. When using Windows nodes, this tag should be added to the skipargument.

Link: https://github.com/kubernetes/community/blob/master/contributors/devel/sig-testing/e2e-tests.md, https://github.com/kubernetes/community/blob/master/contributors/devel/sig-testing/e2e-tests.md

...

Kubernetes performance testing can be done using the end-to-end tests by measuring the response time for different API calls after different operations (example: launch 100 containers, measure time until they respond).