Argo CD Operator E2E Test Guide¶
E2E tests are written using KUTTL.
This test suite assumes that an Argo CD Operator is installed on the cluster or running locally using
ARGOCD_CLUSTER_CONFIG_NAMESPACES=argocd-e2e-cluster-config make install run.
The system executing the tests must have following tools installed:
kuttlkubectl plugin (>= v0.11.1)
jqfor parsing JSON data
There should be a
kubeconfig pointing to your cluster, user should have full admin privileges (i.e.
E2E tests utilize GNU Grep under the hood. Please make sure that you have the GNU compatible
If you are on OSX you can install GNU compatible grep using the below command. The package is installed as
ggrep by default. Please set this(ggrep) as an alias to
brew install grep
Use the below commands to install GNU compatible
grep on OSX.
Also, note that the e2e tests for Redis HA mode require a cluster with at least three worker nodes. A local three-worker node cluster can be created using k3d
Running the tests¶
In any case, you should have set up your
kubeconfig in such a way that your
default context points to the cluster you want to test. You can use the
kubectl login ... command to set this up for you.
Run e2e tests¶
Run Operator locally and execute e2e tests¶
Running manual with kuttl¶
kubectl kuttl test ./tests/k8s --config ./tests/kuttl-tests.yaml
Running single tests¶
Sometimes (e.g. when initially writing a test or troubleshooting an existing
one), you may want to run single test cases isolated. To do so, you can pass
the name of the test using
kubectl kuttl test ./tests/k8s --config ./tests/kuttl-tests.yaml --test 1-004_validate_namespace_scoped_install
The name of the test is the name of the directory containing its steps and assertions.
If you are troubleshooting, you may want to prevent
kuttl from deleting the
test's namespace afterwards. In order to do so, just pass the additional flag
--skip-delete to above command.
Writing new tests¶
Name of the test¶
Each test comes in its own directory, containing all its test steps. The name of the test is defined by the name of this directory.
The name of the test should be short, but expressive. The format for naming a
test is currently
<test ID>_<short description>.
<test ID> is the serial number of the test as defined in the Test Plan
<short description> is exactly that, a short description of
what happens in the test.
Name of the test steps¶
Each test step is a unique YAML file within the test's directory. The name of the step is defined by its file name.
The test steps must be named
XX-<name>.yaml. This is a
and cannot be overriden.
XX is a number (prefixed with
0, so step
<name> is a free form value for the test step.
There are two reserved words you cannot use for
assertcontains positive assertions (i.e. resources that must exist) and
errorscontains negative assertions (i.e. resources that must not exist)
Refer to the kuttl documentation for more information.
Documentation is important, even for tests. You can should provide inline
documentation in your YAML files (using comments) and a
README.md in your
test case's directory. The
README.md should provide some context for the
test case, e.g. what it tries to assert for under which circumstances. This
will help others in troubleshooting failing tests.
kuttl unfortunately neither encourages or supports re-use of your test steps
and assertions yet.
Generally, you should try to use
errors declaration whenever
possible and viable. For some cases, you may need to use custom scripts to
get the results you are looking for.
Scripts can be executed in a
kuttl.dev/TestStep resources from a usual test
Your script probably will retrieve some information, and asserts it state. If the assertion fails, the script should exit with a code > 0, and also print some information why it failed, e.g.
apiVersion: kuttl.dev/v1beta1 kind: TestStep commands: - script: | # Get some piece of information... if test "$result" != "expected"; then echo "Expectation failed, should 'expected', is '$result'" exit 1 fi
Also, you may want to use
set -e and
set -o pipefail at the top of your
script to catch unexpected errors as test case failures, e.g.
apiVersion: kuttl.dev/v1beta1 kind: TestStep commands: - script: | set -e set -o pipefail # rest of your script
Getting values of a resource's environment variables¶
YAML declarations used in
errors files unfortunately don't handle
arrays very well yet. You will always have to specify the complete expectation,
i.e. the complete array.
If you are just interested in a certain variable, and don't care about the rest,
you can use a script similar to the following using
jq. E.g. to get the value
of a variable named
FOO for the
argocd-server deployment in the test's
apiVersion: kuttl.dev/v1beta1 kind: TestStep commands: - script: | val=$(kubectl get -n $NAMESPACE deployments argocd-server -o json \ | jq -r '.spec.templates.spec.containers.env|select(.name=="FOO").value') if test "$val" != "bar"; then echo "Expectation failed for for env FOO in argocd-server: should 'bar', is '$val'" exit 1 fi