1# Testing
2
3The following guidelines summarize the testing policy for docker/cli.
4
5## Unit Test Suite
6
7All code changes should have unit test coverage.
8
9Error cases should be tested with unit tests.
10
11Bug fixes should be covered by new unit tests or additional assertions in
12existing unit tests.
13
14### Details
15
16The unit test suite follows the standard Go testing convention. Tests are
17located in the package directory in `_test.go` files.
18
19Unit tests should be named using the convention:
20
21```
22Test<Function Name><Test Case Name>
23```
24
25[Table tests](https://github.com/golang/go/wiki/TableDrivenTests) should be used
26where appropriate, but may not be appropriate in all cases.
27
28Assertions should be made using
29[gotest.tools/assert](https://godoc.org/gotest.tools/assert).
30
31Fakes, and testing utilities can be found in
32[internal/test](https://godoc.org/github.com/docker/cli/internal/test) and
33[gotest.tools](https://godoc.org/gotest.tools).
34
35## End-to-End Test Suite
36
37The end-to-end test suite tests a cli binary against a real API backend.
38
39### Guidelines
40
41Each feature (subcommand) should have a single end-to-end test for
42the success case. The test should include all (or most) flags/options supported
43by that feature.
44
45In some rare cases a couple additional end-to-end tests may be written for a
46sufficiently complex and critical feature (ex: `container run`, `service
47create`, `service update`, and `docker build` may have ~3-5 cases each).
48
49In some rare cases a sufficiently critical error paths may have a single
50end-to-end test case.
51
52In all other cases the behaviour should be covered by unit tests.
53
54If a code change adds a new flag, that flag should be added to the existing
55"success case" end-to-end test.
56
57If a code change fixes a bug, that bug fix should be covered either by adding
58assertions to the existing end-to-end test, or with one or more unit test.
59
60### Details
61
62The end-to-end test suite is located in
63[./e2e](https://github.com/docker/cli/tree/master/e2e). Each directory in `e2e`
64corresponds to a directory in `cli/command` and contains the tests for that
65subcommand. Files in each directory should be named `<command>_test.go` where
66command is the basename of the command (ex: the test for `docker stack deploy`
67is found in `e2e/stack/deploy_test.go`).
68
69Tests should be named using the convention:
70
71```
72Test<Command Basename>[<Test Case Name>]
73```
74
75where the test case name is only required when there are multiple test cases for
76a single command.
77
78End-to-end test should run the `docker` binary using
79[gotestyourself/icmd](https://godoc.org/github.com/gotestyourself/gotestyourself/icmd)
80and make assertions about the exit code, stdout, stderr, and local file system.
81
82Any Docker image or registry operations should use `registry:5000/<image name>`
83to communicate with the local instance of the registry. To load
84additional fixture images to the registry see
85[scripts/test/e2e/run](https://github.com/docker/cli/blob/master/scripts/test/e2e/run).
View as plain text