1=====================
2Retryable Reads Tests
3=====================
4
5.. contents::
6
7----
8
9Introduction
10============
11
12The YAML and JSON files in this directory tree are platform-independent tests
13that drivers can use to prove their conformance to the Retryable Reads spec.
14
15Prose tests, which are not easily expressed in YAML, are also presented
16in this file. Those tests will need to be manually implemented by each driver.
17
18Tests will require a MongoClient created with options defined in the tests.
19Integration tests will require a running MongoDB cluster with server versions
204.0 or later.
21
22N.B. The spec specifies 3.6 as the minimum server version: however,
23``failCommand`` is not supported on 3.6, so for now, testing requires MongoDB
244.0. Once `DRIVERS-560`_ is resolved, we will attempt to adapt its live failure
25integration tests to test Retryable Reads on MongoDB 3.6.
26
27.. _DRIVERS-560: https://jira.mongodb.org/browse/DRIVERS-560
28
29Server Fail Point
30=================
31
32See: `Server Fail Point`_ in the Transactions spec test suite.
33
34.. _Server Fail Point: ../../transactions/tests#server-fail-point
35
36Disabling Fail Point after Test Execution
37-----------------------------------------
38
39After each test that configures a fail point, drivers should disable the
40``failCommand`` fail point to avoid spurious failures in
41subsequent tests. The fail point may be disabled like so::
42
43 db.runCommand({
44 configureFailPoint: "failCommand",
45 mode: "off"
46 });
47
48Network Error Tests
49===================
50
51Network error tests are expressed in YAML and should be run against a standalone,
52shard cluster, or single-node replica set.
53
54
55Test Format
56-----------
57
58Each YAML file has the following keys:
59
60- ``runOn`` (optional): An array of server version and/or topology requirements
61 for which the tests can be run. If the test environment satisfies one or more
62 of these requirements, the tests may be executed; otherwise, this file should
63 be skipped. If this field is omitted, the tests can be assumed to have no
64 particular requirements and should be executed. Each element will have some or
65 all of the following fields:
66
67 - ``minServerVersion`` (optional): The minimum server version (inclusive)
68 required to successfully run the tests. If this field is omitted, it should
69 be assumed that there is no lower bound on the required server version.
70
71 - ``maxServerVersion`` (optional): The maximum server version (inclusive)
72 against which the tests can be run successfully. If this field is omitted,
73 it should be assumed that there is no upper bound on the required server
74 version.
75
76 - ``topology`` (optional): An array of server topologies against which the
77 tests can be run successfully. Valid topologies are "single",
78 "replicaset", "sharded", and "load-balanced". If this field is omitted,
79 the default is all topologies (i.e. ``["single", "replicaset", "sharded",
80 "load-balanced"]``).
81
82 - ``serverless``: Optional string. Whether or not the test should be run on
83 serverless instances imitating sharded clusters. Valid values are "require",
84 "forbid", and "allow". If "require", the test MUST only be run on serverless
85 instances. If "forbid", the test MUST NOT be run on serverless instances. If
86 omitted or "allow", this option has no effect.
87
88 The test runner MUST be informed whether or not serverless is being used in
89 order to determine if this requirement is met (e.g. through an environment
90 variable or configuration option). Since the serverless proxy imitates a
91 mongos, the runner is not capable of determining this by issuing a server
92 command such as ``buildInfo`` or ``hello``.
93
94- ``database_name`` and ``collection_name``: Optional. The database and
95 collection to use for testing.
96
97- ``bucket_name``: Optional. The GridFS bucket name to use for testing.
98
99- ``data``: The data that should exist in the collection(s) under test before
100 each test run. This will typically be an array of documents to be inserted
101 into the collection under test (i.e. ``collection_name``); however, this field
102 may also be an object mapping collection names to arrays of documents to be
103 inserted into the specified collection.
104
105- ``tests``: An array of tests that are to be run independently of each other.
106 Each test will have some or all of the following fields:
107
108 - ``description``: The name of the test.
109
110 - ``clientOptions``: Optional, parameters to pass to MongoClient().
111
112 - ``useMultipleMongoses`` (optional): If ``true``, the MongoClient for this
113 test should be initialized with multiple mongos seed addresses. If ``false``
114 or omitted, only a single mongos address should be specified. This field has
115 no effect for non-sharded topologies.
116
117 - ``skipReason``: Optional, string describing why this test should be skipped.
118
119 - ``failPoint``: Optional, a server fail point to enable, expressed as the
120 configureFailPoint command to run on the admin database.
121
122 - ``operations``: An array of documents describing an operation to be
123 executed. Each document has the following fields:
124
125 - ``name``: The name of the operation on ``object``.
126
127 - ``object``: The name of the object to perform the operation on. Can be
128 "database", "collection", "client", or "gridfsbucket."
129
130 - ``arguments``: Optional, the names and values of arguments.
131
132 - ``result``: Optional. The return value from the operation, if any. This
133 field may be a scalar (e.g. in the case of a count), a single document, or
134 an array of documents in the case of a multi-document read.
135
136 - ``error``: Optional. If ``true``, the test should expect an error or
137 exception.
138
139 - ``expectations``: Optional list of command-started events.
140
141GridFS Tests
142------------
143
144GridFS tests are denoted by when the YAML file contains ``bucket_name``.
145The ``data`` field will also be an object, which maps collection names
146(e.g. ``fs.files``) to an array of documents that should be inserted into
147the specified collection.
148
149``fs.files`` and ``fs.chunks`` should be created in the database
150specified by ``database_name``. This could be done via inserts or by
151creating GridFSBuckets—using the GridFS ``bucketName`` (see
152`GridFSBucket spec`_) specified by ``bucket_name`` field in the YAML
153file—and calling ``upload_from_stream_with_id`` with the appropriate
154data.
155
156``Download`` tests should be tested against ``GridFS.download_to_stream``.
157``DownloadByName`` tests should be tested against
158``GridFS.download_to_stream_by_name``.
159
160
161.. _GridFSBucket spec: https://github.com/mongodb/specifications/blob/master/source/gridfs/gridfs-spec.rst#configurable-gridfsbucket-class
162
163
164Speeding Up Tests
165-----------------
166
167Drivers can greatly reduce the execution time of tests by setting `heartbeatFrequencyMS`_
168and `minHeartbeatFrequencyMS`_ (internally) to a small value (e.g. 5ms), below what
169is normally permitted in the SDAM spec. If a test specifies an explicit value for
170heartbeatFrequencyMS (e.g. client or URI options), drivers MUST use that value.
171
172.. _minHeartbeatFrequencyMS: ../../server-discovery-and-monitoring/server-discovery-and-monitoring.rst#minheartbeatfrequencyms
173.. _heartbeatFrequencyMS: ../../server-discovery-and-monitoring/server-discovery-and-monitoring.rst#heartbeatfrequencyms
174
175Optional Enumeration Commands
176=============================
177
178A driver only needs to test the optional enumeration commands it has chosen to
179implement (e.g. ``Database.listCollectionNames()``).
180
181PoolClearedError Retryability Test
182==================================
183
184This test will be used to ensure drivers properly retry after encountering PoolClearedErrors.
185This test MUST be implemented by any driver that implements the CMAP specification.
186
1871. Create a client with maxPoolSize=1 and retryReads=true. If testing against a
188 sharded deployment, be sure to connect to only a single mongos.
189
1902. Enable the following failpoint::
191
192 {
193 configureFailPoint: "failCommand",
194 mode: { times: 1 },
195 data: {
196 failCommands: ["find"],
197 errorCode: 91,
198 blockConnection: true,
199 blockTimeMS: 1000
200 }
201 }
202
2033. Start two threads and attempt to perform a ``findOne`` simultaneously on both.
204
2054. Verify that both ``findOne`` attempts succeed.
206
2075. Via CMAP monitoring, assert that the first check out succeeds.
208
2096. Via CMAP monitoring, assert that a PoolClearedEvent is then emitted.
210
2117. Via CMAP monitoring, assert that the second check out then fails due to a
212 connection error.
213
2148. Via Command Monitoring, assert that exactly three ``find`` CommandStartedEvents
215 were observed in total.
216
2179. Disable the failpoint.
218
219
220Changelog
221=========
222
223:2019-03-19: Add top-level ``runOn`` field to denote server version and/or
224 topology requirements requirements for the test file. Removes the
225 ``minServerVersion`` and ``topology`` top-level fields, which are
226 now expressed within ``runOn`` elements.
227
228 Add test-level ``useMultipleMongoses`` field.
229
230:2020-09-16: Suggest lowering heartbeatFrequencyMS in addition to minHeartbeatFrequencyMS.
231
232:2021-03-23: Add prose test for retrying PoolClearedErrors
233
234:2021-04-29: Add ``load-balanced`` to test topology requirements.
View as plain text