1.. role:: javascript(code)
2 :language: javascript
3
4========================================
5Connection Monitoring and Pooling (CMAP)
6========================================
7
8.. contents::
9
10--------
11
12Introduction
13============
14
15The YAML and JSON files in this directory are platform-independent tests that
16drivers can use to prove their conformance to the Connection Monitoring and Pooling (CMAP) Spec.
17
18Several prose tests, which are not easily expressed in YAML, are also presented
19in this file. Those tests will need to be manually implemented by each driver.
20
21Common Test Format
22==================
23
24Each YAML file has the following keys:
25
26- ``version``: A version number indicating the expected format of the spec tests (current version = 1)
27- ``style``: A string indicating what style of tests this file contains. Contains one of the following:
28
29 - ``"unit"``: a test that may be run without connecting to a MongoDB deployment.
30 - ``"integration"``: a test that MUST be run against a real MongoDB deployment.
31
32- ``description``: A text description of what the test is meant to assert
33
34Unit Test Format:
35=================
36
37All Unit Tests have some of the following fields:
38
39- ``poolOptions``: If present, connection pool options to use when creating a pool;
40 both `standard ConnectionPoolOptions <https://github.com/mongodb/specifications/blob/master/source/connection-monitoring-and-pooling/connection-monitoring-and-pooling.rst#connection-pool-options-1>`__
41 and the following test-specific options are allowed:
42
43 - ``backgroundThreadIntervalMS``: A time interval between the end of a
44 `Background Thread Run <https://github.com/mongodb/specifications/blob/master/source/connection-monitoring-and-pooling/connection-monitoring-and-pooling.rst#background-thread>`__
45 and the beginning of the next Run. If a Connection Pool does not implement a Background Thread, the Test Runner MUST ignore the option.
46 If the option is not specified, an implementation is free to use any value it finds reasonable.
47
48 Possible values (0 is not allowed):
49
50 - A negative value: never begin a Run.
51 - A positive value: the interval between Runs in milliseconds.
52
53- ``operations``: A list of operations to perform. All operations support the following fields:
54
55 - ``name``: A string describing which operation to issue.
56 - ``thread``: The name of the thread in which to run this operation. If not specified, runs in the default thread
57
58- ``error``: Indicates that the main thread is expected to error during this test. An error may include of the following fields:
59
60 - ``type``: the type of error emitted
61 - ``message``: the message associated with that error
62 - ``address``: Address of pool emitting error
63
64- ``events``: An array of all connection monitoring events expected to occur while running ``operations``. An event may contain any of the following fields
65
66 - ``type``: The type of event emitted
67 - ``address``: The address of the pool emitting the event
68 - ``connectionId``: The id of a connection associated with the event
69 - ``options``: Options used to create the pool
70 - ``reason``: A reason giving more information on why the event was emitted
71
72- ``ignore``: An array of event names to ignore
73
74Valid Unit Test Operations are the following:
75
76- ``start(target)``: Starts a new thread named ``target``
77
78 - ``target``: The name of the new thread to start
79
80- ``wait(ms)``: Sleep the current thread for ``ms`` milliseconds
81
82 - ``ms``: The number of milliseconds to sleep the current thread for
83
84- ``waitForThread(target)``: wait for thread ``target`` to finish executing. Propagate any errors to the main thread.
85
86 - ``target``: The name of the thread to wait for.
87
88- ``waitForEvent(event, count, timeout)``: block the current thread until ``event`` has occurred ``count`` times
89
90 - ``event``: The name of the event
91 - ``count``: The number of times the event must occur (counting from the start of the test)
92 - ``timeout``: If specified, time out with an error after waiting for this many milliseconds without seeing the required events
93
94- ``label = pool.checkOut()``: call ``checkOut`` on pool, returning the checked out connection
95
96 - ``label``: If specified, associate this label with the returned connection, so that it may be referenced in later operations
97
98- ``pool.checkIn(connection)``: call ``checkIn`` on pool
99
100 - ``connection``: A string label identifying which connection to check in. Should be a label that was previously set with ``checkOut``
101
102- ``pool.clear()``: call ``clear`` on Pool
103- ``pool.close()``: call ``close`` on Pool
104- ``pool.ready()``: call ``ready`` on Pool
105
106
107Integration Test Format
108=======================
109
110The integration test format is identical to the unit test format with
111the addition of the following fields to each test:
112
113- ``runOn`` (optional): An array of server version and/or topology requirements
114 for which the tests can be run. If the test environment satisfies one or more
115 of these requirements, the tests may be executed; otherwise, this test should
116 be skipped. If this field is omitted, the tests can be assumed to have no
117 particular requirements and should be executed. Each element will have some or
118 all of the following fields:
119
120 - ``minServerVersion`` (optional): The minimum server version (inclusive)
121 required to successfully run the tests. If this field is omitted, it should
122 be assumed that there is no lower bound on the required server version.
123
124 - ``maxServerVersion`` (optional): The maximum server version (inclusive)
125 against which the tests can be run successfully. If this field is omitted,
126 it should be assumed that there is no upper bound on the required server
127 version.
128
129- ``failPoint``: optional, a document containing a ``configureFailPoint``
130 command to run against the endpoint being used for the test.
131
132- ``poolOptions.appName`` (optional): appName attribute to be set in connections, which will be affected by the fail point.
133
134Spec Test Match Function
135========================
136
137The definition of MATCH or MATCHES in the Spec Test Runner is as follows:
138
139- MATCH takes two values, ``expected`` and ``actual``
140- Notation is "Assert [actual] MATCHES [expected]
141- Assertion passes if ``expected`` is a subset of ``actual``, with the values ``42`` and ``"42"`` acting as placeholders for "any value"
142
143Pseudocode implementation of ``actual`` MATCHES ``expected``:
144
145::
146
147 If expected is "42" or 42:
148 Assert that actual exists (is not null or undefined)
149 Else:
150 Assert that actual is of the same JSON type as expected
151 If expected is a JSON array:
152 For every idx/value in expected:
153 Assert that actual[idx] MATCHES value
154 Else if expected is a JSON object:
155 For every key/value in expected
156 Assert that actual[key] MATCHES value
157 Else:
158 Assert that expected equals actual
159
160Unit Test Runner:
161=================
162
163For the unit tests, the behavior of a Connection is irrelevant beyond the need to asserting ``connection.id``. Drivers MAY use a mock connection class for testing the pool behavior in unit tests
164
165For each YAML file with ``style: unit``:
166
167- Create a Pool ``pool``, subscribe and capture any Connection Monitoring events emitted in order.
168
169 - If ``poolOptions`` is specified, use those options to initialize both pools
170 - The returned pool must have an ``address`` set as a string value.
171
172- Process each ``operation`` in ``operations`` (on the main thread)
173
174 - If a ``thread`` is specified, the main thread MUST schedule the operation to execute in the corresponding thread. Otherwise, execute the operation directly in the main thread.
175
176- If ``error`` is presented
177
178 - Assert that an actual error ``actualError`` was thrown by the main thread
179 - Assert that ``actualError`` MATCHES ``error``
180
181- Else:
182
183 - Assert that no errors were thrown by the main thread
184
185- calculate ``actualEvents`` as every Connection Event emitted whose ``type`` is not in ``ignore``
186- if ``events`` is not empty, then for every ``idx``/``expectedEvent`` in ``events``
187
188 - Assert that ``actualEvents[idx]`` exists
189 - Assert that ``actualEvents[idx]`` MATCHES ``expectedEvent``
190
191
192It is important to note that the ``ignore`` list is used for calculating ``actualEvents``, but is NOT used for the ``waitForEvent`` command
193
194Integration Test Runner
195=======================
196
197The steps to run the integration tests are the same as those used to run the
198unit tests with the following modifications:
199
200- The integration tests MUST be run against an actual endpoint. If the
201 deployment being tested contains multiple endpoints, then the runner MUST
202 only use one of them to run the tests against.
203
204- For each test, if `failPoint` is specified, its value is a
205 ``configureFailPoint`` command. Run the command on the admin database of the
206 endpoint being tested to enable the fail point.
207
208- At the end of each test, any enabled fail point MUST be disabled to avoid
209 spurious failures in subsequent tests. The fail point may be disabled like
210 so::
211
212 db.adminCommand({
213 configureFailPoint: <fail point name>,
214 mode: "off"
215 });
216
217
218Prose Tests
219===========
220
221The following tests have not yet been automated, but MUST still be tested
222
223#. All ConnectionPoolOptions MUST be specified at the MongoClient level
224#. All ConnectionPoolOptions MUST be the same for all pools created by a MongoClient
225#. A user MUST be able to specify all ConnectionPoolOptions via a URI string
226#. A user MUST be able to subscribe to Connection Monitoring Events in a manner idiomatic to their language and driver
227#. When a check out attempt fails because connection set up throws an error,
228 assert that a ConnectionCheckOutFailedEvent with reason="connectionError" is emitted.
View as plain text