1.. role:: javascript(code)
2 :language: javascript
3
4==============
5Change Streams
6==============
7
8.. contents::
9
10--------
11
12Introduction
13============
14
15The YAML and JSON files in this directory are platform-independent tests that
16drivers can use to prove their conformance to the Change Streams Spec.
17
18Several prose tests, which are not easily expressed in YAML, are also presented
19in this file. Those tests will need to be manually implemented by each driver.
20
21Subdirectories for Test Formats
22-------------------------------
23
24This document describes the legacy format for change streams tests.
25Tests in this legacy format are located under ``./legacy/``.
26
27New change streams tests should be written in the `unified test format <../../unified-test-format/unified-test-format.rst>`__
28and placed under ``./unified/``.
29
30Spec Test Format
31================
32
33Each YAML file has the following keys:
34
35- ``database_name``: The default database
36- ``collection_name``: The default collection
37- ``database2_name``: Another database
38- ``collection2_name``: Another collection
39- ``tests``: An array of tests that are to be run independently of each other.
40 Each test will have some of the following fields:
41
42 - ``description``: The name of the test.
43 - ``minServerVersion``: The minimum server version to run this test against. If not present, assume there is no minimum server version.
44 - ``maxServerVersion``: Reserved for later use
45 - ``failPoint``: Optional configureFailPoint command document to run to configure a fail point on the primary server.
46 - ``target``: The entity on which to run the change stream. Valid values are:
47
48 - ``collection``: Watch changes on collection ``database_name.collection_name``
49 - ``database``: Watch changes on database ``database_name``
50 - ``client``: Watch changes on entire clusters
51 - ``topology``: An array of server topologies against which to run the test.
52 Valid topologies are ``single``, ``replicaset``, ``sharded``, and "load-balanced".
53 - ``changeStreamPipeline``: An array of additional aggregation pipeline stages to add to the change stream
54 - ``changeStreamOptions``: Additional options to add to the changeStream
55 - ``operations``: Array of documents, each describing an operation. Each document has the following fields:
56
57 - ``database``: Database against which to run the operation
58 - ``collection``: Collection against which to run the operation
59 - ``name``: Name of the command to run
60 - ``arguments`` (optional): Object of arguments for the command (ex: document to insert)
61
62 - ``expectations``: Optional list of command-started events in Extended JSON format
63 - ``result``: Document with ONE of the following fields:
64
65 - ``error``: Describes an error received during the test
66 - ``success``: An Extended JSON array of documents expected to be received from the changeStream
67
68Spec Test Match Function
69========================
70
71The definition of MATCH or MATCHES in the Spec Test Runner is as follows:
72
73- MATCH takes two values, ``expected`` and ``actual``
74- Notation is "Assert [actual] MATCHES [expected]
75- Assertion passes if ``expected`` is a subset of ``actual``, with the value ``42`` acting as placeholders for "any value"
76
77Pseudocode implementation of ``actual`` MATCHES ``expected``:
78
79::
80
81 If expected is "42" or 42:
82 Assert that actual exists (is not null or undefined)
83 Else:
84 Assert that actual is of the same JSON type as expected
85 If expected is a JSON array:
86 For every idx/value in expected:
87 Assert that actual[idx] MATCHES value
88 Else if expected is a JSON object:
89 For every key/value in expected
90 Assert that actual[key] MATCHES value
91 Else:
92 Assert that expected equals actual
93
94The expected values for ``result.success`` and ``expectations`` are written in Extended JSON. Drivers may adopt any of the following approaches to comparisons, as long as they are consistent:
95
96- Convert ``actual`` to Extended JSON and compare to ``expected``
97- Convert ``expected`` and ``actual`` to BSON, and compare them
98- Convert ``expected`` and ``actual`` to native equivalents of JSON, and compare them
99
100Spec Test Runner
101================
102
103Before running the tests
104
105- Create a MongoClient ``globalClient``, and connect to the server.
106When executing tests against a sharded cluster, ``globalClient`` must only connect to one mongos. This is because tests
107that set failpoints will only work consistently if both the ``configureFailPoint`` and failing commands are sent to the
108same mongos.
109
110For each YAML file, for each element in ``tests``:
111
112- If ``topology`` does not include the topology of the server instance(s), skip this test.
113- Use ``globalClient`` to
114
115 - Drop the database ``database_name``
116 - Drop the database ``database2_name``
117 - Create the database ``database_name`` and the collection ``database_name.collection_name``
118 - Create the database ``database2_name`` and the collection ``database2_name.collection2_name``
119 - If the the ``failPoint`` field is present, configure the fail point on the primary server. See
120 `Server Fail Point <../../transactions/tests#server-fail-point>`_ in the
121 Transactions spec test documentation for more information.
122
123- Create a new MongoClient ``client``
124- Begin monitoring all APM events for ``client``. (If the driver uses global listeners, filter out all events that do not originate with ``client``). Filter out any "internal" commands (e.g. ``hello`` or legacy hello)
125- Using ``client``, create a changeStream ``changeStream`` against the specified ``target``. Use ``changeStreamPipeline`` and ``changeStreamOptions`` if they are non-empty. Capture any error.
126- If there was no error, use ``globalClient`` and run every operation in ``operations`` in serial against the server until all operations have been executed or an error is thrown. Capture any error.
127- If there was no error and ``result.error`` is set, iterate ``changeStream`` once and capture any error.
128- If there was no error and ``result.success`` is non-empty, iterate ``changeStream`` until it returns as many changes as there are elements in the ``result.success`` array or an error is thrown. Capture any error.
129- Close ``changeStream``
130- If there was an error:
131
132 - Assert that an error was expected for the test.
133 - Assert that the error MATCHES ``result.error``
134
135- Else:
136
137 - Assert that no error was expected for the test
138 - Assert that the changes received from ``changeStream`` MATCH the results in ``result.success``
139
140- If there are any ``expectations``
141
142 - For each (``expected``, ``idx``) in ``expectations``
143 - If ``actual[idx]`` is a ``killCursors`` event, skip it and move to ``actual[idx+1]``.
144 - Else assert that ``actual[idx]`` MATCHES ``expected``
145 - Note: the change stream test command event expectations cover a
146 prefix subset of all command events published by the driver.
147 The test runner MUST verify that, if there are N expectations, that the
148 first N events published by the driver match the expectations, and
149 MUST NOT inspect any subsequent events published by the driver.
150
151- Close the MongoClient ``client``
152
153After running all tests
154
155- Close the MongoClient ``globalClient``
156- Drop database ``database_name``
157- Drop database ``database2_name``
158
159Iterating the Change Stream
160---------------------------
161
162Although synchronous drivers must provide a `non-blocking mode of iteration <../change-streams.rst#not-blocking-on-iteration>`_, asynchronous drivers may not have such a mechanism. Those drivers with only a blocking mode of iteration should be careful not to iterate the change stream unnecessarily, as doing so could cause the test runner to block indefinitely. For this reason, the test runner procedure above advises drivers to take a conservative approach to iteration.
163
164If the test expects an error and one was not thrown by either creating the change stream or executing the test's operations, iterating the change stream once allows for an error to be thrown by a ``getMore`` command. If the test does not expect any error, the change stream should be iterated only until it returns as many result documents as are expected by the test.
165
166Testing on Sharded Clusters
167---------------------------
168
169When writing data on sharded clusters, majority-committed data does not always show up in the response of the first
170``getMore`` command after the data is written. This is because in sharded clusters, no data from shard A may be returned
171until all other shard reports an entry that sorts after the change in shard A.
172
173To account for this, drivers MUST NOT rely on change stream documents in certain batches. For example, if expecting two
174documents in a change stream, these may not be part of the same ``getMore`` response, or even be produced in two
175subsequent ``getMore`` responses. Drivers MUST allow for a ``getMore`` to produce empty batches when testing on a
176sharded cluster. By default, this can take up to 10 seconds, but can be controlled by enabling the ``writePeriodicNoops``
177server parameter and configuring the ``periodNoopIntervalSecs`` parameter. Choosing lower values allows for running
178change stream tests with smaller timeouts.
179
180Prose Tests
181===========
182
183The following tests have not yet been automated, but MUST still be tested. All tests SHOULD be run on both replica sets and sharded clusters unless otherwise specified:
184
185#. ``ChangeStream`` must continuously track the last seen ``resumeToken``
186#. ``ChangeStream`` will throw an exception if the server response is missing the resume token (if wire version is < 8, this is a driver-side error; for 8+, this is a server-side error)
187#. After receiving a ``resumeToken``, ``ChangeStream`` will automatically resume one time on a resumable error with the initial pipeline and options, except for the addition/update of a ``resumeToken``.
188#. ``ChangeStream`` will not attempt to resume on any error encountered while executing an ``aggregate`` command. Note that retryable reads may retry ``aggregate`` commands. Drivers should be careful to distinguish retries from resume attempts. Alternatively, drivers may specify ``retryReads=false`` or avoid using a `retryable error <../../retryable-reads/retryable-reads.rst#retryable-error>`_ for this test.
189#. **Removed**
190#. ``ChangeStream`` will perform server selection before attempting to resume, using initial ``readPreference``
191#. Ensure that a cursor returned from an aggregate command with a cursor id and an initial empty batch is not closed on the driver side.
192#. The ``killCursors`` command sent during the "Resume Process" must not be allowed to throw an exception.
193#. ``$changeStream`` stage for ``ChangeStream`` against a server ``>=4.0`` and ``<4.0.7`` that has not received any results yet MUST include a ``startAtOperationTime`` option when resuming a change stream.
194#. **Removed**
195#. For a ``ChangeStream`` under these conditions:
196
197 - Running against a server ``>=4.0.7``.
198 - The batch is empty or has been iterated to the last document.
199
200 Expected result:
201
202 - ``getResumeToken`` must return the ``postBatchResumeToken`` from the current command response.
203
204#. For a ``ChangeStream`` under these conditions:
205
206 - Running against a server ``<4.0.7``.
207 - The batch is empty or has been iterated to the last document.
208
209 Expected result:
210
211 - ``getResumeToken`` must return the ``_id`` of the last document returned if one exists.
212 - ``getResumeToken`` must return ``resumeAfter`` from the initial aggregate if the option was specified.
213 - If ``resumeAfter`` was not specified, the ``getResumeToken`` result must be empty.
214
215#. For a ``ChangeStream`` under these conditions:
216
217 - The batch is not empty.
218 - The batch has been iterated up to but not including the last element.
219
220 Expected result:
221
222 - ``getResumeToken`` must return the ``_id`` of the previous document returned.
223
224#. For a ``ChangeStream`` under these conditions:
225
226 - The batch is not empty.
227 - The batch hasn’t been iterated at all.
228 - Only the initial ``aggregate`` command has been executed.
229
230 Expected result:
231
232 - ``getResumeToken`` must return ``startAfter`` from the initial aggregate if the option was specified.
233 - ``getResumeToken`` must return ``resumeAfter`` from the initial aggregate if the option was specified.
234 - If neither the ``startAfter`` nor ``resumeAfter`` options were specified, the ``getResumeToken`` result must be empty.
235
236 Note that this test cannot be run against sharded topologies because in that case the initial ``aggregate`` command only establishes cursors on the shards and always returns an empty ``firstBatch``.
237
238#. **Removed**
239#. **Removed**
240#. ``$changeStream`` stage for ``ChangeStream`` started with ``startAfter`` against a server ``>=4.1.1`` that has not received any results yet MUST include a ``startAfter`` option and MUST NOT include a ``resumeAfter`` option when resuming a change stream.
241#. ``$changeStream`` stage for ``ChangeStream`` started with ``startAfter`` against a server ``>=4.1.1`` that has received at least one result MUST include a ``resumeAfter`` option and MUST NOT include a ``startAfter`` option when resuming a change stream.
View as plain text