...

Text file src/go.mongodb.org/mongo-driver/testdata/client-side-operations-timeout/README.rst

Documentation: go.mongodb.org/mongo-driver/testdata/client-side-operations-timeout

     1======================================
     2Client Side Operations Timeouts Tests
     3======================================
     4
     5.. contents::
     6
     7----
     8
     9Introduction
    10============
    11
    12This document describes the tests that drivers MUST run to validate the behavior of the timeoutMS option. These tests
    13are broken up into automated YAML/JSON tests and additional prose tests.
    14
    15Spec Tests
    16==========
    17
    18This directory contains a set of YAML and JSON spec tests. Drivers MUST run these as described in the "Unified Test
    19Runner" specification. Because the tests introduced in this specification are timing-based, there is a risk that some
    20of them may intermittently fail without any bugs being present in the driver. As a mitigatio, drivers MAY execute
    21these tests in two new Evergreen tasks that use single-node replica sets: one with only authentication enabled and
    22another with both authentication and TLS enabled. Drivers that choose to do so SHOULD use the ``single-node-auth.json``
    23and ``single-node-auth-ssl.json`` files in the ``drivers-evergreen-tools`` repository to create these clusters.
    24
    25Prose Tests
    26===========
    27
    28There are some tests that cannot be expressed in the unified YAML/JSON format. For each of these tests, drivers MUST
    29create a MongoClient without the ``timeoutMS`` option set (referred to as ``internalClient``). Any fail points set
    30during a test MUST be unset using ``internalClient`` after the test has been executed. All MongoClient instances
    31created for tests MUST be configured with read/write concern ``majority``, read preference ``primary``, and command
    32monitoring enabled to listen for ``command_started`` events.
    33
    34Multi-batch writes
    35~~~~~~~~~~~~~~~~~~
    36
    37This test MUST only run against server versions 4.4 and higher.
    38
    39#. Using ``internalClient``, drop the ``db.coll`` collection.
    40#. Using ``internalClient``, set the following fail point:
    41
    42   .. code:: javascript
    43
    44       {
    45           configureFailPoint: "failCommand",
    46           mode: {
    47               times: 2
    48           },
    49           data: {
    50               failCommands: ["insert"],
    51               blockConnection: true,
    52               blockTimeMS: 15
    53           }
    54       }
    55
    56#. Create a new MongoClient (referred to as ``client``) with ``timeoutMS=20``.
    57#. Using ``client``, insert 100,001 empty documents in a single ``insertMany`` call.
    58
    59   - Expect this to fail with a timeout error.
    60
    61#. Verify that two ``insert`` commands were executed against ``db.coll`` as part of the ``insertMany`` call.
    62
    63maxTimeMS is not set for commands sent to mongocryptd
    64~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    65
    66This test MUST only be run against enterprise server versions 4.2 and higher.
    67
    68#. Launch a mongoryptd process on 23000.
    69#. Create a MongoClient (referred to as ``client``) using the URI ``mongodb://localhost:23000/?timeoutMS=1000``.
    70#. Using ``client``, execute the ``{ ping: 1 }`` command against the ``admin`` database.
    71#. Verify via command monitoring that the ``ping`` command sent did not contain a ``maxTimeMS`` field.
    72
    73ClientEncryption
    74~~~~~~~~~~~~~~~~
    75
    76Each test under this category MUST only be run against server versions 4.4 and higher. In these tests,
    77``LOCAL_MASTERKEY`` refers to the following base64:
    78
    79.. code:: javascript
    80
    81  Mng0NCt4ZHVUYUJCa1kxNkVyNUR1QURhZ2h2UzR2d2RrZzh0cFBwM3R6NmdWMDFBMUN3YkQ5aXRRMkhGRGdQV09wOGVNYUMxT2k3NjZKelhaQmRCZGJkTXVyZG9uSjFk
    82
    83For each test, perform the following setup:
    84
    85#. Using ``internalClient``, drop and create the ``keyvault.datakeys`` collection.
    86#. Create a MongoClient (referred to as ``keyVaultClient``) with ``timeoutMS=10``.
    87#. Create a ``ClientEncryption`` object that wraps ``keyVaultClient`` (referred to as ``clientEncryption``). Configure this object with ``keyVaultNamespace`` set to ``keyvault.datakeys`` and the following KMS providers map:
    88
    89   .. code:: javascript
    90
    91       {
    92           "local": { "key": <base64 decoding of LOCAL_MASTERKEY> }
    93       }
    94
    95createDataKey
    96`````````````
    97
    98#. Using ``internalClient``, set the following fail point:
    99
   100   .. code:: javascript
   101
   102       {
   103           configureFailPoint: "failCommand",
   104           mode: {
   105               times: 1
   106           },
   107           data: {
   108               failCommands: ["insert"],
   109               blockConnection: true,
   110               blockTimeMS: 15
   111           }
   112       }
   113
   114#. Call ``clientEncryption.createDataKey()`` with the ``local`` KMS provider.
   115
   116   - Expect this to fail with a timeout error.
   117
   118#. Verify that an ``insert`` command was executed against to ``keyvault.datakeys`` as part of the ``createDataKey`` call.
   119
   120encrypt
   121```````
   122
   123#. Call ``client_encryption.createDataKey()`` with the ``local`` KMS provider.
   124
   125   - Expect a BSON binary with subtype 4 to be returned, referred to as ``datakeyId``.
   126
   127#. Using ``internalClient``, set the following fail point:
   128
   129   .. code:: javascript
   130
   131       {
   132           configureFailPoint: "failCommand",
   133           mode: {
   134               times: 1
   135           },
   136           data: {
   137               failCommands: ["find"],
   138               blockConnection: true,
   139               blockTimeMS: 15
   140           }
   141       }
   142
   143#. Call ``clientEncryption.encrypt()`` with the value ``hello``, the algorithm ``AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic``, and the keyId ``datakeyId``.
   144
   145   - Expect this to fail with a timeout error.
   146
   147#. Verify that a ``find`` command was executed against the ``keyvault.datakeys`` collection as part of the ``encrypt`` call.
   148
   149decrypt
   150```````
   151
   152#. Call ``clientEncryption.createDataKey()`` with the ``local`` KMS provider.
   153
   154   - Expect this to return a BSON binary with subtype 4, referred to as ``dataKeyId``.
   155
   156#. Call ``clientEncryption.encrypt()`` with the value ``hello``, the algorithm ``AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic``, and the keyId ``dataKeyId``.
   157
   158   - Expect this to return a BSON binary with subtype 6, referred to as ``encrypted``.
   159
   160#. Close and re-create the ``keyVaultClient`` and ``clientEncryption`` objects.
   161
   162#. Using ``internalClient``, set the following fail point:
   163
   164   .. code:: javascript
   165
   166       {
   167           configureFailPoint: "failCommand",
   168           mode: {
   169               times: 1
   170           },
   171           data: {
   172               failCommands: ["find"],
   173               blockConnection: true,
   174               blockTimeMS: 15
   175           }
   176       }
   177
   178#. Call ``clientEncryption.decrypt()`` with the value ``encrypted``.
   179
   180   - Expect this to fail with a timeout error.
   181
   182#. Verify that a ``find`` command was executed against the ``keyvault.datakeys`` collection as part of the ``decrypt`` call.
   183
   184Background Connection Pooling
   185~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   186
   187The tests in this section MUST only be run if the server version is 4.4 or higher and the URI has authentication
   188fields (i.e. a username and password). Each test in this section requires drivers to create a MongoClient and then wait
   189for some CMAP events to be published. Drivers MUST wait for up to 10 seconds and fail the test if the specified events
   190are not published within that time.
   191
   192timeoutMS used for handshake commands
   193`````````````````````````````````````
   194
   195#. Using ``internalClient``, set the following fail point:
   196
   197   .. code:: javascript
   198
   199       {
   200           configureFailPoint: "failCommand",
   201           mode: {
   202               times: 1
   203           },
   204           data: {
   205               failCommands: ["saslContinue"],
   206               blockConnection: true,
   207               blockTimeMS: 15,
   208               appName: "timeoutBackgroundPoolTest"
   209           }
   210       }
   211
   212#. Create a MongoClient (referred to as ``client``) configured with the following:
   213
   214   - ``minPoolSize`` of 1
   215   - ``timeoutMS`` of 10
   216   - ``appName`` of ``timeoutBackgroundPoolTest``
   217   - CMAP monitor configured to listen for ``ConnectionCreatedEvent`` and ``ConnectionClosedEvent`` events.
   218
   219#. Wait for a ``ConnectionCreatedEvent`` and a ``ConnectionClosedEvent`` to be published.
   220
   221timeoutMS is refreshed for each handshake command
   222`````````````````````````````````````````````````
   223
   224#. Using ``internalClient``, set the following fail point:
   225
   226   .. code:: javascript
   227
   228       {
   229           configureFailPoint: "failCommand",
   230           mode: "alwaysOn",
   231           data: {
   232               failCommands: ["isMaster", "saslContinue"],
   233               blockConnection: true,
   234               blockTimeMS: 15,
   235               appName: "refreshTimeoutBackgroundPoolTest"
   236           }
   237       }
   238
   239#. Create a MongoClient (referred to as ``client``) configured with the following:
   240
   241   - ``minPoolSize`` of 1
   242   - ``timeoutMS`` of 20
   243   - ``appName`` of ``refreshTimeoutBackgroundPoolTest``
   244   - CMAP monitor configured to listen for ``ConnectionCreatedEvent`` and ``ConnectionReady`` events.
   245
   246#. Wait for a ``ConnectionCreatedEvent`` and a ``ConnectionReady`` to be published.
   247
   248Blocking Iteration Methods
   249~~~~~~~~~~~~~~~~~~~~~~~~~~
   250
   251Tests in this section MUST only be run against server versions 4.4 and higher and only apply to drivers that have a
   252blocking method for cursor iteration that executes ``getMore`` commands in a loop until a document is available or an
   253error occurs.
   254
   255Tailable cursors
   256````````````````
   257
   258#. Using ``internalClient``, drop the ``db.coll`` collection.
   259#. Using ``internalClient``, insert the document ``{ x: 1 }`` into ``db.coll``.
   260#. Using ``internalClient``, set the following fail point:
   261
   262   .. code:: javascript
   263
   264       {
   265           configureFailPoint: "failCommand",
   266           mode: "alwaysOn",
   267           data: {
   268               failCommands: ["getMore"],
   269               blockConnection: true,
   270               blockTimeMS: 15
   271           }
   272       }
   273
   274#. Create a new MongoClient (referred to as ``client``) with ``timeoutMS=20``.
   275#. Using ``client``, create a tailable cursor on ``db.coll`` with ``cursorType=tailable``.
   276
   277   - Expect this to succeed and return a cursor with a non-zero ID.
   278
   279#. Call either a blocking or non-blocking iteration method on the cursor.
   280
   281   - Expect this to succeed and return the document ``{ x: 1 }`` without sending a ``getMore`` command.
   282
   283#. Call the blocking iteration method on the resulting cursor.
   284
   285   - Expect this to fail with a timeout error.
   286
   287#. Verify that a ``find`` command and two ``getMore`` commands were executed against the ``db.coll`` collection during the test.
   288
   289Change Streams
   290``````````````
   291
   292#. Using ``internalClient``, drop the ``db.coll`` collection.
   293#. Using ``internalClient``, set the following fail point:
   294
   295   .. code:: javascript
   296
   297       {
   298           configureFailPoint: "failCommand",
   299           mode: "alwaysOn",
   300           data: {
   301               failCommands: ["getMore"],
   302               blockConnection: true,
   303               blockTimeMS: 15
   304           }
   305       }
   306
   307#. Create a new MongoClient (referred to as ``client``) with ``timeoutMS=20``.
   308#. Using ``client``, use the ``watch`` helper to create a change stream against ``db.coll``.
   309
   310   - Expect this to succeed and return a change stream with a non-zero ID.
   311
   312#. Call the blocking iteration method on the resulting change stream.
   313
   314   - Expect this to fail with a timeout error.
   315
   316#. Verify that an ``aggregate`` command and two ``getMore`` commands were executed against the ``db.coll`` collection during the test.
   317
   318GridFS - Upload
   319~~~~~~~~~~~~~~~
   320
   321Tests in this section MUST only be run against server versions 4.4 and higher.
   322
   323uploads via openUploadStream can be timed out
   324`````````````````````````````````````````````
   325
   326#. Using ``internalClient``, drop and re-create the ``db.fs.files`` and ``db.fs.chunks`` collections.
   327#. Using ``internalClient``, set the following fail point:
   328
   329   .. code:: javascript
   330
   331       {
   332           configureFailPoint: "failCommand",
   333           mode: { times: 1 },
   334           data: {
   335               failCommands: ["insert"],
   336               blockConnection: true,
   337               blockTimeMS: 15
   338           }
   339       }
   340
   341#. Create a new MongoClient (referred to as ``client``) with ``timeoutMS=10``.
   342#. Using ``client``, create a GridFS bucket (referred to as ``bucket``) that wraps the ``db`` database.
   343#. Call ``bucket.open_upload_stream()`` with the filename ``filename`` to create an upload stream (referred to as ``uploadStream``).
   344
   345   - Expect this to succeed and return a non-null stream.
   346
   347#. Using ``uploadStream``, upload a single ``0x12`` byte.
   348#. Call ``uploadStream.close()`` to flush the stream and insert chunks.
   349
   350   - Expect this to fail with a timeout error.
   351
   352Aborting an upload stream can be timed out
   353``````````````````````````````````````````
   354
   355This test only applies to drivers that provide an API to abort a GridFS upload stream.
   356
   357#. Using ``internalClient``, drop and re-create the ``db.fs.files`` and ``db.fs.chunks`` collections.
   358#. Using ``internalClient``, set the following fail point:
   359
   360   .. code:: javascript
   361
   362       {
   363           configureFailPoint: "failCommand",
   364           mode: { times: 1 },
   365           data: {
   366               failCommands: ["delete"],
   367               blockConnection: true,
   368               blockTimeMS: 15
   369           }
   370       }
   371
   372#. Create a new MongoClient (referred to as ``client``) with ``timeoutMS=10``.
   373#. Using ``client``, create a GridFS bucket (referred to as ``bucket``) that wraps the ``db`` database with ``chunkSizeBytes=2``.
   374#. Call ``bucket.open_upload_stream()`` with the filename ``filename`` to create an upload stream (referred to as ``uploadStream``).
   375
   376   - Expect this to succeed and return a non-null stream.
   377
   378#. Using ``uploadStream``, upload the bytes ``[0x01, 0x02, 0x03, 0x04]``.
   379#. Call ``uploadStream.abort()``.
   380
   381   - Expect this to fail with a timeout error.
   382
   383GridFS - Download
   384~~~~~~~~~~~~~~~~~
   385
   386This test MUST only be run against server versions 4.4 and higher.
   387
   388#. Using ``internalClient``, drop and re-create the ``db.fs.files`` and ``db.fs.chunks`` collections.
   389#. Using ``internalClient``, insert the following document into the ``db.fs.files`` collection:
   390
   391   .. code:: javascript
   392
   393       {
   394          "_id": {
   395            "$oid": "000000000000000000000005"
   396          },
   397          "length": 10,
   398          "chunkSize": 4,
   399          "uploadDate": {
   400            "$date": "1970-01-01T00:00:00.000Z"
   401          },
   402          "md5": "57d83cd477bfb1ccd975ab33d827a92b",
   403          "filename": "length-10",
   404          "contentType": "application/octet-stream",
   405          "aliases": [],
   406          "metadata": {}
   407       }
   408
   409#. Create a new MongoClient (referred to as ``client``) with ``timeoutMS=10``.
   410#. Using ``client``, create a GridFS bucket (referred to as ``bucket``) that wraps the ``db`` database.
   411#. Call ``bucket.open_download_stream`` with the id ``{ "$oid": "000000000000000000000005" }`` to create a download stream (referred to as ``downloadStream``).
   412
   413   - Expect this to succeed and return a non-null stream.
   414
   415#. Using ``internalClient``, set the following fail point:
   416
   417   .. code:: javascript
   418
   419       {
   420           configureFailPoint: "failCommand",
   421           mode: { times: 1 },
   422           data: {
   423               failCommands: ["find"],
   424               blockConnection: true,
   425               blockTimeMS: 15
   426           }
   427       }
   428
   429#. Read from the ``downloadStream``.
   430
   431   - Expect this to fail with a timeout error.
   432
   433#. Verify that two ``find`` commands were executed during the read: one against ``db.fs.files`` and another against ``db.fs.chunks``.
   434
   435Server Selection
   436~~~~~~~~~~~~~~~~
   437
   438serverSelectionTimeoutMS honored if timeoutMS is not set
   439````````````````````````````````````````````````````````
   440
   441#. Create a MongoClient (referred to as ``client``) with URI ``mongodb://invalid/?serverSelectionTimeoutMS=10``.
   442
   443#. Using ``client``, execute the command ``{ ping: 1 }`` against the ``admin`` database.
   444
   445   - Expect this to fail with a server selection timeout error after no more than 15ms.
   446
   447timeoutMS honored for server selection if it's lower than serverSelectionTimeoutMS
   448``````````````````````````````````````````````````````````````````````````````````
   449
   450#. Create a MongoClient (referred to as ``client``) with URI ``mongodb://invalid/?timeoutMS=10&serverSelectionTimeoutMS=20``.
   451
   452#. Using ``client``, run the command ``{ ping: 1 }`` against the ``admin`` database.
   453
   454   - Expect this to fail with a server selection timeout error after no more than 15ms.
   455
   456serverSelectionTimeoutMS honored for server selection if it's lower than timeoutMS
   457``````````````````````````````````````````````````````````````````````````````````
   458
   459#. Create a MongoClient (referred to as ``client``) with URI ``mongodb://invalid/?timeoutMS=20&serverSelectionTimeoutMS=10``.
   460
   461#. Using ``client``, run the command ``{ ping: 1 }`` against the ``admin`` database.
   462
   463   - Expect this to fail with a server selection timeout error after no more than 15ms.
   464
   465serverSelectionTimeoutMS honored for server selection if timeoutMS=0
   466````````````````````````````````````````````````````````````````````
   467
   468#. Create a MongoClient (referred to as ``client``) with URI ``mongodb://invalid/?timeoutMS=0&serverSelectionTimeoutMS=10``.
   469
   470#. Using ``client``, run the command ``{ ping: 1 }`` against the ``admin`` database.
   471
   472   - Expect this to fail with a server selection timeout error after no more than 15ms.
   473
   474timeoutMS honored for connection handshake commands if it's lower than serverSelectionTimeoutMS
   475```````````````````````````````````````````````````````````````````````````````````````````````
   476
   477This test MUST only be run if the server version is 4.4 or higher and the URI has authentication fields (i.e. a
   478username and password).
   479
   480#. Using ``internalClient``, set the following fail point:
   481
   482   .. code:: javascript
   483
   484       {
   485           configureFailPoint: failCommand,
   486           mode: { times: 1 },
   487           data: {
   488               failCommands: ["saslContinue"],
   489               blockConnection: true,
   490               blockTimeMS: 15
   491           }
   492       }
   493
   494#. Create a new MongoClient (referred to as ``client``) with ``timeoutMS=10`` and ``serverSelectionTimeoutMS=20``.
   495#. Using ``client``, insert the document ``{ x: 1 }`` into collection ``db.coll``.
   496
   497   - Expect this to fail with a timeout error after no more than 15ms.
   498
   499serverSelectionTimeoutMS honored for connection handshake commands if it's lower than timeoutMS
   500```````````````````````````````````````````````````````````````````````````````````````````````
   501
   502This test MUST only be run if the server version is 4.4 or higher and the URI has authentication fields (i.e. a
   503username and password).
   504
   505#. Using ``internalClient``, set the following fail point:
   506
   507   .. code:: javascript
   508
   509       {
   510           configureFailPoint: failCommand,
   511           mode: { times: 1 },
   512           data: {
   513               failCommands: ["saslContinue"],
   514               blockConnection: true,
   515               blockTimeMS: 15
   516           }
   517       }
   518
   519#. Create a new MongoClient (referred to as ``client``) with ``timeoutMS=20`` and ``serverSelectionTimeoutMS=10``.
   520#. Using ``client``, insert the document ``{ x: 1 }`` into collection ``db.coll``.
   521
   522   - Expect this to fail with a timeout error after no more than 15ms.
   523
   524endSession
   525~~~~~~~~~~
   526
   527This test MUST only be run against replica sets and sharded clusters with server version 4.4 or higher. It MUST be
   528run three times: once with the timeout specified via the MongoClient ``timeoutMS`` option, once with the timeout
   529specified via the ClientSession ``defaultTimeoutMS`` option, and once more with the timeout specified via the
   530``timeoutMS`` option for the ``endSession`` operation. In all cases, the timeout MUST be set to 10 milliseconds.
   531
   532#. Using ``internalClient``, drop the ``db.coll`` collection.
   533#. Using ``internalClient``, set the following fail point:
   534
   535   .. code:: javascript
   536
   537       {
   538           configureFailPoint: failCommand,
   539           mode: { times: 1 },
   540           data: {
   541               failCommands: ["abortTransaction"],
   542               blockConnection: true,
   543               blockTimeMS: 15
   544           }
   545       }
   546
   547#. Create a new MongoClient (referred to as ``client``) and an explicit ClientSession derived from that MongoClient (referred to as ``session``).
   548#. Execute the following code:
   549
   550   .. code:: typescript
   551
   552       coll = client.database("db").collection("coll")
   553       session.start_transaction()
   554       coll.insert_one({x: 1}, session=session)
   555
   556#. Using ``session``, execute ``session.end_session``
   557
   558   - Expect this to fail with a timeout error after no more than 15ms.
   559
   560Convenient Transactions
   561~~~~~~~~~~~~~~~~~~~~~~~
   562
   563Tests in this section MUST only run against replica sets and sharded clusters with server versions 4.4 or higher.
   564
   565timeoutMS is refreshed for abortTransaction if the callback fails
   566`````````````````````````````````````````````````````````````````
   567
   568#. Using ``internalClient``, drop the ``db.coll`` collection.
   569#. Using ``internalClient``, set the following fail point:
   570
   571   .. code:: javascript
   572
   573       {
   574           configureFailPoint: failCommand,
   575           mode: { times: 2 },
   576           data: {
   577               failCommands: ["insert", "abortTransaction"],
   578               blockConnection: true,
   579               blockTimeMS: 15
   580           }
   581       }
   582
   583#. Create a new MongoClient (referred to as ``client``) configured with ``timeoutMS=10`` and an explicit ClientSession derived from that MongoClient (referred to as ``session``).
   584#. Using ``session``, execute a ``withTransaction`` operation with the following callback:
   585
   586   .. code:: typescript
   587
   588       def callback() {
   589           coll = client.database("db").collection("coll")
   590           coll.insert_one({ _id: 1 }, session=session)
   591       }
   592
   593#. Expect the previous ``withTransaction`` call to fail with a timeout error.
   594#. Verify that the following events were published during the ``withTransaction`` call:
   595
   596   #. ``command_started`` and ``command_failed`` events for an ``insert`` command.
   597   #. ``command_started`` and ``command_failed`` events for an ``abortTransaction`` command.
   598
   599Unit Tests
   600==========
   601
   602The tests enumerated in this section could not be expressed in either spec or prose format. Drivers SHOULD implement
   603these if it is possible to do so using the driver's existing test infrastructure.
   604
   605- Operations should ignore ``waitQueueTimeoutMS`` if ``timeoutMS`` is also set.
   606- If ``timeoutMS`` is set for an operation, the remaining ``timeoutMS`` value should apply to connection checkout after a server has been selected.
   607- If ``timeoutMS`` is not set for an operation, ``waitQueueTimeoutMS`` should apply to connection checkout after a server has been selected.
   608- If a new connection is required to execute an operation, ``min(remaining computedServerSelectionTimeout, connectTimeoutMS)`` should apply to socket establishment.
   609- For drivers that have control over OCSP behavior, ``min(remaining computedServerSelectionTimeout, 5 seconds)`` should apply to HTTP requests against OCSP responders.
   610- If ``timeoutMS`` is unset, operations fail after two non-consecutive socket timeouts.
   611- The remaining ``timeoutMS`` value should apply to HTTP requests against KMS servers for CSFLE.
   612- The remaining ``timeoutMS`` value should apply to commands sent to mongocryptd as part of automatic encryption.
   613- When doing ``minPoolSize`` maintenance, ``connectTimeoutMS`` is used as the timeout for socket establishment.

View as plain text