dev/MultiforaDB
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

tests/spec: assert expectEvents

Browse Source 
The headline is not the delta, it is that `pass` changed meaning. 354 of the
487 cases declare `expectEvents` and until now the runner read none of them,
so a case could send the wrong command entirely and still be counted a pass
as long as the *result* came back right. The old column was an upper bound by
construction. 193/99/195 becomes 159/133/195, and the two numbers are not
comparable.

Two rules decide how far the assertion reaches, both taken from the spec
rather than from what would be convenient:

  - `command` and `reply` match as *root* documents
    (unified-test-format.md:1020-1022, :1037-1039). The driver hangs `lsid`,
    `$db` and `maxTimeMS` off nearly everything it sends; as nested documents
    essentially the whole corpus would fail on keys no expectation was ever
    written to mention, and the number would say nothing.
  - the event list is exact in number and order, not a prefix
    (unified-test-format.md:3088-3091). 23 cases expect an empty list and a
    prefix rule would pass every one of them without looking.

The assertion runs after the operations, so a wrong result is still reported
as a wrong result rather than being masked, and after the listeners are
disabled, so the teardown's own commands cannot reach the buffer.

`cmap` and `sdam` event types, `ignoreExtraEvents`, and any event field beyond
`command`/`reply`/`commandName`/`databaseName` are reported unsupported at the
point of assertion. None occurs in this corpus -- all 354 blocks are
`eventType: command`, carrying 349 `commandStartedEvent` and 6
`commandSucceededEvent` -- so nothing is being quietly waived.

All 34 newly-failing cases, triaged. Not one is a wrong answer from the
engine; every one is a command the driver never sent:

  - 22x `command.writeConcern: missing` -- runner gap, and the sharpest thing
    this commit found. `buildEntities` drops `collectionOptions` on the floor,
    so `writeConcern: {w: 0}` never reached the driver and every "unacknowledged
    write" case in the corpus has been running an acknowledged write. They
    passed because the results of the two agree. This is precisely the class of
    error the instrument was built to find, and it was invisible to the result
    column.
  - 5x `command.sort.<key>: missing` -- runner gap. The driver holds a sort as
    a JS `Map` (lib/sort.js), so `Object.keys` on it is empty and the matcher
    reports every expected key as absent. Measured, not guessed: EJSON prints a
    `Map` exactly like a document, which is why the dump looks correct.
  - 4x `command.bypassDocumentValidation: missing` -- unclassified. The option
    is absent from the wire for the `false` cases; the driver only forwards it
    when true on some paths (lib/operations/find_and_modify.js:19), and whether
    the runner also drops it has not been established.
  - 2x `command.comment: missing` on getMore -- server gap, most likely. The
    driver gates it on `maxWireVersion >= 9` (lib/operations/get_more.js:43)
    and this engine advertises 8 while reporting itself as 4.4.0, which is
    wire 9. The inconsistency is ours.
  - 1x `command.maxTimeMS: expected 6000, got 10000` -- the CSOT rewrite, dealt
    with in the next commit.

Each of those gets its own commit, and none of them is fixed here: a check and
the fix for what the check caught do not belong in one change.
This commit is contained in:
A.Shakhmatov
2026-08-09 13:11:43 +03:00
parent 6560aec915
commit 97e3e3a556
2 changed files with 110 additions and 22 deletions
Download Patch File Download Diff File Expand all files Collapse all files

54
tests/spec/run.js
View File

@@ -606,6 +606,59 @@ async function buildEntities(url, createEntities, clients, events) {
return { map, clients, events };
}
// Two rules decide how much of the corpus this assertion can reach, and both
// come from the spec rather than from taste:
//
// - `command` and `reply` are matched as *root* documents
// (unified-test-format.md:1020-1022 and :1037-1039). The driver puts
// `lsid`, `$db` and `$readPreference` on nearly everything it sends;
// matched as nested documents, almost every case in the corpus would fail
// on keys its expectation was never written to mention.
// - the event list is exact in number and order, not a prefix
// (unified-test-format.md:3088-3091). 23 cases here expect an empty list,
// and a prefix rule would pass every one of them without looking.
function verifyEvents(expectEvents, entities) {
if (!expectEvents) return;
for (const spec of expectEvents) {
const type = spec.eventType || 'command';
if (type !== 'command') throw new Unsupported(`${type} events`);
if (spec.ignoreExtraEvents) throw new Unsupported('ignoreExtraEvents');
const buf = entities.events.get(spec.client);
// Not treated as an empty list: a client that never subscribed and a
// client that saw nothing are the same thing to a comparison and very
// different things to a runner, and the second one is a runner bug
// that would quietly pass the 23 empty-list cases.
if (!buf) fail(`events ${spec.client}`, 'the client entity is not observing command events');
const expected = spec.events || [];
if (expected.length !== buf.length) {
fail(`events ${spec.client}`, `expected ${expected.length} events, observed ${buf.length}` +
` [${buf.map((e) => `${e.kind}:${e.ev.commandName}`).join(', ')}]`);
}
expected.forEach((e, i) => matchEvent(e, buf[i], entities, `events ${spec.client}[${i}]`));
}
}
function matchEvent(expected, actual, entities, pathStr) {
const [name, body] = Object.entries(expected)[0];
if (!(name in COMMAND_EVENTS)) throw new Unsupported(`event ${name}`);
const kind = name.replace(/Event$/, '');
if (actual.kind !== kind) fail(pathStr, `expected ${kind}, got ${actual.kind} of ${actual.ev.commandName}`);
for (const [k, v] of Object.entries(body || {})) {
switch (k) {
case 'command':
case 'reply':
match(v, actual.ev[k], entities, `${pathStr}.${k}`, true);
break;
case 'commandName':
case 'databaseName':
match(v, actual.ev[k], entities, `${pathStr}.${k}`, false);
break;
default:
throw new Unsupported(`event assertion ${name}.${k}`);
}
}
}
async function verifyOutcome(outcome, entities) {
if (!outcome) return;
for (const spec of outcome) {
@@ -649,6 +702,7 @@ async function runFile(file, url, server) {
const entities = await buildEntities(url, doc.createEntities, clients, events);
for (const op of test.operations) await runOne(op, entities);
disableEvents(events);
verifyEvents(test.expectEvents, entities);
await verifyOutcome(test.outcome, entities);
}, CASE_TIMEOUT_MS, `case exceeded ${CASE_TIMEOUT_MS} ms`);
out.pass++;

78
tests/spec/scorecard.txt
View File

@@ -13,7 +13,7 @@
# semantics; ignoring them makes some cases pass that a full runner would
# fail, so treat `pass` as an upper bound until M1 wires events up.
total 193 pass 99 fail 195 skip 175 files 0 errored
total 159 pass 133 fail 195 skip 175 files 0 errored
# per-file: name pass fail skip
aggregate-allowdiskuse.json 3 0 0
@@ -25,40 +25,40 @@ aggregate-out-readConcern.json 0 0 4
aggregate-out.json 0 2 0
aggregate-rawdata.json 1 0 1
aggregate-write-readPreference.json 0 0 4
aggregate.json 5 0 2
aggregate.json 4 1 2
bulkWrite-arrayFilters.json 0 3 0
bulkWrite-collation.json 0 2 0
bulkWrite-comment.json 2 0 1
bulkWrite-delete-hint-serverError.json 0 0 2
bulkWrite-delete-hint.json 2 0 0
bulkWrite-deleteMany-hint-unacknowledged.json 2 0 2
bulkWrite-deleteMany-hint-unacknowledged.json 0 2 2
bulkWrite-deleteMany-let.json 0 1 1
bulkWrite-deleteMany-rawdata.json 1 0 1
bulkWrite-deleteOne-hint-unacknowledged.json 2 0 2
bulkWrite-deleteOne-hint-unacknowledged.json 0 2 2
bulkWrite-deleteOne-let.json 0 1 1
bulkWrite-deleteOne-rawdata.json 1 0 1
bulkWrite-errorResponse.json 0 0 1
bulkWrite-insertOne-dots_and_dollars.json 3 1 1
bulkWrite-replaceOne-dots_and_dollars.json 2 1 1
bulkWrite-replaceOne-hint-unacknowledged.json 2 0 0
bulkWrite-replaceOne-hint-unacknowledged.json 0 2 0
bulkWrite-replaceOne-let.json 0 1 1
bulkWrite-replaceOne-rawdata.json 1 0 1
bulkWrite-replaceOne-sort.json 1 0 1
bulkWrite-replaceOne-sort.json 0 1 1
bulkWrite-update-hint.json 3 0 0
bulkWrite-update-validation.json 3 0 0
bulkWrite-updateMany-dots_and_dollars.json 0 0 4
bulkWrite-updateMany-hint-unacknowledged.json 2 0 0
bulkWrite-updateMany-hint-unacknowledged.json 0 2 0
bulkWrite-updateMany-let.json 0 1 1
bulkWrite-updateMany-pipeline.json 0 1 0
bulkWrite-updateMany-rawdata.json 0 1 1
bulkWrite-updateOne-dots_and_dollars.json 0 0 4
bulkWrite-updateOne-hint-unacknowledged.json 2 0 0
bulkWrite-updateOne-hint-unacknowledged.json 0 2 0
bulkWrite-updateOne-let.json 0 1 1
bulkWrite-updateOne-pipeline.json 0 1 0
bulkWrite-updateOne-rawdata.json 0 1 1
bulkWrite-updateOne-sort.json 1 0 1
bulkWrite-updateOne-sort.json 0 1 1
bulkWrite.json 10 0 0
bypassDocumentValidation.json 8 1 0
bypassDocumentValidation.json 4 5 0
client-bulkWrite-delete-options.json 0 0 2
client-bulkWrite-delete-rawdata.json 0 0 2
client-bulkWrite-errorResponse.json 0 0 1
@@ -88,7 +88,7 @@ db-aggregate.json 0 2 0
deleteMany-collation.json 0 1 0
deleteMany-comment.json 2 0 1
deleteMany-hint-serverError.json 0 0 2
deleteMany-hint-unacknowledged.json 2 0 2
deleteMany-hint-unacknowledged.json 0 2 2
deleteMany-hint.json 2 0 0
deleteMany-let.json 0 1 1
deleteMany-rawdata.json 1 0 1
@@ -97,7 +97,7 @@ deleteOne-collation.json 0 1 0
deleteOne-comment.json 2 0 1
deleteOne-errorResponse.json 0 0 1
deleteOne-hint-serverError.json 0 0 2
deleteOne-hint-unacknowledged.json 2 0 2
deleteOne-hint-unacknowledged.json 0 2 2
deleteOne-hint.json 2 0 0
deleteOne-let.json 0 1 1
deleteOne-rawdata.json 1 0 1
@@ -109,15 +109,15 @@ distinct-rawdata.json 0 1 1
distinct.json 0 2 0
estimatedDocumentCount-comment.json 1 1 1
estimatedDocumentCount-rawdata.json 1 0 1
estimatedDocumentCount.json 3 1 2
estimatedDocumentCount.json 2 2 2
find-allowdiskuse-serverError.json 0 0 2
find-allowdiskuse.json 3 0 0
find-collation.json 0 1 0
find-comment.json 1 2 2
find-comment.json 0 3 2
find-let.json 0 1 1
find-rawdata.json 1 0 1
find.json 5 0 0
findOne.json 2 0 0
findOne.json 1 1 0
findOneAndDelete-collation.json 0 1 0
findOneAndDelete-comment.json 2 0 1
findOneAndDelete-hint-serverError.json 0 0 2
@@ -153,25 +153,25 @@ insertMany-dots_and_dollars.json 3 1 1
insertMany-rawdata.json 1 0 1
insertMany.json 3 0 0
insertOne-comment.json 2 0 1
insertOne-dots_and_dollars.json 6 2 1
insertOne-dots_and_dollars.json 5 3 1
insertOne-errorResponse.json 0 0 1
insertOne-rawdata.json 1 0 1
insertOne.json 1 0 0
replaceOne-collation.json 0 1 0
replaceOne-comment.json 2 0 1
replaceOne-dots_and_dollars.json 3 1 1
replaceOne-hint-unacknowledged.json 2 0 0
replaceOne-dots_and_dollars.json 2 2 1
replaceOne-hint-unacknowledged.json 0 2 0
replaceOne-hint.json 2 0 0
replaceOne-let.json 0 1 1
replaceOne-rawdata.json 1 0 1
replaceOne-sort.json 1 0 1
replaceOne-sort.json 0 1 1
replaceOne-validation.json 1 0 0
replaceOne.json 5 0 0
updateMany-arrayFilters.json 0 3 0
updateMany-collation.json 0 1 0
updateMany-comment.json 2 0 1
updateMany-dots_and_dollars.json 0 0 4
updateMany-hint-unacknowledged.json 2 0 0
updateMany-hint-unacknowledged.json 0 2 0
updateMany-hint.json 2 0 0
updateMany-let.json 0 1 1
updateMany-pipeline.json 0 1 0
@@ -183,12 +183,12 @@ updateOne-collation.json 0 1 0
updateOne-comment.json 2 0 1
updateOne-dots_and_dollars.json 0 0 4
updateOne-errorResponse.json 0 0 1
updateOne-hint-unacknowledged.json 2 0 0
updateOne-hint-unacknowledged.json 0 2 0
updateOne-hint.json 2 0 0
updateOne-let.json 0 1 1
updateOne-pipeline.json 0 1 0
updateOne-rawdata.json 1 0 1
updateOne-sort.json 1 0 1
updateOne-sort.json 0 1 1
updateOne-validation.json 1 0 0
updateOne.json 4 0 0
@@ -210,6 +210,7 @@ aggregate-out.json FAIL Aggregate with $out and batch size of 0 MongoServerError
aggregate-rawdata.json SKIP Aggregate with rawData option needs server >= 8.2.0
aggregate-write-readPreference.json SKIP * needs topology replicaset/sharded/load-balanced
aggregate.json SKIP aggregate with a document comment - pre 4.4 needs server <= 4.2.99
aggregate.json FAIL aggregate with comment sets comment on getMore events client0[1].command.comment: missing from actual
aggregate.json SKIP aggregate with comment does not set comment on getMore - pre 4.4 needs server <= 4.3.99
bulkWrite-arrayFilters.json FAIL BulkWrite updateOne with arrayFilters outcome crud-tests.test[0].y: expected an array, got {"$[i]":{"b":2}}
bulkWrite-arrayFilters.json FAIL BulkWrite updateMany with arrayFilters outcome crud-tests.test[0].y: expected an array, got {"$[i]":{"b":2}}
@@ -220,11 +221,15 @@ bulkWrite-comment.json SKIP BulkWrite with comment - pre 4.4 needs server <= 4.2
bulkWrite-delete-hint-serverError.json SKIP * needs server <= 4.3.3
bulkWrite-deleteMany-hint-unacknowledged.json SKIP Unacknowledged deleteMany with hint string fails with client-side error on pre-4.4 server needs server <= 4.2.99
bulkWrite-deleteMany-hint-unacknowledged.json SKIP Unacknowledged deleteMany with hint document fails with client-side error on pre-4.4 server needs server <= 4.2.99
bulkWrite-deleteMany-hint-unacknowledged.json FAIL Unacknowledged deleteMany with hint string on 4.4+ server events client0[0].command.writeConcern: missing from actual
bulkWrite-deleteMany-hint-unacknowledged.json FAIL Unacknowledged deleteMany with hint document on 4.4+ server events client0[0].command.writeConcern: missing from actual
bulkWrite-deleteMany-let.json SKIP BulkWrite deleteMany with let option needs server >= 5.0
bulkWrite-deleteMany-let.json FAIL BulkWrite deleteMany with let option unsupported (server-side error) bulkWrite: expected an error, the operation succeeded
bulkWrite-deleteMany-rawdata.json SKIP BulkWrite deleteMany with rawData option needs server >= 8.2.0
bulkWrite-deleteOne-hint-unacknowledged.json SKIP Unacknowledged deleteOne with hint string fails with client-side error on pre-4.4 server needs server <= 4.2.99
bulkWrite-deleteOne-hint-unacknowledged.json SKIP Unacknowledged deleteOne with hint document fails with client-side error on pre-4.4 server needs server <= 4.2.99
bulkWrite-deleteOne-hint-unacknowledged.json FAIL Unacknowledged deleteOne with hint string on 4.4+ server events client0[0].command.writeConcern: missing from actual
bulkWrite-deleteOne-hint-unacknowledged.json FAIL Unacknowledged deleteOne with hint document on 4.4+ server events client0[0].command.writeConcern: missing from actual
bulkWrite-deleteOne-let.json SKIP BulkWrite deleteOne with let option needs server >= 5.0
bulkWrite-deleteOne-let.json FAIL BulkWrite deleteOne with let option unsupported (server-side error) bulkWrite: expected an error, the operation succeeded
bulkWrite-deleteOne-rawdata.json SKIP BulkWrite deleteOne with rawData option needs server >= 8.2.0
@@ -233,14 +238,19 @@ bulkWrite-insertOne-dots_and_dollars.json SKIP Inserting document with top-level
bulkWrite-insertOne-dots_and_dollars.json FAIL Inserting document with top-level dollar-prefixed key on pre-5.0 server yields server-side error bulkWrite: expected an error, the operation succeeded
bulkWrite-replaceOne-dots_and_dollars.json SKIP Replacing document with dollar-prefixed key in embedded doc on 5.0+ server needs server >= 5.0
bulkWrite-replaceOne-dots_and_dollars.json FAIL Replacing document with dollar-prefixed key in embedded doc on pre-5.0 server yields server-side error bulkWrite: expected an error, the operation succeeded
bulkWrite-replaceOne-hint-unacknowledged.json FAIL Unacknowledged replaceOne with hint string on 4.2+ server events client0[0].command.writeConcern: missing from actual
bulkWrite-replaceOne-hint-unacknowledged.json FAIL Unacknowledged replaceOne with hint document on 4.2+ server events client0[0].command.writeConcern: missing from actual
bulkWrite-replaceOne-let.json SKIP BulkWrite replaceOne with let option needs server >= 5.0
bulkWrite-replaceOne-let.json FAIL BulkWrite replaceOne with let option unsupported (server-side error) bulkWrite: expected an error, the operation succeeded
bulkWrite-replaceOne-rawdata.json SKIP BulkWrite replaceOne with rawData option needs server >= 8.2.0
bulkWrite-replaceOne-sort.json SKIP BulkWrite replaceOne with sort option needs server >= 8.0
bulkWrite-replaceOne-sort.json FAIL BulkWrite replaceOne with sort option unsupported (server-side error) events client0[0].command.updates[0].sort._id: missing from actual
bulkWrite-updateMany-dots_and_dollars.json SKIP Updating document to set top-level dollar-prefixed key on 5.0+ server needs server >= 5.0
bulkWrite-updateMany-dots_and_dollars.json SKIP Updating document to set top-level dotted key on 5.0+ server needs server >= 5.0
bulkWrite-updateMany-dots_and_dollars.json SKIP Updating document to set dollar-prefixed key in embedded doc on 5.0+ server needs server >= 5.0
bulkWrite-updateMany-dots_and_dollars.json SKIP Updating document to set dotted key in embedded doc on 5.0+ server needs server >= 5.0
bulkWrite-updateMany-hint-unacknowledged.json FAIL Unacknowledged updateMany with hint string on 4.2+ server events client0[0].command.writeConcern: missing from actual
bulkWrite-updateMany-hint-unacknowledged.json FAIL Unacknowledged updateMany with hint document on 4.2+ server events client0[0].command.writeConcern: missing from actual
bulkWrite-updateMany-let.json SKIP BulkWrite updateMany with let option needs server >= 5.0
bulkWrite-updateMany-let.json FAIL BulkWrite updateMany with let option unsupported (server-side error) bulkWrite: error message "update spec requires u" does not contain "'update.let' is an unknown field"
bulkWrite-updateMany-pipeline.json FAIL UpdateMany in bulk write using pipelines MongoBulkWriteError: update spec requires u
@@ -250,13 +260,20 @@ bulkWrite-updateOne-dots_and_dollars.json SKIP Updating document to set top-leve
bulkWrite-updateOne-dots_and_dollars.json SKIP Updating document to set top-level dotted key on 5.0+ server needs server >= 5.0
bulkWrite-updateOne-dots_and_dollars.json SKIP Updating document to set dollar-prefixed key in embedded doc on 5.0+ server needs server >= 5.0
bulkWrite-updateOne-dots_and_dollars.json SKIP Updating document to set dotted key in embedded doc on 5.0+ server needs server >= 5.0
bulkWrite-updateOne-hint-unacknowledged.json FAIL Unacknowledged updateOne with hint string on 4.2+ server events client0[0].command.writeConcern: missing from actual
bulkWrite-updateOne-hint-unacknowledged.json FAIL Unacknowledged updateOne with hint document on 4.2+ server events client0[0].command.writeConcern: missing from actual
bulkWrite-updateOne-let.json SKIP BulkWrite updateOne with let option needs server >= 5.0
bulkWrite-updateOne-let.json FAIL BulkWrite updateOne with let option unsupported (server-side error) bulkWrite: error message "update spec requires u" does not contain "'update.let' is an unknown field"
bulkWrite-updateOne-pipeline.json FAIL UpdateOne in bulk write using pipelines MongoBulkWriteError: update spec requires u
bulkWrite-updateOne-rawdata.json SKIP BulkWrite updateOne with rawData option needs server >= 8.2.0
bulkWrite-updateOne-rawdata.json FAIL BulkWrite updateOne with rawData option on less than 8.2.0 - ignore argument MongoBulkWriteError: update spec requires u
bulkWrite-updateOne-sort.json SKIP BulkWrite updateOne with sort option needs server >= 8.0
bulkWrite-updateOne-sort.json FAIL BulkWrite updateOne with sort option unsupported (server-side error) events client0[0].command.updates[0].sort._id: missing from actual
bypassDocumentValidation.json FAIL Aggregate with $out passes bypassDocumentValidation: false MongoServerError: Unrecognized pipeline stage name: '$out'
bypassDocumentValidation.json FAIL BulkWrite passes bypassDocumentValidation: false events client0[0].command.bypassDocumentValidation: missing from actual
bypassDocumentValidation.json FAIL FindOneAndReplace passes bypassDocumentValidation: false events client0[0].command.bypassDocumentValidation: missing from actual
bypassDocumentValidation.json FAIL FindOneAndUpdate passes bypassDocumentValidation: false events client0[0].command.bypassDocumentValidation: missing from actual
bypassDocumentValidation.json FAIL InsertMany passes bypassDocumentValidation: false events client0[0].command.bypassDocumentValidation: missing from actual
client-bulkWrite-delete-options.json SKIP * needs server >= 8.0
client-bulkWrite-delete-rawdata.json SKIP client bulk write delete with rawData option needs server >= 8.2.0
client-bulkWrite-delete-rawdata.json SKIP client bulk write delete with rawData option on less than 8.2.0 - ignore argument needs server >= 8.0
@@ -304,6 +321,8 @@ deleteMany-comment.json SKIP deleteMany with comment - pre 4.4 needs server <= 4
deleteMany-hint-serverError.json SKIP * needs server <= 4.3.3
deleteMany-hint-unacknowledged.json SKIP Unacknowledged deleteMany with hint string fails with client-side error on pre-4.4 server needs server <= 4.2.99
deleteMany-hint-unacknowledged.json SKIP Unacknowledged deleteMany with hint document fails with client-side error on pre-4.4 server needs server <= 4.2.99
deleteMany-hint-unacknowledged.json FAIL Unacknowledged deleteMany with hint string on 4.4+ server events client0[0].command.writeConcern: missing from actual
deleteMany-hint-unacknowledged.json FAIL Unacknowledged deleteMany with hint document on 4.4+ server events client0[0].command.writeConcern: missing from actual
deleteMany-let.json SKIP deleteMany with let option needs server >= 5.0
deleteMany-let.json FAIL deleteMany with let option unsupported (server-side error) deleteMany: expected an error, the operation succeeded
deleteMany-rawdata.json SKIP deleteMany with rawData option needs server >= 8.2.0
@@ -313,6 +332,8 @@ deleteOne-errorResponse.json SKIP delete operations support errorResponse assert
deleteOne-hint-serverError.json SKIP * needs server <= 4.3.3
deleteOne-hint-unacknowledged.json SKIP Unacknowledged deleteOne with hint string fails with client-side error on pre-4.4 server needs server <= 4.2.99
deleteOne-hint-unacknowledged.json SKIP Unacknowledged deleteOne with hint document fails with client-side error on pre-4.4 server needs server <= 4.2.99
deleteOne-hint-unacknowledged.json FAIL Unacknowledged deleteOne with hint string on 4.4+ server events client0[0].command.writeConcern: missing from actual
deleteOne-hint-unacknowledged.json FAIL Unacknowledged deleteOne with hint document on 4.4+ server events client0[0].command.writeConcern: missing from actual
deleteOne-let.json SKIP deleteOne with let option needs server >= 5.0
deleteOne-let.json FAIL deleteOne with let option unsupported (server-side error) deleteOne: expected an error, the operation succeeded
deleteOne-rawdata.json SKIP deleteOne with rawData option needs server >= 8.2.0
@@ -327,6 +348,7 @@ distinct.json FAIL Distinct with a filter MongoServerError: no such command: 'di
estimatedDocumentCount-comment.json SKIP estimatedDocumentCount with document comment needs server >= 4.4.14
estimatedDocumentCount-comment.json FAIL estimatedDocumentCount with document comment - pre 4.4.14, server error estimatedDocumentCount: expected an error, the operation succeeded
estimatedDocumentCount-rawdata.json SKIP Estimated document count with rawData option needs server >= 8.2.0
estimatedDocumentCount.json FAIL estimatedDocumentCount with maxTimeMS events client0[0].command.maxTimeMS: expected 6000, got 10000
estimatedDocumentCount.json SKIP estimatedDocumentCount errors correctly--command error runner: failPoint
estimatedDocumentCount.json SKIP estimatedDocumentCount errors correctly--socket error runner: failPoint
estimatedDocumentCount.json FAIL estimatedDocumentCount works correctly on views estimatedDocumentCount: expected 2, got 0
@@ -335,10 +357,12 @@ find-collation.json FAIL Find with a collation find: expected 1 elements, got 0
find-comment.json FAIL find with string comment find[0]: unexpected extra keys ["x"]
find-comment.json FAIL find with document comment find[0]: unexpected extra keys ["x"]
find-comment.json SKIP find with document comment - pre 4.4 needs server <= 4.2.99
find-comment.json FAIL find with comment sets comment on getMore events client0[1].command.comment: missing from actual
find-comment.json SKIP find with comment does not set comment on getMore - pre 4.4 needs server <= 4.3.99
find-let.json SKIP Find with let option needs server >= 5.0
find-let.json FAIL Find with let option unsupported (server-side error) find: expected an error, the operation succeeded
find-rawdata.json SKIP Find with rawData option needs server >= 8.2.0
findOne.json FAIL FindOne with filter, sort, and skip events client0[0].command.sort._id: missing from actual
findOneAndDelete-collation.json FAIL FindOneAndDelete when one document matches with collation findOneAndDelete: expected a document, got null
findOneAndDelete-comment.json SKIP findOneAndDelete with comment - pre 4.4 needs server <= 4.2.99
findOneAndDelete-hint-serverError.json SKIP * needs server <= 4.3.3
@@ -399,16 +423,21 @@ insertOne-comment.json SKIP insertOne with comment - pre 4.4 needs server <= 4.2
insertOne-dots_and_dollars.json SKIP Inserting document with top-level dollar-prefixed key on 5.0+ server needs server >= 5.0
insertOne-dots_and_dollars.json FAIL Inserting document with top-level dollar-prefixed key on pre-5.0 server yields server-side error insertOne: expected an error, the operation succeeded
insertOne-dots_and_dollars.json FAIL Inserting document with dollar-prefixed key in _id yields server-side error insertOne: expected an error, the operation succeeded
insertOne-dots_and_dollars.json FAIL Unacknowledged write using dollar-prefixed or dotted keys may be silently rejected on pre-5.0 server events client0[0].command.writeConcern: missing from actual
insertOne-errorResponse.json SKIP insert operations support errorResponse assertions runner: failPoint
insertOne-rawdata.json SKIP insertOne with rawData option needs server >= 8.2.0
replaceOne-collation.json FAIL ReplaceOne when one document matches with collation replaceOne.matchedCount: expected 1, got 0
replaceOne-comment.json SKIP ReplaceOne with comment - pre 4.4 needs server <= 4.2.99
replaceOne-dots_and_dollars.json SKIP Replacing document with dollar-prefixed key in embedded doc on 5.0+ server needs server >= 5.0
replaceOne-dots_and_dollars.json FAIL Replacing document with dollar-prefixed key in embedded doc on pre-5.0 server yields server-side error replaceOne: expected an error, the operation succeeded
replaceOne-dots_and_dollars.json FAIL Unacknowledged write using dollar-prefixed or dotted keys may be silently rejected on pre-5.0 server events client0[0].command.writeConcern: missing from actual
replaceOne-hint-unacknowledged.json FAIL Unacknowledged replaceOne with hint string on 4.2+ server events client0[0].command.writeConcern: missing from actual
replaceOne-hint-unacknowledged.json FAIL Unacknowledged replaceOne with hint document on 4.2+ server events client0[0].command.writeConcern: missing from actual
replaceOne-let.json SKIP ReplaceOne with let option needs server >= 5.0
replaceOne-let.json FAIL ReplaceOne with let option unsupported (server-side error) replaceOne: expected an error, the operation succeeded
replaceOne-rawdata.json SKIP ReplaceOne with rawData option needs server >= 8.2.0
replaceOne-sort.json SKIP ReplaceOne with sort option needs server >= 8.0
replaceOne-sort.json FAIL replaceOne with sort option unsupported (server-side error) events client0[0].command.updates[0].sort._id: missing from actual
updateMany-arrayFilters.json FAIL UpdateMany when no documents match arrayFilters updateMany.modifiedCount: expected 0, got 2
updateMany-arrayFilters.json FAIL UpdateMany when one document matches arrayFilters updateMany.modifiedCount: expected 1, got 2
updateMany-arrayFilters.json FAIL UpdateMany when multiple documents match arrayFilters outcome crud-v1.coll[0].y: expected an array, got {"$[i]":{"b":2}}
@@ -418,6 +447,8 @@ updateMany-dots_and_dollars.json SKIP Updating document to set top-level dollar-
updateMany-dots_and_dollars.json SKIP Updating document to set top-level dotted key on 5.0+ server needs server >= 5.0
updateMany-dots_and_dollars.json SKIP Updating document to set dollar-prefixed key in embedded doc on 5.0+ server needs server >= 5.0
updateMany-dots_and_dollars.json SKIP Updating document to set dotted key in embedded doc on 5.0+ server needs server >= 5.0
updateMany-hint-unacknowledged.json FAIL Unacknowledged updateMany with hint string on 4.2+ server events client0[0].command.writeConcern: missing from actual
updateMany-hint-unacknowledged.json FAIL Unacknowledged updateMany with hint document on 4.2+ server events client0[0].command.writeConcern: missing from actual
updateMany-let.json SKIP updateMany with let option needs server >= 5.0
updateMany-let.json FAIL updateMany with let option unsupported (server-side error) updateMany: error message "update spec requires u" does not contain "'update.let' is an unknown field"
updateMany-pipeline.json FAIL UpdateMany using pipelines MongoServerError: update spec requires u
@@ -434,8 +465,11 @@ updateOne-dots_and_dollars.json SKIP Updating document to set top-level dotted k
updateOne-dots_and_dollars.json SKIP Updating document to set dollar-prefixed key in embedded doc on 5.0+ server needs server >= 5.0
updateOne-dots_and_dollars.json SKIP Updating document to set dotted key in embedded doc on 5.0+ server needs server >= 5.0
updateOne-errorResponse.json SKIP update operations support errorResponse assertions runner: failPoint
updateOne-hint-unacknowledged.json FAIL Unacknowledged updateOne with hint string on 4.2+ server events client0[0].command.writeConcern: missing from actual
updateOne-hint-unacknowledged.json FAIL Unacknowledged updateOne with hint document on 4.2+ server events client0[0].command.writeConcern: missing from actual
updateOne-let.json SKIP UpdateOne with let option needs server >= 5.0
updateOne-let.json FAIL UpdateOne with let option unsupported (server-side error) updateOne: error message "update spec requires u" does not contain "'update.let' is an unknown field"
updateOne-pipeline.json FAIL UpdateOne using pipelines MongoServerError: update spec requires u
updateOne-rawdata.json SKIP UpdateOne with rawData option needs server >= 8.2.0
updateOne-sort.json SKIP UpdateOne with sort option needs server >= 8.0
updateOne-sort.json FAIL updateOne with sort option unsupported (server-side error) events client0[0].command.updates[0].sort._id: missing from actual