1f141ef619
A whole command that did not exist: five corpus cases answered "no such
command". Two things about it were measured against mongod 8.3.7 rather
than recalled, and the first is not what anyone would guess.
- The answer is **sorted in canonical BSON order**, not in the order the
values were met. `{s: "b"}, {s: "a"}, {s: null}` answers
`[null, "a", "b"]`. Insertion order is the obvious implementation, it
passes every test anybody would think to write by hand against
`[11, 22, 33]`, and it is wrong.
- Deduping is the same comparator, so an int32 `1` and a double `1.0`
collapse while `null` and `"1"` survive.
Both fall out of `bson.compare`, which `$sort` and `$min` already use --
and that is not luck: mongod accumulates into a `BSONElementSet` ordered by
the same `woCompare`. The rest reuses the shared read path: byte-walked
`collect_values_bytes` for the key, so the traversal, the multikey descent
and the numeric path segments are the ones the matcher and the index
already agree on.
Also measured: a terminal array contributes its elements exactly one level
deep (`[[7, 8], 9]` gives `[7, 8]` and `9`, never 7 and 8); a missing field
contributes nothing where an explicit null contributes null; an absent
collection, an absent database and an empty key are each `ok: 1` with an
empty array rather than an error; `query` absent and `query: null` are both
an empty filter; a missing `key` is IDLFailedToParse (40414) while a
wrong-typed one is TypeMismatch (14).
Running the identical probe against both servers now agrees on every
semantic row. Three divergences remain, all outside this command and
recorded in PLAN §6: an unknown query operator matches nothing instead of
erroring (shared with find/count/aggregate, and the same class as M2's six
silent wrong answers), a non-string collection name is refused by dispatch
as BadValue where mongod says InvalidNamespace, and an unknown top-level
field is tolerated -- deliberately, since `comment` and `rawData` arrive
through that door and the corpus requires both be ignored.
crud scorecard: 201 pass / 90 fail -> 204 / 87. distinct.json 0/2 -> 2/0,
distinct-rawdata 0/1 -> 1/0. distinct-comment nets zero: its "no such
command" is replaced by the pre-4.4.14 document-comment case, which
`estimatedDocumentCount` already carries as a standing failure -- and
emulating a bug fixed in 4.4.14 for one command would make the two
disagree. distinct-collation still needs M8, but now fails with the honest
"expected 1 elements, got 2".
198/198 unit (7 new) in ReleaseFast and ReleaseSafe, 83/83 fuzz, aggregation
corpus 70/0, full e2e matrix and crash-fuzz green.
436 lines
34 KiB
Plaintext
436 lines
34 KiB
Plaintext
# MongoDB spec-test scorecard (PLAN D2) -- crud + aggregate, unified format
|
|
# specs: mongodb/specifications @ 615e0f9 (pinned in tests/spec/fetch.sh)
|
|
# driver: mongodb@7.5.0 (pinned in tests/e2e/package-lock.json)
|
|
# server: MultiforaDB reporting version 4.4.0, maxWireVersion 9
|
|
# reproduce: bash tests/spec/fetch.sh && node tests/spec/run.js --scorecard
|
|
#
|
|
# What SKIP means here, so the totals are not read as better than they are:
|
|
# - the suite needs a feature whose milestone has not landed (sessions M4,
|
|
# auth M7, failPoints, gridfs) -- counted as skip, never as pass;
|
|
# - or the runner itself does not implement the operation/matcher yet.
|
|
# expectEvents IS asserted: the commands the driver sent are compared to the
|
|
# expectation exactly, in number and in order, with `command` and `reply`
|
|
# matched as root documents. A pass therefore means the engine answered
|
|
# correctly *and* was asked the right question. Not comparable to any
|
|
# scorecard recorded before that landed, where `pass` was an upper bound.
|
|
# Still unasserted, each reported as skip where it is asserted, never as
|
|
# pass: cmap and sdam events, ignoreExtraEvents, hasServiceId,
|
|
# hasServerConnectionId, and maxTimeMS in an expected command (CSOT rewrites
|
|
# it -- the only assertion this runner declines to make).
|
|
|
|
total 204 pass 87 fail 196 skip 175 files 0 errored
|
|
|
|
# per-file: name pass fail skip
|
|
aggregate-allowdiskuse.json 3 0 0
|
|
aggregate-collation.json 0 1 0
|
|
aggregate-let.json 0 2 2
|
|
aggregate-merge-errorResponse.json 0 0 1
|
|
aggregate-merge.json 5 0 0
|
|
aggregate-out-readConcern.json 0 0 4
|
|
aggregate-out.json 2 0 0
|
|
aggregate-rawdata.json 1 0 1
|
|
aggregate-write-readPreference.json 0 0 4
|
|
aggregate.json 5 0 2
|
|
bulkWrite-arrayFilters.json 0 3 0
|
|
bulkWrite-collation.json 0 2 0
|
|
bulkWrite-comment.json 2 0 1
|
|
bulkWrite-delete-hint-serverError.json 0 0 2
|
|
bulkWrite-delete-hint.json 2 0 0
|
|
bulkWrite-deleteMany-hint-unacknowledged.json 2 0 2
|
|
bulkWrite-deleteMany-let.json 0 1 1
|
|
bulkWrite-deleteMany-rawdata.json 1 0 1
|
|
bulkWrite-deleteOne-hint-unacknowledged.json 2 0 2
|
|
bulkWrite-deleteOne-let.json 0 1 1
|
|
bulkWrite-deleteOne-rawdata.json 1 0 1
|
|
bulkWrite-errorResponse.json 0 0 1
|
|
bulkWrite-insertOne-dots_and_dollars.json 3 1 1
|
|
bulkWrite-replaceOne-dots_and_dollars.json 2 1 1
|
|
bulkWrite-replaceOne-hint-unacknowledged.json 2 0 0
|
|
bulkWrite-replaceOne-let.json 0 1 1
|
|
bulkWrite-replaceOne-rawdata.json 1 0 1
|
|
bulkWrite-replaceOne-sort.json 1 0 1
|
|
bulkWrite-update-hint.json 3 0 0
|
|
bulkWrite-update-validation.json 3 0 0
|
|
bulkWrite-updateMany-dots_and_dollars.json 0 0 4
|
|
bulkWrite-updateMany-hint-unacknowledged.json 2 0 0
|
|
bulkWrite-updateMany-let.json 0 1 1
|
|
bulkWrite-updateMany-pipeline.json 0 1 0
|
|
bulkWrite-updateMany-rawdata.json 0 1 1
|
|
bulkWrite-updateOne-dots_and_dollars.json 0 0 4
|
|
bulkWrite-updateOne-hint-unacknowledged.json 2 0 0
|
|
bulkWrite-updateOne-let.json 0 1 1
|
|
bulkWrite-updateOne-pipeline.json 0 1 0
|
|
bulkWrite-updateOne-rawdata.json 0 1 1
|
|
bulkWrite-updateOne-sort.json 1 0 1
|
|
bulkWrite.json 10 0 0
|
|
bypassDocumentValidation.json 4 5 0
|
|
client-bulkWrite-delete-options.json 0 0 2
|
|
client-bulkWrite-delete-rawdata.json 0 0 2
|
|
client-bulkWrite-errorResponse.json 0 0 1
|
|
client-bulkWrite-errors.json 0 0 9
|
|
client-bulkWrite-mixed-namespaces.json 0 0 1
|
|
client-bulkWrite-options.json 0 0 7
|
|
client-bulkWrite-ordered.json 0 0 3
|
|
client-bulkWrite-partialResults.json 0 0 9
|
|
client-bulkWrite-replaceOne-rawdata.json 0 0 2
|
|
client-bulkWrite-replaceOne-sort.json 0 0 1
|
|
client-bulkWrite-results.json 0 0 3
|
|
client-bulkWrite-update-options.json 0 0 4
|
|
client-bulkWrite-update-pipeline.json 0 0 2
|
|
client-bulkWrite-update-rawdata.json 0 0 2
|
|
client-bulkWrite-update-validation.json 0 0 3
|
|
client-bulkWrite-updateOne-sort.json 0 0 1
|
|
count-collation.json 1 0 1
|
|
count-empty.json 2 0 1
|
|
count-rawdata.json 0 0 2
|
|
count.json 4 0 3
|
|
countDocuments-comment.json 2 0 1
|
|
countDocuments-rawdata.json 1 0 1
|
|
create-null-ids.json 0 6 1
|
|
db-aggregate-rawdata.json 0 1 1
|
|
db-aggregate-write-readPreference.json 0 0 4
|
|
db-aggregate.json 0 2 0
|
|
deleteMany-collation.json 0 1 0
|
|
deleteMany-comment.json 2 0 1
|
|
deleteMany-hint-serverError.json 0 0 2
|
|
deleteMany-hint-unacknowledged.json 2 0 2
|
|
deleteMany-hint.json 2 0 0
|
|
deleteMany-let.json 0 1 1
|
|
deleteMany-rawdata.json 1 0 1
|
|
deleteMany.json 2 0 0
|
|
deleteOne-collation.json 0 1 0
|
|
deleteOne-comment.json 2 0 1
|
|
deleteOne-errorResponse.json 0 0 1
|
|
deleteOne-hint-serverError.json 0 0 2
|
|
deleteOne-hint-unacknowledged.json 2 0 2
|
|
deleteOne-hint.json 2 0 0
|
|
deleteOne-let.json 0 1 1
|
|
deleteOne-rawdata.json 1 0 1
|
|
deleteOne.json 3 0 0
|
|
distinct-collation.json 0 1 0
|
|
distinct-comment.json 1 1 1
|
|
distinct-hint.json 0 0 2
|
|
distinct-rawdata.json 1 0 1
|
|
distinct.json 2 0 0
|
|
estimatedDocumentCount-comment.json 1 1 1
|
|
estimatedDocumentCount-rawdata.json 1 0 1
|
|
estimatedDocumentCount.json 2 1 3
|
|
find-allowdiskuse-serverError.json 0 0 2
|
|
find-allowdiskuse.json 3 0 0
|
|
find-collation.json 0 1 0
|
|
find-comment.json 1 2 2
|
|
find-let.json 0 1 1
|
|
find-rawdata.json 1 0 1
|
|
find.json 5 0 0
|
|
findOne.json 2 0 0
|
|
findOneAndDelete-collation.json 0 1 0
|
|
findOneAndDelete-comment.json 2 0 1
|
|
findOneAndDelete-hint-serverError.json 0 0 2
|
|
findOneAndDelete-hint-unacknowledged.json 2 0 2
|
|
findOneAndDelete-hint.json 2 0 0
|
|
findOneAndDelete-let.json 0 1 1
|
|
findOneAndDelete-rawdata.json 1 0 1
|
|
findOneAndDelete.json 3 0 0
|
|
findOneAndReplace-collation.json 0 1 0
|
|
findOneAndReplace-comment.json 2 0 1
|
|
findOneAndReplace-dots_and_dollars.json 2 1 1
|
|
findOneAndReplace-hint-serverError.json 0 0 2
|
|
findOneAndReplace-hint-unacknowledged.json 2 0 2
|
|
findOneAndReplace-hint.json 2 0 0
|
|
findOneAndReplace-let.json 0 1 1
|
|
findOneAndReplace-rawdata.json 1 0 1
|
|
findOneAndReplace-upsert.json 2 2 0
|
|
findOneAndReplace.json 4 2 0
|
|
findOneAndUpdate-arrayFilters.json 0 3 0
|
|
findOneAndUpdate-collation.json 0 1 0
|
|
findOneAndUpdate-comment.json 0 2 1
|
|
findOneAndUpdate-dots_and_dollars.json 0 0 4
|
|
findOneAndUpdate-errorResponse.json 0 1 1
|
|
findOneAndUpdate-hint-serverError.json 0 0 2
|
|
findOneAndUpdate-hint-unacknowledged.json 2 0 2
|
|
findOneAndUpdate-hint.json 2 0 0
|
|
findOneAndUpdate-let.json 0 1 1
|
|
findOneAndUpdate-pipeline.json 0 1 0
|
|
findOneAndUpdate-rawdata.json 0 1 1
|
|
findOneAndUpdate.json 5 3 0
|
|
insertMany-comment.json 2 0 1
|
|
insertMany-dots_and_dollars.json 3 1 1
|
|
insertMany-rawdata.json 1 0 1
|
|
insertMany.json 3 0 0
|
|
insertOne-comment.json 2 0 1
|
|
insertOne-dots_and_dollars.json 6 2 1
|
|
insertOne-errorResponse.json 0 0 1
|
|
insertOne-rawdata.json 1 0 1
|
|
insertOne.json 1 0 0
|
|
replaceOne-collation.json 0 1 0
|
|
replaceOne-comment.json 2 0 1
|
|
replaceOne-dots_and_dollars.json 3 1 1
|
|
replaceOne-hint-unacknowledged.json 2 0 0
|
|
replaceOne-hint.json 2 0 0
|
|
replaceOne-let.json 0 1 1
|
|
replaceOne-rawdata.json 1 0 1
|
|
replaceOne-sort.json 1 0 1
|
|
replaceOne-validation.json 1 0 0
|
|
replaceOne.json 5 0 0
|
|
updateMany-arrayFilters.json 0 3 0
|
|
updateMany-collation.json 0 1 0
|
|
updateMany-comment.json 2 0 1
|
|
updateMany-dots_and_dollars.json 0 0 4
|
|
updateMany-hint-unacknowledged.json 2 0 0
|
|
updateMany-hint.json 2 0 0
|
|
updateMany-let.json 0 1 1
|
|
updateMany-pipeline.json 0 1 0
|
|
updateMany-rawdata.json 1 0 1
|
|
updateMany-validation.json 1 0 0
|
|
updateMany.json 4 0 0
|
|
updateOne-arrayFilters.json 0 5 0
|
|
updateOne-collation.json 0 1 0
|
|
updateOne-comment.json 2 0 1
|
|
updateOne-dots_and_dollars.json 0 0 4
|
|
updateOne-errorResponse.json 0 0 1
|
|
updateOne-hint-unacknowledged.json 2 0 0
|
|
updateOne-hint.json 2 0 0
|
|
updateOne-let.json 0 1 1
|
|
updateOne-pipeline.json 0 1 0
|
|
updateOne-rawdata.json 1 0 1
|
|
updateOne-sort.json 1 0 1
|
|
updateOne-validation.json 1 0 0
|
|
updateOne.json 4 0 0
|
|
|
|
# every non-passing case, with its reason
|
|
aggregate-collation.json FAIL Aggregate with collation aggregate: expected 1 elements, got 0
|
|
aggregate-let.json SKIP Aggregate with let option needs server >= 5.0
|
|
aggregate-let.json FAIL Aggregate with let option unsupported (server-side error) aggregate: expected an error, the operation succeeded
|
|
aggregate-let.json SKIP Aggregate to collection with let option needs server >= 5.0
|
|
aggregate-let.json FAIL Aggregate to collection with let option unsupported (server-side error) aggregate: expected an error, the operation succeeded
|
|
aggregate-merge-errorResponse.json SKIP aggregate $merge DuplicateKey error is accessible needs server >= 5.1
|
|
aggregate-out-readConcern.json SKIP * needs topology replicaset/sharded
|
|
aggregate-rawdata.json SKIP Aggregate with rawData option needs server >= 8.2.0
|
|
aggregate-write-readPreference.json SKIP * needs topology replicaset/sharded/load-balanced
|
|
aggregate.json SKIP aggregate with a document comment - pre 4.4 needs server <= 4.2.99
|
|
aggregate.json SKIP aggregate with comment does not set comment on getMore - pre 4.4 needs server <= 4.3.99
|
|
bulkWrite-arrayFilters.json FAIL BulkWrite updateOne with arrayFilters outcome crud-tests.test[0].y: expected an array, got {"$[i]":{"b":2}}
|
|
bulkWrite-arrayFilters.json FAIL BulkWrite updateMany with arrayFilters outcome crud-tests.test[0].y: expected an array, got {"$[i]":{"b":2}}
|
|
bulkWrite-arrayFilters.json FAIL BulkWrite with arrayFilters bulkWrite.modifiedCount: expected 3, got 2
|
|
bulkWrite-collation.json FAIL BulkWrite with delete operations and collation bulkWrite.deletedCount: expected 4, got 0
|
|
bulkWrite-collation.json FAIL BulkWrite with update operations and collation bulkWrite.matchedCount: expected 6, got 2
|
|
bulkWrite-comment.json SKIP BulkWrite with comment - pre 4.4 needs server <= 4.2.99
|
|
bulkWrite-delete-hint-serverError.json SKIP * needs server <= 4.3.3
|
|
bulkWrite-deleteMany-hint-unacknowledged.json SKIP Unacknowledged deleteMany with hint string fails with client-side error on pre-4.4 server needs server <= 4.2.99
|
|
bulkWrite-deleteMany-hint-unacknowledged.json SKIP Unacknowledged deleteMany with hint document fails with client-side error on pre-4.4 server needs server <= 4.2.99
|
|
bulkWrite-deleteMany-let.json SKIP BulkWrite deleteMany with let option needs server >= 5.0
|
|
bulkWrite-deleteMany-let.json FAIL BulkWrite deleteMany with let option unsupported (server-side error) bulkWrite: expected an error, the operation succeeded
|
|
bulkWrite-deleteMany-rawdata.json SKIP BulkWrite deleteMany with rawData option needs server >= 8.2.0
|
|
bulkWrite-deleteOne-hint-unacknowledged.json SKIP Unacknowledged deleteOne with hint string fails with client-side error on pre-4.4 server needs server <= 4.2.99
|
|
bulkWrite-deleteOne-hint-unacknowledged.json SKIP Unacknowledged deleteOne with hint document fails with client-side error on pre-4.4 server needs server <= 4.2.99
|
|
bulkWrite-deleteOne-let.json SKIP BulkWrite deleteOne with let option needs server >= 5.0
|
|
bulkWrite-deleteOne-let.json FAIL BulkWrite deleteOne with let option unsupported (server-side error) bulkWrite: expected an error, the operation succeeded
|
|
bulkWrite-deleteOne-rawdata.json SKIP BulkWrite deleteOne with rawData option needs server >= 8.2.0
|
|
bulkWrite-errorResponse.json SKIP bulkWrite operations support errorResponse assertions runner: failPoint
|
|
bulkWrite-insertOne-dots_and_dollars.json SKIP Inserting document with top-level dollar-prefixed key on 5.0+ server needs server >= 5.0
|
|
bulkWrite-insertOne-dots_and_dollars.json FAIL Inserting document with top-level dollar-prefixed key on pre-5.0 server yields server-side error bulkWrite: expected an error, the operation succeeded
|
|
bulkWrite-replaceOne-dots_and_dollars.json SKIP Replacing document with dollar-prefixed key in embedded doc on 5.0+ server needs server >= 5.0
|
|
bulkWrite-replaceOne-dots_and_dollars.json FAIL Replacing document with dollar-prefixed key in embedded doc on pre-5.0 server yields server-side error bulkWrite: expected an error, the operation succeeded
|
|
bulkWrite-replaceOne-let.json SKIP BulkWrite replaceOne with let option needs server >= 5.0
|
|
bulkWrite-replaceOne-let.json FAIL BulkWrite replaceOne with let option unsupported (server-side error) bulkWrite: expected an error, the operation succeeded
|
|
bulkWrite-replaceOne-rawdata.json SKIP BulkWrite replaceOne with rawData option needs server >= 8.2.0
|
|
bulkWrite-replaceOne-sort.json SKIP BulkWrite replaceOne with sort option needs server >= 8.0
|
|
bulkWrite-updateMany-dots_and_dollars.json SKIP Updating document to set top-level dollar-prefixed key on 5.0+ server needs server >= 5.0
|
|
bulkWrite-updateMany-dots_and_dollars.json SKIP Updating document to set top-level dotted key on 5.0+ server needs server >= 5.0
|
|
bulkWrite-updateMany-dots_and_dollars.json SKIP Updating document to set dollar-prefixed key in embedded doc on 5.0+ server needs server >= 5.0
|
|
bulkWrite-updateMany-dots_and_dollars.json SKIP Updating document to set dotted key in embedded doc on 5.0+ server needs server >= 5.0
|
|
bulkWrite-updateMany-let.json SKIP BulkWrite updateMany with let option needs server >= 5.0
|
|
bulkWrite-updateMany-let.json FAIL BulkWrite updateMany with let option unsupported (server-side error) bulkWrite: error message "update spec requires u" does not contain "'update.let' is an unknown field"
|
|
bulkWrite-updateMany-pipeline.json FAIL UpdateMany in bulk write using pipelines MongoBulkWriteError: update spec requires u
|
|
bulkWrite-updateMany-rawdata.json SKIP BulkWrite updateMany with rawData option needs server >= 8.2.0
|
|
bulkWrite-updateMany-rawdata.json FAIL BulkWrite updateMany with rawData option on less than 8.2.0 - ignore argument MongoBulkWriteError: update spec requires u
|
|
bulkWrite-updateOne-dots_and_dollars.json SKIP Updating document to set top-level dollar-prefixed key on 5.0+ server needs server >= 5.0
|
|
bulkWrite-updateOne-dots_and_dollars.json SKIP Updating document to set top-level dotted key on 5.0+ server needs server >= 5.0
|
|
bulkWrite-updateOne-dots_and_dollars.json SKIP Updating document to set dollar-prefixed key in embedded doc on 5.0+ server needs server >= 5.0
|
|
bulkWrite-updateOne-dots_and_dollars.json SKIP Updating document to set dotted key in embedded doc on 5.0+ server needs server >= 5.0
|
|
bulkWrite-updateOne-let.json SKIP BulkWrite updateOne with let option needs server >= 5.0
|
|
bulkWrite-updateOne-let.json FAIL BulkWrite updateOne with let option unsupported (server-side error) bulkWrite: error message "update spec requires u" does not contain "'update.let' is an unknown field"
|
|
bulkWrite-updateOne-pipeline.json FAIL UpdateOne in bulk write using pipelines MongoBulkWriteError: update spec requires u
|
|
bulkWrite-updateOne-rawdata.json SKIP BulkWrite updateOne with rawData option needs server >= 8.2.0
|
|
bulkWrite-updateOne-rawdata.json FAIL BulkWrite updateOne with rawData option on less than 8.2.0 - ignore argument MongoBulkWriteError: update spec requires u
|
|
bulkWrite-updateOne-sort.json SKIP BulkWrite updateOne with sort option needs server >= 8.0
|
|
bypassDocumentValidation.json FAIL Aggregate with $out passes bypassDocumentValidation: false events client0[0].command.bypassDocumentValidation: missing from actual
|
|
bypassDocumentValidation.json FAIL BulkWrite passes bypassDocumentValidation: false events client0[0].command.bypassDocumentValidation: missing from actual
|
|
bypassDocumentValidation.json FAIL FindOneAndReplace passes bypassDocumentValidation: false events client0[0].command.bypassDocumentValidation: missing from actual
|
|
bypassDocumentValidation.json FAIL FindOneAndUpdate passes bypassDocumentValidation: false events client0[0].command.bypassDocumentValidation: missing from actual
|
|
bypassDocumentValidation.json FAIL InsertMany passes bypassDocumentValidation: false events client0[0].command.bypassDocumentValidation: missing from actual
|
|
client-bulkWrite-delete-options.json SKIP * needs server >= 8.0
|
|
client-bulkWrite-delete-rawdata.json SKIP client bulk write delete with rawData option needs server >= 8.2.0
|
|
client-bulkWrite-delete-rawdata.json SKIP client bulk write delete with rawData option on less than 8.2.0 - ignore argument needs server >= 8.0
|
|
client-bulkWrite-errorResponse.json SKIP * needs server >= 8.0
|
|
client-bulkWrite-errors.json SKIP * needs server >= 8.0
|
|
client-bulkWrite-mixed-namespaces.json SKIP * needs server >= 8.0
|
|
client-bulkWrite-options.json SKIP * needs server >= 8.0
|
|
client-bulkWrite-ordered.json SKIP * needs server >= 8.0
|
|
client-bulkWrite-partialResults.json SKIP * needs server >= 8.0
|
|
client-bulkWrite-replaceOne-rawdata.json SKIP client bulkWrite replaceOne with rawData option needs server >= 8.2.0
|
|
client-bulkWrite-replaceOne-rawdata.json SKIP client bulk write replaceOne with rawData option on less than 8.2.0 - ignore argument needs server >= 8.0
|
|
client-bulkWrite-replaceOne-sort.json SKIP * needs server >= 8.0
|
|
client-bulkWrite-results.json SKIP * needs server >= 8.0
|
|
client-bulkWrite-update-options.json SKIP * needs server >= 8.0
|
|
client-bulkWrite-update-pipeline.json SKIP * needs server >= 8.0
|
|
client-bulkWrite-update-rawdata.json SKIP client bulk write update with rawData option needs server >= 8.2.0
|
|
client-bulkWrite-update-rawdata.json SKIP client bulk write update with rawData option on less than 8.2.0 - ignore argument needs server >= 8.0
|
|
client-bulkWrite-update-validation.json SKIP client bulkWrite replaceOne prohibits atomic modifiers runner: operation clientBulkWrite
|
|
client-bulkWrite-update-validation.json SKIP client bulkWrite updateOne requires atomic modifiers runner: operation clientBulkWrite
|
|
client-bulkWrite-update-validation.json SKIP client bulkWrite updateMany requires atomic modifiers runner: operation clientBulkWrite
|
|
client-bulkWrite-updateOne-sort.json SKIP * needs server >= 8.0
|
|
count-collation.json SKIP Deprecated count with collation runner: operation count
|
|
count-empty.json SKIP Deprecated count with empty collection runner: operation count
|
|
count-rawdata.json SKIP Deprecated count with rawData option needs server >= 8.2.0
|
|
count-rawdata.json SKIP Deprecated count with rawData option on less than 8.2.0 - ignore argument runner: operation count
|
|
count.json SKIP Deprecated count without a filter runner: operation count
|
|
count.json SKIP Deprecated count with a filter runner: operation count
|
|
count.json SKIP Deprecated count with skip and limit runner: operation count
|
|
countDocuments-comment.json SKIP countDocuments with document comment on less than 4.4.0 - server error needs server <= 4.3.99
|
|
countDocuments-rawdata.json SKIP Count documents with rawData option needs server >= 8.2.0
|
|
create-null-ids.json FAIL inserting _id with type null via insertOne countDocuments: expected 1, got 0
|
|
create-null-ids.json FAIL inserting _id with type null via insertMany countDocuments: expected 1, got 0
|
|
create-null-ids.json FAIL inserting _id with type null via updateOne countDocuments: expected 1, got 0
|
|
create-null-ids.json FAIL inserting _id with type null via updateMany countDocuments: expected 1, got 0
|
|
create-null-ids.json FAIL inserting _id with type null via replaceOne countDocuments: expected 1, got 0
|
|
create-null-ids.json FAIL inserting _id with type null via bulkWrite countDocuments: expected 1, got 0
|
|
create-null-ids.json SKIP inserting _id with type null via clientBulkWrite needs server >= 8.0
|
|
db-aggregate-rawdata.json SKIP Aggregate with rawData option needs server >= 8.2.0, needs topology replicaset
|
|
db-aggregate-rawdata.json FAIL Aggregate with rawData option on less than 8.2.0 - ignore argument MongoServerError: command requires a collection name
|
|
db-aggregate-write-readPreference.json SKIP * needs topology replicaset
|
|
db-aggregate.json FAIL Aggregate with $listLocalSessions MongoServerError: command requires a collection name
|
|
db-aggregate.json FAIL Aggregate with $listLocalSessions and allowDiskUse MongoServerError: command requires a collection name
|
|
deleteMany-collation.json FAIL DeleteMany when many documents match with collation deleteMany.deletedCount: expected 2, got 0
|
|
deleteMany-comment.json SKIP deleteMany with comment - pre 4.4 needs server <= 4.2.99
|
|
deleteMany-hint-serverError.json SKIP * needs server <= 4.3.3
|
|
deleteMany-hint-unacknowledged.json SKIP Unacknowledged deleteMany with hint string fails with client-side error on pre-4.4 server needs server <= 4.2.99
|
|
deleteMany-hint-unacknowledged.json SKIP Unacknowledged deleteMany with hint document fails with client-side error on pre-4.4 server needs server <= 4.2.99
|
|
deleteMany-let.json SKIP deleteMany with let option needs server >= 5.0
|
|
deleteMany-let.json FAIL deleteMany with let option unsupported (server-side error) deleteMany: expected an error, the operation succeeded
|
|
deleteMany-rawdata.json SKIP deleteMany with rawData option needs server >= 8.2.0
|
|
deleteOne-collation.json FAIL DeleteOne when many documents matches with collation deleteOne.deletedCount: expected 1, got 0
|
|
deleteOne-comment.json SKIP deleteOne with comment - pre 4.4 needs server <= 4.2.99
|
|
deleteOne-errorResponse.json SKIP delete operations support errorResponse assertions runner: failPoint
|
|
deleteOne-hint-serverError.json SKIP * needs server <= 4.3.3
|
|
deleteOne-hint-unacknowledged.json SKIP Unacknowledged deleteOne with hint string fails with client-side error on pre-4.4 server needs server <= 4.2.99
|
|
deleteOne-hint-unacknowledged.json SKIP Unacknowledged deleteOne with hint document fails with client-side error on pre-4.4 server needs server <= 4.2.99
|
|
deleteOne-let.json SKIP deleteOne with let option needs server >= 5.0
|
|
deleteOne-let.json FAIL deleteOne with let option unsupported (server-side error) deleteOne: expected an error, the operation succeeded
|
|
deleteOne-rawdata.json SKIP deleteOne with rawData option needs server >= 8.2.0
|
|
distinct-collation.json FAIL Distinct with a collation distinct: expected 1 elements, got 2
|
|
distinct-comment.json SKIP distinct with document comment needs server >= 4.4.14
|
|
distinct-comment.json FAIL distinct with document comment - pre 4.4, server error distinct: expected an error, the operation succeeded
|
|
distinct-hint.json SKIP * needs server >= 7.1.0
|
|
distinct-rawdata.json SKIP distinct with rawData option needs server >= 8.2.0
|
|
estimatedDocumentCount-comment.json SKIP estimatedDocumentCount with document comment needs server >= 4.4.14
|
|
estimatedDocumentCount-comment.json FAIL estimatedDocumentCount with document comment - pre 4.4.14, server error estimatedDocumentCount: expected an error, the operation succeeded
|
|
estimatedDocumentCount-rawdata.json SKIP Estimated document count with rawData option needs server >= 8.2.0
|
|
estimatedDocumentCount.json SKIP estimatedDocumentCount with maxTimeMS runner: maxTimeMS in an expected command (CSOT rewrites it)
|
|
estimatedDocumentCount.json SKIP estimatedDocumentCount errors correctly--command error runner: failPoint
|
|
estimatedDocumentCount.json SKIP estimatedDocumentCount errors correctly--socket error runner: failPoint
|
|
estimatedDocumentCount.json FAIL estimatedDocumentCount works correctly on views estimatedDocumentCount: expected 2, got 0
|
|
find-allowdiskuse-serverError.json SKIP * needs server <= 4.3.0
|
|
find-collation.json FAIL Find with a collation find: expected 1 elements, got 0
|
|
find-comment.json FAIL find with string comment find[0]: unexpected extra keys ["x"]
|
|
find-comment.json FAIL find with document comment find[0]: unexpected extra keys ["x"]
|
|
find-comment.json SKIP find with document comment - pre 4.4 needs server <= 4.2.99
|
|
find-comment.json SKIP find with comment does not set comment on getMore - pre 4.4 needs server <= 4.3.99
|
|
find-let.json SKIP Find with let option needs server >= 5.0
|
|
find-let.json FAIL Find with let option unsupported (server-side error) find: expected an error, the operation succeeded
|
|
find-rawdata.json SKIP Find with rawData option needs server >= 8.2.0
|
|
findOneAndDelete-collation.json FAIL FindOneAndDelete when one document matches with collation findOneAndDelete: expected a document, got null
|
|
findOneAndDelete-comment.json SKIP findOneAndDelete with comment - pre 4.4 needs server <= 4.2.99
|
|
findOneAndDelete-hint-serverError.json SKIP * needs server <= 4.3.3
|
|
findOneAndDelete-hint-unacknowledged.json SKIP Unacknowledged findOneAndDelete with hint string fails with client-side error on pre-4.4 server needs server <= 4.2.99
|
|
findOneAndDelete-hint-unacknowledged.json SKIP Unacknowledged findOneAndDelete with hint document fails with client-side error on pre-4.4 server needs server <= 4.2.99
|
|
findOneAndDelete-let.json SKIP findOneAndDelete with let option needs server >= 5.0
|
|
findOneAndDelete-let.json FAIL findOneAndDelete with let option unsupported (server-side error) findOneAndDelete: expected an error, the operation succeeded
|
|
findOneAndDelete-rawdata.json SKIP findOneAndDelete with rawData option needs server >= 8.2.0
|
|
findOneAndReplace-collation.json FAIL FindOneAndReplace when one document matches with collation returning the document after modification findOneAndReplace: expected a document, got null
|
|
findOneAndReplace-comment.json SKIP findOneAndReplace with comment - pre 4.4 needs server <= 4.2.99
|
|
findOneAndReplace-dots_and_dollars.json SKIP Replacing document with dollar-prefixed key in embedded doc on 5.0+ server needs server >= 5.0
|
|
findOneAndReplace-dots_and_dollars.json FAIL Replacing document with dollar-prefixed key in embedded doc on pre-5.0 server yields server-side error findOneAndReplace: expected an error, the operation succeeded
|
|
findOneAndReplace-hint-serverError.json SKIP * needs server <= 4.3.0
|
|
findOneAndReplace-hint-unacknowledged.json SKIP Unacknowledged findOneAndReplace with hint string fails with client-side error on pre-4.4 server needs server <= 4.2.99
|
|
findOneAndReplace-hint-unacknowledged.json SKIP Unacknowledged findOneAndReplace with hint document fails with client-side error on pre-4.4 server needs server <= 4.2.99
|
|
findOneAndReplace-let.json SKIP findOneAndReplace with let option needs server >= 5.0
|
|
findOneAndReplace-let.json FAIL findOneAndReplace with let option unsupported (server-side error) findOneAndReplace: expected an error, the operation succeeded
|
|
findOneAndReplace-rawdata.json SKIP findOneAndReplace with rawData option needs server >= 8.2.0
|
|
findOneAndReplace-upsert.json FAIL FindOneAndReplace when no documents match without id specified with upsert returning the document after modification findOneAndReplace: expected a document, got null
|
|
findOneAndReplace-upsert.json FAIL FindOneAndReplace when no documents match with id specified with upsert returning the document after modification findOneAndReplace: expected a document, got null
|
|
findOneAndReplace.json FAIL FindOneAndReplace when many documents match returning the document after modification findOneAndReplace.x: expected 32, got 22
|
|
findOneAndReplace.json FAIL FindOneAndReplace when one document matches returning the document after modification findOneAndReplace.x: expected 32, got 22
|
|
findOneAndUpdate-arrayFilters.json FAIL FindOneAndUpdate when no document matches arrayFilters outcome crud-v1.coll[0].y: expected an array, got {"$[i]":{"b":2}}
|
|
findOneAndUpdate-arrayFilters.json FAIL FindOneAndUpdate when one document matches arrayFilters outcome crud-v1.coll[0].y: expected an array, got {"$[i]":{"b":2}}
|
|
findOneAndUpdate-arrayFilters.json FAIL FindOneAndUpdate when multiple documents match arrayFilters outcome crud-v1.coll[0].y: expected an array, got {"$[i]":{"b":2}}
|
|
findOneAndUpdate-collation.json FAIL FindOneAndUpdate when many documents match with collation returning the document before modification findOneAndUpdate: expected a document, got null
|
|
findOneAndUpdate-comment.json FAIL findOneAndUpdate with string comment MongoServerError: update must be a document
|
|
findOneAndUpdate-comment.json FAIL findOneAndUpdate with document comment MongoServerError: update must be a document
|
|
findOneAndUpdate-comment.json SKIP findOneAndUpdate with comment - pre 4.4 needs server <= 4.2.99
|
|
findOneAndUpdate-dots_and_dollars.json SKIP Updating document to set top-level dollar-prefixed key on 5.0+ server needs server >= 5.0
|
|
findOneAndUpdate-dots_and_dollars.json SKIP Updating document to set top-level dotted key on 5.0+ server needs server >= 5.0
|
|
findOneAndUpdate-dots_and_dollars.json SKIP Updating document to set dollar-prefixed key in embedded doc on 5.0+ server needs server >= 5.0
|
|
findOneAndUpdate-dots_and_dollars.json SKIP Updating document to set dotted key in embedded doc on 5.0+ server needs server >= 5.0
|
|
findOneAndUpdate-errorResponse.json FAIL findOneAndUpdate DuplicateKey error is accessible findOneAndUpdate.errorResponse.keyPattern: missing from actual
|
|
findOneAndUpdate-errorResponse.json SKIP findOneAndUpdate document validation errInfo is accessible needs server >= 5.0
|
|
findOneAndUpdate-hint-serverError.json SKIP * needs server <= 4.3.0
|
|
findOneAndUpdate-hint-unacknowledged.json SKIP Unacknowledged findOneAndUpdate with hint string fails with client-side error on pre-4.4 server needs server <= 4.2.99
|
|
findOneAndUpdate-hint-unacknowledged.json SKIP Unacknowledged findOneAndUpdate with hint document fails with client-side error on pre-4.4 server needs server <= 4.2.99
|
|
findOneAndUpdate-let.json SKIP findOneAndUpdate with let option needs server >= 5.0
|
|
findOneAndUpdate-let.json FAIL findOneAndUpdate with let option unsupported (server-side error) findOneAndUpdate: expected an error, the operation succeeded
|
|
findOneAndUpdate-pipeline.json FAIL FindOneAndUpdate using pipelines MongoServerError: update must be a document
|
|
findOneAndUpdate-rawdata.json SKIP findOneAndUpdate with rawData option needs server >= 8.2.0
|
|
findOneAndUpdate-rawdata.json FAIL findOneAndUpdate with rawData option on less than 8.2.0 - ignore argument MongoServerError: update must be a document
|
|
findOneAndUpdate.json FAIL FindOneAndUpdate when many documents match returning the document after modification findOneAndUpdate.x: expected 23, got 22
|
|
findOneAndUpdate.json FAIL FindOneAndUpdate when one document matches returning the document after modification findOneAndUpdate.x: expected 23, got 22
|
|
findOneAndUpdate.json FAIL FindOneAndUpdate when no documents match with upsert returning the document after modification findOneAndUpdate: expected a document, got null
|
|
insertMany-comment.json SKIP insertMany with comment - pre 4.4 needs server <= 4.2.99
|
|
insertMany-dots_and_dollars.json SKIP Inserting document with top-level dollar-prefixed key on 5.0+ server needs server >= 5.0
|
|
insertMany-dots_and_dollars.json FAIL Inserting document with top-level dollar-prefixed key on pre-5.0 server yields server-side error insertMany: expected an error, the operation succeeded
|
|
insertMany-rawdata.json SKIP insertMany with rawData option needs server >= 8.2.0
|
|
insertOne-comment.json SKIP insertOne with comment - pre 4.4 needs server <= 4.2.99
|
|
insertOne-dots_and_dollars.json SKIP Inserting document with top-level dollar-prefixed key on 5.0+ server needs server >= 5.0
|
|
insertOne-dots_and_dollars.json FAIL Inserting document with top-level dollar-prefixed key on pre-5.0 server yields server-side error insertOne: expected an error, the operation succeeded
|
|
insertOne-dots_and_dollars.json FAIL Inserting document with dollar-prefixed key in _id yields server-side error insertOne: expected an error, the operation succeeded
|
|
insertOne-errorResponse.json SKIP insert operations support errorResponse assertions runner: failPoint
|
|
insertOne-rawdata.json SKIP insertOne with rawData option needs server >= 8.2.0
|
|
replaceOne-collation.json FAIL ReplaceOne when one document matches with collation replaceOne.matchedCount: expected 1, got 0
|
|
replaceOne-comment.json SKIP ReplaceOne with comment - pre 4.4 needs server <= 4.2.99
|
|
replaceOne-dots_and_dollars.json SKIP Replacing document with dollar-prefixed key in embedded doc on 5.0+ server needs server >= 5.0
|
|
replaceOne-dots_and_dollars.json FAIL Replacing document with dollar-prefixed key in embedded doc on pre-5.0 server yields server-side error replaceOne: expected an error, the operation succeeded
|
|
replaceOne-let.json SKIP ReplaceOne with let option needs server >= 5.0
|
|
replaceOne-let.json FAIL ReplaceOne with let option unsupported (server-side error) replaceOne: expected an error, the operation succeeded
|
|
replaceOne-rawdata.json SKIP ReplaceOne with rawData option needs server >= 8.2.0
|
|
replaceOne-sort.json SKIP ReplaceOne with sort option needs server >= 8.0
|
|
updateMany-arrayFilters.json FAIL UpdateMany when no documents match arrayFilters updateMany.modifiedCount: expected 0, got 2
|
|
updateMany-arrayFilters.json FAIL UpdateMany when one document matches arrayFilters updateMany.modifiedCount: expected 1, got 2
|
|
updateMany-arrayFilters.json FAIL UpdateMany when multiple documents match arrayFilters outcome crud-v1.coll[0].y: expected an array, got {"$[i]":{"b":2}}
|
|
updateMany-collation.json FAIL UpdateMany when many documents match with collation updateMany.matchedCount: expected 2, got 1
|
|
updateMany-comment.json SKIP UpdateMany with comment - pre 4.4 needs server <= 4.2.99
|
|
updateMany-dots_and_dollars.json SKIP Updating document to set top-level dollar-prefixed key on 5.0+ server needs server >= 5.0
|
|
updateMany-dots_and_dollars.json SKIP Updating document to set top-level dotted key on 5.0+ server needs server >= 5.0
|
|
updateMany-dots_and_dollars.json SKIP Updating document to set dollar-prefixed key in embedded doc on 5.0+ server needs server >= 5.0
|
|
updateMany-dots_and_dollars.json SKIP Updating document to set dotted key in embedded doc on 5.0+ server needs server >= 5.0
|
|
updateMany-let.json SKIP updateMany with let option needs server >= 5.0
|
|
updateMany-let.json FAIL updateMany with let option unsupported (server-side error) updateMany: error message "update spec requires u" does not contain "'update.let' is an unknown field"
|
|
updateMany-pipeline.json FAIL UpdateMany using pipelines MongoServerError: update spec requires u
|
|
updateMany-rawdata.json SKIP updateMany with rawData option needs server >= 8.2.0
|
|
updateOne-arrayFilters.json FAIL UpdateOne when no document matches arrayFilters updateOne.modifiedCount: expected 0, got 1
|
|
updateOne-arrayFilters.json FAIL UpdateOne when one document matches arrayFilters outcome crud-v1.coll[0].y: expected an array, got {"$[i]":{"b":2}}
|
|
updateOne-arrayFilters.json FAIL UpdateOne when multiple documents match arrayFilters outcome crud-v1.coll[0].y: expected an array, got {"$[i]":{"b":2}}
|
|
updateOne-arrayFilters.json FAIL UpdateOne when no documents match multiple arrayFilters updateOne.modifiedCount: expected 0, got 1
|
|
updateOne-arrayFilters.json FAIL UpdateOne when one document matches multiple arrayFilters outcome crud-v1.coll[2].y: expected an array, got {"$[i]":{"c":{"$[j]":{"d":0}}}}
|
|
updateOne-collation.json FAIL UpdateOne when one document matches with collation updateOne.matchedCount: expected 1, got 0
|
|
updateOne-comment.json SKIP UpdateOne with comment - pre 4.4 needs server <= 4.2.99
|
|
updateOne-dots_and_dollars.json SKIP Updating document to set top-level dollar-prefixed key on 5.0+ server needs server >= 5.0
|
|
updateOne-dots_and_dollars.json SKIP Updating document to set top-level dotted key on 5.0+ server needs server >= 5.0
|
|
updateOne-dots_and_dollars.json SKIP Updating document to set dollar-prefixed key in embedded doc on 5.0+ server needs server >= 5.0
|
|
updateOne-dots_and_dollars.json SKIP Updating document to set dotted key in embedded doc on 5.0+ server needs server >= 5.0
|
|
updateOne-errorResponse.json SKIP update operations support errorResponse assertions runner: failPoint
|
|
updateOne-let.json SKIP UpdateOne with let option needs server >= 5.0
|
|
updateOne-let.json FAIL UpdateOne with let option unsupported (server-side error) updateOne: error message "update spec requires u" does not contain "'update.let' is an unknown field"
|
|
updateOne-pipeline.json FAIL UpdateOne using pipelines MongoServerError: update spec requires u
|
|
updateOne-rawdata.json SKIP UpdateOne with rawData option needs server >= 8.2.0
|
|
updateOne-sort.json SKIP UpdateOne with sort option needs server >= 8.0
|