dev/MultiforaDB
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
bb8cdd964b16d5659d28b35e0c6bb22a55c1383d
MultiforaDB/tests/spec/scorecard.txt
A.Shakhmatov bb8cdd964b tests/spec: the scorecard no longer disclaims expectEvents 
The disclaimer was accurate for as long as it stood -- events were not read,
so `pass` was an upper bound and saying otherwise would have been a lie about
the number. It is now a lie in the other direction, so it goes, replaced by
what is actually true: events are compared exactly, in number and in order,
which is what makes a pass mean the engine answered correctly *and* was asked
the right question. The header says plainly that scorecards recorded before
this are not comparable, and enumerates what is still skipped inside events
rather than leaving "asserted" to be read as "asserted completely".

Two facts in the docs had gone stale and are corrected here because this is
the commit that rereads them:

  - README said `--op-timeout-ms` defaults to 3 s. It has been 10 s since the
    commit that explains, at length and directly above the constant, why 3 s
    was wrong. A stale number in exactly the place that warns against
    tightening it is worse than no number.
  - `MAX_SCHEMA` is [1, 24]; the comment above it still claimed 1.0-1.9.

Totals unchanged at 159/132/196 -- this commit only rewrites prose, and the
scorecard is re-recorded so its header matches the runner that produced it.
2026-08-09 13:14:58 +03:00

481 lines
41 KiB
Plaintext
Raw Blame History

# MongoDB spec-test scorecard (PLAN D2) -- crud + aggregate, unified format
# specs: mongodb/specifications @ 615e0f9 (pinned in tests/spec/fetch.sh)
# driver: mongodb@7.5.0 (pinned in tests/e2e/package-lock.json)
# server: MultiforaDB reporting version 4.4.0, maxWireVersion 8
# reproduce: bash tests/spec/fetch.sh && node tests/spec/run.js --scorecard
#
# What SKIP means here, so the totals are not read as better than they are:
# - the suite needs a feature whose milestone has not landed (sessions M4,
# auth M7, failPoints, gridfs) -- counted as skip, never as pass;
# - or the runner itself does not implement the operation/matcher yet.
# expectEvents IS asserted: the commands the driver sent are compared to the
# expectation exactly, in number and in order, with `command` and `reply`
# matched as root documents. A pass therefore means the engine answered
# correctly *and* was asked the right question. Not comparable to any
# scorecard recorded before that landed, where `pass` was an upper bound.
# Still unasserted, each reported as skip where it is asserted, never as
# pass: cmap and sdam events, ignoreExtraEvents, hasServiceId,
# hasServerConnectionId, and maxTimeMS in an expected command (CSOT rewrites
# it -- the only assertion this runner declines to make).
total 159 pass 132 fail 196 skip 175 files 0 errored
# per-file: name pass fail skip
aggregate-allowdiskuse.json 3 0 0
aggregate-collation.json 0 1 0
aggregate-let.json 0 2 2
aggregate-merge-errorResponse.json 0 0 1
aggregate-merge.json 0 5 0
aggregate-out-readConcern.json 0 0 4
aggregate-out.json 0 2 0
aggregate-rawdata.json 1 0 1
aggregate-write-readPreference.json 0 0 4
aggregate.json 4 1 2
bulkWrite-arrayFilters.json 0 3 0
bulkWrite-collation.json 0 2 0
bulkWrite-comment.json 2 0 1
bulkWrite-delete-hint-serverError.json 0 0 2
bulkWrite-delete-hint.json 2 0 0
bulkWrite-deleteMany-hint-unacknowledged.json 0 2 2
bulkWrite-deleteMany-let.json 0 1 1
bulkWrite-deleteMany-rawdata.json 1 0 1
bulkWrite-deleteOne-hint-unacknowledged.json 0 2 2
bulkWrite-deleteOne-let.json 0 1 1
bulkWrite-deleteOne-rawdata.json 1 0 1
bulkWrite-errorResponse.json 0 0 1
bulkWrite-insertOne-dots_and_dollars.json 3 1 1
bulkWrite-replaceOne-dots_and_dollars.json 2 1 1
bulkWrite-replaceOne-hint-unacknowledged.json 0 2 0
bulkWrite-replaceOne-let.json 0 1 1
bulkWrite-replaceOne-rawdata.json 1 0 1
bulkWrite-replaceOne-sort.json 0 1 1
bulkWrite-update-hint.json 3 0 0
bulkWrite-update-validation.json 3 0 0
bulkWrite-updateMany-dots_and_dollars.json 0 0 4
bulkWrite-updateMany-hint-unacknowledged.json 0 2 0
bulkWrite-updateMany-let.json 0 1 1
bulkWrite-updateMany-pipeline.json 0 1 0
bulkWrite-updateMany-rawdata.json 0 1 1
bulkWrite-updateOne-dots_and_dollars.json 0 0 4
bulkWrite-updateOne-hint-unacknowledged.json 0 2 0
bulkWrite-updateOne-let.json 0 1 1
bulkWrite-updateOne-pipeline.json 0 1 0
bulkWrite-updateOne-rawdata.json 0 1 1
bulkWrite-updateOne-sort.json 0 1 1
bulkWrite.json 10 0 0
bypassDocumentValidation.json 4 5 0
client-bulkWrite-delete-options.json 0 0 2
client-bulkWrite-delete-rawdata.json 0 0 2
client-bulkWrite-errorResponse.json 0 0 1
client-bulkWrite-errors.json 0 0 9
client-bulkWrite-mixed-namespaces.json 0 0 1
client-bulkWrite-options.json 0 0 7
client-bulkWrite-ordered.json 0 0 3
client-bulkWrite-partialResults.json 0 0 9
client-bulkWrite-replaceOne-rawdata.json 0 0 2
client-bulkWrite-replaceOne-sort.json 0 0 1
client-bulkWrite-results.json 0 0 3
client-bulkWrite-update-options.json 0 0 4
client-bulkWrite-update-pipeline.json 0 0 2
client-bulkWrite-update-rawdata.json 0 0 2
client-bulkWrite-update-validation.json 0 0 3
client-bulkWrite-updateOne-sort.json 0 0 1
count-collation.json 1 0 1
count-empty.json 2 0 1
count-rawdata.json 0 0 2
count.json 4 0 3
countDocuments-comment.json 2 0 1
countDocuments-rawdata.json 1 0 1
create-null-ids.json 0 6 1
db-aggregate-rawdata.json 0 1 1
db-aggregate-write-readPreference.json 0 0 4
db-aggregate.json 0 2 0
deleteMany-collation.json 0 1 0
deleteMany-comment.json 2 0 1
deleteMany-hint-serverError.json 0 0 2
deleteMany-hint-unacknowledged.json 0 2 2
deleteMany-hint.json 2 0 0
deleteMany-let.json 0 1 1
deleteMany-rawdata.json 1 0 1
deleteMany.json 2 0 0
deleteOne-collation.json 0 1 0
deleteOne-comment.json 2 0 1
deleteOne-errorResponse.json 0 0 1
deleteOne-hint-serverError.json 0 0 2
deleteOne-hint-unacknowledged.json 0 2 2
deleteOne-hint.json 2 0 0
deleteOne-let.json 0 1 1
deleteOne-rawdata.json 1 0 1
deleteOne.json 3 0 0
distinct-collation.json 0 1 0
distinct-comment.json 1 1 1
distinct-hint.json 0 0 2
distinct-rawdata.json 0 1 1
distinct.json 0 2 0
estimatedDocumentCount-comment.json 1 1 1
estimatedDocumentCount-rawdata.json 1 0 1
estimatedDocumentCount.json 2 1 3
find-allowdiskuse-serverError.json 0 0 2
find-allowdiskuse.json 3 0 0
find-collation.json 0 1 0
find-comment.json 0 3 2
find-let.json 0 1 1
find-rawdata.json 1 0 1
find.json 5 0 0
findOne.json 1 1 0
findOneAndDelete-collation.json 0 1 0
findOneAndDelete-comment.json 2 0 1
findOneAndDelete-hint-serverError.json 0 0 2
findOneAndDelete-hint-unacknowledged.json 0 2 2
findOneAndDelete-hint.json 2 0 0
findOneAndDelete-let.json 0 1 1
findOneAndDelete-rawdata.json 1 0 1
findOneAndDelete.json 3 0 0
findOneAndReplace-collation.json 0 1 0
findOneAndReplace-comment.json 2 0 1
findOneAndReplace-dots_and_dollars.json 2 1 1
findOneAndReplace-hint-serverError.json 0 0 2
findOneAndReplace-hint-unacknowledged.json 0 2 2
findOneAndReplace-hint.json 2 0 0
findOneAndReplace-let.json 0 1 1
findOneAndReplace-rawdata.json 1 0 1
findOneAndReplace-upsert.json 2 2 0
findOneAndReplace.json 4 2 0
findOneAndUpdate-arrayFilters.json 0 3 0
findOneAndUpdate-collation.json 0 1 0
findOneAndUpdate-comment.json 0 2 1
findOneAndUpdate-dots_and_dollars.json 0 0 4
findOneAndUpdate-errorResponse.json 0 1 1
findOneAndUpdate-hint-serverError.json 0 0 2
findOneAndUpdate-hint-unacknowledged.json 0 2 2
findOneAndUpdate-hint.json 2 0 0
findOneAndUpdate-let.json 0 1 1
findOneAndUpdate-pipeline.json 0 1 0
findOneAndUpdate-rawdata.json 0 1 1
findOneAndUpdate.json 5 3 0
insertMany-comment.json 2 0 1
insertMany-dots_and_dollars.json 3 1 1
insertMany-rawdata.json 1 0 1
insertMany.json 3 0 0
insertOne-comment.json 2 0 1
insertOne-dots_and_dollars.json 5 3 1
insertOne-errorResponse.json 0 0 1
insertOne-rawdata.json 1 0 1
insertOne.json 1 0 0
replaceOne-collation.json 0 1 0
replaceOne-comment.json 2 0 1
replaceOne-dots_and_dollars.json 2 2 1
replaceOne-hint-unacknowledged.json 0 2 0
replaceOne-hint.json 2 0 0
replaceOne-let.json 0 1 1
replaceOne-rawdata.json 1 0 1
replaceOne-sort.json 0 1 1
replaceOne-validation.json 1 0 0
replaceOne.json 5 0 0
updateMany-arrayFilters.json 0 3 0
updateMany-collation.json 0 1 0
updateMany-comment.json 2 0 1
updateMany-dots_and_dollars.json 0 0 4
updateMany-hint-unacknowledged.json 0 2 0
updateMany-hint.json 2 0 0
updateMany-let.json 0 1 1
updateMany-pipeline.json 0 1 0
updateMany-rawdata.json 1 0 1
updateMany-validation.json 1 0 0
updateMany.json 4 0 0
updateOne-arrayFilters.json 0 5 0
updateOne-collation.json 0 1 0
updateOne-comment.json 2 0 1
updateOne-dots_and_dollars.json 0 0 4
updateOne-errorResponse.json 0 0 1
updateOne-hint-unacknowledged.json 0 2 0
updateOne-hint.json 2 0 0
updateOne-let.json 0 1 1
updateOne-pipeline.json 0 1 0
updateOne-rawdata.json 1 0 1
updateOne-sort.json 0 1 1
updateOne-validation.json 1 0 0
updateOne.json 4 0 0
# every non-passing case, with its reason
aggregate-collation.json FAIL Aggregate with collation aggregate: expected 1 elements, got 0
aggregate-let.json SKIP Aggregate with let option needs server >= 5.0
aggregate-let.json FAIL Aggregate with let option unsupported (server-side error) aggregate: expected an error, the operation succeeded
aggregate-let.json SKIP Aggregate to collection with let option needs server >= 5.0
aggregate-let.json FAIL Aggregate to collection with let option unsupported (server-side error) aggregate: error message "Unrecognized pipeline stage name: '$out'" does not contain "unrecognized field 'let'"
aggregate-merge-errorResponse.json SKIP aggregate $merge DuplicateKey error is accessible needs server >= 5.1
aggregate-merge.json FAIL Aggregate with $merge MongoServerError: Unrecognized pipeline stage name: '$merge'
aggregate-merge.json FAIL Aggregate with $merge and batch size of 0 MongoServerError: Unrecognized pipeline stage name: '$merge'
aggregate-merge.json FAIL Aggregate with $merge and majority readConcern MongoServerError: Unrecognized pipeline stage name: '$merge'
aggregate-merge.json FAIL Aggregate with $merge and local readConcern MongoServerError: Unrecognized pipeline stage name: '$merge'
aggregate-merge.json FAIL Aggregate with $merge and available readConcern MongoServerError: Unrecognized pipeline stage name: '$merge'
aggregate-out-readConcern.json SKIP * needs topology replicaset/sharded
aggregate-out.json FAIL Aggregate with $out MongoServerError: Unrecognized pipeline stage name: '$out'
aggregate-out.json FAIL Aggregate with $out and batch size of 0 MongoServerError: Unrecognized pipeline stage name: '$out'
aggregate-rawdata.json SKIP Aggregate with rawData option needs server >= 8.2.0
aggregate-write-readPreference.json SKIP * needs topology replicaset/sharded/load-balanced
aggregate.json SKIP aggregate with a document comment - pre 4.4 needs server <= 4.2.99
aggregate.json FAIL aggregate with comment sets comment on getMore events client0[1].command.comment: missing from actual
aggregate.json SKIP aggregate with comment does not set comment on getMore - pre 4.4 needs server <= 4.3.99
bulkWrite-arrayFilters.json FAIL BulkWrite updateOne with arrayFilters outcome crud-tests.test[0].y: expected an array, got {"$[i]":{"b":2}}
bulkWrite-arrayFilters.json FAIL BulkWrite updateMany with arrayFilters outcome crud-tests.test[0].y: expected an array, got {"$[i]":{"b":2}}
bulkWrite-arrayFilters.json FAIL BulkWrite with arrayFilters bulkWrite.modifiedCount: expected 3, got 2
bulkWrite-collation.json FAIL BulkWrite with delete operations and collation bulkWrite.deletedCount: expected 4, got 0
bulkWrite-collation.json FAIL BulkWrite with update operations and collation bulkWrite.matchedCount: expected 6, got 2
bulkWrite-comment.json SKIP BulkWrite with comment - pre 4.4 needs server <= 4.2.99
bulkWrite-delete-hint-serverError.json SKIP * needs server <= 4.3.3
bulkWrite-deleteMany-hint-unacknowledged.json SKIP Unacknowledged deleteMany with hint string fails with client-side error on pre-4.4 server needs server <= 4.2.99
bulkWrite-deleteMany-hint-unacknowledged.json SKIP Unacknowledged deleteMany with hint document fails with client-side error on pre-4.4 server needs server <= 4.2.99
bulkWrite-deleteMany-hint-unacknowledged.json FAIL Unacknowledged deleteMany with hint string on 4.4+ server events client0[0].command.writeConcern: missing from actual
bulkWrite-deleteMany-hint-unacknowledged.json FAIL Unacknowledged deleteMany with hint document on 4.4+ server events client0[0].command.writeConcern: missing from actual
bulkWrite-deleteMany-let.json SKIP BulkWrite deleteMany with let option needs server >= 5.0
bulkWrite-deleteMany-let.json FAIL BulkWrite deleteMany with let option unsupported (server-side error) bulkWrite: expected an error, the operation succeeded
bulkWrite-deleteMany-rawdata.json SKIP BulkWrite deleteMany with rawData option needs server >= 8.2.0
bulkWrite-deleteOne-hint-unacknowledged.json SKIP Unacknowledged deleteOne with hint string fails with client-side error on pre-4.4 server needs server <= 4.2.99
bulkWrite-deleteOne-hint-unacknowledged.json SKIP Unacknowledged deleteOne with hint document fails with client-side error on pre-4.4 server needs server <= 4.2.99
bulkWrite-deleteOne-hint-unacknowledged.json FAIL Unacknowledged deleteOne with hint string on 4.4+ server events client0[0].command.writeConcern: missing from actual
bulkWrite-deleteOne-hint-unacknowledged.json FAIL Unacknowledged deleteOne with hint document on 4.4+ server events client0[0].command.writeConcern: missing from actual
bulkWrite-deleteOne-let.json SKIP BulkWrite deleteOne with let option needs server >= 5.0
bulkWrite-deleteOne-let.json FAIL BulkWrite deleteOne with let option unsupported (server-side error) bulkWrite: expected an error, the operation succeeded
bulkWrite-deleteOne-rawdata.json SKIP BulkWrite deleteOne with rawData option needs server >= 8.2.0
bulkWrite-errorResponse.json SKIP bulkWrite operations support errorResponse assertions runner: failPoint
bulkWrite-insertOne-dots_and_dollars.json SKIP Inserting document with top-level dollar-prefixed key on 5.0+ server needs server >= 5.0
bulkWrite-insertOne-dots_and_dollars.json FAIL Inserting document with top-level dollar-prefixed key on pre-5.0 server yields server-side error bulkWrite: expected an error, the operation succeeded
bulkWrite-replaceOne-dots_and_dollars.json SKIP Replacing document with dollar-prefixed key in embedded doc on 5.0+ server needs server >= 5.0
bulkWrite-replaceOne-dots_and_dollars.json FAIL Replacing document with dollar-prefixed key in embedded doc on pre-5.0 server yields server-side error bulkWrite: expected an error, the operation succeeded
bulkWrite-replaceOne-hint-unacknowledged.json FAIL Unacknowledged replaceOne with hint string on 4.2+ server events client0[0].command.writeConcern: missing from actual
bulkWrite-replaceOne-hint-unacknowledged.json FAIL Unacknowledged replaceOne with hint document on 4.2+ server events client0[0].command.writeConcern: missing from actual
bulkWrite-replaceOne-let.json SKIP BulkWrite replaceOne with let option needs server >= 5.0
bulkWrite-replaceOne-let.json FAIL BulkWrite replaceOne with let option unsupported (server-side error) bulkWrite: expected an error, the operation succeeded
bulkWrite-replaceOne-rawdata.json SKIP BulkWrite replaceOne with rawData option needs server >= 8.2.0
bulkWrite-replaceOne-sort.json SKIP BulkWrite replaceOne with sort option needs server >= 8.0
bulkWrite-replaceOne-sort.json FAIL BulkWrite replaceOne with sort option unsupported (server-side error) events client0[0].command.updates[0].sort._id: missing from actual
bulkWrite-updateMany-dots_and_dollars.json SKIP Updating document to set top-level dollar-prefixed key on 5.0+ server needs server >= 5.0
bulkWrite-updateMany-dots_and_dollars.json SKIP Updating document to set top-level dotted key on 5.0+ server needs server >= 5.0
bulkWrite-updateMany-dots_and_dollars.json SKIP Updating document to set dollar-prefixed key in embedded doc on 5.0+ server needs server >= 5.0
bulkWrite-updateMany-dots_and_dollars.json SKIP Updating document to set dotted key in embedded doc on 5.0+ server needs server >= 5.0
bulkWrite-updateMany-hint-unacknowledged.json FAIL Unacknowledged updateMany with hint string on 4.2+ server events client0[0].command.writeConcern: missing from actual
bulkWrite-updateMany-hint-unacknowledged.json FAIL Unacknowledged updateMany with hint document on 4.2+ server events client0[0].command.writeConcern: missing from actual
bulkWrite-updateMany-let.json SKIP BulkWrite updateMany with let option needs server >= 5.0
bulkWrite-updateMany-let.json FAIL BulkWrite updateMany with let option unsupported (server-side error) bulkWrite: error message "update spec requires u" does not contain "'update.let' is an unknown field"
bulkWrite-updateMany-pipeline.json FAIL UpdateMany in bulk write using pipelines MongoBulkWriteError: update spec requires u
bulkWrite-updateMany-rawdata.json SKIP BulkWrite updateMany with rawData option needs server >= 8.2.0
bulkWrite-updateMany-rawdata.json FAIL BulkWrite updateMany with rawData option on less than 8.2.0 - ignore argument MongoBulkWriteError: update spec requires u
bulkWrite-updateOne-dots_and_dollars.json SKIP Updating document to set top-level dollar-prefixed key on 5.0+ server needs server >= 5.0
bulkWrite-updateOne-dots_and_dollars.json SKIP Updating document to set top-level dotted key on 5.0+ server needs server >= 5.0
bulkWrite-updateOne-dots_and_dollars.json SKIP Updating document to set dollar-prefixed key in embedded doc on 5.0+ server needs server >= 5.0
bulkWrite-updateOne-dots_and_dollars.json SKIP Updating document to set dotted key in embedded doc on 5.0+ server needs server >= 5.0
bulkWrite-updateOne-hint-unacknowledged.json FAIL Unacknowledged updateOne with hint string on 4.2+ server events client0[0].command.writeConcern: missing from actual
bulkWrite-updateOne-hint-unacknowledged.json FAIL Unacknowledged updateOne with hint document on 4.2+ server events client0[0].command.writeConcern: missing from actual
bulkWrite-updateOne-let.json SKIP BulkWrite updateOne with let option needs server >= 5.0
bulkWrite-updateOne-let.json FAIL BulkWrite updateOne with let option unsupported (server-side error) bulkWrite: error message "update spec requires u" does not contain "'update.let' is an unknown field"
bulkWrite-updateOne-pipeline.json FAIL UpdateOne in bulk write using pipelines MongoBulkWriteError: update spec requires u
bulkWrite-updateOne-rawdata.json SKIP BulkWrite updateOne with rawData option needs server >= 8.2.0
bulkWrite-updateOne-rawdata.json FAIL BulkWrite updateOne with rawData option on less than 8.2.0 - ignore argument MongoBulkWriteError: update spec requires u
bulkWrite-updateOne-sort.json SKIP BulkWrite updateOne with sort option needs server >= 8.0
bulkWrite-updateOne-sort.json FAIL BulkWrite updateOne with sort option unsupported (server-side error) events client0[0].command.updates[0].sort._id: missing from actual
bypassDocumentValidation.json FAIL Aggregate with $out passes bypassDocumentValidation: false MongoServerError: Unrecognized pipeline stage name: '$out'
bypassDocumentValidation.json FAIL BulkWrite passes bypassDocumentValidation: false events client0[0].command.bypassDocumentValidation: missing from actual
bypassDocumentValidation.json FAIL FindOneAndReplace passes bypassDocumentValidation: false events client0[0].command.bypassDocumentValidation: missing from actual
bypassDocumentValidation.json FAIL FindOneAndUpdate passes bypassDocumentValidation: false events client0[0].command.bypassDocumentValidation: missing from actual
bypassDocumentValidation.json FAIL InsertMany passes bypassDocumentValidation: false events client0[0].command.bypassDocumentValidation: missing from actual
client-bulkWrite-delete-options.json SKIP * needs server >= 8.0
client-bulkWrite-delete-rawdata.json SKIP client bulk write delete with rawData option needs server >= 8.2.0
client-bulkWrite-delete-rawdata.json SKIP client bulk write delete with rawData option on less than 8.2.0 - ignore argument needs server >= 8.0
client-bulkWrite-errorResponse.json SKIP * needs server >= 8.0
client-bulkWrite-errors.json SKIP * needs server >= 8.0
client-bulkWrite-mixed-namespaces.json SKIP * needs server >= 8.0
client-bulkWrite-options.json SKIP * needs server >= 8.0
client-bulkWrite-ordered.json SKIP * needs server >= 8.0
client-bulkWrite-partialResults.json SKIP * needs server >= 8.0
client-bulkWrite-replaceOne-rawdata.json SKIP client bulkWrite replaceOne with rawData option needs server >= 8.2.0
client-bulkWrite-replaceOne-rawdata.json SKIP client bulk write replaceOne with rawData option on less than 8.2.0 - ignore argument needs server >= 8.0
client-bulkWrite-replaceOne-sort.json SKIP * needs server >= 8.0
client-bulkWrite-results.json SKIP * needs server >= 8.0
client-bulkWrite-update-options.json SKIP * needs server >= 8.0
client-bulkWrite-update-pipeline.json SKIP * needs server >= 8.0
client-bulkWrite-update-rawdata.json SKIP client bulk write update with rawData option needs server >= 8.2.0
client-bulkWrite-update-rawdata.json SKIP client bulk write update with rawData option on less than 8.2.0 - ignore argument needs server >= 8.0
client-bulkWrite-update-validation.json SKIP client bulkWrite replaceOne prohibits atomic modifiers runner: operation clientBulkWrite
client-bulkWrite-update-validation.json SKIP client bulkWrite updateOne requires atomic modifiers runner: operation clientBulkWrite
client-bulkWrite-update-validation.json SKIP client bulkWrite updateMany requires atomic modifiers runner: operation clientBulkWrite
client-bulkWrite-updateOne-sort.json SKIP * needs server >= 8.0
count-collation.json SKIP Deprecated count with collation runner: operation count
count-empty.json SKIP Deprecated count with empty collection runner: operation count
count-rawdata.json SKIP Deprecated count with rawData option needs server >= 8.2.0
count-rawdata.json SKIP Deprecated count with rawData option on less than 8.2.0 - ignore argument runner: operation count
count.json SKIP Deprecated count without a filter runner: operation count
count.json SKIP Deprecated count with a filter runner: operation count
count.json SKIP Deprecated count with skip and limit runner: operation count
countDocuments-comment.json SKIP countDocuments with document comment on less than 4.4.0 - server error needs server <= 4.3.99
countDocuments-rawdata.json SKIP Count documents with rawData option needs server >= 8.2.0
create-null-ids.json FAIL inserting _id with type null via insertOne countDocuments: expected 1, got 0
create-null-ids.json FAIL inserting _id with type null via insertMany countDocuments: expected 1, got 0
create-null-ids.json FAIL inserting _id with type null via updateOne countDocuments: expected 1, got 0
create-null-ids.json FAIL inserting _id with type null via updateMany countDocuments: expected 1, got 0
create-null-ids.json FAIL inserting _id with type null via replaceOne countDocuments: expected 1, got 0
create-null-ids.json FAIL inserting _id with type null via bulkWrite countDocuments: expected 1, got 0
create-null-ids.json SKIP inserting _id with type null via clientBulkWrite needs server >= 8.0
db-aggregate-rawdata.json SKIP Aggregate with rawData option needs server >= 8.2.0, needs topology replicaset
db-aggregate-rawdata.json FAIL Aggregate with rawData option on less than 8.2.0 - ignore argument MongoServerError: command requires a collection name
db-aggregate-write-readPreference.json SKIP * needs topology replicaset
db-aggregate.json FAIL Aggregate with $listLocalSessions MongoServerError: command requires a collection name
db-aggregate.json FAIL Aggregate with $listLocalSessions and allowDiskUse MongoServerError: command requires a collection name
deleteMany-collation.json FAIL DeleteMany when many documents match with collation deleteMany.deletedCount: expected 2, got 0
deleteMany-comment.json SKIP deleteMany with comment - pre 4.4 needs server <= 4.2.99
deleteMany-hint-serverError.json SKIP * needs server <= 4.3.3
deleteMany-hint-unacknowledged.json SKIP Unacknowledged deleteMany with hint string fails with client-side error on pre-4.4 server needs server <= 4.2.99
deleteMany-hint-unacknowledged.json SKIP Unacknowledged deleteMany with hint document fails with client-side error on pre-4.4 server needs server <= 4.2.99
deleteMany-hint-unacknowledged.json FAIL Unacknowledged deleteMany with hint string on 4.4+ server events client0[0].command.writeConcern: missing from actual
deleteMany-hint-unacknowledged.json FAIL Unacknowledged deleteMany with hint document on 4.4+ server events client0[0].command.writeConcern: missing from actual
deleteMany-let.json SKIP deleteMany with let option needs server >= 5.0
deleteMany-let.json FAIL deleteMany with let option unsupported (server-side error) deleteMany: expected an error, the operation succeeded
deleteMany-rawdata.json SKIP deleteMany with rawData option needs server >= 8.2.0
deleteOne-collation.json FAIL DeleteOne when many documents matches with collation deleteOne.deletedCount: expected 1, got 0
deleteOne-comment.json SKIP deleteOne with comment - pre 4.4 needs server <= 4.2.99
deleteOne-errorResponse.json SKIP delete operations support errorResponse assertions runner: failPoint
deleteOne-hint-serverError.json SKIP * needs server <= 4.3.3
deleteOne-hint-unacknowledged.json SKIP Unacknowledged deleteOne with hint string fails with client-side error on pre-4.4 server needs server <= 4.2.99
deleteOne-hint-unacknowledged.json SKIP Unacknowledged deleteOne with hint document fails with client-side error on pre-4.4 server needs server <= 4.2.99
deleteOne-hint-unacknowledged.json FAIL Unacknowledged deleteOne with hint string on 4.4+ server events client0[0].command.writeConcern: missing from actual
deleteOne-hint-unacknowledged.json FAIL Unacknowledged deleteOne with hint document on 4.4+ server events client0[0].command.writeConcern: missing from actual
deleteOne-let.json SKIP deleteOne with let option needs server >= 5.0
deleteOne-let.json FAIL deleteOne with let option unsupported (server-side error) deleteOne: expected an error, the operation succeeded
deleteOne-rawdata.json SKIP deleteOne with rawData option needs server >= 8.2.0
distinct-collation.json FAIL Distinct with a collation MongoServerError: no such command: 'distinct'
distinct-comment.json SKIP distinct with document comment needs server >= 4.4.14
distinct-comment.json FAIL distinct with string comment MongoServerError: no such command: 'distinct'
distinct-hint.json SKIP * needs server >= 7.1.0
distinct-rawdata.json SKIP distinct with rawData option needs server >= 8.2.0
distinct-rawdata.json FAIL distinct with rawData option on less than 8.2.0 - ignore argument MongoServerError: no such command: 'distinct'
distinct.json FAIL Distinct without a filter MongoServerError: no such command: 'distinct'
distinct.json FAIL Distinct with a filter MongoServerError: no such command: 'distinct'
estimatedDocumentCount-comment.json SKIP estimatedDocumentCount with document comment needs server >= 4.4.14
estimatedDocumentCount-comment.json FAIL estimatedDocumentCount with document comment - pre 4.4.14, server error estimatedDocumentCount: expected an error, the operation succeeded
estimatedDocumentCount-rawdata.json SKIP Estimated document count with rawData option needs server >= 8.2.0
estimatedDocumentCount.json SKIP estimatedDocumentCount with maxTimeMS runner: maxTimeMS in an expected command (CSOT rewrites it)
estimatedDocumentCount.json SKIP estimatedDocumentCount errors correctly--command error runner: failPoint
estimatedDocumentCount.json SKIP estimatedDocumentCount errors correctly--socket error runner: failPoint
estimatedDocumentCount.json FAIL estimatedDocumentCount works correctly on views estimatedDocumentCount: expected 2, got 0
find-allowdiskuse-serverError.json SKIP * needs server <= 4.3.0
find-collation.json FAIL Find with a collation find: expected 1 elements, got 0
find-comment.json FAIL find with string comment find[0]: unexpected extra keys ["x"]
find-comment.json FAIL find with document comment find[0]: unexpected extra keys ["x"]
find-comment.json SKIP find with document comment - pre 4.4 needs server <= 4.2.99
find-comment.json FAIL find with comment sets comment on getMore events client0[1].command.comment: missing from actual
find-comment.json SKIP find with comment does not set comment on getMore - pre 4.4 needs server <= 4.3.99
find-let.json SKIP Find with let option needs server >= 5.0
find-let.json FAIL Find with let option unsupported (server-side error) find: expected an error, the operation succeeded
find-rawdata.json SKIP Find with rawData option needs server >= 8.2.0
findOne.json FAIL FindOne with filter, sort, and skip events client0[0].command.sort._id: missing from actual
findOneAndDelete-collation.json FAIL FindOneAndDelete when one document matches with collation findOneAndDelete: expected a document, got null
findOneAndDelete-comment.json SKIP findOneAndDelete with comment - pre 4.4 needs server <= 4.2.99
findOneAndDelete-hint-serverError.json SKIP * needs server <= 4.3.3
findOneAndDelete-hint-unacknowledged.json SKIP Unacknowledged findOneAndDelete with hint string fails with client-side error on pre-4.4 server needs server <= 4.2.99
findOneAndDelete-hint-unacknowledged.json SKIP Unacknowledged findOneAndDelete with hint document fails with client-side error on pre-4.4 server needs server <= 4.2.99
findOneAndDelete-hint-unacknowledged.json FAIL Unacknowledged findOneAndDelete with hint string on 4.4+ server findOneAndDelete: expected null, got {"_id":2,"x":22}
findOneAndDelete-hint-unacknowledged.json FAIL Unacknowledged findOneAndDelete with hint document on 4.4+ server findOneAndDelete: expected null, got {"_id":2,"x":22}
findOneAndDelete-let.json SKIP findOneAndDelete with let option needs server >= 5.0
findOneAndDelete-let.json FAIL findOneAndDelete with let option unsupported (server-side error) findOneAndDelete: expected an error, the operation succeeded
findOneAndDelete-rawdata.json SKIP findOneAndDelete with rawData option needs server >= 8.2.0
findOneAndReplace-collation.json FAIL FindOneAndReplace when one document matches with collation returning the document after modification findOneAndReplace: expected a document, got null
findOneAndReplace-comment.json SKIP findOneAndReplace with comment - pre 4.4 needs server <= 4.2.99
findOneAndReplace-dots_and_dollars.json SKIP Replacing document with dollar-prefixed key in embedded doc on 5.0+ server needs server >= 5.0
findOneAndReplace-dots_and_dollars.json FAIL Replacing document with dollar-prefixed key in embedded doc on pre-5.0 server yields server-side error findOneAndReplace: expected an error, the operation succeeded
findOneAndReplace-hint-serverError.json SKIP * needs server <= 4.3.0
findOneAndReplace-hint-unacknowledged.json SKIP Unacknowledged findOneAndReplace with hint string fails with client-side error on pre-4.4 server needs server <= 4.2.99
findOneAndReplace-hint-unacknowledged.json SKIP Unacknowledged findOneAndReplace with hint document fails with client-side error on pre-4.4 server needs server <= 4.2.99
findOneAndReplace-hint-unacknowledged.json FAIL Unacknowledged findOneAndReplace with hint string on 4.4+ server findOneAndReplace: expected null, got {"_id":2,"x":22}
findOneAndReplace-hint-unacknowledged.json FAIL Unacknowledged findOneAndReplace with hint document on 4.4+ server findOneAndReplace: expected null, got {"_id":2,"x":22}
findOneAndReplace-let.json SKIP findOneAndReplace with let option needs server >= 5.0
findOneAndReplace-let.json FAIL findOneAndReplace with let option unsupported (server-side error) findOneAndReplace: expected an error, the operation succeeded
findOneAndReplace-rawdata.json SKIP findOneAndReplace with rawData option needs server >= 8.2.0
findOneAndReplace-upsert.json FAIL FindOneAndReplace when no documents match without id specified with upsert returning the document after modification findOneAndReplace: expected a document, got null
findOneAndReplace-upsert.json FAIL FindOneAndReplace when no documents match with id specified with upsert returning the document after modification findOneAndReplace: expected a document, got null
findOneAndReplace.json FAIL FindOneAndReplace when many documents match returning the document after modification findOneAndReplace.x: expected 32, got 22
findOneAndReplace.json FAIL FindOneAndReplace when one document matches returning the document after modification findOneAndReplace.x: expected 32, got 22
findOneAndUpdate-arrayFilters.json FAIL FindOneAndUpdate when no document matches arrayFilters outcome crud-v1.coll[0].y: expected an array, got {"$[i]":{"b":2}}
findOneAndUpdate-arrayFilters.json FAIL FindOneAndUpdate when one document matches arrayFilters outcome crud-v1.coll[0].y: expected an array, got {"$[i]":{"b":2}}
findOneAndUpdate-arrayFilters.json FAIL FindOneAndUpdate when multiple documents match arrayFilters outcome crud-v1.coll[0].y: expected an array, got {"$[i]":{"b":2}}
findOneAndUpdate-collation.json FAIL FindOneAndUpdate when many documents match with collation returning the document before modification findOneAndUpdate: expected a document, got null
findOneAndUpdate-comment.json FAIL findOneAndUpdate with string comment MongoServerError: update must be a document
findOneAndUpdate-comment.json FAIL findOneAndUpdate with document comment MongoServerError: update must be a document
findOneAndUpdate-comment.json SKIP findOneAndUpdate with comment - pre 4.4 needs server <= 4.2.99
findOneAndUpdate-dots_and_dollars.json SKIP Updating document to set top-level dollar-prefixed key on 5.0+ server needs server >= 5.0
findOneAndUpdate-dots_and_dollars.json SKIP Updating document to set top-level dotted key on 5.0+ server needs server >= 5.0
findOneAndUpdate-dots_and_dollars.json SKIP Updating document to set dollar-prefixed key in embedded doc on 5.0+ server needs server >= 5.0
findOneAndUpdate-dots_and_dollars.json SKIP Updating document to set dotted key in embedded doc on 5.0+ server needs server >= 5.0
findOneAndUpdate-errorResponse.json FAIL findOneAndUpdate DuplicateKey error is accessible findOneAndUpdate.errorResponse.keyPattern: missing from actual
findOneAndUpdate-errorResponse.json SKIP findOneAndUpdate document validation errInfo is accessible needs server >= 5.0
findOneAndUpdate-hint-serverError.json SKIP * needs server <= 4.3.0
findOneAndUpdate-hint-unacknowledged.json SKIP Unacknowledged findOneAndUpdate with hint string fails with client-side error on pre-4.4 server needs server <= 4.2.99
findOneAndUpdate-hint-unacknowledged.json SKIP Unacknowledged findOneAndUpdate with hint document fails with client-side error on pre-4.4 server needs server <= 4.2.99
findOneAndUpdate-hint-unacknowledged.json FAIL Unacknowledged findOneAndUpdate with hint string on 4.4+ server findOneAndUpdate: expected null, got {"_id":2,"x":22}
findOneAndUpdate-hint-unacknowledged.json FAIL Unacknowledged findOneAndUpdate with hint document on 4.4+ server findOneAndUpdate: expected null, got {"_id":2,"x":22}
findOneAndUpdate-let.json SKIP findOneAndUpdate with let option needs server >= 5.0
findOneAndUpdate-let.json FAIL findOneAndUpdate with let option unsupported (server-side error) findOneAndUpdate: expected an error, the operation succeeded
findOneAndUpdate-pipeline.json FAIL FindOneAndUpdate using pipelines MongoServerError: update must be a document
findOneAndUpdate-rawdata.json SKIP findOneAndUpdate with rawData option needs server >= 8.2.0
findOneAndUpdate-rawdata.json FAIL findOneAndUpdate with rawData option on less than 8.2.0 - ignore argument MongoServerError: update must be a document
findOneAndUpdate.json FAIL FindOneAndUpdate when many documents match returning the document after modification findOneAndUpdate.x: expected 23, got 22
findOneAndUpdate.json FAIL FindOneAndUpdate when one document matches returning the document after modification findOneAndUpdate.x: expected 23, got 22
findOneAndUpdate.json FAIL FindOneAndUpdate when no documents match with upsert returning the document after modification findOneAndUpdate: expected a document, got null
insertMany-comment.json SKIP insertMany with comment - pre 4.4 needs server <= 4.2.99
insertMany-dots_and_dollars.json SKIP Inserting document with top-level dollar-prefixed key on 5.0+ server needs server >= 5.0
insertMany-dots_and_dollars.json FAIL Inserting document with top-level dollar-prefixed key on pre-5.0 server yields server-side error insertMany: expected an error, the operation succeeded
insertMany-rawdata.json SKIP insertMany with rawData option needs server >= 8.2.0
insertOne-comment.json SKIP insertOne with comment - pre 4.4 needs server <= 4.2.99
insertOne-dots_and_dollars.json SKIP Inserting document with top-level dollar-prefixed key on 5.0+ server needs server >= 5.0
insertOne-dots_and_dollars.json FAIL Inserting document with top-level dollar-prefixed key on pre-5.0 server yields server-side error insertOne: expected an error, the operation succeeded
insertOne-dots_and_dollars.json FAIL Inserting document with dollar-prefixed key in _id yields server-side error insertOne: expected an error, the operation succeeded
insertOne-dots_and_dollars.json FAIL Unacknowledged write using dollar-prefixed or dotted keys may be silently rejected on pre-5.0 server events client0[0].command.writeConcern: missing from actual
insertOne-errorResponse.json SKIP insert operations support errorResponse assertions runner: failPoint
insertOne-rawdata.json SKIP insertOne with rawData option needs server >= 8.2.0
replaceOne-collation.json FAIL ReplaceOne when one document matches with collation replaceOne.matchedCount: expected 1, got 0
replaceOne-comment.json SKIP ReplaceOne with comment - pre 4.4 needs server <= 4.2.99
replaceOne-dots_and_dollars.json SKIP Replacing document with dollar-prefixed key in embedded doc on 5.0+ server needs server >= 5.0
replaceOne-dots_and_dollars.json FAIL Replacing document with dollar-prefixed key in embedded doc on pre-5.0 server yields server-side error replaceOne: expected an error, the operation succeeded
replaceOne-dots_and_dollars.json FAIL Unacknowledged write using dollar-prefixed or dotted keys may be silently rejected on pre-5.0 server events client0[0].command.writeConcern: missing from actual
replaceOne-hint-unacknowledged.json FAIL Unacknowledged replaceOne with hint string on 4.2+ server events client0[0].command.writeConcern: missing from actual
replaceOne-hint-unacknowledged.json FAIL Unacknowledged replaceOne with hint document on 4.2+ server events client0[0].command.writeConcern: missing from actual
replaceOne-let.json SKIP ReplaceOne with let option needs server >= 5.0
replaceOne-let.json FAIL ReplaceOne with let option unsupported (server-side error) replaceOne: expected an error, the operation succeeded
replaceOne-rawdata.json SKIP ReplaceOne with rawData option needs server >= 8.2.0
replaceOne-sort.json SKIP ReplaceOne with sort option needs server >= 8.0
replaceOne-sort.json FAIL replaceOne with sort option unsupported (server-side error) events client0[0].command.updates[0].sort._id: missing from actual
updateMany-arrayFilters.json FAIL UpdateMany when no documents match arrayFilters updateMany.modifiedCount: expected 0, got 2
updateMany-arrayFilters.json FAIL UpdateMany when one document matches arrayFilters updateMany.modifiedCount: expected 1, got 2
updateMany-arrayFilters.json FAIL UpdateMany when multiple documents match arrayFilters outcome crud-v1.coll[0].y: expected an array, got {"$[i]":{"b":2}}
updateMany-collation.json FAIL UpdateMany when many documents match with collation updateMany.matchedCount: expected 2, got 1
updateMany-comment.json SKIP UpdateMany with comment - pre 4.4 needs server <= 4.2.99
updateMany-dots_and_dollars.json SKIP Updating document to set top-level dollar-prefixed key on 5.0+ server needs server >= 5.0
updateMany-dots_and_dollars.json SKIP Updating document to set top-level dotted key on 5.0+ server needs server >= 5.0
updateMany-dots_and_dollars.json SKIP Updating document to set dollar-prefixed key in embedded doc on 5.0+ server needs server >= 5.0
updateMany-dots_and_dollars.json SKIP Updating document to set dotted key in embedded doc on 5.0+ server needs server >= 5.0
updateMany-hint-unacknowledged.json FAIL Unacknowledged updateMany with hint string on 4.2+ server events client0[0].command.writeConcern: missing from actual
updateMany-hint-unacknowledged.json FAIL Unacknowledged updateMany with hint document on 4.2+ server events client0[0].command.writeConcern: missing from actual
updateMany-let.json SKIP updateMany with let option needs server >= 5.0
updateMany-let.json FAIL updateMany with let option unsupported (server-side error) updateMany: error message "update spec requires u" does not contain "'update.let' is an unknown field"
updateMany-pipeline.json FAIL UpdateMany using pipelines MongoServerError: update spec requires u
updateMany-rawdata.json SKIP updateMany with rawData option needs server >= 8.2.0
updateOne-arrayFilters.json FAIL UpdateOne when no document matches arrayFilters updateOne.modifiedCount: expected 0, got 1
updateOne-arrayFilters.json FAIL UpdateOne when one document matches arrayFilters outcome crud-v1.coll[0].y: expected an array, got {"$[i]":{"b":2}}
updateOne-arrayFilters.json FAIL UpdateOne when multiple documents match arrayFilters outcome crud-v1.coll[0].y: expected an array, got {"$[i]":{"b":2}}
updateOne-arrayFilters.json FAIL UpdateOne when no documents match multiple arrayFilters updateOne.modifiedCount: expected 0, got 1
updateOne-arrayFilters.json FAIL UpdateOne when one document matches multiple arrayFilters outcome crud-v1.coll[2].y: expected an array, got {"$[i]":{"c":{"$[j]":{"d":0}}}}
updateOne-collation.json FAIL UpdateOne when one document matches with collation updateOne.matchedCount: expected 1, got 0
updateOne-comment.json SKIP UpdateOne with comment - pre 4.4 needs server <= 4.2.99
updateOne-dots_and_dollars.json SKIP Updating document to set top-level dollar-prefixed key on 5.0+ server needs server >= 5.0
updateOne-dots_and_dollars.json SKIP Updating document to set top-level dotted key on 5.0+ server needs server >= 5.0
updateOne-dots_and_dollars.json SKIP Updating document to set dollar-prefixed key in embedded doc on 5.0+ server needs server >= 5.0
updateOne-dots_and_dollars.json SKIP Updating document to set dotted key in embedded doc on 5.0+ server needs server >= 5.0
updateOne-errorResponse.json SKIP update operations support errorResponse assertions runner: failPoint
updateOne-hint-unacknowledged.json FAIL Unacknowledged updateOne with hint string on 4.2+ server events client0[0].command.writeConcern: missing from actual
updateOne-hint-unacknowledged.json FAIL Unacknowledged updateOne with hint document on 4.2+ server events client0[0].command.writeConcern: missing from actual
updateOne-let.json SKIP UpdateOne with let option needs server >= 5.0
updateOne-let.json FAIL UpdateOne with let option unsupported (server-side error) updateOne: error message "update spec requires u" does not contain "'update.let' is an unknown field"
updateOne-pipeline.json FAIL UpdateOne using pipelines MongoServerError: update spec requires u
updateOne-rawdata.json SKIP UpdateOne with rawData option needs server >= 8.2.0
updateOne-sort.json SKIP UpdateOne with sort option needs server >= 8.0
updateOne-sort.json FAIL updateOne with sort option unsupported (server-side error) events client0[0].command.updates[0].sort._id: missing from actual
Reference in New Issue View Git Blame Copy Permalink