dev/MultiforaDB
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

commands: $out and $merge, written by the dispatch epilogue

Browse Source 
The seven reachable failures of M2, and the first commit of the milestone to
move the scorecard: 194/97/196 -> 201/90/196, with `aggregate-*.json` going
9 pass / 13 fail to 16 pass / 6 fail. The seven that moved are exactly the
seven priced as reachable, and the six that remain are exactly the six
attributed to M2.5 ($addFields, the expression engine), M4 ($listLocalSessions)
and M8 (collation).

Both stages write to a collection the pipeline is not reading, and three things
stood against doing that in the handler: `aggregate` is a `.read` command,
dispatch takes locks from a static table keyed on the command name before the
handler runs, and `Collection.lock` allows exactly one collection lock at a
time. So the handler computes the output under the locks it has and leaves it
in `Context.pending_write`; the epilogue applies it with nothing held, beside
the commit and the checkpoint already there. The `.read`/`.write` contract is
amended in its own comment rather than quietly broken.

`pending_write` is cleared at the top of every dispatch, so a handler that
errors before setting one cannot leave the previous command's write to fire. A
failed write replaces the pipeline's `ok: 1` with the failure, because a client
told the aggregation succeeded would believe the collection had been written.

What the stages do not implement is refused, not ignored: `$merge`'s
`whenMatched`, `whenNotMatched`, `on` and `let` all select behaviour this
server does not have, and a `whenMatched: "fail"` that silently merged would be
the same lie Tier 0 spent three commits removing. Codes measured against
mongod 8.3.7. `$out` and `$merge` answer byte-identically to it on both the
replace and the upsert case.

NOT ATOMIC, and said out loud in the code rather than left to be discovered.
mongod replaces an `$out` target atomically; this engine has no
cross-collection atomicity and no rename to build one from, so a crash between
the drop and the last insert leaves the target holding part of the new output
where MongoDB would leave the whole of the old. The fix is
write-to-temp-and-rename and rename is a command that does not exist here.

The test's mutation is the argument for the epilogue in one line: apply the
write inside the `$out` branch and it deadlocks rather than fails.

191/191 unit tests in ReleaseFast and ReleaseSafe, 83/83 fuzz, e2e 49, e2e2
concurrent, e2e3 16, e2e4 17, e2e6 72, e2e7 86.
This commit was merged in pull request #2.
This commit is contained in:
A.Shakhmatov
2026-08-09 21:02:01 +03:00
parent 89eae1cd9c
commit 8ebeb9d4ec
3 changed files with 299 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files

17
tests/spec/scorecard.txt
View File

@@ -18,16 +18,16 @@
# hasServerConnectionId, and maxTimeMS in an expected command (CSOT rewrites
# it -- the only assertion this runner declines to make).
total 194 pass 97 fail 196 skip 175 files 0 errored
total 201 pass 90 fail 196 skip 175 files 0 errored
# per-file: name pass fail skip
aggregate-allowdiskuse.json 3 0 0
aggregate-collation.json 0 1 0
aggregate-let.json 0 2 2
aggregate-merge-errorResponse.json 0 0 1
aggregate-merge.json 0 5 0
aggregate-merge.json 5 0 0
aggregate-out-readConcern.json 0 0 4
aggregate-out.json 0 2 0
aggregate-out.json 2 0 0
aggregate-rawdata.json 1 0 1
aggregate-write-readPreference.json 0 0 4
aggregate.json 5 0 2
@@ -202,16 +202,9 @@ aggregate-collation.json FAIL Aggregate with collation aggregate: expected 1 ele
aggregate-let.json SKIP Aggregate with let option needs server >= 5.0
aggregate-let.json FAIL Aggregate with let option unsupported (server-side error) aggregate: expected an error, the operation succeeded
aggregate-let.json SKIP Aggregate to collection with let option needs server >= 5.0
aggregate-let.json FAIL Aggregate to collection with let option unsupported (server-side error) aggregate: error message "Unrecognized pipeline stage name: '$out'" does not contain "unrecognized field 'let'"
aggregate-let.json FAIL Aggregate to collection with let option unsupported (server-side error) aggregate: expected an error, the operation succeeded
aggregate-merge-errorResponse.json SKIP aggregate $merge DuplicateKey error is accessible needs server >= 5.1
aggregate-merge.json FAIL Aggregate with $merge MongoServerError: Unrecognized pipeline stage name: '$merge'
aggregate-merge.json FAIL Aggregate with $merge and batch size of 0 MongoServerError: Unrecognized pipeline stage name: '$merge'
aggregate-merge.json FAIL Aggregate with $merge and majority readConcern MongoServerError: Unrecognized pipeline stage name: '$merge'
aggregate-merge.json FAIL Aggregate with $merge and local readConcern MongoServerError: Unrecognized pipeline stage name: '$merge'
aggregate-merge.json FAIL Aggregate with $merge and available readConcern MongoServerError: Unrecognized pipeline stage name: '$merge'
aggregate-out-readConcern.json SKIP * needs topology replicaset/sharded
aggregate-out.json FAIL Aggregate with $out MongoServerError: Unrecognized pipeline stage name: '$out'
aggregate-out.json FAIL Aggregate with $out and batch size of 0 MongoServerError: Unrecognized pipeline stage name: '$out'
aggregate-rawdata.json SKIP Aggregate with rawData option needs server >= 8.2.0
aggregate-write-readPreference.json SKIP * needs topology replicaset/sharded/load-balanced
aggregate.json SKIP aggregate with a document comment - pre 4.4 needs server <= 4.2.99
@@ -261,7 +254,7 @@ bulkWrite-updateOne-pipeline.json FAIL UpdateOne in bulk write using pipelines M
bulkWrite-updateOne-rawdata.json SKIP BulkWrite updateOne with rawData option needs server >= 8.2.0
bulkWrite-updateOne-rawdata.json FAIL BulkWrite updateOne with rawData option on less than 8.2.0 - ignore argument MongoBulkWriteError: update spec requires u
bulkWrite-updateOne-sort.json SKIP BulkWrite updateOne with sort option needs server >= 8.0
bypassDocumentValidation.json FAIL Aggregate with $out passes bypassDocumentValidation: false MongoServerError: Unrecognized pipeline stage name: '$out'
bypassDocumentValidation.json FAIL Aggregate with $out passes bypassDocumentValidation: false events client0[0].command.bypassDocumentValidation: missing from actual
bypassDocumentValidation.json FAIL BulkWrite passes bypassDocumentValidation: false events client0[0].command.bypassDocumentValidation: missing from actual
bypassDocumentValidation.json FAIL FindOneAndReplace passes bypassDocumentValidation: false events client0[0].command.bypassDocumentValidation: missing from actual
bypassDocumentValidation.json FAIL FindOneAndUpdate passes bypassDocumentValidation: false events client0[0].command.bypassDocumentValidation: missing from actual