commands: $out and $merge, written by the dispatch epilogue
The seven reachable failures of M2, and the first commit of the milestone to move the scorecard: 194/97/196 -> 201/90/196, with `aggregate-*.json` going 9 pass / 13 fail to 16 pass / 6 fail. The seven that moved are exactly the seven priced as reachable, and the six that remain are exactly the six attributed to M2.5 ($addFields, the expression engine), M4 ($listLocalSessions) and M8 (collation). Both stages write to a collection the pipeline is not reading, and three things stood against doing that in the handler: `aggregate` is a `.read` command, dispatch takes locks from a static table keyed on the command name before the handler runs, and `Collection.lock` allows exactly one collection lock at a time. So the handler computes the output under the locks it has and leaves it in `Context.pending_write`; the epilogue applies it with nothing held, beside the commit and the checkpoint already there. The `.read`/`.write` contract is amended in its own comment rather than quietly broken. `pending_write` is cleared at the top of every dispatch, so a handler that errors before setting one cannot leave the previous command's write to fire. A failed write replaces the pipeline's `ok: 1` with the failure, because a client told the aggregation succeeded would believe the collection had been written. What the stages do not implement is refused, not ignored: `$merge`'s `whenMatched`, `whenNotMatched`, `on` and `let` all select behaviour this server does not have, and a `whenMatched: "fail"` that silently merged would be the same lie Tier 0 spent three commits removing. Codes measured against mongod 8.3.7. `$out` and `$merge` answer byte-identically to it on both the replace and the upsert case. NOT ATOMIC, and said out loud in the code rather than left to be discovered. mongod replaces an `$out` target atomically; this engine has no cross-collection atomicity and no rename to build one from, so a crash between the drop and the last insert leaves the target holding part of the new output where MongoDB would leave the whole of the old. The fix is write-to-temp-and-rename and rename is a command that does not exist here. The test's mutation is the argument for the epilogue in one line: apply the write inside the `$out` branch and it deadlocks rather than fails. 191/191 unit tests in ReleaseFast and ReleaseSafe, 83/83 fuzz, e2e 49, e2e2 concurrent, e2e3 16, e2e4 17, e2e6 72, e2e7 86.
This commit was merged in pull request #2.
This commit is contained in:
@@ -339,5 +339,21 @@ chosen: mongod's `$out` replaces the target collection *atomically*, and this
|
||||
engine has no cross-collection atomicity. A pipeline that fails after writing
|
||||
half its output must not leave the target half-replaced.
|
||||
|
||||
**Nothing is implemented past Tier 0 until that is decided.**
|
||||
**Decision: (b), the epilogue.** Taken after the options were set out. The
|
||||
handler computes the output under the locks it already holds and leaves it in
|
||||
`Context.pending_write`; the epilogue applies it once every lock is released,
|
||||
beside the commit and the checkpoint that already live there. The
|
||||
`.read`/`.write` contract's comment was amended to say so and why.
|
||||
|
||||
Measured after: `aggregate-*.json` went 9 pass / 13 fail to 16 pass / 6 fail,
|
||||
and the whole corpus 194/97/196 to 201/90/196. The seven that moved are exactly
|
||||
the seven priced as reachable above, and the six that remain are exactly the
|
||||
six attributed to M2.5, M4 and M8.
|
||||
|
||||
**The durability question is answered honestly rather than solved.** `$out` is
|
||||
*not* atomic here: it drops the target and inserts, so a crash in between
|
||||
leaves part of the new output where mongod would leave the whole of the old.
|
||||
The shape that fixes it is write-to-temp-and-rename, and this server has no
|
||||
rename command. Recorded in `apply_pending_write`'s own comment as well, since
|
||||
that is where somebody will be standing when it matters.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user