M1: doc-level free list, sessions, and a spec runner that no longer overstates #1

Merged
dev merged 37 commits from m1-cursors into main 2026-08-09 16:15:34 +00:00
Showing only changes of commit e65da2740e - Show all commits

73
PLAN.md
View File

@@ -721,6 +721,70 @@ between `compact` and `checkpoint`; it is out of this scope because it wants
its own design pass, and because the free list must not add a second instance its own design pass, and because the free list must not add a second instance
of the same shape. of the same shape.
### The spec runner starts reading `expectEvents`
354 of the 487 cases declare `expectEvents` and the runner read none of them,
so a case could send the wrong command entirely and still be counted a pass as
long as the result came back right. The old `pass` column was an upper bound by
construction and said so; it is now an assertion that the engine answered
correctly **and** was asked the right question. **Scorecards recorded before
this are not comparable with ones recorded after.**
The totals moved 168/124/195 → 194/97/196 across the commits, but the path
matters more than the endpoints: turning the assertion on cost 34 passes, and
every one of them was a defect the result column could not see.
What it found, in the order it found them:
1. **The runner dropped `collectionOptions`.** Every collection entity was
built as `db.collection(name)`, so the 15 entities declaring
`writeConcern: {w: 0}` never got it — **every "unacknowledged write" case in
the corpus was running an acknowledged write.** They passed because the two
produce results a `$$unsetOrMatches` expectation accepts either way. Only
the command on the wire distinguished them, and nothing read the command.
2. **The wire version disagreed with the version string.** `buildInfo` said
4.4.0, the handshake said maxWireVersion 8, which is 4.2. A driver believes
the wire version: it refused *client-side* to send `hint` on an
unacknowledged delete or findAndModify, and withheld `comment` from
getMore, listCollections and listDatabases. 16 cases. The 8 was not
arbitrary — it was tied to keeping drivers off the streaming hello protocol
— but that turned out to rest entirely on omitting `topologyVersion`, which
is checked in the driver and is the whole mechanism. A test now asserts the
two numbers agree, since drifting apart silently was the actual defect.
3. **`$$unsetOrMatches` was changing root-ness.** The operator wraps a value,
it does not reposition it; the runner matched what stood behind it as a
nested document. 25 cases, all of them results the engine had right.
4. **An event's command is not the shape the driver sends.** A sort is held as
a JS `Map`, so `Object.keys` on it is empty and every expected key read as
missing. It hides well: EJSON prints a Map exactly like a document, so the
event dump reads as evidence the matcher is wrong about something else.
Two assertions are declined, both enumerated in the runner and in
`scorecard.txt`, and neither can hide anything the engine did:
- **`maxTimeMS`** — the harness's own doing. Every client carries CSOT
`timeoutMS`, which overwrites `maxTimeMS` with the remaining budget, so the
value on the wire is ours. Refused unconditionally rather than only when it
would fail, so it cannot become a pass by coincidence. One case, and dropping
`timeoutMS` instead would cost far more — it is what replaced the outer race
that once produced ~190 phantom timeout FAILs.
- **`cmap`/`sdam` event types, `ignoreExtraEvents`, `hasServiceId`,
`hasServerConnectionId`** — none occurs in this corpus; reported unsupported
where asserted rather than waived.
**Left failing on purpose: `bypassDocumentValidation: false`, 4 cases.**
mongodb@7.5.0 strips the field unless it is exactly `true` on the bulk and
findAndModify paths (`lib/bulk/common.js:292`,
`lib/operations/find_and_modify.js:19`) while sending it correctly for single
-document operations, so 4 sibling cases pass and 4 fail on a difference that
is entirely the driver's. The field is built client-side and never reaches the
engine. A refusal was written and thrown away: made unconditional it also
skipped the 4 that legitimately pass, and made conditional it would be a
skip-when-it-would-fail rule, which is the shape that turns a scorecard into
flattery. Four undeserved entries in the fail column is the cheaper error, and
this note is the correction. Revisit when the driver is bumped — which already
has to be its own commit with its own re-recorded scorecard.
--- ---
## 6. Deferred designs (grill each at its milestone) ## 6. Deferred designs (grill each at its milestone)
@@ -759,10 +823,11 @@ of the same shape.
the anchor rewritten — resuming at it returned updated documents twice, the anchor rewritten — resuming at it returned updated documents twice,
caught by draining a collection being updated underneath. caught by draining a collection being updated underneath.
Still open in M1: the doc-level free list, sessions plumbing (`lsid` Still open in M1: the doc-level free list and sessions plumbing (`lsid`
accepted), and command-monitoring (`expectEvents`) in the spec runner. The accepted). The eight reclamation bugs above were cleared first, as
eight reclamation bugs above were cleared first, as preconditions for the preconditions for the free list rather than as work of their own;
free list rather than as work of their own. command-monitoring (`expectEvents`) landed next, so that the free list and
sessions are measured by an instrument that is no longer known to overstate.
**A prerequisite the free list must honour**, recorded here while it is **A prerequisite the free list must honour**, recorded here while it is
still being designed: *an offset that was ever a record start must remain a still being designed: *an offset that was ever a record start must remain a
record start.* `doc_bytes` reads a `u32` length prefix in place, so an record start.* `doc_bytes` reads a `u32` length prefix in place, so an