M1: doc-level free list, sessions, and a spec runner that no longer overstates #1

Merged
dev merged 37 commits from m1-cursors into main 2026-08-09 16:15:34 +00:00
Showing only changes of commit e65da2740e - Show all commits

73
PLAN.md
View File

@@ -721,6 +721,70 @@ between `compact` and `checkpoint`; it is out of this scope because it wants
its own design pass, and because the free list must not add a second instance
of the same shape.
### The spec runner starts reading `expectEvents`
354 of the 487 cases declare `expectEvents` and the runner read none of them,
so a case could send the wrong command entirely and still be counted a pass as
long as the result came back right. The old `pass` column was an upper bound by
construction and said so; it is now an assertion that the engine answered
correctly **and** was asked the right question. **Scorecards recorded before
this are not comparable with ones recorded after.**
The totals moved 168/124/195 → 194/97/196 across the commits, but the path
matters more than the endpoints: turning the assertion on cost 34 passes, and
every one of them was a defect the result column could not see.
What it found, in the order it found them:
1. **The runner dropped `collectionOptions`.** Every collection entity was
built as `db.collection(name)`, so the 15 entities declaring
`writeConcern: {w: 0}` never got it — **every "unacknowledged write" case in
the corpus was running an acknowledged write.** They passed because the two
produce results a `$$unsetOrMatches` expectation accepts either way. Only
the command on the wire distinguished them, and nothing read the command.
2. **The wire version disagreed with the version string.** `buildInfo` said
4.4.0, the handshake said maxWireVersion 8, which is 4.2. A driver believes
the wire version: it refused *client-side* to send `hint` on an
unacknowledged delete or findAndModify, and withheld `comment` from
getMore, listCollections and listDatabases. 16 cases. The 8 was not
arbitrary — it was tied to keeping drivers off the streaming hello protocol
— but that turned out to rest entirely on omitting `topologyVersion`, which
is checked in the driver and is the whole mechanism. A test now asserts the
two numbers agree, since drifting apart silently was the actual defect.
3. **`$$unsetOrMatches` was changing root-ness.** The operator wraps a value,
it does not reposition it; the runner matched what stood behind it as a
nested document. 25 cases, all of them results the engine had right.
4. **An event's command is not the shape the driver sends.** A sort is held as
a JS `Map`, so `Object.keys` on it is empty and every expected key read as
missing. It hides well: EJSON prints a Map exactly like a document, so the
event dump reads as evidence the matcher is wrong about something else.
Two assertions are declined, both enumerated in the runner and in
`scorecard.txt`, and neither can hide anything the engine did:
- **`maxTimeMS`** — the harness's own doing. Every client carries CSOT
`timeoutMS`, which overwrites `maxTimeMS` with the remaining budget, so the
value on the wire is ours. Refused unconditionally rather than only when it
would fail, so it cannot become a pass by coincidence. One case, and dropping
`timeoutMS` instead would cost far more — it is what replaced the outer race
that once produced ~190 phantom timeout FAILs.
- **`cmap`/`sdam` event types, `ignoreExtraEvents`, `hasServiceId`,
`hasServerConnectionId`** — none occurs in this corpus; reported unsupported
where asserted rather than waived.
**Left failing on purpose: `bypassDocumentValidation: false`, 4 cases.**
mongodb@7.5.0 strips the field unless it is exactly `true` on the bulk and
findAndModify paths (`lib/bulk/common.js:292`,
`lib/operations/find_and_modify.js:19`) while sending it correctly for single
-document operations, so 4 sibling cases pass and 4 fail on a difference that
is entirely the driver's. The field is built client-side and never reaches the
engine. A refusal was written and thrown away: made unconditional it also
skipped the 4 that legitimately pass, and made conditional it would be a
skip-when-it-would-fail rule, which is the shape that turns a scorecard into
flattery. Four undeserved entries in the fail column is the cheaper error, and
this note is the correction. Revisit when the driver is bumped — which already
has to be its own commit with its own re-recorded scorecard.
---
## 6. Deferred designs (grill each at its milestone)
@@ -759,10 +823,11 @@ of the same shape.
the anchor rewritten — resuming at it returned updated documents twice,
caught by draining a collection being updated underneath.
Still open in M1: the doc-level free list, sessions plumbing (`lsid`
accepted), and command-monitoring (`expectEvents`) in the spec runner. The
eight reclamation bugs above were cleared first, as preconditions for the
free list rather than as work of their own.
Still open in M1: the doc-level free list and sessions plumbing (`lsid`
accepted). The eight reclamation bugs above were cleared first, as
preconditions for the free list rather than as work of their own;
command-monitoring (`expectEvents`) landed next, so that the free list and
sessions are measured by an instrument that is no longer known to overstate.
**A prerequisite the free list must honour**, recorded here while it is
still being designed: *an offset that was ever a record start must remain a
record start.* `doc_bytes` reads a `u32` length prefix in place, so an