PLAN D2 makes the official specification suites the gate for command semantics;
D7.6 asks for the harness to exist at M0 with a recorded baseline. This is that
harness, pinned on both sides -- mongodb/specifications @ 615e0f9 and
mongodb@7.5.0 -- because a scorecard is only comparable across milestones if a
delta cannot be an upstream test change.
It implements the unified format's Evaluating Matches algorithm as written,
including the two rules that decide whether a pass is earned: extra keys are
tolerated only in a root document, and numeric types compare flexibly. Anything
unimplemented is a SKIP with a reason, never a pass, and the one assertion class
not yet checked -- expectEvents, i.e. command monitoring -- is disclosed at the
top of the scorecard so `pass` reads as an upper bound.
First honest run: 131 pass, 161 fail, 195 skip over 175 files, zero timeouts.
Getting there took four attempts, and the failures are documented in the README
because each would have shipped a scorecard claiming a compatibility gap that
did not exist. Two were genuine leaks in this runner (clients left open when a
case timed out; clients registered for cleanup only after `await connect()`,
plus abandoned cases still creating more). The third I misdiagnosed as machine
load. The fourth attempt found the real cause: a leaked catalog lock in the
engine, fixed separately, which alone accounts for the jump from 45 passes to
131.
So the runner carries its own guards: per-operation CSOT timeouts so work is
never abandoned, an active-handle census per file, an end-of-run tripwire for
stray timers, a hard stop if the server dies rather than emitting hundreds of
misleading ECONNREFUSED failures, and --skip/--limit for bisecting a run whose
failures depend on position. The README states the rule plainly -- a long
unbroken tail of timeouts is a harness bug until proven otherwise -- and the two
commands that settle it.
Also fixes bench-run.sh, which copied its report over bench-latest.txt
unconditionally, including after a run that only warned -- so a degraded run
could silently replace the baseline that PLAN D7.5 makes a milestone gate.
_id uniqueness was a `coll.docs.contains` probe. The docs hashmap is going
away (PLAN A3), so it has to move to the _id_ tree -- and the tree answers
better, because it is keyed on bson.encode_key, which is canonical where
serialize_value is not. int32 1, int64 1 and double 1.0 are now one _id, as
they are in MongoDB (A4).
_id_ is built and checked first, so a write violating both it and a unique
secondary reports _id_, which is what MongoDB reports. It returns
error.DuplicateKey with `dup_index` left null, which is exactly what
commands.zig's E11000 rendering already treats as "the _id_ index", so the
wire-visible message is unchanged and that file needed no edit.
check_unique's exclude-self became optional and is null on an insert. That was
a latent bug of its own: a replace must ignore its own existing entries, but an
insert has none, and passing the document's id there hides a collision whose
entry carries that same id -- precisely the case _id_ exists to catch. Only
_id_ could reach it, since a secondary collision is between different
documents.
Two corrections found while doing this, both worth reading:
PLAN A4 claimed a database already holding {_id: int32 1} and {_id: int64 1}
loses one on reopen. It does not. Replay evicts through the docs map, keyed on
serialize_value, so both survive; the tree is bulk-built afterwards with
enforcement off, which tolerates duplicate keys and warns. The loss arrives
only with the commit that drops the map, and that is where it needs a
pre-flight scan. Amended.
dispatch_insert asserted only `ok: 1`, but a rejected document comes back as a
writeError alongside it -- so the mixed-type corpus silently shrank from ten
documents to nine when _id_ became unique, and every test over it still passed.
The helper now rejects writeErrors and asserts the inserted count; it caught
the shrink immediately. The corpus keeps an int64 _id on a distinct value, and
the collision it used to stand in for is asserted directly.
Also adds Index.lookup_exact, which the commands that currently probe the docs
map will need. Exact byte equality rather than cmp_prefix, because {a: 1}'s
encoding is a proper prefix of {a: 1, b: 2}'s and a prefix match would claim a
document is present when it is not.
Mutation-checked, all three red: unique=false on id_index; exclude=id_key on
insert; eql -> cmp_prefix in lookup_exact.
The M0 implementation review found three of D1-D9 wrong or incomplete. The
originals stay in place with pointers to a new amendments section, so a later
session can see what changed rather than reading a rewritten history.
A1: D4 as written is unsound. The doc and overflow slabs are append-only, so
replay repairs them, but B+tree node pages are mutated in place -- after a
crash the file holds an arbitrary mix of written-back and not-written-back
pages, and once D6.3 truncates the log the data file is the only copy below
the watermark. A half-persisted tree is unrecoverable. So the checkpoint needs
shadow paging: no page below the last watermark's allocation mark is ever
stored into, and the watermark write is the atomic switch. Knock-on: node ids
cannot be page numbers, because Node.parent/next/prev are back-pointers by id
and copy-on-write would cascade; an in-RAM id->page table per index keeps every
persisted id at its current width and gives COW one pointer to fix.
A2: follows from A1 -- a page free list is a prerequisite, not the
defense-in-depth D6.2 assumed, because COW abandons every page it touches in
every epoch. The churn gate stays, retargeted at document garbage.
A3: section 5 step 4's trap was misidentified. store_record already copies
keys, so "entries must own their key bytes" is work that does not need doing.
The real problem is that a leaf record has nowhere to put a slab offset; the
resolution is to make the payload that offset, in every index, and delete
Entry.id rather than re-own it.
A4: making _id_ a unique index keys uniqueness on the canonical encode_key
rather than serialize_value, so int32 1 / int64 1 / double 1.0 collide as they
do in MongoDB -- a compatibility improvement, with a documented one-way
migration hazard for a database that already holds two such documents.
Also records that Engine.seq is never restored on open (harmless today, silent
data loss once a watermark exists), and two bugs the new spec harness found.
AGENTS.md carries the same rules into the operating guide: ground rules grow
from 7 to 9, and the old rule 6 is corrected with a note saying why.
