AGENTS.md

@@ -13,9 +13,11 @@ indexes, and TTL/unique/sparse/compound index support.
 
 **Forward plan**: the project's direction — a full-fledged embedded,
 tens-of-GB, maximally MongoDB-compatible database — is decided and written
-in **[PLAN.md](PLAN.md)**. The current milestone is **M0 (mmap + WAL
+in **[PLAN.md](PLAN.md)**. M0 (mmap + WAL storage foundation) has landed;
-storage foundation)**. Before starting any work, read PLAN.md; its decision
+the current milestone is **M1 (cursors + wire polish)**, whose cursor work is
-record (D1-D9) and ground rules are binding.
+done — see `src/cursor.zig` and `tests/e2e/e2e7.js`. Before starting any
+work, read PLAN.md; its decision record (D1-D9) and ground rules are
+binding.
 
 ## Read first, in order
 
@@ -71,6 +73,7 @@ node tests/e2e/e2e2.js crash-b # restart, verify all 50 survived
 node tests/e2e/e2e3.js         # secondary indexes
 node tests/e2e/e2e4.js         # TTL indexes (server must run --ttl-sweep-secs 1)
 node tests/e2e/e2e6.js         # self-contained full lifecycle (spawns its own server, incl. kill -9)
+node tests/e2e/e2e7.js         # self-contained cursors (spawns its own servers; needs no server running)
 ```
 
 Which suites to run for a given change:
@@ -78,6 +81,7 @@ Which suites to run for a given change:
 - anything touching the write path or log format → the crash pair
   (e2e2 crash-a/b) and e2e6
 - anything touching indexes → e2e3.js and e2e4.js
+- anything touching cursors, batching or the reply size → e2e7.js
 - everything → all of the above
 
 `tests/e2e/README.md` has the full matrix, ports, and harness docs
@@ -180,7 +184,8 @@ src/server.zig   TCP accept loop, per-connection handlers, TTL sweep monitor
 src/db.zig       engine: db → collection → _id → document maps, slab storage
 src/storage.zig  append-only log: blocks, LZ4, XxHash3, replay, compaction
 src/query.zig    filter matcher, regex engine, sort, projection
-src/index.zig    B+tree indexes: entries, search, query planner
+src/index.zig    B+tree indexes: entries, search, query planner, scan resume
+src/cursor.zig   server-side cursor state: registry, batch policy, expiry
 src/update.zig   update operators with dot-path navigation
 src/main.zig     CLI: --port, --bind, --db, --ttl-sweep-secs, --compact-threshold
 ```
@@ -193,6 +198,8 @@ src/main.zig     CLI: --port, --bind, --db, --ttl-sweep-secs, --compact-threshol
    baseline.
 3. Commit scorecard and benchmark results with each milestone (PLAN D9) so
    progress stays verifiable across sessions.
-4. Deferred designs (cursors, aggregation, transactions, change streams,
+4. Deferred designs (aggregation, transactions, change streams, C API) are
-   C API) are deliberately *not* specified yet — grill the design with the
+   deliberately *not* specified yet — grill the design with the human before
-   human before implementing (PLAN section 6).
+   implementing (PLAN section 6). Cursors are no longer among them: the
+   design was settled and implemented in M1, and `src/cursor.zig`'s module
+   comment is where it is written down.

PLAN.md

@@ -320,6 +320,80 @@ takes the first one's place. That commit is where this needs handling — a
 pre-flight scan for compare-equal `_id`s, refusing to drop the map silently
 while any exist — not here.
 
+### Amendment A5 — the doc-level free list, and what it did not fix (amends A2, closes D7.4)
+
+D7.4 left M0 with a bound rather than a target: 1.65× delete-heavy, 2.47×
+update-heavy against a hoped-for ~1.3×, and the stated conclusion that
+doc-level free lists were an M1 item. They are built. The mechanism is
+measured, it works, and **the steady-state ratio did not move**. Both halves
+of that are the amendment.
+
+**What was built.** A collection's slab carries a dense map of dead bytes per
+`map_align` window — two bytes per window, so 2.7 MB for a 21 GB slab — and a
+checkpoint hands back every window with nothing live left in it, splitting the
+runs around what is kept. The window is the unit because it is the smallest
+thing that can be given back at all: `mark_appendable` refuses an unaligned
+start and `protect_stable` rounds outwards. Counting is the whole liveness
+test, because `evict_doc` removes a document's index entries before marking its
+bytes dead, so "no live bytes in this window" and "nothing references these
+bytes" are the same statement. Reclamation lives inside `checkpoint` rather
+than beside it so that the run split and the `free_pages` become durable under
+one `publish`; there is no new record type, no new catalog version and no
+replay path. `alloc_slab_run` is a second policy in the same allocator, because
+`take_free`'s best fit — which exists to stop one-page copy-on-write requests
+dismantling the extents — can never match a request for 2048 pages against
+runs that come back a few windows at a time.
+
+**What it does not fix, and why no threshold reaches it.** The data file never
+shrinks, so `file / live` is a high-water mark, and the mark is set once by the
+one thing reclamation cannot avoid: a rebuild needs a whole second copy of the
+live data before the first can be freed. Live + garbage-at-trigger + copy is
+the peak, and it is reached in the first round, before any free pool exists to
+build the copy out of. Rebuilding earlier lowers the garbage term and nothing
+else; rebuilding later raises it. So ~2× is the floor of a rebuild-based
+design, and tuning is the wrong instrument. Measured occupancy tells the other
+half of the story: 1.06–1.26× in use against a 2.46× file, with 934 MB
+reclaimed over the run.
+
+**The successor, named here so the next session does not re-derive it.**
+Incremental compaction through a doc-id → offset indirection layer, which is
+rejected option (b) of the M1 design, promoted. It is the only thing that
+removes the second copy: a rebuild becomes a move of one document at a time
+with the map updated behind it. The cost is the one that got it rejected — the
+map has to be persistent and crash-safe, i.e. a second copy-on-write B+tree per
+collection and a second random read on the point-lookup path — and it undoes
+A3. That is a milestone, not a knob. Second and cheaper: return free space to
+the filesystem, since 52% of the steady-state file is space the database owns
+and is not using; it needs the file never to shrink below what the fallback
+generation references, which is its own crash-safety design pass.
+
+**One planned commit was declined on the measurement.** A same-length replace
+could be written in place when the document's bytes are on an unpublished page,
+and the plan made it conditional on the update line still being the worst. It
+is the worst — but the measurement says why, and it is not write amplification:
+the peak is set by the first rebuild's second copy, so generating less garbage
+moves the number by nothing. It would buy a write-path change, on the path that
+runs after the log record is durable, for no gate movement. Left undone with
+the reason, rather than done because it was listed.
+
+**Small documents behave exactly as forecast**, and the forecast being written
+down in advance is what makes it a result. 200-byte documents reclaim nothing
+at all — a 16 KiB system page holds ~70 of them and they never all die at once
+— and the counters show a mechanism correctly doing nothing rather than one
+misfiring. The payoff scales as `doc_size / map_align`, so 4 KiB pages read
+four times better on the same code.
+
+**One thing the gate found that the design had not.** A checkpoint is what
+reclaims and a checkpoint is armed by log volume, but a delete logs only an
+`_id`. Deleting half a 190 MB collection moved the log by a couple of megabytes
+so no checkpoint ran, the garbage sailed past the rebuild threshold, and the
+rebuild reset the window map it would have used — six rounds, six rebuilds,
+1 MB reclaimed. `compact` now checkpoints before it copies, which is also the
+right order on its own terms: the cheap half of the job first, and the
+per-collection gate judges what reclamation left. Same six rounds, 256 MB.
+Numbers and reproduction in `tests/e2e/results/m0-gates.txt` under `[M1.1]` and
+`[M1.2]`.
+
 ---
 
 ## 3. Milestones and gates
@@ -631,13 +705,270 @@ The pattern worth noting for M1: both were found by *running* the suites, not by
 reading them, and the second was only visible because the first stopped masking
 it.
 
+### Before the free list: eight bugs the M1 design work turned up
+
+The doc-level free list multiplies traffic through exactly the reclamation
+paths, so those paths were read closely before anything was built. Three of the
+eight were found that way, by reading. Five were found by the tests written for
+the other three — the same lesson M0's gate taught, arriving one milestone
+early: a reclamation bug is invisible until something runs long enough, or
+concurrently enough, to reach the state that exposes it.
+
+1. **`write_freelist` counted its entries before allocating its own pages.**
+   The allocation goes through `take_free`, which `swapRemove`s an exact-fit
+   entry, so the loop wrote one entry fewer than the count it had already
+   committed to and the hash landed eight bytes short. `read_freelist` then
+   declared the list corrupt and dropped all of it. A one-page freelist stream
+   and a one-page hole are both the common case, so this fired at essentially
+   every reopen: the free list has been discarded on restart since it existed.
+
+2. **The free lists were read and written with no lock.** `free_pages` appended
+   to `free_pending` while `publish` rotated the three lists under `alloc_lock`,
+   and `write_freelist` walked all three while a concurrent `free_pages` could
+   reallocate them. Found by the test written for (1).
+
+3. **A checkpoint never gave back the generation it replaced.** The catalog
+   stream and the freelist stream are allocated fresh every publish and were
+   never freed, so an idle server grew forever.
+
+4. **A checkpoint could publish a watermark above the durable log tail.** The
+   snapshot's `seq` check does not catch a writer that appended *before* the
+   walk started and has not committed yet; the window is as wide as an fsync.
+   This was an assertion, so the failure mode was a server abort under exactly
+   the load that makes checkpoints frequent — and without the assertion it is
+   the loss of an acknowledged write, since the truncation that follows a
+   checkpoint would discard the record. Now a retry: seal and re-snapshot.
+
+5. **A document append could land in the published image.** `slab_reserve`
+   checks that the append cursor is writable; `publish` can clear the
+   unpublished set between that check and `slab_append`'s copy, because the log
+   append and its fsync sit in between. SIGBUS where `protect_stable` is
+   compiled in, a silent overwrite of durable data in ReleaseFast, where it is
+   not. Fixed with a pager-level append lock, held shared by appenders and
+   exclusively by `publish`; measured at no cost on the write path (8 clients ×
+   1500 inserts at `{w:1,j:true}`: 23779–24879 docs/s before, 23823–24452
+   after).
+
+6. **`write_catalog` read `slab_extents` under only the shared catalog lock**,
+   while `slab_reserve` appended to that ArrayList under the collection's.
+
+7. **The engine's counters were shared by writers holding no lock in common.**
+   `live_docs`, `dead_docs`, `live_bytes` and `dead_bytes` are updated by a
+   writer holding its own collection's lock and the catalog's shared — so two
+   writers on different collections lose each other's updates, and a reader had
+   no way to see the totals consistently with the per-collection figures they
+   are supposed to equal. Now `counter_lock`, a leaf, with a `Counters`
+   snapshot for the two readers that compare them. The only one of the eight
+   that is **not** mutation-checked: it aborted three of eight ReleaseSafe runs
+   once (8) made the checkpoint's consistency check reachable, and then would
+   not re-trigger in 34 further runs, on the reverted fix and on the pre-fix
+   revision alike. The rate depends on machine load. It stands on inspection,
+   and the concurrency test now asserts the identity once everything is quiet
+   rather than relying on catching the race in the act.
+
+8. **The slab did not count what the appender skips.** `slab_used` only ever
+   grew by a document's length, so the gap left when a checkpoint pushes the
+   append cursor up to a system page, and the tail of an extent abandoned for a
+   document that no longer fits, were counted nowhere — real garbage,
+   invisible to the trigger that decides whether a rebuild is worth doing, and
+   part of the 1.65×/2.47× the churn gate measured.
+
+Accounting is now an identity rather than four independent counters:
+`dead_bytes` is the sum of `slab_used - live_bytes` over the collections that
+exist. `read_catalog` recomputes it on open instead of trusting the watermark's
+hint, `compact` recomputes it from what a rebuild leaves behind instead of
+zeroing it, and `write_catalog` returns both sums for the checkpoint to assert
+under the quiescence condition it already had. That assertion is what surfaced
+(7) and what mutation-checks (5) and (8).
+
+Two of the eight — the drop that charged a dropped collection's live bytes to
+`dead_bytes`, and the counters — also changed what the watermark is for: its
+`dead_bytes` field is now a hint for anything inspecting the header, not a
+source of truth, because a collection dropped after the last checkpoint is gone
+from the catalog and would still be charged for in the hint.
+
+**Still open, deliberately.** `rebuild_collection` frees the pages it abandoned
+while holding only the collection's lock, and a concurrent `checkpoint` may
+already have snapshotted a catalog that claims them. The `seq` retry does not
+see it, because a rebuild appends no log record. The fix is mutual exclusion
+between `compact` and `checkpoint`; it is out of this scope because it wants
+its own design pass, and because the free list must not add a second instance
+of the same shape.
+
+*Where this stands after the free list.* It did add a second instance —
+reclamation frees pages as a checkpoint phase, and two checkpoints can be in
+flight — so that half is closed: `checkpoint` takes a lock of its own. Two
+things came out of doing it. The publish was never the exposure, because it
+already runs under `log_lock`; and the whole class is now *detectable* rather
+than only arguable, because `write_catalog` asserts per run that the pager has
+not already been given it, in test and Debug builds. That assertion is proven
+to fire. The original instance — `compact`'s rebuild walk against a concurrent
+checkpoint — is unchanged and still wants the design pass.
+
+### The spec runner starts reading `expectEvents`
+
+354 of the 487 cases declare `expectEvents` and the runner read none of them,
+so a case could send the wrong command entirely and still be counted a pass as
+long as the result came back right. The old `pass` column was an upper bound by
+construction and said so; it is now an assertion that the engine answered
+correctly **and** was asked the right question. **Scorecards recorded before
+this are not comparable with ones recorded after.**
+
+The totals moved 168/124/195 → 194/97/196 across the commits, but the path
+matters more than the endpoints: turning the assertion on cost 34 passes, and
+every one of them was a defect the result column could not see.
+
+What it found, in the order it found them:
+
+1. **The runner dropped `collectionOptions`.** Every collection entity was
+   built as `db.collection(name)`, so the 15 entities declaring
+   `writeConcern: {w: 0}` never got it — **every "unacknowledged write" case in
+   the corpus was running an acknowledged write.** They passed because the two
+   produce results a `$$unsetOrMatches` expectation accepts either way. Only
+   the command on the wire distinguished them, and nothing read the command.
+2. **The wire version disagreed with the version string.** `buildInfo` said
+   4.4.0, the handshake said maxWireVersion 8, which is 4.2. A driver believes
+   the wire version: it refused *client-side* to send `hint` on an
+   unacknowledged delete or findAndModify, and withheld `comment` from
+   getMore, listCollections and listDatabases. 16 cases. The 8 was not
+   arbitrary — it was tied to keeping drivers off the streaming hello protocol
+   — but that turned out to rest entirely on omitting `topologyVersion`, which
+   is checked in the driver and is the whole mechanism. A test now asserts the
+   two numbers agree, since drifting apart silently was the actual defect.
+3. **`$$unsetOrMatches` was changing root-ness.** The operator wraps a value,
+   it does not reposition it; the runner matched what stood behind it as a
+   nested document. 25 cases, all of them results the engine had right.
+4. **An event's command is not the shape the driver sends.** A sort is held as
+   a JS `Map`, so `Object.keys` on it is empty and every expected key read as
+   missing. It hides well: EJSON prints a Map exactly like a document, so the
+   event dump reads as evidence the matcher is wrong about something else.
+
+Two assertions are declined, both enumerated in the runner and in
+`scorecard.txt`, and neither can hide anything the engine did:
+
+- **`maxTimeMS`** — the harness's own doing. Every client carries CSOT
+  `timeoutMS`, which overwrites `maxTimeMS` with the remaining budget, so the
+  value on the wire is ours. Refused unconditionally rather than only when it
+  would fail, so it cannot become a pass by coincidence. One case, and dropping
+  `timeoutMS` instead would cost far more — it is what replaced the outer race
+  that once produced ~190 phantom timeout FAILs.
+- **`cmap`/`sdam` event types, `ignoreExtraEvents`, `hasServiceId`,
+  `hasServerConnectionId`** — none occurs in this corpus; reported unsupported
+  where asserted rather than waived.
+
+**Left failing on purpose: `bypassDocumentValidation: false`, 4 cases.**
+mongodb@7.5.0 strips the field unless it is exactly `true` on the bulk and
+findAndModify paths (`lib/bulk/common.js:292`,
+`lib/operations/find_and_modify.js:19`) while sending it correctly for single
+-document operations, so 4 sibling cases pass and 4 fail on a difference that
+is entirely the driver's. The field is built client-side and never reaches the
+engine. A refusal was written and thrown away: made unconditional it also
+skipped the 4 that legitimately pass, and made conditional it would be a
+skip-when-it-would-fail rule, which is the shape that turns a scorecard into
+flattery. Four undeserved entries in the fail column is the cheaper error, and
+this note is the correction. Revisit when the driver is bumped — which already
+has to be its own commit with its own re-recorded scorecard.
+
 ---
 
 ## 6. Deferred designs (grill each at its milestone)
 
-- **M1 cursors**: cursor id allocation, idle expiration, batchSize
+- **M1 cursors** — *settled and implemented.* The design lives in
-  semantics, getMore against a lagging/compactable engine, cursor state
+  `src/cursor.zig`'s module comment; the decisions it records, and how each
-  lifecycle across compaction.
+  was reached:
+  - **Cursor ids** are `(nonce << 20) | slot`, always positive, never 0. The
+    nonce is not decoration: without it a recycled slot serves one client
+    another's documents, which is the worst failure this feature could have.
+  - **batchSize semantics** were *measured against mongod 8.3.7*, not
+    recalled, and three assumptions were wrong: a bare `getMore` does **not**
+    inherit the find's batchSize (4998 of 5000 documents come back), a
+    namespace mismatch is `Unauthorized` (13) rather than `CursorNotFound`,
+    and `CursorInUse` is 143 rather than the 12051 an earlier note claimed.
+    `internalQueryFindCommandBatchSize` is 101, `cursorTimeoutMillis`
+    600000, `clientCursorMonitorFrequencySecs` 4.
+  - **Never look ahead**: a batch that met its target leaves the cursor open
+    even when the source is in fact exhausted, so four documents at
+    `batchSize: 2` take three commands. The pinned suites assert that count.
+  - **Idle expiration** is a second monitor fiber, separate from the TTL one:
+    the cadences differ by an order of magnitude, and a TTL sweep failure
+    must not stop cursors being reclaimed. The registry is fixed-capacity and
+    evicts the least recently used cursor, whose client sees the same
+    `CursorNotFound` an idle timeout gives.
+  - **Against a lagging/compactable engine**, what survives depends on what
+    the cursor remembers, so the check is per-source: a repack changes no
+    key, so a streaming cursor resumes; slab offsets all move, so an offsets
+    cursor is killed with `QueryPlanKilled`; a snapshot needs no collection at
+    all. `Collection.layout_epoch` and `Index.epoch` are the tokens.
+  - **Resume** anchors on `(key, off)` plus a position hint gated on
+    `Index.epoch`, with an exact-order band walk bounded by
+    `resume_walk_max`. Two hazards found while implementing: a deleted anchor
+    must resume at its band position or the rest of an equal-key band is
+    silently dropped, and on a *unique* index a same-key entry can only be
+    the anchor rewritten — resuming at it returned updated documents twice,
+    caught by draining a collection being updated underneath.
+
+  The doc-level free list is built; see amendment A5 for what it did and did
+  not achieve, and `[M1.1]`/`[M1.2]` in the results file for the numbers. The
+  eight reclamation bugs above were cleared first, as preconditions for it
+  rather than as work of their own; command-monitoring (`expectEvents`) landed
+  next, so that what followed is measured by an instrument no longer known to
+  overstate.
+  **The prerequisite it had to honour** — *an offset that was ever a record
+  start must remain a record start*, because `doc_bytes` reads a `u32` length
+  prefix in place and an offsets cursor holds exactly such offsets — is met
+  structurally rather than by checking: a window is handed back only when every
+  byte in it is dead, which means every document touching it has already been
+  through `evict_doc` and out of every index. What remains is the cursor
+  holding a *saved* offset list, and that is answered the way a rebuild answers
+  it, by bumping `layout_epoch` when and only when a collection actually gave
+  something back.
+- **M1 sessions** — *settled and implemented.* `lsid` is parsed, validated and
+  deliberately acted on in no way; `txnNumber`, `startTransaction` and
+  `autocommit` are refused; `endSessions` validates the array it discards.
+  Every code and message was **measured against mongod 8.3.7** with a raw
+  OP_MSG probe, because the driver overwrites `lsid` with its own session and a
+  malformed one cannot be sent through it. Three measurements contradicted the
+  design they were checking:
+  - **Unknown fields inside `lsid` are rejected** (IDLUnknownField 40415). The
+    design said to tolerate them, reasoning that the server tolerates unknown
+    fields everywhere. It does not, here.
+  - **`uid` is accepted** — the hash of the credentials owning the session,
+    which a driver sends as soon as authentication is on. Rejecting it would
+    have broken every command in M7.
+  - **An unknown command with a malformed `lsid` answers CommandNotFound**, so
+    command lookup precedes session validation, which is where the check sits.
+
+  The refusals, so M4 does not reopen them:
+  - **No session registry.** A session here would own nothing: no transactions
+    to scope, no retryable writes (a driver disables them for a standalone),
+    and cursors that outlive their connection for reasons of their own. It
+    would be a mutex on the dispatch path guarding state nothing reads. M4's
+    transaction state machine gets to say what shape it needs.
+  - **`lsid` is not echoed.** Measured: mongod answers a well-formed one with
+    exactly `{ok: 1}`. A driver reads only `$clusterTime` and `operationTime`
+    back, and a standalone sends neither — correctly, since without
+    `operationTime` there is nothing for `afterClusterTime` to attach to and
+    causal consistency stays off.
+  - **`startSession` and `refreshSessions` are not implemented.** Both are real
+    mongod commands, but a driver calls neither — it generates session ids
+    locally — so CommandNotFound is the honest answer. Candidates for M4.
+
+  Five divergences from mongod remain, all deliberate. Three share one cause:
+  mongod keeps a per-command table of which commands accept `txnNumber` at all
+  and answers Location50889 or OperationNotSupportedInTransaction 263 for those
+  that do not, *before* reaching the standalone refusal. We have no such table
+  and give the standalone answer uniformly, so replies are identical for every
+  CRUD command — everything a driver would send these fields on — and differ
+  only on things like `ping`, where mongod is more specific rather than
+  differently right. The other two are `startSession`/`refreshSessions` above,
+  with `commitTransaction` alongside them, reachable only by a client whose
+  write this server has already refused.
+
+  **Effect on the scorecard: exactly zero, and that was the prediction.**
+  194/97/196 before and after. The corpus has no `session` entity, no operation
+  taking a `session` argument, and no `lsid` assertion. The value here is
+  protocol hygiene, not a number — and after Stage 1 a wrongly-refused command
+  would have shown up as a changed event stream rather than silently.
 - **M2 aggregation**: stage/expression tiers, which spec-test files are
   the gate, whether $lookup/$unwind/facet make the first cut.
 - **M4 transactions**: snapshot isolation over mmap (COW vs undo), read

README.md

@@ -12,8 +12,9 @@ maximally MongoDB-compatible database — its decision record, milestones
 and gates live in [PLAN.md](PLAN.md). Milestone 0 (mmap + WAL storage
 foundation) has landed; its measured gate results are in
 [`tests/e2e/results/m0-gates.txt`](tests/e2e/results/m0-gates.txt).
-Milestone 1 (cursors, and the doc-level free list the churn gate showed is
+Milestone 1 is in progress: server-side cursors have landed (see
-needed) is next.
+**Cursors** below); the doc-level free list the churn gate showed is needed
+is still open.
 
 ## Quick start
 
@@ -145,10 +146,43 @@ whole pass, so the interval is the tuning knob: expiry is never more
 precise than `--ttl-sweep-secs`, and a very large TTL index wants a
 longer one.
 
+## Cursors
+
+`find`, `aggregate`, `listCollections` and `listIndexes` return real cursor
+ids, and `getMore`/`killCursors` work. Batching follows MongoDB: a first
+batch of 101 documents unless `batchSize` says otherwise, a `getMore` with
+no `batchSize` bounded only by the 16 MiB batch cap, `batchSize: 0` as an
+empty batch with a live cursor, and `limit` honoured across batches. Every
+default here was measured against a real `mongod` rather than assumed.
+
+A cursor holds no lock between requests, so what it remembers has to survive
+arbitrary concurrent writes. Three shapes, picked by the query:
+
+| query | what the cursor keeps |
+| --- | --- |
+| a whole-index walk (`find({})`, or a sort an index provides) | the last key and offset it yielded — O(key), whatever the collection size |
+| a narrowed index plan | the matching offsets, 8 bytes each |
+| a sort no index provides, or aggregate/listing output | a snapshot of the remaining documents |
+
+The first is what lets a cursor walk a collection larger than memory. It
+also survives a compaction, because a repack changes no key; the offsets
+form cannot, and says so with `QueryPlanKilled` rather than returning
+documents from the wrong place.
+
+Cursors are not pinned to the connection that created them, so a `getMore`
+may arrive on any connection — which is what the driver specification
+allows. They are reclaimed when exhausted, when killed, or after
+`--cursor-timeout-ms` idle (default 600000, MongoDB's own
+`cursorTimeoutMillis`); `--max-open-cursors` bounds the registry and evicts
+the least recently used cursor at capacity, whose client then sees the same
+`CursorNotFound` an idle timeout gives.
+
 ## Not (yet) implemented
 
 - Authentication (SCRAM) — run without credentials
-- Real cursors (all results are returned in one batch, cursor id 0)
+- Tailable/awaitData cursors, which need capped collections; a tailable
+  `find` is rejected, exactly as MongoDB rejects one on a non-capped
+  collection
 - Transactions, change streams, replicasets
 - Compression (OP_COMPRESSED)
 - `collMod`, so an index's `expireAfterSeconds` cannot be changed in

src/bson.zig

@@ -74,6 +74,35 @@ pub const Value = union(enum) {
         };
     }
 
+    /// The name mongod uses for this type in a TypeMismatch message ("is the
+    /// wrong type 'int', expected type 'object'"). Its own names, not Zig's:
+    /// a driver that matches on the text is matching on these.
+    pub fn type_name(self: Value) []const u8 {
+        return switch (self) {
+            .double => "double",
+            .string => "string",
+            .doc => "object",
+            .array => "array",
+            .binary => "binData",
+            .object_id => "objectId",
+            .bool => "bool",
+            .datetime => "date",
+            .null => "null",
+            .regex => "regex",
+            .code => "javascript",
+            .symbol => "symbol",
+            .int32 => "int",
+            .timestamp => "timestamp",
+            .int64 => "long",
+            .decimal128 => "decimal",
+            .min_key => "minKey",
+            .max_key => "maxKey",
+            // An unparsed value keeps only its tag byte, and the tags this
+            // union does not name are the ones nothing here inspects.
+            .opaque_val => "unknown",
+        };
+    }
+
     pub fn is_number(self: Value) bool {
         return switch (self) {
             .double, .int32, .int64 => true,

src/cursor.zig

@@ -0,0 +1,979 @@
+//! Server-side cursor state: what a `find`/`aggregate` leaves behind so a later
+//! `getMore` can carry on, and the fixed-capacity registry that holds it.
+//!
+//! This module is deliberately *pure*: it owns state and policy, never
+//! execution. It does not import `db.zig` or `commands.zig`, so `db.Engine` can
+//! own a `Store` with no import cycle, and the batch policy below is testable
+//! with no engine, no socket and no allocator. Filling a batch stays in
+//! `commands.zig`, which already owns orchestration the way `index.zig` owns
+//! planning.
+//!
+//! **The one rule the whole batching protocol follows: never look ahead.** A
+//! batch ends either because it reached its target -- and the cursor stays open
+//! -- or because the source reported EOF, and then the cursor closes with
+//! `id: 0` in that same reply. A batch that reached its target leaves the cursor
+//! open *even when the source happens to be exhausted*. So four documents at
+//! `batchSize: 2` need a third command answering `nextBatch: []` with `id: 0`;
+//! that empty terminal batch is correct, not a bug, and the pinned spec suites
+//! assert exactly that command count.
+//!
+//! ## What a cursor is allowed to remember
+//!
+//! A cursor holds no lock between requests, so everything it saves must survive
+//! arbitrary concurrent mutation. Nothing here is a pointer, and the two things
+//! that look like stable addresses are not:
+//!
+//!   - A tree position `(leaf, slot)` is invalidated by `Index.reset_tree`,
+//!     which clears the node table so ids 0 and 1 become a live but *unrelated*
+//!     root and leaf. Guarded by `Stream.index_epoch`.
+//!   - A slab offset is invalidated by `rebuild_collection`, which moves every
+//!     document. Guarded by `layout_epoch`.
+//!
+//! Both are checked as error returns rather than assertions, because a client
+//! can reach either one by keeping a cursor open across maintenance.
+
+const std = @import("std");
+const bson = @import("bson.zig");
+const index = @import("index.zig");
+// Always active, including in the default ReleaseFast build -- see assert.zig.
+const assert = @import("assert.zig").assert;
+const assert_msg = @import("assert.zig").assert_msg;
+
+// ---------------------------------------------------------------------------
+// Bounds
+// ---------------------------------------------------------------------------
+
+/// Longest index key a `.stream` cursor will anchor on. Tied to the B+tree's
+/// own "this record is normal" threshold rather than picked: a key past it has
+/// already spilled to the overflow slab, so the tree itself considers it
+/// exceptional. Also what makes the anchor a fixed inline array instead of an
+/// allocation.
+///
+/// Unbounded, this is a memory denial of service and not a subtle one:
+/// `bson.encode_key` escapes NULs, so a 16 MB string doubles, and a compound
+/// index may carry 32 of them.
+pub const anchor_key_max: usize = 1024;
+
+comptime {
+    // The bound is only defensible if it really is the tree's spill threshold.
+    // If the page size or the spill fraction ever changes, this fails to
+    // compile rather than silently becoming an arbitrary number.
+    std.debug.assert(anchor_key_max == index.inline_limit);
+}
+
+pub const ns_db_max: usize = 64;
+pub const ns_coll_max: usize = 192;
+pub const index_name_max: usize = 128;
+
+/// Documents in a first batch when the client named no `batchSize`. MongoDB's
+/// own default (`internalQueryFindCommandBatchSize`).
+pub const default_first_batch: u32 = 101;
+
+/// Cap on a batch's document payload: `maxBsonObjectSize`, which is also what
+/// leaves room for the reply envelope inside the 48 MiB message limit.
+pub const batch_bytes_max: u64 = 16 * 1024 * 1024;
+
+/// Idle milliseconds before the sweep reaps a cursor. MongoDB's
+/// `cursorTimeoutMillis`.
+pub const default_idle_timeout_ms: i64 = 10 * 60 * 1000;
+
+/// Slots in the registry unless configured otherwise.
+pub const default_capacity: u32 = 4096;
+
+/// Low bits of a cursor id that address its slot; the rest is the nonce.
+const slot_bits: u6 = 20;
+const slot_mask: u64 = (@as(u64, 1) << slot_bits) - 1;
+/// Nonce width, leaving the sign bit clear so every id is a positive i64.
+const nonce_mask: u64 = (@as(u64, 1) << (63 - slot_bits)) - 1;
+
+pub const max_capacity: u32 = @intCast(slot_mask);
+
+// ---------------------------------------------------------------------------
+// Cursor state
+// ---------------------------------------------------------------------------
+
+/// A namespace, by value. A cursor cannot hold a `*Collection`: `drop` frees
+/// it, and the pointer would dangle exactly the way the M0 notes on
+/// heap-allocating collections describe.
+pub const Ns = struct {
+    db: []const u8,
+    coll: []const u8,
+};
+
+/// Where the remaining documents come from.
+pub const Source = union(enum) {
+    /// An index-ordered scan, resumed from a value-typed anchor. O(key)
+    /// memory, so this is the shape that lets a cursor walk a collection far
+    /// larger than memory -- the reason M0 made whole-index scans stream.
+    stream: Stream,
+    /// Matched slab offsets, 8 bytes each, which `scan_sorted` has already
+    /// materialized for a narrowed plan.
+    ///
+    /// Safe against an offset the coming document free list has recycled,
+    /// because every batch re-applies the full filter -- the index invariant.
+    /// A recycled offset is therefore either rejected or resolves to a
+    /// document that genuinely matches. It needs one guarantee from the free
+    /// list, recorded in PLAN: an offset that was ever a record start must
+    /// stay a record start, since `doc_bytes` reads a length prefix in place.
+    offsets: struct { items: []u64, next: u32 = 0 },
+    /// Canonical BSON bytes owned by the cursor's arena, for results with no
+    /// stable backing store to point at: a sort no index provides, and
+    /// aggregate/listCollections/listIndexes output.
+    buffered: struct { docs: []const []const u8, next: u32 = 0 },
+};
+
+/// A resumable index scan. Every field is a value; nothing here is a pointer
+/// into the tree, the slab or the request that created it.
+pub const Stream = struct {
+    /// Empty means the implicit `_id_` index. Re-resolved by name on every
+    /// `getMore`, so a `dropIndexes` cannot leave a dangling `*Index`.
+    index_name_buf: [index_name_max]u8 = undefined,
+    index_name_len: u8 = 0,
+    /// Bumped by `reset_tree`/`replace_root_with_leaf`; if it moved, the hint
+    /// below addresses a different tree and must not be trusted.
+    index_epoch: u64 = 0,
+    backward: bool = false,
+    anchor_buf: [anchor_key_max]u8 = undefined,
+    anchor_len: u16 = 0,
+    anchor_off: u64 = 0,
+    /// Entries sharing the anchor's key that this cursor has already yielded.
+    /// Without it, an anchor whose document was deleted between batches would
+    /// resume past the entire equal-key band -- on a three-value index that is
+    /// millions of documents silently missing.
+    band_index: u64 = 0,
+    /// Last known position of the anchor. A hint, never trusted without
+    /// re-reading the entry there: it turns resume from a walk down the
+    /// equal-key band into O(1), which is what keeps a low-cardinality
+    /// `sort({status: 1})` from costing O(band) per batch.
+    hint_leaf: u32 = 0,
+    hint_slot: u32 = 0,
+
+    pub fn index_name(self: *const Stream) []const u8 {
+        return self.index_name_buf[0..self.index_name_len];
+    }
+
+    pub fn anchor_key(self: *const Stream) []const u8 {
+        return self.anchor_buf[0..self.anchor_len];
+    }
+
+    /// Whether anything has been yielded yet. Derived rather than stored: an
+    /// encoded index key always begins with `bson.encode_key`'s rank byte, so it
+    /// is never empty, and a separate `started` flag would be a second field that
+    /// has to agree with this one.
+    ///
+    /// It can legitimately be false on a live cursor: `batchSize: 0` returns an
+    /// empty first batch without consuming anything, and such a cursor starts at
+    /// `iter()`/`iter_reverse()` rather than resuming.
+    pub fn started(self: *const Stream) bool {
+        return self.anchor_len > 0;
+    }
+
+    /// Record the entry just yielded as the point to resume after.
+    ///
+    /// Asserts the anchor advances in scan order. This is the single check most
+    /// likely to catch a resume bug: going backwards duplicates documents,
+    /// standing still makes `getMore` loop forever, and both are far easier to
+    /// see here than in a client's result set. Equal keys are legal (a
+    /// duplicate band), which is exactly why `band_index` also has to move.
+    pub fn advance(self: *Stream, key: []const u8, off: u64, leaf: u32, slot: u32) void {
+        assert(key.len <= anchor_key_max);
+        // What makes `started()` derivable, so pin it here rather than trust it.
+        assert_msg(key.len > 0, "an encoded index key is never empty");
+        if (self.started()) {
+            const order = std.mem.order(u8, key, self.anchor_key());
+            if (self.backward) {
+                assert_msg(order != .gt, "a reverse cursor's anchor moved forward");
+            } else {
+                assert_msg(order != .lt, "a forward cursor's anchor moved backward");
+            }
+            if (order == .eq) {
+                assert_msg(
+                    off != self.anchor_off or self.band_index > 0,
+                    "a cursor re-anchored on the entry it just yielded",
+                );
+                // Still inside the anchor's band, so the position within it has
+                // to move or a resume could not tell the two entries apart.
+                self.band_index += 1;
+            } else {
+                self.band_index = 0;
+            }
+        }
+        @memcpy(self.anchor_buf[0..key.len], key);
+        self.anchor_len = @intCast(key.len);
+        self.anchor_off = off;
+        self.hint_leaf = leaf;
+        self.hint_slot = slot;
+    }
+};
+
+pub const Cursor = struct {
+    /// Positive and never 0: `id: 0` is "no cursor" on the wire.
+    id: i64,
+    ns_db_buf: [ns_db_max]u8 = undefined,
+    ns_db_len: u8 = 0,
+    ns_coll_buf: [ns_coll_max]u8 = undefined,
+    ns_coll_len: u8 = 0,
+    /// Bumped when a rebuild moves documents, so a saved offset or anchor
+    /// offset is stale. Also the drop detector.
+    layout_epoch: u64 = 0,
+    /// Serialized so they outlive the request that parsed them: a parsed
+    /// `[]bson.Pair` points into the per-request message arena, and the reply
+    /// arena is reset on every request.
+    filter_bytes: []const u8 = &.{},
+    proj_bytes: []const u8 = &.{},
+    /// Documents still owed across all remaining batches; null is unbounded.
+    /// Reaching 0 is an EOF *source*, which is what closes the cursor in the
+    /// very batch that exhausts the limit rather than one round trip later.
+    /// Optional rather than "0 means unbounded" precisely because 0 has to keep
+    /// its literal meaning here.
+    remaining_limit: ?u64 = null,
+    /// The client's `batchSize`, reused when a `getMore` names none.
+    batch_size: ?u32 = null,
+    /// Exempt from the idle sweep. Still killable by `killCursors` and by
+    /// eviction -- a fixed-capacity registry cannot promise "never expires".
+    no_timeout: bool = false,
+    /// A request is using this cursor right now. Concurrent use is rejected
+    /// rather than queued: queueing lets one client turn a single cursor into a
+    /// connection-count denial of service.
+    pinned: bool = false,
+    /// `killCursors` arrived while pinned; the in-flight request frees it.
+    kill_requested: bool = false,
+    last_use_ms: i64 = 0,
+    arena: std.heap.ArenaAllocator,
+    source: Source,
+
+    pub fn ns(self: *const Cursor) Ns {
+        return .{
+            .db = self.ns_db_buf[0..self.ns_db_len],
+            .coll = self.ns_coll_buf[0..self.ns_coll_len],
+        };
+    }
+
+    pub fn ns_matches(self: *const Cursor, other: Ns) bool {
+        const own = self.ns();
+        return std.mem.eql(u8, own.db, other.db) and std.mem.eql(u8, own.coll, other.coll);
+    }
+};
+
+/// Everything a caller must decide before a cursor can exist. Grouped so
+/// `open` cannot be called with an argument silently in the wrong position.
+pub const OpenSpec = struct {
+    ns: Ns,
+    layout_epoch: u64,
+    filter_bytes: []const u8 = &.{},
+    proj_bytes: []const u8 = &.{},
+    remaining_limit: ?u64 = null,
+    batch_size: ?u32 = null,
+    no_timeout: bool = false,
+    source: Source,
+};
+
+pub const OpenError = error{
+    /// The namespace does not fit the fixed buffers. Callers degrade to a
+    /// single batch rather than failing the query.
+    NameTooLong,
+    /// Every slot is pinned by an in-flight request.
+    TooManyCursors,
+    OutOfMemory,
+    /// Taking the store mutex was cancelled (shutdown).
+    Canceled,
+};
+
+pub const PinError = error{
+    CursorNotFound,
+    /// The id exists but belongs to another namespace. Distinct from
+    /// `CursorNotFound` because mongod answers this with `Unauthorized` (13),
+    /// not 43, and leaves the cursor alive -- the request is wrong, not the
+    /// cursor.
+    CursorNamespaceMismatch,
+    CursorInUse,
+    Canceled,
+};
+
+/// Owned storage for a namespace copied out of the store, so an error message
+/// can name a cursor's namespace without holding the store's mutex or a pointer
+/// into its slots.
+pub const NsBuf = struct {
+    db_buf: [ns_db_max]u8 = undefined,
+    db_len: u8 = 0,
+    coll_buf: [ns_coll_max]u8 = undefined,
+    coll_len: u8 = 0,
+
+    pub fn ns(self: *const NsBuf) Ns {
+        return .{ .db = self.db_buf[0..self.db_len], .coll = self.coll_buf[0..self.coll_len] };
+    }
+
+    fn set(self: *NsBuf, from: Ns) void {
+        @memcpy(self.db_buf[0..from.db.len], from.db);
+        self.db_len = @intCast(from.db.len);
+        @memcpy(self.coll_buf[0..from.coll.len], from.coll);
+        self.coll_len = @intCast(from.coll.len);
+    }
+};
+
+pub const KillOutcome = enum { killed, not_found };
+
+// ---------------------------------------------------------------------------
+// The registry
+// ---------------------------------------------------------------------------
+
+pub const Store = struct {
+    /// Guards every field below. A **leaf** lock: no other lock -- catalog,
+    /// collection, log -- is ever acquired while it is held, so it cannot
+    /// participate in a cycle. In particular a `getMore` copies what it needs
+    /// out, releases this, and only then iterates under the collection lock;
+    /// otherwise the reaper would block behind a full scan.
+    mutex: std.Io.Mutex = .init,
+    /// Boxed, not inline. A `Cursor` inlines its anchor and namespace buffers and
+    /// so is ~1.5 KiB; at the default capacity an inline table would be 6.2 MiB
+    /// allocated and zeroed at *every* `Engine.open` -- paid by every embedded
+    /// user and by all ~47 engine opens in the unit suite, to hold zero cursors.
+    /// A pointer table is 32 KiB and the cursor itself is allocated when one
+    /// actually exists, which is also when its arena is created anyway.
+    slots: []?*Cursor,
+    /// Mixed into every id so ids are not guessable across processes, and so a
+    /// reused slot rejects the previous id exactly. Without this a stale
+    /// `getMore` can address a recycled slot and read another client's cursor.
+    nonce: u64,
+    live: u32 = 0,
+    idle_timeout_ms: i64 = default_idle_timeout_ms,
+    gpa: std.mem.Allocator,
+
+    pub fn init(
+        gpa: std.mem.Allocator,
+        io: std.Io,
+        capacity: u32,
+        idle_timeout_ms: i64,
+    ) !Store {
+        assert(capacity > 0 and capacity <= max_capacity);
+        var seed: [8]u8 = undefined;
+        io.random(&seed);
+        const slots = try gpa.alloc(?*Cursor, capacity);
+        @memset(slots, null);
+        return .{
+            .slots = slots,
+            // A zero nonce would make the first slot's id equal to its index,
+            // and slot 0's id would be 0 -- which means "no cursor".
+            .nonce = std.mem.readInt(u64, &seed, .little) | 1,
+            .idle_timeout_ms = idle_timeout_ms,
+            .gpa = gpa,
+        };
+    }
+
+    pub fn deinit(self: *Store) void {
+        for (self.slots) |maybe| {
+            if (maybe) |c| destroy_cursor(self.gpa, c);
+        }
+        self.gpa.free(self.slots);
+        self.slots = &.{};
+    }
+
+    /// Free a cursor: its arena first, then the box the slot pointed at.
+    fn destroy_cursor(gpa: std.mem.Allocator, c: *Cursor) void {
+        c.arena.deinit();
+        gpa.destroy(c);
+    }
+
+    fn slot_of(id: i64) usize {
+        return @intCast(@as(u64, @bitCast(id)) & slot_mask);
+    }
+
+    /// Build the id for `slot` at the store's current nonce, then advance the
+    /// nonce so the next cursor in this slot gets a different id.
+    fn mint(self: *Store, slot: usize) i64 {
+        const n = self.nonce & nonce_mask;
+        self.nonce +%= 1;
+        const raw = (n << slot_bits) | @as(u64, @intCast(slot));
+        const id: i64 = @intCast(raw & ~(@as(u64, 1) << 63));
+        // Both properties are load-bearing on the wire and in lookup.
+        assert_msg(id > 0, "a cursor id must be a positive int64");
+        assert_msg(slot_of(id) == slot, "a cursor id must address its own slot");
+        return id;
+    }
+
+    /// Register a cursor and return its id, or null when the caller should
+    /// answer in a single batch instead (`NameTooLong` is not worth failing a
+    /// query over -- the degradation is exactly today's behaviour).
+    ///
+    /// Takes ownership of `spec.source` and of the arena backing it.
+    pub fn open(
+        self: *Store,
+        io: std.Io,
+        now_ms: i64,
+        arena: std.heap.ArenaAllocator,
+        spec: OpenSpec,
+    ) OpenError!i64 {
+        if (spec.ns.db.len > ns_db_max or spec.ns.coll.len > ns_coll_max) {
+            return error.NameTooLong;
+        }
+        try self.mutex.lock(io);
+        defer self.mutex.unlock(io);
+
+        const slot = self.free_slot(now_ms) orelse return error.TooManyCursors;
+        assert(self.slots[slot] == null);
+
+        const c = try self.gpa.create(Cursor);
+        errdefer self.gpa.destroy(c);
+        c.* = .{
+            .id = self.mint(slot),
+            .layout_epoch = spec.layout_epoch,
+            .filter_bytes = spec.filter_bytes,
+            .proj_bytes = spec.proj_bytes,
+            .remaining_limit = spec.remaining_limit,
+            .batch_size = spec.batch_size,
+            .no_timeout = spec.no_timeout,
+            .last_use_ms = now_ms,
+            .arena = arena,
+            .source = spec.source,
+        };
+        @memcpy(c.ns_db_buf[0..spec.ns.db.len], spec.ns.db);
+        c.ns_db_len = @intCast(spec.ns.db.len);
+        @memcpy(c.ns_coll_buf[0..spec.ns.coll.len], spec.ns.coll);
+        c.ns_coll_len = @intCast(spec.ns.coll.len);
+
+        self.slots[slot] = c;
+        self.live += 1;
+        return c.id;
+    }
+
+    /// An empty slot: a genuinely free one, else the least-recently-used
+    /// unpinned cursor. Evicting is legal and cheap to reason about, because
+    /// the victim's client gets `CursorNotFound` on its next `getMore` -- the
+    /// same answer an idle timeout gives, which every driver already handles.
+    /// Caller holds the mutex.
+    fn free_slot(self: *Store, now_ms: i64) ?usize {
+        var lru: ?usize = null;
+        var lru_ms: i64 = std.math.maxInt(i64);
+        for (self.slots, 0..) |maybe, i| {
+            const c = maybe orelse return i;
+            // Reap on the way past, so a store that has gone quiet does not
+            // wait for the sweep tick to reclaim what already expired.
+            if (self.expired(c, now_ms)) {
+                self.destroy(i);
+                return i;
+            }
+            if (c.pinned) continue;
+            if (c.last_use_ms < lru_ms) {
+                lru_ms = c.last_use_ms;
+                lru = i;
+            }
+        }
+        if (lru) |i| {
+            self.destroy(i);
+            return i;
+        }
+        return null;
+    }
+
+    /// Caller holds the mutex.
+    fn expired(self: *const Store, c: *const Cursor, now_ms: i64) bool {
+        if (c.pinned or c.no_timeout or self.idle_timeout_ms <= 0) return false;
+        return now_ms -| c.last_use_ms >= self.idle_timeout_ms;
+    }
+
+    /// Caller holds the mutex.
+    fn destroy(self: *Store, slot: usize) void {
+        const c = self.slots[slot] orelse return;
+        assert_msg(!c.pinned, "a pinned cursor must not be destroyed under its user");
+        destroy_cursor(self.gpa, c);
+        self.slots[slot] = null;
+        self.live -= 1;
+    }
+
+    /// Claim a cursor for one request. The returned pointer is stable only
+    /// until `release`, and only the pinning request may touch it.
+    ///
+    /// The namespace check is not cosmetic: `dispatch` locks the collection
+    /// named in the *message*, so a `getMore` quoting one cursor's id and
+    /// another collection's name would otherwise iterate the first collection's
+    /// index while holding the second collection's lock. A mismatch leaves the
+    /// cursor alive -- it is the request that is wrong, not the cursor.
+    pub fn pin(self: *Store, io: std.Io, id: i64, ns: Ns, now_ms: i64) PinError!*Cursor {
+        try self.mutex.lock(io);
+        defer self.mutex.unlock(io);
+        if (id <= 0) return error.CursorNotFound;
+        const slot = slot_of(id);
+        if (slot >= self.slots.len) return error.CursorNotFound;
+        const c = self.slots[slot] orelse return error.CursorNotFound;
+        // Compare the whole id, not just the slot: this is what makes a
+        // recycled slot reject its predecessor's id.
+        if (c.id != id) return error.CursorNotFound;
+        if (self.expired(c, now_ms)) {
+            self.destroy(slot);
+            return error.CursorNotFound;
+        }
+        if (!c.ns_matches(ns)) return error.CursorNamespaceMismatch;
+        if (c.pinned) return error.CursorInUse;
+        c.pinned = true;
+        c.last_use_ms = now_ms;
+        return c;
+    }
+
+    /// The namespace a live cursor belongs to, copied out. Only used to build
+    /// the namespace-mismatch error message, so a second lock acquisition on an
+    /// error path is the right trade for not threading an out-parameter through
+    /// the success path.
+    pub fn ns_of(self: *Store, io: std.Io, id: i64, out: *NsBuf) bool {
+        self.mutex.lock(io) catch return false;
+        defer self.mutex.unlock(io);
+        if (id <= 0) return false;
+        const slot = slot_of(id);
+        if (slot >= self.slots.len) return false;
+        const c = self.slots[slot] orelse return false;
+        if (c.id != id) return false;
+        out.set(c.ns());
+        return true;
+    }
+
+    /// Hand a pinned cursor back. `exhausted` destroys it, and so does a
+    /// `killCursors` that arrived while it was pinned.
+    pub fn release(self: *Store, io: std.Io, c: *Cursor, now_ms: i64, exhausted: bool) void {
+        self.mutex.lock(io) catch {
+            // Cancellation while returning a cursor would otherwise leave it
+            // pinned forever, unreachable and un-reapable. Unpinning without
+            // the lock is the lesser evil: the field is only ever written by
+            // the one request that owns the pin.
+            c.pinned = false;
+            return;
+        };
+        defer self.mutex.unlock(io);
+        assert_msg(c.pinned, "released a cursor that was not pinned");
+        c.pinned = false;
+        c.last_use_ms = now_ms;
+        if (exhausted or c.kill_requested) {
+            const slot = slot_of(c.id);
+            assert(self.slots[slot].? == c);
+            self.destroy(slot);
+        }
+    }
+
+    /// `killCursors` for one id. A pinned cursor is marked and reported killed:
+    /// the client's intent is satisfied, and the in-flight request frees it on
+    /// release. Storage is never freed under a running request.
+    pub fn kill(self: *Store, io: std.Io, id: i64, ns: Ns) KillOutcome {
+        self.mutex.lock(io) catch return .not_found;
+        defer self.mutex.unlock(io);
+        if (id <= 0) return .not_found;
+        const slot = slot_of(id);
+        if (slot >= self.slots.len) return .not_found;
+        const c = self.slots[slot] orelse return .not_found;
+        if (c.id != id or !c.ns_matches(ns)) return .not_found;
+        if (c.pinned) {
+            c.kill_requested = true;
+            return .killed;
+        }
+        self.destroy(slot);
+        return .killed;
+    }
+
+    /// Kill every cursor on a namespace. Called when the collection or its
+    /// database is dropped: a later `getMore` would fail anyway, since the
+    /// cursor holds names rather than a pointer, but reaping here frees the
+    /// slots at once and keeps the open-cursor metric honest.
+    pub fn kill_namespace(
+        self: *Store,
+        io: std.Io,
+        db_name: []const u8,
+        coll_name: ?[]const u8,
+    ) u32 {
+        self.mutex.lock(io) catch return 0;
+        defer self.mutex.unlock(io);
+        var n: u32 = 0;
+        for (self.slots, 0..) |maybe, i| {
+            const c = maybe orelse continue;
+            const own = c.ns();
+            if (!std.mem.eql(u8, own.db, db_name)) continue;
+            if (coll_name) |name| {
+                if (!std.mem.eql(u8, own.coll, name)) continue;
+            }
+            if (c.pinned) {
+                c.kill_requested = true;
+            } else {
+                self.destroy(i);
+            }
+            n += 1;
+        }
+        return n;
+    }
+
+    /// Reap idle cursors. Returns how many went.
+    pub fn sweep(self: *Store, io: std.Io, now_ms: i64) u32 {
+        self.mutex.lock(io) catch return 0;
+        defer self.mutex.unlock(io);
+        if (self.live == 0) return 0;
+        var n: u32 = 0;
+        for (self.slots, 0..) |maybe, i| {
+            const c = maybe orelse continue;
+            if (!self.expired(c, now_ms)) continue;
+            self.destroy(i);
+            n += 1;
+        }
+        return n;
+    }
+};
+
+// ---------------------------------------------------------------------------
+// Batch policy
+// ---------------------------------------------------------------------------
+
+/// What `offer` decided about one document.
+pub const Offered = enum {
+    appended,
+    /// The batch is full. **Not** EOF: the cursor stays open, and this document
+    /// has not been consumed -- the caller must hand it to the next batch.
+    batch_full,
+};
+
+/// Accumulates one batch and owns the two limits that end it.
+///
+/// Split out from the emit path so the whole policy is a pure function of
+/// `(target, emitted, bytes, size)` and can be unit-tested without an engine,
+/// a socket or an allocator. The subtle parts are all here: a target of 0 is
+/// unbounded (a `getMore` naming no `batchSize`), a `batchSize: 0` first batch
+/// is a target that is *reached immediately*, and the byte cap must still let
+/// the first document through or an oversized document would wedge the cursor
+/// forever, returning empty batches with no progress.
+pub const BatchBuilder = struct {
+    /// Documents wanted; null means no document target, fill to the byte cap.
+    /// Optional rather than "0 means unbounded" because `batchSize: 0` is a real
+    /// request for an empty batch, and conflating the two returned the whole
+    /// collection where mongod returns nothing.
+    target: ?u32,
+    bytes_max: u64 = batch_bytes_max,
+    emitted: u32 = 0,
+    bytes: u64 = 0,
+
+    pub fn init(target: ?u32) BatchBuilder {
+        return .{ .target = target };
+    }
+
+    /// Whether the batch has already met its document target, checked before
+    /// pulling from the source so a full batch never consumes a document it
+    /// cannot carry.
+    pub fn full(self: *const BatchBuilder) bool {
+        const t = self.target orelse return false;
+        return self.emitted >= t;
+    }
+
+    /// Account for a document of `size` serialized bytes.
+    pub fn offer(self: *BatchBuilder, size: u64) Offered {
+        assert_msg(!self.full(), "offered a document to a batch that was already full");
+        // The at-least-one rule: an empty batch takes the document whatever it
+        // measures. Stored documents cannot exceed the cap (inserts enforce
+        // 16 MiB), so this only arises for a generated one.
+        if (self.emitted > 0 and self.bytes + size > self.bytes_max) return .batch_full;
+        self.emitted += 1;
+        self.bytes += size;
+        return .appended;
+    }
+};
+
+/// The document target for a batch: the client's `batchSize` if it named one,
+/// otherwise 101 for a first batch and *no* document target for a `getMore`.
+///
+/// Both defaults are measured against mongod 8.3.7 rather than assumed.
+/// `internalQueryFindCommandBatchSize` reports 101, and a `getMore` carrying no
+/// `batchSize` after a `find` with `batchSize: 2` returns 4998 of 5000
+/// documents -- so a bare `getMore` is bounded by bytes alone and does *not*
+/// inherit the `batchSize` the cursor was created with.
+pub fn batch_target(batch_size: ?u32, first: bool) ?u32 {
+    if (batch_size) |n| return n;
+    return if (first) default_first_batch else null;
+}
+
+// ---------------------------------------------------------------------------
+// Tests
+// ---------------------------------------------------------------------------
+
+const testing = std.testing;
+
+/// A Store for tests, with a threaded Io so the mutex is real.
+const TestStore = struct {
+    threaded: std.Io.Threaded,
+    store: Store,
+
+    fn init(capacity: u32, idle_timeout_ms: i64) !TestStore {
+        var self: TestStore = undefined;
+        self.threaded = std.Io.Threaded.init(testing.allocator, .{});
+        self.store = try Store.init(
+            testing.allocator,
+            self.threaded.io(),
+            capacity,
+            idle_timeout_ms,
+        );
+        return self;
+    }
+
+    fn io(self: *TestStore) std.Io {
+        return self.threaded.io();
+    }
+
+    fn deinit(self: *TestStore) void {
+        self.store.deinit();
+        self.threaded.deinit();
+    }
+
+    fn open_one(self: *TestStore, coll: []const u8, now_ms: i64) !i64 {
+        const arena = std.heap.ArenaAllocator.init(testing.allocator);
+        return self.store.open(self.io(), now_ms, arena, .{
+            .ns = .{ .db = "t", .coll = coll },
+            .layout_epoch = 0,
+            .source = .{ .buffered = .{ .docs = &.{} } },
+        });
+    }
+};
+
+test "cursor ids are positive, address their slot, and never repeat" {
+    var ts = try TestStore.init(4, default_idle_timeout_ms);
+    defer ts.deinit();
+
+    var seen: [16]i64 = undefined;
+    for (0..16) |i| {
+        const id = try ts.open_one("c", 0);
+        try testing.expect(id > 0);
+        // Freeing the slot immediately means the next open reuses it, which is
+        // exactly the case the nonce has to survive.
+        const killed = ts.store.kill(ts.io(), id, .{ .db = "t", .coll = "c" });
+        try testing.expectEqual(KillOutcome.killed, killed);
+        seen[i] = id;
+    }
+    for (seen, 0..) |a, i| {
+        for (seen[i + 1 ..]) |b| try testing.expect(a != b);
+    }
+}
+
+test "a recycled slot rejects the id it used to hold" {
+    // The guard that keeps one client from reading another's cursor. Without
+    // the nonce in the id, `slot_of(stale) == slot_of(fresh)` and the stale
+    // getMore would be served the new cursor's documents.
+    var ts = try TestStore.init(1, default_idle_timeout_ms);
+    defer ts.deinit();
+    const ns = Ns{ .db = "t", .coll = "c" };
+
+    const stale = try ts.open_one("c", 0);
+    try testing.expectEqual(KillOutcome.killed, ts.store.kill(ts.io(), stale, ns));
+    const fresh = try ts.open_one("c", 0);
+
+    try testing.expectEqual(Store.slot_of(stale), Store.slot_of(fresh));
+    try testing.expect(stale != fresh);
+    try testing.expectError(error.CursorNotFound, ts.store.pin(ts.io(), stale, ns, 0));
+    _ = try ts.store.pin(ts.io(), fresh, ns, 0);
+}
+
+test "pin rejects a wrong namespace and a second holder, and leaves the cursor alive" {
+    var ts = try TestStore.init(4, default_idle_timeout_ms);
+    defer ts.deinit();
+    const ns = Ns{ .db = "t", .coll = "c" };
+    const id = try ts.open_one("c", 0);
+
+    // A wrong namespace must not kill the cursor: the request is wrong, not
+    // the cursor, and the client is allowed to retry correctly. Reported apart
+    // from CursorNotFound because mongod answers it with Unauthorized (13).
+    const wrong_coll = Ns{ .db = "t", .coll = "other" };
+    const wrong_db = Ns{ .db = "other", .coll = "c" };
+    const mismatch = error.CursorNamespaceMismatch;
+    try testing.expectError(mismatch, ts.store.pin(ts.io(), id, wrong_coll, 0));
+    try testing.expectError(mismatch, ts.store.pin(ts.io(), id, wrong_db, 0));
+
+    var found: NsBuf = .{};
+    try testing.expect(ts.store.ns_of(ts.io(), id, &found));
+    try testing.expectEqualStrings("t", found.ns().db);
+    try testing.expectEqualStrings("c", found.ns().coll);
+
+    const c = try ts.store.pin(ts.io(), id, ns, 0);
+    try testing.expectError(error.CursorInUse, ts.store.pin(ts.io(), id, ns, 0));
+    ts.store.release(ts.io(), c, 1, false);
+    // Released, so it can be pinned again.
+    const again = try ts.store.pin(ts.io(), id, ns, 2);
+    ts.store.release(ts.io(), again, 3, true);
+    try testing.expectError(error.CursorNotFound, ts.store.pin(ts.io(), id, ns, 4));
+}
+
+test "a full store evicts the least recently used unpinned cursor" {
+    var ts = try TestStore.init(3, default_idle_timeout_ms);
+    defer ts.deinit();
+    const ns = Ns{ .db = "t", .coll = "c" };
+
+    const a = try ts.open_one("c", 100);
+    const b = try ts.open_one("c", 200);
+    const c = try ts.open_one("c", 300);
+    // Touch `a` so `b` becomes the least recently used.
+    const pinned_a = try ts.store.pin(ts.io(), a, ns, 400);
+    ts.store.release(ts.io(), pinned_a, 400, false);
+
+    const d = try ts.open_one("c", 500);
+    try testing.expectError(error.CursorNotFound, ts.store.pin(ts.io(), b, ns, 500));
+    for ([_]i64{ a, c, d }) |id| {
+        const live = try ts.store.pin(ts.io(), id, ns, 500);
+        ts.store.release(ts.io(), live, 500, false);
+    }
+}
+
+test "a store whose every slot is pinned refuses rather than evicting" {
+    var ts = try TestStore.init(2, default_idle_timeout_ms);
+    defer ts.deinit();
+    const ns = Ns{ .db = "t", .coll = "c" };
+    const a = try ts.open_one("c", 0);
+    const b = try ts.open_one("c", 0);
+    _ = try ts.store.pin(ts.io(), a, ns, 0);
+    _ = try ts.store.pin(ts.io(), b, ns, 0);
+    try testing.expectError(error.TooManyCursors, ts.open_one("c", 0));
+}
+
+test "the sweep reaps idle cursors and spares noCursorTimeout" {
+    var ts = try TestStore.init(4, 1000);
+    defer ts.deinit();
+    const ns = Ns{ .db = "t", .coll = "c" };
+
+    const perishable = try ts.open_one("c", 0);
+    const arena = std.heap.ArenaAllocator.init(testing.allocator);
+    const immortal = try ts.store.open(ts.io(), 0, arena, .{
+        .ns = ns,
+        .layout_epoch = 0,
+        .no_timeout = true,
+        .source = .{ .buffered = .{ .docs = &.{} } },
+    });
+
+    // Just short of the timeout: nothing goes.
+    try testing.expectEqual(@as(u32, 0), ts.store.sweep(ts.io(), 999));
+    try testing.expectEqual(@as(u32, 1), ts.store.sweep(ts.io(), 1000));
+    try testing.expectError(error.CursorNotFound, ts.store.pin(ts.io(), perishable, ns, 1000));
+
+    // The exempt one survives an interval it would otherwise have died in...
+    try testing.expectEqual(@as(u32, 0), ts.store.sweep(ts.io(), 100_000));
+    const live = try ts.store.pin(ts.io(), immortal, ns, 100_000);
+    ts.store.release(ts.io(), live, 100_000, false);
+    // ...but is still killable explicitly.
+    try testing.expectEqual(KillOutcome.killed, ts.store.kill(ts.io(), immortal, ns));
+}
+
+test "killCursors reports a pinned cursor killed and frees it on release" {
+    var ts = try TestStore.init(4, default_idle_timeout_ms);
+    defer ts.deinit();
+    const ns = Ns{ .db = "t", .coll = "c" };
+    const id = try ts.open_one("c", 0);
+    const c = try ts.store.pin(ts.io(), id, ns, 0);
+
+    try testing.expectEqual(KillOutcome.killed, ts.store.kill(ts.io(), id, ns));
+    // Still pinned, so its storage must not have been freed under the request.
+    try testing.expect(c.kill_requested);
+    // Not exhausted, but the pending kill wins.
+    ts.store.release(ts.io(), c, 1, false);
+    try testing.expectError(error.CursorNotFound, ts.store.pin(ts.io(), id, ns, 2));
+    try testing.expectEqual(KillOutcome.not_found, ts.store.kill(ts.io(), id, ns));
+}
+
+test "kill_namespace reaps a collection's cursors and leaves the rest" {
+    var ts = try TestStore.init(8, default_idle_timeout_ms);
+    defer ts.deinit();
+    const doomed = try ts.open_one("doomed", 0);
+    const spared = try ts.open_one("spared", 0);
+
+    try testing.expectEqual(@as(u32, 1), ts.store.kill_namespace(ts.io(), "t", "doomed"));
+    const doomed_ns = Ns{ .db = "t", .coll = "doomed" };
+    try testing.expectError(error.CursorNotFound, ts.store.pin(ts.io(), doomed, doomed_ns, 0));
+    const live = try ts.store.pin(ts.io(), spared, .{ .db = "t", .coll = "spared" }, 0);
+    ts.store.release(ts.io(), live, 0, false);
+
+    // Whole-database form.
+    try testing.expectEqual(@as(u32, 1), ts.store.kill_namespace(ts.io(), "t", null));
+    try testing.expectEqual(@as(u32, 0), ts.store.live);
+}
+
+test "a namespace too long for the fixed buffers declines rather than failing" {
+    var ts = try TestStore.init(2, default_idle_timeout_ms);
+    defer ts.deinit();
+    const long = "c" ** (ns_coll_max + 1);
+    try testing.expectError(error.NameTooLong, ts.open_one(long, 0));
+}
+
+test "batch_target: 101 for a first batch, unbounded for a getMore, honoured when given" {
+    try testing.expectEqual(@as(?u32, default_first_batch), batch_target(null, true));
+    // A bare getMore has no document target at all. Measured against mongod:
+    // it does not inherit the batchSize the cursor was created with.
+    try testing.expectEqual(@as(?u32, null), batch_target(null, false));
+    try testing.expectEqual(@as(?u32, 7), batch_target(7, true));
+
+    // batchSize: 0 is a real target of zero -- an empty first batch with a live
+    // cursor, which drivers use to obtain a cursor cheaply. It must NOT read as
+    // "unbounded": conflating the two returns the whole collection where mongod
+    // returns nothing, which is exactly the bug this optional prevents.
+    try testing.expectEqual(@as(?u32, 0), batch_target(0, true));
+    var zero = BatchBuilder.init(batch_target(0, true));
+    try testing.expect(zero.full());
+    var bare = BatchBuilder.init(batch_target(null, false));
+    try testing.expect(!bare.full());
+}
+
+test "BatchBuilder stops at its document target" {
+    var b = BatchBuilder.init(2);
+    try testing.expect(!b.full());
+    try testing.expectEqual(Offered.appended, b.offer(10));
+    try testing.expect(!b.full());
+    try testing.expectEqual(Offered.appended, b.offer(10));
+    try testing.expect(b.full());
+    try testing.expectEqual(@as(u32, 2), b.emitted);
+}
+
+test "BatchBuilder: a null target is unbounded by documents" {
+    var b = BatchBuilder.init(null);
+    for (0..5000) |_| {
+        try testing.expect(!b.full());
+        try testing.expectEqual(Offered.appended, b.offer(1));
+    }
+    try testing.expect(!b.full());
+}
+
+test "BatchBuilder stops on bytes, but always takes at least one document" {
+    // Hitting the byte cap must not read as EOF, or the cursor would close and
+    // silently drop the rest of the result.
+    var b = BatchBuilder.init(null);
+    b.bytes_max = 100;
+    try testing.expectEqual(Offered.appended, b.offer(60));
+    try testing.expectEqual(Offered.batch_full, b.offer(60));
+    // The refused document was not accounted for, so the caller can hand it to
+    // the next batch.
+    try testing.expectEqual(@as(u32, 1), b.emitted);
+    try testing.expectEqual(@as(u64, 60), b.bytes);
+
+    // An oversized document on an empty batch goes through anyway: refusing it
+    // would wedge the cursor, returning empty batches and never progressing.
+    var solo = BatchBuilder.init(null);
+    solo.bytes_max = 100;
+    try testing.expectEqual(Offered.appended, solo.offer(1_000_000));
+    try testing.expectEqual(@as(u32, 1), solo.emitted);
+    try testing.expectEqual(Offered.batch_full, solo.offer(1));
+}
+
+test "Stream.advance records the anchor and counts an equal-key band" {
+    var s = Stream{};
+    try testing.expect(!s.started());
+
+    s.advance("aaa", 10, 3, 4);
+    try testing.expect(s.started());
+    try testing.expectEqualStrings("aaa", s.anchor_key());
+    try testing.expectEqual(@as(u64, 10), s.anchor_off);
+    try testing.expectEqual(@as(u32, 3), s.hint_leaf);
+    try testing.expectEqual(@as(u32, 4), s.hint_slot);
+    try testing.expectEqual(@as(u64, 0), s.band_index);
+
+    // Same key, different document: still inside the band, so the position
+    // within it has to advance or a resume could not tell them apart.
+    s.advance("aaa", 11, 3, 5);
+    try testing.expectEqual(@as(u64, 1), s.band_index);
+    s.advance("aaa", 12, 3, 6);
+    try testing.expectEqual(@as(u64, 2), s.band_index);
+
+    // A new key ends the band.
+    s.advance("bbb", 13, 3, 7);
+    try testing.expectEqual(@as(u64, 0), s.band_index);
+    try testing.expectEqualStrings("bbb", s.anchor_key());
+}
+
+test "Stream.advance accepts a reverse cursor moving down" {
+    var s = Stream{ .backward = true };
+    s.advance("ccc", 1, 1, 5);
+    s.advance("bbb", 2, 1, 4);
+    s.advance("aaa", 3, 1, 3);
+    try testing.expectEqualStrings("aaa", s.anchor_key());
+}

src/index.zig

@@ -106,7 +106,9 @@ const page_size = 4096;
 /// Bytes of node payload: a 32-byte header plus the slotted region.
 const page_data = page_size - 32;
 /// Records longer than a quarter of a node spill to the overflow slab.
-const inline_limit = page_size / 4;
+/// Public because it is also the bound a resumable cursor anchors within: a key
+/// past it has already spilled, so the tree itself treats it as exceptional.
+pub const inline_limit = page_size / 4;
 /// Upper bound on the slots one node can hold, since every slot costs at
 /// least its own size. Bounds the split scratch.
 const max_slots = page_data / slot_size;
@@ -234,6 +236,19 @@ pub const Index = struct {
     depth: u32,
     /// Total entries, maintained incrementally.
     entry_count: usize,
+    /// Bumped whenever a node id stops meaning what it meant, which is the one
+    /// thing that makes a saved `(leaf, slot)` position dangerous rather than
+    /// merely stale. Node ids are otherwise append-only (`alloc_node`, and
+    /// `drop_child` abandons a page without recycling its id), and `page()`
+    /// resolves ids through `node_pages`, so copy-on-write and checkpoints move
+    /// pages without disturbing ids. Only `reset_tree` and
+    /// `replace_root_with_leaf` reuse an id for different contents.
+    ///
+    /// A resumable cursor keeps a position hint to avoid walking an equal-key
+    /// band on every `getMore`; it must compare this first. Without it the hint
+    /// would address a live but unrelated leaf after a compaction and the cursor
+    /// would iterate a tree that no longer exists.
+    epoch: u64,
     /// Repack scratch: any single node's record bytes fit here.
     scratch: [page_data]u8,
     /// Promoted-key scratch: inline keys being propagated up a split are
@@ -268,6 +283,7 @@ pub const Index = struct {
             .leaf_count = 0,
             .depth = 0,
             .entry_count = 0,
+            .epoch = 0,
             .scratch = undefined,
             .promo = undefined,
         };
@@ -475,7 +491,7 @@ pub const Index = struct {
         // 16 MB).
         const want_pages: u32 = @intCast(@max(
             ovf_extent_pages,
-            (overflow_bytes + pgr.page_size - 1) / pgr.page_size,
+            pgr.pages_for(overflow_bytes),
         ));
         try self.pager.reserve_pages(&self.hold, want_pages);
         const first = self.pager.alloc_pages_assume_reserved(&self.hold, want_pages);
@@ -614,6 +630,10 @@ pub const Index = struct {
         self.depth = 0;
         self.entry_count = 0;
         self.multikey = false;
+        // Node ids 0 and 1 were just re-created as different nodes, so every
+        // position anyone saved into the old tree now points somewhere valid
+        // and wrong. This is the bump that tells them apart.
+        self.epoch += 1;
     }
 
     /// Remove every entry for `id`, in one pass over the leaves. Infallible.
@@ -777,6 +797,14 @@ pub const Index = struct {
         leaf: u32,
         slot: u32,
 
+        /// `next`, plus the position of the entry it yielded. Exact because
+        /// `next` leaves `leaf` alone on the call that yields and has already
+        /// incremented `slot` past the entry.
+        pub fn positioned(self: *Iter) ?Positioned {
+            const e = self.next() orelse return null;
+            return .{ .key = e.key, .off = e.off, .leaf = self.leaf, .slot = self.slot - 1 };
+        }
+
         pub fn next(self: *Iter) ?EntryRef {
             const ix = self.ix;
             while (self.leaf != 0) {
@@ -798,6 +826,70 @@ pub const Index = struct {
         return .{ .ix = self, .leaf = self.first_leaf, .slot = 0 };
     }
 
+    /// TEMPORARY: does the parent's child order match the leaf chain, and is each
+    /// separator really its child's first key?
+    pub fn dbg_root(self: *const Index) void {
+        const rt = self.page(self.root);
+        std.debug.print("  root={d} count={d} first_child={d} depth={d}\n", .{ self.root, rt.count, rt.first_child, self.depth });
+        // chain position of every leaf
+        var pos_of = std.mem.zeroes([512]i32);
+        for (&pos_of) |*v| v.* = -1;
+        var lf = self.first_leaf;
+        var pos: i32 = 0;
+        while (lf != 0) : (pos += 1) {
+            if (lf < pos_of.len) pos_of[lf] = pos;
+            lf = self.page(lf).next;
+        }
+        var prev_pos: i32 = pos_of[rt.first_child];
+        var i: u32 = 0;
+        while (i < rt.count) : (i += 1) {
+            const child = get_slot(rt, i).extra;
+            const cp = if (child < pos_of.len) pos_of[child] else -2;
+            const sep = self.key_of(self.root, i);
+            const cfirst = if (self.page(child).count > 0) self.key_of(child, 0) else "";
+            const sep_wrong = cfirst.len > 0 and !std.mem.eql(u8, sep, cfirst);
+            const order_wrong = cp != prev_pos + 1;
+            if (sep_wrong or order_wrong) {
+                std.debug.print("    slot[{d}] child={d} chainpos={d} (prev {d}){s}{s}\n      sep  ={x}\n      first={x}\n", .{
+                    i,                                 child,                                cp,  prev_pos,
+                    if (order_wrong) " ORDER" else "", if (sep_wrong) " SEP!=FIRST" else "", sep, cfirst,
+                });
+            }
+            prev_pos = cp;
+        }
+    }
+
+    /// Debug aid: how many distinct keys are present in the leaf chain but not
+    /// findable by descending from the root.
+    ///
+    /// Iteration and descent are two independent ways to reach an entry, and a
+    /// query only ever uses descent. `count()` cannot tell them apart -- it
+    /// returns a stored counter -- so an index whose leaves are intact but whose
+    /// interior nodes no longer route to them looks perfectly healthy by every
+    /// other measure, and silently answers a query with fewer documents than it
+    /// holds. That is what the crash fuzzer caught: a handful of key values
+    /// returning nothing while every other value was exact.
+    ///
+    /// O(distinct keys x depth). For assertions and tests, not for the hot path.
+    pub fn unreachable_key_count(self: *const Index) u32 {
+        var bad: u32 = 0;
+        var it = self.iter();
+        var prev: ?[]const u8 = null;
+        while (it.next()) |e| {
+            if (prev) |p| {
+                if (std.mem.eql(u8, p, e.key)) continue;
+            }
+            prev = e.key;
+            var probe = self.seek(e.key);
+            const first = probe.next() orelse {
+                bad += 1;
+                continue;
+            };
+            if (cmp_prefix(e.key, first.key) != .eq) bad += 1;
+        }
+        return bad;
+    }
+
     /// Reverse ordered iteration. Leaves are doubly linked and `prev` has
     /// always been maintained -- nothing walked it until now, so a descending
     /// scan had to materialize every candidate and reverse the list. This turns
@@ -808,6 +900,13 @@ pub const Index = struct {
         /// One past the slot to yield next, so 0 means this leaf is done.
         slot: u32,
 
+        /// As `Iter.positioned`, but `RevIter.next` decrements *onto* the entry
+        /// it yields, so the slot needs no adjustment.
+        pub fn positioned(self: *RevIter) ?Positioned {
+            const e = self.next() orelse return null;
+            return .{ .key = e.key, .off = e.off, .leaf = self.leaf, .slot = self.slot };
+        }
+
         pub fn next(self: *RevIter) ?EntryRef {
             const ix = self.ix;
             while (self.leaf != 0) {
@@ -855,6 +954,174 @@ pub const Index = struct {
         return .{ .ix = self, .leaf = b.leaf, .slot = b.slot };
     }
 
+    // -- resuming an interrupted scan ---------------------------------------
+
+    /// Entries a resume will walk past before giving up and reporting `capped`.
+    /// A bound rather than a hope: `seek` lands at the *start* of an equal-key
+    /// band, so without one a key with millions of duplicates would make every
+    /// batch cost O(band) and a full drain quadratic.
+    pub const resume_walk_max: u32 = 1 << 16;
+
+    /// One entry, with enough of its position to resume after it next time.
+    pub const Positioned = struct {
+        key: []const u8,
+        off: u64,
+        leaf: u32,
+        slot: u32,
+    };
+
+    /// A resumed forward walk. `capped` means the anchor could not be located
+    /// within `resume_walk_max` steps, so the position is not trustworthy and
+    /// the caller must fail rather than return documents from the wrong place.
+    pub const Resumed = struct { it: Iter, capped: bool = false };
+    pub const ResumedRev = struct { it: RevIter, capped: bool = false };
+
+    /// Does `(leaf, slot)` still hold exactly `(key, off)`?
+    ///
+    /// A hint is never believed, only checked, and the checks are ordered so the
+    /// cheap structural ones run first: `off_of` asserts `is_leaf` with
+    /// `std.debug.assert`, which in ReleaseFast is a promise to the optimizer
+    /// rather than a check, so `is_leaf` must be tested for real beforehand.
+    ///
+    /// Node ids are append-only, so a stale id is always in bounds; what makes a
+    /// hint dangerous rather than merely wrong is `reset_tree` re-creating ids 0
+    /// and 1 as different nodes, and `Index.epoch` is what the caller compares
+    /// for that.
+    fn hint_holds(self: *const Index, leaf: u32, slot: u32, key: []const u8, off: u64) bool {
+        if (leaf == 0 or leaf >= self.node_pages.items.len) return false;
+        const node = self.page(leaf);
+        if (node.is_leaf != 1) return false;
+        if (slot >= node.count) return false;
+        if (self.off_of(leaf, slot) != off) return false;
+        return std.mem.eql(u8, self.key_of(leaf, slot), key);
+    }
+
+    /// Locate the anchor `(key, off)` by walking its equal-key band.
+    ///
+    /// Comparison is `std.mem.order`, not `cmp_prefix`, because the band is
+    /// defined as the entries whose key is byte-equal to the anchor's and prefix
+    /// semantics would call `"ab"` and `"abc"` equal. In fairness the two happen
+    /// to agree on where this function resumes -- the fallback is positional, and
+    /// the first out-of-band entry is the same entry either way -- so this is a
+    /// clarity choice, not a bug fix; an attempted mutation to `cmp_prefix` does
+    /// not change any observable result. What does matter is that `lower_bound`
+    /// uses prefix semantics and therefore errs *before* the band, never past it,
+    /// so the walk cannot start beyond the anchor and skip it.
+    ///
+    /// When the anchor is gone, "gone" turns out to mean two different things and
+    /// they want opposite answers:
+    ///
+    ///   - **Deleted.** A sibling has moved up into the anchor's band position, and
+    ///     that sibling has not been returned yet. Resume *at* band position
+    ///     `band_index`. Resuming after the whole band instead would silently drop
+    ///     every remaining member, which on a three-value index is most of the
+    ///     collection.
+    ///   - **Updated.** The document was rewritten, so its key is unchanged but its
+    ///     offset moved. The entry at the anchor's band position *is* the anchor,
+    ///     already returned. Resume *after* it.
+    ///
+    /// The index cannot tell these apart in general -- both look like "same key,
+    /// different offset". On a **unique** index it can: two entries cannot share a
+    /// key, so a same-key entry is necessarily the same document, hence the update
+    /// case, hence resume past the band. That covers `_id_` and so every unsorted
+    /// scan and every `_id` sort, which is where an update-during-drain otherwise
+    /// returns a document twice -- observed as duplicate `_id`s draining a
+    /// collection that was being updated underneath.
+    ///
+    /// On a non-unique index the positional fallback stands, so an updated document
+    /// may come back a second time. That is legal: MongoDB documents that a
+    /// non-snapshot cursor may return a document more than once if an intervening
+    /// write moves it.
+    fn band_resume(self: *const Index, key: []const u8, off: u64, band_index: u64) Resumed {
+        var it = self.seek(key);
+        var fallback: ?Iter = null;
+        var pos: u64 = 0;
+        var steps: u32 = 0;
+        while (steps < resume_walk_max) : (steps += 1) {
+            // The iterator state that would yield the entry we are about to
+            // look at, i.e. "resume *at* this entry".
+            const before = it;
+            const e = it.next() orelse break;
+            if (std.mem.order(u8, e.key, key) != .eq) {
+                // Past the band. Prefer the fallback if the band held one.
+                return .{ .it = fallback orelse before };
+            }
+            if (e.off == off) return .{ .it = it }; // resume just after the anchor
+            // On a unique index a same-key entry can only be the anchor itself,
+            // rewritten, so there is no sibling to fall back to.
+            if (!self.unique and pos == band_index and fallback == null) fallback = before;
+            pos += 1;
+        }
+        if (steps == resume_walk_max) return .{ .it = it, .capped = true };
+        return .{ .it = fallback orelse it };
+    }
+
+    /// A forward walk positioned just after `(key, off)`.
+    ///
+    /// O(1) whenever the hint still holds, which is the case unless something
+    /// wrote to that exact leaf between batches. The band walk is the fallback,
+    /// and it is what the walk bound exists to contain.
+    pub fn resume_forward(
+        self: *const Index,
+        key: []const u8,
+        off: u64,
+        band_index: u64,
+        hint_leaf: u32,
+        hint_slot: u32,
+        hint_trusted: bool,
+    ) Resumed {
+        if (hint_trusted and self.hint_holds(hint_leaf, hint_slot, key, off)) {
+            return .{ .it = .{ .ix = self, .leaf = hint_leaf, .slot = hint_slot + 1 } };
+        }
+        return self.band_resume(key, off, band_index);
+    }
+
+    /// A reverse walk positioned just before `(key, off)` in key order, i.e. at
+    /// the next entry a descending scan owes.
+    ///
+    /// `RevIter` decrements before yielding, so slot `s` yields `s - 1` -- the
+    /// entry immediately below the anchor -- and crosses into `prev` when the
+    /// anchor sat at slot 0.
+    ///
+    /// Known limitation, and it is a deliberate trade. When the anchor is gone
+    /// *and* it had duplicates, this resumes below the whole band rather than at
+    /// the anchor's position within it, so the band's remaining members are not
+    /// returned. Placing a reverse fallback exactly would need the band's length,
+    /// which is only known after walking it, hence a second walk on a path that
+    /// requires a descending scan over a duplicate-heavy index whose anchor was
+    /// deleted mid-cursor. Forward resumes -- every unsorted scan and every
+    /// ascending sort -- use `band_index` and have no such gap.
+    pub fn resume_reverse(
+        self: *const Index,
+        key: []const u8,
+        off: u64,
+        hint_leaf: u32,
+        hint_slot: u32,
+        hint_trusted: bool,
+    ) ResumedRev {
+        if (hint_trusted and self.hint_holds(hint_leaf, hint_slot, key, off)) {
+            return .{ .it = .{ .ix = self, .leaf = hint_leaf, .slot = hint_slot } };
+        }
+        // Find the anchor by walking forward, then turn around on it.
+        var it = self.seek(key);
+        var steps: u32 = 0;
+        while (steps < resume_walk_max) : (steps += 1) {
+            const e = it.next() orelse break;
+            if (std.mem.order(u8, e.key, key) != .eq) break; // past the band
+            if (e.off == off) {
+                // `it` has already stepped past the anchor, so the anchor sat at
+                // `it.slot - 1` and a RevIter there yields the entry below it.
+                return .{ .it = .{ .ix = self, .leaf = it.leaf, .slot = it.slot - 1 } };
+            }
+        }
+        if (steps == resume_walk_max) {
+            return .{ .it = .{ .ix = self, .leaf = 0, .slot = 0 }, .capped = true };
+        }
+        // Anchor gone: resume below the band.
+        const b = self.lower_bound(key);
+        return .{ .it = .{ .ix = self, .leaf = b.leaf, .slot = b.slot } };
+    }
+
     // -- serialization ------------------------------------------------------
 
     /// The canonical spec document bytes
@@ -1242,6 +1509,36 @@ pub const Index = struct {
 
     /// Separator position in an internal node: after any equal keys, so the
     /// "last separator <= key" descent lands on the newest right child.
+    /// The slot at which a new right sibling of `left` belongs: immediately after
+    /// `left`'s own position among this node's children.
+    ///
+    /// Deliberately positional, not a search for the promoted key. The two agree
+    /// only while separators are distinct. When several children share a
+    /// separator -- ten distinct values across thousands of documents, so each
+    /// value spans dozens of leaves -- `separator_pos` returns the slot after the
+    /// *whole* equal-key run, which puts the new sibling at the end of that run
+    /// while the leaf chain has it immediately after `left`.
+    ///
+    /// Parent child order then no longer matches leaf chain order, and that is
+    /// the one thing a lookup cannot survive: `descend_lower` picks the last
+    /// child of the equal run, and `lookup_eq` walks forward from there over keys
+    /// that are *smaller* than the one it wants, so it stops at the first
+    /// mismatch and reports nothing. The entries are all present, the chain is
+    /// correctly ordered, `count()` is right -- and a query returns an empty
+    /// result. Found by tests/fuzz/crash-fuzz.js after ~700 heavy cycles as
+    /// `find({k: 3})` returning 0 of 401 documents while every other key was
+    /// exact.
+    fn child_slot_after(self: *const Index, node_id: u32, left: u32) u32 {
+        const node = self.page(node_id);
+        if (node.first_child == left) return 0;
+        var i: u32 = 0;
+        while (i < node.count) : (i += 1) {
+            if (get_slot(node, i).extra == left) return i + 1;
+        }
+        assert_msg(false, "a split's left sibling must be a child of the node taking its separator");
+        return node.count;
+    }
+
     fn separator_pos(self: *const Index, node_id: u32, key: []const u8) u32 {
         const node = self.page(node_id);
         var lo: u32 = 0;
@@ -1352,7 +1649,7 @@ pub const Index = struct {
         }
         const child = self.descend_insert(node_id, key);
         const res = self.insert_rec(child, key, off) orelse return null;
-        return self.insert_separator(node_id, res);
+        return self.insert_separator(node_id, child, res);
     }
 
     /// Split a full leaf around the record being inserted. The new record
@@ -1426,7 +1723,7 @@ pub const Index = struct {
 
     /// Insert a promoted separator into an internal node, splitting it when
     /// full. Returns the next promotion, or null.
-    fn insert_separator(self: *Index, node_id: u32, split: Split) ?Split {
+    fn insert_separator(self: *Index, node_id: u32, left: u32, split: Split) ?Split {
         // The incoming key may live in the promo buffer, which a nested
         // split_internal (below) would overwrite with its own promoted key;
         // spilled keys already live in the immutable slab. Copy inline keys
@@ -1443,7 +1740,7 @@ pub const Index = struct {
             self.repack_keep_prefix(node_id, self.page(node_id).count);
         }
         if (self.fits(node_id, key.len)) {
-            self.store_record(node_id, self.separator_pos(node_id, key), .{
+            self.store_record(node_id, self.child_slot_after(node_id, left), .{
                 .key = key,
                 .child = split.right,
                 .spill_off = split.spill_off,
@@ -1451,7 +1748,7 @@ pub const Index = struct {
             self.page_mut(split.right).parent = node_id;
             return null;
         }
-        return self.split_internal(node_id, key, split.spill_off, split.right);
+        return self.split_internal(node_id, left, key, split.spill_off, split.right);
     }
 
     /// Split a full internal node around the separator being inserted: the
@@ -1462,13 +1759,15 @@ pub const Index = struct {
     fn split_internal(
         self: *Index,
         node_id: u32,
+        left: u32,
         key: []const u8,
         spill_off: ?u64,
         child: u32,
     ) Split {
         const old_count = self.page(node_id).count;
         std.debug.assert(old_count >= 2);
-        const pos = self.separator_pos(node_id, key);
+        // Positional, for the reason `child_slot_after` documents.
+        const pos = self.child_slot_after(node_id, left);
         const n = old_count + 1;
 
         var costs: [max_slots + 1]u32 = undefined;
@@ -1614,6 +1913,9 @@ pub const Index = struct {
         self.first_leaf = self.root;
         self.leaf_count = 1;
         self.depth = 0;
+        // The root's id is unchanged but it is a leaf now, so a saved position
+        // that named it as an internal node describes a different tree shape.
+        self.epoch += 1;
     }
 
     /// Pack the sorted staging array into a fresh tree: leaves filled in
@@ -2345,6 +2647,50 @@ fn simple_index(
     return Index.init(gpa, pager, "test", keys[0..paths.len], unique, sparse, null);
 }
 
+test "a bulk build with many duplicate keys stays reachable for every key" {
+    // The shape the crash fuzzer failed on: ~4000 entries over 10 distinct key
+    // values, so each value spans several leaves and the interior separators
+    // repeat. `find({k:v})` came back empty for a few values and exactly right
+    // for the rest, with `count()` still reporting every entry -- entries that
+    // exist and cannot be reached.
+    const gpa = testing.allocator;
+    var ix = try simple_index(gpa, test_pager(), &.{"k"}, false, false);
+    defer ix.deinit(gpa);
+
+    const n_docs: usize = 4000;
+    const n_keys: i32 = 10;
+    var docs: std.ArrayListUnmanaged([]u8) = .empty;
+    defer {
+        for (docs.items) |d| gpa.free(d);
+        docs.deinit(gpa);
+    }
+    for (0..n_docs) |i| {
+        const k: i32 = @intCast(@mod(@as(i32, @intCast(i)), n_keys));
+        const pairs = [_]bson.Pair{.{ .key = "k", .value = .{ .int32 = k } }};
+        const bytes = try bytes_of(gpa, &pairs);
+        try docs.append(gpa, bytes);
+        try ix.append_doc_entries(gpa, bytes, @intCast(i + 1));
+    }
+    _ = try ix.finish_bulk(gpa, false);
+    try testing.expectEqual(n_docs, ix.count());
+
+    // Iteration must see every entry: that separates "never inserted" from
+    // "inserted and unreachable from the root".
+    var walked: usize = 0;
+    var wit = ix.iter();
+    while (wit.next()) |_| walked += 1;
+    try testing.expectEqual(n_docs, walked);
+
+    // And every key must be reachable by descent, which is what a query does.
+    var k: i32 = 0;
+    while (k < n_keys) : (k += 1) {
+        var out: std.ArrayListUnmanaged(u64) = .empty;
+        defer out.deinit(gpa);
+        try ix.lookup_eq(gpa, &.{.{ .int32 = k }}, &out);
+        try testing.expectEqual(n_docs / @as(usize, @intCast(n_keys)), out.items.len);
+    }
+}
+
 /// Look up the documents under `key` and compare with the expected offsets.
 /// A leaf record's payload is a slab offset now, so tests identify documents by
 /// small distinct numbers rather than by byte-string ids.
@@ -2884,6 +3230,159 @@ test "incremental inserts and removals stay identical to a brute-force model" {
     }
 }
 
+test "splitting inside a run of equal separators keeps every key reachable" {
+    // The crash fuzzer's exact shape, and the reason the two differentials above
+    // miss it: bulk-pack first, *then* keep inserting.
+    //
+    // A packed tree has full leaves, so the next inserts split leaves in the
+    // middle of a run of equal separators -- and a new right sibling placed by
+    // key rather than by position lands at the end of that run, so the parent's
+    // child order stops matching the leaf chain. A purely incremental build
+    // leaves leaves half full and rarely produces the geometry.
+    const gpa = testing.allocator;
+    var ix = try simple_index(gpa, test_pager(), &.{"k"}, false, false);
+    defer ix.deinit(gpa);
+
+    const n_keys: i32 = 10;
+    const packed_docs: usize = 4000;
+    const grown_docs: usize = 2000;
+
+    // Phase 1: bulk pack, which fills every leaf.
+    for (0..packed_docs) |i| {
+        const k: i32 = @intCast(@mod(@as(i32, @intCast(i)), n_keys));
+        const d = try bytes_of(gpa, &.{.{ .key = "k", .value = .{ .int32 = k } }});
+        defer gpa.free(d);
+        try ix.append_doc_entries(gpa, d, @intCast(i + 1));
+    }
+    _ = try ix.finish_bulk(gpa, false);
+    ix.pager.release_reservation(&ix.hold);
+
+    // Phase 2: grow it in random key order, which is what puts a split on the
+    // leaf *before* an equal-key run -- the case where the new right sibling's
+    // promoted key equals the run's key while its chain position is at the run's
+    // start. Round-robin never produces it: every insert routes to the last
+    // child of its run, and a split there belongs at the end of the run anyway.
+    var prng = std.Random.DefaultPrng.init(0xbad_5eed);
+    const rand = prng.random();
+    for (0..grown_docs) |j| {
+        const k = rand.intRangeLessThan(i32, 0, n_keys);
+        const d = try bytes_of(gpa, &.{.{ .key = "k", .value = .{ .int32 = k } }});
+        defer gpa.free(d);
+        _ = try ix.add_doc(gpa, d, @intCast(packed_docs + j + 1), false);
+        ix.pager.release_reservation(&ix.hold);
+        if (j % 25 == 0) {
+            testing.expectEqual(@as(u32, 0), ix.unreachable_key_count()) catch |err| {
+                std.debug.print("  went unreachable after {d} grown inserts\n", .{j});
+                return err;
+            };
+        }
+    }
+
+    try testing.expectEqual(packed_docs + grown_docs, ix.count());
+    // Every entry the chain holds must be findable by descent too.
+    try testing.expectEqual(@as(u32, 0), ix.unreachable_key_count());
+
+    // And a lookup must find as many entries as iteration holds for that key.
+    var k: i32 = 0;
+    while (k < n_keys) : (k += 1) {
+        var enc: std.ArrayListUnmanaged(u8) = .empty;
+        defer enc.deinit(gpa);
+        try bson.encode_key(bson.Value{ .int32 = k }, gpa, &enc);
+        var want: usize = 0;
+        var wit = ix.iter();
+        while (wit.next()) |e| {
+            if (cmp_prefix(enc.items, e.key) == .eq) want += 1;
+        }
+        var out: std.ArrayListUnmanaged(u64) = .empty;
+        defer out.deinit(gpa);
+        try ix.lookup_eq(gpa, &.{.{ .int32 = k }}, &out);
+        testing.expectEqual(want, out.items.len) catch |err| {
+            std.debug.print("  key {d}: reachable {d}, iteration holds {d} (entry_count {d})\n", .{
+                k, out.items.len, want, ix.count(),
+            });
+            return err;
+        };
+    }
+}
+
+test "an incrementally built index with few distinct keys stays reachable" {
+    // The crash fuzzer's shape, which the existing differentials miss: a single
+    // key with only ten distinct values over thousands of documents, so each
+    // value spans dozens of leaves and most interior separators are duplicates.
+    // The compound-key differential above uses ~961 combinations over 600
+    // inserts, which is almost no duplication at all.
+    //
+    // Symptom being hunted: `lookup_eq` returning nothing for a few values while
+    // every other value is exactly right, and `count()` still reporting every
+    // entry -- entries that exist and cannot be reached by descent.
+    const gpa = testing.allocator;
+    var prng = std.Random.DefaultPrng.init(0xd0_0d_1e);
+    const rand = prng.random();
+    var ix = try simple_index(gpa, test_pager(), &.{"k"}, false, false);
+    defer ix.deinit(gpa);
+
+    const n_keys: i32 = 10;
+    const n: usize = 3000;
+    var keys: std.ArrayListUnmanaged(i32) = .empty;
+    defer keys.deinit(gpa);
+    var live: std.ArrayListUnmanaged(bool) = .empty;
+    defer live.deinit(gpa);
+
+    // Interleave inserts and removals, which is what a real workload does and
+    // what leaves half-empty leaves and one-child internal nodes behind.
+    for (0..n) |i| {
+        const k = rand.intRangeAtMost(i32, 0, n_keys - 1);
+        const off: u64 = @intCast(i + 1);
+        const d = try bytes_of(gpa, &.{.{ .key = "k", .value = .{ .int32 = k } }});
+        defer gpa.free(d);
+        _ = try ix.add_doc(gpa, d, off, false);
+        // A bare index test has no engine to do this at a write boundary, and
+        // without it the promise accumulates and grows the shared test file
+        // without bound.
+        ix.pager.release_reservation(&ix.hold);
+        try keys.append(gpa, k);
+        try live.append(gpa, true);
+
+        // Remove an earlier document every few inserts.
+        if (i > 20 and i % 3 == 0) {
+            const victim = rand.intRangeLessThan(usize, 0, keys.items.len);
+            if (live.items[victim]) {
+                const vd = try bytes_of(gpa, &.{.{ .key = "k", .value = .{ .int32 = keys.items[victim] } }});
+                defer gpa.free(vd);
+                ix.remove_doc(gpa, vd, @intCast(victim + 1));
+                live.items[victim] = false;
+            }
+        }
+
+        // Every value must be reachable by descent, and the counts must match a
+        // brute-force pass over the model. Checked periodically rather than every
+        // step: this is 10 descents over a tree of thousands of entries.
+        if (i % 250 != 0 and i != n - 1) continue;
+        var k_check: i32 = 0;
+        while (k_check < n_keys) : (k_check += 1) {
+            var want: usize = 0;
+            for (keys.items, live.items) |kk, is_live| {
+                if (is_live and kk == k_check) want += 1;
+            }
+            var out: std.ArrayListUnmanaged(u64) = .empty;
+            defer out.deinit(gpa);
+            try ix.lookup_eq(gpa, &.{.{ .int32 = k_check }}, &out);
+            testing.expectEqual(want, out.items.len) catch |err| {
+                std.debug.print(
+                    "  at insert {d}: key {d} reachable {d}, expected {d} (entry_count {d})\n",
+                    .{ i, k_check, out.items.len, want, ix.count() },
+                );
+                return err;
+            };
+        }
+        // And iteration must see exactly as many entries as the tree claims.
+        var walked: usize = 0;
+        var wit = ix.iter();
+        while (wit.next()) |_| walked += 1;
+        try testing.expectEqual(ix.count(), walked);
+    }
+}
+
 /// One document's facts in the incremental-mutation differential.
 const ModelFact = struct { a: i32, b: i32, off: u64 };
 
@@ -3379,3 +3878,258 @@ test "planner picks eq run, ranges, and bails on sparse null" {
         try testing.expect((try plan(gpa, null, &.{&ix}, &f, &.{})) == null);
     }
 }
+
+/// Drain `ix` by resuming every `stride` entries, the way a cursor with that
+/// batch size would, and return the offsets in the order they came out.
+fn drain_resuming(
+    gpa: std.mem.Allocator,
+    ix: *const Index,
+    stride: u32,
+    backward: bool,
+    out: *std.ArrayListUnmanaged(u64),
+) !void {
+    var started = false;
+    var anchor: std.ArrayListUnmanaged(u8) = .empty;
+    defer anchor.deinit(gpa);
+    var anchor_off: u64 = 0;
+    var band_index: u64 = 0;
+    var hint_leaf: u32 = 0;
+    var hint_slot: u32 = 0;
+
+    while (true) {
+        // One "batch": open a walk where the last one stopped.
+        var fwd: Index.Iter = undefined;
+        var rev: Index.RevIter = undefined;
+        if (!started) {
+            if (backward) rev = ix.iter_reverse() else fwd = ix.iter();
+        } else if (backward) {
+            const r = ix.resume_reverse(anchor.items, anchor_off, hint_leaf, hint_slot, true);
+            try testing.expect(!r.capped);
+            rev = r.it;
+        } else {
+            const r = ix.resume_forward(
+                anchor.items,
+                anchor_off,
+                band_index,
+                hint_leaf,
+                hint_slot,
+                true,
+            );
+            try testing.expect(!r.capped);
+            fwd = r.it;
+        }
+
+        var n: u32 = 0;
+        while (n < stride) : (n += 1) {
+            const e = if (backward)
+                rev.positioned()
+            else
+                fwd.positioned();
+            const got = e orelse return;
+            try out.append(gpa, got.off);
+            if (started and std.mem.eql(u8, anchor.items, got.key)) {
+                band_index += 1;
+            } else {
+                band_index = 0;
+            }
+            anchor.clearRetainingCapacity();
+            try anchor.appendSlice(gpa, got.key);
+            anchor_off = got.off;
+            hint_leaf = got.leaf;
+            hint_slot = got.slot;
+            started = true;
+        }
+    }
+}
+
+test "a resumed walk yields exactly what an uninterrupted one does" {
+    // The property the whole streaming cursor rests on: stopping and restarting
+    // a scan changes nothing about what it returns, in either direction, at any
+    // batch size, including one entry at a time.
+    //
+    // Mutation-checked: changing `resume_forward`'s `hint_slot + 1` to
+    // `hint_slot` makes every batch boundary repeat an entry, and this test goes
+    // red. (A third mutation was tried and rejected as meaningless: swapping
+    // `std.mem.order` for `cmp_prefix` inside `band_resume` changes nothing
+    // observable, so no test can catch it -- see the note there.)
+    //
+    // Note this test always resumes from a *valid* hint, since nothing mutates
+    // the tree between its batches. The band walk is covered by the two tests
+    // below, which invalidate the hint on purpose.
+    const gpa = testing.allocator;
+
+    // Three corpora, each hard for a different reason: distinct keys spanning
+    // several leaves and an interior level; a low-cardinality index whose bands
+    // span leaves; and *variable-length string keys in prefix relationships*
+    // ("a" < "ab" < "abc"), which is the only shape that can tell `std.mem.order`
+    // apart from `cmp_prefix` -- fixed-width integer keys never differ, so an
+    // integer-only corpus cannot catch that mistake at all.
+    const Shape = enum { distinct, duplicates, prefixes };
+    for ([_]Shape{ .distinct, .duplicates, .prefixes }) |shape| {
+        var ix = try simple_index(gpa, test_pager(), &.{"a"}, false, false);
+        defer ix.deinit(gpa);
+
+        const n = 400;
+        var key_buf: [40]u8 = undefined;
+        for (0..n) |i| {
+            const value: bson.Value = switch (shape) {
+                .distinct => .{ .int32 = @intCast(i + 1) },
+                .duplicates => .{ .int32 = @intCast(i % 3) },
+                // Every key is a prefix of the next in its group of eight, so
+                // each band start is also a proper prefix of later keys.
+                .prefixes => blk: {
+                    const written = try std.fmt.bufPrint(&key_buf, "k{d}", .{i / 8});
+                    const depth = (i % 8) + 1;
+                    @memset(key_buf[written.len .. written.len + depth], 'x');
+                    break :blk .{ .string = key_buf[0 .. written.len + depth] };
+                },
+            };
+            const d = try bytes_of(gpa, &.{
+                .{ .key = "_id", .value = .{ .int32 = @intCast(i + 1) } },
+                .{ .key = "a", .value = value },
+            });
+            defer gpa.free(d);
+            _ = try ix.add_doc(gpa, d, @intCast(i + 1), false);
+        }
+        try testing.expect(ix.depth >= 1);
+
+        for ([_]bool{ false, true }) |backward| {
+            var whole: std.ArrayListUnmanaged(u64) = .empty;
+            defer whole.deinit(gpa);
+            if (backward) {
+                var it = ix.iter_reverse();
+                while (it.next()) |e| try whole.append(gpa, e.off);
+            } else {
+                var it = ix.iter();
+                while (it.next()) |e| try whole.append(gpa, e.off);
+            }
+            try testing.expectEqual(@as(usize, n), whole.items.len);
+
+            for ([_]u32{ 1, 2, 7, 101, 399, 400, 1000 }) |stride| {
+                var resumed: std.ArrayListUnmanaged(u64) = .empty;
+                defer resumed.deinit(gpa);
+                try drain_resuming(gpa, &ix, stride, backward, &resumed);
+                try testing.expectEqualSlices(u64, whole.items, resumed.items);
+            }
+        }
+    }
+}
+
+test "a resume survives a split between batches" {
+    // A cursor holds no lock, so the tree it comes back to is not the tree it
+    // left. Inserting mid-drain moves entries between leaves and invalidates the
+    // position hint, which is exactly what the anchor is for.
+    const gpa = testing.allocator;
+    var ix = try simple_index(gpa, test_pager(), &.{"a"}, false, false);
+    defer ix.deinit(gpa);
+
+    const n = 200;
+    for (0..n) |i| {
+        // Even keys only, so the inserts below land between existing entries.
+        const d = try bytes_of(gpa, &.{
+            .{ .key = "_id", .value = .{ .int32 = @intCast(i + 1) } },
+            .{ .key = "a", .value = .{ .int32 = @intCast((i + 1) * 2) } },
+        });
+        defer gpa.free(d);
+        _ = try ix.add_doc(gpa, d, @intCast(i + 1), false);
+    }
+
+    var seen: std.ArrayListUnmanaged(u64) = .empty;
+    defer seen.deinit(gpa);
+    var anchor: std.ArrayListUnmanaged(u8) = .empty;
+    defer anchor.deinit(gpa);
+    var anchor_off: u64 = 0;
+    var hint_leaf: u32 = 0;
+    var hint_slot: u32 = 0;
+    var started = false;
+    var next_id: i32 = 10_000;
+
+    while (true) {
+        var it = if (!started) ix.iter() else blk: {
+            const r = ix.resume_forward(anchor.items, anchor_off, 0, hint_leaf, hint_slot, true);
+            try testing.expect(!r.capped);
+            break :blk r.it;
+        };
+        var n_in_batch: u32 = 0;
+        while (n_in_batch < 5) : (n_in_batch += 1) {
+            const got = it.positioned() orelse break;
+            try seen.append(gpa, got.off);
+            anchor.clearRetainingCapacity();
+            try anchor.appendSlice(gpa, got.key);
+            anchor_off = got.off;
+            hint_leaf = got.leaf;
+            hint_slot = got.slot;
+            started = true;
+        }
+        if (n_in_batch < 5) break;
+
+        // Between batches, insert odd keys across the whole range: guaranteed to
+        // split leaves and to appear both before and after the anchor.
+        for (0..20) |k| {
+            next_id += 1;
+            const d = try bytes_of(gpa, &.{
+                .{ .key = "_id", .value = .{ .int32 = next_id } },
+                .{ .key = "a", .value = .{ .int32 = @intCast(k * 19 + 1) } },
+            });
+            defer gpa.free(d);
+            _ = try ix.add_doc(gpa, d, @intCast(next_id), false);
+        }
+    }
+
+    // The 200 originals must each appear exactly once. Entries inserted behind
+    // the cursor may or may not show up -- that is ordinary non-snapshot cursor
+    // behaviour -- but nothing may be duplicated or lost.
+    var originals: u32 = 0;
+    var counts = std.AutoHashMap(u64, u32).init(gpa);
+    defer counts.deinit();
+    for (seen.items) |off| {
+        const e = try counts.getOrPutValue(off, 0);
+        e.value_ptr.* += 1;
+        try testing.expectEqual(@as(u32, 1), e.value_ptr.*); // no duplicates
+        if (off <= n) originals += 1;
+    }
+    try testing.expectEqual(@as(u32, n), originals);
+}
+
+test "a resume whose anchor was deleted keeps the rest of its band" {
+    // The failure this guards against is silent and large: with the anchor gone,
+    // resuming after the whole equal-key band drops every remaining member, and
+    // on a low-cardinality index that is most of the collection.
+    //
+    // Mutation-checked: `pos == band_index + 1` in `band_resume` shifts the
+    // resume by one entry and this test goes red.
+    const gpa = testing.allocator;
+    var ix = try simple_index(gpa, test_pager(), &.{"a"}, false, false);
+    defer ix.deinit(gpa);
+
+    // One key, 50 documents: a single band.
+    for (0..50) |i| {
+        const d = try bytes_of(gpa, &.{
+            .{ .key = "_id", .value = .{ .int32 = @intCast(i + 1) } },
+            .{ .key = "a", .value = .{ .int32 = 7 } },
+        });
+        defer gpa.free(d);
+        _ = try ix.add_doc(gpa, d, @intCast(i + 1), false);
+    }
+
+    // Yield three, then delete the third -- the anchor itself.
+    var it = ix.iter();
+    var third: Index.Positioned = undefined;
+    var band_index: u64 = 0;
+    for (0..3) |i| {
+        third = it.positioned().?;
+        if (i > 0) band_index += 1;
+    }
+    const anchor_key = try gpa.dupe(u8, third.key);
+    defer gpa.free(anchor_key);
+    ix.remove_off(third.off);
+
+    const r = ix.resume_forward(anchor_key, third.off, band_index, third.leaf, third.slot, true);
+    try testing.expect(!r.capped);
+    var rest: u32 = 0;
+    var walk = r.it;
+    while (walk.next()) |_| rest += 1;
+
+    // 50 inserted, 1 deleted, 2 already returned before the anchor: 47 left.
+    try testing.expectEqual(@as(u32, 47), rest);
+}

src/lib.zig

@@ -10,6 +10,7 @@ pub const db = @import("db.zig");
 pub const query = @import("query.zig");
 pub const update = @import("update.zig");
 pub const index = @import("index.zig");
+pub const cursor = @import("cursor.zig");
 pub const pager = @import("pager.zig");
 
 test {
@@ -23,5 +24,6 @@ test {
     _ = @import("query.zig");
     _ = @import("update.zig");
     _ = @import("index.zig");
+    _ = @import("cursor.zig");
     _ = @import("pager.zig");
 }

src/main.zig

@@ -10,6 +10,18 @@ const usage =
     \\  --db <path>   database file (default multiforadb.log)
     \\  --ttl-sweep-secs <n>
     \\                seconds between TTL index sweeps (default 60, 0 disables)
+    \\  --cursor-timeout-ms <n>
+    \\                idle milliseconds before an open cursor is reaped
+    \\                (default 600000, matching MongoDB's cursorTimeoutMillis;
+    \\                0 disables expiry)
+    \\  --cursor-sweep-secs <n>
+    \\                seconds between idle-cursor sweeps (default 4, matching
+    \\                MongoDB's clientCursorMonitorFrequencySecs; 0 disables)
+    \\  --max-open-cursors <n>
+    \\                cursor registry capacity (default 4096). At capacity the
+    \\                least recently used cursor is evicted, and its client
+    \\                sees CursorNotFound -- the same answer an idle timeout
+    \\                gives, which every driver already handles.
     \\  --compact-threshold <bytes>
     \\                minimum log bytes between compactions; suffixes k/m/g
     \\                (default 16m). The actual trigger also scales with the
@@ -44,34 +56,77 @@ fn parse_size_suffix(v: []const u8) ?u64 {
     return n * mult;
 }
 
-pub fn main(init: std.process.Init) !void {
+/// Everything the CLI can set. Parsed apart from `main` so the option table has
-    var port: u16 = 27017;
+/// room to grow without main outgrowing the 70-line limit.
-    var bind_ip: []const u8 = "127.0.0.1";
+const Options = struct {
-    var db_path: []const u8 = "multiforadb.log";
+    port: u16 = 27017,
-    var ttl_sweep_secs: i64 = 60;
+    bind_ip: []const u8 = "127.0.0.1",
-    var compact_threshold: u64 = 16 * 1024 * 1024;
+    db_path: []const u8 = "multiforadb.log",
+    ttl_sweep_secs: i64 = 60,
+    compact_threshold: u64 = 16 * 1024 * 1024,
+    cursor_timeout_ms: i64 = mongo.cursor.default_idle_timeout_ms,
+    cursor_sweep_secs: i64 = 4,
+    max_open_cursors: u32 = mongo.cursor.default_capacity,
+    /// Set when --help was given: print usage and exit without opening anything.
+    help: bool = false,
+};
 
+pub fn main(init: std.process.Init) !void {
+    const opts = try parse_args(init) orelse {
+        try std.Io.File.writeStreamingAll(.stdout(), init.io, usage);
+        return;
+    };
+
+    const oid_gen = mongo.bson.ObjectIdGen.init(init.io);
+    var engine = try mongo.db.Engine.open(init.gpa, init.io, opts.db_path);
+    defer engine.deinit();
+    engine.compact_threshold = opts.compact_threshold;
+    // The engine builds its registry with the defaults so an embedded caller
+    // needs no configuration; the CLI replaces it when asked for something else.
+    try engine.reconfigure_cursors(opts.max_open_cursors, opts.cursor_timeout_ms);
+    std.debug.print("multiforadb: opened database '{s}' (compact threshold {d})\n", .{
+        opts.db_path,
+        opts.compact_threshold,
+    });
+
+    var server = mongo.server.Server{
+        .gpa = init.gpa,
+        .port = opts.port,
+        .bind_ip = opts.bind_ip,
+        .oid_gen = oid_gen,
+        .connection_counter = .init(1),
+        .engine = &engine,
+        .start_time = std.Io.Timestamp.now(init.io, .real),
+        .ttl_sweep_secs = opts.ttl_sweep_secs,
+        .cursor_sweep_secs = opts.cursor_sweep_secs,
+    };
+    try server.run();
+}
+
+/// Null means --help: the caller prints usage and exits.
+fn parse_args(init: std.process.Init) !?Options {
+    var o = Options{};
     var it = std.process.Args.Iterator.init(init.minimal.args);
     defer it.deinit();
     _ = it.next(); // program name
     while (it.next()) |arg| {
         if (std.mem.eql(u8, arg, "--port")) {
             const v = it.next() orelse return error.MissingValue;
-            port = std.fmt.parseInt(u16, v, 10) catch {
+            o.port = std.fmt.parseInt(u16, v, 10) catch {
                 std.debug.print("multiforadb: invalid port '{s}'\n", .{v});
                 return error.InvalidPort;
             };
         } else if (std.mem.eql(u8, arg, "--bind")) {
-            bind_ip = it.next() orelse return error.MissingValue;
+            o.bind_ip = it.next() orelse return error.MissingValue;
         } else if (std.mem.eql(u8, arg, "--db")) {
-            db_path = it.next() orelse return error.MissingValue;
+            o.db_path = it.next() orelse return error.MissingValue;
         } else if (std.mem.eql(u8, arg, "--ttl-sweep-secs")) {
             const v = it.next() orelse return error.MissingValue;
-            // i64 is the width std.Io.Duration.fromSeconds takes, so the
+            // i64 is the width std.Io.Duration.fromSeconds takes, so the value
-            // value reaches the sweeper without a cast; negatives are the
+            // reaches the sweeper without a cast; negatives are the only thing
-            // only thing parseInt would otherwise let through.
+            // parseInt would otherwise let through.
-            ttl_sweep_secs = std.fmt.parseInt(i64, v, 10) catch -1;
+            o.ttl_sweep_secs = std.fmt.parseInt(i64, v, 10) catch -1;
-            if (ttl_sweep_secs < 0) {
+            if (o.ttl_sweep_secs < 0) {
                 std.debug.print("multiforadb: invalid ttl sweep interval '{s}'\n", .{v});
                 return error.InvalidTtlSweepSecs;
             }
@@ -85,34 +140,45 @@ pub fn main(init: std.process.Init) !void {
                 std.debug.print("multiforadb: compact threshold must be at least 1m\n", .{});
                 return error.InvalidCompactThreshold;
             }
-            compact_threshold = parsed;
+            o.compact_threshold = parsed;
         } else if (std.mem.eql(u8, arg, "--help") or std.mem.eql(u8, arg, "-h")) {
-            try std.Io.File.writeStreamingAll(.stdout(), init.io, usage);
+            return null;
-            return;
+        } else if (try parse_cursor_flag(arg, &it, &o)) {
+            // Handled: one of the cursor-registry flags.
         } else {
             std.debug.print("multiforadb: unknown option '{s}'\n{s}", .{ arg, usage });
             return error.UnknownOption;
         }
     }
-
+    return o;
-    const oid_gen = mongo.bson.ObjectIdGen.init(init.io);
+}
-    var engine = try mongo.db.Engine.open(init.gpa, init.io, db_path);
+
-    defer engine.deinit();
+/// The cursor-registry flags, grouped so `parse_args` stays one flat table of
-    engine.compact_threshold = compact_threshold;
+/// options. Returns whether `arg` was one of them; consumes its value if so.
-    std.debug.print("multiforadb: opened database '{s}' (compact threshold {d})\n", .{
+fn parse_cursor_flag(arg: []const u8, it: *std.process.Args.Iterator, o: *Options) !bool {
-        db_path,
+    if (std.mem.eql(u8, arg, "--cursor-timeout-ms")) {
-        compact_threshold,
+        const v = it.next() orelse return error.MissingValue;
-    });
+        o.cursor_timeout_ms = std.fmt.parseInt(i64, v, 10) catch -1;
-
+        if (o.cursor_timeout_ms < 0) {
-    var server = mongo.server.Server{
+            std.debug.print("multiforadb: invalid cursor timeout '{s}'\n", .{v});
-        .gpa = init.gpa,
+            return error.InvalidCursorTimeout;
-        .port = port,
+        }
-        .bind_ip = bind_ip,
+    } else if (std.mem.eql(u8, arg, "--cursor-sweep-secs")) {
-        .oid_gen = oid_gen,
+        const v = it.next() orelse return error.MissingValue;
-        .connection_counter = .init(1),
+        o.cursor_sweep_secs = std.fmt.parseInt(i64, v, 10) catch -1;
-        .engine = &engine,
+        if (o.cursor_sweep_secs < 0) {
-        .start_time = std.Io.Timestamp.now(init.io, .real),
+            std.debug.print("multiforadb: invalid cursor sweep interval '{s}'\n", .{v});
-        .ttl_sweep_secs = ttl_sweep_secs,
+            return error.InvalidCursorSweepSecs;
-    };
+        }
-    try server.run();
+    } else if (std.mem.eql(u8, arg, "--max-open-cursors")) {
+        const v = it.next() orelse return error.MissingValue;
+        o.max_open_cursors = std.fmt.parseInt(u32, v, 10) catch 0;
+        if (o.max_open_cursors == 0 or o.max_open_cursors > mongo.cursor.max_capacity) {
+            std.debug.print("multiforadb: invalid max open cursors '{s}'\n", .{v});
+            return error.InvalidMaxOpenCursors;
+        }
+    } else {
+        return false;
+    }
+    return true;
 }

src/pager.zig

@@ -27,7 +27,8 @@
 //!     [56..64)    u64 freelist_len
 //!     [64..72)    u64 prev_generation -- kept intact for fallback
 //!     [72..80)    u64 live_docs    -- cached hint
-//!     [80..88)    u64 dead_bytes   -- cached, drives the rebuild trigger
+//!     [80..88)    u64 dead_bytes   -- cached hint; the engine recomputes it
+//!                                     from the catalog on open
 //!     [88..4088)  reserved (zero)
 //!     [4088..4096) u64 xxhash3 over [0..4088)
 //!   pages 3..     data, handed out by a tail-bump extent allocator
@@ -107,6 +108,10 @@ const header_hashed_len: usize = 24;
 /// cursors up to this rather than to page_size.
 pub const map_align = std.heap.page_size_min;
 
+/// Logical pages per system page. At least one: the comptime block below only
+/// requires one of the two sizes to divide the other.
+pub const pages_per_map_align: u32 = @max(1, map_align / page_size);
+
 /// Growth granularity. Large enough that growth is rare and each `setLength`
 /// covers many allocations, and a multiple of every supported system page size.
 const grow_chunk_pages: u32 = 2048; // 8 MiB
@@ -134,6 +139,22 @@ pub const Extent = struct {
 /// What a checkpoint publishes. Everything here is authoritative except the
 /// two cached counters, which are hints the engine recomputes if they look
 /// wrong.
+/// Pages a run of `len` bytes occupies, rounded up. The one place the
+/// partial-page rule is written.
+pub fn pages_for(len: u64) u32 {
+    return @intCast((len + page_size - 1) / page_size);
+}
+
+/// One generation's two streams, as page runs. Page counts rather than byte
+/// lengths because the free list speaks pages and because the free-list stream
+/// is allocated from an upper bound that its final length can fall short of.
+pub const Streams = struct {
+    catalog_page: u32 = 0,
+    catalog_pages: u32 = 0,
+    freelist_page: u32 = 0,
+    freelist_pages: u32 = 0,
+};
+
 pub const Watermark = struct {
     generation: u64 = 0,
     /// The log sequence this image covers. The crash-recovery invariant is that
@@ -226,6 +247,21 @@ pub const Pager = struct {
     /// the log append -- that is what per-consumer reservations buy.
     alloc_lock: std.Io.Mutex,
 
+    /// Separates a publish from the appends that are mid-flight.
+    ///
+    /// An appender asks `is_unpublished_at` whether its cursor is still
+    /// writable, and copies bytes there afterwards. `publish` clears the whole
+    /// unpublished set and mprotects the image between those two steps, so the
+    /// answer was stale by the time it was used and the copy landed in the
+    /// published image: SIGBUS where the protection is compiled in, a silent
+    /// store into the durable image in ReleaseFast, where it is not.
+    ///
+    /// Shared by appenders so writers on different collections still run
+    /// concurrently -- the decomposition ROADMAP item 5 measured is not given
+    /// back. Exclusive only for the tail of a publish, which happens once per
+    /// checkpoint.
+    append_lock: std.Io.RwLock,
+
     /// True when this file was created by this open (no checkpoint to load).
     fresh: bool,
 
@@ -235,6 +271,22 @@ pub const Pager = struct {
     /// The generation the next checkpoint will publish.
     generation: u64,
 
+    /// Where the catalog and free-list streams of the two most recent
+    /// generations live, so the one that falls out of reference can be freed.
+    ///
+    /// Both streams are written into *fresh* pages at every publish, so that a
+    /// crash leaves the previous copy intact. Nothing ever gave those pages
+    /// back: a database checkpointing every 32 MiB of log leaked two runs per
+    /// checkpoint forever, and the catalog carries a `u32` per index node page,
+    /// so at the tens-of-GB target that is hundreds of KB each time.
+    ///
+    /// Process-local rather than in the watermark: only a running pager needs to
+    /// know, because an open reads the slot it is loading and the other slot is
+    /// the fallback. Two generations because a publish overwrites the slot of
+    /// the generation *two* back -- that is the one nothing can reach again.
+    streams_cur: Streams,
+    streams_prev: Streams,
+
     /// Pages below this belong to the last published image and must never be
     /// stored into (PLAN amendment A1). Zero until a checkpoint publishes one,
     /// which is why copy-on-write is inert before then.
@@ -274,6 +326,13 @@ pub const Pager = struct {
 
     /// See `track_dirty`. Pages written since the last sync.
     dirty: if (track_dirty) std.AutoHashMapUnmanaged(u32, void) else void,
+    /// Guards `dirty`, and only `dirty`. Writers to the file itself are kept
+    /// apart by the locks their *owners* hold -- a collection's, an index's --
+    /// and the pages two of them touch never overlap. This set is the one thing
+    /// they share: two appenders on different collections record into the same
+    /// hash map at the same time, which is a torn map rather than a torn page.
+    /// Test-only instrumentation, so it costs the server nothing.
+    dirty_lock: if (track_dirty) std.Io.Mutex else void,
     /// Whether the last `protect_image` actually took effect. Checked by a test:
     /// an mprotect that silently fails would leave the belt looking present and
     /// doing nothing, which is worse than not having it.
@@ -327,15 +386,19 @@ pub const Pager = struct {
             .alloc_tail = page_first_data,
             .reserved_pages = 0,
             .alloc_lock = .init,
+            .append_lock = .init,
             .fresh = created,
             .loaded = .{},
             .generation = 0,
+            .streams_cur = .{},
+            .streams_prev = .{},
             .stable_pages = 0,
             .unpublished = .{},
             .free_ready = .empty,
             .free_hold = .empty,
             .free_pending = .empty,
             .dirty = if (track_dirty) .empty else {},
+            .dirty_lock = if (track_dirty) .init else {},
             .protect_ok = false,
         };
         // The bit set is the one heap allocation `self` owns before `deinit` can
@@ -417,7 +480,7 @@ pub const Pager = struct {
     /// recycling hands back.
     pub inline fn page_mut(self: *Pager, p: u32) *align(page_size) [page_size]u8 {
         assert_msg(p < self.mapped_pages, "write to a page past the mapped end of the data file");
-        if (track_dirty) self.dirty.put(self.gpa, p, {}) catch {};
+        self.note_dirty(p, p);
         return @ptrCast(@alignCast(self.reserve.ptr + (@as(usize, p) << page_shift)));
     }
 
@@ -496,6 +559,18 @@ pub const Pager = struct {
         return self.is_unpublished(@intCast(off >> page_shift));
     }
 
+    /// Hold off the next publish while an append decides where to put its bytes
+    /// and puts them there. Uncancelable and infallible: the append runs after
+    /// the log record is durable, where there is nowhere to report a failure.
+    /// See `append_lock`.
+    pub fn lock_append(self: *Pager) void {
+        self.append_lock.lockSharedUncancelable(self.io);
+    }
+
+    pub fn unlock_append(self: *Pager) void {
+        self.append_lock.unlockShared(self.io);
+    }
+
     /// The one page a write may legitimately land on below the stable mark: a
     /// watermark slot. Overwriting the *inactive* slot is the whole mechanism --
     /// alternating by generation parity is what makes it safe, where every other
@@ -507,7 +582,7 @@ pub const Pager = struct {
     inline fn page_mut_slot(self: *Pager, p: u32) *align(page_size) [page_size]u8 {
         assert(p == page_watermark_a or p == page_watermark_b);
         assert_msg(p < self.mapped_pages, "write to a watermark slot past the mapped end");
-        if (track_dirty) self.dirty.put(self.gpa, p, {}) catch {};
+        self.note_dirty(p, p);
         return @ptrCast(@alignCast(self.reserve.ptr + (@as(usize, p) << page_shift)));
     }
 
@@ -527,13 +602,21 @@ pub const Pager = struct {
     pub inline fn bytes_mut(self: *Pager, off: u64, len: usize) []u8 {
         assert_msg(off + len <= @as(u64, self.mapped_pages) << page_shift, "write past the mapped end of the data file");
         if (track_dirty) {
-            var pg_i: u32 = @intCast(off >> page_shift);
+            self.note_dirty(@intCast(off >> page_shift), @intCast((off + len - 1) >> page_shift));
-            const last: u32 = @intCast((off + len - 1) >> page_shift);
-            while (pg_i <= last) : (pg_i += 1) self.dirty.put(self.gpa, pg_i, {}) catch {};
         }
         return self.reserve[@intCast(off)..][0..len];
     }
 
+    /// Record pages `first..=last` as written since the last sync. See
+    /// `dirty_lock` for why this is the one shared structure on the write path.
+    inline fn note_dirty(self: *Pager, first: u32, last: u32) void {
+        if (!track_dirty) return;
+        self.dirty_lock.lockUncancelable(self.io);
+        defer self.dirty_lock.unlock(self.io);
+        var p = first;
+        while (p <= last) : (p += 1) self.dirty.put(self.gpa, p, {}) catch {};
+    }
+
     // -- allocation ---------------------------------------------------------
 
     /// Hand out `n` contiguous pages, growing the file if needed. For callers
@@ -557,6 +640,13 @@ pub const Pager = struct {
     pub fn reserve_pages(self: *Pager, hold: *Reservation, n: u32) !void {
         self.alloc_lock.lockUncancelable(self.io);
         defer self.alloc_lock.unlock(self.io);
+        return self.reserve_pages_locked(hold, n);
+    }
+
+    /// For callers already holding `alloc_lock`. The lock is not reentrant, so
+    /// the split is what lets `write_freelist` hold it across reading the lists
+    /// *and* allocating the pages it writes them into.
+    fn reserve_pages_locked(self: *Pager, hold: *Reservation, n: u32) !void {
         // Additive: room for every promise outstanding anywhere *plus* this one.
         // Two consumers reserving before the same log append must both be able to
         // rely on their promise.
@@ -615,10 +705,88 @@ pub const Pager = struct {
         } else {
             self.free_ready.items[i] = .{ .first = e.first + n, .pages = e.pages - n };
         }
-        self.unprotect(first, n);
         return first;
     }
 
+    /// Take a run off the free list for a document slab: at least `min_pages`,
+    /// at most `max_pages`, starting and ending on a system-page boundary.
+    /// Returns null when nothing on the list qualifies, and the caller bumps the
+    /// tail instead.
+    ///
+    /// `take_free` cannot serve this, and that is the whole reason this exists.
+    /// It is deliberately best fit -- smallest sufficient run -- so that the
+    /// thousands of single-page copy-on-write requests per generation cannot
+    /// dismantle the large runs. A slab extent asks for 2048 pages, and window
+    /// reclamation gives back runs a few pages at a time, so with an exact-size
+    /// rule the free list could fill up with reclaimed slab that no slab request
+    /// would ever take: the pages come back, the file keeps growing, the ratio
+    /// does not move. That is the failure this whole milestone is measured
+    /// against.
+    ///
+    /// So this one takes a partial run when it cannot get a whole one, and is
+    /// allowed to trim a larger one. There is no cannibalisation to fear here:
+    /// the request is itself large (the caller's floor is 1 MiB), so what it
+    /// leaves behind is still a usable run rather than a hole. The one-page
+    /// requests still go through `take_free` unchanged, and its pinned mutation
+    /// test is untouched.
+    ///
+    /// The alignment is not cosmetic. `map_align` is the granularity writeback
+    /// works at and the granularity reclamation gives back at, so a run that
+    /// starts mid-system-page both wastes its first window and shares a kernel
+    /// page with whatever occupies the rest of it -- which for a page still in
+    /// the published image is the tearing `mark_appendable` refuses to risk.
+    pub fn alloc_slab_run(self: *Pager, hold: *Reservation, min_pages: u32, max_pages: u32) ?Extent {
+        assert(min_pages > 0);
+        assert(min_pages <= max_pages);
+        self.alloc_lock.lockUncancelable(self.io);
+        defer self.alloc_lock.unlock(self.io);
+
+        // A split can leave a piece at each end, so one entry may become two.
+        // Out of memory before anything is disturbed: the caller falls back to
+        // bumping the tail, which is what it would have done anyway.
+        self.free_ready.ensureUnusedCapacity(self.gpa, 1) catch return null;
+
+        var best: ?usize = null;
+        var best_first: u32 = 0;
+        var best_take: u32 = 0;
+        var best_src: u32 = 0;
+        for (self.free_ready.items, 0..) |e, i| {
+            const from = std.mem.alignForward(u32, e.first, pages_per_map_align);
+            const to = std.mem.alignBackward(u32, e.first + e.pages, pages_per_map_align);
+            if (to <= from) continue;
+            const usable = to - from;
+            if (usable < min_pages) continue;
+            const take = @min(usable, max_pages);
+            // The longest run available, so the collection switches extents as
+            // rarely as possible -- every switch abandons what is left of the
+            // one before it. Ties go to the smallest source run, which leaves
+            // the big ones as whole as it can. `best_take` starts at zero and
+            // every candidate takes at least `min_pages`, so "nothing yet" is
+            // already encoded.
+            if (take > best_take or (take == best_take and e.pages < best_src)) {
+                best = i;
+                best_first = from;
+                best_take = take;
+                best_src = e.pages;
+            }
+        }
+        const i = best orelse return null;
+        const e = self.free_ready.items[i];
+        const head = best_first - e.first;
+        const tail_first = best_first + best_take;
+        const tail = (e.first + e.pages) - tail_first;
+        if (head > 0) {
+            self.free_ready.items[i] = .{ .first = e.first, .pages = head };
+            if (tail > 0) self.free_ready.appendAssumeCapacity(.{ .first = tail_first, .pages = tail });
+        } else if (tail > 0) {
+            self.free_ready.items[i] = .{ .first = tail_first, .pages = tail };
+        } else {
+            _ = self.free_ready.swapRemove(i);
+        }
+        self.claim_locked(hold, best_first, best_take);
+        return .{ .first = best_first, .pages = best_take };
+    }
+
     /// Merge runs that touch, so the holes single-page frees leave behind can add
     /// up to an extent again. Without it the free list only ever fragments: every
     /// generation returns thousands of one-page copy-on-write victims, and an
@@ -646,36 +814,46 @@ pub const Pager = struct {
     }
 
     pub fn alloc_pages_assume_reserved(self: *Pager, hold: *Reservation, n: u32) u32 {
-        assert(n > 0);
         self.alloc_lock.lockUncancelable(self.io);
         defer self.alloc_lock.unlock(self.io);
-        assert_msg(
+        return self.alloc_assume_reserved_locked(hold, n);
-            n <= hold.pages,
+    }
-            "page allocation overran reserve_pages' promise",
+
-        );
+    /// For callers already holding `alloc_lock`; see `reserve_pages_locked`.
-        assert_msg(
+    fn alloc_assume_reserved_locked(self: *Pager, hold: *Reservation, n: u32) u32 {
-            n <= self.reserved_pages,
+        assert(n > 0);
-            "page allocation overran the pager's total promise",
-        );
         assert_msg(
             self.alloc_tail + n <= self.mapped_pages,
             "page allocation past the mapped end of the data file",
         );
-        self.reserved_pages -= n;
-        hold.pages -= n;
         // Reuse before growing. Without this the free list is decorative and the
         // file grows without bound under churn, because copy-on-write abandons
         // every page it touches in every generation (PLAN amendment A2).
         if (self.take_free(n)) |recycled| {
-            self.mark_unpublished(recycled, n);
+            self.claim_locked(hold, recycled, n);
             return recycled;
         }
         const first = self.alloc_tail;
         self.alloc_tail += n;
-        self.mark_unpublished(first, n);
+        self.claim_locked(hold, first, n);
         return first;
     }
 
+    /// Charge `[first, first+pages)` against the reservation and make it
+    /// writable. The one place a claim is booked, because there are two
+    /// allocation policies above it and hand-copying this is how they drift --
+    /// the copy in `alloc_slab_run` had already lost one of the preconditions.
+    fn claim_locked(self: *Pager, hold: *Reservation, first: u32, pages: u32) void {
+        assert_msg(pages <= hold.pages, "page allocation overran reserve_pages' promise");
+        assert_msg(pages <= self.reserved_pages, "page allocation overran the pager's total promise");
+        self.reserved_pages -= pages;
+        hold.pages -= pages;
+        // A recycled page was inside a published image once, so its protection
+        // has to be lifted before it is handed out again.
+        self.unprotect(first, pages);
+        self.mark_unpublished(first, pages);
+    }
+
     fn mark_unpublished(self: *Pager, first: u32, n: u32) void {
         // `grow_to` sizes the set to the mapping, and `reserve_pages` has already
         // grown the mapping past this run, so the range is in bounds.
@@ -704,7 +882,11 @@ pub const Pager = struct {
             try std.posix.msync(self.reserve[0..len], std.posix.MSF.SYNC);
         }
         try self.file.sync(self.io);
-        if (track_dirty) self.dirty.clearRetainingCapacity();
+        if (track_dirty) {
+            self.dirty_lock.lockUncancelable(self.io);
+            defer self.dirty_lock.unlock(self.io);
+            self.dirty.clearRetainingCapacity();
+        }
     }
 
     /// Make the published image read-only at the hardware level. See
@@ -771,9 +953,21 @@ pub const Pager = struct {
 
         // Round up to a growth chunk, and to the system page size, so a
         // 16 KiB-page host never gets a partial mapping request.
+        //
+        // `alignForwardAnyAlign`, not `alignForward`: the chunk is a *proportion*
+        // of the current size once the file passes 64 MiB, and `mapped_pages / 8`
+        // is not a power of two. `alignForward` asserts that it is -- so this
+        // panicked in safe builds and, worse, in ReleaseFast (where the assert is
+        // compiled out) computed `(addr + align - 1) & ~(align - 1)` with a
+        // non-power-of-two mask, which can round *down*. A mapping longer than the
+        // file is the one thing this function exists to prevent: a store into a
+        // mapped page past end-of-file raises SIGBUS, which no error path catches.
+        //
+        // Never noticed because no unit test grew a pager past 64 MiB, which is
+        // where the chunk stops being `grow_chunk_pages`.
         const chunk = @max(grow_chunk_pages, self.mapped_pages / 8);
-        var new_pages = std.mem.alignForward(u32, want_pages, chunk);
+        var new_pages = std.mem.alignForwardAnyAlign(u32, want_pages, chunk);
-        const sys_pages: u32 = @intCast(map_align / page_size);
+        const sys_pages: u32 = pages_per_map_align;
         if (sys_pages > 1) new_pages = std.mem.alignForward(u32, new_pages, sys_pages);
 
         if (@as(u64, new_pages) << page_shift > self.reserve.len) {
@@ -835,6 +1029,18 @@ pub const Pager = struct {
             // Everything the published image references is off limits to
             // writes from here on.
             self.stable_pages = wm.alloc_tail;
+            // So the next publish but one gives this generation's streams back.
+            // Page counts are derived from the lengths here rather than
+            // remembered, which can be one page short for a free-list stream
+            // whose final length fell inside the page its bound reserved. That
+            // loses at most one page, once per open, against the unbounded leak
+            // this replaces.
+            self.streams_cur = .{
+                .catalog_page = wm.catalog_page,
+                .catalog_pages = pages_for(wm.catalog_len),
+                .freelist_page = wm.freelist_page,
+                .freelist_pages = pages_for(wm.freelist_len),
+            };
             try self.read_freelist(wm);
         } else if (self.watermark_attempted()) {
             // Only worth saying when a watermark was *written* and cannot be
@@ -949,8 +1155,34 @@ pub const Pager = struct {
         // made durable -- so a test can discard everything else and assert the
         // image still loads. Clearing it here would make that check vacuous.
 
-        // 5. only now is the new image current: advance the stable mark and
+        // 5. the streams of the generation two back are now unreachable: the
+        //    slot that named them is the one this publish just overwrote, and
+        //    the two slots hold the new generation and its predecessor. Give
+        //    their pages back -- via the free list, so they are still withheld
+        //    for two more generations like anything else.
+        //
+        //    Before the lock below, which `free_pages` takes for itself.
+        try self.free_pages(self.streams_prev.catalog_page, self.streams_prev.catalog_pages);
+        try self.free_pages(self.streams_prev.freelist_page, self.streams_prev.freelist_pages);
+        self.streams_prev = self.streams_cur;
+        self.streams_cur = .{
+            .catalog_page = wm.catalog_page,
+            .catalog_pages = pages_for(wm.catalog_len),
+            .freelist_page = fl.first,
+            .freelist_pages = fl.pages,
+        };
+
+        // 6. only now is the new image current: advance the stable mark and
         //    rotate the free lists by one generation.
+        //
+        //    Exclusive against the appenders for this step alone. Freezing the
+        //    image while one of them is between "is my cursor still writable"
+        //    and the copy that relies on the answer is what put documents into
+        //    the durable image; holding them off here is what makes the answer
+        //    still true when it is used. Taken before `alloc_lock`, the order
+        //    `mark_appendable` uses on the appender's side.
+        self.append_lock.lockUncancelable(self.io);
+        defer self.append_lock.unlock(self.io);
         self.generation = wm.generation;
         self.loaded = wm;
         // The free lists and the unpublished set are allocator state, so the
@@ -973,11 +1205,44 @@ pub const Pager = struct {
 
     /// Give back a run of pages. They become reusable two generations later --
     /// see the field comment on `free_ready`.
+    ///
+    /// Under the allocation lock, like every other mutation of the free lists.
+    /// It was not, and the hot caller is `page_mut_cow`, which runs under a
+    /// *collection* lock: two collections doing copy-on-write concurrently
+    /// appended to the same list, and `publish` rotated all three lists
+    /// underneath them. No caller holds the lock already -- `page_mut_cow` takes
+    /// it inside `alloc_pages` and has released it by here -- so this cannot
+    /// recurse.
     pub fn free_pages(self: *Pager, first: u32, pages: u32) !void {
         if (pages == 0) return;
+        self.alloc_lock.lockUncancelable(self.io);
+        defer self.alloc_lock.unlock(self.io);
         try self.free_pending.append(self.gpa, .{ .first = first, .pages = pages });
     }
 
+    /// Whether any page of `[first, first+pages)` has been handed to the free
+    /// list. A consumer that still claims one is claiming a page the pager is
+    /// about to give to somebody else, and the symptom is a document quietly
+    /// overwritten rather than anything failing -- so this is the detector the
+    /// enlarged free list deserves, and it is why the free lists are readable
+    /// from outside at all.
+    ///
+    /// Walks three lists, so it is for assertions in test and Debug builds.
+    pub fn owns_freed(self: *Pager, first: u32, pages: u32) bool {
+        self.alloc_lock.lockUncancelable(self.io);
+        defer self.alloc_lock.unlock(self.io);
+        for ([_][]const Extent{
+            self.free_pending.items,
+            self.free_hold.items,
+            self.free_ready.items,
+        }) |list| {
+            for (list) |e| {
+                if (first < e.first + e.pages and e.first < first + pages) return true;
+            }
+        }
+        return false;
+    }
+
     /// Pages available for immediate reuse.
     pub fn free_ready_pages(self: *const Pager) u32 {
         var n: u32 = 0;
@@ -985,14 +1250,37 @@ pub const Pager = struct {
         return n;
     }
 
-    fn write_freelist(self: *Pager) !struct { first: u32, len: u64 } {
+    fn write_freelist(self: *Pager) !struct { first: u32, len: u64, pages: u32 } {
-        const count = self.free_ready.items.len + self.free_hold.items.len + self.free_pending.items.len;
+        // Size from an upper bound, then count the entries actually written.
-        const len: u64 = 8 + @as(u64, count) * 8 + 8;
+        //
-        const pages: u32 = @intCast((len + page_size - 1) / page_size);
+        // The allocation below takes its pages off this very list, and an exact
-        const first = try self.alloc_pages(pages);
+        // fit removes the entry it took (`take_free`). A count captured
-        const buf = self.bytes_mut(@as(u64, first) << page_shift, @intCast(len));
+        // beforehand therefore claims one entry more than the loop writes: the
+        // hash lands eight bytes short of where `read_freelist` looks for it and
+        // the whole list is dropped as corrupt on the next open. The stream is
+        // one page and a one-page run is the commonest thing on the list, so
+        // that is the ordinary case rather than a corner.
+        //
+        // `take_free` never *adds* an entry, so the bound holds and one
+        // allocation is enough.
+        //
+        // Under `alloc_lock` for the whole of it, allocation included. Reading
+        // the three lists is as much a use of them as appending is: a concurrent
+        // `free_pages` -- copy-on-write in some collection, which a checkpoint
+        // does not exclude -- grows `free_pending` while the loop below walks it,
+        // and a growth that reallocates leaves the loop on freed memory. A free
+        // that lands after this point simply waits for the next generation's
+        // list; the page stays allocated one generation longer, which is the
+        // safe direction.
+        self.alloc_lock.lockUncancelable(self.io);
+        defer self.alloc_lock.unlock(self.io);
+        const bound = self.free_ready.items.len + self.free_hold.items.len + self.free_pending.items.len;
+        const pages: u32 = pages_for(8 + bound * 8 + 8);
+        var hold: Reservation = .{};
+        try self.reserve_pages_locked(&hold, pages);
+        const first = self.alloc_assume_reserved_locked(&hold, pages);
+        const buf = self.bytes_mut(@as(u64, first) << page_shift, @as(usize, pages) << page_shift);
         @memset(buf, 0);
-        std.mem.writeInt(u64, buf[0..8], count, .little);
         var at: usize = 8;
         for ([_][]const Extent{
             self.free_ready.items,
@@ -1005,8 +1293,17 @@ pub const Pager = struct {
                 at += 8;
             }
         }
+        const count = (at - 8) / 8;
+        assert_msg(
+            count == bound or count + 1 == bound,
+            "the free list changed size while it was being written",
+        );
+        std.mem.writeInt(u64, buf[0..8], count, .little);
         std.mem.writeInt(u64, buf[at..][0..8], header_hash(buf[0..at]), .little);
-        return .{ .first = first, .len = len };
+        // `pages` and not `pages_for(len)`: the allocation was sized from the
+        // bound, and the whole run has to go back when this generation falls
+        // out of reference.
+        return .{ .first = first, .len = @as(u64, at) + 8, .pages = pages };
     }
 
     /// Load the free list a watermark points at. A damaged one is dropped with a
@@ -1524,6 +1821,55 @@ test "freed pages are withheld for two generations and survive a reopen" {
     try testing.expectEqual(@as(u32, 2), again.free_ready_pages());
 }
 
+test "the persisted free list survives allocating its own pages" {
+    // `write_freelist` allocates the pages it is about to write into, and that
+    // allocation goes through `take_free` like any other. On an exact fit the
+    // entry is removed, so a count captured beforehand describes one entry more
+    // than the loop writes, the hash lands short of where the reader looks, and
+    // the whole list is dropped as corrupt on the next open.
+    //
+    // The stream is one page and a one-page run is the commonest thing on the
+    // list, so this is the normal case, not a corner. The two-generation test
+    // above misses it because its free run is two pages and the stream asks for
+    // one: shrinking an entry keeps the count right, only removing it does not.
+    //
+    // Mutation check: compute `count` before `alloc_pages` again and the reopen
+    // assertion goes red with "data file free list is corrupt".
+    var threaded: std.Io.Threaded = .init_single_threaded;
+    defer threaded.deinit();
+    const io = threaded.io();
+    var tp = try TmpPager.init(io, 64 << 20);
+    defer tp.deinit();
+
+    // Five pages, of which three go back one at a time with a gap between each
+    // -- adjacent runs would be coalesced back into one and the list would be
+    // too short for a lost entry to show.
+    const base = try tp.pg().alloc_pages(5);
+    try tp.pg().publish(.{ .seq = 1 });
+    try tp.pg().free_pages(base, 1);
+    try tp.pg().free_pages(base + 2, 1);
+    try tp.pg().free_pages(base + 4, 1);
+
+    try tp.pg().publish(.{ .seq = 2 }); // pending -> hold
+    try tp.pg().publish(.{ .seq = 3 }); // hold -> ready
+    try testing.expectEqual(@as(u32, 3), tp.pg().free_ready_pages());
+
+    // This is the publish that trips it: the free list is now non-empty and
+    // holds a run of exactly the one page the stream needs.
+    try tp.pg().publish(.{ .seq = 4 });
+    // Not a fixed number: a publish also recycles the streams of the generation
+    // two back, so what is on the list is the three frees above minus whatever
+    // the stream allocations took plus whatever they gave back. The invariant
+    // under test is that a reopen agrees with it, whatever it is.
+    const ready_before = tp.pg().free_ready_pages();
+    try testing.expect(ready_before > 0);
+    tp.close();
+
+    var again = try reopen(io, tp.path);
+    defer again.deinit();
+    try testing.expectEqual(ready_before, again.free_ready_pages());
+}
+
 test "the watermark is never published before the pages it describes" {
     // The load-bearing ordering of the whole design: every page a watermark
     // describes is durable before the watermark that describes it. Reverse them
@@ -1736,6 +2082,168 @@ test "one-page requests do not carve up the runs the extents need" {
     try testing.expectEqual(tail_before, pg.alloc_tail);
 }
 
+test "a slab run comes off the free list aligned, or not at all" {
+    // `alloc_slab_run` is a second allocation policy in the same allocator, so
+    // what it must not do is as important as what it must:
+    //
+    //  - what it hands out starts and ends on a system-page boundary, because
+    //    that is the granularity writeback tears at and the granularity
+    //    reclamation gives back at;
+    //  - a run too short to be worth an extent is left alone;
+    //  - the pieces it trims off stay on the free list rather than leaking.
+    //
+    // Mutation checks: drop the `alignForward`/`alignBackward` and the first
+    // assertion goes red on the odd-page run; drop the `usable < min_pages`
+    // test and the short run is handed out.
+    var threaded: std.Io.Threaded = .init_single_threaded;
+    defer threaded.deinit();
+    const io = threaded.io();
+    var tp = try TmpPager.init(io, 64 << 20);
+    defer tp.deinit();
+    const pg = tp.pg();
+    const spp: u32 = pages_per_map_align;
+
+    // A long run deliberately starting one 4 KiB page past a boundary, and a
+    // short one, kept apart so coalescing cannot merge them.
+    _ = try pg.alloc_pages(std.mem.alignForward(u32, pg.alloc_tail, spp) - pg.alloc_tail + 1);
+    const long = try pg.alloc_pages(400);
+    _ = try pg.alloc_pages(1); // separator, never freed
+    const short = try pg.alloc_pages(8);
+    _ = try pg.alloc_pages(1); // separator, never freed
+    try testing.expect(long % spp != 0);
+    try pg.publish(.{ .seq = 1 });
+    try pg.free_pages(long, 400);
+    try pg.free_pages(short, 8);
+    try pg.publish(.{ .seq = 2 });
+    try pg.publish(.{ .seq = 3 });
+    const ready_before = pg.free_ready_pages();
+
+    var hold: Reservation = .{};
+    try pg.reserve_pages(&hold, 256);
+    const run = pg.alloc_slab_run(&hold, 64, 256) orelse return error.TestUnexpectedResult;
+    pg.release_reservation(&hold);
+    try testing.expectEqual(@as(u32, 0), run.first % spp);
+    try testing.expectEqual(@as(u32, 0), run.pages % spp);
+    try testing.expectEqual(@as(u32, 256), run.pages);
+    // It came out of the long run, not the short one, and past its unaligned
+    // first page.
+    try testing.expect(run.first > long);
+    try testing.expect(run.first < long + 400);
+    // Everything not handed out is still on the list.
+    try testing.expectEqual(ready_before - 256, pg.free_ready_pages());
+
+    // The short run is below the floor and stays where it is, whatever is asked
+    // of it; nothing else is left long enough either.
+    var hold2: Reservation = .{};
+    try pg.reserve_pages(&hold2, 256);
+    try testing.expect(pg.alloc_slab_run(&hold2, 200, 256) == null);
+    pg.release_reservation(&hold2);
+    try testing.expectEqual(ready_before - 256, pg.free_ready_pages());
+}
+
+test "a quiet checkpoint stops growing the file" {
+    // Both streams a publish writes are allocated fresh every time, so that a
+    // crash leaves the previous copy readable. Nothing gave them back, and a
+    // database checkpoints on log volume rather than on having anything to say
+    // -- so an idle server grew the data file forever.
+    //
+    // Steady state is not zero growth: a publish allocates this generation's
+    // streams and frees the ones from two generations back, and those take two
+    // more publishes to become reusable. So a few pages are always in flight,
+    // and the assertion is that the number does not track the publish count.
+    //
+    // Mutation check: drop the two `free_pages` calls from `publish` and this
+    // goes red at 40-something pages instead of a handful.
+    var threaded: std.Io.Threaded = .init_single_threaded;
+    defer threaded.deinit();
+    const io = threaded.io();
+    var tp = try TmpPager.init(io, 64 << 20);
+    defer tp.deinit();
+
+    // Let the rotation reach its steady state before measuring.
+    for (1..5) |i| try tp.pg().publish(.{ .seq = i });
+    const settled = tp.pg().alloc_tail;
+
+    for (5..45) |i| try tp.pg().publish(.{ .seq = i });
+    try testing.expect(tp.pg().alloc_tail - settled <= 8);
+}
+
+test "concurrent frees lose no pages while a publish rotates the lists" {
+    // `free_pages` mutates the same three lists `publish` rotates, and its hot
+    // caller is `page_mut_cow` under a *collection* lock -- so two collections
+    // copying nodes concurrently were appending to one `ArrayList` unserialized
+    // while a checkpoint moved it out from under them.
+    //
+    // Pages are conserved across the rotation and across coalescing, so the sum
+    // over all three lists is the invariant to assert. Probabilistic by nature,
+    // as any test of a data race is: it says nothing when green and is only
+    // evidence when red. Mutation check: drop the lock from `free_pages` and
+    // this fails or crashes within a few runs.
+    const gpa = testing.allocator;
+    var threaded: std.Io.Threaded = .init(gpa, .{});
+    defer threaded.deinit();
+    const io = threaded.io();
+    var tp = try TmpPager.init(io, 64 << 20);
+    defer tp.deinit();
+
+    const freers = 4;
+    const per_freer = 200;
+    // One page each, allocated up front so no fiber is also growing the file.
+    var pages: [freers * per_freer]u32 = undefined;
+    for (&pages) |*p| p.* = try tp.pg().alloc_pages(1);
+    try tp.pg().publish(.{ .seq = 1 });
+
+    const Worker = struct {
+        fn freer(p: *Pager, run: []const u32) error{Canceled}!void {
+            for (run) |page| p.free_pages(page, 1) catch return error.Canceled;
+        }
+        fn publisher(p: *Pager, seq: *std.atomic.Value(u64)) error{Canceled}!void {
+            for (0..8) |_| {
+                p.publish(.{ .seq = seq.fetchAdd(1, .monotonic) }) catch return error.Canceled;
+            }
+        }
+    };
+
+    var seq = std.atomic.Value(u64).init(2);
+    var group: std.Io.Group = .init;
+    defer group.cancel(io);
+    for (0..freers) |i| {
+        group.async(io, Worker.freer, .{ tp.pg(), pages[i * per_freer ..][0..per_freer] });
+    }
+    group.async(io, Worker.publisher, .{ tp.pg(), &seq });
+    try group.await(io);
+
+    // Page identity, not a total: the publisher's own free-list streams are
+    // allocated *off this list*, so a plain count would be short by however many
+    // publishes found a fit. Every page still on the list must therefore be one
+    // the freers put there, exactly once -- a lost or half-written append shows
+    // up as a duplicate or as a page nobody freed, neither of which recycling
+    // can produce.
+    const lo = pages[0];
+    var seen = try std.DynamicBitSetUnmanaged.initEmpty(gpa, pages.len);
+    defer seen.deinit(gpa);
+    var on_list: usize = 0;
+    for ([_][]const Extent{
+        tp.pg().free_ready.items,
+        tp.pg().free_hold.items,
+        tp.pg().free_pending.items,
+    }) |list| for (list) |e| {
+        for (0..e.pages) |i| {
+            const p = e.first + @as(u32, @intCast(i));
+            // Pages outside the set the freers own are the publisher's own
+            // stream runs coming back two generations later; they are not what
+            // this test is about.
+            if (p < lo or p - lo >= pages.len) continue;
+            try testing.expect(!seen.isSet(p - lo));
+            seen.set(p - lo);
+            on_list += 1;
+        }
+    };
+    // The only pages missing are the ones a publish recycled into its stream,
+    // and there were nine publishes at one page each.
+    try testing.expect(pages.len - on_list <= 9);
+}
+
 test "freed pages that touch merge back into a usable run" {
     // Mutation check: drop the `coalesce_free_ready()` call from `publish`. Red
     // -- the four one-page frees below stay four separate holes and the run of

src/server.zig

@@ -20,6 +20,9 @@ pub const Server = struct {
     /// because that is what std.Io.Duration.fromSeconds takes — the CLI
     /// rejects negatives.
     ttl_sweep_secs: i64,
+    /// Seconds between idle-cursor sweeps; 0 leaves that monitor unspawned.
+    /// mongod's own `clientCursorMonitorFrequencySecs` default is 4.
+    cursor_sweep_secs: i64,
 
     pub fn run(self: *Server) !void {
         // Unbounded async limit: connection handlers otherwise fall back to
@@ -43,6 +46,12 @@ pub const Server = struct {
         // The TTL monitor is just another member of the connection group, so
         // the `group.cancel` above stops it with everything else.
         if (self.ttl_sweep_secs > 0) group.async(io, ttl_monitor, .{ io, self });
+        // A separate fiber rather than a branch inside ttl_monitor, for two
+        // reasons: the cadences differ by more than an order of magnitude (4 s
+        // against 60 s), and a TTL sweep that fails must not stop cursors being
+        // reclaimed. It is also spawned when TTL sweeping is disabled entirely,
+        // which is the configuration the spec runner uses.
+        if (self.cursor_sweep_secs > 0) group.async(io, cursor_monitor, .{ io, self });
 
         while (true) {
             const stream = listener.accept(io) catch |err| switch (err) {
@@ -87,6 +96,20 @@ fn ttl_monitor(io: std.Io, server: *Server) error{Canceled}!void {
     }
 }
 
+/// Reap cursors nobody has touched for `cursor_timeout_ms`, until the group is
+/// canceled. Takes no engine lock: a cursor owns its own arena, and the store's
+/// mutex is a leaf.
+fn cursor_monitor(io: std.Io, server: *Server) error{Canceled}!void {
+    const interval: std.Io.Duration = .fromSeconds(server.cursor_sweep_secs);
+    while (true) {
+        // Sleep first, for the same reason the TTL monitor does: at startup
+        // there is nothing to reap and the listener wants the CPU.
+        try std.Io.sleep(io, interval, .awake);
+        const now_ms = std.Io.Timestamp.now(io, .real).toMilliseconds();
+        _ = server.engine.cursors.sweep(io, now_ms);
+    }
+}
+
 /// Entry point required by `Group.async`: must return only `error.Canceled`.
 fn handle_connection(io: std.Io, stream: std.Io.net.Stream, server: *Server) error{Canceled}!void {
     handle_connection_inner(io, stream, server) catch {};

src/wire.zig

@@ -185,6 +185,81 @@ pub const Message = struct {
         };
     }
 
+    /// A logical session id: a UUID, so 16 bytes of binary subtype 4.
+    pub const SessionId = [16]u8;
+
+    /// Every way an `lsid` can be malformed, named after what is wrong rather
+    /// than after the error the caller will send: the codes belong to the
+    /// command layer, which is where they were measured.
+    pub const LsidError = error{
+        LsidNotDocument,
+        LsidUnknownField,
+        LsidIdMissing,
+        LsidIdNotBinary,
+        LsidIdNotUuid,
+        LsidIdWrongLength,
+        LsidInternalSession,
+        LsidTxnNumberWithoutTxnUuid,
+    };
+
+    /// The logical session id, or null when the command carries no `lsid`.
+    ///
+    /// An accessor over the command envelope, like `db_name`: called from
+    /// dispatch, never from `parse`, because a malformed session id is a
+    /// command that gets an error reply and not a connection that gets torn
+    /// down.
+    ///
+    /// The fields it tolerates were measured against mongod 8.3.7, not
+    /// recalled, and the measurement contradicted the assumption it was
+    /// written on. mongod does *not* ignore unknown fields inside `lsid` --
+    /// it answers IDLUnknownField -- and it does accept `uid`, the hash of
+    /// the credentials that own the session, which a driver sends as soon as
+    /// authentication is on. Both matter to us: the first because tolerating
+    /// what the server rejects is the kind of divergence that only shows up
+    /// under a driver nobody tested, the second because M7 would otherwise
+    /// break every command.
+    pub fn lsid(self: *const Message) LsidError!?SessionId {
+        const v = bson.get_pair(self.body.pairs, "lsid") orelse return null;
+        const doc = switch (v) {
+            .doc => |d| d,
+            else => return error.LsidNotDocument,
+        };
+
+        var id: ?bson.Binary = null;
+        var has_txn_number = false;
+        var has_txn_uuid = false;
+        for (doc) |pair| {
+            if (std.mem.eql(u8, pair.key, "id")) {
+                id = switch (pair.value) {
+                    .binary => |b| b,
+                    else => return error.LsidIdNotBinary,
+                };
+            } else if (std.mem.eql(u8, pair.key, "uid")) {
+                // Accepted and ignored: it identifies the user a session
+                // belongs to, and this server has exactly one.
+            } else if (std.mem.eql(u8, pair.key, "txnNumber")) {
+                has_txn_number = true;
+            } else if (std.mem.eql(u8, pair.key, "txnUUID")) {
+                has_txn_uuid = true;
+            } else {
+                return error.LsidUnknownField;
+            }
+        }
+
+        // A `txnNumber` inside the session id is not the retryable-write one
+        // outside it: together with `txnUUID` the two name an *internal*
+        // session, which only exists to run a transaction on another
+        // session's behalf. Neither can mean anything here, and mongod
+        // refuses them on a standalone too.
+        if (has_txn_number and !has_txn_uuid) return error.LsidTxnNumberWithoutTxnUuid;
+        if (has_txn_uuid) return error.LsidInternalSession;
+
+        const bin = id orelse return error.LsidIdMissing;
+        if (bin.subtype != 4) return error.LsidIdNotUuid;
+        if (bin.data.len != 16) return error.LsidIdWrongLength;
+        return bin.data[0..16].*;
+    }
+
     /// Documents of a batch argument (`documents`, `updates`, `deletes`).
     /// Drivers send them either as an OP_MSG document sequence or as an array
     /// inside the command body; callers should not have to care which. The
@@ -299,9 +374,20 @@ fn begin_message(
 }
 
 /// Patch in the total length of the message started at `len_pos`.
+///
+/// The bound is `max_message_size`, the same 48 MiB we advertise to drivers as
+/// `maxMessageSizeBytes`, not `maxInt(u32)`. A reply past what we told the
+/// client to expect is not a large reply, it is a desynchronized connection:
+/// the driver reads the length, refuses or mis-frames it, and every later
+/// command on that socket reads the wrong bytes. Failing here turns that into
+/// one honest error on the request that caused it.
+///
+/// Reachable today: nothing caps how many documents a `find` puts in its single
+/// batch, so ~3000 documents of 16 KiB clears 48 MB. The cursor batch budget
+/// makes it unreachable, which is the point of keeping this as the backstop.
 fn end_message(out: *std.ArrayListUnmanaged(u8), len_pos: usize) !void {
     const total = out.items.len - len_pos;
-    if (total > std.math.maxInt(u32)) return error.MessageTooLarge;
+    if (total > max_message_size) return error.MessageTooLarge;
     std.mem.writeInt(u32, out.items[len_pos..][0..4], @intCast(total), .little);
 }
 
@@ -367,6 +453,94 @@ test "reply serializes to a parseable message" {
     try testing.expectEqual(@as(usize, 0), msg.seqs.len);
 }
 
+/// An OP_MSG carrying one body section, for tests that care about the command
+/// envelope rather than about framing.
+fn fake_op_msg(gpa: std.mem.Allocator, pairs: []const bson.Pair) !Message {
+    var doc: std.ArrayListUnmanaged(u8) = .empty;
+    defer doc.deinit(gpa);
+    try bson.write_doc(pairs, gpa, &doc);
+
+    var buf: std.ArrayListUnmanaged(u8) = .empty;
+    defer buf.deinit(gpa);
+    try buf.appendSlice(gpa, &[_]u8{ 0, 0, 0, 0, 7, 0, 0, 0, 0, 0, 0, 0, 0xDD, 0x07, 0, 0, 0, 0, 0, 0 });
+    try buf.append(gpa, 0x00);
+    try buf.appendSlice(gpa, doc.items);
+    std.mem.writeInt(u32, buf.items[0..4], @intCast(buf.items.len), .little);
+    return Message.parse(gpa, buf.items);
+}
+
+test "a session id is read out of a command" {
+    const uuid = [_]u8{0xAB} ** 16;
+    var msg = try fake_op_msg(testing.allocator, &.{
+        .{ .key = "ping", .value = .{ .int32 = 1 } },
+        .{ .key = "lsid", .value = .{ .doc = &.{
+            .{ .key = "id", .value = .{ .binary = .{ .subtype = 4, .data = &uuid } } },
+        } } },
+    });
+    defer msg.deinit();
+    try testing.expectEqual(uuid, (try msg.lsid()).?);
+}
+
+test "a command with no lsid has no session" {
+    var msg = try fake_op_msg(testing.allocator, &.{.{ .key = "ping", .value = .{ .int32 = 1 } }});
+    defer msg.deinit();
+    try testing.expect((try msg.lsid()) == null);
+}
+
+test "a session id carrying a user hash is still a session id" {
+    // `uid` arrives as soon as authentication is on (M7). Rejecting it as an
+    // unknown field would break every command the moment that lands, which is
+    // exactly the kind of divergence a measurement against a real server is
+    // for -- mongod 8.3.7 answers ok:1 to this.
+    const uuid = [_]u8{0x11} ** 16;
+    var msg = try fake_op_msg(testing.allocator, &.{
+        .{ .key = "ping", .value = .{ .int32 = 1 } },
+        .{ .key = "lsid", .value = .{ .doc = &.{
+            .{ .key = "id", .value = .{ .binary = .{ .subtype = 4, .data = &uuid } } },
+            .{ .key = "uid", .value = .{ .binary = .{ .subtype = 0, .data = &[_]u8{0} ** 32 } } },
+        } } },
+    });
+    defer msg.deinit();
+    try testing.expectEqual(uuid, (try msg.lsid()).?);
+}
+
+test "every malformed session id is named" {
+    const uuid = [_]u8{0x22} ** 16;
+    const cases = [_]struct { want: anyerror, lsid: bson.Value }{
+        .{ .want = error.LsidNotDocument, .lsid = .{ .int32 = 5 } },
+        .{ .want = error.LsidIdMissing, .lsid = .{ .doc = &.{} } },
+        .{ .want = error.LsidIdNotBinary, .lsid = .{ .doc = &.{
+            .{ .key = "id", .value = .{ .string = "nope" } },
+        } } },
+        .{ .want = error.LsidIdNotUuid, .lsid = .{ .doc = &.{
+            .{ .key = "id", .value = .{ .binary = .{ .subtype = 0, .data = &uuid } } },
+        } } },
+        .{ .want = error.LsidIdWrongLength, .lsid = .{ .doc = &.{
+            .{ .key = "id", .value = .{ .binary = .{ .subtype = 4, .data = uuid[0..15] } } },
+        } } },
+        .{ .want = error.LsidUnknownField, .lsid = .{ .doc = &.{
+            .{ .key = "id", .value = .{ .binary = .{ .subtype = 4, .data = &uuid } } },
+            .{ .key = "bogus", .value = .{ .int32 = 1 } },
+        } } },
+        .{ .want = error.LsidTxnNumberWithoutTxnUuid, .lsid = .{ .doc = &.{
+            .{ .key = "id", .value = .{ .binary = .{ .subtype = 4, .data = &uuid } } },
+            .{ .key = "txnNumber", .value = .{ .int64 = 1 } },
+        } } },
+        .{ .want = error.LsidInternalSession, .lsid = .{ .doc = &.{
+            .{ .key = "id", .value = .{ .binary = .{ .subtype = 4, .data = &uuid } } },
+            .{ .key = "txnUUID", .value = .{ .binary = .{ .subtype = 4, .data = &uuid } } },
+        } } },
+    };
+    for (cases) |c| {
+        var msg = try fake_op_msg(testing.allocator, &.{
+            .{ .key = "ping", .value = .{ .int32 = 1 } },
+            .{ .key = "lsid", .value = c.lsid },
+        });
+        defer msg.deinit();
+        try testing.expectError(c.want, msg.lsid());
+    }
+}
+
 test "reject non-OP_MSG non-OP_QUERY opcodes" {
     var buf: [20]u8 = undefined;
     std.mem.writeInt(u32, buf[0..4], 20, .little);
@@ -411,3 +585,31 @@ test "parse OP_QUERY handshake" {
     try testing.expectEqual(@as(i32, op_code_query), msg.op_code);
     try testing.expectEqualStrings("isMaster", msg.command_name());
 }
+
+test "a reply past the advertised message size fails to build" {
+    // The guard exists because exceeding it desynchronizes the connection
+    // rather than merely making one reply large, so it must be an error return
+    // and not a truncation. One oversized string is the cheapest way past it
+    // without allocating 48 MB of documents.
+    const gpa = testing.allocator;
+    var reply = Reply.init(gpa);
+    defer reply.deinit();
+
+    const big = try reply.arena_alloc().alloc(u8, max_message_size + 1);
+    @memset(big, 'x');
+    try reply.put_ok();
+    try reply.put("payload", .{ .string = big });
+
+    var out: std.ArrayListUnmanaged(u8) = .empty;
+    defer out.deinit(gpa);
+    try testing.expectError(error.MessageTooLarge, reply.build(gpa, 1, 1, &out));
+
+    // And a reply comfortably inside the bound still builds, so the guard is
+    // not simply rejecting everything.
+    var small = Reply.init(gpa);
+    defer small.deinit();
+    try small.put_ok();
+    out.clearRetainingCapacity();
+    try small.build(gpa, 1, 1, &out);
+    try testing.expect(out.items.len < max_message_size);
+}

tests/e2e/README.md

@@ -41,6 +41,19 @@ node tests/e2e/e2e6.js              # 73 checks, ~15 s, needs no running server
 E2E6_PORT=27300 node tests/e2e/e2e6.js   # different port if 27220 is taken
 ```
 
+`e2e7.js` is the cursor suite, self-contained for a different reason: cursor
+behaviour is only observable with non-default flags. It spawns three servers in
+turn -- default flags for batching/streaming/aggregate, then
+`--cursor-timeout-ms 800 --cursor-sweep-secs 1 --max-open-cursors 4` for idle
+expiry and registry capacity, then a restart on the same database to confirm a
+cursor does not survive one. Most of it uses raw `runCommand`, because the
+driver hides `cursor.id` and that is the thing under test:
+
+```sh
+node tests/e2e/e2e7.js              # 86 checks, needs no running server
+E2E7_PORT=27310 node tests/e2e/e2e7.js   # different port if 27230 is taken
+```
+
 Rebuild with `zig build` after any change under `src/` before restarting the
 server: `zig build test` compiles the test binary only and leaves
 `zig-out/bin/multiforadb` stale, so the suites keep running against the old

tests/e2e/churn.js

@@ -0,0 +1,382 @@
+// The churn gate: how large the data file settles at, relative to the live
+// data, under sustained rewriting.
+//
+// This is the measurement PLAN D7.4 was decided on, and until now it was the
+// only gate in `tests/e2e/results/m0-gates.txt` without a `reproduce:` line --
+// the numbers were real but the harness was not committed, so nobody could
+// re-run them against a change. That is what this file fixes.
+//
+// It spawns its own server on a fresh database, so it needs nothing running:
+//
+//   node tests/e2e/churn.js --docs 40000 --doc-size 16k --index \
+//        --mode delete-refill --rounds 6
+//   node tests/e2e/churn.js --docs 40000 --doc-size 16k --index \
+//        --mode update --multiple 5
+//
+// What to read. The ratio alone does not say whether reclamation is working:
+// a file that grows and is periodically halved by a rebuild averages out to a
+// respectable number. So every round prints `reclaimed`, `allocTail` and
+// `compactions` beside it. Reclamation is carrying the workload when
+// `reclaimed` climbs while `allocTail` stays put. If instead `allocTail` grows
+// by the full write volume of each round, the free list is decorative however
+// good the ratio looks -- and that is a failure even at 1.2x.
+//
+// Small documents are expected not to improve on the delete-refill line, and
+// that is a pass rather than a fault. Reclamation gives back whole system
+// pages, so a 16 KiB page holds ~82 documents of 200 bytes and the chance all
+// 82 are dead at once is nil. The mechanism to check there is that the
+// counters decay correctly, not that the ratio moves.
+//
+// Options:
+//   --docs <n>          documents in the collection (default 40000)
+//   --doc-size <n[k|m]> payload bytes per document (default 16k)
+//   --index             create one secondary index over a churned field
+//   --mode <m>          delete-refill (default) | update
+//   --rounds <n>        rounds in each mode; update mode splits its writes
+//                       across them (default 6)
+//   --multiple <n>      update mode: total writes as a multiple of --docs
+//   --target <x>        fail unless the steady-state ratio is at or under x
+//   --port <n>          listen port (default 27320)
+//   --keep              leave the database file behind
+const { MongoClient } = require('mongodb');
+const { spawn } = require('child_process');
+const fs = require('fs');
+const path = require('path');
+
+function parseSize(s) {
+  const m = String(s).match(/^(\d+)([kKmMgG]?)$/);
+  if (!m) throw new Error(`bad size: ${s}`);
+  const mult = { '': 1, k: 1 << 10, m: 1 << 20, g: 1 << 30 }[m[2].toLowerCase()];
+  return Number(m[1]) * mult;
+}
+
+function parseArgs(argv) {
+  const o = {
+    docs: 40000, docSize: 16 << 10, index: false, mode: 'delete-refill',
+    rounds: 6, multiple: 5, target: null, port: 27320, keep: false,
+  };
+  for (let i = 0; i < argv.length; i++) {
+    const a = argv[i];
+    const next = () => {
+      if (i + 1 >= argv.length) throw new Error(`${a} needs a value`);
+      return argv[++i];
+    };
+    switch (a) {
+      case '--docs': o.docs = Number(next()); break;
+      case '--doc-size': o.docSize = parseSize(next()); break;
+      case '--index': o.index = true; break;
+      case '--mode': o.mode = next(); break;
+      case '--rounds': o.rounds = Number(next()); break;
+      case '--multiple': o.multiple = Number(next()); break;
+      case '--target': o.target = Number(next()); break;
+      case '--port': o.port = Number(next()); break;
+      case '--keep': o.keep = true; break;
+      default: throw new Error(`unknown option ${a}`);
+    }
+  }
+  if (o.mode !== 'delete-refill' && o.mode !== 'update') {
+    throw new Error(`--mode must be delete-refill or update, got ${o.mode}`);
+  }
+  return o;
+}
+
+const opt = parseArgs(process.argv.slice(2));
+const BIN = process.env.MFDB_BIN || path.resolve(__dirname, '../../zig-out/bin/multiforadb');
+const DBFILE = process.env.CHURN_DB ||
+  path.resolve(__dirname, `../../.zig-cache/churn-${opt.port}.log`);
+const URL = `mongodb://127.0.0.1:${opt.port}`;
+
+const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
+
+// A fixed seed, so two runs churn the same documents in the same order and a
+// difference between them is the code rather than the dice.
+let seed = 0x9e3779b9;
+function rnd() {
+  seed ^= seed << 13; seed >>>= 0;
+  seed ^= seed >> 17;
+  seed ^= seed << 5; seed >>>= 0;
+  return seed / 0x100000000;
+}
+const pick = (n) => Math.floor(rnd() * n);
+
+let server = null;
+let serverDead = false;
+// Bounded: the listeners below run for the life of the process and only the
+// last few lines are ever read, so an unbounded string would hold a rope
+// proportional to everything the server ever said.
+let serverLog = '';
+const noteServerLog = (d) => {
+  serverLog = (serverLog + d).slice(-65536);
+};
+
+function cleanup() {
+  if (server && !serverDead) {
+    try { server.kill('SIGKILL'); } catch {}
+  }
+}
+process.on('exit', cleanup);
+process.on('SIGINT', () => { cleanup(); process.exit(130); });
+process.on('SIGTERM', () => { cleanup(); process.exit(143); });
+
+function startServer() {
+  return new Promise((resolve, reject) => {
+    fs.rmSync(DBFILE, { force: true });
+    fs.rmSync(DBFILE + '.data', { force: true });
+    serverDead = false;
+    server = spawn(BIN, ['--port', String(opt.port), '--db', DBFILE], {
+      stdio: ['ignore', 'pipe', 'pipe'],
+    });
+    server.stdout.on('data', noteServerLog);
+    server.stderr.on('data', noteServerLog);
+    server.on('error', (e) => reject(new Error(`cannot start ${BIN}: ${e.message}`)));
+    server.on('exit', (code, sig) => {
+      // A child that dies must fail the start, or the poll below would find a
+      // *stale* server on the same port and measure the wrong database.
+      serverDead = true;
+      if (code !== null && sig === null) noteServerLog(`\n[child exited rc=${code}]`);
+    });
+    const deadline = Date.now() + 15000;
+    (async () => {
+      while (Date.now() < deadline) {
+        if (serverDead) {
+          reject(new Error(`server child exited during start (port ${opt.port} busy?)\n${serverLog}`));
+          return;
+        }
+        // A fresh client per attempt, deliberately: a MongoClient whose first
+        // connect fails tears its topology down and every later command on it
+        // fails the same way, so reusing one turns "not up yet" into "never
+        // comes up".
+        const c = new MongoClient(URL, { serverSelectionTimeoutMS: 1000 });
+        try {
+          await c.connect();
+          await c.db('admin').command({ ping: 1 });
+          await c.close();
+          return resolve();
+        } catch {
+          try { await c.close(); } catch {}
+          await sleep(100);
+        }
+      }
+      reject(new Error(`server did not come up on :${opt.port}\n${serverLog}`));
+    })();
+  });
+}
+
+async function stopServer() {
+  if (!server) return;
+  const exited = new Promise((r) => server.once('exit', r));
+  server.kill('SIGTERM');
+  await Promise.race([exited, sleep(5000)]);
+  serverDead = true;
+  server = null;
+}
+
+function dataFileBytes() {
+  try {
+    return fs.statSync(DBFILE + '.data').size;
+  } catch {
+    return 0;
+  }
+}
+
+const MB = (n) => (n / (1 << 20)).toFixed(1);
+
+// Live bytes are computed here rather than read off the server, so the ratio
+// means the same thing whatever binary is under test. A `multifora` section is
+// a recent addition; without it the counters print as n/a and the ratio -- the
+// number the gate is actually about -- is still measured, which is what makes
+// a run against an older build comparable.
+let docBytes = 0;
+
+async function stats(client, count) {
+  const s = await client.db('admin').command({ serverStatus: 1 });
+  const m = s.multifora || null;
+  return {
+    live: count * docBytes,
+    count,
+    file: dataFileBytes(),
+    m: m && {
+      live: Number(m.liveBytes),
+      dead: Number(m.deadBytes),
+      reclaimed: Number(m.reclaimedBytes),
+      runs: Number(m.slabRuns),
+      freeReady: Number(m.freeReadyBytes),
+      allocTail: Number(m.allocTailBytes),
+      compactions: Number(m.compactions),
+    },
+  };
+}
+
+function report(label, s) {
+  const ratio = s.live > 0 ? s.file / s.live : 0;
+  let line = `  ${label.padEnd(12)} ratio ${ratio.toFixed(2)}x  file ${MB(s.file)}MB  live ${MB(s.live)}MB`;
+  if (s.m) {
+    // What the database is actually occupying, as opposed to what it has ever
+    // had to occupy. The file never shrinks, so `ratio` is a high-water mark
+    // and cannot come down however well reclamation works; `inUse` is the
+    // number that moves when it does.
+    const inUse = s.m.allocTail - s.m.freeReady;
+    line += `  inUse ${(inUse / s.live).toFixed(2)}x` +
+      `  dead ${MB(s.m.dead)}MB  reclaimed ${MB(s.m.reclaimed)}MB` +
+      `  allocTail ${MB(s.m.allocTail)}MB  freeReady ${MB(s.m.freeReady)}MB` +
+      `  runs ${s.m.runs}  compactions ${s.m.compactions}`;
+  } else {
+    line += '  (no multifora section: counters n/a)';
+  }
+  console.log(line);
+  return ratio;
+}
+
+// One document of about `opt.docSize` payload bytes. `k` is the field a
+// secondary index covers and an update rewrites, so index maintenance is part
+// of the churn rather than a constant.
+const PAD = 'x'.repeat(Math.max(1, opt.docSize));
+function makeDoc(id) {
+  return { _id: id, k: id % 1000, pad: PAD };
+}
+
+// Batches sized so one insertMany stays well under the 48 MB wire limit
+// whatever --doc-size is.
+function batchSize() {
+  return Math.max(1, Math.min(1000, Math.floor((8 << 20) / (opt.docSize + 64))));
+}
+
+async function insertRange(coll, from, to) {
+  const bs = batchSize();
+  for (let i = from; i < to; i += bs) {
+    const docs = [];
+    for (let j = i; j < Math.min(i + bs, to); j++) docs.push(makeDoc(j));
+    await coll.insertMany(docs, { ordered: false });
+  }
+}
+
+async function main() {
+  console.log(
+    `churn: ${opt.docs} x ${opt.docSize} B, mode ${opt.mode}` +
+    `${opt.index ? ', one secondary index' : ''}` +
+    `${opt.mode === 'delete-refill' ? `, ${opt.rounds} rounds` : `, ${opt.multiple}x writes`}`,
+  );
+  console.log(`churn: platform ${process.platform}/${process.arch}, binary ${BIN}`);
+  await startServer();
+  const client = new MongoClient(URL, { serverSelectionTimeoutMS: 5000 });
+  await client.connect();
+  const db = client.db('churn');
+  const coll = db.collection('c');
+
+  const t0 = Date.now();
+  // The exact serialized size of one document, so `live` is a real byte count
+  // rather than the payload size the caller asked for.
+  docBytes = require('mongodb').BSON.serialize(makeDoc(0)).length;
+  await insertRange(coll, 0, opt.docs);
+  if (opt.index) await coll.createIndex({ k: 1 });
+  const base = await stats(client, opt.docs);
+  report('loaded', base);
+
+  const ratios = [];
+  let nextId = opt.docs;
+  if (opt.mode === 'delete-refill') {
+    // Half the collection dies and is replaced by fresh documents, so the live
+    // size is flat and everything the file gains is garbage that was not
+    // reclaimed.
+    const half = Math.floor(opt.docs / 2);
+    // The ids actually live, so a round deletes exactly `half` documents and
+    // the collection stays the same size. Sampling blind from the id space
+    // re-picks already-dead ids, which deletes fewer than it inserts and turns
+    // a flat-live measurement into a growing one.
+    const live = Array.from({ length: opt.docs }, (_, i) => i);
+    for (let r = 0; r < opt.rounds; r++) {
+      const ids = [];
+      for (let i = 0; i < half; i++) {
+        const at = pick(live.length);
+        ids.push(live[at]);
+        live[at] = live[live.length - 1];
+        live.pop();
+      }
+      // One delete spec per id, not one `$in` over thousands of them. The
+      // server's planner refuses to use an index for an `$in` wider than
+      // `index.max_combos` (100), so a 5000-element one falls back to a full
+      // collection scan that re-filters every document against every member --
+      // quadratic in `--docs`, and it was the whole of this harness's runtime:
+      // the documented 40k x 16 KiB gate took 62 s and takes 11 s now, and the
+      // 150k x 200 B line went from ~480 s to 3 s. Reported ratios are
+      // unchanged, which is the point: this was the instrument's cost, not the
+      // database's.
+      const bs = 5000;
+      for (let i = 0; i < ids.length; i += bs) {
+        await coll.bulkWrite(
+          ids.slice(i, i + bs).map((id) => ({ deleteOne: { filter: { _id: id } } })),
+          { ordered: false },
+        );
+      }
+      await insertRange(coll, nextId, nextId + ids.length);
+      for (let i = 0; i < ids.length; i++) live.push(nextId + i);
+      nextId += ids.length;
+      ratios.push(report(`round ${r + 1}`, await stats(client, opt.docs)));
+    }
+  } else {
+    // The same documents rewritten over and over: every rewrite leaves the old
+    // copy behind, and old copies die in insertion order, which is the best
+    // case for reclaiming whole windows.
+    const total = opt.docs * opt.multiple;
+    const per = Math.floor(total / opt.rounds);
+    for (let r = 0; r < opt.rounds; r++) {
+      let done = 0;
+      while (done < per) {
+        const ops = [];
+        for (let i = 0; i < Math.min(2000, per - done); i++) {
+          const id = pick(opt.docs);
+          ops.push({ updateOne: { filter: { _id: id }, update: { $set: { k: pick(1000) } } } });
+        }
+        await coll.bulkWrite(ops, { ordered: false });
+        done += ops.length;
+      }
+      ratios.push(report(`round ${r + 1}`, await stats(client, opt.docs)));
+    }
+  }
+
+  // The one real count in the run, and the only one the check below needs.
+  const count = await coll.countDocuments({});
+  const final = await stats(client, count);
+  const elapsed = ((Date.now() - t0) / 1000).toFixed(0);
+  console.log(`churn: ${count} documents live at the end, ${elapsed}s`);
+
+  // Steady state is the second half of the rounds: the first ones are still
+  // filling the file out and say nothing about where it settles.
+  const tail = ratios.slice(Math.floor(ratios.length / 2));
+  const steady = tail.reduce((a, b) => a + b, 0) / tail.length;
+  const drift = tail.length > 1 ? tail[tail.length - 1] - tail[0] : 0;
+  console.log(
+    `churn: steady state ${steady.toFixed(2)}x over the last ${tail.length} rounds, ` +
+    `drift ${drift >= 0 ? '+' : ''}${drift.toFixed(2)}x`,
+  );
+  if (final.m) {
+    console.log(
+      `churn: reclaimed ${MB(final.m.reclaimed)}MB total, ` +
+      `${final.m.compactions} collection rebuilds`,
+    );
+  }
+
+  await client.close();
+  if (!opt.keep) {
+    fs.rmSync(DBFILE, { force: true });
+    fs.rmSync(DBFILE + '.data', { force: true });
+  }
+  await stopServer();
+
+  if (count !== opt.docs) {
+    console.log(`CHURN_FAIL: ${count} documents live, expected ${opt.docs}`);
+    process.exit(1);
+  }
+  if (opt.target !== null && steady > opt.target) {
+    console.log(`CHURN_FAIL: steady state ${steady.toFixed(2)}x is above the ${opt.target}x target`);
+    process.exit(1);
+  }
+  console.log('CHURN_OK');
+}
+
+main().catch((e) => {
+  console.error('CHURN_FAIL', e);
+  console.log('--- server log tail ---');
+  console.log(serverLog.split('\n').slice(-40).join('\n'));
+  process.exit(1);
+});

tests/e2e/e2e7.js

@@ -0,0 +1,474 @@
+// E2E part 7: server-side cursors, self-contained.
+//
+// Spawns its own multiforadb servers, because cursor behaviour is only
+// observable with non-default flags (a short idle timeout, a tiny registry) and
+// with raw `runCommand` — the driver hides `cursor.id`, which is the thing under
+// test.
+//
+//   node tests/e2e/e2e7.js
+//
+// Env:  E2E7_PORT  listen port (default 27230)
+//       MFDB_BIN   server binary (default ../../zig-out/bin/multiforadb)
+//       E2E7_KEEP  keep the log files after the run
+//
+// The one rule most of this file is about: **never look ahead.** A batch ends
+// either because it reached its target — cursor stays open — or because the
+// source reported EOF, and only then does the cursor close with `id: 0`. So four
+// documents at `batchSize: 2` require a third command answering an empty
+// `nextBatch` with `id: 0`. That empty terminal batch is correct, and it is what
+// real mongod does (measured, not assumed — see checks 5 and 6).
+const { MongoClient, Long } = require('mongodb');
+const { spawn } = require('child_process');
+const fs = require('fs');
+const path = require('path');
+
+const PORT = Number(process.env.E2E7_PORT || 27230);
+const BIN = process.env.MFDB_BIN || path.resolve(__dirname, '../../zig-out/bin/multiforadb');
+const DBFILE = path.resolve(__dirname, '../../.zig-cache/e2e7-cursors.log');
+const URL = `mongodb://127.0.0.1:${PORT}`;
+
+const results = [];
+function check(name, cond, detail = '') {
+  results.push({ name, ok: !!cond, detail: String(detail) });
+  if (!cond) console.error(`  x ${name} ${detail}`);
+}
+function eq(name, got, want) {
+  const ok = JSON.stringify(got) === JSON.stringify(want);
+  check(name, ok, ok ? '' : `got ${JSON.stringify(got)} want ${JSON.stringify(want)}`);
+}
+const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
+
+/// The error code a command produces, or 0 when it succeeds.
+async function codeOf(fn) {
+  try {
+    await fn();
+    return 0;
+  } catch (e) {
+    return e.code === undefined ? -1 : e.code;
+  }
+}
+
+let server = null;
+let serverLog = '';
+let serverDead = false;
+
+function cleanup() {
+  if (server && !serverDead) {
+    try { server.kill('SIGKILL'); } catch {}
+  }
+}
+process.on('exit', cleanup);
+process.on('SIGINT', () => { cleanup(); process.exit(130); });
+process.on('SIGTERM', () => { cleanup(); process.exit(143); });
+
+function startServer(args, fresh = true) {
+  return new Promise((resolve, reject) => {
+    if (fresh) {
+      fs.rmSync(DBFILE, { force: true });
+      fs.rmSync(DBFILE + '.data', { force: true });
+    }
+    serverDead = false;
+    serverLog = '';
+    server = spawn(BIN, ['--port', String(PORT), '--db', DBFILE, ...args], {
+      stdio: ['ignore', 'pipe', 'pipe'],
+    });
+    server.stdout.on('data', (d) => (serverLog += d));
+    server.stderr.on('data', (d) => (serverLog += d));
+    server.on('error', (e) => reject(new Error(`cannot start ${BIN}: ${e.message}`)));
+    server.on('exit', (code, sig) => {
+      serverDead = true;
+      if (code !== null && sig === null) serverLog += `\n[child exited rc=${code}]`;
+    });
+    const deadline = Date.now() + 15000;
+    (async () => {
+      while (Date.now() < deadline) {
+        if (serverDead) {
+          reject(new Error(`server exited during start (port ${PORT} busy?)\n${serverLog}`));
+          return;
+        }
+        const c = new MongoClient(URL, { serverSelectionTimeoutMS: 1000 });
+        try {
+          await c.connect();
+          await c.db('admin').command({ ping: 1 });
+          await c.close();
+          return resolve();
+        } catch {
+          try { await c.close(); } catch {}
+          await sleep(100);
+        }
+      }
+      reject(new Error(`server did not come up on :${PORT}\n${serverLog}`));
+    })();
+  });
+}
+
+async function stopServer(sig = 'SIGTERM') {
+  if (!server) return;
+  const exited = new Promise((r) => server.once('exit', r));
+  server.kill(sig);
+  await Promise.race([exited, sleep(5000)]);
+  serverDead = true;
+  server = null;
+}
+
+// ---------------------------------------------------------------------------
+// Phase A — batching, lifecycle and errors, on default flags
+// ---------------------------------------------------------------------------
+
+async function phaseA(db) {
+  const col = db.collection('c');
+  await col.deleteMany({});
+  await col.insertMany([...Array(250)].map((_, i) => ({ _id: i + 1, x: i })));
+
+  // 1. A cursor is a real cursor: nonzero id, a namespace with both parts.
+  let r = await db.command({ find: 'c', filter: {}, batchSize: 2 });
+  eq('1 batchSize 2 returns 2', r.cursor.firstBatch.length, 2);
+  check('1 cursor id is nonzero', r.cursor.id > 0, r.cursor.id);
+  eq('1 ns is db.coll', r.cursor.ns, 'e2e7.c');
+
+  // 2. The default first batch is 101, MongoDB's own
+  //    internalQueryFindCommandBatchSize.
+  eq('2 default first batch is 101', (await db.command({ find: 'c', filter: {} })).cursor.firstBatch.length, 101);
+
+  // 3. A getMore naming no batchSize is bounded by bytes, not by the batchSize
+  //    the cursor was created with. Measured against mongod 8.3.7: find with
+  //    batchSize 2 then a bare getMore returns 4998 of 5000 documents.
+  const id3 = (await db.command({ find: 'c', filter: {}, batchSize: 2 })).cursor.id;
+  let g = await db.command({ getMore: id3, collection: 'c' });
+  eq('3 a bare getMore drains the rest', g.cursor.nextBatch.length, 248);
+  eq('3 and closes at EOF', String(g.cursor.id), '0');
+
+  // 4. A getMore's batchSize applies to that batch only.
+  const id4 = (await db.command({ find: 'c', filter: {}, batchSize: 2 })).cursor.id;
+  g = await db.command({ getMore: id4, collection: 'c', batchSize: 3 });
+  eq('4 getMore batchSize 3', g.cursor.nextBatch.map((d) => d._id), [3, 4, 5]);
+  check('4 still open', g.cursor.id > 0);
+
+  // 5. No look-ahead: 4 documents at batchSize 2 needs a third command whose
+  //    nextBatch is empty. Closing on "batch full and source dry" would break
+  //    the command counts the pinned spec suites assert.
+  const four = db.collection('four');
+  await four.deleteMany({});
+  await four.insertMany([1, 2, 3, 4].map((i) => ({ _id: i })));
+  r = await db.command({ find: 'four', filter: {}, batchSize: 2 });
+  g = await db.command({ getMore: r.cursor.id, collection: 'four', batchSize: 2 });
+  eq('5 second full batch is 2 documents', g.cursor.nextBatch.length, 2);
+  check('5 and leaves the cursor open', g.cursor.id > 0);
+  g = await db.command({ getMore: g.cursor.id, collection: 'four', batchSize: 2 });
+  eq('5 terminal batch is empty and closed', [g.cursor.nextBatch.length, String(g.cursor.id)], [0, '0']);
+
+  // 6. limit is an EOF source, so the batch that exhausts it also closes the
+  //    cursor. This is why the driver sends batchSize = limit + 1.
+  r = await db.command({ find: 'c', filter: {}, sort: { _id: 1 }, limit: 4, batchSize: 5 });
+  eq('6 limit 4 batchSize 5 closes in one reply', String(r.cursor.id), '0');
+  eq('6 and returns exactly the limit', r.cursor.firstBatch.map((d) => d._id), [1, 2, 3, 4]);
+  r = await db.command({ find: 'c', filter: {}, sort: { _id: 1 }, limit: 4, batchSize: 2 });
+  check('6 limit 4 batchSize 2 stays open', r.cursor.id > 0);
+  g = await db.command({ getMore: r.cursor.id, collection: 'c', batchSize: 2 });
+  eq('6 the batch reaching the limit closes', String(g.cursor.id), '0');
+  eq('6 across batches, limit still honoured', g.cursor.nextBatch.map((d) => d._id), [3, 4]);
+
+  // 7. skip is consumed once, at creation, and never re-applied.
+  r = await db.command({ find: 'c', filter: {}, skip: 20, batchSize: 3 });
+  eq('7 skip 20 starts at 21', r.cursor.firstBatch.map((d) => d._id), [21, 22, 23]);
+  g = await db.command({ getMore: r.cursor.id, collection: 'c', batchSize: 3 });
+  eq('7 skip not re-applied on getMore', g.cursor.nextBatch.map((d) => d._id), [24, 25, 26]);
+
+  // 8. batchSize 0 is a real request for an empty batch with a live cursor, not
+  //    "unbounded". Drivers use it to obtain a cursor cheaply. Nothing may be
+  //    consumed.
+  r = await db.command({ find: 'c', filter: {}, batchSize: 0 });
+  eq('8 batchSize 0 returns nothing', r.cursor.firstBatch.length, 0);
+  check('8 but a live cursor', r.cursor.id > 0);
+  g = await db.command({ getMore: r.cursor.id, collection: 'c', batchSize: 1 });
+  eq('8 nothing was consumed', g.cursor.nextBatch.map((d) => d._id), [1]);
+
+  // 9. singleBatch, and its wire-legacy form, a negative limit.
+  eq('9 singleBatch closes', String((await db.command({ find: 'c', filter: {}, batchSize: 2, singleBatch: true })).cursor.id), '0');
+  r = await db.command({ find: 'c', filter: {}, limit: -3 });
+  eq('9 negative limit is one batch', [r.cursor.firstBatch.length, String(r.cursor.id)], [3, '0']);
+
+  // 10. A cursor is not pinned to the connection that created it: the driver
+  //     spec allows a getMore from any connection to the same server.
+  const other = new MongoClient(URL);
+  await other.connect();
+  const shared = (await db.command({ find: 'c', filter: {}, batchSize: 2 })).cursor.id;
+  eq('10 getMore from another connection', await codeOf(() => other.db('e2e7').command({ getMore: shared, collection: 'c', batchSize: 2 })), 0);
+  await other.close();
+
+  // 11. A getMore naming the wrong collection is Unauthorized (13), not 43, and
+  //     leaves the cursor alive — the request is wrong, not the cursor.
+  //     Measured against mongod, which answers exactly this code.
+  const live = (await db.command({ find: 'c', filter: {}, batchSize: 2 })).cursor.id;
+  eq('11 wrong collection is Unauthorized 13', await codeOf(() => db.command({ getMore: live, collection: 'four' })), 13);
+  eq('11 the cursor survived it', (await db.command({ getMore: live, collection: 'c', batchSize: 1 })).cursor.nextBatch.length, 1);
+
+  // 12. killCursors: all four arrays, and the right partitioning.
+  let k = await db.command({ killCursors: 'c', cursors: [live] });
+  eq('12 a live cursor is killed', k.cursorsKilled.map(String), [String(live)]);
+  check('12 all four arrays present', ['cursorsKilled', 'cursorsNotFound', 'cursorsAlive', 'cursorsUnknown'].every((f) => Array.isArray(k[f])), Object.keys(k).join(','));
+  k = await db.command({ killCursors: 'c', cursors: [live] });
+  eq('12 killing it twice reports notFound', k.cursorsNotFound.map(String), [String(live)]);
+  const other_ns = (await db.command({ find: 'four', filter: {}, batchSize: 1 })).cursor.id;
+  k = await db.command({ killCursors: 'c', cursors: [other_ns] });
+  eq('12 a wrong-namespace id reports notFound', k.cursorsNotFound.map(String), [String(other_ns)]);
+  eq('12 and that cursor still lives', await codeOf(() => db.command({ getMore: other_ns, collection: 'four', batchSize: 1 })), 0);
+
+  // 13. Malformed and unknown ids.
+  eq('13 getMore after kill is 43', await codeOf(() => db.command({ getMore: live, collection: 'c' })), 43);
+  eq('13 id 0 is 43', await codeOf(() => db.command({ getMore: Long.fromNumber(0), collection: 'c' })), 43);
+  eq('13 an unknown id is 43', await codeOf(() => db.command({ getMore: Long.fromString('987654321'), collection: 'c' })), 43);
+  eq('13 a non-numeric id is TypeMismatch 14', await codeOf(() => db.command({ getMore: 'nope', collection: 'c' })), 14);
+  eq('13 a missing collection is BadValue 2', await codeOf(() => db.command({ getMore: Long.fromNumber(1) })), 2);
+
+  // 14. tailable is refused, which is parity: mongod rejects it on a non-capped
+  //     collection and this engine has none. Ignoring it would make a driver's
+  //     tail loop exit, which the application reads as data loss.
+  eq('14 tailable is BadValue 2', await codeOf(() => db.command({ find: 'c', filter: {}, tailable: true })), 2);
+  eq('14 awaitData alone is BadValue 2', await codeOf(() => db.command({ find: 'c', filter: {}, awaitData: true })), 2);
+
+  // 15. The driver's own iteration, which is the point of all of the above.
+  const ids = (await col.find({}).batchSize(7).toArray()).map((d) => d._id);
+  eq('15 driver drains 250 at batchSize 7', ids.length, 250);
+  eq('15 no duplicates and no gaps', [new Set(ids).size, Math.min(...ids), Math.max(...ids)], [250, 1, 250]);
+  eq('15 sort+skip+limit unchanged', (await col.find({ _id: { $gt: 2 } }, { sort: { _id: 1 }, skip: 2, limit: 2 }).toArray()).map((d) => d._id), [5, 6]);
+  // A sort no index provides must materialize; it still has to drain correctly.
+  eq('15 an unindexed sort drains in order', (await col.find({}, { sort: { x: -1 } }).batchSize(10).toArray()).map((d) => d.x)[0], 249);
+}
+
+// ---------------------------------------------------------------------------
+// Phase B — streaming cursors: resume across writes, and what survives a rebuild
+// ---------------------------------------------------------------------------
+
+async function phaseB(db) {
+  const col = db.collection('s');
+  await col.deleteMany({});
+  await col.insertMany([...Array(300)].map((_, i) => ({ _id: i + 1, a: i % 5, pad: 'q'.repeat(200) })));
+
+  // 16. A whole-index walk holds a key, not a list, so it resumes across writes
+  //     that move documents. The bug this caught: an update rewrites a document
+  //     to a new offset, and resuming by band position returned it twice.
+  let r = await db.command({ find: 's', filter: {}, batchSize: 10 });
+  const seen = new Set(r.cursor.firstBatch.map((d) => d._id));
+  let dupes = 0;
+  let id = r.cursor.id;
+  let rounds = 0;
+  let errored = 0;
+  while (String(id) !== '0' && rounds++ < 200) {
+    // Churn between every batch: updates rewrite documents, which both moves
+    // them in the slab and can split leaves.
+    await col.updateMany({ _id: { $lt: 60 } }, { $inc: { n: 1 } });
+    let g;
+    try {
+      g = await db.command({ getMore: id, collection: 's', batchSize: 10 });
+    } catch (e) {
+      errored = e.code;
+      break;
+    }
+    for (const d of g.cursor.nextBatch) {
+      if (seen.has(d._id)) dupes++;
+      seen.add(d._id);
+    }
+    id = g.cursor.id;
+  }
+  eq('16 draining across churn did not error', errored, 0);
+  eq('16 no document came back twice', dupes, 0);
+  eq('16 every document was returned', seen.size, 300);
+  check('16 and nothing outside the collection', [...seen].every((v) => v >= 1 && v <= 300));
+
+  // 17. Both directions stream, over the _id_ index and a secondary one.
+  eq('17 ascending _id sort drains', (await col.find({}, { sort: { _id: 1 } }).batchSize(9).toArray()).length, 300);
+  const desc = (await col.find({}, { sort: { _id: -1 } }).batchSize(9).toArray()).map((d) => d._id);
+  eq('17 descending drains in order', [desc.length, desc[0], desc[299]], [300, 300, 1]);
+  await col.createIndex({ a: 1 });
+  const bya = await col.find({}, { sort: { a: 1 } }).batchSize(11).toArray();
+  eq('17 a secondary-index sort drains', bya.length, 300);
+  check('17 and in the index order', bya.every((d, i) => i === 0 || bya[i - 1].a <= d.a));
+
+  // 18. Dropping the index a stream is following cannot be resumed — the walk
+  //     has nothing left to walk. That must be a clean error, not garbage.
+  await col.createIndex({ b: 1 });
+  const onb = (await db.command({ find: 's', filter: {}, sort: { b: 1 }, batchSize: 3 })).cursor.id;
+  await col.dropIndex('b_1');
+  eq('18 dropping the streamed index is 175', await codeOf(() => db.command({ getMore: onb, collection: 's', batchSize: 3 })), 175);
+
+  // 19. Dropping the collection kills every kind of cursor.
+  const doomed = (await db.command({ find: 's', filter: {}, batchSize: 3 })).cursor.id;
+  await col.drop();
+  const dc = await codeOf(() => db.command({ getMore: doomed, collection: 's', batchSize: 3 }));
+  check('19 dropping the collection kills the cursor', dc === 175 || dc === 43, dc);
+}
+
+// ---------------------------------------------------------------------------
+// Phase C — aggregate, the listing commands, and count
+// ---------------------------------------------------------------------------
+
+async function phaseC(db) {
+  const col = db.collection('g');
+  await col.deleteMany({});
+  await col.insertMany([...Array(250)].map((_, i) => ({ _id: i + 1, g: i % 40, v: i })));
+
+  // 20. aggregate batches through cursor.batchSize; a bare cursor is the default.
+  let r = await db.command({ aggregate: 'g', pipeline: [], cursor: { batchSize: 3 } });
+  eq('20 aggregate batchSize 3', r.cursor.firstBatch.length, 3);
+  check('20 aggregate cursor is real', r.cursor.id > 0);
+  eq('20 aggregate ns', r.cursor.ns, 'e2e7.g');
+  const g20 = await db.command({ getMore: r.cursor.id, collection: 'g', batchSize: 5 });
+  eq('20 aggregate getMore continues', g20.cursor.nextBatch.map((d) => d._id), [4, 5, 6, 7, 8]);
+  eq('20 bare cursor defaults to 101', (await db.command({ aggregate: 'g', pipeline: [], cursor: {} })).cursor.firstBatch.length, 101);
+  eq('20 driver aggregate drains', (await col.aggregate([], { batchSize: 7 }).toArray()).length, 250);
+  const groups = await col.aggregate([{ $group: { _id: '$g', n: { $sum: 1 } } }], { batchSize: 6 }).toArray();
+  eq('20 $group drains across batches', [groups.length, groups.reduce((a, x) => a + x.n, 0)], [40, 250]);
+  eq('20 $count stage', await col.aggregate([{ $count: 'total' }]).toArray(), [{ total: 250 }]);
+
+  // 21. listCollections' namespace. It used to be "<db>." with an empty
+  //     collection part, and the driver throws client-side on a namespace like
+  //     that — so the moment the cursor stopped being id 0 it would have broken.
+  for (let i = 0; i < 12; i++) await db.createCollection('k' + i);
+  r = await db.command({ listCollections: 1, cursor: { batchSize: 4 } });
+  eq('21 listCollections ns has a collection part', r.cursor.ns, 'e2e7.$cmd.listCollections');
+  eq('21 listCollections honours batchSize', r.cursor.firstBatch.length, 4);
+  check('21 listCollections cursor is real', r.cursor.id > 0);
+  const g21 = await db.command({ getMore: r.cursor.id, collection: '$cmd.listCollections', batchSize: 100 });
+  check('21 its getMore works', g21.cursor.nextBatch.length >= 8, g21.cursor.nextBatch.length);
+  const listed = await db.listCollections({}, { batchSize: 3 }).toArray();
+  check('21 driver listCollections drains', listed.length >= 13, listed.length);
+
+  // 22. listIndexes.
+  await col.createIndex({ v: 1 });
+  await col.createIndex({ g: 1 });
+  await col.createIndex({ v: -1, g: 1 });
+  r = await db.command({ listIndexes: 'g', cursor: { batchSize: 2 } });
+  eq('22 listIndexes honours batchSize', r.cursor.firstBatch.length, 2);
+  eq('22 listIndexes ns', r.cursor.ns, 'e2e7.g');
+  eq('22 driver listIndexes drains', (await col.listIndexes({ batchSize: 1 }).toArray()).length, 4);
+
+  // 23. count honoured neither skip nor limit before, which made
+  //     countDocuments(f, {limit}) a silent wrong answer.
+  eq('23 count plain', (await db.command({ count: 'g' })).n, 250);
+  eq('23 count limit', (await db.command({ count: 'g', limit: 10 })).n, 10);
+  eq('23 count skip', (await db.command({ count: 'g', skip: 240 })).n, 10);
+  eq('23 count skip and limit', (await db.command({ count: 'g', skip: 245, limit: 10 })).n, 5);
+  eq('23 count skip past the end', (await db.command({ count: 'g', skip: 1000 })).n, 0);
+  eq('23 count with a query and limit', (await db.command({ count: 'g', query: { g: 0 }, limit: 3 })).n, 3);
+  eq('23 driver countDocuments limit', await col.countDocuments({}, { limit: 7 }), 7);
+
+  // 24. A batch is capped by bytes as well as by documents, so a large-document
+  //     result splits instead of building a reply past the advertised message
+  //     size. 40 documents of ~1 MiB cannot all fit one 16 MiB batch.
+  const big = db.collection('big');
+  await big.deleteMany({});
+  const pad = 'p'.repeat(1024 * 1024 - 64);
+  for (let i = 0; i < 40; i++) await big.insertOne({ _id: i + 1, pad });
+  r = await db.command({ find: 'big', filter: {}, batchSize: 40 });
+  check('24 the byte cap split the batch', r.cursor.firstBatch.length >= 1 && r.cursor.firstBatch.length <= 16, r.cursor.firstBatch.length);
+  check('24 and left the cursor open', r.cursor.id > 0);
+  eq('24 the whole result still drains', (await big.find({}).batchSize(40).toArray()).length, 40);
+}
+
+// ---------------------------------------------------------------------------
+// Phase D — expiry, capacity, and what a restart does
+// ---------------------------------------------------------------------------
+
+async function phaseD(db) {
+  const col = db.collection('e');
+  await col.deleteMany({});
+  await col.insertMany([...Array(50)].map((_, i) => ({ _id: i + 1 })));
+
+  // 25. An idle cursor is reaped; noCursorTimeout exempts one from that.
+  const perishable = (await db.command({ find: 'e', filter: {}, batchSize: 2 })).cursor.id;
+  const immortal = (await db.command({ find: 'e', filter: {}, batchSize: 2, noCursorTimeout: true })).cursor.id;
+  await sleep(300);
+  eq('25 before the timeout it is alive', await codeOf(() => db.command({ getMore: perishable, collection: 'e', batchSize: 1 })), 0);
+  await sleep(2500);
+  eq('25 an idle cursor is reaped', await codeOf(() => db.command({ getMore: perishable, collection: 'e', batchSize: 1 })), 43);
+  eq('25 noCursorTimeout survives', await codeOf(() => db.command({ getMore: immortal, collection: 'e', batchSize: 1 })), 0);
+  const k = await db.command({ killCursors: 'e', cursors: [immortal] });
+  eq('25 but is still killable', k.cursorsKilled.map(String), [String(immortal)]);
+
+  // 26. A full registry evicts the least recently used cursor rather than
+  //     refusing the new one. The victim sees the same 43 an idle timeout gives,
+  //     which every driver already handles.
+  const ids = [];
+  for (let i = 0; i < 5; i++) ids.push((await db.command({ find: 'e', filter: {}, batchSize: 1 })).cursor.id);
+  eq('26 the oldest was evicted', await codeOf(() => db.command({ getMore: ids[0], collection: 'e', batchSize: 1 })), 43);
+  const alive = [];
+  for (const id of ids.slice(1)) alive.push(await codeOf(() => db.command({ getMore: id, collection: 'e', batchSize: 1 })));
+  eq('26 the newest four are alive', alive, [0, 0, 0, 0]);
+}
+
+async function phaseE(db, staleId) {
+  // 27. Cursors do not survive a restart, and a stale id must be a clean 43 —
+  //     not a hang, and not an empty batch claiming the result ended.
+  eq('27 a cursor from before the restart is 43', await codeOf(() => db.command({ getMore: staleId, collection: 'e', batchSize: 1 })), 43);
+  const fresh = await db.command({ find: 'e', filter: {}, batchSize: 2 });
+  check('27 and new cursors work after a restart', fresh.cursor.id > 0);
+}
+
+async function main() {
+  // The same guard e2e6.js and big.js carry: without it a missing binary
+  // surfaces as a generic spawn error instead of saying what to do about it.
+  if (!fs.existsSync(BIN)) {
+    console.error(`E2E7_FAIL server binary not found: ${BIN}\n  run: zig build`);
+    process.exit(1);
+  }
+
+  // Phase A-C on default cursor flags.
+  await startServer(['--ttl-sweep-secs', '0', '--compact-threshold', '1m'], true);
+  let client = new MongoClient(URL);
+  await client.connect();
+  let db = client.db('e2e7');
+  console.log('phase A: batching, lifecycle, errors');
+  await phaseA(db);
+  console.log('phase B: streaming cursors across writes');
+  await phaseB(db);
+  console.log('phase C: aggregate, listings, count');
+  await phaseC(db);
+  await client.close();
+  await stopServer('SIGTERM');
+
+  // Phase D needs a short timeout and a tiny registry.
+  console.log('phase D: idle expiry and registry capacity');
+  await startServer(
+    ['--ttl-sweep-secs', '0', '--cursor-timeout-ms', '800', '--cursor-sweep-secs', '1', '--max-open-cursors', '4'],
+    true,
+  );
+  client = new MongoClient(URL);
+  await client.connect();
+  db = client.db('e2e7');
+  await phaseD(db);
+  const staleId = (await db.command({ find: 'e', filter: {}, batchSize: 1 })).cursor.id;
+  await client.close();
+  await stopServer('SIGTERM');
+
+  // Phase E: the same database, a new process.
+  console.log('phase E: a cursor does not survive a restart');
+  await startServer(['--ttl-sweep-secs', '0'], false);
+  client = new MongoClient(URL);
+  await client.connect();
+  await phaseE(client.db('e2e7'), staleId);
+  await client.close();
+
+  if (process.env.E2E7_KEEP !== '1') {
+    fs.rmSync(DBFILE, { force: true });
+    fs.rmSync(DBFILE + '.data', { force: true });
+  }
+  await stopServer('SIGTERM');
+
+  const failed = results.filter((r) => !r.ok);
+  console.log(`\n${results.length - failed.length}/${results.length} checks passed`);
+  if (failed.length) {
+    console.log('FAILED:', failed.map((f) => f.name).join(', '));
+    console.log('--- server log tail ---');
+    console.log(serverLog.split('\n').slice(-30).join('\n'));
+    process.exit(1);
+  }
+  console.log('E2E7_OK');
+}
+
+main().catch((e) => {
+  console.error('E2E7_FAIL', e);
+  console.log('--- server log tail ---');
+  console.log(serverLog.split('\n').slice(-40).join('\n'));
+  process.exit(1);
+});

tests/e2e/results/m0-gates.txt

@@ -157,3 +157,101 @@
 # not a write`. The scorecard above is the M0 figure and is left as measured;
 # those two commits took it to 163 pass / 129 fail, and `tests/spec/scorecard.txt`
 # always holds the current one.
+
+
+# ===========================================================================
+# M1 — doc-level free list (PLAN amendment A5)
+# ===========================================================================
+#
+# Same machine, same driver. Server at the M1 commit named per block,
+# ReleaseFast. These are the numbers D7.4 said an M1 item owed.
+
+[M1.1] churn gate — the doc-level free list
+  reproduce: node tests/e2e/churn.js --docs 40000 --doc-size 16k --index \
+               --mode delete-refill --rounds 6
+             node tests/e2e/churn.js --docs 40000 --doc-size 16k --index \
+               --mode update --multiple 5
+  baseline:  the same harness against the end-of-Stage-2 binary (e416ad1),
+             which has no reclamation and no `multifora` section.
+
+  The harness is committed this time (`tests/e2e/churn.js`), which is half the
+  point of the block: D7.4's numbers were real and unrepeatable. Each line
+  above runs in 6-10 s.
+
+                                  Stage 2      M1        target
+  delete half and refill, 6x      1.94x        1.94x     <= 1.45x   NOT MET
+  random $set over 5x the coll.   2.46x        2.46x     <= 1.60x   NOT MET
+  Both flat, drift +0.00x over the last three rounds.
+
+  D7.4 recorded 1.65x for the delete line. This harness reads 1.94x for the
+  *same binary* D7.4 was measured against the descendants of, so that gap is
+  the harness, not a regression: the ad-hoc version sampled ids to delete
+  blindly, which re-picks already-dead ids, deletes fewer than it inserts and
+  measures a collection that is quietly growing. The update line reproduces
+  D7.4 exactly (2.46 vs 2.47).
+
+  THE RATIO DID NOT MOVE AND THE MECHANISM WORKS. Both are true, and the
+  counters are what separate them:
+
+    round 6, update line:  reclaimed 934.0MB   dead 83.9MB
+                           allocTail 1523.5MB  freeReady 797.0MB
+                           inUse 1.16x         file/live 2.46x
+
+  Reclamation returned 934 MB over the run and the collection is occupying
+  1.16x its live data. What 2.46x measures is the data file's high-water mark,
+  and the file never shrinks. The mark is set once, in round 1, by the one
+  thing reclamation cannot avoid: a rebuild needs a whole second copy of the
+  live data before the first copy can be freed. 626 MB live + the garbage
+  standing at the moment it fires + 626 MB of copy is the number, and it is
+  reached before any free pool exists to build the copy out of.
+
+  So the floor for a rebuild-based design is ~2x, and no threshold reaches it.
+  Rebuilding earlier lowers the garbage term and raises nothing; rebuilding
+  later raises it. The plan anticipated this exact outcome and said what to do
+  about it, which is to write it down rather than tune: the remaining lever is
+  incremental compaction -- a doc-id-to-offset indirection layer, so a rebuild
+  moves documents without a second copy of everything. That is amendment A5's
+  successor and it is a milestone of its own, not a knob.
+
+  A second lever, cheaper and not attempted: give free space back to the
+  filesystem. `freeReady` stands at 797 MB with `allocTail` flat, so 52% of the
+  file is space the database owns and is not using. Returning the tail-adjacent
+  part of it needs the file never to shrink below what the fallback generation
+  references, which is a crash-safety argument and its own design pass.
+
+  What did change, and is the reason the mechanism is worth keeping:
+
+    delete-refill, 12k x 16 KiB   reclaimed 1 MB  ->  256 MB  (six rounds)
+
+  The first figure is what reclamation achieved before `compact` was made to
+  checkpoint before it copies. A checkpoint is what reclaims and a checkpoint
+  is armed by log volume; a delete logs only an `_id`, so deleting half a
+  collection moved the log by a couple of megabytes, no checkpoint ran, and the
+  rebuild got there first every time and reset the window map it would have
+  used. The harness found that on its first serious run, which is the argument
+  for committing it.
+
+[M1.2] churn gate — 200-byte documents
+  reproduce: node tests/e2e/churn.js --docs 150000 --doc-size 200 --index \
+               --mode delete-refill --rounds 4        (3 s)
+  Predicted in advance, in the plan, as a pass rather than a fault:
+
+    reclaimed 0.0 MB over four rounds, exactly as forecast.
+    ratio 3.93x on both the Stage 2 binary and M1 -- identical, flat.
+
+  Reclamation hands back whole system pages. A 16 KiB page on this machine
+  holds ~70 documents of 200 bytes and the chance that all 70 are dead at once
+  under uniform deletion is nil, so nothing is ever handed back. The forecast
+  said the ratio would not improve and the counters would show a mechanism
+  that correctly does nothing, rather than one that silently misfires; that is
+  what they show.
+
+  Read the ratio on this line with care. `live` counts document bytes, and at
+  200 bytes the two index trees are comparable in size to the documents
+  themselves -- the file is already 2.18x at load, before any churn. That
+  overhead is index structure, not slab garbage, and it is not what this gate
+  is about.
+
+  The payoff of window reclamation scales as doc_size / map_align, so a 4 KiB
+  system page (x86-64 Linux) reads four times better on the same code. Every
+  number in this file is Apple Silicon with 16 KiB pages.

tests/fuzz/crash-fuzz.js

@@ -561,8 +561,41 @@ async function verify(client, base, r, cycleNo) {
       .map((d) => canon(d))
       .sort();
     if (got.length !== want.length || got.some((c, i) => c !== want[i])) {
+      // Decisive diagnostic: the same question asked without the index. `dbDocs`
+      // came from find({}) on this same reopened server, so filtering it here
+      // says whether the *documents* are wrong or only the index's answer about
+      // them. An index that returns fewer documents than a scan is the canonical
+      // under-approximation -- candidates are generated from the index and the
+      // full filter is only re-applied to those, so a missing entry is a
+      // silently missing result.
+      const scanGot = dbDocs.filter((d) => d.k === v).map((d) => canon(d)).sort();
       throw new Fail(`cycle ${cycleNo}: find({k:${v}}) mismatch at prefix ${matched}`, {
-        cycleNo, v, matched, got, want,
+        cycleNo,
+        v,
+        matched,
+        verdict:
+          scanGot.length === want.length && scanGot.every((c, i) => c === want[i])
+            ? 'INDEX under-approximates: a scan of the same server returns the expected documents'
+            : 'DOCUMENTS differ too: the scan does not match the model either',
+        indexReturned: got.length,
+        scanReturned: scanGot.length,
+        modelExpected: want.length,
+        // Per-key totals, so a single lost leaf is distinguishable from an
+        // index that came back empty.
+        perKeyIndexVsScan: await (async () => {
+          const rows = [];
+          for (let u = 0; u < 10; u++) {
+            const idx = (await coll.find({ k: u }).toArray()).length;
+            const scan = dbDocs.filter((d) => d.k === u).length;
+            rows.push({ k: u, index: idx, scan });
+          }
+          return rows;
+        })(),
+        indexes: dbIndexes.map((i) => i.name),
+        totalDocs: dbDocs.length,
+        serverLog: serverLog.slice(-2000),
+        got,
+        want,
       });
     }
   }

tests/spec/README.md

@@ -41,29 +41,43 @@ result is a real pass:
 
 Supported: `client`/`database`/`collection` entities, `initialData`,
 `outcome`, `expectError` (code, codeName, contains, labels, errorResponse),
-`saveResultAsEntity`, `runOnRequirements` gating, and the `$$type`,
+`expectEvents`, `saveResultAsEntity`, `runOnRequirements` gating, and the
-`$$exists`, `$$unsetOrMatches`, `$$matchesEntity`, `$$matchesHexBytes`
+`$$type`, `$$exists`, `$$unsetOrMatches`, `$$matchesEntity`,
-operators.
+`$$matchesHexBytes` operators.
 
-**Not asserted yet: `expectEvents`** (command monitoring). Those assertions are
+`expectEvents` compares the commands the driver actually sent against the
-about the command shape the driver emits rather than result semantics. Ignoring
+expectation, **exact in number and in order**, with `command` and `reply`
-them lets some cases pass that a complete runner would fail, so **treat `pass`
+matched as root documents so the driver's own additions (`lsid`, `$db`) are
-as an upper bound** until M1 wires events up. This is stated again at the top of
+allowed. It is what makes a pass mean the engine answered correctly *and* was
-`scorecard.txt` so the number is never read out of context.
+asked the right question — 354 of the 487 cases declare events, and before this
+was asserted a case could send the wrong command and still be counted a pass.
+**Scorecards recorded before it landed are not comparable**; there, `pass` was
+an upper bound by construction.
+
+Still unasserted within events, each reported as SKIP at the point of
+assertion: `cmap` and `sdam` event types, `ignoreExtraEvents`, `hasServiceId`,
+`hasServerConnectionId`, and `maxTimeMS` in an expected command — the runner
+puts CSOT `timeoutMS` on every client, and CSOT overwrites `maxTimeMS` with
+what is left of that budget, so the value on the wire is the harness's. That is
+the only assertion this runner declines to make; one case in the corpus is
+affected.
 
 Not supported, each reported as SKIP with a reason and never as PASS: session
 and bucket entities (M4 / GridFS), `failPoint`, client-side encryption,
 `testRunner` operations, and any operation or matcher the runner does not know.
 
+`MFDB_DUMP_EVENTS=1` prints each case's observed command stream, which is the
+fastest way to tell a wrong answer from a command the driver never sent.
+
 ## Reading the scorecard
 
 `scorecard.txt` records the totals, a per-file breakdown, and every
 non-passing case with its reason. The distinction that matters:
 
 - **FAIL** — the engine answered, and answered differently from the spec. Real
-  work. An operation that never answered inside `--op-timeout-ms` (default 3 s,
+  work. An operation that never answered inside `--op-timeout-ms` (default
-  enforced by the driver itself via CSOT `timeoutMS`) is also a FAIL, because
+  10 s, enforced by the driver itself via CSOT `timeoutMS`) is also a FAIL,
-  "no answer" is a result. There is a second, much longer `--case-timeout-ms`
+  because "no answer" is a result. There is a second, much longer `--case-timeout-ms`
   backstop for a hang the driver cannot see; if it ever fires, treat the run
   with suspicion — see the trap below.
 - **SKIP** — nobody claims anything. Either the suite needs a feature whose

tests/spec/run.js

@@ -31,8 +31,9 @@ const SUITE_DIR = path.join(__dirname, 'specifications', 'source', 'crud', 'test
 const SCORECARD = path.join(__dirname, 'scorecard.txt');
 const REPO = path.join(__dirname, '..', '..');
 
-// The runner implements schema 1.0-1.9 features that CRUD tests actually use.
+// The runner implements the schema features that CRUD tests actually use, up
-// A file declaring more than this is skipped whole rather than half-run.
+// to this version. A file declaring more than this is skipped whole rather
+// than half-run.
 const MAX_SCHEMA = [1, 24];
 
 const argv = process.argv.slice(2);
@@ -233,7 +234,7 @@ function scalarEqual(expected, actual) {
 // one: only a root document may carry keys the expectation does not mention.
 function match(expected, actual, entities, pathStr = '', root = true) {
     const sk = specialKey(expected);
-    if (sk) return special(sk, expected[sk], actual, entities, pathStr, true);
+    if (sk) return special(sk, expected[sk], actual, entities, pathStr, true, root);
 
     if (isPlainDoc(expected)) {
         if (!isPlainDoc(actual)) fail(pathStr, `expected a document, got ${describe(actual)}`);
@@ -241,7 +242,7 @@ function match(expected, actual, entities, pathStr = '', root = true) {
             const kp = pathStr ? `${pathStr}.${k}` : k;
             const vsk = specialKey(v);
             if (vsk) {
-                const consumed = special(vsk, v[vsk], actual[k], entities, kp, Object.prototype.hasOwnProperty.call(actual, k));
+                const consumed = special(vsk, v[vsk], actual[k], entities, kp, Object.prototype.hasOwnProperty.call(actual, k), false);
                 if (!consumed) continue;
                 continue;
             }
@@ -268,7 +269,11 @@ function match(expected, actual, entities, pathStr = '', root = true) {
 }
 
 // Returns whether the actual value still has to be matched by the caller.
-function special(op, arg, actual, entities, pathStr, present) {
+// `root` is the root-ness of the value the operator stands in for: these
+// operators wrap a value, they do not reposition it. A `$$unsetOrMatches` at
+// the top of an `expectResult` still matches a root document, so the actual
+// document may carry fields the expectation does not mention.
+function special(op, arg, actual, entities, pathStr, present, root) {
     switch (op) {
         case '$$exists':
             if (arg && !present) fail(pathStr, 'expected the key to exist');
@@ -283,11 +288,11 @@ function special(op, arg, actual, entities, pathStr, present) {
         }
         case '$$unsetOrMatches':
             if (!present || actual === undefined) return false;
-            match(arg, actual, entities, pathStr, false);
+            match(arg, actual, entities, pathStr, root);
             return false;
         case '$$matchesEntity': {
             if (!(arg in entities.map)) fail(pathStr, `entity ${arg} not found`);
-            match(entities.map[arg], actual, entities, pathStr, false);
+            match(entities.map[arg], actual, entities, pathStr, root);
             return false;
         }
         case '$$matchesHexBytes':
@@ -314,11 +319,38 @@ class Unsupported extends Error {}
 // Argument keys the spec passes positionally rather than as driver options.
 const POSITIONAL = new Set(['filter', 'document', 'documents', 'update', 'replacement', 'pipeline', 'fieldName', 'models', 'requests', 'keys', 'name', 'indexes', 'command', 'session', 'entity', 'to']);
 
+// Driver options the driver only honours as a JavaScript number. The suites are
+// parsed with `EJSON.parse(text, {relaxed: false})` so that `$numberLong` and
+// friends keep their exact BSON type in *data* -- but that also turns a plain
+// JSON `2` in an *option* into a BSON Int32 object, and the driver gates every
+// one of these on `typeof options.skip === 'number'`
+// (node_modules/mongodb/lib/operations/find.js:68-95). A BSON wrapper therefore
+// failed the check and the option was dropped on the floor: `skip`, `limit` and
+// `batchSize` never reached the wire at all, and three find.json cases failed
+// with the *unclipped* match count while the engine was applying both correctly.
+// Read as an engine bug for a whole milestone. Coerce by name, not by shape:
+// unwrapping every numeric-looking value would rewrite the wire type of the
+// `comment` and `hint` values that other suites assert on.
+const NUMERIC_OPTIONS = new Set([
+    'skip',
+    'limit',
+    'batchSize',
+    'maxTimeMS',
+    'maxAwaitTimeMS',
+    'expireAfterSeconds',
+]);
+
+function numeric_option(v) {
+    if (v === null || typeof v !== 'object' || typeof v.valueOf !== 'function') return v;
+    const n = v.valueOf();
+    return typeof n === 'number' ? n : v;
+}
+
 function options(args, drop = []) {
     const o = {};
     for (const [k, v] of Object.entries(args || {})) {
         if (POSITIONAL.has(k) || drop.includes(k)) continue;
-        o[k] = v;
+        o[k] = NUMERIC_OPTIONS.has(k) ? numeric_option(v) : v;
     }
     return Object.keys(o).length ? o : undefined;
 }
@@ -459,9 +491,88 @@ async function seedInitialData(initialData) {
     }
 }
 
+// ---------------------------------------------------------------------------
+// Command monitoring
+// ---------------------------------------------------------------------------
+
+// The three event types a `client` entity can ask to observe that this runner
+// can produce, mapped to the driver's own event names. `cmap` and `sdam` types
+// are simply not collected; a test that goes on to *assert* them is reported
+// unsupported at that point rather than here, so declaring an observation this
+// runner ignores costs a case nothing.
+const COMMAND_EVENTS = {
+    commandStartedEvent: 'commandStarted',
+    commandSucceededEvent: 'commandSucceeded',
+    commandFailedEvent: 'commandFailed',
+};
+
+// Sensitive commands, per the command-logging-and-monitoring spec's Security
+// section. Events for these are dropped unless the entity sets
+// `observeSensitiveCommands` (unified-test-format.md:3070-3075). None of them
+// can be issued by this engine yet -- there is no auth and no user management
+// before M7 -- so this is here to keep the rule where the rule belongs rather
+// than to filter anything today.
+const SENSITIVE_COMMANDS = new Set([
+    'authenticate', 'saslstart', 'saslcontinue', 'getnonce', 'createuser',
+    'updateuser', 'copydbgetnonce', 'copydbsaslstart', 'copydb',
+]);
+
+function isSensitive(ev) {
+    const name = String(ev.commandName || '').toLowerCase();
+    if (SENSITIVE_COMMANDS.has(name)) return true;
+    // `hello` and legacy hello are sensitive only when they carry
+    // `speculativeAuthenticate`, which the driver does not report either way --
+    // it redacts both the command and the reply to an empty document, and the
+    // spec says to infer sensitivity from exactly that.
+    if (name === 'hello' || name === 'ismaster') {
+        const body = ev.command || ev.reply;
+        return !!body && Object.keys(body).length === 0;
+    }
+    return false;
+}
+
+// Listeners are disabled rather than removed, and disabled before the outcome
+// check rather than after it (unified-test-format.md:3081): the teardown that
+// follows a case issues commands of its own, and a buffer that kept growing
+// through it would make the assertion a function of the harness.
+function disableEvents(events) {
+    if (process.env.MFDB_DUMP_EVENTS) {
+        for (const [id, buf] of events) console.log(`      events ${id}: ${buf.map((e) => `${e.kind}:${e.ev.commandName}`).join(', ')}`);
+    }
+    for (const buf of events.values()) buf.enabled = false;
+}
+
+// `collectionOrDatabaseOptions` (unified-test-format.md, entity definitions).
+// Anything outside this set is refused rather than silently ignored -- that is
+// the whole lesson of the bug this function exists to fix.
+const ENTITY_OPTIONS = new Set(['readConcern', 'readPreference', 'writeConcern']);
+
+// These documents are parsed with `relaxed: false` like everything else in the
+// file, so a plain JSON `0` arrives as a BSON Int32 -- and the driver gates
+// `writeConcern.w` on `typeof w === 'number'`, which is exactly the trap
+// NUMERIC_OPTIONS documents for operation options. Unwrapping wholesale is safe
+// here in a way it is not there: these are settings the driver consumes, not
+// values any assertion ever compares.
+function plainOptions(spec, what) {
+    for (const k of Object.keys(spec || {})) {
+        if (!ENTITY_OPTIONS.has(k)) throw new Unsupported(`${what} ${k}`);
+    }
+    const walk = (v) => {
+        if (Array.isArray(v)) return v.map(walk);
+        if (typeof v === 'object' && v !== null && numeric(v) !== null) return numeric(v);
+        if (!isPlainDoc(v)) return v;
+        const out = {};
+        for (const [k, x] of Object.entries(v)) out[k] = walk(x);
+        return out;
+    };
+    return walk(spec || {});
+}
+
 // `clients` is supplied by the caller so that entities created before a
 // failure are still closed: returning them only on success is what leaked.
-async function buildEntities(url, createEntities, clients) {
+// `events` is supplied for the same reason -- a case that dies partway still
+// has to be able to turn its listeners off.
+async function buildEntities(url, createEntities, clients, events) {
     const map = {};
     for (const spec of createEntities || []) {
         const [kind, def] = Object.entries(spec)[0];
@@ -475,10 +586,12 @@ async function buildEntities(url, createEntities, clients) {
                 // buildEntities -- can create further clients *after* cleanup
                 // has already run. That is what leaked, and what turned into
                 // 190 phantom timeout FAILs.
+                const observed = (def.observeEvents || []).filter((e) => e in COMMAND_EVENTS);
                 const c = new MongoClient(url, Object.assign({
                     serverSelectionTimeoutMS: 2000,
                     connectTimeoutMS: 2000,
                     timeoutMS: OP_TIMEOUT_MS,
+                    monitorCommands: observed.length > 0,
                 }, def.uriOptions || {}));
                 // Registered before connect, so a client whose connect throws
                 // or is abandoned is still closed by the caller.
@@ -489,18 +602,128 @@ async function buildEntities(url, createEntities, clients) {
                     await c.close().catch(() => { });
                     throw new Error('case abandoned');
                 }
+                // Subscribed after connect, so the handshake this client just
+                // performed is not in its own buffer.
+                if (observed.length) {
+                    const ignore = new Set((def.ignoreCommandMonitoringEvents || []).map((s) => String(s).toLowerCase()));
+                    const buf = [];
+                    buf.enabled = true;
+                    events.set(def.id, buf);
+                    for (const name of observed) {
+                        const kind = name.replace(/Event$/, '');
+                        c.on(COMMAND_EVENTS[name], (ev) => {
+                            if (!buf.enabled) return;
+                            if (ignore.has(String(ev.commandName).toLowerCase())) return;
+                            if (!def.observeSensitiveCommands && isSensitive(ev)) return;
+                            buf.push({ kind, ev });
+                        });
+                    }
+                }
                 map[def.id] = c;
                 break;
             }
-            case 'database': map[def.id] = map[def.client].db(def.databaseName); break;
+            case 'database':
-            case 'collection': map[def.id] = map[def.database].collection(def.collectionName); break;
+                map[def.id] = map[def.client].db(def.databaseName, plainOptions(def.databaseOptions, 'databaseOptions'));
+                break;
+            case 'collection':
+                map[def.id] = map[def.database].collection(def.collectionName, plainOptions(def.collectionOptions, 'collectionOptions'));
+                break;
             case 'session': throw new Unsupported('session entities (M4)');
             case 'bucket': throw new Unsupported('gridfs bucket entities');
             case 'clientEncryption': throw new Unsupported('client-side encryption');
             default: throw new Unsupported(`entity type ${kind}`);
         }
     }
-    return { map, clients };
+    return { map, clients, events };
+}
+
+// Two rules decide how much of the corpus this assertion can reach, and both
+// come from the spec rather than from taste:
+//
+//   - `command` and `reply` are matched as *root* documents
+//     (unified-test-format.md:1020-1022 and :1037-1039). The driver puts
+//     `lsid`, `$db` and `$readPreference` on nearly everything it sends;
+//     matched as nested documents, almost every case in the corpus would fail
+//     on keys its expectation was never written to mention.
+//   - the event list is exact in number and order, not a prefix
+//     (unified-test-format.md:3088-3091). 23 cases here expect an empty list,
+//     and a prefix rule would pass every one of them without looking.
+function verifyEvents(expectEvents, entities) {
+    if (!expectEvents) return;
+    for (const spec of expectEvents) {
+        const type = spec.eventType || 'command';
+        if (type !== 'command') throw new Unsupported(`${type} events`);
+        if (spec.ignoreExtraEvents) throw new Unsupported('ignoreExtraEvents');
+        const buf = entities.events.get(spec.client);
+        // Not treated as an empty list: a client that never subscribed and a
+        // client that saw nothing are the same thing to a comparison and very
+        // different things to a runner, and the second one is a runner bug
+        // that would quietly pass the 23 empty-list cases.
+        if (!buf) fail(`events ${spec.client}`, 'the client entity is not observing command events');
+        const expected = spec.events || [];
+        if (expected.length !== buf.length) {
+            fail(`events ${spec.client}`, `expected ${expected.length} events, observed ${buf.length}` +
+                ` [${buf.map((e) => `${e.kind}:${e.ev.commandName}`).join(', ')}]`);
+        }
+        expected.forEach((e, i) => matchEvent(e, buf[i], entities, `events ${spec.client}[${i}]`));
+    }
+}
+
+// A monitored event hands over the command as the driver holds it in memory,
+// which is not always the shape it puts on the wire: a sort is a JS `Map`
+// (driver lib/sort.js). `Object.keys` on a Map is empty, so the matcher
+// reported every key of an expected sort as missing from a command that in
+// fact carried it. EJSON serializes a Map exactly like a document, which is
+// why a dump of the event looks perfectly correct and this had to be measured
+// rather than read. Converted only for the comparison, and at every depth,
+// since a sort also appears inside `updates[i]`.
+function wireShape(v) {
+    if (v instanceof Map) {
+        const out = {};
+        for (const [k, x] of v) out[k] = wireShape(x);
+        return out;
+    }
+    if (Array.isArray(v)) return v.map(wireShape);
+    if (!isPlainDoc(v)) return v;
+    const out = {};
+    for (const [k, x] of Object.entries(v)) out[k] = wireShape(x);
+    return out;
+}
+
+function matchEvent(expected, actual, entities, pathStr) {
+    const [name, body] = Object.entries(expected)[0];
+    if (!(name in COMMAND_EVENTS)) throw new Unsupported(`event ${name}`);
+    const kind = name.replace(/Event$/, '');
+    if (actual.kind !== kind) fail(pathStr, `expected ${kind}, got ${actual.kind} of ${actual.ev.commandName}`);
+    for (const [k, v] of Object.entries(body || {})) {
+        switch (k) {
+            case 'command':
+                // The one assertion this runner cannot make, and the only
+                // escape hatch in it. Every client entity carries CSOT
+                // `timeoutMS` (see OP_TIMEOUT_MS), and CSOT overwrites
+                // `maxTimeMS` on every command with what is left of that
+                // budget -- so the value on the wire is the harness's, not the
+                // test's. Dropping `timeoutMS` is not the alternative: it is
+                // what replaced the outer race that produced ~190 phantom
+                // timeout FAILs, and it would cost far more than one case.
+                // Refused unconditionally rather than only when the values
+                // differ, so this can never turn into a pass by coincidence.
+                if (isPlainDoc(v) && Object.prototype.hasOwnProperty.call(v, 'maxTimeMS')) {
+                    throw new Unsupported('maxTimeMS in an expected command (CSOT rewrites it)');
+                }
+                match(v, wireShape(actual.ev[k]), entities, `${pathStr}.${k}`, true);
+                break;
+            case 'reply':
+                match(v, wireShape(actual.ev[k]), entities, `${pathStr}.${k}`, true);
+                break;
+            case 'commandName':
+            case 'databaseName':
+                match(v, actual.ev[k], entities, `${pathStr}.${k}`, false);
+                break;
+            default:
+                throw new Unsupported(`event assertion ${name}.${k}`);
+        }
+    }
 }
 
 async function verifyOutcome(outcome, entities) {
@@ -539,11 +762,14 @@ async function runFile(file, url, server) {
         // Owned out here, not by buildEntities, so a case that dies partway
         // still has every client it managed to open closed below.
         const clients = [];
+        const events = new Map();
         try {
             await withTimeout(async () => {
                 await seedInitialData(doc.initialData);
-                const entities = await buildEntities(url, doc.createEntities, clients);
+                const entities = await buildEntities(url, doc.createEntities, clients, events);
                 for (const op of test.operations) await runOne(op, entities);
+                disableEvents(events);
+                verifyEvents(test.expectEvents, entities);
                 await verifyOutcome(test.outcome, entities);
             }, CASE_TIMEOUT_MS, `case exceeded ${CASE_TIMEOUT_MS} ms`);
             out.pass++;
@@ -556,6 +782,7 @@ async function runFile(file, url, server) {
             // its continuation may still be running and about to open another
             // client, which buildEntities closes itself on seeing this.
             clients.abandoned = true;
+            disableEvents(events);
             for (const c of clients) await c.close().catch(() => { });
         }
     }
@@ -722,10 +949,15 @@ function scorecardText(results, tot, errored, server, nfiles) {
     L.push('#   - the suite needs a feature whose milestone has not landed (sessions M4,');
     L.push('#     auth M7, failPoints, gridfs) -- counted as skip, never as pass;');
     L.push('#   - or the runner itself does not implement the operation/matcher yet.');
-    L.push('# Deliberately NOT asserted yet: expectEvents (command monitoring). Those');
+    L.push('# expectEvents IS asserted: the commands the driver sent are compared to the');
-    L.push('# assertions are about driver-visible command shape rather than result');
+    L.push('# expectation exactly, in number and in order, with `command` and `reply`');
-    L.push('# semantics; ignoring them makes some cases pass that a full runner would');
+    L.push('# matched as root documents. A pass therefore means the engine answered');
-    L.push('# fail, so treat `pass` as an upper bound until M1 wires events up.');
+    L.push('# correctly *and* was asked the right question. Not comparable to any');
+    L.push('# scorecard recorded before that landed, where `pass` was an upper bound.');
+    L.push('# Still unasserted, each reported as skip where it is asserted, never as');
+    L.push('# pass: cmap and sdam events, ignoreExtraEvents, hasServiceId,');
+    L.push('# hasServerConnectionId, and maxTimeMS in an expected command (CSOT rewrites');
+    L.push('# it -- the only assertion this runner declines to make).');
     L.push('');
     L.push(`total\t${tot.pass} pass\t${tot.fail} fail\t${tot.skip} skip\t${nfiles} files\t${errored} errored`);
     L.push('');

tests/spec/scorecard.txt

@@ -1,19 +1,24 @@
 # MongoDB spec-test scorecard (PLAN D2) -- crud + aggregate, unified format
 # specs: mongodb/specifications @ 615e0f9 (pinned in tests/spec/fetch.sh)
 # driver: mongodb@7.5.0 (pinned in tests/e2e/package-lock.json)
-# server: MultiforaDB reporting version 4.4.0, maxWireVersion 8
+# server: MultiforaDB reporting version 4.4.0, maxWireVersion 9
 # reproduce: bash tests/spec/fetch.sh && node tests/spec/run.js --scorecard
 #
 # What SKIP means here, so the totals are not read as better than they are:
 #   - the suite needs a feature whose milestone has not landed (sessions M4,
 #     auth M7, failPoints, gridfs) -- counted as skip, never as pass;
 #   - or the runner itself does not implement the operation/matcher yet.
-# Deliberately NOT asserted yet: expectEvents (command monitoring). Those
+# expectEvents IS asserted: the commands the driver sent are compared to the
-# assertions are about driver-visible command shape rather than result
+# expectation exactly, in number and in order, with `command` and `reply`
-# semantics; ignoring them makes some cases pass that a full runner would
+# matched as root documents. A pass therefore means the engine answered
-# fail, so treat `pass` as an upper bound until M1 wires events up.
+# correctly *and* was asked the right question. Not comparable to any
+# scorecard recorded before that landed, where `pass` was an upper bound.
+# Still unasserted, each reported as skip where it is asserted, never as
+# pass: cmap and sdam events, ignoreExtraEvents, hasServiceId,
+# hasServerConnectionId, and maxTimeMS in an expected command (CSOT rewrites
+# it -- the only assertion this runner declines to make).
 
-total	163 pass	129 fail	195 skip	175 files	0 errored
+total	194 pass	97 fail	196 skip	175 files	0 errored
 
 # per-file: name	pass	fail	skip
 aggregate-allowdiskuse.json	3	0	0
@@ -31,34 +36,34 @@ bulkWrite-collation.json	0	2	0
 bulkWrite-comment.json	2	0	1
 bulkWrite-delete-hint-serverError.json	0	0	2
 bulkWrite-delete-hint.json	2	0	0
-bulkWrite-deleteMany-hint-unacknowledged.json	0	2	2
+bulkWrite-deleteMany-hint-unacknowledged.json	2	0	2
 bulkWrite-deleteMany-let.json	0	1	1
 bulkWrite-deleteMany-rawdata.json	1	0	1
-bulkWrite-deleteOne-hint-unacknowledged.json	0	2	2
+bulkWrite-deleteOne-hint-unacknowledged.json	2	0	2
 bulkWrite-deleteOne-let.json	0	1	1
 bulkWrite-deleteOne-rawdata.json	1	0	1
 bulkWrite-errorResponse.json	0	0	1
 bulkWrite-insertOne-dots_and_dollars.json	3	1	1
 bulkWrite-replaceOne-dots_and_dollars.json	2	1	1
-bulkWrite-replaceOne-hint-unacknowledged.json	0	2	0
+bulkWrite-replaceOne-hint-unacknowledged.json	2	0	0
 bulkWrite-replaceOne-let.json	0	1	1
 bulkWrite-replaceOne-rawdata.json	1	0	1
 bulkWrite-replaceOne-sort.json	1	0	1
 bulkWrite-update-hint.json	3	0	0
 bulkWrite-update-validation.json	3	0	0
 bulkWrite-updateMany-dots_and_dollars.json	0	0	4
-bulkWrite-updateMany-hint-unacknowledged.json	0	2	0
+bulkWrite-updateMany-hint-unacknowledged.json	2	0	0
 bulkWrite-updateMany-let.json	0	1	1
 bulkWrite-updateMany-pipeline.json	0	1	0
 bulkWrite-updateMany-rawdata.json	0	1	1
 bulkWrite-updateOne-dots_and_dollars.json	0	0	4
-bulkWrite-updateOne-hint-unacknowledged.json	0	2	0
+bulkWrite-updateOne-hint-unacknowledged.json	2	0	0
 bulkWrite-updateOne-let.json	0	1	1
 bulkWrite-updateOne-pipeline.json	0	1	0
 bulkWrite-updateOne-rawdata.json	0	1	1
 bulkWrite-updateOne-sort.json	1	0	1
 bulkWrite.json	10	0	0
-bypassDocumentValidation.json	8	1	0
+bypassDocumentValidation.json	4	5	0
 client-bulkWrite-delete-options.json	0	0	2
 client-bulkWrite-delete-rawdata.json	0	0	2
 client-bulkWrite-errorResponse.json	0	0	1
@@ -78,7 +83,7 @@ client-bulkWrite-updateOne-sort.json	0	0	1
 count-collation.json	1	0	1
 count-empty.json	2	0	1
 count-rawdata.json	0	0	2
-count.json	3	1	3
+count.json	4	0	3
 countDocuments-comment.json	2	0	1
 countDocuments-rawdata.json	1	0	1
 create-null-ids.json	0	6	1
@@ -88,7 +93,7 @@ db-aggregate.json	0	2	0
 deleteMany-collation.json	0	1	0
 deleteMany-comment.json	2	0	1
 deleteMany-hint-serverError.json	0	0	2
-deleteMany-hint-unacknowledged.json	0	2	2
+deleteMany-hint-unacknowledged.json	2	0	2
 deleteMany-hint.json	2	0	0
 deleteMany-let.json	0	1	1
 deleteMany-rawdata.json	1	0	1
@@ -97,7 +102,7 @@ deleteOne-collation.json	0	1	0
 deleteOne-comment.json	2	0	1
 deleteOne-errorResponse.json	0	0	1
 deleteOne-hint-serverError.json	0	0	2
-deleteOne-hint-unacknowledged.json	0	2	2
+deleteOne-hint-unacknowledged.json	2	0	2
 deleteOne-hint.json	2	0	0
 deleteOne-let.json	0	1	1
 deleteOne-rawdata.json	1	0	1
@@ -109,19 +114,19 @@ distinct-rawdata.json	0	1	1
 distinct.json	0	2	0
 estimatedDocumentCount-comment.json	1	1	1
 estimatedDocumentCount-rawdata.json	1	0	1
-estimatedDocumentCount.json	3	1	2
+estimatedDocumentCount.json	2	1	3
 find-allowdiskuse-serverError.json	0	0	2
 find-allowdiskuse.json	3	0	0
 find-collation.json	0	1	0
 find-comment.json	1	2	2
 find-let.json	0	1	1
 find-rawdata.json	1	0	1
-find.json	2	3	0
+find.json	5	0	0
-findOne.json	1	1	0
+findOne.json	2	0	0
 findOneAndDelete-collation.json	0	1	0
 findOneAndDelete-comment.json	2	0	1
 findOneAndDelete-hint-serverError.json	0	0	2
-findOneAndDelete-hint-unacknowledged.json	0	2	2
+findOneAndDelete-hint-unacknowledged.json	2	0	2
 findOneAndDelete-hint.json	2	0	0
 findOneAndDelete-let.json	0	1	1
 findOneAndDelete-rawdata.json	1	0	1
@@ -130,7 +135,7 @@ findOneAndReplace-collation.json	0	1	0
 findOneAndReplace-comment.json	2	0	1
 findOneAndReplace-dots_and_dollars.json	2	1	1
 findOneAndReplace-hint-serverError.json	0	0	2
-findOneAndReplace-hint-unacknowledged.json	0	2	2
+findOneAndReplace-hint-unacknowledged.json	2	0	2
 findOneAndReplace-hint.json	2	0	0
 findOneAndReplace-let.json	0	1	1
 findOneAndReplace-rawdata.json	1	0	1
@@ -142,25 +147,25 @@ findOneAndUpdate-comment.json	0	2	1
 findOneAndUpdate-dots_and_dollars.json	0	0	4
 findOneAndUpdate-errorResponse.json	0	1	1
 findOneAndUpdate-hint-serverError.json	0	0	2
-findOneAndUpdate-hint-unacknowledged.json	0	2	2
+findOneAndUpdate-hint-unacknowledged.json	2	0	2
 findOneAndUpdate-hint.json	2	0	0
 findOneAndUpdate-let.json	0	1	1
 findOneAndUpdate-pipeline.json	0	1	0
 findOneAndUpdate-rawdata.json	0	1	1
 findOneAndUpdate.json	5	3	0
 insertMany-comment.json	2	0	1
-insertMany-dots_and_dollars.json	0	4	1
+insertMany-dots_and_dollars.json	3	1	1
 insertMany-rawdata.json	1	0	1
-insertMany.json	2	1	0
+insertMany.json	3	0	0
 insertOne-comment.json	2	0	1
-insertOne-dots_and_dollars.json	5	3	1
+insertOne-dots_and_dollars.json	6	2	1
 insertOne-errorResponse.json	0	0	1
 insertOne-rawdata.json	1	0	1
 insertOne.json	1	0	0
 replaceOne-collation.json	0	1	0
 replaceOne-comment.json	2	0	1
 replaceOne-dots_and_dollars.json	3	1	1
-replaceOne-hint-unacknowledged.json	0	2	0
+replaceOne-hint-unacknowledged.json	2	0	0
 replaceOne-hint.json	2	0	0
 replaceOne-let.json	0	1	1
 replaceOne-rawdata.json	1	0	1
@@ -171,7 +176,7 @@ updateMany-arrayFilters.json	0	3	0
 updateMany-collation.json	0	1	0
 updateMany-comment.json	2	0	1
 updateMany-dots_and_dollars.json	0	0	4
-updateMany-hint-unacknowledged.json	0	2	0
+updateMany-hint-unacknowledged.json	2	0	0
 updateMany-hint.json	2	0	0
 updateMany-let.json	0	1	1
 updateMany-pipeline.json	0	1	0
@@ -183,7 +188,7 @@ updateOne-collation.json	0	1	0
 updateOne-comment.json	2	0	1
 updateOne-dots_and_dollars.json	0	0	4
 updateOne-errorResponse.json	0	0	1
-updateOne-hint-unacknowledged.json	0	2	0
+updateOne-hint-unacknowledged.json	2	0	0
 updateOne-hint.json	2	0	0
 updateOne-let.json	0	1	1
 updateOne-pipeline.json	0	1	0
@@ -220,15 +225,11 @@ bulkWrite-comment.json	SKIP	BulkWrite with comment - pre 4.4	needs server <= 4.2
 bulkWrite-delete-hint-serverError.json	SKIP	*	needs server <= 4.3.3
 bulkWrite-deleteMany-hint-unacknowledged.json	SKIP	Unacknowledged deleteMany with hint string fails with client-side error on pre-4.4 server	needs server <= 4.2.99
 bulkWrite-deleteMany-hint-unacknowledged.json	SKIP	Unacknowledged deleteMany with hint document fails with client-side error on pre-4.4 server	needs server <= 4.2.99
-bulkWrite-deleteMany-hint-unacknowledged.json	FAIL	Unacknowledged deleteMany with hint string on 4.4+ server	bulkWrite: unexpected extra keys ["insertedCount","matchedCount","modifiedCount","deletedCount","upsertedCount","upsertedIds","insertedIds"]
-bulkWrite-deleteMany-hint-unacknowledged.json	FAIL	Unacknowledged deleteMany with hint document on 4.4+ server	bulkWrite: unexpected extra keys ["insertedCount","matchedCount","modifiedCount","deletedCount","upsertedCount","upsertedIds","insertedIds"]
 bulkWrite-deleteMany-let.json	SKIP	BulkWrite deleteMany with let option	needs server >= 5.0
 bulkWrite-deleteMany-let.json	FAIL	BulkWrite deleteMany with let option unsupported (server-side error)	bulkWrite: expected an error, the operation succeeded
 bulkWrite-deleteMany-rawdata.json	SKIP	BulkWrite deleteMany with rawData option	needs server >= 8.2.0
 bulkWrite-deleteOne-hint-unacknowledged.json	SKIP	Unacknowledged deleteOne with hint string fails with client-side error on pre-4.4 server	needs server <= 4.2.99
 bulkWrite-deleteOne-hint-unacknowledged.json	SKIP	Unacknowledged deleteOne with hint document fails with client-side error on pre-4.4 server	needs server <= 4.2.99
-bulkWrite-deleteOne-hint-unacknowledged.json	FAIL	Unacknowledged deleteOne with hint string on 4.4+ server	bulkWrite: unexpected extra keys ["insertedCount","matchedCount","modifiedCount","deletedCount","upsertedCount","upsertedIds","insertedIds"]
-bulkWrite-deleteOne-hint-unacknowledged.json	FAIL	Unacknowledged deleteOne with hint document on 4.4+ server	bulkWrite: unexpected extra keys ["insertedCount","matchedCount","modifiedCount","deletedCount","upsertedCount","upsertedIds","insertedIds"]
 bulkWrite-deleteOne-let.json	SKIP	BulkWrite deleteOne with let option	needs server >= 5.0
 bulkWrite-deleteOne-let.json	FAIL	BulkWrite deleteOne with let option unsupported (server-side error)	bulkWrite: expected an error, the operation succeeded
 bulkWrite-deleteOne-rawdata.json	SKIP	BulkWrite deleteOne with rawData option	needs server >= 8.2.0
@@ -237,8 +238,6 @@ bulkWrite-insertOne-dots_and_dollars.json	SKIP	Inserting document with top-level
 bulkWrite-insertOne-dots_and_dollars.json	FAIL	Inserting document with top-level dollar-prefixed key on pre-5.0 server yields server-side error	bulkWrite: expected an error, the operation succeeded
 bulkWrite-replaceOne-dots_and_dollars.json	SKIP	Replacing document with dollar-prefixed key in embedded doc on 5.0+ server	needs server >= 5.0
 bulkWrite-replaceOne-dots_and_dollars.json	FAIL	Replacing document with dollar-prefixed key in embedded doc on pre-5.0 server yields server-side error	bulkWrite: expected an error, the operation succeeded
-bulkWrite-replaceOne-hint-unacknowledged.json	FAIL	Unacknowledged replaceOne with hint string on 4.2+ server	bulkWrite: unexpected extra keys ["insertedCount","matchedCount","modifiedCount","deletedCount","upsertedCount","upsertedIds","insertedIds"]
-bulkWrite-replaceOne-hint-unacknowledged.json	FAIL	Unacknowledged replaceOne with hint document on 4.2+ server	bulkWrite: unexpected extra keys ["insertedCount","matchedCount","modifiedCount","deletedCount","upsertedCount","upsertedIds","insertedIds"]
 bulkWrite-replaceOne-let.json	SKIP	BulkWrite replaceOne with let option	needs server >= 5.0
 bulkWrite-replaceOne-let.json	FAIL	BulkWrite replaceOne with let option unsupported (server-side error)	bulkWrite: expected an error, the operation succeeded
 bulkWrite-replaceOne-rawdata.json	SKIP	BulkWrite replaceOne with rawData option	needs server >= 8.2.0
@@ -247,8 +246,6 @@ bulkWrite-updateMany-dots_and_dollars.json	SKIP	Updating document to set top-lev
 bulkWrite-updateMany-dots_and_dollars.json	SKIP	Updating document to set top-level dotted key on 5.0+ server	needs server >= 5.0
 bulkWrite-updateMany-dots_and_dollars.json	SKIP	Updating document to set dollar-prefixed key in embedded doc on 5.0+ server	needs server >= 5.0
 bulkWrite-updateMany-dots_and_dollars.json	SKIP	Updating document to set dotted key in embedded doc on 5.0+ server	needs server >= 5.0
-bulkWrite-updateMany-hint-unacknowledged.json	FAIL	Unacknowledged updateMany with hint string on 4.2+ server	bulkWrite: unexpected extra keys ["insertedCount","matchedCount","modifiedCount","deletedCount","upsertedCount","upsertedIds","insertedIds"]
-bulkWrite-updateMany-hint-unacknowledged.json	FAIL	Unacknowledged updateMany with hint document on 4.2+ server	bulkWrite: unexpected extra keys ["insertedCount","matchedCount","modifiedCount","deletedCount","upsertedCount","upsertedIds","insertedIds"]
 bulkWrite-updateMany-let.json	SKIP	BulkWrite updateMany with let option	needs server >= 5.0
 bulkWrite-updateMany-let.json	FAIL	BulkWrite updateMany with let option unsupported (server-side error)	bulkWrite: error message "update spec requires u" does not contain "'update.let' is an unknown field"
 bulkWrite-updateMany-pipeline.json	FAIL	UpdateMany in bulk write using pipelines	MongoBulkWriteError: update spec requires u
@@ -258,8 +255,6 @@ bulkWrite-updateOne-dots_and_dollars.json	SKIP	Updating document to set top-leve
 bulkWrite-updateOne-dots_and_dollars.json	SKIP	Updating document to set top-level dotted key on 5.0+ server	needs server >= 5.0
 bulkWrite-updateOne-dots_and_dollars.json	SKIP	Updating document to set dollar-prefixed key in embedded doc on 5.0+ server	needs server >= 5.0
 bulkWrite-updateOne-dots_and_dollars.json	SKIP	Updating document to set dotted key in embedded doc on 5.0+ server	needs server >= 5.0
-bulkWrite-updateOne-hint-unacknowledged.json	FAIL	Unacknowledged updateOne with hint string on 4.2+ server	bulkWrite: unexpected extra keys ["insertedCount","matchedCount","modifiedCount","deletedCount","upsertedCount","upsertedIds","insertedIds"]
-bulkWrite-updateOne-hint-unacknowledged.json	FAIL	Unacknowledged updateOne with hint document on 4.2+ server	bulkWrite: unexpected extra keys ["insertedCount","matchedCount","modifiedCount","deletedCount","upsertedCount","upsertedIds","insertedIds"]
 bulkWrite-updateOne-let.json	SKIP	BulkWrite updateOne with let option	needs server >= 5.0
 bulkWrite-updateOne-let.json	FAIL	BulkWrite updateOne with let option unsupported (server-side error)	bulkWrite: error message "update spec requires u" does not contain "'update.let' is an unknown field"
 bulkWrite-updateOne-pipeline.json	FAIL	UpdateOne in bulk write using pipelines	MongoBulkWriteError: update spec requires u
@@ -267,6 +262,10 @@ bulkWrite-updateOne-rawdata.json	SKIP	BulkWrite updateOne with rawData option	ne
 bulkWrite-updateOne-rawdata.json	FAIL	BulkWrite updateOne with rawData option on less than 8.2.0 - ignore argument	MongoBulkWriteError: update spec requires u
 bulkWrite-updateOne-sort.json	SKIP	BulkWrite updateOne with sort option	needs server >= 8.0
 bypassDocumentValidation.json	FAIL	Aggregate with $out passes bypassDocumentValidation: false	MongoServerError: Unrecognized pipeline stage name: '$out'
+bypassDocumentValidation.json	FAIL	BulkWrite passes bypassDocumentValidation: false	events client0[0].command.bypassDocumentValidation: missing from actual
+bypassDocumentValidation.json	FAIL	FindOneAndReplace passes bypassDocumentValidation: false	events client0[0].command.bypassDocumentValidation: missing from actual
+bypassDocumentValidation.json	FAIL	FindOneAndUpdate passes bypassDocumentValidation: false	events client0[0].command.bypassDocumentValidation: missing from actual
+bypassDocumentValidation.json	FAIL	InsertMany passes bypassDocumentValidation: false	events client0[0].command.bypassDocumentValidation: missing from actual
 client-bulkWrite-delete-options.json	SKIP	*	needs server >= 8.0
 client-bulkWrite-delete-rawdata.json	SKIP	client bulk write delete with rawData option	needs server >= 8.2.0
 client-bulkWrite-delete-rawdata.json	SKIP	client bulk write delete with rawData option on less than 8.2.0 - ignore argument	needs server >= 8.0
@@ -292,7 +291,6 @@ count-collation.json	SKIP	Deprecated count with collation	runner: operation coun
 count-empty.json	SKIP	Deprecated count with empty collection	runner: operation count
 count-rawdata.json	SKIP	Deprecated count with rawData option	needs server >= 8.2.0
 count-rawdata.json	SKIP	Deprecated count with rawData option on less than 8.2.0 - ignore argument	runner: operation count
-count.json	FAIL	Count documents with skip and limit	countDocuments: expected 2, got 3
 count.json	SKIP	Deprecated count without a filter	runner: operation count
 count.json	SKIP	Deprecated count with a filter	runner: operation count
 count.json	SKIP	Deprecated count with skip and limit	runner: operation count
@@ -315,8 +313,6 @@ deleteMany-comment.json	SKIP	deleteMany with comment - pre 4.4	needs server <= 4
 deleteMany-hint-serverError.json	SKIP	*	needs server <= 4.3.3
 deleteMany-hint-unacknowledged.json	SKIP	Unacknowledged deleteMany with hint string fails with client-side error on pre-4.4 server	needs server <= 4.2.99
 deleteMany-hint-unacknowledged.json	SKIP	Unacknowledged deleteMany with hint document fails with client-side error on pre-4.4 server	needs server <= 4.2.99
-deleteMany-hint-unacknowledged.json	FAIL	Unacknowledged deleteMany with hint string on 4.4+ server	deleteMany: unexpected extra keys ["deletedCount"]
-deleteMany-hint-unacknowledged.json	FAIL	Unacknowledged deleteMany with hint document on 4.4+ server	deleteMany: unexpected extra keys ["deletedCount"]
 deleteMany-let.json	SKIP	deleteMany with let option	needs server >= 5.0
 deleteMany-let.json	FAIL	deleteMany with let option unsupported (server-side error)	deleteMany: expected an error, the operation succeeded
 deleteMany-rawdata.json	SKIP	deleteMany with rawData option	needs server >= 8.2.0
@@ -326,8 +322,6 @@ deleteOne-errorResponse.json	SKIP	delete operations support errorResponse assert
 deleteOne-hint-serverError.json	SKIP	*	needs server <= 4.3.3
 deleteOne-hint-unacknowledged.json	SKIP	Unacknowledged deleteOne with hint string fails with client-side error on pre-4.4 server	needs server <= 4.2.99
 deleteOne-hint-unacknowledged.json	SKIP	Unacknowledged deleteOne with hint document fails with client-side error on pre-4.4 server	needs server <= 4.2.99
-deleteOne-hint-unacknowledged.json	FAIL	Unacknowledged deleteOne with hint string on 4.4+ server	deleteOne: unexpected extra keys ["deletedCount"]
-deleteOne-hint-unacknowledged.json	FAIL	Unacknowledged deleteOne with hint document on 4.4+ server	deleteOne: unexpected extra keys ["deletedCount"]
 deleteOne-let.json	SKIP	deleteOne with let option	needs server >= 5.0
 deleteOne-let.json	FAIL	deleteOne with let option unsupported (server-side error)	deleteOne: expected an error, the operation succeeded
 deleteOne-rawdata.json	SKIP	deleteOne with rawData option	needs server >= 8.2.0
@@ -342,6 +336,7 @@ distinct.json	FAIL	Distinct with a filter	MongoServerError: no such command: 'di
 estimatedDocumentCount-comment.json	SKIP	estimatedDocumentCount with document comment	needs server >= 4.4.14
 estimatedDocumentCount-comment.json	FAIL	estimatedDocumentCount with document comment - pre 4.4.14, server error	estimatedDocumentCount: expected an error, the operation succeeded
 estimatedDocumentCount-rawdata.json	SKIP	Estimated document count with rawData option	needs server >= 8.2.0
+estimatedDocumentCount.json	SKIP	estimatedDocumentCount with maxTimeMS	runner: maxTimeMS in an expected command (CSOT rewrites it)
 estimatedDocumentCount.json	SKIP	estimatedDocumentCount errors correctly--command error	runner: failPoint
 estimatedDocumentCount.json	SKIP	estimatedDocumentCount errors correctly--socket error	runner: failPoint
 estimatedDocumentCount.json	FAIL	estimatedDocumentCount works correctly on views	estimatedDocumentCount: expected 2, got 0
@@ -354,17 +349,11 @@ find-comment.json	SKIP	find with comment does not set comment on getMore - pre 4
 find-let.json	SKIP	Find with let option	needs server >= 5.0
 find-let.json	FAIL	Find with let option unsupported (server-side error)	find: expected an error, the operation succeeded
 find-rawdata.json	SKIP	Find with rawData option	needs server >= 8.2.0
-find.json	FAIL	Find with filter, sort, skip, and limit	find: expected 2 elements, got 4
-find.json	FAIL	Find with limit, sort, and batchsize	find: expected 4 elements, got 6
-find.json	FAIL	Find with batchSize equal to limit	find: expected 4 elements, got 5
-findOne.json	FAIL	FindOne with filter, sort, and skip	findOne._id: expected 5, got 3
 findOneAndDelete-collation.json	FAIL	FindOneAndDelete when one document matches with collation	findOneAndDelete: expected a document, got null
 findOneAndDelete-comment.json	SKIP	findOneAndDelete with comment - pre 4.4	needs server <= 4.2.99
 findOneAndDelete-hint-serverError.json	SKIP	*	needs server <= 4.3.3
 findOneAndDelete-hint-unacknowledged.json	SKIP	Unacknowledged findOneAndDelete with hint string fails with client-side error on pre-4.4 server	needs server <= 4.2.99
 findOneAndDelete-hint-unacknowledged.json	SKIP	Unacknowledged findOneAndDelete with hint document fails with client-side error on pre-4.4 server	needs server <= 4.2.99
-findOneAndDelete-hint-unacknowledged.json	FAIL	Unacknowledged findOneAndDelete with hint string on 4.4+ server	findOneAndDelete: expected null, got {"_id":2,"x":22}
-findOneAndDelete-hint-unacknowledged.json	FAIL	Unacknowledged findOneAndDelete with hint document on 4.4+ server	findOneAndDelete: expected null, got {"_id":2,"x":22}
 findOneAndDelete-let.json	SKIP	findOneAndDelete with let option	needs server >= 5.0
 findOneAndDelete-let.json	FAIL	findOneAndDelete with let option unsupported (server-side error)	findOneAndDelete: expected an error, the operation succeeded
 findOneAndDelete-rawdata.json	SKIP	findOneAndDelete with rawData option	needs server >= 8.2.0
@@ -375,8 +364,6 @@ findOneAndReplace-dots_and_dollars.json	FAIL	Replacing document with dollar-pref
 findOneAndReplace-hint-serverError.json	SKIP	*	needs server <= 4.3.0
 findOneAndReplace-hint-unacknowledged.json	SKIP	Unacknowledged findOneAndReplace with hint string fails with client-side error on pre-4.4 server	needs server <= 4.2.99
 findOneAndReplace-hint-unacknowledged.json	SKIP	Unacknowledged findOneAndReplace with hint document fails with client-side error on pre-4.4 server	needs server <= 4.2.99
-findOneAndReplace-hint-unacknowledged.json	FAIL	Unacknowledged findOneAndReplace with hint string on 4.4+ server	findOneAndReplace: expected null, got {"_id":2,"x":22}
-findOneAndReplace-hint-unacknowledged.json	FAIL	Unacknowledged findOneAndReplace with hint document on 4.4+ server	findOneAndReplace: expected null, got {"_id":2,"x":22}
 findOneAndReplace-let.json	SKIP	findOneAndReplace with let option	needs server >= 5.0
 findOneAndReplace-let.json	FAIL	findOneAndReplace with let option unsupported (server-side error)	findOneAndReplace: expected an error, the operation succeeded
 findOneAndReplace-rawdata.json	SKIP	findOneAndReplace with rawData option	needs server >= 8.2.0
@@ -400,8 +387,6 @@ findOneAndUpdate-errorResponse.json	SKIP	findOneAndUpdate document validation er
 findOneAndUpdate-hint-serverError.json	SKIP	*	needs server <= 4.3.0
 findOneAndUpdate-hint-unacknowledged.json	SKIP	Unacknowledged findOneAndUpdate with hint string fails with client-side error on pre-4.4 server	needs server <= 4.2.99
 findOneAndUpdate-hint-unacknowledged.json	SKIP	Unacknowledged findOneAndUpdate with hint document fails with client-side error on pre-4.4 server	needs server <= 4.2.99
-findOneAndUpdate-hint-unacknowledged.json	FAIL	Unacknowledged findOneAndUpdate with hint string on 4.4+ server	findOneAndUpdate: expected null, got {"_id":2,"x":22}
-findOneAndUpdate-hint-unacknowledged.json	FAIL	Unacknowledged findOneAndUpdate with hint document on 4.4+ server	findOneAndUpdate: expected null, got {"_id":2,"x":22}
 findOneAndUpdate-let.json	SKIP	findOneAndUpdate with let option	needs server >= 5.0
 findOneAndUpdate-let.json	FAIL	findOneAndUpdate with let option unsupported (server-side error)	findOneAndUpdate: expected an error, the operation succeeded
 findOneAndUpdate-pipeline.json	FAIL	FindOneAndUpdate using pipelines	MongoServerError: update must be a document
@@ -413,24 +398,17 @@ findOneAndUpdate.json	FAIL	FindOneAndUpdate when no documents match with upsert
 insertMany-comment.json	SKIP	insertMany with comment - pre 4.4	needs server <= 4.2.99
 insertMany-dots_and_dollars.json	SKIP	Inserting document with top-level dollar-prefixed key on 5.0+ server	needs server >= 5.0
 insertMany-dots_and_dollars.json	FAIL	Inserting document with top-level dollar-prefixed key on pre-5.0 server yields server-side error	insertMany: expected an error, the operation succeeded
-insertMany-dots_and_dollars.json	FAIL	Inserting document with top-level dotted key	insertMany: unexpected extra keys ["insertedCount"]
-insertMany-dots_and_dollars.json	FAIL	Inserting document with dollar-prefixed key in embedded doc	insertMany: unexpected extra keys ["insertedCount"]
-insertMany-dots_and_dollars.json	FAIL	Inserting document with dotted key in embedded doc	insertMany: unexpected extra keys ["insertedCount"]
 insertMany-rawdata.json	SKIP	insertMany with rawData option	needs server >= 8.2.0
-insertMany.json	FAIL	InsertMany with non-existing documents	insertMany: unexpected extra keys ["insertedCount"]
 insertOne-comment.json	SKIP	insertOne with comment - pre 4.4	needs server <= 4.2.99
 insertOne-dots_and_dollars.json	SKIP	Inserting document with top-level dollar-prefixed key on 5.0+ server	needs server >= 5.0
 insertOne-dots_and_dollars.json	FAIL	Inserting document with top-level dollar-prefixed key on pre-5.0 server yields server-side error	insertOne: expected an error, the operation succeeded
 insertOne-dots_and_dollars.json	FAIL	Inserting document with dollar-prefixed key in _id yields server-side error	insertOne: expected an error, the operation succeeded
-insertOne-dots_and_dollars.json	FAIL	Unacknowledged write using dollar-prefixed or dotted keys may be silently rejected on pre-5.0 server	insertOne: unexpected extra keys ["insertedId"]
 insertOne-errorResponse.json	SKIP	insert operations support errorResponse assertions	runner: failPoint
 insertOne-rawdata.json	SKIP	insertOne with rawData option	needs server >= 8.2.0
 replaceOne-collation.json	FAIL	ReplaceOne when one document matches with collation	replaceOne.matchedCount: expected 1, got 0
 replaceOne-comment.json	SKIP	ReplaceOne with comment - pre 4.4	needs server <= 4.2.99
 replaceOne-dots_and_dollars.json	SKIP	Replacing document with dollar-prefixed key in embedded doc on 5.0+ server	needs server >= 5.0
 replaceOne-dots_and_dollars.json	FAIL	Replacing document with dollar-prefixed key in embedded doc on pre-5.0 server yields server-side error	replaceOne: expected an error, the operation succeeded
-replaceOne-hint-unacknowledged.json	FAIL	Unacknowledged replaceOne with hint string on 4.2+ server	replaceOne: unexpected extra keys ["modifiedCount","upsertedId","upsertedCount","matchedCount"]
-replaceOne-hint-unacknowledged.json	FAIL	Unacknowledged replaceOne with hint document on 4.2+ server	replaceOne: unexpected extra keys ["modifiedCount","upsertedId","upsertedCount","matchedCount"]
 replaceOne-let.json	SKIP	ReplaceOne with let option	needs server >= 5.0
 replaceOne-let.json	FAIL	ReplaceOne with let option unsupported (server-side error)	replaceOne: expected an error, the operation succeeded
 replaceOne-rawdata.json	SKIP	ReplaceOne with rawData option	needs server >= 8.2.0
@@ -444,8 +422,6 @@ updateMany-dots_and_dollars.json	SKIP	Updating document to set top-level dollar-
 updateMany-dots_and_dollars.json	SKIP	Updating document to set top-level dotted key on 5.0+ server	needs server >= 5.0
 updateMany-dots_and_dollars.json	SKIP	Updating document to set dollar-prefixed key in embedded doc on 5.0+ server	needs server >= 5.0
 updateMany-dots_and_dollars.json	SKIP	Updating document to set dotted key in embedded doc on 5.0+ server	needs server >= 5.0
-updateMany-hint-unacknowledged.json	FAIL	Unacknowledged updateMany with hint string on 4.2+ server	updateMany: unexpected extra keys ["modifiedCount","upsertedId","upsertedCount","matchedCount"]
-updateMany-hint-unacknowledged.json	FAIL	Unacknowledged updateMany with hint document on 4.2+ server	updateMany: unexpected extra keys ["modifiedCount","upsertedId","upsertedCount","matchedCount"]
 updateMany-let.json	SKIP	updateMany with let option	needs server >= 5.0
 updateMany-let.json	FAIL	updateMany with let option unsupported (server-side error)	updateMany: error message "update spec requires u" does not contain "'update.let' is an unknown field"
 updateMany-pipeline.json	FAIL	UpdateMany using pipelines	MongoServerError: update spec requires u
@@ -462,8 +438,6 @@ updateOne-dots_and_dollars.json	SKIP	Updating document to set top-level dotted k
 updateOne-dots_and_dollars.json	SKIP	Updating document to set dollar-prefixed key in embedded doc on 5.0+ server	needs server >= 5.0
 updateOne-dots_and_dollars.json	SKIP	Updating document to set dotted key in embedded doc on 5.0+ server	needs server >= 5.0
 updateOne-errorResponse.json	SKIP	update operations support errorResponse assertions	runner: failPoint
-updateOne-hint-unacknowledged.json	FAIL	Unacknowledged updateOne with hint string on 4.2+ server	updateOne: unexpected extra keys ["modifiedCount","upsertedId","upsertedCount","matchedCount"]
-updateOne-hint-unacknowledged.json	FAIL	Unacknowledged updateOne with hint document on 4.2+ server	updateOne: unexpected extra keys ["modifiedCount","upsertedId","upsertedCount","matchedCount"]
 updateOne-let.json	SKIP	UpdateOne with let option	needs server >= 5.0
 updateOne-let.json	FAIL	UpdateOne with let option unsupported (server-side error)	updateOne: error message "update spec requires u" does not contain "'update.let' is an unknown field"
 updateOne-pipeline.json	FAIL	UpdateOne using pipelines	MongoServerError: update spec requires u