plan/results: the M1 churn numbers

Amendment A5 and the `[M1.1]`/`[M1.2]` blocks. What they record is a result
with two halves, and the value is in keeping both:

The mechanism works. 934 MB reclaimed over the update run, occupancy at
1.06-1.26x its live data, and the counters that say so are in `serverStatus`
rather than inferred.

The ratio did not move. 1.94x delete-heavy and 2.46x update-heavy, identical
to the end-of-Stage-2 binary measured with the same harness. `file / live` is
a high-water mark because the data file never shrinks, and the mark is set in
the first round by the one thing reclamation cannot avoid: a rebuild needs a
whole second copy of the live data before the first can be freed. So ~2x is
the floor of a rebuild-based design and no threshold reaches it -- rebuilding
earlier lowers the garbage term and nothing else, rebuilding later raises it.
The plan said in advance what to do if this happened, which was to write it
down rather than tune, and to name incremental compaction through a
doc-id-to-offset indirection layer as the successor. Recorded, with its cost:
a second copy-on-write B+tree per collection, a second random read on point
lookup, and it undoes A3.

A second lever is named that the plan had not: 52% of the steady-state file is
space the database owns and is not using, so returning it to the filesystem is
worth more here than reclaiming harder. It needs the file never to shrink
below what the fallback generation references, which is its own crash-safety
pass.

D7.4's 1.65x for the delete line is corrected to 1.94x, and the correction is
the harness rather than a regression -- the same 1.94x comes out of the binary
that predates any of this work. The old ad-hoc version sampled ids to delete
blindly, which re-picks dead ones, so it deleted fewer documents than it
inserted and measured a collection that was quietly growing. The update line
reproduces D7.4 exactly, 2.46 against 2.47.

200-byte documents reclaim nothing, exactly as forecast, and the forecast
being written down beforehand is what makes that a result instead of a
disappointment. 3.93x on both binaries. Also noted, because the number invites
misreading: at that document size the two index trees are comparable to the
documents themselves and the file is already 2.18x before any churn -- index
structure, not slab garbage.

PLAN.md

@@ -320,6 +320,71 @@ takes the first one's place. That commit is where this needs handling — a
 pre-flight scan for compare-equal `_id`s, refusing to drop the map silently
 while any exist — not here.
 
+### Amendment A5 — the doc-level free list, and what it did not fix (amends A2, closes D7.4)
+
+D7.4 left M0 with a bound rather than a target: 1.65× delete-heavy, 2.47×
+update-heavy against a hoped-for ~1.3×, and the stated conclusion that
+doc-level free lists were an M1 item. They are built. The mechanism is
+measured, it works, and **the steady-state ratio did not move**. Both halves
+of that are the amendment.
+
+**What was built.** A collection's slab carries a dense map of dead bytes per
+`map_align` window — two bytes per window, so 2.7 MB for a 21 GB slab — and a
+checkpoint hands back every window with nothing live left in it, splitting the
+runs around what is kept. The window is the unit because it is the smallest
+thing that can be given back at all: `mark_appendable` refuses an unaligned
+start and `protect_stable` rounds outwards. Counting is the whole liveness
+test, because `evict_doc` removes a document's index entries before marking its
+bytes dead, so "no live bytes in this window" and "nothing references these
+bytes" are the same statement. Reclamation lives inside `checkpoint` rather
+than beside it so that the run split and the `free_pages` become durable under
+one `publish`; there is no new record type, no new catalog version and no
+replay path. `alloc_slab_run` is a second policy in the same allocator, because
+`take_free`'s best fit — which exists to stop one-page copy-on-write requests
+dismantling the extents — can never match a request for 2048 pages against
+runs that come back a few windows at a time.
+
+**What it does not fix, and why no threshold reaches it.** The data file never
+shrinks, so `file / live` is a high-water mark, and the mark is set once by the
+one thing reclamation cannot avoid: a rebuild needs a whole second copy of the
+live data before the first can be freed. Live + garbage-at-trigger + copy is
+the peak, and it is reached in the first round, before any free pool exists to
+build the copy out of. Rebuilding earlier lowers the garbage term and nothing
+else; rebuilding later raises it. So ~2× is the floor of a rebuild-based
+design, and tuning is the wrong instrument. Measured occupancy tells the other
+half of the story: 1.06–1.26× in use against a 2.46× file, with 934 MB
+reclaimed over the run.
+
+**The successor, named here so the next session does not re-derive it.**
+Incremental compaction through a doc-id → offset indirection layer, which is
+rejected option (b) of the M1 design, promoted. It is the only thing that
+removes the second copy: a rebuild becomes a move of one document at a time
+with the map updated behind it. The cost is the one that got it rejected — the
+map has to be persistent and crash-safe, i.e. a second copy-on-write B+tree per
+collection and a second random read on the point-lookup path — and it undoes
+A3. That is a milestone, not a knob. Second and cheaper: return free space to
+the filesystem, since 52% of the steady-state file is space the database owns
+and is not using; it needs the file never to shrink below what the fallback
+generation references, which is its own crash-safety design pass.
+
+**Small documents behave exactly as forecast**, and the forecast being written
+down in advance is what makes it a result. 200-byte documents reclaim nothing
+at all — a 16 KiB system page holds ~70 of them and they never all die at once
+— and the counters show a mechanism correctly doing nothing rather than one
+misfiring. The payoff scales as `doc_size / map_align`, so 4 KiB pages read
+four times better on the same code.
+
+**One thing the gate found that the design had not.** A checkpoint is what
+reclaims and a checkpoint is armed by log volume, but a delete logs only an
+`_id`. Deleting half a 190 MB collection moved the log by a couple of megabytes
+so no checkpoint ran, the garbage sailed past the rebuild threshold, and the
+rebuild reset the window map it would have used — six rounds, six rebuilds,
+1 MB reclaimed. `compact` now checkpoints before it copies, which is also the
+right order on its own terms: the cheap half of the job first, and the
+per-collection gate judges what reclamation left. Same six rounds, 256 MB.
+Numbers and reproduction in `tests/e2e/results/m0-gates.txt` under `[M1.1]` and
+`[M1.2]`.
+
 ---
 
 ## 3. Milestones and gates
@@ -721,6 +786,16 @@ between `compact` and `checkpoint`; it is out of this scope because it wants
 its own design pass, and because the free list must not add a second instance
 of the same shape.
 
+*Where this stands after the free list.* It did add a second instance —
+reclamation frees pages as a checkpoint phase, and two checkpoints can be in
+flight — so that half is closed: `checkpoint` takes a lock of its own. Two
+things came out of doing it. The publish was never the exposure, because it
+already runs under `log_lock`; and the whole class is now *detectable* rather
+than only arguable, because `write_catalog` asserts per run that the pager has
+not already been given it, in test and Debug builds. That assertion is proven
+to fire. The original instance — `compact`'s rebuild walk against a concurrent
+checkpoint — is unchanged and still wants the design pass.
+
 ### The spec runner starts reading `expectEvents`
 
 354 of the 487 cases declare `expectEvents` and the runner read none of them,
@@ -823,15 +898,21 @@ has to be its own commit with its own re-recorded scorecard.
     the anchor rewritten — resuming at it returned updated documents twice,
     caught by draining a collection being updated underneath.
 
-  Still open in M1: the doc-level free list. The eight reclamation bugs above
-  were cleared first, as preconditions for the free list rather than as work of
-  their own; command-monitoring (`expectEvents`) landed next, so that what
-  followed is measured by an instrument no longer known to overstate.
-  **A prerequisite the free list must honour**, recorded here while it is
-  still being designed: *an offset that was ever a record start must remain a
-  record start.* `doc_bytes` reads a `u32` length prefix in place, so an
-  offset landing mid-record after a re-split is a garbage-length read rather
-  than a wrong answer — and an offsets cursor holds exactly such offsets.
+  The doc-level free list is built; see amendment A5 for what it did and did
+  not achieve, and `[M1.1]`/`[M1.2]` in the results file for the numbers. The
+  eight reclamation bugs above were cleared first, as preconditions for it
+  rather than as work of their own; command-monitoring (`expectEvents`) landed
+  next, so that what followed is measured by an instrument no longer known to
+  overstate.
+  **The prerequisite it had to honour** — *an offset that was ever a record
+  start must remain a record start*, because `doc_bytes` reads a `u32` length
+  prefix in place and an offsets cursor holds exactly such offsets — is met
+  structurally rather than by checking: a window is handed back only when every
+  byte in it is dead, which means every document touching it has already been
+  through `evict_doc` and out of every index. What remains is the cursor
+  holding a *saved* offset list, and that is answered the way a rebuild answers
+  it, by bumping `layout_epoch` when and only when a collection actually gave
+  something back.
 - **M1 sessions** — *settled and implemented.* `lsid` is parsed, validated and
   deliberately acted on in no way; `txnNumber`, `startTransaction` and
   `autocommit` are refused; `endSessions` validates the array it discards.