138b7f706fb34a7008a6ed7c31fc5d9efaa85ccd
5 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
| 138b7f706f |
db/storage: reclaim the log once a checkpoint covers it
The point of a lagging checkpoint: a record whose effect the data file already holds is redundant, so the log can go back to just its header. Without this the log only grows and every open pays for every write ever made. Ordering, which is the whole safety argument: publish the watermark, *then* truncate. The other way round, a crash between them leaves the records gone from the log and absent from any image. A failed truncation is a warning rather than an error -- it costs space and replay time, and loses nothing, so it must not fail a checkpoint that already succeeded. Also wires checkpointing up, which nothing did before. `note_checkpoint` arms it when the log passes a threshold, and the write epilogue and the TTL monitor both claim it -- outside any collection lock, for the same reason compaction runs there: it takes the log lock. The threshold is separate from the compaction one on purpose: compaction is about the garbage share of the data, a checkpoint is about how much replay an open would otherwise do. -- Two things the tests taught me. The first version measured the log before the checkpoint and found 16 bytes -- just the header. Appends buffer in the log's open block and only a commit seals and writes it, so there was nothing on disk to shrink. The test commits first now, and says why. And the "no valid watermark" warning fired for every young database, which is its normal state before the first checkpoint. It now distinguishes a watermark that was *written and cannot be read* from one that was never written -- warning about the ordinary case is how people learn to ignore the warning that matters. Mutation-checked, red: skipping the truncation. Not covered, and the test says so: moving the truncation before the publish, whose failure mode is a crash landing between the two. That needs process-level crash injection, which an in-process test cannot express. |
|||
| d7f7ebb994 |
pager: copy-on-write above the stable mark
The invariant everything else in the crash story rests on (PLAN amendment A1): no page belonging to the last published image is ever stored into, so recovery is `image + replay(seq > watermark)` and the image's bytes are exactly what the watermark described. `page_mut_cow` takes a *pointer to the owner's page number*. That is the load- bearing detail: copy-on-write relocates the page, so the owner has to be told, and a second reference would still aim at the abandoned copy. For the B+tree the owner is the id->page table slot -- which is precisely why node ids are not page numbers. Inert until a checkpoint publishes something, since the stable mark starts at zero. Append-only consumers keep writing in place, except that a checkpoint landing mid-extent freezes the page their tail points into, so both slabs now start a fresh extent rather than writing inside the image. Waste is bounded by one extent per collection per checkpoint. The free list is wired into allocation, which it was not before: copy-on-write abandons every page it touches in every generation, so without reuse the file grows by `generations x touched_set` without bound. That is the difference between a free list being defense-in-depth and being a prerequisite (A2). -- Three things I got wrong on the way, all worth recording. I added a `p >= stable_pages` assert to `page_mut` and had to take it back out. A page recycled off the free list *is* below the mark and *is* legitimately writable -- freed two generations ago, referenced by no live image -- so the page number alone cannot tell a violation from a reuse. The invariant is enforced the two ways A1 actually describes: structurally through `page_mut_cow`, and mechanically through mprotect. The comment says so, since the assert looks like an obvious thing to add. The watermark slots needed a narrow exception, because overwriting the inactive slot is the publication mechanism rather than a violation. It is a separate non-public accessor that asserts its argument is a slot, so it cannot become a general escape hatch. And the mprotect belt: `std.posix.mprotect` does not exist in Zig 0.16, so it is a libc call. It compiled only in ReleaseFast, where the branch is comptime- eliminated -- ReleaseSafe caught that immediately, which is the argument for running both. What the belt's test asserts is that the protection is really applied, not that a violating write faults. A SIGSEGV cannot be caught in-process, and the fault is the OS's behaviour rather than this code's; an mprotect that failed silently would leave a belt that looks present and does nothing, which is the failure worth guarding here. Stated in the test rather than implied. Mutation-checked, all red: COW returning without copying; copying without moving the slot; copying when already above the mark; never reusing a freed page. |
|||
| 2e7f72074f |
index: the node arena and overflow slab live in the data file
The last structures move onto the pager, so the whole engine's storage is now
one mapped file plus the WAL.
Node ids are deliberately *not* page numbers. PLAN amendment A1 explains why:
`Node.parent`, `next`, `prev` and an internal slot's `extra` are back-pointers
by id, so copy-on-write moving a page would force every node referring to it to
move as well -- COWing one leaf cascades through the leaf level, one internal
node through its whole subtree. An in-RAM id->page table makes the table slot
the single owner of a page number, so COW has exactly one pointer to fix. It
costs one dependent load per node access and 4 bytes per node, about 5.6 MB at
100M documents, against the 64-100 bytes *per document* this milestone removes.
The overflow slab becomes extents too, so `Slot.off` for a spilled record is an
absolute file offset -- the same change documents went through.
--
Two bugs, both found by measuring rather than by reading, and both worth
recording because the second one would have been invisible until the churn gate.
The reservation was a tail mark, and it cannot be: an upsert reserves tree pages
for every index *and* slab room for the document, all before one log append. The
second reserver overwrote the first one's promise and the first one's allocation
then asserted. Caught on a 512 MB load by the tripwire added in the
`reserve_for` commit, which is the entire reason that assert exists. It is a
count now, and the multi-consumer ordering is pinned by a test.
And a reservation was never released. It is scoped to one write -- taken before
the log append so the publish cannot fail -- but a tree reservation covers the
worst case of several splits while a typical insert causes none, so the promise
accumulated by a handful of pages per write and dragged the file up with it. The
data file was **1.89 GB for 512 MB of documents**; releasing the unclaimed
promise at the end of each write brings it to 551 MB, or 1.08x, which is the
extent slack and the node pages.
--
Measured on one harness, 512 MB / 16 KB docs, against the in-RAM engine this
replaces:
bulk insert throughput 742.6 MB/s -> 736.4 MB/s
insertOne (sequential) 0.20 ms -> 0.22 ms
createIndex({k: 1}) 26.8 ms -> 16.5 ms
countDocuments({}) 2.1 ms -> 1.1 ms
findOne({k: 500}) indexed 0.75 ms -> 0.56 ms
find({p: range}).count() 6.6 ms -> 4.6 ms
aggregate $group by k 5.8 ms -> 3.8 ms
updateMany({k: 7}, {$inc}) 1.2 ms -> 1.0 ms
Reads gain from one contiguous mapping; the two write rows are within noise of
flat. RSS is still unchanged and still cannot improve, for the reason given in
the previous commit: every open replays the whole log and rebuilds everything.
The dev harnesses each open their own data file now. `zig build fuzz` caught all
four of them, again.
|
|||
| c3e7368477 |
pager: watermark double buffer, page free list, and the ordering that matters
Still engine-unused; this completes the data file's own machinery so the structures can move onto it next. The watermark is what a checkpoint publishes: the log sequence the image covers, the allocation extent, and where the catalog and free list live. Two slots, written by generation parity, so a torn write can never leave zero valid slots -- writing a new generation over the only copy of the old one could. The newer slot whose hash validates wins, and validation happens *before* the generation comparison, or a torn slot wins whenever its garbage generation happens to be larger. A slot that validates is trusted, so it is checked for sense as well as integrity: a rehashed slot with an impossible alloc_tail would otherwise be believed and would describe a file that does not exist. Both slots unreadable opens anyway, with a loud warning and no checkpoint, which means a full replay. Ground rule 4: refusing to start costs more than the warning does. The free list releases pages two generations after they are freed. That is not caution -- it is what keeps generation N-1 a usable image, since its pages stay allocated while N is current, so a torn watermark or a bad catalog can fall back instead of discarding the database. -- Two things worth reading, because both were wrong first. Publish captured alloc_tail *before* writing the free list, so the pages the free list occupies fell outside the published image and the next open would have handed them out again while the watermark still pointed at them. Found by the first test written against it. And the ordering invariant -- every page a watermark describes is durable before the watermark that describes it -- was untestable, which is worse than untested. A kill -9 does not lose page-cache writes, so deleting the sync leaves every test green while making a real power loss unrecoverable. So the pager now records, in test builds only, which pages have been written since the last sync, and a test discards exactly those from the file before reopening. That simulates the one failure a process kill cannot produce. `track_dirty` is `builtin.is_test`, so `page_mut` carries no branch in a real build. Mutation-checked, and the precise result is in the comments because the obvious mutation is not a violation: publish syncs twice before the watermark, so removing either call alone is harmless and correctly stays green. Removing both, or deferring them past the watermark, goes red. Also red: always writing slot A; accepting an impossible alloc_tail; releasing freed pages a generation early. Noted as belt-and-braces rather than claimed as covered: the generation-zero check, which the hash already rejects. |
|||
| 04d56f5b66 |
pager: the data file, its page allocator and the mmap over a fixed reservation
New src/pager.zig, engine-unused at this commit: the structures move onto it in the commits that follow, and landing it alone keeps that change reviewable. The file is an array of 4 KiB pages with a tail-bump extent allocator. PLAN D6.1 asked for a region table; it cannot be one, because there is a node arena per index and a document slab per collection, so the region count is dynamic and unbounded and N contiguous regions cannot all grow at the tail. One page array means exactly one growth path, so the write-then-extend discipline lives in exactly one place, and `ls`/`du` stay honest for the backup story. The mapping is a PROT_NONE anonymous NORESERVE reservation that new file-backed suffixes are mmap'd into with MAP_FIXED. That is one VMA and zero committed pages, and it means **the base never moves for the life of the process**, so a pointer handed out before a growth is still valid after it. The ArrayList-backed arena this replaces could not promise that -- the promoted-key scratch buffer in index.zig exists solely to work around it, and phase8 records a dangling-slab-pointer bug of exactly that shape. Growth is `setLength` and *then* `mmap`, never the reverse: a store into a mapped page past end-of-file raises SIGBUS, which no error path can catch. The accessors assert against `mapped_pages`, so a violation is a panic with a message instead of a signal. `page_mut` is deliberately the only way to obtain a writable page. Copy-on-write hooks in there (commit 12), and funnelling every write through one function is what makes that a change of one body rather than of every caller. `sync` is msync + fsync and only the checkpoint calls it. Between checkpoints dirty pages may sit in the page cache indefinitely, because recovery is `image + replay(seq > watermark)` and the image's pages are never written *differently* -- which is what keeps the write path at exactly one fsync, the WAL's (PLAN amendment A1). This is the one place in src/ that reaches for std.posix, against the house style, and the module comment says why: std.Io.File.MemoryMap prefaults by default, exposes no NORESERVE/FIXED/address hint so it cannot express a reservation, and its setLength is mremap on Linux and unsupported on darwin. Mutation-checked, all four red: remapping the prefix at a kernel-chosen address; dropping the uuid comparison (which is what stops one database's log being replayed onto another's checkpoint); dropping the header hash comparison; and moving setLength after mmap. An empty file is treated as absent rather than as corruption, so a create that died before its header landed still opens. |
