db: compaction becomes a data-file rebuild

`compact` used to re-emit every live document into a fresh log and rename it over
the old one. That is the wrong shape twice over now: the log is not where the data
lives, and a re-emitted record carries a sequence a later watermark can cover,
which would make the next open skip it (PLAN section 4). The log re-emission is
deleted; the checkpoint at the end reclaims the log instead.

What it reclaims is what a checkpoint cannot. A checkpoint publishes the
structures where they already are, and it cannot move a document, because every
index leaf holds that document's physical offset. So reclaiming a replaced
document's bytes means rewriting the documents *and* repacking every index
against the new offsets, together -- which is the whole of `rebuild_collection`.
Documents are copied in _id order, so the new slab reads sequentially afterwards.

Old extents and old node pages go to the free list rather than being reused
immediately, so a crash mid-rebuild simply loses the rebuild: the previous
watermark still describes the previous layout, intact.

Adds `Collection.slab_used`, because `slab_tail` cannot answer "how many bytes
are in use" -- it is an absolute file offset and jumps forward with each new
extent. That is also the number the rebuild trigger wants.

--

The test is the part worth reading. My first version asserted that every document
was still findable and had the replaced contents, and it was nearly useless: two
mutations -- not repacking the indexes at all, and not republishing the docs-map
offsets -- both left it green. Freed extents go on the free list rather than being
overwritten, so a stale offset still reads a perfectly plausible document.

What actually distinguishes a repacked index from a stale one is *where* the
offset points: after a rebuild every live offset must fall inside an extent the
collection currently owns. Asserting that, plus that the index and the map agree,
turns all three mutations red -- including repacking `_id_` but forgetting the
secondaries.
This commit is contained in:
2026-08-03 21:48:04 +03:00
parent 138b7f706f
commit 148e03ac9f
2 changed files with 248 additions and 122 deletions

View File

@@ -579,6 +579,30 @@ pub const Index = struct {
return duplicate;
}
/// Throw the tree away and start from an empty root, so a rebuild can pack a
/// fresh one. The old pages go on the free list, which withholds them for two
/// generations -- the image that still references them stays intact.
pub fn reset_tree(self: *Index, gpa: std.mem.Allocator) !void {
for (self.node_pages.items) |p| try self.pager.free_pages(p, 1);
for (self.ovf_extents.items) |e| try self.pager.free_pages(e.first, e.pages);
self.node_pages.clearRetainingCapacity();
self.ovf_extents.clearRetainingCapacity();
self.ovf_tail = 0;
self.ovf_end = 0;
try self.node_pages.ensureUnusedCapacity(gpa, 2);
try self.pager.reserve_pages(2);
self.node_pages.appendAssumeCapacity(self.pager.alloc_pages_assume_reserved(1));
self.node_pages.appendAssumeCapacity(self.pager.alloc_pages_assume_reserved(1));
self.page_mut(0).* = empty_node(0);
self.page_mut(1).* = empty_node(1);
self.root = 1;
self.first_leaf = 1;
self.leaf_count = 1;
self.depth = 0;
self.entry_count = 0;
self.multikey = false;
}
/// Remove every entry for `id`, in one pass over the leaves. Infallible.
/// Used directly by remove_id's own callers and as the fallback when
/// regeneration cannot locate entries.