db: compaction becomes a data-file rebuild
`compact` used to re-emit every live document into a fresh log and rename it over the old one. That is the wrong shape twice over now: the log is not where the data lives, and a re-emitted record carries a sequence a later watermark can cover, which would make the next open skip it (PLAN section 4). The log re-emission is deleted; the checkpoint at the end reclaims the log instead. What it reclaims is what a checkpoint cannot. A checkpoint publishes the structures where they already are, and it cannot move a document, because every index leaf holds that document's physical offset. So reclaiming a replaced document's bytes means rewriting the documents *and* repacking every index against the new offsets, together -- which is the whole of `rebuild_collection`. Documents are copied in _id order, so the new slab reads sequentially afterwards. Old extents and old node pages go to the free list rather than being reused immediately, so a crash mid-rebuild simply loses the rebuild: the previous watermark still describes the previous layout, intact. Adds `Collection.slab_used`, because `slab_tail` cannot answer "how many bytes are in use" -- it is an absolute file offset and jumps forward with each new extent. That is also the number the rebuild trigger wants. -- The test is the part worth reading. My first version asserted that every document was still findable and had the replaced contents, and it was nearly useless: two mutations -- not repacking the indexes at all, and not republishing the docs-map offsets -- both left it green. Freed extents go on the free list rather than being overwritten, so a stale offset still reads a perfectly plausible document. What actually distinguishes a repacked index from a stale one is *where* the offset points: after a rebuild every live offset must fall inside an extent the collection currently owns. Asserting that, plus that the index and the map agree, turns all three mutations red -- including repacking `_id_` but forgetting the secondaries.
This commit is contained in:
@@ -579,6 +579,30 @@ pub const Index = struct {
|
||||
return duplicate;
|
||||
}
|
||||
|
||||
/// Throw the tree away and start from an empty root, so a rebuild can pack a
|
||||
/// fresh one. The old pages go on the free list, which withholds them for two
|
||||
/// generations -- the image that still references them stays intact.
|
||||
pub fn reset_tree(self: *Index, gpa: std.mem.Allocator) !void {
|
||||
for (self.node_pages.items) |p| try self.pager.free_pages(p, 1);
|
||||
for (self.ovf_extents.items) |e| try self.pager.free_pages(e.first, e.pages);
|
||||
self.node_pages.clearRetainingCapacity();
|
||||
self.ovf_extents.clearRetainingCapacity();
|
||||
self.ovf_tail = 0;
|
||||
self.ovf_end = 0;
|
||||
try self.node_pages.ensureUnusedCapacity(gpa, 2);
|
||||
try self.pager.reserve_pages(2);
|
||||
self.node_pages.appendAssumeCapacity(self.pager.alloc_pages_assume_reserved(1));
|
||||
self.node_pages.appendAssumeCapacity(self.pager.alloc_pages_assume_reserved(1));
|
||||
self.page_mut(0).* = empty_node(0);
|
||||
self.page_mut(1).* = empty_node(1);
|
||||
self.root = 1;
|
||||
self.first_leaf = 1;
|
||||
self.leaf_count = 1;
|
||||
self.depth = 0;
|
||||
self.entry_count = 0;
|
||||
self.multikey = false;
|
||||
}
|
||||
|
||||
/// Remove every entry for `id`, in one pass over the leaves. Infallible.
|
||||
/// Used directly by remove_id's own callers and as the fallback when
|
||||
/// regeneration cannot locate entries.
|
||||
|
||||
Reference in New Issue
Block a user