From 343b25adc8db2cf7e54a0ca049dce7af01c0b809 Mon Sep 17 00:00:00 2001 From: "A.Shakhmatov" Date: Sun, 9 Aug 2026 17:43:28 +0300 Subject: [PATCH] db: a rebuild reclaims before it copies Found by the churn harness, and it is the difference between reclamation working and reclamation being unreachable. A checkpoint is what hands back empty slab windows, and a checkpoint is armed by log volume. A delete logs only an `_id`. So deleting half of a 190 MB collection moved the log by a couple of megabytes, no checkpoint ran, and the garbage sailed straight past the rebuild threshold -- the rebuild got there first every time and reset the window map it would have used. Measured before this: six rounds of delete-and-refill, six rebuilds, 1 MB reclaimed. After: the same six rounds, 256 MB reclaimed. The fix is one line of ordering. `compact` now checkpoints before it walks the collections, so the cheap half of the job runs first: a checkpoint hands back whole windows for the cost of one publish, where a rebuild copies every live byte in the database. The per-collection gate then judges what reclamation left rather than what it was about to take, so a collection whose garbage was all in empty windows is not rewritten at all. No new threshold and no new state -- the gate that decides is the one added in "a rebuild copies only the collections that have garbage", now reading a post-reclamation number. 186/186 unit tests in ReleaseFast and ReleaseSafe, 83/83 fuzz, the full e2e matrix, crash-fuzz 60 cycles. --- src/db.zig | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/src/db.zig b/src/db.zig index c6f96c7..6cd38a3 100644 --- a/src/db.zig +++ b/src/db.zig @@ -1890,6 +1890,20 @@ pub const Engine = struct { if (self.compacting.swap(true, .acq_rel)) return; defer self.compacting.store(false, .release); + // Reclamation first, because it is the cheap half of the same job: a + // checkpoint hands back whole windows for the cost of one publish, + // where a rebuild copies every live byte in the database. Whatever it + // takes, the per-collection gate below no longer sees, so a collection + // whose garbage was all in empty windows is not rewritten at all. + // + // This is not a refinement, it is what makes reclamation reachable + // under a delete-heavy workload. A checkpoint is otherwise armed by log + // volume, and a delete logs only an `_id` -- so deleting half a 190 MB + // collection moves the log by a couple of megabytes and no checkpoint + // runs, while the garbage sails past the rebuild threshold. Measured + // with the churn harness: six rounds, six rebuilds, 1 MB reclaimed. + try self.checkpoint(); + try self.catalog_lock.lockShared(self.io); var rebuild_err: ?anyerror = null; var db_it = self.dbs.iterator();