db: a rebuild reclaims before it copies
Found by the churn harness, and it is the difference between reclamation working and reclamation being unreachable. A checkpoint is what hands back empty slab windows, and a checkpoint is armed by log volume. A delete logs only an `_id`. So deleting half of a 190 MB collection moved the log by a couple of megabytes, no checkpoint ran, and the garbage sailed straight past the rebuild threshold -- the rebuild got there first every time and reset the window map it would have used. Measured before this: six rounds of delete-and-refill, six rebuilds, 1 MB reclaimed. After: the same six rounds, 256 MB reclaimed. The fix is one line of ordering. `compact` now checkpoints before it walks the collections, so the cheap half of the job runs first: a checkpoint hands back whole windows for the cost of one publish, where a rebuild copies every live byte in the database. The per-collection gate then judges what reclamation left rather than what it was about to take, so a collection whose garbage was all in empty windows is not rewritten at all. No new threshold and no new state -- the gate that decides is the one added in "a rebuild copies only the collections that have garbage", now reading a post-reclamation number. 186/186 unit tests in ReleaseFast and ReleaseSafe, 83/83 fuzz, the full e2e matrix, crash-fuzz 60 cycles.
This commit is contained in:
14
src/db.zig
14
src/db.zig
@@ -1890,6 +1890,20 @@ pub const Engine = struct {
|
||||
if (self.compacting.swap(true, .acq_rel)) return;
|
||||
defer self.compacting.store(false, .release);
|
||||
|
||||
// Reclamation first, because it is the cheap half of the same job: a
|
||||
// checkpoint hands back whole windows for the cost of one publish,
|
||||
// where a rebuild copies every live byte in the database. Whatever it
|
||||
// takes, the per-collection gate below no longer sees, so a collection
|
||||
// whose garbage was all in empty windows is not rewritten at all.
|
||||
//
|
||||
// This is not a refinement, it is what makes reclamation reachable
|
||||
// under a delete-heavy workload. A checkpoint is otherwise armed by log
|
||||
// volume, and a delete logs only an `_id` -- so deleting half a 190 MB
|
||||
// collection moves the log by a couple of megabytes and no checkpoint
|
||||
// runs, while the garbage sails past the rebuild threshold. Measured
|
||||
// with the churn harness: six rounds, six rebuilds, 1 MB reclaimed.
|
||||
try self.checkpoint();
|
||||
|
||||
try self.catalog_lock.lockShared(self.io);
|
||||
var rebuild_err: ?anyerror = null;
|
||||
var db_it = self.dbs.iterator();
|
||||
|
||||
Reference in New Issue
Block a user