db/storage: reclaim the log once a checkpoint covers it
The point of a lagging checkpoint: a record whose effect the data file already holds is redundant, so the log can go back to just its header. Without this the log only grows and every open pays for every write ever made. Ordering, which is the whole safety argument: publish the watermark, *then* truncate. The other way round, a crash between them leaves the records gone from the log and absent from any image. A failed truncation is a warning rather than an error -- it costs space and replay time, and loses nothing, so it must not fail a checkpoint that already succeeded. Also wires checkpointing up, which nothing did before. `note_checkpoint` arms it when the log passes a threshold, and the write epilogue and the TTL monitor both claim it -- outside any collection lock, for the same reason compaction runs there: it takes the log lock. The threshold is separate from the compaction one on purpose: compaction is about the garbage share of the data, a checkpoint is about how much replay an open would otherwise do. -- Two things the tests taught me. The first version measured the log before the checkpoint and found 16 bytes -- just the header. Appends buffer in the log's open block and only a commit seals and writes it, so there was nothing on disk to shrink. The test commits first now, and says why. And the "no valid watermark" warning fired for every young database, which is its normal state before the first checkpoint. It now distinguishes a watermark that was *written and cannot be read* from one that was never written -- warning about the ordinary case is how people learn to ignore the warning that matters. Mutation-checked, red: skipping the truncation. Not covered, and the test says so: moving the truncation before the publish, whose failure mode is a crash landing between the two. That needs process-level crash injection, which an in-process test cannot express.
This commit is contained in:
@@ -626,15 +626,29 @@ pub const Pager = struct {
|
||||
// writes from here on.
|
||||
self.stable_pages = wm.alloc_tail;
|
||||
try self.read_freelist(wm);
|
||||
} else if (self.file_pages > page_first_data) {
|
||||
} else if (self.watermark_attempted()) {
|
||||
// Only worth saying when a watermark was *written* and cannot be
|
||||
// read: a data file that simply never reached its first checkpoint
|
||||
// is the normal state of a young database, and warning about it
|
||||
// trains people to ignore the warning that matters.
|
||||
std.debug.print(
|
||||
"multiforadb: WARNING: data file '{s}' has no valid watermark; " ++
|
||||
"multiforadb: WARNING: data file '{s}' has a damaged watermark; " ++
|
||||
"treating it as having no checkpoint and replaying the log in full\n",
|
||||
.{self.path},
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// Whether either slot holds anything at all. A never-checkpointed file has
|
||||
/// both slots as written by create: all zeroes.
|
||||
fn watermark_attempted(self: *const Pager) bool {
|
||||
for ([_]u32{ page_watermark_a, page_watermark_b }) |p| {
|
||||
if (p >= self.mapped_pages) continue;
|
||||
for (self.page(p)) |byte| if (byte != 0) return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
fn read_slot(self: *const Pager, p: u32) ?Watermark {
|
||||
if (p >= self.mapped_pages) return null;
|
||||
const b = self.page(p);
|
||||
|
||||
Reference in New Issue
Block a user