db/storage: reclaim the log once a checkpoint covers it

The point of a lagging checkpoint: a record whose effect the data file already
holds is redundant, so the log can go back to just its header. Without this the
log only grows and every open pays for every write ever made.

Ordering, which is the whole safety argument: publish the watermark, *then*
truncate. The other way round, a crash between them leaves the records gone from
the log and absent from any image. A failed truncation is a warning rather than
an error -- it costs space and replay time, and loses nothing, so it must not
fail a checkpoint that already succeeded.

Also wires checkpointing up, which nothing did before. `note_checkpoint` arms it
when the log passes a threshold, and the write epilogue and the TTL monitor both
claim it -- outside any collection lock, for the same reason compaction runs
there: it takes the log lock. The threshold is separate from the compaction one
on purpose: compaction is about the garbage share of the data, a checkpoint is
about how much replay an open would otherwise do.

--

Two things the tests taught me.

The first version measured the log before the checkpoint and found 16 bytes --
just the header. Appends buffer in the log's open block and only a commit seals
and writes it, so there was nothing on disk to shrink. The test commits first
now, and says why.

And the "no valid watermark" warning fired for every young database, which is
its normal state before the first checkpoint. It now distinguishes a watermark
that was *written and cannot be read* from one that was never written -- warning
about the ordinary case is how people learn to ignore the warning that matters.

Mutation-checked, red: skipping the truncation. Not covered, and the test says so:
moving the truncation before the publish, whose failure mode is a crash landing
between the two. That needs process-level crash injection, which an in-process
test cannot express.
This commit is contained in:
2026-08-03 21:35:03 +03:00
parent 58e645b969
commit 138b7f706f
5 changed files with 166 additions and 2 deletions

View File

@@ -460,6 +460,27 @@ pub const Log = struct {
/// Appends never sync (see `append_record`), so nothing is durable until
/// this returns -- which is why Engine.commit calls it exactly once per
/// write command, coalescing every writer in flight into a single fsync.
/// Discard every record, keeping only the file header.
///
/// Only a checkpoint may call this, and only after the watermark that covers
/// these records is durable. The whole point of a lagging checkpoint is that
/// the log can be reclaimed once the data file holds its effect -- and the
/// order is not negotiable: truncate before the watermark is durable and a
/// crash in between leaves records gone from the log and absent from the
/// image.
///
/// Caller holds the log lock. The open block is dropped rather than sealed:
/// its records are below the watermark too, so writing them out would only
/// be work the next open throws away.
pub fn truncate_to_header(self: *Log) !void {
self.block.clearRetainingCapacity();
try self.file.setLength(self.io, file_header_len);
try self.file.sync(self.io);
self.end_pos = file_header_len;
self.log_bytes = 0;
self.data_bytes = 0;
}
pub fn sync(self: *Log) !void {
try self.seal_block();
try self.file.sync(self.io);