pager: reuse the data file when there is no checkpoint to honour

`Pager.open` set `alloc_tail` to the end of the existing file -- "everything
already in the file is allocated until a watermark narrows it down". Safe when
a watermark exists. When one does not, it is the opposite of safe: nothing in
the file is referenced, the log is the whole truth and replay is about to
rebuild the slab, the trees and the overflow from it, so every reopen started
allocating *above* the previous copy. With no watermark there is also no free
list, so the old copy was never given back. Linear growth per open, unbounded.

It does not need a crash. A database small enough never to reach the 32 MB
checkpoint threshold never publishes a watermark at all, so *every* clean
reopen took this path:

  20 documents inserted per cycle, 12 reopen cycles
  before   17, 34, 50, 67, 85, 102, 118, 135, 168, 201, 236, 269 MB
  after    17 MB, flat

240 documents in a 269 MB file, heading for `DatabaseTooLarge`. The crash
fuzzer shows the same thing under a real workload -- 60 crash/reopen cycles
with ~460 documents ended at 2735 MB before, 17 MB after, with the prefix
invariant holding either way. That number was sitting in its own output as
`data=2735MB` and reads as normal until you divide it by the document count.

The file is deliberately not truncated. The mapping already covers these pages
and `grow_to` extends the file only when the mapping is too small, so
shortening the file behind a mapping that still spans it would turn a later
write into SIGBUS. Reusing from the front is what the unbounded growth needed;
giving the disk back is a separate change to the same function.

Mutation: leave `alloc_tail` at the file end -- red on the new test, which
opens, writes and closes three times without a checkpoint and requires the
third tail to be within one slab extent of the first.
This commit is contained in:
2026-08-04 00:40:30 +03:00
parent 1814020df9
commit 319a515b89
2 changed files with 68 additions and 0 deletions

View File

@@ -352,6 +352,30 @@ pub const Pager = struct {
// narrows it down.
self.alloc_tail = @max(page_first_data, self.file_pages);
try self.load_watermark();
if (self.loaded.generation == 0) {
// No usable checkpoint: either none was ever published, or both
// watermark slots were unreadable. Then *nothing* in this file is
// referenced -- the log is the whole truth and replay is about to
// rebuild the slab, the trees and the overflow from it -- so the
// file is free space, not allocated space.
//
// Leaving `alloc_tail` at the file end instead made every reopen
// append a fresh copy above the last one, with no watermark and
// therefore no free list to ever give the old copy back. Linear
// growth per open, unbounded, and it does not need a crash: a
// database small enough never to reach the checkpoint threshold
// never publishes a watermark at all, so *every* clean reopen
// took this path. Measured at 20 documents per cycle: +17 MB per
// reopen, 269 MB after twelve, on course for DatabaseTooLarge.
//
// The file is not truncated here on purpose. The mapping is
// already established over these pages, and `grow_to` extends the
// file only when the mapping is too small -- so shortening the
// file behind a mapping that still covers it would turn a later
// write into SIGBUS. Reusing from the front bounds the file at
// its high-water mark, which is what the unbounded growth needed.
self.alloc_tail = page_first_data;
}
}
return self;
}