dev/MultiforaDB
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

pager: a page reservation belongs to its consumer, not to the pager

Browse Source 
The promise `reserve_pages` makes was a single counter on the pager, and the
first concurrent benchmark since the data file landed aborted the server on
it, reliably, at four clients:

  assertion failed: page allocation overran reserve_pages' promise
    src/index.zig:955 in alloc_node
    src/db.zig:794  in upsert

Two upserts on different collections hold different collection locks, so they
run at the same time. Each ends by dropping "whatever is still promised" --
and `release_reservation` zeroed the shared counter, so the first to publish
released the second's promise while the second was still between its log
append and its supposedly infallible allocation. The tripwire fired, which is
the good outcome; the bad one is a growth that never happened and a store past
the mapped end.

This is PLAN risk 3 ("a shared pager makes alloc_tail and free_pending a
global mutex on every insert"), whose mitigation -- private pre-allocated runs
-- was never built. So: `pager.Reservation` is a per-consumer promise, held by
every Index, every Collection (for its doc slab) and the checkpoint, and each
one releases only its own. The pager keeps the sum, which is all `grow_to`
needs. `Engine.release_write_reservations` drops exactly the buckets one
upsert reserved through.

The allocator's own state -- the tail, the total, the free lists, the
unpublished set, file growth -- is now behind `alloc_lock`, taken
uncancelable. It is never held across the log append: that is precisely what
per-consumer reservations buy, and why group commit is unaffected.

  concurrent durable insertOne   4 clients  21697 docs/s   (was aborting)
                                16 clients  30678 docs/s

Mutation: make `release_reservation` zero `self.reserved_pages` again. Red on
the new pager test and on three command tests.
This commit is contained in:
Alexey Shakhmatov
2026-08-03 22:51:15 +03:00
parent 5228ed740a
commit 4b70ce6da9
3 changed files with 156 additions and 41 deletions
Download Patch File Download Diff File Expand all files Collapse all files

40
src/db.zig
View File

@@ -50,6 +50,10 @@ pub const Collection = struct {
/// rebuild. `slab_tail` cannot answer that -- it is an absolute file offset,
/// so it jumps forward whenever a fresh extent is taken.
slab_used: u64,
/// This collection's outstanding page promise, for the document slab. Per
/// collection because concurrent writers must not release each other's --
/// see `pager.Reservation`.
hold: pgr.Reservation,
/// Of those bytes, the ones still reachable. `slab_used - live_bytes` is
/// this collection's slab garbage, which only a rebuild reclaims. Kept per
/// collection so dropping one can move the right amount from the engine's
@@ -91,6 +95,7 @@ pub const Collection = struct {
.slab_end = 0,
.slab_used = 0,
.live_bytes = 0,
.hold = .{},
.indexes = .empty,
.id_index = undefined,
};
@@ -140,8 +145,8 @@ pub const Collection = struct {
slab_extent_pages,
(len + pgr.page_size - 1) / pgr.page_size,
));
try self.pager.reserve_pages(want_pages);
const first = self.pager.alloc_pages_assume_reserved(want_pages);
try self.pager.reserve_pages(&self.hold, want_pages);
const first = self.pager.alloc_pages_assume_reserved(&self.hold, want_pages);
try self.slab_extents.append(gpa, .{ .first = first, .pages = want_pages });
self.slab_tail = @as(u64, first) << pgr.page_shift;
self.slab_end = self.slab_tail + (@as(u64, want_pages) << pgr.page_shift);
@@ -261,6 +266,10 @@ pub const Engine = struct {
/// 16 KiB documents and one of 40 B documents look identical.
live_bytes: u64 = 0,
dead_bytes: u64 = 0,
/// The checkpoint's own page promise, for the catalog and free-list pages it
/// writes. Separate from any collection's for the same reason those are
/// separate from each other.
hold: pgr.Reservation = .{},
/// Set when the log has grown enough since the last checkpoint to be worth
/// reclaiming. Read by the write epilogue and the TTL monitor, both of which
/// run without holding a collection lock.
@@ -420,6 +429,15 @@ pub const Engine = struct {
/// own total (a rebuild appends through it too, and its copies are live by
/// definition); the engine's total only moves when a document actually
/// becomes live, which a rebuild's copies do not.
/// Drop the unclaimed part of every promise a write to this collection took:
/// the slab's and one per index. Called after the write is published, under
/// the same collection lock the reservations were taken under.
fn release_write_reservations(self: *Engine, coll: *Collection) void {
self.pager.release_reservation(&coll.hold);
self.pager.release_reservation(&coll.id_index.hold);
for (coll.indexes.items) |ix| self.pager.release_reservation(&ix.hold);
}
fn publish_doc_bytes(self: *Engine, coll: *Collection, bytes: []const u8) u64 {
const off = coll.slab_append(bytes);
self.live_bytes += bytes.len;
@@ -794,8 +812,10 @@ pub const Engine = struct {
b.ix.insert_entries(&b.built, off);
}
// The write is published; anything the reservations above did not claim
// is dead. Leaving it promised would grow the file on every write.
self.pager.release_reservation();
// is dead. Leaving it promised would grow the file on every write. Every
// consumer this upsert reserved through, and only those: another
// collection may be mid-write on another thread.
self.release_write_reservations(coll);
self.note_compact();
self.note_checkpoint();
}
@@ -919,7 +939,7 @@ pub const Engine = struct {
try ix.append_doc_entries(self.gpa, coll.doc_bytes(entry.off), entry.off);
}
_ = try ix.finish_bulk(self.gpa, true);
self.pager.release_reservation();
self.pager.release_reservation(&ix.hold);
// Reserve the collection slot, then persist and publish.
try coll.indexes.ensureUnusedCapacity(self.gpa, 1);
@@ -1284,7 +1304,7 @@ pub const Engine = struct {
const bytes = doc_bytes_in(self.pager, entry.off);
try coll.slab_reserve(self.gpa, bytes.len);
const new_off = coll.slab_append(bytes);
self.pager.release_reservation();
self.pager.release_reservation(&coll.hold);
try moved.append(self.gpa, .{ .off = new_off });
}
// Republish the offsets.
@@ -1313,7 +1333,7 @@ pub const Engine = struct {
// Duplicates are tolerated here for the same reason they are on open:
// refusing would make a maintenance task able to take the database down.
_ = ix.finish_bulk(self.gpa, false) catch |err| return err;
self.pager.release_reservation();
self.pager.release_reservation(&ix.hold);
}
/// Re-emit one collection's index specs and documents into the compacted
@@ -1367,7 +1387,7 @@ pub const Engine = struct {
};
}
// Tolerated, not enforced: the database must always open.
defer self.pager.release_reservation();
defer self.pager.release_reservation(&ix.hold);
if (try ix.finish_bulk(self.gpa, false)) {
std.debug.print(
"multiforadb: WARNING: unique index '{s}' has duplicate keys in existing " ++
@@ -1710,7 +1730,7 @@ pub const Engine = struct {
self.log_lock.unlock(self.io);
return err;
};
self.pager.release_reservation();
self.pager.release_reservation(&self.hold);
// The watermark is durable, so every record it covers is now
// redundant. Strictly after the publish: the other order loses data
// if a crash lands between them.
@@ -1844,7 +1864,7 @@ fn apply_record(ctx: *anyopaque, record: storage.Record, doc: *bson.Document) an
} else {
self.index_one(&coll.id_index, doc_bytes, off);
}
self.pager.release_reservation();
self.release_write_reservations(coll);
// The _id_ entry is added after replay, in build_all_indexes,
// together with the secondary indexes.
//