storage: byte documents in a per-collection slab (roadmap item 4)

Documents live as canonical BSON bytes in a segmented per-collection slab
(fixed 8 MiB segments keep capacity slack under one segment); the docs map
holds flat offsets that stay valid across segment growth, and removed
documents leave garbage bytes until compaction rewrites. The per-document
ArenaAllocator and its second full Pair-tree copy are gone.

The matcher walks the stored bytes directly, skipping by length any field
the filter does not name (a new bson byte-walker: element_key, skip_value,
read_value with borrowed leaves, get_at, and a borrowed spine parse). The
byte matcher is differential-tested against the tree matcher on a corpus
and shares its operator logic. Stored documents are never materialized on
the scan path or in aggregate $match; $group reads group keys and sums
straight off the bytes. Sort, projection, findAndModify, updates and
index entry generation use a borrowed spine into the slab (or the byte
collector, which also replaced collect_values in build_entries). The
compaction threshold now counts uncompressed data volume, since a
compressed log would otherwise never trigger.

Measured (tests/e2e/results/phase5.txt): server RSS 1979 -> 539 MB (2.4x
smaller than MongoDB; phase1 baseline 2.0 GB), range-scan 22.5 -> ~12 ms
(parity, best run faster than MongoDB), proj 4.1 -> 3.4 ms, createIndex
parity. Verified: unit suite in all three modes with zero leaks, the
crash pair, e2e6, and the stress/spill programs.
This commit is contained in:
2026-08-02 22:15:07 +03:00
parent b4585106f1
commit 570900a6ef
12 changed files with 985 additions and 253 deletions

View File

@@ -102,6 +102,10 @@ pub const Log = struct {
path: []const u8,
end_pos: u64,
log_bytes: u64, // bytes written since the log was last rewritten
/// Uncompressed record bytes appended since the log was last rewritten —
/// the data volume, which the compaction threshold is really about (the
/// on-disk size shrinks with compression and would under-trigger).
data_bytes: u64 = 0,
codec: u8,
// Reused record-framing buffer. Appends are single-writer (the engine's
// exclusive lock), so one buffer avoids a realloc cycle per record.
@@ -224,7 +228,6 @@ pub const Log = struct {
// runs on input already proven intact.
const file_len = self.file.length(self.io) catch return error.InvalidLog;
if (pos + total >= file_len) return;
std.debug.print("mongo-lite: corrupt block hash at {d}\n", .{pos});
return error.InvalidLog;
}
@@ -331,6 +334,7 @@ pub const Log = struct {
std.mem.writeInt(u32, buf.items[0..4], total, .little);
std.mem.writeInt(u64, buf.items[4..12], record_hash(buf.items[12..]), .little);
self.data_bytes += buf.items.len;
// Seal the current block when the next record would push it past the
// target; a single oversized record keeps its own block.
if (self.block.items.len > 0 and self.block.items.len + buf.items.len > block_target) {