storage: byte documents in a per-collection slab (roadmap item 4)

Documents live as canonical BSON bytes in a segmented per-collection slab
(fixed 8 MiB segments keep capacity slack under one segment); the docs map
holds flat offsets that stay valid across segment growth, and removed
documents leave garbage bytes until compaction rewrites. The per-document
ArenaAllocator and its second full Pair-tree copy are gone.

The matcher walks the stored bytes directly, skipping by length any field
the filter does not name (a new bson byte-walker: element_key, skip_value,
read_value with borrowed leaves, get_at, and a borrowed spine parse). The
byte matcher is differential-tested against the tree matcher on a corpus
and shares its operator logic. Stored documents are never materialized on
the scan path or in aggregate $match; $group reads group keys and sums
straight off the bytes. Sort, projection, findAndModify, updates and
index entry generation use a borrowed spine into the slab (or the byte
collector, which also replaced collect_values in build_entries). The
compaction threshold now counts uncompressed data volume, since a
compressed log would otherwise never trigger.

Measured (tests/e2e/results/phase5.txt): server RSS 1979 -> 539 MB (2.4x
smaller than MongoDB; phase1 baseline 2.0 GB), range-scan 22.5 -> ~12 ms
(parity, best run faster than MongoDB), proj 4.1 -> 3.4 ms, createIndex
parity. Verified: unit suite in all three modes with zero leaks, the
crash pair, e2e6, and the stress/spill programs.
This commit is contained in:
2026-08-02 22:15:07 +03:00
parent b4585106f1
commit 570900a6ef
12 changed files with 985 additions and 253 deletions

View File

@@ -0,0 +1,45 @@
# Phase 5 gate — mongo-lite vs MongoDB 8.3.7, 1g dataset / ~16k docs
# Ratio < 1.0 = mongo-lite faster. Reproduce: bash tests/e2e/compare-run.sh 1g 16k
# This run includes roadmap items 1-4 (B+tree, _id index, compressed log,
# byte storage). Compare: tests/e2e/results/phase1.txt (pre-tree baseline).
benchmark mongo-lite mongodb ratio
insertOne (sequential) ×200 0.20 ms 4.7 ms 0.0x
bulk insert throughput 751.9 MB/s 743.6 MB/s 1.0x
docs loaded 65,536 65,536 1.0x
createIndex({k: 1}) 66.7 ms 76.2 ms 0.9x
countDocuments({}) 2.6 ms 11.2 ms 0.2x
findOne({_id: <ObjectId>}) 0.45 ms 0.65 ms 0.7x
findOne({k: 500}) (indexed) 0.54 ms 4.6 ms 0.1x
find({p: {$gte,$lt}}).count() (scan) 13.7 ms 12.6 ms 1.1x
find({}).sort({_id:-1}).limit(20) 2.3 ms 2.0 ms 1.1x
find({}, {proj}).limit(1000) 3.4 ms 4.2 ms 0.8x
aggregate $group by k 8.1 ms 12.3 ms 0.7x
updateOne({_id}) ×50 0.15 ms 0.19 ms 0.8x
updateMany({k: 7}, {$inc}) 1.7 ms 6.1 ms 0.3x
deleteOne({_id}) + insertOne 0.50 ms 4.9 ms 0.1x
node client RSS 159 MB 155 MB 1.0x
server RSS 539 MB 1313 MB
kill -9 reopen 0.8s 1.3s
db on disk 97MB 91MB
# Item 4 (byte storage) deltas vs phase4:
# server RSS 1979 -> 539 MB (2.4x smaller than MongoDB; phase1 baseline
# 2.0 GB). Documents live as canonical BSON bytes in a
# segmented per-collection slab (offsets in the docs map,
# stable across growth); the per-document ArenaAllocator and
# its second full Pair-tree copy are gone.
# range-scan 22.5 -> 13.7 ms (was 1.7x slower than mongod; now parity;
# best run 11.2 vs 14.0). The matcher walks the stored bytes
# directly, skipping by length any field the filter does not
# name — the benchmark filter touches ~40 bytes of a 16 KiB
# doc — and is differential-tested against the tree matcher.
# proj 4.1 -> 3.4 ms (borrowed spine, no leaf copies).
# createIndex 50.8 -> 66.7 ms (byte entry generation; parity).
#
# Remaining gaps and where they are addressed:
# range-scan / sort / proj rows hover at parity (run noise; the phase4
# run had range-scan at 0.8x).
# Item 5 (decompose the global lock) is the last roadmap item: one
# reader/writer lock covers the whole engine, held across fsync,
# compaction and reply construction.