dev/MultiforaDB
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

index: B+tree over the encoded keys (roadmap item 1)

Browse Source 
Replace Index.entries (one sorted array) with a B+tree so writes into an
already-built index stop being quadratic. Nodes are fixed 4 KiB slotted
pages in a flat u32-addressed ArrayListUnmanaged(Node); records longer
than a quarter page spill to an append-only overflow slab (BSON strings
reach 16 MB). Leaves are doubly linked for ordered iteration; the flat
node array stays one contiguous byte range for a later checkpoint.

Insertion descends by separator and splits leaves/internals upward,
promoting keys via a stable copy (a nested split can otherwise clobber
the promoted-key scratch). Deletion does not rebalance: emptied leaves
are unlinked and dropped from their parent, internal nodes may carry one
child, and dead pages are abandoned in place (node memory peaks at the
tree's peak size, exactly what the old array's capacity did). Lookups
are lower-bound seeks plus leaf-chain band scans, so equal keys may
span leaves freely. Bulk build (append_doc_entries + finish_bulk) sorts
a staging array and packs leaves bottom-up. reserve_for now takes the
built entries and reserves exact overflow bytes plus a worst-case node
count, keeping insert_entries infallible after the log append.

db.zig: TTL sweep now seeks the minimum-datetime encoded key and walks
the contiguous datetime band, stopping at the cutoff or type change.

Measured (tests/e2e/results/phase2.txt): updateMany 17.3 -> 1.8 ms
(2.8x slower than MongoDB -> 3.7x faster), createIndex 62 -> 51 ms.

Verified: unit suite ReleaseFast/ReleaseSafe/Debug (incl. the existing
lookup_range and remove_doc differentials, plus a new incremental
insert/remove differential against a brute-force model), the crash pair,
e2e3/e2e4/e2e6, and dev stress tests for depth-2 splits, full drains,
and spilled records through internal levels.
This commit is contained in:
Alexey Shakhmatov
2026-08-02 21:10:25 +03:00
parent 71112b0ff7
commit 61fe952125
8 changed files with 1450 additions and 218 deletions
Download Patch File Download Diff File Expand all files Collapse all files

38
tests/e2e/results/phase2.txt Normal file
View File

@@ -0,0 +1,38 @@
# Phase 2 gate — mongo-lite vs MongoDB 8.3.7, 1g dataset / ~16k docs
# Ratio < 1.0 = mongo-lite faster. Reproduce: bash tests/e2e/compare-run.sh 1g 16k
# This run includes roadmap item 1 (B+tree over the encoded keys).
# Compare: tests/e2e/results/phase1.txt (pre-tree baseline).
benchmark mongo-lite mongodb ratio
insertOne (sequential) ×200 0.19 ms 4.1 ms 0.0x
bulk insert throughput 810.4 MB/s 714.4 MB/s 1.1x
docs loaded 65,536 65,536 1.0x
createIndex({k: 1}) 50.8 ms 82.4 ms 0.6x
countDocuments({}) 1.5 ms 13.8 ms 0.1x
findOne({_id: <ObjectId>}) 0.57 ms 0.67 ms 0.9x
findOne({k: 500}) (indexed) 0.57 ms 1.8 ms 0.3x
find({p: {$gte,$lt}}).count() (scan) 20.3 ms 12.9 ms 1.6x
find({}).sort({_id:-1}).limit(20) 6.2 ms 2.7 ms 2.3x
find({}, {proj}).limit(1000) 3.7 ms 4.5 ms 0.8x
aggregate $group by k 11.5 ms 15.5 ms 0.7x
updateOne({_id}) ×50 0.17 ms 0.19 ms 0.9x
updateMany({k: 7}, {$inc}) 1.8 ms 6.7 ms 0.3x
deleteOne({_id}) + insertOne 0.50 ms 5.0 ms 0.1x
node client RSS 152 MB 156 MB 1.0x
server RSS 1974 MB 1474 MB
kill -9 reopen 0.8s 1.3s
db on disk 1028MB 96MB
# Item 1 (B+tree over encoded keys) deltas vs phase1:
# updateMany 17.3 -> 1.8 ms (2.8x slower than mongod -> 3.7x faster):
# entry removal was a per-entry binary search into a sorted
# array with an orderedRemove memmove behind it; now it is a
# descent plus a leaf-local slot removal.
# createIndex 62.4 -> 50.8 ms (bulk packing replaces append+sort)
# findOne(k:500) indexed 0.64 -> 0.57 ms (unchanged shape, tree search)
#
# Remaining gaps and where they are addressed:
# db on disk 11x -> Phase 3 (block-compressed log)
# sort+limit 2.3x -> Phase 2 (ordered _id index)
# range-scan 1.6x -> Phase 4 (contiguous byte storage, not the matcher)
# server RSS 1.3x -> Phase 4 (per-document arena -> byte storage)