Aleksey Shakhmatov
61fe952125
Replace Index.entries (one sorted array) with a B+tree so writes into an already-built index stop being quadratic. Nodes are fixed 4 KiB slotted pages in a flat u32-addressed ArrayListUnmanaged(Node); records longer than a quarter page spill to an append-only overflow slab (BSON strings reach 16 MB). Leaves are doubly linked for ordered iteration; the flat node array stays one contiguous byte range for a later checkpoint. Insertion descends by separator and splits leaves/internals upward, promoting keys via a stable copy (a nested split can otherwise clobber the promoted-key scratch). Deletion does not rebalance: emptied leaves are unlinked and dropped from their parent, internal nodes may carry one child, and dead pages are abandoned in place (node memory peaks at the tree's peak size, exactly what the old array's capacity did). Lookups are lower-bound seeks plus leaf-chain band scans, so equal keys may span leaves freely. Bulk build (append_doc_entries + finish_bulk) sorts a staging array and packs leaves bottom-up. reserve_for now takes the built entries and reserves exact overflow bytes plus a worst-case node count, keeping insert_entries infallible after the log append. db.zig: TTL sweep now seeks the minimum-datetime encoded key and walks the contiguous datetime band, stopping at the cutoff or type change. Measured (tests/e2e/results/phase2.txt): updateMany 17.3 -> 1.8 ms (2.8x slower than MongoDB -> 3.7x faster), createIndex 62 -> 51 ms. Verified: unit suite ReleaseFast/ReleaseSafe/Debug (incl. the existing lookup_range and remove_doc differentials, plus a new incremental insert/remove differential against a brute-force model), the crash pair, e2e3/e2e4/e2e6, and dev stress tests for depth-2 splits, full drains, and spilled records through internal levels.
39 lines
2.8 KiB
Plaintext
39 lines
2.8 KiB
Plaintext
# Phase 2 gate — mongo-lite vs MongoDB 8.3.7, 1g dataset / ~16k docs
|
||
# Ratio < 1.0 = mongo-lite faster. Reproduce: bash tests/e2e/compare-run.sh 1g 16k
|
||
# This run includes roadmap item 1 (B+tree over the encoded keys).
|
||
# Compare: tests/e2e/results/phase1.txt (pre-tree baseline).
|
||
|
||
benchmark mongo-lite mongodb ratio
|
||
insertOne (sequential) ×200 0.19 ms 4.1 ms 0.0x
|
||
bulk insert throughput 810.4 MB/s 714.4 MB/s 1.1x
|
||
docs loaded 65,536 65,536 1.0x
|
||
createIndex({k: 1}) 50.8 ms 82.4 ms 0.6x
|
||
countDocuments({}) 1.5 ms 13.8 ms 0.1x
|
||
findOne({_id: <ObjectId>}) 0.57 ms 0.67 ms 0.9x
|
||
findOne({k: 500}) (indexed) 0.57 ms 1.8 ms 0.3x
|
||
find({p: {$gte,$lt}}).count() (scan) 20.3 ms 12.9 ms 1.6x
|
||
find({}).sort({_id:-1}).limit(20) 6.2 ms 2.7 ms 2.3x
|
||
find({}, {proj}).limit(1000) 3.7 ms 4.5 ms 0.8x
|
||
aggregate $group by k 11.5 ms 15.5 ms 0.7x
|
||
updateOne({_id}) ×50 0.17 ms 0.19 ms 0.9x
|
||
updateMany({k: 7}, {$inc}) 1.8 ms 6.7 ms 0.3x
|
||
deleteOne({_id}) + insertOne 0.50 ms 5.0 ms 0.1x
|
||
node client RSS 152 MB 156 MB 1.0x
|
||
server RSS 1974 MB 1474 MB
|
||
kill -9 reopen 0.8s 1.3s
|
||
db on disk 1028MB 96MB
|
||
|
||
# Item 1 (B+tree over encoded keys) deltas vs phase1:
|
||
# updateMany 17.3 -> 1.8 ms (2.8x slower than mongod -> 3.7x faster):
|
||
# entry removal was a per-entry binary search into a sorted
|
||
# array with an orderedRemove memmove behind it; now it is a
|
||
# descent plus a leaf-local slot removal.
|
||
# createIndex 62.4 -> 50.8 ms (bulk packing replaces append+sort)
|
||
# findOne(k:500) indexed 0.64 -> 0.57 ms (unchanged shape, tree search)
|
||
#
|
||
# Remaining gaps and where they are addressed:
|
||
# db on disk 11x -> Phase 3 (block-compressed log)
|
||
# sort+limit 2.3x -> Phase 2 (ordered _id index)
|
||
# range-scan 1.6x -> Phase 4 (contiguous byte storage, not the matcher)
|
||
# server RSS 1.3x -> Phase 4 (per-document arena -> byte storage)
|