Aleksey Shakhmatov
ac464f2b92
compare-run.sh answers "how do we compare to MongoDB"; it says nothing
about whether a change made things better or worse than last week. Add a
harness that records each run and diffs it against the previous one.
bench-run.sh wraps compare-run.sh, adds a concurrent durable-write
comparison (concurrent.js: N clients each doing sequential insertOne with
{w:1, j:true}, exercising the group-commit path under real contention),
writes a versioned name<TAB>value report to results/bench-<timestamp>.txt,
and prints a diff of our numbers against results/bench-latest.txt.
Also:
- compare-run.sh polled with fixed sleeps, which are flaky once earlier
benchmark phases have warmed the machine; both servers now wait on a
real driver connection instead.
- a dispatch error only reached the client as a generic InternalError,
with nothing on the server side naming the failing command; log the
connection, command and error name before replacing the reply.
73 lines
4.7 KiB
Plaintext
73 lines
4.7 KiB
Plaintext
# Phase 8 gate — mongo-lite vs MongoDB 8.3.7, 1g dataset / ~16k docs
|
||
# Ratio < 1.0 = mongo-lite faster. Reproduce: bash tests/e2e/bench-run.sh 1g 16k
|
||
# Working tree: c8d547f + uncommitted diff (epilogue-commit/group-commit work) +
|
||
# three bugs found and fixed while benchmarking (details at the bottom).
|
||
# Machine-readable copy: tests/e2e/results/bench-latest.txt (auto-diffed on
|
||
# every bench-run.sh invocation).
|
||
|
||
benchmark mongo-lite mongodb ratio
|
||
insertOne (sequential) ×200 0.20 ms 14.8 ms 0.0x
|
||
bulk insert throughput 732.4 MB/s 546.2 MB/s 1.3x
|
||
docs loaded 65,536 65,536 1.0x
|
||
createIndex({k: 1}) 74.4 ms 83.6 ms 0.9x
|
||
countDocuments({}) 3.1 ms 12.4 ms 0.3x
|
||
findOne({_id: <ObjectId>}) 0.59 ms 0.61 ms 1.0x
|
||
findOne({k: 500}) (indexed) 0.56 ms 0.94 ms 0.6x
|
||
find({p: {$gte,$lt}}).count() (scan) 13.0 ms 15.0 ms 0.9x
|
||
find({}).sort({_id:-1}).limit(20) 2.2 ms 2.2 ms 1.0x
|
||
find({}, {proj}).limit(1000) 3.5 ms 4.4 ms 0.8x
|
||
aggregate $group by k 8.3 ms 13.2 ms 0.6x
|
||
updateOne({_id}) ×50 0.14 ms 0.20 ms 0.7x
|
||
updateMany({k: 7}, {$inc}) 1.9 ms 6.4 ms 0.3x
|
||
deleteOne({_id}) + insertOne 0.58 ms 4.9 ms 0.1x
|
||
node client RSS 158 MB 157 MB 1.0x
|
||
server RSS 553 MB 1486 MB
|
||
kill -9 reopen 0.8s 1.3s
|
||
db on disk 97MB 106MB
|
||
|
||
# NEW: concurrent durable writes (sequential insertOne per client, {w:1, j:true})
|
||
# through the official driver, both servers, same workload:
|
||
# clients | mongodb | mongo-lite | ratio
|
||
# 1 | 232 | 8,435 | 36x
|
||
# 4 | 491 | 22,480 | 46x
|
||
# 8 | 966 | 25,920 | 27x
|
||
# 16 | 1,842 | 31,079 | 17x
|
||
# 32 | 3,632 | 34,041 | 9x
|
||
# (mongo-lite scales to ~34k durable docs/s at 32 clients; mongod tops out
|
||
# around 3.6k on this machine's fsync-per-write path.)
|
||
|
||
# THREE BUGS FOUND AND FIXED WHILE BENCHMARKING
|
||
# All three were invisible to the single-connection compare.js suite and only
|
||
# surfaced under the new concurrent harness (tests/e2e/concurrent.js).
|
||
#
|
||
# 1. Group-commit deadlock (>=6 concurrent writers froze permanently).
|
||
# std.Io.Condition.signal() delivers exactly ONE wakeup. The commit
|
||
# leader's completion woke a single follower on `while (committing)` and
|
||
# stranded the rest; an append's signal could also be consumed by a
|
||
# follower, stranding the leader on the pending_appends drain. Signals
|
||
# were also sent without commit_lock, leaving a lost-wakeup window
|
||
# between a waiter's epoch snapshot and its futex sleep.
|
||
# Fix (src/db.zig): all commit_done signals hold commit_lock and use
|
||
# broadcast(). Reproduced on HEAD too (introduced by the group-commit
|
||
# work, not the uncommitted diff).
|
||
#
|
||
# 2. Racy assertion in commit(): assert(pending_appends == 0) after the
|
||
# drain loop. A NEW append can start at any moment (the increment takes
|
||
# no commit_lock), so the assert fired spuriously under load and crashed
|
||
# the server (32 concurrent clients, "commit would seal while appends
|
||
# are still in flight"). The seal's real invariant holds via log_lock
|
||
# ordering; the instant check was wrong. Fix: removed the assert.
|
||
#
|
||
# 3. Slab segment-start corruption in Collection.slab_append: a pointer
|
||
# `last` into slab.items dangled across the slab.append() that starts the
|
||
# next segment (the list can reallocate), so the new segment's start
|
||
# offset was computed from freed memory. Docs in the affected segment
|
||
# were then located at wrong offsets, surfacing as InvalidBson on reads
|
||
# with a batch limit >~50 (or a crash in Debug builds). Position varied
|
||
# between runs (heap contents vary). Fix: read the last segment's length
|
||
# as a value before the append.
|
||
#
|
||
# Verification after the fixes: unit suite green, e2e6 72/72 (incl. kill -9
|
||
# mid-write recovery), crash pair, concurrent 8-client, indexes, TTL all
|
||
# green; concurrent stress 1/4/8/16/32 clients x5 sequential rounds clean.
|