tests/e2e: iteration-to-iteration benchmark harness

compare-run.sh answers "how do we compare to MongoDB"; it says nothing
about whether a change made things better or worse than last week. Add a
harness that records each run and diffs it against the previous one.

bench-run.sh wraps compare-run.sh, adds a concurrent durable-write
comparison (concurrent.js: N clients each doing sequential insertOne with
{w:1, j:true}, exercising the group-commit path under real contention),
writes a versioned name<TAB>value report to results/bench-<timestamp>.txt,
and prints a diff of our numbers against results/bench-latest.txt.

Also:
- compare-run.sh polled with fixed sleeps, which are flaky once earlier
  benchmark phases have warmed the machine; both servers now wait on a
  real driver connection instead.
- a dispatch error only reached the client as a generic InternalError,
  with nothing on the server side naming the failing command; log the
  connection, command and error name before replacing the reply.
This commit is contained in:
2026-08-03 12:33:28 +03:00
parent 720540860a
commit ac464f2b92
10 changed files with 441 additions and 3 deletions

View File

@@ -28,7 +28,17 @@ echo; echo "### mongod (MongoDB $(mongod --version | grep -oE 'v[0-9.]+' | head
mongod --dbpath "$CMPDIR/mongod" --port $MD_PORT --bind_ip 127.0.0.1 \
--quiet >"$MD_OUT" 2>&1 &
MD_PID=$!
sleep 2
# Poll until mongod answers; a fixed sleep is flaky right after other
# benchmark phases have warmed the machine.
wait_ready() { # $1 = url
for _ in $(seq 1 90); do
NODE_PATH="tests/e2e/node_modules" node -e "require('mongodb').MongoClient.connect(process.argv[1],{serverSelectionTimeoutMS:800}).then(c=>c.close().then(()=>process.exit(0))).catch(()=>process.exit(1))" "$1" 2>/dev/null \
&& return 0
sleep 1
done
return 1
}
wait_ready "mongodb://127.0.0.1:$MD_PORT" || { echo "mongod never became ready" >&2; exit 1; }
node tests/e2e/compare.js --url "mongodb://127.0.0.1:$MD_PORT" --label mongodb --size "$SIZE" --doc-size "$DOC" \
> "$CMPDIR/mongo-report.txt" 2>&1 || { echo "mongodb bench failed:"; tail -5 "$CMPDIR/mongo-report.txt"; }
MD_RSS=$(ps -o rss= -p $MD_PID | awk '{printf "%.0f", $1/1024}')
@@ -57,7 +67,7 @@ echo; echo "### mongo-lite (recommended config: --compact-threshold 1g)"
zig build -Doptimize=ReleaseFast 2>&1 | grep -c "^error" | grep -q "^0" || { echo "build failed"; exit 1; }
./zig-out/bin/mongo-lite --port $ML_PORT --db "$ML_LOG" --compact-threshold 1g >"$ML_OUT" 2>&1 &
ML_PID=$!
sleep 1
wait_ready "mongodb://127.0.0.1:$ML_PORT" || { echo "mongo-lite never became ready" >&2; exit 1; }
node tests/e2e/compare.js --url "mongodb://127.0.0.1:$ML_PORT" --label mongo-lite --size "$SIZE" --doc-size "$DOC" \
> "$CMPDIR/ml-report.txt" 2>&1 || { echo "mongo-lite bench failed:"; tail -5 "$CMPDIR/ml-report.txt"; }
ML_RSS=$(ps -o rss= -p $ML_PID | awk '{printf "%.0f", $1/1024}')