tests/e2e: iteration-to-iteration benchmark harness
compare-run.sh answers "how do we compare to MongoDB"; it says nothing
about whether a change made things better or worse than last week. Add a
harness that records each run and diffs it against the previous one.
bench-run.sh wraps compare-run.sh, adds a concurrent durable-write
comparison (concurrent.js: N clients each doing sequential insertOne with
{w:1, j:true}, exercising the group-commit path under real contention),
writes a versioned name<TAB>value report to results/bench-<timestamp>.txt,
and prints a diff of our numbers against results/bench-latest.txt.
Also:
- compare-run.sh polled with fixed sleeps, which are flaky once earlier
benchmark phases have warmed the machine; both servers now wait on a
real driver connection instead.
- a dispatch error only reached the client as a generic InternalError,
with nothing on the server side naming the failing command; log the
connection, command and error name before replacing the reply.
This commit is contained in:
@@ -143,9 +143,10 @@ fn handle_connection_inner(io: std.Io, stream: std.Io.net.Stream, server: *Serve
|
|||||||
|
|
||||||
reply.reset();
|
reply.reset();
|
||||||
|
|
||||||
commands.dispatch(&ctx, &msg, &reply) catch {
|
commands.dispatch(&ctx, &msg, &reply) catch |err| {
|
||||||
// Discard any partial reply (the client would read the first
|
// Discard any partial reply (the client would read the first
|
||||||
// ok field, which may already say 1) and send a clean error.
|
// ok field, which may already say 1) and send a clean error.
|
||||||
|
std.debug.print("mongo-lite: dispatch error on conn {d} cmd {s}: {s}\n", .{ connection_id, msg.command_name(), @errorName(err) });
|
||||||
reply.pairs.clearRetainingCapacity();
|
reply.pairs.clearRetainingCapacity();
|
||||||
reply.put_error(
|
reply.put_error(
|
||||||
@intFromEnum(commands.ErrorCode.internal_error),
|
@intFromEnum(commands.ErrorCode.internal_error),
|
||||||
|
|||||||
@@ -97,3 +97,19 @@ on :27019, runs the same driver workload against each (durable writes:
|
|||||||
mongo-lite fsyncs per command, mongod runs with `j: true`), measures kill -9
|
mongo-lite fsyncs per command, mongod runs with `j: true`), measures kill -9
|
||||||
reopen for both, and prints a side-by-side table. `compare.js` alone runs
|
reopen for both, and prints a side-by-side table. `compare.js` alone runs
|
||||||
one side (see its `--help`-style header comment).
|
one side (see its `--help`-style header comment).
|
||||||
|
|
||||||
|
### Iteration-to-iteration comparison: `bench-run.sh` + `concurrent.js`
|
||||||
|
|
||||||
|
```sh
|
||||||
|
bash tests/e2e/bench-run.sh [size] [doc-size] ["clients..."] # e.g. 1g 16k "1 4 8 16 32"
|
||||||
|
```
|
||||||
|
|
||||||
|
Runs the main suite (`compare-run.sh`) plus a concurrent durable-write
|
||||||
|
comparison (`concurrent.js`, N clients each doing sequential `insertOne`
|
||||||
|
with `{w:1, j:true}` — the group-commit path under real contention), then
|
||||||
|
writes a machine-readable, versioned report to
|
||||||
|
`tests/e2e/results/bench-<timestamp>.txt` and prints a diff of the
|
||||||
|
mongo-lite numbers against the previous run (`results/bench-latest.txt`).
|
||||||
|
The report has `[main]` / `[concurrency]` / `[meta]` sections with
|
||||||
|
`name<TAB>value` rows; `bench-run.sh 1g 16k` reproduces the phase8 gate
|
||||||
|
(see `results/phase8.txt`).
|
||||||
|
|||||||
152
tests/e2e/bench-run.sh
Normal file
152
tests/e2e/bench-run.sh
Normal file
@@ -0,0 +1,152 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
# Full benchmark run, saved for iteration-to-iteration comparison.
|
||||||
|
#
|
||||||
|
# bash tests/e2e/bench-run.sh [size] [doc-size] [clients...]
|
||||||
|
# (defaults: 1g, 16k, "1 4 8 16 32"; clients apply to the concurrent phase)
|
||||||
|
#
|
||||||
|
# Runs the compare-run.sh main suite (mongo-lite vs mongod, same driver,
|
||||||
|
# durable writes) plus the concurrent durable-write comparison, then writes
|
||||||
|
# a versioned machine-readable report to tests/e2e/results/ and prints a
|
||||||
|
# diff of the mongo-lite numbers against the previous run.
|
||||||
|
set -u
|
||||||
|
cd "$(dirname "$0")/../.."
|
||||||
|
SIZE="${1:-1g}"; DOC="${2:-16k}"; CLIENTS="${3:-1 4 8 16 32}"
|
||||||
|
PER_CLIENT=2000
|
||||||
|
RESDIR=tests/e2e/results
|
||||||
|
mkdir -p "$RESDIR"
|
||||||
|
NOW=$(date +%Y%m%d-%H%M%S)
|
||||||
|
REV=$(git rev-parse --short HEAD 2>/dev/null || echo unknown)
|
||||||
|
DIRTY=$(git status --porcelain | grep -q . && echo "+dirty" || echo "")
|
||||||
|
REPORT="$RESDIR/bench-$NOW.txt"
|
||||||
|
LATEST="$RESDIR/bench-latest.txt"
|
||||||
|
TMP=$(mktemp -d /tmp/bench-run.XXXXXX)
|
||||||
|
trap 'rm -rf "$TMP"' EXIT
|
||||||
|
|
||||||
|
# ---- main suite ----------------------------------------------------------
|
||||||
|
bash tests/e2e/compare-run.sh "$SIZE" "$DOC" > "$TMP/compare.out" 2>&1
|
||||||
|
if [ $? -ne 0 ]; then
|
||||||
|
echo "main suite failed — see $TMP/compare.out" >&2
|
||||||
|
tail -20 "$TMP/compare.out" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# ---- concurrent durable writes --------------------------------------------
|
||||||
|
echo "### concurrent durable insertOne (clients: $CLIENTS)" >&2
|
||||||
|
rm -rf "$TMP/mongod" && mkdir -p "$TMP/mongod"
|
||||||
|
mongod --dbpath "$TMP/mongod" --port 27018 --bind_ip 127.0.0.1 --quiet >"$TMP/md.out" 2>&1 &
|
||||||
|
MD_PID=$!
|
||||||
|
./zig-out/bin/mongo-lite --port 27019 --db "$TMP/ml.log" --compact-threshold 1g >"$TMP/ml.out" 2>&1 &
|
||||||
|
ML_PID=$!
|
||||||
|
# Poll both servers with the real driver until they answer (fresh mongod
|
||||||
|
# dbpaths can take several seconds; a fixed sleep is flaky).
|
||||||
|
wait_ready() { # $1 = url
|
||||||
|
for _ in $(seq 1 90); do
|
||||||
|
NODE_PATH="tests/e2e/node_modules" node -e "require('mongodb').MongoClient.connect(process.argv[1],{serverSelectionTimeoutMS:800}).then(c=>c.close().then(()=>process.exit(0))).catch(()=>process.exit(1))" "$1" 2>/dev/null \
|
||||||
|
&& return 0
|
||||||
|
sleep 1
|
||||||
|
done
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
wait_ready mongodb://127.0.0.1:27018 || { echo "mongod never became ready" >&2; kill -9 $MD_PID $ML_PID 2>/dev/null; exit 1; }
|
||||||
|
wait_ready mongodb://127.0.0.1:27019 || { echo "mongo-lite never became ready" >&2; kill -9 $MD_PID $ML_PID 2>/dev/null; exit 1; }
|
||||||
|
CC="$TMP/cc.txt"
|
||||||
|
for C in $CLIENTS; do
|
||||||
|
MD=$(node tests/e2e/concurrent.js --url mongodb://127.0.0.1:27018 --label mongodb --clients "$C" --per-client "$PER_CLIENT" 2>/dev/null | tail -1)
|
||||||
|
ML=$(node tests/e2e/concurrent.js --url mongodb://127.0.0.1:27019 --label mongo-lite --clients "$C" --per-client "$PER_CLIENT" 2>/dev/null | tail -1)
|
||||||
|
ML_D=$(echo "$ML" | sed -E 's/.*\t([0-9.]+) docs\/s.*/\1/')
|
||||||
|
MD_D=$(echo "$MD" | sed -E 's/.*\t([0-9.]+) docs\/s.*/\1/')
|
||||||
|
if [ -z "$ML_D" ] || [ -z "$MD_D" ]; then
|
||||||
|
echo "WARNING: concurrent run at $C clients produced no result (ml='$ML' md='$MD')" >&2
|
||||||
|
fi
|
||||||
|
RATIO=$(node -e "const m=Number('$ML_D'),d=Number('$MD_D');console.log(d>0?(m/d).toFixed(1)+'x':'—')")
|
||||||
|
echo "clients $C $ML_D $MD_D $RATIO" | tee -a "$CC"
|
||||||
|
done
|
||||||
|
kill -9 $MD_PID $ML_PID 2>/dev/null; wait 2>/dev/null
|
||||||
|
|
||||||
|
# ---- assemble the report --------------------------------------------------
|
||||||
|
{
|
||||||
|
echo "# mongo-lite vs MongoDB benchmark"
|
||||||
|
echo "# date: $(date -u +%Y-%m-%dT%H:%M:%SZ) git: $REV$DIRTY"
|
||||||
|
echo "# args: size=$SIZE doc-size=$DOC wc=j clients=$CLIENTS per-client=$PER_CLIENT"
|
||||||
|
echo "# reproduce: bash tests/e2e/bench-run.sh $SIZE $DOC \"$CLIENTS\""
|
||||||
|
echo
|
||||||
|
echo "[main]"
|
||||||
|
node -e '
|
||||||
|
const fs = require("fs");
|
||||||
|
const read = (p) => {
|
||||||
|
const m = {};
|
||||||
|
for (const line of fs.readFileSync(p, "utf8").split("\n")) {
|
||||||
|
const i = line.indexOf("\t");
|
||||||
|
if (i > 0) m[line.slice(0, i)] = line.slice(i + 1).replace(/\t.*$/, "");
|
||||||
|
}
|
||||||
|
return m;
|
||||||
|
};
|
||||||
|
const a = read("/tmp/mongo-cmp/ml-report.txt");
|
||||||
|
const b = read("/tmp/mongo-cmp/mongo-report.txt");
|
||||||
|
const keys = ["insertOne (sequential) ×200","bulk insert throughput","docs loaded","createIndex({k: 1})",
|
||||||
|
"countDocuments({})","findOne({_id: <ObjectId>})","findOne({k: 500}) (indexed)","find({p: {$gte,$lt}}).count() (scan)",
|
||||||
|
"find({}).sort({_id:-1}).limit(20)","find({}, {proj}).limit(1000)","aggregate $group by k",
|
||||||
|
"updateOne({_id}) ×50","updateMany({k: 7}, {$inc})","deleteOne({_id}) + insertOne","node client RSS"];
|
||||||
|
for (const k of keys) console.log(`${k}\t${a[k] || "—"}\t${b[k] || "—"}`);
|
||||||
|
'
|
||||||
|
echo
|
||||||
|
echo "[concurrency]"
|
||||||
|
cat "$CC"
|
||||||
|
echo
|
||||||
|
echo "[meta]"
|
||||||
|
node -e '
|
||||||
|
const fs = require("fs");
|
||||||
|
const m = JSON.parse(fs.readFileSync("/tmp/mongo-cmp/meta.json", "utf8"));
|
||||||
|
for (const [k, v] of Object.entries(m)) console.log(`${k}\t${v}`);
|
||||||
|
'
|
||||||
|
} > "$REPORT"
|
||||||
|
|
||||||
|
# ---- diff against the previous run ---------------------------------------
|
||||||
|
echo "report: $REPORT"
|
||||||
|
if [ -f "$LATEST" ]; then
|
||||||
|
echo; echo "### mongo-lite numbers vs previous run ($(head -2 "$LATEST" | tail -1 | sed 's/# //'))"
|
||||||
|
node -e '
|
||||||
|
const fs = require("fs");
|
||||||
|
const old = fs.readFileSync(process.argv[1], "utf8");
|
||||||
|
const neu = fs.readFileSync(process.argv[2], "utf8");
|
||||||
|
const section = (txt, name) => {
|
||||||
|
const i = txt.indexOf("[" + name + "]");
|
||||||
|
if (i < 0) return new Map();
|
||||||
|
const body = txt.slice(i).split(/\n\[/)[0];
|
||||||
|
const m = new Map();
|
||||||
|
for (const line of body.split("\n")) {
|
||||||
|
const p = line.split("\t");
|
||||||
|
if (p.length >= 3 && !line.startsWith("#")) {
|
||||||
|
// concurrency rows are "clients\t<N>\t<docs/s>..." — key on the
|
||||||
|
// client count, not the literal "clients".
|
||||||
|
const key = p[0] === "clients" ? p[0] + p[1] : p[0];
|
||||||
|
const val = p[0] === "clients" ? p[2] : p[1];
|
||||||
|
m.set(key, val);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return m;
|
||||||
|
};
|
||||||
|
const o = section(old, "main"), n = section(neu, "main");
|
||||||
|
const fmt = (v) => v.padEnd(20);
|
||||||
|
console.log(`benchmark`.padEnd(42) + fmt("old") + fmt("new") + "delta");
|
||||||
|
for (const [k, v] of n) {
|
||||||
|
if (!o.has(k)) continue;
|
||||||
|
const ov = o.get(k);
|
||||||
|
const num = (s) => parseFloat(s);
|
||||||
|
const d = isFinite(num(ov)) && isFinite(num(v)) && num(ov) > 0 ? ((num(v) - num(ov)) / num(ov) * 100).toFixed(0) + "%" : "";
|
||||||
|
console.log(k.padEnd(42) + fmt(ov) + fmt(v) + d);
|
||||||
|
}
|
||||||
|
// concurrency (mongo-lite docs/s per client count)
|
||||||
|
const oc = section(old, "concurrency"), nc = section(neu, "concurrency");
|
||||||
|
console.log("\nconcurrency — mongo-lite docs/s:");
|
||||||
|
for (const [k, v] of nc) {
|
||||||
|
const ov = oc.get(k);
|
||||||
|
const d = ov && Number(ov) > 0 ? ((Number(v) - Number(ov)) / Number(ov) * 100).toFixed(0) + "%" : "";
|
||||||
|
console.log(`clients ${k.slice(7)}:`.padEnd(12) + `${ov ?? "—"}`.padEnd(14) + `${v}`.padEnd(14) + d);
|
||||||
|
}
|
||||||
|
' "$LATEST" "$REPORT"
|
||||||
|
else
|
||||||
|
echo "(no previous run to diff — this is the baseline)"
|
||||||
|
fi
|
||||||
|
cp "$REPORT" "$LATEST"
|
||||||
|
echo; echo "latest: $LATEST"
|
||||||
@@ -28,7 +28,17 @@ echo; echo "### mongod (MongoDB $(mongod --version | grep -oE 'v[0-9.]+' | head
|
|||||||
mongod --dbpath "$CMPDIR/mongod" --port $MD_PORT --bind_ip 127.0.0.1 \
|
mongod --dbpath "$CMPDIR/mongod" --port $MD_PORT --bind_ip 127.0.0.1 \
|
||||||
--quiet >"$MD_OUT" 2>&1 &
|
--quiet >"$MD_OUT" 2>&1 &
|
||||||
MD_PID=$!
|
MD_PID=$!
|
||||||
sleep 2
|
# Poll until mongod answers; a fixed sleep is flaky right after other
|
||||||
|
# benchmark phases have warmed the machine.
|
||||||
|
wait_ready() { # $1 = url
|
||||||
|
for _ in $(seq 1 90); do
|
||||||
|
NODE_PATH="tests/e2e/node_modules" node -e "require('mongodb').MongoClient.connect(process.argv[1],{serverSelectionTimeoutMS:800}).then(c=>c.close().then(()=>process.exit(0))).catch(()=>process.exit(1))" "$1" 2>/dev/null \
|
||||||
|
&& return 0
|
||||||
|
sleep 1
|
||||||
|
done
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
wait_ready "mongodb://127.0.0.1:$MD_PORT" || { echo "mongod never became ready" >&2; exit 1; }
|
||||||
node tests/e2e/compare.js --url "mongodb://127.0.0.1:$MD_PORT" --label mongodb --size "$SIZE" --doc-size "$DOC" \
|
node tests/e2e/compare.js --url "mongodb://127.0.0.1:$MD_PORT" --label mongodb --size "$SIZE" --doc-size "$DOC" \
|
||||||
> "$CMPDIR/mongo-report.txt" 2>&1 || { echo "mongodb bench failed:"; tail -5 "$CMPDIR/mongo-report.txt"; }
|
> "$CMPDIR/mongo-report.txt" 2>&1 || { echo "mongodb bench failed:"; tail -5 "$CMPDIR/mongo-report.txt"; }
|
||||||
MD_RSS=$(ps -o rss= -p $MD_PID | awk '{printf "%.0f", $1/1024}')
|
MD_RSS=$(ps -o rss= -p $MD_PID | awk '{printf "%.0f", $1/1024}')
|
||||||
@@ -57,7 +67,7 @@ echo; echo "### mongo-lite (recommended config: --compact-threshold 1g)"
|
|||||||
zig build -Doptimize=ReleaseFast 2>&1 | grep -c "^error" | grep -q "^0" || { echo "build failed"; exit 1; }
|
zig build -Doptimize=ReleaseFast 2>&1 | grep -c "^error" | grep -q "^0" || { echo "build failed"; exit 1; }
|
||||||
./zig-out/bin/mongo-lite --port $ML_PORT --db "$ML_LOG" --compact-threshold 1g >"$ML_OUT" 2>&1 &
|
./zig-out/bin/mongo-lite --port $ML_PORT --db "$ML_LOG" --compact-threshold 1g >"$ML_OUT" 2>&1 &
|
||||||
ML_PID=$!
|
ML_PID=$!
|
||||||
sleep 1
|
wait_ready "mongodb://127.0.0.1:$ML_PORT" || { echo "mongo-lite never became ready" >&2; exit 1; }
|
||||||
node tests/e2e/compare.js --url "mongodb://127.0.0.1:$ML_PORT" --label mongo-lite --size "$SIZE" --doc-size "$DOC" \
|
node tests/e2e/compare.js --url "mongodb://127.0.0.1:$ML_PORT" --label mongo-lite --size "$SIZE" --doc-size "$DOC" \
|
||||||
> "$CMPDIR/ml-report.txt" 2>&1 || { echo "mongo-lite bench failed:"; tail -5 "$CMPDIR/ml-report.txt"; }
|
> "$CMPDIR/ml-report.txt" 2>&1 || { echo "mongo-lite bench failed:"; tail -5 "$CMPDIR/ml-report.txt"; }
|
||||||
ML_RSS=$(ps -o rss= -p $ML_PID | awk '{printf "%.0f", $1/1024}')
|
ML_RSS=$(ps -o rss= -p $ML_PID | awk '{printf "%.0f", $1/1024}')
|
||||||
|
|||||||
45
tests/e2e/concurrent.js
Normal file
45
tests/e2e/concurrent.js
Normal file
@@ -0,0 +1,45 @@
|
|||||||
|
// Concurrent durable-write benchmark: N clients each do M sequential
|
||||||
|
// insertOne ({w:1, j:true} by default) into their own collection, reporting
|
||||||
|
// aggregate docs/s. Exercises the group-commit path under real contention.
|
||||||
|
//
|
||||||
|
// node tests/e2e/concurrent.js --url mongodb://127.0.0.1:27019 --label mongo-lite
|
||||||
|
// [--clients 8] [--per-client 2000] [--wc j|none]
|
||||||
|
//
|
||||||
|
// Output (stdout, tab-separated, one line):
|
||||||
|
// <label> clients=N <docs/s> (M docs in S s)
|
||||||
|
const { MongoClient, ObjectId } = require('mongodb');
|
||||||
|
|
||||||
|
let opt = { url: null, label: null, clients: 8, perClient: 2000, wc: 'j' };
|
||||||
|
for (let i = 2; i < process.argv.length; i++) {
|
||||||
|
const a = process.argv[i];
|
||||||
|
if (a === '--url') opt.url = process.argv[++i];
|
||||||
|
else if (a === '--label') opt.label = process.argv[++i];
|
||||||
|
else if (a === '--clients') opt.clients = Number(process.argv[++i]);
|
||||||
|
else if (a === '--per-client') opt.perClient = Number(process.argv[++i]);
|
||||||
|
else if (a === '--wc') opt.wc = process.argv[++i];
|
||||||
|
else { console.error(`unknown option ${a}`); process.exit(2); }
|
||||||
|
}
|
||||||
|
if (!opt.url) { console.error('--url required'); process.exit(2); }
|
||||||
|
opt.label = opt.label || opt.url.replace(/^mongodb:\/\//, '');
|
||||||
|
const WC = opt.wc === 'j' ? { writeConcern: { w: 1, j: true } } : {};
|
||||||
|
|
||||||
|
async function worker(i) {
|
||||||
|
const c = new MongoClient(opt.url, { maxPoolSize: 4, serverSelectionTimeoutMS: 15000 });
|
||||||
|
await c.connect();
|
||||||
|
// One collection per worker in a scratch db, so no cross-worker contention
|
||||||
|
// beyond the engine's own commit path.
|
||||||
|
const coll = c.db(`cc${Date.now() % 100000}`).collection(`w${i}`);
|
||||||
|
for (let j = 0; j < opt.perClient; j++) {
|
||||||
|
await coll.insertOne({ _id: new ObjectId(), k: j, payload: 'x'.repeat(256) }, WC);
|
||||||
|
}
|
||||||
|
await c.close();
|
||||||
|
}
|
||||||
|
|
||||||
|
(async () => {
|
||||||
|
const t0 = process.hrtime.bigint();
|
||||||
|
await Promise.all(Array.from({ length: opt.clients }, (_, i) => worker(i)));
|
||||||
|
const s = Number(process.hrtime.bigint() - t0) / 1e9;
|
||||||
|
const total = opt.clients * opt.perClient;
|
||||||
|
console.log(`${opt.label}\tclients=${opt.clients}\t${(total / s).toFixed(0)} docs/s\t(${total} docs in ${s.toFixed(2)} s)`);
|
||||||
|
process.exit(0);
|
||||||
|
})().catch((e) => { console.error('FAIL', e); process.exit(1); });
|
||||||
34
tests/e2e/results/bench-20260803-112548.txt
Normal file
34
tests/e2e/results/bench-20260803-112548.txt
Normal file
@@ -0,0 +1,34 @@
|
|||||||
|
# mongo-lite vs MongoDB benchmark
|
||||||
|
# date: 2026-08-03T08:26:46Z git: c8d547f+dirty
|
||||||
|
# args: size=128m doc-size=8k wc=j clients=1 2 4 per-client=2000
|
||||||
|
# reproduce: bash tests/e2e/bench-run.sh 128m 8k "1 2 4"
|
||||||
|
|
||||||
|
[main]
|
||||||
|
insertOne (sequential) ×200 0.20 ms 5.9 ms
|
||||||
|
bulk insert throughput 664.4 MB/s 357.9 MB/s
|
||||||
|
docs loaded 16,384 16,384
|
||||||
|
createIndex({k: 1}) 10.4 ms 45.2 ms
|
||||||
|
countDocuments({}) 0.73 ms 8.6 ms
|
||||||
|
findOne({_id: <ObjectId>}) 0.35 ms 0.62 ms
|
||||||
|
findOne({k: 500}) (indexed) 0.36 ms 1.2 ms
|
||||||
|
find({p: {$gte,$lt}}).count() (scan) 2.4 ms 3.2 ms
|
||||||
|
find({}).sort({_id:-1}).limit(20) 2.0 ms 2.2 ms
|
||||||
|
find({}, {proj}).limit(1000) — 4.6 ms
|
||||||
|
aggregate $group by k — 4.6 ms
|
||||||
|
updateOne({_id}) ×50 — 0.26 ms
|
||||||
|
updateMany({k: 7}, {$inc}) — 5.7 ms
|
||||||
|
deleteOne({_id}) + insertOne — 4.8 ms
|
||||||
|
node client RSS — 136 MB
|
||||||
|
|
||||||
|
[concurrency]
|
||||||
|
clients 1 8117 175 46.4x
|
||||||
|
clients 2 17102 230 74.4x
|
||||||
|
clients 4 21967 352 62.4x
|
||||||
|
|
||||||
|
[meta]
|
||||||
|
ml_rss_mb 158
|
||||||
|
md_rss_mb 251
|
||||||
|
ml_reopen 0.3s
|
||||||
|
md_reopen 0.4s
|
||||||
|
ml_disk_mb 7MB
|
||||||
|
md_disk_mb 7MB
|
||||||
36
tests/e2e/results/bench-20260803-113439.txt
Normal file
36
tests/e2e/results/bench-20260803-113439.txt
Normal file
@@ -0,0 +1,36 @@
|
|||||||
|
# mongo-lite vs MongoDB benchmark
|
||||||
|
# date: 2026-08-03T08:36:09Z git: c8d547f+dirty
|
||||||
|
# args: size=1g doc-size=16k wc=j clients=1 4 8 16 32 per-client=2000
|
||||||
|
# reproduce: bash tests/e2e/bench-run.sh 1g 16k "1 4 8 16 32"
|
||||||
|
|
||||||
|
[main]
|
||||||
|
insertOne (sequential) ×200 0.20 ms 4.9 ms
|
||||||
|
bulk insert throughput 748.8 MB/s 685.2 MB/s
|
||||||
|
docs loaded 65,536 65,536
|
||||||
|
createIndex({k: 1}) 79.8 ms 86.6 ms
|
||||||
|
countDocuments({}) 3.4 ms 13.6 ms
|
||||||
|
findOne({_id: <ObjectId>}) 0.71 ms 0.61 ms
|
||||||
|
findOne({k: 500}) (indexed) 0.66 ms 2.0 ms
|
||||||
|
find({p: {$gte,$lt}}).count() (scan) 13.5 ms 12.4 ms
|
||||||
|
find({}).sort({_id:-1}).limit(20) 2.3 ms 2.6 ms
|
||||||
|
find({}, {proj}).limit(1000) 3.6 ms 4.4 ms
|
||||||
|
aggregate $group by k 9.5 ms 13.7 ms
|
||||||
|
updateOne({_id}) ×50 0.14 ms 0.18 ms
|
||||||
|
updateMany({k: 7}, {$inc}) 2.1 ms 6.1 ms
|
||||||
|
deleteOne({_id}) + insertOne 0.73 ms 5.0 ms
|
||||||
|
node client RSS 160 MB 153 MB
|
||||||
|
|
||||||
|
[concurrency]
|
||||||
|
clients 1 8816 203 43.4x
|
||||||
|
clients 4 22142 472 46.9x
|
||||||
|
clients 8 27716 967 28.7x
|
||||||
|
clients 16 32031 1858 17.2x
|
||||||
|
clients 32 3759 0.0x
|
||||||
|
|
||||||
|
[meta]
|
||||||
|
ml_rss_mb 547
|
||||||
|
md_rss_mb 1317
|
||||||
|
ml_reopen 0.8s
|
||||||
|
md_reopen 1.3s
|
||||||
|
ml_disk_mb 97MB
|
||||||
|
md_disk_mb 89MB
|
||||||
36
tests/e2e/results/bench-20260803-115200.txt
Normal file
36
tests/e2e/results/bench-20260803-115200.txt
Normal file
@@ -0,0 +1,36 @@
|
|||||||
|
# mongo-lite vs MongoDB benchmark
|
||||||
|
# date: 2026-08-03T08:53:32Z git: c8d547f+dirty
|
||||||
|
# args: size=1g doc-size=16k wc=j clients=1 4 8 16 32 per-client=2000
|
||||||
|
# reproduce: bash tests/e2e/bench-run.sh 1g 16k "1 4 8 16 32"
|
||||||
|
|
||||||
|
[main]
|
||||||
|
insertOne (sequential) ×200 0.20 ms 14.8 ms
|
||||||
|
bulk insert throughput 732.4 MB/s 546.2 MB/s
|
||||||
|
docs loaded 65,536 65,536
|
||||||
|
createIndex({k: 1}) 74.4 ms 83.6 ms
|
||||||
|
countDocuments({}) 3.1 ms 12.4 ms
|
||||||
|
findOne({_id: <ObjectId>}) 0.59 ms 0.61 ms
|
||||||
|
findOne({k: 500}) (indexed) 0.56 ms 0.94 ms
|
||||||
|
find({p: {$gte,$lt}}).count() (scan) 13.0 ms 15.0 ms
|
||||||
|
find({}).sort({_id:-1}).limit(20) 2.2 ms 2.2 ms
|
||||||
|
find({}, {proj}).limit(1000) 3.5 ms 4.4 ms
|
||||||
|
aggregate $group by k 8.3 ms 13.2 ms
|
||||||
|
updateOne({_id}) ×50 0.14 ms 0.20 ms
|
||||||
|
updateMany({k: 7}, {$inc}) 1.9 ms 6.4 ms
|
||||||
|
deleteOne({_id}) + insertOne 0.58 ms 4.9 ms
|
||||||
|
node client RSS 158 MB 157 MB
|
||||||
|
|
||||||
|
[concurrency]
|
||||||
|
clients 1 8435 232 36.4x
|
||||||
|
clients 4 22480 491 45.8x
|
||||||
|
clients 8 25920 966 26.8x
|
||||||
|
clients 16 31079 1842 16.9x
|
||||||
|
clients 32 34041 3632 9.4x
|
||||||
|
|
||||||
|
[meta]
|
||||||
|
ml_rss_mb 553
|
||||||
|
md_rss_mb 1486
|
||||||
|
ml_reopen 0.8s
|
||||||
|
md_reopen 1.3s
|
||||||
|
ml_disk_mb 97MB
|
||||||
|
md_disk_mb 106MB
|
||||||
36
tests/e2e/results/bench-latest.txt
Normal file
36
tests/e2e/results/bench-latest.txt
Normal file
@@ -0,0 +1,36 @@
|
|||||||
|
# mongo-lite vs MongoDB benchmark
|
||||||
|
# date: 2026-08-03T08:53:32Z git: c8d547f+dirty
|
||||||
|
# args: size=1g doc-size=16k wc=j clients=1 4 8 16 32 per-client=2000
|
||||||
|
# reproduce: bash tests/e2e/bench-run.sh 1g 16k "1 4 8 16 32"
|
||||||
|
|
||||||
|
[main]
|
||||||
|
insertOne (sequential) ×200 0.20 ms 14.8 ms
|
||||||
|
bulk insert throughput 732.4 MB/s 546.2 MB/s
|
||||||
|
docs loaded 65,536 65,536
|
||||||
|
createIndex({k: 1}) 74.4 ms 83.6 ms
|
||||||
|
countDocuments({}) 3.1 ms 12.4 ms
|
||||||
|
findOne({_id: <ObjectId>}) 0.59 ms 0.61 ms
|
||||||
|
findOne({k: 500}) (indexed) 0.56 ms 0.94 ms
|
||||||
|
find({p: {$gte,$lt}}).count() (scan) 13.0 ms 15.0 ms
|
||||||
|
find({}).sort({_id:-1}).limit(20) 2.2 ms 2.2 ms
|
||||||
|
find({}, {proj}).limit(1000) 3.5 ms 4.4 ms
|
||||||
|
aggregate $group by k 8.3 ms 13.2 ms
|
||||||
|
updateOne({_id}) ×50 0.14 ms 0.20 ms
|
||||||
|
updateMany({k: 7}, {$inc}) 1.9 ms 6.4 ms
|
||||||
|
deleteOne({_id}) + insertOne 0.58 ms 4.9 ms
|
||||||
|
node client RSS 158 MB 157 MB
|
||||||
|
|
||||||
|
[concurrency]
|
||||||
|
clients 1 8435 232 36.4x
|
||||||
|
clients 4 22480 491 45.8x
|
||||||
|
clients 8 25920 966 26.8x
|
||||||
|
clients 16 31079 1842 16.9x
|
||||||
|
clients 32 34041 3632 9.4x
|
||||||
|
|
||||||
|
[meta]
|
||||||
|
ml_rss_mb 553
|
||||||
|
md_rss_mb 1486
|
||||||
|
ml_reopen 0.8s
|
||||||
|
md_reopen 1.3s
|
||||||
|
ml_disk_mb 97MB
|
||||||
|
md_disk_mb 106MB
|
||||||
72
tests/e2e/results/phase8.txt
Normal file
72
tests/e2e/results/phase8.txt
Normal file
@@ -0,0 +1,72 @@
|
|||||||
|
# Phase 8 gate — mongo-lite vs MongoDB 8.3.7, 1g dataset / ~16k docs
|
||||||
|
# Ratio < 1.0 = mongo-lite faster. Reproduce: bash tests/e2e/bench-run.sh 1g 16k
|
||||||
|
# Working tree: c8d547f + uncommitted diff (epilogue-commit/group-commit work) +
|
||||||
|
# three bugs found and fixed while benchmarking (details at the bottom).
|
||||||
|
# Machine-readable copy: tests/e2e/results/bench-latest.txt (auto-diffed on
|
||||||
|
# every bench-run.sh invocation).
|
||||||
|
|
||||||
|
benchmark mongo-lite mongodb ratio
|
||||||
|
insertOne (sequential) ×200 0.20 ms 14.8 ms 0.0x
|
||||||
|
bulk insert throughput 732.4 MB/s 546.2 MB/s 1.3x
|
||||||
|
docs loaded 65,536 65,536 1.0x
|
||||||
|
createIndex({k: 1}) 74.4 ms 83.6 ms 0.9x
|
||||||
|
countDocuments({}) 3.1 ms 12.4 ms 0.3x
|
||||||
|
findOne({_id: <ObjectId>}) 0.59 ms 0.61 ms 1.0x
|
||||||
|
findOne({k: 500}) (indexed) 0.56 ms 0.94 ms 0.6x
|
||||||
|
find({p: {$gte,$lt}}).count() (scan) 13.0 ms 15.0 ms 0.9x
|
||||||
|
find({}).sort({_id:-1}).limit(20) 2.2 ms 2.2 ms 1.0x
|
||||||
|
find({}, {proj}).limit(1000) 3.5 ms 4.4 ms 0.8x
|
||||||
|
aggregate $group by k 8.3 ms 13.2 ms 0.6x
|
||||||
|
updateOne({_id}) ×50 0.14 ms 0.20 ms 0.7x
|
||||||
|
updateMany({k: 7}, {$inc}) 1.9 ms 6.4 ms 0.3x
|
||||||
|
deleteOne({_id}) + insertOne 0.58 ms 4.9 ms 0.1x
|
||||||
|
node client RSS 158 MB 157 MB 1.0x
|
||||||
|
server RSS 553 MB 1486 MB
|
||||||
|
kill -9 reopen 0.8s 1.3s
|
||||||
|
db on disk 97MB 106MB
|
||||||
|
|
||||||
|
# NEW: concurrent durable writes (sequential insertOne per client, {w:1, j:true})
|
||||||
|
# through the official driver, both servers, same workload:
|
||||||
|
# clients | mongodb | mongo-lite | ratio
|
||||||
|
# 1 | 232 | 8,435 | 36x
|
||||||
|
# 4 | 491 | 22,480 | 46x
|
||||||
|
# 8 | 966 | 25,920 | 27x
|
||||||
|
# 16 | 1,842 | 31,079 | 17x
|
||||||
|
# 32 | 3,632 | 34,041 | 9x
|
||||||
|
# (mongo-lite scales to ~34k durable docs/s at 32 clients; mongod tops out
|
||||||
|
# around 3.6k on this machine's fsync-per-write path.)
|
||||||
|
|
||||||
|
# THREE BUGS FOUND AND FIXED WHILE BENCHMARKING
|
||||||
|
# All three were invisible to the single-connection compare.js suite and only
|
||||||
|
# surfaced under the new concurrent harness (tests/e2e/concurrent.js).
|
||||||
|
#
|
||||||
|
# 1. Group-commit deadlock (>=6 concurrent writers froze permanently).
|
||||||
|
# std.Io.Condition.signal() delivers exactly ONE wakeup. The commit
|
||||||
|
# leader's completion woke a single follower on `while (committing)` and
|
||||||
|
# stranded the rest; an append's signal could also be consumed by a
|
||||||
|
# follower, stranding the leader on the pending_appends drain. Signals
|
||||||
|
# were also sent without commit_lock, leaving a lost-wakeup window
|
||||||
|
# between a waiter's epoch snapshot and its futex sleep.
|
||||||
|
# Fix (src/db.zig): all commit_done signals hold commit_lock and use
|
||||||
|
# broadcast(). Reproduced on HEAD too (introduced by the group-commit
|
||||||
|
# work, not the uncommitted diff).
|
||||||
|
#
|
||||||
|
# 2. Racy assertion in commit(): assert(pending_appends == 0) after the
|
||||||
|
# drain loop. A NEW append can start at any moment (the increment takes
|
||||||
|
# no commit_lock), so the assert fired spuriously under load and crashed
|
||||||
|
# the server (32 concurrent clients, "commit would seal while appends
|
||||||
|
# are still in flight"). The seal's real invariant holds via log_lock
|
||||||
|
# ordering; the instant check was wrong. Fix: removed the assert.
|
||||||
|
#
|
||||||
|
# 3. Slab segment-start corruption in Collection.slab_append: a pointer
|
||||||
|
# `last` into slab.items dangled across the slab.append() that starts the
|
||||||
|
# next segment (the list can reallocate), so the new segment's start
|
||||||
|
# offset was computed from freed memory. Docs in the affected segment
|
||||||
|
# were then located at wrong offsets, surfacing as InvalidBson on reads
|
||||||
|
# with a batch limit >~50 (or a crash in Debug builds). Position varied
|
||||||
|
# between runs (heap contents vary). Fix: read the last segment's length
|
||||||
|
# as a value before the append.
|
||||||
|
#
|
||||||
|
# Verification after the fixes: unit suite green, e2e6 72/72 (incl. kill -9
|
||||||
|
# mid-write recovery), crash pair, concurrent 8-client, indexes, TTL all
|
||||||
|
# green; concurrent stress 1/4/8/16/32 clients x5 sequential rounds clean.
|
||||||
Reference in New Issue
Block a user