tests/e2e: iteration-to-iteration benchmark harness
compare-run.sh answers "how do we compare to MongoDB"; it says nothing
about whether a change made things better or worse than last week. Add a
harness that records each run and diffs it against the previous one.
bench-run.sh wraps compare-run.sh, adds a concurrent durable-write
comparison (concurrent.js: N clients each doing sequential insertOne with
{w:1, j:true}, exercising the group-commit path under real contention),
writes a versioned name<TAB>value report to results/bench-<timestamp>.txt,
and prints a diff of our numbers against results/bench-latest.txt.
Also:
- compare-run.sh polled with fixed sleeps, which are flaky once earlier
benchmark phases have warmed the machine; both servers now wait on a
real driver connection instead.
- a dispatch error only reached the client as a generic InternalError,
with nothing on the server side naming the failing command; log the
connection, command and error name before replacing the reply.
This commit is contained in:
@@ -97,3 +97,19 @@ on :27019, runs the same driver workload against each (durable writes:
|
||||
mongo-lite fsyncs per command, mongod runs with `j: true`), measures kill -9
|
||||
reopen for both, and prints a side-by-side table. `compare.js` alone runs
|
||||
one side (see its `--help`-style header comment).
|
||||
|
||||
### Iteration-to-iteration comparison: `bench-run.sh` + `concurrent.js`
|
||||
|
||||
```sh
|
||||
bash tests/e2e/bench-run.sh [size] [doc-size] ["clients..."] # e.g. 1g 16k "1 4 8 16 32"
|
||||
```
|
||||
|
||||
Runs the main suite (`compare-run.sh`) plus a concurrent durable-write
|
||||
comparison (`concurrent.js`, N clients each doing sequential `insertOne`
|
||||
with `{w:1, j:true}` — the group-commit path under real contention), then
|
||||
writes a machine-readable, versioned report to
|
||||
`tests/e2e/results/bench-<timestamp>.txt` and prints a diff of the
|
||||
mongo-lite numbers against the previous run (`results/bench-latest.txt`).
|
||||
The report has `[main]` / `[concurrency]` / `[meta]` sections with
|
||||
`name<TAB>value` rows; `bench-run.sh 1g 16k` reproduces the phase8 gate
|
||||
(see `results/phase8.txt`).
|
||||
|
||||
Reference in New Issue
Block a user