dev/MultiforaDB
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

tests/spec: the scorecard no longer disclaims expectEvents

Browse Source 
The disclaimer was accurate for as long as it stood -- events were not read,
so `pass` was an upper bound and saying otherwise would have been a lie about
the number. It is now a lie in the other direction, so it goes, replaced by
what is actually true: events are compared exactly, in number and in order,
which is what makes a pass mean the engine answered correctly *and* was asked
the right question. The header says plainly that scorecards recorded before
this are not comparable, and enumerates what is still skipped inside events
rather than leaving "asserted" to be read as "asserted completely".

Two facts in the docs had gone stale and are corrected here because this is
the commit that rereads them:

  - README said `--op-timeout-ms` defaults to 3 s. It has been 10 s since the
    commit that explains, at length and directly above the constant, why 3 s
    was wrong. A stale number in exactly the place that warns against
    tightening it is worse than no number.
  - `MAX_SCHEMA` is [1, 24]; the comment above it still claimed 1.0-1.9.

Totals unchanged at 159/132/196 -- this commit only rewrites prose, and the
scorecard is re-recorded so its header matches the runner that produced it.
This commit is contained in:
A.Shakhmatov
2026-08-09 13:14:58 +03:00
parent 54ad124c18
commit bb8cdd964b
3 changed files with 46 additions and 21 deletions
Download Patch File Download Diff File Expand all files Collapse all files

18
tests/spec/run.js
View File

@@ -31,8 +31,9 @@ const SUITE_DIR = path.join(__dirname, 'specifications', 'source', 'crud', 'test
const SCORECARD = path.join(__dirname, 'scorecard.txt');
const REPO = path.join(__dirname, '..', '..');
// The runner implements schema 1.0-1.9 features that CRUD tests actually use.
// A file declaring more than this is skipped whole rather than half-run.
// The runner implements the schema features that CRUD tests actually use, up
// to this version. A file declaring more than this is skipped whole rather
// than half-run.
const MAX_SCHEMA = [1, 24];
const argv = process.argv.slice(2);
@@ -897,10 +898,15 @@ function scorecardText(results, tot, errored, server, nfiles) {
L.push('# - the suite needs a feature whose milestone has not landed (sessions M4,');
L.push('# auth M7, failPoints, gridfs) -- counted as skip, never as pass;');
L.push('# - or the runner itself does not implement the operation/matcher yet.');
L.push('# Deliberately NOT asserted yet: expectEvents (command monitoring). Those');
L.push('# assertions are about driver-visible command shape rather than result');
L.push('# semantics; ignoring them makes some cases pass that a full runner would');
L.push('# fail, so treat `pass` as an upper bound until M1 wires events up.');
L.push('# expectEvents IS asserted: the commands the driver sent are compared to the');
L.push('# expectation exactly, in number and in order, with `command` and `reply`');
L.push('# matched as root documents. A pass therefore means the engine answered');
L.push('# correctly *and* was asked the right question. Not comparable to any');
L.push('# scorecard recorded before that landed, where `pass` was an upper bound.');
L.push('# Still unasserted, each reported as skip where it is asserted, never as');
L.push('# pass: cmap and sdam events, ignoreExtraEvents, hasServiceId,');
L.push('# hasServerConnectionId, and maxTimeMS in an expected command (CSOT rewrites');
L.push('# it -- the only assertion this runner declines to make).');
L.push('');
L.push(`total\t${tot.pass} pass\t${tot.fail} fail\t${tot.skip} skip\t${nfiles} files\t${errored} errored`);
L.push('');