tests/spec: a recorded corpus for the positional operators

M3's gate as named -- "remaining crud coverage; e2e3/e2e4 green" -- cannot
see this work. e2e3 and e2e4 contain zero positional paths, and the pinned
crud corpus covers `$[<identifier>]` only: no `$[]` case, no bare `$` case
anywhere in it. Both stayed green through a bug that replaced an array with
`{"$[i]": {...}}` and answered ok: 1. So M3 brings its own corpus, built the
way M2.5's was: inputs authored in `sources/`, every expectation recorded
from mongod 8.3.7, run through the shared runner with `--suite-dir`.

51 cases across the three spellings. It stands at 15 pass / 36 fail against
the refusal, which is the intended shape -- `expressions.json` was recorded
at 1 pass / 26 fail before the evaluator and is green now. The 15 that pass
are refusals where this server's code already matches; of the 36, 31 are the
constructs answering "not implemented" and 5 are refusals whose code
differs, four of them arrayFilters validation this server cannot do because
it never parses the option.

Every case records its `outcome`, refusals included. That is deliberate and
it is the whole point of the file: a refusal that left the document mangled
is indistinguishable from a clean one in `expectError` alone, and a mangled
document is what this corpus exists to catch.

What recording it settled, none of it guessable, the first contradicting
what the design review assumed:

  - `y.$[i].c.$[i].d`, one identifier reused at two levels, is **accepted**
    -- not a duplicate-identifier error
  - `$[]` over an empty array is a no-op with modifiedCount 0
  - `$[]` over an array with a non-document element is error 28, where every
    other path failure here is 2
  - a positional segment never creates: a missing or non-array path is an
    error, where `$set: {'a.b': 1}` would construct one
  - an upsert gets no special case -- it fails for the same reason
  - `$` writes only the first matching element, and is refused when the query
    never touched the array
  - any of the three in first position is refused, as is `$` twice in a path
  - `arrayFilters` alongside a replacement is ignored rather than refused

A case needing its own documents reseeds through operations rather than
`initialData`, because the format's `initialData` is per file and the runner
seeds it once per test. That keeps one file per operator instead of one file
per document shape.

crud scorecard unchanged at 204/87, aggregation corpus still 70/0.
This commit was merged in pull request #6.
This commit is contained in:
A.Shakhmatov
2026-08-10 19:00:01 +03:00
parent 3c2ac38fd9
commit e5a84c0598
10 changed files with 4554 additions and 2 deletions

View File

@@ -0,0 +1,119 @@
# The positional-update corpus
MongoDB has three ways to say "descend into this array" in an update path:
| | since | pinned crud corpus |
|---|---|---|
| `$[<identifier>]` with `arrayFilters` | 3.6 | 4 files, 14 cases |
| `$[]` all-positional | 3.6 | **nothing** |
| `$` positional | ancient | **nothing** |
All three shared one code path in this engine, and that path overwrote the
array it was supposed to walk — `y: [{b: 3}, {b: 1}]` became
`y: {"$[i]": {"b": 2}}`, every element discarded, `ok: 1`. Only the
`$[<identifier>]` third of it was externally visible, so a fix measured
against the pinned corpus alone would have gone green while two of the three
still destroyed data. That is `docs/M3_ARRAYFILTERS_DESIGN_REVIEW.md` §3, and
this directory is its consequence.
## The one rule
**Inputs are authored here; expectations are measured against a real mongod.**
The same rule as `tests/spec/aggregate/`, for the same reason: a corpus we
write end to end can encode our own bugs as expectations and will then agree
with us forever. It is not a hypothetical risk here — the design review's own
guesses about `$[]`, about missing paths and about upserts were all wrong
before mongod was asked.
`sources/*.json` holds documents and operations and nothing else. `record.js`
asks mongod 8.3.7 what each one answers.
## Running it
```sh
node tests/spec/run.js --suite-dir tests/spec/positional
```
The same runner as the crud corpus, pointed elsewhere — the entity model, the
matchers, `outcome` verification and the skip accounting come for free.
`--scorecard` is refused with `--suite-dir`: `tests/spec/scorecard.txt` is the
crud corpus's record and the milestones are compared against it.
## Re-recording
```sh
mongod --port 27099 --dbpath /tmp/mongo-corpus &
node tests/spec/positional/record.js --mongod-port 27099
```
Writes `<name>.json` for every `sources/<name>.json`. The generated files are
committed: they *are* the corpus, and regenerating them is how a disagreement
with mongod gets re-measured rather than argued about.
Three things to know when adding cases:
- **Every case records its `outcome`, including the refusals.** A refusal that
left the document mangled looks identical to a clean one in `expectError`
alone, and a mangled document is the entire reason this corpus exists.
- **Errors record the code, not the message.** Message text is mongod's to
change between releases. Several of these messages also embed a
shell-syntax rendering of the offending element, which no formatter in this
server produces.
- **A case needing its own documents reseeds through operations**
(`deleteMany` + `insertMany`), not `initialData`: the unified format's
`initialData` is per file and the runner seeds it once per test. This keeps
one file per operator rather than one file per document shape.
Every construct here is 3.6 or older, so nothing depends on a server newer
than the 4.4 this one reports. A case that did should be left out rather than
annotated — recording it from mongod 8.x and judging it against a 4.4 answer
measures the version gap, not the engine.
## Where it stands
Recorded against mongod 8.3.7, run against the server at the positional
refusal:
```
filtered.json 8 pass 19 fail 0 skip
all-positional.json 3 pass 9 fail 0 skip
first-positional.json 4 pass 8 fail 0 skip
```
Red by construction and by design. The 15 that pass are the refusals where
this server's code already matches mongod's; of the 36 that fail, 31 are the
constructs themselves answering "not implemented by this server", and 5 are
refusals whose code differs — four of them the `arrayFilters` validation this
server cannot do yet because it never parses the option.
That is the intended shape. `tests/spec/aggregate/expressions.json` was
recorded at 1 pass / 26 fail before the evaluator existed and is green now;
this is the same gate at the same stage.
## What recording it settled
None of this is guessable, and the first row contradicts what the review
assumed:
| | mongod |
|---|---|
| `y.$[i].c.$[i].d`, one identifier reused at two levels | **accepted** — not a duplicate-identifier error |
| `$[]` over an empty array | no-op, `modifiedCount: 0` |
| `$[]` where one element is not a document | error **28**, where the other path failures are 2 |
| `$[]` or `$[i]` where the path is missing, or is not an array | error 2 — a positional segment never *creates* |
| an upsert that would insert | error 2; upsert gets no special case |
| `$` when the query never touched the array | error 2 |
| `$` where several elements match | only the first is written |
| `$`, `$[]` or `$[i]` in first position | error 2 |
| `$` twice in one path | error 2 |
| `arrayFilters` alongside a replacement | ignored, no error |
| an array filter the update never uses | error 9 |
| a filter with no top-level field, two of them, or a duplicate identifier | error 9 |
The 2-versus-9 split is *nearly* "9 is the `arrayFilters` array judged on its
own, 2 is anything needing the update or the stored document" — and it is not
a clean rule: an identifier that is not a lowercase alphanumeric name carries
the same `Error parsing array filter` prefix as the 9s and is a 2. Recorded as
measured rather than tidied, because deriving the codes from the rule would
get that row wrong.