tests/spec: a recorded corpus for the positional operators #6

dev · 2026-08-10T16:50:37Z

dev commented

2026-08-10 16:50:37 +00:00

M3's gate as named -- "remaining crud coverage; e2e3/e2e4 green" -- cannot see
this work. e2e3 and e2e4 contain zero positional paths, and the pinned crud
corpus covers $[<identifier>] only: no $[] case, no bare $ case anywhere in
it. Both stayed green through a bug that replaced an array with {"$[i]": {...}}
and answered ok: 1.

So M3 brings its own corpus, built the way M2.5's was: inputs authored in
sources/, every expectation recorded from mongod 8.3.7, run through the shared
runner with --suite-dir.

Where it stands

51 cases across the three spellings:

filtered.json           8 pass   19 fail   0 skip
all-positional.json     3 pass    9 fail   0 skip
first-positional.json   4 pass    8 fail   0 skip

Red by construction and by design -- tests/spec/aggregate/expressions.json was
recorded at 1 pass / 26 fail before the evaluator existed and is green now; this
is the same gate at the same stage. The 15 that pass are refusals where this
server's code already matches mongod's. Of the 36 that fail, 31 are the
constructs answering "not implemented by this server", and 5 are refusals whose
code differs -- four of them arrayFilters validation this server cannot do
because it never parses the option.

Every case records its `outcome`, refusals included

Deliberate, and the whole point of the file: a refusal that left the document
mangled is indistinguishable from a clean one in expectError alone, and a
mangled document is what this corpus exists to catch.

What recording it settled

None of it guessable, and the first row contradicts what the design review
assumed:

	mongod
`y.$[i].c.$[i].d`, one identifier reused at two levels	accepted — not a duplicate-identifier error
`$[]` over an empty array	no-op, `modifiedCount: 0`
`$[]` where one element is not a document	error 28, where every other path failure is 2
`$[]` / `$[i]` where the path is missing or not an array	error 2 — a positional segment never creates
an upsert that would insert	error 2; no special case
`$` where several elements match	only the first is written
`$` when the query never touched the array	error 2
any of the three in first position, or `$` twice in a path	error 2
`arrayFilters` alongside a replacement	ignored, no error

The 2-versus-9 split is nearly "9 is the arrayFilters array judged on its
own, 2 is anything needing the update or the stored document" -- and it is not a
clean rule: an identifier that is not a lowercase alphanumeric name carries the
same Error parsing array filter prefix as the 9s and is a 2. Recorded as
measured rather than tidied.

Notes

A case needing its own documents reseeds through operations (deleteMany +
insertMany) rather than initialData, because the unified format's
initialData is per file and the runner seeds it once per test. That keeps one
file per operator instead of one file per document shape.

PLAN §3's M3 row now names this corpus as the gate, with §6 recording why the
old one could not see the work.

Verification

crud scorecard unchanged at 204/87, aggregation corpus still 70/0, and the
--scorecard + --suite-dir guard still refuses.

M3's gate as named -- "remaining crud coverage; e2e3/e2e4 green" -- **cannot see this work**. `e2e3` and `e2e4` contain zero positional paths, and the pinned crud corpus covers `$[<identifier>]` only: no `$[]` case, no bare `$` case anywhere in it. Both stayed green through a bug that replaced an array with `{"$[i]": {...}}` and answered `ok: 1`. So M3 brings its own corpus, built the way M2.5's was: inputs authored in `sources/`, every expectation recorded from mongod 8.3.7, run through the shared runner with `--suite-dir`. ## Where it stands 51 cases across the three spellings: ``` filtered.json 8 pass 19 fail 0 skip all-positional.json 3 pass 9 fail 0 skip first-positional.json 4 pass 8 fail 0 skip ``` Red by construction and by design -- `tests/spec/aggregate/expressions.json` was recorded at 1 pass / 26 fail before the evaluator existed and is green now; this is the same gate at the same stage. The 15 that pass are refusals where this server's code already matches mongod's. Of the 36 that fail, 31 are the constructs answering "not implemented by this server", and 5 are refusals whose code differs -- four of them `arrayFilters` validation this server cannot do because it never parses the option. ## Every case records its `outcome`, refusals included Deliberate, and the whole point of the file: a refusal that left the document mangled is indistinguishable from a clean one in `expectError` alone, and a mangled document is what this corpus exists to catch. ## What recording it settled None of it guessable, and the first row contradicts what the design review assumed: | | mongod | |---|---| | `y.$[i].c.$[i].d`, one identifier reused at two levels | **accepted** — not a duplicate-identifier error | | `$[]` over an empty array | no-op, `modifiedCount: 0` | | `$[]` where one element is not a document | error **28**, where every other path failure is 2 | | `$[]` / `$[i]` where the path is missing or not an array | error 2 — a positional segment never *creates* | | an upsert that would insert | error 2; no special case | | `$` where several elements match | only the first is written | | `$` when the query never touched the array | error 2 | | any of the three in first position, or `$` twice in a path | error 2 | | `arrayFilters` alongside a replacement | ignored, no error | The 2-versus-9 split is *nearly* "9 is the `arrayFilters` array judged on its own, 2 is anything needing the update or the stored document" -- and it is not a clean rule: an identifier that is not a lowercase alphanumeric name carries the same `Error parsing array filter` prefix as the 9s and is a 2. Recorded as measured rather than tidied. ## Notes A case needing its own documents reseeds through operations (`deleteMany` + `insertMany`) rather than `initialData`, because the unified format's `initialData` is per file and the runner seeds it once per test. That keeps one file per operator instead of one file per document shape. PLAN §3's M3 row now names this corpus as the gate, with §6 recording why the old one could not see the work. ## Verification crud scorecard unchanged at 204/87, aggregation corpus still 70/0, and the `--scorecard` + `--suite-dir` guard still refuses.

dev added 1 commit 2026-08-10 16:50:37 +00:00

tests/spec: a recorded corpus for the positional operators e5a84c0598

M3's gate as named -- "remaining crud coverage; e2e3/e2e4 green" -- cannot
see this work. e2e3 and e2e4 contain zero positional paths, and the pinned
crud corpus covers `$[<identifier>]` only: no `$[]` case, no bare `$` case
anywhere in it. Both stayed green through a bug that replaced an array with
`{"$[i]": {...}}` and answered ok: 1. So M3 brings its own corpus, built the
way M2.5's was: inputs authored in `sources/`, every expectation recorded
from mongod 8.3.7, run through the shared runner with `--suite-dir`.

51 cases across the three spellings. It stands at 15 pass / 36 fail against
the refusal, which is the intended shape -- `expressions.json` was recorded
at 1 pass / 26 fail before the evaluator and is green now. The 15 that pass
are refusals where this server's code already matches; of the 36, 31 are the
constructs answering "not implemented" and 5 are refusals whose code
differs, four of them arrayFilters validation this server cannot do because
it never parses the option.

Every case records its `outcome`, refusals included. That is deliberate and
it is the whole point of the file: a refusal that left the document mangled
is indistinguishable from a clean one in `expectError` alone, and a mangled
document is what this corpus exists to catch.

What recording it settled, none of it guessable, the first contradicting
what the design review assumed:

  - `y.$[i].c.$[i].d`, one identifier reused at two levels, is **accepted**
    -- not a duplicate-identifier error
  - `$[]` over an empty array is a no-op with modifiedCount 0
  - `$[]` over an array with a non-document element is error 28, where every
    other path failure here is 2
  - a positional segment never creates: a missing or non-array path is an
    error, where `$set: {'a.b': 1}` would construct one
  - an upsert gets no special case -- it fails for the same reason
  - `$` writes only the first matching element, and is refused when the query
    never touched the array
  - any of the three in first position is refused, as is `$` twice in a path
  - `arrayFilters` alongside a replacement is ignored rather than refused

A case needing its own documents reseeds through operations rather than
`initialData`, because the format's `initialData` is per file and the runner
seeds it once per test. That keeps one file per operator instead of one file
per document shape.

crud scorecard unchanged at 204/87, aggregation corpus still 70/0.

dev merged commit e5a84c0598 into main

2026-08-10 16:50:47 +00:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: dev/MultiforaDB#6