From 44788ae681ce71a824df949d07c3229e97f6b278 Mon Sep 17 00:00:00 2001 From: "A.Shakhmatov" Date: Mon, 10 Aug 2026 21:33:45 +0300 Subject: [PATCH] plan/spec: the operator corpus goes green MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 102/102. PLAN's M3 row now names both corpora as the gate, because neither the pinned crud suite nor e2e3/e2e4 can see this work: the eight operators are barely in the pinned corpus and `$push`'s modifiers are not in it at all. §6 records what the corpus found that was nobody's operator -- an upsert never reporting the `_id` it generated -- and the three things left open on purpose: `$bit`, a multi-field `$sort` key inside `$push`, and the 64-path bound on the conflict check. Full matrix at this commit: 247/247 unit tests in ReleaseFast and ReleaseSafe, 83/83 fuzz, operators 102/0, positional 51/0, aggregation 70/0, pinned crud 218/73/196 unmoved, e2e and crash-fuzz green. --- PLAN.md | 24 +++++++++++++++++++++++- tests/spec/operators/README.md | 28 +++++++++++++++++----------- 2 files changed, 40 insertions(+), 12 deletions(-) diff --git a/PLAN.md b/PLAN.md index 6c0d1e6..b2b0a0d 100644 --- a/PLAN.md +++ b/PLAN.md @@ -467,7 +467,7 @@ answers, and the trade is only acceptable because the lie is removed first. | M1 | **Cursors + wire polish** | getMore / killCursors / batchSize; server-side cursor state with idle timeout; sessions plumbing (lsid accepted) as drivers send it; hello advertisement updates; **`moreToCome` on requests** (see the bug below); command-monitoring assertions in the spec runner | crud spec suite green; e2e green | | M2 | **The `aggregate` command surface** | `$out` and `$merge` (7 of the 13 failures), and refusing every pipeline construct the engine does not implement instead of answering `0` (amendment A6). The other 6 failures are blocked on M2.5, M4 and M8 — see `docs/M2_DESIGN_REVIEW.md` §7 | `aggregate-*.json`: 0 fail among the 7 reachable cases | | M2.5 | **The aggregation engine** | expression evaluator, per-stage document iterator, the accumulators, `$unwind`; `$lookup`/`$facet` explicitly out of the first cut (amendment A6) | a purpose-built stage corpus, every expectation measured against mongod | -| M3 | **Update operators + index types** | `distinct` (**done**); positional paths refused rather than destructive (**done**); `$`/`$[]`/`$[]` + `arrayFilters` implemented (**done**); then $setOnInsert, $addToSet, $mul, $min/$max, $pop, $pullAll, $currentDate, pipeline updates; partial + hashed indexes | `tests/spec/positional/` 0 fail (51 cases, recorded from mongod — **green**; the named gate could not see this work, see below); remaining crud coverage; e2e3/e2e4 green | +| M3 | **Update operators + index types** | `distinct` (**done**); positional paths refused rather than destructive (**done**); `$`/`$[]`/`$[]` + `arrayFilters` (**done**); $setOnInsert, $addToSet, $mul, $min/$max, $pop, $pullAll, $currentDate + `$push`'s modifiers (**done**); then pipeline updates; partial + hashed indexes | `tests/spec/positional/` 0 fail (51 cases) and `tests/spec/operators/` 0 fail (102 cases), both recorded from mongod — **green**; the named gate could not see either, see below; remaining crud coverage; e2e3/e2e4 green | | M4 | **Sessions + transactions** | logical sessions, snapshot isolation on the mmap engine, write concern at commit | sessions + transactions spec suites green | | M5 | **Change streams** | change feed + resume tokens (likely log-seq based), getMore integration | change-streams spec suite green | | M6 | **Admin/ops commands** | dbStats, collStats, serverStatus, ping, buildInfo, listDatabases filters, dropDatabase durability (log it) | mongosh UX smoke; e2e green | @@ -1123,6 +1123,28 @@ has to be its own commit with its own re-recorded scorecard. agrees. Worth one commit, and it needs its own measurements first: the padding case (`y.3.b` past the end) is a *legal* creation on mongod, so the fix is not "refuse a non-document element". +- **M3's second corpus is `tests/spec/operators/`.** The eight operators PLAN + §3 names all answered `bad update` with code 2, one message for every + question — and `$push`'s `$slice`, `$position` and `$sort` were parsed, + accepted and *dropped*. `{$each: [3, 4], $slice: -3}` appended both values, + sliced nothing and answered ok: 1 with modifiedCount: 1, which is the same + class of wrong answer the positional operators were. 102 cases, recorded red + at 18/84 and driven green. + + It found one thing that was nobody's operator: **an upsert never reported the + `_id` it generated**. `Engine.insert` writes a generated `_id` into the bytes + and leaves the caller's tree without it, so `upsertedId` came back null and + `findOneAndUpdate` with `returnDocument: after` returned a document with no + `_id` — the only name the client has for a document it has never seen. Fixed + in `build_upsert_doc`; the pinned crud corpus never noticed because its + upsert cases match `upsertedId` loosely. + + Left open, measured and deliberate: `$bit` is not implemented (mongod has it, + no driver in `tests/` sends it, and it is not in the M3 row); `$sort` inside + `$push` accepts only a one-field key document, where mongod allows several; + and the conflict check stops after 64 distinct paths in one update rather + than refusing a legal wide update. + - **M3 update operators** — open. `distinct` landed first because it was a whole missing command with no dependencies, and measuring it turned up three things worth keeping, none of which are `distinct`'s to fix: diff --git a/tests/spec/operators/README.md b/tests/spec/operators/README.md index 7caae44..5848e54 100644 --- a/tests/spec/operators/README.md +++ b/tests/spec/operators/README.md @@ -58,21 +58,27 @@ as a bare number would not have said so. ## Where it stands -Recorded against mongod 8.3.7, run against the server before any of it was -implemented: +Recorded against mongod 8.3.7. Green: ``` -array-ops.json 2 pass 24 fail 0 skip -current-date.json 3 pass 8 fail 0 skip -modifiers.json 7 pass 7 fail 0 skip -numeric.json 0 pass 21 fail 0 skip -push-modifiers.json 6 pass 15 fail 0 skip -set-on-insert.json 0 pass 9 fail 0 skip +array-ops.json 26 pass 0 fail 0 skip +current-date.json 11 pass 0 fail 0 skip +modifiers.json 14 pass 0 fail 0 skip +numeric.json 21 pass 0 fail 0 skip +push-modifiers.json 21 pass 0 fail 0 skip +set-on-insert.json 9 pass 0 fail 0 skip ``` -Red by construction. The 18 that pass are the shapes this server already -answers the way mongod does — mostly refusals that happen to agree, plus the -`$push` cases with no modifier on them. +It was recorded red — 18 pass / 84 fail against a server with none of these +operators — and driven green by seven commits. The 18 that passed then were +the shapes this server already answered mongod's way, mostly refusals that +happened to agree. + +It also found a bug nothing else here had: an upsert never reported the `_id` +it generated, so `updateOne(..., {upsert: true}).upsertedId` was null and +`findOneAndUpdate` with `returnDocument: after` returned a document with no +`_id`. This is the first corpus here that upserts into an empty collection and +then looks at what came back. ## What recording it settled