From 1892cb796951d91c483dec7116bf0e81dccce355 Mon Sep 17 00:00:00 2001 From: "A.Shakhmatov" Date: Mon, 10 Aug 2026 20:40:41 +0300 Subject: [PATCH] plan/spec: the positional gate goes green MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Scorecard 204 -> 218 pass, 87 -> 73 fail: the fourteen `arrayFilters` cases across five files, which is the whole of what the two implementation commits were expected to move and nothing else. Positional corpus 51/51. The three divergences measured on the way are written into PLAN §6 rather than left in commit messages: `$` with two predicates on one array that no element satisfies together, an array filter with a top-level `$and`/`$or`, and a literal index into a scalar element -- the last being the one place the positional walk and the plain indexed path now answer differently, which is worth a commit of its own and needs its own measurements first. The design review gets an outcome note, since two of its guesses were wrong and the corpus is where that was settled. --- PLAN.md | 33 +++++++++++++++++++++++---- docs/M3_ARRAYFILTERS_DESIGN_REVIEW.md | 8 +++++++ tests/spec/positional/README.md | 28 ++++++++++++----------- tests/spec/scorecard.txt | 24 ++++--------------- 4 files changed, 56 insertions(+), 37 deletions(-) diff --git a/PLAN.md b/PLAN.md index 502bc41..6c0d1e6 100644 --- a/PLAN.md +++ b/PLAN.md @@ -467,7 +467,7 @@ answers, and the trade is only acceptable because the lie is removed first. | M1 | **Cursors + wire polish** | getMore / killCursors / batchSize; server-side cursor state with idle timeout; sessions plumbing (lsid accepted) as drivers send it; hello advertisement updates; **`moreToCome` on requests** (see the bug below); command-monitoring assertions in the spec runner | crud spec suite green; e2e green | | M2 | **The `aggregate` command surface** | `$out` and `$merge` (7 of the 13 failures), and refusing every pipeline construct the engine does not implement instead of answering `0` (amendment A6). The other 6 failures are blocked on M2.5, M4 and M8 — see `docs/M2_DESIGN_REVIEW.md` §7 | `aggregate-*.json`: 0 fail among the 7 reachable cases | | M2.5 | **The aggregation engine** | expression evaluator, per-stage document iterator, the accumulators, `$unwind`; `$lookup`/`$facet` explicitly out of the first cut (amendment A6) | a purpose-built stage corpus, every expectation measured against mongod | -| M3 | **Update operators + index types** | `distinct` (**done**); positional paths refused rather than destructive (**done**); then `$`/`$[]`/`$[]` implemented, $setOnInsert, $addToSet, $mul, $min/$max, $pop, $pullAll, $currentDate, pipeline updates; partial + hashed indexes | `tests/spec/positional/` 0 fail (51 cases, recorded from mongod — the named gate could not see this work, see below); remaining crud coverage; e2e3/e2e4 green | +| M3 | **Update operators + index types** | `distinct` (**done**); positional paths refused rather than destructive (**done**); `$`/`$[]`/`$[]` + `arrayFilters` implemented (**done**); then $setOnInsert, $addToSet, $mul, $min/$max, $pop, $pullAll, $currentDate, pipeline updates; partial + hashed indexes | `tests/spec/positional/` 0 fail (51 cases, recorded from mongod — **green**; the named gate could not see this work, see below); remaining crud coverage; e2e3/e2e4 green | | M4 | **Sessions + transactions** | logical sessions, snapshot isolation on the mmap engine, write concern at commit | sessions + transactions spec suites green | | M5 | **Change streams** | change feed + resume tokens (likely log-seq based), getMore integration | change-streams spec suite green | | M6 | **Admin/ops commands** | dbStats, collStats, serverStatus, ping, buildInfo, listDatabases filters, dropDatabase durability (log it) | mongosh UX smoke; e2e green | @@ -586,9 +586,10 @@ gaps (`bad update`, `update must be a document` — M3), unimplemented commands `bulkWrite`/`insertMany`. That list, not the total, is the milestone backlog. All three named commands have since landed: `$out`/`$merge` in M2, `distinct` -as M3's first commit. The backlog the same grouping gives today is -`arrayFilters` (14 cases), the `findOneAndUpdate`/`findOneAndReplace` shapes -(~10), pipeline-form updates (~10), and `create-null-ids` (6). +as M3's first commit, and the `arrayFilters` cluster (14 cases) with M3's +positional operators. The backlog the same grouping gives today is the +`findOneAndUpdate`/`findOneAndReplace` shapes (~10), pipeline-form updates +(~10), and `create-null-ids` (6). --- @@ -1091,7 +1092,7 @@ has to be its own commit with its own re-recorded scorecard. `sources/`, every expectation recorded from mongod 8.3.7, run through the shared runner with `--suite-dir`. - It stands at 15 pass / 36 fail against the refusal, which is the intended + It stood at 15 pass / 36 fail against the refusal, which was the intended shape — `expressions.json` was 1/26 before the evaluator and is green now. Recording it settled a dozen things, and the first contradicts what the design review assumed: `y.$[i].c.$[i].d` reusing one identifier at two @@ -1100,6 +1101,28 @@ has to be its own commit with its own re-recorded scorecard. is error 28 where every other path failure is 2; a positional segment never creates, so a missing or non-array path is an error where `$set: {'a.b': 1}` would construct; and an upsert gets no special case. + + **Green as of the two implementation commits**, 51/51, and the pinned crud + scorecard moved 204 → 218 with it. Three divergences from mongod were + measured on the way and left in place rather than guessed at: + - **`$` with two predicates on one array.** `{"y.b": 3, "y.c": 2}` matches + `[{b: 3, c: 1}, {b: 1, c: 2}]` without either element satisfying both, and + the document still matched. mongod writes element 1; this writes element 0. + mongod's answer is an artefact of which predicate last wrote its match + position — the same query with the two predicates *reversed* still gives 1 + — so there is no rule here to copy, only a behaviour to record. + - **An array filter with a top-level `$and`/`$or`.** mongod accepts + `[{$or: [{"i.b": 3}]}]` and finds the identifier inside it; this refuses + with 9, the same answer it gives `[{}]`. The identifier is read off + top-level field names, and an operator is not one. + - **A literal index into a scalar element.** `$set: {"y.0.b": 1}` on + `y: [null]` is PathNotViable (28) on mongod; here it replaces the null + with `{b: 1}`. The positional walk refuses this — `Walk.element` is where + the corpus measures it — but the plain indexed path still takes + `set_path`'s creating branch. One rule, reached two ways, and only one way + agrees. Worth one commit, and it needs its own measurements first: the + padding case (`y.3.b` past the end) is a *legal* creation on mongod, so + the fix is not "refuse a non-document element". - **M3 update operators** — open. `distinct` landed first because it was a whole missing command with no dependencies, and measuring it turned up three things worth keeping, none of which are `distinct`'s to fix: diff --git a/docs/M3_ARRAYFILTERS_DESIGN_REVIEW.md b/docs/M3_ARRAYFILTERS_DESIGN_REVIEW.md index 3ce9277..d5a1fcc 100644 --- a/docs/M3_ARRAYFILTERS_DESIGN_REVIEW.md +++ b/docs/M3_ARRAYFILTERS_DESIGN_REVIEW.md @@ -10,6 +10,14 @@ Everything below was measured on 2026-08-10 against mongod 8.3.7 on `:27099` and this server at `5942f5e` on `:27020`, running the identical probe against both. +**Outcome.** All of it landed, in the order §6 recommended: the refusal +(`f04e712`), the corpus (`e5a84c0`), then the implementation in two commits. +The corpus is 51/51 and the pinned crud scorecard moved 204 → 218. Two of the +review's own guesses were wrong and the corpus caught both — a reused +identifier is accepted, and `$[]` in first position answers the array-filter +message rather than the `$` one. The divergences that remain are listed in +PLAN §6 under the M3 gate; this document is not the place to track them. + --- ## 1. The plan names a feature; the measurement found data loss diff --git a/tests/spec/positional/README.md b/tests/spec/positional/README.md index 2059254..1a9cbdc 100644 --- a/tests/spec/positional/README.md +++ b/tests/spec/positional/README.md @@ -72,24 +72,26 @@ measures the version gap, not the engine. ## Where it stands -Recorded against mongod 8.3.7, run against the server at the positional -refusal: +Recorded against mongod 8.3.7. Green: ``` -filtered.json 8 pass 19 fail 0 skip -all-positional.json 3 pass 9 fail 0 skip -first-positional.json 4 pass 8 fail 0 skip +filtered.json 27 pass 0 fail 0 skip +all-positional.json 12 pass 0 fail 0 skip +first-positional.json 12 pass 0 fail 0 skip ``` -Red by construction and by design. The 15 that pass are the refusals where -this server's code already matches mongod's; of the 36 that fail, 31 are the -constructs themselves answering "not implemented by this server", and 5 are -refusals whose code differs — four of them the `arrayFilters` validation this -server cannot do yet because it never parses the option. +It was recorded red — 15 pass / 36 fail against the positional refusal — and +driven green by the two implementation commits, the same shape +`tests/spec/aggregate/expressions.json` had at 1 pass / 26 fail before the +expression evaluator existed. The 15 that passed then were the refusals where +this server already agreed with mongod, which is the only part of a red gate +that is worth anything: it says the corpus is measuring the server and not +the harness. -That is the intended shape. `tests/spec/aggregate/expressions.json` was -recorded at 1 pass / 26 fail before the evaluator existed and is green now; -this is the same gate at the same stage. +Three answers here still differ from mongod and are green only because no +case covers them; they are written down in PLAN §6 rather than papered over — +`$` with two disagreeing predicates on one array, an array filter with a +top-level `$and`/`$or`, and a literal index into a scalar element. ## What recording it settled diff --git a/tests/spec/scorecard.txt b/tests/spec/scorecard.txt index 31134c4..2f6eef4 100644 --- a/tests/spec/scorecard.txt +++ b/tests/spec/scorecard.txt @@ -18,7 +18,7 @@ # hasServerConnectionId, and maxTimeMS in an expected command (CSOT rewrites # it -- the only assertion this runner declines to make). -total 204 pass 87 fail 196 skip 175 files 0 errored +total 218 pass 73 fail 196 skip 175 files 0 errored # per-file: name pass fail skip aggregate-allowdiskuse.json 3 0 0 @@ -31,7 +31,7 @@ aggregate-out.json 2 0 0 aggregate-rawdata.json 1 0 1 aggregate-write-readPreference.json 0 0 4 aggregate.json 5 0 2 -bulkWrite-arrayFilters.json 0 3 0 +bulkWrite-arrayFilters.json 3 0 0 bulkWrite-collation.json 0 2 0 bulkWrite-comment.json 2 0 1 bulkWrite-delete-hint-serverError.json 0 0 2 @@ -141,7 +141,7 @@ findOneAndReplace-let.json 0 1 1 findOneAndReplace-rawdata.json 1 0 1 findOneAndReplace-upsert.json 2 2 0 findOneAndReplace.json 4 2 0 -findOneAndUpdate-arrayFilters.json 0 3 0 +findOneAndUpdate-arrayFilters.json 3 0 0 findOneAndUpdate-collation.json 0 1 0 findOneAndUpdate-comment.json 0 2 1 findOneAndUpdate-dots_and_dollars.json 0 0 4 @@ -172,7 +172,7 @@ replaceOne-rawdata.json 1 0 1 replaceOne-sort.json 1 0 1 replaceOne-validation.json 1 0 0 replaceOne.json 5 0 0 -updateMany-arrayFilters.json 0 3 0 +updateMany-arrayFilters.json 3 0 0 updateMany-collation.json 0 1 0 updateMany-comment.json 2 0 1 updateMany-dots_and_dollars.json 0 0 4 @@ -183,7 +183,7 @@ updateMany-pipeline.json 0 1 0 updateMany-rawdata.json 1 0 1 updateMany-validation.json 1 0 0 updateMany.json 4 0 0 -updateOne-arrayFilters.json 0 5 0 +updateOne-arrayFilters.json 5 0 0 updateOne-collation.json 0 1 0 updateOne-comment.json 2 0 1 updateOne-dots_and_dollars.json 0 0 4 @@ -209,9 +209,6 @@ aggregate-rawdata.json SKIP Aggregate with rawData option needs server >= 8.2.0 aggregate-write-readPreference.json SKIP * needs topology replicaset/sharded/load-balanced aggregate.json SKIP aggregate with a document comment - pre 4.4 needs server <= 4.2.99 aggregate.json SKIP aggregate with comment does not set comment on getMore - pre 4.4 needs server <= 4.3.99 -bulkWrite-arrayFilters.json FAIL BulkWrite updateOne with arrayFilters MongoBulkWriteError: the positional operator '$[i]' in path 'y.$[i].b' is not implemented by this server -bulkWrite-arrayFilters.json FAIL BulkWrite updateMany with arrayFilters MongoBulkWriteError: the positional operator '$[i]' in path 'y.$[i].b' is not implemented by this server -bulkWrite-arrayFilters.json FAIL BulkWrite with arrayFilters MongoBulkWriteError: the positional operator '$[i]' in path 'y.$[i].b' is not implemented by this server bulkWrite-collation.json FAIL BulkWrite with delete operations and collation bulkWrite.deletedCount: expected 4, got 0 bulkWrite-collation.json FAIL BulkWrite with update operations and collation bulkWrite.matchedCount: expected 6, got 2 bulkWrite-comment.json SKIP BulkWrite with comment - pre 4.4 needs server <= 4.2.99 @@ -361,9 +358,6 @@ findOneAndReplace-upsert.json FAIL FindOneAndReplace when no documents match wit findOneAndReplace-upsert.json FAIL FindOneAndReplace when no documents match with id specified with upsert returning the document after modification findOneAndReplace: expected a document, got null findOneAndReplace.json FAIL FindOneAndReplace when many documents match returning the document after modification findOneAndReplace.x: expected 32, got 22 findOneAndReplace.json FAIL FindOneAndReplace when one document matches returning the document after modification findOneAndReplace.x: expected 32, got 22 -findOneAndUpdate-arrayFilters.json FAIL FindOneAndUpdate when no document matches arrayFilters MongoServerError: the positional operator '$[i]' in path 'y.$[i].b' is not implemented by this server -findOneAndUpdate-arrayFilters.json FAIL FindOneAndUpdate when one document matches arrayFilters MongoServerError: the positional operator '$[i]' in path 'y.$[i].b' is not implemented by this server -findOneAndUpdate-arrayFilters.json FAIL FindOneAndUpdate when multiple documents match arrayFilters MongoServerError: the positional operator '$[i]' in path 'y.$[i].b' is not implemented by this server findOneAndUpdate-collation.json FAIL FindOneAndUpdate when many documents match with collation returning the document before modification findOneAndUpdate: expected a document, got null findOneAndUpdate-comment.json FAIL findOneAndUpdate with string comment MongoServerError: update must be a document findOneAndUpdate-comment.json FAIL findOneAndUpdate with document comment MongoServerError: update must be a document @@ -403,9 +397,6 @@ replaceOne-let.json SKIP ReplaceOne with let option needs server >= 5.0 replaceOne-let.json FAIL ReplaceOne with let option unsupported (server-side error) replaceOne: expected an error, the operation succeeded replaceOne-rawdata.json SKIP ReplaceOne with rawData option needs server >= 8.2.0 replaceOne-sort.json SKIP ReplaceOne with sort option needs server >= 8.0 -updateMany-arrayFilters.json FAIL UpdateMany when no documents match arrayFilters MongoServerError: the positional operator '$[i]' in path 'y.$[i].b' is not implemented by this server -updateMany-arrayFilters.json FAIL UpdateMany when one document matches arrayFilters MongoServerError: the positional operator '$[i]' in path 'y.$[i].b' is not implemented by this server -updateMany-arrayFilters.json FAIL UpdateMany when multiple documents match arrayFilters MongoServerError: the positional operator '$[i]' in path 'y.$[i].b' is not implemented by this server updateMany-collation.json FAIL UpdateMany when many documents match with collation updateMany.matchedCount: expected 2, got 1 updateMany-comment.json SKIP UpdateMany with comment - pre 4.4 needs server <= 4.2.99 updateMany-dots_and_dollars.json SKIP Updating document to set top-level dollar-prefixed key on 5.0+ server needs server >= 5.0 @@ -416,11 +407,6 @@ updateMany-let.json SKIP updateMany with let option needs server >= 5.0 updateMany-let.json FAIL updateMany with let option unsupported (server-side error) updateMany: error message "update spec requires u" does not contain "'update.let' is an unknown field" updateMany-pipeline.json FAIL UpdateMany using pipelines MongoServerError: update spec requires u updateMany-rawdata.json SKIP updateMany with rawData option needs server >= 8.2.0 -updateOne-arrayFilters.json FAIL UpdateOne when no document matches arrayFilters MongoServerError: the positional operator '$[i]' in path 'y.$[i].b' is not implemented by this server -updateOne-arrayFilters.json FAIL UpdateOne when one document matches arrayFilters MongoServerError: the positional operator '$[i]' in path 'y.$[i].b' is not implemented by this server -updateOne-arrayFilters.json FAIL UpdateOne when multiple documents match arrayFilters MongoServerError: the positional operator '$[i]' in path 'y.$[i].b' is not implemented by this server -updateOne-arrayFilters.json FAIL UpdateOne when no documents match multiple arrayFilters MongoServerError: the positional operator '$[i]' in path 'y.$[i].c.$[j].d' is not implemented by this server -updateOne-arrayFilters.json FAIL UpdateOne when one document matches multiple arrayFilters MongoServerError: the positional operator '$[i]' in path 'y.$[i].c.$[j].d' is not implemented by this server updateOne-collation.json FAIL UpdateOne when one document matches with collation updateOne.matchedCount: expected 1, got 0 updateOne-comment.json SKIP UpdateOne with comment - pre 4.4 needs server <= 4.2.99 updateOne-dots_and_dollars.json SKIP Updating document to set top-level dollar-prefixed key on 5.0+ server needs server >= 5.0