diff --git a/PLAN.md b/PLAN.md index 502bc41..6c0d1e6 100644 --- a/PLAN.md +++ b/PLAN.md @@ -467,7 +467,7 @@ answers, and the trade is only acceptable because the lie is removed first. | M1 | **Cursors + wire polish** | getMore / killCursors / batchSize; server-side cursor state with idle timeout; sessions plumbing (lsid accepted) as drivers send it; hello advertisement updates; **`moreToCome` on requests** (see the bug below); command-monitoring assertions in the spec runner | crud spec suite green; e2e green | | M2 | **The `aggregate` command surface** | `$out` and `$merge` (7 of the 13 failures), and refusing every pipeline construct the engine does not implement instead of answering `0` (amendment A6). The other 6 failures are blocked on M2.5, M4 and M8 — see `docs/M2_DESIGN_REVIEW.md` §7 | `aggregate-*.json`: 0 fail among the 7 reachable cases | | M2.5 | **The aggregation engine** | expression evaluator, per-stage document iterator, the accumulators, `$unwind`; `$lookup`/`$facet` explicitly out of the first cut (amendment A6) | a purpose-built stage corpus, every expectation measured against mongod | -| M3 | **Update operators + index types** | `distinct` (**done**); positional paths refused rather than destructive (**done**); then `$`/`$[]`/`$[]` implemented, $setOnInsert, $addToSet, $mul, $min/$max, $pop, $pullAll, $currentDate, pipeline updates; partial + hashed indexes | `tests/spec/positional/` 0 fail (51 cases, recorded from mongod — the named gate could not see this work, see below); remaining crud coverage; e2e3/e2e4 green | +| M3 | **Update operators + index types** | `distinct` (**done**); positional paths refused rather than destructive (**done**); `$`/`$[]`/`$[]` + `arrayFilters` implemented (**done**); then $setOnInsert, $addToSet, $mul, $min/$max, $pop, $pullAll, $currentDate, pipeline updates; partial + hashed indexes | `tests/spec/positional/` 0 fail (51 cases, recorded from mongod — **green**; the named gate could not see this work, see below); remaining crud coverage; e2e3/e2e4 green | | M4 | **Sessions + transactions** | logical sessions, snapshot isolation on the mmap engine, write concern at commit | sessions + transactions spec suites green | | M5 | **Change streams** | change feed + resume tokens (likely log-seq based), getMore integration | change-streams spec suite green | | M6 | **Admin/ops commands** | dbStats, collStats, serverStatus, ping, buildInfo, listDatabases filters, dropDatabase durability (log it) | mongosh UX smoke; e2e green | @@ -586,9 +586,10 @@ gaps (`bad update`, `update must be a document` — M3), unimplemented commands `bulkWrite`/`insertMany`. That list, not the total, is the milestone backlog. All three named commands have since landed: `$out`/`$merge` in M2, `distinct` -as M3's first commit. The backlog the same grouping gives today is -`arrayFilters` (14 cases), the `findOneAndUpdate`/`findOneAndReplace` shapes -(~10), pipeline-form updates (~10), and `create-null-ids` (6). +as M3's first commit, and the `arrayFilters` cluster (14 cases) with M3's +positional operators. The backlog the same grouping gives today is the +`findOneAndUpdate`/`findOneAndReplace` shapes (~10), pipeline-form updates +(~10), and `create-null-ids` (6). --- @@ -1091,7 +1092,7 @@ has to be its own commit with its own re-recorded scorecard. `sources/`, every expectation recorded from mongod 8.3.7, run through the shared runner with `--suite-dir`. - It stands at 15 pass / 36 fail against the refusal, which is the intended + It stood at 15 pass / 36 fail against the refusal, which was the intended shape — `expressions.json` was 1/26 before the evaluator and is green now. Recording it settled a dozen things, and the first contradicts what the design review assumed: `y.$[i].c.$[i].d` reusing one identifier at two @@ -1100,6 +1101,28 @@ has to be its own commit with its own re-recorded scorecard. is error 28 where every other path failure is 2; a positional segment never creates, so a missing or non-array path is an error where `$set: {'a.b': 1}` would construct; and an upsert gets no special case. + + **Green as of the two implementation commits**, 51/51, and the pinned crud + scorecard moved 204 → 218 with it. Three divergences from mongod were + measured on the way and left in place rather than guessed at: + - **`$` with two predicates on one array.** `{"y.b": 3, "y.c": 2}` matches + `[{b: 3, c: 1}, {b: 1, c: 2}]` without either element satisfying both, and + the document still matched. mongod writes element 1; this writes element 0. + mongod's answer is an artefact of which predicate last wrote its match + position — the same query with the two predicates *reversed* still gives 1 + — so there is no rule here to copy, only a behaviour to record. + - **An array filter with a top-level `$and`/`$or`.** mongod accepts + `[{$or: [{"i.b": 3}]}]` and finds the identifier inside it; this refuses + with 9, the same answer it gives `[{}]`. The identifier is read off + top-level field names, and an operator is not one. + - **A literal index into a scalar element.** `$set: {"y.0.b": 1}` on + `y: [null]` is PathNotViable (28) on mongod; here it replaces the null + with `{b: 1}`. The positional walk refuses this — `Walk.element` is where + the corpus measures it — but the plain indexed path still takes + `set_path`'s creating branch. One rule, reached two ways, and only one way + agrees. Worth one commit, and it needs its own measurements first: the + padding case (`y.3.b` past the end) is a *legal* creation on mongod, so + the fix is not "refuse a non-document element". - **M3 update operators** — open. `distinct` landed first because it was a whole missing command with no dependencies, and measuring it turned up three things worth keeping, none of which are `distinct`'s to fix: diff --git a/docs/M3_ARRAYFILTERS_DESIGN_REVIEW.md b/docs/M3_ARRAYFILTERS_DESIGN_REVIEW.md index 3ce9277..d5a1fcc 100644 --- a/docs/M3_ARRAYFILTERS_DESIGN_REVIEW.md +++ b/docs/M3_ARRAYFILTERS_DESIGN_REVIEW.md @@ -10,6 +10,14 @@ Everything below was measured on 2026-08-10 against mongod 8.3.7 on `:27099` and this server at `5942f5e` on `:27020`, running the identical probe against both. +**Outcome.** All of it landed, in the order §6 recommended: the refusal +(`f04e712`), the corpus (`e5a84c0`), then the implementation in two commits. +The corpus is 51/51 and the pinned crud scorecard moved 204 → 218. Two of the +review's own guesses were wrong and the corpus caught both — a reused +identifier is accepted, and `$[]` in first position answers the array-filter +message rather than the `$` one. The divergences that remain are listed in +PLAN §6 under the M3 gate; this document is not the place to track them. + --- ## 1. The plan names a feature; the measurement found data loss diff --git a/tests/spec/positional/README.md b/tests/spec/positional/README.md index 2059254..1a9cbdc 100644 --- a/tests/spec/positional/README.md +++ b/tests/spec/positional/README.md @@ -72,24 +72,26 @@ measures the version gap, not the engine. ## Where it stands -Recorded against mongod 8.3.7, run against the server at the positional -refusal: +Recorded against mongod 8.3.7. Green: ``` -filtered.json 8 pass 19 fail 0 skip -all-positional.json 3 pass 9 fail 0 skip -first-positional.json 4 pass 8 fail 0 skip +filtered.json 27 pass 0 fail 0 skip +all-positional.json 12 pass 0 fail 0 skip +first-positional.json 12 pass 0 fail 0 skip ``` -Red by construction and by design. The 15 that pass are the refusals where -this server's code already matches mongod's; of the 36 that fail, 31 are the -constructs themselves answering "not implemented by this server", and 5 are -refusals whose code differs — four of them the `arrayFilters` validation this -server cannot do yet because it never parses the option. +It was recorded red — 15 pass / 36 fail against the positional refusal — and +driven green by the two implementation commits, the same shape +`tests/spec/aggregate/expressions.json` had at 1 pass / 26 fail before the +expression evaluator existed. The 15 that passed then were the refusals where +this server already agreed with mongod, which is the only part of a red gate +that is worth anything: it says the corpus is measuring the server and not +the harness. -That is the intended shape. `tests/spec/aggregate/expressions.json` was -recorded at 1 pass / 26 fail before the evaluator existed and is green now; -this is the same gate at the same stage. +Three answers here still differ from mongod and are green only because no +case covers them; they are written down in PLAN §6 rather than papered over — +`$` with two disagreeing predicates on one array, an array filter with a +top-level `$and`/`$or`, and a literal index into a scalar element. ## What recording it settled diff --git a/tests/spec/scorecard.txt b/tests/spec/scorecard.txt index 31134c4..2f6eef4 100644 --- a/tests/spec/scorecard.txt +++ b/tests/spec/scorecard.txt @@ -18,7 +18,7 @@ # hasServerConnectionId, and maxTimeMS in an expected command (CSOT rewrites # it -- the only assertion this runner declines to make). -total 204 pass 87 fail 196 skip 175 files 0 errored +total 218 pass 73 fail 196 skip 175 files 0 errored # per-file: name pass fail skip aggregate-allowdiskuse.json 3 0 0 @@ -31,7 +31,7 @@ aggregate-out.json 2 0 0 aggregate-rawdata.json 1 0 1 aggregate-write-readPreference.json 0 0 4 aggregate.json 5 0 2 -bulkWrite-arrayFilters.json 0 3 0 +bulkWrite-arrayFilters.json 3 0 0 bulkWrite-collation.json 0 2 0 bulkWrite-comment.json 2 0 1 bulkWrite-delete-hint-serverError.json 0 0 2 @@ -141,7 +141,7 @@ findOneAndReplace-let.json 0 1 1 findOneAndReplace-rawdata.json 1 0 1 findOneAndReplace-upsert.json 2 2 0 findOneAndReplace.json 4 2 0 -findOneAndUpdate-arrayFilters.json 0 3 0 +findOneAndUpdate-arrayFilters.json 3 0 0 findOneAndUpdate-collation.json 0 1 0 findOneAndUpdate-comment.json 0 2 1 findOneAndUpdate-dots_and_dollars.json 0 0 4 @@ -172,7 +172,7 @@ replaceOne-rawdata.json 1 0 1 replaceOne-sort.json 1 0 1 replaceOne-validation.json 1 0 0 replaceOne.json 5 0 0 -updateMany-arrayFilters.json 0 3 0 +updateMany-arrayFilters.json 3 0 0 updateMany-collation.json 0 1 0 updateMany-comment.json 2 0 1 updateMany-dots_and_dollars.json 0 0 4 @@ -183,7 +183,7 @@ updateMany-pipeline.json 0 1 0 updateMany-rawdata.json 1 0 1 updateMany-validation.json 1 0 0 updateMany.json 4 0 0 -updateOne-arrayFilters.json 0 5 0 +updateOne-arrayFilters.json 5 0 0 updateOne-collation.json 0 1 0 updateOne-comment.json 2 0 1 updateOne-dots_and_dollars.json 0 0 4 @@ -209,9 +209,6 @@ aggregate-rawdata.json SKIP Aggregate with rawData option needs server >= 8.2.0 aggregate-write-readPreference.json SKIP * needs topology replicaset/sharded/load-balanced aggregate.json SKIP aggregate with a document comment - pre 4.4 needs server <= 4.2.99 aggregate.json SKIP aggregate with comment does not set comment on getMore - pre 4.4 needs server <= 4.3.99 -bulkWrite-arrayFilters.json FAIL BulkWrite updateOne with arrayFilters MongoBulkWriteError: the positional operator '$[i]' in path 'y.$[i].b' is not implemented by this server -bulkWrite-arrayFilters.json FAIL BulkWrite updateMany with arrayFilters MongoBulkWriteError: the positional operator '$[i]' in path 'y.$[i].b' is not implemented by this server -bulkWrite-arrayFilters.json FAIL BulkWrite with arrayFilters MongoBulkWriteError: the positional operator '$[i]' in path 'y.$[i].b' is not implemented by this server bulkWrite-collation.json FAIL BulkWrite with delete operations and collation bulkWrite.deletedCount: expected 4, got 0 bulkWrite-collation.json FAIL BulkWrite with update operations and collation bulkWrite.matchedCount: expected 6, got 2 bulkWrite-comment.json SKIP BulkWrite with comment - pre 4.4 needs server <= 4.2.99 @@ -361,9 +358,6 @@ findOneAndReplace-upsert.json FAIL FindOneAndReplace when no documents match wit findOneAndReplace-upsert.json FAIL FindOneAndReplace when no documents match with id specified with upsert returning the document after modification findOneAndReplace: expected a document, got null findOneAndReplace.json FAIL FindOneAndReplace when many documents match returning the document after modification findOneAndReplace.x: expected 32, got 22 findOneAndReplace.json FAIL FindOneAndReplace when one document matches returning the document after modification findOneAndReplace.x: expected 32, got 22 -findOneAndUpdate-arrayFilters.json FAIL FindOneAndUpdate when no document matches arrayFilters MongoServerError: the positional operator '$[i]' in path 'y.$[i].b' is not implemented by this server -findOneAndUpdate-arrayFilters.json FAIL FindOneAndUpdate when one document matches arrayFilters MongoServerError: the positional operator '$[i]' in path 'y.$[i].b' is not implemented by this server -findOneAndUpdate-arrayFilters.json FAIL FindOneAndUpdate when multiple documents match arrayFilters MongoServerError: the positional operator '$[i]' in path 'y.$[i].b' is not implemented by this server findOneAndUpdate-collation.json FAIL FindOneAndUpdate when many documents match with collation returning the document before modification findOneAndUpdate: expected a document, got null findOneAndUpdate-comment.json FAIL findOneAndUpdate with string comment MongoServerError: update must be a document findOneAndUpdate-comment.json FAIL findOneAndUpdate with document comment MongoServerError: update must be a document @@ -403,9 +397,6 @@ replaceOne-let.json SKIP ReplaceOne with let option needs server >= 5.0 replaceOne-let.json FAIL ReplaceOne with let option unsupported (server-side error) replaceOne: expected an error, the operation succeeded replaceOne-rawdata.json SKIP ReplaceOne with rawData option needs server >= 8.2.0 replaceOne-sort.json SKIP ReplaceOne with sort option needs server >= 8.0 -updateMany-arrayFilters.json FAIL UpdateMany when no documents match arrayFilters MongoServerError: the positional operator '$[i]' in path 'y.$[i].b' is not implemented by this server -updateMany-arrayFilters.json FAIL UpdateMany when one document matches arrayFilters MongoServerError: the positional operator '$[i]' in path 'y.$[i].b' is not implemented by this server -updateMany-arrayFilters.json FAIL UpdateMany when multiple documents match arrayFilters MongoServerError: the positional operator '$[i]' in path 'y.$[i].b' is not implemented by this server updateMany-collation.json FAIL UpdateMany when many documents match with collation updateMany.matchedCount: expected 2, got 1 updateMany-comment.json SKIP UpdateMany with comment - pre 4.4 needs server <= 4.2.99 updateMany-dots_and_dollars.json SKIP Updating document to set top-level dollar-prefixed key on 5.0+ server needs server >= 5.0 @@ -416,11 +407,6 @@ updateMany-let.json SKIP updateMany with let option needs server >= 5.0 updateMany-let.json FAIL updateMany with let option unsupported (server-side error) updateMany: error message "update spec requires u" does not contain "'update.let' is an unknown field" updateMany-pipeline.json FAIL UpdateMany using pipelines MongoServerError: update spec requires u updateMany-rawdata.json SKIP updateMany with rawData option needs server >= 8.2.0 -updateOne-arrayFilters.json FAIL UpdateOne when no document matches arrayFilters MongoServerError: the positional operator '$[i]' in path 'y.$[i].b' is not implemented by this server -updateOne-arrayFilters.json FAIL UpdateOne when one document matches arrayFilters MongoServerError: the positional operator '$[i]' in path 'y.$[i].b' is not implemented by this server -updateOne-arrayFilters.json FAIL UpdateOne when multiple documents match arrayFilters MongoServerError: the positional operator '$[i]' in path 'y.$[i].b' is not implemented by this server -updateOne-arrayFilters.json FAIL UpdateOne when no documents match multiple arrayFilters MongoServerError: the positional operator '$[i]' in path 'y.$[i].c.$[j].d' is not implemented by this server -updateOne-arrayFilters.json FAIL UpdateOne when one document matches multiple arrayFilters MongoServerError: the positional operator '$[i]' in path 'y.$[i].c.$[j].d' is not implemented by this server updateOne-collation.json FAIL UpdateOne when one document matches with collation updateOne.matchedCount: expected 1, got 0 updateOne-comment.json SKIP UpdateOne with comment - pre 4.4 needs server <= 4.2.99 updateOne-dots_and_dollars.json SKIP Updating document to set top-level dollar-prefixed key on 5.0+ server needs server >= 5.0