From eaf8d7c677b42ee11810d80a7489360dc43b12ad Mon Sep 17 00:00:00 2001 From: "A.Shakhmatov" Date: Mon, 10 Aug 2026 21:02:51 +0300 Subject: [PATCH 1/9] tests/spec: a recorded corpus for the update operators MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit PLAN §3 lists eight operators for M3 -- `$setOnInsert`, `$addToSet`, `$mul`, `$min`, `$max`, `$pop`, `$pullAll`, `$currentDate` -- and the pinned crud corpus says almost nothing about any of them. A probe running the identical update against mongod 8.3.7 and this server found all eight answering `bad update`, code 2, one message for every question. It also found the thing this directory exists for: `$push`'s `$slice`, `$position` and `$sort` are **silently ignored**. `{$each: [3, 4], $slice: -3}` appends both values, slices nothing, and answers ok: 1 with modifiedCount: 1. A missing operator is an error the client can see; a modifier that is parsed, accepted and then dropped is the same class of wrong answer the positional operators were. 103 cases in six files, 18 pass / 84 fail -- red by construction, like `tests/spec/aggregate/expressions.json` at 1/26 and the positional corpus at 15/36. Inputs authored in `sources/`, every expectation measured. Two things here cannot be recorded as values, and both become a `$$type` assertion rather than being left out: a `$currentDate` field is whatever the clock said (named per case in `volatile`, so a real stored date can still be pinned one day), and an upsert that inserts gets a generated ObjectId (that one automatic -- no source authors an ObjectId). Everything else is compared exactly. Files are canonical extended JSON, which the runner already parses that way: `$mul` overflowing an int32 produces an int64, and writing `4000000000` as a bare number would not have said so. What recording it settled, none of it guessable: - `$mul` of a missing field writes **0**, not the operand; of a non-numeric field, or by one, TypeMismatch (14). - `$min`/`$max` are not numeric operators. They compare in BSON canonical order, so `$min: {s: 5}` on `s: "b"` writes 5, and a missing field is always written. - two operators writing one field is **ConflictingUpdateOperators (40)** -- `$min`+`$max`, `$set`+`$inc`, `$setOnInsert`+`$set`. A whole error class this server does not have. - `$addToSet` compares documents whole, **field order included**: `{a:1,b:2}` and `{b:2,a:1}` are two values. But `2` and `2.0` are one. - `$push` modifiers are only modifiers when `$each` is there: `{$slice: 1}` alone is a value to push. With it, the order is position, then sort the whole array, then slice. - `$currentDate` with `false` still writes a date. - `$setOnInsert` may write `_id` on an insert, where `$set` may not. - an unknown modifier is FailedToParse (9), not BadValue. One case was authored and then removed: `{b: 1, $set: {c: 1}}` never reaches a server -- the driver rejects it -- so there was no answer to record and the case would have asserted nothing. --- tests/spec/operators/README.md | 103 ++++++++ tests/spec/operators/array-ops.json | 1 + tests/spec/operators/current-date.json | 1 + tests/spec/operators/modifiers.json | 1 + tests/spec/operators/numeric.json | 1 + tests/spec/operators/push-modifiers.json | 1 + tests/spec/operators/record.js | 222 ++++++++++++++++++ tests/spec/operators/set-on-insert.json | 1 + tests/spec/operators/sources/array-ops.json | 153 ++++++++++++ .../spec/operators/sources/current-date.json | 81 +++++++ tests/spec/operators/sources/modifiers.json | 88 +++++++ tests/spec/operators/sources/numeric.json | 122 ++++++++++ .../operators/sources/push-modifiers.json | 122 ++++++++++ .../spec/operators/sources/set-on-insert.json | 99 ++++++++ tests/spec/run.js | 6 +- 15 files changed, 999 insertions(+), 3 deletions(-) create mode 100644 tests/spec/operators/README.md create mode 100644 tests/spec/operators/array-ops.json create mode 100644 tests/spec/operators/current-date.json create mode 100644 tests/spec/operators/modifiers.json create mode 100644 tests/spec/operators/numeric.json create mode 100644 tests/spec/operators/push-modifiers.json create mode 100644 tests/spec/operators/record.js create mode 100644 tests/spec/operators/set-on-insert.json create mode 100644 tests/spec/operators/sources/array-ops.json create mode 100644 tests/spec/operators/sources/current-date.json create mode 100644 tests/spec/operators/sources/modifiers.json create mode 100644 tests/spec/operators/sources/numeric.json create mode 100644 tests/spec/operators/sources/push-modifiers.json create mode 100644 tests/spec/operators/sources/set-on-insert.json diff --git a/tests/spec/operators/README.md b/tests/spec/operators/README.md new file mode 100644 index 0000000..7caae44 --- /dev/null +++ b/tests/spec/operators/README.md @@ -0,0 +1,103 @@ +# The update-operator corpus + +PLAN §3 lists eight update operators for M3 — `$setOnInsert`, `$addToSet`, +`$mul`, `$min`, `$max`, `$pop`, `$pullAll`, `$currentDate` — and the pinned +crud corpus says almost nothing about any of them. A probe running the +identical update against mongod 8.3.7 and this server found: + +- all eight answering `bad update`, code 2, one message for every question; +- `$push`'s `$slice`, `$position` and `$sort` **silently ignored**. + `{$each: [3, 4], $slice: -3}` appended both values, sliced nothing, and + answered `ok: 1` with `modifiedCount: 1`. + +The second is the reason this directory exists rather than a list of TODOs. +A missing operator is an error the client can see; a modifier that is parsed, +accepted and then dropped is the same class of wrong answer the positional +operators were — the client asked for one thing and was told it got it. + +## The one rule + +**Inputs are authored here; expectations are measured against a real mongod.** + +``` +tests/spec/operators/ + sources/*.json documents + operations, authored + record.js runs them against mongod, writes the expectations + *.json generated, unified format, do not hand-edit +``` + +```sh +mongod --port 27099 --dbpath +node tests/spec/operators/record.js --mongod-port 27099 +node tests/spec/run.js --suite-dir tests/spec/operators +``` + +Same discipline as `tests/spec/positional/` and `tests/spec/aggregate/`, and +for the same reason: a corpus written end to end here can encode our own bugs +as expectations and then agree with us forever. + +## What cannot be recorded as a value + +Two things in this corpus are not predictable, and both are replaced by a +`$$type` assertion rather than left out: + +- a `$currentDate` field is whatever the clock said. The case names it in + `volatile`, per case, so a corpus that one day wants to pin a real stored + date still can. +- an upsert that inserts gets a generated ObjectId. That one is automatic: + no source file authors an ObjectId, so the rule is unambiguous. + +Everything else about the document — which fields exist, in what order, +holding what — is still compared exactly. A case that dropped a field still +fails. + +Files are written with canonical extended JSON (`relaxed: false`), which the +runner already parses that way. It is verbose and it is exact: `$mul` +overflowing an int32 produces an int64, and a corpus that wrote `4000000000` +as a bare number would not have said so. + +## Where it stands + +Recorded against mongod 8.3.7, run against the server before any of it was +implemented: + +``` +array-ops.json 2 pass 24 fail 0 skip +current-date.json 3 pass 8 fail 0 skip +modifiers.json 7 pass 7 fail 0 skip +numeric.json 0 pass 21 fail 0 skip +push-modifiers.json 6 pass 15 fail 0 skip +set-on-insert.json 0 pass 9 fail 0 skip +``` + +Red by construction. The 18 that pass are the shapes this server already +answers the way mongod does — mostly refusals that happen to agree, plus the +`$push` cases with no modifier on them. + +## What recording it settled + +None of this is guessable, and several rows contradict the obvious reading: + +| | mongod | +|---|---| +| `$mul` of a missing field | writes **0**, not the operand | +| `$mul` of a non-numeric field, or by one | TypeMismatch (14) | +| `$min`/`$max` across types | compares in BSON canonical order, so `$min: {s: 5}` on `s: "b"` writes 5 | +| `$min`/`$max` of a missing field | always writes | +| two operators writing one field | **ConflictingUpdateOperators (40)** — `$min`+`$max`, `$set`+`$inc`, `$setOnInsert`+`$set` | +| `$addToSet` of a document | compares whole, **field order included**: `{a:1,b:2}` and `{b:2,a:1}` are two values | +| `$addToSet` of `2` and `2.0` | one value | +| `$pop` of an empty or missing field | no-op, not an error | +| `$pop` with an argument that is not ±1 | FailedToParse (9); on a non-array field, TypeMismatch (14) | +| `$pullAll` with a non-array argument | BadValue (2) | +| `$push` modifiers without `$each` | **not modifiers at all** — `{$slice: 1}` is pushed as a value | +| `$push` modifier order | insert at `$position`, then `$sort` the whole array, then `$slice` | +| `$position` negative | counted from the end | +| `$currentDate` with `false` | still writes a date; the boolean's value is ignored | +| `$currentDate` with anything but a bool or `{$type: date\|timestamp}` | BadValue (2) | +| `$setOnInsert` writing `_id` on an insert | **allowed**, unlike `$set` | +| an unknown modifier | FailedToParse (9), not BadValue | + +One case was authored and then removed: `{b: 1, $set: {c: 1}}` is rejected by +the driver before it reaches a server, so there is no server answer to record +and the case would have asserted nothing. diff --git a/tests/spec/operators/array-ops.json b/tests/spec/operators/array-ops.json new file mode 100644 index 0000000..7692dba --- /dev/null +++ b/tests/spec/operators/array-ops.json @@ -0,0 +1 @@ +{"description":"array-ops","schemaVersion":"1.4","createEntities":[{"client":{"id":"client0"}},{"database":{"id":"database0","client":"client0","databaseName":"operator-corpus"}},{"collection":{"id":"collection0","database":"database0","collectionName":"coll"}}],"initialData":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"t":[{"$numberInt":"1"},{"$numberInt":"2"},{"$numberInt":"3"}]}]}],"tests":[{"description":"$addToSet appends a value the array does not hold","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$addToSet":{"t":{"$numberInt":"4"}}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"1"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"t":[{"$numberInt":"1"},{"$numberInt":"2"},{"$numberInt":"3"},{"$numberInt":"4"}]}]}]},{"description":"$addToSet on a value already there writes nothing","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$addToSet":{"t":{"$numberInt":"2"}}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"0"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"t":[{"$numberInt":"1"},{"$numberInt":"2"},{"$numberInt":"3"}]}]}]},{"description":"$addToSet on a missing field creates the array","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$addToSet":{"gone":{"$numberInt":"1"}}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"1"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"t":[{"$numberInt":"1"},{"$numberInt":"2"},{"$numberInt":"3"}],"gone":[{"$numberInt":"1"}]}]}]},{"description":"$addToSet $each adds only what is missing","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$addToSet":{"t":{"$each":[{"$numberInt":"2"},{"$numberInt":"4"},{"$numberInt":"5"}]}}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"1"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"t":[{"$numberInt":"1"},{"$numberInt":"2"},{"$numberInt":"3"},{"$numberInt":"4"},{"$numberInt":"5"}]}]}]},{"description":"$addToSet $each with duplicates inside it","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$addToSet":{"t":{"$each":[{"$numberInt":"7"},{"$numberInt":"7"}]}}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"1"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"t":[{"$numberInt":"1"},{"$numberInt":"2"},{"$numberInt":"3"},{"$numberInt":"7"}]}]}]},{"description":"$addToSet compares documents whole","operations":[{"object":"collection0","name":"deleteMany","arguments":{"filter":{}}},{"object":"collection0","name":"insertMany","arguments":{"documents":[{"_id":{"$numberInt":"1"},"t":[{"a":{"$numberInt":"1"}}]}]}},{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$addToSet":{"t":{"a":{"$numberInt":"1"}}}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"0"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"t":[{"a":{"$numberInt":"1"}}]}]}]},{"description":"$addToSet distinguishes documents by field order","operations":[{"object":"collection0","name":"deleteMany","arguments":{"filter":{}}},{"object":"collection0","name":"insertMany","arguments":{"documents":[{"_id":{"$numberInt":"1"},"t":[{"a":{"$numberInt":"1"},"b":{"$numberInt":"2"}}]}]}},{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$addToSet":{"t":{"b":{"$numberInt":"2"},"a":{"$numberInt":"1"}}}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"1"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"t":[{"a":{"$numberInt":"1"},"b":{"$numberInt":"2"}},{"b":{"$numberInt":"2"},"a":{"$numberInt":"1"}}]}]}]},{"description":"$addToSet treats an int and an equal double as one value","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$addToSet":{"t":{"$numberInt":"2"}}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"0"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"t":[{"$numberInt":"1"},{"$numberInt":"2"},{"$numberInt":"3"}]}]}]},{"description":"$addToSet on a non-array field","operations":[{"object":"collection0","name":"deleteMany","arguments":{"filter":{}}},{"object":"collection0","name":"insertMany","arguments":{"documents":[{"_id":{"$numberInt":"1"},"t":{"$numberInt":"5"}}]}},{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$addToSet":{"t":{"$numberInt":"1"}}}},"expectError":{"isError":true,"errorCode":{"$numberInt":"2"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"t":{"$numberInt":"5"}}]}]},{"description":"$pop removes the last element","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$pop":{"t":{"$numberInt":"1"}}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"1"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"t":[{"$numberInt":"1"},{"$numberInt":"2"}]}]}]},{"description":"$pop with -1 removes the first","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$pop":{"t":{"$numberInt":"-1"}}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"1"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"t":[{"$numberInt":"2"},{"$numberInt":"3"}]}]}]},{"description":"$pop of an empty array writes nothing","operations":[{"object":"collection0","name":"deleteMany","arguments":{"filter":{}}},{"object":"collection0","name":"insertMany","arguments":{"documents":[{"_id":{"$numberInt":"1"},"t":[]}]}},{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$pop":{"t":{"$numberInt":"1"}}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"0"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"t":[]}]}]},{"description":"$pop of a missing field writes nothing","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$pop":{"gone":{"$numberInt":"1"}}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"0"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"t":[{"$numberInt":"1"},{"$numberInt":"2"},{"$numberInt":"3"}]}]}]},{"description":"$pop of a one-element array leaves an empty one","operations":[{"object":"collection0","name":"deleteMany","arguments":{"filter":{}}},{"object":"collection0","name":"insertMany","arguments":{"documents":[{"_id":{"$numberInt":"1"},"t":[{"$numberInt":"9"}]}]}},{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$pop":{"t":{"$numberInt":"1"}}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"1"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"t":[]}]}]},{"description":"$pop with an argument that is neither 1 nor -1","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$pop":{"t":{"$numberInt":"2"}}}},"expectError":{"isError":true,"errorCode":{"$numberInt":"9"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"t":[{"$numberInt":"1"},{"$numberInt":"2"},{"$numberInt":"3"}]}]}]},{"description":"$pop with 1.0, which is 1","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$pop":{"t":{"$numberInt":"1"}}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"1"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"t":[{"$numberInt":"1"},{"$numberInt":"2"}]}]}]},{"description":"$pop of a non-array field","operations":[{"object":"collection0","name":"deleteMany","arguments":{"filter":{}}},{"object":"collection0","name":"insertMany","arguments":{"documents":[{"_id":{"$numberInt":"1"},"t":{"$numberInt":"5"}}]}},{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$pop":{"t":{"$numberInt":"1"}}}},"expectError":{"isError":true,"errorCode":{"$numberInt":"14"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"t":{"$numberInt":"5"}}]}]},{"description":"$pullAll removes every copy of every listed value","operations":[{"object":"collection0","name":"deleteMany","arguments":{"filter":{}}},{"object":"collection0","name":"insertMany","arguments":{"documents":[{"_id":{"$numberInt":"1"},"t":[{"$numberInt":"1"},{"$numberInt":"2"},{"$numberInt":"3"},{"$numberInt":"2"}]}]}},{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$pullAll":{"t":[{"$numberInt":"2"}]}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"1"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"t":[{"$numberInt":"1"},{"$numberInt":"3"}]}]}]},{"description":"$pullAll with several values","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$pullAll":{"t":[{"$numberInt":"1"},{"$numberInt":"3"}]}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"1"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"t":[{"$numberInt":"2"}]}]}]},{"description":"$pullAll matching nothing writes nothing","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$pullAll":{"t":[{"$numberInt":"9"}]}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"0"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"t":[{"$numberInt":"1"},{"$numberInt":"2"},{"$numberInt":"3"}]}]}]},{"description":"$pullAll compares documents whole, not by predicate","operations":[{"object":"collection0","name":"deleteMany","arguments":{"filter":{}}},{"object":"collection0","name":"insertMany","arguments":{"documents":[{"_id":{"$numberInt":"1"},"t":[{"a":{"$numberInt":"1"}},{"a":{"$numberInt":"2"}}]}]}},{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$pullAll":{"t":[{"a":{"$numberInt":"1"}}]}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"1"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"t":[{"a":{"$numberInt":"2"}}]}]}]},{"description":"$pullAll on a missing field writes nothing","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$pullAll":{"gone":[{"$numberInt":"1"}]}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"0"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"t":[{"$numberInt":"1"},{"$numberInt":"2"},{"$numberInt":"3"}]}]}]},{"description":"$pullAll with an argument that is not an array","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$pullAll":{"t":{"$numberInt":"1"}}}},"expectError":{"isError":true,"errorCode":{"$numberInt":"2"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"t":[{"$numberInt":"1"},{"$numberInt":"2"},{"$numberInt":"3"}]}]}]},{"description":"$pullAll emptying the array","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$pullAll":{"t":[{"$numberInt":"1"},{"$numberInt":"2"},{"$numberInt":"3"}]}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"1"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"t":[]}]}]},{"description":"$addToSet through a positional segment","operations":[{"object":"collection0","name":"deleteMany","arguments":{"filter":{}}},{"object":"collection0","name":"insertMany","arguments":{"documents":[{"_id":{"$numberInt":"1"},"y":[{"t":[{"$numberInt":"1"}]},{"t":[{"$numberInt":"2"}]}]}]}},{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$addToSet":{"y.$[].t":{"$numberInt":"1"}}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"1"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"y":[{"t":[{"$numberInt":"1"}]},{"t":[{"$numberInt":"2"},{"$numberInt":"1"}]}]}]}]},{"description":"$pop down a dotted path","operations":[{"object":"collection0","name":"deleteMany","arguments":{"filter":{}}},{"object":"collection0","name":"insertMany","arguments":{"documents":[{"_id":{"$numberInt":"1"},"n":{"t":[{"$numberInt":"1"},{"$numberInt":"2"}]}}]}},{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$pop":{"n.t":{"$numberInt":"-1"}}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"1"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"n":{"t":[{"$numberInt":"2"}]}}]}]}]} diff --git a/tests/spec/operators/current-date.json b/tests/spec/operators/current-date.json new file mode 100644 index 0000000..b987ba7 --- /dev/null +++ b/tests/spec/operators/current-date.json @@ -0,0 +1 @@ +{"description":"current-date","schemaVersion":"1.4","createEntities":[{"client":{"id":"client0"}},{"database":{"id":"database0","client":"client0","databaseName":"operator-corpus"}},{"collection":{"id":"collection0","database":"database0","collectionName":"coll"}}],"initialData":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"}}]}],"tests":[{"description":"$currentDate with true writes a date","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$currentDate":{"d":true}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"1"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"d":{"$$type":"date"}}]}]},{"description":"$currentDate with an explicit date type","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$currentDate":{"d":{"$type":"date"}}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"1"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"d":{"$$type":"date"}}]}]},{"description":"$currentDate with the timestamp type","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$currentDate":{"d":{"$type":"timestamp"}}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"1"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"d":{"$$type":"timestamp"}}]}]},{"description":"$currentDate overwrites a field that is already there","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$currentDate":{"a":true}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"1"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$$type":"date"}}]}]},{"description":"$currentDate down a dotted path","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$currentDate":{"n.d":true}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"1"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"n":{"d":{"$$type":"date"}}}]}]},{"description":"$currentDate beside another operator","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$currentDate":{"d":true},"$inc":{"a":{"$numberInt":"1"}}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"1"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"2"},"d":{"$$type":"date"}}]}]},{"description":"$currentDate with false, which is still a date","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$currentDate":{"d":false}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"1"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"d":{"$$type":"date"}}]}]},{"description":"$currentDate with an unknown type name","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$currentDate":{"d":{"$type":"nope"}}}},"expectError":{"isError":true,"errorCode":{"$numberInt":"2"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"}}]}]},{"description":"$currentDate with a document that is not $type","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$currentDate":{"d":{"a":{"$numberInt":"1"}}}}},"expectError":{"isError":true,"errorCode":{"$numberInt":"2"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"}}]}]},{"description":"$currentDate with a number","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$currentDate":{"d":{"$numberInt":"1"}}}},"expectError":{"isError":true,"errorCode":{"$numberInt":"2"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"}}]}]},{"description":"$currentDate on an upsert that inserts","operations":[{"object":"collection0","name":"deleteMany","arguments":{"filter":{}}},{"object":"collection0","name":"updateOne","arguments":{"filter":{"k":{"$numberInt":"1"}},"update":{"$currentDate":{"d":true}},"upsert":true},"expectResult":{"matchedCount":{"$numberInt":"0"},"modifiedCount":{"$numberInt":"0"},"upsertedCount":{"$numberInt":"1"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$$type":"objectId"},"k":{"$numberInt":"1"},"d":{"$$type":"date"}}]}]}]} diff --git a/tests/spec/operators/modifiers.json b/tests/spec/operators/modifiers.json new file mode 100644 index 0000000..17fae6b --- /dev/null +++ b/tests/spec/operators/modifiers.json @@ -0,0 +1 @@ +{"description":"modifiers","schemaVersion":"1.4","createEntities":[{"client":{"id":"client0"}},{"database":{"id":"database0","client":"client0","databaseName":"operator-corpus"}},{"collection":{"id":"collection0","database":"database0","collectionName":"coll"}}],"initialData":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"t":[{"$numberInt":"1"},{"$numberInt":"2"}]}]}],"tests":[{"description":"an unknown modifier","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$bogus":{"a":{"$numberInt":"1"}}}},"expectError":{"isError":true,"errorCode":{"$numberInt":"9"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"t":[{"$numberInt":"1"},{"$numberInt":"2"}]}]}]},{"description":"an unknown modifier beside a known one","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$set":{"b":{"$numberInt":"1"}},"$bogus":{"a":{"$numberInt":"1"}}}},"expectError":{"isError":true,"errorCode":{"$numberInt":"9"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"t":[{"$numberInt":"1"},{"$numberInt":"2"}]}]}]},{"description":"a known modifier whose argument is not a document","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$set":{"$numberInt":"1"}}},"expectError":{"isError":true,"errorCode":{"$numberInt":"9"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"t":[{"$numberInt":"1"},{"$numberInt":"2"}]}]}]},{"description":"an empty argument to a known modifier","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$set":{}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"0"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"t":[{"$numberInt":"1"},{"$numberInt":"2"}]}]}]},{"description":"a data field after an operator","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$set":{"c":{"$numberInt":"1"}},"b":{"$numberInt":"1"}}},"expectError":{"isError":true,"errorCode":{"$numberInt":"9"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"t":[{"$numberInt":"1"},{"$numberInt":"2"}]}]}]},{"description":"two operators writing the same field","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$set":{"a":{"$numberInt":"2"}},"$inc":{"a":{"$numberInt":"1"}}}},"expectError":{"isError":true,"errorCode":{"$numberInt":"40"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"t":[{"$numberInt":"1"},{"$numberInt":"2"}]}]}]},{"description":"$inc on a non-numeric field","operations":[{"object":"collection0","name":"deleteMany","arguments":{"filter":{}}},{"object":"collection0","name":"insertMany","arguments":{"documents":[{"_id":{"$numberInt":"1"},"a":"x"}]}},{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$inc":{"a":{"$numberInt":"1"}}}},"expectError":{"isError":true,"errorCode":{"$numberInt":"14"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"a":"x"}]}]},{"description":"$inc by a non-numeric operand","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$inc":{"a":"x"}}},"expectError":{"isError":true,"errorCode":{"$numberInt":"14"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"t":[{"$numberInt":"1"},{"$numberInt":"2"}]}]}]},{"description":"$push onto a non-array field","operations":[{"object":"collection0","name":"deleteMany","arguments":{"filter":{}}},{"object":"collection0","name":"insertMany","arguments":{"documents":[{"_id":{"$numberInt":"1"},"t":{"$numberInt":"5"}}]}},{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$push":{"t":{"$numberInt":"1"}}}},"expectError":{"isError":true,"errorCode":{"$numberInt":"2"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"t":{"$numberInt":"5"}}]}]},{"description":"$pull on a non-array field","operations":[{"object":"collection0","name":"deleteMany","arguments":{"filter":{}}},{"object":"collection0","name":"insertMany","arguments":{"documents":[{"_id":{"$numberInt":"1"},"t":{"$numberInt":"5"}}]}},{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$pull":{"t":{"$numberInt":"1"}}}},"expectError":{"isError":true,"errorCode":{"$numberInt":"2"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"t":{"$numberInt":"5"}}]}]},{"description":"$unset of a field that is not there","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$unset":{"gone":""}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"0"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"t":[{"$numberInt":"1"},{"$numberInt":"2"}]}]}]},{"description":"$rename onto a field that already exists","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$rename":{"a":"t"}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"1"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"t":{"$numberInt":"1"}}]}]},{"description":"$rename of a field that is not there","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$rename":{"gone":"b"}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"0"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"t":[{"$numberInt":"1"},{"$numberInt":"2"}]}]}]},{"description":"an update that writes nothing still reports a match","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$set":{"a":{"$numberInt":"1"}}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"0"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"t":[{"$numberInt":"1"},{"$numberInt":"2"}]}]}]}]} diff --git a/tests/spec/operators/numeric.json b/tests/spec/operators/numeric.json new file mode 100644 index 0000000..81796be --- /dev/null +++ b/tests/spec/operators/numeric.json @@ -0,0 +1 @@ +{"description":"numeric","schemaVersion":"1.4","createEntities":[{"client":{"id":"client0"}},{"database":{"id":"database0","client":"client0","databaseName":"operator-corpus"}},{"collection":{"id":"collection0","database":"database0","collectionName":"coll"}}],"initialData":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"5"},"s":"b","d":{"$numberDouble":"2.5"}}]}],"tests":[{"description":"$mul multiplies an int by an int","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$mul":{"a":{"$numberInt":"2"}}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"1"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"10"},"s":"b","d":{"$numberDouble":"2.5"}}]}]},{"description":"$mul of a missing field","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$mul":{"gone":{"$numberInt":"5"}}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"1"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"5"},"s":"b","d":{"$numberDouble":"2.5"},"gone":{"$numberInt":"0"}}]}]},{"description":"$mul mixes int and double","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$mul":{"d":{"$numberInt":"2"}}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"1"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"5"},"s":"b","d":{"$numberInt":"5"}}]}]},{"description":"$mul by zero","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$mul":{"a":{"$numberInt":"0"}}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"1"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"0"},"s":"b","d":{"$numberDouble":"2.5"}}]}]},{"description":"$mul overflowing an int32","operations":[{"object":"collection0","name":"deleteMany","arguments":{"filter":{}}},{"object":"collection0","name":"insertMany","arguments":{"documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"2000000000"}}]}},{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$mul":{"a":{"$numberInt":"2"}}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"1"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberLong":"4000000000"}}]}]},{"description":"$mul of a non-numeric field","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$mul":{"s":{"$numberInt":"2"}}}},"expectError":{"isError":true,"errorCode":{"$numberInt":"14"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"5"},"s":"b","d":{"$numberDouble":"2.5"}}]}]},{"description":"$mul by a non-numeric operand","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$mul":{"a":"x"}}},"expectError":{"isError":true,"errorCode":{"$numberInt":"14"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"5"},"s":"b","d":{"$numberDouble":"2.5"}}]}]},{"description":"$mul down a dotted path","operations":[{"object":"collection0","name":"deleteMany","arguments":{"filter":{}}},{"object":"collection0","name":"insertMany","arguments":{"documents":[{"_id":{"$numberInt":"1"},"n":{"a":{"$numberInt":"3"}}}]}},{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$mul":{"n.a":{"$numberInt":"4"}}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"1"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"n":{"a":{"$numberInt":"12"}}}]}]},{"description":"$min writes when the operand is lower","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$min":{"a":{"$numberInt":"3"}}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"1"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"3"},"s":"b","d":{"$numberDouble":"2.5"}}]}]},{"description":"$min leaves the field when the operand is higher","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$min":{"a":{"$numberInt":"7"}}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"0"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"5"},"s":"b","d":{"$numberDouble":"2.5"}}]}]},{"description":"$min of a missing field always writes","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$min":{"gone":{"$numberInt":"7"}}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"1"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"5"},"s":"b","d":{"$numberDouble":"2.5"},"gone":{"$numberInt":"7"}}]}]},{"description":"$min against a field of another type","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$min":{"s":{"$numberInt":"5"}}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"1"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"5"},"s":{"$numberInt":"5"},"d":{"$numberDouble":"2.5"}}]}]},{"description":"$min compares an int and an equal double","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$min":{"a":{"$numberInt":"5"}}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"0"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"5"},"s":"b","d":{"$numberDouble":"2.5"}}]}]},{"description":"$max writes when the operand is higher","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$max":{"a":{"$numberInt":"7"}}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"1"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"7"},"s":"b","d":{"$numberDouble":"2.5"}}]}]},{"description":"$max leaves the field when the operand is lower","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$max":{"a":{"$numberInt":"3"}}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"0"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"5"},"s":"b","d":{"$numberDouble":"2.5"}}]}]},{"description":"$max of a missing field always writes","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$max":{"gone":{"$numberInt":"7"}}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"1"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"5"},"s":"b","d":{"$numberDouble":"2.5"},"gone":{"$numberInt":"7"}}]}]},{"description":"$max against a field of another type","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$max":{"s":{"$numberInt":"5"}}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"0"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"5"},"s":"b","d":{"$numberDouble":"2.5"}}]}]},{"description":"$max against a null field","operations":[{"object":"collection0","name":"deleteMany","arguments":{"filter":{}}},{"object":"collection0","name":"insertMany","arguments":{"documents":[{"_id":{"$numberInt":"1"},"a":null}]}},{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$max":{"a":{"$numberInt":"1"}}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"1"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"}}]}]},{"description":"$min and $max on the same field in one update","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$min":{"a":{"$numberInt":"3"}},"$max":{"a":{"$numberInt":"9"}}}},"expectError":{"isError":true,"errorCode":{"$numberInt":"40"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"5"},"s":"b","d":{"$numberDouble":"2.5"}}]}]},{"description":"$min through an array index","operations":[{"object":"collection0","name":"deleteMany","arguments":{"filter":{}}},{"object":"collection0","name":"insertMany","arguments":{"documents":[{"_id":{"$numberInt":"1"},"t":[{"$numberInt":"5"},{"$numberInt":"5"}]}]}},{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$min":{"t.0":{"$numberInt":"1"}}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"1"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"t":[{"$numberInt":"1"},{"$numberInt":"5"}]}]}]},{"description":"$mul through a positional segment","operations":[{"object":"collection0","name":"deleteMany","arguments":{"filter":{}}},{"object":"collection0","name":"insertMany","arguments":{"documents":[{"_id":{"$numberInt":"1"},"t":[{"a":{"$numberInt":"2"}},{"a":{"$numberInt":"3"}}]}]}},{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$mul":{"t.$[].a":{"$numberInt":"10"}}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"1"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"t":[{"a":{"$numberInt":"20"}},{"a":{"$numberInt":"30"}}]}]}]}]} diff --git a/tests/spec/operators/push-modifiers.json b/tests/spec/operators/push-modifiers.json new file mode 100644 index 0000000..d61e933 --- /dev/null +++ b/tests/spec/operators/push-modifiers.json @@ -0,0 +1 @@ +{"description":"push-modifiers","schemaVersion":"1.4","createEntities":[{"client":{"id":"client0"}},{"database":{"id":"database0","client":"client0","databaseName":"operator-corpus"}},{"collection":{"id":"collection0","database":"database0","collectionName":"coll"}}],"initialData":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"t":[{"$numberInt":"1"},{"$numberInt":"2"}]}]}],"tests":[{"description":"$slice keeps the last n","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$push":{"t":{"$each":[{"$numberInt":"3"},{"$numberInt":"4"}],"$slice":{"$numberInt":"-3"}}}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"1"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"t":[{"$numberInt":"2"},{"$numberInt":"3"},{"$numberInt":"4"}]}]}]},{"description":"$slice keeps the first n","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$push":{"t":{"$each":[{"$numberInt":"3"},{"$numberInt":"4"}],"$slice":{"$numberInt":"3"}}}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"1"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"t":[{"$numberInt":"1"},{"$numberInt":"2"},{"$numberInt":"3"}]}]}]},{"description":"$slice of zero empties the array","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$push":{"t":{"$each":[{"$numberInt":"3"}],"$slice":{"$numberInt":"0"}}}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"1"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"t":[]}]}]},{"description":"$slice larger than the array keeps all of it","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$push":{"t":{"$each":[{"$numberInt":"3"}],"$slice":{"$numberInt":"10"}}}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"1"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"t":[{"$numberInt":"1"},{"$numberInt":"2"},{"$numberInt":"3"}]}]}]},{"description":"$slice with an empty $each truncates without adding","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$push":{"t":{"$each":[],"$slice":{"$numberInt":"1"}}}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"1"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"t":[{"$numberInt":"1"}]}]}]},{"description":"$position inserts at the front","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$push":{"t":{"$each":[{"$numberInt":"9"}],"$position":{"$numberInt":"0"}}}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"1"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"t":[{"$numberInt":"9"},{"$numberInt":"1"},{"$numberInt":"2"}]}]}]},{"description":"$position inserts in the middle","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$push":{"t":{"$each":[{"$numberInt":"9"}],"$position":{"$numberInt":"1"}}}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"1"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"t":[{"$numberInt":"1"},{"$numberInt":"9"},{"$numberInt":"2"}]}]}]},{"description":"$position past the end appends","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$push":{"t":{"$each":[{"$numberInt":"9"}],"$position":{"$numberInt":"99"}}}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"1"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"t":[{"$numberInt":"1"},{"$numberInt":"2"},{"$numberInt":"9"}]}]}]},{"description":"$position counted from the end","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$push":{"t":{"$each":[{"$numberInt":"9"}],"$position":{"$numberInt":"-1"}}}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"1"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"t":[{"$numberInt":"1"},{"$numberInt":"9"},{"$numberInt":"2"}]}]}]},{"description":"$sort ascending over scalars","operations":[{"object":"collection0","name":"deleteMany","arguments":{"filter":{}}},{"object":"collection0","name":"insertMany","arguments":{"documents":[{"_id":{"$numberInt":"1"},"t":[{"$numberInt":"3"},{"$numberInt":"1"}]}]}},{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$push":{"t":{"$each":[{"$numberInt":"2"}],"$sort":{"$numberInt":"1"}}}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"1"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"t":[{"$numberInt":"1"},{"$numberInt":"2"},{"$numberInt":"3"}]}]}]},{"description":"$sort descending over scalars","operations":[{"object":"collection0","name":"deleteMany","arguments":{"filter":{}}},{"object":"collection0","name":"insertMany","arguments":{"documents":[{"_id":{"$numberInt":"1"},"t":[{"$numberInt":"3"},{"$numberInt":"1"}]}]}},{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$push":{"t":{"$each":[{"$numberInt":"2"}],"$sort":{"$numberInt":"-1"}}}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"1"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"t":[{"$numberInt":"3"},{"$numberInt":"2"},{"$numberInt":"1"}]}]}]},{"description":"$sort on a field of the elements","operations":[{"object":"collection0","name":"deleteMany","arguments":{"filter":{}}},{"object":"collection0","name":"insertMany","arguments":{"documents":[{"_id":{"$numberInt":"1"},"t":[{"a":{"$numberInt":"3"}},{"a":{"$numberInt":"1"}}]}]}},{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$push":{"t":{"$each":[{"a":{"$numberInt":"2"}}],"$sort":{"a":{"$numberInt":"1"}}}}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"1"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"t":[{"a":{"$numberInt":"1"}},{"a":{"$numberInt":"2"}},{"a":{"$numberInt":"3"}}]}]}]},{"description":"$sort and $slice together","operations":[{"object":"collection0","name":"deleteMany","arguments":{"filter":{}}},{"object":"collection0","name":"insertMany","arguments":{"documents":[{"_id":{"$numberInt":"1"},"t":[{"$numberInt":"3"},{"$numberInt":"1"}]}]}},{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$push":{"t":{"$each":[{"$numberInt":"2"}],"$sort":{"$numberInt":"1"},"$slice":{"$numberInt":"2"}}}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"1"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"t":[{"$numberInt":"1"},{"$numberInt":"2"}]}]}]},{"description":"$position and $slice together","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$push":{"t":{"$each":[{"$numberInt":"9"}],"$position":{"$numberInt":"0"},"$slice":{"$numberInt":"2"}}}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"1"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"t":[{"$numberInt":"9"},{"$numberInt":"1"}]}]}]},{"description":"$slice without $each","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$push":{"t":{"$slice":{"$numberInt":"1"}}}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"1"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"t":[{"$numberInt":"1"},{"$numberInt":"2"},{"$slice":{"$numberInt":"1"}}]}]}]},{"description":"$each that is not an array","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$push":{"t":{"$each":{"$numberInt":"3"}}}}},"expectError":{"isError":true,"errorCode":{"$numberInt":"2"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"t":[{"$numberInt":"1"},{"$numberInt":"2"}]}]}]},{"description":"$slice that is not a number","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$push":{"t":{"$each":[{"$numberInt":"3"}],"$slice":"x"}}}},"expectError":{"isError":true,"errorCode":{"$numberInt":"2"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"t":[{"$numberInt":"1"},{"$numberInt":"2"}]}]}]},{"description":"$position that is not a number","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$push":{"t":{"$each":[{"$numberInt":"3"}],"$position":"x"}}}},"expectError":{"isError":true,"errorCode":{"$numberInt":"2"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"t":[{"$numberInt":"1"},{"$numberInt":"2"}]}]}]},{"description":"an unknown modifier beside $each","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$push":{"t":{"$each":[{"$numberInt":"3"}],"$bogus":{"$numberInt":"1"}}}}},"expectError":{"isError":true,"errorCode":{"$numberInt":"2"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"t":[{"$numberInt":"1"},{"$numberInt":"2"}]}]}]},{"description":"a document operand that is not modifiers at all is a value","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$push":{"t":{"a":{"$numberInt":"1"}}}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"1"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"t":[{"$numberInt":"1"},{"$numberInt":"2"},{"a":{"$numberInt":"1"}}]}]}]},{"description":"$push $each onto a missing field","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{},"update":{"$push":{"gone":{"$each":[{"$numberInt":"1"},{"$numberInt":"2"}]}}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"1"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"t":[{"$numberInt":"1"},{"$numberInt":"2"}],"gone":[{"$numberInt":"1"},{"$numberInt":"2"}]}]}]}]} diff --git a/tests/spec/operators/record.js b/tests/spec/operators/record.js new file mode 100644 index 0000000..d151a5f --- /dev/null +++ b/tests/spec/operators/record.js @@ -0,0 +1,222 @@ +// Record an update-operator corpus by asking a real mongod what the answer is. +// +// Eight operators PLAN §3 lists for M3 -- `$setOnInsert`, `$addToSet`, `$mul`, +// `$min`, `$max`, `$pop`, `$pullAll`, `$currentDate` -- plus `$push`'s +// `$slice`/`$position`/`$sort` modifiers. The pinned crud corpus contains +// almost nothing about any of them, and a probe against both servers found +// every one of the eight answering `bad update` here, and the three `$push` +// modifiers *silently ignored*: `{$each: [3, 4], $slice: -3}` appended without +// slicing and answered ok: 1. Same discipline as `tests/spec/positional/` and +// `tests/spec/aggregate/`: inputs are authored in `sources/`, expectations are +// measured here. +// +// node tests/spec/operators/record.js --mongod-port 27099 +// +// Options: +// --mongod-port a running mongod to measure against (default 27099) +// --only record just one source file +const fs = require('fs'); +const path = require('path'); +// The same pinned driver `run.js` uses, resolved the same way. +const DRIVER = path.join(__dirname, '..', '..', 'e2e', 'node_modules', 'mongodb'); +const { MongoClient } = require(DRIVER); +const { EJSON, ObjectId, Timestamp } = require(path.join(DRIVER, 'lib', 'bson.js')); + +const argv = process.argv.slice(2); +function opt(name, dflt) { + const i = argv.indexOf('--' + name); + if (i < 0) return dflt; + const v = argv[i + 1]; + return v === undefined || v.startsWith('--') ? true : v; +} + +const PORT = parseInt(opt('mongod-port', '27099'), 10); +const ONLY = opt('only', null); +const SRC_DIR = path.join(__dirname, 'sources'); +const DB_NAME = 'operator-corpus'; +const COLL = 'coll'; + +// Every construct here predates 4.0, so nothing depends on a server newer than +// the 4.4 this one reports. +const SCHEMA_VERSION = '1.4'; + +const UPDATE_KEYS = ['matchedCount', 'modifiedCount', 'upsertedCount']; + +/// Values nobody can predict, replaced by an assertion about their type. +/// +/// Two kinds, and both are the corpus staying honest rather than the corpus +/// looking away: a `$currentDate` field is whatever the clock said, and an +/// upsert that inserts gets a generated ObjectId. Everything else about the +/// document -- which fields exist, in what order, holding what -- is still +/// compared exactly, so a case that lost a field still fails. +/// +/// The ObjectId rule is automatic because it is unambiguous: no source file +/// authors one. A date is not: `volatile` names those per case, so a corpus +/// that one day wants to pin a real stored date still can. +function maskGenerated(value) { + if (value instanceof ObjectId) return { $$type: 'objectId' }; + if (Array.isArray(value)) return value.map(maskGenerated); + if (value && typeof value === 'object' && value.constructor === Object) { + const out = {}; + for (const [k, v] of Object.entries(value)) out[k] = maskGenerated(v); + return out; + } + return value; +} + +function maskVolatile(docs, paths) { + for (const p of paths) { + for (const doc of docs) { + const segs = p.split('.'); + let cur = doc; + for (const s of segs.slice(0, -1)) cur = cur === undefined ? undefined : cur[s]; + const last = segs[segs.length - 1]; + if (cur === undefined || !(last in cur)) { + throw new Error(`volatile path '${p}' is not in the recorded document ` + + `${JSON.stringify(doc)} -- the case did not write what it said it would`); + } + const v = cur[last]; + if (v instanceof Date) cur[last] = { $$type: 'date' }; + else if (v instanceof Timestamp) cur[last] = { $$type: 'timestamp' }; + else throw new Error(`volatile path '${p}' holds ${v}, which is neither a date nor a timestamp`); + } + } +} + +async function main() { + if (!fs.existsSync(SRC_DIR)) { + console.error(`missing ${SRC_DIR}`); + process.exit(2); + } + const client = new MongoClient(`mongodb://127.0.0.1:${PORT}`, { serverSelectionTimeoutMS: 3000 }); + try { + await client.connect(); + } catch (e) { + console.error(`no mongod on :${PORT} -- start one first:\n` + + ` mongod --port ${PORT} --dbpath \n${e.message}`); + process.exit(2); + } + const build = await client.db('admin').command({ buildInfo: 1 }); + console.log(`recording against mongod ${build.version} on :${PORT}`); + + const sources = fs.readdirSync(SRC_DIR).filter((f) => f.endsWith('.json')) + .filter((f) => !ONLY || f === ONLY || f === ONLY + '.json') + .sort(); + if (!sources.length) { + console.error('no source files'); + process.exit(2); + } + + for (const file of sources) { + const src = JSON.parse(fs.readFileSync(path.join(SRC_DIR, file), 'utf8')); + const name = path.basename(file, '.json'); + const out = await record(client, name, src); + const text = EJSON.stringify(out, { relaxed: false, indent: 2 }); + fs.writeFileSync(path.join(__dirname, `${name}.json`), text + '\n'); + const errs = out.tests.filter((t) => t.operations[t.operations.length - 1].expectError).length; + console.log(` ${name}: ${out.tests.length} cases, ${errs} of them errors`); + } + await client.close(); + console.log('RECORDED'); +} + +async function record(client, name, src) { + const coll = client.db(DB_NAME).collection(COLL); + const tests = []; + + for (const c of src.cases) { + const documents = c.documents === undefined ? src.documents : c.documents; + await coll.drop().catch(() => {}); + if (documents.length) await coll.insertMany(structuredClone(documents)); + + // A case that brings its own documents reseeds through *operations*: + // the unified format's `initialData` is per file, and the runner seeds + // it once per test. + const setup = []; + if (c.documents !== undefined) { + setup.push({ object: 'collection0', name: 'deleteMany', arguments: { filter: {} } }); + if (documents.length) { + setup.push({ + object: 'collection0', + name: 'insertMany', + arguments: { documents: structuredClone(documents) }, + }); + } + } + const op = { + object: 'collection0', + name: c.operation, + arguments: structuredClone(c.arguments), + }; + try { + op.expectResult = maskGenerated(await invoke(coll, c.operation, structuredClone(c.arguments))); + } catch (e) { + // The code, not the message: message text is mongod's to change + // between releases, and several of these embed a rendering of the + // offending BSON that no formatter here produces. + delete op.expectResult; + op.expectError = { isError: true, errorCode: e.code }; + } + + // Recorded for every case including the refusals: a refusal that left + // the document half-written would look identical to a clean one in + // `expectError` alone. + const after = await coll.find({}, { sort: { _id: 1 } }).toArray(); + if (c.volatile) maskVolatile(after, c.volatile); + + tests.push({ + description: c.description, + operations: [...setup, op], + outcome: [{ + collectionName: COLL, + databaseName: DB_NAME, + documents: after.map(maskGenerated), + }], + }); + } + await coll.drop().catch(() => {}); + + return { + description: name, + schemaVersion: SCHEMA_VERSION, + // Recorded, not authored. Regenerate with tests/spec/operators/record.js. + createEntities: [ + { client: { id: 'client0' } }, + { database: { id: 'database0', client: 'client0', databaseName: DB_NAME } }, + { collection: { id: 'collection0', database: 'database0', collectionName: COLL } }, + ], + initialData: [{ collectionName: COLL, databaseName: DB_NAME, documents: src.documents }], + tests, + }; +} + +async function invoke(coll, name, args) { + const { filter, update, replacement, ...rest } = args; + switch (name) { + case 'updateOne': + return pick(await coll.updateOne(filter, update, rest), UPDATE_KEYS); + case 'updateMany': + return pick(await coll.updateMany(filter, update, rest), UPDATE_KEYS); + case 'replaceOne': + return pick(await coll.replaceOne(filter, replacement, rest), UPDATE_KEYS); + case 'findOneAndUpdate': { + const r = await coll.findOneAndUpdate(filter, update, rest); + // Driver 5+ returns the document itself; 4.x wrapped it in + // `{value}`. run.js tolerates both the same way. + return r && typeof r === 'object' && 'value' in r && 'ok' in r ? r.value : r; + } + default: + throw new Error(`record.js does not know the operation '${name}'`); + } +} + +function pick(result, keys) { + const o = {}; + for (const k of keys) if (result[k] !== undefined) o[k] = result[k]; + return o; +} + +main().catch((e) => { + console.error('RECORD_FAIL', e); + process.exit(1); +}); diff --git a/tests/spec/operators/set-on-insert.json b/tests/spec/operators/set-on-insert.json new file mode 100644 index 0000000..0a65ee1 --- /dev/null +++ b/tests/spec/operators/set-on-insert.json @@ -0,0 +1 @@ +{"description":"set-on-insert","schemaVersion":"1.4","createEntities":[{"client":{"id":"client0"}},{"database":{"id":"database0","client":"client0","databaseName":"operator-corpus"}},{"collection":{"id":"collection0","database":"database0","collectionName":"coll"}}],"initialData":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"k":{"$numberInt":"1"},"a":{"$numberInt":"100"}}]}],"tests":[{"description":"$setOnInsert writes when the upsert inserts","operations":[{"object":"collection0","name":"deleteMany","arguments":{"filter":{}}},{"object":"collection0","name":"updateOne","arguments":{"filter":{"k":{"$numberInt":"1"}},"update":{"$setOnInsert":{"a":{"$numberInt":"1"}},"$set":{"b":{"$numberInt":"2"}}},"upsert":true},"expectResult":{"matchedCount":{"$numberInt":"0"},"modifiedCount":{"$numberInt":"0"},"upsertedCount":{"$numberInt":"1"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$$type":"objectId"},"k":{"$numberInt":"1"},"a":{"$numberInt":"1"},"b":{"$numberInt":"2"}}]}]},{"description":"$setOnInsert is ignored when the upsert updates","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{"k":{"$numberInt":"1"}},"update":{"$setOnInsert":{"a":{"$numberInt":"1"}},"$set":{"b":{"$numberInt":"2"}}},"upsert":true},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"1"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"k":{"$numberInt":"1"},"a":{"$numberInt":"100"},"b":{"$numberInt":"2"}}]}]},{"description":"$setOnInsert alone on a plain update is a no-op","operations":[{"object":"collection0","name":"updateOne","arguments":{"filter":{"k":{"$numberInt":"1"}},"update":{"$setOnInsert":{"a":{"$numberInt":"1"}}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"0"},"upsertedCount":{"$numberInt":"0"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"1"},"k":{"$numberInt":"1"},"a":{"$numberInt":"100"}}]}]},{"description":"$setOnInsert alone on an insert is the whole document","operations":[{"object":"collection0","name":"deleteMany","arguments":{"filter":{}}},{"object":"collection0","name":"updateOne","arguments":{"filter":{"k":{"$numberInt":"7"}},"update":{"$setOnInsert":{"a":{"$numberInt":"1"}}},"upsert":true},"expectResult":{"matchedCount":{"$numberInt":"0"},"modifiedCount":{"$numberInt":"0"},"upsertedCount":{"$numberInt":"1"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$$type":"objectId"},"k":{"$numberInt":"7"},"a":{"$numberInt":"1"}}]}]},{"description":"$setOnInsert writes a dotted path on insert","operations":[{"object":"collection0","name":"deleteMany","arguments":{"filter":{}}},{"object":"collection0","name":"updateOne","arguments":{"filter":{"k":{"$numberInt":"1"}},"update":{"$setOnInsert":{"x.y":{"$numberInt":"5"}}},"upsert":true},"expectResult":{"matchedCount":{"$numberInt":"0"},"modifiedCount":{"$numberInt":"0"},"upsertedCount":{"$numberInt":"1"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$$type":"objectId"},"k":{"$numberInt":"1"},"x":{"y":{"$numberInt":"5"}}}]}]},{"description":"$setOnInsert may not touch _id","operations":[{"object":"collection0","name":"deleteMany","arguments":{"filter":{}}},{"object":"collection0","name":"updateOne","arguments":{"filter":{"k":{"$numberInt":"1"}},"update":{"$setOnInsert":{"_id":{"$numberInt":"9"}}},"upsert":true},"expectResult":{"matchedCount":{"$numberInt":"0"},"modifiedCount":{"$numberInt":"0"},"upsertedCount":{"$numberInt":"1"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$numberInt":"9"},"k":{"$numberInt":"1"}}]}]},{"description":"$setOnInsert and $set naming the same field on an insert","operations":[{"object":"collection0","name":"deleteMany","arguments":{"filter":{}}},{"object":"collection0","name":"updateOne","arguments":{"filter":{"k":{"$numberInt":"1"}},"update":{"$setOnInsert":{"a":{"$numberInt":"1"}},"$set":{"a":{"$numberInt":"2"}}},"upsert":true},"expectError":{"isError":true,"errorCode":{"$numberInt":"40"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[]}]},{"description":"findOneAndUpdate reports the document $setOnInsert built","operations":[{"object":"collection0","name":"deleteMany","arguments":{"filter":{}}},{"object":"collection0","name":"findOneAndUpdate","arguments":{"filter":{"k":{"$numberInt":"3"}},"update":{"$setOnInsert":{"a":{"$numberInt":"1"}}},"upsert":true,"returnDocument":"after"},"expectResult":{"_id":{"$$type":"objectId"},"k":{"$numberInt":"3"},"a":{"$numberInt":"1"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$$type":"objectId"},"k":{"$numberInt":"3"},"a":{"$numberInt":"1"}}]}]},{"description":"updateMany with $setOnInsert on an upsert that inserts","operations":[{"object":"collection0","name":"deleteMany","arguments":{"filter":{}}},{"object":"collection0","name":"updateMany","arguments":{"filter":{"k":{"$numberInt":"1"}},"update":{"$setOnInsert":{"a":{"$numberInt":"1"}}},"upsert":true},"expectResult":{"matchedCount":{"$numberInt":"0"},"modifiedCount":{"$numberInt":"0"},"upsertedCount":{"$numberInt":"1"}}}],"outcome":[{"collectionName":"coll","databaseName":"operator-corpus","documents":[{"_id":{"$$type":"objectId"},"k":{"$numberInt":"1"},"a":{"$numberInt":"1"}}]}]}]} diff --git a/tests/spec/operators/sources/array-ops.json b/tests/spec/operators/sources/array-ops.json new file mode 100644 index 0000000..49f2a15 --- /dev/null +++ b/tests/spec/operators/sources/array-ops.json @@ -0,0 +1,153 @@ +{ + "_comment": [ + "Inputs only. Expectations are measured -- see record.js.", + "`$addToSet`, `$pop` and `$pullAll`: the three array operators that are", + "neither `$push` nor `$pull`. Each has a shape that decides whether it", + "wrote at all -- a set that already held the value, a pop of an empty", + "array, a pullAll matching nothing -- and `modifiedCount` is the only", + "place that shows, which is why every case records an outcome too." + ], + "documents": [{ "_id": 1, "t": [1, 2, 3] }], + "cases": [ + { + "description": "$addToSet appends a value the array does not hold", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$addToSet": { "t": 4 } } } + }, + { + "description": "$addToSet on a value already there writes nothing", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$addToSet": { "t": 2 } } } + }, + { + "description": "$addToSet on a missing field creates the array", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$addToSet": { "gone": 1 } } } + }, + { + "description": "$addToSet $each adds only what is missing", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$addToSet": { "t": { "$each": [2, 4, 5] } } } } + }, + { + "description": "$addToSet $each with duplicates inside it", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$addToSet": { "t": { "$each": [7, 7] } } } } + }, + { + "description": "$addToSet compares documents whole", + "documents": [{ "_id": 1, "t": [{ "a": 1 }] }], + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$addToSet": { "t": { "a": 1 } } } } + }, + { + "description": "$addToSet distinguishes documents by field order", + "documents": [{ "_id": 1, "t": [{ "a": 1, "b": 2 }] }], + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$addToSet": { "t": { "b": 2, "a": 1 } } } } + }, + { + "description": "$addToSet treats an int and an equal double as one value", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$addToSet": { "t": 2.0 } } } + }, + { + "description": "$addToSet on a non-array field", + "documents": [{ "_id": 1, "t": 5 }], + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$addToSet": { "t": 1 } } } + }, + { + "description": "$pop removes the last element", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$pop": { "t": 1 } } } + }, + { + "description": "$pop with -1 removes the first", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$pop": { "t": -1 } } } + }, + { + "description": "$pop of an empty array writes nothing", + "documents": [{ "_id": 1, "t": [] }], + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$pop": { "t": 1 } } } + }, + { + "description": "$pop of a missing field writes nothing", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$pop": { "gone": 1 } } } + }, + { + "description": "$pop of a one-element array leaves an empty one", + "documents": [{ "_id": 1, "t": [9] }], + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$pop": { "t": 1 } } } + }, + { + "description": "$pop with an argument that is neither 1 nor -1", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$pop": { "t": 2 } } } + }, + { + "description": "$pop with 1.0, which is 1", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$pop": { "t": 1.0 } } } + }, + { + "description": "$pop of a non-array field", + "documents": [{ "_id": 1, "t": 5 }], + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$pop": { "t": 1 } } } + }, + { + "description": "$pullAll removes every copy of every listed value", + "documents": [{ "_id": 1, "t": [1, 2, 3, 2] }], + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$pullAll": { "t": [2] } } } + }, + { + "description": "$pullAll with several values", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$pullAll": { "t": [1, 3] } } } + }, + { + "description": "$pullAll matching nothing writes nothing", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$pullAll": { "t": [9] } } } + }, + { + "description": "$pullAll compares documents whole, not by predicate", + "documents": [{ "_id": 1, "t": [{ "a": 1 }, { "a": 2 }] }], + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$pullAll": { "t": [{ "a": 1 }] } } } + }, + { + "description": "$pullAll on a missing field writes nothing", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$pullAll": { "gone": [1] } } } + }, + { + "description": "$pullAll with an argument that is not an array", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$pullAll": { "t": 1 } } } + }, + { + "description": "$pullAll emptying the array", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$pullAll": { "t": [1, 2, 3] } } } + }, + { + "description": "$addToSet through a positional segment", + "documents": [{ "_id": 1, "y": [{ "t": [1] }, { "t": [2] }] }], + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$addToSet": { "y.$[].t": 1 } } } + }, + { + "description": "$pop down a dotted path", + "documents": [{ "_id": 1, "n": { "t": [1, 2] } }], + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$pop": { "n.t": -1 } } } + } + ] +} diff --git a/tests/spec/operators/sources/current-date.json b/tests/spec/operators/sources/current-date.json new file mode 100644 index 0000000..d50fd5b --- /dev/null +++ b/tests/spec/operators/sources/current-date.json @@ -0,0 +1,81 @@ +{ + "_comment": [ + "Inputs only. Expectations are measured -- see record.js.", + "`$currentDate`, the one update operator whose result cannot be recorded", + "as a value: it is whatever the clock said. `volatile` names the fields", + "whose recorded value is replaced by a `$$type` assertion, so the case", + "still pins the type, the position and every other field of the document", + "-- everything except the number nobody can predict." + ], + "documents": [{ "_id": 1, "a": 1 }], + "cases": [ + { + "description": "$currentDate with true writes a date", + "volatile": ["d"], + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$currentDate": { "d": true } } } + }, + { + "description": "$currentDate with an explicit date type", + "volatile": ["d"], + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$currentDate": { "d": { "$type": "date" } } } } + }, + { + "description": "$currentDate with the timestamp type", + "volatile": ["d"], + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$currentDate": { "d": { "$type": "timestamp" } } } } + }, + { + "description": "$currentDate overwrites a field that is already there", + "volatile": ["a"], + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$currentDate": { "a": true } } } + }, + { + "description": "$currentDate down a dotted path", + "volatile": ["n.d"], + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$currentDate": { "n.d": true } } } + }, + { + "description": "$currentDate beside another operator", + "volatile": ["d"], + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$currentDate": { "d": true }, "$inc": { "a": 1 } } } + }, + { + "description": "$currentDate with false, which is still a date", + "volatile": ["d"], + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$currentDate": { "d": false } } } + }, + { + "description": "$currentDate with an unknown type name", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$currentDate": { "d": { "$type": "nope" } } } } + }, + { + "description": "$currentDate with a document that is not $type", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$currentDate": { "d": { "a": 1 } } } } + }, + { + "description": "$currentDate with a number", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$currentDate": { "d": 1 } } } + }, + { + "description": "$currentDate on an upsert that inserts", + "documents": [], + "volatile": ["d"], + "operation": "updateOne", + "arguments": { + "filter": { "k": 1 }, + "update": { "$currentDate": { "d": true } }, + "upsert": true + } + } + ] +} diff --git a/tests/spec/operators/sources/modifiers.json b/tests/spec/operators/sources/modifiers.json new file mode 100644 index 0000000..cd17fec --- /dev/null +++ b/tests/spec/operators/sources/modifiers.json @@ -0,0 +1,88 @@ +{ + "_comment": [ + "Inputs only. Expectations are measured -- see record.js.", + "The operator table itself, rather than any one operator: what an update", + "document may be, and what happens to a name that is not in the table.", + "This server answers `bad update` with code 2 to every one of these, which", + "is one message covering several different questions.", + "A `{b: 1, $set: {...}}` case was authored and then removed: the driver", + "rejects it before it reaches a server, so there is no server answer to", + "record and a case with no `errorCode` would assert nothing." + ], + "documents": [{ "_id": 1, "a": 1, "t": [1, 2] }], + "cases": [ + { + "description": "an unknown modifier", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$bogus": { "a": 1 } } } + }, + { + "description": "an unknown modifier beside a known one", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$set": { "b": 1 }, "$bogus": { "a": 1 } } } + }, + { + "description": "a known modifier whose argument is not a document", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$set": 1 } } + }, + { + "description": "an empty argument to a known modifier", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$set": {} } } + }, + { + "description": "a data field after an operator", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$set": { "c": 1 }, "b": 1 } } + }, + { + "description": "two operators writing the same field", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$set": { "a": 2 }, "$inc": { "a": 1 } } } + }, + { + "description": "$inc on a non-numeric field", + "documents": [{ "_id": 1, "a": "x" }], + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$inc": { "a": 1 } } } + }, + { + "description": "$inc by a non-numeric operand", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$inc": { "a": "x" } } } + }, + { + "description": "$push onto a non-array field", + "documents": [{ "_id": 1, "t": 5 }], + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$push": { "t": 1 } } } + }, + { + "description": "$pull on a non-array field", + "documents": [{ "_id": 1, "t": 5 }], + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$pull": { "t": 1 } } } + }, + { + "description": "$unset of a field that is not there", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$unset": { "gone": "" } } } + }, + { + "description": "$rename onto a field that already exists", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$rename": { "a": "t" } } } + }, + { + "description": "$rename of a field that is not there", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$rename": { "gone": "b" } } } + }, + { + "description": "an update that writes nothing still reports a match", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$set": { "a": 1 } } } + } + ] +} diff --git a/tests/spec/operators/sources/numeric.json b/tests/spec/operators/sources/numeric.json new file mode 100644 index 0000000..f53c4fb --- /dev/null +++ b/tests/spec/operators/sources/numeric.json @@ -0,0 +1,122 @@ +{ + "_comment": [ + "Inputs only. Expectations are measured -- see record.js.", + "`$mul`, `$min` and `$max`. The two comparison operators are not numeric", + "at all -- they compare in BSON canonical order, which is why a string", + "field and a number operand have a defined answer -- and the cases here", + "are shaped to say which of the three rules each one follows." + ], + "documents": [{ "_id": 1, "a": 5, "s": "b", "d": 2.5 }], + "cases": [ + { + "description": "$mul multiplies an int by an int", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$mul": { "a": 2 } } } + }, + { + "description": "$mul of a missing field", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$mul": { "gone": 5 } } } + }, + { + "description": "$mul mixes int and double", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$mul": { "d": 2 } } } + }, + { + "description": "$mul by zero", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$mul": { "a": 0 } } } + }, + { + "description": "$mul overflowing an int32", + "documents": [{ "_id": 1, "a": 2000000000 }], + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$mul": { "a": 2 } } } + }, + { + "description": "$mul of a non-numeric field", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$mul": { "s": 2 } } } + }, + { + "description": "$mul by a non-numeric operand", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$mul": { "a": "x" } } } + }, + { + "description": "$mul down a dotted path", + "documents": [{ "_id": 1, "n": { "a": 3 } }], + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$mul": { "n.a": 4 } } } + }, + { + "description": "$min writes when the operand is lower", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$min": { "a": 3 } } } + }, + { + "description": "$min leaves the field when the operand is higher", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$min": { "a": 7 } } } + }, + { + "description": "$min of a missing field always writes", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$min": { "gone": 7 } } } + }, + { + "description": "$min against a field of another type", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$min": { "s": 5 } } } + }, + { + "description": "$min compares an int and an equal double", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$min": { "a": 5.0 } } } + }, + { + "description": "$max writes when the operand is higher", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$max": { "a": 7 } } } + }, + { + "description": "$max leaves the field when the operand is lower", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$max": { "a": 3 } } } + }, + { + "description": "$max of a missing field always writes", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$max": { "gone": 7 } } } + }, + { + "description": "$max against a field of another type", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$max": { "s": 5 } } } + }, + { + "description": "$max against a null field", + "documents": [{ "_id": 1, "a": null }], + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$max": { "a": 1 } } } + }, + { + "description": "$min and $max on the same field in one update", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$min": { "a": 3 }, "$max": { "a": 9 } } } + }, + { + "description": "$min through an array index", + "documents": [{ "_id": 1, "t": [5, 5] }], + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$min": { "t.0": 1 } } } + }, + { + "description": "$mul through a positional segment", + "documents": [{ "_id": 1, "t": [{ "a": 2 }, { "a": 3 }] }], + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$mul": { "t.$[].a": 10 } } } + } + ] +} diff --git a/tests/spec/operators/sources/push-modifiers.json b/tests/spec/operators/sources/push-modifiers.json new file mode 100644 index 0000000..3ae75d2 --- /dev/null +++ b/tests/spec/operators/sources/push-modifiers.json @@ -0,0 +1,122 @@ +{ + "_comment": [ + "Inputs only. Expectations are measured -- see record.js.", + "`$push`'s modifiers: `$each`, `$slice`, `$position`, `$sort`. This server", + "already implements `$push` and `$each` and *silently ignores* the other", + "three -- `{$each: [3, 4], $slice: -3}` appended and did not slice, and", + "answered ok: 1. That is the same class of wrong answer the positional", + "operators were: the client asked for one thing and was told it got it." + ], + "documents": [{ "_id": 1, "t": [1, 2] }], + "cases": [ + { + "description": "$slice keeps the last n", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$push": { "t": { "$each": [3, 4], "$slice": -3 } } } } + }, + { + "description": "$slice keeps the first n", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$push": { "t": { "$each": [3, 4], "$slice": 3 } } } } + }, + { + "description": "$slice of zero empties the array", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$push": { "t": { "$each": [3], "$slice": 0 } } } } + }, + { + "description": "$slice larger than the array keeps all of it", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$push": { "t": { "$each": [3], "$slice": 10 } } } } + }, + { + "description": "$slice with an empty $each truncates without adding", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$push": { "t": { "$each": [], "$slice": 1 } } } } + }, + { + "description": "$position inserts at the front", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$push": { "t": { "$each": [9], "$position": 0 } } } } + }, + { + "description": "$position inserts in the middle", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$push": { "t": { "$each": [9], "$position": 1 } } } } + }, + { + "description": "$position past the end appends", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$push": { "t": { "$each": [9], "$position": 99 } } } } + }, + { + "description": "$position counted from the end", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$push": { "t": { "$each": [9], "$position": -1 } } } } + }, + { + "description": "$sort ascending over scalars", + "documents": [{ "_id": 1, "t": [3, 1] }], + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$push": { "t": { "$each": [2], "$sort": 1 } } } } + }, + { + "description": "$sort descending over scalars", + "documents": [{ "_id": 1, "t": [3, 1] }], + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$push": { "t": { "$each": [2], "$sort": -1 } } } } + }, + { + "description": "$sort on a field of the elements", + "documents": [{ "_id": 1, "t": [{ "a": 3 }, { "a": 1 }] }], + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$push": { "t": { "$each": [{ "a": 2 }], "$sort": { "a": 1 } } } } } + }, + { + "description": "$sort and $slice together", + "documents": [{ "_id": 1, "t": [3, 1] }], + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$push": { "t": { "$each": [2], "$sort": 1, "$slice": 2 } } } } + }, + { + "description": "$position and $slice together", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$push": { "t": { "$each": [9], "$position": 0, "$slice": 2 } } } } + }, + { + "description": "$slice without $each", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$push": { "t": { "$slice": 1 } } } } + }, + { + "description": "$each that is not an array", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$push": { "t": { "$each": 3 } } } } + }, + { + "description": "$slice that is not a number", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$push": { "t": { "$each": [3], "$slice": "x" } } } } + }, + { + "description": "$position that is not a number", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$push": { "t": { "$each": [3], "$position": "x" } } } } + }, + { + "description": "an unknown modifier beside $each", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$push": { "t": { "$each": [3], "$bogus": 1 } } } } + }, + { + "description": "a document operand that is not modifiers at all is a value", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$push": { "t": { "a": 1 } } } } + }, + { + "description": "$push $each onto a missing field", + "operation": "updateOne", + "arguments": { "filter": {}, "update": { "$push": { "gone": { "$each": [1, 2] } } } } + } + ] +} diff --git a/tests/spec/operators/sources/set-on-insert.json b/tests/spec/operators/sources/set-on-insert.json new file mode 100644 index 0000000..040f450 --- /dev/null +++ b/tests/spec/operators/sources/set-on-insert.json @@ -0,0 +1,99 @@ +{ + "_comment": [ + "Inputs only. Expectations are measured -- see record.js.", + "`$setOnInsert`, the only update operator whose meaning depends on which", + "branch of an upsert ran. Every case here is therefore a pair: the same", + "update against a document that exists and against one that does not." + ], + "documents": [{ "_id": 1, "k": 1, "a": 100 }], + "cases": [ + { + "description": "$setOnInsert writes when the upsert inserts", + "documents": [], + "operation": "updateOne", + "arguments": { + "filter": { "k": 1 }, + "update": { "$setOnInsert": { "a": 1 }, "$set": { "b": 2 } }, + "upsert": true + } + }, + { + "description": "$setOnInsert is ignored when the upsert updates", + "operation": "updateOne", + "arguments": { + "filter": { "k": 1 }, + "update": { "$setOnInsert": { "a": 1 }, "$set": { "b": 2 } }, + "upsert": true + } + }, + { + "description": "$setOnInsert alone on a plain update is a no-op", + "operation": "updateOne", + "arguments": { + "filter": { "k": 1 }, + "update": { "$setOnInsert": { "a": 1 } } + } + }, + { + "description": "$setOnInsert alone on an insert is the whole document", + "documents": [], + "operation": "updateOne", + "arguments": { + "filter": { "k": 7 }, + "update": { "$setOnInsert": { "a": 1 } }, + "upsert": true + } + }, + { + "description": "$setOnInsert writes a dotted path on insert", + "documents": [], + "operation": "updateOne", + "arguments": { + "filter": { "k": 1 }, + "update": { "$setOnInsert": { "x.y": 5 } }, + "upsert": true + } + }, + { + "description": "$setOnInsert may not touch _id", + "documents": [], + "operation": "updateOne", + "arguments": { + "filter": { "k": 1 }, + "update": { "$setOnInsert": { "_id": 9 } }, + "upsert": true + } + }, + { + "description": "$setOnInsert and $set naming the same field on an insert", + "documents": [], + "operation": "updateOne", + "arguments": { + "filter": { "k": 1 }, + "update": { "$setOnInsert": { "a": 1 }, "$set": { "a": 2 } }, + "upsert": true + } + }, + { + "description": "findOneAndUpdate reports the document $setOnInsert built", + "documents": [], + "operation": "findOneAndUpdate", + "arguments": { + "filter": { "k": 3 }, + "update": { "$setOnInsert": { "a": 1 } }, + "upsert": true, + "returnDocument": "after" + } + }, + { + "description": "updateMany with $setOnInsert on an upsert that inserts", + "documents": [], + "operation": "updateMany", + "arguments": { + "filter": { "k": 1 }, + "update": { "$setOnInsert": { "a": 1 } }, + "upsert": true + } + } + ] +} diff --git a/tests/spec/run.js b/tests/spec/run.js index 60e3b38..11c5143 100644 --- a/tests/spec/run.js +++ b/tests/spec/run.js @@ -44,9 +44,9 @@ function opt(name, dflt) { } const VERBOSE = !!opt('verbose', false); // The corpus. Defaults to the pinned crud suite; `--suite-dir` points the same -// runner at another one, which is how `tests/spec/aggregate/` and -// `tests/spec/positional/` are run -- both exist because the pinned suite has -// a hole where a whole feature should be. Sharing +// runner at another one, which is how `tests/spec/aggregate/`, +// `tests/spec/positional/` and `tests/spec/operators/` are run -- each exists +// because the pinned suite has a hole where a whole feature should be. Sharing // the runner rather than writing a second one is the point: the entity model, // the matchers, the skip accounting and `expectEvents` all come for free, and a // second runner would drift from this one exactly where it mattered. -- 2.39.5 From 301a4538fdbe3f70c971d64cc2b449326f817e26 Mon Sep 17 00:00:00 2001 From: "A.Shakhmatov" Date: Mon, 10 Aug 2026 21:04:39 +0300 Subject: [PATCH 2/9] update: one function per update operator `apply_operator` was a chain of six inline bodies and about a hundred lines, and M3 adds five more operators to it. One function each, and the shape every operator's argument has -- a document of path/operand pairs -- checked once at the top instead of six times. No behaviour change: 222/222, the same tests, unmoved. --- src/update.zig | 234 ++++++++++++++++++++++++++++--------------------- 1 file changed, 133 insertions(+), 101 deletions(-) diff --git a/src/update.zig b/src/update.zig index 44035cc..e62cce0 100644 --- a/src/update.zig +++ b/src/update.zig @@ -610,110 +610,142 @@ fn apply_operator( value: bson.Value, opts: Options, ) UpdateError!void { - const diag = opts.diag; - if (std.mem.eql(u8, op, "$set")) { - const ops = doc_pairs(value) orelse return error.InvalidUpdate; - for (ops) |p| { - if (std.mem.eql(u8, p.key, "_id")) return error.ImmutableId; - for (try resolve(arena, pairs.items, p.key, opts)) |segs| { - try set_path(arena, pairs, segs, try bson.copy_value(arena, p.value), p.key, diag); - } - } - return; - } - if (std.mem.eql(u8, op, "$unset")) { - const ops = doc_pairs(value) orelse return error.InvalidUpdate; - for (ops) |p| { - for (try resolve(arena, pairs.items, p.key, opts)) |segs| { - unset_path(arena, pairs, segs); - } - } - return; - } - if (std.mem.eql(u8, op, "$inc")) { - const ops = doc_pairs(value) orelse return error.InvalidUpdate; - for (ops) |p| { - for (try resolve(arena, pairs.items, p.key, opts)) |segs| { - const current = get_value(pairs.items, segs) orelse bson.Value{ .int32 = 0 }; - if (!current.is_number() or !p.value.is_number()) return error.InvalidUpdate; - const sum = try numeric_add(current, p.value); - try set_path(arena, pairs, segs, sum, p.key, diag); - } - } - return; - } - if (std.mem.eql(u8, op, "$push")) { - const ops = doc_pairs(value) orelse return error.InvalidUpdate; - for (ops) |p| { - for (try resolve(arena, pairs.items, p.key, opts)) |segs| { - const current_opt = get_value(pairs.items, segs); - var items: std.ArrayListUnmanaged(bson.Value) = .empty; - defer items.deinit(arena); - if (current_opt) |current| { - switch (current) { - .array => |arr| try items.appendSlice(arena, arr), - .null => {}, - else => return error.InvalidUpdate, // non-array field - } - } - if (p.value == .doc) { - if (bson.get_pair(p.value.doc, "$each")) |each| { - const arr = switch (each) { - .array => |a| a, - else => return error.InvalidUpdate, - }; - for (arr) |item| try items.append(arena, try bson.copy_value(arena, item)); - try set_path(arena, pairs, segs, .{ .array = try items.toOwnedSlice(arena) }, p.key, diag); - continue; - } - } - try items.append(arena, try bson.copy_value(arena, p.value)); - try set_path(arena, pairs, segs, .{ .array = try items.toOwnedSlice(arena) }, p.key, diag); - } - } - return; - } - if (std.mem.eql(u8, op, "$pull")) { - const ops = doc_pairs(value) orelse return error.InvalidUpdate; - for (ops) |p| { - for (try resolve(arena, pairs.items, p.key, opts)) |segs| { - const current = get_value(pairs.items, segs) orelse continue; - const arr = switch (current) { - .array => |a| a, - else => return error.InvalidUpdate, - }; - var items: std.ArrayListUnmanaged(bson.Value) = .empty; - defer items.deinit(arena); - for (arr) |elem| { - if (!pull_matches(arena, p.value, elem)) { - try items.append(arena, elem); - } - } - try set_path(arena, pairs, segs, .{ .array = try items.toOwnedSlice(arena) }, p.key, diag); - } - } - return; - } - // `$rename` alone keeps the plain split: `validate` has already refused a - // positional path on either end of it, which is what mongod does too. - if (std.mem.eql(u8, op, "$rename")) { - const ops = doc_pairs(value) orelse return error.InvalidUpdate; - for (ops) |p| { - if (p.value != .string) return error.InvalidUpdate; - if (std.mem.eql(u8, p.key, "_id") or std.mem.eql(u8, p.value.string, "_id")) return error.ImmutableId; - var old_segs: [max_path_segments][]const u8 = undefined; - const old_n = split_path(p.key, &old_segs) orelse return error.InvalidUpdate; - const v = get_value(pairs.items, old_segs[0..old_n]) orelse continue; // no-op when absent - unset_path(arena, pairs, old_segs[0..old_n]); - var new_segs: [max_path_segments][]const u8 = undefined; - const new_n = split_path(p.value.string, &new_segs) orelse return error.InvalidUpdate; - try set_path(arena, pairs, new_segs[0..new_n], v, p.value.string, diag); - } - return; - } + // Every operator's argument is a document of path/operand pairs, so the + // shape is checked once here rather than at the top of each. + const ops = doc_pairs(value) orelse return error.InvalidUpdate; + if (std.mem.eql(u8, op, "$set")) return op_set(arena, pairs, ops, opts); + if (std.mem.eql(u8, op, "$unset")) return op_unset(arena, pairs, ops, opts); + if (std.mem.eql(u8, op, "$inc")) return op_inc(arena, pairs, ops, opts); + if (std.mem.eql(u8, op, "$push")) return op_push(arena, pairs, ops, opts); + if (std.mem.eql(u8, op, "$pull")) return op_pull(arena, pairs, ops, opts); + if (std.mem.eql(u8, op, "$rename")) return op_rename(arena, pairs, ops, opts); return error.InvalidUpdate; } +fn op_set( + arena: std.mem.Allocator, + pairs: *std.ArrayListUnmanaged(bson.Pair), + ops: []const bson.Pair, + opts: Options, +) UpdateError!void { + for (ops) |p| { + if (std.mem.eql(u8, p.key, "_id")) return error.ImmutableId; + for (try resolve(arena, pairs.items, p.key, opts)) |segs| { + try set_path(arena, pairs, segs, try bson.copy_value(arena, p.value), p.key, opts.diag); + } + } +} + +fn op_unset( + arena: std.mem.Allocator, + pairs: *std.ArrayListUnmanaged(bson.Pair), + ops: []const bson.Pair, + opts: Options, +) UpdateError!void { + for (ops) |p| { + for (try resolve(arena, pairs.items, p.key, opts)) |segs| { + unset_path(arena, pairs, segs); + } + } +} + +fn op_inc( + arena: std.mem.Allocator, + pairs: *std.ArrayListUnmanaged(bson.Pair), + ops: []const bson.Pair, + opts: Options, +) UpdateError!void { + for (ops) |p| { + for (try resolve(arena, pairs.items, p.key, opts)) |segs| { + const current = get_value(pairs.items, segs) orelse bson.Value{ .int32 = 0 }; + if (!current.is_number() or !p.value.is_number()) return error.InvalidUpdate; + const sum = try numeric_add(current, p.value); + try set_path(arena, pairs, segs, sum, p.key, opts.diag); + } + } +} + +fn op_push( + arena: std.mem.Allocator, + pairs: *std.ArrayListUnmanaged(bson.Pair), + ops: []const bson.Pair, + opts: Options, +) UpdateError!void { + for (ops) |p| { + for (try resolve(arena, pairs.items, p.key, opts)) |segs| { + const current_opt = get_value(pairs.items, segs); + var items: std.ArrayListUnmanaged(bson.Value) = .empty; + defer items.deinit(arena); + if (current_opt) |current| { + switch (current) { + .array => |arr| try items.appendSlice(arena, arr), + .null => {}, + else => return error.InvalidUpdate, // non-array field + } + } + if (p.value == .doc) { + if (bson.get_pair(p.value.doc, "$each")) |each| { + const arr = switch (each) { + .array => |a| a, + else => return error.InvalidUpdate, + }; + for (arr) |item| try items.append(arena, try bson.copy_value(arena, item)); + try set_path(arena, pairs, segs, .{ .array = try items.toOwnedSlice(arena) }, p.key, opts.diag); + continue; + } + } + try items.append(arena, try bson.copy_value(arena, p.value)); + try set_path(arena, pairs, segs, .{ .array = try items.toOwnedSlice(arena) }, p.key, opts.diag); + } + } +} + +fn op_pull( + arena: std.mem.Allocator, + pairs: *std.ArrayListUnmanaged(bson.Pair), + ops: []const bson.Pair, + opts: Options, +) UpdateError!void { + for (ops) |p| { + for (try resolve(arena, pairs.items, p.key, opts)) |segs| { + const current = get_value(pairs.items, segs) orelse continue; + const arr = switch (current) { + .array => |a| a, + else => return error.InvalidUpdate, + }; + var items: std.ArrayListUnmanaged(bson.Value) = .empty; + defer items.deinit(arena); + for (arr) |elem| { + if (!pull_matches(arena, p.value, elem)) { + try items.append(arena, elem); + } + } + try set_path(arena, pairs, segs, .{ .array = try items.toOwnedSlice(arena) }, p.key, opts.diag); + } + } +} + +/// `$rename` alone keeps the plain split: `validate` has already refused a +/// positional path on either end of it, which is what mongod does too. +fn op_rename( + arena: std.mem.Allocator, + pairs: *std.ArrayListUnmanaged(bson.Pair), + ops: []const bson.Pair, + opts: Options, +) UpdateError!void { + for (ops) |p| { + if (p.value != .string) return error.InvalidUpdate; + if (std.mem.eql(u8, p.key, "_id") or std.mem.eql(u8, p.value.string, "_id")) return error.ImmutableId; + var old_segs: [max_path_segments][]const u8 = undefined; + const old_n = split_path(p.key, &old_segs) orelse return error.InvalidUpdate; + const v = get_value(pairs.items, old_segs[0..old_n]) orelse continue; // no-op when absent + unset_path(arena, pairs, old_segs[0..old_n]); + var new_segs: [max_path_segments][]const u8 = undefined; + const new_n = split_path(p.value.string, &new_segs) orelse return error.InvalidUpdate; + try set_path(arena, pairs, new_segs[0..new_n], v, p.value.string, opts.diag); + } +} + fn doc_pairs(v: bson.Value) ?[]const bson.Pair { return switch (v) { .doc => |pairs| pairs, -- 2.39.5 From c2f682717b6a316d000ac412224aed608033831d Mon Sep 17 00:00:00 2001 From: "A.Shakhmatov" Date: Mon, 10 Aug 2026 21:08:29 +0300 Subject: [PATCH 3/9] update: $mul, $min and $max Three operators, two shapes. `$mul` joins `$inc` in `op_arith` because they differ only in the operation; `$min` and `$max` are not numeric operators at all and get their own. Measured, and each row is a rule that would have been guessed wrong: - `{$mul: {gone: 5}}` writes **0**, not 5. An absent field starts from zero under both operators, which is the identity for one and the annihilator for the other, and mongod picks zero for both. - `$min`/`$max` compare in BSON canonical order, so `{$min: {s: 5}}` on `s: "b"` writes 5 -- a number ranks below a string -- and `{$max: {a: 1}}` on `a: null` writes 1. An absent field is always written: there is nothing to be smaller or larger than. - an int32 product that does not fit widens to int64, the same ladder `numeric_add` already climbed. `$inc` changes answer with them: a non-numeric field or operand was `InvalidUpdate` -> BadValue (2), and mongod answers TypeMismatch (14) with a different sentence for each side. So two errors rather than one, and `$inc` gets the codes it should always have had. numeric.json 0/21 -> 20/21. The one left is `$min` and `$max` on the same field, which is ConflictingUpdateOperators (40) -- a whole error class this server does not have yet, and its own commit. 228/228 unit tests. --- src/commands.zig | 19 +++++ src/update.zig | 205 +++++++++++++++++++++++++++++++++++++++++++++-- 2 files changed, 218 insertions(+), 6 deletions(-) diff --git a/src/commands.zig b/src/commands.zig index 4b5d26f..3b7fa36 100644 --- a/src/commands.zig +++ b/src/commands.zig @@ -4253,6 +4253,25 @@ fn update_refusal(reply: *wire.Reply, err: anyerror, diag: update.Diagnostic) !v "name, found '{s}' and '{s}'", .{ diag.segment, diag.other }, )), + error.NotNumericField => return reply.put_error( + @intFromEnum(ErrorCode.type_mismatch), + "TypeMismatch", + try std.fmt.allocPrint( + arena, + "Cannot apply {s} to a value of non-numeric type. The field '{s}' is of " ++ + "non-numeric type {s}", + .{ diag.segment, diag.path, diag.other }, + ), + ), + error.NotNumericOperand => return reply.put_error( + @intFromEnum(ErrorCode.type_mismatch), + "TypeMismatch", + try std.fmt.allocPrint( + arena, + "Cannot {s} with non-numeric argument at field '{s}'", + .{ if (std.mem.eql(u8, diag.segment, "$inc")) "increment" else "multiply", diag.path }, + ), + ), error.PathNotViable => return reply.put_error( @intFromEnum(ErrorCode.path_not_viable), "PathNotViable", diff --git a/src/update.zig b/src/update.zig index e62cce0..dca3b54 100644 --- a/src/update.zig +++ b/src/update.zig @@ -16,6 +16,12 @@ const query = @import("query.zig"); pub const UpdateError = error{ ImmutableId, InvalidUpdate, + /// `$inc` or `$mul` against a stored field that is not a number, and by an + /// operand that is not one. Both are mongod's `TypeMismatch` (14) rather + /// than the `BadValue` most bad updates answer, and each has its own + /// sentence, so they are two errors. + NotNumericField, + NotNumericOperand, /// A path segment that is not a field to create: a non-numeric name /// applied to an array, or any name applied to a scalar element a /// positional segment selected. mongod's `PathNotViable`. @@ -615,7 +621,10 @@ fn apply_operator( const ops = doc_pairs(value) orelse return error.InvalidUpdate; if (std.mem.eql(u8, op, "$set")) return op_set(arena, pairs, ops, opts); if (std.mem.eql(u8, op, "$unset")) return op_unset(arena, pairs, ops, opts); - if (std.mem.eql(u8, op, "$inc")) return op_inc(arena, pairs, ops, opts); + if (std.mem.eql(u8, op, "$inc")) return op_arith(arena, pairs, ops, opts, .add); + if (std.mem.eql(u8, op, "$mul")) return op_arith(arena, pairs, ops, opts, .mul); + if (std.mem.eql(u8, op, "$min")) return op_extremum(arena, pairs, ops, opts, .lt); + if (std.mem.eql(u8, op, "$max")) return op_extremum(arena, pairs, ops, opts, .gt); if (std.mem.eql(u8, op, "$push")) return op_push(arena, pairs, ops, opts); if (std.mem.eql(u8, op, "$pull")) return op_pull(arena, pairs, ops, opts); if (std.mem.eql(u8, op, "$rename")) return op_rename(arena, pairs, ops, opts); @@ -649,18 +658,58 @@ fn op_unset( } } -fn op_inc( +/// `$inc` and `$mul`, which differ only in the operation and in what an absent +/// field starts from: `$inc` from 0 because adding leaves the operand, `$mul` +/// from 0 because multiplying does too -- so `{$mul: {gone: 5}}` writes 0, not +/// 5. Measured; the natural guess is the other one. +fn op_arith( arena: std.mem.Allocator, pairs: *std.ArrayListUnmanaged(bson.Pair), ops: []const bson.Pair, opts: Options, + comptime kind: enum { add, mul }, +) UpdateError!void { + for (ops) |p| { + // The operand is checked before the paths are resolved, so an operand + // that is not a number is one answer for the whole update rather than + // one per element a positional segment reached. + const op_name = if (kind == .add) "$inc" else "$mul"; + if (!p.value.is_number()) { + note(opts.diag, p.key, op_name); + return error.NotNumericOperand; + } + for (try resolve(arena, pairs.items, p.key, opts)) |segs| { + const current = get_value(pairs.items, segs) orelse bson.Value{ .int32 = 0 }; + if (!current.is_number()) { + if (opts.diag) |d| d.* = .{ .path = p.key, .segment = op_name, .other = current.type_name() }; + return error.NotNumericField; + } + const result = switch (kind) { + .add => try numeric_add(current, p.value), + .mul => try numeric_mul(current, p.value), + }; + try set_path(arena, pairs, segs, result, p.key, opts.diag); + } + } +} + +/// `$min` and `$max`, which are not numeric operators at all: they compare in +/// BSON canonical order, so `{$min: {s: 5}}` on `s: "b"` writes 5 because a +/// number ranks below a string. An absent field is always written -- there is +/// nothing to be smaller or larger than. +fn op_extremum( + arena: std.mem.Allocator, + pairs: *std.ArrayListUnmanaged(bson.Pair), + ops: []const bson.Pair, + opts: Options, + comptime want: std.math.Order, ) UpdateError!void { for (ops) |p| { for (try resolve(arena, pairs.items, p.key, opts)) |segs| { - const current = get_value(pairs.items, segs) orelse bson.Value{ .int32 = 0 }; - if (!current.is_number() or !p.value.is_number()) return error.InvalidUpdate; - const sum = try numeric_add(current, p.value); - try set_path(arena, pairs, segs, sum, p.key, opts.diag); + if (get_value(pairs.items, segs)) |current| { + if (bson.compare(p.value, current) != want) continue; + } + try set_path(arena, pairs, segs, try bson.copy_value(arena, p.value), p.key, opts.diag); } } } @@ -970,6 +1019,35 @@ fn numeric_add(a: bson.Value, b: bson.Value) UpdateError!bson.Value { return .{ .int64 = sum }; } +/// The same widening ladder as `numeric_add`: a double anywhere makes the +/// answer a double, two int32s stay int32 unless the product does not fit, +/// and an int64 overflowing is a refusal rather than a wrap. +fn numeric_mul(a: bson.Value, b: bson.Value) UpdateError!bson.Value { + if (a == .double or b == .double) { + const product: f64 = @floatCast(a.as_f128() * b.as_f128()); + return .{ .double = product }; + } + if (a == .int64 or b == .int64) { + const av: i64 = as_int64(a); + const bv: i64 = as_int64(b); + const product = std.math.mul(i64, av, bv) catch return error.InvalidUpdate; + return .{ .int64 = product }; + } + const product: i64 = @as(i64, a.int32) * b.int32; + if (product >= std.math.minInt(i32) and product <= std.math.maxInt(i32)) { + return .{ .int32 = @intCast(product) }; + } + return .{ .int64 = product }; +} + +fn as_int64(v: bson.Value) i64 { + return switch (v) { + .int32 => |i| i, + .int64 => |i| i, + else => unreachable, + }; +} + // --------------------------------------------------------------------------- // Tests // --------------------------------------------------------------------------- @@ -1671,3 +1749,118 @@ test "a replacement is not a path, so it is not refused" { try testing.expect(doc.get("y") == null); try testing.expectEqual(@as(i32, 1), doc.get("z").?.int32); } + +test "$mul multiplies, and starts a missing field from zero" { + // The measured surprise: `{$mul: {gone: 5}}` writes 0, not 5. Mutation + // check: start `op_arith` from `.{ .int32 = 1 }` for `.mul` and the second + // half goes red -- which is the reading anyone would reach for. + var doc = try doc_with(testing.allocator, &.{ + .{ .key = "a", .value = .{ .int32 = 5 } }, + .{ .key = "d", .value = .{ .double = 2.5 } }, + }); + defer doc.arena.deinit(); + try apply(&doc, &doc_of(&.{ + .{ .key = "$mul", .value = .{ .doc = &.{ + .{ .key = "a", .value = .{ .int32 = 2 } }, + .{ .key = "d", .value = .{ .int32 = 2 } }, + .{ .key = "gone", .value = .{ .int32 = 5 } }, + } } }, + }), .{}); + try testing.expectEqual(@as(i32, 10), doc.get("a").?.int32); + try testing.expectEqual(@as(f64, 5.0), doc.get("d").?.double); + try testing.expectEqual(@as(i32, 0), doc.get("gone").?.int32); +} + +test "$mul widens an int32 product that does not fit" { + var doc = try doc_with(testing.allocator, &.{ + .{ .key = "a", .value = .{ .int32 = 2000000000 } }, + }); + defer doc.arena.deinit(); + try apply(&doc, &doc_of(&.{ + .{ .key = "$mul", .value = .{ .doc = &.{.{ .key = "a", .value = .{ .int32 = 2 } }} } }, + }), .{}); + try testing.expectEqual(@as(i64, 4000000000), doc.get("a").?.int64); +} + +test "$inc and $mul refuse a non-number, on either side" { + // TypeMismatch, not the BadValue the rest of a bad update answers, and the + // field and the operand are different sentences on mongod -- so they are + // different errors here. + var doc = try doc_with(testing.allocator, &.{ + .{ .key = "a", .value = .{ .int32 = 5 } }, + .{ .key = "s", .value = .{ .string = "b" } }, + }); + defer doc.arena.deinit(); + var diag: Diagnostic = .{}; + for ([_][]const u8{ "$inc", "$mul" }) |op| { + try testing.expectError(error.NotNumericField, apply(&doc, &doc_of(&.{ + .{ .key = op, .value = .{ .doc = &.{.{ .key = "s", .value = .{ .int32 = 2 } }} } }, + }), .{ .diag = &diag })); + try testing.expectEqualStrings("string", diag.other); + try testing.expectError(error.NotNumericOperand, apply(&doc, &doc_of(&.{ + .{ .key = op, .value = .{ .doc = &.{.{ .key = "a", .value = .{ .string = "x" } }} } }, + }), .{ .diag = &diag })); + try testing.expectEqualStrings(op, diag.segment); + } + // Neither refusal wrote anything. + try testing.expectEqual(@as(i32, 5), doc.get("a").?.int32); + try testing.expectEqualStrings("b", doc.get("s").?.string); +} + +test "$min and $max compare in BSON order, not numerically" { + // The load-bearing case: `s` holds a string and the operand is a number, + // and there is still a defined answer because a number ranks below a + // string. Mutation check: make `op_extremum` require both to be numbers + // and the two `s` rows go red. + var doc = try doc_with(testing.allocator, &.{ + .{ .key = "a", .value = .{ .int32 = 5 } }, + .{ .key = "s", .value = .{ .string = "b" } }, + .{ .key = "n", .value = .null }, + }); + defer doc.arena.deinit(); + try apply(&doc, &doc_of(&.{ + .{ .key = "$min", .value = .{ .doc = &.{ + .{ .key = "a", .value = .{ .int32 = 7 } }, // higher: not written + .{ .key = "s", .value = .{ .int32 = 5 } }, // a number is below a string + .{ .key = "gone", .value = .{ .int32 = 7 } }, // absent: always written + } } }, + }), .{}); + try testing.expectEqual(@as(i32, 5), doc.get("a").?.int32); + try testing.expectEqual(@as(i32, 5), doc.get("s").?.int32); + try testing.expectEqual(@as(i32, 7), doc.get("gone").?.int32); + + try apply(&doc, &doc_of(&.{ + .{ .key = "$max", .value = .{ .doc = &.{ + .{ .key = "a", .value = .{ .int32 = 3 } }, // lower: not written + .{ .key = "n", .value = .{ .int32 = 1 } }, // a number is above null + } } }, + }), .{}); + try testing.expectEqual(@as(i32, 5), doc.get("a").?.int32); + try testing.expectEqual(@as(i32, 1), doc.get("n").?.int32); +} + +test "$min treats an int and an equal double as the same value" { + // Equal is not less, so nothing is written and the stored type survives. + var doc = try doc_with(testing.allocator, &.{.{ .key = "a", .value = .{ .int32 = 5 } }}); + defer doc.arena.deinit(); + try apply(&doc, &doc_of(&.{ + .{ .key = "$min", .value = .{ .doc = &.{.{ .key = "a", .value = .{ .double = 5.0 } }} } }, + }), .{}); + try testing.expect(doc.get("a").? == .int32); +} + +test "$mul reaches every element a positional segment names" { + var doc = try doc_with(testing.allocator, &.{ + .{ .key = "t", .value = .{ .array = &.{ + .{ .doc = &.{.{ .key = "a", .value = .{ .int32 = 2 } }} }, + .{ .doc = &.{.{ .key = "a", .value = .{ .int32 = 3 } }} }, + } } }, + }); + defer doc.arena.deinit(); + try apply(&doc, &doc_of(&.{ + .{ .key = "$mul", .value = .{ .doc = &.{.{ .key = "t.$[].a", .value = .{ .int32 = 10 } }} } }, + }), .{}); + const t = doc.get("t").?.array; + try testing.expectEqual(@as(i32, 20), t[0].doc[0].value.int32); + try testing.expectEqual(@as(i32, 30), t[1].doc[0].value.int32); +} -- 2.39.5 From 066b32617e21e778607ffbcf9ec3875fa8ecf41c Mon Sep 17 00:00:00 2001 From: "A.Shakhmatov" Date: Mon, 10 Aug 2026 21:11:33 +0300 Subject: [PATCH 4/9] update: $addToSet, $pop and $pullAll `$addToSet`'s identity is `bson.compare` equality, which is already exactly mongod's: an int32 `2` and a double `2.0` are one value, and `{a: 1, b: 2}` and `{b: 2, a: 1}` are two, because `compare_docs` walks the pairs positionally and tie-breaks on the key. Candidates are checked against the array as it grows, so a `$each` holding the same value twice adds it once. `$pullAll` is `$pull`'s neighbour and its opposite: `$pull` takes a predicate, `$pullAll` takes values compared whole. `{$pull: {t: {a: 1}}}` removes elements *having* `a: 1`; `{$pullAll: {t: [{a: 1}]}}` removes elements that *are* `{a: 1}`. Sharing the comparison would have been the natural mistake and is what the test mutates to check. `$pop` is the small one, and its refusals are the measured part: an empty array and an absent field are no-ops, an argument that is not 1 or -1 is FailedToParse (9), and a non-array field is TypeMismatch (14) -- where the identical mistake under `$addToSet` and `$pullAll` is BadValue (2). Three codes for one shape, none of them derivable from the others. `$each` that is not an array is 14 under `$addToSet` and 2 under `$push`, measured on both. array-ops.json 2/26 -> 26/26. 234/234 unit tests. --- src/commands.zig | 36 ++++++ src/update.zig | 286 +++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 322 insertions(+) diff --git a/src/commands.zig b/src/commands.zig index 3b7fa36..2146436 100644 --- a/src/commands.zig +++ b/src/commands.zig @@ -4272,6 +4272,42 @@ fn update_refusal(reply: *wire.Reply, err: anyerror, diag: update.Diagnostic) !v .{ if (std.mem.eql(u8, diag.segment, "$inc")) "increment" else "multiply", diag.path }, ), ), + error.NotAnArrayField => return bad_value(reply, try std.fmt.allocPrint( + arena, + "Cannot apply {s} to non-array field. Field named '{s}' has non-array type", + .{ diag.segment, diag.path }, + )), + error.NotAnArrayPathElement => return reply.put_error( + @intFromEnum(ErrorCode.type_mismatch), + "TypeMismatch", + try std.fmt.allocPrint( + arena, + "Path '{s}' contains an element of non-array type", + .{diag.path}, + ), + ), + // mongod has two sentences here -- "$pop expects 1 or -1, found: 2" + // and "Expected a number in: t: \"x\"" -- both code 9, and both about + // an argument that is not one of the two values `$pop` takes. One + // sentence covering both says the same thing without rendering the + // operand, which no formatter here does. + error.BadPopArgument => return failed_to_parse(reply, try std.fmt.allocPrint( + arena, + "$pop expects 1 or -1, at field '{s}'", + .{diag.path}, + )), + error.PullAllNeedsArray => return bad_value(reply, try std.fmt.allocPrint( + arena, + "$pullAll requires an array argument but was given a {s}", + .{diag.other}, + )), + // `$push` and `$addToSet` disagree about the code for the identical + // mistake: 2 and 14. Measured on both, and not derivable from either. + error.BadEach => if (std.mem.eql(u8, diag.segment, "$addToSet")) return reply.put_error( + @intFromEnum(ErrorCode.type_mismatch), + "TypeMismatch", + "The argument to $each in $addToSet must be an array", + ) else return bad_value(reply, "The argument to $each in $push must be an array"), error.PathNotViable => return reply.put_error( @intFromEnum(ErrorCode.path_not_viable), "PathNotViable", diff --git a/src/update.zig b/src/update.zig index dca3b54..6b3ddce 100644 --- a/src/update.zig +++ b/src/update.zig @@ -22,6 +22,18 @@ pub const UpdateError = error{ /// sentence, so they are two errors. NotNumericField, NotNumericOperand, + /// `$addToSet` or `$pullAll` against a field that is not an array. + /// `BadValue` (2) on mongod, where `$pop` against one is `TypeMismatch`. + NotAnArrayField, + /// `$pop` against a field that is not an array. Same mistake as + /// `NotAnArrayField`, different code -- measured, not derived. + NotAnArrayPathElement, + /// `$pop` by anything that is not 1 or -1. `FailedToParse` (9). + BadPopArgument, + /// `$pullAll` by something that is not an array of values. + PullAllNeedsArray, + /// `$each` that is not an array. + BadEach, /// A path segment that is not a field to create: a non-numeric name /// applied to an array, or any name applied to a scalar element a /// positional segment selected. mongod's `PathNotViable`. @@ -627,6 +639,9 @@ fn apply_operator( if (std.mem.eql(u8, op, "$max")) return op_extremum(arena, pairs, ops, opts, .gt); if (std.mem.eql(u8, op, "$push")) return op_push(arena, pairs, ops, opts); if (std.mem.eql(u8, op, "$pull")) return op_pull(arena, pairs, ops, opts); + if (std.mem.eql(u8, op, "$addToSet")) return op_add_to_set(arena, pairs, ops, opts); + if (std.mem.eql(u8, op, "$pop")) return op_pop(arena, pairs, ops, opts); + if (std.mem.eql(u8, op, "$pullAll")) return op_pull_all(arena, pairs, ops, opts); if (std.mem.eql(u8, op, "$rename")) return op_rename(arena, pairs, ops, opts); return error.InvalidUpdate; } @@ -774,6 +789,139 @@ fn op_pull( } } +/// `$addToSet`: append what the array does not already hold. +/// +/// "Already hold" is `bson.compare` equality, which is exactly mongod's: an +/// int32 `2` and a double `2.0` are one value, and two documents with the same +/// fields in a different order are two -- because `compare_docs` walks the +/// pairs positionally and tie-breaks on the key. +fn op_add_to_set( + arena: std.mem.Allocator, + pairs: *std.ArrayListUnmanaged(bson.Pair), + ops: []const bson.Pair, + opts: Options, +) UpdateError!void { + for (ops) |p| { + // `$each` is a modifier only here and under `$push`; anywhere else a + // document operand is the value being added. + const candidates: []const bson.Value = if (each_of(p.value)) |each| switch (each) { + .array => |a| a, + else => { + note(opts.diag, p.key, "$addToSet"); + return error.BadEach; + }, + } else &.{p.value}; + + for (try resolve(arena, pairs.items, p.key, opts)) |segs| { + var items: std.ArrayListUnmanaged(bson.Value) = .empty; + defer items.deinit(arena); + if (get_value(pairs.items, segs)) |current| switch (current) { + .array => |arr| try items.appendSlice(arena, arr), + .null => {}, + else => { + note(opts.diag, p.key, "$addToSet"); + return error.NotAnArrayField; + }, + }; + for (candidates) |c| { + if (holds(items.items, c)) continue; + try items.append(arena, try bson.copy_value(arena, c)); + } + try set_path(arena, pairs, segs, .{ .array = try items.toOwnedSlice(arena) }, p.key, opts.diag); + } + } +} + +fn holds(items: []const bson.Value, v: bson.Value) bool { + for (items) |item| if (bson.compare(item, v) == .eq) return true; + return false; +} + +/// The `$each` of a modifier document, or null when the operand is a value. +/// +/// Presence of `$each` is what makes an operand a modifier document at all -- +/// measured: `{$push: {t: {$slice: 1}}}` pushes `{$slice: 1}` as a value. +fn each_of(v: bson.Value) ?bson.Value { + const doc = doc_pairs(v) orelse return null; + return bson.get_pair(doc, "$each"); +} + +/// `$pop`: remove one element from an end. `1` is the last, `-1` the first. +/// +/// An empty array and an absent field are both no-ops rather than errors, so +/// the only refusals are the argument and the field's type. +fn op_pop( + arena: std.mem.Allocator, + pairs: *std.ArrayListUnmanaged(bson.Pair), + ops: []const bson.Pair, + opts: Options, +) UpdateError!void { + for (ops) |p| { + if (!p.value.is_number()) { + note(opts.diag, p.key, ""); + return error.BadPopArgument; + } + const n = p.value.as_f128(); + if (n != 1 and n != -1) { + note(opts.diag, p.key, ""); + return error.BadPopArgument; + } + for (try resolve(arena, pairs.items, p.key, opts)) |segs| { + const current = get_value(pairs.items, segs) orelse continue; + const arr = switch (current) { + .array => |a| a, + else => { + note(opts.diag, p.key, "$pop"); + return error.NotAnArrayPathElement; + }, + }; + if (arr.len == 0) continue; + const kept = if (n == 1) arr[0 .. arr.len - 1] else arr[1..]; + try set_path(arena, pairs, segs, .{ .array = try arena.dupe(bson.Value, kept) }, p.key, opts.diag); + } + } +} + +/// `$pullAll`: remove every element equal to any of the listed values. +/// +/// The difference from `$pull` is the whole of it: `$pull` takes a *predicate* +/// and `$pullAll` takes values, compared whole. `{$pull: {t: {a: 1}}}` matches +/// elements having `a: 1`; `{$pullAll: {t: [{a: 1}]}}` matches elements that +/// *are* `{a: 1}`. +fn op_pull_all( + arena: std.mem.Allocator, + pairs: *std.ArrayListUnmanaged(bson.Pair), + ops: []const bson.Pair, + opts: Options, +) UpdateError!void { + for (ops) |p| { + const wanted = switch (p.value) { + .array => |a| a, + else => { + if (opts.diag) |d| d.* = .{ .path = p.key, .other = p.value.type_name() }; + return error.PullAllNeedsArray; + }, + }; + for (try resolve(arena, pairs.items, p.key, opts)) |segs| { + const current = get_value(pairs.items, segs) orelse continue; + const arr = switch (current) { + .array => |a| a, + else => { + note(opts.diag, p.key, "$pullAll"); + return error.NotAnArrayField; + }, + }; + var items: std.ArrayListUnmanaged(bson.Value) = .empty; + defer items.deinit(arena); + for (arr) |elem| { + if (holds(wanted, elem)) continue; + try items.append(arena, elem); + } + try set_path(arena, pairs, segs, .{ .array = try items.toOwnedSlice(arena) }, p.key, opts.diag); + } + } +} + /// `$rename` alone keeps the plain split: `validate` has already refused a /// positional path on either end of it, which is what mongod does too. fn op_rename( @@ -1864,3 +2012,141 @@ test "$mul reaches every element a positional segment names" { try testing.expectEqual(@as(i32, 20), t[0].doc[0].value.int32); try testing.expectEqual(@as(i32, 30), t[1].doc[0].value.int32); } + +test "$addToSet appends only what the array does not hold" { + var doc = try doc_with(testing.allocator, &.{ + .{ .key = "t", .value = .{ .array = &.{ .{ .int32 = 1 }, .{ .int32 = 2 } } } }, + }); + defer doc.arena.deinit(); + try apply(&doc, &doc_of(&.{ + .{ .key = "$addToSet", .value = .{ .doc = &.{ + .{ .key = "t", .value = .{ .doc = &.{ + .{ .key = "$each", .value = .{ .array = &.{ .{ .int32 = 2 }, .{ .int32 = 3 }, .{ .int32 = 3 } } } }, + } } }, + .{ .key = "gone", .value = .{ .int32 = 1 } }, + } } }, + }), .{}); + // 2 is already there, and the two 3s in one `$each` are one value: the + // candidates are checked against the array *as it grows*. + const t = doc.get("t").?.array; + try testing.expectEqual(@as(usize, 3), t.len); + try testing.expectEqual(@as(i32, 3), t[2].int32); + // An absent field becomes a one-element array rather than an error. + try testing.expectEqual(@as(usize, 1), doc.get("gone").?.array.len); +} + +test "$addToSet identity is BSON equality, field order included" { + // Two rows, opposite answers, one comparator. Mutation check: compare with + // anything that ignores key order and the second half goes red -- mongod + // stores both spellings of the same document. + var doc = try doc_with(testing.allocator, &.{ + .{ .key = "n", .value = .{ .array = &.{.{ .int32 = 2 }} } }, + .{ .key = "d", .value = .{ .array = &.{.{ .doc = &.{ + .{ .key = "a", .value = .{ .int32 = 1 } }, + .{ .key = "b", .value = .{ .int32 = 2 } }, + } }} } }, + }); + defer doc.arena.deinit(); + try apply(&doc, &doc_of(&.{ + .{ .key = "$addToSet", .value = .{ .doc = &.{ + // An int32 2 and a double 2.0 are one value. + .{ .key = "n", .value = .{ .double = 2.0 } }, + // The same fields in the other order are two. + .{ .key = "d", .value = .{ .doc = &.{ + .{ .key = "b", .value = .{ .int32 = 2 } }, + .{ .key = "a", .value = .{ .int32 = 1 } }, + } } }, + } } }, + }), .{}); + try testing.expectEqual(@as(usize, 1), doc.get("n").?.array.len); + try testing.expectEqual(@as(usize, 2), doc.get("d").?.array.len); +} + +test "$pop takes one element off an end, and is quiet when there is none" { + var doc = try doc_with(testing.allocator, &.{ + .{ .key = "a", .value = .{ .array = &.{ .{ .int32 = 1 }, .{ .int32 = 2 }, .{ .int32 = 3 } } } }, + .{ .key = "b", .value = .{ .array = &.{ .{ .int32 = 1 }, .{ .int32 = 2 } } } }, + .{ .key = "e", .value = .{ .array = &.{} } }, + }); + defer doc.arena.deinit(); + try apply(&doc, &doc_of(&.{ + .{ .key = "$pop", .value = .{ .doc = &.{ + .{ .key = "a", .value = .{ .int32 = 1 } }, // the last + .{ .key = "b", .value = .{ .double = -1.0 } }, // the first, and -1.0 is -1 + .{ .key = "e", .value = .{ .int32 = 1 } }, // empty: no-op + .{ .key = "gone", .value = .{ .int32 = 1 } }, // absent: no-op + } } }, + }), .{}); + try testing.expectEqual(@as(usize, 2), doc.get("a").?.array.len); + try testing.expectEqual(@as(i32, 2), doc.get("a").?.array[1].int32); + try testing.expectEqual(@as(usize, 1), doc.get("b").?.array.len); + try testing.expectEqual(@as(i32, 2), doc.get("b").?.array[0].int32); + try testing.expectEqual(@as(usize, 0), doc.get("e").?.array.len); + try testing.expect(doc.get("gone") == null); +} + +test "$pop refuses an argument that is not one of its two values" { + var doc = try doc_with(testing.allocator, &.{ + .{ .key = "t", .value = .{ .array = &.{.{ .int32 = 1 }} } }, + }); + defer doc.arena.deinit(); + for ([_]bson.Value{ .{ .int32 = 2 }, .{ .int32 = 0 }, .{ .string = "x" } }) |bad| { + try testing.expectError(error.BadPopArgument, apply(&doc, &doc_of(&.{ + .{ .key = "$pop", .value = .{ .doc = &.{.{ .key = "t", .value = bad }} } }, + }), .{})); + } + try testing.expectEqual(@as(usize, 1), doc.get("t").?.array.len); +} + +test "$pullAll removes values, where $pull removes matches" { + // The whole difference between the two, in one document: `{a: 1}` as a + // `$pull` argument is a predicate and as a `$pullAll` element is a value. + // Mutation check: route `$pullAll` through `pull_matches` and the second + // half goes red -- `{a: 1, b: 2}` would be pulled by a predicate too. + var doc = try doc_with(testing.allocator, &.{ + .{ .key = "n", .value = .{ .array = &.{ + .{ .int32 = 1 }, .{ .int32 = 2 }, .{ .int32 = 3 }, .{ .int32 = 2 }, + } } }, + .{ .key = "d", .value = .{ .array = &.{ + .{ .doc = &.{.{ .key = "a", .value = .{ .int32 = 1 } }} }, + .{ .doc = &.{ + .{ .key = "a", .value = .{ .int32 = 1 } }, + .{ .key = "b", .value = .{ .int32 = 2 } }, + } }, + } } }, + }); + defer doc.arena.deinit(); + try apply(&doc, &doc_of(&.{ + .{ .key = "$pullAll", .value = .{ .doc = &.{ + .{ .key = "n", .value = .{ .array = &.{.{ .int32 = 2 }} } }, + .{ .key = "d", .value = .{ .array = &.{ + .{ .doc = &.{.{ .key = "a", .value = .{ .int32 = 1 } }} }, + } } }, + .{ .key = "gone", .value = .{ .array = &.{.{ .int32 = 1 }} } }, + } } }, + }), .{}); + try testing.expectEqual(@as(usize, 2), doc.get("n").?.array.len); + // Only the element that *is* `{a: 1}` went; the one that merely matches + // that predicate stayed. + try testing.expectEqual(@as(usize, 1), doc.get("d").?.array.len); + try testing.expectEqual(@as(usize, 2), doc.get("d").?.array[0].doc.len); + try testing.expect(doc.get("gone") == null); +} + +test "the array operators refuse a field that is not an array" { + var doc = try doc_with(testing.allocator, &.{.{ .key = "t", .value = .{ .int32 = 5 } }}); + defer doc.arena.deinit(); + try testing.expectError(error.NotAnArrayField, apply(&doc, &doc_of(&.{ + .{ .key = "$addToSet", .value = .{ .doc = &.{.{ .key = "t", .value = .{ .int32 = 1 } }} } }, + }), .{})); + try testing.expectError(error.NotAnArrayPathElement, apply(&doc, &doc_of(&.{ + .{ .key = "$pop", .value = .{ .doc = &.{.{ .key = "t", .value = .{ .int32 = 1 } }} } }, + }), .{})); + try testing.expectError(error.NotAnArrayField, apply(&doc, &doc_of(&.{ + .{ .key = "$pullAll", .value = .{ .doc = &.{.{ .key = "t", .value = .{ .array = &.{.{ .int32 = 1 }} } }} } }, + }), .{})); + try testing.expectError(error.PullAllNeedsArray, apply(&doc, &doc_of(&.{ + .{ .key = "$pullAll", .value = .{ .doc = &.{.{ .key = "t", .value = .{ .int32 = 1 } }} } }, + }), .{})); + try testing.expectEqual(@as(i32, 5), doc.get("t").?.int32); +} -- 2.39.5 From 482ddb3d1a4e718d508df96498266692ce16eac0 Mon Sep 17 00:00:00 2001 From: "A.Shakhmatov" Date: Mon, 10 Aug 2026 21:16:14 +0300 Subject: [PATCH 5/9] update: $push's modifiers are modifiers `$slice`, `$position` and `$sort` were parsed, accepted and dropped. `{$each: [3, 4], $slice: -3}` appended both values, sliced nothing and answered ok: 1 with modifiedCount: 1 -- the same class of wrong answer the positional operators were, and the reason `tests/spec/operators/` exists rather than a list of TODOs. Measured, and the order is the whole of it: insert at `$position`, then `$sort` the array *including* the new elements, then `$slice` the result. - `$slice: n` keeps the first n, `$slice: -n` the **last** n. That half is what a capped log depends on and is the one easy to write backwards; the test mutates exactly it. - `$position` counts back from the end when negative, and clamps at the front rather than wrapping. - `$sort: 1` orders whole elements in BSON order; `$sort: {a: 1}` orders on a field of them, and an element without it sorts as null -- the rank a missing field has everywhere else here. - **without `$each` there are no modifiers at all**: `{$push: {t: {$slice: 1}}}` pushes the document `{$slice: 1}` as a value. That is what makes `$each` the flag rather than a member of the set, and it is measured, not reasoned. An unknown `$`-prefixed key beside `$each`, or a `$slice`/`$position` that is not a number, is BadValue -- refused rather than ignored, which is the point. push-modifiers.json 6/21 -> 21/21. 240/240 unit tests. --- src/commands.zig | 5 + src/update.zig | 310 ++++++++++++++++++++++++++++++++++++++++++++--- 2 files changed, 297 insertions(+), 18 deletions(-) diff --git a/src/commands.zig b/src/commands.zig index 2146436..7907b13 100644 --- a/src/commands.zig +++ b/src/commands.zig @@ -4308,6 +4308,11 @@ fn update_refusal(reply: *wire.Reply, err: anyerror, diag: update.Diagnostic) !v "TypeMismatch", "The argument to $each in $addToSet must be an array", ) else return bad_value(reply, "The argument to $each in $push must be an array"), + error.BadPushModifier => return bad_value(reply, try std.fmt.allocPrint( + arena, + "Unrecognized or invalid $push modifier '{s}' at field '{s}'", + .{ diag.segment, diag.path }, + )), error.PathNotViable => return reply.put_error( @intFromEnum(ErrorCode.path_not_viable), "PathNotViable", diff --git a/src/update.zig b/src/update.zig index 6b3ddce..6fa5d02 100644 --- a/src/update.zig +++ b/src/update.zig @@ -34,6 +34,10 @@ pub const UpdateError = error{ PullAllNeedsArray, /// `$each` that is not an array. BadEach, + /// A `$push` modifier that is unknown, or whose operand is not what it + /// takes: `$slice`/`$position` want a number, `$sort` a number or a + /// one-field document. + BadPushModifier, /// A path segment that is not a field to create: a non-numeric name /// applied to an array, or any name applied to a scalar element a /// positional segment selected. mongod's `PathNotViable`. @@ -729,6 +733,62 @@ fn op_extremum( } } +/// `$push`'s modifiers, in the order mongod applies them. +/// +/// Absent `$each` there are no modifiers at all: `{$push: {t: {$slice: 1}}}` +/// pushes the document `{$slice: 1}` as a value. Measured, and it is what +/// makes `$each` the flag rather than a member of the set. +const PushModifiers = struct { + each: []const bson.Value, + /// Where the new elements go. Negative counts back from the end. + position: ?i64 = null, + /// Ascending/descending over whole elements, or a document naming a field + /// of them. + sort: ?bson.Value = null, + /// Keep the first n, or with a negative n the last -n. Applied last, and + /// on its own it truncates without adding anything. + slice: ?i64 = null, +}; + +fn parse_push_modifiers(spec: []const bson.Pair, opts: Options, path: []const u8) UpdateError!PushModifiers { + var m = PushModifiers{ .each = &.{} }; + for (spec) |p| { + if (std.mem.eql(u8, p.key, "$each")) { + m.each = switch (p.value) { + .array => |a| a, + else => { + note(opts.diag, path, "$push"); + return error.BadEach; + }, + }; + continue; + } + const numeric: ?i64 = if (p.value.is_number()) @intFromFloat(@trunc(p.value.as_f128())) else null; + if (std.mem.eql(u8, p.key, "$position")) { + m.position = numeric orelse return bad_modifier(opts, path, "$position"); + continue; + } + if (std.mem.eql(u8, p.key, "$slice")) { + m.slice = numeric orelse return bad_modifier(opts, path, "$slice"); + continue; + } + if (std.mem.eql(u8, p.key, "$sort")) { + m.sort = p.value; + continue; + } + // An unknown `$`-prefixed key beside `$each` is a typo, not a field of + // a document being pushed: the document is `$each`'s elements, not + // this one. + return bad_modifier(opts, path, p.key); + } + return m; +} + +fn bad_modifier(opts: Options, path: []const u8, name: []const u8) UpdateError { + note(opts.diag, path, name); + return error.BadPushModifier; +} + fn op_push( arena: std.mem.Allocator, pairs: *std.ArrayListUnmanaged(bson.Pair), @@ -736,34 +796,110 @@ fn op_push( opts: Options, ) UpdateError!void { for (ops) |p| { + const mods: ?PushModifiers = if (each_of(p.value) != null) + try parse_push_modifiers(p.value.doc, opts, p.key) + else + null; for (try resolve(arena, pairs.items, p.key, opts)) |segs| { - const current_opt = get_value(pairs.items, segs); var items: std.ArrayListUnmanaged(bson.Value) = .empty; defer items.deinit(arena); - if (current_opt) |current| { - switch (current) { - .array => |arr| try items.appendSlice(arena, arr), - .null => {}, - else => return error.InvalidUpdate, // non-array field + if (get_value(pairs.items, segs)) |current| switch (current) { + .array => |arr| try items.appendSlice(arena, arr), + .null => {}, + else => return error.InvalidUpdate, // non-array field + }; + if (mods) |m| { + try insert_each(arena, &items, m); + if (m.sort) |key| try sort_elements(items.items, key); + if (m.slice) |n| { + const kept = slice_range(items.items.len, n); + std.mem.copyForwards(bson.Value, items.items[0..kept.len], kept.of(items.items)); + items.shrinkRetainingCapacity(kept.len); } + } else { + try items.append(arena, try bson.copy_value(arena, p.value)); } - if (p.value == .doc) { - if (bson.get_pair(p.value.doc, "$each")) |each| { - const arr = switch (each) { - .array => |a| a, - else => return error.InvalidUpdate, - }; - for (arr) |item| try items.append(arena, try bson.copy_value(arena, item)); - try set_path(arena, pairs, segs, .{ .array = try items.toOwnedSlice(arena) }, p.key, opts.diag); - continue; - } - } - try items.append(arena, try bson.copy_value(arena, p.value)); try set_path(arena, pairs, segs, .{ .array = try items.toOwnedSlice(arena) }, p.key, opts.diag); } } } +fn insert_each( + arena: std.mem.Allocator, + items: *std.ArrayListUnmanaged(bson.Value), + m: PushModifiers, +) UpdateError!void { + const at: usize = if (m.position) |pos| blk: { + if (pos >= 0) break :blk @min(@as(usize, @intCast(pos)), items.items.len); + // Counted back from the end, and clamped at the front rather than + // wrapping: `$position: -99` on a two-element array is 0. + const back: usize = @intCast(-pos); + break :blk items.items.len -| back; + } else items.items.len; + for (m.each, 0..) |item, i| { + try items.insert(arena, at + i, try bson.copy_value(arena, item)); + } +} + +/// Which elements a `$slice` keeps. A non-negative `n` keeps the first `n`; a +/// negative one keeps the **last** `-n`, which is the shape a capped log uses +/// and the half that is easy to get backwards. +fn slice_range(len: usize, n: i64) struct { + start: usize, + len: usize, + + fn of(self: @This(), items: []bson.Value) []bson.Value { + return items[self.start .. self.start + self.len]; + } +} { + if (n >= 0) return .{ .start = 0, .len = @min(len, @as(usize, @intCast(n))) }; + const keep = @min(len, @as(usize, @intCast(-n))); + return .{ .start = len - keep, .len = keep }; +} + +/// `$sort: 1` orders whole elements; `$sort: {a: 1}` orders on a field of +/// them, which is only meaningful when they are documents -- an element that +/// is not one, or that lacks the field, sorts as null, the same rank a missing +/// field has everywhere else here. +const ElementSort = struct { + path: ?[]const u8, + descending: bool, + + fn key_of(self: ElementSort, v: bson.Value) bson.Value { + const path = self.path orelse return v; + const sub = switch (v) { + .doc => |d| d, + else => return .null, + }; + var segs: [max_path_segments][]const u8 = undefined; + const n = split_path(path, &segs) orelse return .null; + return get_value(sub, segs[0..n]) orelse .null; + } + + fn less(self: ElementSort, a: bson.Value, b: bson.Value) bool { + const order = bson.compare(self.key_of(a), self.key_of(b)); + return if (self.descending) order == .gt else order == .lt; + } +}; + +fn sort_elements(items: []bson.Value, key: bson.Value) UpdateError!void { + const spec: ElementSort = switch (key) { + .doc => |pairs| blk: { + if (pairs.len != 1) return error.BadPushModifier; + break :blk .{ .path = pairs[0].key, .descending = is_descending(pairs[0].value) }; + }, + else => blk: { + if (!key.is_number()) return error.BadPushModifier; + break :blk .{ .path = null, .descending = is_descending(key) }; + }, + }; + std.mem.sort(bson.Value, items, spec, ElementSort.less); +} + +fn is_descending(v: bson.Value) bool { + return v.is_number() and v.as_f128() < 0; +} + fn op_pull( arena: std.mem.Allocator, pairs: *std.ArrayListUnmanaged(bson.Pair), @@ -2150,3 +2286,141 @@ test "the array operators refuse a field that is not an array" { }), .{})); try testing.expectEqual(@as(i32, 5), doc.get("t").?.int32); } + +test "$push applies its modifiers in mongod's order: position, sort, slice" { + // The order is the whole of it. `$sort` runs over the array *after* the + // new elements are in it, and `$slice` runs last, over the sorted result. + var doc = try doc_with(testing.allocator, &.{ + .{ .key = "t", .value = .{ .array = &.{ .{ .int32 = 3 }, .{ .int32 = 1 } } } }, + }); + defer doc.arena.deinit(); + try apply(&doc, &doc_of(&.{ + .{ .key = "$push", .value = .{ .doc = &.{.{ .key = "t", .value = .{ .doc = &.{ + .{ .key = "$each", .value = .{ .array = &.{.{ .int32 = 2 }} } }, + .{ .key = "$sort", .value = .{ .int32 = 1 } }, + .{ .key = "$slice", .value = .{ .int32 = 2 } }, + } } }} } }, + }), .{}); + const t = doc.get("t").?.array; + try testing.expectEqual(@as(usize, 2), t.len); + try testing.expectEqual(@as(i32, 1), t[0].int32); + try testing.expectEqual(@as(i32, 2), t[1].int32); +} + +test "$slice keeps the first n, or with a negative n the last" { + // Mutation check: keep the first `-n` elements for a negative slice and + // the second half goes red. It is the half a capped log depends on. + const cases = [_]struct { n: i32, want: [3]i32, len: usize }{ + .{ .n = 3, .want = .{ 1, 2, 3 }, .len = 3 }, + .{ .n = -3, .want = .{ 2, 3, 4 }, .len = 3 }, + .{ .n = 0, .want = .{ 0, 0, 0 }, .len = 0 }, + .{ .n = 10, .want = .{ 1, 2, 3 }, .len = 4 }, + }; + for (cases) |c| { + var doc = try doc_with(testing.allocator, &.{ + .{ .key = "t", .value = .{ .array = &.{ .{ .int32 = 1 }, .{ .int32 = 2 } } } }, + }); + defer doc.arena.deinit(); + const spec = [2]bson.Pair{ + .{ .key = "$each", .value = .{ .array = &.{ .{ .int32 = 3 }, .{ .int32 = 4 } } } }, + .{ .key = "$slice", .value = .{ .int32 = c.n } }, + }; + const arg = [1]bson.Pair{.{ .key = "t", .value = .{ .doc = &spec } }}; + try apply(&doc, &doc_of(&.{.{ .key = "$push", .value = .{ .doc = &arg } }}), .{}); + const t = doc.get("t").?.array; + try testing.expectEqual(c.len, t.len); + for (t, 0..) |v, i| { + if (i < 3) try testing.expectEqual(c.want[i], v.int32); + } + } +} + +test "$position inserts where it says, counting back from the end when negative" { + const cases = [_]struct { pos: i32, want: [3]i32 }{ + .{ .pos = 0, .want = .{ 9, 1, 2 } }, + .{ .pos = 1, .want = .{ 1, 9, 2 } }, + .{ .pos = 99, .want = .{ 1, 2, 9 } }, + .{ .pos = -1, .want = .{ 1, 9, 2 } }, + .{ .pos = -99, .want = .{ 9, 1, 2 } }, // clamped at the front, not wrapped + }; + for (cases) |c| { + var doc = try doc_with(testing.allocator, &.{ + .{ .key = "t", .value = .{ .array = &.{ .{ .int32 = 1 }, .{ .int32 = 2 } } } }, + }); + defer doc.arena.deinit(); + const spec = [2]bson.Pair{ + .{ .key = "$each", .value = .{ .array = &.{.{ .int32 = 9 }} } }, + .{ .key = "$position", .value = .{ .int32 = c.pos } }, + }; + const arg = [1]bson.Pair{.{ .key = "t", .value = .{ .doc = &spec } }}; + try apply(&doc, &doc_of(&.{.{ .key = "$push", .value = .{ .doc = &arg } }}), .{}); + const t = doc.get("t").?.array; + try testing.expectEqual(@as(usize, 3), t.len); + for (t, 0..) |v, i| try testing.expectEqual(c.want[i], v.int32); + } +} + +test "$sort orders on a field of the elements" { + var doc = try doc_with(testing.allocator, &.{ + .{ .key = "t", .value = .{ .array = &.{ + .{ .doc = &.{.{ .key = "a", .value = .{ .int32 = 3 } }} }, + .{ .doc = &.{.{ .key = "a", .value = .{ .int32 = 1 } }} }, + } } }, + }); + defer doc.arena.deinit(); + try apply(&doc, &doc_of(&.{ + .{ .key = "$push", .value = .{ .doc = &.{.{ .key = "t", .value = .{ .doc = &.{ + .{ .key = "$each", .value = .{ .array = &.{ + .{ .doc = &.{.{ .key = "a", .value = .{ .int32 = 2 } }} }, + } } }, + .{ .key = "$sort", .value = .{ .doc = &.{.{ .key = "a", .value = .{ .int32 = 1 } }} } }, + } } }} } }, + }), .{}); + const t = doc.get("t").?.array; + try testing.expectEqual(@as(usize, 3), t.len); + for (t, 1..) |v, want| try testing.expectEqual(@as(i32, @intCast(want)), v.doc[0].value.int32); +} + +test "without $each there are no modifiers, only a value" { + // The measured rule that makes `$each` the flag: `{$slice: 1}` on its own + // is a document to push, not an instruction. Mutation check: treat any + // `$`-prefixed key as a modifier and this stores nothing where mongod + // stores a document. + var doc = try doc_with(testing.allocator, &.{ + .{ .key = "t", .value = .{ .array = &.{.{ .int32 = 1 }} } }, + }); + defer doc.arena.deinit(); + try apply(&doc, &doc_of(&.{ + .{ .key = "$push", .value = .{ .doc = &.{.{ .key = "t", .value = .{ .doc = &.{ + .{ .key = "$slice", .value = .{ .int32 = 1 } }, + } } }} } }, + }), .{}); + const t = doc.get("t").?.array; + try testing.expectEqual(@as(usize, 2), t.len); + try testing.expect(t[1] == .doc); +} + +test "a $push modifier that is not one, or is handed the wrong thing, is refused" { + var doc = try doc_with(testing.allocator, &.{ + .{ .key = "t", .value = .{ .array = &.{.{ .int32 = 1 }} } }, + }); + defer doc.arena.deinit(); + const bad = [_]bson.Pair{ + .{ .key = "$bogus", .value = .{ .int32 = 1 } }, + .{ .key = "$slice", .value = .{ .string = "x" } }, + .{ .key = "$position", .value = .{ .string = "x" } }, + }; + for (bad) |m| { + const spec = [2]bson.Pair{ + .{ .key = "$each", .value = .{ .array = &.{.{ .int32 = 3 }} } }, + m, + }; + const arg = [1]bson.Pair{.{ .key = "t", .value = .{ .doc = &spec } }}; + try testing.expectError(error.BadPushModifier, apply( + &doc, + &doc_of(&.{.{ .key = "$push", .value = .{ .doc = &arg } }}), + .{}, + )); + } + try testing.expectEqual(@as(usize, 1), doc.get("t").?.array.len); +} -- 2.39.5 From 1b6753fceca41fe5d2644f47b36772d429853cc3 Mon Sep 17 00:00:00 2001 From: "A.Shakhmatov" Date: Mon, 10 Aug 2026 21:21:32 +0300 Subject: [PATCH 6/9] update: $setOnInsert and $currentDate `$setOnInsert` is `$set` on the branch that inserts and nothing at all on the branch that updates, so `Options` grows the one bit that says which -- and `build_upsert_doc` is the only caller that sets it. It does not go through `op_set` because of the measured exception: writing `_id` is allowed on a document being built and refused on one being rewritten. A document nobody has yet has no identity to change. `$currentDate` reads a clock, and the clock is a parameter. There is no fallback to a global one: this file has no `io` to reach the real clock through, and the handlers pass the same `std.Io.Timestamp` that `ttl_sweep` and the cursor sweep already read. A caller that forgets gets the epoch -- deterministic and obviously wrong -- rather than something that changes between runs. Measured: `$currentDate: {d: false}` writes a date. The boolean says "a date", not "whether", and either value means the same thing. `{$type: "timestamp"}` writes a BSON timestamp, seconds in the high 32 bits and an ordinal in the low ones; mongod fills the ordinal from the oplog, a standalone has none, so it is 1. current-date.json 3/11 -> 11/11, set-on-insert.json 0/9 -> 7/9. The two left are not `$setOnInsert`'s: one is ConflictingUpdateOperators, and the other is a pre-existing bug the corpus found -- an upsert never reports the `_id` it generated, because `Engine.insert` writes it into the bytes and not back into the caller's tree. `updateOne(..., {upsert: true}).upsertedId` is null here and an ObjectId on mongod. Next commit. 244/244 unit tests. --- src/commands.zig | 19 +++++- src/update.zig | 170 +++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 188 insertions(+), 1 deletion(-) diff --git a/src/commands.zig b/src/commands.zig index 7907b13..dfa6e6d 100644 --- a/src/commands.zig +++ b/src/commands.zig @@ -1977,6 +1977,7 @@ fn cmd_update(ctx: *Context, msg: *wire.Message, reply: *wire.Reply) !void { "update.updates.arrayFilters", ) orelse return, .query = q, + .now_ms = std.Io.Timestamp.now(ctx.io, .real).toMilliseconds(), .diag = &diag, }; // Before the scan, not after: an update naming an identifier nothing @@ -2102,6 +2103,7 @@ fn cmd_find_and_modify(ctx: *Context, msg: *wire.Message, reply: *wire.Reply) !v "findAndModify.arrayFilters", ) orelse return, .query = q, + .now_ms = std.Io.Timestamp.now(ctx.io, .real).toMilliseconds(), .diag = &diag, }; if (doc_arg(msg.body.get("update"))) |u| { @@ -4313,6 +4315,17 @@ fn update_refusal(reply: *wire.Reply, err: anyerror, diag: update.Diagnostic) !v "Unrecognized or invalid $push modifier '{s}' at field '{s}'", .{ diag.segment, diag.path }, )), + error.BadCurrentDateType => return bad_value( + reply, + "The '$type' string field is required to be 'date' or 'timestamp': " ++ + "{$currentDate: {field : {$type: 'date'}}}", + ), + error.BadCurrentDateOperand => return bad_value(reply, try std.fmt.allocPrint( + arena, + "{s} is not valid type for $currentDate. Please use a boolean ('true') or a " ++ + "$type expression ({{$type: 'timestamp/date'}}).", + .{diag.other}, + )), error.PathNotViable => return reply.put_error( @intFromEnum(ErrorCode.path_not_viable), "PathNotViable", @@ -4410,7 +4423,11 @@ fn build_upsert_doc( const owned = try arena.create(bson.Document); owned.* = bson.Document{ .arena = std.heap.ArenaAllocator.init(arena), .pairs = try pairs.toOwnedSlice(arena) }; // Apply update operators to build the final doc; _id handled by insert. - try update.apply(owned, &.{ .arena = undefined, .pairs = u_doc }, opts); + // `inserting` is what `$setOnInsert` asks about, and this is the only + // caller that answers yes. + var insert_opts = opts; + insert_opts.inserting = true; + try update.apply(owned, &.{ .arena = undefined, .pairs = u_doc }, insert_opts); return owned; } diff --git a/src/update.zig b/src/update.zig index 6fa5d02..a6d804f 100644 --- a/src/update.zig +++ b/src/update.zig @@ -34,6 +34,10 @@ pub const UpdateError = error{ PullAllNeedsArray, /// `$each` that is not an array. BadEach, + /// `$currentDate` handed a document that is not `{$type: date|timestamp}`. + BadCurrentDateType, + /// `$currentDate` handed something that is neither a bool nor a document. + BadCurrentDateOperand, /// A `$push` modifier that is unknown, or whose operand is not what it /// takes: `$slice`/`$position` want a number, `$sort` a number or a /// one-field document. @@ -115,6 +119,15 @@ pub const Options = struct { /// The command's query. `$` resolves against it, so an update carrying one /// without this is refused rather than guessing at an element. query: ?[]const bson.Pair = null, + /// Whether this application is building a document an upsert will insert. + /// `$setOnInsert` is the one operator that asks. + inserting: bool = false, + /// The clock `$currentDate` reads, in Unix milliseconds. There is no + /// fallback to a global one: this file has no `io` to read the real clock + /// through, and a caller that forgets gets the epoch rather than a value + /// that changes between runs. The command handlers pass the same clock + /// `ttl_sweep` and the cursor sweep already use. + now_ms: i64 = 0, diag: ?*Diagnostic = null, }; @@ -636,6 +649,15 @@ fn apply_operator( // shape is checked once here rather than at the top of each. const ops = doc_pairs(value) orelse return error.InvalidUpdate; if (std.mem.eql(u8, op, "$set")) return op_set(arena, pairs, ops, opts); + // `$setOnInsert` is `$set` on the branch that inserts and nothing at all + // on the branch that updates -- including its `_id` rule, which is why it + // does not go through `op_set`: writing `_id` is allowed on a document + // being built and refused on one being rewritten. Measured. + if (std.mem.eql(u8, op, "$setOnInsert")) { + if (!opts.inserting) return; + return op_set_on_insert(arena, pairs, ops, opts); + } + if (std.mem.eql(u8, op, "$currentDate")) return op_current_date(arena, pairs, ops, opts); if (std.mem.eql(u8, op, "$unset")) return op_unset(arena, pairs, ops, opts); if (std.mem.eql(u8, op, "$inc")) return op_arith(arena, pairs, ops, opts, .add); if (std.mem.eql(u8, op, "$mul")) return op_arith(arena, pairs, ops, opts, .mul); @@ -664,6 +686,72 @@ fn op_set( } } +fn op_set_on_insert( + arena: std.mem.Allocator, + pairs: *std.ArrayListUnmanaged(bson.Pair), + ops: []const bson.Pair, + opts: Options, +) UpdateError!void { + for (ops) |p| { + for (try resolve(arena, pairs.items, p.key, opts)) |segs| { + try set_path(arena, pairs, segs, try bson.copy_value(arena, p.value), p.key, opts.diag); + } + } +} + +/// `$currentDate`: write the clock, as a date or as a BSON timestamp. +/// +/// The operand says which. A bool -- **either** bool, measured: `false` writes +/// a date too, the value is ignored -- means a date; `{$type: "date"}` and +/// `{$type: "timestamp"}` say so; anything else is refused. +fn op_current_date( + arena: std.mem.Allocator, + pairs: *std.ArrayListUnmanaged(bson.Pair), + ops: []const bson.Pair, + opts: Options, +) UpdateError!void { + const now = opts.now_ms; + for (ops) |p| { + const value: bson.Value = switch (try current_date_kind(p.value, opts, p.key)) { + .date => .{ .datetime = now }, + // A BSON timestamp is seconds in the high 32 bits and an ordinal + // in the low ones. mongod fills the ordinal from the oplog; a + // standalone has no oplog, so it is 1 -- the same answer twice in + // one second is the same timestamp, which nothing here reads. + .timestamp => .{ .timestamp = (@as(u64, @intCast(@divFloor(now, 1000))) << 32) | 1 }, + }; + for (try resolve(arena, pairs.items, p.key, opts)) |segs| { + try set_path(arena, pairs, segs, value, p.key, opts.diag); + } + } +} + +fn current_date_kind( + v: bson.Value, + opts: Options, + path: []const u8, +) UpdateError!enum { date, timestamp } { + switch (v) { + .bool => return .date, + .doc => |spec| { + const t = bson.get_pair(spec, "$type") orelse { + note(opts.diag, path, ""); + return error.BadCurrentDateType; + }; + if (t == .string) { + if (std.mem.eql(u8, t.string, "date")) return .date; + if (std.mem.eql(u8, t.string, "timestamp")) return .timestamp; + } + note(opts.diag, path, ""); + return error.BadCurrentDateType; + }, + else => { + if (opts.diag) |d| d.* = .{ .path = path, .other = v.type_name() }; + return error.BadCurrentDateOperand; + }, + } +} + fn op_unset( arena: std.mem.Allocator, pairs: *std.ArrayListUnmanaged(bson.Pair), @@ -2424,3 +2512,85 @@ test "a $push modifier that is not one, or is handed the wrong thing, is refused } try testing.expectEqual(@as(usize, 1), doc.get("t").?.array.len); } + +test "$setOnInsert writes only on the branch that inserts" { + // Mutation check: drop the `opts.inserting` guard and the first half goes + // red -- an update would gain a field that is only supposed to exist on a + // document nobody had before. + var updating = try doc_with(testing.allocator, &.{ + .{ .key = "_id", .value = .{ .int32 = 1 } }, + .{ .key = "a", .value = .{ .int32 = 100 } }, + }); + defer updating.arena.deinit(); + try apply(&updating, &doc_of(&.{ + .{ .key = "$setOnInsert", .value = .{ .doc = &.{.{ .key = "a", .value = .{ .int32 = 1 } }} } }, + .{ .key = "$set", .value = .{ .doc = &.{.{ .key = "b", .value = .{ .int32 = 2 } }} } }, + }), .{}); + try testing.expectEqual(@as(i32, 100), updating.get("a").?.int32); + try testing.expectEqual(@as(i32, 2), updating.get("b").?.int32); + + var inserting = try doc_with(testing.allocator, &.{.{ .key = "k", .value = .{ .int32 = 1 } }}); + defer inserting.arena.deinit(); + try apply(&inserting, &doc_of(&.{ + .{ .key = "$setOnInsert", .value = .{ .doc = &.{.{ .key = "a", .value = .{ .int32 = 1 } }} } }, + }), .{ .inserting = true }); + try testing.expectEqual(@as(i32, 1), inserting.get("a").?.int32); +} + +test "$setOnInsert may write _id, where $set may not" { + // The one place the immutability rule does not apply: a document being + // built has no identity yet to change. + var doc = try doc_with(testing.allocator, &.{.{ .key = "k", .value = .{ .int32 = 1 } }}); + defer doc.arena.deinit(); + try apply(&doc, &doc_of(&.{ + .{ .key = "$setOnInsert", .value = .{ .doc = &.{.{ .key = "_id", .value = .{ .int32 = 9 } }} } }, + }), .{ .inserting = true }); + try testing.expectEqual(@as(i32, 9), doc.get("_id").?.int32); + + try testing.expectError(error.ImmutableId, apply(&doc, &doc_of(&.{ + .{ .key = "$set", .value = .{ .doc = &.{.{ .key = "_id", .value = .{ .int32 = 8 } }} } }, + }), .{ .inserting = true })); +} + +test "$currentDate writes the clock it was handed" { + // The clock is a parameter, so the result is a value rather than a moving + // target -- and the server passes the same one `ttl_sweep` reads. + var doc = try doc_with(testing.allocator, &.{.{ .key = "a", .value = .{ .int32 = 1 } }}); + defer doc.arena.deinit(); + try apply(&doc, &doc_of(&.{ + .{ .key = "$currentDate", .value = .{ .doc = &.{ + .{ .key = "d", .value = .{ .bool = true } }, + // Measured: `false` writes a date too. The boolean says "a date", + // not "whether". + .{ .key = "f", .value = .{ .bool = false } }, + .{ .key = "e", .value = .{ .doc = &.{.{ .key = "$type", .value = .{ .string = "date" } }} } }, + .{ .key = "t", .value = .{ .doc = &.{.{ .key = "$type", .value = .{ .string = "timestamp" } }} } }, + } } }, + }), .{ .now_ms = 1_700_000_000_123 }); + try testing.expectEqual(@as(i64, 1_700_000_000_123), doc.get("d").?.datetime); + try testing.expectEqual(@as(i64, 1_700_000_000_123), doc.get("f").?.datetime); + try testing.expectEqual(@as(i64, 1_700_000_000_123), doc.get("e").?.datetime); + // Seconds in the high 32 bits, an ordinal in the low ones. + try testing.expectEqual(@as(u64, (1_700_000_000 << 32) | 1), doc.get("t").?.timestamp); +} + +test "$currentDate refuses an operand that names no type" { + var doc = try doc_with(testing.allocator, &.{.{ .key = "a", .value = .{ .int32 = 1 } }}); + defer doc.arena.deinit(); + var diag: Diagnostic = .{}; + try testing.expectError(error.BadCurrentDateType, apply(&doc, &doc_of(&.{ + .{ .key = "$currentDate", .value = .{ .doc = &.{ + .{ .key = "d", .value = .{ .doc = &.{.{ .key = "$type", .value = .{ .string = "nope" } }} } }, + } } }, + }), .{ .diag = &diag })); + try testing.expectError(error.BadCurrentDateType, apply(&doc, &doc_of(&.{ + .{ .key = "$currentDate", .value = .{ .doc = &.{ + .{ .key = "d", .value = .{ .doc = &.{.{ .key = "a", .value = .{ .int32 = 1 } }} } }, + } } }, + }), .{ .diag = &diag })); + try testing.expectError(error.BadCurrentDateOperand, apply(&doc, &doc_of(&.{ + .{ .key = "$currentDate", .value = .{ .doc = &.{.{ .key = "d", .value = .{ .int32 = 1 } }} } }, + }), .{ .diag = &diag })); + try testing.expectEqualStrings("int", diag.other); + try testing.expect(doc.get("d") == null); +} -- 2.39.5 From c9575eec0a8f316513f988441ccfc752f4c3390e Mon Sep 17 00:00:00 2001 From: "A.Shakhmatov" Date: Mon, 10 Aug 2026 21:25:12 +0300 Subject: [PATCH 7/9] commands: an upsert reports the _id it generated `Engine.insert` generates an `_id` into the bytes it writes and leaves the caller's tree without one, so every handler that looks afterwards found nothing: `update`'s `upserted` array carried `_id: null`, which the driver surfaces as `upsertedId: null`, and `findAndModify` with `returnDocument: after` returned an upserted document with no `_id` at all. The client has never seen this document. The `_id` in the reply is the only way it can name it again, so both answers were wrong in the way that matters. Settled in `build_upsert_doc` rather than in the storage engine: the upsert path is what decides the identity of the document it builds, and it has to be after the operators run because `$setOnInsert` may supply the `_id` itself. At the front of the document, where MongoDB stores it and where the `_id_` index descends on it. Found by `tests/spec/operators/`, which is the first corpus here to upsert into an empty collection and then look at what came back. The pinned crud corpus does not move -- its upsert cases match `upsertedId` loosely. set-on-insert.json 7/9 -> 8/9. 245/245 unit tests. --- src/commands.zig | 77 ++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 75 insertions(+), 2 deletions(-) diff --git a/src/commands.zig b/src/commands.zig index dfa6e6d..b0c1c1c 100644 --- a/src/commands.zig +++ b/src/commands.zig @@ -1992,7 +1992,7 @@ fn cmd_update(ctx: *Context, msg: *wire.Message, reply: *wire.Reply) !void { if (matched.items.len == 0) { if (upsert) { - const new_doc = build_upsert_doc(reply, q, u_doc, opts) catch |err| + const new_doc = build_upsert_doc(reply, ctx, q, u_doc, opts) catch |err| return update_refusal(reply, err, diag); ctx.engine.insert(db_name, coll_name, new_doc, ctx.oid_gen) catch |err| switch (err) { error.DuplicateKey, error.DuplicateKeyIndex => return duplicate_key_error(ctx, reply, db_name, coll_name, new_doc), @@ -2134,7 +2134,7 @@ fn cmd_find_and_modify(ctx: *Context, msg: *wire.Message, reply: *wire.Reply) !v if (target == null and do_update and upsert) { const u_doc = doc_arg(msg.body.get("update")) orelse return bad_value(reply, "update must be a document"); - const new_doc = build_upsert_doc(reply, q, u_doc, opts) catch |err| + const new_doc = build_upsert_doc(reply, ctx, q, u_doc, opts) catch |err| return update_refusal(reply, err, diag); ctx.engine.insert(db_name, coll_name, new_doc, ctx.oid_gen) catch |err| switch (err) { error.DuplicateKey, error.DuplicateKeyIndex => return duplicate_key_error(ctx, reply, db_name, coll_name, new_doc), @@ -4404,8 +4404,17 @@ fn wrong_type_dynamic( /// Build the document for an upsert: equality fields from the filter, then /// the update operators applied. Owned by the reply arena. +/// +/// The `_id` is settled *here* rather than in the storage engine. `insert` +/// generates one into the bytes it writes and leaves the caller's tree without +/// it, so `updateOne(..., {upsert: true}).upsertedId` came back null and +/// `findOneAndUpdate` with `returnDocument: after` returned a document missing +/// its `_id`. Both are what the client is told about a document it has never +/// seen, and both were wrong. Found by `tests/spec/operators/`, which is the +/// first corpus here to upsert into an empty collection and then look. fn build_upsert_doc( reply: *wire.Reply, + ctx: *Context, q: []const bson.Pair, u_doc: []const bson.Pair, opts: update.Options, @@ -4428,6 +4437,16 @@ fn build_upsert_doc( var insert_opts = opts; insert_opts.inserting = true; try update.apply(owned, &.{ .arena = undefined, .pairs = u_doc }, insert_opts); + // After the operators, because `$setOnInsert` may supply the `_id` itself + // and a generated one would then be the wrong answer. At the front, + // because that is where MongoDB stores it and where the `_id_` index + // descends on it. + if (owned.get("_id") == null) { + const with_id = try arena.alloc(bson.Pair, owned.pairs.len + 1); + with_id[0] = .{ .key = "_id", .value = .{ .object_id = ctx.oid_gen.new(ctx.io) } }; + @memcpy(with_id[1..], owned.pairs); + owned.pairs = with_id; + } return owned; } @@ -7058,3 +7077,57 @@ test "a rebuild kills an offsets cursor and spares a streaming one" { try testing.expectEqual(@as(i64, 0), id); try testing.expectEqual(@as(u32, 60), seen); } + +test "an upsert reports the _id it generated" { + // The client has never seen this document, so the `_id` in the reply is + // the only way it can name it again. `Engine.insert` generates one into + // the bytes it writes and leaves the caller's tree without it, so both + // `upserted` here and `findAndModify`'s returned document used to come + // back without an `_id` at all -- `upsertedId: null` on the driver. + // + // Mutation check: delete the `_id` block in `build_upsert_doc` and both + // halves go red. + var threaded: std.Io.Threaded = .init_single_threaded; + defer threaded.deinit(); + const io = threaded.io(); + var tdb = try TestDb.init(io); + defer tdb.deinit(); + var ctx = tdb.ctx(io); + + const updates = [_]bson.Value{.{ .doc = &.{ + .{ .key = "q", .value = .{ .doc = &.{.{ .key = "k", .value = .{ .int32 = 1 } }} } }, + .{ .key = "u", .value = .{ .doc = &.{ + .{ .key = "$set", .value = .{ .doc = &.{.{ .key = "a", .value = .{ .int32 = 1 } }} } }, + } } }, + .{ .key = "upsert", .value = .{ .bool = true } }, + } }}; + var reply = wire.Reply.init(testing.allocator); + defer reply.deinit(); + var msg = try parse_fake_msg("update", .{ .string = "up" }, &.{ + .{ .key = "updates", .value = .{ .array = &updates } }, + }); + defer msg.deinit(); + try dispatch(&ctx, &msg, &reply); + const upserted = bson.get_pair(reply.pairs.items, "upserted").?.array; + try testing.expectEqual(@as(usize, 1), upserted.len); + try testing.expect(bson.get_pair(upserted[0].doc, "_id").? == .object_id); + + // And an `_id` the update supplied itself is the one that is used, rather + // than being generated over. + const with_id = [_]bson.Value{.{ .doc = &.{ + .{ .key = "q", .value = .{ .doc = &.{.{ .key = "k", .value = .{ .int32 = 2 } }} } }, + .{ .key = "u", .value = .{ .doc = &.{ + .{ .key = "$setOnInsert", .value = .{ .doc = &.{.{ .key = "_id", .value = .{ .int32 = 9 } }} } }, + } } }, + .{ .key = "upsert", .value = .{ .bool = true } }, + } }}; + var reply2 = wire.Reply.init(testing.allocator); + defer reply2.deinit(); + var msg2 = try parse_fake_msg("update", .{ .string = "up" }, &.{ + .{ .key = "updates", .value = .{ .array = &with_id } }, + }); + defer msg2.deinit(); + try dispatch(&ctx, &msg2, &reply2); + const upserted2 = bson.get_pair(reply2.pairs.items, "upserted").?.array; + try testing.expectEqual(@as(i32, 9), bson.get_pair(upserted2[0].doc, "_id").?.int32); +} -- 2.39.5 From 71e3879ae01a0fd591bde0ecef6741788725bdac Mon Sep 17 00:00:00 2001 From: "A.Shakhmatov" Date: Mon, 10 Aug 2026 21:30:35 +0300 Subject: [PATCH 8/9] update: an unknown modifier, and two paths that decide one field Two refusals this server did not have, both measured. **ConflictingUpdateOperators (40).** Two paths in one update where either is a prefix of the other at a segment boundary leave the result depending on which operator ran first, so mongod refuses rather than picking an order: `{$set: {a: 2}, $inc: {a: 1}}`, `{$min: {a: 2}, $max: {a: 9}}`, `{$set: {a: 2, "a.b": 3}}`, and `$rename` counts both its ends. Siblings are fine, and the segment boundary is what makes them fine -- `a.b` and `a.bb` share a string prefix and decide nothing about each other, which is the mutation the test names. The check is quadratic in the number of paths and bounded at 64, so it stays on the stack. Past the bound it stops checking rather than refusing: missing a conflict in a 65-field update is a better answer than refusing a legal one, and no driver writes updates that wide. **Unknown modifier (9).** A name not in the operator table was `InvalidUpdate` -> BadValue; mongod answers FailedToParse and names it. What matters is the option nobody took: skipping an unknown operator would make a typo an update the client believes ran. A known operator handed something that is not a document of fields is the same code. One existing test changed answer and was wrong: it packed `$set: {new: 5}` and `$rename: {new: "renamed"}` into one update and passed only because there was no conflict check. mongod refuses that with 40 -- measured, then the test split into two updates. operators corpus 95/102 -> **102/102**. The pinned crud corpus, the positional corpus and the aggregation corpus are unmoved at 218/73, 51/0 and 70/0. 247/247 unit tests. --- src/commands.zig | 24 +++++ src/update.zig | 267 ++++++++++++++++++++++++++++++++++++++++------- 2 files changed, 256 insertions(+), 35 deletions(-) diff --git a/src/commands.zig b/src/commands.zig index b0c1c1c..5dc74cc 100644 --- a/src/commands.zig +++ b/src/commands.zig @@ -66,6 +66,10 @@ pub const ErrorCode = enum(i32) { duplicate_key = 11000, namespace_exists = 48, failed_to_parse = 9, + /// `ConflictingUpdateOperators`, measured on mongod 8.3.7: two paths in + /// one update where either is a prefix of the other, so which of them + /// decides the result would depend on the order the operators ran in. + conflicting_update_operators = 40, internal_error = 1, /// "Unrecognized pipeline stage name". A `Location` code, so mongod names it /// `Location40324` rather than after any symbol. @@ -4326,6 +4330,26 @@ fn update_refusal(reply: *wire.Reply, err: anyerror, diag: update.Diagnostic) !v "$type expression ({{$type: 'timestamp/date'}}).", .{diag.other}, )), + error.UnknownModifier => return failed_to_parse(reply, try std.fmt.allocPrint( + arena, + "Unknown modifier: {s}. Expected a valid update modifier or pipeline-style " ++ + "update specified as an array", + .{diag.segment}, + )), + error.ModifierNeedsFields => return failed_to_parse(reply, try std.fmt.allocPrint( + arena, + "Modifiers operate on fields but we found type {s} instead", + .{diag.segment}, + )), + error.ConflictingUpdate => return reply.put_error( + @intFromEnum(ErrorCode.conflicting_update_operators), + "ConflictingUpdateOperators", + try std.fmt.allocPrint( + arena, + "Updating the path '{s}' would create a conflict at '{s}'", + .{ diag.path, diag.segment }, + ), + ), error.PathNotViable => return reply.put_error( @intFromEnum(ErrorCode.path_not_viable), "PathNotViable", diff --git a/src/update.zig b/src/update.zig index a6d804f..abdb2ba 100644 --- a/src/update.zig +++ b/src/update.zig @@ -34,6 +34,14 @@ pub const UpdateError = error{ PullAllNeedsArray, /// `$each` that is not an array. BadEach, + /// A name that is not in the operator table. mongod's `FailedToParse` (9), + /// and a refusal rather than a silent skip. + UnknownModifier, + /// A known operator whose argument is not a document of fields. + ModifierNeedsFields, + /// Two paths in one update where either is a prefix of the other: + /// mongod's `ConflictingUpdateOperators` (40). + ConflictingUpdate, /// `$currentDate` handed a document that is not `{$type: date|timestamp}`. BadCurrentDateType, /// `$currentDate` handed something that is neither a bool nor a document. @@ -235,10 +243,20 @@ pub fn validate(update: []const bson.Pair, opts: Options) UpdateError!void { // single case out of seventeen where this server already agreed with it. if (is_replacement(update)) return; try bind_array_filters(opts); + var written: WrittenPaths = .{}; for (update) |op| { - const ops = doc_pairs(op.value) orelse continue; + if (!is_known_operator(op.key)) { + note(opts.diag, "", op.key); + return error.UnknownModifier; + } + const ops = doc_pairs(op.value) orelse { + note(opts.diag, "", op.value.type_name()); + return error.ModifierNeedsFields; + }; const rename = std.mem.eql(u8, op.key, "$rename"); for (ops) |p| { + try note_written(&written, p.key, opts); + if (rename and p.value == .string) try note_written(&written, p.value.string, opts); if (rename) { if (has_positional(p.key)) { note(opts.diag, p.key, ""); @@ -260,6 +278,76 @@ pub fn validate(update: []const bson.Pair, opts: Options) UpdateError!void { } } +/// The operator table. A name not in it is a typo, and a typo that is quietly +/// ignored is an update the client believes happened -- so an unknown modifier +/// is refused rather than skipped, which is what mongod does and what this +/// server did not. +const known_operators = [_][]const u8{ + "$set", "$setOnInsert", "$unset", "$inc", "$mul", + "$min", "$max", "$push", "$pull", "$addToSet", + "$pop", "$pullAll", "$rename", "$currentDate", +}; + +fn is_known_operator(name: []const u8) bool { + for (known_operators) |k| if (std.mem.eql(u8, name, k)) return true; + return false; +} + +/// How many distinct paths one update may write. Well past anything a driver +/// sends, and the bound is what keeps the conflict check on the stack: it is +/// quadratic in the number of paths, which is fine at this size and would not +/// be at an unbounded one. +const max_written_paths = 64; + +/// Record a path this update writes, refusing a second one that collides. +/// +/// Two paths collide when either is a prefix of the other at a segment +/// boundary: `a` and `a` obviously, and `a` and `a.b` because writing the +/// parent decides what the child is. `a.b` and `a.c` are siblings and fine. +/// mongod calls this ConflictingUpdateOperators and it applies across +/// operators and within one: `{$set: {a: 2}, $inc: {a: 1}}` and +/// `{$set: {a: 2, "a.b": 3}}` are both refused. +const WrittenPaths = struct { + items: [max_written_paths][]const u8 = undefined, + n: usize = 0, + + fn slice(self: *const WrittenPaths) []const []const u8 { + return self.items[0..self.n]; + } + + fn append(self: *WrittenPaths, path: []const u8) void { + // Past the bound the check stops rather than the update: refusing a + // legal update because it names 65 fields would be a worse answer than + // missing a conflict in one, and no driver writes updates that wide. + if (self.n == self.items.len) return; + self.items[self.n] = path; + self.n += 1; + } +}; + +fn note_written( + written: *WrittenPaths, + path: []const u8, + opts: Options, +) UpdateError!void { + for (written.slice()) |seen| { + const common = prefix_of(seen, path) orelse continue; + if (opts.diag) |d| d.* = .{ .path = path, .segment = common }; + return error.ConflictingUpdate; + } + written.append(path); +} + +/// The shorter of two paths when one is a prefix of the other at a segment +/// boundary, else null. +fn prefix_of(a: []const u8, b: []const u8) ?[]const u8 { + const short = if (a.len <= b.len) a else b; + const long = if (a.len <= b.len) b else a; + if (!std.mem.startsWith(u8, long, short)) return null; + if (long.len != short.len and long[short.len] != '.') return null; + return short; +} + /// Read each array filter's identifier off its top-level field names. /// /// `{"i.b": 3}` binds `i`; `{"i.b": 3, "i.c": 1}` also binds `i` and is legal; @@ -1450,6 +1538,12 @@ test "$set, $inc, $unset, $rename" { } } }, .{ .key = "$inc", .value = .{ .doc = &.{.{ .key = "user.age", .value = .{ .int32 = 2 } }} } }, .{ .key = "$unset", .value = .{ .doc = &.{.{ .key = "gone", .value = .{ .string = "" } }} } }, + }), .{}); + // A second update, because `$set: {new: 5}` and `$rename: {new: ...}` in + // one document both write `new` and mongod refuses that as a conflict -- + // measured. This test used to pack them together and pass only because + // this server had no conflict check. + try apply(&doc, &doc_of(&.{ .{ .key = "$rename", .value = .{ .doc = &.{.{ .key = "new", .value = .{ .string = "renamed" } }} } }, }), .{}); @@ -1618,8 +1712,10 @@ test "a mixed update document is refused from either side" { var doc = bson.Document{ .arena = std.heap.ArenaAllocator.init(testing.allocator), .pairs = &.{} }; defer doc.arena.deinit(); - // Starts with an operator, so operators are expected throughout. - try testing.expectError(error.InvalidUpdate, apply(&doc, &doc_of(&.{ + // Starts with an operator, so operators are expected throughout -- and a + // field that is not one is read as a modifier by that name, which is what + // mongod calls it too ("Unknown modifier: plain"). + try testing.expectError(error.UnknownModifier, apply(&doc, &doc_of(&.{ .{ .key = "$set", .value = .{ .doc = &.{.{ .key = "a", .value = .{ .int32 = 1 } }} } }, .{ .key = "plain", .value = .{ .int32 = 1 } }, }), .{})); @@ -2191,21 +2287,31 @@ test "$min and $max compare in BSON order, not numerically" { }); defer doc.arena.deinit(); try apply(&doc, &doc_of(&.{ - .{ .key = "$min", .value = .{ .doc = &.{ - .{ .key = "a", .value = .{ .int32 = 7 } }, // higher: not written - .{ .key = "s", .value = .{ .int32 = 5 } }, // a number is below a string - .{ .key = "gone", .value = .{ .int32 = 7 } }, // absent: always written - } } }, + .{ + .key = "$min", + .value = .{ + .doc = &.{ + .{ .key = "a", .value = .{ .int32 = 7 } }, // higher: not written + .{ .key = "s", .value = .{ .int32 = 5 } }, // a number is below a string + .{ .key = "gone", .value = .{ .int32 = 7 } }, // absent: always written + }, + }, + }, }), .{}); try testing.expectEqual(@as(i32, 5), doc.get("a").?.int32); try testing.expectEqual(@as(i32, 5), doc.get("s").?.int32); try testing.expectEqual(@as(i32, 7), doc.get("gone").?.int32); try apply(&doc, &doc_of(&.{ - .{ .key = "$max", .value = .{ .doc = &.{ - .{ .key = "a", .value = .{ .int32 = 3 } }, // lower: not written - .{ .key = "n", .value = .{ .int32 = 1 } }, // a number is above null - } } }, + .{ + .key = "$max", + .value = .{ + .doc = &.{ + .{ .key = "a", .value = .{ .int32 = 3 } }, // lower: not written + .{ .key = "n", .value = .{ .int32 = 1 } }, // a number is above null + }, + }, + }, }), .{}); try testing.expectEqual(@as(i32, 5), doc.get("a").?.int32); try testing.expectEqual(@as(i32, 1), doc.get("n").?.int32); @@ -2272,15 +2378,20 @@ test "$addToSet identity is BSON equality, field order included" { }); defer doc.arena.deinit(); try apply(&doc, &doc_of(&.{ - .{ .key = "$addToSet", .value = .{ .doc = &.{ - // An int32 2 and a double 2.0 are one value. - .{ .key = "n", .value = .{ .double = 2.0 } }, - // The same fields in the other order are two. - .{ .key = "d", .value = .{ .doc = &.{ - .{ .key = "b", .value = .{ .int32 = 2 } }, - .{ .key = "a", .value = .{ .int32 = 1 } }, - } } }, - } } }, + .{ + .key = "$addToSet", + .value = .{ + .doc = &.{ + // An int32 2 and a double 2.0 are one value. + .{ .key = "n", .value = .{ .double = 2.0 } }, + // The same fields in the other order are two. + .{ .key = "d", .value = .{ .doc = &.{ + .{ .key = "b", .value = .{ .int32 = 2 } }, + .{ .key = "a", .value = .{ .int32 = 1 } }, + } } }, + }, + }, + }, }), .{}); try testing.expectEqual(@as(usize, 1), doc.get("n").?.array.len); try testing.expectEqual(@as(usize, 2), doc.get("d").?.array.len); @@ -2294,12 +2405,17 @@ test "$pop takes one element off an end, and is quiet when there is none" { }); defer doc.arena.deinit(); try apply(&doc, &doc_of(&.{ - .{ .key = "$pop", .value = .{ .doc = &.{ - .{ .key = "a", .value = .{ .int32 = 1 } }, // the last - .{ .key = "b", .value = .{ .double = -1.0 } }, // the first, and -1.0 is -1 - .{ .key = "e", .value = .{ .int32 = 1 } }, // empty: no-op - .{ .key = "gone", .value = .{ .int32 = 1 } }, // absent: no-op - } } }, + .{ + .key = "$pop", + .value = .{ + .doc = &.{ + .{ .key = "a", .value = .{ .int32 = 1 } }, // the last + .{ .key = "b", .value = .{ .double = -1.0 } }, // the first, and -1.0 is -1 + .{ .key = "e", .value = .{ .int32 = 1 } }, // empty: no-op + .{ .key = "gone", .value = .{ .int32 = 1 } }, // absent: no-op + }, + }, + }, }), .{}); try testing.expectEqual(@as(usize, 2), doc.get("a").?.array.len); try testing.expectEqual(@as(i32, 2), doc.get("a").?.array[1].int32); @@ -2558,14 +2674,19 @@ test "$currentDate writes the clock it was handed" { var doc = try doc_with(testing.allocator, &.{.{ .key = "a", .value = .{ .int32 = 1 } }}); defer doc.arena.deinit(); try apply(&doc, &doc_of(&.{ - .{ .key = "$currentDate", .value = .{ .doc = &.{ - .{ .key = "d", .value = .{ .bool = true } }, - // Measured: `false` writes a date too. The boolean says "a date", - // not "whether". - .{ .key = "f", .value = .{ .bool = false } }, - .{ .key = "e", .value = .{ .doc = &.{.{ .key = "$type", .value = .{ .string = "date" } }} } }, - .{ .key = "t", .value = .{ .doc = &.{.{ .key = "$type", .value = .{ .string = "timestamp" } }} } }, - } } }, + .{ + .key = "$currentDate", + .value = .{ + .doc = &.{ + .{ .key = "d", .value = .{ .bool = true } }, + // Measured: `false` writes a date too. The boolean says "a date", + // not "whether". + .{ .key = "f", .value = .{ .bool = false } }, + .{ .key = "e", .value = .{ .doc = &.{.{ .key = "$type", .value = .{ .string = "date" } }} } }, + .{ .key = "t", .value = .{ .doc = &.{.{ .key = "$type", .value = .{ .string = "timestamp" } }} } }, + }, + }, + }, }), .{ .now_ms = 1_700_000_000_123 }); try testing.expectEqual(@as(i64, 1_700_000_000_123), doc.get("d").?.datetime); try testing.expectEqual(@as(i64, 1_700_000_000_123), doc.get("f").?.datetime); @@ -2594,3 +2715,79 @@ test "$currentDate refuses an operand that names no type" { try testing.expectEqualStrings("int", diag.other); try testing.expect(doc.get("d") == null); } + +test "two paths in one update may not decide the same field" { + // ConflictingUpdateOperators: writing `a` and `a.b` in one update leaves + // the result depending on which operator ran first, so mongod refuses + // rather than picking an order. Measured, across operators and within one. + var doc = try doc_with(testing.allocator, &.{ + .{ .key = "a", .value = .{ .doc = &.{.{ .key = "b", .value = .{ .int32 = 1 } }} } }, + }); + defer doc.arena.deinit(); + var diag: Diagnostic = .{}; + + // Across two operators, on the same path. + try testing.expectError(error.ConflictingUpdate, apply(&doc, &doc_of(&.{ + .{ .key = "$set", .value = .{ .doc = &.{.{ .key = "a", .value = .{ .int32 = 2 } }} } }, + .{ .key = "$inc", .value = .{ .doc = &.{.{ .key = "a", .value = .{ .int32 = 1 } }} } }, + }), .{ .diag = &diag })); + try testing.expectEqualStrings("a", diag.segment); + + // Across two operators, one path a prefix of the other. + try testing.expectError(error.ConflictingUpdate, apply(&doc, &doc_of(&.{ + .{ .key = "$set", .value = .{ .doc = &.{.{ .key = "a", .value = .{ .int32 = 2 } }} } }, + .{ .key = "$inc", .value = .{ .doc = &.{.{ .key = "a.b", .value = .{ .int32 = 1 } }} } }, + }), .{ .diag = &diag })); + try testing.expectEqualStrings("a.b", diag.path); + try testing.expectEqualStrings("a", diag.segment); + + // Within one operator. + try testing.expectError(error.ConflictingUpdate, apply(&doc, &doc_of(&.{ + .{ .key = "$set", .value = .{ .doc = &.{ + .{ .key = "a", .value = .{ .int32 = 2 } }, + .{ .key = "a.b", .value = .{ .int32 = 3 } }, + } } }, + }), .{ .diag = &diag })); + + // `$rename` writes both ends, so both count. + try testing.expectError(error.ConflictingUpdate, apply(&doc, &doc_of(&.{ + .{ .key = "$rename", .value = .{ .doc = &.{.{ .key = "a", .value = .{ .string = "c" } }} } }, + .{ .key = "$set", .value = .{ .doc = &.{.{ .key = "c", .value = .{ .int32 = 5 } }} } }, + }), .{ .diag = &diag })); + + // Siblings are not a conflict. Mutation check: compare paths with plain + // `startsWith` and this goes red -- `a.b` starts with `a.bb`'s prefix in + // the string sense and neither decides the other. + try apply(&doc, &doc_of(&.{ + .{ .key = "$set", .value = .{ .doc = &.{.{ .key = "a.b", .value = .{ .int32 = 2 } }} } }, + .{ .key = "$inc", .value = .{ .doc = &.{.{ .key = "a.bb", .value = .{ .int32 = 1 } }} } }, + }), .{}); + try testing.expectEqual(@as(i32, 2), doc.get("a").?.doc[0].value.int32); +} + +test "a name that is not an operator is refused, not skipped" { + // It used to be `InvalidUpdate` -> BadValue, which is the right shape but + // the wrong code; mongod answers FailedToParse (9) and names the modifier. + // The reason it matters is the alternative nobody chose: skipping an + // unknown operator would make a typo an update the client believes ran. + var doc = try doc_with(testing.allocator, &.{.{ .key = "a", .value = .{ .int32 = 1 } }}); + defer doc.arena.deinit(); + var diag: Diagnostic = .{}; + try testing.expectError(error.UnknownModifier, apply(&doc, &doc_of(&.{ + .{ .key = "$bogus", .value = .{ .doc = &.{.{ .key = "a", .value = .{ .int32 = 1 } }} } }, + }), .{ .diag = &diag })); + try testing.expectEqualStrings("$bogus", diag.segment); + + // Even beside a good one, and nothing the good one asked for lands. + try testing.expectError(error.UnknownModifier, apply(&doc, &doc_of(&.{ + .{ .key = "$set", .value = .{ .doc = &.{.{ .key = "b", .value = .{ .int32 = 1 } }} } }, + .{ .key = "$bogus", .value = .{ .doc = &.{} } }, + }), .{})); + try testing.expect(doc.get("b") == null); + + // A known operator handed something that is not a document of fields. + try testing.expectError(error.ModifierNeedsFields, apply(&doc, &doc_of(&.{ + .{ .key = "$set", .value = .{ .int32 = 1 } }, + }), .{ .diag = &diag })); + try testing.expectEqualStrings("int", diag.segment); +} -- 2.39.5 From 44788ae681ce71a824df949d07c3229e97f6b278 Mon Sep 17 00:00:00 2001 From: "A.Shakhmatov" Date: Mon, 10 Aug 2026 21:33:45 +0300 Subject: [PATCH 9/9] plan/spec: the operator corpus goes green MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 102/102. PLAN's M3 row now names both corpora as the gate, because neither the pinned crud suite nor e2e3/e2e4 can see this work: the eight operators are barely in the pinned corpus and `$push`'s modifiers are not in it at all. §6 records what the corpus found that was nobody's operator -- an upsert never reporting the `_id` it generated -- and the three things left open on purpose: `$bit`, a multi-field `$sort` key inside `$push`, and the 64-path bound on the conflict check. Full matrix at this commit: 247/247 unit tests in ReleaseFast and ReleaseSafe, 83/83 fuzz, operators 102/0, positional 51/0, aggregation 70/0, pinned crud 218/73/196 unmoved, e2e and crash-fuzz green. --- PLAN.md | 24 +++++++++++++++++++++++- tests/spec/operators/README.md | 28 +++++++++++++++++----------- 2 files changed, 40 insertions(+), 12 deletions(-) diff --git a/PLAN.md b/PLAN.md index 6c0d1e6..b2b0a0d 100644 --- a/PLAN.md +++ b/PLAN.md @@ -467,7 +467,7 @@ answers, and the trade is only acceptable because the lie is removed first. | M1 | **Cursors + wire polish** | getMore / killCursors / batchSize; server-side cursor state with idle timeout; sessions plumbing (lsid accepted) as drivers send it; hello advertisement updates; **`moreToCome` on requests** (see the bug below); command-monitoring assertions in the spec runner | crud spec suite green; e2e green | | M2 | **The `aggregate` command surface** | `$out` and `$merge` (7 of the 13 failures), and refusing every pipeline construct the engine does not implement instead of answering `0` (amendment A6). The other 6 failures are blocked on M2.5, M4 and M8 — see `docs/M2_DESIGN_REVIEW.md` §7 | `aggregate-*.json`: 0 fail among the 7 reachable cases | | M2.5 | **The aggregation engine** | expression evaluator, per-stage document iterator, the accumulators, `$unwind`; `$lookup`/`$facet` explicitly out of the first cut (amendment A6) | a purpose-built stage corpus, every expectation measured against mongod | -| M3 | **Update operators + index types** | `distinct` (**done**); positional paths refused rather than destructive (**done**); `$`/`$[]`/`$[]` + `arrayFilters` implemented (**done**); then $setOnInsert, $addToSet, $mul, $min/$max, $pop, $pullAll, $currentDate, pipeline updates; partial + hashed indexes | `tests/spec/positional/` 0 fail (51 cases, recorded from mongod — **green**; the named gate could not see this work, see below); remaining crud coverage; e2e3/e2e4 green | +| M3 | **Update operators + index types** | `distinct` (**done**); positional paths refused rather than destructive (**done**); `$`/`$[]`/`$[]` + `arrayFilters` (**done**); $setOnInsert, $addToSet, $mul, $min/$max, $pop, $pullAll, $currentDate + `$push`'s modifiers (**done**); then pipeline updates; partial + hashed indexes | `tests/spec/positional/` 0 fail (51 cases) and `tests/spec/operators/` 0 fail (102 cases), both recorded from mongod — **green**; the named gate could not see either, see below; remaining crud coverage; e2e3/e2e4 green | | M4 | **Sessions + transactions** | logical sessions, snapshot isolation on the mmap engine, write concern at commit | sessions + transactions spec suites green | | M5 | **Change streams** | change feed + resume tokens (likely log-seq based), getMore integration | change-streams spec suite green | | M6 | **Admin/ops commands** | dbStats, collStats, serverStatus, ping, buildInfo, listDatabases filters, dropDatabase durability (log it) | mongosh UX smoke; e2e green | @@ -1123,6 +1123,28 @@ has to be its own commit with its own re-recorded scorecard. agrees. Worth one commit, and it needs its own measurements first: the padding case (`y.3.b` past the end) is a *legal* creation on mongod, so the fix is not "refuse a non-document element". +- **M3's second corpus is `tests/spec/operators/`.** The eight operators PLAN + §3 names all answered `bad update` with code 2, one message for every + question — and `$push`'s `$slice`, `$position` and `$sort` were parsed, + accepted and *dropped*. `{$each: [3, 4], $slice: -3}` appended both values, + sliced nothing and answered ok: 1 with modifiedCount: 1, which is the same + class of wrong answer the positional operators were. 102 cases, recorded red + at 18/84 and driven green. + + It found one thing that was nobody's operator: **an upsert never reported the + `_id` it generated**. `Engine.insert` writes a generated `_id` into the bytes + and leaves the caller's tree without it, so `upsertedId` came back null and + `findOneAndUpdate` with `returnDocument: after` returned a document with no + `_id` — the only name the client has for a document it has never seen. Fixed + in `build_upsert_doc`; the pinned crud corpus never noticed because its + upsert cases match `upsertedId` loosely. + + Left open, measured and deliberate: `$bit` is not implemented (mongod has it, + no driver in `tests/` sends it, and it is not in the M3 row); `$sort` inside + `$push` accepts only a one-field key document, where mongod allows several; + and the conflict check stops after 64 distinct paths in one update rather + than refusing a legal wide update. + - **M3 update operators** — open. `distinct` landed first because it was a whole missing command with no dependencies, and measuring it turned up three things worth keeping, none of which are `distinct`'s to fix: diff --git a/tests/spec/operators/README.md b/tests/spec/operators/README.md index 7caae44..5848e54 100644 --- a/tests/spec/operators/README.md +++ b/tests/spec/operators/README.md @@ -58,21 +58,27 @@ as a bare number would not have said so. ## Where it stands -Recorded against mongod 8.3.7, run against the server before any of it was -implemented: +Recorded against mongod 8.3.7. Green: ``` -array-ops.json 2 pass 24 fail 0 skip -current-date.json 3 pass 8 fail 0 skip -modifiers.json 7 pass 7 fail 0 skip -numeric.json 0 pass 21 fail 0 skip -push-modifiers.json 6 pass 15 fail 0 skip -set-on-insert.json 0 pass 9 fail 0 skip +array-ops.json 26 pass 0 fail 0 skip +current-date.json 11 pass 0 fail 0 skip +modifiers.json 14 pass 0 fail 0 skip +numeric.json 21 pass 0 fail 0 skip +push-modifiers.json 21 pass 0 fail 0 skip +set-on-insert.json 9 pass 0 fail 0 skip ``` -Red by construction. The 18 that pass are the shapes this server already -answers the way mongod does — mostly refusals that happen to agree, plus the -`$push` cases with no modifier on them. +It was recorded red — 18 pass / 84 fail against a server with none of these +operators — and driven green by seven commits. The 18 that passed then were +the shapes this server already answered mongod's way, mostly refusals that +happened to agree. + +It also found a bug nothing else here had: an upsert never reported the `_id` +it generated, so `updateOne(..., {upsert: true}).upsertedId` was null and +`findOneAndUpdate` with `returnDocument: after` returned a document with no +`_id`. This is the first corpus here that upserts into an empty collection and +then looks at what came back. ## What recording it settled -- 2.39.5