102/102. PLAN's M3 row now names both corpora as the gate, because neither the
pinned crud suite nor e2e3/e2e4 can see this work: the eight operators are
barely in the pinned corpus and `$push`'s modifiers are not in it at all.
§6 records what the corpus found that was nobody's operator -- an upsert never
reporting the `_id` it generated -- and the three things left open on purpose:
`$bit`, a multi-field `$sort` key inside `$push`, and the 64-path bound on the
conflict check.
Full matrix at this commit: 247/247 unit tests in ReleaseFast and ReleaseSafe,
83/83 fuzz, operators 102/0, positional 51/0, aggregation 70/0, pinned crud
218/73/196 unmoved, e2e and crash-fuzz green.
Two refusals this server did not have, both measured.
**ConflictingUpdateOperators (40).** Two paths in one update where either is a
prefix of the other at a segment boundary leave the result depending on which
operator ran first, so mongod refuses rather than picking an order:
`{$set: {a: 2}, $inc: {a: 1}}`, `{$min: {a: 2}, $max: {a: 9}}`,
`{$set: {a: 2, "a.b": 3}}`, and `$rename` counts both its ends. Siblings are
fine, and the segment boundary is what makes them fine -- `a.b` and `a.bb`
share a string prefix and decide nothing about each other, which is the
mutation the test names.
The check is quadratic in the number of paths and bounded at 64, so it stays
on the stack. Past the bound it stops checking rather than refusing: missing
a conflict in a 65-field update is a better answer than refusing a legal one,
and no driver writes updates that wide.
**Unknown modifier (9).** A name not in the operator table was
`InvalidUpdate` -> BadValue; mongod answers FailedToParse and names it. What
matters is the option nobody took: skipping an unknown operator would make a
typo an update the client believes ran. A known operator handed something
that is not a document of fields is the same code.
One existing test changed answer and was wrong: it packed
`$set: {new: 5}` and `$rename: {new: "renamed"}` into one update and passed
only because there was no conflict check. mongod refuses that with 40 --
measured, then the test split into two updates.
operators corpus 95/102 -> **102/102**. The pinned crud corpus, the positional
corpus and the aggregation corpus are unmoved at 218/73, 51/0 and 70/0.
247/247 unit tests.
`Engine.insert` generates an `_id` into the bytes it writes and leaves the
caller's tree without one, so every handler that looks afterwards found
nothing: `update`'s `upserted` array carried `_id: null`, which the driver
surfaces as `upsertedId: null`, and `findAndModify` with
`returnDocument: after` returned an upserted document with no `_id` at all.
The client has never seen this document. The `_id` in the reply is the only
way it can name it again, so both answers were wrong in the way that matters.
Settled in `build_upsert_doc` rather than in the storage engine: the upsert
path is what decides the identity of the document it builds, and it has to be
after the operators run because `$setOnInsert` may supply the `_id` itself.
At the front of the document, where MongoDB stores it and where the `_id_`
index descends on it.
Found by `tests/spec/operators/`, which is the first corpus here to upsert
into an empty collection and then look at what came back. The pinned crud
corpus does not move -- its upsert cases match `upsertedId` loosely.
set-on-insert.json 7/9 -> 8/9. 245/245 unit tests.
`$setOnInsert` is `$set` on the branch that inserts and nothing at all on the
branch that updates, so `Options` grows the one bit that says which -- and
`build_upsert_doc` is the only caller that sets it. It does not go through
`op_set` because of the measured exception: writing `_id` is allowed on a
document being built and refused on one being rewritten. A document nobody
has yet has no identity to change.
`$currentDate` reads a clock, and the clock is a parameter. There is no
fallback to a global one: this file has no `io` to reach the real clock
through, and the handlers pass the same `std.Io.Timestamp` that `ttl_sweep`
and the cursor sweep already read. A caller that forgets gets the epoch --
deterministic and obviously wrong -- rather than something that changes
between runs.
Measured: `$currentDate: {d: false}` writes a date. The boolean says "a date",
not "whether", and either value means the same thing. `{$type: "timestamp"}`
writes a BSON timestamp, seconds in the high 32 bits and an ordinal in the
low ones; mongod fills the ordinal from the oplog, a standalone has none, so
it is 1.
current-date.json 3/11 -> 11/11, set-on-insert.json 0/9 -> 7/9. The two left
are not `$setOnInsert`'s: one is ConflictingUpdateOperators, and the other is
a pre-existing bug the corpus found -- an upsert never reports the `_id` it
generated, because `Engine.insert` writes it into the bytes and not back into
the caller's tree. `updateOne(..., {upsert: true}).upsertedId` is null here
and an ObjectId on mongod. Next commit.
244/244 unit tests.
`$slice`, `$position` and `$sort` were parsed, accepted and dropped.
`{$each: [3, 4], $slice: -3}` appended both values, sliced nothing and
answered ok: 1 with modifiedCount: 1 -- the same class of wrong answer the
positional operators were, and the reason `tests/spec/operators/` exists
rather than a list of TODOs.
Measured, and the order is the whole of it: insert at `$position`, then
`$sort` the array *including* the new elements, then `$slice` the result.
- `$slice: n` keeps the first n, `$slice: -n` the **last** n. That half is
what a capped log depends on and is the one easy to write backwards; the
test mutates exactly it.
- `$position` counts back from the end when negative, and clamps at the
front rather than wrapping.
- `$sort: 1` orders whole elements in BSON order; `$sort: {a: 1}` orders on
a field of them, and an element without it sorts as null -- the rank a
missing field has everywhere else here.
- **without `$each` there are no modifiers at all**: `{$push: {t: {$slice:
1}}}` pushes the document `{$slice: 1}` as a value. That is what makes
`$each` the flag rather than a member of the set, and it is measured, not
reasoned.
An unknown `$`-prefixed key beside `$each`, or a `$slice`/`$position` that is
not a number, is BadValue -- refused rather than ignored, which is the point.
push-modifiers.json 6/21 -> 21/21. 240/240 unit tests.
`$addToSet`'s identity is `bson.compare` equality, which is already exactly
mongod's: an int32 `2` and a double `2.0` are one value, and `{a: 1, b: 2}`
and `{b: 2, a: 1}` are two, because `compare_docs` walks the pairs
positionally and tie-breaks on the key. Candidates are checked against the
array as it grows, so a `$each` holding the same value twice adds it once.
`$pullAll` is `$pull`'s neighbour and its opposite: `$pull` takes a predicate,
`$pullAll` takes values compared whole. `{$pull: {t: {a: 1}}}` removes
elements *having* `a: 1`; `{$pullAll: {t: [{a: 1}]}}` removes elements that
*are* `{a: 1}`. Sharing the comparison would have been the natural mistake and
is what the test mutates to check.
`$pop` is the small one, and its refusals are the measured part: an empty
array and an absent field are no-ops, an argument that is not 1 or -1 is
FailedToParse (9), and a non-array field is TypeMismatch (14) -- where the
identical mistake under `$addToSet` and `$pullAll` is BadValue (2). Three
codes for one shape, none of them derivable from the others. `$each` that is
not an array is 14 under `$addToSet` and 2 under `$push`, measured on both.
array-ops.json 2/26 -> 26/26. 234/234 unit tests.
Three operators, two shapes. `$mul` joins `$inc` in `op_arith` because they
differ only in the operation; `$min` and `$max` are not numeric operators at
all and get their own.
Measured, and each row is a rule that would have been guessed wrong:
- `{$mul: {gone: 5}}` writes **0**, not 5. An absent field starts from zero
under both operators, which is the identity for one and the annihilator
for the other, and mongod picks zero for both.
- `$min`/`$max` compare in BSON canonical order, so `{$min: {s: 5}}` on
`s: "b"` writes 5 -- a number ranks below a string -- and `{$max: {a: 1}}`
on `a: null` writes 1. An absent field is always written: there is nothing
to be smaller or larger than.
- an int32 product that does not fit widens to int64, the same ladder
`numeric_add` already climbed.
`$inc` changes answer with them: a non-numeric field or operand was
`InvalidUpdate` -> BadValue (2), and mongod answers TypeMismatch (14) with a
different sentence for each side. So two errors rather than one, and `$inc`
gets the codes it should always have had.
numeric.json 0/21 -> 20/21. The one left is `$min` and `$max` on the same
field, which is ConflictingUpdateOperators (40) -- a whole error class this
server does not have yet, and its own commit.
228/228 unit tests.
`apply_operator` was a chain of six inline bodies and about a hundred lines,
and M3 adds five more operators to it. One function each, and the shape every
operator's argument has -- a document of path/operand pairs -- checked once at
the top instead of six times.
No behaviour change: 222/222, the same tests, unmoved.
PLAN §3 lists eight operators for M3 -- `$setOnInsert`, `$addToSet`, `$mul`,
`$min`, `$max`, `$pop`, `$pullAll`, `$currentDate` -- and the pinned crud
corpus says almost nothing about any of them. A probe running the identical
update against mongod 8.3.7 and this server found all eight answering
`bad update`, code 2, one message for every question.
It also found the thing this directory exists for: `$push`'s `$slice`,
`$position` and `$sort` are **silently ignored**. `{$each: [3, 4], $slice: -3}`
appends both values, slices nothing, and answers ok: 1 with modifiedCount: 1.
A missing operator is an error the client can see; a modifier that is parsed,
accepted and then dropped is the same class of wrong answer the positional
operators were.
103 cases in six files, 18 pass / 84 fail -- red by construction, like
`tests/spec/aggregate/expressions.json` at 1/26 and the positional corpus at
15/36. Inputs authored in `sources/`, every expectation measured.
Two things here cannot be recorded as values, and both become a `$$type`
assertion rather than being left out: a `$currentDate` field is whatever the
clock said (named per case in `volatile`, so a real stored date can still be
pinned one day), and an upsert that inserts gets a generated ObjectId (that
one automatic -- no source authors an ObjectId). Everything else is compared
exactly. Files are canonical extended JSON, which the runner already parses
that way: `$mul` overflowing an int32 produces an int64, and writing
`4000000000` as a bare number would not have said so.
What recording it settled, none of it guessable:
- `$mul` of a missing field writes **0**, not the operand; of a non-numeric
field, or by one, TypeMismatch (14).
- `$min`/`$max` are not numeric operators. They compare in BSON canonical
order, so `$min: {s: 5}` on `s: "b"` writes 5, and a missing field is
always written.
- two operators writing one field is **ConflictingUpdateOperators (40)** --
`$min`+`$max`, `$set`+`$inc`, `$setOnInsert`+`$set`. A whole error class
this server does not have.
- `$addToSet` compares documents whole, **field order included**:
`{a:1,b:2}` and `{b:2,a:1}` are two values. But `2` and `2.0` are one.
- `$push` modifiers are only modifiers when `$each` is there: `{$slice: 1}`
alone is a value to push. With it, the order is position, then sort the
whole array, then slice.
- `$currentDate` with `false` still writes a date.
- `$setOnInsert` may write `_id` on an insert, where `$set` may not.
- an unknown modifier is FailedToParse (9), not BadValue.
One case was authored and then removed: `{b: 1, $set: {c: 1}}` never reaches a
server -- the driver rejects it -- so there was no answer to record and the
case would have asserted nothing.
