`$setOnInsert` is `$set` on the branch that inserts and nothing at all on the
branch that updates, so `Options` grows the one bit that says which -- and
`build_upsert_doc` is the only caller that sets it. It does not go through
`op_set` because of the measured exception: writing `_id` is allowed on a
document being built and refused on one being rewritten. A document nobody
has yet has no identity to change.
`$currentDate` reads a clock, and the clock is a parameter. There is no
fallback to a global one: this file has no `io` to reach the real clock
through, and the handlers pass the same `std.Io.Timestamp` that `ttl_sweep`
and the cursor sweep already read. A caller that forgets gets the epoch --
deterministic and obviously wrong -- rather than something that changes
between runs.
Measured: `$currentDate: {d: false}` writes a date. The boolean says "a date",
not "whether", and either value means the same thing. `{$type: "timestamp"}`
writes a BSON timestamp, seconds in the high 32 bits and an ordinal in the
low ones; mongod fills the ordinal from the oplog, a standalone has none, so
it is 1.
current-date.json 3/11 -> 11/11, set-on-insert.json 0/9 -> 7/9. The two left
are not `$setOnInsert`'s: one is ConflictingUpdateOperators, and the other is
a pre-existing bug the corpus found -- an upsert never reports the `_id` it
generated, because `Engine.insert` writes it into the bytes and not back into
the caller's tree. `updateOne(..., {upsert: true}).upsertedId` is null here
and an ObjectId on mongod. Next commit.
244/244 unit tests.
`$slice`, `$position` and `$sort` were parsed, accepted and dropped.
`{$each: [3, 4], $slice: -3}` appended both values, sliced nothing and
answered ok: 1 with modifiedCount: 1 -- the same class of wrong answer the
positional operators were, and the reason `tests/spec/operators/` exists
rather than a list of TODOs.
Measured, and the order is the whole of it: insert at `$position`, then
`$sort` the array *including* the new elements, then `$slice` the result.
- `$slice: n` keeps the first n, `$slice: -n` the **last** n. That half is
what a capped log depends on and is the one easy to write backwards; the
test mutates exactly it.
- `$position` counts back from the end when negative, and clamps at the
front rather than wrapping.
- `$sort: 1` orders whole elements in BSON order; `$sort: {a: 1}` orders on
a field of them, and an element without it sorts as null -- the rank a
missing field has everywhere else here.
- **without `$each` there are no modifiers at all**: `{$push: {t: {$slice:
1}}}` pushes the document `{$slice: 1}` as a value. That is what makes
`$each` the flag rather than a member of the set, and it is measured, not
reasoned.
An unknown `$`-prefixed key beside `$each`, or a `$slice`/`$position` that is
not a number, is BadValue -- refused rather than ignored, which is the point.
push-modifiers.json 6/21 -> 21/21. 240/240 unit tests.
`$addToSet`'s identity is `bson.compare` equality, which is already exactly
mongod's: an int32 `2` and a double `2.0` are one value, and `{a: 1, b: 2}`
and `{b: 2, a: 1}` are two, because `compare_docs` walks the pairs
positionally and tie-breaks on the key. Candidates are checked against the
array as it grows, so a `$each` holding the same value twice adds it once.
`$pullAll` is `$pull`'s neighbour and its opposite: `$pull` takes a predicate,
`$pullAll` takes values compared whole. `{$pull: {t: {a: 1}}}` removes
elements *having* `a: 1`; `{$pullAll: {t: [{a: 1}]}}` removes elements that
*are* `{a: 1}`. Sharing the comparison would have been the natural mistake and
is what the test mutates to check.
`$pop` is the small one, and its refusals are the measured part: an empty
array and an absent field are no-ops, an argument that is not 1 or -1 is
FailedToParse (9), and a non-array field is TypeMismatch (14) -- where the
identical mistake under `$addToSet` and `$pullAll` is BadValue (2). Three
codes for one shape, none of them derivable from the others. `$each` that is
not an array is 14 under `$addToSet` and 2 under `$push`, measured on both.
array-ops.json 2/26 -> 26/26. 234/234 unit tests.
Three operators, two shapes. `$mul` joins `$inc` in `op_arith` because they
differ only in the operation; `$min` and `$max` are not numeric operators at
all and get their own.
Measured, and each row is a rule that would have been guessed wrong:
- `{$mul: {gone: 5}}` writes **0**, not 5. An absent field starts from zero
under both operators, which is the identity for one and the annihilator
for the other, and mongod picks zero for both.
- `$min`/`$max` compare in BSON canonical order, so `{$min: {s: 5}}` on
`s: "b"` writes 5 -- a number ranks below a string -- and `{$max: {a: 1}}`
on `a: null` writes 1. An absent field is always written: there is nothing
to be smaller or larger than.
- an int32 product that does not fit widens to int64, the same ladder
`numeric_add` already climbed.
`$inc` changes answer with them: a non-numeric field or operand was
`InvalidUpdate` -> BadValue (2), and mongod answers TypeMismatch (14) with a
different sentence for each side. So two errors rather than one, and `$inc`
gets the codes it should always have had.
numeric.json 0/21 -> 20/21. The one left is `$min` and `$max` on the same
field, which is ConflictingUpdateOperators (40) -- a whole error class this
server does not have yet, and its own commit.
228/228 unit tests.
`apply_operator` was a chain of six inline bodies and about a hundred lines,
and M3 adds five more operators to it. One function each, and the shape every
operator's argument has -- a document of path/operand pairs -- checked once at
the top instead of six times.
No behaviour change: 222/222, the same tests, unmoved.
`$[]`, `$[<identifier>]` and `$` stop being refused and start being walked.
A path with a positional segment names a *set* of concrete paths rather than
one -- `y.$[].b` on a two-element array names `y.0.b` and `y.1.b` -- so
`resolve` expands it against the document at hand and every operator then
walks paths it already knew how to walk. Nothing below `resolve` knows a
positional segment exists, which is why `$set`, `$inc`, `$unset`, `$push` and
`$pull` all get it at once.
The static half -- which identifier binds to which array filter, whether a
segment may sit in first position, whether a filter went unused -- depends
only on the update and the filters, so it runs once in `validate` before any
document is touched. The document-dependent half is resolution itself: an
absent or non-array path is a refusal there, because array updates address
what is there rather than creating it, unlike `$set` on a plain path.
Codes and messages measured on mongod 8.3.7, not recalled. Sixteen shapes
were run; the ones that changed what this commit does:
- `$[]` in first position answers the *array filter identifier* message,
not the `$` one -- mongod treats the two spellings as one check.
- an array filter may have several top-level fields as long as they all
name the same identifier: `{i.b: 3, i.c: 1}` is legal, `{i.b: 3, j.b: 1}`
is not. So the check is on the name, not on the count.
- an identifier is `[a-z][a-zA-Z0-9]*`: `aB2` yes, `Ab`, `a_b`, `1x` no.
- `$rename` refuses a positional path on *either* end, with its own message
for each, so it keeps the plain split rather than resolving.
- `$unset` of an element leaves a null in its place rather than shortening
the array -- the same answer `$unset: {"y.0": ""}` already gave.
- a scalar element with more path below it is PathNotViable (28), not a
field to create. Without that check the walk would hand `y.1.b` to
`set_path` and it would replace the `7` at `y.1` with `{b: 9}` -- a
smaller copy of the destruction this whole walk replaced.
One divergence, measured and deliberate: `{"y.b": 3, "y.c": 2}` matches
`[{b: 3, c: 1}, {b: 1, c: 2}]` without either element satisfying both, and
`$` then picks element 1 on mongod and element 0 here. mongod's answer is an
artefact of which predicate last wrote its match position; guessing at it
would be worse than recording it. PLAN §6.
The command handlers still pass neither the query nor any array filters, so
over the wire this is `$[]` working and the other two refusing -- for the
right reason and with the right code, which they did not before. The plumbing
is the next commit.
221/221 unit tests in ReleaseFast and ReleaseSafe, 83/83 fuzz.
Positional corpus 15 -> 23 of 51.
`{$set: {"y.$[i].b": 2}}` did not fail to update the array. It replaced
`y: [{b: 3}, {b: 1}]` with `y: {"$[i]": {"b": 2}}` -- every element
discarded -- and answered ok: 1, modifiedCount: 1. Remotely reachable by any
client issuing an ordinary MongoDB update.
`arrayFilters` was not implicated: the string appears nowhere in src/, the
option is accepted off the wire and dropped. The destruction was in the
path, at `set_path`'s "treat as non-array: replace with a doc" branch, so it
fired for all three spellings of "descend into this array" -- `$`, `$[]` and
`$[<ident>]` -- under every operator. `$inc` through `$[i]` stored its
operand rather than incrementing.
Two refusals, because the branch held two different mistakes:
- a positional segment is refused up front, before anything is applied, so
an update naming a good path and a positional one lands neither. Its
code is BadValue (2), which is what mongod answers for every positional
path failure.
- a plain non-numeric segment under an array -- `y.nope.b`, `y.$x.b` -- is
PathNotViable (28), measured. It is never a field to create, which is
the opposite of what `set_path` does for a missing *document* field and
the reason this branch existed at all.
Numeric segments are untouched, including the null padding past the end,
which a test now pins.
Messages are this server's own words. mongod's PathNotViable text embeds a
shell-syntax rendering of the offending element (`Cannot create field 'nope'
in element {y: [ { b: 3 }, { b: 1 } ]}`) and no BSON formatter here produces
it. The code is what the corpus asserts and the code is exact; a half-copy
of the text would be worse than a clear sentence that does not pretend.
Re-running the 17-case probe against both servers: 16 of 17 diverged before,
0 are destructive now, and 5 agree with mongod's code exactly -- every case
where mongod also refuses. The rest refuse where mongod succeeds, which is
the honest not-implemented state and is what the design review chose.
Scorecard unchanged at 204 pass / 87 fail: the 14 arrayFilters cases still
fail, now reporting the refusal rather than a corrupted document. That was
the gate this review picked -- `docs/M3_ARRAYFILTERS_DESIGN_REVIEW.md` §5,
option D -- because the corpus has no `$[]` case and no bare `$` case at
all, so passing it would have certified two live ways to destroy an array.
206/206 unit (8 new) in ReleaseFast and ReleaseSafe, 83/83 fuzz, aggregation
corpus 70/0, full e2e matrix and crash-fuzz green. Both refusals are
mutation-checked: dropping the up-front scan reddens four tests on the
array's contents rather than on the error, and restoring the destructive
branch reddens the PathNotViable test.
`replaceOne`, `findOneAndReplace` and `bulkWrite`'s `replaceOne` all failed
with "bad update". `update.apply` rejected any update document whose first key
was not `$`-prefixed, so a replacement document -- which by definition has no
operators -- could not get through at all.
MongoDB decides on the first field and nothing else: `$`-prefixed means
operators, anything else means the document *is* the new content. An empty
document is a replacement too, and a legal one. `is_replacement` says which,
`apply_replacement` does the work, and because all three call sites already go
through `apply`, that one branch covers the update command, findAndModify and
the upsert builder.
What a replacement means, precisely:
- every field is replaced except `_id`, which is immutable and keeps its
position at the front, where it is stored and where the `_id_` index
descends on it;
- a replacement may restate the same `_id` but not a different one -- that
is `ImmutableId`, because otherwise a rewrite would silently change a
document's identity while the index entry kept the old key;
- when the target has no `_id` yet, the replacement supplies it. That is the
upsert path: `build_upsert_doc` seeds a document from the filter's
equalities, so `replaceOne({_id: 99}, {u: 1}, {upsert: true})` inserts
`{_id: 99, u: 1}` and not a generated ObjectId;
- a mixed document is refused from either side, rather than guessed at.
Two options on update specs are refused rather than ignored:
- `multi` with a replacement (FailedToParse). A replacement describes one
document; applying it to many would leave every match identical apart from
its `_id`.
- `sort`, a MongoDB 8.0 addition this server does not implement. Ignoring it
is the worst of the three answers -- `sort` chooses *which* match to write,
so the client would silently get a different document than it asked for.
spec scorecard 131 pass / 161 fail -> 161 pass / 131 fail
e2e.js 35 checks -> 45
Sixteen spec files improved and none regressed. The two `-sort` files briefly
did: they had been passing on their "server-side error" case, which our
"bad update" failure satisfied by accident, and passing for the wrong reason is
how a gap survives a scorecard.
Five mutations, each verified red: seeding the replacement from the old pairs,
dropping the `_id` comparison, dropping the `_id` a replacement supplies, and
removing the mixed-document guard from either loop.
Note `nModified` is still wrong for a write that changes nothing -- MongoDB
counts a document as modified only if applying the update altered it. That is
the remaining bulkWrite failure and is fixed next, separately.
Wrap signatures and long expressions to the 100-column limit and make every
file zig fmt clean. Semantics-preserving throughout: ignoring whitespace and
the trailing commas that wrapping introduces, every file here is byte-identical
to its predecessor, and the one apparent exception is a warning string split
with `++`, which concatenates at comptime to the same bytes.
src/index.zig and src/commands.zig are reformatted in the commits that follow,
because their reformat is interleaved with in-flight changes to them and
separating the two would need the reformat re-derived rather than moved.
Adds src/index.zig with the full secondary-index machinery: entry
generation mirroring field_matches (array value + elements), BSON-order
sorted entries with binary search, compound prefix and range lookups,
unique/sparse options, the query planner (longest equality/$in run +
optional range, $in cartesian cap, sparse/null bail), and the _id_ fast
path guarded against serialization-ambiguous values (numbers, strings,
symbols, codes, opaque payloads).
query.collect_values is now pub so entry generation can mirror it exactly.
storage.zig gains record_type_index_create/drop; lib.zig exports index.
