tests/spec: a recorded corpus for the update operators

PLAN §3 lists eight operators for M3 -- `$setOnInsert`, `$addToSet`, `$mul`,
`$min`, `$max`, `$pop`, `$pullAll`, `$currentDate` -- and the pinned crud
corpus says almost nothing about any of them. A probe running the identical
update against mongod 8.3.7 and this server found all eight answering
`bad update`, code 2, one message for every question.

It also found the thing this directory exists for: `$push`'s `$slice`,
`$position` and `$sort` are **silently ignored**. `{$each: [3, 4], $slice: -3}`
appends both values, slices nothing, and answers ok: 1 with modifiedCount: 1.
A missing operator is an error the client can see; a modifier that is parsed,
accepted and then dropped is the same class of wrong answer the positional
operators were.

103 cases in six files, 18 pass / 84 fail -- red by construction, like
`tests/spec/aggregate/expressions.json` at 1/26 and the positional corpus at
15/36. Inputs authored in `sources/`, every expectation measured.

Two things here cannot be recorded as values, and both become a `$$type`
assertion rather than being left out: a `$currentDate` field is whatever the
clock said (named per case in `volatile`, so a real stored date can still be
pinned one day), and an upsert that inserts gets a generated ObjectId (that
one automatic -- no source authors an ObjectId). Everything else is compared
exactly. Files are canonical extended JSON, which the runner already parses
that way: `$mul` overflowing an int32 produces an int64, and writing
`4000000000` as a bare number would not have said so.

What recording it settled, none of it guessable:

  - `$mul` of a missing field writes **0**, not the operand; of a non-numeric
    field, or by one, TypeMismatch (14).
  - `$min`/`$max` are not numeric operators. They compare in BSON canonical
    order, so `$min: {s: 5}` on `s: "b"` writes 5, and a missing field is
    always written.
  - two operators writing one field is **ConflictingUpdateOperators (40)** --
    `$min`+`$max`, `$set`+`$inc`, `$setOnInsert`+`$set`. A whole error class
    this server does not have.
  - `$addToSet` compares documents whole, **field order included**:
    `{a:1,b:2}` and `{b:2,a:1}` are two values. But `2` and `2.0` are one.
  - `$push` modifiers are only modifiers when `$each` is there: `{$slice: 1}`
    alone is a value to push. With it, the order is position, then sort the
    whole array, then slice.
  - `$currentDate` with `false` still writes a date.
  - `$setOnInsert` may write `_id` on an insert, where `$set` may not.
  - an unknown modifier is FailedToParse (9), not BadValue.

One case was authored and then removed: `{b: 1, $set: {c: 1}}` never reaches a
server -- the driver rejects it -- so there was no answer to record and the
case would have asserted nothing.

tests/spec/operators/README.md

@@ -0,0 +1,103 @@
+# The update-operator corpus
+
+PLAN §3 lists eight update operators for M3 — `$setOnInsert`, `$addToSet`,
+`$mul`, `$min`, `$max`, `$pop`, `$pullAll`, `$currentDate` — and the pinned
+crud corpus says almost nothing about any of them. A probe running the
+identical update against mongod 8.3.7 and this server found:
+
+- all eight answering `bad update`, code 2, one message for every question;
+- `$push`'s `$slice`, `$position` and `$sort` **silently ignored**.
+  `{$each: [3, 4], $slice: -3}` appended both values, sliced nothing, and
+  answered `ok: 1` with `modifiedCount: 1`.
+
+The second is the reason this directory exists rather than a list of TODOs.
+A missing operator is an error the client can see; a modifier that is parsed,
+accepted and then dropped is the same class of wrong answer the positional
+operators were — the client asked for one thing and was told it got it.
+
+## The one rule
+
+**Inputs are authored here; expectations are measured against a real mongod.**
+
+```
+tests/spec/operators/
+  sources/*.json      documents + operations, authored
+  record.js           runs them against mongod, writes the expectations
+  *.json              generated, unified format, do not hand-edit
+```
+
+```sh
+mongod --port 27099 --dbpath <dir>
+node tests/spec/operators/record.js --mongod-port 27099
+node tests/spec/run.js --suite-dir tests/spec/operators
+```
+
+Same discipline as `tests/spec/positional/` and `tests/spec/aggregate/`, and
+for the same reason: a corpus written end to end here can encode our own bugs
+as expectations and then agree with us forever.
+
+## What cannot be recorded as a value
+
+Two things in this corpus are not predictable, and both are replaced by a
+`$$type` assertion rather than left out:
+
+- a `$currentDate` field is whatever the clock said. The case names it in
+  `volatile`, per case, so a corpus that one day wants to pin a real stored
+  date still can.
+- an upsert that inserts gets a generated ObjectId. That one is automatic:
+  no source file authors an ObjectId, so the rule is unambiguous.
+
+Everything else about the document — which fields exist, in what order,
+holding what — is still compared exactly. A case that dropped a field still
+fails.
+
+Files are written with canonical extended JSON (`relaxed: false`), which the
+runner already parses that way. It is verbose and it is exact: `$mul`
+overflowing an int32 produces an int64, and a corpus that wrote `4000000000`
+as a bare number would not have said so.
+
+## Where it stands
+
+Recorded against mongod 8.3.7, run against the server before any of it was
+implemented:
+
+```
+array-ops.json          2 pass   24 fail   0 skip
+current-date.json       3 pass    8 fail   0 skip
+modifiers.json          7 pass    7 fail   0 skip
+numeric.json            0 pass   21 fail   0 skip
+push-modifiers.json     6 pass   15 fail   0 skip
+set-on-insert.json      0 pass    9 fail   0 skip
+```
+
+Red by construction. The 18 that pass are the shapes this server already
+answers the way mongod does — mostly refusals that happen to agree, plus the
+`$push` cases with no modifier on them.
+
+## What recording it settled
+
+None of this is guessable, and several rows contradict the obvious reading:
+
+| | mongod |
+|---|---|
+| `$mul` of a missing field | writes **0**, not the operand |
+| `$mul` of a non-numeric field, or by one | TypeMismatch (14) |
+| `$min`/`$max` across types | compares in BSON canonical order, so `$min: {s: 5}` on `s: "b"` writes 5 |
+| `$min`/`$max` of a missing field | always writes |
+| two operators writing one field | **ConflictingUpdateOperators (40)** — `$min`+`$max`, `$set`+`$inc`, `$setOnInsert`+`$set` |
+| `$addToSet` of a document | compares whole, **field order included**: `{a:1,b:2}` and `{b:2,a:1}` are two values |
+| `$addToSet` of `2` and `2.0` | one value |
+| `$pop` of an empty or missing field | no-op, not an error |
+| `$pop` with an argument that is not ±1 | FailedToParse (9); on a non-array field, TypeMismatch (14) |
+| `$pullAll` with a non-array argument | BadValue (2) |
+| `$push` modifiers without `$each` | **not modifiers at all** — `{$slice: 1}` is pushed as a value |
+| `$push` modifier order | insert at `$position`, then `$sort` the whole array, then `$slice` |
+| `$position` negative | counted from the end |
+| `$currentDate` with `false` | still writes a date; the boolean's value is ignored |
+| `$currentDate` with anything but a bool or `{$type: date\|timestamp}` | BadValue (2) |
+| `$setOnInsert` writing `_id` on an insert | **allowed**, unlike `$set` |
+| an unknown modifier | FailedToParse (9), not BadValue |
+
+One case was authored and then removed: `{b: 1, $set: {c: 1}}` is rejected by
+the driver before it reaches a server, so there is no server answer to record
+and the case would have asserted nothing.

tests/spec/operators/record.js

@@ -0,0 +1,222 @@
+// Record an update-operator corpus by asking a real mongod what the answer is.
+//
+// Eight operators PLAN §3 lists for M3 -- `$setOnInsert`, `$addToSet`, `$mul`,
+// `$min`, `$max`, `$pop`, `$pullAll`, `$currentDate` -- plus `$push`'s
+// `$slice`/`$position`/`$sort` modifiers. The pinned crud corpus contains
+// almost nothing about any of them, and a probe against both servers found
+// every one of the eight answering `bad update` here, and the three `$push`
+// modifiers *silently ignored*: `{$each: [3, 4], $slice: -3}` appended without
+// slicing and answered ok: 1. Same discipline as `tests/spec/positional/` and
+// `tests/spec/aggregate/`: inputs are authored in `sources/`, expectations are
+// measured here.
+//
+//   node tests/spec/operators/record.js --mongod-port 27099
+//
+// Options:
+//   --mongod-port <n>  a running mongod to measure against (default 27099)
+//   --only <name>      record just one source file
+const fs = require('fs');
+const path = require('path');
+// The same pinned driver `run.js` uses, resolved the same way.
+const DRIVER = path.join(__dirname, '..', '..', 'e2e', 'node_modules', 'mongodb');
+const { MongoClient } = require(DRIVER);
+const { EJSON, ObjectId, Timestamp } = require(path.join(DRIVER, 'lib', 'bson.js'));
+
+const argv = process.argv.slice(2);
+function opt(name, dflt) {
+    const i = argv.indexOf('--' + name);
+    if (i < 0) return dflt;
+    const v = argv[i + 1];
+    return v === undefined || v.startsWith('--') ? true : v;
+}
+
+const PORT = parseInt(opt('mongod-port', '27099'), 10);
+const ONLY = opt('only', null);
+const SRC_DIR = path.join(__dirname, 'sources');
+const DB_NAME = 'operator-corpus';
+const COLL = 'coll';
+
+// Every construct here predates 4.0, so nothing depends on a server newer than
+// the 4.4 this one reports.
+const SCHEMA_VERSION = '1.4';
+
+const UPDATE_KEYS = ['matchedCount', 'modifiedCount', 'upsertedCount'];
+
+/// Values nobody can predict, replaced by an assertion about their type.
+///
+/// Two kinds, and both are the corpus staying honest rather than the corpus
+/// looking away: a `$currentDate` field is whatever the clock said, and an
+/// upsert that inserts gets a generated ObjectId. Everything else about the
+/// document -- which fields exist, in what order, holding what -- is still
+/// compared exactly, so a case that lost a field still fails.
+///
+/// The ObjectId rule is automatic because it is unambiguous: no source file
+/// authors one. A date is not: `volatile` names those per case, so a corpus
+/// that one day wants to pin a real stored date still can.
+function maskGenerated(value) {
+    if (value instanceof ObjectId) return { $$type: 'objectId' };
+    if (Array.isArray(value)) return value.map(maskGenerated);
+    if (value && typeof value === 'object' && value.constructor === Object) {
+        const out = {};
+        for (const [k, v] of Object.entries(value)) out[k] = maskGenerated(v);
+        return out;
+    }
+    return value;
+}
+
+function maskVolatile(docs, paths) {
+    for (const p of paths) {
+        for (const doc of docs) {
+            const segs = p.split('.');
+            let cur = doc;
+            for (const s of segs.slice(0, -1)) cur = cur === undefined ? undefined : cur[s];
+            const last = segs[segs.length - 1];
+            if (cur === undefined || !(last in cur)) {
+                throw new Error(`volatile path '${p}' is not in the recorded document ` +
+                    `${JSON.stringify(doc)} -- the case did not write what it said it would`);
+            }
+            const v = cur[last];
+            if (v instanceof Date) cur[last] = { $$type: 'date' };
+            else if (v instanceof Timestamp) cur[last] = { $$type: 'timestamp' };
+            else throw new Error(`volatile path '${p}' holds ${v}, which is neither a date nor a timestamp`);
+        }
+    }
+}
+
+async function main() {
+    if (!fs.existsSync(SRC_DIR)) {
+        console.error(`missing ${SRC_DIR}`);
+        process.exit(2);
+    }
+    const client = new MongoClient(`mongodb://127.0.0.1:${PORT}`, { serverSelectionTimeoutMS: 3000 });
+    try {
+        await client.connect();
+    } catch (e) {
+        console.error(`no mongod on :${PORT} -- start one first:\n` +
+            `  mongod --port ${PORT} --dbpath <dir>\n${e.message}`);
+        process.exit(2);
+    }
+    const build = await client.db('admin').command({ buildInfo: 1 });
+    console.log(`recording against mongod ${build.version} on :${PORT}`);
+
+    const sources = fs.readdirSync(SRC_DIR).filter((f) => f.endsWith('.json'))
+        .filter((f) => !ONLY || f === ONLY || f === ONLY + '.json')
+        .sort();
+    if (!sources.length) {
+        console.error('no source files');
+        process.exit(2);
+    }
+
+    for (const file of sources) {
+        const src = JSON.parse(fs.readFileSync(path.join(SRC_DIR, file), 'utf8'));
+        const name = path.basename(file, '.json');
+        const out = await record(client, name, src);
+        const text = EJSON.stringify(out, { relaxed: false, indent: 2 });
+        fs.writeFileSync(path.join(__dirname, `${name}.json`), text + '\n');
+        const errs = out.tests.filter((t) => t.operations[t.operations.length - 1].expectError).length;
+        console.log(`  ${name}: ${out.tests.length} cases, ${errs} of them errors`);
+    }
+    await client.close();
+    console.log('RECORDED');
+}
+
+async function record(client, name, src) {
+    const coll = client.db(DB_NAME).collection(COLL);
+    const tests = [];
+
+    for (const c of src.cases) {
+        const documents = c.documents === undefined ? src.documents : c.documents;
+        await coll.drop().catch(() => {});
+        if (documents.length) await coll.insertMany(structuredClone(documents));
+
+        // A case that brings its own documents reseeds through *operations*:
+        // the unified format's `initialData` is per file, and the runner seeds
+        // it once per test.
+        const setup = [];
+        if (c.documents !== undefined) {
+            setup.push({ object: 'collection0', name: 'deleteMany', arguments: { filter: {} } });
+            if (documents.length) {
+                setup.push({
+                    object: 'collection0',
+                    name: 'insertMany',
+                    arguments: { documents: structuredClone(documents) },
+                });
+            }
+        }
+        const op = {
+            object: 'collection0',
+            name: c.operation,
+            arguments: structuredClone(c.arguments),
+        };
+        try {
+            op.expectResult = maskGenerated(await invoke(coll, c.operation, structuredClone(c.arguments)));
+        } catch (e) {
+            // The code, not the message: message text is mongod's to change
+            // between releases, and several of these embed a rendering of the
+            // offending BSON that no formatter here produces.
+            delete op.expectResult;
+            op.expectError = { isError: true, errorCode: e.code };
+        }
+
+        // Recorded for every case including the refusals: a refusal that left
+        // the document half-written would look identical to a clean one in
+        // `expectError` alone.
+        const after = await coll.find({}, { sort: { _id: 1 } }).toArray();
+        if (c.volatile) maskVolatile(after, c.volatile);
+
+        tests.push({
+            description: c.description,
+            operations: [...setup, op],
+            outcome: [{
+                collectionName: COLL,
+                databaseName: DB_NAME,
+                documents: after.map(maskGenerated),
+            }],
+        });
+    }
+    await coll.drop().catch(() => {});
+
+    return {
+        description: name,
+        schemaVersion: SCHEMA_VERSION,
+        // Recorded, not authored. Regenerate with tests/spec/operators/record.js.
+        createEntities: [
+            { client: { id: 'client0' } },
+            { database: { id: 'database0', client: 'client0', databaseName: DB_NAME } },
+            { collection: { id: 'collection0', database: 'database0', collectionName: COLL } },
+        ],
+        initialData: [{ collectionName: COLL, databaseName: DB_NAME, documents: src.documents }],
+        tests,
+    };
+}
+
+async function invoke(coll, name, args) {
+    const { filter, update, replacement, ...rest } = args;
+    switch (name) {
+        case 'updateOne':
+            return pick(await coll.updateOne(filter, update, rest), UPDATE_KEYS);
+        case 'updateMany':
+            return pick(await coll.updateMany(filter, update, rest), UPDATE_KEYS);
+        case 'replaceOne':
+            return pick(await coll.replaceOne(filter, replacement, rest), UPDATE_KEYS);
+        case 'findOneAndUpdate': {
+            const r = await coll.findOneAndUpdate(filter, update, rest);
+            // Driver 5+ returns the document itself; 4.x wrapped it in
+            // `{value}`. run.js tolerates both the same way.
+            return r && typeof r === 'object' && 'value' in r && 'ok' in r ? r.value : r;
+        }
+        default:
+            throw new Error(`record.js does not know the operation '${name}'`);
+    }
+}
+
+function pick(result, keys) {
+    const o = {};
+    for (const k of keys) if (result[k] !== undefined) o[k] = result[k];
+    return o;
+}
+
+main().catch((e) => {
+    console.error('RECORD_FAIL', e);
+    process.exit(1);
+});

tests/spec/operators/sources/array-ops.json

@@ -0,0 +1,153 @@
+{
+  "_comment": [
+    "Inputs only. Expectations are measured -- see record.js.",
+    "`$addToSet`, `$pop` and `$pullAll`: the three array operators that are",
+    "neither `$push` nor `$pull`. Each has a shape that decides whether it",
+    "wrote at all -- a set that already held the value, a pop of an empty",
+    "array, a pullAll matching nothing -- and `modifiedCount` is the only",
+    "place that shows, which is why every case records an outcome too."
+  ],
+  "documents": [{ "_id": 1, "t": [1, 2, 3] }],
+  "cases": [
+    {
+      "description": "$addToSet appends a value the array does not hold",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$addToSet": { "t": 4 } } }
+    },
+    {
+      "description": "$addToSet on a value already there writes nothing",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$addToSet": { "t": 2 } } }
+    },
+    {
+      "description": "$addToSet on a missing field creates the array",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$addToSet": { "gone": 1 } } }
+    },
+    {
+      "description": "$addToSet $each adds only what is missing",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$addToSet": { "t": { "$each": [2, 4, 5] } } } }
+    },
+    {
+      "description": "$addToSet $each with duplicates inside it",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$addToSet": { "t": { "$each": [7, 7] } } } }
+    },
+    {
+      "description": "$addToSet compares documents whole",
+      "documents": [{ "_id": 1, "t": [{ "a": 1 }] }],
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$addToSet": { "t": { "a": 1 } } } }
+    },
+    {
+      "description": "$addToSet distinguishes documents by field order",
+      "documents": [{ "_id": 1, "t": [{ "a": 1, "b": 2 }] }],
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$addToSet": { "t": { "b": 2, "a": 1 } } } }
+    },
+    {
+      "description": "$addToSet treats an int and an equal double as one value",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$addToSet": { "t": 2.0 } } }
+    },
+    {
+      "description": "$addToSet on a non-array field",
+      "documents": [{ "_id": 1, "t": 5 }],
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$addToSet": { "t": 1 } } }
+    },
+    {
+      "description": "$pop removes the last element",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$pop": { "t": 1 } } }
+    },
+    {
+      "description": "$pop with -1 removes the first",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$pop": { "t": -1 } } }
+    },
+    {
+      "description": "$pop of an empty array writes nothing",
+      "documents": [{ "_id": 1, "t": [] }],
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$pop": { "t": 1 } } }
+    },
+    {
+      "description": "$pop of a missing field writes nothing",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$pop": { "gone": 1 } } }
+    },
+    {
+      "description": "$pop of a one-element array leaves an empty one",
+      "documents": [{ "_id": 1, "t": [9] }],
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$pop": { "t": 1 } } }
+    },
+    {
+      "description": "$pop with an argument that is neither 1 nor -1",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$pop": { "t": 2 } } }
+    },
+    {
+      "description": "$pop with 1.0, which is 1",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$pop": { "t": 1.0 } } }
+    },
+    {
+      "description": "$pop of a non-array field",
+      "documents": [{ "_id": 1, "t": 5 }],
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$pop": { "t": 1 } } }
+    },
+    {
+      "description": "$pullAll removes every copy of every listed value",
+      "documents": [{ "_id": 1, "t": [1, 2, 3, 2] }],
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$pullAll": { "t": [2] } } }
+    },
+    {
+      "description": "$pullAll with several values",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$pullAll": { "t": [1, 3] } } }
+    },
+    {
+      "description": "$pullAll matching nothing writes nothing",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$pullAll": { "t": [9] } } }
+    },
+    {
+      "description": "$pullAll compares documents whole, not by predicate",
+      "documents": [{ "_id": 1, "t": [{ "a": 1 }, { "a": 2 }] }],
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$pullAll": { "t": [{ "a": 1 }] } } }
+    },
+    {
+      "description": "$pullAll on a missing field writes nothing",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$pullAll": { "gone": [1] } } }
+    },
+    {
+      "description": "$pullAll with an argument that is not an array",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$pullAll": { "t": 1 } } }
+    },
+    {
+      "description": "$pullAll emptying the array",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$pullAll": { "t": [1, 2, 3] } } }
+    },
+    {
+      "description": "$addToSet through a positional segment",
+      "documents": [{ "_id": 1, "y": [{ "t": [1] }, { "t": [2] }] }],
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$addToSet": { "y.$[].t": 1 } } }
+    },
+    {
+      "description": "$pop down a dotted path",
+      "documents": [{ "_id": 1, "n": { "t": [1, 2] } }],
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$pop": { "n.t": -1 } } }
+    }
+  ]
+}

tests/spec/operators/sources/current-date.json

@@ -0,0 +1,81 @@
+{
+  "_comment": [
+    "Inputs only. Expectations are measured -- see record.js.",
+    "`$currentDate`, the one update operator whose result cannot be recorded",
+    "as a value: it is whatever the clock said. `volatile` names the fields",
+    "whose recorded value is replaced by a `$$type` assertion, so the case",
+    "still pins the type, the position and every other field of the document",
+    "-- everything except the number nobody can predict."
+  ],
+  "documents": [{ "_id": 1, "a": 1 }],
+  "cases": [
+    {
+      "description": "$currentDate with true writes a date",
+      "volatile": ["d"],
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$currentDate": { "d": true } } }
+    },
+    {
+      "description": "$currentDate with an explicit date type",
+      "volatile": ["d"],
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$currentDate": { "d": { "$type": "date" } } } }
+    },
+    {
+      "description": "$currentDate with the timestamp type",
+      "volatile": ["d"],
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$currentDate": { "d": { "$type": "timestamp" } } } }
+    },
+    {
+      "description": "$currentDate overwrites a field that is already there",
+      "volatile": ["a"],
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$currentDate": { "a": true } } }
+    },
+    {
+      "description": "$currentDate down a dotted path",
+      "volatile": ["n.d"],
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$currentDate": { "n.d": true } } }
+    },
+    {
+      "description": "$currentDate beside another operator",
+      "volatile": ["d"],
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$currentDate": { "d": true }, "$inc": { "a": 1 } } }
+    },
+    {
+      "description": "$currentDate with false, which is still a date",
+      "volatile": ["d"],
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$currentDate": { "d": false } } }
+    },
+    {
+      "description": "$currentDate with an unknown type name",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$currentDate": { "d": { "$type": "nope" } } } }
+    },
+    {
+      "description": "$currentDate with a document that is not $type",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$currentDate": { "d": { "a": 1 } } } }
+    },
+    {
+      "description": "$currentDate with a number",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$currentDate": { "d": 1 } } }
+    },
+    {
+      "description": "$currentDate on an upsert that inserts",
+      "documents": [],
+      "volatile": ["d"],
+      "operation": "updateOne",
+      "arguments": {
+        "filter": { "k": 1 },
+        "update": { "$currentDate": { "d": true } },
+        "upsert": true
+      }
+    }
+  ]
+}

tests/spec/operators/sources/modifiers.json

@@ -0,0 +1,88 @@
+{
+  "_comment": [
+    "Inputs only. Expectations are measured -- see record.js.",
+    "The operator table itself, rather than any one operator: what an update",
+    "document may be, and what happens to a name that is not in the table.",
+    "This server answers `bad update` with code 2 to every one of these, which",
+    "is one message covering several different questions.",
+    "A `{b: 1, $set: {...}}` case was authored and then removed: the driver",
+    "rejects it before it reaches a server, so there is no server answer to",
+    "record and a case with no `errorCode` would assert nothing."
+  ],
+  "documents": [{ "_id": 1, "a": 1, "t": [1, 2] }],
+  "cases": [
+    {
+      "description": "an unknown modifier",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$bogus": { "a": 1 } } }
+    },
+    {
+      "description": "an unknown modifier beside a known one",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$set": { "b": 1 }, "$bogus": { "a": 1 } } }
+    },
+    {
+      "description": "a known modifier whose argument is not a document",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$set": 1 } }
+    },
+    {
+      "description": "an empty argument to a known modifier",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$set": {} } }
+    },
+    {
+      "description": "a data field after an operator",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$set": { "c": 1 }, "b": 1 } }
+    },
+    {
+      "description": "two operators writing the same field",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$set": { "a": 2 }, "$inc": { "a": 1 } } }
+    },
+    {
+      "description": "$inc on a non-numeric field",
+      "documents": [{ "_id": 1, "a": "x" }],
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$inc": { "a": 1 } } }
+    },
+    {
+      "description": "$inc by a non-numeric operand",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$inc": { "a": "x" } } }
+    },
+    {
+      "description": "$push onto a non-array field",
+      "documents": [{ "_id": 1, "t": 5 }],
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$push": { "t": 1 } } }
+    },
+    {
+      "description": "$pull on a non-array field",
+      "documents": [{ "_id": 1, "t": 5 }],
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$pull": { "t": 1 } } }
+    },
+    {
+      "description": "$unset of a field that is not there",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$unset": { "gone": "" } } }
+    },
+    {
+      "description": "$rename onto a field that already exists",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$rename": { "a": "t" } } }
+    },
+    {
+      "description": "$rename of a field that is not there",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$rename": { "gone": "b" } } }
+    },
+    {
+      "description": "an update that writes nothing still reports a match",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$set": { "a": 1 } } }
+    }
+  ]
+}

tests/spec/operators/sources/numeric.json

@@ -0,0 +1,122 @@
+{
+  "_comment": [
+    "Inputs only. Expectations are measured -- see record.js.",
+    "`$mul`, `$min` and `$max`. The two comparison operators are not numeric",
+    "at all -- they compare in BSON canonical order, which is why a string",
+    "field and a number operand have a defined answer -- and the cases here",
+    "are shaped to say which of the three rules each one follows."
+  ],
+  "documents": [{ "_id": 1, "a": 5, "s": "b", "d": 2.5 }],
+  "cases": [
+    {
+      "description": "$mul multiplies an int by an int",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$mul": { "a": 2 } } }
+    },
+    {
+      "description": "$mul of a missing field",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$mul": { "gone": 5 } } }
+    },
+    {
+      "description": "$mul mixes int and double",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$mul": { "d": 2 } } }
+    },
+    {
+      "description": "$mul by zero",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$mul": { "a": 0 } } }
+    },
+    {
+      "description": "$mul overflowing an int32",
+      "documents": [{ "_id": 1, "a": 2000000000 }],
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$mul": { "a": 2 } } }
+    },
+    {
+      "description": "$mul of a non-numeric field",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$mul": { "s": 2 } } }
+    },
+    {
+      "description": "$mul by a non-numeric operand",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$mul": { "a": "x" } } }
+    },
+    {
+      "description": "$mul down a dotted path",
+      "documents": [{ "_id": 1, "n": { "a": 3 } }],
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$mul": { "n.a": 4 } } }
+    },
+    {
+      "description": "$min writes when the operand is lower",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$min": { "a": 3 } } }
+    },
+    {
+      "description": "$min leaves the field when the operand is higher",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$min": { "a": 7 } } }
+    },
+    {
+      "description": "$min of a missing field always writes",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$min": { "gone": 7 } } }
+    },
+    {
+      "description": "$min against a field of another type",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$min": { "s": 5 } } }
+    },
+    {
+      "description": "$min compares an int and an equal double",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$min": { "a": 5.0 } } }
+    },
+    {
+      "description": "$max writes when the operand is higher",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$max": { "a": 7 } } }
+    },
+    {
+      "description": "$max leaves the field when the operand is lower",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$max": { "a": 3 } } }
+    },
+    {
+      "description": "$max of a missing field always writes",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$max": { "gone": 7 } } }
+    },
+    {
+      "description": "$max against a field of another type",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$max": { "s": 5 } } }
+    },
+    {
+      "description": "$max against a null field",
+      "documents": [{ "_id": 1, "a": null }],
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$max": { "a": 1 } } }
+    },
+    {
+      "description": "$min and $max on the same field in one update",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$min": { "a": 3 }, "$max": { "a": 9 } } }
+    },
+    {
+      "description": "$min through an array index",
+      "documents": [{ "_id": 1, "t": [5, 5] }],
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$min": { "t.0": 1 } } }
+    },
+    {
+      "description": "$mul through a positional segment",
+      "documents": [{ "_id": 1, "t": [{ "a": 2 }, { "a": 3 }] }],
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$mul": { "t.$[].a": 10 } } }
+    }
+  ]
+}

tests/spec/operators/sources/push-modifiers.json

@@ -0,0 +1,122 @@
+{
+  "_comment": [
+    "Inputs only. Expectations are measured -- see record.js.",
+    "`$push`'s modifiers: `$each`, `$slice`, `$position`, `$sort`. This server",
+    "already implements `$push` and `$each` and *silently ignores* the other",
+    "three -- `{$each: [3, 4], $slice: -3}` appended and did not slice, and",
+    "answered ok: 1. That is the same class of wrong answer the positional",
+    "operators were: the client asked for one thing and was told it got it."
+  ],
+  "documents": [{ "_id": 1, "t": [1, 2] }],
+  "cases": [
+    {
+      "description": "$slice keeps the last n",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$push": { "t": { "$each": [3, 4], "$slice": -3 } } } }
+    },
+    {
+      "description": "$slice keeps the first n",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$push": { "t": { "$each": [3, 4], "$slice": 3 } } } }
+    },
+    {
+      "description": "$slice of zero empties the array",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$push": { "t": { "$each": [3], "$slice": 0 } } } }
+    },
+    {
+      "description": "$slice larger than the array keeps all of it",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$push": { "t": { "$each": [3], "$slice": 10 } } } }
+    },
+    {
+      "description": "$slice with an empty $each truncates without adding",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$push": { "t": { "$each": [], "$slice": 1 } } } }
+    },
+    {
+      "description": "$position inserts at the front",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$push": { "t": { "$each": [9], "$position": 0 } } } }
+    },
+    {
+      "description": "$position inserts in the middle",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$push": { "t": { "$each": [9], "$position": 1 } } } }
+    },
+    {
+      "description": "$position past the end appends",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$push": { "t": { "$each": [9], "$position": 99 } } } }
+    },
+    {
+      "description": "$position counted from the end",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$push": { "t": { "$each": [9], "$position": -1 } } } }
+    },
+    {
+      "description": "$sort ascending over scalars",
+      "documents": [{ "_id": 1, "t": [3, 1] }],
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$push": { "t": { "$each": [2], "$sort": 1 } } } }
+    },
+    {
+      "description": "$sort descending over scalars",
+      "documents": [{ "_id": 1, "t": [3, 1] }],
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$push": { "t": { "$each": [2], "$sort": -1 } } } }
+    },
+    {
+      "description": "$sort on a field of the elements",
+      "documents": [{ "_id": 1, "t": [{ "a": 3 }, { "a": 1 }] }],
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$push": { "t": { "$each": [{ "a": 2 }], "$sort": { "a": 1 } } } } }
+    },
+    {
+      "description": "$sort and $slice together",
+      "documents": [{ "_id": 1, "t": [3, 1] }],
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$push": { "t": { "$each": [2], "$sort": 1, "$slice": 2 } } } }
+    },
+    {
+      "description": "$position and $slice together",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$push": { "t": { "$each": [9], "$position": 0, "$slice": 2 } } } }
+    },
+    {
+      "description": "$slice without $each",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$push": { "t": { "$slice": 1 } } } }
+    },
+    {
+      "description": "$each that is not an array",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$push": { "t": { "$each": 3 } } } }
+    },
+    {
+      "description": "$slice that is not a number",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$push": { "t": { "$each": [3], "$slice": "x" } } } }
+    },
+    {
+      "description": "$position that is not a number",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$push": { "t": { "$each": [3], "$position": "x" } } } }
+    },
+    {
+      "description": "an unknown modifier beside $each",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$push": { "t": { "$each": [3], "$bogus": 1 } } } }
+    },
+    {
+      "description": "a document operand that is not modifiers at all is a value",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$push": { "t": { "a": 1 } } } }
+    },
+    {
+      "description": "$push $each onto a missing field",
+      "operation": "updateOne",
+      "arguments": { "filter": {}, "update": { "$push": { "gone": { "$each": [1, 2] } } } }
+    }
+  ]
+}

tests/spec/operators/sources/set-on-insert.json

@@ -0,0 +1,99 @@
+{
+  "_comment": [
+    "Inputs only. Expectations are measured -- see record.js.",
+    "`$setOnInsert`, the only update operator whose meaning depends on which",
+    "branch of an upsert ran. Every case here is therefore a pair: the same",
+    "update against a document that exists and against one that does not."
+  ],
+  "documents": [{ "_id": 1, "k": 1, "a": 100 }],
+  "cases": [
+    {
+      "description": "$setOnInsert writes when the upsert inserts",
+      "documents": [],
+      "operation": "updateOne",
+      "arguments": {
+        "filter": { "k": 1 },
+        "update": { "$setOnInsert": { "a": 1 }, "$set": { "b": 2 } },
+        "upsert": true
+      }
+    },
+    {
+      "description": "$setOnInsert is ignored when the upsert updates",
+      "operation": "updateOne",
+      "arguments": {
+        "filter": { "k": 1 },
+        "update": { "$setOnInsert": { "a": 1 }, "$set": { "b": 2 } },
+        "upsert": true
+      }
+    },
+    {
+      "description": "$setOnInsert alone on a plain update is a no-op",
+      "operation": "updateOne",
+      "arguments": {
+        "filter": { "k": 1 },
+        "update": { "$setOnInsert": { "a": 1 } }
+      }
+    },
+    {
+      "description": "$setOnInsert alone on an insert is the whole document",
+      "documents": [],
+      "operation": "updateOne",
+      "arguments": {
+        "filter": { "k": 7 },
+        "update": { "$setOnInsert": { "a": 1 } },
+        "upsert": true
+      }
+    },
+    {
+      "description": "$setOnInsert writes a dotted path on insert",
+      "documents": [],
+      "operation": "updateOne",
+      "arguments": {
+        "filter": { "k": 1 },
+        "update": { "$setOnInsert": { "x.y": 5 } },
+        "upsert": true
+      }
+    },
+    {
+      "description": "$setOnInsert may not touch _id",
+      "documents": [],
+      "operation": "updateOne",
+      "arguments": {
+        "filter": { "k": 1 },
+        "update": { "$setOnInsert": { "_id": 9 } },
+        "upsert": true
+      }
+    },
+    {
+      "description": "$setOnInsert and $set naming the same field on an insert",
+      "documents": [],
+      "operation": "updateOne",
+      "arguments": {
+        "filter": { "k": 1 },
+        "update": { "$setOnInsert": { "a": 1 }, "$set": { "a": 2 } },
+        "upsert": true
+      }
+    },
+    {
+      "description": "findOneAndUpdate reports the document $setOnInsert built",
+      "documents": [],
+      "operation": "findOneAndUpdate",
+      "arguments": {
+        "filter": { "k": 3 },
+        "update": { "$setOnInsert": { "a": 1 } },
+        "upsert": true,
+        "returnDocument": "after"
+      }
+    },
+    {
+      "description": "updateMany with $setOnInsert on an upsert that inserts",
+      "documents": [],
+      "operation": "updateMany",
+      "arguments": {
+        "filter": { "k": 1 },
+        "update": { "$setOnInsert": { "a": 1 } },
+        "upsert": true
+      }
+    }
+  ]
+}

tests/spec/run.js

@@ -44,9 +44,9 @@ function opt(name, dflt) {
 }
 const VERBOSE = !!opt('verbose', false);
 // The corpus. Defaults to the pinned crud suite; `--suite-dir` points the same
-// runner at another one, which is how `tests/spec/aggregate/` and
-// `tests/spec/positional/` are run -- both exist because the pinned suite has
-// a hole where a whole feature should be. Sharing
+// runner at another one, which is how `tests/spec/aggregate/`,
+// `tests/spec/positional/` and `tests/spec/operators/` are run -- each exists
+// because the pinned suite has a hole where a whole feature should be. Sharing
 // the runner rather than writing a second one is the point: the entity model,
 // the matchers, the skip accounting and `expectEvents` all come for free, and a
 // second runner would drift from this one exactly where it mattered.