diff --git a/PLAN.md b/PLAN.md index a9a3feb..14ff3b0 100644 --- a/PLAN.md +++ b/PLAN.md @@ -1143,6 +1143,14 @@ has to be its own commit with its own re-recorded scorecard. contrast, was honestly missing -- refused, with the wrong code (2 where mongod says 67) but the right answer. + **`tests/spec/indexes/` is the gate**, recorded red at 3/39 across 42 cases. + A case there is a *sequence* -- create, insert, read, list -- because an + index outlives a `deleteMany` and every case is about which indexes exist. + Recording it corrected two of the review's own guesses: `$in` in a partial + filter is **allowed** (the review grouped it with `$ne` and `$regex`, which + are 67), and the same key with a different filter and no explicit name is + **IndexKeySpecsConflict (86)**, not 67. + `docs/M3_INDEX_TYPES_DESIGN_REVIEW.md` measures both features' rules and argues the planner rule that decides the design: a partial index may only answer a query whose predicates *imply* its filter, so until that test diff --git a/tests/spec/indexes/README.md b/tests/spec/indexes/README.md new file mode 100644 index 0000000..f27dabd --- /dev/null +++ b/tests/spec/indexes/README.md @@ -0,0 +1,79 @@ +# The index corpus + +M3's last row: partial and hashed indexes. `docs/M3_INDEX_TYPES_DESIGN_REVIEW.md` +measured both against mongod 8.3.7 and found they were not the same kind of +gap — hashed was honestly refused, `partialFilterExpression` was accepted and +ignored, and a *unique* partial index therefore refused inserts mongod +accepts. + +It also found that **no test in this repository covered that row, in any +suite**. The pinned corpus is crud and aggregate; `tests/e2e/e2e5.js` and +`e2e6.js` test indexes and write neither a partial nor a hashed spec. So this +directory exists for the same reason `tests/spec/positional/` and +`tests/spec/operators/` do. + +## The one rule + +**Inputs are authored here; expectations are measured against a real mongod.** + +``` +tests/spec/indexes/ + sources/*.json documents + operations, authored + record.js runs them against mongod, writes the expectations + *.json generated, unified format, do not hand-edit +``` + +```sh +mongod --port 27099 --dbpath +node tests/spec/indexes/record.js --mongod-port 27099 +node tests/spec/run.js --suite-dir tests/spec/indexes +``` + +Unlike the other two corpora, a case here is a **sequence**: create an index, +insert against it, read back, list it. An index outlives a `deleteMany`, and +every case is about which indexes exist, so the recorder drops the collection +between cases and walks each case's operations in order — stopping at the +first that throws, which is what a client would see. + +## Where it stands + +Recorded against mongod 8.3.7, run before any of it was implemented: + +``` +hashed.json 0 pass 18 fail 0 skip +partial.json 3 pass 21 fail 0 skip +``` + +Red by construction. The three that pass are the reads a partial index does +not change: this server builds an index over every document, so a query still +finds everything, which is the whole reason the review called the partial gap +smaller than the `arrayFilters` one. + +## What recording it settled + +Two of the review's own guesses were wrong, which is why it was recorded +rather than reasoned: + +| | mongod | +|---|---| +| `$in` in a partial filter | **allowed** — the review listed it with `$ne` and `$regex` | +| same key, different filter, no explicit name | **IndexKeySpecsConflict (86)**, not the 67 the review assumed | + +And what it confirmed: + +| | mongod | +|---|---| +| a unique partial index | constrains only the documents its filter selects — two `{a: 1, t: false}` are fine, two `{a: 9, t: true}` are E11000 | +| a document *leaving* the filter | frees the value it held for another document to take | +| a document *entering* the filter | must take a value nothing inside it holds, or the update is E11000 | +| `$regex`, `$ne` in a filter | 67 | +| a filter that is not a document | TypeMismatch (14) | +| `sparse` + `partialFilterExpression` | 67 — may not be combined | +| `expireAfterSeconds` + a filter | allowed; `listIndexes` reports the filter *before* the expiry | +| an empty filter | allowed, and reported | +| two hashed components | 31303 | +| `unique` on a hashed index | 16764 | +| an array at a hashed path | 16766, **at insert time**, not at creation | +| a key direction that is not 1, -1 or `"hashed"` | 67 | +| a hashed index beside an ascending one on the same field | both exist | +| a range query or a sort over a hashed field | still correct — the planner declines the index rather than misusing it | diff --git a/tests/spec/indexes/hashed.json b/tests/spec/indexes/hashed.json new file mode 100644 index 0000000..736343e --- /dev/null +++ b/tests/spec/indexes/hashed.json @@ -0,0 +1 @@ +{"description":"hashed","schemaVersion":"1.4","createEntities":[{"client":{"id":"client0"}},{"database":{"id":"database0","client":"client0","databaseName":"index-corpus"}},{"collection":{"id":"collection0","database":"database0","collectionName":"coll"}}],"initialData":[{"collectionName":"coll","databaseName":"index-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"s":"x"},{"_id":{"$numberInt":"2"},"a":{"$numberInt":"5"},"s":"y"},{"_id":{"$numberInt":"3"},"s":"z"},{"_id":{"$numberInt":"4"},"a":{"$numberInt":"5"},"s":"x"}]}],"tests":[{"description":"a hashed index is created and names itself for the plugin","operations":[{"object":"collection0","name":"createIndex","arguments":{"keys":{"a":"hashed"}},"expectResult":"a_hashed"},{"object":"collection0","name":"listIndexes","arguments":{},"expectResult":[{"v":{"$numberInt":"2"},"key":{"_id":{"$numberInt":"1"}},"name":"_id_"},{"v":{"$numberInt":"2"},"key":{"a":"hashed"},"name":"a_hashed"}]}],"outcome":[{"collectionName":"coll","databaseName":"index-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"s":"x"},{"_id":{"$numberInt":"2"},"a":{"$numberInt":"5"},"s":"y"},{"_id":{"$numberInt":"3"},"s":"z"},{"_id":{"$numberInt":"4"},"a":{"$numberInt":"5"},"s":"x"}]}]},{"description":"an equality query answers every match","operations":[{"object":"collection0","name":"createIndex","arguments":{"keys":{"a":"hashed"}},"expectResult":"a_hashed"},{"object":"collection0","name":"find","arguments":{"filter":{"a":{"$numberInt":"5"}},"sort":{"_id":{"$numberInt":"1"}}},"expectResult":[{"_id":{"$numberInt":"2"},"a":{"$numberInt":"5"},"s":"y"},{"_id":{"$numberInt":"4"},"a":{"$numberInt":"5"},"s":"x"}]}],"outcome":[{"collectionName":"coll","databaseName":"index-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"s":"x"},{"_id":{"$numberInt":"2"},"a":{"$numberInt":"5"},"s":"y"},{"_id":{"$numberInt":"3"},"s":"z"},{"_id":{"$numberInt":"4"},"a":{"$numberInt":"5"},"s":"x"}]}]},{"description":"equality against a value nothing holds","operations":[{"object":"collection0","name":"createIndex","arguments":{"keys":{"a":"hashed"}},"expectResult":"a_hashed"},{"object":"collection0","name":"find","arguments":{"filter":{"a":{"$numberInt":"99"}},"sort":{"_id":{"$numberInt":"1"}}},"expectResult":[]}],"outcome":[{"collectionName":"coll","databaseName":"index-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"s":"x"},{"_id":{"$numberInt":"2"},"a":{"$numberInt":"5"},"s":"y"},{"_id":{"$numberInt":"3"},"s":"z"},{"_id":{"$numberInt":"4"},"a":{"$numberInt":"5"},"s":"x"}]}]},{"description":"equality against a missing field","operations":[{"object":"collection0","name":"createIndex","arguments":{"keys":{"a":"hashed"}},"expectResult":"a_hashed"},{"object":"collection0","name":"find","arguments":{"filter":{"a":null},"sort":{"_id":{"$numberInt":"1"}}},"expectResult":[{"_id":{"$numberInt":"3"},"s":"z"}]}],"outcome":[{"collectionName":"coll","databaseName":"index-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"s":"x"},{"_id":{"$numberInt":"2"},"a":{"$numberInt":"5"},"s":"y"},{"_id":{"$numberInt":"3"},"s":"z"},{"_id":{"$numberInt":"4"},"a":{"$numberInt":"5"},"s":"x"}]}]},{"description":"a range query still answers every match","operations":[{"object":"collection0","name":"createIndex","arguments":{"keys":{"a":"hashed"}},"expectResult":"a_hashed"},{"object":"collection0","name":"find","arguments":{"filter":{"a":{"$gte":{"$numberInt":"5"}}},"sort":{"_id":{"$numberInt":"1"}}},"expectResult":[{"_id":{"$numberInt":"2"},"a":{"$numberInt":"5"},"s":"y"},{"_id":{"$numberInt":"4"},"a":{"$numberInt":"5"},"s":"x"}]}],"outcome":[{"collectionName":"coll","databaseName":"index-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"s":"x"},{"_id":{"$numberInt":"2"},"a":{"$numberInt":"5"},"s":"y"},{"_id":{"$numberInt":"3"},"s":"z"},{"_id":{"$numberInt":"4"},"a":{"$numberInt":"5"},"s":"x"}]}]},{"description":"a sort on the hashed field is by value, not by hash","operations":[{"object":"collection0","name":"createIndex","arguments":{"keys":{"a":"hashed"}},"expectResult":"a_hashed"},{"object":"collection0","name":"find","arguments":{"filter":{},"sort":{"a":{"$numberInt":"1"}}},"expectResult":[{"_id":{"$numberInt":"3"},"s":"z"},{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"s":"x"},{"_id":{"$numberInt":"2"},"a":{"$numberInt":"5"},"s":"y"},{"_id":{"$numberInt":"4"},"a":{"$numberInt":"5"},"s":"x"}]}],"outcome":[{"collectionName":"coll","databaseName":"index-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"s":"x"},{"_id":{"$numberInt":"2"},"a":{"$numberInt":"5"},"s":"y"},{"_id":{"$numberInt":"3"},"s":"z"},{"_id":{"$numberInt":"4"},"a":{"$numberInt":"5"},"s":"x"}]}]},{"description":"equality across numeric types","operations":[{"object":"collection0","name":"createIndex","arguments":{"keys":{"a":"hashed"}},"expectResult":"a_hashed"},{"object":"collection0","name":"find","arguments":{"filter":{"a":{"$numberInt":"5"}},"sort":{"_id":{"$numberInt":"1"}}},"expectResult":[{"_id":{"$numberInt":"2"},"a":{"$numberInt":"5"},"s":"y"},{"_id":{"$numberInt":"4"},"a":{"$numberInt":"5"},"s":"x"}]}],"outcome":[{"collectionName":"coll","databaseName":"index-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"s":"x"},{"_id":{"$numberInt":"2"},"a":{"$numberInt":"5"},"s":"y"},{"_id":{"$numberInt":"3"},"s":"z"},{"_id":{"$numberInt":"4"},"a":{"$numberInt":"5"},"s":"x"}]}]},{"description":"inserting after the index exists","operations":[{"object":"collection0","name":"createIndex","arguments":{"keys":{"a":"hashed"}},"expectResult":"a_hashed"},{"object":"collection0","name":"insertOne","arguments":{"document":{"_id":{"$numberInt":"5"},"a":{"$numberInt":"5"}}},"expectResult":{"insertedId":{"$numberInt":"5"}}},{"object":"collection0","name":"find","arguments":{"filter":{"a":{"$numberInt":"5"}},"sort":{"_id":{"$numberInt":"1"}}},"expectResult":[{"_id":{"$numberInt":"2"},"a":{"$numberInt":"5"},"s":"y"},{"_id":{"$numberInt":"4"},"a":{"$numberInt":"5"},"s":"x"},{"_id":{"$numberInt":"5"},"a":{"$numberInt":"5"}}]}],"outcome":[{"collectionName":"coll","databaseName":"index-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"s":"x"},{"_id":{"$numberInt":"2"},"a":{"$numberInt":"5"},"s":"y"},{"_id":{"$numberInt":"3"},"s":"z"},{"_id":{"$numberInt":"4"},"a":{"$numberInt":"5"},"s":"x"},{"_id":{"$numberInt":"5"},"a":{"$numberInt":"5"}}]}]},{"description":"deleting through a hashed index","operations":[{"object":"collection0","name":"createIndex","arguments":{"keys":{"a":"hashed"}},"expectResult":"a_hashed"},{"object":"collection0","name":"deleteMany","arguments":{"filter":{"a":{"$numberInt":"5"}}},"expectResult":{"deletedCount":{"$numberInt":"2"}}},{"object":"collection0","name":"find","arguments":{"filter":{},"sort":{"_id":{"$numberInt":"1"}}},"expectResult":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"s":"x"},{"_id":{"$numberInt":"3"},"s":"z"}]}],"outcome":[{"collectionName":"coll","databaseName":"index-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"s":"x"},{"_id":{"$numberInt":"3"},"s":"z"}]}]},{"description":"a hashed component beside a range one","operations":[{"object":"collection0","name":"createIndex","arguments":{"keys":{"a":"hashed","s":{"$numberInt":"1"}}},"expectResult":"a_hashed_s_1"},{"object":"collection0","name":"listIndexes","arguments":{},"expectResult":[{"v":{"$numberInt":"2"},"key":{"_id":{"$numberInt":"1"}},"name":"_id_"},{"v":{"$numberInt":"2"},"key":{"a":"hashed","s":{"$numberInt":"1"}},"name":"a_hashed_s_1"}]}],"outcome":[{"collectionName":"coll","databaseName":"index-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"s":"x"},{"_id":{"$numberInt":"2"},"a":{"$numberInt":"5"},"s":"y"},{"_id":{"$numberInt":"3"},"s":"z"},{"_id":{"$numberInt":"4"},"a":{"$numberInt":"5"},"s":"x"}]}]},{"description":"two hashed components are refused","operations":[{"object":"collection0","name":"createIndex","arguments":{"keys":{"a":"hashed","s":"hashed"}},"expectError":{"isError":true,"errorCode":{"$numberInt":"31303"}}}],"outcome":[{"collectionName":"coll","databaseName":"index-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"s":"x"},{"_id":{"$numberInt":"2"},"a":{"$numberInt":"5"},"s":"y"},{"_id":{"$numberInt":"3"},"s":"z"},{"_id":{"$numberInt":"4"},"a":{"$numberInt":"5"},"s":"x"}]}]},{"description":"a unique hashed index is refused","operations":[{"object":"collection0","name":"createIndex","arguments":{"keys":{"a":"hashed"},"unique":true},"expectError":{"isError":true,"errorCode":{"$numberInt":"16764"}}}],"outcome":[{"collectionName":"coll","databaseName":"index-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"s":"x"},{"_id":{"$numberInt":"2"},"a":{"$numberInt":"5"},"s":"y"},{"_id":{"$numberInt":"3"},"s":"z"},{"_id":{"$numberInt":"4"},"a":{"$numberInt":"5"},"s":"x"}]}]},{"description":"an array at the hashed path is refused, at insert time","operations":[{"object":"collection0","name":"deleteMany","arguments":{"filter":{}}},{"object":"collection0","name":"createIndex","arguments":{"keys":{"a":"hashed"}},"expectResult":"a_hashed"},{"object":"collection0","name":"insertOne","arguments":{"document":{"_id":{"$numberInt":"1"},"a":[{"$numberInt":"1"},{"$numberInt":"2"}]}},"expectError":{"isError":true,"errorCode":{"$numberInt":"16766"}}}],"outcome":[{"collectionName":"coll","databaseName":"index-corpus","documents":[]}]},{"description":"a key direction that is neither 1, -1 nor hashed","operations":[{"object":"collection0","name":"createIndex","arguments":{"keys":{"a":"bogus"}},"expectError":{"isError":true,"errorCode":{"$numberInt":"67"}}}],"outcome":[{"collectionName":"coll","databaseName":"index-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"s":"x"},{"_id":{"$numberInt":"2"},"a":{"$numberInt":"5"},"s":"y"},{"_id":{"$numberInt":"3"},"s":"z"},{"_id":{"$numberInt":"4"},"a":{"$numberInt":"5"},"s":"x"}]}]},{"description":"a sparse hashed index","operations":[{"object":"collection0","name":"createIndex","arguments":{"keys":{"a":"hashed"},"sparse":true},"expectResult":"a_hashed"},{"object":"collection0","name":"listIndexes","arguments":{},"expectResult":[{"v":{"$numberInt":"2"},"key":{"_id":{"$numberInt":"1"}},"name":"_id_"},{"v":{"$numberInt":"2"},"key":{"a":"hashed"},"name":"a_hashed","sparse":true}]}],"outcome":[{"collectionName":"coll","databaseName":"index-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"s":"x"},{"_id":{"$numberInt":"2"},"a":{"$numberInt":"5"},"s":"y"},{"_id":{"$numberInt":"3"},"s":"z"},{"_id":{"$numberInt":"4"},"a":{"$numberInt":"5"},"s":"x"}]}]},{"description":"the same hashed index twice is idempotent","operations":[{"object":"collection0","name":"createIndex","arguments":{"keys":{"a":"hashed"}},"expectResult":"a_hashed"},{"object":"collection0","name":"createIndex","arguments":{"keys":{"a":"hashed"}},"expectResult":"a_hashed"},{"object":"collection0","name":"listIndexes","arguments":{},"expectResult":[{"v":{"$numberInt":"2"},"key":{"_id":{"$numberInt":"1"}},"name":"_id_"},{"v":{"$numberInt":"2"},"key":{"a":"hashed"},"name":"a_hashed"}]}],"outcome":[{"collectionName":"coll","databaseName":"index-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"s":"x"},{"_id":{"$numberInt":"2"},"a":{"$numberInt":"5"},"s":"y"},{"_id":{"$numberInt":"3"},"s":"z"},{"_id":{"$numberInt":"4"},"a":{"$numberInt":"5"},"s":"x"}]}]},{"description":"a hashed index beside an ascending one on the same field","operations":[{"object":"collection0","name":"createIndex","arguments":{"keys":{"a":{"$numberInt":"1"}}},"expectResult":"a_1"},{"object":"collection0","name":"createIndex","arguments":{"keys":{"a":"hashed"}},"expectResult":"a_hashed"},{"object":"collection0","name":"listIndexes","arguments":{},"expectResult":[{"v":{"$numberInt":"2"},"key":{"_id":{"$numberInt":"1"}},"name":"_id_"},{"v":{"$numberInt":"2"},"key":{"a":{"$numberInt":"1"}},"name":"a_1"},{"v":{"$numberInt":"2"},"key":{"a":"hashed"},"name":"a_hashed"}]}],"outcome":[{"collectionName":"coll","databaseName":"index-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"s":"x"},{"_id":{"$numberInt":"2"},"a":{"$numberInt":"5"},"s":"y"},{"_id":{"$numberInt":"3"},"s":"z"},{"_id":{"$numberInt":"4"},"a":{"$numberInt":"5"},"s":"x"}]}]},{"description":"a hashed index is dropped by name","operations":[{"object":"collection0","name":"createIndex","arguments":{"keys":{"a":"hashed"}},"expectResult":"a_hashed"},{"object":"collection0","name":"dropIndex","arguments":{"name":"a_hashed"},"expectResult":{}},{"object":"collection0","name":"listIndexes","arguments":{},"expectResult":[{"v":{"$numberInt":"2"},"key":{"_id":{"$numberInt":"1"}},"name":"_id_"}]}],"outcome":[{"collectionName":"coll","databaseName":"index-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"s":"x"},{"_id":{"$numberInt":"2"},"a":{"$numberInt":"5"},"s":"y"},{"_id":{"$numberInt":"3"},"s":"z"},{"_id":{"$numberInt":"4"},"a":{"$numberInt":"5"},"s":"x"}]}]}]} diff --git a/tests/spec/indexes/partial.json b/tests/spec/indexes/partial.json new file mode 100644 index 0000000..504c54d --- /dev/null +++ b/tests/spec/indexes/partial.json @@ -0,0 +1 @@ +{"description":"partial","schemaVersion":"1.4","createEntities":[{"client":{"id":"client0"}},{"database":{"id":"database0","client":"client0","databaseName":"index-corpus"}},{"collection":{"id":"collection0","database":"database0","collectionName":"coll"}}],"initialData":[{"collectionName":"coll","databaseName":"index-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"s":"x"},{"_id":{"$numberInt":"2"},"a":{"$numberInt":"5"},"s":"y"},{"_id":{"$numberInt":"3"},"s":"z"},{"_id":{"$numberInt":"4"},"a":{"$numberInt":"5"},"s":"x"}]}],"tests":[{"description":"a partial index is created and reports its filter","operations":[{"object":"collection0","name":"createIndex","arguments":{"keys":{"a":{"$numberInt":"1"}},"partialFilterExpression":{"a":{"$gte":{"$numberInt":"5"}}}},"expectResult":"a_1"},{"object":"collection0","name":"listIndexes","arguments":{},"expectResult":[{"v":{"$numberInt":"2"},"key":{"_id":{"$numberInt":"1"}},"name":"_id_"},{"v":{"$numberInt":"2"},"key":{"a":{"$numberInt":"1"}},"name":"a_1","partialFilterExpression":{"a":{"$gte":{"$numberInt":"5"}}}}]}],"outcome":[{"collectionName":"coll","databaseName":"index-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"s":"x"},{"_id":{"$numberInt":"2"},"a":{"$numberInt":"5"},"s":"y"},{"_id":{"$numberInt":"3"},"s":"z"},{"_id":{"$numberInt":"4"},"a":{"$numberInt":"5"},"s":"x"}]}]},{"description":"a query the filter covers still answers every match","operations":[{"object":"collection0","name":"createIndex","arguments":{"keys":{"a":{"$numberInt":"1"}},"partialFilterExpression":{"a":{"$gte":{"$numberInt":"5"}}}},"expectResult":"a_1"},{"object":"collection0","name":"find","arguments":{"filter":{"a":{"$gte":{"$numberInt":"5"}}},"sort":{"_id":{"$numberInt":"1"}}},"expectResult":[{"_id":{"$numberInt":"2"},"a":{"$numberInt":"5"},"s":"y"},{"_id":{"$numberInt":"4"},"a":{"$numberInt":"5"},"s":"x"}]}],"outcome":[{"collectionName":"coll","databaseName":"index-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"s":"x"},{"_id":{"$numberInt":"2"},"a":{"$numberInt":"5"},"s":"y"},{"_id":{"$numberInt":"3"},"s":"z"},{"_id":{"$numberInt":"4"},"a":{"$numberInt":"5"},"s":"x"}]}]},{"description":"a query the filter excludes still answers every match","operations":[{"object":"collection0","name":"createIndex","arguments":{"keys":{"a":{"$numberInt":"1"}},"partialFilterExpression":{"a":{"$gte":{"$numberInt":"5"}}}},"expectResult":"a_1"},{"object":"collection0","name":"find","arguments":{"filter":{"a":{"$numberInt":"1"}},"sort":{"_id":{"$numberInt":"1"}}},"expectResult":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"s":"x"}]}],"outcome":[{"collectionName":"coll","databaseName":"index-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"s":"x"},{"_id":{"$numberInt":"2"},"a":{"$numberInt":"5"},"s":"y"},{"_id":{"$numberInt":"3"},"s":"z"},{"_id":{"$numberInt":"4"},"a":{"$numberInt":"5"},"s":"x"}]}]},{"description":"a query straddling the filter still answers every match","operations":[{"object":"collection0","name":"createIndex","arguments":{"keys":{"a":{"$numberInt":"1"}},"partialFilterExpression":{"a":{"$gte":{"$numberInt":"5"}}}},"expectResult":"a_1"},{"object":"collection0","name":"find","arguments":{"filter":{"a":{"$gte":{"$numberInt":"1"}}},"sort":{"_id":{"$numberInt":"1"}}},"expectResult":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"s":"x"},{"_id":{"$numberInt":"2"},"a":{"$numberInt":"5"},"s":"y"},{"_id":{"$numberInt":"4"},"a":{"$numberInt":"5"},"s":"x"}]},{"object":"collection0","name":"countDocuments","arguments":{"filter":{}},"expectResult":{"$numberInt":"4"}}],"outcome":[{"collectionName":"coll","databaseName":"index-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"s":"x"},{"_id":{"$numberInt":"2"},"a":{"$numberInt":"5"},"s":"y"},{"_id":{"$numberInt":"3"},"s":"z"},{"_id":{"$numberInt":"4"},"a":{"$numberInt":"5"},"s":"x"}]}]},{"description":"a unique partial index accepts duplicates its filter excludes","operations":[{"object":"collection0","name":"deleteMany","arguments":{"filter":{}}},{"object":"collection0","name":"createIndex","arguments":{"keys":{"a":{"$numberInt":"1"}},"unique":true,"partialFilterExpression":{"t":true}},"expectResult":"a_1"},{"object":"collection0","name":"insertMany","arguments":{"documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"t":false},{"_id":{"$numberInt":"2"},"a":{"$numberInt":"1"},"t":false}]},"expectResult":{"insertedCount":{"$numberInt":"2"},"insertedIds":{"0":{"$numberInt":"1"},"1":{"$numberInt":"2"}}}},{"object":"collection0","name":"find","arguments":{"filter":{},"sort":{"_id":{"$numberInt":"1"}}},"expectResult":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"t":false},{"_id":{"$numberInt":"2"},"a":{"$numberInt":"1"},"t":false}]}],"outcome":[{"collectionName":"coll","databaseName":"index-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"t":false},{"_id":{"$numberInt":"2"},"a":{"$numberInt":"1"},"t":false}]}]},{"description":"a unique partial index refuses duplicates its filter selects","operations":[{"object":"collection0","name":"deleteMany","arguments":{"filter":{}}},{"object":"collection0","name":"createIndex","arguments":{"keys":{"a":{"$numberInt":"1"}},"unique":true,"partialFilterExpression":{"t":true}},"expectResult":"a_1"},{"object":"collection0","name":"insertMany","arguments":{"documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"9"},"t":true},{"_id":{"$numberInt":"2"},"a":{"$numberInt":"9"},"t":true}]},"expectError":{"isError":true,"errorCode":{"$numberInt":"11000"}}}],"outcome":[{"collectionName":"coll","databaseName":"index-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"9"},"t":true}]}]},{"description":"a document leaving the filter frees the value it held","operations":[{"object":"collection0","name":"deleteMany","arguments":{"filter":{}}},{"object":"collection0","name":"createIndex","arguments":{"keys":{"a":{"$numberInt":"1"}},"unique":true,"partialFilterExpression":{"t":true}},"expectResult":"a_1"},{"object":"collection0","name":"insertMany","arguments":{"documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"9"},"t":true}]},"expectResult":{"insertedCount":{"$numberInt":"1"},"insertedIds":{"0":{"$numberInt":"1"}}}},{"object":"collection0","name":"updateOne","arguments":{"filter":{"_id":{"$numberInt":"1"}},"update":{"$set":{"t":false}}},"expectResult":{"matchedCount":{"$numberInt":"1"},"modifiedCount":{"$numberInt":"1"},"upsertedCount":{"$numberInt":"0"}}},{"object":"collection0","name":"insertOne","arguments":{"document":{"_id":{"$numberInt":"2"},"a":{"$numberInt":"9"},"t":true}},"expectResult":{"insertedId":{"$numberInt":"2"}}},{"object":"collection0","name":"find","arguments":{"filter":{},"sort":{"_id":{"$numberInt":"1"}}},"expectResult":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"9"},"t":false},{"_id":{"$numberInt":"2"},"a":{"$numberInt":"9"},"t":true}]}],"outcome":[{"collectionName":"coll","databaseName":"index-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"9"},"t":false},{"_id":{"$numberInt":"2"},"a":{"$numberInt":"9"},"t":true}]}]},{"description":"a document entering the filter takes the value it names","operations":[{"object":"collection0","name":"deleteMany","arguments":{"filter":{}}},{"object":"collection0","name":"createIndex","arguments":{"keys":{"a":{"$numberInt":"1"}},"unique":true,"partialFilterExpression":{"t":true}},"expectResult":"a_1"},{"object":"collection0","name":"insertMany","arguments":{"documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"9"},"t":true},{"_id":{"$numberInt":"2"},"a":{"$numberInt":"9"},"t":false}]},"expectResult":{"insertedCount":{"$numberInt":"2"},"insertedIds":{"0":{"$numberInt":"1"},"1":{"$numberInt":"2"}}}},{"object":"collection0","name":"updateOne","arguments":{"filter":{"_id":{"$numberInt":"2"}},"update":{"$set":{"t":true}}},"expectError":{"isError":true,"errorCode":{"$numberInt":"11000"}}}],"outcome":[{"collectionName":"coll","databaseName":"index-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"9"},"t":true},{"_id":{"$numberInt":"2"},"a":{"$numberInt":"9"},"t":false}]}]},{"description":"a filter on $exists","operations":[{"object":"collection0","name":"createIndex","arguments":{"keys":{"a":{"$numberInt":"1"}},"partialFilterExpression":{"a":{"$exists":true}}},"expectResult":"a_1"},{"object":"collection0","name":"listIndexes","arguments":{},"expectResult":[{"v":{"$numberInt":"2"},"key":{"_id":{"$numberInt":"1"}},"name":"_id_"},{"v":{"$numberInt":"2"},"key":{"a":{"$numberInt":"1"}},"name":"a_1","partialFilterExpression":{"a":{"$exists":true}}}]}],"outcome":[{"collectionName":"coll","databaseName":"index-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"s":"x"},{"_id":{"$numberInt":"2"},"a":{"$numberInt":"5"},"s":"y"},{"_id":{"$numberInt":"3"},"s":"z"},{"_id":{"$numberInt":"4"},"a":{"$numberInt":"5"},"s":"x"}]}]},{"description":"a filter with two predicates","operations":[{"object":"collection0","name":"createIndex","arguments":{"keys":{"a":{"$numberInt":"1"}},"partialFilterExpression":{"a":{"$gt":{"$numberInt":"0"}},"s":"x"}},"expectResult":"a_1"},{"object":"collection0","name":"listIndexes","arguments":{},"expectResult":[{"v":{"$numberInt":"2"},"key":{"_id":{"$numberInt":"1"}},"name":"_id_"},{"v":{"$numberInt":"2"},"key":{"a":{"$numberInt":"1"}},"name":"a_1","partialFilterExpression":{"a":{"$gt":{"$numberInt":"0"}},"s":"x"}}]}],"outcome":[{"collectionName":"coll","databaseName":"index-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"s":"x"},{"_id":{"$numberInt":"2"},"a":{"$numberInt":"5"},"s":"y"},{"_id":{"$numberInt":"3"},"s":"z"},{"_id":{"$numberInt":"4"},"a":{"$numberInt":"5"},"s":"x"}]}]},{"description":"a filter with an explicit $and","operations":[{"object":"collection0","name":"createIndex","arguments":{"keys":{"a":{"$numberInt":"1"}},"partialFilterExpression":{"$and":[{"a":{"$gt":{"$numberInt":"0"}}}]}},"expectResult":"a_1"},{"object":"collection0","name":"listIndexes","arguments":{},"expectResult":[{"v":{"$numberInt":"2"},"key":{"_id":{"$numberInt":"1"}},"name":"_id_"},{"v":{"$numberInt":"2"},"key":{"a":{"$numberInt":"1"}},"name":"a_1","partialFilterExpression":{"$and":[{"a":{"$gt":{"$numberInt":"0"}}}]}}]}],"outcome":[{"collectionName":"coll","databaseName":"index-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"s":"x"},{"_id":{"$numberInt":"2"},"a":{"$numberInt":"5"},"s":"y"},{"_id":{"$numberInt":"3"},"s":"z"},{"_id":{"$numberInt":"4"},"a":{"$numberInt":"5"},"s":"x"}]}]},{"description":"a filter on a field the index does not name","operations":[{"object":"collection0","name":"createIndex","arguments":{"keys":{"a":{"$numberInt":"1"}},"partialFilterExpression":{"s":"x"}},"expectResult":"a_1"},{"object":"collection0","name":"listIndexes","arguments":{},"expectResult":[{"v":{"$numberInt":"2"},"key":{"_id":{"$numberInt":"1"}},"name":"_id_"},{"v":{"$numberInt":"2"},"key":{"a":{"$numberInt":"1"}},"name":"a_1","partialFilterExpression":{"s":"x"}}]}],"outcome":[{"collectionName":"coll","databaseName":"index-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"s":"x"},{"_id":{"$numberInt":"2"},"a":{"$numberInt":"5"},"s":"y"},{"_id":{"$numberInt":"3"},"s":"z"},{"_id":{"$numberInt":"4"},"a":{"$numberInt":"5"},"s":"x"}]}]},{"description":"a filter using $regex is refused","operations":[{"object":"collection0","name":"createIndex","arguments":{"keys":{"a":{"$numberInt":"1"}},"partialFilterExpression":{"s":{"$regex":"x"}}},"expectError":{"isError":true,"errorCode":{"$numberInt":"67"}}}],"outcome":[{"collectionName":"coll","databaseName":"index-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"s":"x"},{"_id":{"$numberInt":"2"},"a":{"$numberInt":"5"},"s":"y"},{"_id":{"$numberInt":"3"},"s":"z"},{"_id":{"$numberInt":"4"},"a":{"$numberInt":"5"},"s":"x"}]}]},{"description":"a filter using $ne is refused","operations":[{"object":"collection0","name":"createIndex","arguments":{"keys":{"a":{"$numberInt":"1"}},"partialFilterExpression":{"a":{"$ne":{"$numberInt":"1"}}}},"expectError":{"isError":true,"errorCode":{"$numberInt":"67"}}}],"outcome":[{"collectionName":"coll","databaseName":"index-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"s":"x"},{"_id":{"$numberInt":"2"},"a":{"$numberInt":"5"},"s":"y"},{"_id":{"$numberInt":"3"},"s":"z"},{"_id":{"$numberInt":"4"},"a":{"$numberInt":"5"},"s":"x"}]}]},{"description":"a filter using $in is refused","operations":[{"object":"collection0","name":"createIndex","arguments":{"keys":{"a":{"$numberInt":"1"}},"partialFilterExpression":{"a":{"$in":[{"$numberInt":"1"},{"$numberInt":"2"}]}}},"expectResult":"a_1"}],"outcome":[{"collectionName":"coll","databaseName":"index-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"s":"x"},{"_id":{"$numberInt":"2"},"a":{"$numberInt":"5"},"s":"y"},{"_id":{"$numberInt":"3"},"s":"z"},{"_id":{"$numberInt":"4"},"a":{"$numberInt":"5"},"s":"x"}]}]},{"description":"a filter that is not a document is refused","operations":[{"object":"collection0","name":"createIndex","arguments":{"keys":{"a":{"$numberInt":"1"}},"partialFilterExpression":{"$numberInt":"1"}},"expectError":{"isError":true,"errorCode":{"$numberInt":"14"}}}],"outcome":[{"collectionName":"coll","databaseName":"index-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"s":"x"},{"_id":{"$numberInt":"2"},"a":{"$numberInt":"5"},"s":"y"},{"_id":{"$numberInt":"3"},"s":"z"},{"_id":{"$numberInt":"4"},"a":{"$numberInt":"5"},"s":"x"}]}]},{"description":"an empty filter","operations":[{"object":"collection0","name":"createIndex","arguments":{"keys":{"a":{"$numberInt":"1"}},"partialFilterExpression":{}},"expectResult":"a_1"},{"object":"collection0","name":"listIndexes","arguments":{},"expectResult":[{"v":{"$numberInt":"2"},"key":{"_id":{"$numberInt":"1"}},"name":"_id_"},{"v":{"$numberInt":"2"},"key":{"a":{"$numberInt":"1"}},"name":"a_1","partialFilterExpression":{}}]}],"outcome":[{"collectionName":"coll","databaseName":"index-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"s":"x"},{"_id":{"$numberInt":"2"},"a":{"$numberInt":"5"},"s":"y"},{"_id":{"$numberInt":"3"},"s":"z"},{"_id":{"$numberInt":"4"},"a":{"$numberInt":"5"},"s":"x"}]}]},{"description":"sparse and a partial filter may not be combined","operations":[{"object":"collection0","name":"createIndex","arguments":{"keys":{"a":{"$numberInt":"1"}},"sparse":true,"partialFilterExpression":{"a":{"$gt":{"$numberInt":"0"}}}},"expectError":{"isError":true,"errorCode":{"$numberInt":"67"}}}],"outcome":[{"collectionName":"coll","databaseName":"index-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"s":"x"},{"_id":{"$numberInt":"2"},"a":{"$numberInt":"5"},"s":"y"},{"_id":{"$numberInt":"3"},"s":"z"},{"_id":{"$numberInt":"4"},"a":{"$numberInt":"5"},"s":"x"}]}]},{"description":"a partial filter with a TTL","operations":[{"object":"collection0","name":"createIndex","arguments":{"keys":{"a":{"$numberInt":"1"}},"expireAfterSeconds":{"$numberInt":"100"},"partialFilterExpression":{"a":{"$gt":{"$numberInt":"0"}}}},"expectResult":"a_1"},{"object":"collection0","name":"listIndexes","arguments":{},"expectResult":[{"v":{"$numberInt":"2"},"key":{"_id":{"$numberInt":"1"}},"name":"_id_"},{"v":{"$numberInt":"2"},"key":{"a":{"$numberInt":"1"}},"name":"a_1","partialFilterExpression":{"a":{"$gt":{"$numberInt":"0"}}},"expireAfterSeconds":{"$numberInt":"100"}}]}],"outcome":[{"collectionName":"coll","databaseName":"index-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"s":"x"},{"_id":{"$numberInt":"2"},"a":{"$numberInt":"5"},"s":"y"},{"_id":{"$numberInt":"3"},"s":"z"},{"_id":{"$numberInt":"4"},"a":{"$numberInt":"5"},"s":"x"}]}]},{"description":"a partial index on a compound key","operations":[{"object":"collection0","name":"createIndex","arguments":{"keys":{"a":{"$numberInt":"1"},"s":{"$numberInt":"1"}},"partialFilterExpression":{"a":{"$gt":{"$numberInt":"0"}}}},"expectResult":"a_1_s_1"},{"object":"collection0","name":"listIndexes","arguments":{},"expectResult":[{"v":{"$numberInt":"2"},"key":{"_id":{"$numberInt":"1"}},"name":"_id_"},{"v":{"$numberInt":"2"},"key":{"a":{"$numberInt":"1"},"s":{"$numberInt":"1"}},"name":"a_1_s_1","partialFilterExpression":{"a":{"$gt":{"$numberInt":"0"}}}}]}],"outcome":[{"collectionName":"coll","databaseName":"index-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"s":"x"},{"_id":{"$numberInt":"2"},"a":{"$numberInt":"5"},"s":"y"},{"_id":{"$numberInt":"3"},"s":"z"},{"_id":{"$numberInt":"4"},"a":{"$numberInt":"5"},"s":"x"}]}]},{"description":"two indexes on one key differing only in their filter","operations":[{"object":"collection0","name":"createIndex","arguments":{"keys":{"a":{"$numberInt":"1"}},"name":"one","partialFilterExpression":{"a":{"$gt":{"$numberInt":"0"}}}},"expectResult":"one"},{"object":"collection0","name":"createIndex","arguments":{"keys":{"a":{"$numberInt":"1"}},"name":"two","partialFilterExpression":{"a":{"$gt":{"$numberInt":"4"}}}},"expectResult":"two"},{"object":"collection0","name":"listIndexes","arguments":{},"expectResult":[{"v":{"$numberInt":"2"},"key":{"_id":{"$numberInt":"1"}},"name":"_id_"},{"v":{"$numberInt":"2"},"key":{"a":{"$numberInt":"1"}},"name":"one","partialFilterExpression":{"a":{"$gt":{"$numberInt":"0"}}}},{"v":{"$numberInt":"2"},"key":{"a":{"$numberInt":"1"}},"name":"two","partialFilterExpression":{"a":{"$gt":{"$numberInt":"4"}}}}]}],"outcome":[{"collectionName":"coll","databaseName":"index-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"s":"x"},{"_id":{"$numberInt":"2"},"a":{"$numberInt":"5"},"s":"y"},{"_id":{"$numberInt":"3"},"s":"z"},{"_id":{"$numberInt":"4"},"a":{"$numberInt":"5"},"s":"x"}]}]},{"description":"the same index twice is idempotent","operations":[{"object":"collection0","name":"createIndex","arguments":{"keys":{"a":{"$numberInt":"1"}},"partialFilterExpression":{"a":{"$gt":{"$numberInt":"0"}}}},"expectResult":"a_1"},{"object":"collection0","name":"createIndex","arguments":{"keys":{"a":{"$numberInt":"1"}},"partialFilterExpression":{"a":{"$gt":{"$numberInt":"0"}}}},"expectResult":"a_1"},{"object":"collection0","name":"listIndexes","arguments":{},"expectResult":[{"v":{"$numberInt":"2"},"key":{"_id":{"$numberInt":"1"}},"name":"_id_"},{"v":{"$numberInt":"2"},"key":{"a":{"$numberInt":"1"}},"name":"a_1","partialFilterExpression":{"a":{"$gt":{"$numberInt":"0"}}}}]}],"outcome":[{"collectionName":"coll","databaseName":"index-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"s":"x"},{"_id":{"$numberInt":"2"},"a":{"$numberInt":"5"},"s":"y"},{"_id":{"$numberInt":"3"},"s":"z"},{"_id":{"$numberInt":"4"},"a":{"$numberInt":"5"},"s":"x"}]}]},{"description":"the same name with a different filter conflicts","operations":[{"object":"collection0","name":"createIndex","arguments":{"keys":{"a":{"$numberInt":"1"}},"partialFilterExpression":{"a":{"$gt":{"$numberInt":"0"}}}},"expectResult":"a_1"},{"object":"collection0","name":"createIndex","arguments":{"keys":{"a":{"$numberInt":"1"}},"partialFilterExpression":{"a":{"$gt":{"$numberInt":"4"}}}},"expectError":{"isError":true,"errorCode":{"$numberInt":"86"}}}],"outcome":[{"collectionName":"coll","databaseName":"index-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"s":"x"},{"_id":{"$numberInt":"2"},"a":{"$numberInt":"5"},"s":"y"},{"_id":{"$numberInt":"3"},"s":"z"},{"_id":{"$numberInt":"4"},"a":{"$numberInt":"5"},"s":"x"}]}]},{"description":"a partial index is dropped by name","operations":[{"object":"collection0","name":"createIndex","arguments":{"keys":{"a":{"$numberInt":"1"}},"name":"pa","partialFilterExpression":{"a":{"$gt":{"$numberInt":"0"}}}},"expectResult":"pa"},{"object":"collection0","name":"dropIndex","arguments":{"name":"pa"},"expectResult":{}},{"object":"collection0","name":"listIndexes","arguments":{},"expectResult":[{"v":{"$numberInt":"2"},"key":{"_id":{"$numberInt":"1"}},"name":"_id_"}]}],"outcome":[{"collectionName":"coll","databaseName":"index-corpus","documents":[{"_id":{"$numberInt":"1"},"a":{"$numberInt":"1"},"s":"x"},{"_id":{"$numberInt":"2"},"a":{"$numberInt":"5"},"s":"y"},{"_id":{"$numberInt":"3"},"s":"z"},{"_id":{"$numberInt":"4"},"a":{"$numberInt":"5"},"s":"x"}]}]}]} diff --git a/tests/spec/indexes/record.js b/tests/spec/indexes/record.js new file mode 100644 index 0000000..b67e8d0 --- /dev/null +++ b/tests/spec/indexes/record.js @@ -0,0 +1,186 @@ +// Record an index corpus by asking a real mongod what the answer is. +// +// Partial and hashed indexes: M3's last row. `docs/M3_INDEX_TYPES_DESIGN_REVIEW.md` +// found that `partialFilterExpression` was accepted and ignored here, which +// made a *unique* partial index refuse inserts mongod accepts -- and that no +// test in this repository covered the row, in any suite. The pinned corpus is +// crud and aggregate; `e2e5.js` and `e2e6.js` test indexes but write neither +// a partial nor a hashed spec. +// +// Unlike `tests/spec/operators/`, a case here is a *sequence*: create an +// index, insert against it, read back, list it. So the source format carries +// a list of operations rather than one, and the recorder walks them in order, +// stopping at the first that throws -- which is what a client would see. +// +// node tests/spec/indexes/record.js --mongod-port 27099 +// +// Options: +// --mongod-port a running mongod to measure against (default 27099) +// --only record just one source file +const fs = require('fs'); +const path = require('path'); +const DRIVER = path.join(__dirname, '..', '..', 'e2e', 'node_modules', 'mongodb'); +const { MongoClient } = require(DRIVER); +const { EJSON, ObjectId } = require(path.join(DRIVER, 'lib', 'bson.js')); + +const argv = process.argv.slice(2); +function opt(name, dflt) { + const i = argv.indexOf('--' + name); + if (i < 0) return dflt; + const v = argv[i + 1]; + return v === undefined || v.startsWith('--') ? true : v; +} + +const PORT = parseInt(opt('mongod-port', '27099'), 10); +const ONLY = opt('only', null); +const SRC_DIR = path.join(__dirname, 'sources'); +const DB_NAME = 'index-corpus'; +const COLL = 'coll'; +const SCHEMA_VERSION = '1.4'; + +/// An ObjectId is the one value no source authors, so masking it is +/// unambiguous. Nothing else here is unpredictable: an index name, a key +/// specification and a document count are all decided by the input. +function mask(value) { + if (value instanceof ObjectId) return { $$type: 'objectId' }; + if (Array.isArray(value)) return value.map(mask); + if (value && typeof value === 'object' && value.constructor === Object) { + const out = {}; + for (const [k, v] of Object.entries(value)) out[k] = mask(v); + return out; + } + return value; +} + +async function main() { + const client = new MongoClient(`mongodb://127.0.0.1:${PORT}`, { serverSelectionTimeoutMS: 3000 }); + try { + await client.connect(); + } catch (e) { + console.error(`no mongod on :${PORT} -- start one first:\n` + + ` mongod --port ${PORT} --dbpath \n${e.message}`); + process.exit(2); + } + const build = await client.db('admin').command({ buildInfo: 1 }); + console.log(`recording against mongod ${build.version} on :${PORT}`); + + const sources = fs.readdirSync(SRC_DIR).filter((f) => f.endsWith('.json')) + .filter((f) => !ONLY || f === ONLY || f === ONLY + '.json') + .sort(); + for (const file of sources) { + const src = JSON.parse(fs.readFileSync(path.join(SRC_DIR, file), 'utf8')); + const name = path.basename(file, '.json'); + const out = await record(client, name, src); + fs.writeFileSync(path.join(__dirname, `${name}.json`), EJSON.stringify(out, { relaxed: false, indent: 2 }) + '\n'); + const errs = out.tests.filter((t) => t.operations.some((o) => o.expectError)).length; + console.log(` ${name}: ${out.tests.length} cases, ${errs} of them reaching an error`); + } + await client.close(); + console.log('RECORDED'); +} + +async function record(client, name, src) { + const db = client.db(DB_NAME); + const coll = db.collection(COLL); + const tests = []; + + for (const c of src.cases) { + const documents = c.documents === undefined ? src.documents : c.documents; + // Dropped rather than emptied: an index outlives a `deleteMany`, and + // every case here is about which indexes exist. + await coll.drop().catch(() => {}); + if (documents.length) await coll.insertMany(structuredClone(documents)); + + const operations = []; + // A case that brings its own documents reseeds through operations, + // because `initialData` is per file and seeded once per test. + if (c.documents !== undefined) { + operations.push({ object: 'collection0', name: 'deleteMany', arguments: { filter: {} } }); + if (documents.length) { + operations.push({ + object: 'collection0', + name: 'insertMany', + arguments: { documents: structuredClone(documents) }, + }); + } + } + for (const op of c.ops) { + const rec = { object: 'collection0', name: op.name, arguments: structuredClone(op.arguments) }; + try { + rec.expectResult = mask(await invoke(coll, op.name, structuredClone(op.arguments))); + } catch (e) { + // The code, not the message. mongod's index-specification + // errors embed a rendering of the whole spec, which no + // formatter here produces. + delete rec.expectResult; + rec.expectError = { isError: true, errorCode: e.code }; + operations.push(rec); + break; // a client would stop here too + } + operations.push(rec); + } + + const after = await coll.find({}, { sort: { _id: 1 } }).toArray(); + tests.push({ + description: c.description, + operations, + outcome: [{ collectionName: COLL, databaseName: DB_NAME, documents: after.map(mask) }], + }); + } + await coll.drop().catch(() => {}); + + return { + description: name, + schemaVersion: SCHEMA_VERSION, + // Recorded, not authored. Regenerate with tests/spec/indexes/record.js. + createEntities: [ + { client: { id: 'client0' } }, + { database: { id: 'database0', client: 'client0', databaseName: DB_NAME } }, + { collection: { id: 'collection0', database: 'database0', collectionName: COLL } }, + ], + initialData: [{ collectionName: COLL, databaseName: DB_NAME, documents: src.documents }], + tests, + }; +} + +async function invoke(coll, name, args) { + switch (name) { + case 'createIndex': { + const { keys, ...rest } = args; + return await coll.createIndex(keys, rest); + } + case 'listIndexes': + return await coll.listIndexes().toArray(); + case 'dropIndex': + return plain(await coll.dropIndex(args.name)); + case 'insertMany': + return plain(await coll.insertMany(args.documents)); + case 'insertOne': + return plain(await coll.insertOne(args.document)); + case 'find': + return await coll.find(args.filter || {}, { sort: args.sort }).toArray(); + case 'countDocuments': + return await coll.countDocuments(args.filter || {}); + case 'deleteMany': + return plain(await coll.deleteMany(args.filter || {})); + case 'updateOne': + return plain(await coll.updateOne(args.filter, args.update, args)); + default: + throw new Error(`record.js does not know the operation '${name}'`); + } +} + +/// Driver results carry `acknowledged` and driver-side types; the unified +/// format's expectations are the plain counts and ids. +function plain(result) { + const out = {}; + for (const k of ['insertedCount', 'insertedId', 'insertedIds', 'deletedCount', 'matchedCount', 'modifiedCount', 'upsertedCount']) { + if (result[k] !== undefined) out[k] = result[k]; + } + return out; +} + +main().catch((e) => { + console.error('RECORD_FAIL', e); + process.exit(1); +}); diff --git a/tests/spec/indexes/sources/hashed.json b/tests/spec/indexes/sources/hashed.json new file mode 100644 index 0000000..db5ca90 --- /dev/null +++ b/tests/spec/indexes/sources/hashed.json @@ -0,0 +1,146 @@ +{ + "_comment": [ + "Inputs only. Expectations are measured -- see record.js.", + "Hashed indexes. The tree is ordered by a hash, so it answers equality and", + "nothing else; every read case here exists to pin that a range query and a", + "sort still come back *correct*, because the planner declining to use the", + "index is the only thing that makes them so." + ], + "documents": [ + { "_id": 1, "a": 1, "s": "x" }, + { "_id": 2, "a": 5, "s": "y" }, + { "_id": 3, "s": "z" }, + { "_id": 4, "a": 5, "s": "x" } + ], + "cases": [ + { + "description": "a hashed index is created and names itself for the plugin", + "ops": [ + { "name": "createIndex", "arguments": { "keys": { "a": "hashed" } } }, + { "name": "listIndexes", "arguments": {} } + ] + }, + { + "description": "an equality query answers every match", + "ops": [ + { "name": "createIndex", "arguments": { "keys": { "a": "hashed" } } }, + { "name": "find", "arguments": { "filter": { "a": 5 }, "sort": { "_id": 1 } } } + ] + }, + { + "description": "equality against a value nothing holds", + "ops": [ + { "name": "createIndex", "arguments": { "keys": { "a": "hashed" } } }, + { "name": "find", "arguments": { "filter": { "a": 99 }, "sort": { "_id": 1 } } } + ] + }, + { + "description": "equality against a missing field", + "ops": [ + { "name": "createIndex", "arguments": { "keys": { "a": "hashed" } } }, + { "name": "find", "arguments": { "filter": { "a": null }, "sort": { "_id": 1 } } } + ] + }, + { + "description": "a range query still answers every match", + "ops": [ + { "name": "createIndex", "arguments": { "keys": { "a": "hashed" } } }, + { "name": "find", "arguments": { "filter": { "a": { "$gte": 5 } }, "sort": { "_id": 1 } } } + ] + }, + { + "description": "a sort on the hashed field is by value, not by hash", + "ops": [ + { "name": "createIndex", "arguments": { "keys": { "a": "hashed" } } }, + { "name": "find", "arguments": { "filter": {}, "sort": { "a": 1 } } } + ] + }, + { + "description": "equality across numeric types", + "ops": [ + { "name": "createIndex", "arguments": { "keys": { "a": "hashed" } } }, + { "name": "find", "arguments": { "filter": { "a": 5.0 }, "sort": { "_id": 1 } } } + ] + }, + { + "description": "inserting after the index exists", + "ops": [ + { "name": "createIndex", "arguments": { "keys": { "a": "hashed" } } }, + { "name": "insertOne", "arguments": { "document": { "_id": 5, "a": 5 } } }, + { "name": "find", "arguments": { "filter": { "a": 5 }, "sort": { "_id": 1 } } } + ] + }, + { + "description": "deleting through a hashed index", + "ops": [ + { "name": "createIndex", "arguments": { "keys": { "a": "hashed" } } }, + { "name": "deleteMany", "arguments": { "filter": { "a": 5 } } }, + { "name": "find", "arguments": { "filter": {}, "sort": { "_id": 1 } } } + ] + }, + { + "description": "a hashed component beside a range one", + "ops": [ + { "name": "createIndex", "arguments": { "keys": { "a": "hashed", "s": 1 } } }, + { "name": "listIndexes", "arguments": {} } + ] + }, + { + "description": "two hashed components are refused", + "ops": [ + { "name": "createIndex", "arguments": { "keys": { "a": "hashed", "s": "hashed" } } } + ] + }, + { + "description": "a unique hashed index is refused", + "ops": [ + { "name": "createIndex", "arguments": { "keys": { "a": "hashed" }, "unique": true } } + ] + }, + { + "description": "an array at the hashed path is refused, at insert time", + "documents": [], + "ops": [ + { "name": "createIndex", "arguments": { "keys": { "a": "hashed" } } }, + { "name": "insertOne", "arguments": { "document": { "_id": 1, "a": [1, 2] } } } + ] + }, + { + "description": "a key direction that is neither 1, -1 nor hashed", + "ops": [ + { "name": "createIndex", "arguments": { "keys": { "a": "bogus" } } } + ] + }, + { + "description": "a sparse hashed index", + "ops": [ + { "name": "createIndex", "arguments": { "keys": { "a": "hashed" }, "sparse": true } }, + { "name": "listIndexes", "arguments": {} } + ] + }, + { + "description": "the same hashed index twice is idempotent", + "ops": [ + { "name": "createIndex", "arguments": { "keys": { "a": "hashed" } } }, + { "name": "createIndex", "arguments": { "keys": { "a": "hashed" } } }, + { "name": "listIndexes", "arguments": {} } + ] + }, + { + "description": "a hashed index beside an ascending one on the same field", + "ops": [ + { "name": "createIndex", "arguments": { "keys": { "a": 1 } } }, + { "name": "createIndex", "arguments": { "keys": { "a": "hashed" } } }, + { "name": "listIndexes", "arguments": {} } + ] + }, + { + "description": "a hashed index is dropped by name", + "ops": [ + { "name": "createIndex", "arguments": { "keys": { "a": "hashed" } } }, + { "name": "dropIndex", "arguments": { "name": "a_hashed" } }, + { "name": "listIndexes", "arguments": {} } + ] + } + ] +} diff --git a/tests/spec/indexes/sources/partial.json b/tests/spec/indexes/sources/partial.json new file mode 100644 index 0000000..b1daade --- /dev/null +++ b/tests/spec/indexes/sources/partial.json @@ -0,0 +1,193 @@ +{ + "_comment": [ + "Inputs only. Expectations are measured -- see record.js.", + "`partialFilterExpression`: which filters may be written, what the index", + "then reports about itself, and the one behaviour that made ignoring the", + "option a wrong answer rather than a missing feature -- a unique partial", + "index constrains only the documents its filter selects." + ], + "documents": [ + { "_id": 1, "a": 1, "s": "x" }, + { "_id": 2, "a": 5, "s": "y" }, + { "_id": 3, "s": "z" }, + { "_id": 4, "a": 5, "s": "x" } + ], + "cases": [ + { + "description": "a partial index is created and reports its filter", + "ops": [ + { "name": "createIndex", "arguments": { "keys": { "a": 1 }, "partialFilterExpression": { "a": { "$gte": 5 } } } }, + { "name": "listIndexes", "arguments": {} } + ] + }, + { + "description": "a query the filter covers still answers every match", + "ops": [ + { "name": "createIndex", "arguments": { "keys": { "a": 1 }, "partialFilterExpression": { "a": { "$gte": 5 } } } }, + { "name": "find", "arguments": { "filter": { "a": { "$gte": 5 } }, "sort": { "_id": 1 } } } + ] + }, + { + "description": "a query the filter excludes still answers every match", + "ops": [ + { "name": "createIndex", "arguments": { "keys": { "a": 1 }, "partialFilterExpression": { "a": { "$gte": 5 } } } }, + { "name": "find", "arguments": { "filter": { "a": 1 }, "sort": { "_id": 1 } } } + ] + }, + { + "description": "a query straddling the filter still answers every match", + "ops": [ + { "name": "createIndex", "arguments": { "keys": { "a": 1 }, "partialFilterExpression": { "a": { "$gte": 5 } } } }, + { "name": "find", "arguments": { "filter": { "a": { "$gte": 1 } }, "sort": { "_id": 1 } } }, + { "name": "countDocuments", "arguments": { "filter": {} } } + ] + }, + { + "description": "a unique partial index accepts duplicates its filter excludes", + "documents": [], + "ops": [ + { "name": "createIndex", "arguments": { "keys": { "a": 1 }, "unique": true, "partialFilterExpression": { "t": true } } }, + { "name": "insertMany", "arguments": { "documents": [{ "_id": 1, "a": 1, "t": false }, { "_id": 2, "a": 1, "t": false }] } }, + { "name": "find", "arguments": { "filter": {}, "sort": { "_id": 1 } } } + ] + }, + { + "description": "a unique partial index refuses duplicates its filter selects", + "documents": [], + "ops": [ + { "name": "createIndex", "arguments": { "keys": { "a": 1 }, "unique": true, "partialFilterExpression": { "t": true } } }, + { "name": "insertMany", "arguments": { "documents": [{ "_id": 1, "a": 9, "t": true }, { "_id": 2, "a": 9, "t": true }] } } + ] + }, + { + "description": "a document leaving the filter frees the value it held", + "documents": [], + "ops": [ + { "name": "createIndex", "arguments": { "keys": { "a": 1 }, "unique": true, "partialFilterExpression": { "t": true } } }, + { "name": "insertMany", "arguments": { "documents": [{ "_id": 1, "a": 9, "t": true }] } }, + { "name": "updateOne", "arguments": { "filter": { "_id": 1 }, "update": { "$set": { "t": false } } } }, + { "name": "insertOne", "arguments": { "document": { "_id": 2, "a": 9, "t": true } } }, + { "name": "find", "arguments": { "filter": {}, "sort": { "_id": 1 } } } + ] + }, + { + "description": "a document entering the filter takes the value it names", + "documents": [], + "ops": [ + { "name": "createIndex", "arguments": { "keys": { "a": 1 }, "unique": true, "partialFilterExpression": { "t": true } } }, + { "name": "insertMany", "arguments": { "documents": [{ "_id": 1, "a": 9, "t": true }, { "_id": 2, "a": 9, "t": false }] } }, + { "name": "updateOne", "arguments": { "filter": { "_id": 2 }, "update": { "$set": { "t": true } } } } + ] + }, + { + "description": "a filter on $exists", + "ops": [ + { "name": "createIndex", "arguments": { "keys": { "a": 1 }, "partialFilterExpression": { "a": { "$exists": true } } } }, + { "name": "listIndexes", "arguments": {} } + ] + }, + { + "description": "a filter with two predicates", + "ops": [ + { "name": "createIndex", "arguments": { "keys": { "a": 1 }, "partialFilterExpression": { "a": { "$gt": 0 }, "s": "x" } } }, + { "name": "listIndexes", "arguments": {} } + ] + }, + { + "description": "a filter with an explicit $and", + "ops": [ + { "name": "createIndex", "arguments": { "keys": { "a": 1 }, "partialFilterExpression": { "$and": [{ "a": { "$gt": 0 } }] } } }, + { "name": "listIndexes", "arguments": {} } + ] + }, + { + "description": "a filter on a field the index does not name", + "ops": [ + { "name": "createIndex", "arguments": { "keys": { "a": 1 }, "partialFilterExpression": { "s": "x" } } }, + { "name": "listIndexes", "arguments": {} } + ] + }, + { + "description": "a filter using $regex is refused", + "ops": [ + { "name": "createIndex", "arguments": { "keys": { "a": 1 }, "partialFilterExpression": { "s": { "$regex": "x" } } } } + ] + }, + { + "description": "a filter using $ne is refused", + "ops": [ + { "name": "createIndex", "arguments": { "keys": { "a": 1 }, "partialFilterExpression": { "a": { "$ne": 1 } } } } + ] + }, + { + "description": "a filter using $in is refused", + "ops": [ + { "name": "createIndex", "arguments": { "keys": { "a": 1 }, "partialFilterExpression": { "a": { "$in": [1, 2] } } } } + ] + }, + { + "description": "a filter that is not a document is refused", + "ops": [ + { "name": "createIndex", "arguments": { "keys": { "a": 1 }, "partialFilterExpression": 1 } } + ] + }, + { + "description": "an empty filter", + "ops": [ + { "name": "createIndex", "arguments": { "keys": { "a": 1 }, "partialFilterExpression": {} } }, + { "name": "listIndexes", "arguments": {} } + ] + }, + { + "description": "sparse and a partial filter may not be combined", + "ops": [ + { "name": "createIndex", "arguments": { "keys": { "a": 1 }, "sparse": true, "partialFilterExpression": { "a": { "$gt": 0 } } } } + ] + }, + { + "description": "a partial filter with a TTL", + "ops": [ + { "name": "createIndex", "arguments": { "keys": { "a": 1 }, "expireAfterSeconds": 100, "partialFilterExpression": { "a": { "$gt": 0 } } } }, + { "name": "listIndexes", "arguments": {} } + ] + }, + { + "description": "a partial index on a compound key", + "ops": [ + { "name": "createIndex", "arguments": { "keys": { "a": 1, "s": 1 }, "partialFilterExpression": { "a": { "$gt": 0 } } } }, + { "name": "listIndexes", "arguments": {} } + ] + }, + { + "description": "two indexes on one key differing only in their filter", + "ops": [ + { "name": "createIndex", "arguments": { "keys": { "a": 1 }, "name": "one", "partialFilterExpression": { "a": { "$gt": 0 } } } }, + { "name": "createIndex", "arguments": { "keys": { "a": 1 }, "name": "two", "partialFilterExpression": { "a": { "$gt": 4 } } } }, + { "name": "listIndexes", "arguments": {} } + ] + }, + { + "description": "the same index twice is idempotent", + "ops": [ + { "name": "createIndex", "arguments": { "keys": { "a": 1 }, "partialFilterExpression": { "a": { "$gt": 0 } } } }, + { "name": "createIndex", "arguments": { "keys": { "a": 1 }, "partialFilterExpression": { "a": { "$gt": 0 } } } }, + { "name": "listIndexes", "arguments": {} } + ] + }, + { + "description": "the same name with a different filter conflicts", + "ops": [ + { "name": "createIndex", "arguments": { "keys": { "a": 1 }, "partialFilterExpression": { "a": { "$gt": 0 } } } }, + { "name": "createIndex", "arguments": { "keys": { "a": 1 }, "partialFilterExpression": { "a": { "$gt": 4 } } } } + ] + }, + { + "description": "a partial index is dropped by name", + "ops": [ + { "name": "createIndex", "arguments": { "keys": { "a": 1 }, "name": "pa", "partialFilterExpression": { "a": { "$gt": 0 } } } }, + { "name": "dropIndex", "arguments": { "name": "pa" } }, + { "name": "listIndexes", "arguments": {} } + ] + } + ] +} diff --git a/tests/spec/run.js b/tests/spec/run.js index 11c5143..67322c2 100644 --- a/tests/spec/run.js +++ b/tests/spec/run.js @@ -45,8 +45,9 @@ function opt(name, dflt) { const VERBOSE = !!opt('verbose', false); // The corpus. Defaults to the pinned crud suite; `--suite-dir` points the same // runner at another one, which is how `tests/spec/aggregate/`, -// `tests/spec/positional/` and `tests/spec/operators/` are run -- each exists -// because the pinned suite has a hole where a whole feature should be. Sharing +// `tests/spec/positional/`, `tests/spec/operators/` and `tests/spec/indexes/` +// are run -- each exists because the pinned suite has a hole where a whole +// feature should be. Sharing // the runner rather than writing a second one is the point: the entity model, // the matchers, the skip accounting and `expectEvents` all come for free, and a // second runner would drift from this one exactly where it mattered.