M3: the index corpus, recorded red #11

dev · 2026-08-10T19:34:08Z

dev commented

2026-08-10 19:34:08 +00:00

Step 2 of the order docs/M3_INDEX_TYPES_DESIGN_REVIEW.md set out: the gate,
recorded, before the implementation it measures.

42 cases in two files, red at 3/39.

This is the first of the four recorded corpora here whose subject nothing in
the repository tested at all. The pinned suite is crud and aggregate;
tests/e2e/e2e5.js and e2e6.js test indexes and write neither a partial nor
a hashed spec. The row that made the review urgent -- a unique partial index
refusing an insert mongod accepts -- was covered by no test, in any suite.

The three that pass are the reads a partial index does not change: this server
indexes every document, so a query still finds everything. That is exactly why
the review called this a smaller fire than arrayFilters, and it is worth
having the corpus say so rather than the prose.

A case here is a sequence

Create an index, insert against it, read back, list it. An index outlives a
deleteMany and every case is about which indexes exist, so the recorder
drops the collection between cases and walks each case's operations in order,
stopping at the first that throws -- which is what a client would see.

That is a different shape from tests/spec/operators/, where a case is one
update, and it is why this corpus brings its own recorder rather than
extending that one.

Recording it corrected the review twice

Which is the argument for recording rather than reasoning, and the same thing
happened when the positional corpus was recorded against its own review.

$in in a partial filter is allowed. The review grouped it with $ne
and $regex, which are 67.
Same key, different filter, no explicit name is IndexKeySpecsConflict
(86), not the 67 the review assumed.

What it pins for the implementation

a unique partial index constrains only the documents its filter selects: two
{a: 1, t: false} are accepted, two {a: 9, t: true} are E11000;
a document leaving the filter frees the value it held, for another to
take;
a document entering it must take a value nothing inside holds, or the
update is E11000;
sparse and partialFilterExpression may not be combined (67);
expireAfterSeconds and a filter may, and listIndexes reports the filter
before the expiry;
an empty filter is legal and is reported;
two hashed components is 31303, unique on a hashed index is 16764, and an
array at a hashed path is 16766 at insert time rather than at creation;
a range query or a sort over a hashed field still returns the right answer,
because the planner declines the index rather than misusing it.

Verification

Nothing else moves: 249/249 unit tests, operators 125/0, positional 51/0,
aggregation 70/0, pinned crud 228/63/196.

Step 2 of the order `docs/M3_INDEX_TYPES_DESIGN_REVIEW.md` set out: the gate, recorded, before the implementation it measures. 42 cases in two files, **red at 3/39**. This is the first of the four recorded corpora here whose subject nothing in the repository tested at all. The pinned suite is crud and aggregate; `tests/e2e/e2e5.js` and `e2e6.js` test indexes and write neither a partial nor a hashed spec. The row that made the review urgent -- a unique partial index refusing an insert mongod accepts -- was covered by no test, in any suite. The three that pass are the reads a partial index does not change: this server indexes every document, so a query still finds everything. That is exactly why the review called this a smaller fire than `arrayFilters`, and it is worth having the corpus say so rather than the prose. ## A case here is a sequence Create an index, insert against it, read back, list it. An index outlives a `deleteMany` and every case is about which indexes exist, so the recorder drops the collection between cases and walks each case's operations in order, stopping at the first that throws -- which is what a client would see. That is a different shape from `tests/spec/operators/`, where a case is one update, and it is why this corpus brings its own recorder rather than extending that one. ## Recording it corrected the review twice Which is the argument for recording rather than reasoning, and the same thing happened when the positional corpus was recorded against its own review. - **`$in` in a partial filter is allowed.** The review grouped it with `$ne` and `$regex`, which are 67. - **Same key, different filter, no explicit name is IndexKeySpecsConflict (86)**, not the 67 the review assumed. ## What it pins for the implementation - a unique partial index constrains only the documents its filter selects: two `{a: 1, t: false}` are accepted, two `{a: 9, t: true}` are E11000; - a document **leaving** the filter frees the value it held, for another to take; - a document **entering** it must take a value nothing inside holds, or the update is E11000; - `sparse` and `partialFilterExpression` may not be combined (67); - `expireAfterSeconds` and a filter may, and `listIndexes` reports the filter *before* the expiry; - an empty filter is legal and is reported; - two hashed components is 31303, `unique` on a hashed index is 16764, and an array at a hashed path is 16766 **at insert time** rather than at creation; - a range query or a sort over a hashed field still returns the right answer, because the planner declines the index rather than misusing it. ## Verification Nothing else moves: 249/249 unit tests, operators 125/0, positional 51/0, aggregation 70/0, pinned crud 228/63/196. ## Next Step 3: partial indexes, maintained and `unique`-enforcing, with the planner declining to read from them until the implication test exists. Then hashed, equality-only. Both need one catalog field, and `write_index_catalog`'s flags byte has spare bits, so `catalog_version` stays 1.

dev added 2 commits 2026-08-10 19:34:09 +00:00

tests/spec: a recorded corpus for partial and hashed indexes 879a6bb07b

M3's last row, and the first of the four corpora here whose subject nothing in
the repository tested at all: the pinned suite is crud and aggregate, and
`e2e5.js`/`e2e6.js` write neither a partial nor a hashed spec.

42 cases in two files, recorded red at 3/39. The three that pass are the reads
a partial index does not change -- this server indexes every document, so a
query still finds everything, which is exactly why the review called this a
smaller fire than `arrayFilters`.

A case here is a *sequence* rather than one operation: create an index, insert
against it, read back, list it. So the recorder walks a case's operations in
order and stops at the first that throws, which is what a client would see,
and drops the collection between cases because an index outlives a
`deleteMany`.

Two of the design review's own guesses were wrong, which is the argument for
recording rather than reasoning:

  - **`$in` in a partial filter is allowed.** The review grouped it with `$ne`
    and `$regex`, which are 67.
  - **Same key, different filter, no explicit name is IndexKeySpecsConflict
    (86)**, not the 67 the review assumed.

What it confirmed, and what the implementation now has to satisfy: a unique
partial index constrains only the documents its filter selects; a document
*leaving* the filter frees the value it held; a document *entering* it must
take a value nothing inside holds, or the update is E11000. `sparse` and
`partialFilterExpression` may not be combined (67). `expireAfterSeconds` and a
filter may, and `listIndexes` reports the filter before the expiry. Two hashed
components is 31303, `unique` on a hashed index is 16764, and an array at a
hashed path is 16766 *at insert time* rather than at creation.

plan: the index corpus is M3's last gate 1c098dc48c

42 cases at 3/39, and the two rows where recording it corrected the review:
`$in` in a partial filter is allowed, and a same-key different-filter clash
with no explicit name is IndexKeySpecsConflict (86) rather than 67.

Everything else at this commit is unmoved: 249/249 unit tests, operators
125/0, positional 51/0, aggregation 70/0, pinned crud 228/63/196.

dev merged commit 1c098dc48c into main

2026-08-10 19:34:18 +00:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: dev/MultiforaDB#11