M3: design review of the index types, and the partialFilterExpression refusal #10

Merged
dev merged 2 commits from m3-index-types into main 2026-08-10 19:19:07 +00:00

2 Commits

Author SHA1 Message Date
A.Shakhmatov
21002528be commands: refuse partialFilterExpression instead of ignoring it
`createIndex({a: 1}, {partialFilterExpression: ...})` answered success, built
the index over every document, and left the option out of `listIndexes`.

An over-inclusive index still answers reads correctly -- it holds a superset,
never a subset -- so this was not the array-destroying class of bug. `unique`
is where it stopped being harmless. Measured on mongod 8.3.7:

  createIndex({a: 1}, {unique: true, partialFilterExpression: {t: true}})
  insertMany([{a: 1, t: false}, {a: 1, t: false}])
    mongod: accepted -- neither document is in the index, so neither collides
    ours:   E11000 duplicate key error, dup key: {a: 1}

A legal insert refused. Unique-within-a-subset -- unique email among active
accounts, unique external id among synced rows -- is the whole point of the
option, so every use of it hit this.

Refused at creation, which is the same judgement `cmd_update` already makes
about an update spec's `sort`: "ignoring the field would be the worst of the
three possible answers." The two alternatives here were to keep enforcing
`unique` over the wrong set, or to echo the option back from `listIndexes`
while not honouring it, which is a larger lie than saying no.

CannotCreateIndex (67), joining the five existing rows of the same test with
the mutation that reddens them written beside it. The implementation is the
rest of M3's last row and comes next, against a corpus that does not exist
yet -- nothing in this repository covered the row above, in any suite.

249/249 unit tests, pinned crud corpus unmoved at 228/63/196.
2026-08-10 22:12:08 +03:00
A.Shakhmatov
ce4ff1fd63 docs: M3 design review -- partial and hashed indexes
M3's last row names them together. They are not the same kind of gap, and the
measurement is what says so.

**Hashed is honestly missing**: `createIndex({a: "hashed"})` is refused. Wrong
code -- 2 where mongod says 67, and it has four more specific ones besides --
but the right answer, and queries keep working because there is simply no
index.

**Partial is accepted and ignored.** `createIndex` reports success,
`listIndexes` does not mention `partialFilterExpression`, and the index is
built over every document rather than the ones the filter selects.

An over-inclusive index still answers reads correctly, which is worth saying
plainly: it holds a superset, never a subset. `unique` is where that stops
being true --

  createIndex({a: 1}, {unique: true, partialFilterExpression: {t: true}})
  insertMany([{a: 1, t: false}, {a: 1, t: false}])
    mongod: accepted, neither document is in the index
    ours:   E11000 duplicate key error

-- a legal insert refused. Unique-within-a-subset is the whole point of the
option, and every use of it is an insert this server rejects.

Nothing in the repository covers that row: the pinned suite is crud and
aggregate, and neither `e2e5.js` nor `e2e6.js` writes a partial or hashed
spec. So this needs its own recorded corpus, like the positional operators and
the update operators before it.

The review measures both feature's rules -- which predicates a partial filter
may hold, why `sparse` and `partialFilterExpression` may not be combined, the
four hashed refusals and the one that fires at insert time rather than at
creation -- and argues the planner rule that matters: a partial index may only
answer a query whose predicates *imply* its filter, so until that test exists
the safe rule is to maintain the index and never read from it. Too few
documents is the one failure worse than no index at all.

Recommended order: refuse `partialFilterExpression`, record the corpus, then
implement partial, then hashed, then the implication test. Both need one
catalog field each, and `write_index_catalog`'s flags byte has spare bits, so
`catalog_version` stays 1 -- the same argument the free list used.
2026-08-10 22:09:17 +03:00