index: a partial index may answer a query that implies its filter
Step 5, the last of `docs/M3_INDEX_TYPES_DESIGN_REVIEW.md`'s order and the
last item in M3's row. A partial index was maintained and enforced
`unique`, and every read scanned -- correct, but the speedup the option
exists for was never earned.
The test is one-sided by construction: a `false` costs a scan, a `true`
has to be right, because returning too few documents is the one failure
worse than having no index at all. Two routes, and a filter conjunct is
implied if either answers yes:
**Route one, the query pins a value.** Run the *real matcher* against a
stand-in document holding that value at the path, rather than
reimplementing eight operators against a comparison that would then have
two definitions. Sound because every operator `check_partial_filter`
admits is existential -- "some value at this path satisfies it" -- so a
document with more values at the path satisfies it at least as easily,
and every document the query matches has the pinned value among its
values there. Covers `$eq`, `$in`, `$type`, `$exists` and the bounds in
one stroke.
Two shapes break that argument and are refused rather than approximated,
and both have a row in the test table:
- an **array** value. `{a: [1, 2]}` matches `{a: [[1, 2], 3]}`, whose
values at `a` do not include 1 or 2 -- only one level is expanded, so
the real document's value set is not a superset of the stand-in's.
- a **null** value. `{a: null}` also matches a document with no `a`,
which has no values at the path rather than more of them. The
stand-in alone would report `{a: {$exists: true}}` as implied, so an
empty document is tested too and both have to agree. This is the rule
the previous commit's null fix made necessary and possible in the
same breath.
**Route two, bounds.** The only route needing neither side to name a
document: `{a: {$gt: 5}}` implies `{a: {$gt: 0}}`. Inclusivity is where
it is decided -- `$gte: 0` admits the endpoint that `$gt: 0` excludes.
Soundness is judged against *this server's* matcher, not mongod's. Both
halves of the question run the same code: `query.matches_bytes` decides
the index's contents in `build_entries` and re-filters every candidate
the plan yields. Where this server's comparison differs from mongod's
(PLAN §6: the comparison operators are not type-bracketed) both halves
are wrong together, which is a matching bug and not a lost document.
`$or` on the filter's side is implied by one implied branch: sufficient,
not necessary, since a query can imply a disjunction without implying a
disjunct.
**What each gate can and cannot see, measured with two mutations.** With
`query_implies_filter` forced to `true`, `partial.json` goes 23/7 and
every failure reads "expected N, got N-1" -- the exact shape of the bug.
With it forced to `false` -- the behaviour this commit replaces -- the
corpus is 30/30, because no client can observe *that* an index was used,
only that an answer went missing. So the corpus guards soundness and the
unit test on `plan()` is the only thing that sees the feature work at
all; both are needed and the commit says which does which.
Six corpus cases added, each pairing a query that implies the filter with
one that does not and touches the same field: a query leaving the
filter's field out, an `$in` straddling the filter, a query for null
against an `$exists` filter, equalities inside and outside a range
filter, a sort a partial index could serve, and a unique partial index
read. `partial.json` 24 -> 30 cases, `tests/spec/indexes/` 42 -> 48.
Verified: 257/257 unit tests in ReleaseFast and ReleaseSafe, 88/88 fuzz,
all four corpora 0 fail, pinned scorecard unchanged at 228/63/196, the
full e2e matrix and crash-fuzz green.
This commit is contained in:
453
src/index.zig
453
src/index.zig
@@ -2659,6 +2659,264 @@ pub const Plan = struct {
|
||||
}
|
||||
};
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Partial-index implication
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// Whether every document this query matches is one the partial filter also
|
||||
/// matches — the test that decides when a partial index may answer a read.
|
||||
///
|
||||
/// A partial index holds a subset of the collection, so reading from it is
|
||||
/// only correct when the query cannot match a document the filter left out.
|
||||
/// Too few documents is the one failure worse than having no index at all, so
|
||||
/// every answer here is one-sided: a `false` costs a scan, a `true` has to be
|
||||
/// right. Everything below is written to be wrong in that direction.
|
||||
///
|
||||
/// Soundness is judged against *this server's* matcher, not mongod's. Both
|
||||
/// halves of the question are decided by the same code: `query.matches_bytes`
|
||||
/// is what `build_entries` consults to decide the index's contents, and what
|
||||
/// every candidate this plan yields is re-filtered through. Where this
|
||||
/// server's comparison differs from mongod's — PLAN §6 records that the
|
||||
/// comparison operators are not type-bracketed — both halves are wrong
|
||||
/// together, which is a matching bug and not a lost document.
|
||||
///
|
||||
/// Only the operators `check_partial_filter` admits can appear on the filter
|
||||
/// side, and every one of them is *existential*: "some value at this path
|
||||
/// satisfies it". That is what the two routes below rest on.
|
||||
fn query_implies_filter(
|
||||
gpa: std.mem.Allocator,
|
||||
filter: []const bson.Pair,
|
||||
clauses: []const Clause,
|
||||
) query.QueryError!bool {
|
||||
for (filter) |f| {
|
||||
if (!try conjunct_implied(gpa, f, clauses)) return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
// Explicit rather than inferred: this and `query_implies_filter` call each
|
||||
// other, which an inferred error set cannot resolve.
|
||||
fn conjunct_implied(
|
||||
gpa: std.mem.Allocator,
|
||||
f: bson.Pair,
|
||||
clauses: []const Clause,
|
||||
) query.QueryError!bool {
|
||||
if (f.key.len > 0 and f.key[0] == '$') {
|
||||
const branches = switch (f.value) {
|
||||
.array => |a| a,
|
||||
else => return false,
|
||||
};
|
||||
if (std.mem.eql(u8, f.key, "$and")) {
|
||||
for (branches) |b| {
|
||||
const sub = switch (b) {
|
||||
.doc => |d| d,
|
||||
else => return false,
|
||||
};
|
||||
if (!try query_implies_filter(gpa, sub, clauses)) return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
if (std.mem.eql(u8, f.key, "$or")) {
|
||||
// One implied branch is enough. Implying none is not proof that
|
||||
// the disjunction fails -- a query can imply `{$or: [A, B]}`
|
||||
// without implying either -- so this is sufficient and not
|
||||
// necessary, which is the side to be wrong on.
|
||||
for (branches) |b| {
|
||||
const sub = switch (b) {
|
||||
.doc => |d| d,
|
||||
else => continue,
|
||||
};
|
||||
if (try query_implies_filter(gpa, sub, clauses)) return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
// The query's constraint on this path, accumulated exactly the way
|
||||
// `evaluate_index` accumulates it. Whatever survives is a *necessary*
|
||||
// condition of matching the query: an operator this does not understand
|
||||
// clears the info rather than weakening it.
|
||||
var qi = CompInfo{};
|
||||
for (clauses) |cl| {
|
||||
if (std.mem.eql(u8, cl.path, f.key)) analyze_clause(cl.value, &qi);
|
||||
}
|
||||
if (try pinned_values_imply(gpa, f.key, f.value, qi)) return true;
|
||||
return bounds_imply(f.value, qi);
|
||||
}
|
||||
|
||||
/// Route one: the query pins the path to a known value, or to one of a known
|
||||
/// set, and each of them satisfies the filter.
|
||||
fn pinned_values_imply(
|
||||
gpa: std.mem.Allocator,
|
||||
path: []const u8,
|
||||
fv: bson.Value,
|
||||
qi: CompInfo,
|
||||
) query.QueryError!bool {
|
||||
if (qi.eq) |v| return value_implies(gpa, path, v, fv);
|
||||
if (qi.in_values) |list| {
|
||||
// `{$in: []}` matches nothing and so implies everything, but an index
|
||||
// read of a query with no answers is not worth a special case.
|
||||
if (list.len == 0) return false;
|
||||
for (list) |m| {
|
||||
if (!try value_implies(gpa, path, m, fv)) return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
/// Whether a query pinning `v` at `path` implies the filter predicate `fv`
|
||||
/// there.
|
||||
///
|
||||
/// Answered by running the real matcher against a stand-in document holding
|
||||
/// exactly `v` at `path`, rather than by reimplementing eight operators
|
||||
/// against a comparison that would then have two definitions. It is sound
|
||||
/// because the filter's operators are existential, so a document with *more*
|
||||
/// values at the path satisfies them at least as easily -- and every document
|
||||
/// the query matches has `v` among its values there.
|
||||
///
|
||||
/// Two shapes break that argument and are refused rather than approximated:
|
||||
///
|
||||
/// - **`v` is an array.** `{a: [1, 2]}` matches `{a: [[1, 2], 3]}`, whose
|
||||
/// values at `a` are the outer array, `[1, 2]` and `3` -- only one level
|
||||
/// is expanded, so 1 and 2 are not among them and the real document's
|
||||
/// value set is not a superset of the stand-in's.
|
||||
/// - **`v` is null.** `{a: null}` also matches a document with no `a` at
|
||||
/// all, which has *no* values at the path rather than more of them. The
|
||||
/// stand-in alone would report `{a: {$exists: true}}` as implied, so an
|
||||
/// empty document is tested too and both have to agree.
|
||||
fn value_implies(
|
||||
gpa: std.mem.Allocator,
|
||||
path: []const u8,
|
||||
v: bson.Value,
|
||||
fv: bson.Value,
|
||||
) query.QueryError!bool {
|
||||
if (v == .array) return false;
|
||||
var arena = std.heap.ArenaAllocator.init(gpa);
|
||||
defer arena.deinit();
|
||||
const a = arena.allocator();
|
||||
const stand_in = (try nest_value(a, path, v)) orelse return false;
|
||||
const pred = [_]bson.Pair{.{ .key = path, .value = fv }};
|
||||
const f_doc = bson.Document{ .arena = undefined, .pairs = &pred };
|
||||
if (!try query.matches(a, &f_doc, &stand_in)) return false;
|
||||
if (v == .null) {
|
||||
const empty = bson.Document{ .arena = undefined, .pairs = &.{} };
|
||||
if (!try query.matches(a, &f_doc, &empty)) return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
/// The deepest path a stand-in document is built for. Past it the answer is
|
||||
/// "cannot tell", which costs a scan.
|
||||
const max_path_segments = 16;
|
||||
|
||||
/// A document holding `v` at a dotted `path`. Null when a segment is numeric:
|
||||
/// `a.0` may address an array element, and a document with a literal "0"
|
||||
/// field is a different question than the one being asked.
|
||||
fn nest_value(
|
||||
a: std.mem.Allocator,
|
||||
path: []const u8,
|
||||
v: bson.Value,
|
||||
) std.mem.Allocator.Error!?bson.Document {
|
||||
var segs: [max_path_segments][]const u8 = undefined;
|
||||
var n: usize = 0;
|
||||
var it = std.mem.splitScalar(u8, path, '.');
|
||||
while (it.next()) |s| {
|
||||
if (s.len == 0 or n == segs.len) return null;
|
||||
if (std.fmt.parseInt(usize, s, 10)) |_| return null else |_| {}
|
||||
segs[n] = s;
|
||||
n += 1;
|
||||
}
|
||||
if (n == 0) return null;
|
||||
var value = v;
|
||||
var i = n;
|
||||
while (i > 1) {
|
||||
i -= 1;
|
||||
const inner = try a.alloc(bson.Pair, 1);
|
||||
inner[0] = .{ .key = segs[i], .value = value };
|
||||
value = .{ .doc = inner };
|
||||
}
|
||||
const outer = try a.alloc(bson.Pair, 1);
|
||||
outer[0] = .{ .key = segs[0], .value = value };
|
||||
return .{ .arena = undefined, .pairs = outer };
|
||||
}
|
||||
|
||||
/// Route two: the query's bounds on a path are at least as tight as the
|
||||
/// filter's. The only route that needs neither side to name a document --
|
||||
/// `{a: {$gt: 5}}` implies `{a: {$gt: 0}}` with no value in common.
|
||||
///
|
||||
/// Restricted to the operators whose implication *is* an order comparison.
|
||||
/// Anything else in the filter predicate leaves this to route one.
|
||||
fn bounds_imply(fv: bson.Value, qi: CompInfo) bool {
|
||||
const ops = switch (fv) {
|
||||
.doc => |d| d,
|
||||
else => return false,
|
||||
};
|
||||
if (!query.all_operator_keys(ops)) return false;
|
||||
|
||||
var lo: ?bson.Value = null;
|
||||
var lo_incl = false;
|
||||
var hi: ?bson.Value = null;
|
||||
var hi_incl = false;
|
||||
var wants_exists = false;
|
||||
for (ops) |op| {
|
||||
if (std.mem.eql(u8, op.key, "$gt")) {
|
||||
lo = op.value;
|
||||
lo_incl = false;
|
||||
} else if (std.mem.eql(u8, op.key, "$gte")) {
|
||||
lo = op.value;
|
||||
lo_incl = true;
|
||||
} else if (std.mem.eql(u8, op.key, "$lt")) {
|
||||
hi = op.value;
|
||||
hi_incl = false;
|
||||
} else if (std.mem.eql(u8, op.key, "$lte")) {
|
||||
hi = op.value;
|
||||
hi_incl = true;
|
||||
} else if (std.mem.eql(u8, op.key, "$exists")) {
|
||||
// `$exists: false` is anti-monotone -- more values at a path make
|
||||
// it *less* true -- so none of the reasoning here applies to it.
|
||||
if (op.value != .bool or !op.value.bool) return false;
|
||||
wants_exists = true;
|
||||
} else {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
if (lo == null and hi == null and !wants_exists) return false;
|
||||
|
||||
if (lo) |f| {
|
||||
const q = qi.lo orelse return false;
|
||||
if (!bound_implies(q, qi.lo_incl, f, lo_incl, .gt)) return false;
|
||||
}
|
||||
if (hi) |f| {
|
||||
const q = qi.hi orelse return false;
|
||||
if (!bound_implies(q, qi.hi_incl, f, hi_incl, .lt)) return false;
|
||||
}
|
||||
// A bound predicate never matches a document with no value at the path:
|
||||
// the comparison loop has nothing to run over. So any bound at all on the
|
||||
// query's side settles `$exists: true`.
|
||||
if (wants_exists and qi.lo == null and qi.hi == null) return false;
|
||||
return true;
|
||||
}
|
||||
|
||||
/// Whether a query bound is at least as tight as a filter bound on the same
|
||||
/// side. `tighter` is the direction that narrows: `.gt` for a lower bound,
|
||||
/// `.lt` for an upper one.
|
||||
fn bound_implies(
|
||||
q: bson.Value,
|
||||
q_incl: bool,
|
||||
f: bson.Value,
|
||||
f_incl: bool,
|
||||
tighter: std.math.Order,
|
||||
) bool {
|
||||
const o = bson.compare(q, f);
|
||||
if (o == tighter) return true;
|
||||
if (o != .eq) return false;
|
||||
// The same endpoint: the query implies the filter unless it admits that
|
||||
// endpoint where the filter excludes it.
|
||||
return f_incl or !q_incl;
|
||||
}
|
||||
|
||||
/// Pick the index (if any) that can generate a superset of the matching
|
||||
/// documents: the one covering the longest leading run of equality/$in
|
||||
/// predicates, optionally with a range on the next key. Returns null when
|
||||
@@ -2690,13 +2948,11 @@ pub fn plan(
|
||||
}
|
||||
}
|
||||
for (indexes) |ix| {
|
||||
// A partial index holds a *subset* of the collection, so answering a
|
||||
// query from it is only correct when the query's predicates imply its
|
||||
// filter. That implication test does not exist yet, and reading from
|
||||
// the index without it returns too few documents -- the one failure
|
||||
// worse than having no index at all. So it is maintained, it enforces
|
||||
// `unique`, and reads scan. PLAN §6.
|
||||
if (ix.partial != null) continue;
|
||||
// A partial index holds a *subset* of the collection, so it may only
|
||||
// answer a query that cannot match a document its filter left out.
|
||||
if (ix.partial) |f| {
|
||||
if (!try query_implies_filter(gpa, f.pairs, clauses.items)) continue;
|
||||
}
|
||||
var cand = (try evaluate_index(gpa, ix, clauses.items, sort)) orelse continue;
|
||||
if (best) |b| {
|
||||
if (plan_better(&cand, &b)) {
|
||||
@@ -4248,6 +4504,189 @@ test "the _id index plan covers equality, ranges and _id sort order" {
|
||||
}
|
||||
}
|
||||
|
||||
test "a partial index answers only a query that implies its filter" {
|
||||
const gpa = testing.allocator;
|
||||
|
||||
// Every row is a (partial filter, query) pair and whether the query is
|
||||
// allowed to read the index. The two halves this table is really about
|
||||
// are opposite failures: a `true` that should be false loses documents
|
||||
// silently, a `false` that should be true only costs a scan. So the rows
|
||||
// are weighted towards the first.
|
||||
const Row = struct {
|
||||
filter: []const bson.Pair,
|
||||
query: []const bson.Pair,
|
||||
implied: bool,
|
||||
why: []const u8,
|
||||
};
|
||||
const eq_true: []const bson.Pair = &.{.{ .key = "t", .value = .{ .bool = true } }};
|
||||
const gt_zero: []const bson.Pair = &.{
|
||||
.{ .key = "n", .value = .{ .doc = &.{.{ .key = "$gt", .value = .{ .int32 = 0 } }} } },
|
||||
};
|
||||
const exists: []const bson.Pair = &.{
|
||||
.{ .key = "n", .value = .{ .doc = &.{.{ .key = "$exists", .value = .{ .bool = true } }} } },
|
||||
};
|
||||
|
||||
const rows = [_]Row{
|
||||
// -- route one: the query pins a value -----------------------------
|
||||
.{ .why = "the same equality", .implied = true, .filter = eq_true, .query = &.{
|
||||
.{ .key = "a", .value = .{ .int32 = 1 } },
|
||||
.{ .key = "t", .value = .{ .bool = true } },
|
||||
} },
|
||||
.{ .why = "the opposite equality", .implied = false, .filter = eq_true, .query = &.{
|
||||
.{ .key = "a", .value = .{ .int32 = 1 } },
|
||||
.{ .key = "t", .value = .{ .bool = false } },
|
||||
} },
|
||||
.{ .why = "no predicate on the filter's path", .implied = false, .filter = eq_true, .query = &.{
|
||||
.{ .key = "a", .value = .{ .int32 = 1 } },
|
||||
} },
|
||||
.{ .why = "an equality inside the filter's range", .implied = true, .filter = gt_zero, .query = &.{
|
||||
.{ .key = "n", .value = .{ .int32 = 5 } },
|
||||
} },
|
||||
.{ .why = "an equality outside it", .implied = false, .filter = gt_zero, .query = &.{
|
||||
.{ .key = "n", .value = .{ .int32 = -5 } },
|
||||
} },
|
||||
.{ .why = "an $in wholly inside", .implied = true, .filter = gt_zero, .query = &.{
|
||||
.{ .key = "n", .value = .{ .doc = &.{.{ .key = "$in", .value = .{ .array = &.{
|
||||
.{ .int32 = 5 },
|
||||
.{ .int32 = 9 },
|
||||
} } }} } },
|
||||
} },
|
||||
.{ .why = "an $in straddling it", .implied = false, .filter = gt_zero, .query = &.{
|
||||
.{ .key = "n", .value = .{ .doc = &.{.{ .key = "$in", .value = .{ .array = &.{
|
||||
.{ .int32 = 5 },
|
||||
.{ .int32 = 0 },
|
||||
} } }} } },
|
||||
} },
|
||||
.{ .why = "an equality implies $exists", .implied = true, .filter = exists, .query = &.{
|
||||
.{ .key = "n", .value = .{ .int32 = 5 } },
|
||||
} },
|
||||
// `{n: null}` matches a document with no `n` at all, so it cannot
|
||||
// imply that `n` exists. This is the row that makes the stand-in
|
||||
// document insufficient on its own.
|
||||
.{ .why = "equality to null does not imply $exists", .implied = false, .filter = exists, .query = &.{
|
||||
.{ .key = "n", .value = .null },
|
||||
} },
|
||||
// ...but it does imply an equality to null, and the index does hold
|
||||
// those documents: `build_entries` runs the same matcher.
|
||||
.{ .why = "equality to null implies equality to null", .implied = true, .query = &.{
|
||||
.{ .key = "n", .value = .null },
|
||||
}, .filter = &.{.{ .key = "n", .value = .null }} },
|
||||
// An array value cannot stand in for itself: `{a: [1, 2]}` matches
|
||||
// `{a: [[1, 2], 3]}`, whose values at `a` do not include 1 or 2.
|
||||
.{ .why = "an array equality is refused", .implied = false, .filter = gt_zero, .query = &.{
|
||||
.{ .key = "n", .value = .{ .array = &.{ .{ .int32 = 5 }, .{ .int32 = 9 } } } },
|
||||
} },
|
||||
|
||||
// -- route two: bounds ---------------------------------------------
|
||||
.{ .why = "a tighter lower bound", .implied = true, .filter = gt_zero, .query = &.{
|
||||
.{ .key = "n", .value = .{ .doc = &.{.{ .key = "$gt", .value = .{ .int32 = 5 } }} } },
|
||||
} },
|
||||
.{ .why = "the identical bound", .implied = true, .filter = gt_zero, .query = &.{
|
||||
.{ .key = "n", .value = .{ .doc = &.{.{ .key = "$gt", .value = .{ .int32 = 0 } }} } },
|
||||
} },
|
||||
.{ .why = "a looser lower bound", .implied = false, .filter = gt_zero, .query = &.{
|
||||
.{ .key = "n", .value = .{ .doc = &.{.{ .key = "$gt", .value = .{ .int32 = -1 } }} } },
|
||||
} },
|
||||
// $gte 0 admits 0 where the filter's $gt 0 excludes it -- the whole
|
||||
// of what inclusivity decides.
|
||||
.{ .why = "the same endpoint, inclusive against exclusive", .implied = false, .filter = gt_zero, .query = &.{
|
||||
.{ .key = "n", .value = .{ .doc = &.{.{ .key = "$gte", .value = .{ .int32 = 0 } }} } },
|
||||
} },
|
||||
.{ .why = "a bound implies $exists", .implied = true, .filter = exists, .query = &.{
|
||||
.{ .key = "n", .value = .{ .doc = &.{.{ .key = "$gt", .value = .{ .int32 = 5 } }} } },
|
||||
} },
|
||||
.{ .why = "an upper bound does not imply a lower one", .implied = false, .filter = gt_zero, .query = &.{
|
||||
.{ .key = "n", .value = .{ .doc = &.{.{ .key = "$lt", .value = .{ .int32 = 5 } }} } },
|
||||
} },
|
||||
// An operator the analysis does not understand clears the info, so
|
||||
// the tight bound beside it cannot be leaned on.
|
||||
.{ .why = "an unusable operator beside a bound", .implied = false, .filter = gt_zero, .query = &.{
|
||||
.{ .key = "n", .value = .{ .doc = &.{
|
||||
.{ .key = "$gt", .value = .{ .int32 = 5 } },
|
||||
.{ .key = "$bogus", .value = .{ .int32 = 1 } },
|
||||
} } },
|
||||
} },
|
||||
|
||||
// -- the filter's own shape ----------------------------------------
|
||||
.{ .why = "both conjuncts implied", .implied = true, .query = &.{
|
||||
.{ .key = "t", .value = .{ .bool = true } },
|
||||
.{ .key = "n", .value = .{ .int32 = 5 } },
|
||||
}, .filter = &.{
|
||||
.{ .key = "t", .value = .{ .bool = true } },
|
||||
.{ .key = "n", .value = .{ .doc = &.{.{ .key = "$gt", .value = .{ .int32 = 0 } }} } },
|
||||
} },
|
||||
.{ .why = "one conjunct implied is not enough", .implied = false, .query = &.{
|
||||
.{ .key = "t", .value = .{ .bool = true } },
|
||||
}, .filter = &.{
|
||||
.{ .key = "t", .value = .{ .bool = true } },
|
||||
.{ .key = "n", .value = .{ .doc = &.{.{ .key = "$gt", .value = .{ .int32 = 0 } }} } },
|
||||
} },
|
||||
.{ .why = "one branch of the filter's $or", .implied = true, .query = &.{
|
||||
.{ .key = "t", .value = .{ .bool = true } },
|
||||
}, .filter = &.{.{ .key = "$or", .value = .{ .array = &.{
|
||||
.{ .doc = &.{.{ .key = "t", .value = .{ .bool = true } }} },
|
||||
.{ .doc = &.{.{ .key = "n", .value = .{ .int32 = 9 } }} },
|
||||
} } }} },
|
||||
.{ .why = "no branch of the filter's $or", .implied = false, .query = &.{
|
||||
.{ .key = "t", .value = .{ .bool = false } },
|
||||
}, .filter = &.{.{ .key = "$or", .value = .{ .array = &.{
|
||||
.{ .doc = &.{.{ .key = "t", .value = .{ .bool = true } }} },
|
||||
.{ .doc = &.{.{ .key = "n", .value = .{ .int32 = 9 } }} },
|
||||
} } }} },
|
||||
// The query's own $or is skipped by `flatten_clauses`, so its members
|
||||
// are not predicates the query is known to enforce.
|
||||
.{ .why = "a query $or supplies nothing", .implied = false, .filter = eq_true, .query = &.{
|
||||
.{ .key = "$or", .value = .{ .array = &.{
|
||||
.{ .doc = &.{.{ .key = "t", .value = .{ .bool = true } }} },
|
||||
} } },
|
||||
} },
|
||||
// ...but a query $and does: it is a conjunction like the top level.
|
||||
.{ .why = "a query $and supplies its members", .implied = true, .filter = eq_true, .query = &.{
|
||||
.{ .key = "$and", .value = .{ .array = &.{
|
||||
.{ .doc = &.{.{ .key = "t", .value = .{ .bool = true } }} },
|
||||
} } },
|
||||
} },
|
||||
// A dotted path is nested into the stand-in document...
|
||||
.{ .why = "a dotted filter path", .implied = true, .query = &.{
|
||||
.{ .key = "m.t", .value = .{ .bool = true } },
|
||||
}, .filter = &.{.{ .key = "m.t", .value = .{ .bool = true } }} },
|
||||
// ...unless a segment is numeric, where it might mean an array index.
|
||||
.{ .why = "a numeric path segment is refused", .implied = false, .query = &.{
|
||||
.{ .key = "m.0", .value = .{ .bool = true } },
|
||||
}, .filter = &.{.{ .key = "m.0", .value = .{ .bool = true } }} },
|
||||
};
|
||||
|
||||
for (rows) |r| {
|
||||
errdefer std.debug.print("row: {s}\n", .{r.why});
|
||||
var clauses: std.ArrayListUnmanaged(Clause) = .empty;
|
||||
defer clauses.deinit(gpa);
|
||||
try flatten_clauses(gpa, r.query, &clauses);
|
||||
try testing.expectEqual(r.implied, try query_implies_filter(gpa, r.filter, clauses.items));
|
||||
}
|
||||
|
||||
// And the whole point of the answer: a partial index the query implies is
|
||||
// planned, one it does not is left alone. Mutation check: make
|
||||
// `query_implies_filter` return true unconditionally and the second
|
||||
// expectation here starts producing a plan -- which is the shape of a
|
||||
// read that silently returns too few documents.
|
||||
var ix = try simple_index(gpa, test_pager(), &.{"a"}, false, false);
|
||||
defer ix.deinit(gpa);
|
||||
try ix.set_partial(gpa, eq_true);
|
||||
{
|
||||
const f = [_]bson.Pair{
|
||||
.{ .key = "a", .value = .{ .int32 = 1 } },
|
||||
.{ .key = "t", .value = .{ .bool = true } },
|
||||
};
|
||||
var p = (try plan(gpa, null, &.{&ix}, &f, &.{})).?;
|
||||
defer p.deinit(gpa);
|
||||
try testing.expectEqual(@as(usize, 1), p.key_len());
|
||||
}
|
||||
{
|
||||
const f = [_]bson.Pair{.{ .key = "a", .value = .{ .int32 = 1 } }};
|
||||
try testing.expect((try plan(gpa, null, &.{&ix}, &f, &.{})) == null);
|
||||
}
|
||||
}
|
||||
|
||||
test "planner picks eq run, ranges, and bails on sparse null" {
|
||||
const gpa = testing.allocator;
|
||||
var ix = try simple_index(gpa, test_pager(), &.{ "a", "b" }, false, false);
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -79,6 +79,85 @@
|
||||
{ "name": "updateOne", "arguments": { "filter": { "_id": 2 }, "update": { "$set": { "t": true } } } }
|
||||
]
|
||||
},
|
||||
{
|
||||
"_comment": [
|
||||
"The reads below exist for the implication test: a partial index may",
|
||||
"only answer a query that cannot match a document its filter left",
|
||||
"out. Each pairs a query that implies the filter with one that does",
|
||||
"not and touches the same field, so an implication test that says yes",
|
||||
"too readily loses the documents outside the filter -- which is a",
|
||||
"wrong answer a result comparison can see, unlike the index being",
|
||||
"used at all, which no client can observe."
|
||||
],
|
||||
"description": "a query that leaves the filter's field out still sees past it",
|
||||
"documents": [
|
||||
{ "_id": 1, "a": 1, "t": true },
|
||||
{ "_id": 2, "a": 1, "t": false },
|
||||
{ "_id": 3, "a": 2, "t": true }
|
||||
],
|
||||
"ops": [
|
||||
{ "name": "createIndex", "arguments": { "keys": { "a": 1 }, "partialFilterExpression": { "t": true } } },
|
||||
{ "name": "find", "arguments": { "filter": { "a": 1 }, "sort": { "_id": 1 } } },
|
||||
{ "name": "find", "arguments": { "filter": { "a": 1, "t": true }, "sort": { "_id": 1 } } },
|
||||
{ "name": "find", "arguments": { "filter": { "a": 1, "t": false }, "sort": { "_id": 1 } } }
|
||||
]
|
||||
},
|
||||
{
|
||||
"description": "an $in straddling the filter still answers every match",
|
||||
"ops": [
|
||||
{ "name": "createIndex", "arguments": { "keys": { "a": 1 }, "partialFilterExpression": { "a": { "$gte": 5 } } } },
|
||||
{ "name": "find", "arguments": { "filter": { "a": { "$in": [1, 5] } }, "sort": { "_id": 1 } } },
|
||||
{ "name": "find", "arguments": { "filter": { "a": { "$in": [5, 9] } }, "sort": { "_id": 1 } } }
|
||||
]
|
||||
},
|
||||
{
|
||||
"description": "a query for null does not belong to an index that requires the field",
|
||||
"documents": [
|
||||
{ "_id": 1, "a": 1 },
|
||||
{ "_id": 2 },
|
||||
{ "_id": 3, "a": null }
|
||||
],
|
||||
"ops": [
|
||||
{ "name": "createIndex", "arguments": { "keys": { "a": 1 }, "partialFilterExpression": { "a": { "$exists": true } } } },
|
||||
{ "name": "find", "arguments": { "filter": { "a": null }, "sort": { "_id": 1 } } },
|
||||
{ "name": "find", "arguments": { "filter": { "a": 1 }, "sort": { "_id": 1 } } }
|
||||
]
|
||||
},
|
||||
{
|
||||
"description": "an equality inside the filter's range, and one outside it",
|
||||
"ops": [
|
||||
{ "name": "createIndex", "arguments": { "keys": { "a": 1 }, "partialFilterExpression": { "a": { "$gt": 3 } } } },
|
||||
{ "name": "find", "arguments": { "filter": { "a": 5 }, "sort": { "_id": 1 } } },
|
||||
{ "name": "find", "arguments": { "filter": { "a": 1 }, "sort": { "_id": 1 } } },
|
||||
{ "name": "find", "arguments": { "filter": { "a": { "$gt": 4 } }, "sort": { "_id": 1 } } },
|
||||
{ "name": "find", "arguments": { "filter": { "a": { "$gt": 0 } }, "sort": { "_id": 1 } } }
|
||||
]
|
||||
},
|
||||
{
|
||||
"description": "a sort a partial index could serve",
|
||||
"documents": [
|
||||
{ "_id": 1, "a": 3, "t": true },
|
||||
{ "_id": 2, "a": 1, "t": false },
|
||||
{ "_id": 3, "a": 2, "t": true }
|
||||
],
|
||||
"ops": [
|
||||
{ "name": "createIndex", "arguments": { "keys": { "a": 1 }, "partialFilterExpression": { "t": true } } },
|
||||
{ "name": "find", "arguments": { "filter": { "t": true }, "sort": { "a": 1 } } },
|
||||
{ "name": "find", "arguments": { "filter": {}, "sort": { "a": 1 } } }
|
||||
]
|
||||
},
|
||||
{
|
||||
"description": "a unique partial index answers the read it constrains",
|
||||
"documents": [
|
||||
{ "_id": 1, "a": 9, "t": true },
|
||||
{ "_id": 2, "a": 9, "t": false }
|
||||
],
|
||||
"ops": [
|
||||
{ "name": "createIndex", "arguments": { "keys": { "a": 1 }, "unique": true, "partialFilterExpression": { "t": true } } },
|
||||
{ "name": "find", "arguments": { "filter": { "a": 9, "t": true }, "sort": { "_id": 1 } } },
|
||||
{ "name": "find", "arguments": { "filter": { "a": 9 }, "sort": { "_id": 1 } } }
|
||||
]
|
||||
},
|
||||
{
|
||||
"description": "a filter on $exists",
|
||||
"ops": [
|
||||
|
||||
Reference in New Issue
Block a user