index/db: locate index entries by regeneration, not by scanning

remove_id walks every entry in the index comparing ids, so evicting one
document cost O(index size) per index -- on a 65,536-document collection
that is 65,536 id comparisons to remove a single entry, and it ran on
every update and every delete.

Entry generation is a pure function of the document, so remove_doc
regenerates the entries the document contributed and binary-searches for
each one. evict_doc now removes the document from the docs map first and
hands the document itself to the index, while both it and the map key are
still alive.

  updateMany({k: 7}, {$inc}) over 65,536 documents, measured A/B:
    15.4ms -> 5.5ms   (MongoDB 8.3.7: 6.1ms)

It is infallible by construction. Regeneration allocates and can fail,
and a document the index could not key contributed nothing to remove; in
either case it falls back to the scan, which is always correct. It also
falls back if the regenerated entries are not all found, so a
disagreement degrades to slow rather than leaving a stale index. That
last guard is defensive only -- regeneration is deterministic, so the
test below does not reach it.

The equivalence is checked directly rather than by example: two identical
indexes are built over documents exercising multikey arrays with repeats,
missing fields and both sparse settings, then emptied document by
document in shuffled order -- one through remove_doc, one through
remove_id -- asserting the entry arrays stay byte-identical at every
step. Verified it fails when removal order is reversed, which is the way
positional removal actually breaks.

Insertion still memmoves the tail. That, and ordered leaf iteration, are
what the tree is still for.

Note for later: the updateOne({_id}) and deleteOne({_id}) paths are slow
here for an unrelated reason -- an integer _id is rejected by
value_fast_path_safe, so each one is a full collection scan. The encoded
keys already make that guard unnecessary; removing it belongs with the
_id index.

Verified: 78 unit tests under ReleaseFast and ReleaseSafe, e2e
29/16/17/3/2, the crash pair, e2e6 72/72.
This commit is contained in:
2026-08-02 19:47:44 +03:00
parent 6feacc21cd
commit 2e508d3ecf
2 changed files with 141 additions and 4 deletions

View File

@@ -131,12 +131,15 @@ pub const Engine = struct {
/// Drop the document stored under `id_key`, freeing it and its key.
/// No-op when the id is absent. This is the single chokepoint where a
/// document dies, so index entries are removed here — before
/// old.value.deinit() and gpa.free(old.key) — keeping the entry aliasing
/// (values into the document arena, id into the docs map key) safe.
/// document dies, so index entries are removed here — while the
/// document and the docs map key are both still alive, which is what
/// keeps `Entry.id`'s aliasing of that key safe.
///
/// The document itself is handed to the index: entries are located by
/// regenerating them from it, which is far cheaper than scanning.
fn evict_doc(self: *Engine, coll: *Collection, id_key: []const u8) void {
for (coll.indexes.items) |*ix| ix.remove_id(self.gpa, id_key);
const old = coll.docs.fetchRemove(id_key) orelse return;
for (coll.indexes.items) |*ix| ix.remove_doc(self.gpa, old.value, old.key);
old.value.*.deinit();
self.gpa.destroy(old.value);
self.gpa.free(old.key);