Aleksey Shakhmatov
f2844e7894
Every reply came back in a single batch with `cursor.id = 0`, `getMore` was a
stub answering an empty `nextBatch` on the literal namespace `test.$cmd`, and
nothing read `batchSize`. That caps the useful collection size at what fits in
one 48 MiB message, which is the opposite of the tens-of-GB target and the
reason M0 made whole-index scans stream: the streaming candidate generator
existed with no consumer that could suspend.
## What a cursor is allowed to remember
A cursor holds no lock between requests, so everything it saves has to survive
arbitrary concurrent mutation. Nothing here is a pointer, and the two things
that look like stable addresses are not: `reset_tree` re-creates node ids 0 and
1 as different nodes, and `rebuild_collection` moves every document. Three
sources, chosen by query shape, each with a different memory contract:
- **stream** -- an index-ordered walk resumed from a `(key, off)` anchor plus a
`(leaf, slot)` hint. O(key) state, so this is what lets a cursor walk a
collection larger than memory. Survives a rebuild, because a repack changes
no key.
- **offsets** -- the matched slab offsets a narrowed plan already materialized,
8 bytes each. Killed by a rebuild with `QueryPlanKilled`, because those
offsets now name unrelated bytes.
- **buffered** -- canonical BSON copies, for a sort no index provides and for
aggregate/listing output. Depends on nothing, which is what lets a listing
hold a cursor over a `$cmd.*` namespace no collection backs.
`Collection.layout_epoch` and `Index.epoch` are the invalidation tokens, both
checked as error returns rather than assertions since a client reaches them by
keeping a cursor open across maintenance.
## Resume
`resume_forward`/`resume_reverse` are O(1) while the hint holds and fall back to
an exact-order band walk bounded by `resume_walk_max`. Without the hint, `seek`
lands at the *start* of an equal-key band, so `sort({status: 1})` over three
distinct values across 10M documents would cost ~5e10 comparisons to drain.
Two hazards found by draining a collection while writing to it, neither
predictable from reading the code:
- A deleted anchor must resume at its *band position*, or the rest of an
equal-key band is silently dropped -- most of the collection on a
low-cardinality index. Hence `band_index`.
- On a **unique** index a same-key entry can only be the anchor rewritten, so
resuming at it returned updated documents twice. Observed as duplicate `_id`s
while updating underneath a drain.
## Protocol
Measured against mongod 8.3.7 rather than recalled, which corrected three
assumptions: a bare `getMore` does *not* inherit the find's `batchSize` (4998 of
5000 documents come back), a namespace mismatch is `Unauthorized` (13) not
`CursorNotFound`, and `CursorInUse` is 143 not 12051.
`internalQueryFindCommandBatchSize` is 101, `cursorTimeoutMillis` 600000,
`clientCursorMonitorFrequencySecs` 4.
The rule everything follows is **never look ahead**: a batch that met its target
leaves the cursor open even when the source is in fact exhausted, so four
documents at `batchSize: 2` take three commands. `limit` acts as an EOF source,
which is what makes `batchSize == limit` close in one round trip. `skip` is
consumed once. `batchSize: 0` returns an empty batch with a live cursor.
Cursor ids are `(nonce << 20) | slot`, always positive. The nonce is not
decoration: without it a recycled slot serves one client another's documents.
Cursors are not connection-pinned, since the driver spec allows a `getMore` on
any connection to the same server; they end at exhaustion, `killCursors`, or the
idle sweep (a second monitor fiber, separate from the TTL one because the
cadences differ by an order of magnitude and a TTL failure must not stop
reclamation). The registry is fixed-capacity and evicts the least recently used
cursor, whose client sees the same 43 an idle timeout gives.
Fixed alongside, because cursors are what expose them:
- `listCollections` reported `"<db>."` with an *empty* collection part, which
makes the driver throw client-side -- so it would have broken the moment its
cursor stopped being id 0. Now `<db>.$cmd.listCollections`, as mongod uses.
- `count` ignored `skip` and `limit` entirely.
- `wire.end_message` now bounds a reply by the 48 MiB we advertise rather than
by `maxInt(u32)`; a reply past what we told the client to expect is not a large
reply, it is a desynchronized connection.
- Two `codeName` strings were wrong: 72 is `InvalidOptions` (MongoDB has no
`InvalidArgument`), and 40324 reports as `Location40324`.
## Verification
Unit 160/160 in ReleaseFast and ReleaseSafe; `tests/e2e/e2e7.js` adds 86 cursor
checks across five phases (batching/lifecycle/errors, streaming across churn,
aggregate+listings+count, expiry+capacity, restart) and is self-contained
because cursor behaviour is only observable with non-default flags. No
regressions: e2e 49, e2e3 16, e2e4 17, e2e2 2, e2e6 72. Spec 168 pass / 124
fail, +5 against the previous scorecard.
Mutation-checked, per the repo's second ground rule: `hint_slot + 1`, the
`band_index` off-by-one, both epoch bumps, the id nonce, the at-least-one-
document rule, and `stream_shape` returning null each turn the intended test
red. One claim was withdrawn rather than kept -- swapping `std.mem.order` for
`cmp_prefix` in the band walk changes nothing observable, so the comment now
says so instead of asserting a check that does not hold.
3836 lines
174 KiB
Zig
3836 lines
174 KiB
Zig
//! MongoDB command dispatch. Each command fills `reply` with its result;
|
|
//! unknown commands and failures produce error replies with real codes.
|
|
|
|
const std = @import("std");
|
|
const builtin = @import("builtin");
|
|
const bson = @import("bson.zig");
|
|
const wire = @import("wire.zig");
|
|
const db = @import("db.zig");
|
|
const Collection = db.Collection;
|
|
const query = @import("query.zig");
|
|
const update = @import("update.zig");
|
|
const index = @import("index.zig");
|
|
const cursor = @import("cursor.zig");
|
|
// Always active, including in the default ReleaseFast build -- see assert.zig.
|
|
const assert = @import("assert.zig").assert;
|
|
const assert_msg = @import("assert.zig").assert_msg;
|
|
|
|
pub const Context = struct {
|
|
gpa: std.mem.Allocator,
|
|
io: std.Io,
|
|
oid_gen: *bson.ObjectIdGen,
|
|
connection_id: u32,
|
|
client_desc: []const u8,
|
|
engine: *db.Engine,
|
|
server_start: std.Io.Timestamp,
|
|
};
|
|
|
|
pub const ErrorCode = enum(i32) {
|
|
command_not_found = 59,
|
|
bad_value = 2,
|
|
/// 72 is MongoDB's `InvalidOptions`. There is no `InvalidArgument` in its
|
|
/// table at all, so that is the name this used to send.
|
|
invalid_options = 72,
|
|
namespace_not_found = 26,
|
|
index_not_found = 27,
|
|
duplicate_key = 11000,
|
|
namespace_exists = 48,
|
|
failed_to_parse = 9,
|
|
internal_error = 1,
|
|
/// "Unrecognized pipeline stage name". A `Location` code, so mongod names it
|
|
/// `Location40324` rather than after any symbol.
|
|
location_unrecognized_stage = 40324,
|
|
index_options_conflict = 85,
|
|
cannot_create_index = 67,
|
|
invalid_index_specification_option = 197,
|
|
// Cursor codes. Taken from MongoDB's own error_codes.js rather than
|
|
// recalled -- CursorInUse in particular is 143, not the 12051 that turns up
|
|
// in older notes.
|
|
cursor_not_found = 43,
|
|
cursor_in_use = 143,
|
|
query_plan_killed = 175,
|
|
unauthorized = 13,
|
|
type_mismatch = 14,
|
|
operation_failed = 96,
|
|
};
|
|
|
|
/// Which lock (if any) a command needs on the engine. Contract: only
|
|
/// `.write` commands may call engine mutation functions (insert, replace,
|
|
/// remove, drop*, get_or_create_collection, compact); `.read` commands may
|
|
/// only read (`get_collection`, `database_names`, `collection_names`);
|
|
/// `.none` commands must not touch the engine at all.
|
|
///
|
|
/// This describes engine *data* only. The cursor store is a separate resource
|
|
/// with its own leaf mutex, and any kind may mutate it: `getMore` is `.read`
|
|
/// because it only reads documents, even though it advances cursor state, and
|
|
/// `killCursors` is `.none` because the store is all it touches.
|
|
const CommandKind = enum { none, read, write };
|
|
|
|
/// Lock shape for one command, acquired by dispatch: the catalog lock mode
|
|
/// and whether the command's target collection is locked (shared for reads,
|
|
/// exclusive for writes/DDL). The target collection is the message field
|
|
/// named after the command (find/count/insert/...), which every
|
|
/// collection-targeting command uses except `getMore` -- see `Command.coll_field`.
|
|
const LockShape = struct {
|
|
catalog: enum { none, shared, exclusive } = .none,
|
|
coll: enum { none, shared, exclusive } = .none,
|
|
};
|
|
|
|
const Command = struct {
|
|
name: []const u8,
|
|
kind: CommandKind,
|
|
locks: LockShape = .{},
|
|
/// Body field naming the target collection, when it is not the command's
|
|
/// own field. Only `getMore` needs it: its own value is an int64 cursor id.
|
|
coll_field: ?[]const u8 = null,
|
|
handler: *const fn (*Context, *wire.Message, *wire.Reply) anyerror!void,
|
|
};
|
|
|
|
/// The one place a command exists: name, lock class, and handler declared
|
|
/// together so a new command cannot be given a handler but no lock.
|
|
const command_table = [_]Command{
|
|
// Handshake, admin info, no-ops: never touch the engine.
|
|
.{ .name = "hello", .kind = .none, .handler = cmd_hello },
|
|
.{ .name = "isMaster", .kind = .none, .handler = cmd_is_master },
|
|
.{ .name = "ismaster", .kind = .none, .handler = cmd_is_master },
|
|
.{ .name = "ping", .kind = .none, .handler = cmd_ping },
|
|
.{ .name = "buildInfo", .kind = .none, .handler = cmd_build_info },
|
|
.{ .name = "getParameter", .kind = .none, .handler = cmd_get_parameter },
|
|
.{ .name = "whatsmyuri", .kind = .none, .handler = cmd_whatsmyuri },
|
|
.{ .name = "hostInfo", .kind = .none, .handler = cmd_host_info },
|
|
.{ .name = "getCmdLineOpts", .kind = .none, .handler = cmd_get_cmd_line_opts },
|
|
.{ .name = "serverStatus", .kind = .none, .handler = cmd_server_status },
|
|
.{ .name = "endSessions", .kind = .none, .handler = cmd_end_sessions },
|
|
.{ .name = "connectionStatus", .kind = .none, .handler = cmd_connection_status },
|
|
// killCursors touches only the cursor store, so it needs no lock -- and its
|
|
// own field really is the collection name, unlike getMore's.
|
|
.{ .name = "killCursors", .kind = .none, .handler = cmd_kill_cursors },
|
|
// Read-only: scan the engine without mutating it.
|
|
.{ .name = "find", .kind = .read, .locks = .{ .catalog = .shared, .coll = .shared }, .handler = cmd_find },
|
|
// getMore is the one command whose collection is not its own field: its
|
|
// value is an int64 cursor id, so dispatch reads `collection` instead.
|
|
.{
|
|
.name = "getMore",
|
|
.kind = .read,
|
|
.locks = .{ .catalog = .shared, .coll = .shared },
|
|
.coll_field = "collection",
|
|
.handler = cmd_get_more,
|
|
},
|
|
.{ .name = "count", .kind = .read, .locks = .{ .catalog = .shared, .coll = .shared }, .handler = cmd_count },
|
|
.{ .name = "aggregate", .kind = .read, .locks = .{ .catalog = .shared, .coll = .shared }, .handler = cmd_aggregate },
|
|
.{ .name = "listDatabases", .kind = .read, .locks = .{ .catalog = .shared }, .handler = cmd_list_databases },
|
|
.{ .name = "listCollections", .kind = .read, .locks = .{ .catalog = .shared }, .handler = cmd_list_collections },
|
|
// Writes: the target collection exclusively; create/drop take the
|
|
// catalog exclusively (they mutate the maps).
|
|
.{ .name = "create", .kind = .write, .locks = .{ .catalog = .exclusive, .coll = .exclusive }, .handler = cmd_create },
|
|
.{ .name = "drop", .kind = .write, .locks = .{ .catalog = .exclusive, .coll = .exclusive }, .handler = cmd_drop },
|
|
.{ .name = "dropDatabase", .kind = .write, .locks = .{ .catalog = .exclusive }, .handler = cmd_drop_database },
|
|
.{ .name = "createIndexes", .kind = .write, .locks = .{ .catalog = .shared, .coll = .exclusive }, .handler = cmd_create_indexes },
|
|
.{ .name = "dropIndexes", .kind = .write, .locks = .{ .catalog = .shared, .coll = .exclusive }, .handler = cmd_drop_indexes },
|
|
.{ .name = "insert", .kind = .write, .locks = .{ .catalog = .shared, .coll = .exclusive }, .handler = cmd_insert },
|
|
.{ .name = "update", .kind = .write, .locks = .{ .catalog = .shared, .coll = .exclusive }, .handler = cmd_update },
|
|
.{ .name = "delete", .kind = .write, .locks = .{ .catalog = .shared, .coll = .exclusive }, .handler = cmd_delete },
|
|
.{ .name = "findAndModify", .kind = .write, .locks = .{ .catalog = .shared, .coll = .exclusive }, .handler = cmd_find_and_modify },
|
|
.{ .name = "listIndexes", .kind = .read, .locks = .{ .catalog = .shared, .coll = .shared }, .handler = cmd_list_indexes },
|
|
};
|
|
|
|
/// Command name to its index in `command_table`, resolved at comptime so a
|
|
/// request costs one hash instead of a walk down the whole table comparing
|
|
/// strings.
|
|
const command_index = blk: {
|
|
var kvs: [command_table.len]struct { []const u8, usize } = undefined;
|
|
for (&command_table, 0..) |c, i| kvs[i] = .{ c.name, i };
|
|
break :blk std.StaticStringMap(usize).initComptime(kvs);
|
|
};
|
|
|
|
pub fn dispatch(ctx: *Context, msg: *wire.Message, reply: *wire.Reply) !void {
|
|
const name = msg.command_name();
|
|
const cmd = if (command_index.get(name)) |i| &command_table[i] else {
|
|
var buf: [256]u8 = undefined;
|
|
const errmsg = try std.fmt.bufPrint(&buf, "no such command: '{s}'", .{name});
|
|
return reply.put_error(@intFromEnum(ErrorCode.command_not_found), "CommandNotFound", errmsg);
|
|
};
|
|
|
|
// Lock the catalog (shared for most commands, exclusive for DDL), then
|
|
// the target collection, then run the handler. The collection lock is
|
|
// taken while the catalog lock is held, so a concurrent drop can never
|
|
// free the collection out from under us.
|
|
// Resolve the namespace *before* taking any lock. It used to happen after
|
|
// the catalog lock, with `orelse return` on both parts -- and a plain
|
|
// `return` is not an error return, so it ran neither the errdefer below nor
|
|
// the explicit unlocks after the handler. The catalog lock was simply
|
|
// leaked, shared, forever.
|
|
//
|
|
// A database-level command reaches it: `db.aggregate(...)` sends
|
|
// `{aggregate: 1}`, whose value is not a string, so str_arg returns null.
|
|
// The symptom was baffling because a leaked *shared* lock is invisible to
|
|
// readers -- ping and listDatabases kept answering in microseconds -- while
|
|
// the next write that needs the catalog exclusive to create a collection
|
|
// blocks forever. It presented as an unrelated client-side timeout one
|
|
// command later.
|
|
var ns: ?struct { db: []const u8, coll: []const u8 } = null;
|
|
if (cmd.locks.coll != .none) {
|
|
const db_name = msg.db_name() orelse
|
|
return bad_value(reply, "command requires a $db");
|
|
const field = cmd.coll_field orelse name;
|
|
const coll_name = str_arg(msg.body.get(field)) orelse
|
|
return bad_value(reply, "command requires a collection name");
|
|
ns = .{ .db = db_name, .coll = coll_name };
|
|
}
|
|
|
|
switch (cmd.locks.catalog) {
|
|
.none => {},
|
|
.shared => try ctx.engine.lock_catalog(false),
|
|
.exclusive => try ctx.engine.lock_catalog(true),
|
|
}
|
|
var catalog_held = cmd.locks.catalog != .none;
|
|
var coll: ?*Collection = null;
|
|
errdefer {
|
|
if (coll) |c| ctx.engine.unlock_collection(c, cmd.locks.coll == .exclusive);
|
|
if (catalog_held) ctx.engine.unlock_catalog(cmd.locks.catalog == .exclusive);
|
|
}
|
|
|
|
if (ns) |n| {
|
|
// Write commands may create the collection on first use; the catalog
|
|
// lock is upgraded to exclusive for that, then restored to shared.
|
|
const create = cmd.kind == .write and cmd.locks.catalog == .shared;
|
|
if (try ctx.engine.lock_collection(n.db, n.coll, cmd.locks.coll == .exclusive, create)) |c| {
|
|
coll = c;
|
|
}
|
|
}
|
|
|
|
const result = cmd.handler(ctx, msg, reply);
|
|
|
|
// Release the collection and catalog locks before the commit: the
|
|
// commit may block on other writers' appends, and must never do so
|
|
// while holding a collection lock.
|
|
if (coll) |c| ctx.engine.unlock_collection(c, cmd.locks.coll == .exclusive);
|
|
coll = null;
|
|
if (catalog_held) ctx.engine.unlock_catalog(cmd.locks.catalog == .exclusive);
|
|
catalog_held = false;
|
|
if (cmd.locks.coll == .exclusive) {
|
|
// Durability (seal + fsync) coalesces across concurrent writers. A
|
|
// commit error deliberately wins over the handler's captured `result`:
|
|
// whether the write reached disk matters more to the client than why
|
|
// the write itself was unhappy.
|
|
// commit() asserts its own postcondition (committed_seq >= this
|
|
// command's seq) internally. Re-checking it here is not possible
|
|
// without the log lock, and taking it just to assert would add a real
|
|
// race in exchange for a weaker check than the one already made.
|
|
try ctx.engine.commit();
|
|
// The write is durable by now, so a compaction failure is a maintenance
|
|
// problem and not the client's. Report it and hand the request back
|
|
// rather than turning an applied write into an error the client retries.
|
|
// A checkpoint reclaims the log, so an open stops paying for every write
|
|
// ever made. Runs here, with no lock held, for the same reason
|
|
// compaction does: it takes the log lock and must not do that while
|
|
// holding a collection lock.
|
|
if (ctx.engine.take_checkpoint()) {
|
|
ctx.engine.checkpoint() catch |err| {
|
|
// Durability is unaffected -- the log still holds everything.
|
|
// The cost is a slower next open, which is not the client's
|
|
// problem, so report and carry on.
|
|
std.debug.print("multiforadb: checkpoint failed: {s}\n", .{@errorName(err)});
|
|
};
|
|
}
|
|
if (ctx.engine.take_compact()) {
|
|
ctx.engine.compact() catch |err| {
|
|
std.debug.print("multiforadb: compaction failed: {s}\n", .{@errorName(err)});
|
|
ctx.engine.request_compact();
|
|
};
|
|
}
|
|
}
|
|
return result;
|
|
}
|
|
|
|
// ---------------------------------------------------------------------------
|
|
// Handshake / administration
|
|
// ---------------------------------------------------------------------------
|
|
|
|
fn add_server_info(ctx: *Context, reply: *wire.Reply) !void {
|
|
try reply.put("isWritablePrimary", .{ .bool = true });
|
|
try reply.put("maxBsonObjectSize", .{ .int32 = wire.max_bson_object_size });
|
|
try reply.put("maxMessageSizeBytes", .{ .int32 = 48000000 });
|
|
try reply.put("maxWriteBatchSize", .{ .int32 = 100000 });
|
|
const now = std.Io.Timestamp.now(ctx.io, .real);
|
|
try reply.put("localTime", .{ .datetime = now.toMilliseconds() });
|
|
try reply.put("logicalSessionTimeoutMinutes", .{ .int32 = 30 });
|
|
try reply.put("connectionId", .{ .int32 = @intCast(ctx.connection_id) });
|
|
try reply.put("minWireVersion", .{ .int32 = 0 });
|
|
try reply.put("maxWireVersion", .{ .int32 = 8 });
|
|
try reply.put("readOnly", .{ .bool = false });
|
|
|
|
// Deliberately no `topologyVersion`. A driver treats its presence as
|
|
// "this server supports the streaming (awaitable) hello protocol" and
|
|
// switches monitoring to an exhaust hello: it sends one hello with
|
|
// maxAwaitTimeMS and the OP_MSG exhaustAllowed flag, then expects a
|
|
// stream of unsolicited replies each carrying moreToCome. We answer
|
|
// once with moreToCome clear and go back to reading, so the driver
|
|
// fails the heartbeat ("Server ended moreToCome unexpectedly"), drops
|
|
// the connection and resets its pool — a connect/disconnect loop once
|
|
// per heartbeat, which is what MongoDB Compass showed. Omitting the
|
|
// field keeps monitoring on the polling path, which we do implement,
|
|
// and matches maxWireVersion 8: streaming hello arrived in wire 9.
|
|
}
|
|
|
|
fn cmd_hello(ctx: *Context, _: *wire.Message, reply: *wire.Reply) !void {
|
|
try add_server_info(ctx, reply);
|
|
try reply.put_ok();
|
|
}
|
|
|
|
fn cmd_is_master(ctx: *Context, _: *wire.Message, reply: *wire.Reply) !void {
|
|
try add_server_info(ctx, reply);
|
|
try reply.put("ismaster", .{ .bool = true });
|
|
try reply.put("helloOk", .{ .bool = true });
|
|
try reply.put_ok();
|
|
}
|
|
|
|
fn cmd_ping(_: *Context, _: *wire.Message, reply: *wire.Reply) !void {
|
|
try reply.put_ok();
|
|
}
|
|
|
|
fn cmd_build_info(_: *Context, _: *wire.Message, reply: *wire.Reply) !void {
|
|
try reply.put("version", .{ .string = "4.4.0" });
|
|
try reply.put("gitVersion", .{ .string = "multiforadb" });
|
|
try reply.put("versionArray", .{ .array = try int_array(reply, &.{ 4, 4, 0, 0 }) });
|
|
try reply.put("openssl", .{ .doc = &.{} });
|
|
try reply.put("loaderFlags", .{ .string = "" });
|
|
try reply.put("compilerInfo", .{ .string = "zig 0.16.0" });
|
|
try reply.put("allocator", .{ .string = "system" });
|
|
try reply.put("javascriptEngine", .{ .string = "none" });
|
|
try reply.put("bits", .{ .int32 = 64 });
|
|
try reply.put("debug", .{ .bool = false });
|
|
try reply.put("maxBsonObjectSize", .{ .int32 = wire.max_bson_object_size });
|
|
try reply.put("storageEngines", .{ .array = try str_array(reply, &.{"wiredTiger"}) });
|
|
try reply.put_ok();
|
|
}
|
|
|
|
fn cmd_get_parameter(_: *Context, msg: *wire.Message, reply: *wire.Reply) !void {
|
|
// mongosh probes featureCompatibilityVersion; respond per requested key.
|
|
const params = doc_arg(msg.body.get("getParameter")) orelse
|
|
return invalid_arg(reply, "getParameter requires a document");
|
|
if (bson.get_pair(params, "featureCompatibilityVersion") == null) {
|
|
return invalid_arg(reply, "no option found to get");
|
|
}
|
|
const fcv = try reply.arena_alloc().alloc(bson.Pair, 1);
|
|
fcv[0] = .{ .key = "version", .value = .{ .string = "4.4" } };
|
|
try reply.put("featureCompatibilityVersion", .{ .doc = fcv });
|
|
try reply.put_ok();
|
|
}
|
|
|
|
fn cmd_whatsmyuri(ctx: *Context, _: *wire.Message, reply: *wire.Reply) !void {
|
|
try reply.put("you", .{ .string = ctx.client_desc });
|
|
try reply.put_ok();
|
|
}
|
|
|
|
fn cmd_host_info(ctx: *Context, _: *wire.Message, reply: *wire.Reply) !void {
|
|
const now = std.Io.Timestamp.now(ctx.io, .real);
|
|
const cpu_count: usize = std.Thread.getCpuCount() catch 1;
|
|
|
|
const system = try reply.arena_alloc().alloc(bson.Pair, 6);
|
|
system[0] = .{ .key = "currentTime", .value = .{ .datetime = now.toMilliseconds() } };
|
|
system[1] = .{ .key = "hostname", .value = .{ .string = "localhost" } };
|
|
system[2] = .{ .key = "cpuAddrSize", .value = .{ .int32 = 64 } };
|
|
system[3] = .{ .key = "memSizeMB", .value = .{ .int32 = 0 } };
|
|
system[4] = .{ .key = "numCores", .value = .{ .int32 = @intCast(cpu_count) } };
|
|
system[5] = .{ .key = "cpuArch", .value = .{ .string = @tagName(builtin.cpu.arch) } };
|
|
try reply.put("system", .{ .doc = system });
|
|
|
|
const os = try reply.arena_alloc().alloc(bson.Pair, 3);
|
|
os[0] = .{ .key = "type", .value = .{ .string = @tagName(builtin.os.tag) } };
|
|
os[1] = .{ .key = "name", .value = .{ .string = @tagName(builtin.os.tag) } };
|
|
os[2] = .{ .key = "version", .value = .{ .string = "unknown" } };
|
|
try reply.put("os", .{ .doc = os });
|
|
|
|
try reply.put("extra", .{ .doc = &.{} });
|
|
try reply.put_ok();
|
|
}
|
|
|
|
fn cmd_get_cmd_line_opts(_: *Context, _: *wire.Message, reply: *wire.Reply) !void {
|
|
const argv = try reply.arena_alloc().alloc(bson.Pair, 2);
|
|
argv[0] = .{ .key = "dbpath", .value = .{ .string = "multiforadb.log" } };
|
|
argv[1] = .{ .key = "port", .value = .{ .int32 = 27017 } };
|
|
try reply.put("argv", .{ .array = &.{} });
|
|
try reply.put("parsed", .{ .doc = argv });
|
|
try reply.put_ok();
|
|
}
|
|
|
|
fn cmd_server_status(ctx: *Context, _: *wire.Message, reply: *wire.Reply) !void {
|
|
const now = std.Io.Timestamp.now(ctx.io, .real);
|
|
const uptime: i64 = std.Io.Timestamp.durationTo(ctx.server_start, now).toSeconds();
|
|
|
|
try reply.put("host", .{ .string = "localhost" });
|
|
try reply.put("version", .{ .string = "4.4.0" });
|
|
try reply.put("process", .{ .string = "mongod" });
|
|
try reply.put("uptime", .{ .double = @floatFromInt(uptime) });
|
|
try reply.put("localTime", .{ .datetime = now.toMilliseconds() });
|
|
const connections = try reply.arena_alloc().alloc(bson.Pair, 1);
|
|
connections[0] = .{ .key = "current", .value = .{ .int32 = @intCast(ctx.connection_id) } };
|
|
try reply.put("connections", .{ .doc = connections });
|
|
try reply.put_ok();
|
|
}
|
|
|
|
fn cmd_end_sessions(_: *Context, _: *wire.Message, reply: *wire.Reply) !void {
|
|
try reply.put_ok();
|
|
}
|
|
|
|
fn cmd_connection_status(_: *Context, _: *wire.Message, reply: *wire.Reply) !void {
|
|
const auth_info = try reply.arena_alloc().alloc(bson.Pair, 2);
|
|
auth_info[0] = .{ .key = "authenticatedUsers", .value = .{ .array = &.{} } };
|
|
auth_info[1] = .{ .key = "authenticatedUserRoles", .value = .{ .array = &.{} } };
|
|
try reply.put("authInfo", .{ .doc = auth_info });
|
|
try reply.put_ok();
|
|
}
|
|
|
|
fn cmd_list_databases(ctx: *Context, _: *wire.Message, reply: *wire.Reply) !void {
|
|
var names: std.ArrayListUnmanaged([]const u8) = .empty;
|
|
defer names.deinit(ctx.gpa);
|
|
try ctx.engine.database_names(&names);
|
|
|
|
const values = try reply.arena_alloc().alloc(bson.Value, names.items.len);
|
|
for (names.items, 0..) |n, i| {
|
|
const entry = try reply.arena_alloc().alloc(bson.Pair, 3);
|
|
entry[0] = .{ .key = "name", .value = .{ .string = try reply.arena_alloc().dupe(u8, n) } };
|
|
entry[1] = .{ .key = "sizeOnDisk", .value = .{ .double = 0 } };
|
|
entry[2] = .{ .key = "empty", .value = .{ .bool = true } };
|
|
values[i] = .{ .doc = entry };
|
|
}
|
|
try reply.put("databases", .{ .array = values });
|
|
try reply.put("totalSize", .{ .int32 = 0 });
|
|
try reply.put("totalSizeMb", .{ .int32 = 0 });
|
|
try reply.put_ok();
|
|
}
|
|
|
|
/// The collection part of the namespace a `listCollections` cursor reports.
|
|
/// mongod uses this pseudo-collection, and the exact string matters: the previous
|
|
/// `"<db>."` had an *empty* collection part, and the driver throws client-side
|
|
/// when it tries to build a getMore or killCursors from a namespace like that --
|
|
/// so the moment such a cursor stopped being id 0 it would have broken.
|
|
const list_collections_ns: []const u8 = "$cmd.listCollections";
|
|
|
|
fn cmd_list_collections(ctx: *Context, msg: *wire.Message, reply: *wire.Reply) !void {
|
|
const db_name = msg.db_name() orelse return invalid_arg(reply, "listCollections requires $db");
|
|
const batch_size = (try aggregate_batch_size(reply, msg) orelse return).value;
|
|
var names: std.ArrayListUnmanaged([]const u8) = .empty;
|
|
defer names.deinit(ctx.gpa);
|
|
try ctx.engine.collection_names(db_name, &names);
|
|
|
|
const arena = reply.arena_alloc();
|
|
const docs = try arena.alloc(*const bson.Document, names.items.len);
|
|
for (names.items, 0..) |n, i| {
|
|
const entry = try arena.alloc(bson.Pair, 3);
|
|
entry[0] = .{ .key = "name", .value = .{ .string = try arena.dupe(u8, n) } };
|
|
entry[1] = .{ .key = "type", .value = .{ .string = "collection" } };
|
|
entry[2] = .{ .key = "options", .value = .{ .doc = &.{} } };
|
|
docs[i] = try doc_from_pairs(arena, entry);
|
|
}
|
|
try emit_first_batch(ctx, reply, db_name, list_collections_ns, null, docs, batch_size);
|
|
try reply.put_ok();
|
|
}
|
|
|
|
fn cmd_create(ctx: *Context, msg: *wire.Message, reply: *wire.Reply) !void {
|
|
const db_name = msg.db_name() orelse return invalid_arg(reply, "create requires $db");
|
|
const coll_name = str_arg(msg.body.get("create")) orelse return bad_value(reply, "create requires a collection name");
|
|
_ = try ctx.engine.get_or_create_collection(db_name, coll_name);
|
|
try reply.put_ok();
|
|
}
|
|
|
|
fn cmd_drop(ctx: *Context, msg: *wire.Message, reply: *wire.Reply) !void {
|
|
const db_name = msg.db_name() orelse return invalid_arg(reply, "drop requires $db");
|
|
const coll_name = str_arg(msg.body.get("drop")) orelse return bad_value(reply, "drop requires a collection name");
|
|
if (!try ctx.engine.drop_collection(db_name, coll_name)) {
|
|
return reply.put_error(@intFromEnum(ErrorCode.namespace_not_found), "NamespaceNotFound", "ns not found");
|
|
}
|
|
try reply.put_ok();
|
|
}
|
|
|
|
fn cmd_drop_database(ctx: *Context, msg: *wire.Message, reply: *wire.Reply) !void {
|
|
const db_name = msg.db_name() orelse return invalid_arg(reply, "dropDatabase requires $db");
|
|
_ = try ctx.engine.drop_database(db_name);
|
|
try reply.put("dropped", .{ .string = try reply.arena_alloc().dupe(u8, db_name) });
|
|
try reply.put_ok();
|
|
}
|
|
|
|
// ---------------------------------------------------------------------------
|
|
// Indexes
|
|
// ---------------------------------------------------------------------------
|
|
|
|
fn cmd_create_indexes(ctx: *Context, msg: *wire.Message, reply: *wire.Reply) !void {
|
|
const db_name = msg.db_name() orelse return invalid_arg(reply, "createIndexes requires $db");
|
|
const coll_name = str_arg(msg.body.get("createIndexes")) orelse return bad_value(reply, "createIndexes requires a collection name");
|
|
const indexes = switch (msg.body.get("indexes") orelse return bad_value(reply, "createIndexes requires indexes")) {
|
|
.array => |a| a,
|
|
else => return bad_value(reply, "indexes must be an array"),
|
|
};
|
|
|
|
const existed = ctx.engine.get_collection(db_name, coll_name) != null;
|
|
const coll = try ctx.engine.get_or_create_collection(db_name, coll_name);
|
|
const num_before: i32 = @intCast(coll.indexes.items.len + 1); // + the _id_ index
|
|
|
|
for (indexes) |spec_v| {
|
|
const spec = switch (spec_v) {
|
|
.doc => |p| p,
|
|
else => return bad_value(reply, "indexes must be documents"),
|
|
};
|
|
const key_value = bson.get_pair(spec, "key") orelse return bad_value(reply, "index spec requires key");
|
|
const key_pairs = switch (key_value) {
|
|
.doc => |p| p,
|
|
else => return bad_value(reply, "key must be a document"),
|
|
};
|
|
if (key_pairs.len == 0) return bad_value(reply, "cannot create index with an empty key");
|
|
|
|
// {_id: 1} is the implicit index: an idempotent no-op. Any other
|
|
// secondary index touching _id is rejected.
|
|
var has_id = false;
|
|
for (key_pairs) |p| {
|
|
if (std.mem.eql(u8, p.key, "_id")) has_id = true;
|
|
}
|
|
if (has_id) {
|
|
const only = key_pairs[0].value;
|
|
const is_id_index = key_pairs.len == 1 and ((only == .int32 and only.int32 == 1) or
|
|
(only == .int64 and only.int64 == 1) or
|
|
(only == .double and only.double == 1.0));
|
|
if (is_id_index) {
|
|
// The no-op must not swallow options that would change what
|
|
// the index does — MongoDB rejects a TTL _id index rather
|
|
// than quietly ignoring the expiry.
|
|
if (bson.get_pair(spec, "expireAfterSeconds") != null) {
|
|
return reply.put_error(
|
|
@intFromEnum(ErrorCode.invalid_index_specification_option),
|
|
"InvalidIndexSpecificationOption",
|
|
"the field 'expireAfterSeconds' is not valid for an _id " ++
|
|
"index specification",
|
|
);
|
|
}
|
|
continue;
|
|
}
|
|
return bad_value(reply, "cannot create a secondary index on the _id field");
|
|
}
|
|
const name = bson.get_pair(spec, "name") orelse bson.Value.null;
|
|
if (name == .string and std.mem.eql(u8, name.string, "_id_")) {
|
|
return bad_value(reply, "cannot create index with name '_id_'");
|
|
}
|
|
|
|
const spec_doc = bson.Document{ .arena = undefined, .pairs = spec };
|
|
_ = ctx.engine.create_index(db_name, coll_name, &spec_doc) catch |err| switch (err) {
|
|
error.InvalidIndexSpec => return bad_value(reply, "invalid index spec"),
|
|
error.TtlOnCompoundIndex => return reply.put_error(
|
|
@intFromEnum(ErrorCode.cannot_create_index),
|
|
"CannotCreateIndex",
|
|
"TTL indexes are single-field indexes, compound indexes do not support TTL",
|
|
),
|
|
error.InvalidExpireAfterSeconds => return reply.put_error(
|
|
@intFromEnum(ErrorCode.cannot_create_index),
|
|
"CannotCreateIndex",
|
|
"TTL index 'expireAfterSeconds' option must be a whole number " ++
|
|
"between 0 and 2147483647",
|
|
),
|
|
error.IndexOptionsConflict => return reply.put_error(@intFromEnum(ErrorCode.index_options_conflict), "IndexOptionsConflict", "index already exists with a different specification"),
|
|
error.DuplicateKeyIndex => {
|
|
const ix_name = if (name == .string) name.string else "index";
|
|
const msg_text = try e11000_message(reply, db_name, coll_name, ix_name, try render_spec_key(reply, key_pairs));
|
|
return reply.put_error(@intFromEnum(ErrorCode.duplicate_key), "DuplicateKey", msg_text);
|
|
},
|
|
error.ParallelArrays => return bad_value(reply, "cannot index parallel arrays"),
|
|
else => return err,
|
|
};
|
|
}
|
|
|
|
try reply.put("createdCollectionAutomatically", .{ .bool = !existed });
|
|
try reply.put("numIndexesBefore", .{ .int32 = num_before });
|
|
try reply.put("numIndexesAfter", .{ .int32 = @intCast(coll.indexes.items.len + 1) });
|
|
try reply.put_ok();
|
|
}
|
|
|
|
fn cmd_list_indexes(ctx: *Context, msg: *wire.Message, reply: *wire.Reply) !void {
|
|
const db_name = msg.db_name() orelse return invalid_arg(reply, "listIndexes requires $db");
|
|
const coll_name = str_arg(msg.body.get("listIndexes")) orelse return bad_value(reply, "listIndexes requires a collection name");
|
|
const coll = ctx.engine.get_collection(db_name, coll_name) orelse
|
|
return reply.put_error(@intFromEnum(ErrorCode.namespace_not_found), "NamespaceNotFound", "ns not found");
|
|
|
|
const batch_size = (try aggregate_batch_size(reply, msg) orelse return).value;
|
|
// The _id_ index first, then the secondaries.
|
|
const arena = reply.arena_alloc();
|
|
const values = try arena.alloc(*const bson.Document, coll.indexes.items.len + 1);
|
|
const id_pairs = try arena.alloc(bson.Pair, 2);
|
|
id_pairs[0] = .{ .key = "v", .value = .{ .int32 = 2 } };
|
|
id_pairs[1] = .{ .key = "key", .value = .{ .doc = &.{.{ .key = "_id", .value = .{ .int32 = 1 } }} } };
|
|
values[0] = try doc_from_pairs(arena, try index_pairs_append(reply, id_pairs, "_id_"));
|
|
for (coll.indexes.items, 0..) |ix, i| {
|
|
// The pairs live in the reply arena (freed with it); the docs array
|
|
// below references them.
|
|
var pairs: std.ArrayListUnmanaged(bson.Pair) = .empty;
|
|
try ix.spec_pairs(arena, &pairs);
|
|
values[1 + i] = try doc_from_pairs(arena, pairs.items);
|
|
}
|
|
try emit_first_batch(ctx, reply, db_name, coll_name, null, values, batch_size);
|
|
try reply.put_ok();
|
|
}
|
|
|
|
/// The _id_ index entry: {v, key: {_id: 1}, name: "_id_"}.
|
|
fn index_pairs_append(
|
|
reply: *wire.Reply,
|
|
pairs: []const bson.Pair,
|
|
name: []const u8,
|
|
) ![]const bson.Pair {
|
|
const arena = reply.arena_alloc();
|
|
const with_name = try arena.alloc(bson.Pair, pairs.len + 1);
|
|
@memcpy(with_name[0..pairs.len], pairs);
|
|
with_name[pairs.len] = .{ .key = "name", .value = .{ .string = try arena.dupe(u8, name) } };
|
|
return with_name;
|
|
}
|
|
|
|
fn cmd_drop_indexes(ctx: *Context, msg: *wire.Message, reply: *wire.Reply) !void {
|
|
const db_name = msg.db_name() orelse return invalid_arg(reply, "dropIndexes requires $db");
|
|
const coll_name = str_arg(msg.body.get("dropIndexes")) orelse return bad_value(reply, "dropIndexes requires a collection name");
|
|
const coll = ctx.engine.get_collection(db_name, coll_name) orelse
|
|
return reply.put_error(@intFromEnum(ErrorCode.namespace_not_found), "NamespaceNotFound", "ns not found");
|
|
const n_indexes_was: i32 = @intCast(coll.indexes.items.len + 1);
|
|
|
|
const arg = msg.body.get("index") orelse return bad_value(reply, "dropIndexes requires index");
|
|
if (arg == .string and std.mem.eql(u8, arg.string, "*")) {
|
|
// Drop every secondary index. Copy the names first: each drop
|
|
// mutates the collection's index list.
|
|
var names: std.ArrayListUnmanaged([]const u8) = .empty;
|
|
defer names.deinit(ctx.gpa);
|
|
for (coll.indexes.items) |ix| try names.append(ctx.gpa, ix.name);
|
|
for (names.items) |nm| _ = try ctx.engine.drop_index(db_name, coll_name, nm);
|
|
} else if (arg == .string) {
|
|
if (std.mem.eql(u8, arg.string, "_id_")) {
|
|
return invalid_arg(reply, "cannot drop the _id index");
|
|
}
|
|
if (!try ctx.engine.drop_index(db_name, coll_name, arg.string)) {
|
|
return reply.put_error(@intFromEnum(ErrorCode.index_not_found), "IndexNotFound", "index not found with name");
|
|
}
|
|
} else if (arg == .doc) {
|
|
// A key document: drop the index with a matching key pattern.
|
|
const key_value = bson.get_pair(arg.doc, "key") orelse bson.Value{ .doc = arg.doc };
|
|
const key_pairs = switch (key_value) {
|
|
.doc => |p| p,
|
|
else => return bad_value(reply, "dropIndexes index must be a name, key document, or '*'"),
|
|
};
|
|
const target = index.find_by_key_pattern(coll.indexes.items, key_pairs) orelse
|
|
return reply.put_error(@intFromEnum(ErrorCode.index_not_found), "IndexNotFound", "index not found with key pattern");
|
|
_ = try ctx.engine.drop_index(db_name, coll_name, target.name);
|
|
} else {
|
|
return bad_value(reply, "dropIndexes index must be a name, key document, or '*'");
|
|
}
|
|
|
|
try reply.put("nIndexesWas", .{ .int32 = n_indexes_was });
|
|
try reply.put_ok();
|
|
}
|
|
|
|
/// The E11000 text drivers parse. One definition for both create-time and
|
|
/// write-time conflicts; they differ only in how the dup key is rendered.
|
|
fn e11000_message(
|
|
reply: *wire.Reply,
|
|
db_name: []const u8,
|
|
coll_name: []const u8,
|
|
index_name: []const u8,
|
|
key_text: []const u8,
|
|
) ![]const u8 {
|
|
return std.fmt.allocPrint(
|
|
reply.arena_alloc(),
|
|
"E11000 duplicate key error collection: {s}.{s} index: {s} dup key: {s}",
|
|
.{ db_name, coll_name, index_name, key_text },
|
|
);
|
|
}
|
|
|
|
/// Render the key pattern with placeholder values for a createIndexes
|
|
/// duplicate-key error (no specific document is involved).
|
|
fn render_spec_key(reply: *wire.Reply, key_pairs: []const bson.Pair) ![]const u8 {
|
|
const arena = reply.arena_alloc();
|
|
var out: std.ArrayListUnmanaged(u8) = .empty;
|
|
defer out.deinit(arena);
|
|
try out.append(arena, '{');
|
|
for (key_pairs, 0..) |p, i| {
|
|
if (i > 0) try out.appendSlice(arena, ", ");
|
|
try out.appendSlice(arena, p.key);
|
|
try out.appendSlice(arena, ": ?");
|
|
}
|
|
try out.append(arena, '}');
|
|
return out.toOwnedSlice(arena);
|
|
}
|
|
|
|
// ---------------------------------------------------------------------------
|
|
// CRUD
|
|
// ---------------------------------------------------------------------------
|
|
|
|
fn cmd_insert(ctx: *Context, msg: *wire.Message, reply: *wire.Reply) !void {
|
|
const db_name = msg.db_name() orelse return invalid_arg(reply, "insert requires $db");
|
|
const coll_name = str_arg(msg.body.get("insert")) orelse return bad_value(reply, "insert requires a collection name");
|
|
|
|
const docs = try batch_arg(msg, reply, "insert", "documents") orelse return;
|
|
|
|
var inserted: i64 = 0;
|
|
var write_errors: std.ArrayListUnmanaged(bson.Value) = .empty;
|
|
defer write_errors.deinit(reply.arena_alloc());
|
|
|
|
// Group commit: one fsync for the whole batch instead of one per document.
|
|
// Nothing to open or close here -- appends never sync, and the dispatch
|
|
// epilogue is the single commit point. It runs on every return path, so a
|
|
// failed doc (writeErrors) or a hard error still syncs what was appended,
|
|
// and does it after the collection lock is released.
|
|
for (docs, 0..) |*doc, i| {
|
|
if (ctx.engine.insert(db_name, coll_name, doc, ctx.oid_gen)) |_| {
|
|
inserted += 1;
|
|
} else |err| {
|
|
switch (err) {
|
|
error.DuplicateKey, error.DuplicateKeyIndex => {
|
|
const e = try reply.arena_alloc().alloc(bson.Pair, 3);
|
|
e[0] = .{ .key = "index", .value = .{ .int32 = @intCast(i) } };
|
|
e[1] = .{ .key = "code", .value = .{ .int32 = @intFromEnum(ErrorCode.duplicate_key) } };
|
|
e[2] = .{ .key = "errmsg", .value = .{ .string = try duplicate_key_message(ctx, reply, db_name, coll_name, doc) } };
|
|
try write_errors.append(reply.arena_alloc(), .{ .doc = e });
|
|
},
|
|
else => return err,
|
|
}
|
|
}
|
|
}
|
|
|
|
try reply.put("n", .{ .int32 = @intCast(inserted) });
|
|
if (write_errors.items.len > 0) {
|
|
// Copy into the reply arena: write_errors is freed when this command
|
|
// returns, before the reply is serialized.
|
|
const arr = try reply.arena_alloc().alloc(bson.Value, write_errors.items.len);
|
|
@memcpy(arr, write_errors.items);
|
|
try reply.put("writeErrors", .{ .array = arr });
|
|
}
|
|
try reply.put_ok();
|
|
}
|
|
|
|
/// Parse a `batchSize`-shaped option. Null means the error reply is already
|
|
/// written. `zero_is_default` distinguishes `find`, where 0 is a real request for
|
|
/// an empty batch, from `getMore`, where mongod reads it as "no document target"
|
|
/// -- which is also what an absent field means.
|
|
fn batch_size_arg(
|
|
reply: *wire.Reply,
|
|
v: bson.Value,
|
|
label: []const u8,
|
|
zero_is_default: bool,
|
|
) !??u32 {
|
|
const n = int_value(v) orelse {
|
|
const text = try std.fmt.allocPrint(reply.arena_alloc(), "{s} must be a number", .{label});
|
|
try bad_value(reply, text);
|
|
return null;
|
|
};
|
|
if (n < 0) {
|
|
const text = try std.fmt.allocPrint(
|
|
reply.arena_alloc(),
|
|
"{s} value must be non-negative",
|
|
.{label},
|
|
);
|
|
try bad_value(reply, text);
|
|
return null;
|
|
}
|
|
if (n == 0 and zero_is_default) return @as(?u32, null);
|
|
return @as(?u32, std.math.cast(u32, n) orelse std.math.maxInt(u32));
|
|
}
|
|
|
|
/// The `find` options that shape the cursor rather than the query.
|
|
const CursorOpts = struct {
|
|
batch_size: ?u32 = null,
|
|
/// One batch and no cursor. Set explicitly, and also by a negative `limit`.
|
|
single_batch: bool = false,
|
|
no_timeout: bool = false,
|
|
};
|
|
|
|
/// Parse the cursor-shaping options, or write an error reply and return null.
|
|
fn parse_cursor_opts(reply: *wire.Reply, msg: *wire.Message) !?CursorOpts {
|
|
// Every tailable form is refused, and that is parity rather than a gap:
|
|
// mongod rejects a tailable cursor on a non-capped collection, and this
|
|
// engine has no capped collections at all. Silently ignoring the flag would
|
|
// be worse than erroring -- the cursor would report EOF and a driver's tail
|
|
// loop would exit, which reads to the application as data loss.
|
|
if (bool_arg(msg.body.get("tailable")) orelse false) {
|
|
try bad_value(reply, "tailable cursor requested on non capped collection");
|
|
return null;
|
|
}
|
|
if (bool_arg(msg.body.get("awaitData")) orelse false) {
|
|
try bad_value(reply, "Cannot set 'awaitData' without also setting 'tailable'");
|
|
return null;
|
|
}
|
|
|
|
var opts = CursorOpts{};
|
|
if (msg.body.get("batchSize")) |v| {
|
|
opts.batch_size = try batch_size_arg(reply, v, "batchSize", false) orelse return null;
|
|
}
|
|
opts.single_batch = bool_arg(msg.body.get("singleBatch")) orelse false;
|
|
opts.no_timeout = bool_arg(msg.body.get("noCursorTimeout")) orelse false;
|
|
return opts;
|
|
}
|
|
|
|
/// Whether a source may outlive the request that built it.
|
|
///
|
|
/// Only a snapshot can fail this: it is the one arm that pins bytes proportional
|
|
/// to the result set, and holding a large one for the idle timeout times the
|
|
/// number of live cursors is exactly what `cursor_buffer_max` exists to prevent.
|
|
/// Over the bound the caller emits everything in one batch instead -- which is
|
|
/// what this server did before cursors existed.
|
|
fn source_keepable(source: cursor.Source) bool {
|
|
const buffered = switch (source) {
|
|
.buffered => |b| b,
|
|
else => return true,
|
|
};
|
|
var total: u64 = 0;
|
|
for (buffered.docs) |d| total += d.len;
|
|
return total <= cursor_buffer_max;
|
|
}
|
|
|
|
fn cmd_find(ctx: *Context, msg: *wire.Message, reply: *wire.Reply) !void {
|
|
const db_name = msg.db_name() orelse return invalid_arg(reply, "find requires $db");
|
|
const coll_name = str_arg(msg.body.get("find")) orelse return bad_value(reply, "find requires a collection name");
|
|
const filter = doc_arg(msg.body.get("filter")) orelse return bad_value(reply, "filter must be a document");
|
|
const sort_keys = try parse_sort_keys(reply, msg.body.get("sort"));
|
|
const proj_pairs = doc_arg(msg.body.get("projection"));
|
|
const skip: u64 = int_arg(msg.body.get("skip")) orelse 0;
|
|
const limit_raw = int_value(msg.body.get("limit")) orelse 0;
|
|
const opts = try parse_cursor_opts(reply, msg) orelse return;
|
|
|
|
// A negative limit is the historical `ntoreturn` shape: this many documents
|
|
// in exactly one batch. Drivers translate it before sending, but the wire
|
|
// form is still legal.
|
|
const single_batch = opts.single_batch or limit_raw < 0;
|
|
const limit: u64 = @abs(limit_raw);
|
|
const target = cursor.batch_target(opts.batch_size, true);
|
|
|
|
// A find on a namespace that does not exist is an empty cursor, not an
|
|
// error, and the ordinary path below says exactly that.
|
|
const coll = ctx.engine.get_collection(db_name, coll_name);
|
|
|
|
var arena = std.heap.ArenaAllocator.init(ctx.gpa);
|
|
var arena_owned = false;
|
|
defer if (!arena_owned) arena.deinit();
|
|
|
|
// Whether this reply is the whole answer, which is what makes the top-k
|
|
// sort shortcut legal -- it leaves everything past k unspecified, and an
|
|
// open cursor would later need those documents.
|
|
const closes_here = single_batch or (limit > 0 and limit <= (target orelse 0));
|
|
|
|
var feed = try find_feed(ctx, reply, &arena, coll, .{
|
|
.db_name = db_name,
|
|
.coll_name = coll_name,
|
|
.filter = filter,
|
|
.sort_keys = sort_keys,
|
|
.skip = skip,
|
|
.limit = limit,
|
|
.closes_here = closes_here,
|
|
});
|
|
|
|
var values: std.ArrayListUnmanaged(bson.Value) = .empty;
|
|
const exhausted = try fill_batch(ctx, reply, coll, &feed, filter, proj_pairs, target, &values);
|
|
|
|
var cursor_id: i64 = 0;
|
|
if (!exhausted and !single_batch and source_keepable(feed.source)) {
|
|
cursor_id = keep_find_cursor(ctx, &arena, coll, &feed, .{
|
|
.db = db_name,
|
|
.coll = coll_name,
|
|
}, filter, proj_pairs, opts);
|
|
arena_owned = cursor_id != 0;
|
|
}
|
|
// No cursor, but documents still to come: one batch holding the rest.
|
|
if (cursor_id == 0 and !exhausted and !single_batch) {
|
|
_ = try fill_batch(ctx, reply, coll, &feed, filter, proj_pairs, null, &values);
|
|
}
|
|
|
|
const ns = try format_namespace(reply, db_name, coll_name);
|
|
const batch = try cursor_doc(reply, cursor_id, ns, "firstBatch", values.items);
|
|
try reply.put("cursor", .{ .doc = batch });
|
|
try reply.put_ok();
|
|
}
|
|
|
|
/// Register the remainder of a `find` as a cursor, returning its id or 0.
|
|
fn keep_find_cursor(
|
|
ctx: *Context,
|
|
arena: *std.heap.ArenaAllocator,
|
|
coll: ?*Collection,
|
|
feed: *const Feed,
|
|
ns: cursor.Ns,
|
|
filter: []const bson.Pair,
|
|
proj_pairs: ?[]const bson.Pair,
|
|
opts: CursorOpts,
|
|
) i64 {
|
|
// Serialized before the arena is handed over: `open_cursor` takes it by
|
|
// value, so anything allocated after that call would be invisible to the copy
|
|
// the cursor keeps.
|
|
const a = arena.allocator();
|
|
const filter_bytes = serialize_pairs(a, filter) catch return 0;
|
|
const proj_bytes = if (proj_pairs) |pp| (serialize_pairs(a, pp) catch return 0) else "";
|
|
return open_cursor(ctx, arena.*, .{
|
|
.ns = ns,
|
|
.layout_epoch = if (coll) |c| c.layout_epoch else 0,
|
|
.filter_bytes = filter_bytes,
|
|
.proj_bytes = proj_bytes,
|
|
.remaining_limit = feed.remaining_limit,
|
|
.batch_size = opts.batch_size,
|
|
.no_timeout = opts.no_timeout,
|
|
// feed.source, not the source it started as: the first batch advanced it,
|
|
// and for a stream that advance *is* the resume point.
|
|
.source = feed.source,
|
|
});
|
|
}
|
|
|
|
/// Everything `find_feed` needs that is not a lock or an arena.
|
|
const FindRequest = struct {
|
|
db_name: []const u8,
|
|
coll_name: []const u8,
|
|
filter: []const bson.Pair,
|
|
sort_keys: []const query.SortKey,
|
|
skip: u64,
|
|
limit: u64,
|
|
closes_here: bool,
|
|
};
|
|
|
|
/// Choose the source for a `find` and open the feed that fills its first batch.
|
|
///
|
|
/// A whole-index walk is served without materializing anything: the cursor
|
|
/// remembers a key and an offset, so this is what makes `find({})` over a
|
|
/// collection larger than memory possible at all. Every other shape collects its
|
|
/// matches first, exactly as before cursors existed.
|
|
fn find_feed(
|
|
ctx: *Context,
|
|
reply: *wire.Reply,
|
|
arena: *std.heap.ArenaAllocator,
|
|
coll: ?*Collection,
|
|
req: FindRequest,
|
|
) !Feed {
|
|
const remaining: ?u64 = if (req.limit == 0) null else req.limit;
|
|
|
|
// One plan for the whole request: `stream_shape` reads it to decide whether a
|
|
// resumable walk is possible, and the fallback scan reuses it instead of
|
|
// planning the same query a second time.
|
|
var plan_opt = if (coll) |c|
|
|
try index.plan(ctx.gpa, &c.id_index, c.indexes.items, req.filter, req.sort_keys)
|
|
else
|
|
null;
|
|
defer if (plan_opt) |*p| p.deinit(ctx.gpa);
|
|
const plan: ?*const index.Plan = if (plan_opt) |*p| p else null;
|
|
|
|
if (coll) |c| {
|
|
if (stream_shape(c, plan, req.sort_keys)) |st| {
|
|
var feed = Feed{ .source = .{ .stream = st }, .remaining_limit = remaining };
|
|
feed.scan = open_scan(c, &feed.source.stream);
|
|
// `skip` is consumed once, here, through the same anchor the batch
|
|
// uses -- which is what lets a cursor whose entire first batch falls
|
|
// inside the skipped prefix still resume from the right place.
|
|
try stream_skip(ctx, c, &feed, req.filter, req.skip);
|
|
return feed;
|
|
}
|
|
}
|
|
|
|
// Documents needed to fill the page, counting the skipped prefix; 0 means
|
|
// unbounded. Only an index-ordered scan may stop there.
|
|
const page_end: usize = if (req.limit == 0) 0 else blk: {
|
|
const skip_usize = std.math.cast(usize, req.skip) orelse break :blk 0;
|
|
break :blk skip_usize +| req.limit;
|
|
};
|
|
// Lives only until `find_source` copies what it needs into `arena`.
|
|
var matched: std.ArrayListUnmanaged(u64) = .empty;
|
|
defer matched.deinit(ctx.gpa);
|
|
var index_sorted = false;
|
|
_ = try scan_planned(
|
|
ctx,
|
|
req.db_name,
|
|
req.coll_name,
|
|
req.filter,
|
|
page_end,
|
|
&matched,
|
|
req.sort_keys,
|
|
&index_sorted,
|
|
plan,
|
|
);
|
|
if (coll == null) {
|
|
assert_msg(matched.items.len == 0, "find matched in a collection that does not exist");
|
|
}
|
|
const source = try find_source(reply, arena, coll, matched.items, .{
|
|
.sort_keys = req.sort_keys,
|
|
.index_sorted = index_sorted,
|
|
.skip = req.skip,
|
|
.page_end = page_end,
|
|
.closes_here = req.closes_here,
|
|
});
|
|
return .{ .source = source, .remaining_limit = remaining };
|
|
}
|
|
|
|
/// How `find_source` should turn a match list into a source.
|
|
const SourceShape = struct {
|
|
sort_keys: []const query.SortKey,
|
|
index_sorted: bool,
|
|
skip: u64,
|
|
page_end: usize,
|
|
closes_here: bool,
|
|
};
|
|
|
|
/// Build the source `find` pulls from, with `skip` already consumed.
|
|
///
|
|
/// The query shape decides which source is possible. An index-ordered scan
|
|
/// yields offsets -- 8 bytes apiece, and valid until the collection is rebuilt.
|
|
/// A sort no index provides had to materialize and order every match, so there
|
|
/// is no ordered offset list to point at and the remainder is snapshotted.
|
|
fn find_source(
|
|
reply: *wire.Reply,
|
|
arena: *std.heap.ArenaAllocator,
|
|
coll: ?*Collection,
|
|
matched: []const u64,
|
|
shape: SourceShape,
|
|
) !cursor.Source {
|
|
const c = coll orelse return .{ .offsets = .{ .items = &.{} } };
|
|
if (shape.sort_keys.len == 0 or shape.index_sorted) {
|
|
// Already in the order the client asked for, so skip is a slice.
|
|
const rest = if (shape.skip < matched.len) matched[shape.skip..] else &.{};
|
|
return .{ .offsets = .{ .items = try arena.allocator().dupe(u64, rest) } };
|
|
}
|
|
|
|
// Ordering needs the values: materialize into the reply arena and sort.
|
|
const reply_arena = reply.arena_alloc();
|
|
var tree_docs: std.ArrayListUnmanaged(*const bson.Document) = .empty;
|
|
for (matched) |off| try tree_docs.append(reply_arena, try doc_tree(reply_arena, c, off));
|
|
if (shape.closes_here and shape.page_end > 0 and shape.page_end *| 4 <= tree_docs.items.len) {
|
|
try query.sort_docs_top_k(reply_arena, tree_docs.items, shape.sort_keys, shape.page_end);
|
|
} else {
|
|
try query.sort_docs(reply_arena, tree_docs.items, shape.sort_keys);
|
|
}
|
|
const rest = if (shape.skip < tree_docs.items.len) tree_docs.items[shape.skip..] else &.{};
|
|
return buffered_source(arena.allocator(), rest);
|
|
}
|
|
|
|
/// Serialize `pairs` into `arena`. A cursor cannot keep the parsed form: those
|
|
/// pairs point into the per-request message arena.
|
|
fn serialize_pairs(arena: std.mem.Allocator, pairs: []const bson.Pair) ![]const u8 {
|
|
var buf: std.ArrayListUnmanaged(u8) = .empty;
|
|
try bson.write_doc(pairs, arena, &buf);
|
|
return buf.items;
|
|
}
|
|
|
|
/// Collect the documents in `db_name.coll_name` matching `filter`, stopping
|
|
/// after `limit` matches (0 = unlimited). Returns the number matched; `out`
|
|
/// may be null when only the count is wanted. Every command that scans a
|
|
/// collection goes through here, so an index only needs this one call site.
|
|
///
|
|
/// Candidate generation order: the _id_ fast path (docs map lookup), then a
|
|
/// secondary-index plan, then a plain scan. Every candidate is re-checked
|
|
/// with the unchanged filter, so an index that over-approximates is merely
|
|
/// slow — never wrong. With no index created and no usable _id clause, the
|
|
/// plain-scan path is the only one reached.
|
|
fn scan_matching(
|
|
ctx: *Context,
|
|
db_name: []const u8,
|
|
coll_name: []const u8,
|
|
filter: []const bson.Pair,
|
|
limit: usize,
|
|
out: ?*std.ArrayListUnmanaged(u64),
|
|
) !usize {
|
|
return scan_sorted(ctx, db_name, coll_name, filter, limit, out, &.{}, null);
|
|
}
|
|
|
|
/// `scan_matching` plus the option of having an index produce the ordering.
|
|
/// When `sorted` is given it reports whether the candidates came out in
|
|
/// `sort` order, in which case the caller must not sort them again — and
|
|
/// `limit` is then a genuine early stop rather than an arbitrary subset.
|
|
fn scan_sorted(
|
|
ctx: *Context,
|
|
db_name: []const u8,
|
|
coll_name: []const u8,
|
|
filter: []const bson.Pair,
|
|
limit: usize,
|
|
out: ?*std.ArrayListUnmanaged(u64),
|
|
sort: []const query.SortKey,
|
|
sorted: ?*bool,
|
|
) !usize {
|
|
return scan_planned(ctx, db_name, coll_name, filter, limit, out, sort, sorted, null);
|
|
}
|
|
|
|
/// `scan_sorted` with the plan supplied. `find` decides between a resumable walk
|
|
/// and a materializing scan by looking at the plan, so without this it would plan
|
|
/// once to choose and `scan_sorted` would plan the identical query again --
|
|
/// `index.plan` flattens the filter's clauses and evaluates every index, both
|
|
/// allocating, on the hot read path.
|
|
///
|
|
/// `prebuilt` is borrowed: the caller keeps ownership, including its `deinit`.
|
|
fn scan_planned(
|
|
ctx: *Context,
|
|
db_name: []const u8,
|
|
coll_name: []const u8,
|
|
filter: []const bson.Pair,
|
|
limit: usize,
|
|
out: ?*std.ArrayListUnmanaged(u64),
|
|
sort: []const query.SortKey,
|
|
sorted: ?*bool,
|
|
prebuilt: ?*const index.Plan,
|
|
) !usize {
|
|
if (sorted) |flag| flag.* = false;
|
|
// Stopping early is only meaningful when the candidates come out in the
|
|
// order the caller asked for. Without a sort any subset of that size is
|
|
// a valid page; with one, the limit is honoured only if an index turns
|
|
// out to supply the ordering.
|
|
var lim: usize = if (sort.len == 0) limit else 0;
|
|
const coll = ctx.engine.get_collection(db_name, coll_name) orelse return 0;
|
|
var n: usize = 0;
|
|
|
|
// Index plan (the implicit _id_ index first, then the secondaries):
|
|
// candidates in index order, re-filtered. A candidate *is* a slab offset
|
|
// now, so the map lookup that used to translate an id into one is gone --
|
|
// and so is the accidental safety net it provided: a stale entry used to be
|
|
// dropped silently by `orelse continue`, where now it resolves to
|
|
// superseded-but-parseable bytes that the re-applied filter might accept.
|
|
// Loud beats silent: a wrong answer a test can see beats a missing
|
|
// candidate nothing can.
|
|
// Candidates arrive as a stream so that a whole-index read never
|
|
// materializes: at the tens-of-GB target a `countDocuments({})` would
|
|
// otherwise build a list of every offset in the collection before the
|
|
// first one is examined. A narrowed plan still materializes, because its
|
|
// multikey/$in dedupe genuinely needs the whole set, and it is bounded by
|
|
// selectivity.
|
|
var offs: std.ArrayListUnmanaged(u64) = .empty;
|
|
defer offs.deinit(ctx.gpa);
|
|
var cands: index.Candidates = undefined;
|
|
// Only plan here when the caller did not; theirs is borrowed, so only ours
|
|
// is freed.
|
|
var owned_plan = if (prebuilt != null)
|
|
null
|
|
else
|
|
try index.plan(ctx.gpa, &coll.id_index, coll.indexes.items, filter, sort);
|
|
defer if (owned_plan) |*p| p.deinit(ctx.gpa);
|
|
const plan_opt: ?*const index.Plan = prebuilt orelse if (owned_plan) |*p| p else null;
|
|
|
|
if (plan_opt) |plan| {
|
|
if (sorted) |flag| flag.* = plan.provides_sort;
|
|
if (plan.provides_sort) lim = limit;
|
|
if (plan.full_scan()) {
|
|
cands = if (plan.backward)
|
|
.{ .scan_rev = plan.index.iter_reverse() }
|
|
else
|
|
.{ .scan = plan.index.iter() };
|
|
} else {
|
|
try plan.search(ctx.gpa, &offs);
|
|
cands = .{ .list = .{ .items = offs.items } };
|
|
}
|
|
} else {
|
|
// No usable predicate: every document, in _id order. The docs map was
|
|
// the fallback here, and its iteration order was the hash's; walking
|
|
// the _id_ index instead is ordered, streams, and does not depend on a
|
|
// structure that is going away.
|
|
cands = .{ .scan = coll.id_index.iter() };
|
|
}
|
|
|
|
while (cands.next()) |off| {
|
|
if (!try query.matches_bytes(ctx.gpa, filter, coll.doc_bytes(off))) continue;
|
|
if (out) |list| try list.append(ctx.gpa, off);
|
|
n += 1;
|
|
if (lim != 0 and n >= lim) break;
|
|
}
|
|
return n;
|
|
}
|
|
|
|
/// A stored document (a slab offset) materialized as a borrowed spine in
|
|
/// `arena`: keys and leaf values point into the slab's stable bytes, only
|
|
/// the pair/value skeleton is allocated. The arena owns the skeleton, so
|
|
/// the result is never deinit'd — the reply arena frees it with the reply.
|
|
fn doc_tree(arena: std.mem.Allocator, coll: *const Collection, off: u64) !*const bson.Document {
|
|
return doc_tree_bytes(arena, coll.doc_bytes(off));
|
|
}
|
|
|
|
/// The same borrowed spine over bytes that are already in hand -- the slab for a
|
|
/// live scan, or a cursor's own snapshot for a buffered one.
|
|
fn doc_tree_bytes(arena: std.mem.Allocator, bytes: []const u8) !*const bson.Document {
|
|
const doc = try arena.create(bson.Document);
|
|
doc.* = bson.Document{ .arena = undefined, .pairs = try bson.spine(arena, bytes) };
|
|
return doc;
|
|
}
|
|
|
|
// ---------------------------------------------------------------------------
|
|
// Cursors: filling a batch, and the sources a batch pulls from
|
|
// ---------------------------------------------------------------------------
|
|
|
|
/// Largest snapshot a cursor will copy into its own arena. Mirrors MongoDB's
|
|
/// 32 MiB in-memory sort limit, and only the shapes that had to materialize
|
|
/// anyway can reach it.
|
|
///
|
|
/// Over the bound the cursor is *declined* and the whole result goes out in one
|
|
/// batch -- exactly what this server did before cursors existed. Declining is
|
|
/// the right direction to fail: the alternative, holding the snapshot anyway,
|
|
/// pins it for the idle timeout times the number of live cursors.
|
|
const cursor_buffer_max: u64 = 32 * 1024 * 1024;
|
|
|
|
/// The part of a cursor that a batch consumes. Kept separate from
|
|
/// `cursor.Cursor` so `find` can fill its first batch and only then decide
|
|
/// whether a cursor needs to exist at all.
|
|
const Feed = struct {
|
|
source: cursor.Source,
|
|
/// Documents still owed across every remaining batch; null is unbounded.
|
|
remaining_limit: ?u64,
|
|
/// A `.stream` source's live position in the tree, valid for **this request
|
|
/// only**. It is deliberately not part of `cursor.Source`: a tree position
|
|
/// must never outlive the collection lock that made it safe to hold, which is
|
|
/// the whole reason the stored form is a value-typed anchor instead.
|
|
scan: ?Scan = null,
|
|
|
|
fn limit_exhausted(self: *const Feed) bool {
|
|
const rem = self.remaining_limit orelse return false;
|
|
return rem == 0;
|
|
}
|
|
};
|
|
|
|
/// A walk over one index, in one direction, for the duration of one request.
|
|
const Scan = struct {
|
|
walk: union(enum) {
|
|
fwd: index.Index.Iter,
|
|
rev: index.Index.RevIter,
|
|
},
|
|
/// The entry `peek_bytes` has produced but the batch has not yet accepted.
|
|
/// Held because a batch that turns out to be full must not consume it.
|
|
pending: ?index.Index.Positioned = null,
|
|
|
|
fn next(self: *Scan) ?index.Index.Positioned {
|
|
return switch (self.walk) {
|
|
.fwd => |*it| it.positioned(),
|
|
.rev => |*it| it.positioned(),
|
|
};
|
|
}
|
|
};
|
|
|
|
/// The next document the source will yield, *without* consuming it -- a batch
|
|
/// that turns out to be full must not swallow a document it cannot carry.
|
|
///
|
|
/// Candidates whose document no longer matches are consumed and skipped here.
|
|
/// That re-check is the index invariant (`src/index.zig`) applied per batch, and
|
|
/// it is also what makes a saved offset safe once documents start being
|
|
/// recycled: a reused offset either fails the filter or resolves to a document
|
|
/// that genuinely matches it.
|
|
fn peek_bytes(
|
|
ctx: *Context,
|
|
coll: ?*Collection,
|
|
filter: []const bson.Pair,
|
|
feed: *Feed,
|
|
) !?[]const u8 {
|
|
switch (feed.source) {
|
|
// A snapshot of documents that already matched, and that nothing can
|
|
// mutate underneath us -- so it is not re-filtered.
|
|
.buffered => |*b| {
|
|
if (b.next >= b.docs.len) return null;
|
|
return b.docs[b.next];
|
|
},
|
|
.offsets => |*o| {
|
|
const c = coll orelse return null;
|
|
while (o.next < o.items.len) {
|
|
const bytes = c.doc_bytes(o.items[o.next]);
|
|
if (try query.matches_bytes(ctx.gpa, filter, bytes)) return bytes;
|
|
o.next += 1;
|
|
}
|
|
return null;
|
|
},
|
|
.stream => {
|
|
const c = coll orelse return null;
|
|
const sc = &(feed.scan orelse return null);
|
|
if (sc.pending) |p| return c.doc_bytes(p.off);
|
|
while (sc.next()) |p| {
|
|
const bytes = c.doc_bytes(p.off);
|
|
if (!try query.matches_bytes(ctx.gpa, filter, bytes)) {
|
|
// A candidate the filter rejects is still progress: the
|
|
// anchor must move past it, or a resume would walk it again.
|
|
feed.source.stream.advance(p.key, p.off, p.leaf, p.slot);
|
|
continue;
|
|
}
|
|
sc.pending = p;
|
|
return bytes;
|
|
}
|
|
return null;
|
|
},
|
|
}
|
|
}
|
|
|
|
fn consume_one(feed: *Feed) void {
|
|
switch (feed.source) {
|
|
.buffered => |*b| b.next += 1,
|
|
.offsets => |*o| o.next += 1,
|
|
.stream => |*st| {
|
|
const sc = &(feed.scan orelse return);
|
|
const p = sc.pending orelse return;
|
|
st.advance(p.key, p.off, p.leaf, p.slot);
|
|
sc.pending = null;
|
|
},
|
|
}
|
|
}
|
|
|
|
/// Resolve the index a `.stream` cursor names. Empty is the implicit `_id_`,
|
|
/// which is deliberately kept out of `Collection.indexes` and so is not findable
|
|
/// by name.
|
|
fn stream_index(coll: *Collection, name: []const u8) ?*const index.Index {
|
|
if (name.len == 0) return &coll.id_index;
|
|
return coll.find_index(name);
|
|
}
|
|
|
|
/// Open this request's walk over the index a `.stream` cursor is following.
|
|
///
|
|
/// Null means the cursor can never produce another document: the index it was
|
|
/// following is gone. The caller turns that into `QueryPlanKilled` rather than
|
|
/// an empty batch, because an empty batch would claim the result set ended.
|
|
fn open_scan(coll: *Collection, st: *cursor.Stream) ?Scan {
|
|
const ix = stream_index(coll, st.index_name()) orelse return null;
|
|
// A hint is a node id plus a slot, and `reset_tree` recycles ids 0 and 1 as
|
|
// different nodes -- so the hint is only meaningful at the epoch it was
|
|
// taken. The *anchor* is unaffected: it is key bytes and an offset, both
|
|
// values, so an epoch change costs a band walk rather than correctness.
|
|
const trusted = st.index_epoch == ix.epoch;
|
|
st.index_epoch = ix.epoch;
|
|
|
|
if (!st.started()) {
|
|
// `batchSize: 0` leaves a cursor with no anchor yet, so a first getMore
|
|
// starts the walk from the beginning rather than resuming.
|
|
return .{ .walk = if (st.backward)
|
|
.{ .rev = ix.iter_reverse() }
|
|
else
|
|
.{ .fwd = ix.iter() } };
|
|
}
|
|
if (st.backward) {
|
|
const r = ix.resume_reverse(
|
|
st.anchor_key(),
|
|
st.anchor_off,
|
|
st.hint_leaf,
|
|
st.hint_slot,
|
|
trusted,
|
|
);
|
|
if (r.capped) return null;
|
|
return .{ .walk = .{ .rev = r.it } };
|
|
}
|
|
const r = ix.resume_forward(
|
|
st.anchor_key(),
|
|
st.anchor_off,
|
|
st.band_index,
|
|
st.hint_leaf,
|
|
st.hint_slot,
|
|
trusted,
|
|
);
|
|
if (r.capped) return null;
|
|
return .{ .walk = .{ .fwd = r.it } };
|
|
}
|
|
|
|
/// Whether this query can be served by walking one index end to end, and if so
|
|
/// which index and in which direction.
|
|
///
|
|
/// This is the shape that lets a cursor outlive its result set: it holds a key
|
|
/// and an offset instead of a list, so `find({})` over a collection larger than
|
|
/// memory costs O(key) of cursor state rather than 8 bytes per matching
|
|
/// document. Everything else keeps the materialized sources.
|
|
///
|
|
/// Two shapes qualify. A query the planner declines outright (`{}` with no sort)
|
|
/// walks the `_id_` index forward -- every document has an `_id` and the index is
|
|
/// not sparse, so a full walk cannot miss one. And a plan whose `full_scan()`
|
|
/// holds is by construction a whole-index walk in the requested order; note that
|
|
/// `full_scan()` implies `provides_sort`, since a plan with no run, no range and
|
|
/// no sort direction is declined before it is built.
|
|
///
|
|
/// A narrowed plan is excluded on purpose: it dedupes `$in` and multikey
|
|
/// candidates across the whole set, which a stream cannot do without remembering
|
|
/// what it has already emitted.
|
|
fn stream_shape(
|
|
coll: *Collection,
|
|
plan_opt: ?*const index.Plan,
|
|
sort: []const query.SortKey,
|
|
) ?cursor.Stream {
|
|
var st = cursor.Stream{};
|
|
if (plan_opt) |plan| {
|
|
if (!plan.full_scan()) return null;
|
|
st.backward = plan.backward;
|
|
if (plan.index != &coll.id_index) {
|
|
if (plan.index.name.len > cursor.index_name_max) return null;
|
|
@memcpy(st.index_name_buf[0..plan.index.name.len], plan.index.name);
|
|
st.index_name_len = @intCast(plan.index.name.len);
|
|
}
|
|
st.index_epoch = plan.index.epoch;
|
|
return st;
|
|
}
|
|
// No usable predicate and no ordering to honour: every document in _id
|
|
// order, which is what the old materializing fallback did too.
|
|
if (sort.len != 0) return null;
|
|
st.index_epoch = coll.id_index.epoch;
|
|
return st;
|
|
}
|
|
|
|
/// Consume `skip` matching documents without emitting them, advancing the anchor
|
|
/// as it goes so a resume does not walk them again.
|
|
fn stream_skip(
|
|
ctx: *Context,
|
|
coll: *Collection,
|
|
feed: *Feed,
|
|
filter: []const bson.Pair,
|
|
skip: u64,
|
|
) !void {
|
|
var left = skip;
|
|
while (left > 0) {
|
|
_ = try peek_bytes(ctx, coll, filter, feed) orelse return;
|
|
consume_one(feed);
|
|
left -= 1;
|
|
}
|
|
}
|
|
|
|
/// Fill one batch from `feed`, projecting into the reply arena. Returns whether
|
|
/// the source is **exhausted**.
|
|
///
|
|
/// Exhaustion is observed, never predicted: a batch that reached its document
|
|
/// target returns false even when the source happens to have nothing left, so
|
|
/// the cursor stays open and the client gets one more (possibly empty) batch.
|
|
/// Predicting it here would close the cursor a round trip early and break the
|
|
/// command-count assertions in the pinned spec suites.
|
|
fn fill_batch(
|
|
ctx: *Context,
|
|
reply: *wire.Reply,
|
|
coll: ?*Collection,
|
|
feed: *Feed,
|
|
filter: []const bson.Pair,
|
|
proj_pairs: ?[]const bson.Pair,
|
|
target: ?u32,
|
|
out: *std.ArrayListUnmanaged(bson.Value),
|
|
) !bool {
|
|
var builder = cursor.BatchBuilder.init(target);
|
|
const arena = reply.arena_alloc();
|
|
while (true) {
|
|
// Checked before the target so the batch that takes the last document
|
|
// the limit allows is itself the one that closes the cursor. This is
|
|
// what lets `batchSize == limit` finish in a single round trip.
|
|
if (feed.limit_exhausted()) return true;
|
|
if (builder.full()) return false;
|
|
const bytes = try peek_bytes(ctx, coll, filter, feed) orelse return true;
|
|
// The stored length is exact with no projection and an upper bound with
|
|
// one, since `query.project` only ever drops fields.
|
|
if (builder.offer(bytes.len) == .batch_full) return false;
|
|
const doc = try doc_tree_bytes(arena, bytes);
|
|
try out.append(arena, try project_doc(reply, doc, proj_pairs));
|
|
consume_one(feed);
|
|
if (feed.remaining_limit) |rem| feed.remaining_limit = rem - 1;
|
|
}
|
|
}
|
|
|
|
/// Register `feed`'s remainder as a cursor and return its id, or 0 when no
|
|
/// cursor is needed or one could not be had.
|
|
///
|
|
/// Every "could not" path degrades to `id: 0` rather than to an error: the
|
|
/// client then has the batch it was given and no cursor, which is precisely how
|
|
/// this server behaved before cursors existed.
|
|
fn open_cursor(ctx: *Context, arena: std.heap.ArenaAllocator, spec: cursor.OpenSpec) i64 {
|
|
var owned = arena;
|
|
return ctx.engine.cursors.open(ctx.io, now_ms(ctx), owned, spec) catch |err| switch (err) {
|
|
// A namespace too long for the fixed buffers, a store whose every slot
|
|
// is pinned, or an allocation failure. None is worth failing a query
|
|
// whose documents are already in the reply.
|
|
error.NameTooLong, error.TooManyCursors, error.OutOfMemory, error.Canceled => {
|
|
owned.deinit();
|
|
return 0;
|
|
},
|
|
};
|
|
}
|
|
|
|
fn now_ms(ctx: *Context) i64 {
|
|
return std.Io.Timestamp.now(ctx.io, .real).toMilliseconds();
|
|
}
|
|
|
|
/// Copy `docs` into `arena` as canonical BSON, for a result with no stable
|
|
/// backing store to point at.
|
|
fn buffered_source(
|
|
arena: std.mem.Allocator,
|
|
docs: []const *const bson.Document,
|
|
) !cursor.Source {
|
|
const out = try arena.alloc([]const u8, docs.len);
|
|
for (docs, 0..) |d, i| out[i] = try serialize_pairs(arena, d.pairs);
|
|
return .{ .buffered = .{ .docs = out } };
|
|
}
|
|
|
|
/// Wrap pairs already living in an arena as a borrowed document, so generated
|
|
/// results can go through the same batch path as stored ones.
|
|
fn doc_from_pairs(arena: std.mem.Allocator, pairs: []const bson.Pair) !*const bson.Document {
|
|
const doc = try arena.create(bson.Document);
|
|
doc.* = bson.Document{ .arena = undefined, .pairs = pairs };
|
|
return doc;
|
|
}
|
|
|
|
/// Emit a first batch from documents already materialized in the reply arena,
|
|
/// registering a cursor for whatever does not fit in it.
|
|
///
|
|
/// The snapshot source is the only one available here: these documents are either
|
|
/// generated (`$group`, a listing) or the output of a pipeline that has already
|
|
/// materialized its window, so there is no stable structure to point back into.
|
|
/// That also makes the resulting cursor independent of its collection, which is
|
|
/// what lets a listing hold a cursor over a `$cmd.*` namespace.
|
|
///
|
|
/// `ns_coll` is the collection part of the reported namespace and the one a
|
|
/// `getMore` must name -- for a listing that is `$cmd.listCollections`, not the
|
|
/// empty string that used to be reported. An empty collection part makes the
|
|
/// driver throw client-side before it even sends the getMore.
|
|
fn emit_first_batch(
|
|
ctx: *Context,
|
|
reply: *wire.Reply,
|
|
ns_db: []const u8,
|
|
ns_coll: []const u8,
|
|
proj_pairs: ?[]const bson.Pair,
|
|
docs: []const *const bson.Document,
|
|
batch_size: ?u32,
|
|
) !void {
|
|
var arena = std.heap.ArenaAllocator.init(ctx.gpa);
|
|
var arena_owned = false;
|
|
defer if (!arena_owned) arena.deinit();
|
|
|
|
const source = try buffered_source(arena.allocator(), docs);
|
|
var feed = Feed{ .source = source, .remaining_limit = null };
|
|
var values: std.ArrayListUnmanaged(bson.Value) = .empty;
|
|
const target = cursor.batch_target(batch_size, true);
|
|
const exhausted = try fill_batch(ctx, reply, null, &feed, &.{}, proj_pairs, target, &values);
|
|
|
|
var cursor_id: i64 = 0;
|
|
if (!exhausted and source_keepable(feed.source)) {
|
|
cursor_id = open_cursor(ctx, arena, .{
|
|
.ns = .{ .db = ns_db, .coll = ns_coll },
|
|
// Nothing about this cursor depends on the collection's layout.
|
|
.layout_epoch = 0,
|
|
.batch_size = batch_size,
|
|
.source = feed.source,
|
|
});
|
|
arena_owned = cursor_id != 0;
|
|
}
|
|
if (cursor_id == 0 and !exhausted) {
|
|
_ = try fill_batch(ctx, reply, null, &feed, &.{}, proj_pairs, null, &values);
|
|
}
|
|
|
|
const ns = try format_namespace(reply, ns_db, ns_coll);
|
|
const batch = try cursor_doc(reply, cursor_id, ns, "firstBatch", values.items);
|
|
try reply.put("cursor", .{ .doc = batch });
|
|
}
|
|
|
|
/// `batchSize` out of an `aggregate`'s `cursor` option. A bare `cursor: {}` means
|
|
/// the default; a missing `cursor` is accepted as the same thing, which is looser
|
|
/// than mongod (it requires the field) but cannot surprise any driver.
|
|
fn aggregate_batch_size(reply: *wire.Reply, msg: *wire.Message) !?struct { value: ?u32 } {
|
|
const spec = doc_arg(msg.body.get("cursor")) orelse return .{ .value = null };
|
|
const v = bson.get_pair(spec, "batchSize") orelse return .{ .value = null };
|
|
const parsed = try batch_size_arg(reply, v, "cursor.batchSize", false) orelse return null;
|
|
return .{ .value = parsed };
|
|
}
|
|
|
|
fn project_doc(
|
|
reply: *wire.Reply,
|
|
doc: *const bson.Document,
|
|
proj_pairs: ?[]const bson.Pair,
|
|
) !bson.Value {
|
|
if (proj_pairs) |pp| {
|
|
var out: std.ArrayListUnmanaged(bson.Pair) = .empty;
|
|
errdefer out.deinit(reply.arena_alloc());
|
|
try query.project(reply.arena_alloc(), doc, &.{ .arena = undefined, .pairs = pp }, &out);
|
|
return .{ .doc = out.items };
|
|
}
|
|
return .{ .doc = try bson.copy_pairs(reply.arena_alloc(), doc.pairs) };
|
|
}
|
|
|
|
fn cmd_update(ctx: *Context, msg: *wire.Message, reply: *wire.Reply) !void {
|
|
const db_name = msg.db_name() orelse return invalid_arg(reply, "update requires $db");
|
|
const coll_name = str_arg(msg.body.get("update")) orelse return bad_value(reply, "update requires a collection name");
|
|
|
|
const specs = try batch_arg(msg, reply, "update", "updates") orelse return;
|
|
|
|
var n_matched: i64 = 0;
|
|
var n_modified: i64 = 0;
|
|
var upserted: std.ArrayListUnmanaged(bson.Pair) = .empty;
|
|
defer upserted.deinit(reply.arena_alloc());
|
|
var write_errors: std.ArrayListUnmanaged(bson.Value) = .empty;
|
|
defer write_errors.deinit(reply.arena_alloc());
|
|
|
|
// Group commit for multi-document updates: one fsync per command, issued
|
|
// by the dispatch epilogue once the collection lock is released.
|
|
for (specs, 0..) |*spec, si| {
|
|
const q = doc_arg(spec.get("q")) orelse return bad_value(reply, "update spec requires q");
|
|
const u_doc = doc_arg(spec.get("u")) orelse return bad_value(reply, "update spec requires u");
|
|
const multi = bool_arg(spec.get("multi")) orelse false;
|
|
const upsert = bool_arg(spec.get("upsert")) orelse false;
|
|
// `sort` on an update spec picks *which* match to write when the filter
|
|
// matches several -- a MongoDB 8.0 addition. This server advertises 4.4,
|
|
// and ignoring the field would be the worst of the three possible
|
|
// answers: the client asked for a specific document and would silently
|
|
// get a different one. So refuse it, which is also what a real 4.4 does
|
|
// with an unknown update-spec field.
|
|
if (spec.get("sort") != null) {
|
|
return failed_to_parse(reply, "Unknown option to update: sort");
|
|
}
|
|
// A replacement describes one document, so there is no coherent meaning
|
|
// for applying it to many: every match would end up identical apart from
|
|
// its `_id`. MongoDB rejects the combination rather than doing that.
|
|
if (multi and update.is_replacement(u_doc)) {
|
|
return failed_to_parse(reply, "multi update is not supported for replacement-style update");
|
|
}
|
|
|
|
var matched: std.ArrayListUnmanaged(u64) = .empty;
|
|
defer matched.deinit(ctx.gpa);
|
|
_ = try scan_matching(ctx, db_name, coll_name, q, if (multi) 0 else 1, &matched);
|
|
|
|
if (matched.items.len == 0) {
|
|
if (upsert) {
|
|
const new_doc = try build_upsert_doc(reply, q, u_doc);
|
|
ctx.engine.insert(db_name, coll_name, new_doc, ctx.oid_gen) catch |err| switch (err) {
|
|
error.DuplicateKey, error.DuplicateKeyIndex => return duplicate_key_error(ctx, reply, db_name, coll_name, new_doc),
|
|
else => return err,
|
|
};
|
|
const id = new_doc.get("_id") orelse bson.Value.null;
|
|
const u = try reply.arena_alloc().alloc(bson.Pair, 2);
|
|
u[0] = .{ .key = "index", .value = .{ .int32 = @intCast(si) } };
|
|
u[1] = .{ .key = "_id", .value = try bson.copy_value(reply.arena_alloc(), id) };
|
|
try upserted.append(reply.arena_alloc(), .{ .key = "u", .value = .{ .doc = u } });
|
|
n_matched += 1;
|
|
}
|
|
continue;
|
|
}
|
|
|
|
n_matched += @intCast(matched.items.len);
|
|
const coll = ctx.engine.get_collection(db_name, coll_name) orelse return;
|
|
for (matched.items) |off| {
|
|
// Work on a copy: the log write must precede any visible change,
|
|
// and a rejected update must not corrupt the stored document.
|
|
const doc = try doc_tree(reply.arena_alloc(), coll, off);
|
|
const copy = try clone_doc(reply, doc);
|
|
update.apply(copy, &.{ .arena = undefined, .pairs = u_doc }) catch |err| switch (err) {
|
|
error.ImmutableId, error.InvalidUpdate => return bad_value(reply, "bad update"),
|
|
else => return err,
|
|
};
|
|
const written = ctx.engine.replace(db_name, coll_name, copy, ctx.oid_gen) catch |err| switch (err) {
|
|
error.DuplicateKey, error.DuplicateKeyIndex => {
|
|
const e = try reply.arena_alloc().alloc(bson.Pair, 3);
|
|
e[0] = .{ .key = "index", .value = .{ .int32 = @intCast(si) } };
|
|
e[1] = .{ .key = "code", .value = .{ .int32 = @intFromEnum(ErrorCode.duplicate_key) } };
|
|
e[2] = .{ .key = "errmsg", .value = .{ .string = try duplicate_key_message(ctx, reply, db_name, coll_name, copy) } };
|
|
try write_errors.append(reply.arena_alloc(), .{ .doc = e });
|
|
continue;
|
|
},
|
|
else => return err,
|
|
};
|
|
// `n` counts matches, `nModified` counts documents the update
|
|
// actually altered. A write that would store the same bytes is
|
|
// neither logged nor counted here.
|
|
if (written == .modified) n_modified += 1;
|
|
}
|
|
}
|
|
|
|
try reply.put("n", .{ .int32 = @intCast(n_matched) });
|
|
try reply.put("nModified", .{ .int32 = @intCast(n_modified) });
|
|
if (write_errors.items.len > 0) {
|
|
const arr = try reply.arena_alloc().alloc(bson.Value, write_errors.items.len);
|
|
@memcpy(arr, write_errors.items);
|
|
try reply.put("writeErrors", .{ .array = arr });
|
|
}
|
|
if (upserted.items.len > 0) {
|
|
const arr = try reply.arena_alloc().alloc(bson.Value, upserted.items.len);
|
|
for (upserted.items, 0..) |u, i| arr[i] = u.value;
|
|
try reply.put("upserted", .{ .array = arr });
|
|
}
|
|
try reply.put_ok();
|
|
}
|
|
|
|
fn cmd_delete(ctx: *Context, msg: *wire.Message, reply: *wire.Reply) !void {
|
|
const db_name = msg.db_name() orelse return invalid_arg(reply, "delete requires $db");
|
|
const coll_name = str_arg(msg.body.get("delete")) orelse return bad_value(reply, "delete requires a collection name");
|
|
|
|
const specs = try batch_arg(msg, reply, "delete", "deletes") orelse return;
|
|
|
|
var n_deleted: i64 = 0;
|
|
// Group commit for multi-document deletes: one fsync per command, issued
|
|
// by the dispatch epilogue once the collection lock is released.
|
|
for (specs) |*spec| {
|
|
const q = doc_arg(spec.get("q")) orelse return bad_value(reply, "delete spec requires q");
|
|
const limit = int_value(spec.get("limit")) orelse 1;
|
|
|
|
var matched: std.ArrayListUnmanaged(u64) = .empty;
|
|
defer matched.deinit(ctx.gpa);
|
|
_ = try scan_matching(ctx, db_name, coll_name, q, if (limit == 1) 1 else 0, &matched);
|
|
if (ctx.engine.get_collection(db_name, coll_name)) |coll| {
|
|
var id_arena = std.heap.ArenaAllocator.init(ctx.gpa);
|
|
defer id_arena.deinit();
|
|
for (matched.items) |off| {
|
|
const id = (try bson.get_at(id_arena.allocator(), coll.doc_bytes(off), "_id")) orelse continue;
|
|
if (try ctx.engine.remove_by_id(db_name, coll_name, id)) n_deleted += 1;
|
|
}
|
|
}
|
|
}
|
|
|
|
try reply.put("n", .{ .int32 = @intCast(n_deleted) });
|
|
try reply.put_ok();
|
|
}
|
|
|
|
fn cmd_find_and_modify(ctx: *Context, msg: *wire.Message, reply: *wire.Reply) !void {
|
|
const db_name = msg.db_name() orelse return invalid_arg(reply, "findAndModify requires $db");
|
|
const coll_name = str_arg(msg.body.get("findAndModify")) orelse return bad_value(reply, "findAndModify requires a collection name");
|
|
|
|
const q = doc_arg(msg.body.get("query")) orelse &.{};
|
|
const sort_keys = try parse_sort_keys(reply, msg.body.get("sort"));
|
|
const remove = bool_arg(msg.body.get("remove")) orelse false;
|
|
const do_update = msg.body.get("update") != null;
|
|
const upsert = bool_arg(msg.body.get("upsert")) orelse false;
|
|
const ret_new = bool_arg(msg.body.get("new")) orelse false;
|
|
const proj_pairs = doc_arg(msg.body.get("fields"));
|
|
|
|
if (remove and do_update) return bad_value(reply, "remove and update are mutually exclusive");
|
|
if (!remove and !do_update) return bad_value(reply, "must specify update or remove");
|
|
|
|
var matched: std.ArrayListUnmanaged(u64) = .empty;
|
|
defer matched.deinit(ctx.gpa);
|
|
// Without a sort, only the first match is ever used.
|
|
_ = try scan_matching(ctx, db_name, coll_name, q, if (sort_keys.len > 0) 0 else 1, &matched);
|
|
const arena = reply.arena_alloc();
|
|
const coll = ctx.engine.get_collection(db_name, coll_name) orelse return;
|
|
// findAndModify reads and rewrites the document, so materialize the
|
|
// (usually tiny) match set as trees in the reply arena.
|
|
var matched_docs: std.ArrayListUnmanaged(*const bson.Document) = .empty;
|
|
for (matched.items) |off| try matched_docs.append(arena, try doc_tree(arena, coll, off));
|
|
if (sort_keys.len > 0) {
|
|
try query.sort_docs(arena, matched_docs.items, sort_keys);
|
|
}
|
|
|
|
const target = if (matched_docs.items.len > 0) matched_docs.items[0] else null;
|
|
|
|
// Each branch decides what the reply says; the tail below emits it once.
|
|
var n: i32 = 0;
|
|
var updated_existing = false;
|
|
var upserted_id: ?bson.Value = null;
|
|
var value: bson.Value = .null;
|
|
|
|
if (target == null and do_update and upsert) {
|
|
const u_doc = doc_arg(msg.body.get("update")) orelse return bad_value(reply, "update must be a document");
|
|
const new_doc = try build_upsert_doc(reply, q, u_doc);
|
|
ctx.engine.insert(db_name, coll_name, new_doc, ctx.oid_gen) catch |err| switch (err) {
|
|
error.DuplicateKey, error.DuplicateKeyIndex => return duplicate_key_error(ctx, reply, db_name, coll_name, new_doc),
|
|
else => return err,
|
|
};
|
|
n = 1;
|
|
upserted_id = try bson.copy_value(arena, new_doc.get("_id") orelse bson.Value.null);
|
|
if (ret_new) value = try project_doc(reply, new_doc, proj_pairs);
|
|
} else if (target != null and remove) {
|
|
n = 1;
|
|
// Project before removing: this reads the stored document.
|
|
value = try project_doc(reply, target.?, proj_pairs);
|
|
_ = try ctx.engine.remove_by_id(db_name, coll_name, target.?.get("_id") orelse unreachable);
|
|
} else if (target != null and do_update) {
|
|
const u_doc = doc_arg(msg.body.get("update")) orelse return bad_value(reply, "update must be a document");
|
|
const before = try bson.copy_pairs(arena, target.?.pairs);
|
|
const copy = try clone_doc(reply, target.?);
|
|
update.apply(copy, &.{ .arena = undefined, .pairs = u_doc }) catch |err| switch (err) {
|
|
error.ImmutableId, error.InvalidUpdate => return bad_value(reply, "bad update"),
|
|
else => return err,
|
|
};
|
|
// findAndModify reports `n` (matched) and `updatedExisting`, neither of
|
|
// which distinguishes a no-op, so whether it wrote is not needed here.
|
|
_ = try ctx.engine.replace(db_name, coll_name, copy, ctx.oid_gen);
|
|
n = 1;
|
|
updated_existing = true;
|
|
value = if (ret_new) try project_doc(reply, copy, proj_pairs) else .{ .doc = before };
|
|
} // else: no match and no upsert — an empty result
|
|
|
|
var leo: std.ArrayListUnmanaged(bson.Pair) = .empty;
|
|
defer leo.deinit(arena);
|
|
try leo.append(arena, .{ .key = "n", .value = .{ .int32 = n } });
|
|
try leo.append(arena, .{ .key = "updatedExisting", .value = .{ .bool = updated_existing } });
|
|
if (upserted_id) |id| try leo.append(arena, .{ .key = "upserted", .value = id });
|
|
try reply.put("value", value);
|
|
try reply.put("lastErrorObject", .{ .doc = try leo.toOwnedSlice(arena) });
|
|
try reply.put_ok();
|
|
}
|
|
|
|
fn cmd_count(ctx: *Context, msg: *wire.Message, reply: *wire.Reply) !void {
|
|
const db_name = msg.db_name() orelse return invalid_arg(reply, "count requires $db");
|
|
const coll_name = str_arg(msg.body.get("count")) orelse return bad_value(reply, "count requires a collection name");
|
|
const q = doc_arg(msg.body.get("query")) orelse &.{};
|
|
// `count` takes skip and limit like `find` does, and ignoring them was a
|
|
// silent wrong answer for `countDocuments(f, {limit})`.
|
|
const skip: u64 = int_arg(msg.body.get("skip")) orelse 0;
|
|
const limit: u64 = @abs(int_value(msg.body.get("limit")) orelse 0);
|
|
|
|
// Counting only needs to know whether the matches reach skip + limit, so the
|
|
// scan may stop there. Unlike `find` this needs no index to be an early stop:
|
|
// the *count* of a window does not depend on which documents fall in it.
|
|
const ceiling: usize = if (limit == 0) 0 else blk: {
|
|
const s = std.math.cast(usize, skip) orelse break :blk 0;
|
|
const l = std.math.cast(usize, limit) orelse break :blk 0;
|
|
break :blk s +| l;
|
|
};
|
|
const matched = try scan_matching(ctx, db_name, coll_name, q, ceiling, null);
|
|
const after_skip = matched -| (std.math.cast(usize, skip) orelse matched);
|
|
const n = if (limit == 0) after_skip else @min(after_skip, limit);
|
|
try reply.put("n", .{ .int32 = @intCast(n) });
|
|
try reply.put_ok();
|
|
}
|
|
|
|
fn cmd_aggregate(ctx: *Context, msg: *wire.Message, reply: *wire.Reply) !void {
|
|
const db_name = msg.db_name() orelse return invalid_arg(reply, "aggregate requires $db");
|
|
const coll_name = str_arg(msg.body.get("aggregate")) orelse return bad_value(reply, "aggregate requires a collection name");
|
|
|
|
const batch_size = (try aggregate_batch_size(reply, msg) orelse return).value;
|
|
const pipeline_value = msg.body.get("pipeline") orelse return bad_value(reply, "aggregate requires pipeline");
|
|
var stages = switch (pipeline_value) {
|
|
.array => |a| a,
|
|
else => return bad_value(reply, "pipeline must be an array"),
|
|
};
|
|
|
|
// countDocuments() reaches us as [{$match: F}?, {$group: {_id: <literal>,
|
|
// n: {$sum: 1}}}]. The general path answers that by materializing every
|
|
// matching document and then throwing them all away, so recognize the
|
|
// shape and answer it from a counting scan instead.
|
|
if (try count_only_pipeline(reply, stages)) |shape| {
|
|
const n = try scan_matching(ctx, db_name, coll_name, shape.filter, 0, null);
|
|
// No documents means no groups at all, not a group holding zero —
|
|
// same as the general path, which builds groups per document.
|
|
var docs: []const *const bson.Document = &.{};
|
|
if (n > 0) {
|
|
const arena = reply.arena_alloc();
|
|
const pairs = try arena.alloc(bson.Pair, 1 + shape.accs.len);
|
|
pairs[0] = .{ .key = "_id", .value = try bson.copy_value(arena, shape.id_value) };
|
|
for (shape.accs, 0..) |acc, i| {
|
|
// Mirrors run_group's coercion exactly: an integral sum in
|
|
// int32 range comes back as int32, otherwise a double.
|
|
const sum: f64 = @as(f64, @floatFromInt(n)) * acc.term;
|
|
pairs[1 + i] = .{
|
|
.key = acc.key,
|
|
.value = if (sum == @floor(sum) and sum <= 2_147_483_647 and sum >= -2_147_483_648)
|
|
.{ .int32 = @intFromFloat(sum) }
|
|
else
|
|
.{ .double = sum },
|
|
};
|
|
}
|
|
const doc = try doc_from_pairs(arena, pairs);
|
|
const one = try arena.alloc(*const bson.Document, 1);
|
|
one[0] = doc;
|
|
docs = one;
|
|
}
|
|
try emit_first_batch(ctx, reply, db_name, coll_name, null, docs, batch_size);
|
|
return reply.put_ok();
|
|
}
|
|
|
|
// The pipeline operates on a stream of documents; each stage transforms
|
|
// the current window [start, end). Before $group the stream holds slab
|
|
// offsets (matched in place, never materialized); $group replaces it
|
|
// with generated group documents, so the stream flips to tree form.
|
|
var offs: std.ArrayListUnmanaged(u64) = .empty;
|
|
defer offs.deinit(ctx.gpa);
|
|
var trees: std.ArrayListUnmanaged(*const bson.Document) = .empty;
|
|
defer trees.deinit(ctx.gpa);
|
|
var in_trees = false;
|
|
// No such collection is an empty result, not an absent one. A bare
|
|
// `return` here sent a reply with no `ok` field at all, which the driver
|
|
// reports as the uninformative `MongoServerError: n/a` -- and it is what
|
|
// `db.aggregate(...)` hits, because a database-level aggregate names no
|
|
// collection. MongoDB answers an aggregate over a missing collection with
|
|
// an empty cursor.
|
|
const coll = ctx.engine.get_collection(db_name, coll_name) orelse {
|
|
try emit_first_batch(ctx, reply, db_name, coll_name, null, &.{}, batch_size);
|
|
return reply.put_ok();
|
|
};
|
|
// A leading $match is pushed down into an indexed candidate scan; the
|
|
// stage is then dropped from the pipeline so it is not applied twice.
|
|
if (stages.len > 0 and stages[0] == .doc and stages[0].doc.len > 0 and std.mem.eql(u8, stages[0].doc[0].key, "$match")) {
|
|
const filter = doc_arg(stages[0].doc[0].value) orelse return bad_value(reply, "$match requires a document");
|
|
_ = try scan_matching(ctx, db_name, coll_name, filter, 0, &offs);
|
|
stages = stages[1..];
|
|
} else {
|
|
// Every document, in _id order. A pipeline materializes its stream
|
|
// anyway (stages need random access to the window), so this one stays a
|
|
// list -- but it comes from the _id_ index rather than the docs map,
|
|
// which is ordered and does not depend on a structure that is going
|
|
// away. Streaming the whole pipeline is M1's cursor work.
|
|
var it = coll.id_index.iter();
|
|
while (it.next()) |e| try offs.append(ctx.gpa, e.off);
|
|
}
|
|
|
|
var start: usize = 0;
|
|
var end: usize = offs.items.len;
|
|
var count_stage: ?[]const u8 = null;
|
|
var proj_pairs: ?[]const bson.Pair = null;
|
|
|
|
for (stages) |stage_v| {
|
|
const stage = switch (stage_v) {
|
|
.doc => |pairs| pairs,
|
|
else => return bad_value(reply, "pipeline stages must be documents"),
|
|
};
|
|
if (stage.len == 0) continue;
|
|
const stage_name = stage[0].key;
|
|
if (std.mem.eql(u8, stage_name, "$match")) {
|
|
const filter = doc_arg(stage[0].value) orelse return bad_value(reply, "$match requires a document");
|
|
if (!in_trees) {
|
|
var kept: std.ArrayListUnmanaged(u64) = .empty;
|
|
defer kept.deinit(ctx.gpa);
|
|
for (offs.items[start..end]) |off| {
|
|
if (try query.matches_bytes(ctx.gpa, filter, coll.doc_bytes(off))) {
|
|
try kept.append(ctx.gpa, off);
|
|
}
|
|
}
|
|
offs.deinit(ctx.gpa);
|
|
offs = kept;
|
|
kept = .empty;
|
|
} else {
|
|
var kept: std.ArrayListUnmanaged(*const bson.Document) = .empty;
|
|
defer kept.deinit(ctx.gpa);
|
|
for (trees.items[start..end]) |d| {
|
|
if (try query.matches(ctx.gpa, &.{ .arena = undefined, .pairs = filter }, d)) {
|
|
try kept.append(ctx.gpa, d);
|
|
}
|
|
}
|
|
trees.deinit(ctx.gpa);
|
|
trees = kept;
|
|
kept = .empty;
|
|
}
|
|
start = 0;
|
|
end = (if (in_trees) trees.items.len else offs.items.len);
|
|
} else if (std.mem.eql(u8, stage_name, "$sort")) {
|
|
const keys = try parse_sort_keys(reply, stage[0].value);
|
|
if (keys.len > 0) {
|
|
const arena = reply.arena_alloc();
|
|
if (!in_trees) {
|
|
// Sorting needs the values; materialize and switch the
|
|
// stream to tree form for the rest of the pipeline.
|
|
//
|
|
// The list buffer must come from `ctx.gpa`, because that is
|
|
// what frees it: ownership moves to `trees`, and `trees` is
|
|
// released by this function's `defer trees.deinit(ctx.gpa)`
|
|
// and by the $match branch above. Building it from the
|
|
// reply arena instead handed a gpa-free an arena-owned
|
|
// pointer -- a remote, client-triggerable invalid free that
|
|
// macOS malloc turns into SIGTRAP with no panic text, so it
|
|
// read as "the connection closed". Any pipeline with $sort
|
|
// and no preceding $group reached it.
|
|
//
|
|
// The *documents* stay in the arena on purpose: it outlives
|
|
// the command, and only the ArrayList's own allocator has
|
|
// to match its deinit.
|
|
var all: std.ArrayListUnmanaged(*const bson.Document) = .empty;
|
|
errdefer all.deinit(ctx.gpa);
|
|
for (offs.items) |off| try all.append(ctx.gpa, try doc_tree(arena, coll, off));
|
|
trees.deinit(ctx.gpa);
|
|
trees = all;
|
|
all = .empty;
|
|
in_trees = true;
|
|
}
|
|
try query.sort_docs(arena, trees.items[start..end], keys);
|
|
}
|
|
} else if (std.mem.eql(u8, stage_name, "$skip")) {
|
|
const n = try stage_count(reply, stage[0].value, "$skip") orelse return;
|
|
start = @min(start + n, end);
|
|
} else if (std.mem.eql(u8, stage_name, "$limit")) {
|
|
const n = try stage_count(reply, stage[0].value, "$limit") orelse return;
|
|
end = @min(end, start + n);
|
|
} else if (std.mem.eql(u8, stage_name, "$project")) {
|
|
proj_pairs = doc_arg(stage[0].value);
|
|
} else if (std.mem.eql(u8, stage_name, "$group")) {
|
|
const gp = doc_arg(stage[0].value) orelse return bad_value(reply, "$group requires a document");
|
|
const grouped_opt = try run_group(ctx, reply, coll, gp, offs.items[start..end]);
|
|
var grouped = grouped_opt orelse return;
|
|
// Group results replace the stream: later stages see groups.
|
|
offs.deinit(ctx.gpa);
|
|
offs = .empty;
|
|
trees.deinit(ctx.gpa);
|
|
trees = grouped;
|
|
grouped = .empty;
|
|
in_trees = true;
|
|
start = 0;
|
|
end = trees.items.len;
|
|
} else if (std.mem.eql(u8, stage_name, "$count")) {
|
|
count_stage = switch (stage[0].value) {
|
|
.string => |s| s,
|
|
else => return bad_value(reply, "$count requires a string"),
|
|
};
|
|
} else {
|
|
const msg_text = try std.fmt.allocPrint(reply.arena_alloc(), "Unrecognized pipeline stage name: '{s}'", .{stage_name});
|
|
// 40324 is right for "unrecognized stage" but its name is not
|
|
// `InvalidPipelineOperator` (that is 168). mongod reports numeric
|
|
// Location codes under a `Location<n>` name -- verified by asking a
|
|
// real mongod for an unknown stage.
|
|
const code = @intFromEnum(ErrorCode.location_unrecognized_stage);
|
|
return reply.put_error(code, "Location40324", msg_text);
|
|
}
|
|
}
|
|
|
|
if (count_stage) |name| {
|
|
const len = if (in_trees) trees.items[start..end].len else offs.items[start..end].len;
|
|
const c = try reply.arena_alloc().alloc(bson.Pair, 1);
|
|
c[0] = .{ .key = try reply.arena_alloc().dupe(u8, name), .value = .{ .int32 = @intCast(len) } };
|
|
const one = try reply.arena_alloc().alloc(*const bson.Document, 1);
|
|
one[0] = try doc_from_pairs(reply.arena_alloc(), c);
|
|
try emit_first_batch(ctx, reply, db_name, coll_name, null, one, batch_size);
|
|
} else {
|
|
const arena = reply.arena_alloc();
|
|
if (in_trees) {
|
|
const window = trees.items[start..end];
|
|
try emit_first_batch(ctx, reply, db_name, coll_name, proj_pairs, window, batch_size);
|
|
} else {
|
|
var page: std.ArrayListUnmanaged(*const bson.Document) = .empty;
|
|
for (offs.items[start..end]) |off| try page.append(arena, try doc_tree(arena, coll, off));
|
|
const window = page.items;
|
|
try emit_first_batch(ctx, reply, db_name, coll_name, proj_pairs, window, batch_size);
|
|
}
|
|
}
|
|
try reply.put_ok();
|
|
}
|
|
|
|
/// A pipeline whose whole answer is the number of matching documents.
|
|
const CountShape = struct {
|
|
filter: []const bson.Pair,
|
|
/// The literal every document groups under.
|
|
id_value: bson.Value,
|
|
accs: []const Acc,
|
|
|
|
const Acc = struct { key: []const u8, term: f64 };
|
|
};
|
|
|
|
/// Recognize `[{$match: F}?, {$group: {_id: <literal>, k: {$sum: <number>}}}]`
|
|
/// — the shape a driver sends for countDocuments().
|
|
///
|
|
/// Deliberately conservative: a `_id` of `"$field"`, an accumulator over a
|
|
/// field, or any other stage needs the documents themselves, so anything
|
|
/// that is not exactly this shape returns null and takes the general path.
|
|
fn count_only_pipeline(reply: *wire.Reply, stages: []const bson.Value) !?CountShape {
|
|
if (stages.len == 0 or stages.len > 2) return null;
|
|
|
|
var filter: []const bson.Pair = &.{};
|
|
if (stages.len == 2) {
|
|
const first = switch (stages[0]) {
|
|
.doc => |p| p,
|
|
else => return null,
|
|
};
|
|
if (first.len != 1 or !std.mem.eql(u8, first[0].key, "$match")) return null;
|
|
filter = doc_arg(first[0].value) orelse return null;
|
|
}
|
|
|
|
const last = switch (stages[stages.len - 1]) {
|
|
.doc => |p| p,
|
|
else => return null,
|
|
};
|
|
if (last.len != 1 or !std.mem.eql(u8, last[0].key, "$group")) return null;
|
|
const gp = doc_arg(last[0].value) orelse return null;
|
|
|
|
const id_value = bson.get_pair(gp, "_id") orelse return null;
|
|
switch (id_value) {
|
|
// A field path or a computed id groups per document.
|
|
.string => |s| if (s.len > 0 and s[0] == '$') return null,
|
|
.doc, .array => return null,
|
|
else => {},
|
|
}
|
|
|
|
var accs: std.ArrayListUnmanaged(CountShape.Acc) = .empty;
|
|
for (gp) |p| {
|
|
if (std.mem.eql(u8, p.key, "_id")) continue;
|
|
const spec = switch (p.value) {
|
|
.doc => |d| d,
|
|
else => return null,
|
|
};
|
|
if (spec.len != 1 or !std.mem.eql(u8, spec[0].key, "$sum")) return null;
|
|
const term: f64 = switch (spec[0].value) {
|
|
.int32 => |i| @floatFromInt(i),
|
|
.int64 => |i| @floatFromInt(i),
|
|
.double => |d| d,
|
|
// $sum over a field depends on the documents.
|
|
else => return null,
|
|
};
|
|
try accs.append(reply.arena_alloc(), .{ .key = p.key, .term = term });
|
|
}
|
|
|
|
return .{ .filter = filter, .id_value = id_value, .accs = accs.items };
|
|
}
|
|
|
|
/// Minimal $group: supports `_id` of null/literal/"$field" and `$sum`
|
|
/// accumulators (constant or "$field").
|
|
fn run_group(
|
|
ctx: *Context,
|
|
reply: *wire.Reply,
|
|
coll: *const Collection,
|
|
group_pairs: []const bson.Pair,
|
|
docs: []const u64,
|
|
) !?std.ArrayListUnmanaged(*const bson.Document) {
|
|
const arena = reply.arena_alloc();
|
|
const id_expr = bson.get_pair(group_pairs, "_id") orelse {
|
|
try bad_value(reply, "$group requires _id");
|
|
return null;
|
|
};
|
|
|
|
var accs: std.ArrayListUnmanaged(bson.Pair) = .empty;
|
|
defer accs.deinit(arena);
|
|
for (group_pairs) |p| {
|
|
if (std.mem.eql(u8, p.key, "_id")) continue;
|
|
try accs.append(arena, p);
|
|
}
|
|
|
|
const Group = struct {
|
|
id_value: bson.Value,
|
|
sums: []f64,
|
|
};
|
|
var groups: std.StringHashMapUnmanaged(Group) = .empty;
|
|
defer groups.deinit(ctx.gpa);
|
|
// StringHashMapUnmanaged does not copy keys; keep them alive until done.
|
|
var keys_owned: std.ArrayListUnmanaged([]u8) = .empty;
|
|
defer {
|
|
for (keys_owned.items) |k| ctx.gpa.free(k);
|
|
keys_owned.deinit(ctx.gpa);
|
|
}
|
|
|
|
var id_key_buf: std.ArrayListUnmanaged(u8) = .empty;
|
|
defer id_key_buf.deinit(ctx.gpa);
|
|
// Byte-walk materializations (nested group keys) live here.
|
|
var walk_arena = std.heap.ArenaAllocator.init(ctx.gpa);
|
|
defer walk_arena.deinit();
|
|
for (docs) |off| {
|
|
const doc = coll.doc_bytes(off);
|
|
const id_value: bson.Value = switch (id_expr) {
|
|
.string => |s| if (s.len > 0 and s[0] == '$') (try query_path_value_bytes(walk_arena.allocator(), doc, s[1..])) orelse .null else id_expr,
|
|
else => id_expr,
|
|
};
|
|
id_key_buf.clearRetainingCapacity();
|
|
try bson.write_value(id_value, ctx.gpa, &id_key_buf);
|
|
|
|
const gop = try groups.getOrPut(ctx.gpa, id_key_buf.items);
|
|
if (!gop.found_existing) {
|
|
// Only a new group needs an owned copy of the key; the map
|
|
// borrows it, so keys_owned keeps it alive until we are done.
|
|
const key = try ctx.gpa.dupe(u8, id_key_buf.items);
|
|
try keys_owned.append(ctx.gpa, key);
|
|
gop.key_ptr.* = key;
|
|
const sums = try ctx.gpa.alloc(f64, accs.items.len);
|
|
@memset(sums, 0);
|
|
gop.value_ptr.* = .{ .id_value = id_value, .sums = sums };
|
|
}
|
|
for (accs.items, 0..) |acc, i| {
|
|
var expr = acc.value;
|
|
// Unwrap {$sum: <expr>} accumulator documents.
|
|
if (expr == .doc) {
|
|
if (bson.get_pair(expr.doc, "$sum")) |inner| {
|
|
expr = inner;
|
|
} else continue;
|
|
}
|
|
const term: f64 = switch (expr) {
|
|
.int32 => |n| @floatFromInt(n),
|
|
.int64 => |n| @floatFromInt(n),
|
|
.double => |n| n,
|
|
.string => |s| if (s.len > 0 and s[0] == '$')
|
|
switch ((try query_path_value_bytes(walk_arena.allocator(), doc, s[1..])) orelse .null) {
|
|
.int32 => |n| @floatFromInt(n),
|
|
.int64 => |n| @floatFromInt(n),
|
|
.double => |n| n,
|
|
else => 0,
|
|
}
|
|
else
|
|
0,
|
|
else => 0,
|
|
};
|
|
gop.value_ptr.sums[i] += term;
|
|
}
|
|
}
|
|
|
|
var out: std.ArrayListUnmanaged(*const bson.Document) = .empty;
|
|
errdefer out.deinit(ctx.gpa);
|
|
var it = groups.iterator();
|
|
// free sum arrays
|
|
defer {
|
|
var git = groups.iterator();
|
|
while (git.next()) |e| ctx.gpa.free(e.value_ptr.sums);
|
|
}
|
|
while (it.next()) |entry| {
|
|
const npairs = 1 + accs.items.len;
|
|
const pairs = try arena.alloc(bson.Pair, npairs);
|
|
pairs[0] = .{ .key = "_id", .value = try bson.copy_value(arena, entry.value_ptr.id_value) };
|
|
for (accs.items, 0..) |acc, i| {
|
|
const sum: f64 = entry.value_ptr.sums[i];
|
|
const sum_value: bson.Value = if (sum == @floor(sum) and sum <= 2_147_483_647 and sum >= -2_147_483_648)
|
|
.{ .int32 = @intFromFloat(sum) }
|
|
else
|
|
.{ .double = sum };
|
|
pairs[1 + i] = .{ .key = acc.key, .value = sum_value };
|
|
}
|
|
const doc = try arena.create(bson.Document);
|
|
doc.* = bson.Document{ .arena = undefined, .pairs = pairs };
|
|
try out.append(ctx.gpa, doc);
|
|
}
|
|
return out;
|
|
}
|
|
|
|
/// Resolve a simple "$field" path expression inside a document.
|
|
fn query_path_value_bytes(
|
|
gpa: std.mem.Allocator,
|
|
bytes: []const u8,
|
|
path: []const u8,
|
|
) !?bson.Value {
|
|
var cur: bson.Value = undefined;
|
|
var it = std.mem.splitScalar(u8, path, '.');
|
|
const first = it.next() orelse return null;
|
|
cur = (try bson.get_at(gpa, bytes, first)) orelse return null;
|
|
while (it.next()) |seg| {
|
|
cur = switch (cur) {
|
|
.doc => |pairs| bson.get_pair(pairs, seg) orelse return null,
|
|
else => return null,
|
|
};
|
|
}
|
|
return cur;
|
|
}
|
|
|
|
/// A cursor id from the wire. Accepts int32 as well as int64, so a hand-written
|
|
/// `runCommand` is not rejected on a technicality; the driver always sends a
|
|
/// long.
|
|
fn cursor_id_arg(v: ?bson.Value) ?i64 {
|
|
return switch (v orelse return null) {
|
|
.int64 => |i| i,
|
|
.int32 => |i| i,
|
|
else => null,
|
|
};
|
|
}
|
|
|
|
fn cursor_not_found(reply: *wire.Reply, id: i64) !void {
|
|
const text = try std.fmt.allocPrint(reply.arena_alloc(), "cursor id {d} not found", .{id});
|
|
return reply.put_error(@intFromEnum(ErrorCode.cursor_not_found), "CursorNotFound", text);
|
|
}
|
|
|
|
fn query_plan_killed(reply: *wire.Reply, why: []const u8) !void {
|
|
const text = try std.fmt.allocPrint(
|
|
reply.arena_alloc(),
|
|
"query plan killed :: caused by :: {s}",
|
|
.{why},
|
|
);
|
|
const code = @intFromEnum(ErrorCode.query_plan_killed);
|
|
return reply.put_error(code, "QueryPlanKilled", text);
|
|
}
|
|
|
|
/// Whether this cursor can still be answered from `coll`, or why not.
|
|
///
|
|
/// What a rebuild since the cursor was created costs depends entirely on what the
|
|
/// cursor remembers. `.offsets` holds slab offsets and a rebuild moved every
|
|
/// document, so those offsets now name unrelated bytes. `.stream` holds key
|
|
/// bytes, and a repack changes no key -- only the anchor's offset and position
|
|
/// hint go stale, and both are checked before they are believed, so the walk
|
|
/// resumes at the right key and the offsets it yields come fresh out of the tree.
|
|
/// `.buffered` holds copies of the documents and needs no collection at all,
|
|
/// which is what lets a listing hold a cursor over a `$cmd.*` namespace.
|
|
fn cursor_still_valid(c: *cursor.Cursor, coll: ?*Collection) ?[]const u8 {
|
|
const live = coll orelse {
|
|
// A snapshot needs no collection at all, which is what lets a listing
|
|
// hold a cursor over a `$cmd.*` namespace nothing backs.
|
|
return if (c.source == .buffered) null else "collection dropped";
|
|
};
|
|
if (live.layout_epoch == c.layout_epoch) return null;
|
|
if (c.source == .offsets) return "collection rebuilt";
|
|
// Survived the rebuild: adopt the new layout so the next getMore does not
|
|
// re-examine it.
|
|
c.layout_epoch = live.layout_epoch;
|
|
return null;
|
|
}
|
|
|
|
/// The stored filter and projection, reparsed for this request. They were
|
|
/// serialized into the cursor's own arena because the parsed forms pointed into
|
|
/// the request that created them; `spine` parses the structure and borrows the
|
|
/// leaf bytes, so this costs no copy of the filter's strings.
|
|
fn cursor_query(reply: *wire.Reply, c: *const cursor.Cursor) !struct {
|
|
filter: []const bson.Pair,
|
|
proj: ?[]const bson.Pair,
|
|
} {
|
|
const arena = reply.arena_alloc();
|
|
return .{
|
|
.filter = if (c.filter_bytes.len == 0) &.{} else try bson.spine(arena, c.filter_bytes),
|
|
.proj = if (c.proj_bytes.len == 0) null else try bson.spine(arena, c.proj_bytes),
|
|
};
|
|
}
|
|
|
|
/// The shape of a `getMore` request, or null once the error reply is written.
|
|
fn parse_get_more(reply: *wire.Reply, msg: *wire.Message) !?struct { id: i64, batch_size: ?u32 } {
|
|
const id = cursor_id_arg(msg.body.get("getMore")) orelse {
|
|
const text = "BSON field 'getMore.getMore' is the wrong type";
|
|
try reply.put_error(@intFromEnum(ErrorCode.type_mismatch), "TypeMismatch", text);
|
|
return null;
|
|
};
|
|
var batch_size: ?u32 = null;
|
|
if (msg.body.get("batchSize")) |v| {
|
|
batch_size = try batch_size_arg(reply, v, "batchSize", true) orelse return null;
|
|
}
|
|
return .{ .id = id, .batch_size = batch_size };
|
|
}
|
|
|
|
/// Claim the cursor for this request, or write the error reply and return null.
|
|
///
|
|
/// The namespace check inside `pin` is load-bearing rather than cosmetic:
|
|
/// dispatch locks the collection the *message* names, so a getMore quoting one
|
|
/// cursor's id and another collection's name would otherwise iterate the first
|
|
/// collection's index while holding the second collection's lock.
|
|
fn pin_cursor(ctx: *Context, reply: *wire.Reply, id: i64, ns: cursor.Ns) !?*cursor.Cursor {
|
|
return ctx.engine.cursors.pin(ctx.io, id, ns, now_ms(ctx)) catch |err| switch (err) {
|
|
error.CursorNotFound => {
|
|
try cursor_not_found(reply, id);
|
|
return null;
|
|
},
|
|
error.CursorNamespaceMismatch => {
|
|
var found: cursor.NsBuf = .{};
|
|
const arena = reply.arena_alloc();
|
|
const text = if (ctx.engine.cursors.ns_of(ctx.io, id, &found))
|
|
try std.fmt.allocPrint(
|
|
arena,
|
|
"Requested getMore on namespace '{s}.{s}', but cursor belongs to" ++
|
|
" a different namespace {s}.{s}",
|
|
.{ ns.db, ns.coll, found.ns().db, found.ns().coll },
|
|
)
|
|
else
|
|
try std.fmt.allocPrint(
|
|
arena,
|
|
"Requested getMore on namespace '{s}.{s}', but cursor belongs to" ++
|
|
" a different namespace",
|
|
.{ ns.db, ns.coll },
|
|
);
|
|
try reply.put_error(@intFromEnum(ErrorCode.unauthorized), "Unauthorized", text);
|
|
return null;
|
|
},
|
|
error.CursorInUse => {
|
|
const text = try std.fmt.allocPrint(
|
|
reply.arena_alloc(),
|
|
"cursor id {d} is already in use",
|
|
.{id},
|
|
);
|
|
try reply.put_error(@intFromEnum(ErrorCode.cursor_in_use), "CursorInUse", text);
|
|
return null;
|
|
},
|
|
error.Canceled => return err,
|
|
};
|
|
}
|
|
|
|
fn cmd_get_more(ctx: *Context, msg: *wire.Message, reply: *wire.Reply) !void {
|
|
const db_name = msg.db_name() orelse return invalid_arg(reply, "getMore requires $db");
|
|
const coll_name = str_arg(msg.body.get("collection")) orelse
|
|
return bad_value(reply, "Field 'collection' must be of type string");
|
|
if (coll_name.len == 0) return bad_value(reply, "Collection names cannot be empty");
|
|
const req = try parse_get_more(reply, msg) orelse return;
|
|
const id = req.id;
|
|
|
|
const ns = cursor.Ns{ .db = db_name, .coll = coll_name };
|
|
const c = try pin_cursor(ctx, reply, id, ns) orelse return;
|
|
// Set to true by every path that must not leave the cursor behind, including
|
|
// the error paths below: a cursor whose collection is gone can never answer
|
|
// again, so keeping it would only occupy a slot until the idle sweep.
|
|
var done = false;
|
|
defer ctx.engine.cursors.release(ctx.io, c, now_ms(ctx), done);
|
|
|
|
const coll = ctx.engine.get_collection(db_name, coll_name);
|
|
if (cursor_still_valid(c, coll)) |why| {
|
|
done = true;
|
|
return query_plan_killed(reply, why);
|
|
}
|
|
const q = try cursor_query(reply, c);
|
|
|
|
var feed = Feed{ .source = c.source, .remaining_limit = c.remaining_limit };
|
|
// Persist on every exit, not just the success path. For a stream the advance
|
|
// *is* the resume anchor, so an early return that skipped this would report a
|
|
// batch and then hand the same documents out again on the next getMore.
|
|
// Registered after `release`'s defer, so it runs before it.
|
|
defer {
|
|
c.source = feed.source;
|
|
c.remaining_limit = feed.remaining_limit;
|
|
}
|
|
if (feed.source == .stream) {
|
|
// Only a stream needs the collection here; `open_scan` walks its index.
|
|
assert_msg(coll != null, "a streaming cursor reached getMore with no collection");
|
|
// Reopening the walk is where a resume actually happens. Null means the
|
|
// index the cursor was following is gone, or its anchor could not be
|
|
// located within the walk bound -- either way there is no honest way to
|
|
// continue, and an empty batch would falsely claim the result ended.
|
|
feed.scan = open_scan(coll.?, &feed.source.stream);
|
|
if (feed.scan == null) {
|
|
done = true;
|
|
return query_plan_killed(reply, "the index this cursor was following is gone");
|
|
}
|
|
}
|
|
var values: std.ArrayListUnmanaged(bson.Value) = .empty;
|
|
// Deliberately not `batch_size orelse c.batch_size`: mongod does not carry
|
|
// the find's batchSize into a bare getMore. Measured -- find with
|
|
// batchSize 2 then a bare getMore returns 4998 of 5000 documents.
|
|
const target = cursor.batch_target(req.batch_size, false);
|
|
done = try fill_batch(ctx, reply, coll, &feed, q.filter, q.proj, target, &values);
|
|
|
|
const ns_str = try format_namespace(reply, db_name, coll_name);
|
|
const live_id: i64 = if (done) 0 else id;
|
|
const batch = try cursor_doc(reply, live_id, ns_str, "nextBatch", values.items);
|
|
try reply.put("cursor", .{ .doc = batch });
|
|
try reply.put_ok();
|
|
}
|
|
|
|
fn cmd_kill_cursors(ctx: *Context, msg: *wire.Message, reply: *wire.Reply) !void {
|
|
const db_name = msg.db_name() orelse return invalid_arg(reply, "killCursors requires $db");
|
|
// Unlike getMore, killCursors' own field really is the collection name.
|
|
const coll_name = str_arg(msg.body.get("killCursors")) orelse
|
|
return bad_value(reply, "killCursors requires a collection name");
|
|
const cursors_arg = msg.body.get("cursors") orelse
|
|
return bad_value(reply, "killCursors requires a cursors array");
|
|
const ids = switch (cursors_arg) {
|
|
.array => |a| a,
|
|
else => return bad_value(reply, "cursors must be an array"),
|
|
};
|
|
|
|
const arena = reply.arena_alloc();
|
|
var killed: std.ArrayListUnmanaged(bson.Value) = .empty;
|
|
var not_found: std.ArrayListUnmanaged(bson.Value) = .empty;
|
|
const ns = cursor.Ns{ .db = db_name, .coll = coll_name };
|
|
for (ids) |v| {
|
|
const id = cursor_id_arg(v) orelse {
|
|
try not_found.append(arena, v);
|
|
continue;
|
|
};
|
|
// A namespace mismatch reports not-found rather than erroring:
|
|
// killCursors is best-effort by design, and the driver ignores its reply
|
|
// entirely.
|
|
switch (ctx.engine.cursors.kill(ctx.io, id, ns)) {
|
|
.killed => try killed.append(arena, .{ .int64 = id }),
|
|
.not_found => try not_found.append(arena, .{ .int64 = id }),
|
|
}
|
|
}
|
|
|
|
try reply.put("cursorsKilled", .{ .array = killed.items });
|
|
try reply.put("cursorsNotFound", .{ .array = not_found.items });
|
|
// Empty by construction: a cursor pinned by an in-flight getMore is marked
|
|
// and reported killed, since the client's intent is satisfied and the
|
|
// request frees it on release. `cursorsUnknown` exists for shape -- every
|
|
// outcome here is classified.
|
|
try reply.put("cursorsAlive", .{ .array = &.{} });
|
|
try reply.put("cursorsUnknown", .{ .array = &.{} });
|
|
try reply.put_ok();
|
|
}
|
|
|
|
// ---------------------------------------------------------------------------
|
|
// Helpers
|
|
// ---------------------------------------------------------------------------
|
|
|
|
/// Deep-copy a stored document into the reply arena so updates can be
|
|
/// applied off the live doc.
|
|
fn clone_doc(reply: *wire.Reply, doc: *const bson.Document) !*bson.Document {
|
|
const arena = reply.arena_alloc();
|
|
const owned = try arena.create(bson.Document);
|
|
owned.* = .{
|
|
.arena = std.heap.ArenaAllocator.init(arena),
|
|
.pairs = try bson.copy_pairs(arena, doc.pairs),
|
|
};
|
|
return owned;
|
|
}
|
|
|
|
/// Build the document for an upsert: equality fields from the filter, then
|
|
/// the update operators applied. Owned by the reply arena.
|
|
fn build_upsert_doc(
|
|
reply: *wire.Reply,
|
|
q: []const bson.Pair,
|
|
u_doc: []const bson.Pair,
|
|
) !*bson.Document {
|
|
const arena = reply.arena_alloc();
|
|
var pairs: std.ArrayListUnmanaged(bson.Pair) = .empty;
|
|
defer pairs.deinit(arena);
|
|
for (q) |p| {
|
|
const is_operator = p.key.len > 0 and p.key[0] == '$';
|
|
const is_embedded_operator = p.value == .doc and query.all_operator_keys(p.value.doc);
|
|
if (!is_operator and !is_embedded_operator) {
|
|
try pairs.append(arena, .{ .key = try arena.dupe(u8, p.key), .value = try bson.copy_value(arena, p.value) });
|
|
}
|
|
}
|
|
const owned = try arena.create(bson.Document);
|
|
owned.* = bson.Document{ .arena = std.heap.ArenaAllocator.init(arena), .pairs = try pairs.toOwnedSlice(arena) };
|
|
// Apply update operators to build the final doc; _id handled by insert.
|
|
update.apply(owned, &.{ .arena = undefined, .pairs = u_doc }) catch |err| switch (err) {
|
|
error.ImmutableId, error.InvalidUpdate => return error.InvalidUpdate,
|
|
else => return err,
|
|
};
|
|
return owned;
|
|
}
|
|
|
|
fn parse_sort_keys(reply: *wire.Reply, value: ?bson.Value) ![]const query.SortKey {
|
|
const pairs = doc_arg(value) orelse return &.{};
|
|
const out = try reply.arena_alloc().alloc(query.SortKey, pairs.len);
|
|
for (pairs, 0..) |p, i| {
|
|
const descending = switch (p.value) {
|
|
.int32 => |n| n < 0,
|
|
.int64 => |n| n < 0,
|
|
.double => |n| n < 0,
|
|
.string => |s| std.mem.eql(u8, s, "desc"),
|
|
else => false,
|
|
};
|
|
out[i] = .{
|
|
.path = try reply.arena_alloc().dupe(u8, p.key),
|
|
.descending = descending,
|
|
};
|
|
}
|
|
return out;
|
|
}
|
|
|
|
fn doc_arg(v: ?bson.Value) ?[]const bson.Pair {
|
|
return switch (v orelse return null) {
|
|
.doc => |pairs| pairs,
|
|
else => null,
|
|
};
|
|
}
|
|
|
|
fn str_arg(v: ?bson.Value) ?[]const u8 {
|
|
return switch (v orelse return null) {
|
|
.string => |s| s,
|
|
else => null,
|
|
};
|
|
}
|
|
|
|
fn bool_arg(v: ?bson.Value) ?bool {
|
|
return switch (v orelse return null) {
|
|
.bool => |b| b,
|
|
.int32 => |i| i != 0,
|
|
else => null,
|
|
};
|
|
}
|
|
|
|
fn int_arg(v: ?bson.Value) ?u64 {
|
|
const i = int_value(v) orelse return null;
|
|
return if (i < 0) null else @intCast(i);
|
|
}
|
|
|
|
/// Numeric argument as a signed integer, with no clamping — callers that care
|
|
/// about the sign (find's limit, delete's limit, $skip) apply their own rule.
|
|
fn int_value(v: ?bson.Value) ?i64 {
|
|
return switch (v orelse return null) {
|
|
.int32 => |i| i,
|
|
.int64 => |i| i,
|
|
// lossyCast saturates instead of trapping on out-of-range doubles.
|
|
.double => |d| std.math.lossyCast(i64, d),
|
|
else => null,
|
|
};
|
|
}
|
|
|
|
/// A `$skip`/`$limit` stage operand: a non-negative document count. Writes
|
|
/// the error reply and returns null when the operand is not a number.
|
|
fn stage_count(reply: *wire.Reply, v: bson.Value, stage: []const u8) !?usize {
|
|
const n = int_value(v) orelse {
|
|
const text = try std.fmt.allocPrint(reply.arena_alloc(), "{s} requires a number", .{stage});
|
|
try bad_value(reply, text);
|
|
return null;
|
|
};
|
|
return @intCast(@max(0, n));
|
|
}
|
|
|
|
/// Fetch a batch argument, writing the standard error reply and returning
|
|
/// null when it is missing or malformed.
|
|
fn batch_arg(
|
|
msg: *wire.Message,
|
|
reply: *wire.Reply,
|
|
cmd: []const u8,
|
|
name: []const u8,
|
|
) !?[]const bson.Document {
|
|
return msg.batch(name) catch |err| {
|
|
const arena = reply.arena_alloc();
|
|
const text = switch (err) {
|
|
error.MissingBatch => try std.fmt.allocPrint(arena, "{s} requires {s}", .{ cmd, name }),
|
|
error.BatchNotArray => try std.fmt.allocPrint(arena, "{s} must be an array", .{name}),
|
|
error.BatchElementNotDoc => try std.fmt.allocPrint(arena, "{s} must be documents", .{name}),
|
|
error.OutOfMemory => return error.OutOfMemory,
|
|
};
|
|
try bad_value(reply, text);
|
|
return null;
|
|
};
|
|
}
|
|
|
|
fn invalid_arg(reply: *wire.Reply, msg: []const u8) !void {
|
|
return reply.put_error(@intFromEnum(ErrorCode.invalid_options), "InvalidOptions", msg);
|
|
}
|
|
|
|
fn bad_value(reply: *wire.Reply, msg: []const u8) !void {
|
|
return reply.put_error(@intFromEnum(ErrorCode.bad_value), "BadValue", msg);
|
|
}
|
|
|
|
fn failed_to_parse(reply: *wire.Reply, msg: []const u8) !void {
|
|
return reply.put_error(@intFromEnum(ErrorCode.failed_to_parse), "FailedToParse", msg);
|
|
}
|
|
|
|
/// The E11000 text, shared by the top-level error reply and the per-document
|
|
/// `writeErrors` entries of a batch insert. Uses the collection's
|
|
/// dup_index (set by a rejected unique-index write) when the conflict came
|
|
/// from a secondary index; otherwise it is the _id_ index. Per-collection
|
|
/// so concurrent writers on other collections cannot clobber it.
|
|
fn duplicate_key_message(
|
|
ctx: *Context,
|
|
reply: *wire.Reply,
|
|
db_name: []const u8,
|
|
coll_name: []const u8,
|
|
doc: *const bson.Document,
|
|
) ![]const u8 {
|
|
var index_name: []const u8 = "_id_";
|
|
var key_text: []const u8 = undefined;
|
|
const coll = ctx.engine.get_collection(db_name, coll_name);
|
|
if (coll) |c| {
|
|
if (c.dup_index) |name| {
|
|
index_name = name;
|
|
key_text = try render_dup_key(ctx, reply, db_name, coll_name, name, doc);
|
|
return e11000_message(reply, db_name, coll_name, index_name, key_text);
|
|
}
|
|
}
|
|
key_text = try serialize_value_compact(reply, doc.get("_id") orelse bson.Value.null);
|
|
return e11000_message(reply, db_name, coll_name, index_name, key_text);
|
|
}
|
|
|
|
/// Render the dup key of a secondary index from the offending document: the
|
|
/// document's values for the index key pattern.
|
|
fn render_dup_key(
|
|
ctx: *Context,
|
|
reply: *wire.Reply,
|
|
db_name: []const u8,
|
|
coll_name: []const u8,
|
|
index_name: []const u8,
|
|
doc: *const bson.Document,
|
|
) ![]const u8 {
|
|
const arena = reply.arena_alloc();
|
|
const coll = ctx.engine.get_collection(db_name, coll_name) orelse
|
|
// Index not found (defensive): fall back to the document's _id.
|
|
return serialize_value_compact(reply, doc.get("_id") orelse bson.Value.null);
|
|
const ix = coll.find_index(index_name) orelse
|
|
return serialize_value_compact(reply, doc.get("_id") orelse bson.Value.null);
|
|
|
|
var out: std.ArrayListUnmanaged(u8) = .empty;
|
|
defer out.deinit(arena);
|
|
try out.append(arena, '{');
|
|
for (ix.keys, 0..) |k, i| {
|
|
if (i > 0) try out.appendSlice(arena, ", ");
|
|
try out.appendSlice(arena, k.path);
|
|
try out.appendSlice(arena, ": ");
|
|
var values: std.ArrayListUnmanaged(bson.Value) = .empty;
|
|
defer values.deinit(arena);
|
|
try query.collect_values(arena, doc.pairs, k.path, &values, 0);
|
|
const v: bson.Value = if (values.items.len > 0) values.items[0] else .null;
|
|
try out.appendSlice(arena, try serialize_value_compact(reply, v));
|
|
}
|
|
try out.append(arena, '}');
|
|
return out.toOwnedSlice(arena);
|
|
}
|
|
|
|
fn duplicate_key_error(
|
|
ctx: *Context,
|
|
reply: *wire.Reply,
|
|
db_name: []const u8,
|
|
coll_name: []const u8,
|
|
doc: *const bson.Document,
|
|
) !void {
|
|
const msg_text = try duplicate_key_message(ctx, reply, db_name, coll_name, doc);
|
|
return reply.put_error(@intFromEnum(ErrorCode.duplicate_key), "DuplicateKey", msg_text);
|
|
}
|
|
|
|
/// Compact extended-JSON-ish rendering of a value for error messages.
|
|
fn serialize_value_compact(reply: *wire.Reply, v: bson.Value) ![]const u8 {
|
|
const arena = reply.arena_alloc();
|
|
return switch (v) {
|
|
.int32 => |i| try std.fmt.allocPrint(arena, "{d}", .{i}),
|
|
.int64 => |i| try std.fmt.allocPrint(arena, "{d}", .{i}),
|
|
.double => |d| try std.fmt.allocPrint(arena, "{d}", .{d}),
|
|
.string => |s| try std.fmt.allocPrint(arena, "'{s}'", .{s}),
|
|
.bool => |b| try std.fmt.allocPrint(arena, "{}", .{b}),
|
|
.object_id => |oid| blk: {
|
|
const hex = std.fmt.bytesToHex(oid[0..], .lower);
|
|
break :blk try std.fmt.allocPrint(arena, "ObjectId('{s}')", .{hex});
|
|
},
|
|
else => "{ ... }",
|
|
};
|
|
}
|
|
|
|
/// Build a { id: <n>, ns: "...", <batch_key>: [...] } cursor document.
|
|
pub fn cursor_doc(
|
|
reply: *wire.Reply,
|
|
cursor_id: i64,
|
|
ns: []const u8,
|
|
batch_key: []const u8,
|
|
docs: []const bson.Value,
|
|
) ![]const bson.Pair {
|
|
const c = try reply.arena_alloc().alloc(bson.Pair, 3);
|
|
c[0] = .{ .key = "id", .value = .{ .int64 = cursor_id } };
|
|
c[1] = .{ .key = "ns", .value = .{ .string = ns } };
|
|
c[2] = .{ .key = batch_key, .value = .{ .array = docs } };
|
|
return c;
|
|
}
|
|
|
|
fn format_namespace(reply: *wire.Reply, db_name: []const u8, coll_name: []const u8) ![]const u8 {
|
|
return std.fmt.allocPrint(reply.arena_alloc(), "{s}.{s}", .{ db_name, coll_name });
|
|
}
|
|
|
|
fn int_array(reply: *wire.Reply, values: []const i32) ![]const bson.Value {
|
|
const out = try reply.arena_alloc().alloc(bson.Value, values.len);
|
|
for (values, 0..) |v, i| out[i] = .{ .int32 = v };
|
|
return out;
|
|
}
|
|
|
|
fn str_array(reply: *wire.Reply, values: []const []const u8) ![]const bson.Value {
|
|
const out = try reply.arena_alloc().alloc(bson.Value, values.len);
|
|
for (values, 0..) |v, i| out[i] = .{ .string = try reply.arena_alloc().dupe(u8, v) };
|
|
return out;
|
|
}
|
|
|
|
// ---------------------------------------------------------------------------
|
|
// Tests
|
|
// ---------------------------------------------------------------------------
|
|
|
|
const testing = std.testing;
|
|
|
|
/// Temp-log-backed engine plus the Context a command test dispatches with.
|
|
const TestDb = struct {
|
|
tmp: std.testing.TmpDir,
|
|
path: []u8,
|
|
engine: db.Engine,
|
|
gen: bson.ObjectIdGen,
|
|
|
|
fn init(io: std.Io) !TestDb {
|
|
const tmp = std.testing.tmpDir(.{});
|
|
const path = try std.fmt.allocPrint(testing.allocator, ".zig-cache/tmp/{s}/cmd.log", .{tmp.sub_path});
|
|
return .{
|
|
.tmp = tmp,
|
|
.path = path,
|
|
.engine = try db.Engine.open(testing.allocator, io, path),
|
|
.gen = bson.ObjectIdGen.init(io),
|
|
};
|
|
}
|
|
|
|
fn deinit(self: *TestDb) void {
|
|
self.engine.deinit();
|
|
self.tmp.cleanup();
|
|
testing.allocator.free(self.path);
|
|
}
|
|
|
|
fn ctx(self: *TestDb, io: std.Io) Context {
|
|
return test_ctx(io, &self.engine, &self.gen, 1);
|
|
}
|
|
};
|
|
|
|
fn test_ctx(io: std.Io, engine: *db.Engine, gen: *bson.ObjectIdGen, connection_id: u32) Context {
|
|
return .{
|
|
.gpa = testing.allocator,
|
|
.io = io,
|
|
.oid_gen = gen,
|
|
.connection_id = connection_id,
|
|
.client_desc = "127.0.0.1:0",
|
|
.engine = engine,
|
|
.server_start = std.Io.Timestamp.now(io, .real),
|
|
};
|
|
}
|
|
|
|
test "ping and hello replies parse" {
|
|
var threaded: std.Io.Threaded = .init_single_threaded;
|
|
defer threaded.deinit();
|
|
const io = threaded.io();
|
|
var tdb = try TestDb.init(io);
|
|
defer tdb.deinit();
|
|
var ctx = tdb.ctx(io);
|
|
|
|
var reply = wire.Reply.init(testing.allocator);
|
|
defer reply.deinit();
|
|
var msg = try parse_fake_msg("ping", .null, &.{});
|
|
defer msg.deinit();
|
|
try dispatch(&ctx, &msg, &reply);
|
|
try testing.expectEqual(@as(f64, 1.0), reply.pairs.items[0].value.double);
|
|
|
|
var reply2 = wire.Reply.init(testing.allocator);
|
|
defer reply2.deinit();
|
|
var msg2 = try parse_fake_msg("hello", .null, &.{});
|
|
defer msg2.deinit();
|
|
try dispatch(&ctx, &msg2, &reply2);
|
|
const ok = bson.get_pair(reply2.pairs.items, "ok").?;
|
|
try testing.expectEqual(@as(f64, 1.0), ok.double);
|
|
const primary = bson.get_pair(reply2.pairs.items, "isWritablePrimary").?;
|
|
try testing.expect(primary.bool);
|
|
try testing.expectEqual(@as(i32, 8), bson.get_pair(reply2.pairs.items, "maxWireVersion").?.int32);
|
|
}
|
|
|
|
test "handshake does not advertise the streaming hello protocol" {
|
|
// `topologyVersion` in a hello/isMaster reply tells the driver it may
|
|
// monitor with an exhaust hello and expect moreToCome replies we never
|
|
// send; the driver then kills the connection every heartbeat.
|
|
var threaded: std.Io.Threaded = .init_single_threaded;
|
|
defer threaded.deinit();
|
|
const io = threaded.io();
|
|
var tdb = try TestDb.init(io);
|
|
defer tdb.deinit();
|
|
var ctx = tdb.ctx(io);
|
|
|
|
for ([_][]const u8{ "hello", "isMaster", "ismaster" }) |cmd| {
|
|
var reply = wire.Reply.init(testing.allocator);
|
|
defer reply.deinit();
|
|
var msg = try parse_fake_msg(cmd, .null, &.{});
|
|
defer msg.deinit();
|
|
try dispatch(&ctx, &msg, &reply);
|
|
try testing.expectEqual(@as(f64, 1.0), bson.get_pair(reply.pairs.items, "ok").?.double);
|
|
try testing.expect(bson.get_pair(reply.pairs.items, "topologyVersion") == null);
|
|
}
|
|
}
|
|
|
|
test "unknown command gives CommandNotFound" {
|
|
var threaded: std.Io.Threaded = .init_single_threaded;
|
|
defer threaded.deinit();
|
|
const io = threaded.io();
|
|
var tdb = try TestDb.init(io);
|
|
defer tdb.deinit();
|
|
var ctx = tdb.ctx(io);
|
|
var reply = wire.Reply.init(testing.allocator);
|
|
defer reply.deinit();
|
|
var msg = try parse_fake_msg("nonsenseCmd", .null, &.{});
|
|
defer msg.deinit();
|
|
try dispatch(&ctx, &msg, &reply);
|
|
try testing.expectEqual(@as(f64, 0.0), bson.get_pair(reply.pairs.items, "ok").?.double);
|
|
try testing.expectEqual(@as(i32, 59), bson.get_pair(reply.pairs.items, "code").?.int32);
|
|
try testing.expectEqualStrings("CommandNotFound", bson.get_pair(reply.pairs.items, "codeName").?.string);
|
|
}
|
|
|
|
test "getParameter responds for featureCompatibilityVersion" {
|
|
var threaded: std.Io.Threaded = .init_single_threaded;
|
|
defer threaded.deinit();
|
|
const io = threaded.io();
|
|
var tdb = try TestDb.init(io);
|
|
defer tdb.deinit();
|
|
var ctx = tdb.ctx(io);
|
|
var reply = wire.Reply.init(testing.allocator);
|
|
defer reply.deinit();
|
|
var fcv_doc = try testing.allocator.alloc(bson.Pair, 1);
|
|
defer testing.allocator.free(fcv_doc);
|
|
fcv_doc[0] = .{ .key = "featureCompatibilityVersion", .value = .{ .int32 = 1 } };
|
|
var msg = try parse_fake_msg("getParameter", .{ .doc = fcv_doc }, &.{});
|
|
defer msg.deinit();
|
|
try dispatch(&ctx, &msg, &reply);
|
|
const fcv = bson.get_pair(reply.pairs.items, "featureCompatibilityVersion").?;
|
|
try testing.expectEqualStrings("4.4", fcv.doc[0].value.string);
|
|
}
|
|
|
|
test "insert counts only successful inserts" {
|
|
var threaded: std.Io.Threaded = .init_single_threaded;
|
|
defer threaded.deinit();
|
|
const io = threaded.io();
|
|
var tdb = try TestDb.init(io);
|
|
defer tdb.deinit();
|
|
var ctx = tdb.ctx(io);
|
|
|
|
// documents: [{_id:1}, {_id:1} (dup), {_id:2}] → n: 2 + one writeError.
|
|
const docs = [_]bson.Value{
|
|
.{ .doc = &.{.{ .key = "_id", .value = .{ .int32 = 1 } }} },
|
|
.{ .doc = &.{.{ .key = "_id", .value = .{ .int32 = 1 } }} },
|
|
.{ .doc = &.{.{ .key = "_id", .value = .{ .int32 = 2 } }} },
|
|
};
|
|
var reply = wire.Reply.init(testing.allocator);
|
|
defer reply.deinit();
|
|
var msg = try parse_fake_msg("insert", .{ .string = "users" }, &.{
|
|
.{ .key = "documents", .value = .{ .array = &docs } },
|
|
});
|
|
defer msg.deinit();
|
|
try dispatch(&ctx, &msg, &reply);
|
|
try testing.expectEqual(@as(i64, 2), bson.get_pair(reply.pairs.items, "n").?.int32);
|
|
const errors = bson.get_pair(reply.pairs.items, "writeErrors").?;
|
|
try testing.expectEqual(@as(usize, 1), errors.array.len);
|
|
const first = errors.array[0].doc[0];
|
|
try testing.expectEqualStrings("index", first.key);
|
|
try testing.expectEqual(@as(i64, 1), first.value.int32);
|
|
}
|
|
|
|
fn parse_fake_msg(name: []const u8, value: bson.Value, extra: []const bson.Pair) !wire.Message {
|
|
var out: std.ArrayListUnmanaged(u8) = .empty;
|
|
defer out.deinit(testing.allocator);
|
|
const pairs = try testing.allocator.alloc(bson.Pair, 2 + extra.len);
|
|
defer testing.allocator.free(pairs);
|
|
pairs[0] = .{ .key = name, .value = value };
|
|
pairs[1] = .{ .key = "$db", .value = .{ .string = "test" } };
|
|
@memcpy(pairs[2..], extra);
|
|
try bson.write_doc(pairs, testing.allocator, &out);
|
|
var msg: std.ArrayListUnmanaged(u8) = .empty;
|
|
defer msg.deinit(testing.allocator);
|
|
try msg.appendSlice(testing.allocator, &[_]u8{ 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0xDD, 0x07, 0, 0, 0, 0, 0, 0 });
|
|
try msg.append(testing.allocator, 0x00);
|
|
try msg.appendSlice(testing.allocator, out.items);
|
|
std.mem.writeInt(u32, msg.items[0..4], @intCast(msg.items.len), .little);
|
|
std.mem.writeInt(i32, msg.items[12..16], wire.op_code_msg, .little);
|
|
return wire.Message.parse(testing.allocator, msg.items);
|
|
}
|
|
|
|
test "concurrent insert/find commands on a threaded Io" {
|
|
// Exercises dispatch's lock classification end-to-end: writer fibers run
|
|
// `insert` under the exclusive lock, reader fibers run `count`/`find`
|
|
// under the shared lock. Every committed insert must be visible once all
|
|
// writers finish, and no reader may observe more docs than can exist.
|
|
var threaded: std.Io.Threaded = .init(testing.allocator, .{});
|
|
defer threaded.deinit();
|
|
const io = threaded.io();
|
|
var gen = bson.ObjectIdGen.init(io);
|
|
|
|
var tmp = std.testing.tmpDir(.{});
|
|
defer tmp.cleanup();
|
|
const path = try std.fmt.allocPrint(testing.allocator, ".zig-cache/tmp/{s}/conc.log", .{tmp.sub_path});
|
|
defer testing.allocator.free(path);
|
|
var engine = try db.Engine.open(testing.allocator, io, path);
|
|
defer engine.deinit();
|
|
|
|
const writers = 4;
|
|
const readers = 4;
|
|
const per_writer: i32 = 150;
|
|
const total: i32 = writers * per_writer;
|
|
var next_id = std.atomic.Value(i32).init(1);
|
|
var remaining = std.atomic.Value(usize).init(@intCast(total));
|
|
|
|
const Worker = struct {
|
|
fn writer(
|
|
iow: std.Io,
|
|
eng: *db.Engine,
|
|
id_counter: *std.atomic.Value(i32),
|
|
pending: *std.atomic.Value(usize),
|
|
fiber_id: u32,
|
|
total_writes: i32,
|
|
) error{Canceled}!void {
|
|
var wgen = bson.ObjectIdGen.init(iow);
|
|
var ctx = test_ctx(iow, eng, &wgen, fiber_id);
|
|
while (true) {
|
|
const id = id_counter.fetchAdd(1, .monotonic);
|
|
if (id > total_writes) return;
|
|
const docs = [_]bson.Value{.{ .doc = &.{.{ .key = "_id", .value = .{ .int32 = id } }} }};
|
|
var msg = parse_fake_msg("insert", .{ .string = "users" }, &.{
|
|
.{ .key = "documents", .value = .{ .array = &docs } },
|
|
}) catch return error.Canceled;
|
|
defer msg.deinit();
|
|
var reply = wire.Reply.init(testing.allocator);
|
|
defer reply.deinit();
|
|
dispatch(&ctx, &msg, &reply) catch return error.Canceled;
|
|
_ = pending.fetchSub(1, .monotonic);
|
|
}
|
|
}
|
|
|
|
fn reader(
|
|
iow: std.Io,
|
|
eng: *db.Engine,
|
|
pending: *std.atomic.Value(usize),
|
|
fiber_id: u32,
|
|
total_writes: i32,
|
|
) error{Canceled}!void {
|
|
var rgen = bson.ObjectIdGen.init(iow);
|
|
var ctx = test_ctx(iow, eng, &rgen, fiber_id);
|
|
while (pending.load(.acquire) > 0) {
|
|
var msg = parse_fake_msg("count", .{ .string = "users" }, &.{}) catch return error.Canceled;
|
|
defer msg.deinit();
|
|
var reply = wire.Reply.init(testing.allocator);
|
|
defer reply.deinit();
|
|
dispatch(&ctx, &msg, &reply) catch return error.Canceled;
|
|
const n = bson.get_pair(reply.pairs.items, "n") orelse return error.Canceled;
|
|
if (n.int32 > total_writes) return error.Canceled;
|
|
}
|
|
}
|
|
};
|
|
|
|
var group: std.Io.Group = .init;
|
|
defer group.cancel(io);
|
|
for (0..readers) |i| group.async(io, Worker.reader, .{ io, &engine, &remaining, @intCast(i + 1), total });
|
|
for (0..writers) |i| group.async(io, Worker.writer, .{ io, &engine, &next_id, &remaining, @intCast(i + 1), total });
|
|
try group.await(io);
|
|
|
|
var ctx = test_ctx(io, &engine, &gen, 0);
|
|
var msg = try parse_fake_msg("count", .{ .string = "users" }, &.{});
|
|
defer msg.deinit();
|
|
var reply = wire.Reply.init(testing.allocator);
|
|
defer reply.deinit();
|
|
try dispatch(&ctx, &msg, &reply);
|
|
try testing.expectEqual(total, bson.get_pair(reply.pairs.items, "n").?.int32);
|
|
}
|
|
|
|
// -- index command tests ----------------------------------------------------
|
|
|
|
/// Dispatch an insert of the given documents (each a bson.Value .doc).
|
|
fn dispatch_insert(
|
|
tdb: *TestDb,
|
|
io: std.Io,
|
|
coll_name: []const u8,
|
|
docs: []const bson.Value,
|
|
) !void {
|
|
var ctx = tdb.ctx(io);
|
|
var msg = try parse_fake_msg("insert", .{ .string = coll_name }, &.{
|
|
.{ .key = "documents", .value = .{ .array = docs } },
|
|
});
|
|
defer msg.deinit();
|
|
var reply = wire.Reply.init(testing.allocator);
|
|
defer reply.deinit();
|
|
try dispatch(&ctx, &msg, &reply);
|
|
try testing.expectEqual(@as(f64, 1.0), bson.get_pair(reply.pairs.items, "ok").?.double);
|
|
// `ok: 1` is not success for an insert batch: a rejected document comes
|
|
// back as a writeError alongside it. Asserting only `ok` let a corpus
|
|
// silently shrink -- when _id_ became a unique index, the mixed-type
|
|
// corpus below lost its int64 1 document and every test over it still
|
|
// passed, over nine documents instead of ten.
|
|
if (bson.get_pair(reply.pairs.items, "writeErrors")) |we| {
|
|
std.debug.print("dispatch_insert: unexpected writeErrors: {any}\n", .{we});
|
|
return error.TestUnexpectedResult;
|
|
}
|
|
try testing.expectEqual(@as(i32, @intCast(docs.len)), bson.get_pair(reply.pairs.items, "n").?.int32);
|
|
}
|
|
|
|
/// Dispatch createIndexes for one spec.
|
|
fn dispatch_create_index(tdb: *TestDb, io: std.Io, coll_name: []const u8, spec: bson.Value) !void {
|
|
var ctx = tdb.ctx(io);
|
|
const specs = [_]bson.Value{spec};
|
|
var msg = try parse_fake_msg("createIndexes", .{ .string = coll_name }, &.{
|
|
.{ .key = "indexes", .value = .{ .array = &specs } },
|
|
});
|
|
defer msg.deinit();
|
|
var reply = wire.Reply.init(testing.allocator);
|
|
defer reply.deinit();
|
|
try dispatch(&ctx, &msg, &reply);
|
|
try testing.expectEqual(@as(f64, 1.0), bson.get_pair(reply.pairs.items, "ok").?.double);
|
|
}
|
|
|
|
/// Dispatch find and append the serialized _id of every result to `out`.
|
|
fn dispatch_find_ids(
|
|
tdb: *TestDb,
|
|
io: std.Io,
|
|
coll_name: []const u8,
|
|
filter: []const bson.Pair,
|
|
out: *std.ArrayListUnmanaged([]u8),
|
|
) !void {
|
|
var ctx = tdb.ctx(io);
|
|
var msg = try parse_fake_msg("find", .{ .string = coll_name }, &.{
|
|
.{ .key = "filter", .value = .{ .doc = filter } },
|
|
});
|
|
defer msg.deinit();
|
|
var reply = wire.Reply.init(testing.allocator);
|
|
defer reply.deinit();
|
|
try dispatch(&ctx, &msg, &reply);
|
|
try testing.expectEqual(@as(f64, 1.0), bson.get_pair(reply.pairs.items, "ok").?.double);
|
|
const cur = bson.get_pair(reply.pairs.items, "cursor") orelse return error.TestUnexpectedResult;
|
|
const batch = switch (bson.get_pair(cur.doc, "firstBatch") orelse return error.TestUnexpectedResult) {
|
|
.array => |a| a,
|
|
else => return error.TestUnexpectedResult,
|
|
};
|
|
for (batch) |d| {
|
|
const idv = bson.get_pair(d.doc, "_id") orelse continue;
|
|
try out.append(testing.allocator, try bson.serialize_value(testing.allocator, idv));
|
|
}
|
|
std.mem.sort([]u8, out.items, {}, less_u8);
|
|
}
|
|
|
|
fn less_u8(_: void, a: []u8, b: []u8) bool {
|
|
return std.mem.order(u8, a, b) == .lt;
|
|
}
|
|
|
|
test "createIndexes, listIndexes, dropIndexes, and idempotent re-create" {
|
|
var threaded: std.Io.Threaded = .init_single_threaded;
|
|
defer threaded.deinit();
|
|
const io = threaded.io();
|
|
var tdb = try TestDb.init(io);
|
|
defer tdb.deinit();
|
|
|
|
// createIndexes builds the index immediately.
|
|
{
|
|
var ctx = tdb.ctx(io);
|
|
const specs = [_]bson.Value{.{ .doc = &.{
|
|
.{ .key = "key", .value = .{ .doc = &.{.{ .key = "email", .value = .{ .int32 = 1 } }} } },
|
|
.{ .key = "name", .value = .{ .string = "email_1" } },
|
|
} }};
|
|
var msg = try parse_fake_msg("createIndexes", .{ .string = "users" }, &.{
|
|
.{ .key = "indexes", .value = .{ .array = &specs } },
|
|
});
|
|
defer msg.deinit();
|
|
var reply = wire.Reply.init(testing.allocator);
|
|
defer reply.deinit();
|
|
try dispatch(&ctx, &msg, &reply);
|
|
try testing.expectEqual(@as(f64, 1.0), bson.get_pair(reply.pairs.items, "ok").?.double);
|
|
try testing.expectEqual(@as(i64, 1), bson.get_pair(reply.pairs.items, "numIndexesBefore").?.int32);
|
|
try testing.expectEqual(@as(i64, 2), bson.get_pair(reply.pairs.items, "numIndexesAfter").?.int32);
|
|
}
|
|
|
|
// Idempotent re-create of the same spec.
|
|
try dispatch_create_index(&tdb, io, "users", .{ .doc = &.{
|
|
.{ .key = "key", .value = .{ .doc = &.{.{ .key = "email", .value = .{ .int32 = 1 } }} } },
|
|
.{ .key = "name", .value = .{ .string = "email_1" } },
|
|
} });
|
|
|
|
// listIndexes: _id_ first, then the secondary.
|
|
{
|
|
var ctx = tdb.ctx(io);
|
|
var msg = try parse_fake_msg("listIndexes", .{ .string = "users" }, &.{});
|
|
defer msg.deinit();
|
|
var reply = wire.Reply.init(testing.allocator);
|
|
defer reply.deinit();
|
|
try dispatch(&ctx, &msg, &reply);
|
|
try testing.expectEqual(@as(f64, 1.0), bson.get_pair(reply.pairs.items, "ok").?.double);
|
|
const cur = bson.get_pair(reply.pairs.items, "cursor").?;
|
|
const batch = bson.get_pair(cur.doc, "firstBatch").?.array;
|
|
try testing.expectEqual(@as(usize, 2), batch.len);
|
|
try testing.expectEqualStrings("_id_", bson.get_pair(batch[0].doc, "name").?.string);
|
|
try testing.expectEqualStrings("email_1", bson.get_pair(batch[1].doc, "name").?.string);
|
|
const key_pairs = bson.get_pair(batch[1].doc, "key").?.doc;
|
|
try testing.expect(bson.get_pair(key_pairs, "email") != null);
|
|
}
|
|
|
|
// listIndexes on a missing collection: NamespaceNotFound.
|
|
{
|
|
var ctx = tdb.ctx(io);
|
|
var msg = try parse_fake_msg("listIndexes", .{ .string = "nope" }, &.{});
|
|
defer msg.deinit();
|
|
var reply = wire.Reply.init(testing.allocator);
|
|
defer reply.deinit();
|
|
try dispatch(&ctx, &msg, &reply);
|
|
try testing.expectEqual(@as(i32, 26), bson.get_pair(reply.pairs.items, "code").?.int32);
|
|
}
|
|
|
|
// dropIndexes("*") removes the secondary but keeps _id_.
|
|
{
|
|
var ctx = tdb.ctx(io);
|
|
var msg = try parse_fake_msg("dropIndexes", .{ .string = "users" }, &.{
|
|
.{ .key = "index", .value = .{ .string = "*" } },
|
|
});
|
|
defer msg.deinit();
|
|
var reply = wire.Reply.init(testing.allocator);
|
|
defer reply.deinit();
|
|
try dispatch(&ctx, &msg, &reply);
|
|
try testing.expectEqual(@as(f64, 1.0), bson.get_pair(reply.pairs.items, "ok").?.double);
|
|
try testing.expectEqual(@as(i64, 2), bson.get_pair(reply.pairs.items, "nIndexesWas").?.int32);
|
|
}
|
|
try testing.expectEqual(@as(usize, 0), tdb.engine.get_collection("test", "users").?.indexes.items.len);
|
|
|
|
// dropIndexes("_id_") errors.
|
|
{
|
|
var ctx = tdb.ctx(io);
|
|
var msg = try parse_fake_msg("dropIndexes", .{ .string = "users" }, &.{
|
|
.{ .key = "index", .value = .{ .string = "_id_" } },
|
|
});
|
|
defer msg.deinit();
|
|
var reply = wire.Reply.init(testing.allocator);
|
|
defer reply.deinit();
|
|
try dispatch(&ctx, &msg, &reply);
|
|
try testing.expectEqual(@as(i32, 72), bson.get_pair(reply.pairs.items, "code").?.int32);
|
|
}
|
|
}
|
|
|
|
test "TTL index round-trips through createIndexes/listIndexes; bad specs give 67" {
|
|
var threaded: std.Io.Threaded = .init_single_threaded;
|
|
defer threaded.deinit();
|
|
const io = threaded.io();
|
|
var tdb = try TestDb.init(io);
|
|
defer tdb.deinit();
|
|
|
|
// A driver sends expireAfterSeconds as a double.
|
|
try dispatch_create_index(&tdb, io, "sessions", .{ .doc = &.{
|
|
.{
|
|
.key = "key",
|
|
.value = .{ .doc = &.{
|
|
.{ .key = "expireAt", .value = .{ .int32 = 1 } },
|
|
} },
|
|
},
|
|
|
|
.{ .key = "name", .value = .{ .string = "expireAt_1" } },
|
|
.{ .key = "expireAfterSeconds", .value = .{ .double = 60.0 } },
|
|
} });
|
|
try testing.expectEqual(@as(?i64, 60), tdb.engine.get_collection("test", "sessions").?.indexes.items[0].ttl);
|
|
|
|
// listIndexes reports it back.
|
|
{
|
|
var ctx = tdb.ctx(io);
|
|
var msg = try parse_fake_msg("listIndexes", .{ .string = "sessions" }, &.{});
|
|
defer msg.deinit();
|
|
var reply = wire.Reply.init(testing.allocator);
|
|
defer reply.deinit();
|
|
try dispatch(&ctx, &msg, &reply);
|
|
const cur = bson.get_pair(reply.pairs.items, "cursor").?;
|
|
const batch = bson.get_pair(cur.doc, "firstBatch").?.array;
|
|
try testing.expectEqual(@as(usize, 2), batch.len);
|
|
try testing.expectEqual(@as(i32, 60), bson.get_pair(batch[1].doc, "expireAfterSeconds").?.int32);
|
|
// The _id_ entry never carries one.
|
|
try testing.expect(bson.get_pair(batch[0].doc, "expireAfterSeconds") == null);
|
|
}
|
|
|
|
// Same name and key, different expiry: IndexOptionsConflict, as in
|
|
// MongoDB (changing it is a collMod, which this server does not have).
|
|
const bad = [_]struct { spec: bson.Value, code: i32 }{
|
|
.{ .code = 85, .spec = .{ .doc = &.{
|
|
.{ .key = "key", .value = .{ .doc = &.{.{ .key = "expireAt", .value = .{ .int32 = 1 } }} } },
|
|
.{ .key = "name", .value = .{ .string = "expireAt_1" } },
|
|
.{ .key = "expireAfterSeconds", .value = .{ .int32 = 90 } },
|
|
} } },
|
|
// TTL on a compound key.
|
|
.{ .code = 67, .spec = .{ .doc = &.{
|
|
.{ .key = "key", .value = .{ .doc = &.{
|
|
.{ .key = "a", .value = .{ .int32 = 1 } },
|
|
.{ .key = "b", .value = .{ .int32 = 1 } },
|
|
} } },
|
|
.{ .key = "expireAfterSeconds", .value = .{ .int32 = 60 } },
|
|
} } },
|
|
// Negative and non-numeric expiries.
|
|
.{ .code = 67, .spec = .{ .doc = &.{
|
|
.{ .key = "key", .value = .{ .doc = &.{.{ .key = "a", .value = .{ .int32 = 1 } }} } },
|
|
.{ .key = "expireAfterSeconds", .value = .{ .int32 = -1 } },
|
|
} } },
|
|
.{ .code = 67, .spec = .{ .doc = &.{
|
|
.{ .key = "key", .value = .{ .doc = &.{.{ .key = "a", .value = .{ .int32 = 1 } }} } },
|
|
.{ .key = "expireAfterSeconds", .value = .{ .string = "60" } },
|
|
} } },
|
|
// Past MongoDB's 2147483647 bound.
|
|
.{ .code = 67, .spec = .{ .doc = &.{
|
|
.{ .key = "key", .value = .{ .doc = &.{.{ .key = "a", .value = .{ .int32 = 1 } }} } },
|
|
.{ .key = "expireAfterSeconds", .value = .{ .int64 = index.max_expire_after_seconds + 1 } },
|
|
} } },
|
|
// {_id: 1} is otherwise an idempotent no-op, but an expiry on it
|
|
// would be silently dropped, so it is rejected instead.
|
|
.{ .code = 197, .spec = .{ .doc = &.{
|
|
.{ .key = "key", .value = .{ .doc = &.{.{ .key = "_id", .value = .{ .int32 = 1 } }} } },
|
|
.{ .key = "expireAfterSeconds", .value = .{ .int32 = 60 } },
|
|
} } },
|
|
};
|
|
for (bad) |case| {
|
|
var ctx = tdb.ctx(io);
|
|
const specs = [_]bson.Value{case.spec};
|
|
var msg = try parse_fake_msg("createIndexes", .{ .string = "sessions" }, &.{
|
|
.{ .key = "indexes", .value = .{ .array = &specs } },
|
|
});
|
|
defer msg.deinit();
|
|
var reply = wire.Reply.init(testing.allocator);
|
|
defer reply.deinit();
|
|
try dispatch(&ctx, &msg, &reply);
|
|
try testing.expectEqual(case.code, bson.get_pair(reply.pairs.items, "code").?.int32);
|
|
}
|
|
// Nothing partial was registered by the rejected specs.
|
|
try testing.expectEqual(@as(usize, 1), tdb.engine.get_collection("test", "sessions").?.indexes.items.len);
|
|
}
|
|
|
|
test "unique index constraint returns 11000 through insert and update" {
|
|
var threaded: std.Io.Threaded = .init_single_threaded;
|
|
defer threaded.deinit();
|
|
const io = threaded.io();
|
|
var tdb = try TestDb.init(io);
|
|
defer tdb.deinit();
|
|
|
|
try dispatch_create_index(&tdb, io, "users", .{ .doc = &.{
|
|
.{ .key = "key", .value = .{ .doc = &.{.{ .key = "email", .value = .{ .int32 = 1 } }} } },
|
|
.{ .key = "name", .value = .{ .string = "email_1" } },
|
|
.{ .key = "unique", .value = .{ .bool = true } },
|
|
} });
|
|
|
|
const docs = [_]bson.Value{
|
|
.{ .doc = &.{ .{ .key = "_id", .value = .{ .int32 = 1 } }, .{ .key = "email", .value = .{ .string = "a@x.io" } } } },
|
|
.{ .doc = &.{ .{ .key = "_id", .value = .{ .int32 = 2 } }, .{ .key = "email", .value = .{ .string = "a@x.io" } } } },
|
|
.{ .doc = &.{ .{ .key = "_id", .value = .{ .int32 = 3 } }, .{ .key = "email", .value = .{ .string = "b@x.io" } } } },
|
|
};
|
|
var ctx = tdb.ctx(io);
|
|
var msg = try parse_fake_msg("insert", .{ .string = "users" }, &.{
|
|
.{ .key = "documents", .value = .{ .array = &docs } },
|
|
});
|
|
defer msg.deinit();
|
|
var reply = wire.Reply.init(testing.allocator);
|
|
defer reply.deinit();
|
|
try dispatch(&ctx, &msg, &reply);
|
|
// 2 inserted, 1 writeError with code 11000 naming the index.
|
|
try testing.expectEqual(@as(i64, 2), bson.get_pair(reply.pairs.items, "n").?.int32);
|
|
const errors = bson.get_pair(reply.pairs.items, "writeErrors").?.array;
|
|
try testing.expectEqual(@as(usize, 1), errors.len);
|
|
try testing.expectEqual(@as(i64, 11000), bson.get_pair(errors[0].doc, "code").?.int32);
|
|
const errmsg = bson.get_pair(errors[0].doc, "errmsg").?.string;
|
|
try testing.expect(std.mem.indexOf(u8, errmsg, "email_1") != null);
|
|
try testing.expect(std.mem.indexOf(u8, errmsg, "E11000") != null);
|
|
|
|
// An update that collides is reported as a writeError with code 11000.
|
|
const updates = [_]bson.Value{.{ .doc = &.{
|
|
.{ .key = "q", .value = .{ .doc = &.{.{ .key = "_id", .value = .{ .int32 = 3 } }} } },
|
|
.{ .key = "u", .value = .{ .doc = &.{.{ .key = "$set", .value = .{ .doc = &.{.{ .key = "email", .value = .{ .string = "a@x.io" } }} } }} } },
|
|
} }};
|
|
var msg2 = try parse_fake_msg("update", .{ .string = "users" }, &.{
|
|
.{ .key = "updates", .value = .{ .array = &updates } },
|
|
});
|
|
defer msg2.deinit();
|
|
var reply2 = wire.Reply.init(testing.allocator);
|
|
defer reply2.deinit();
|
|
try dispatch(&ctx, &msg2, &reply2);
|
|
try testing.expectEqual(@as(f64, 1.0), bson.get_pair(reply2.pairs.items, "ok").?.double);
|
|
try testing.expectEqual(@as(i64, 0), bson.get_pair(reply2.pairs.items, "nModified").?.int32);
|
|
const up_errs = bson.get_pair(reply2.pairs.items, "writeErrors").?.array;
|
|
try testing.expectEqual(@as(usize, 1), up_errs.len);
|
|
try testing.expectEqual(@as(i64, 11000), bson.get_pair(up_errs[0].doc, "code").?.int32);
|
|
}
|
|
|
|
/// Free a list of serialized ids (each element is gpa-owned).
|
|
fn free_id_list(gpa: std.mem.Allocator, list: *std.ArrayListUnmanaged([]u8)) void {
|
|
for (list.items) |id| gpa.free(id);
|
|
list.deinit(gpa);
|
|
}
|
|
|
|
/// Free the serialized ids and reset the list, keeping capacity.
|
|
fn clear_id_list(gpa: std.mem.Allocator, list: *std.ArrayListUnmanaged([]u8)) void {
|
|
for (list.items) |id| gpa.free(id);
|
|
list.clearRetainingCapacity();
|
|
}
|
|
|
|
test "aggregate $sort without a preceding $group sorts and frees correctly" {
|
|
// Regression test for a remote, client-triggerable invalid free: the
|
|
// $sort stage materialized its document list from the reply arena and
|
|
// handed it to `trees`, which is freed with the gpa. Two things made it
|
|
// survive for so long, and this test is shaped to close both:
|
|
//
|
|
// - every existing aggregate test sorts *after* a $group, which leaves
|
|
// the stream already materialized so the guilty branch never runs.
|
|
// So this pipeline must have $sort with NO $group before it.
|
|
// - the symptom was allocator-dependent (macOS malloc aborted; other
|
|
// allocators may not notice). testing.allocator detects an invalid
|
|
// free itself, which is what gives this teeth in every mode.
|
|
//
|
|
// Mutation check: change the `all.append(ctx.gpa, ...)` back to
|
|
// `all.append(arena, ...)` in cmd_aggregate and this goes red.
|
|
var threaded = std.Io.Threaded.init(testing.allocator, .{});
|
|
defer threaded.deinit();
|
|
const io = threaded.io();
|
|
|
|
var tdb = try TestDb.init(io);
|
|
defer tdb.deinit();
|
|
|
|
// Insert out of order so a missing sort is visible, not coincidental.
|
|
try dispatch_insert(&tdb, io, "agg", &.{
|
|
.{ .doc = &.{ .{ .key = "_id", .value = .{ .int32 = 1 } }, .{ .key = "x", .value = .{ .int32 = 30 } } } },
|
|
.{ .doc = &.{ .{ .key = "_id", .value = .{ .int32 = 2 } }, .{ .key = "x", .value = .{ .int32 = 10 } } } },
|
|
.{ .doc = &.{ .{ .key = "_id", .value = .{ .int32 = 3 } }, .{ .key = "x", .value = .{ .int32 = 20 } } } },
|
|
});
|
|
|
|
const sort_stage = bson.Value{ .doc = &.{.{ .key = "$sort", .value = .{ .doc = &.{.{ .key = "x", .value = .{ .int32 = 1 } }} } }} };
|
|
const stages = [_]bson.Value{sort_stage};
|
|
|
|
var ctx = tdb.ctx(io);
|
|
var msg = try parse_fake_msg("aggregate", .{ .string = "agg" }, &.{
|
|
.{ .key = "pipeline", .value = .{ .array = &stages } },
|
|
.{ .key = "cursor", .value = .{ .doc = &.{} } },
|
|
});
|
|
defer msg.deinit();
|
|
var reply = wire.Reply.init(testing.allocator);
|
|
defer reply.deinit();
|
|
try dispatch(&ctx, &msg, &reply);
|
|
|
|
try testing.expectEqual(@as(f64, 1.0), bson.get_pair(reply.pairs.items, "ok").?.double);
|
|
const cur = bson.get_pair(reply.pairs.items, "cursor") orelse return error.TestUnexpectedResult;
|
|
const batch = switch (bson.get_pair(cur.doc, "firstBatch") orelse return error.TestUnexpectedResult) {
|
|
.array => |a| a,
|
|
else => return error.TestUnexpectedResult,
|
|
};
|
|
try testing.expectEqual(@as(usize, 3), batch.len);
|
|
// Ascending by x means _id order 2, 3, 1.
|
|
const want = [_]i32{ 2, 3, 1 };
|
|
for (batch, want) |d, id| {
|
|
try testing.expectEqual(id, bson.get_pair(d.doc, "_id").?.int32);
|
|
}
|
|
}
|
|
|
|
test "count_only_pipeline accepts only shapes a count can answer" {
|
|
// The fast path skips materializing documents, so mis-accepting a
|
|
// pipeline would silently return a wrong aggregate rather than a slow
|
|
// one. Pin exactly which shapes it claims.
|
|
var reply = wire.Reply.init(testing.allocator);
|
|
defer reply.deinit();
|
|
|
|
const group_count = bson.Value{ .doc = &.{.{ .key = "$group", .value = .{ .doc = &.{
|
|
.{ .key = "_id", .value = .null },
|
|
.{ .key = "n", .value = .{ .doc = &.{.{ .key = "$sum", .value = .{ .int32 = 1 } }} } },
|
|
} } }} };
|
|
const match_k = bson.Value{ .doc = &.{.{ .key = "$match", .value = .{ .doc = &.{.{ .key = "k", .value = .{ .int32 = 2 } }} } }} };
|
|
|
|
// Accepted: the two shapes countDocuments() produces.
|
|
try testing.expect(try count_only_pipeline(&reply, &.{group_count}) != null);
|
|
const with_match = try count_only_pipeline(&reply, &.{ match_k, group_count });
|
|
try testing.expect(with_match != null);
|
|
try testing.expectEqual(@as(usize, 1), with_match.?.filter.len);
|
|
try testing.expectEqualStrings("k", with_match.?.filter[0].key);
|
|
|
|
// Rejected: grouping by a field value needs the documents.
|
|
try testing.expect(try count_only_pipeline(&reply, &.{.{ .doc = &.{.{ .key = "$group", .value = .{ .doc = &.{
|
|
.{ .key = "_id", .value = .{ .string = "$k" } },
|
|
.{ .key = "n", .value = .{ .doc = &.{.{ .key = "$sum", .value = .{ .int32 = 1 } }} } },
|
|
} } }} }}) == null);
|
|
|
|
// Rejected: summing a field, not a constant.
|
|
try testing.expect(try count_only_pipeline(&reply, &.{.{ .doc = &.{.{ .key = "$group", .value = .{ .doc = &.{
|
|
.{ .key = "_id", .value = .null },
|
|
.{ .key = "t", .value = .{ .doc = &.{.{ .key = "$sum", .value = .{ .string = "$x" } }} } },
|
|
} } }} }}) == null);
|
|
|
|
// Rejected: an accumulator we do not model at all.
|
|
try testing.expect(try count_only_pipeline(&reply, &.{.{ .doc = &.{.{ .key = "$group", .value = .{ .doc = &.{
|
|
.{ .key = "_id", .value = .null },
|
|
.{ .key = "m", .value = .{ .doc = &.{.{ .key = "$max", .value = .{ .string = "$x" } }} } },
|
|
} } }} }}) == null);
|
|
|
|
// Rejected: any extra stage, since it could reshape the result.
|
|
try testing.expect(try count_only_pipeline(&reply, &.{ match_k, group_count, .{ .doc = &.{.{ .key = "$limit", .value = .{ .int32 = 1 } }} } }) == null);
|
|
// Rejected: a leading stage that is not $match.
|
|
try testing.expect(try count_only_pipeline(&reply, &.{ .{ .doc = &.{.{ .key = "$sort", .value = .{ .doc = &.{.{ .key = "k", .value = .{ .int32 = 1 } }} } }} }, group_count }) == null);
|
|
// Rejected: empty pipeline.
|
|
try testing.expect(try count_only_pipeline(&reply, &.{}) == null);
|
|
}
|
|
|
|
test "a sorted full scan over a multikey index returns each document once" {
|
|
// scan_sorted streams a whole-index read instead of materializing it, which
|
|
// is what keeps a countDocuments({}) from building a list of every offset in
|
|
// the collection. But one document contributes several entries to a multikey
|
|
// index, so walking that index end to end yields it once per array element.
|
|
// The materializing path deduped; a stream cannot, so Plan.full_scan()
|
|
// refuses multikey indexes and this shape keeps materializing.
|
|
//
|
|
// The shape is `find({}).sort({tags: 1})`: no filter, so the only reason to
|
|
// use an index at all is that it supplies the order.
|
|
//
|
|
// Mutation check, and it is worth stating precisely because the obvious
|
|
// version of it does nothing: two independent guards refuse this, so
|
|
// removing either one alone leaves the test green. `index_provides_sort`
|
|
// returns null for a multikey index, and `Plan.full_scan` refuses one
|
|
// again. Remove *both* and each document comes back three times. So this
|
|
// test pins the pair, not either guard -- which is the useful property,
|
|
// since it is the behaviour that matters rather than which check delivers
|
|
// it.
|
|
var threaded: std.Io.Threaded = .init_single_threaded;
|
|
defer threaded.deinit();
|
|
const io = threaded.io();
|
|
var tdb = try TestDb.init(io);
|
|
defer tdb.deinit();
|
|
|
|
try dispatch_insert(&tdb, io, "mk", &.{
|
|
.{ .doc = &.{
|
|
.{ .key = "_id", .value = .{ .int32 = 1 } },
|
|
.{ .key = "tags", .value = .{ .array = &.{ .{ .int32 = 1 }, .{ .int32 = 2 }, .{ .int32 = 3 } } } },
|
|
} },
|
|
.{ .doc = &.{
|
|
.{ .key = "_id", .value = .{ .int32 = 2 } },
|
|
.{ .key = "tags", .value = .{ .array = &.{ .{ .int32 = 4 }, .{ .int32 = 5 }, .{ .int32 = 6 } } } },
|
|
} },
|
|
});
|
|
try dispatch_create_index(&tdb, io, "mk", .{ .doc = &.{
|
|
.{ .key = "key", .value = .{ .doc = &.{.{ .key = "tags", .value = .{ .int32 = 1 } }} } },
|
|
.{ .key = "name", .value = .{ .string = "tags_1" } },
|
|
} });
|
|
|
|
var ctx = tdb.ctx(io);
|
|
var msg = try parse_fake_msg("find", .{ .string = "mk" }, &.{
|
|
.{ .key = "filter", .value = .{ .doc = &.{} } },
|
|
.{ .key = "sort", .value = .{ .doc = &.{.{ .key = "tags", .value = .{ .int32 = 1 } }} } },
|
|
});
|
|
defer msg.deinit();
|
|
var reply = wire.Reply.init(testing.allocator);
|
|
defer reply.deinit();
|
|
try dispatch(&ctx, &msg, &reply);
|
|
try testing.expectEqual(@as(f64, 1.0), bson.get_pair(reply.pairs.items, "ok").?.double);
|
|
const cur = bson.get_pair(reply.pairs.items, "cursor") orelse return error.TestUnexpectedResult;
|
|
const batch = switch (bson.get_pair(cur.doc, "firstBatch") orelse return error.TestUnexpectedResult) {
|
|
.array => |arr| arr,
|
|
else => return error.TestUnexpectedResult,
|
|
};
|
|
// Two documents, not six.
|
|
try testing.expectEqual(@as(usize, 2), batch.len);
|
|
}
|
|
|
|
test "a command with no collection name errors and holds no lock" {
|
|
// Regression for a leaked catalog lock. dispatch resolved the namespace
|
|
// *after* taking the catalog lock, with `orelse return` -- and a plain
|
|
// return runs neither the errdefer nor the explicit unlocks, so the lock
|
|
// was held shared forever. `db.aggregate(...)` sends {aggregate: 1}, whose
|
|
// value is not a string, so it reached exactly that path.
|
|
//
|
|
// Two assertions, because the first alone would have passed before the fix
|
|
// for the wrong reason: the reply must be a real error (it used to be an
|
|
// empty document, which a driver reports as the useless "n/a"), and a
|
|
// subsequent write that has to take the catalog exclusive to create a
|
|
// collection must still complete. The second is the lock check.
|
|
//
|
|
// Mutation check: restore the `orelse return` pair after the lock
|
|
// acquisition and this test hangs on the insert rather than failing -- the
|
|
// suite times out. That is a deadlock, so it cannot be asserted more
|
|
// politely from a single fiber; the e2e suite covers the same sequence
|
|
// where a hang surfaces as a client timeout instead.
|
|
var threaded: std.Io.Threaded = .init_single_threaded;
|
|
defer threaded.deinit();
|
|
const io = threaded.io();
|
|
var tdb = try TestDb.init(io);
|
|
defer tdb.deinit();
|
|
|
|
{
|
|
var ctx = tdb.ctx(io);
|
|
// {aggregate: 1} -- a database-level aggregate, no collection named.
|
|
var msg = try parse_fake_msg("aggregate", .{ .int32 = 1 }, &.{
|
|
.{ .key = "pipeline", .value = .{ .array = &.{} } },
|
|
.{ .key = "cursor", .value = .{ .doc = &.{} } },
|
|
});
|
|
defer msg.deinit();
|
|
var reply = wire.Reply.init(testing.allocator);
|
|
defer reply.deinit();
|
|
try dispatch(&ctx, &msg, &reply);
|
|
// A proper error, not an empty reply.
|
|
try testing.expectEqual(@as(f64, 0.0), bson.get_pair(reply.pairs.items, "ok").?.double);
|
|
try testing.expectEqual(
|
|
@as(i32, @intFromEnum(ErrorCode.bad_value)),
|
|
bson.get_pair(reply.pairs.items, "code").?.int32,
|
|
);
|
|
}
|
|
|
|
// The lock assertion: this insert creates a collection, which upgrades the
|
|
// catalog lock to exclusive. With the lock leaked it never returns.
|
|
try dispatch_insert(&tdb, io, "after", &.{
|
|
.{ .doc = &.{.{ .key = "_id", .value = .{ .int32 = 1 } }} },
|
|
});
|
|
}
|
|
|
|
test "compare-equal _id encodings collide under the unique _id_ index" {
|
|
// _id uniqueness moved from a docs-map probe keyed on serialize_value to
|
|
// the _id_ B+tree, keyed on the canonical bson.encode_key (PLAN A3/A4).
|
|
// That changes observable behavior, in MongoDB's direction: int32 1,
|
|
// int64 1 and double 1.0 are one _id, not three.
|
|
//
|
|
// Mutation check: setting `unique = false` back on id_index makes these
|
|
// collisions vanish (this test and "insert counts only successful inserts"
|
|
// both go red).
|
|
//
|
|
// The other half of the change is covered elsewhere, which is worth
|
|
// knowing so nobody re-checks it here: passing `id_key` instead of null as
|
|
// check_unique's exclude on the insert path is caught by "insert counts
|
|
// only successful inserts", not by this test. Two documents with the
|
|
// *same* encoding share an id_key, so exclude-self hides the collision;
|
|
// int32 1 and int64 1 have different id_keys, so this test survives that
|
|
// mutation. The two tests are complementary, not redundant.
|
|
var threaded: std.Io.Threaded = .init_single_threaded;
|
|
defer threaded.deinit();
|
|
const io = threaded.io();
|
|
var tdb = try TestDb.init(io);
|
|
defer tdb.deinit();
|
|
|
|
try dispatch_insert(&tdb, io, "ids", &.{
|
|
.{ .doc = &.{.{ .key = "_id", .value = .{ .int32 = 1 } }} },
|
|
});
|
|
|
|
// Each of these is the same _id as int32 1.
|
|
for ([_]bson.Value{
|
|
.{ .doc = &.{.{ .key = "_id", .value = .{ .int64 = 1 } }} },
|
|
.{ .doc = &.{.{ .key = "_id", .value = .{ .double = 1.0 } }} },
|
|
}) |dup| {
|
|
var ctx = tdb.ctx(io);
|
|
const one = [_]bson.Value{dup};
|
|
var msg = try parse_fake_msg("insert", .{ .string = "ids" }, &.{
|
|
.{ .key = "documents", .value = .{ .array = &one } },
|
|
});
|
|
defer msg.deinit();
|
|
var reply = wire.Reply.init(testing.allocator);
|
|
defer reply.deinit();
|
|
try dispatch(&ctx, &msg, &reply);
|
|
const we = bson.get_pair(reply.pairs.items, "writeErrors") orelse return error.TestUnexpectedResult;
|
|
const first = we.array[0];
|
|
try testing.expectEqual(@as(i32, 11000), bson.get_pair(first.doc, "code").?.int32);
|
|
// The index named in the message is the one MongoDB names.
|
|
const errmsg = bson.get_pair(first.doc, "errmsg").?.string;
|
|
try testing.expect(std.mem.indexOf(u8, errmsg, "_id_") != null);
|
|
}
|
|
|
|
// A different rank is a different _id: "1" is not 1.
|
|
try dispatch_insert(&tdb, io, "ids", &.{
|
|
.{ .doc = &.{.{ .key = "_id", .value = .{ .string = "1" } }} },
|
|
});
|
|
|
|
// And the collision is not merely rejection: a lookup by any of the
|
|
// equivalent encodings finds the one stored document.
|
|
for ([_]bson.Value{ .{ .int32 = 1 }, .{ .int64 = 1 }, .{ .double = 1.0 } }) |probe| {
|
|
var ids: std.ArrayListUnmanaged([]u8) = .empty;
|
|
defer {
|
|
for (ids.items) |id| testing.allocator.free(id);
|
|
ids.deinit(testing.allocator);
|
|
}
|
|
try dispatch_find_ids(&tdb, io, "ids", &.{.{ .key = "_id", .value = probe }}, &ids);
|
|
try testing.expectEqual(@as(usize, 1), ids.items.len);
|
|
}
|
|
}
|
|
|
|
test "indexed queries are equivalent to scans over a mixed corpus" {
|
|
var threaded: std.Io.Threaded = .init_single_threaded;
|
|
defer threaded.deinit();
|
|
const io = threaded.io();
|
|
var tdb = try TestDb.init(io);
|
|
defer tdb.deinit();
|
|
|
|
// Corpus deliberately mixes numeric _id encodings, arrays, nested docs,
|
|
// missing fields, explicit nulls, and duplicate values.
|
|
//
|
|
// It used to carry int32 1 and int64 1 as two documents, to exercise the
|
|
// old docs-map fast path (they compare equal but serialize differently).
|
|
// _id_ is a unique index now, keyed on the canonical bson.encode_key, so
|
|
// those two *are* the same _id and the second is rejected -- which is
|
|
// MongoDB's behavior. The int64 encoding still appears below, on a
|
|
// distinct value; the collision itself is asserted in "compare-equal _id
|
|
// encodings collide under the unique _id_ index".
|
|
const corpus = [_]bson.Value{
|
|
.{ .doc = &.{
|
|
.{ .key = "_id", .value = .{ .int32 = 1 } },
|
|
.{ .key = "a", .value = .{ .int32 = 10 } },
|
|
.{ .key = "b", .value = .{ .string = "x" } },
|
|
.{ .key = "tags", .value = .{ .array = &.{ .{ .string = "a" }, .{ .string = "b" } } } },
|
|
} },
|
|
.{ .doc = &.{
|
|
.{ .key = "_id", .value = .{ .int64 = 2 } },
|
|
.{ .key = "a", .value = .{ .int32 = 20 } },
|
|
.{ .key = "b", .value = .{ .string = "y" } },
|
|
.{ .key = "tags", .value = .{ .array = &.{} } },
|
|
} },
|
|
.{ .doc = &.{
|
|
.{ .key = "_id", .value = .{ .int32 = 3 } },
|
|
.{ .key = "a", .value = .{ .double = 30.0 } },
|
|
.{ .key = "b", .value = .null },
|
|
.{ .key = "tags", .value = .{ .array = &.{.{ .string = "a" }} } },
|
|
} },
|
|
.{ .doc = &.{
|
|
.{ .key = "_id", .value = .{ .string = "s4" } },
|
|
.{ .key = "a", .value = .{ .int32 = 10 } },
|
|
.{ .key = "b", .value = .{ .string = "z" } },
|
|
.{ .key = "tags", .value = .{ .array = &.{ .{ .int32 = 10 }, .{ .int32 = 20 } } } },
|
|
} },
|
|
.{ .doc = &.{
|
|
.{ .key = "_id", .value = .{ .int32 = 5 } },
|
|
.{ .key = "a", .value = .null },
|
|
.{ .key = "b", .value = .{ .string = "x" } },
|
|
} },
|
|
.{ .doc = &.{
|
|
.{ .key = "_id", .value = .{ .int32 = 6 } },
|
|
.{ .key = "a", .value = .{ .doc = &.{.{ .key = "n", .value = .{ .int32 = 1 } }} } },
|
|
.{ .key = "b", .value = .{ .string = "w" } },
|
|
} },
|
|
.{ .doc = &.{
|
|
.{ .key = "_id", .value = .{ .int32 = 7 } },
|
|
.{ .key = "a", .value = .{ .array = &.{ .{ .int32 = 1 }, .{ .int32 = 2 } } } },
|
|
.{ .key = "b", .value = .{ .string = "q" } },
|
|
} },
|
|
.{ .doc = &.{
|
|
.{ .key = "_id", .value = .{ .int32 = 8 } },
|
|
.{ .key = "c", .value = .{ .int32 = 99 } },
|
|
} },
|
|
.{ .doc = &.{
|
|
.{ .key = "_id", .value = .{ .int32 = 9 } },
|
|
.{ .key = "a", .value = .{ .int32 = 40 } },
|
|
.{ .key = "b", .value = .{ .array = &.{ .{ .int32 = 5 }, .{ .int32 = 6 } } } },
|
|
} },
|
|
};
|
|
try dispatch_insert(&tdb, io, "eq", &corpus);
|
|
|
|
const filters = [_]struct { pairs: []const bson.Pair }{
|
|
.{ .pairs = &.{} },
|
|
.{ .pairs = &.{.{ .key = "a", .value = .{ .int32 = 10 } }} },
|
|
.{ .pairs = &.{.{ .key = "a", .value = .{ .doc = &.{.{ .key = "$eq", .value = .{ .int32 = 20 } }} } }} },
|
|
.{ .pairs = &.{.{ .key = "a", .value = .{ .doc = &.{.{ .key = "$gt", .value = .{ .int32 = 15 } }} } }} },
|
|
.{ .pairs = &.{.{ .key = "a", .value = .{ .doc = &.{.{ .key = "$gte", .value = .{ .int32 = 20 } }} } }} },
|
|
.{ .pairs = &.{.{ .key = "a", .value = .{ .doc = &.{.{ .key = "$lt", .value = .{ .int32 = 30 } }} } }} },
|
|
.{ .pairs = &.{.{ .key = "a", .value = .{ .doc = &.{.{ .key = "$lte", .value = .{ .int32 = 30 } }} } }} },
|
|
.{ .pairs = &.{.{ .key = "a", .value = .{ .doc = &.{.{ .key = "$in", .value = .{ .array = &.{ .{ .int32 = 10 }, .{ .int32 = 30 } } } }} } }} },
|
|
.{ .pairs = &.{.{ .key = "a", .value = .{ .doc = &.{ .{ .key = "$gt", .value = .{ .int32 = 5 } }, .{ .key = "$lt", .value = .{ .int32 = 25 } } } } }} },
|
|
.{ .pairs = &.{.{ .key = "b", .value = .{ .string = "x" } }} },
|
|
.{ .pairs = &.{.{ .key = "b", .value = .null }} },
|
|
.{ .pairs = &.{ .{ .key = "a", .value = .{ .int32 = 10 } }, .{ .key = "b", .value = .{ .string = "x" } } } },
|
|
.{ .pairs = &.{.{ .key = "tags", .value = .{ .string = "a" } }} },
|
|
.{ .pairs = &.{.{ .key = "tags", .value = .{ .array = &.{ .{ .string = "a" }, .{ .string = "b" } } } }} },
|
|
.{ .pairs = &.{.{ .key = "tags", .value = .{ .doc = &.{.{ .key = "$all", .value = .{ .array = &.{.{ .string = "a" }} } }} } }} },
|
|
.{ .pairs = &.{.{ .key = "a", .value = .{ .doc = &.{.{ .key = "$regex", .value = .{ .string = "^1" } }} } }} },
|
|
.{ .pairs = &.{.{ .key = "c", .value = .{ .doc = &.{.{ .key = "$exists", .value = .{ .bool = false } }} } }} },
|
|
.{ .pairs = &.{.{ .key = "a", .value = .null }} },
|
|
.{ .pairs = &.{.{ .key = "_id", .value = .{ .int32 = 1 } }} },
|
|
.{ .pairs = &.{.{ .key = "_id", .value = .{ .string = "s4" } }} },
|
|
.{ .pairs = &.{.{ .key = "_id", .value = .{ .doc = &.{.{ .key = "$in", .value = .{ .array = &.{ .{ .int32 = 1 }, .{ .string = "s4" } } } }} } }} },
|
|
.{ .pairs = &.{.{ .key = "$and", .value = .{ .array = &.{
|
|
.{ .doc = &.{.{ .key = "a", .value = .{ .doc = &.{.{ .key = "$gte", .value = .{ .int32 = 10 } }} } }} },
|
|
.{ .doc = &.{.{ .key = "a", .value = .{ .doc = &.{.{ .key = "$lte", .value = .{ .int32 = 30 } }} } }} },
|
|
} } }} },
|
|
.{ .pairs = &.{.{ .key = "a", .value = .{ .array = &.{ .{ .int32 = 1 }, .{ .int32 = 2 } } } }} },
|
|
.{ .pairs = &.{.{ .key = "a", .value = .{ .doc = &.{.{ .key = "$gt", .value = .{ .int32 = 100 } }} } }} },
|
|
.{ .pairs = &.{.{ .key = "$or", .value = .{ .array = &.{
|
|
.{ .doc = &.{.{ .key = "a", .value = .{ .int32 = 10 } }} },
|
|
.{ .doc = &.{.{ .key = "b", .value = .{ .string = "y" } }} },
|
|
} } }} },
|
|
.{ .pairs = &.{.{ .key = "a", .value = .{ .doc = &.{.{ .key = "$gt", .value = .null }} } }} },
|
|
.{ .pairs = &.{ .{ .key = "a", .value = .{ .int32 = 10 } }, .{ .key = "tags", .value = .{ .int32 = 10 } } } },
|
|
};
|
|
|
|
// First with a compound (a, b) index, then after dropping it — and the
|
|
// _id fast path is exercised by the _id filters in both runs.
|
|
try dispatch_create_index(&tdb, io, "eq", .{ .doc = &.{
|
|
.{ .key = "key", .value = .{ .doc = &.{
|
|
.{ .key = "a", .value = .{ .int32 = 1 } },
|
|
.{ .key = "b", .value = .{ .int32 = 1 } },
|
|
} } },
|
|
.{ .key = "name", .value = .{ .string = "a_1_b_1" } },
|
|
} });
|
|
|
|
// Run every filter through the index (the (a, b) plan plus the _id fast
|
|
// path), then drop the index and require identical results from the
|
|
// scan. The corpus's int32/int64 _id pair exercises the fast-path guard
|
|
// (numbers fall back to a scan in both runs).
|
|
var indexed_results: std.ArrayListUnmanaged(std.ArrayListUnmanaged([]u8)) = .empty;
|
|
defer {
|
|
for (indexed_results.items) |*l| free_id_list(testing.allocator, l);
|
|
indexed_results.deinit(testing.allocator);
|
|
}
|
|
for (filters) |f| {
|
|
var list: std.ArrayListUnmanaged([]u8) = .empty;
|
|
errdefer free_id_list(testing.allocator, &list);
|
|
try dispatch_find_ids(&tdb, io, "eq", f.pairs, &list);
|
|
try indexed_results.append(testing.allocator, list);
|
|
}
|
|
{
|
|
var ctx = tdb.ctx(io);
|
|
var msg = try parse_fake_msg("dropIndexes", .{ .string = "eq" }, &.{
|
|
.{ .key = "index", .value = .{ .string = "*" } },
|
|
});
|
|
defer msg.deinit();
|
|
var reply = wire.Reply.init(testing.allocator);
|
|
defer reply.deinit();
|
|
try dispatch(&ctx, &msg, &reply);
|
|
}
|
|
var scanned: std.ArrayListUnmanaged([]u8) = .empty;
|
|
defer free_id_list(testing.allocator, &scanned);
|
|
for (filters, 0..) |f, fi| {
|
|
clear_id_list(testing.allocator, &scanned);
|
|
try dispatch_find_ids(&tdb, io, "eq", f.pairs, &scanned);
|
|
const indexed = indexed_results.items[fi];
|
|
try testing.expectEqual(scanned.items.len, indexed.items.len);
|
|
for (scanned.items, indexed.items) |a, b| try testing.expectEqualSlices(u8, a, b);
|
|
}
|
|
|
|
// Same corpus with a sparse (a, b) index: the sparse/null bail keeps
|
|
// {a: null} and null-range filters correct.
|
|
try dispatch_create_index(&tdb, io, "eq", .{ .doc = &.{
|
|
.{ .key = "key", .value = .{ .doc = &.{
|
|
.{ .key = "a", .value = .{ .int32 = 1 } },
|
|
.{ .key = "b", .value = .{ .int32 = 1 } },
|
|
} } },
|
|
.{ .key = "name", .value = .{ .string = "a_1_b_1_sparse" } },
|
|
.{ .key = "sparse", .value = .{ .bool = true } },
|
|
} });
|
|
for (indexed_results.items) |*l| free_id_list(testing.allocator, l);
|
|
indexed_results.clearRetainingCapacity();
|
|
for (filters) |f| {
|
|
var list: std.ArrayListUnmanaged([]u8) = .empty;
|
|
errdefer free_id_list(testing.allocator, &list);
|
|
try dispatch_find_ids(&tdb, io, "eq", f.pairs, &list);
|
|
try indexed_results.append(testing.allocator, list);
|
|
}
|
|
{
|
|
var ctx = tdb.ctx(io);
|
|
var msg = try parse_fake_msg("dropIndexes", .{ .string = "eq" }, &.{
|
|
.{ .key = "index", .value = .{ .string = "*" } },
|
|
});
|
|
defer msg.deinit();
|
|
var reply = wire.Reply.init(testing.allocator);
|
|
defer reply.deinit();
|
|
try dispatch(&ctx, &msg, &reply);
|
|
}
|
|
for (filters, 0..) |f, fi| {
|
|
clear_id_list(testing.allocator, &scanned);
|
|
try dispatch_find_ids(&tdb, io, "eq", f.pairs, &scanned);
|
|
const indexed = indexed_results.items[fi];
|
|
try testing.expectEqual(scanned.items.len, indexed.items.len);
|
|
for (scanned.items, indexed.items) |a, b| try testing.expectEqualSlices(u8, a, b);
|
|
}
|
|
}
|
|
|
|
/// Insert `n` documents `{_id: 1..n, a: i % 5, pad}` in one batch.
|
|
///
|
|
/// Goes through `dispatch_insert` rather than dispatching itself, because that
|
|
/// helper checks `ok`, `writeErrors` *and* `n` -- and its comment records what
|
|
/// omitting those cost last time: a corpus silently lost a document and every
|
|
/// test over it still passed. A cursor test whose premise is "60 documents" must
|
|
/// not be able to become "0 documents" quietly.
|
|
const seed_pad = "0123456789012345678901234567890123456789";
|
|
|
|
fn seed_docs(tdb: *TestDb, io: std.Io, coll: []const u8, n: i32) !void {
|
|
const arena = testing.allocator;
|
|
const docs = try arena.alloc(bson.Value, @intCast(n));
|
|
defer {
|
|
for (docs) |d| arena.free(d.doc);
|
|
arena.free(docs);
|
|
}
|
|
for (docs, 0..) |*d, i| {
|
|
const pairs = try arena.alloc(bson.Pair, 3);
|
|
pairs[0] = .{ .key = "_id", .value = .{ .int32 = @intCast(i + 1) } };
|
|
pairs[1] = .{ .key = "a", .value = .{ .int32 = @intCast(@mod(i + 1, 5)) } };
|
|
pairs[2] = .{ .key = "pad", .value = .{ .string = seed_pad } };
|
|
d.* = .{ .doc = pairs };
|
|
}
|
|
try dispatch_insert(tdb, io, coll, docs);
|
|
}
|
|
|
|
/// Run `find` and return (cursor id, first-batch length).
|
|
fn dispatch_find(
|
|
ctx: *Context,
|
|
coll: []const u8,
|
|
filter: []const bson.Pair,
|
|
batch_size: i32,
|
|
) !struct { id: i64, n: usize } {
|
|
var reply = wire.Reply.init(testing.allocator);
|
|
defer reply.deinit();
|
|
var msg = try parse_fake_msg("find", .{ .string = coll }, &.{
|
|
.{ .key = "filter", .value = .{ .doc = filter } },
|
|
.{ .key = "batchSize", .value = .{ .int32 = batch_size } },
|
|
});
|
|
defer msg.deinit();
|
|
try dispatch(ctx, &msg, &reply);
|
|
const cur = bson.get_pair(reply.pairs.items, "cursor") orelse return error.TestUnexpectedResult;
|
|
const id = bson.get_pair(cur.doc, "id").?.int64;
|
|
const batch = bson.get_pair(cur.doc, "firstBatch").?.array;
|
|
return .{ .id = id, .n = batch.len };
|
|
}
|
|
|
|
/// Run `getMore` and return the error code, or 0 on success.
|
|
fn dispatch_get_more(ctx: *Context, coll: []const u8, id: i64) !i32 {
|
|
var reply = wire.Reply.init(testing.allocator);
|
|
defer reply.deinit();
|
|
var msg = try parse_fake_msg("getMore", .{ .int64 = id }, &.{
|
|
.{ .key = "collection", .value = .{ .string = coll } },
|
|
.{ .key = "batchSize", .value = .{ .int32 = 5 } },
|
|
});
|
|
defer msg.deinit();
|
|
try dispatch(ctx, &msg, &reply);
|
|
if (bson.get_pair(reply.pairs.items, "code")) |c| return c.int32;
|
|
return 0;
|
|
}
|
|
|
|
test "a rebuild kills an offsets cursor and spares a streaming one" {
|
|
// This is also the test that proves the streaming source is *selected*: a
|
|
// whole-index walk and an indexed-predicate scan are given different sources,
|
|
// and a rebuild is exactly what tells them apart. If `find({})` quietly fell
|
|
// back to materializing offsets, both would die here.
|
|
//
|
|
// A rebuild is triggered directly rather than through churn, because whether
|
|
// churn crosses the compaction threshold is not something a test should have
|
|
// to guess at.
|
|
var threaded: std.Io.Threaded = .init_single_threaded;
|
|
defer threaded.deinit();
|
|
const io = threaded.io();
|
|
var tdb = try TestDb.init(io);
|
|
defer tdb.deinit();
|
|
var ctx = tdb.ctx(io);
|
|
|
|
try seed_docs(&tdb, io, "c", 60);
|
|
|
|
// A whole-index walk: no predicate, no sort.
|
|
const walk = try dispatch_find(&ctx, "c", &.{}, 5);
|
|
try testing.expectEqual(@as(usize, 5), walk.n);
|
|
try testing.expect(walk.id != 0);
|
|
// A narrowed plan over the _id_ index, which materializes its candidates.
|
|
const narrowed = try dispatch_find(&ctx, "c", &.{
|
|
.{ .key = "_id", .value = .{ .doc = &.{.{ .key = "$gte", .value = .{ .int32 = 1 } }} } },
|
|
}, 5);
|
|
try testing.expect(narrowed.id != 0);
|
|
|
|
// Both still work before the rebuild, so the difference below is the rebuild.
|
|
try testing.expectEqual(@as(i32, 0), try dispatch_get_more(&ctx, "c", walk.id));
|
|
try testing.expectEqual(@as(i32, 0), try dispatch_get_more(&ctx, "c", narrowed.id));
|
|
|
|
try ctx.engine.compact();
|
|
|
|
// The stream remembers key bytes, which a repack does not change.
|
|
try testing.expectEqual(@as(i32, 0), try dispatch_get_more(&ctx, "c", walk.id));
|
|
// The offsets name bytes that have moved, so continuing would be a wrong
|
|
// answer; QueryPlanKilled says so instead.
|
|
try testing.expectEqual(
|
|
@intFromEnum(ErrorCode.query_plan_killed),
|
|
try dispatch_get_more(&ctx, "c", narrowed.id),
|
|
);
|
|
// And a second getMore on the killed cursor no longer knows it at all.
|
|
try testing.expectEqual(
|
|
@intFromEnum(ErrorCode.cursor_not_found),
|
|
try dispatch_get_more(&ctx, "c", narrowed.id),
|
|
);
|
|
|
|
// The surviving stream drains to exactly the 60 documents, once each.
|
|
// Three batches of 5 are already out: the first batch, the getMore before the
|
|
// rebuild, and the one just after it.
|
|
var seen: u32 = 5 + 5 + 5;
|
|
var id = walk.id;
|
|
var rounds: u32 = 0;
|
|
while (rounds < 100) : (rounds += 1) {
|
|
var reply = wire.Reply.init(testing.allocator);
|
|
defer reply.deinit();
|
|
var msg = try parse_fake_msg("getMore", .{ .int64 = id }, &.{
|
|
.{ .key = "collection", .value = .{ .string = "c" } },
|
|
.{ .key = "batchSize", .value = .{ .int32 = 7 } },
|
|
});
|
|
defer msg.deinit();
|
|
try dispatch(&ctx, &msg, &reply);
|
|
const cur = bson.get_pair(reply.pairs.items, "cursor").?;
|
|
seen += @intCast(bson.get_pair(cur.doc, "nextBatch").?.array.len);
|
|
id = bson.get_pair(cur.doc, "id").?.int64;
|
|
if (id == 0) break;
|
|
}
|
|
try testing.expectEqual(@as(i64, 0), id);
|
|
try testing.expectEqual(@as(u32, 60), seen);
|
|
}
|