index: hold encoded byte keys instead of Value slices
Entry.key becomes the order-preserving byte encoding of the indexed
values, concatenated column by column, instead of a slice of Values.
Comparing two entries is now a memcmp.
The old representation allocated one Value slice per entry, and each
Value in it pointed into a different document's arena -- so a binary
search over the entry array was a chain of pointer chases across the
heap, and every comparison walked the key component by component
dispatching on BSON type. Byte keys make the comparison contiguous and
type-free, and the key no longer aliases the document at all.
createIndex over 65,536 documents:
{k: 1} 56ms -> 44ms
{s: 1} unique 53ms -> 31ms
{p: 1, k: -1} 54ms -> 37ms
(both already down from ~650ms before the bulk build)
The search API still takes Values and encodes at the call site: lookups
happen per query, not per document, so there is nothing to gain from
pushing the encoding out to callers, and Plan keeps its current shape.
Prefix search compares raw byte prefixes, which is sound because every
column encoding is self-delimiting -- a prefix of an encoded key is
exactly the encoding of its leading columns. For the same reason a
complete column encoding can never be a proper prefix of another, so
finish_bulk's duplicate test is now a plain byte equality.
The TTL sweep read entry keys as Values to find datetimes. It now uses
bson.encoded_leading_datetime, which checks the column's tag and decodes
eight bytes rather than the whole key. Still a linear walk for the reason
the existing comment gives.
Key direction is deliberately still not applied to the encoding.
Complementing descending columns would let a sort read the array
forwards, but nothing exploits that yet, and doing it now would change
the array's order for no gain. It belongs with the sort-aware planner.
remove_id is still a linear scan and insertion still memmoves the tail:
those are the tree's job, not this change's.
Verified: 77 unit tests under ReleaseFast and ReleaseSafe, e2e
29/16/17/3/2, the crash pair, e2e6 72/72, and the randomized
lookup_range test that checks bounds against a brute-force filter.
This commit is contained in:
15
src/bson.zig
15
src/bson.zig
@@ -617,8 +617,9 @@ pub fn encode_key(v: Value, gpa: std.mem.Allocator, out: *std.ArrayListUnmanaged
|
||||
// Flip the sign bit so the two's-complement order becomes unsigned
|
||||
// byte order.
|
||||
.datetime => |ms| {
|
||||
std.debug.assert(rank(v) + 1 == encoded_datetime_tag);
|
||||
var buf: [8]u8 = undefined;
|
||||
std.mem.writeInt(u64, &buf, @as(u64, @bitCast(ms)) ^ (1 << 63), .big);
|
||||
std.mem.writeInt(u64, &buf, @as(u64, @bitCast(ms)) ^ (@as(u64, 1) << 63), .big);
|
||||
try out.appendSlice(gpa, &buf);
|
||||
},
|
||||
.timestamp => |ts| {
|
||||
@@ -638,6 +639,18 @@ pub fn encode_key(v: Value, gpa: std.mem.Allocator, out: *std.ArrayListUnmanaged
|
||||
}
|
||||
}
|
||||
|
||||
/// The tag `encode_key` writes for a datetime.
|
||||
pub const encoded_datetime_tag: u8 = 9 + 1;
|
||||
|
||||
/// The datetime in an encoded key's first column, or null when that column
|
||||
/// holds anything else. Lets a TTL sweep read entry keys without decoding
|
||||
/// them back into Values.
|
||||
pub fn encoded_leading_datetime(key: []const u8) ?i64 {
|
||||
if (key.len < 1 + 8 or key[0] != encoded_datetime_tag) return null;
|
||||
const biased = std.mem.readInt(u64, key[1..9], .big);
|
||||
return @bitCast(biased ^ (@as(u64, 1) << 63));
|
||||
}
|
||||
|
||||
/// Byte string, terminated so it stays self-delimiting, with the terminator
|
||||
/// ordering below any content.
|
||||
///
|
||||
|
||||
Reference in New Issue
Block a user