Encodes a Value so that std.mem.order over the bytes reproduces
bson.compare exactly. This is the foundation for the encoded-key index:
it lets an index binary-search, range-scan and eventually be stored as
raw bytes, instead of carrying Value trees whose every comparison chases
pointers into a different document's arena.
Layout is [rank + 1] then a self-delimiting payload; the +1 keeps 0x00
out of the tag space so it can terminate variable-length payloads.
The parts that are easy to get wrong, and why they are the way they are:
Numbers encode the f128 that compare already widens int32, int64 and
double to -- exactly, for all three. So int32 1, int64 1 and double 1.0
produce identical bytes, which is the cross-type equality that numeric
index lookups need, and is precisely what value_fast_path_safe exists
today to work around. Negatives are bit-inverted and positives get the
sign bit set, making the IEEE order lexicographic. -0.0 normalizes to
+0.0 (they compare equal) and every NaN encodes as all-ones (compare
makes NaN greatest and all NaNs equal).
Byte strings escape 0x00 as 00 FF and terminate with 00 00. A BSON
string may contain NUL, so a bare terminator would be ambiguous;
escaping fixes ordering at the same time, since a real NUL then sorts
above the terminator and any byte >= 01 does too. "Shorter is less"
falls out to match std.mem.order, which also gives documents and arrays
their length tie-break for free.
Binary length-prefixes because compare_binary orders by length first,
but opaque_val escapes instead: compare ignores its kind and orders the
data lexicographically, not by length.
Correctness rests entirely on the order equivalence, so it is checked
exhaustively rather than by example: every ordered pair of a corpus
spanning all fifteen ranks and their boundaries (numeric cross-type and
sign, NaN, both zeros, infinities, embedded NULs, prefix relationships,
empty and nested documents and arrays, binary subtypes) is compared both
ways. A second test concatenates two-column keys and checks they
reproduce component-wise order, which is what makes compound keys and
prefix search sound. Verified both fail when escaping is dropped, when
-0.0 is not normalized, when the binary length prefix is wrong, and when
NaN stops being greatest.
Nothing uses the encoding yet; the index still holds Value keys.
Aleksey Shakhmatov
ea1f0f09cf