dev/MultiforaDB
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

results: the M0 gates, measured

Browse Source 
PLAN D7's six items, with the numbers and the command that reproduces each in
tests/e2e/results/m0-gates.txt. Unit tests green in both optimize modes, the
whole e2e matrix green, the spec scorecard byte-identical at 131/161/195, and
the large smoke run at the scale D7.3 asked for:

  21.47 GB collection (1,310,720 x 16 KiB)
  data file                 21.75 GB      (+1.3% over the documents)
  log after the load        2.5 MB        (checkpoints reclaim it)
  kill -9 then reopen       0.5 s         (0.5 s at 4 GB too -- flat)
  RSS after reopen          237 MB        (1.1% of the data)
  count after restart       1,310,720     last document byte-intact
  acked writes after kill   200/200

That is the milestone's claim, measured: an open costs the working set rather
than the size of the database. Before M0 the same measurement was 523 MB
resident for a 512 MB database, because recovering each document's `_id` meant
reading every document at open.

Two gates need reading rather than a tick, and m0-gates.txt says so where a
reader would otherwise take a tick for granted.

The churn gate settles at 1.65x live data (delete-heavy) to 2.47x
(update-heavy), flat, above the ~1.3x amendment A2 hoped for. Rebuild-only
reclamation cannot reach that: it needs a whole second copy of the live data
before the first can be freed. The gate existed to decide whether doc-level
free lists are needed after M0, and that is the answer.

Benchmark parity holds for every read and latency row inside the run-to-run
spread, and bulk insert regresses 24% (732 -> 555 MB/s), reproducibly across
three runs. Risk 1 as written: document bytes now reach the disk uncompressed
on top of the LZ4 log. createIndex improves 62% from the same change.

Three measurement bugs fixed while running the gates, because each would have
put a false number in the README:

  - `compare-run.sh` measured "db on disk" as `du` of the log alone against
    `du` of mongod's whole dbpath. It reported 20 MB for a 1 GB collection --
    the documents had moved to <db>.data. Honest figure, measured: 914 MB of
    allocated blocks against mongod's compressed 85 MB.
  - `big.js` counted "compaction events" as "the log shrank", which is a
    *checkpoint* now. It claimed 12 compaction rewrites during a pure insert
    load, which has no garbage to compact.
  - `big.js` labelled peak RSS "in-memory engine: docs live in RAM" and its
    summary said the collection was held "fully in RAM". Both were true of the
    engine this milestone replaced.

README: the storage section described an all-in-RAM engine; the comparison
table mixed one old run's body with three new rows; and `findOne({_id})` was
documented as a full scan for integer ids, which the ordered `_id_` index made
false (2 ms against 55 s for a scan of the same 21.5 GB collection). The table
is now best-of-three for both servers, with the measured variance stated, since
two runs of the same binary moved the sub-10 ms rows by 27-51%.
This commit is contained in:
Alexey Shakhmatov
2026-08-03 23:09:43 +03:00
parent 4b70ce6da9
commit 504179acd1
9 changed files with 505 additions and 87 deletions
Download Patch File Download Diff File Expand all files Collapse all files

80
PLAN.md
View File

@@ -532,13 +532,79 @@ discipline lives, and `ls`/`du` stay honest for D6.6's backup story.
| 16 | drop the docs hashmap | original step 7 |
| 17 | gates | D7's six items; churn gate per amended D6.2 |
M0 is done when D7's six items pass. Two harness fixes are prerequisites for
the gate rather than the work: `bench-run.sh` copies its report over
`bench-latest.txt` unconditionally, including after a run that only warned,
so the baseline being defended can be clobbered; and the four B+tree dev
harnesses (`spill.zig`, `spill2.zig`, `stress.zig`, `fuzz_split.zig`) are in
no build step, so `zig build test` will not notice an API break in the one
place that fuzzes splits and >1 KB keys.
All seventeen commits are landed. Both harness fixes that were prerequisites
for the gate rather than the work are in: `bench-run.sh` no longer copies its
report over `bench-latest.txt` after a degraded run (it used to, so the
baseline being defended could be clobbered), and the four B+tree dev harnesses
(`spill.zig`, `spill2.zig`, `stress.zig`, `fuzz_split.zig`) now have a
`zig build fuzz` step — they were in no build step, so `zig build test` could
not notice an API break in the one place that fuzzes splits and >1 KB keys.
### The gate result
Measured numbers, and the command that reproduces each, are in
`tests/e2e/results/m0-gates.txt`. In short:
| D7 | gate | outcome |
|---|---|---|
| 1 | unit tests RF + RS | pass, 122/122 both |
| 2 | e2e matrix unchanged | pass, every suite |
| 3 | 20-30 GB smoke, RSS ≈ working set, fast reopen | pass at 21.5 GB |
| 4 | churn gate | measured, bounded, above the hoped-for 1.3x |
| 5 | benchmark parity | pass except bulk insert, -24% |
| 6 | spec runner + scorecard | pass, scorecard unchanged |
Two of those need reading rather than a tick.
**D7.4** hoped for ~1.3x of live data and settles at 1.65x (delete-heavy) to
2.47x (update-heavy), flat in both cases. Before the gate's own findings were
fixed it was 4.1x and *climbing linearly* — nothing was being reclaimed at
all. The gate's stated purpose (amendment A2) was to decide whether doc-level
free lists are needed after M0, and the answer is yes, in M1: a rebuild needs
a whole second copy of the live data before the first can be freed, so
rebuild-only reclamation cannot reach 1.3x however it is tuned. What M0 owed
was a bound, and there is one.
**D7.5** says "no phase8 row regresses". Bulk insert throughput regresses
24%, reproducibly (732 → 555 MB/s). That is risk 1 as written: document bytes
now reach the disk uncompressed in the data file on top of the LZ4 log, and
that writeback bandwidth is new. Every other row is inside this machine's
run-to-run spread, which two consecutive runs of the same binary showed to be
27-51% on the sub-10 ms rows — so the gate is met for the read and latency
rows and not for bulk load. `createIndex` improves 62%, also reproducibly,
from the same change.
### What the gates found
Five bugs, none of which any unit test or e2e suite had reached, and four of
which need either a long-running workload or a second concurrent writer to
appear at all:
1. The compaction trigger had been dead since commit 14 — it gated on
`log.data_bytes`, and truncating the log at every checkpoint zeroes that.
2. `page_mut_cow` asked `p >= stable_pages`, which is false for a recycled
page, so every write to one copied and freed it again.
3. First fit let copy-on-write's one-page requests shave the extents the doc
slab needs, so the free list drained every generation.
4. A rebuild's freed space took two unrelated checkpoints to become reusable,
so the next rebuild grew the file instead of reusing it.
5. `reserve_pages`' promise was a single counter on the pager, and two upserts
on different collections release it independently — so the first to finish
revoked the second's promise mid-write. This aborted the server at four
concurrent clients on the first concurrent benchmark since the data file
landed. Risk 3, whose mitigation (private pre-allocated runs) was listed in
this document and never built.
The lesson worth carrying into M1 is the shape of 1-4: every one is a
*reclamation* bug, invisible to any test that does not run long enough to
reach a steady state. The unit suite proved each mechanism works once. Only
the churn gate showed that none of them worked twice.
A compatibility gap also surfaced, out of M0's scope and left alone:
`update.apply` (`src/update.zig:18`) rejects any update document whose first
key is not `$`, so `replaceOne`, `findOneAndReplace` and `bulkWrite`'s
`replaceOne` all fail with "bad update". It is a CRUD feature rather than
storage, and it is already inside what the 161 spec failures cover.
---

148
README.md
View File

@@ -10,7 +10,10 @@ driver, PyMongo — connect over TCP and just work.
The direction from this MVP — a full-fledged embedded, tens-of-GB,
maximally MongoDB-compatible database — its decision record, milestones
and gates live in [PLAN.md](PLAN.md). Milestone 0 (mmap + WAL storage
foundation) is next.
foundation) has landed; its measured gate results are in
[`tests/e2e/results/m0-gates.txt`](tests/e2e/results/m0-gates.txt).
Milestone 1 (cursors, and the doc-level free list the churn gate showed is
needed) is next.
## Quick start
@@ -155,11 +158,42 @@ longer one.
## Working with large collections
Everything lives in RAM (db → collection → _id → document maps) and every
write command is logged with `fsync` before it is acknowledged (one sync per
command via group commit — a 500-doc `insertMany` syncs once, not 500
times), so multi-GB collections work, with cost/behavior notes measured by
the `tests/e2e/big.js` harness (12-core/32 GB Mac):
Documents, B+tree pages and overflow records live in an mmap'd data file
(`<db>.data`), with the append-only log as the write-ahead log in front of it.
Every write command is logged with `fsync` before it is acknowledged (one sync
per command via group commit — a 500-doc `insertMany` syncs once, not 500
times); a periodic checkpoint publishes the data file and truncates the log.
So resident memory is the working set rather than the size of the database, and
an open does not replay everything ever written. Measured by the
`tests/e2e/big.js` harness on a 12-core/32 GB Mac:
| 21.5 GB collection (1.3M × 16 KiB) | |
|---|---|
| data file | 21.75 GB (+1.3% over the documents) |
| log after the load | 2.5 MB — checkpoints reclaim it |
| `kill -9` then reopen | 0.5 s (also 0.5 s at 4 GB) |
| resident after reopen | 237 MB — 1.1% of the data |
| documents after restart | all 1,310,720, last one byte-intact |
| acked writes surviving `kill -9` | 200/200 |
Notes on the cost side, from the same run:
- **A bulk load still touches everything it writes.** Peak resident during the
20 GB load was 18.5 GB: writing 21 GB of pages dirties 21 GB of pages, and
the kernel keeps them until it wants the memory back. The mmap win is in
reopen and steady-state reads, not in bulk ingest.
- **Bulk insert costs about a quarter of its old throughput** (732 → 555 MB/s
at 1 GB), because document bytes now reach the disk uncompressed in the data
file on top of the compressed log. This was the anticipated trade for the
rows above; see `m0-gates.txt` for the untried mitigations.
- **A cold full scan reads the whole collection from disk** — ~55 s for 21 GB,
about 390 MB/s. Index the fields you filter on; `countDocuments({})` with no
filter is a full scan by definition.
- **Churn is bounded but not tight.** Under sustained rewriting the data file
settles at 1.65× (delete-heavy) to 2.47× (update-heavy) the live data and
stays there. Reclamation is by whole-collection rebuild, which needs a second
copy of the live data before it can free the first; a doc-level free list is
the M1 fix.
- **Build in ReleaseFast** — `zig build` defaults to it. A Debug server is
10-200x slower on every path (the matcher alone was 70 µs/doc in Debug
@@ -173,12 +207,16 @@ the `tests/e2e/big.js` harness (12-core/32 GB Mac):
only a floor below which small logs are left alone. (It used to fire
every 16 MB regardless, rewriting the whole log each time: quadratic
total traffic, and the reason bulk loads needed a raised threshold.)
- **`findOne({_id})` is O(1) only for ObjectId ids.** Integer, int64 and
double ids compare equal but hash differently, so the docs-map fast path
is skipped and every `_id` lookup becomes a full scan. Use the driver's
default ObjectIds (or a secondary index) on big collections. The
order-preserving key encoding already removes the ambiguity that forces
this; lifting the restriction waits on an ordered `_id` index.
- **`findOne({_id})` is an index descent for every `_id` type.** It used to
be O(1) for ObjectIds and a *full scan* for integer, int64 and double ids,
which compare equal but hashed differently. The hash map is gone: `_id_` is
an ordered B+tree over the canonical key encoding, so all of those are one
descent. Measured on the 21.5 GB collection with integer ids:
`findOne({_id})` 2 ms, against 55 s for a scan of the same collection.
One consequence to know about: because the encoding is canonical, `1`
(int32), `1` (int64) and `1.0` (double) are now the *same* `_id` — which
matches MongoDB, and which a database written by an older build will warn
loudly about on first open if it holds two such documents.
- **Secondary-index entry insert is O(n)** (sorted array — see v1 limits
above), so inserting into a collection that already has an index is
quadratic. Building an index over existing data is not: entries are
@@ -194,37 +232,65 @@ With the ReleaseFast default build (MongoDB 8.3.7 on the same Mac):
| benchmark | MultiforaDB | mongodb | winner |
|---|---|---|---|
| insertOne (sequential) | 0.20 ms | 4.7 ms | **MultiforaDB ×24** |
| bulk insert (insertMany) | 752 MB/s | 744 MB/s | MultiforaDB |
| createIndex({k: 1}) | 67 ms | 76 ms | **MultiforaDB** |
| countDocuments({}) | 2.6 ms | 11.2 ms | **MultiforaDB ×4** |
| findOne({_id}) | 0.45 ms | 0.65 ms | **MultiforaDB** |
| findOne indexed | 0.54 ms | 4.6 ms | **MultiforaDB ×8** |
| range-scan count | 13.7 ms | 12.6 ms | mongodb ×1.1 |
| sort + limit(20), on `_id` | 2.3 ms | 2.0 ms | mongodb ×1.1 |
| sort + limit(20), indexed field | 1.0 ms | — | — |
| aggregate $group | 8.1 ms | 12.3 ms | **MultiforaDB** |
| updateOne({_id}) | 0.15 ms | 0.19 ms | **MultiforaDB** |
| updateMany (65 docs) | 1.7 ms | 6.1 ms | **MultiforaDB ×3.6** |
| deleteOne + insert | 0.50 ms | 4.9 ms | **MultiforaDB ×10** |
| server RSS | 539 MB | 1.3 GB | **MultiforaDB ×2.4** |
| kill -9 → reopen | 0.8 s | 1.3 s | **MultiforaDB** |
| db on disk | 97 MB | 91 MB | mongodb |
| insertOne (sequential) ×200 | 0.20 ms | 4.4 ms | **MultiforaDB ×22** |
| bulk insert (insertMany) | 555 MB/s | 739 MB/s | mongodb ×1.3 |
| createIndex({k: 1}) | 27.9 ms | 70 ms | **MultiforaDB ×3** |
| countDocuments({}) | 2.6 ms | 11.3 ms | **MultiforaDB ×4** |
| findOne({_id}) | 0.60 ms | 0.61 ms | parity |
| findOne indexed | 0.54 ms | 1.1 ms | **MultiforaDB ×2** |
| range-scan count | 11.2 ms | 12.5 ms | **MultiforaDB** |
| sort + limit(20) on `_id` | 1.5 ms | 2.0 ms | **MultiforaDB** |
| projection + limit(1000) | 3.6 ms | 4.3 ms | **MultiforaDB** |
| aggregate $group | 7.4 ms | 13.4 ms | **MultiforaDB ×2** |
| updateOne({_id}) ×50 | 0.15 ms | 0.21 ms | **MultiforaDB** |
| updateMany (65 docs) | 1.9 ms | 5.9 ms | **MultiforaDB ×3** |
| deleteOne + insert | 0.63 ms | 4.9 ms | **MultiforaDB ×8** |
| concurrent durable writes, 32 clients | 32,817/s | 3,765/s | **MultiforaDB ×9** |
| server RSS after the load | 1.06 GB | 1.18 GB | MultiforaDB |
| kill -9 → reopen | 0.3 s | 1.3 s | **MultiforaDB ×4** |
| db on disk | 914 MB | 85 MB | **mongodb ×11** |
The engine now holds every document as canonical BSON bytes in a
segmented per-collection slab (no per-document arena, no second Pair-tree
copy), which is why RSS is a quarter of MongoDB's and the range scan —
matching against the bytes directly, skipping fields by length — runs at
parity. The log is LZ4-compressed in 256 KiB blocks, so the on-disk size
matches MongoDB's compressed files. Bulk insert is compress-bound (the
LZ4 codec runs at ~1.7 GB/s; deflate would cap writes below the insert
rate, which is why the roadmap chose LZ4).
Each cell is the best of three runs of the same suite, for both servers. That
is not fussiness: two consecutive runs of the *same* binary moved the sub-10 ms
rows by 27–51% on this machine, so a single run's ratios say more about the
minute they were taken in than about either database. Treat differences under
about 1.5× as noise.
Reproduce the table with `bash tests/e2e/compare-run.sh 1g 16k`; the
pre-tree baseline is recorded in `tests/e2e/results/phase1.txt`, and the
runs with the B+tree, ordered `_id` index, compressed log and byte
storage (roadmap items 1–4) in `tests/e2e/results/phase2.txt` through
`phase5.txt`.
Two rows in that table changed direction with the mmap foundation and are
worth being explicit about.
**`db on disk`** was 97 MB against mongod's 91 MB when the log was the only
copy of the data and LZ4 compressed it. The data file does not compress
documents: 65,536 × 16 KiB documents now occupy 914 MB of allocated blocks
(`du`; `ls` shows 1.09 GB, the difference being the sparse tail the file is
grown into) against mongod's compressed 85 MB. Per-page or per-extent
compression is the fix, and it is not in M0. If you are comparing against a
report from before this was written, note that the row used to measure the log
file alone — which said 20 MB for a 1 GB collection, because the documents had
moved to `<db>.data`. Fixed in `compare-run.sh`.
**`server RSS`** is measured right after writing the whole dataset, so it
reflects a bulk load having dirtied every page it wrote, not steady state. The
number that speaks to the architecture is resident memory *after a reopen*:
237 MB for a 21.5 GB collection (see the section above). Before M0 the same
measurement was 523 MB for a 512 MB collection, because recovering each
document's `_id` read every document at open.
The range scan runs at parity because matching happens against the stored BSON
bytes directly, skipping fields by length, with no per-document arena and no
second Pair-tree copy. The log is still LZ4-compressed in 256 KiB blocks; since
it is now truncated at every checkpoint, its size no longer tracks the
database's.
Reproduce the whole thing — main suite, concurrency sweep and the meta rows —
with `bash tests/e2e/bench-run.sh 1g 16k "1 4 8 16 32"`, which writes a
timestamped report to `tests/e2e/results/` and diffs it against the last one.
The pre-tree baseline is in `tests/e2e/results/phase1.txt`; the runs with the
B+tree, ordered `_id` index, compressed log and byte storage (roadmap items
1–4) in `phase2.txt` through `phase5.txt`; the all-in-RAM engine this table's
predecessor measured in `phase8.txt`; and the mmap foundation's own gate
results, including what each of those rows cost or gained, in
`m0-gates.txt`.
### What is left (highest impact first)

41
tests/e2e/big.js
View File

@@ -119,10 +119,13 @@ async function stopServer(sig = 'SIGKILL') {
serverDead = true;
server = null;
}
async function rssMB() {
// Synchronous on purpose. The async version sampled inside setInterval and a
// short run could finish before any sample landed, reporting RSS 0 -- which reads
// as "measured and tiny" rather than "not measured".
function rssMB() {
if (!server) return 0;
try {
const out = (await new Promise((r) => require('child_process').exec(`ps -o rss= -p ${server.pid}`, (e, so) => r(so || '')))).trim();
const out = require('child_process').execSync(`ps -o rss= -p ${server.pid}`, { encoding: 'utf8' }).trim();
return Math.round(Number(out) / 1024);
} catch { return 0; }
}
@@ -164,9 +167,9 @@ async function main() {
const payload = 'x'.repeat(payloadLen);
const { ObjectId } = require('mongodb');
const t0 = Date.now();
const logSamples = [{ t: 0, size: fs.existsSync(DBFILE) ? fs.statSync(DBFILE).size : 0, rss: 0 }];
const sampler = setInterval(async () => {
logSamples.push({ t: Date.now() - t0, size: fs.existsSync(DBFILE) ? fs.statSync(DBFILE).size : 0, rss: await rssMB() });
const logSamples = [{ t: 0, size: fs.existsSync(DBFILE) ? fs.statSync(DBFILE).size : 0, rss: rssMB() }];
const sampler = setInterval(() => {
logSamples.push({ t: Date.now() - t0, size: fs.existsSync(DBFILE) ? fs.statSync(DBFILE).size : 0, rss: rssMB() });
}, 2000);
// Rate curve: avg MB/s per progress chunk, to show how throughput changes
@@ -215,12 +218,18 @@ async function main() {
row('throughput', `${(bytes / 1e6 / (insertMs / 1000)).toFixed(1)} MB/s (${(inserted / (insertMs / 1000)).toFixed(0)} docs/s)`);
row('rate curve (MB/s per chunk)', rates.join(' → ') || 'n/a');
const compactions = logSamples.filter((s, i) => i > 0 && s.size < logSamples[i - 1].size - 2 * 1024 * 1024).length;
// A shrinking log means a *checkpoint* now, not a compaction: the checkpoint
// publishes the data file and truncates the log to its header. Compaction
// (which rewrites the data file) leaves no signature in the log's size, so
// this counter cannot see it and must not claim to.
const truncations = logSamples.filter((s, i) => i > 0 && s.size < logSamples[i - 1].size - 2 * 1024 * 1024).length;
const sizes = logSamples.map((s) => s.size);
row('log file size (min → final)', `${fmt(Math.min(...sizes))} → ${fmt(sizes[sizes.length - 1])}`);
row('compaction events observed', compactions, '(log shrank by >2MB between samples)');
const dataFileSize = fs.existsSync(DBFILE + '.data') ? fs.statSync(DBFILE + '.data').size : 0;
row('data file size', fmt(dataFileSize));
row('log truncations (checkpoints)', truncations, '(log shrank by >2MB between samples)');
const peakRss = Math.max(...logSamples.map((s) => s.rss));
row('peak server RSS', `${peakRss} MB`, '(in-memory engine: docs live in RAM)');
row('peak server RSS', `${peakRss} MB`, '(a write touches its pages; see the after-reopen row)');
// ---- find / read -------------------------------------------------------
console.log('\n== find / read on full dataset ==');
@@ -298,7 +307,12 @@ async function main() {
await client.close();
await stopServer('SIGKILL');
const reopenMs = await startServer();
row('kill -9 then reopen (replay of full log)', `${(reopenMs / 1000).toFixed(1)} s`);
row('kill -9 then reopen', `${(reopenMs / 1000).toFixed(1)} s`);
// The point of the mmap work: what an open pays for is the working set, not
// the size of the database. Sampled right after the server answers its first
// ping, before any query has touched a document.
const rssAfterReopen = rssMB();
row('RSS after reopen (working set)', `${rssAfterReopen} MB`);
const c2 = new MongoClient(URL, { serverSelectionTimeoutMS: 10000 });
await c2.connect();
const db2 = c2.db('big');
@@ -335,10 +349,13 @@ async function main() {
await stopServer('SIGKILL');
console.log('\n== summary ==');
console.log(` multiforadb handles a ${fmt(bytes)} collection fully in RAM (RSS ${peakRss} MB)`);
// Values captured while the server was alive: by the time the summary prints,
// it has been stopped and its files removed, so sampling here reports zero.
console.log(` multiforadb handles a ${fmt(bytes)} collection in a ${fmt(dataFileSize)} data file`);
console.log(` peak RSS during load ${peakRss} MB; after reopen ${rssAfterReopen} MB — the working set, not the data size`);
console.log(` insert: ${(bytes / 1e6 / (insertMs / 1000)).toFixed(1)} MB/s — fsync per write is by design (crash safety)`);
if (compactions > 0) {
console.log(` ${compactions} compaction rewrites observed: every 16MB of writes rewrites the whole log — for multi-GB loads the cumulative rewrite traffic dominates`);
if (truncations > 0) {
console.log(` ${truncations} checkpoints reclaimed the log during the load, which is why it ends at ${fmt(sizes[sizes.length - 1])} rather than ${fmt(bytes)}`);
}
console.log('BIG_OK');
}

7
tests/e2e/compare-run.sh
View File

@@ -71,7 +71,12 @@ wait_ready "mongodb://127.0.0.1:$MFDB_PORT" || { echo "multiforadb never became
node tests/e2e/compare.js --url "mongodb://127.0.0.1:$MFDB_PORT" --label multiforadb --size "$SIZE" --doc-size "$DOC" \
> "$CMPDIR/mfdb-report.txt" 2>&1 || { echo "multiforadb bench failed:"; tail -5 "$CMPDIR/mfdb-report.txt"; }
MFDB_RSS=$(ps -o rss= -p $MFDB_PID | awk '{printf "%.0f", $1/1024}')
MFDB_DISK=$(du -sm "$MFDB_LOG" | awk '{print $1}')
# Both files. The documents live in "$MFDB_LOG".data since the mmap foundation
# landed, and the log is truncated at every checkpoint -- so measuring the log
# alone reported 20 MB for a 1 GB collection, against a `du` over mongod's whole
# dbpath. `du` rather than `ls`: the data file is grown with setLength and is
# sparse until written, and allocated blocks are what actually costs disk.
MFDB_DISK=$(du -scm "$MFDB_LOG" "$MFDB_LOG.data" 2>/dev/null | tail -1 | awk '{print $1}')
echo; echo "### multiforadb kill -9 + reopen (replay)"
kill -9 $MFDB_PID; wait $MFDB_PID 2>/dev/null

36
tests/e2e/results/bench-20260803-223947.txt Normal file
View File

@@ -0,0 +1,36 @@
# multiforadb vs MongoDB benchmark
# date: 2026-08-03T19:42:00Z git: 5228ed7+dirty
# args: size=1g doc-size=16k wc=j clients=1 4 8 16 32 per-client=2000
# reproduce: bash tests/e2e/bench-run.sh 1g 16k "1 4 8 16 32"
[main]
insertOne (sequential) ×200 0.20 ms 4.9 ms
bulk insert throughput 463.4 MB/s 581.3 MB/s
docs loaded 65,536 65,536
createIndex({k: 1}) 27.9 ms 78.1 ms
countDocuments({}) 2.6 ms 12.8 ms
findOne({_id: <ObjectId>}) 0.64 ms 0.72 ms
findOne({k: 500}) (indexed) 0.59 ms 1.6 ms
find({p: {$gte,$lt}}).count() (scan) 14.1 ms 12.5 ms
find({}).sort({_id:-1}).limit(20) 1.5 ms 2.4 ms
find({}, {proj}).limit(1000) 3.6 ms 4.7 ms
aggregate $group by k 8.4 ms 13.4 ms
updateOne({_id}) ×50 0.15 ms 0.23 ms
updateMany({k: 7}, {$inc}) 1.9 ms 7.0 ms
deleteOne({_id}) + insertOne 0.63 ms 5.8 ms
node client RSS 153 MB 155 MB
[concurrency]
clients 1 8354 204 41.0x
clients 4 465 0.0x
clients 8 978 0.0x
clients 16 1877 0.0x
clients 32 3765 0.0x
[meta]
mfdb_rss_mb 1059
md_rss_mb 1203
mfdb_reopen 0.3s
md_reopen 1.3s
mfdb_disk_mb 20MB
md_disk_mb 83MB

36
tests/e2e/results/bench-20260803-225128.txt Normal file
View File

@@ -0,0 +1,36 @@
# multiforadb vs MongoDB benchmark
# date: 2026-08-03T19:53:01Z git: 4b70ce6+dirty
# args: size=1g doc-size=16k wc=j clients=1 4 8 16 32 per-client=2000
# reproduce: bash tests/e2e/bench-run.sh 1g 16k "1 4 8 16 32"
[main]
insertOne (sequential) ×200 0.22 ms 4.4 ms
bulk insert throughput 555.5 MB/s 738.5 MB/s
docs loaded 65,536 65,536
createIndex({k: 1}) 28.0 ms 70.0 ms
countDocuments({}) 2.8 ms 12.3 ms
findOne({_id: <ObjectId>}) 0.60 ms 0.61 ms
findOne({k: 500}) (indexed) 0.54 ms 1.1 ms
find({p: {$gte,$lt}}).count() (scan) 11.2 ms 13.2 ms
find({}).sort({_id:-1}).limit(20) 1.5 ms 2.0 ms
find({}, {proj}).limit(1000) 3.8 ms 4.3 ms
aggregate $group by k 7.4 ms 15.8 ms
updateOne({_id}) ×50 0.18 ms 0.21 ms
updateMany({k: 7}, {$inc}) 1.9 ms 5.9 ms
deleteOne({_id}) + insertOne 0.67 ms 5.1 ms
node client RSS 163 MB 155 MB
[concurrency]
clients 1 8373 218 38.4x
clients 4 20880 491 42.5x
clients 8 26349 964 27.3x
clients 16 29594 1710 17.3x
clients 32 32235 3389 9.5x
[meta]
mfdb_rss_mb 1059
md_rss_mb 1242
mfdb_reopen 0.3s
md_reopen 1.3s
mfdb_disk_mb 20MB
md_disk_mb 87MB

36
tests/e2e/results/bench-20260803-225322.txt Normal file
View File

@@ -0,0 +1,36 @@
# multiforadb vs MongoDB benchmark
# date: 2026-08-03T19:54:54Z git: 4b70ce6+dirty
# args: size=1g doc-size=16k wc=j clients=1 4 8 16 32 per-client=2000
# reproduce: bash tests/e2e/bench-run.sh 1g 16k "1 4 8 16 32"
[main]
insertOne (sequential) ×200 0.21 ms 4.4 ms
bulk insert throughput 550.4 MB/s 721.6 MB/s
docs loaded 65,536 65,536
createIndex({k: 1}) 28.9 ms 72.5 ms
countDocuments({}) 3.6 ms 11.3 ms
findOne({_id: <ObjectId>}) 0.80 ms 0.76 ms
findOne({k: 500}) (indexed) 0.73 ms 5.4 ms
find({p: {$gte,$lt}}).count() (scan) 14.5 ms 15.4 ms
find({}).sort({_id:-1}).limit(20) 1.9 ms 2.6 ms
find({}, {proj}).limit(1000) 3.8 ms 4.9 ms
aggregate $group by k 11.2 ms 15.5 ms
updateOne({_id}) ×50 0.17 ms 0.21 ms
updateMany({k: 7}, {$inc}) 2.0 ms 6.1 ms
deleteOne({_id}) + insertOne 0.69 ms 4.9 ms
node client RSS 150 MB 156 MB
[concurrency]
clients 1 7709 229 33.7x
clients 4 20122 493 40.8x
clients 8 26210 977 26.8x
clients 16 27794 1895 14.7x
clients 32 32817 3249 10.1x
[meta]
mfdb_rss_mb 1060
md_rss_mb 1178
mfdb_reopen 0.3s
md_reopen 1.3s
mfdb_disk_mb 20MB
md_disk_mb 85MB

52
tests/e2e/results/bench-latest.txt
View File

@@ -1,36 +1,36 @@
# mongo-lite vs MongoDB benchmark
# date: 2026-08-03T08:53:32Z git: c8d547f+dirty
# multiforadb vs MongoDB benchmark
# date: 2026-08-03T19:54:54Z git: 4b70ce6+dirty
# args: size=1g doc-size=16k wc=j clients=1 4 8 16 32 per-client=2000
# reproduce: bash tests/e2e/bench-run.sh 1g 16k "1 4 8 16 32"
[main]
insertOne (sequential) ×200 0.20 ms 14.8 ms
bulk insert throughput 732.4 MB/s 546.2 MB/s
insertOne (sequential) ×200 0.21 ms 4.4 ms
bulk insert throughput 550.4 MB/s 721.6 MB/s
docs loaded 65,536 65,536
createIndex({k: 1}) 74.4 ms 83.6 ms
countDocuments({}) 3.1 ms 12.4 ms
findOne({_id: <ObjectId>}) 0.59 ms 0.61 ms
findOne({k: 500}) (indexed) 0.56 ms 0.94 ms
find({p: {$gte,$lt}}).count() (scan) 13.0 ms 15.0 ms
find({}).sort({_id:-1}).limit(20) 2.2 ms 2.2 ms
find({}, {proj}).limit(1000) 3.5 ms 4.4 ms
aggregate $group by k 8.3 ms 13.2 ms
updateOne({_id}) ×50 0.14 ms 0.20 ms
updateMany({k: 7}, {$inc}) 1.9 ms 6.4 ms
deleteOne({_id}) + insertOne 0.58 ms 4.9 ms
node client RSS 158 MB 157 MB
createIndex({k: 1}) 28.9 ms 72.5 ms
countDocuments({}) 3.6 ms 11.3 ms
findOne({_id: <ObjectId>}) 0.80 ms 0.76 ms
findOne({k: 500}) (indexed) 0.73 ms 5.4 ms
find({p: {$gte,$lt}}).count() (scan) 14.5 ms 15.4 ms
find({}).sort({_id:-1}).limit(20) 1.9 ms 2.6 ms
find({}, {proj}).limit(1000) 3.8 ms 4.9 ms
aggregate $group by k 11.2 ms 15.5 ms
updateOne({_id}) ×50 0.17 ms 0.21 ms
updateMany({k: 7}, {$inc}) 2.0 ms 6.1 ms
deleteOne({_id}) + insertOne 0.69 ms 4.9 ms
node client RSS 150 MB 156 MB
[concurrency]
clients 1 8435 232 36.4x
clients 4 22480 491 45.8x
clients 8 25920 966 26.8x
clients 16 31079 1842 16.9x
clients 32 34041 3632 9.4x
clients 1 7709 229 33.7x
clients 4 20122 493 40.8x
clients 8 26210 977 26.8x
clients 16 27794 1895 14.7x
clients 32 32817 3249 10.1x
[meta]
ml_rss_mb 553
md_rss_mb 1486
ml_reopen 0.8s
mfdb_rss_mb 1060
md_rss_mb 1178
mfdb_reopen 0.3s
md_reopen 1.3s
ml_disk_mb 97MB
md_disk_mb 106MB
mfdb_disk_mb 20MB
md_disk_mb 85MB

156
tests/e2e/results/m0-gates.txt Normal file
View File

@@ -0,0 +1,156 @@
# M0 gate results — mmap + WAL storage foundation (PLAN D7)
#
# Machine: Apple Silicon, macOS 25.5.0, 16 KiB system pages, APFS.
# Server: MultiforaDB at the commit named below, ReleaseFast.
# Driver: mongodb@7.5.0 (pinned in tests/e2e/package-lock.json).
# mongod: 8.3.7, for the parity rows.
#
# Read this alongside PLAN.md §5. Every number here is reproducible with the
# command printed under it; where a gate was not met, the number is recorded as
# measured and the reason is stated rather than the workload being tuned until
# it passed.
[D7.1] unit tests, ReleaseFast and ReleaseSafe
zig build test -Doptimize=ReleaseFast 122/122 pass
zig build test -Doptimize=ReleaseSafe 122/122 pass
zig build fuzz fuzz_split, spill, spill2, stress clean
Both modes matter: `protect_stable` (the mprotect belt over the published
image) is comptime-off in ReleaseFast, and `std.posix.mprotect` not existing
in Zig 0.16 was a ReleaseSafe-only compile error.
[D7.2] end-to-end suites — all green
reproduce: zig-out/bin/multiforadb --port 27020 --db /tmp/mfdb-e2e.log \
--ttl-sweep-secs 1 & then node tests/e2e/<suite>
e2e.js CRUD, operators, aggregate, errors 35/35
e2e3.js secondary indexes: unique/sparse/compound 16/16
e2e4.js TTL indexes: expiry and rejected specs 17/17
e2e5.js miscellaneous command surface 3/3
e2e2.js concurrent, 4 writers + 4 readers 2/2
e2e2.js crash-a / crash-b (kill -9, then verify) 1/1 then 3/3
e2e6.js compaction and log growth (own server) 72/72
[D7.3] large-collection smoke — big.js
reproduce: node tests/e2e/big.js --size 20g --doc-size 16k
20g run 4g run
documents 1,310,720 262,144 (x 16 KiB)
live data 21.47 GB 4.29 GB
data file 21.75 GB 4.35 GB (+1.3%, +1.4%)
log file, min -> final 16 B -> 2.5 MB 16 B -> 7.7 MB
bulk insert 411.0 MB/s 560.3 MB/s
peak server RSS during load 18,492 MB 3,318 MB
kill -9 then reopen 0.5 s 0.5 s
RSS after reopen 237 MB (1.1%) 130 MB (3.0%)
count after restart 1,310,720 OK 262,144 OK
last doc intact after restart payload 16254B OK
kill -9 after 200 acked writes 200/200 200/200
This is the milestone's central claim, measured: an open costs the working
set, not the size of the database, and it no longer replays the whole log.
Reopen is flat at 0.5 s from 4 GB to 21.5 GB, and RSS after reopen falls as a
*share* of the data as the database grows — which is what "RSS = working set"
has to mean to be worth anything.
Before M0 the same shape of run reported 523 MB resident after reopen for a
512 MB database, because recovering each document's `_id` meant reading every
document at open. That scan is what the mmap foundation deleted.
Two other things the 20g run shows, honestly. Peak RSS during the *load* is
18.5 GB: writing 21 GB touches 21 GB of pages and the kernel keeps them until
it needs the memory, so a bulk load is not where the mmap win shows up. And a
cold full scan of 21 GB costs ~55 s — countDocuments, the range scan and the
unindexed updateMany all land there, ~390 MB/s off the disk — which is correct
and is the reason M1's cursors and index-only paths matter.
[D7.4] churn gate — doc/slab garbage (PLAN amendment A2 retargets D6.2 here)
40,000 x 16 KiB documents (655 MB live), one secondary index, steady state
measured as data-file size / live data after the ratio stopped moving.
before the fixes after
delete half and refill, 6 rounds 4.10x, climbing 1.65x, flat
random $set over 5x the collection 3.58x, climbing 2.47x, flat
"Climbing" is the whole finding: nothing was being reclaimed at all. Three
bugs, all fixed in `db/pager: reclaim what churn abandons` — a compaction
trigger that had read `log.data_bytes` since before the log was truncated at
every checkpoint, a numeric stable mark that made recycled pages look
published, and a first-fit free list that let one-page requests dismantle
the extents. A fourth fix (a rebuild publishes twice, so the space it frees
is reusable immediately) took the interleaved case from 3.58x to 2.47x.
The gate hoped for ~1.3x and this is above it, structurally: a rebuild needs
a whole second copy of the live data before the first can be freed, and
two-generation retention holds the old copy through two more publishes. The
gate's purpose was to decide whether doc-level free lists are needed after
M0. They are; that is an M1 item. What M0 owes is a bound, and there is one.
[D7.5] benchmark parity vs the pre-mmap engine
reproduce: bash tests/e2e/bench-run.sh 1g 16k "1 4 8 16 32"
baseline: tests/e2e/results/phase8.txt (c8d547f, all-in-RAM engine)
M0: three runs, tests/e2e/results/bench-2026080{3-223947,3-225128,3-225322}.txt
Read the range, not a single delta. Two consecutive runs of the *same* binary
moved the sub-10 ms rows by 27-51% on this machine, so any one comparison
reads whatever the noise did that minute. Best-of-three against the baseline:
row pre-mmap M0 min..max best
insertOne (sequential) x200 0.20 ms 0.20..0.22 ms +0%
bulk insert throughput 732.4 MB/s 463.4..555.5 MB/s -24%
createIndex({k: 1}) 74.4 ms 27.9..28.9 ms -62%
countDocuments({}) 3.1 ms 2.6..3.6 ms -16%
findOne({_id: <ObjectId>}) 0.59 ms 0.60..0.80 ms +2%
findOne({k: 500}) (indexed) 0.56 ms 0.54..0.73 ms -4%
find({p: {$gte,$lt}}).count() (scan) 13.0 ms 11.2..14.5 ms -14%
find({}).sort({_id:-1}).limit(20) 2.2 ms 1.5..1.9 ms -32%
find({}, {proj}).limit(1000) 3.5 ms 3.6..3.8 ms +3%
aggregate $group by k 8.3 ms 7.4..11.2 ms -11%
updateOne({_id}) x50 0.14 ms 0.15..0.18 ms +7%
updateMany({k: 7}, {$inc}) 1.9 ms 1.9..2.0 ms +0%
deleteOne({_id}) + insertOne 0.58 ms 0.63..0.69 ms +9%
node client RSS 158 MB 150..163 MB -5%
concurrent durable insertOne, docs/s:
clients 1 8,435 7,709..8,373 -1%
clients 4 22,480 20,122..20,880 -7%
clients 8 25,920 26,210..26,349 +2%
clients 16 31,079 27,794..29,594 -5%
clients 32 34,041 32,235..32,817 -4%
One reproducible regression: bulk insert, -24%, steady across all three runs.
This is PLAN risk 1 exactly as written -- document bytes now reach the disk
uncompressed in the data file on top of the LZ4 log, and that writeback
bandwidth is new. Every other row is inside the run-to-run spread. The gate
as stated was "no phase8 row regressed"; it is met for the read and latency
rows and not for bulk load, which is the trade the milestone makes and which
risk 1 anticipated. Untried mitigations, in the order worth trying:
MADV_HUGEPAGE / a larger growth chunk, and not writing doc bytes twice
(log and slab) for a bulk path that could log an extent reference instead.
createIndex at -62% is the other reproducible number, and it comes from the
same change: a bulk build now packs pages in the mapping instead of growing
an ArrayList.
[D7.6] MongoDB spec-test scorecard
reproduce: bash tests/spec/fetch.sh && node tests/spec/run.js --scorecard
total 131 pass 161 fail 195 skip 175 files 0 errored
Byte-identical to the scorecard recorded before the storage rewrite. That is
the intended result and it is worth stating plainly: M0 replaced the document
store, the node arena and the overflow slab, added copy-on-write, a watermark
and a checkpoint, and moved every index leaf's payload -- and changed no
observable CRUD or aggregate semantics.
# WHAT THE GATES FOUND
# Five bugs, none of which any unit test or e2e suite had reached:
# 1. The compaction trigger had been dead since commit 14 (log truncation
# zeroes the counter it gated on). Nothing reclaimed doc-slab garbage.
# 2. `page_mut_cow` asked `p >= stable_pages`, which is false for a recycled
# page, so every write to one copied and freed it again.
# 3. First fit let copy-on-write's one-page requests shave the extents the
# doc slab needs, so the free list drained every generation and the file
# grew anyway.
# 4. A rebuild's freed space took two unrelated checkpoints to become
# reusable, so the next rebuild grew the file instead of reusing it.
# 5. `reserve_pages`' promise was a single counter on the pager. Two upserts
# on different collections release it independently, so the first to
# finish revoked the second's promise mid-write -- aborting the server at
# four concurrent clients, on the first concurrent benchmark since the
# data file landed. PLAN risk 3, whose mitigation was never built.
# Four of the five are invisible without a workload that runs long enough to
# reach a steady state, or wide enough to have two writers. That is the
# argument for keeping both the churn gate and the concurrent benchmark in the
# gate list rather than treating them as optional.
#
# ONE COMPATIBILITY GAP FOUND, NOT FIXED (out of M0's scope, storage-only)
# `update.apply` (src/update.zig:18) rejects any update document whose first
# key is not `$`, so replacement-style writes -- replaceOne, findOneAndReplace,
# bulkWrite's replaceOne -- fail with "bad update". Found while building the
# churn workload. It is a CRUD feature, not storage, and it is already part of
# what the 161 spec failures cover.