diff --git a/PLAN.md b/PLAN.md index a3aa62a..58ef9d1 100644 --- a/PLAN.md +++ b/PLAN.md @@ -532,13 +532,79 @@ discipline lives, and `ls`/`du` stay honest for D6.6's backup story. | 16 | drop the docs hashmap | original step 7 | | 17 | gates | D7's six items; churn gate per amended D6.2 | -M0 is done when D7's six items pass. Two harness fixes are prerequisites for -the gate rather than the work: `bench-run.sh` copies its report over -`bench-latest.txt` unconditionally, including after a run that only warned, -so the baseline being defended can be clobbered; and the four B+tree dev -harnesses (`spill.zig`, `spill2.zig`, `stress.zig`, `fuzz_split.zig`) are in -no build step, so `zig build test` will not notice an API break in the one -place that fuzzes splits and >1 KB keys. +All seventeen commits are landed. Both harness fixes that were prerequisites +for the gate rather than the work are in: `bench-run.sh` no longer copies its +report over `bench-latest.txt` after a degraded run (it used to, so the +baseline being defended could be clobbered), and the four B+tree dev harnesses +(`spill.zig`, `spill2.zig`, `stress.zig`, `fuzz_split.zig`) now have a +`zig build fuzz` step — they were in no build step, so `zig build test` could +not notice an API break in the one place that fuzzes splits and >1 KB keys. + +### The gate result + +Measured numbers, and the command that reproduces each, are in +`tests/e2e/results/m0-gates.txt`. In short: + +| D7 | gate | outcome | +|---|---|---| +| 1 | unit tests RF + RS | pass, 122/122 both | +| 2 | e2e matrix unchanged | pass, every suite | +| 3 | 20-30 GB smoke, RSS ≈ working set, fast reopen | pass at 21.5 GB | +| 4 | churn gate | measured, bounded, above the hoped-for 1.3x | +| 5 | benchmark parity | pass except bulk insert, -24% | +| 6 | spec runner + scorecard | pass, scorecard unchanged | + +Two of those need reading rather than a tick. + +**D7.4** hoped for ~1.3x of live data and settles at 1.65x (delete-heavy) to +2.47x (update-heavy), flat in both cases. Before the gate's own findings were +fixed it was 4.1x and *climbing linearly* — nothing was being reclaimed at +all. The gate's stated purpose (amendment A2) was to decide whether doc-level +free lists are needed after M0, and the answer is yes, in M1: a rebuild needs +a whole second copy of the live data before the first can be freed, so +rebuild-only reclamation cannot reach 1.3x however it is tuned. What M0 owed +was a bound, and there is one. + +**D7.5** says "no phase8 row regresses". Bulk insert throughput regresses +24%, reproducibly (732 → 555 MB/s). That is risk 1 as written: document bytes +now reach the disk uncompressed in the data file on top of the LZ4 log, and +that writeback bandwidth is new. Every other row is inside this machine's +run-to-run spread, which two consecutive runs of the same binary showed to be +27-51% on the sub-10 ms rows — so the gate is met for the read and latency +rows and not for bulk load. `createIndex` improves 62%, also reproducibly, +from the same change. + +### What the gates found + +Five bugs, none of which any unit test or e2e suite had reached, and four of +which need either a long-running workload or a second concurrent writer to +appear at all: + +1. The compaction trigger had been dead since commit 14 — it gated on + `log.data_bytes`, and truncating the log at every checkpoint zeroes that. +2. `page_mut_cow` asked `p >= stable_pages`, which is false for a recycled + page, so every write to one copied and freed it again. +3. First fit let copy-on-write's one-page requests shave the extents the doc + slab needs, so the free list drained every generation. +4. A rebuild's freed space took two unrelated checkpoints to become reusable, + so the next rebuild grew the file instead of reusing it. +5. `reserve_pages`' promise was a single counter on the pager, and two upserts + on different collections release it independently — so the first to finish + revoked the second's promise mid-write. This aborted the server at four + concurrent clients on the first concurrent benchmark since the data file + landed. Risk 3, whose mitigation (private pre-allocated runs) was listed in + this document and never built. + +The lesson worth carrying into M1 is the shape of 1-4: every one is a +*reclamation* bug, invisible to any test that does not run long enough to +reach a steady state. The unit suite proved each mechanism works once. Only +the churn gate showed that none of them worked twice. + +A compatibility gap also surfaced, out of M0's scope and left alone: +`update.apply` (`src/update.zig:18`) rejects any update document whose first +key is not `$`, so `replaceOne`, `findOneAndReplace` and `bulkWrite`'s +`replaceOne` all fail with "bad update". It is a CRUD feature rather than +storage, and it is already inside what the 161 spec failures cover. --- diff --git a/README.md b/README.md index 9aa52ef..e8eaf07 100644 --- a/README.md +++ b/README.md @@ -10,7 +10,10 @@ driver, PyMongo — connect over TCP and just work. The direction from this MVP — a full-fledged embedded, tens-of-GB, maximally MongoDB-compatible database — its decision record, milestones and gates live in [PLAN.md](PLAN.md). Milestone 0 (mmap + WAL storage -foundation) is next. +foundation) has landed; its measured gate results are in +[`tests/e2e/results/m0-gates.txt`](tests/e2e/results/m0-gates.txt). +Milestone 1 (cursors, and the doc-level free list the churn gate showed is +needed) is next. ## Quick start @@ -155,11 +158,42 @@ longer one. ## Working with large collections -Everything lives in RAM (db → collection → _id → document maps) and every -write command is logged with `fsync` before it is acknowledged (one sync per -command via group commit — a 500-doc `insertMany` syncs once, not 500 -times), so multi-GB collections work, with cost/behavior notes measured by -the `tests/e2e/big.js` harness (12-core/32 GB Mac): +Documents, B+tree pages and overflow records live in an mmap'd data file +(`.data`), with the append-only log as the write-ahead log in front of it. +Every write command is logged with `fsync` before it is acknowledged (one sync +per command via group commit — a 500-doc `insertMany` syncs once, not 500 +times); a periodic checkpoint publishes the data file and truncates the log. +So resident memory is the working set rather than the size of the database, and +an open does not replay everything ever written. Measured by the +`tests/e2e/big.js` harness on a 12-core/32 GB Mac: + +| 21.5 GB collection (1.3M × 16 KiB) | | +|---|---| +| data file | 21.75 GB (+1.3% over the documents) | +| log after the load | 2.5 MB — checkpoints reclaim it | +| `kill -9` then reopen | 0.5 s (also 0.5 s at 4 GB) | +| resident after reopen | 237 MB — 1.1% of the data | +| documents after restart | all 1,310,720, last one byte-intact | +| acked writes surviving `kill -9` | 200/200 | + +Notes on the cost side, from the same run: + +- **A bulk load still touches everything it writes.** Peak resident during the + 20 GB load was 18.5 GB: writing 21 GB of pages dirties 21 GB of pages, and + the kernel keeps them until it wants the memory back. The mmap win is in + reopen and steady-state reads, not in bulk ingest. +- **Bulk insert costs about a quarter of its old throughput** (732 → 555 MB/s + at 1 GB), because document bytes now reach the disk uncompressed in the data + file on top of the compressed log. This was the anticipated trade for the + rows above; see `m0-gates.txt` for the untried mitigations. +- **A cold full scan reads the whole collection from disk** — ~55 s for 21 GB, + about 390 MB/s. Index the fields you filter on; `countDocuments({})` with no + filter is a full scan by definition. +- **Churn is bounded but not tight.** Under sustained rewriting the data file + settles at 1.65× (delete-heavy) to 2.47× (update-heavy) the live data and + stays there. Reclamation is by whole-collection rebuild, which needs a second + copy of the live data before it can free the first; a doc-level free list is + the M1 fix. - **Build in ReleaseFast** — `zig build` defaults to it. A Debug server is 10-200x slower on every path (the matcher alone was 70 µs/doc in Debug @@ -173,12 +207,16 @@ the `tests/e2e/big.js` harness (12-core/32 GB Mac): only a floor below which small logs are left alone. (It used to fire every 16 MB regardless, rewriting the whole log each time: quadratic total traffic, and the reason bulk loads needed a raised threshold.) -- **`findOne({_id})` is O(1) only for ObjectId ids.** Integer, int64 and - double ids compare equal but hash differently, so the docs-map fast path - is skipped and every `_id` lookup becomes a full scan. Use the driver's - default ObjectIds (or a secondary index) on big collections. The - order-preserving key encoding already removes the ambiguity that forces - this; lifting the restriction waits on an ordered `_id` index. +- **`findOne({_id})` is an index descent for every `_id` type.** It used to + be O(1) for ObjectIds and a *full scan* for integer, int64 and double ids, + which compare equal but hashed differently. The hash map is gone: `_id_` is + an ordered B+tree over the canonical key encoding, so all of those are one + descent. Measured on the 21.5 GB collection with integer ids: + `findOne({_id})` 2 ms, against 55 s for a scan of the same collection. + One consequence to know about: because the encoding is canonical, `1` + (int32), `1` (int64) and `1.0` (double) are now the *same* `_id` — which + matches MongoDB, and which a database written by an older build will warn + loudly about on first open if it holds two such documents. - **Secondary-index entry insert is O(n)** (sorted array — see v1 limits above), so inserting into a collection that already has an index is quadratic. Building an index over existing data is not: entries are @@ -194,37 +232,65 @@ With the ReleaseFast default build (MongoDB 8.3.7 on the same Mac): | benchmark | MultiforaDB | mongodb | winner | |---|---|---|---| -| insertOne (sequential) | 0.20 ms | 4.7 ms | **MultiforaDB ×24** | -| bulk insert (insertMany) | 752 MB/s | 744 MB/s | MultiforaDB | -| createIndex({k: 1}) | 67 ms | 76 ms | **MultiforaDB** | -| countDocuments({}) | 2.6 ms | 11.2 ms | **MultiforaDB ×4** | -| findOne({_id}) | 0.45 ms | 0.65 ms | **MultiforaDB** | -| findOne indexed | 0.54 ms | 4.6 ms | **MultiforaDB ×8** | -| range-scan count | 13.7 ms | 12.6 ms | mongodb ×1.1 | -| sort + limit(20), on `_id` | 2.3 ms | 2.0 ms | mongodb ×1.1 | -| sort + limit(20), indexed field | 1.0 ms | — | — | -| aggregate $group | 8.1 ms | 12.3 ms | **MultiforaDB** | -| updateOne({_id}) | 0.15 ms | 0.19 ms | **MultiforaDB** | -| updateMany (65 docs) | 1.7 ms | 6.1 ms | **MultiforaDB ×3.6** | -| deleteOne + insert | 0.50 ms | 4.9 ms | **MultiforaDB ×10** | -| server RSS | 539 MB | 1.3 GB | **MultiforaDB ×2.4** | -| kill -9 → reopen | 0.8 s | 1.3 s | **MultiforaDB** | -| db on disk | 97 MB | 91 MB | mongodb | +| insertOne (sequential) ×200 | 0.20 ms | 4.4 ms | **MultiforaDB ×22** | +| bulk insert (insertMany) | 555 MB/s | 739 MB/s | mongodb ×1.3 | +| createIndex({k: 1}) | 27.9 ms | 70 ms | **MultiforaDB ×3** | +| countDocuments({}) | 2.6 ms | 11.3 ms | **MultiforaDB ×4** | +| findOne({_id}) | 0.60 ms | 0.61 ms | parity | +| findOne indexed | 0.54 ms | 1.1 ms | **MultiforaDB ×2** | +| range-scan count | 11.2 ms | 12.5 ms | **MultiforaDB** | +| sort + limit(20) on `_id` | 1.5 ms | 2.0 ms | **MultiforaDB** | +| projection + limit(1000) | 3.6 ms | 4.3 ms | **MultiforaDB** | +| aggregate $group | 7.4 ms | 13.4 ms | **MultiforaDB ×2** | +| updateOne({_id}) ×50 | 0.15 ms | 0.21 ms | **MultiforaDB** | +| updateMany (65 docs) | 1.9 ms | 5.9 ms | **MultiforaDB ×3** | +| deleteOne + insert | 0.63 ms | 4.9 ms | **MultiforaDB ×8** | +| concurrent durable writes, 32 clients | 32,817/s | 3,765/s | **MultiforaDB ×9** | +| server RSS after the load | 1.06 GB | 1.18 GB | MultiforaDB | +| kill -9 → reopen | 0.3 s | 1.3 s | **MultiforaDB ×4** | +| db on disk | 914 MB | 85 MB | **mongodb ×11** | -The engine now holds every document as canonical BSON bytes in a -segmented per-collection slab (no per-document arena, no second Pair-tree -copy), which is why RSS is a quarter of MongoDB's and the range scan — -matching against the bytes directly, skipping fields by length — runs at -parity. The log is LZ4-compressed in 256 KiB blocks, so the on-disk size -matches MongoDB's compressed files. Bulk insert is compress-bound (the -LZ4 codec runs at ~1.7 GB/s; deflate would cap writes below the insert -rate, which is why the roadmap chose LZ4). +Each cell is the best of three runs of the same suite, for both servers. That +is not fussiness: two consecutive runs of the *same* binary moved the sub-10 ms +rows by 27–51% on this machine, so a single run's ratios say more about the +minute they were taken in than about either database. Treat differences under +about 1.5× as noise. -Reproduce the table with `bash tests/e2e/compare-run.sh 1g 16k`; the -pre-tree baseline is recorded in `tests/e2e/results/phase1.txt`, and the -runs with the B+tree, ordered `_id` index, compressed log and byte -storage (roadmap items 1–4) in `tests/e2e/results/phase2.txt` through -`phase5.txt`. +Two rows in that table changed direction with the mmap foundation and are +worth being explicit about. + +**`db on disk`** was 97 MB against mongod's 91 MB when the log was the only +copy of the data and LZ4 compressed it. The data file does not compress +documents: 65,536 × 16 KiB documents now occupy 914 MB of allocated blocks +(`du`; `ls` shows 1.09 GB, the difference being the sparse tail the file is +grown into) against mongod's compressed 85 MB. Per-page or per-extent +compression is the fix, and it is not in M0. If you are comparing against a +report from before this was written, note that the row used to measure the log +file alone — which said 20 MB for a 1 GB collection, because the documents had +moved to `.data`. Fixed in `compare-run.sh`. + +**`server RSS`** is measured right after writing the whole dataset, so it +reflects a bulk load having dirtied every page it wrote, not steady state. The +number that speaks to the architecture is resident memory *after a reopen*: +237 MB for a 21.5 GB collection (see the section above). Before M0 the same +measurement was 523 MB for a 512 MB collection, because recovering each +document's `_id` read every document at open. + +The range scan runs at parity because matching happens against the stored BSON +bytes directly, skipping fields by length, with no per-document arena and no +second Pair-tree copy. The log is still LZ4-compressed in 256 KiB blocks; since +it is now truncated at every checkpoint, its size no longer tracks the +database's. + +Reproduce the whole thing — main suite, concurrency sweep and the meta rows — +with `bash tests/e2e/bench-run.sh 1g 16k "1 4 8 16 32"`, which writes a +timestamped report to `tests/e2e/results/` and diffs it against the last one. +The pre-tree baseline is in `tests/e2e/results/phase1.txt`; the runs with the +B+tree, ordered `_id` index, compressed log and byte storage (roadmap items +1–4) in `phase2.txt` through `phase5.txt`; the all-in-RAM engine this table's +predecessor measured in `phase8.txt`; and the mmap foundation's own gate +results, including what each of those rows cost or gained, in +`m0-gates.txt`. ### What is left (highest impact first) diff --git a/tests/e2e/big.js b/tests/e2e/big.js index e76e505..29024bd 100644 --- a/tests/e2e/big.js +++ b/tests/e2e/big.js @@ -119,10 +119,13 @@ async function stopServer(sig = 'SIGKILL') { serverDead = true; server = null; } -async function rssMB() { +// Synchronous on purpose. The async version sampled inside setInterval and a +// short run could finish before any sample landed, reporting RSS 0 -- which reads +// as "measured and tiny" rather than "not measured". +function rssMB() { if (!server) return 0; try { - const out = (await new Promise((r) => require('child_process').exec(`ps -o rss= -p ${server.pid}`, (e, so) => r(so || '')))).trim(); + const out = require('child_process').execSync(`ps -o rss= -p ${server.pid}`, { encoding: 'utf8' }).trim(); return Math.round(Number(out) / 1024); } catch { return 0; } } @@ -164,9 +167,9 @@ async function main() { const payload = 'x'.repeat(payloadLen); const { ObjectId } = require('mongodb'); const t0 = Date.now(); - const logSamples = [{ t: 0, size: fs.existsSync(DBFILE) ? fs.statSync(DBFILE).size : 0, rss: 0 }]; - const sampler = setInterval(async () => { - logSamples.push({ t: Date.now() - t0, size: fs.existsSync(DBFILE) ? fs.statSync(DBFILE).size : 0, rss: await rssMB() }); + const logSamples = [{ t: 0, size: fs.existsSync(DBFILE) ? fs.statSync(DBFILE).size : 0, rss: rssMB() }]; + const sampler = setInterval(() => { + logSamples.push({ t: Date.now() - t0, size: fs.existsSync(DBFILE) ? fs.statSync(DBFILE).size : 0, rss: rssMB() }); }, 2000); // Rate curve: avg MB/s per progress chunk, to show how throughput changes @@ -215,12 +218,18 @@ async function main() { row('throughput', `${(bytes / 1e6 / (insertMs / 1000)).toFixed(1)} MB/s (${(inserted / (insertMs / 1000)).toFixed(0)} docs/s)`); row('rate curve (MB/s per chunk)', rates.join(' → ') || 'n/a'); - const compactions = logSamples.filter((s, i) => i > 0 && s.size < logSamples[i - 1].size - 2 * 1024 * 1024).length; + // A shrinking log means a *checkpoint* now, not a compaction: the checkpoint + // publishes the data file and truncates the log to its header. Compaction + // (which rewrites the data file) leaves no signature in the log's size, so + // this counter cannot see it and must not claim to. + const truncations = logSamples.filter((s, i) => i > 0 && s.size < logSamples[i - 1].size - 2 * 1024 * 1024).length; const sizes = logSamples.map((s) => s.size); row('log file size (min → final)', `${fmt(Math.min(...sizes))} → ${fmt(sizes[sizes.length - 1])}`); - row('compaction events observed', compactions, '(log shrank by >2MB between samples)'); + const dataFileSize = fs.existsSync(DBFILE + '.data') ? fs.statSync(DBFILE + '.data').size : 0; + row('data file size', fmt(dataFileSize)); + row('log truncations (checkpoints)', truncations, '(log shrank by >2MB between samples)'); const peakRss = Math.max(...logSamples.map((s) => s.rss)); - row('peak server RSS', `${peakRss} MB`, '(in-memory engine: docs live in RAM)'); + row('peak server RSS', `${peakRss} MB`, '(a write touches its pages; see the after-reopen row)'); // ---- find / read ------------------------------------------------------- console.log('\n== find / read on full dataset =='); @@ -298,7 +307,12 @@ async function main() { await client.close(); await stopServer('SIGKILL'); const reopenMs = await startServer(); - row('kill -9 then reopen (replay of full log)', `${(reopenMs / 1000).toFixed(1)} s`); + row('kill -9 then reopen', `${(reopenMs / 1000).toFixed(1)} s`); + // The point of the mmap work: what an open pays for is the working set, not + // the size of the database. Sampled right after the server answers its first + // ping, before any query has touched a document. + const rssAfterReopen = rssMB(); + row('RSS after reopen (working set)', `${rssAfterReopen} MB`); const c2 = new MongoClient(URL, { serverSelectionTimeoutMS: 10000 }); await c2.connect(); const db2 = c2.db('big'); @@ -335,10 +349,13 @@ async function main() { await stopServer('SIGKILL'); console.log('\n== summary =='); - console.log(` multiforadb handles a ${fmt(bytes)} collection fully in RAM (RSS ${peakRss} MB)`); + // Values captured while the server was alive: by the time the summary prints, + // it has been stopped and its files removed, so sampling here reports zero. + console.log(` multiforadb handles a ${fmt(bytes)} collection in a ${fmt(dataFileSize)} data file`); + console.log(` peak RSS during load ${peakRss} MB; after reopen ${rssAfterReopen} MB — the working set, not the data size`); console.log(` insert: ${(bytes / 1e6 / (insertMs / 1000)).toFixed(1)} MB/s — fsync per write is by design (crash safety)`); - if (compactions > 0) { - console.log(` ${compactions} compaction rewrites observed: every 16MB of writes rewrites the whole log — for multi-GB loads the cumulative rewrite traffic dominates`); + if (truncations > 0) { + console.log(` ${truncations} checkpoints reclaimed the log during the load, which is why it ends at ${fmt(sizes[sizes.length - 1])} rather than ${fmt(bytes)}`); } console.log('BIG_OK'); } diff --git a/tests/e2e/compare-run.sh b/tests/e2e/compare-run.sh index 40687c4..a27b720 100644 --- a/tests/e2e/compare-run.sh +++ b/tests/e2e/compare-run.sh @@ -71,7 +71,12 @@ wait_ready "mongodb://127.0.0.1:$MFDB_PORT" || { echo "multiforadb never became node tests/e2e/compare.js --url "mongodb://127.0.0.1:$MFDB_PORT" --label multiforadb --size "$SIZE" --doc-size "$DOC" \ > "$CMPDIR/mfdb-report.txt" 2>&1 || { echo "multiforadb bench failed:"; tail -5 "$CMPDIR/mfdb-report.txt"; } MFDB_RSS=$(ps -o rss= -p $MFDB_PID | awk '{printf "%.0f", $1/1024}') -MFDB_DISK=$(du -sm "$MFDB_LOG" | awk '{print $1}') +# Both files. The documents live in "$MFDB_LOG".data since the mmap foundation +# landed, and the log is truncated at every checkpoint -- so measuring the log +# alone reported 20 MB for a 1 GB collection, against a `du` over mongod's whole +# dbpath. `du` rather than `ls`: the data file is grown with setLength and is +# sparse until written, and allocated blocks are what actually costs disk. +MFDB_DISK=$(du -scm "$MFDB_LOG" "$MFDB_LOG.data" 2>/dev/null | tail -1 | awk '{print $1}') echo; echo "### multiforadb kill -9 + reopen (replay)" kill -9 $MFDB_PID; wait $MFDB_PID 2>/dev/null diff --git a/tests/e2e/results/bench-20260803-223947.txt b/tests/e2e/results/bench-20260803-223947.txt new file mode 100644 index 0000000..0e59c1f --- /dev/null +++ b/tests/e2e/results/bench-20260803-223947.txt @@ -0,0 +1,36 @@ +# multiforadb vs MongoDB benchmark +# date: 2026-08-03T19:42:00Z git: 5228ed7+dirty +# args: size=1g doc-size=16k wc=j clients=1 4 8 16 32 per-client=2000 +# reproduce: bash tests/e2e/bench-run.sh 1g 16k "1 4 8 16 32" + +[main] +insertOne (sequential) ×200 0.20 ms 4.9 ms +bulk insert throughput 463.4 MB/s 581.3 MB/s +docs loaded 65,536 65,536 +createIndex({k: 1}) 27.9 ms 78.1 ms +countDocuments({}) 2.6 ms 12.8 ms +findOne({_id: }) 0.64 ms 0.72 ms +findOne({k: 500}) (indexed) 0.59 ms 1.6 ms +find({p: {$gte,$lt}}).count() (scan) 14.1 ms 12.5 ms +find({}).sort({_id:-1}).limit(20) 1.5 ms 2.4 ms +find({}, {proj}).limit(1000) 3.6 ms 4.7 ms +aggregate $group by k 8.4 ms 13.4 ms +updateOne({_id}) ×50 0.15 ms 0.23 ms +updateMany({k: 7}, {$inc}) 1.9 ms 7.0 ms +deleteOne({_id}) + insertOne 0.63 ms 5.8 ms +node client RSS 153 MB 155 MB + +[concurrency] +clients 1 8354 204 41.0x +clients 4 465 0.0x +clients 8 978 0.0x +clients 16 1877 0.0x +clients 32 3765 0.0x + +[meta] +mfdb_rss_mb 1059 +md_rss_mb 1203 +mfdb_reopen 0.3s +md_reopen 1.3s +mfdb_disk_mb 20MB +md_disk_mb 83MB diff --git a/tests/e2e/results/bench-20260803-225128.txt b/tests/e2e/results/bench-20260803-225128.txt new file mode 100644 index 0000000..66d8d8a --- /dev/null +++ b/tests/e2e/results/bench-20260803-225128.txt @@ -0,0 +1,36 @@ +# multiforadb vs MongoDB benchmark +# date: 2026-08-03T19:53:01Z git: 4b70ce6+dirty +# args: size=1g doc-size=16k wc=j clients=1 4 8 16 32 per-client=2000 +# reproduce: bash tests/e2e/bench-run.sh 1g 16k "1 4 8 16 32" + +[main] +insertOne (sequential) ×200 0.22 ms 4.4 ms +bulk insert throughput 555.5 MB/s 738.5 MB/s +docs loaded 65,536 65,536 +createIndex({k: 1}) 28.0 ms 70.0 ms +countDocuments({}) 2.8 ms 12.3 ms +findOne({_id: }) 0.60 ms 0.61 ms +findOne({k: 500}) (indexed) 0.54 ms 1.1 ms +find({p: {$gte,$lt}}).count() (scan) 11.2 ms 13.2 ms +find({}).sort({_id:-1}).limit(20) 1.5 ms 2.0 ms +find({}, {proj}).limit(1000) 3.8 ms 4.3 ms +aggregate $group by k 7.4 ms 15.8 ms +updateOne({_id}) ×50 0.18 ms 0.21 ms +updateMany({k: 7}, {$inc}) 1.9 ms 5.9 ms +deleteOne({_id}) + insertOne 0.67 ms 5.1 ms +node client RSS 163 MB 155 MB + +[concurrency] +clients 1 8373 218 38.4x +clients 4 20880 491 42.5x +clients 8 26349 964 27.3x +clients 16 29594 1710 17.3x +clients 32 32235 3389 9.5x + +[meta] +mfdb_rss_mb 1059 +md_rss_mb 1242 +mfdb_reopen 0.3s +md_reopen 1.3s +mfdb_disk_mb 20MB +md_disk_mb 87MB diff --git a/tests/e2e/results/bench-20260803-225322.txt b/tests/e2e/results/bench-20260803-225322.txt new file mode 100644 index 0000000..29d08a6 --- /dev/null +++ b/tests/e2e/results/bench-20260803-225322.txt @@ -0,0 +1,36 @@ +# multiforadb vs MongoDB benchmark +# date: 2026-08-03T19:54:54Z git: 4b70ce6+dirty +# args: size=1g doc-size=16k wc=j clients=1 4 8 16 32 per-client=2000 +# reproduce: bash tests/e2e/bench-run.sh 1g 16k "1 4 8 16 32" + +[main] +insertOne (sequential) ×200 0.21 ms 4.4 ms +bulk insert throughput 550.4 MB/s 721.6 MB/s +docs loaded 65,536 65,536 +createIndex({k: 1}) 28.9 ms 72.5 ms +countDocuments({}) 3.6 ms 11.3 ms +findOne({_id: }) 0.80 ms 0.76 ms +findOne({k: 500}) (indexed) 0.73 ms 5.4 ms +find({p: {$gte,$lt}}).count() (scan) 14.5 ms 15.4 ms +find({}).sort({_id:-1}).limit(20) 1.9 ms 2.6 ms +find({}, {proj}).limit(1000) 3.8 ms 4.9 ms +aggregate $group by k 11.2 ms 15.5 ms +updateOne({_id}) ×50 0.17 ms 0.21 ms +updateMany({k: 7}, {$inc}) 2.0 ms 6.1 ms +deleteOne({_id}) + insertOne 0.69 ms 4.9 ms +node client RSS 150 MB 156 MB + +[concurrency] +clients 1 7709 229 33.7x +clients 4 20122 493 40.8x +clients 8 26210 977 26.8x +clients 16 27794 1895 14.7x +clients 32 32817 3249 10.1x + +[meta] +mfdb_rss_mb 1060 +md_rss_mb 1178 +mfdb_reopen 0.3s +md_reopen 1.3s +mfdb_disk_mb 20MB +md_disk_mb 85MB diff --git a/tests/e2e/results/bench-latest.txt b/tests/e2e/results/bench-latest.txt index c661221..29d08a6 100644 --- a/tests/e2e/results/bench-latest.txt +++ b/tests/e2e/results/bench-latest.txt @@ -1,36 +1,36 @@ -# mongo-lite vs MongoDB benchmark -# date: 2026-08-03T08:53:32Z git: c8d547f+dirty +# multiforadb vs MongoDB benchmark +# date: 2026-08-03T19:54:54Z git: 4b70ce6+dirty # args: size=1g doc-size=16k wc=j clients=1 4 8 16 32 per-client=2000 # reproduce: bash tests/e2e/bench-run.sh 1g 16k "1 4 8 16 32" [main] -insertOne (sequential) ×200 0.20 ms 14.8 ms -bulk insert throughput 732.4 MB/s 546.2 MB/s +insertOne (sequential) ×200 0.21 ms 4.4 ms +bulk insert throughput 550.4 MB/s 721.6 MB/s docs loaded 65,536 65,536 -createIndex({k: 1}) 74.4 ms 83.6 ms -countDocuments({}) 3.1 ms 12.4 ms -findOne({_id: }) 0.59 ms 0.61 ms -findOne({k: 500}) (indexed) 0.56 ms 0.94 ms -find({p: {$gte,$lt}}).count() (scan) 13.0 ms 15.0 ms -find({}).sort({_id:-1}).limit(20) 2.2 ms 2.2 ms -find({}, {proj}).limit(1000) 3.5 ms 4.4 ms -aggregate $group by k 8.3 ms 13.2 ms -updateOne({_id}) ×50 0.14 ms 0.20 ms -updateMany({k: 7}, {$inc}) 1.9 ms 6.4 ms -deleteOne({_id}) + insertOne 0.58 ms 4.9 ms -node client RSS 158 MB 157 MB +createIndex({k: 1}) 28.9 ms 72.5 ms +countDocuments({}) 3.6 ms 11.3 ms +findOne({_id: }) 0.80 ms 0.76 ms +findOne({k: 500}) (indexed) 0.73 ms 5.4 ms +find({p: {$gte,$lt}}).count() (scan) 14.5 ms 15.4 ms +find({}).sort({_id:-1}).limit(20) 1.9 ms 2.6 ms +find({}, {proj}).limit(1000) 3.8 ms 4.9 ms +aggregate $group by k 11.2 ms 15.5 ms +updateOne({_id}) ×50 0.17 ms 0.21 ms +updateMany({k: 7}, {$inc}) 2.0 ms 6.1 ms +deleteOne({_id}) + insertOne 0.69 ms 4.9 ms +node client RSS 150 MB 156 MB [concurrency] -clients 1 8435 232 36.4x -clients 4 22480 491 45.8x -clients 8 25920 966 26.8x -clients 16 31079 1842 16.9x -clients 32 34041 3632 9.4x +clients 1 7709 229 33.7x +clients 4 20122 493 40.8x +clients 8 26210 977 26.8x +clients 16 27794 1895 14.7x +clients 32 32817 3249 10.1x [meta] -ml_rss_mb 553 -md_rss_mb 1486 -ml_reopen 0.8s +mfdb_rss_mb 1060 +md_rss_mb 1178 +mfdb_reopen 0.3s md_reopen 1.3s -ml_disk_mb 97MB -md_disk_mb 106MB +mfdb_disk_mb 20MB +md_disk_mb 85MB diff --git a/tests/e2e/results/m0-gates.txt b/tests/e2e/results/m0-gates.txt new file mode 100644 index 0000000..e9880ed --- /dev/null +++ b/tests/e2e/results/m0-gates.txt @@ -0,0 +1,156 @@ +# M0 gate results — mmap + WAL storage foundation (PLAN D7) +# +# Machine: Apple Silicon, macOS 25.5.0, 16 KiB system pages, APFS. +# Server: MultiforaDB at the commit named below, ReleaseFast. +# Driver: mongodb@7.5.0 (pinned in tests/e2e/package-lock.json). +# mongod: 8.3.7, for the parity rows. +# +# Read this alongside PLAN.md §5. Every number here is reproducible with the +# command printed under it; where a gate was not met, the number is recorded as +# measured and the reason is stated rather than the workload being tuned until +# it passed. + +[D7.1] unit tests, ReleaseFast and ReleaseSafe + zig build test -Doptimize=ReleaseFast 122/122 pass + zig build test -Doptimize=ReleaseSafe 122/122 pass + zig build fuzz fuzz_split, spill, spill2, stress clean + Both modes matter: `protect_stable` (the mprotect belt over the published + image) is comptime-off in ReleaseFast, and `std.posix.mprotect` not existing + in Zig 0.16 was a ReleaseSafe-only compile error. + +[D7.2] end-to-end suites — all green + reproduce: zig-out/bin/multiforadb --port 27020 --db /tmp/mfdb-e2e.log \ + --ttl-sweep-secs 1 & then node tests/e2e/ + e2e.js CRUD, operators, aggregate, errors 35/35 + e2e3.js secondary indexes: unique/sparse/compound 16/16 + e2e4.js TTL indexes: expiry and rejected specs 17/17 + e2e5.js miscellaneous command surface 3/3 + e2e2.js concurrent, 4 writers + 4 readers 2/2 + e2e2.js crash-a / crash-b (kill -9, then verify) 1/1 then 3/3 + e2e6.js compaction and log growth (own server) 72/72 + +[D7.3] large-collection smoke — big.js + reproduce: node tests/e2e/big.js --size 20g --doc-size 16k + 20g run 4g run + documents 1,310,720 262,144 (x 16 KiB) + live data 21.47 GB 4.29 GB + data file 21.75 GB 4.35 GB (+1.3%, +1.4%) + log file, min -> final 16 B -> 2.5 MB 16 B -> 7.7 MB + bulk insert 411.0 MB/s 560.3 MB/s + peak server RSS during load 18,492 MB 3,318 MB + kill -9 then reopen 0.5 s 0.5 s + RSS after reopen 237 MB (1.1%) 130 MB (3.0%) + count after restart 1,310,720 OK 262,144 OK + last doc intact after restart payload 16254B OK + kill -9 after 200 acked writes 200/200 200/200 + This is the milestone's central claim, measured: an open costs the working + set, not the size of the database, and it no longer replays the whole log. + Reopen is flat at 0.5 s from 4 GB to 21.5 GB, and RSS after reopen falls as a + *share* of the data as the database grows — which is what "RSS = working set" + has to mean to be worth anything. + Before M0 the same shape of run reported 523 MB resident after reopen for a + 512 MB database, because recovering each document's `_id` meant reading every + document at open. That scan is what the mmap foundation deleted. + Two other things the 20g run shows, honestly. Peak RSS during the *load* is + 18.5 GB: writing 21 GB touches 21 GB of pages and the kernel keeps them until + it needs the memory, so a bulk load is not where the mmap win shows up. And a + cold full scan of 21 GB costs ~55 s — countDocuments, the range scan and the + unindexed updateMany all land there, ~390 MB/s off the disk — which is correct + and is the reason M1's cursors and index-only paths matter. + +[D7.4] churn gate — doc/slab garbage (PLAN amendment A2 retargets D6.2 here) + 40,000 x 16 KiB documents (655 MB live), one secondary index, steady state + measured as data-file size / live data after the ratio stopped moving. + before the fixes after + delete half and refill, 6 rounds 4.10x, climbing 1.65x, flat + random $set over 5x the collection 3.58x, climbing 2.47x, flat + "Climbing" is the whole finding: nothing was being reclaimed at all. Three + bugs, all fixed in `db/pager: reclaim what churn abandons` — a compaction + trigger that had read `log.data_bytes` since before the log was truncated at + every checkpoint, a numeric stable mark that made recycled pages look + published, and a first-fit free list that let one-page requests dismantle + the extents. A fourth fix (a rebuild publishes twice, so the space it frees + is reusable immediately) took the interleaved case from 3.58x to 2.47x. + The gate hoped for ~1.3x and this is above it, structurally: a rebuild needs + a whole second copy of the live data before the first can be freed, and + two-generation retention holds the old copy through two more publishes. The + gate's purpose was to decide whether doc-level free lists are needed after + M0. They are; that is an M1 item. What M0 owes is a bound, and there is one. + +[D7.5] benchmark parity vs the pre-mmap engine + reproduce: bash tests/e2e/bench-run.sh 1g 16k "1 4 8 16 32" + baseline: tests/e2e/results/phase8.txt (c8d547f, all-in-RAM engine) + M0: three runs, tests/e2e/results/bench-2026080{3-223947,3-225128,3-225322}.txt + Read the range, not a single delta. Two consecutive runs of the *same* binary + moved the sub-10 ms rows by 27-51% on this machine, so any one comparison + reads whatever the noise did that minute. Best-of-three against the baseline: + row pre-mmap M0 min..max best + insertOne (sequential) x200 0.20 ms 0.20..0.22 ms +0% + bulk insert throughput 732.4 MB/s 463.4..555.5 MB/s -24% + createIndex({k: 1}) 74.4 ms 27.9..28.9 ms -62% + countDocuments({}) 3.1 ms 2.6..3.6 ms -16% + findOne({_id: }) 0.59 ms 0.60..0.80 ms +2% + findOne({k: 500}) (indexed) 0.56 ms 0.54..0.73 ms -4% + find({p: {$gte,$lt}}).count() (scan) 13.0 ms 11.2..14.5 ms -14% + find({}).sort({_id:-1}).limit(20) 2.2 ms 1.5..1.9 ms -32% + find({}, {proj}).limit(1000) 3.5 ms 3.6..3.8 ms +3% + aggregate $group by k 8.3 ms 7.4..11.2 ms -11% + updateOne({_id}) x50 0.14 ms 0.15..0.18 ms +7% + updateMany({k: 7}, {$inc}) 1.9 ms 1.9..2.0 ms +0% + deleteOne({_id}) + insertOne 0.58 ms 0.63..0.69 ms +9% + node client RSS 158 MB 150..163 MB -5% + concurrent durable insertOne, docs/s: + clients 1 8,435 7,709..8,373 -1% + clients 4 22,480 20,122..20,880 -7% + clients 8 25,920 26,210..26,349 +2% + clients 16 31,079 27,794..29,594 -5% + clients 32 34,041 32,235..32,817 -4% + One reproducible regression: bulk insert, -24%, steady across all three runs. + This is PLAN risk 1 exactly as written -- document bytes now reach the disk + uncompressed in the data file on top of the LZ4 log, and that writeback + bandwidth is new. Every other row is inside the run-to-run spread. The gate + as stated was "no phase8 row regressed"; it is met for the read and latency + rows and not for bulk load, which is the trade the milestone makes and which + risk 1 anticipated. Untried mitigations, in the order worth trying: + MADV_HUGEPAGE / a larger growth chunk, and not writing doc bytes twice + (log and slab) for a bulk path that could log an extent reference instead. + createIndex at -62% is the other reproducible number, and it comes from the + same change: a bulk build now packs pages in the mapping instead of growing + an ArrayList. + +[D7.6] MongoDB spec-test scorecard + reproduce: bash tests/spec/fetch.sh && node tests/spec/run.js --scorecard + total 131 pass 161 fail 195 skip 175 files 0 errored + Byte-identical to the scorecard recorded before the storage rewrite. That is + the intended result and it is worth stating plainly: M0 replaced the document + store, the node arena and the overflow slab, added copy-on-write, a watermark + and a checkpoint, and moved every index leaf's payload -- and changed no + observable CRUD or aggregate semantics. + +# WHAT THE GATES FOUND +# Five bugs, none of which any unit test or e2e suite had reached: +# 1. The compaction trigger had been dead since commit 14 (log truncation +# zeroes the counter it gated on). Nothing reclaimed doc-slab garbage. +# 2. `page_mut_cow` asked `p >= stable_pages`, which is false for a recycled +# page, so every write to one copied and freed it again. +# 3. First fit let copy-on-write's one-page requests shave the extents the +# doc slab needs, so the free list drained every generation and the file +# grew anyway. +# 4. A rebuild's freed space took two unrelated checkpoints to become +# reusable, so the next rebuild grew the file instead of reusing it. +# 5. `reserve_pages`' promise was a single counter on the pager. Two upserts +# on different collections release it independently, so the first to +# finish revoked the second's promise mid-write -- aborting the server at +# four concurrent clients, on the first concurrent benchmark since the +# data file landed. PLAN risk 3, whose mitigation was never built. +# Four of the five are invisible without a workload that runs long enough to +# reach a steady state, or wide enough to have two writers. That is the +# argument for keeping both the churn gate and the concurrent benchmark in the +# gate list rather than treating them as optional. +# +# ONE COMPATIBILITY GAP FOUND, NOT FIXED (out of M0's scope, storage-only) +# `update.apply` (src/update.zig:18) rejects any update document whose first +# key is not `$`, so replacement-style writes -- replaceOne, findOneAndReplace, +# bulkWrite's replaceOne -- fail with "bad update". Found while building the +# churn workload. It is a CRUD feature, not storage, and it is already part of +# what the 161 spec failures cover.