db/storage: close three data-loss paths in commit and compaction
Follow-up hardening on the group-commit work fromecd28d9andc8d547f. The signal -> broadcast fix and the append-drained wakeup under commit_lock were correct but incomplete; each of the three defects below could lose or corrupt data that had already been acknowledged. - log_append's cleanup defer took commit_lock with `lock(...) catch {}` and then unlocked unconditionally. Mutex.lock is Cancelable!void and Mutex.unlock treats an already-unlocked mutex as `unreachable`, so a cancellation there (client disconnect, shutdown) released a mutex the fiber never held: a panic in ReleaseSafe and silent memory corruption in the default ReleaseFast build. A cleanup path must not be a cancellation point, so it uses lockUncancelable. - Engine.commit waited with `catch return`, which returns *success* from !void. A failed wait therefore told the caller its write was on disk and the dispatch epilogue replied ok without a seal or an fsync -- the same failure classc8d547ffixed, reached through the error path instead of the happy one. The follower wait now propagates; the leader's drain is uncancelable, since once `committing` is set every other writer is parked behind it and the drain is bounded anyway. - Two compactions could run at once. They share one `<log>.tmp` path and each ends in a rename onto the log, so one truncates and rewrites the file the other is about to publish, and then that one renames whatever it finds over the live log. compact now claims an atomic `compacting` slot and a second caller returns; the guard sits on the resource rather than in take_compact, so direct callers (tests included) are covered too. compact_pending became atomic while we were there: note_compact sets it under a *collection* lock and the epilogue read it under none. Also in compaction: the tmp file is opened with a new Log.create that truncates. Log.open keeps an existing file's bytes and only rewinds end_pos to the header, so a longer tmp left by a crashed or retried rewrite kept its tail -- and those trailing blocks are intact and hash-correct, so replay applied them as live records once the rename published the file, resurrecting deleted documents. The `deleteFile ... catch {}` that used to stand in for this is gone, and with it a swallowed error the invariant rested on. The retry loop is bounded at 8 attempts (each one rewrites the whole log before the seq check can reject it, so an unbounded retry livelocks under sustained writes); giving up re-arms the request instead of failing the write. A compaction failure no longer fails the write whose epilogue triggered it: the write is durable by then, so the error is reported and the request re-armed rather than turned into an error the client retries. Assertions: db.zig, commands.zig and storage.zig had none, which is why every defect in this series was found by a stress run rather than at the moment of corruption. src/assert.zig adds an assert that survives ReleaseFast -- std.debug.assert lowers to `unreachable`, which in this project's default build is not a skipped check but a promise to the optimizer, exactly the wrong lowering for a durability invariant that might be false. Eleven of them now cover the commit watermark, the in-flight append count, the compaction snapshot, and the live/dead counters (whose u64 subtraction would otherwise underflow into a live count that suppresses compaction forever). cmd_find asserts that a missing collection implies no matches instead of silently emitting an empty page for a query that did match. Dead state removed: Log.defer_sync was never read (only written once by compact), yet its doc comment instructed callers to follow a defer_sync protocol that no longer exists and has no effect if followed. Engine's begin_batch/end_batch were both `_ = self`, so three call sites announced a batch boundary that wasn't there. Both are gone and the comments now describe the real contract: appends never sync, Log.sync is the only commit point. Tests: Log.create's truncation is pinned by a storage test that replays after reusing a path, and the compaction guard by a db test that drives the flag directly -- both confirmed to fail without their fix. The threaded test added alongside them is a smoke test only, and says so: both races have windows too narrow to hit reliably (compact_snapshot_coll holds each collection's write lock while snapshotting, so two compactions serialize there and an insert cannot re-arm compact_pending meanwhile), and it passes with the guard removed. Includes an unrelated fix that was already in the working tree: slab_append held a pointer into slab.items across an append to that same list, which could dangle after a realloc and corrupt the new segment's start offset. Verified: unit suite in ReleaseFast/ReleaseSafe/Debug; e2e, e2e2 concurrent, e2e3, e2e4, e2e5, e2e6 (72/72) and the kill -9 crash pair, with no assertion firing anywhere.
This commit is contained in:
@@ -1,5 +1,6 @@
|
||||
// mongo-lite core library. Public entry point for tests and the server.
|
||||
|
||||
pub const assert = @import("assert.zig");
|
||||
pub const bson = @import("bson.zig");
|
||||
pub const wire = @import("wire.zig");
|
||||
pub const commands = @import("commands.zig");
|
||||
@@ -11,6 +12,7 @@ pub const update = @import("update.zig");
|
||||
pub const index = @import("index.zig");
|
||||
|
||||
test {
|
||||
_ = @import("assert.zig");
|
||||
_ = @import("bson.zig");
|
||||
_ = @import("wire.zig");
|
||||
_ = @import("commands.zig");
|
||||
|
||||
Reference in New Issue
Block a user