fix(store): a full disk mid-command leaves the books whole
Max-testing pass on ledger-store, fault injection (rank 1). A full disk is simulated with no code seam: the test re-runs its own binary as a child under bash with the file size capped (ulimit -f, SIGXFSZ ignored), so writes past the cap fail as on a full disk. Two bugs, both of which left the books unopenable: 1. SQLite side: applying one journal line runs several statements (rows, events mirror, last_applied_seq) in autocommit. When the disk filled partway, the rows committed but last_applied_seq didn't, and the next open's catch-up re-applied the line into "UNIQUE constraint failed: entries.id". Each line now applies inside a savepoint: all or nothing. (docs/receipt-store.md promised exactly this heal; it wasn't atomic.) 2. Journal side: an append that failed partway left a partial line, and any later line would land after it, mid-journal. append now cuts the file back to its length before the write on any error. Tests (tests/faults.rs, its own binary because the child re-runs it): a_full_disk_mid_command_leaves_the_books_whole (the database fills first) and a_full_disk_mid_append_leaves_only_whole_journal_lines (the journal writer alone). Both shown to fail without their fix. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GWVo7x3UHw5kwCUaWxiejX
This commit is contained in:
1 parent
6625c8cea0
commit
02d1f20f90
3 files changed
+243
-3
No files matched your search
@@ -162,9 +162,21 @@ impl JournalWriter {
|
||||
prev: self.prev_hash.clone(),
|
||||
};
|
||||
let bytes = line.to_canonical();
|
||||
self.file.write_all(bytes.as_bytes())?;
|
||||
self.file.write_all(b"\n")?;
|
||||
self.file.sync_all()?;
|
||||
// A write that fails partway (a full disk) must leave nothing
|
||||
// behind: cut the file back to where it was. Otherwise a partial
|
||||
// line stays, and every later line lands after it, mid-journal
|
||||
// (found by the full-disk fault test, 2026-10-13).
|
||||
let before = self.file.metadata()?.len();
|
||||
let written = self
|
||||
.file
|
||||
.write_all(bytes.as_bytes())
|
||||
.and_then(|()| self.file.write_all(b"\n"))
|
||||
.and_then(|()| self.file.sync_all());
|
||||
if let Err(e) = written {
|
||||
let _ = self.file.set_len(before);
|
||||
let _ = self.file.sync_all();
|
||||
return Err(e.into());
|
||||
}
|
||||
self.prev_hash = sha256_hex(bytes.as_bytes());
|
||||
self.next_seq += 1;
|
||||
Ok(line)
|
||||
|
||||
@@ -3498,6 +3498,29 @@ pub(crate) fn apply_line_to_sqlite(
|
||||
conn: &Connection,
|
||||
line: &JournalLine,
|
||||
) -> Result<(), StoreError> {
|
||||
// All or nothing, per line: the rows, the events mirror, and
|
||||
// last_applied_seq commit together. Without this a write that failed
|
||||
// partway (a full disk) committed the rows but not last_applied_seq,
|
||||
// and the next open's catch-up re-applied the line into a UNIQUE
|
||||
// violation: the books would not open (found by the full-disk fault
|
||||
// test, 2026-10-13). A savepoint, so it nests inside a caller's
|
||||
// transaction.
|
||||
conn.execute_batch("SAVEPOINT apply_line")?;
|
||||
match apply_line_rows(conn, line) {
|
||||
Ok(()) => {
|
||||
conn.execute_batch("RELEASE apply_line")?;
|
||||
Ok(())
|
||||
}
|
||||
Err(e) => {
|
||||
let _ = conn.execute_batch("ROLLBACK TO apply_line; RELEASE apply_line");
|
||||
Err(e)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// The statements behind [`apply_line_to_sqlite`], run inside its
|
||||
/// savepoint.
|
||||
fn apply_line_rows(conn: &Connection, line: &JournalLine) -> Result<(), StoreError> {
|
||||
let p = &line.payload;
|
||||
match line.line_type.as_str() {
|
||||
"store.created" => {}
|
||||
|
||||
Reference in new issue
Block a user