fix(store): a full disk mid-command leaves the books whole

Max-testing pass on ledger-store, fault injection (rank 1). A full disk
is simulated with no code seam: the test re-runs its own binary as a
child under bash with the file size capped (ulimit -f, SIGXFSZ
ignored), so writes past the cap fail as on a full disk. Two bugs, both
of which left the books unopenable:

1. SQLite side: applying one journal line runs several statements (rows,
   events mirror, last_applied_seq) in autocommit. When the disk filled
   partway, the rows committed but last_applied_seq didn't, and the next
   open's catch-up re-applied the line into "UNIQUE constraint failed:
   entries.id". Each line now applies inside a savepoint: all or
   nothing. (docs/receipt-store.md promised exactly this heal; it
   wasn't atomic.)
2. Journal side: an append that failed partway left a partial line, and
   any later line would land after it, mid-journal. append now cuts the
   file back to its length before the write on any error.

Tests (tests/faults.rs, its own binary because the child re-runs it):
a_full_disk_mid_command_leaves_the_books_whole (the database fills
first) and a_full_disk_mid_append_leaves_only_whole_journal_lines (the
journal writer alone). Both shown to fail without their fix.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GWVo7x3UHw5kwCUaWxiejX
This commit is contained in:
swaitsandClaude Opus 5.5 committed 2026-10-13 17:16:43 -06:00
1 parent 6625c8cea0
commit 02d1f20f90
3 files changed
+243 -3

No files matched your search

+15 -3
View File
@@ -162,9 +162,21 @@ impl JournalWriter {
prev: self.prev_hash.clone(),
};
let bytes = line.to_canonical();
self.file.write_all(bytes.as_bytes())?;
self.file.write_all(b"\n")?;
self.file.sync_all()?;
// A write that fails partway (a full disk) must leave nothing
// behind: cut the file back to where it was. Otherwise a partial
// line stays, and every later line lands after it, mid-journal
// (found by the full-disk fault test, 2026-10-13).
let before = self.file.metadata()?.len();
let written = self
.file
.write_all(bytes.as_bytes())
.and_then(|()| self.file.write_all(b"\n"))
.and_then(|()| self.file.sync_all());
if let Err(e) = written {
let _ = self.file.set_len(before);
let _ = self.file.sync_all();
return Err(e.into());
}
self.prev_hash = sha256_hex(bytes.as_bytes());
self.next_seq += 1;
Ok(line)
+23
View File
@@ -3498,6 +3498,29 @@ pub(crate) fn apply_line_to_sqlite(
conn: &Connection,
line: &JournalLine,
) -> Result<(), StoreError> {
// All or nothing, per line: the rows, the events mirror, and
// last_applied_seq commit together. Without this a write that failed
// partway (a full disk) committed the rows but not last_applied_seq,
// and the next open's catch-up re-applied the line into a UNIQUE
// violation: the books would not open (found by the full-disk fault
// test, 2026-10-13). A savepoint, so it nests inside a caller's
// transaction.
conn.execute_batch("SAVEPOINT apply_line")?;
match apply_line_rows(conn, line) {
Ok(()) => {
conn.execute_batch("RELEASE apply_line")?;
Ok(())
}
Err(e) => {
let _ = conn.execute_batch("ROLLBACK TO apply_line; RELEASE apply_line");
Err(e)
}
}
}
/// The statements behind [`apply_line_to_sqlite`], run inside its
/// savepoint.
fn apply_line_rows(conn: &Connection, line: &JournalLine) -> Result<(), StoreError> {
let p = &line.payload;
match line.line_type.as_str() {
"store.created" => {}