wanderbooks/docs/receipt-store.md

# Receipt & document store + persistence spec

Three pieces: the **journal** (the source of truth), the **blob store**
(the bytes), and **SQLite** (the queryable projection). All immutable-first,
matching the ledger's philosophy: the books never forget, and neither does
the filing cabinet.

## The journal: the source of truth

SQLite is the query engine, not the source of truth. The source of truth is
an **append-only journal of domain events**: one JSON object per line
(JSONL), hash-chained. This is the deliberate answer to "insanely durable,
easily backupable, journable":

- **Durable.** Every state change is an immutable fact appended to
  `journal.jsonl`, fsync'd per append. Each line carries the SHA-256 of the
  previous line's canonical bytes (`prev`), forming a tamper-evident chain:
  bit rot, torn writes, and casual editing are detected on read/verify, not
  discovered at tax time. Honest scope: this detects corruption and casual
  tampering. It does not stop a determined attacker with full write access
  from rewriting the chain — nothing local does.
- **Backupable.** The backup is `cp -r $WANDERBOOKS_DATA /backup`: the
  journal file plus the blob directory. Plain files, no WAL-file timing
  windows, no special backup API. A torn tail from a mid-copy is detectable
  (the last line fails hash/parse) and recoverable (truncate to the last
  good seq — the projection never saw it). The live journal recovers the
  same way on its own: an append cut off by a crash or power loss (bytes
  after the last newline) was never acknowledged, so opening the books
  saves those bytes to `journal.jsonl.torn.<time>`, truncates to the last
  newline, and journals a `journal.tail_repaired` line; `books.digest`
  reports it. Damage before the last newline is corruption and still
  refuses to open. Better: the journal is
  git-friendly, so an optional `wanderbooks backup` that commits it to a
  private git repo gives versioned, distributed, deduplicated backup with
  full history.
- **Journable.** It literally is a journal: `{seq, ts, type, payload, prev}`.
  Greppable, diffable, replayable. The audit trail isn't a feature bolted
  on — it's the storage format.

### Write path

There is exactly one write path: **append to the journal, fsync, then apply
to SQLite.** No dual writes, no background applier thread (the volume
doesn't justify it): the MCP server appends, fsyncs, applies the new lines
to the projection synchronously, and returns. A crash between append and
apply is safe — on startup the applier replays every journal line with
`seq > last_applied_seq`.

### Line format

```json
{"seq": 42, "ts": "2026-10-10T18:20:00Z", "type": "entry.posted",
 "payload": { "...": "..." }, "prev": "abc123..."}
```

- `seq`: 1-based, gapless, assigned by the writer.
- `ts`: UTC, RFC 3339. Recorded, never trusted for ordering (`seq` orders).
- `type`: `account.added`, `entry.posted`, `entry.reversed`,
  `receipt.attached`, `receipt.linked`, `receipt.unlinked`,
  `document.filed`, `attachment.withdrawn` (plus the A/R, A/P, asset,
  period, payroll, and class events listed on `JournalLine`).
- `payload`: the full domain facts (account fields, entry with lines, blob
  sha, ...). Everything the applier needs; nothing it must look up elsewhere.
- `prev`: hex SHA-256 of the previous line's canonical JSON (keys sorted, no
  whitespace). The genesis line uses `"prev": "000...0"` (64 zeros).

### Drafts are journaled (durable)

Draft entries are durable: `entry.created` and `entry.line_added` journal
lines record the draft as the agent builds it, so drafts (and receipts
attached to them) survive server restarts. Drafts never touch the SQLite
projection until posted — reports only see posted entries. A draft that
never posts leaves its `created`/`line_added` lines in the journal, which
is correct: the journal records what happened, including abandoned work.

### Verification

`wanderbooks verify` (a `ledger-store` API, later a CLI subcommand):
re-read the journal, recompute the hash chain, replay every line into a fresh
in-memory `Ledger`, and compare its reports against the SQLite projection's.
Any mismatch is a loud failure. Run it on startup (cheap at this scale) and
before backups.

## Blob store

- **Content-addressed.** The key is SHA-256 of the exact bytes. Path:
  `$DATA/blobs/<first2>/<next2>/<full-hex>`. Same bytes, same path, forever.
- **Immutable.** Blobs are never modified or deleted by the application. A
  retention/deletion policy is an explicit future human decision, not an
  accident of implementation.
- **Deduplicated.** Attaching the same receipt twice stores one blob and two
  attachment rows. `attach_receipt` / `file_document` return
  `deduplicated: true` when the blob already existed.
- **Limits.** 25 MB per file by default, configurable. MIME type is accepted
  from the client and stored; server-side sniffing is future work.
- **Metadata** lives in SQLite (`blobs` table), not in the filesystem:
  `sha256 PK, size_bytes, mime_type, original_name, created_at`.

## Attachments & documents

- `attachments`: `id PK, blob_sha FK, entry_id FK NULL, category,
  description, created_at`.
  - `entry_id` is nullable: a receipt can be filed before its journal entry
    exists, then linked with `books.link_receipt`.
  - `category` mirrors today's Drive layout: `formation`, `legal`, `tax`,
    `accounting` (today's 01 Formation / 02 Legal / 03 Tax / 04 Accounting),
    plus `receipts`.
- Documents filed via `books.file_document` and receipts via
  `books.attach_receipt` share the blob store; they differ only in metadata.
- **Receipts change by events, never by edits.** `receipt.unlinked`
  (`attachment_id`, `entry_id`, `reason`) detaches a receipt from its entry;
  `attachment.withdrawn` (`attachment_id`, `reason`) retires an unlinked
  receipt or a document (`withdrawn_at`, `withdrawn_reason` columns, schema
  v5). Linking requires an unlinked, non-withdrawn attachment, so evidence
  only leaves an entry through a recorded unlink. Blobs are untouched: a
  withdrawn file is still hash-verified and readable, and the journal shows
  when it was filed, linked, unlinked, and withdrawn, and why.
- The store is policy-free: it records attachments faithfully. Whether a
  posting *requires* a receipt is decided by the MCP layer's receipt policy
  (`docs/mcp-interface.md`), which reads the `attachments` table.

## `ledger-store`: the SQLite projection

SQLite is the read-optimized projection of the journal. Queries, reports,
the MCP server's reads, and the future GUI all hit SQLite — and SQLite is
**disposable by design**: `rebuild` drops it and replays the journal.
Suspected corruption is fixed by rebuild, never by forensics.

- **Location.** `$WANDERBOOKS_DATA/wanderbooks.db`, falling back to
  `~/.local/share/wanderbooks/wanderbooks.db`. The journal lives next to it
  at `$WANDERBOOKS_DATA/journal.jsonl`; blobs under
  `$WANDERBOOKS_DATA/blobs/`. WAL mode on, `synchronous=FULL`,
  `integrity_check` on startup.
- **Schema.** `meta(key, value)` — `schema_version`, `base_currency`,
  `last_applied_seq`; `accounts`, `entries`, `lines`, `blobs`, `attachments`
  as above; `events(seq, type, payload, created_at)` — the applier's output,
  a queryable mirror of the journal stream and the feed the future GUI
  tails. (Poll `WHERE seq > ?`; later SSE over the HTTP transport. This is
  the correct answer to "watch its files to update live" — watch the event
  stream, not the files.)
- **Amounts as TEXT, never REAL.** Unchanged and still load-bearing: a REAL
  column would silently reintroduce binary floats into the money path.
- **Append-only, three layers.** (1) The engine's API never mutates posted
  state. (2) The journal is append-only by file discipline plus the hash
  chain. (3) The applier only ever INSERTs into the projection. The SQLite
  triggers from the earlier draft are deliberately dropped: they protected
  the projection, which is disposable — the journal is the trust anchor, and
  piling mechanisms on the disposable layer adds complexity without adding
  safety. Simplicity is a durability feature too.
- **Migrations.** Numbered SQL files, applied in order, bumping
  `meta.schema_version`. Every migration is tested by rebuilding a
  projection from the journal and comparing reports before/after. A newer
  schema version than the binary understands is a hard error, not a guess.

## Migration from Google Drive

Backfill walks the Drive `Wanderland Labs LLC` folders and files each
document via the equivalent of `books.file_document` with its category.
The Drive folders stay exactly as they are until the capability-parity
cutover rule (see `docs/capability-parity.md`) is satisfied. Migration is an
implementation task, not part of this spec — but the spec is written so the
migration is a straightforward script, not a redesign.