docs(plan): Q-9 answered (D-61): the reader crate, unit U0d after S4.5
The owner, 2026-10-29, the question tool: the reader (`extract.rs`) moves into a small library crate of its own, `ledger-reader`, that `ledger-store` and `ledger-import` both depend on, so the writer sheds the typesetter and the crate map is layered again; half a day for the implementing team once S4.5 is in. 8.2 gains the unit U0d in the template (its Done-when is `cargo tree`), 8.0.1 its row, 8.0.3 the crate name (D-53's closed `ledger-extract` subprocess stays closed), 10.1 D-61, 10.3 closes Q-9. STATUS.md, HANDOFF.md and architecture.md say the same; the arrow stands as built until U0d lands. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HS8WZziDnyxunC23MUxrLp
This commit is contained in:
1 parent
1111157ed1
commit
25b5258036
4 files changed
+37
-18
No files matched your search
@@ -31,7 +31,7 @@ the specification in `bookspec/` (which never depends on the engine).
|
||||
| `ledger-core` | the semantics engine: money, chart, journal, `Ledger` commands, reports, periods, assets, schedules, parties, A/R and A/P, payroll, projects, settings and conventions, the kernel and its Kani harnesses | nothing |
|
||||
| `ledger-vault` | the TIN vault: XChaCha20-Poly1305 records under the book's key | nothing |
|
||||
| `ledger-config` | the `wanderbooks.toml` codec (adoption in, `books.export_settings` out) and the receipt policy until P4 moves it into the core | `ledger-core` |
|
||||
| `ledger-store` | the source of truth: the hash-chained JSONL journal, its format gate, the SQLite projection (schema 18; a file per schema), the blob store, backups, the access log, `Snapshot` for readers | `ledger-core`, `ledger-import` (the file reader, P6; Q-9) |
|
||||
| `ledger-store` | the source of truth: the hash-chained JSONL journal, its format gate, the SQLite projection (schema 18; a file per schema), the blob store, backups, the access log, `Snapshot` for readers | `ledger-core`, `ledger-import` (the file reader, P6; `ledger-reader` after U0d, D-61) |
|
||||
| `ledger-import` | formats: QuickBooks CSV in; IIF, XLSX, Typst PDFs (statements, the invoice, labels) out | `ledger-core` |
|
||||
| `ledger-package` | the shared read-only view over `Store` or `Snapshot` (`PackageBooks`): the CPA package, the 1099 worksheet, the invoice document, Needs attention | `ledger-core`, `ledger-store`, `ledger-config`, `ledger-import`, `ledger-vault` |
|
||||
| `ledger-mcp` | the books server: the MCP tools (122), resources, the handbook, scopes, the door and the access log, backups' scheduling | all of the above |
|
||||
@@ -50,14 +50,14 @@ test `crates/ledger-core/tests/oracle_gnucash.rs` with `oracle/gnucash/`
|
||||
(`PackageBooks` is the shared view; extraction is in-process in
|
||||
`ledger-import`, P6).
|
||||
|
||||
Under review (Q-9, asked 2026-10-29 when P6 landed): P6 made
|
||||
`ledger-store` depend on `ledger-import` so the writer can extract a
|
||||
file's text at attach time and on the backfill command. That keeps the
|
||||
direction inward (`ledger-import` depends on the core alone and must
|
||||
never depend on the store) but pulls Typst, the fonts and hayro into the
|
||||
store's dependency closure. The plan's recommendation is a small library
|
||||
crate holding the reader that both depend on; until the owner decides,
|
||||
the arrow stands as drawn above.
|
||||
Decided (Q-9 → D-61, 2026-10-29): P6 made `ledger-store` depend on
|
||||
`ledger-import` so the writer can extract a file's text at attach time
|
||||
and on the backfill command. That keeps the direction inward
|
||||
(`ledger-import` depends on the core alone and must never depend on the
|
||||
store) but pulls Typst, the fonts and hayro into the store's dependency
|
||||
closure. Unit U0d (plan 8.2, after S4.5) moves the reader into a small
|
||||
library crate, `ledger-reader`, that both depend on; until it lands the
|
||||
arrow stands as drawn above.
|
||||
|
||||
`ledger-core` itself (`crates/ledger-core/src/`): `money`, `account`,
|
||||
`journal`, `ledger` (the command surface and replay), `kernel` (every
|
||||
|
||||
+2
-2
@@ -18,14 +18,14 @@ You are the engineering manager (EM) of a team implementing the remaining work o
|
||||
|
||||
- `main` is `bfd453b6`. Landed since the hand-off: U0a, U0b, U0c, P7, P9a, P13 steps 1-3, P4b S4.1, S4.2, S4.3 (2026-10-26 to 2026-10-27), then S4.4, S5, S6, S7 and P6 (2026-10-29, merged from the paused branches by the planning session). Live on the owner's books: P0 to P3, Projects, P4b S1 (`document_standing`); everything behind `documents_v2` waits for the owner's upgrade (Q-5).
|
||||
- Nothing is built and unmerged. Branch `ci-fixes` (`76285434`, a fuzz target's renamed path and a regression test) is the one branch left to merge.
|
||||
- Not started: S4.5 (next), P4, P5, P8, P10, P11a (starts now: D-56; step 0 first), P11b, P12, P13 steps 4-6.
|
||||
- Not started: S4.5 (next), U0d (after S4.5), P4, P5, P8, P10, P11a (starts now: D-56; step 0 first), P11b, P12, P13 steps 4-6.
|
||||
- The plan's section 8 was rewritten on 2026-10-26 for this hand-off and revised on 2026-10-28 (section 0, "What changed on 2026-10-28"; `plan/2026-10-28_DRIFT_REPORT.md`). Sections 1 to 7 are the older design and carry strike-throughs (`~~text~~ → D-n`) and "As built" notes where the build or a decision departed; where section 8 and an earlier section disagree, section 8 is right.
|
||||
- Mutation testing is paused until unit P12 (decision D-33). Workers never run `cargo mutants`; each unit writes a "For P12" paragraph in `docs/testing.md` instead (recipe R-G).
|
||||
- The owner's live books are never touched by anyone on this team. The only books a worker uses are test fixtures and throwaway books made by `wanderbooks setup` in a scratch root. A unit's new writes are refused on the owner's books until the owner's format upgrade; merging never waits for that.
|
||||
|
||||
### The paused worktrees (2026-10-27 pause) landed on 2026-10-29
|
||||
|
||||
Nothing is left to resume. The merges went in the 8.0.2 order with the conflicts that paragraph predicted; what P6's first complete gate found is in its As-built block (8.2) and `plan/STATUS.md`. Two lessons for the next pause: a checkpoint whose gate was interrupted is not a reviewed unit (P6's five unreached tests and the write-off interaction cost a day of gates), and in this colocated repo never run a `jj` command while a `git merge` is in progress in the same checkout (jj's auto-import resets git's index; one merge had to be redone in a scratch worktree). Q-5 and Q-8 were answered 2026-10-28 (D-57, D-58); Q-9 (P6's `ledger-store` → `ledger-import` dependency) is with the owner.
|
||||
Nothing is left to resume. The merges went in the 8.0.2 order with the conflicts that paragraph predicted; what P6's first complete gate found is in its As-built block (8.2) and `plan/STATUS.md`. Two lessons for the next pause: a checkpoint whose gate was interrupted is not a reviewed unit (P6's five unreached tests and the write-off interaction cost a day of gates), and in this colocated repo never run a `jj` command while a `git merge` is in progress in the same checkout (jj's auto-import resets git's index; one merge had to be redone in a scratch worktree). Q-5 and Q-8 were answered 2026-10-28 (D-57, D-58); Q-9 on 2026-10-29 (D-61: the reader moves into its own crate, unit U0d, after S4.5).
|
||||
|
||||
### How to run things
|
||||
|
||||
|
||||
+7
-5
@@ -42,12 +42,14 @@ reproduce locally (the hand-written case passes; ten local minutes
|
||||
found nothing); the job now keeps its crash input as an artifact for
|
||||
the next run.
|
||||
|
||||
**Open for the owner (the question tool):** Q-9, whether
|
||||
`ledger-store` keeps its dependency on `ledger-import` (P6 made the
|
||||
writer read a file's text as it is filed).
|
||||
**Answered 2026-10-29 (the question tool):** Q-9, the writer's dependency
|
||||
on the formats crate: the reader moves into a small crate of its own
|
||||
after S4.5 (D-61, unit U0d, half a day). And D-60, the owner's rule for
|
||||
how much checking a merge gets: a compile and the unit's own tests per
|
||||
merge; the exhaustive gates once per batch.
|
||||
|
||||
**Next, in order:** (1) merge `ci-fixes`; (2) S4.5, the last P4b slice;
|
||||
(3) P11a step 0 (the macOS and Windows keyring stores, the Windows data
|
||||
**Next, in order:** (1) S4.5, the last P4b slice (`ci-fixes` merged,
|
||||
`95a54d69`); (2) U0d, the reader crate, beside it; (3) P11a step 0 (the macOS and Windows keyring stores, the Windows data
|
||||
root and pipes, a cross-platform CI build) and the packaging trial,
|
||||
staffed now; (4) P4 (adding `confirmed` to the settlement tools too),
|
||||
then the owner's one format upgrade (D-57), P5, P8, P10; P13 steps 4-6
|
||||
|
||||
+19
-2
@@ -819,6 +819,7 @@ One row per unit of work. A unit is what one engineer takes into one git worktre
|
||||
| U0a | Split `server.rs` by tool group | landed 2026-10-27 (`45396bdc`) | none | U0b, U0c, S4.1, S4.2, S7.1 (core steps only) | none | 3-4 |
|
||||
| U0b | The attention engine over `PackageBooks` | landed 2026-10-27 (`26d3a994`) | none | U0a, U0c, S4.1, S4.2 | none | 4-6 |
|
||||
| U0c | `docs/testing.md` sections for S1-B, S2, S3; `docs/mcp-interface.md` cleanup of stale P4b notes | landed 2026-10-26 (`198aa2a1`) | none | everything | none | 0.5-1 |
|
||||
| U0d | The reader crate: `extract.rs` moves out of `ledger-import` so `ledger-store` sheds the typesetter (Q-9 → D-61) | not started; after S4.5 | P6 (landed) | everything | none | 0.5 |
|
||||
| P4b S4 | `documents_v2`: numbers, terms, line accounts, snapshots, preview and confirm, the issued PDF filed, party email and terms, invoicing settings | in progress: S4.1-S4.3 landed 2026-10-27 (`d19fb464`); S4.4 landed 2026-10-29 (`d329bd6f`, gate 2,883 tests green); S4.5 not started, **next** | S4.1 and S4.2 need nothing; S4.3 needs S4.1, S4.2; S4.4 needs U0a, S4.3; S4.5 needs S4.4 | S5 (after S4.2), S7 (core after S1), P6, P9a, P11a, P13 | `documents_v2` (one upgrade after S7, D-52) | 12-16 |
|
||||
| P4b S5 | Sent tracking | landed 2026-10-29 (`3a3f03d8`; schema 17; law 35) | S1; S4.2 for the defaults (works without it, see the unit) | S4.3 onward, S6 | `documents_v2` | 3-4 |
|
||||
| P4b S6 | Paying on terms: multi-document payments, discounts, the processor fee, bills due, aging exports | landed 2026-10-29 (`be580598`; no schema change; law 36); `confirmed` deferred to P4 | S4.1 (`Terms`), U0a | S5, S7, P6 | `documents_v2` | 8-10 |
|
||||
@@ -887,6 +888,7 @@ These files are edited by nearly every unit. Each has one owner (the EM, or a na
|
||||
|
||||
| Kind | Name | Unit |
|
||||
|---|---|---|
|
||||
| crate | `ledger-reader` (a library; D-53's `ledger-extract` subprocess stays closed) | U0d |
|
||||
| journal feature | `documents_v2` | S4, S5, S6, S7 (one feature for the four; D-52) |
|
||||
| journal feature | `drafts_v2` | P4 |
|
||||
| journal feature | `source_revisions` | P5 |
|
||||
@@ -1285,6 +1287,20 @@ Each unit follows the 8.0.4 template. "A-n" is an owner answer numbered in 8.3.1
|
||||
- The timeline's reversed items are labelled, not struck through: "Payment reversed" for the reversal and "Payment (reversed later)" for the payment, which is what the S2 paragraph names as P12's targets (`crates/ledger-gui/src/pages/invoices.rs`). Verified against the shipped page, not against plan 8.1 S2's "struck through".
|
||||
- Build item 2 assumed the A/R and A/P text already matched S1-S3. It does not in one place: `books.documents`' `party` was said to take a customer's or vendor's name or id, while the handler matches that name (ignoring case) or the document's own id, never the customer's or vendor's id. The doc now says the name; the schema's own description (`crates/ledger-mcp/src/types.rs`, `DocumentsParams::party`) still offers an id, which is a code finding (resolve the id, or narrow the schema) for a unit that may change code, and this one may not.
|
||||
|
||||
#### U0d: the reader crate (Q-9, D-61)
|
||||
|
||||
1. **Status:** not started; after S4.5 (D-61: not urgent, never before S4.5 or P11a step 0).
|
||||
2. **Depends on:** P6 (landed). No owner go-ahead.
|
||||
3. **Feature and schema:** none. No journal line changes; no money moves.
|
||||
4. **Build:**
|
||||
1. New crate `crates/ledger-reader` (name allocated in 8.0.3; a plain library, in-process as today, not D-53's closed subprocess): move `crates/ledger-import/src/extract.rs` there whole (`pdf_text`, `pdf_lines`, `page_png`, `page_count`, `text_of`, `extract`, the caps `MAX_BYTES`, `MAX_PAGES`, `MAX_TEXT_BYTES`, `VERSION`, `ExtractError`, the `catch_unwind` around the reader), with the dependencies only it needs (`hayro`, `hayro-interpret`, `kurbo`, `zip`, `quick-xml`); `ledger-import` re-exports it as `ledger_import::extract` so no caller changes in this step (`crates/ledger-import/src/lib.rs (changed)`).
|
||||
2. `ledger-store` depends on `ledger-reader` instead of `ledger-import` (`crates/ledger-store/Cargo.toml (changed)`; `store.rs` imports move from `ledger_import::extract` to `ledger_reader`); `ledger-mcp`, `ledger-gui`, `ledger-host` keep reading through the one module (via the re-export or the crate directly, whichever `clippy` prefers; one of the two, not both).
|
||||
3. `docs/architecture.md`: the crate table gains the row, the store's row drops `ledger-import`, the "evidence pipeline" section names the crate; the sentence marked "under review (Q-9)" is replaced by D-61. `AGENTS.md`'s architecture paragraph if it lists crates. `docs/testing.md` For-P12 paragraph of P6 points at the new crate path.
|
||||
5. **Files:** owns `crates/ledger-reader/`; touches `crates/ledger-import/{Cargo.toml,src/lib.rs}`, `crates/ledger-store/Cargo.toml`, `crates/ledger-store/src/store.rs` (imports only), `Cargo.toml` (workspace members), the two fuzz targets' manifest, `docs/architecture.md`, `docs/testing.md`.
|
||||
6. **Tests and proofs:** no new test: the move is behavior-preserving, and P6's `extraction_deterministic`, `xlsx_cells_are_decimal_strings`, `a_pdf_that_crashes_the_reader_is_failed_not_fatal` move with the module and pass unchanged; `fuzz_extract_pdf` and `fuzz_extract_xlsx` build. No kernel, Kani or Lean change.
|
||||
7. **Done when:** (1) `cargo tree -p ledger-store -e normal` shows no `typst*` and no `ledger-import`; (2) `cargo tree -p ledger-import -e normal` shows no `ledger-store` (as today); (3) the three P6 extraction tests and the two fuzz targets pass where they now live; (4) `docs/architecture.md` has no "under review" sentence; (5) the batch's gate (D-60) green.
|
||||
8. **As built:** none yet.
|
||||
|
||||
#### P4b S4: `documents_v2`
|
||||
|
||||
1. **Status:** in progress: S4.1 (`13282cf8`), S4.2 (`bf91c5a7`) and S4.3 (`d19fb464`) landed 2026-10-27; S4.4 landed 2026-10-29 (`d329bd6f`, merged clean from branch `S4.4` at `002fb9f8`); S4.5 not started, the next unit.
|
||||
@@ -2078,6 +2094,7 @@ The owner decisions of 2026-10-16 (at the top of this document) replace the form
|
||||
| D-58 | Q-8 answered (owner, 2026-10-28, the question tool: "Yes, all stand as built") | **D-47 to D-54 are the owner's decisions**, no longer the review's recommendations: reversals stay posted (D-47), the vault stays v1 (D-48), the P4b deferrals (D-49), the preview digest with no key (D-50), the P9 split (D-51), one P4b upgrade (D-52), no jobs and no new crates (D-53), the P11 split (D-54). The "the owner may revisit" clauses on those rows are closed. |
|
||||
| D-59 | Mutation testing in CI (owner, 2026-10-29, asked how the always-red weekly mutation shards should run until P12: "Don't think we need to run the long ass shit like mutation testing in CI. It's more for local test runs.") | **Mutation testing is a local run, never a CI job.** The mutation shards are removed from `scheduled.yml`, which keeps the proofs, the GnuCash oracle and the fuzz targets weekly; `just mutants-*` stays for the desktop, and P12 runs the whole pass there (D-33). The 2026-10-26 weekly run's survivors (real ones in every crate, and timeouts) are P12's list. |
|
||||
| D-60 | How much checking each merge gets (owner, 2026-10-29, after the paused wave took twelve hours to land with a full gate around every merge: "merge, and then just ... compile it once, make sure it compiles. Maybe run some specific tests related to that and then move to the next one, merge ... And then after you have done either several of them or even all of them, then run the super exhaustive test suites") | **Per merge, a compile and the unit's own tests; the exhaustive gates (`just check`, `just prove`, fuzz) once per batch, at the end, and their findings fixed together.** GitHub's `check` job stays the gate on every push. `plan/HANDOFF.md` "Reviewing and merging" carries the rule; AGENTS.md's "before every commit" is read as before every push of a batch, not before every merge inside one. | Every remaining unit; the EM's merge procedure |
|
||||
| D-61 | Q-9 answered (owner, 2026-10-29, the question tool: "Split the reader out later") | **The reader (`extract.rs`) moves into a small library crate of its own, `ledger-reader`, that `ledger-store` and `ledger-import` both depend on; a half-day unit (U0d) for the implementing team after S4.5.** Until then the arrow stands as built; `ledger-import` never depends on `ledger-store`. | U0d; `docs/architecture.md` |
|
||||
|
||||
### 10.2 Deferred
|
||||
|
||||
@@ -2086,7 +2103,7 @@ The owner decisions of 2026-10-16 (at the top of this document) replace the form
|
||||
|
||||
### 10.3 Open questions for the owner, and the closed ones
|
||||
|
||||
**Open (2026-10-29): Q-9 now; Q-4, Q-6, Q-7 when their units start.** Each is asked one at a time, in plain English; none blocks the units before it. Q-5 and Q-8 were answered on 2026-10-28 (D-57, D-58) and are kept below for the record.
|
||||
**Open (2026-10-29): Q-4, Q-6, Q-7, each when its unit starts.** One at a time, in plain English; none blocks the units before it. Q-5, Q-8 and Q-9 were answered (D-57, D-58, D-61) and are kept below for the record. Q-5 and Q-8 were answered on 2026-10-28 (D-57, D-58) and are kept below for the record.
|
||||
|
||||
**Q-4 (before P11a step 5).** Releases: should the app check updates against our own signed release list as well (two keys, one on your hardware token and one offline, a version that only goes up, an expiry), as decided on 2026-10-18 (D-25), or is the packaging tool's own single-key signature plus Apple's and Microsoft's code signing enough? The second is simpler and already refuses an older version; the first also stops a frozen or replayed update list. Recommended: the first, as decided, since the key work is small once the token exists.
|
||||
|
||||
@@ -2096,7 +2113,7 @@ The owner decisions of 2026-10-16 (at the top of this document) replace the form
|
||||
|
||||
**Q-7 (when P11b starts).** When the app connects Codex to a company's books folder, may it mark that folder as trusted in Codex's own settings, or should Codex ask you itself the first time? Recommended: let Codex ask.
|
||||
|
||||
**Q-9 (asked 2026-10-29, after P6 landed).** P6 made the part of the program that writes the books (`ledger-store`) depend on the part that reads and writes file formats (`ledger-import`), so a receipt's text is read the moment it is filed. That is the right behavior, but `ledger-import` also carries the PDF typesetter the invoice is printed with, so the books writer now builds and tests with a typesetter it never uses, and the crate map has an arrow it did not have (`docs/architecture.md`). Three choices: (a) keep it as built and write the rule down (the format crate must never depend on the writer; it doesn't today); (b) move the reader (`extract.rs`: the PDF text and page reader, the spreadsheet reader) into a small library crate of its own that both depend on (not D-53's closed `ledger-extract` subprocess: a plain library, in-process as today; the EM allocates its name in 8.0.3), so the writer sheds the typesetter and the map is layered again (about half a day for the implementing team, no behavior change); (c) take reading out of the writer and make every caller read the file and hand the text in, which removes the arrow but makes "a file is read as it is filed" a duty each caller can forget. Recommended: (b), as a small unit for the implementing team once S4.5 is in, because it keeps P6's guarantee and the architecture's layering, and it is mechanical; (a) is acceptable if build times in `ledger-store` never bother anyone.
|
||||
**Q-9 (closed 2026-10-29 → D-61: split the reader out later, unit U0d).** P6 made the part of the program that writes the books (`ledger-store`) depend on the part that reads and writes file formats (`ledger-import`), so a receipt's text is read the moment it is filed. That is the right behavior, but `ledger-import` also carries the PDF typesetter the invoice is printed with, so the books writer now builds and tests with a typesetter it never uses, and the crate map has an arrow it did not have (`docs/architecture.md`). Three choices: (a) keep it as built and write the rule down (the format crate must never depend on the writer; it doesn't today); (b) move the reader (`extract.rs`: the PDF text and page reader, the spreadsheet reader) into a small library crate of its own that both depend on (not D-53's closed `ledger-extract` subprocess: a plain library, in-process as today; the EM allocates its name in 8.0.3), so the writer sheds the typesetter and the map is layered again (about half a day for the implementing team, no behavior change); (c) take reading out of the writer and make every caller read the file and hand the text in, which removes the arrow but makes "a file is read as it is filed" a duty each caller can forget. Recommended: (b), as a small unit for the implementing team once S4.5 is in, because it keeps P6's guarantee and the architecture's layering, and it is mechanical; (a) is acceptable if build times in `ledger-store` never bother anyone.
|
||||
|
||||
**Q-8 (closed 2026-10-28 → D-58: all stand).** The 2026-10-26 review made eight engineering decisions in your name (D-47 to D-54: reversals stay posted with no reversal drafts; the vault stays v1; the P4b deferrals; the preview digest with no key; the P9 and P11 splits; one `documents_v2` upgrade; no jobs and no new crates), and S5, S6, S7 and P6 were built on them. Do they stand? Recommended: yes, as built; any you reverse becomes a new unit.
|
||||
|
||||
|
||||
Reference in new issue
Block a user