test(agent): scripted evals follow only next_steps; the surface over the wire (P5)
Agent reason: the envelope is only worth having if an agent can do its work by following it. crates/ledger-cli/tests/agent_evals/ holds six scripted transcripts, no model: onboarding to a first post, a receipt-backed post, a draft that waits for its receipt and posts when it arrives, a reversal, a reconciliation, a month-end close. agent_evals_follow_only_next_steps plays each against a real books server: after books.start a step may call only a tool (or fetch only a prompt) the previous result's next_steps named, with the arguments the step gave and the ones the agent decides; setup calls (the chart and the books an agent finds) are not the agent's moves. agent_surface.rs holds the over-the-wire tests of the rest: the journal in words (order, subject and type filters, paging, a bad subject refused), source corrections through the three tools, the prompts rendered against their files and refused when malformed, the resource templates advertised and read, books.start v2 whole and under 8 KiB with the self-report in the access log only, a new book told to onboard, and a mutating call answering with the head the journal has and a refusal naming its remedy. Docs: testing.md's P5 section (every test, and the For-P12 paragraph naming what a mutant would flip), parity.md (no feature moved; one row), and architecture.md (the envelope and the guards); mcp-interface.md's overview sections now describe the surface as built. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HS8WZziDnyxunC23MUxrLp
This commit is contained in:
1 parent
a319a64c3a
commit
7bac25496e
12 files changed
+1358
-5
No files matched your search
@@ -223,3 +223,18 @@ backdated entries included: defect 6), `zero_net_accounts`,
|
||||
`drafts_by_state` and the reconciliation split; reports gained
|
||||
`as_of_status` and `strict`; the CPA package is refused while a draft
|
||||
dated in it is open (4.8).
|
||||
|
||||
### P5: the agent surface (2026-10-29)
|
||||
|
||||
No semantic feature moved, so no status changed and the floor stands. P5
|
||||
changes how an agent is told what to do, not how a figure is computed: the
|
||||
result envelope on every tool result (`next_steps`, `guards`, `warnings`,
|
||||
`head`), `books.start` v2, the prompts, the resource templates,
|
||||
`books.journal_history` (the journal in words) and source corrections
|
||||
(`source_revisions`: `convention.source_revised`, `decision.source_revised`,
|
||||
`open_item.source_revised`). One new row, `partial` until a hand-verified
|
||||
golden covers it:
|
||||
|
||||
| Area | Counterpart (a cross-check, not the spec) | Status | Notes |
|
||||
|---|---|---|---|
|
||||
| Audit trail in words (`books.journal_history`, the `describe` sentences) | QBO "Audit log", Xero "History and notes" | partial | tests `journal_history_traces_what_the_books_did`, `every_event_type_has_a_sentence`; the viewer keeps its own sentences (the shared describer is the server's) |
|
||||
Reference in new issue
Block a user