test(agent): scripted evals follow only next_steps; the surface over the wire (P5)
Agent reason: the envelope is only worth having if an agent can do its work by following it. crates/ledger-cli/tests/agent_evals/ holds six scripted transcripts, no model: onboarding to a first post, a receipt-backed post, a draft that waits for its receipt and posts when it arrives, a reversal, a reconciliation, a month-end close. agent_evals_follow_only_next_steps plays each against a real books server: after books.start a step may call only a tool (or fetch only a prompt) the previous result's next_steps named, with the arguments the step gave and the ones the agent decides; setup calls (the chart and the books an agent finds) are not the agent's moves. agent_surface.rs holds the over-the-wire tests of the rest: the journal in words (order, subject and type filters, paging, a bad subject refused), source corrections through the three tools, the prompts rendered against their files and refused when malformed, the resource templates advertised and read, books.start v2 whole and under 8 KiB with the self-report in the access log only, a new book told to onboard, and a mutating call answering with the head the journal has and a refusal naming its remedy. Docs: testing.md's P5 section (every test, and the For-P12 paragraph naming what a mutant would flip), parity.md (no feature moved; one row), and architecture.md (the envelope and the guards); mcp-interface.md's overview sections now describe the surface as built. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HS8WZziDnyxunC23MUxrLp
This commit is contained in:
1 parent
a319a64c3a
commit
7bac25496e
12 files changed
+1358
-5
No files matched your search
@@ -12,12 +12,16 @@ locked to one assistant.
|
||||
|
||||
| Crate | Role | Spec |
|
||||
|---|---|---|
|
||||
| `ledger-core` | the engine: money, entries, the laws; every rule the tools enforce | `docs/spec/rules.md`, `docs/verification.md` |
|
||||
| `ledger-package` | the read-only views shared with the viewer: reports, aging, the CPA package, Needs attention (the one list `books.start` and the viewer both draw) | `docs/architecture.md` |
|
||||
| `ledger-store` | the hash-chained journal (the source of truth), the SQLite projection, the blob store | `docs/receipt-store.md` |
|
||||
| `ledger-mcp` | the books server: the tools, resources and handbook over store + core | this document |
|
||||
| `ledger-host` | the agent bridge (`wanderbooks mcp`), the host and its leases, tokens | `plan/served.md` 3.3, 3.9 |
|
||||
|
||||
(The overview sections of this document date from the first MCP phase; P5
|
||||
rewrites them. The per-tool sections are held to the code by a test.)
|
||||
(P5 rewrote the overview sections for the surface as it is: the result
|
||||
envelope, `books.start` v2, the prompts, the resource templates and the
|
||||
journal in words, below. The per-tool sections are held to the code by a
|
||||
test: `every_tool_is_documented_and_every_documented_tool_exists`.)
|
||||
|
||||
Rust, `rmcp` SDK (the Rust MCP SDK). The MCP crate holds no accounting
|
||||
logic: it translates tool calls into `Ledger` commands and formats results.
|
||||
@@ -389,8 +393,8 @@ books' files; no tool title or description, connect text or handbook
|
||||
section names `wanderbooks.toml` (`tool_text_lint`). A rule set by law that the books
|
||||
don't know for a year names its official source (the statute or
|
||||
publication, with its address) and says to tell the human.
|
||||
Tool descriptions are at most 700 characters; eight longer ones are listed
|
||||
in `tool_descriptions_fit_700_characters`, to be shortened in P5.
|
||||
Tool descriptions are at most 700 characters (`tool_descriptions_fit_700_characters`
|
||||
allows none longer; the long text lives in the handbook).
|
||||
|
||||
**Annotations.** Every tool has a `title` and a `readOnlyHint`; every tool
|
||||
that changes anything also has a `destructiveHint`, true for these 31:
|
||||
|
||||
Reference in new issue
Block a user