merge: P6, the evidence pipeline (branch P6, checkpoint ce366408)

The last of the paused branches, from its "not for merge" checkpoint:
extraction in ledger-import (text, pages, page images; capped, caught),
journaled as attachment.text_extracted; dedup by hash with
books.find_attachment and books.consolidate_duplicates; annotations and
books.set_entry_settles; the company record categories; the two fuzz
targets; feature attachment_text. Nineteen files conflicted with the
four P4b merges, resolved as plan 8.0.2 says: P6 retakes schema 18 (it
had taken 16, which S4.3 holds; S5 holds 17); the tool table is 129 and
the destructive count 31; both sides' appends kept with their seams
restored; one B-27 paragraph naming every feature; in the old-books
fixture P6's hand-written reimbursement is entry 7 (its ids collided
with S4.3's invoice); the replay fuzzer's list gains S7's
invoice.written_off; S4.3's issued-PDF test expects the extraction
record P6 journals right after a filing.

One integration defect between S7 and P6, found by the arap property
test and fixed here: under P6's dedup, a write-off whose evidence bytes
were filed already (the same collection letter for a second write-off of
the same invoice) was refused only after its entry had been journaled,
at the final link to the invoice ("already linked"). The write-off now
resolves the dedup before anything is journaled and keeps a link the
record already has to the invoice instead of refusing it; an explicit
existing_attachment_id already on an entry is still refused (S7's
one-receipt-once rule). P6's gate never ran before the pause; it runs on
main after this merge, and its ledger-store -> ledger-import dependency
is the open review point (plan 8.0.1).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HS8WZziDnyxunC23MUxrLp
This commit is contained in:
swaitsandClaude Fable 5.1 committed 2026-10-28 23:08:59 -06:00
commit 2ad192bbcd
70 files changed
+4618 -384

No files matched your search

+24
View File
@@ -165,6 +165,15 @@ pub enum Request {
/// The owner passphrase, checked by the server (INVARIANT 7).
passphrase: Passphrase,
},
/// Read one attachment's text now and journal it (plan P6 step 2):
/// the backfill, one attachment at a time, so a long books dir can be
/// resumed.
Extract {
/// The books.
id: String,
/// The attachment to read.
attachment_id: u64,
},
/// End a token.
TokenRevoke {
/// Its books.
@@ -477,6 +486,18 @@ pub enum Response {
},
/// A format upgrade made (or none needed).
FormatUpgraded(FormatUpgraded),
/// An attachment's text read and journaled (plan P6 step 2).
Extracted {
/// The attachment.
attachment_id: u64,
/// "ok", "none" or "failed".
status: String,
/// How many pages it has.
pages: u32,
/// The text blob's SHA-256, when there is text.
#[serde(default, skip_serializing_if = "Option::is_none")]
text_sha256: Option<String>,
},
/// Refused, with why (no path in the data root: INVARIANT 3).
Refused(String),
}
@@ -530,6 +551,9 @@ pub fn call_within(
Request::UpgradeFormat { .. } => {
Some("`wanderbooks books list` says whether the books still wait for it")
}
Request::Extract { .. } => Some(
"`wanderbooks books extract BOOK ATTACHMENT` reads it again; it journals once per extractor version",
),
};
let unknown = move |why: String| match look {
None => CallError::Unknown(why),
+28
View File
@@ -497,6 +497,34 @@ impl Books {
expect_head,
} => self.restore(id, source, replace, confirmed, expect_head),
Request::UpgradeFormat { id, passphrase } => self.upgrade_format(id, &passphrase),
Request::Extract { id, attachment_id } => {
let id = match BookId::parse(&id) {
Ok(id) => id,
Err(e) => return refused(e),
};
let store = match self.held().get(id.as_str()) {
Some(Book::Open { store, .. }) => store.clone(),
Some(Book::Failed(why)) => {
return Response::Refused(format!("those books didn't open: {why}"));
}
None => {
return Response::Refused(format!(
"there are no books called {:?}",
id.as_str()
));
}
};
match store.with_store_mut(|s| s.extract_attachment(attachment_id)) {
Ok(Ok(x)) => Response::Extracted {
attachment_id: x.attachment_id,
status: x.status.key().to_string(),
pages: x.pages,
text_sha256: x.text_sha256,
},
Ok(Err(e)) => Response::Refused(self.hide(e.to_string())),
Err(why) => Response::Refused(self.hide(why)),
}
}
Request::BackUpNow { id, new_disk } => {
let id = match BookId::parse(&id) {
Ok(id) => id,
+6 -1
View File
@@ -56,7 +56,7 @@ const C: Needs = PostWhenConfirmed;
const K: Needs = PostForApprovalRules;
/// Every tool and what it needs (INVARIANT 1).
pub const TABLE: [(&str, Needs); 125] = [
pub const TABLE: [(&str, Needs); 129] = [
("books.handbook", R),
("books.start", R),
// Settings are committed at once: the settings commit of the plan.
@@ -191,6 +191,11 @@ pub const TABLE: [(&str, Needs); 125] = [
("books.back_up_now", A),
// P4b S7: a bad-debt write-off (preview, then post).
("books.write_off_invoice", C),
// P6: evidence (plan/served.md P6).
("books.find_attachment", R),
("books.annotate_attachment", W),
("books.set_entry_settles", W),
("books.consolidate_duplicates", C),
];
/// What `tool` needs, given its arguments; None for a tool this server
+99 -30
View File
@@ -576,6 +576,7 @@ impl BooksServer {
+ Self::reconciliation_router()
+ Self::parties_router()
+ Self::attachments_router()
+ Self::evidence_router()
+ Self::settings_router()
+ Self::chart_router()
+ Self::payroll_router()
@@ -1407,6 +1408,8 @@ mod chart;
mod documents;
#[path = "tools/entries.rs"]
mod entries;
#[path = "tools/evidence.rs"]
mod evidence;
#[path = "tools/helpers.rs"]
mod helpers;
#[path = "tools/parties.rs"]
@@ -1442,44 +1445,58 @@ pub(crate) use helpers::*;
// ServerHandler
// ---------------------------------------------------------------------------
/// What a resource read gives: JSON/Markdown text, or a file's bytes (a
/// receipt's page picture, plan P6).
enum ResourceBody {
/// Text the agent reads.
Text(String),
/// Bytes, base64 in the MCP answer.
Blob { mime: String, bytes: Vec<u8> },
}
impl BooksServer {
/// Render a resource URI to its JSON text. Used by `read_resource`.
async fn resource_text(&self, uri: &str) -> Result<String, String> {
/// Render a resource URI to what it gives. Used by `read_resource`.
async fn resource_body(&self, uri: &str) -> Result<ResourceBody, String> {
// Split off the query string for URIs like books://digest?since=...
let (path, query) = match uri.split_once('?') {
Some((p, q)) => (p, q),
None => (uri, ""),
};
fn json<T: serde::Serialize>(v: &T) -> Result<ResourceBody, String> {
serde_json::to_string_pretty(v)
.map(ResourceBody::Text)
.map_err(|e| e.to_string())
}
match path {
"books://start" => {
let Json(out) = self.start(Parameters(kept::StartParams::default())).await?;
serde_json::to_string_pretty(&out).map_err(|e| e.to_string())
json(&out)
}
"books://settings" => {
let Json(out) = self.settings_tool().await?;
serde_json::to_string_pretty(&out).map_err(|e| e.to_string())
json(&out)
}
"books://conventions" => {
let Json(out) = self
.conventions(Parameters(kept::ConventionsParams::default()))
.await?;
serde_json::to_string_pretty(&out).map_err(|e| e.to_string())
json(&out)
}
"books://chart" => {
let Json(out) = self.chart().await?;
serde_json::to_string_pretty(&out).map_err(|e| e.to_string())
json(&out)
}
"books://reports/trial-balance" => {
let Json(out) = self
.trial_balance(Parameters(TrialBalanceParams::default()))
.await?;
serde_json::to_string_pretty(&out).map_err(|e| e.to_string())
json(&out)
}
"books://reports/balance-sheet" => {
let Json(out) = self
.balance_sheet(Parameters(BalanceSheetParams { as_of: None }))
.await?;
serde_json::to_string_pretty(&out).map_err(|e| e.to_string())
json(&out)
}
"books://reports/profit-and-loss" => {
let Json(out) = self
@@ -1489,16 +1506,16 @@ impl BooksServer {
class: None,
}))
.await?;
serde_json::to_string_pretty(&out).map_err(|e| e.to_string())
json(&out)
}
"books://digest" => {
let since = digest_since(query, today());
let Json(out) = self.digest(Parameters(DigestParams { since })).await?;
serde_json::to_string_pretty(&out).map_err(|e| e.to_string())
json(&out)
}
handbook::URI => {
let Json(out) = self.handbook(Parameters(HandbookParams::default())).await?;
serde_json::to_string_pretty(&out).map_err(|e| e.to_string())
json(&out)
}
_ if path.starts_with(&format!("{}/", handbook::URI)) => {
let id = &path[handbook::URI.len() + 1..];
@@ -1508,21 +1525,59 @@ impl BooksServer {
self.currency(),
self.document_line_projects(),
)
.map(ResourceBody::Text)
.ok_or_else(|| format!("unknown handbook section: {id}"))
}
_ if path.starts_with("books://receipts/") => {
let id: u64 = path
.strip_prefix("books://receipts/")
.unwrap_or("")
_ if (path.starts_with("books://receipts/")
|| path.starts_with("books://attachments/")) =>
{
// `books://attachments/{id}/page/{n}`: one page as a PNG
// (plan P6; D-46: a file is handed over, but a resource
// read is the agent's own fetch of a picture, never a
// path).
let prefix = if path.starts_with("books://receipts/") {
"books://receipts/"
} else {
"books://attachments/"
};
let rest = &path[prefix.len()..];
if let Some((id, page)) = rest.split_once("/page/") {
let id: u64 = id
.parse()
.map_err(|_| format!("bad attachment resource URI: {uri}"))?;
let page: u32 = page
.parse()
.map_err(|_| format!("bad page in {uri}: give a page number from 1"))?;
if page == 0 {
return Err("page numbers start at 1".into());
}
let store = self.lock()?;
let meta = find_attachment(&store, id)?;
let bytes = store.attachment_bytes(id).map_err(|e| e.to_string())?;
let png = ledger_import::extract::page_png(
&bytes,
usize::try_from(page - 1).unwrap_or(usize::MAX),
1600,
)
.map_err(|e| e.to_string())?;
let _ = meta;
return Ok(ResourceBody::Blob {
mime: "image/png".into(),
bytes: png,
});
}
let id: u64 = rest
.parse()
.map_err(|_| format!("bad receipt resource URI: {uri}"))?;
.map_err(|_| format!("bad attachment resource URI: {uri}"))?;
let Json(out) = self
.get_receipt(Parameters(GetReceiptParams {
attachment_id: id,
view: None,
page: None,
max_px: None,
}))
.await?;
serde_json::to_string_pretty(&out).map_err(|e| e.to_string())
json(&out)
}
_ => Err(format!("unknown resource: {uri}")),
}
@@ -1842,29 +1897,43 @@ impl BooksServer {
let uri = request.uri.clone();
let server = self.clone();
let at = uri.clone();
let text = run_blocking(&self.store, move || {
Box::pin(async move { in_timezone_async(timezone, server.resource_text(&at)).await })
let body = run_blocking(&self.store, move || {
Box::pin(async move { in_timezone_async(timezone, server.resource_body(&at)).await })
})
.await?;
// A refusal quotes the URI it was given: masked like a tool's.
let text = text.map_err(|e| {
let body = body.map_err(|e| {
redact_error(rmcp::ErrorData::resource_not_found(
uri.clone(),
Some(serde_json::json!({"reason": e})),
))
})?;
let mime_type = if uri.starts_with(&format!("{}/", handbook::URI)) {
"text/markdown"
} else {
"application/json"
let contents = match body {
ResourceBody::Text(text) => {
let mime_type = if uri.starts_with(&format!("{}/", handbook::URI)) {
"text/markdown"
} else {
"application/json"
};
ResourceContents::TextResourceContents {
uri,
mime_type: Some(mime_type.to_string()),
text,
meta: None,
}
}
ResourceBody::Blob { mime, bytes } => {
use base64::Engine;
ResourceContents::BlobResourceContents {
uri,
mime_type: Some(mime),
blob: base64::engine::general_purpose::STANDARD.encode(bytes),
meta: None,
}
}
};
Ok(ReadResourceResponse::Complete(ReadResourceResult::new(
vec![ResourceContents::TextResourceContents {
uri,
mime_type: Some(mime_type.to_string()),
text,
meta: None,
}],
vec![contents],
)))
}
}
+14 -3
View File
@@ -42,7 +42,7 @@ fn annotations_complete() {
const D: Hints = (Some(false), Some(true), None);
const WI: Hints = (Some(false), Some(false), Some(true));
const DI: Hints = (Some(false), Some(true), Some(true));
let table: [(&str, Hints); 125] = [
let table: [(&str, Hints); 129] = [
("books.handbook", R),
("books.start", R),
("books.set_setting", DI),
@@ -164,6 +164,11 @@ fn annotations_complete() {
("books.project", R),
("books.profit_and_loss_by_class", R),
("books.export_cpa_package", W),
// P6: evidence (plan/served.md P6).
("books.find_attachment", R),
("books.annotate_attachment", W),
("books.set_entry_settles", W),
("books.consolidate_duplicates", D),
("books.access_log", R),
("books.backup", R),
("books.back_up_now", WI),
@@ -208,8 +213,8 @@ fn annotations_complete() {
"{name}: openWorldHint"
);
}
// docs/mcp-interface.md names the 30 destructive tools.
assert_eq!(table.iter().filter(|(_, h)| h.1 == Some(true)).count(), 30);
// docs/mcp-interface.md names the 31 destructive tools.
assert_eq!(table.iter().filter(|(_, h)| h.1 == Some(true)).count(), 31);
}
/// Every refusal an agent can be given whose text is fixed here, or
@@ -472,6 +477,12 @@ fn no_tool_takes_a_path() {
&mut props,
);
for p in props {
// An annotation's `source` (plan P6 step 5) is what a human
// says a receipt came from ("the owner's email"), kept as text
// and never read as a place on this computer.
if t.name == "books.annotate_attachment" && p == "source" {
continue;
}
let lower = p.to_ascii_lowercase();
// Names that hold a place to read or write: whole words,
// so `link_to`, `*_file_name`, `tin_on_file` and `date_to`
+2 -89
View File
@@ -157,7 +157,7 @@ pub fn refuse(
if holds_sha(vault_dir, &ledger_store::journal::sha256_hex(bytes))? {
return Err(TIN_FILE.to_string());
}
let Some(text) = text_of(bytes, mime_type, file_name) else {
let Some(text) = ledger_import::extract::text_of(bytes, mime_type, file_name) else {
return Ok(());
};
let text: Vec<char> = text.chars().map(fold).collect();
@@ -278,97 +278,10 @@ fn holds_printed_tin(text: &[char], own_ein: Option<u32>) -> bool {
})
}
/// Whether a file of this type or name is text: csv, tsv, txt, json, iif,
/// xml, html, md and their kin.
fn text_like(mime_type: &str, file_name: &str) -> bool {
let mime = mime_type.trim().to_ascii_lowercase();
let ext = Path::new(file_name)
.extension()
.and_then(|e| e.to_str())
.map(str::to_ascii_lowercase)
.unwrap_or_default();
mime.starts_with("text/")
|| [
"json",
"xml",
"csv",
"x-iif",
"javascript",
"x-yaml",
"yaml",
]
.iter()
.any(|t| mime.ends_with(&format!("/{t}")) || mime.ends_with(&format!("+{t}")))
|| [
"csv", "tsv", "txt", "text", "json", "iif", "xml", "html", "htm", "md", "markdown",
"yaml", "yml", "tab", "prn", "log", "ofx", "qfx", "qif",
]
.contains(&ext.as_str())
}
/// The bytes as text, if they are text (INVARIANT 3): UTF-16 with its
/// byte-order mark, UTF-8, or, for a text type or name, whatever they
/// decode to. Text with NUL bytes and no byte-order mark (UTF-16 written
/// without one is valid UTF-8, its digits split by NULs) is read every
/// way it can be (as it is, as UTF-16 both ways round, and with its NULs
/// taken out), each reading on a line of its own: so is bytes that aren't
/// UTF-8 but are NUL-interleaved as UTF-16 is.
fn text_of(bytes: &[u8], mime_type: &str, file_name: &str) -> Option<String> {
let utf16 = |body: &[u8], le: bool| {
let units: Vec<u16> = body
.as_chunks::<2>()
.0
.iter()
.map(|&c| {
if le {
u16::from_le_bytes(c)
} else {
u16::from_be_bytes(c)
}
})
.collect();
String::from_utf16_lossy(&units)
};
match bytes {
[0xFF, 0xFE, body @ ..] => Some(utf16(body, true)),
[0xFE, 0xFF, body @ ..] => Some(utf16(body, false)),
_ => {
let plain = match std::str::from_utf8(bytes) {
Ok(t) => Some(t.to_string()),
Err(_) if text_like(mime_type, file_name) => {
Some(String::from_utf8_lossy(bytes).into_owned())
}
Err(_) => None,
};
if !bytes.contains(&0) || plain.is_none() && !nul_interleaved(bytes) {
return plain;
}
let without_nuls: Vec<u8> = bytes.iter().copied().filter(|&b| b != 0).collect();
let readings = [
plain.unwrap_or_default(),
utf16(bytes, true),
utf16(bytes, false),
String::from_utf8_lossy(&without_nuls).into_owned(),
];
Some(readings.join("\n"))
}
}
}
/// Whether `bytes` look like UTF-16 without a byte-order mark: of their
/// byte pairs, at least half have a NUL on one side (the high byte of a
/// Latin character) and at most an eighth on the other.
fn nul_interleaved(bytes: &[u8]) -> bool {
let pairs = bytes.as_chunks::<2>().0;
let nuls = |side: usize| pairs.iter().filter(|p| p[side] == 0).count();
let (first, second) = (nuls(0), nuls(1));
let n = pairs.len();
n > 0 && (2 * first >= n && 8 * second <= n || 2 * second >= n && 8 * first <= n)
}
#[cfg(test)]
mod tests {
use super::*;
use ledger_import::extract::nul_interleaved;
/// A test's dir, removed when the test ends, pass or panic.
struct TestDir(std::path::PathBuf);
+83 -21
View File
@@ -138,7 +138,7 @@ impl BooksServer {
#[tool(
name = "books.file_document",
title = "File a document",
description = "File a document: category formation, legal, tax, accounting, statement (a bank or card statement: then books.record_statement), contract, photo, warranty, import_source (a bank export), or other. link_to links it to subjects in the same call (an account, an asset, a party, a project, the company: see books.link_attachment), all checked first. The bytes: upload_id, from books.begin_upload and `wanderbooks upload` (a file never passes through you). Never a W-9 (books.set_w9 w9_upload_id puts it in the vault). A file holding a full TIN is refused: one the server read or wrote with TINs, or text with a TIN the vault keeps or an SSN or EIN printed 3-2-4 or 2-7 with hyphens.",
description = "File a document: category formation, legal, tax, accounting, statement (then books.record_statement), contract, photo, warranty, import_source (a bank export), notices, correspondence, plans, history, or other. link_to links it to subjects in the same call (see books.link_attachment), all checked first. The bytes: upload_id, from books.begin_upload and `wanderbooks upload` (a file never passes through you). Never a W-9 (books.set_w9). A file holding a full TIN is refused, as is text with one the vault keeps or an SSN or EIN printed 3-2-4 or 2-7.",
annotations(
read_only_hint = false,
destructive_hint = false,
@@ -152,10 +152,11 @@ impl BooksServer {
let p = params.0;
match p.category.as_str() {
"formation" | "legal" | "tax" | "accounting" | "statement" | "contract" | "photo"
| "warranty" | "import_source" | "other" => {}
| "warranty" | "import_source" | "notices" | "correspondence" | "plans" | "history"
| "other" => {}
_ => {
return Err(format!(
"bad category {:?}: expected formation, legal, tax, accounting, statement, contract, photo, warranty, import_source, or other",
"bad category {:?}: expected formation, legal, tax, accounting, statement, contract, photo, warranty, import_source, notices, correspondence, plans, history, or other",
p.category
));
}
@@ -454,7 +455,7 @@ impl BooksServer {
#[tool(
name = "books.get_receipt",
title = "Get a receipt",
description = "A receipt or document: what it is and everything it is linked to. view bytes adds the file, handed over: run its download (wanderbooks download) to save a copy in a folder you name; its bytes never come back here.",
description = "A receipt or document: what it is, what it is linked to, its first 8 KiB of text and its pages. view text gives the whole extracted text (past 64 KiB as a handed-over text file); view page_image with page (1 is the first) gives one page as a PNG, handed over (D-46, D-15: no OCR). view bytes hands over the file itself. Text first; a page picture only when the text is empty or wrong; never read a file's bytes into your context.",
annotations(read_only_hint = true, open_world_hint = false)
)]
pub(super) async fn get_receipt(
@@ -462,29 +463,90 @@ impl BooksServer {
params: Parameters<GetReceiptParams>,
) -> Result<Json<GetReceiptOut>, String> {
let p = params.0;
let bytes_wanted = match p.view.as_deref().unwrap_or("meta") {
"meta" => false,
"bytes" => true,
other => {
return Err(format!(
"view {:?}: meta (the default) or bytes",
other.chars().take(20).collect::<String>()
));
}
};
let view = p.view.as_deref().unwrap_or("meta");
if !matches!(view, "meta" | "text" | "page_image" | "bytes") {
return Err(format!(
"view {:?}: meta (the default), text, page_image or bytes",
view.chars().take(20).collect::<String>()
));
}
let store = self.lock()?;
let meta = find_attachment(&store, p.attachment_id)?;
let extraction = store.extraction(meta.id).map_err(|e| e.to_string())?;
let pages = extraction.as_ref().map_or(0, |x| x.pages);
let text_status = extraction
.as_ref()
.map_or("not_extracted", |x| x.status.key())
.to_string();
let annotation = store.ledger().annotation(meta.id).cloned();
let full_text = store.extraction_text(meta.id).map_err(|e| e.to_string())?;
let book = book_name(&store);
// Handed over, read hash-verified (D-46): never its bytes here.
let file = if bytes_wanted {
let bytes = store
.attachment_bytes(p.attachment_id)
let mut file = None;
// Every view carries the meta answer's first 8 KiB of text (view
// `text` replaces it with the whole text, or hands a file over).
let mut text = None;
let mut text_truncated = false;
if let Some(t) = &full_text {
let cut = floor_char_boundary(t, 8 * 1024);
text_truncated = cut < t.len();
text = Some(t[..cut].to_string());
}
match view {
"meta" => {}
"text" => match full_text {
None => {
return Err(format!(
"attachment {} has no extracted text (text_status {text_status}); view page_image to read its pages",
meta.id
));
}
Some(t) if t.len() <= 64 * 1024 => text = Some(t),
Some(t) => {
// Past 64 KiB the whole text is the handed file, not
// the preview.
text = None;
text_truncated = false;
let name = format!("{}-text.txt", stem(&meta.file_name));
file = Some(self.hand_over(&book, &name, "text/plain", t.into_bytes())?);
}
},
"page_image" => {
let page = p.page.ok_or("view page_image needs page: 1 is the first")?;
let max_px = p.max_px.unwrap_or(1600);
if !(1..=8192).contains(&max_px) {
return Err("max_px must be 1 to 8192".into());
}
let bytes = store.attachment_bytes(meta.id).map_err(|e| e.to_string())?;
if meta.mime_type != "application/pdf" && !bytes.starts_with(b"%PDF") {
return Err(format!(
"attachment {} isn't a PDF: view bytes for the file itself",
meta.id
));
}
let png = ledger_import::extract::page_png(
&bytes,
usize::try_from(page - 1).unwrap_or(usize::MAX),
max_px,
)
.map_err(|e| e.to_string())?;
Some(self.hand_over(&book_name(&store), &meta.file_name, &meta.mime_type, bytes)?)
} else {
None
};
let name = format!("{}-p{page}.png", stem(&meta.file_name));
file = Some(self.hand_over(&book, &name, "image/png", png)?);
}
"bytes" => {
let bytes = store.attachment_bytes(meta.id).map_err(|e| e.to_string())?;
file = Some(self.hand_over(&book, &meta.file_name, &meta.mime_type, bytes)?);
}
_ => unreachable!("view checked"),
}
Ok(Json(GetReceiptOut {
file,
text,
text_truncated,
text_status,
pages,
receipt_number: annotation.as_ref().and_then(|a| a.fields.receipt_number),
annotation: annotation.as_ref().map(annotation_out),
links: links_out(&store, meta.id)?,
description: meta.description.clone(),
attachment_id: meta.id,
+412
View File
@@ -0,0 +1,412 @@
//! Evidence beyond a file's bytes: finding it, annotating it, and
//! consolidating duplicates (plan/served.md P6 steps 4 and 5).
//!
//! ```text
//! GOAL: a receipt found by what it is or says, the facts a human read off
//! it kept on the record, and identical files held once, linked
//! wherever they are evidence.
//! SOURCE: plan/served.md P6 step 4 (defect 4) and step 5 (defect 10);
//! IZ-18 (no FIFO guess: `entry.settles` only); Rev. Proc. 98-25
//! (an audit trail); rules.md B-44.
//! ```
use super::*;
/// A date as the tool takes it.
fn date_of(text: &str) -> Result<NaiveDate, String> {
NaiveDate::parse_from_str(text.trim(), "%Y-%m-%d")
.map_err(|_| format!("{text:?} isn't a date: give YYYY-MM-DD"))
}
#[tool_router(router = evidence_router, vis = "pub(super)")]
impl BooksServer {
/// ```text
/// GOAL: find a filed file by its exact bytes, its receipt number, or
/// what its text says.
/// SOURCE: plan/served.md P6 step 4; B-44.
/// INVARIANTS:
/// 1. EXACTLY one of sha256, receipt_number and text is given.
/// 2. Withdrawn files are listed too, marked (a duplicate is found
/// before it is consolidated).
/// 3. A text search is case-insensitive over the extracted text.
/// SCENARIOS:
/// 1. IF none or more than one is given: SHOULD refuse.
/// 2. IF sha256 is not 64 hex digits: SHOULD refuse.
/// 3. IF nothing matches: SHOULD answer with an empty list.
/// ```
#[tool(
name = "books.find_attachment",
title = "Find a filed file",
description = "Find receipts and documents by exactly one of: sha256 (the file's exact hash), receipt_number (what books.annotate_attachment recorded), or text (words its extracted text says, case-insensitively). Withdrawn files are listed too, marked, so a duplicate is found before books.consolidate_duplicates. Each comes with all its links.",
annotations(read_only_hint = true, open_world_hint = false)
)]
async fn find_attachment(
&self,
params: Parameters<FindAttachmentParams>,
) -> Result<Json<FindAttachmentOut>, String> {
let p = params.0;
let given = [
p.sha256.is_some(),
p.receipt_number.is_some(),
p.text.is_some(),
]
.into_iter()
.filter(|g| *g)
.count();
if given != 1 {
return Err(
"give exactly one of sha256, receipt_number or text: they find a file three ways"
.into(),
);
}
let store = self.lock()?;
let mut found = if let Some(sha) = p.sha256 {
let sha = sha.trim().to_lowercase();
if sha.len() != 64 || !sha.bytes().all(|b| b.is_ascii_hexdigit()) {
return Err("sha256 is 64 hexadecimal digits".into());
}
store
.list_attachments(None)
.map_err(|e| e.to_string())?
.into_iter()
.filter(|a| a.blob_sha.eq_ignore_ascii_case(&sha))
.collect()
} else if let Some(n) = p.receipt_number {
store
.attachments_with_receipt_number(n)
.map_err(|e| e.to_string())?
} else {
let text = p.text.unwrap_or_default();
let text = text.trim();
if text.chars().count() < 2 {
return Err("text is at least two characters".into());
}
store
.attachments_holding_text(text)
.map_err(|e| e.to_string())?
};
found.sort_by_key(|a| a.id);
let attachments = found
.into_iter()
.map(|a| receipt_out(&store, a))
.collect::<Result<_, _>>()?;
Ok(Json(FindAttachmentOut { attachments }))
}
/// ```text
/// GOAL: record what a human read off a file, without touching the file
/// or the ledger's amounts.
/// SOURCE: plan/served.md P6 step 5 (defect 10); B-44.
/// INVARIANTS:
/// 1. Journaled as `attachment.annotated`; a refusal journals nothing.
/// 2. The fields are the closed list of `AnnotationFieldsParam`; at
/// least one field or the note is given.
/// 3. A new annotation replaces the file's earlier one; the file
/// stays filed and its bytes unchanged.
/// 4. The answer names the next receipt number (one past the highest
/// annotated).
/// SCENARIOS:
/// 1. IF the attachment is unknown or withdrawn: SHOULD refuse.
/// 2. IF nothing is given, an amount is negative or malformed, or a
/// date is outside the calendar: SHOULD refuse.
/// ```
#[tool(
name = "books.annotate_attachment",
title = "Annotate a filed file",
description = "Record what a human read off a receipt or document: receipt_number, payee, amount (decimal string), date (YYYY-MM-DD), instrument, category, source, reimbursable_to, and a note. It changes no amount and no file; a later annotation replaces this one. The answer gives next_receipt_number (one past the highest annotated), so receipt numbers run without reuse. Amounts are exact decimal strings.",
annotations(
read_only_hint = false,
destructive_hint = false,
open_world_hint = false
)
)]
async fn annotate_attachment(
&self,
params: Parameters<AnnotateAttachmentParams>,
) -> Result<Json<AnnotateAttachmentOut>, String> {
let p = params.0;
let currency = self.currency();
let amount = match p.fields.amount.as_deref() {
None => None,
Some(a) => Some(a.trim().parse::<Decimal>().map_err(|_| {
format!("{a:?} isn't an amount: give a decimal string like \"12.34\"")
})?),
};
let date = match p.fields.date.as_deref() {
None => None,
Some(d) => Some(date_of(d)?),
};
let annotation = ledger_core::Annotation {
attachment_id: p.attachment_id,
fields: ledger_core::AnnotationFields {
receipt_number: p.fields.receipt_number,
payee: p.fields.payee,
amount,
date,
instrument: p.fields.instrument,
category: p.fields.category,
source: p.fields.source,
reimbursable_to: p.fields.reimbursable_to,
},
note: p.note,
};
let _ = currency;
let mut store = self.lock()?;
let stored = store
.annotate_attachment(annotation)
.map_err(|e| e.to_string())?;
let next = store.ledger().next_receipt_number();
Ok(Json(AnnotateAttachmentOut {
attachment_id: stored.attachment_id,
fields: annotation_out(&stored),
next_receipt_number: next,
}))
}
/// ```text
/// GOAL: record which entries a payment settles, so "the member was
/// paid back" is read from the books and never guessed.
/// SOURCE: plan/served.md P6 step 5 (IZ-18: no FIFO guess).
/// INVARIANTS:
/// 1. Journaled as `entry.settles`; a refusal journals nothing.
/// 2. Every named entry exists, none is the paying entry itself, none
/// is named twice.
/// 3. An empty list clears the record.
/// SCENARIOS:
/// 1. IF the paying entry or a settled entry is unknown, or the list
/// names the paying entry or one twice: SHOULD refuse.
/// ```
#[tool(
name = "books.set_entry_settles",
title = "Record what a payment settles",
description = "Record which entries an entry pays back, such as a reimbursement to a member settling the expenses they paid personally. Needs attention stops asking about a member-paid entry once a payment names it here. Empty clears the record. It changes no amount: it names what the payment settles.",
annotations(
read_only_hint = false,
destructive_hint = false,
open_world_hint = false
)
)]
async fn set_entry_settles(
&self,
params: Parameters<SetEntrySettlesParams>,
) -> Result<Json<SetEntrySettlesOut>, String> {
let p = params.0;
let mut store = self.lock()?;
store
.set_entry_settles(p.entry_id, &p.settles)
.map_err(|e| e.to_string())?;
Ok(Json(SetEntrySettlesOut {
ok: true,
entry_id: p.entry_id,
settles: p.settles,
}))
}
/// ```text
/// GOAL: hold identical files once, linked wherever they were evidence,
/// without ever leaving an entry the receipt policy covers with
/// none.
/// SOURCE: plan/served.md P6 step 4 (defect 4); rules.md B-44.
/// INVARIANTS:
/// 1. Records sharing one blob SHA are one set; the oldest is kept
/// and linked to every subject the others had.
/// 2. Every other record is unlinked, then withdrawn as a duplicate
/// (`duplicate_of`), journaled; nothing is deleted.
/// 3. Every touched entry's evidence is re-checked before the first
/// journal append: an entry that would be left with none and
/// needs one refuses the whole call, journaling nothing.
/// 4. Without `confirmed`, a preview that changes nothing.
/// SCENARIOS:
/// 1. IF the kept record is withdrawn and cannot stand in for a
/// duplicate that is a posted entry's only evidence: SHOULD
/// refuse (receipt_required).
/// 2. IF no two records share bytes: SHOULD answer with no groups.
/// ```
#[tool(
name = "books.consolidate_duplicates",
title = "Consolidate duplicate files",
description = "Find receipts and documents filed more than once with the same bytes. For each set it keeps the oldest, links it to every subject the others had, unlinks the others, and withdraws them as duplicates of it (nothing is deleted). Refused whole when it would leave a posted entry the receipt policy covers with no evidence. Without confirmed, a preview that changes nothing.",
annotations(
read_only_hint = false,
destructive_hint = true,
open_world_hint = false
)
)]
async fn consolidate_duplicates(
&self,
params: Parameters<ConsolidateDuplicatesParams>,
) -> Result<Json<ConsolidateDuplicatesOut>, String> {
use std::collections::{BTreeMap, BTreeSet};
let confirmed = params.0.confirmed.unwrap_or(false);
let mut store = self.lock()?;
let all = store.list_attachments(None).map_err(|e| e.to_string())?;
let mut by_sha: BTreeMap<String, Vec<ledger_store::Attachment>> = BTreeMap::new();
for a in all {
by_sha.entry(a.blob_sha.clone()).or_default().push(a);
}
// One plan per set of two or more records.
struct Group {
sha: String,
keep: ledger_store::Attachment,
dups: Vec<ledger_store::Attachment>,
link: BTreeSet<ledger_store::Subject>,
unlink: Vec<(u64, ledger_store::Subject)>,
}
let mut groups: Vec<Group> = Vec::new();
for (sha, mut records) in by_sha {
if records.len() < 2 {
continue;
}
records.sort_by_key(|a| a.id);
let keep = records.remove(0);
let mut link: BTreeSet<ledger_store::Subject> = BTreeSet::new();
let mut unlink = Vec::new();
let mut live_dups = Vec::new();
for d in records {
if d.withdrawn_at.is_some() {
continue;
}
for sub in subjects_of(&store, &d)? {
if !covered(&store, &keep, sub)? {
link.insert(sub);
}
if !matches!(sub, ledger_store::Subject::Entry(e) if Some(e) == d.entry_id) {
unlink.push((d.id, sub));
}
}
live_dups.push(d);
}
if live_dups.is_empty() {
continue;
}
groups.push(Group {
sha,
keep,
dups: live_dups,
link,
unlink,
});
}
// Consolidating writes `duplicate_of`: the books must read the
// feature before anything at all is journaled (plan P6, R-A).
if !groups.is_empty() {
store
.ensure_feature(ledger_store::format::ATTACHMENT_TEXT)
.map_err(|e| e.to_string())?;
}
// The evidence gate, before anything is journaled: an entry that
// would be left with none, and needs one, refuses the whole call.
for g in &groups {
let keep_stands = g.keep.withdrawn_at.is_none();
for sub in &g.link {
if let ledger_store::Subject::Entry(e) = sub {
let mut left = store
.list_attachments(Some(*e))
.map_err(|e| e.to_string())?
.into_iter()
.filter(|a| a.withdrawn_at.is_none())
.filter(|a| !g.dups.iter().any(|d| d.id == a.id))
.count();
if keep_stands && !covered(&store, &g.keep, *sub)? {
left += 1;
}
if left == 0 {
self.ensure_receipt_not_required(&store, *e)
.map_err(|why| {
format!(
"not consolidated: entry {e}'s only evidence is a duplicate, and the kept file can't stand in for it ({why})"
)
})?;
}
}
}
}
let out_groups: Vec<DuplicateGroupOut> = groups
.iter()
.map(|g| DuplicateGroupOut {
sha256: g.sha.clone(),
keep: g.keep.id,
withdraw: g.dups.iter().map(|d| d.id).collect(),
links: g.link.iter().map(|s| subject_out(&store, *s)).collect(),
})
.collect();
let withdrawn = out_groups.iter().map(|g| g.withdraw.len()).sum();
if !confirmed {
return Ok(Json(ConsolidateDuplicatesOut {
status: "preview".into(),
groups: out_groups,
withdrawn,
}));
}
for g in &groups {
for sub in &g.link {
store
.link_attachment(g.keep.id, *sub)
.map_err(|e| e.to_string())?;
}
let reason = format!("duplicate of #{}", g.keep.id);
for (id, sub) in &g.unlink {
// A link a previous step took off (it was on the kept file
// already) isn't there to take off again.
if store
.attachment_links(*id)
.map_err(|e| e.to_string())?
.contains(sub)
{
store
.unlink_attachment(*id, *sub, &reason)
.map_err(|e| e.to_string())?;
}
}
for d in &g.dups {
if d.entry_id.is_some() {
store
.unlink_receipt(d.id, &reason)
.map_err(|e| e.to_string())?;
}
store
.withdraw_attachment_duplicate(d.id, &reason, g.keep.id)
.map_err(|e| e.to_string())?;
}
}
Ok(Json(ConsolidateDuplicatesOut {
status: "consolidated".into(),
groups: out_groups,
withdrawn,
}))
}
}
/// Every subject a filing is evidence of: the entry it is filed on, and
/// every explicit link.
fn subjects_of(
store: &Store,
a: &ledger_store::Attachment,
) -> Result<Vec<ledger_store::Subject>, String> {
let mut out: Vec<ledger_store::Subject> = a
.entry_id
.map(ledger_store::Subject::Entry)
.into_iter()
.collect();
out.extend(store.attachment_links(a.id).map_err(|e| e.to_string())?);
out.sort();
out.dedup();
Ok(out)
}
/// Whether `a` is already evidence of `sub`.
fn covered(
store: &Store,
a: &ledger_store::Attachment,
sub: ledger_store::Subject,
) -> Result<bool, String> {
if let ledger_store::Subject::Entry(e) = sub
&& a.entry_id == Some(e)
{
return Ok(true);
}
Ok(store
.attachment_links(a.id)
.map_err(|e| e.to_string())?
.contains(&sub))
}
+8 -2
View File
@@ -48,13 +48,14 @@ pub(crate) const INLINE_ARGS: [&str; 5] = [
/// Why `args` can't be taken by `tool` (INVARIANT 5): an argument that
/// carries a file's bytes or text. books.begin_upload's own sha256 is the
/// file's hash, not its bytes.
/// file's hash, not its bytes, and books.find_attachment's is what it
/// searches by (plan P6 step 4): neither file passes through the call.
pub(crate) fn inline_refusal(
tool: &str,
args: Option<&serde_json::Map<String, serde_json::Value>>,
) -> Option<String> {
let given = args?.keys().find(|k| INLINE_ARGS.contains(&k.as_str()))?;
if tool == "books.begin_upload" && given == "sha256" {
if matches!(tool, "books.begin_upload" | "books.find_attachment") && given == "sha256" {
return None;
}
Some(format!(
@@ -368,6 +369,11 @@ mod tests {
inline_refusal("books.begin_upload", Some(&args("sha256"))),
None
);
// A search by a file's hash names no bytes (plan P6).
assert_eq!(
inline_refusal("books.find_attachment", Some(&args("sha256"))),
None
);
assert!(inline_refusal("books.begin_upload", Some(&args("content_base64"))).is_some());
assert_eq!(inline_refusal("books.attach_receipt", None), None);
}
+38
View File
@@ -726,3 +726,41 @@ pub(crate) fn sent_out(s: &ledger_core::arap::Sent) -> SentOut {
pub(crate) fn last_sent(sends: &[ledger_core::arap::Sent]) -> Option<String> {
sends.iter().map(|s| s.on).max().map(|d| d.to_string())
}
/// An annotation as a tool answers with it (plan P6 step 5).
pub(crate) fn annotation_out(a: &ledger_core::Annotation) -> AnnotationOut {
AnnotationOut {
receipt_number: a.fields.receipt_number,
payee: a.fields.payee.clone(),
amount: a.fields.amount.map(money),
date: a.fields.date.map(|d| d.to_string()),
instrument: a.fields.instrument.clone(),
category: a.fields.category.clone(),
source: a.fields.source.clone(),
reimbursable_to: a.fields.reimbursable_to.clone(),
note: a.note.clone(),
}
}
/// A file name's stem, for a page picture's or a text file's name: what is
/// before the last dot, or "attachment" when there is nothing.
pub(crate) fn stem(file_name: &str) -> String {
let stem = file_name
.rsplit_once('.')
.map_or(file_name, |(stem, _)| stem)
.trim();
if stem.is_empty() {
"attachment".to_string()
} else {
stem.chars().take(80).collect()
}
}
/// The largest character boundary at or below `at` in `text`.
pub(crate) fn floor_char_boundary(text: &str, at: usize) -> usize {
let mut cut = at.min(text.len());
while !text.is_char_boundary(cut) {
cut -= 1;
}
cut
}
+193 -5
View File
@@ -1849,11 +1849,21 @@ pub struct ListReceiptsOut {
pub struct GetReceiptParams {
/// Attachment id.
pub attachment_id: u64,
/// What to return: "meta" (default: what it is and what it's linked
/// to), or "bytes" (also the file, handed over: the command that
/// saves a copy, never its bytes; D-46).
/// What to return: "meta" (default: what it is, what it's linked to,
/// the first 8 KiB of its text and its page count), "text" (the whole
/// extracted text, or a handed-over text file past 64 KiB),
/// "page_image" (one page as a PNG, handed over: give `page`), or
/// "bytes" (the file itself, handed over: the command that saves a
/// copy, never its bytes; D-46).
#[serde(default)]
pub view: Option<String>,
/// For view "page_image": the page, 1 for the first.
#[serde(default)]
pub page: Option<u32>,
/// For view "page_image": the picture's longest side in pixels
/// (default 1600, at most 8192).
#[serde(default)]
pub max_px: Option<u32>,
}
/// Result of `books.get_receipt`: what it is and what it's linked to, and
@@ -1878,10 +1888,31 @@ pub struct GetReceiptOut {
pub description: Option<String>,
/// Everything it is linked to (the entry it is filed on first).
pub links: Vec<SubjectOut>,
/// With view `bytes`: the file, handed over: run its `download` to
/// save a copy in a folder you name.
/// With view `bytes` or `page_image`: the file (or the page picture),
/// handed over: run its `download` to save a copy in a folder you
/// name.
#[serde(skip_serializing_if = "Option::is_none")]
pub file: Option<HandedFile>,
/// The first 8 KiB of the extracted text (view `meta`), or the whole
/// text (view `text`, when it fits in 64 KiB).
#[serde(skip_serializing_if = "Option::is_none")]
pub text: Option<String>,
/// True when `text` is only the first 8 KiB of a longer extraction.
#[serde(default)]
pub text_truncated: bool,
/// What reading the file found: "ok", "none", "failed", or
/// "not_extracted" (books this version didn't read yet:
/// `wanderbooks books extract`).
pub text_status: String,
/// How many pages the file has (0 when it has none).
#[serde(default)]
pub pages: u32,
/// The receipt number annotated on it, if any.
#[serde(skip_serializing_if = "Option::is_none")]
pub receipt_number: Option<u32>,
/// Everything annotated about it, if anything.
#[serde(skip_serializing_if = "Option::is_none")]
pub annotation: Option<AnnotationOut>,
}
// ---------------------------------------------------------------------------
@@ -5353,3 +5384,160 @@ pub struct WriteOffInvoiceOut {
/// The invoice's derived figures after it.
pub document: WriteOffDocumentOut,
}
// P6: evidence (plan/served.md P6: extraction, dedup, annotations)
// ---------------------------------------------------------------------------
/// One annotation's fields, as a human reads them off a file (plan P6 step
/// 5). Every field is optional; the amount is an exact decimal string in
/// the books' own currency.
#[derive(Debug, Default, Clone, Deserialize, JsonSchema)]
pub struct AnnotationFieldsParam {
/// The number printed on the receipt. Engine numbers run from the
/// highest one annotated (`next_receipt_number` in the answer).
pub receipt_number: Option<u32>,
/// Who was paid.
pub payee: Option<String>,
/// The amount printed on the receipt, a decimal string such as
/// "12.34".
pub amount: Option<String>,
/// The date printed on the receipt, YYYY-MM-DD.
pub date: Option<String>,
/// How it was paid: a payment method's id or label.
pub instrument: Option<String>,
/// What kind of record it is (the document categories).
pub category: Option<String>,
/// Where it came from.
pub source: Option<String>,
/// Who is to be reimbursed for it, if anyone.
pub reimbursable_to: Option<String>,
}
/// One recorded annotation, as read back.
#[derive(Debug, Clone, Serialize, JsonSchema)]
pub struct AnnotationOut {
/// The number printed on the receipt, if recorded.
#[serde(skip_serializing_if = "Option::is_none")]
pub receipt_number: Option<u32>,
/// Who was paid, if recorded.
#[serde(skip_serializing_if = "Option::is_none")]
pub payee: Option<String>,
/// The amount printed on the receipt, decimal string, if recorded.
#[serde(skip_serializing_if = "Option::is_none")]
pub amount: Option<String>,
/// The date printed on the receipt, if recorded.
#[serde(skip_serializing_if = "Option::is_none")]
pub date: Option<String>,
/// How it was paid, if recorded.
#[serde(skip_serializing_if = "Option::is_none")]
pub instrument: Option<String>,
/// What kind of record it is, if recorded.
#[serde(skip_serializing_if = "Option::is_none")]
pub category: Option<String>,
/// Where it came from, if recorded.
#[serde(skip_serializing_if = "Option::is_none")]
pub source: Option<String>,
/// Who is to be reimbursed for it, if recorded.
#[serde(skip_serializing_if = "Option::is_none")]
pub reimbursable_to: Option<String>,
/// A free note, if recorded.
#[serde(skip_serializing_if = "Option::is_none")]
pub note: Option<String>,
}
/// Parameters for `books.find_attachment`: exactly one of its three.
#[derive(Debug, Deserialize, JsonSchema)]
#[serde(deny_unknown_fields)]
pub struct FindAttachmentParams {
/// The exact SHA-256 of the file's bytes.
pub sha256: Option<String>,
/// The receipt number recorded on it (books.annotate_attachment).
pub receipt_number: Option<u32>,
/// Text it says, case-insensitively (its extracted text; see
/// books.get_receipt view text).
pub text: Option<String>,
}
/// Result of `books.find_attachment`.
#[derive(Debug, Serialize, JsonSchema)]
pub struct FindAttachmentOut {
/// Every match, withdrawn ones included (marked), oldest first.
pub attachments: Vec<ReceiptOut>,
}
/// Parameters for `books.annotate_attachment`.
#[derive(Debug, Deserialize, JsonSchema)]
#[serde(deny_unknown_fields)]
pub struct AnnotateAttachmentParams {
/// The attachment.
pub attachment_id: u64,
/// What was read off it (the closed field list).
pub fields: AnnotationFieldsParam,
/// A free note.
pub note: Option<String>,
}
/// Result of `books.annotate_attachment`.
#[derive(Debug, Serialize, JsonSchema)]
pub struct AnnotateAttachmentOut {
/// The attachment.
pub attachment_id: u64,
/// The annotation as stored (it replaces an earlier one).
pub fields: AnnotationOut,
/// The next receipt number the books would give: one past the highest
/// annotated.
pub next_receipt_number: u32,
}
/// Parameters for `books.set_entry_settles`.
#[derive(Debug, Deserialize, JsonSchema)]
#[serde(deny_unknown_fields)]
pub struct SetEntrySettlesParams {
/// The entry that pays, such as a reimbursement to a member.
pub entry_id: u64,
/// The entries it settles (what it pays back). Empty clears the record.
pub settles: Vec<u64>,
}
/// Result of `books.set_entry_settles`.
#[derive(Debug, Serialize, JsonSchema)]
pub struct SetEntrySettlesOut {
/// True when recorded.
pub ok: bool,
/// The entry that pays.
pub entry_id: u64,
/// The entries it settles now.
pub settles: Vec<u64>,
}
/// One set of attachments holding the same bytes (plan P6 step 4).
#[derive(Debug, Serialize, JsonSchema)]
pub struct DuplicateGroupOut {
/// The SHA-256 they share.
pub sha256: String,
/// The attachment kept (the oldest).
pub keep: u64,
/// The attachments withdrawn as duplicates of it.
pub withdraw: Vec<u64>,
/// The subjects the kept file is linked to on their behalf.
pub links: Vec<SubjectOut>,
}
/// Parameters for `books.consolidate_duplicates`.
#[derive(Debug, Deserialize, JsonSchema)]
#[serde(deny_unknown_fields)]
pub struct ConsolidateDuplicatesParams {
/// True: consolidate. Without it, a preview that changes nothing.
pub confirmed: Option<bool>,
}
/// Result of `books.consolidate_duplicates`.
#[derive(Debug, Serialize, JsonSchema)]
pub struct ConsolidateDuplicatesOut {
/// "preview" or "consolidated".
pub status: String,
/// Each set of attachments holding one file's bytes.
pub groups: Vec<DuplicateGroupOut>,
/// How many attachments were (or would be) withdrawn.
pub withdrawn: usize,
}