merge: P6, the evidence pipeline (branch P6, checkpoint ce366408)

The last of the paused branches, from its "not for merge" checkpoint:
extraction in ledger-import (text, pages, page images; capped, caught),
journaled as attachment.text_extracted; dedup by hash with
books.find_attachment and books.consolidate_duplicates; annotations and
books.set_entry_settles; the company record categories; the two fuzz
targets; feature attachment_text. Nineteen files conflicted with the
four P4b merges, resolved as plan 8.0.2 says: P6 retakes schema 18 (it
had taken 16, which S4.3 holds; S5 holds 17); the tool table is 129 and
the destructive count 31; both sides' appends kept with their seams
restored; one B-27 paragraph naming every feature; in the old-books
fixture P6's hand-written reimbursement is entry 7 (its ids collided
with S4.3's invoice); the replay fuzzer's list gains S7's
invoice.written_off; S4.3's issued-PDF test expects the extraction
record P6 journals right after a filing.

One integration defect between S7 and P6, found by the arap property
test and fixed here: under P6's dedup, a write-off whose evidence bytes
were filed already (the same collection letter for a second write-off of
the same invoice) was refused only after its entry had been journaled,
at the final link to the invoice ("already linked"). The write-off now
resolves the dedup before anything is journaled and keeps a link the
record already has to the invoice instead of refusing it; an explicit
existing_attachment_id already on an entry is still refused (S7's
one-receipt-once rule). P6's gate never ran before the pause; it runs on
main after this merge, and its ledger-store -> ledger-import dependency
is the open review point (plan 8.0.1).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HS8WZziDnyxunC23MUxrLp

crates/ledger-core/src/error.rs

@@ -72,6 +72,11 @@ pub enum LedgerError {
     #[error("unknown account id {0}")]
     UnknownAccount(u64),
 
+    /// What the books record about their evidence (an annotation, a
+    /// settlement) is not a record they keep (plan P6, [`crate::evidence`]).
+    #[error("{0}")]
+    InvalidEvidence(String),
+
     /// Placeholder (header) accounts organize the chart; they take no postings.
     #[error("account '{0}' is a placeholder and cannot take postings")]
     PlaceholderAccount(String),

crates/ledger-core/src/evidence.rs

@@ -0,0 +1,120 @@
+//! What the books record about their evidence beyond the file itself
+//! (plan P6 step 5): a receipt's own facts (its number, payee, amount,
+//! date, instrument, where it came from, who is to be reimbursed) and
+//! which entries a payment settles.
+//!
+//! ```text
+//! GOAL: an annotation is a record of what a human read off a file, kept
+//!       apart from the file and the ledger, and a settlement is the one
+//!       record from which "the member was paid back" is derived.
+//! SOURCE: plan/served.md P6 step 5 (defect 10); IZ-18 (no FIFO guess:
+//!         `entry.settles` only); IRS Pub. 583 (records are kept with what
+//!         they support).
+//! INVARIANTS:
+//!   1. An annotation's fields are a closed list; every one is optional,
+//!      and at least one field or the note is present.
+//!   2. A receipt number runs from the highest one annotated
+//!      ([`crate::Ledger::next_receipt_number`]).
+//!   3. A settlement names existing entries, never itself, never one
+//!      twice; it records nothing about amounts.
+//! ```
+
+use chrono::NaiveDate;
+use rust_decimal::Decimal;
+
+use crate::LedgerError;
+
+/// The closed set of facts a human may record about a file (plan P6 step
+/// 5). Every field is optional; the amount is an exact decimal in the
+/// books' own currency, and the date is the date on the document.
+#[derive(Clone, Debug, Default, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
+pub struct AnnotationFields {
+    /// The number printed on the receipt, unique in its own series only.
+    pub receipt_number: Option<u32>,
+    /// Who was paid.
+    pub payee: Option<String>,
+    /// The amount printed on the receipt, exact.
+    pub amount: Option<Decimal>,
+    /// The date printed on the receipt.
+    pub date: Option<NaiveDate>,
+    /// How it was paid: a payment method's id or label.
+    pub instrument: Option<String>,
+    /// What kind of record it is, as the document categories name them.
+    pub category: Option<String>,
+    /// Where it came from (a folder, a person, an email printed to PDF).
+    pub source: Option<String>,
+    /// Who is to be reimbursed for it, if anyone.
+    pub reimbursable_to: Option<String>,
+}
+
+impl AnnotationFields {
+    /// Whether nothing at all is recorded.
+    pub fn is_empty(&self) -> bool {
+        self.receipt_number.is_none()
+            && self.payee.is_none()
+            && self.amount.is_none()
+            && self.date.is_none()
+            && self.instrument.is_none()
+            && self.category.is_none()
+            && self.source.is_none()
+            && self.reimbursable_to.is_none()
+    }
+
+    /// Check the fields before they are journaled (INVARIANTS 1 and 2):
+    /// every text trimmed and non-empty, the number positive, the amount
+    /// not negative, the date in the books' calendar. Returns them as they
+    /// will be kept.
+    pub fn checked(mut self) -> Result<Self, LedgerError> {
+        let trim = |s: Option<String>| -> Option<String> {
+            s.map(|s| s.trim().to_string()).filter(|s| !s.is_empty())
+        };
+        self.payee = trim(self.payee);
+        self.instrument = trim(self.instrument);
+        self.category = trim(self.category);
+        self.source = trim(self.source);
+        self.reimbursable_to = trim(self.reimbursable_to);
+        if self.receipt_number == Some(0) {
+            return Err(LedgerError::InvalidEvidence(
+                "a receipt number is positive".into(),
+            ));
+        }
+        if self.amount.is_some_and(|a| a.is_sign_negative()) {
+            return Err(LedgerError::InvalidEvidence(
+                "an annotated amount is not negative".into(),
+            ));
+        }
+        if let Some(d) = self.date {
+            crate::calendar::check(d)?;
+        }
+        Ok(self)
+    }
+}
+
+/// One recorded annotation: the file, the facts read off it, and an
+/// optional note.
+#[derive(Clone, Debug, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
+pub struct Annotation {
+    /// The attachment the facts are about.
+    pub attachment_id: u64,
+    /// The facts.
+    pub fields: AnnotationFields,
+    /// A free note.
+    pub note: Option<String>,
+}
+
+impl Annotation {
+    /// Check the annotation before it is journaled (INVARIANTS 1 and 2).
+    pub fn checked(mut self) -> Result<Self, LedgerError> {
+        self.fields = self.fields.checked()?;
+        self.note = self
+            .note
+            .map(|n| n.trim().to_string())
+            .filter(|n| !n.is_empty());
+        if self.fields.is_empty() && self.note.is_none() {
+            return Err(LedgerError::InvalidEvidence(
+                "an annotation records at least one field or a note".into(),
+            ));
+        }
+        Ok(self)
+    }
+}

crates/ledger-core/src/ledger.rs

@@ -349,6 +349,12 @@ pub struct Ledger {
     /// Conventions, decisions, open items and approvals
     /// ([`crate::conventions`]).
     conventions: crate::conventions::Conventions,
+    /// The facts recorded about each filed file ([`crate::evidence`],
+    /// `attachment.annotated`), by attachment id.
+    annotations: std::collections::BTreeMap<u64, crate::evidence::Annotation>,
+    /// What each entry settles (`entry.settles`, plan P6): the one record
+    /// a member-paid entry's "reimbursed" is derived from (IZ-18).
+    entry_settles: std::collections::BTreeMap<EntryId, Vec<EntryId>>,
 }
 
 impl Ledger {
@@ -403,6 +409,8 @@ impl Ledger {
             entry_no_party: HashMap::new(),
             settings: crate::settings::Settings::default(),
             conventions: crate::conventions::Conventions::default(),
+            annotations: std::collections::BTreeMap::new(),
+            entry_settles: std::collections::BTreeMap::new(),
         }
     }
 
@@ -816,6 +824,8 @@ impl Ledger {
             ),
             ("settings", self.settings == other.settings),
             ("conventions", self.conventions == other.conventions),
+            ("annotations", self.annotations == other.annotations),
+            ("entry_settles", self.entry_settles == other.entry_settles),
         ]
         .into_iter()
         .filter(|(_, same)| !same)
@@ -8334,6 +8344,101 @@ impl Ledger {
         &self.conventions
     }
 
+    /// The facts recorded about the books' files, by attachment id
+    /// (`attachment.annotated`, plan P6 step 5).
+    pub fn annotations(&self) -> impl Iterator<Item = &crate::evidence::Annotation> {
+        self.annotations.values()
+    }
+
+    /// The facts recorded about one file, if any.
+    pub fn annotation(&self, attachment_id: u64) -> Option<&crate::evidence::Annotation> {
+        self.annotations.get(&attachment_id)
+    }
+
+    /// The next receipt number: one past the highest one annotated, or 1
+    /// when none is (plan P6 step 5: engine numbers run from the max).
+    pub fn next_receipt_number(&self) -> u32 {
+        self.annotations
+            .values()
+            .filter_map(|a| a.fields.receipt_number)
+            .max()
+            .map_or(1, |n| n.saturating_add(1))
+    }
+
+    /// Record what a human read off a file, replacing an earlier
+    /// annotation of it (plan P6 step 5; INVARIANTS 1 and 2 of
+    /// [`crate::evidence`]).
+    pub fn annotate_attachment(
+        &mut self,
+        annotation: crate::evidence::Annotation,
+    ) -> Result<(), LedgerError> {
+        let annotation = annotation.checked()?;
+        self.annotations
+            .insert(annotation.attachment_id, annotation);
+        Ok(())
+    }
+
+    /// [`Self::annotate_attachment`] as journaled (`attachment.annotated`),
+    /// for replay only (rules.md B-28).
+    pub fn replay_annotate_attachment(
+        &mut self,
+        annotation: crate::evidence::Annotation,
+    ) -> Result<(), LedgerError> {
+        self.annotate_attachment(annotation)
+    }
+
+    /// The entries `entry` records settling (`entry.settles`, plan P6).
+    pub fn entry_settles(&self, entry: EntryId) -> &[EntryId] {
+        self.entry_settles.get(&entry).map_or(&[], Vec::as_slice)
+    }
+
+    /// Whether any entry records settling `entry`: the one way a
+    /// member-paid entry reads as reimbursed (IZ-18: no FIFO guess).
+    pub fn settles_entry(&self, entry: EntryId) -> bool {
+        self.entry_settles.values().any(|s| s.contains(&entry))
+    }
+
+    /// Record which entries `entry` settles (a reimbursement naming what
+    /// it pays back, plan P6 step 5). INVARIANT 3: every entry exists,
+    /// none is `entry` itself, none is named twice.
+    pub fn set_entry_settles(
+        &mut self,
+        entry: EntryId,
+        settles: Vec<EntryId>,
+    ) -> Result<(), LedgerError> {
+        if !self.entries.contains_key(&entry) {
+            return Err(LedgerError::UnknownEntry(entry));
+        }
+        let mut seen = std::collections::BTreeSet::new();
+        for s in &settles {
+            if *s == entry {
+                return Err(LedgerError::InvalidEvidence(
+                    "an entry doesn't settle itself".into(),
+                ));
+            }
+            if !self.entries.contains_key(s) {
+                return Err(LedgerError::UnknownEntry(*s));
+            }
+            if !seen.insert(*s) {
+                return Err(LedgerError::InvalidEvidence(format!(
+                    "entry {s} is settled twice by entry {entry}"
+                )));
+            }
+        }
+        self.entry_settles.insert(entry, settles);
+        Ok(())
+    }
+
+    /// [`Self::set_entry_settles`] as journaled (`entry.settles`), for
+    /// replay only (rules.md B-28).
+    pub fn replay_set_entry_settles(
+        &mut self,
+        entry: EntryId,
+        settles: Vec<EntryId>,
+    ) -> Result<(), LedgerError> {
+        self.set_entry_settles(entry, settles)
+    }
+
     /// Check a convention before it is journaled: its id, text and
     /// structure (crate::conventions::check_new), and that every account a
     /// posting template or approval rule names is in the chart (a

crates/ledger-core/src/lib.rs

@@ -25,6 +25,7 @@ pub mod cashflow;
 pub mod classes;
 pub mod conventions;
 pub mod error;
+pub mod evidence;
 pub mod form1099;
 pub mod instruments;
 pub mod journal;
@@ -69,6 +70,7 @@ pub use cashflow::{
 };
 pub use classes::{Class, ClassId, ProjectFacts, ProjectKind, ProjectStatus};
 pub use error::LedgerError;
+pub use evidence::{Annotation, AnnotationFields};
 pub use form1099::{
     Dated1099, Decision1099, ExemptReason, Form1099, Payment1099, PaymentAccounts, Status1099,
     Vendor1099Line, Vendor1099Report, vendor_1099_report,