P3: the IIF in plain ASCII with every change listed, and the CPA package as of one day (defects 7, 21)
The CPA package's general-journal.iif now holds every posted entry through as_of (closing entries left out, D-36), written as plain ASCII with CR LF, DOCNUM the entry id, QuickBooks Desktop's field limits and subaccount types honoured, and every character written differently listed in the export's warnings and the README (grouped by substitution, TINs masked). The 1099 worksheet stops at as_of too. PDFs print CJK, Arabic, Hebrew, Thai and Devanagari (Noto fallbacks, OFL), and so does the viewer. has_full_tin reads Unicode separators. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T3gXMVt7XJPi6ens1or9Dj
This commit is contained in:
48 files changed
+5661
-317
No files matched your search
@@ -171,7 +171,7 @@ pub fn vendor_1099_report_with(
|
||||
threshold: Option<Decimal>,
|
||||
sources: &PaymentAccounts,
|
||||
) -> Result<Vendor1099Report, LedgerError> {
|
||||
vendor_1099_report_with_members(ledger, year, threshold, sources, &[])
|
||||
vendor_1099_report_with_members(ledger, year, None, threshold, sources, &[])
|
||||
}
|
||||
|
||||
/// What an entry paid out of `sources`: credits minus debits to them.
|
||||
@@ -211,15 +211,16 @@ fn vendor_part(payment: Decimal, share: Decimal, target: Decimal) -> Result<Deci
|
||||
}
|
||||
}
|
||||
|
||||
/// [`vendor_1099_report`], counting payments out of `sources`, each
|
||||
/// vendor's part of an entry with several parties resolved with `members`
|
||||
/// (the company's members, from the settings).
|
||||
/// [`vendor_1099_report`], counting payments out of `sources` made in
|
||||
/// `year` (through `through`, when given), each vendor's part of an entry
|
||||
/// with several parties resolved with `members` (the company's members,
|
||||
/// from the settings).
|
||||
///
|
||||
/// ```text
|
||||
/// SOURCE: docs/spec/rules.md B-19 (26 U.S.C. §6041(a): payments made in
|
||||
/// the year; card payments go on 1099-K); crate::shares (an entry
|
||||
/// with several parties, each with their share; DECISION, owner,
|
||||
/// 2026-10-15).
|
||||
/// the year; card payments go on 1099-K), B-33 (a worksheet as of
|
||||
/// a day); crate::shares (an entry with several parties, each with
|
||||
/// their share; DECISION, owner, 2026-10-15).
|
||||
/// DEFINITIONS:
|
||||
/// payment(e): credits minus debits to `sources` on entry e.
|
||||
/// money: crate::parties::money_accounts(ledger, sources.cash), what the
|
||||
@@ -227,8 +228,12 @@ fn vendor_part(payment: Decimal, share: Decimal, target: Decimal) -> Result<Deci
|
||||
/// target(e): crate::shares::share_target(e, money): the entry_flow net,
|
||||
/// else the total debits.
|
||||
/// INVARIANTS:
|
||||
/// 1. Only posted entries dated in `year` and tagged with a vendor
|
||||
/// (Ledger::entry_vendor) count, toward that vendor.
|
||||
/// 1. Only posted entries dated in `year` (and, given `through`, on or
|
||||
/// before it) and tagged with a vendor (Ledger::entry_vendor) count,
|
||||
/// toward that vendor. `through` makes a worksheet as of a day: the
|
||||
/// payments from January 1 through it (§6041(a) and §6041A(a)(2)
|
||||
/// count what is paid "in any calendar year"; a CPA package as of
|
||||
/// June 30 shows the year to date, rules.md B-33).
|
||||
/// 2. An entry naming at most one party (Ledger::entry_parties): the
|
||||
/// vendor's amount is payment(e), exactly as before shares.
|
||||
/// 3. An entry naming several parties (one of them the vendor's party,
|
||||
@@ -263,6 +268,7 @@ fn vendor_part(payment: Decimal, share: Decimal, target: Decimal) -> Result<Deci
|
||||
pub fn vendor_1099_report_with_members(
|
||||
ledger: &Ledger,
|
||||
year: i32,
|
||||
through: Option<chrono::NaiveDate>,
|
||||
threshold: Option<Decimal>,
|
||||
sources: &PaymentAccounts,
|
||||
members: &[crate::PartyMember],
|
||||
@@ -275,7 +281,7 @@ pub fn vendor_1099_report_with_members(
|
||||
let mut paid: std::collections::HashMap<VendorId, Decimal> = Default::default();
|
||||
let mut unresolved = Vec::new();
|
||||
for entry in ledger.posted_entries() {
|
||||
if entry.date().year() != year {
|
||||
if entry.date().year() != year || through.is_some_and(|t| entry.date() > t) {
|
||||
continue;
|
||||
}
|
||||
let Some(vendor) = ledger.entry_vendor(entry.id()) else {
|
||||
|
||||
@@ -13,8 +13,10 @@
|
||||
//! 2026-10-15); IRS truncated TINs: Treas. Reg. §301.6109-4.
|
||||
//! INVARIANTS:
|
||||
//! 1. No party field holds a full taxpayer identification number: any
|
||||
//! run of nine digits (single spaces or hyphens between them allowed)
|
||||
//! is refused. A W-9's TIN is kept as its last four digits only.
|
||||
//! run of nine digits, in any script, separated or not (`has_full_tin`:
|
||||
//! up to three spaces, hyphens, dots, slashes or underscores, in any
|
||||
//! of their Unicode forms, between neighbors) is refused. A W-9's TIN
|
||||
//! is kept as its last four digits only.
|
||||
//! 2. W-9 facts need a 1099 role (contractor or landlord); W-4 facts need
|
||||
//! the employee role. Taxpayer-ID facts (`TinFacts`: the TIN's type,
|
||||
//! last four, and vault record) are allowed for any role: members
|
||||
@@ -272,33 +274,100 @@ pub fn name_key(name: &str) -> String {
|
||||
name.trim().to_lowercase()
|
||||
}
|
||||
|
||||
/// The characters that may sit between a TIN's digits: space, hyphen,
|
||||
/// dot, slash, underscore, and the Unicode dashes and no-break space
|
||||
/// (`291-12-2846`, `291.12.2846`, `291/12/2846`, `291 - 12 - 2846`).
|
||||
/// The characters that may sit between a TIN's digits (`291-12-2846`,
|
||||
/// `291.12.2846`, `291/12/2846`, `291 - 12 - 2846`): in ASCII a space,
|
||||
/// hyphen, dot, slash or underscore (no other, not a tab or a line
|
||||
/// break); past ASCII, every form of them Unicode has, so a TIN can't slip
|
||||
/// through typed in another script (`docs/spec/sources/unicode-tin-text.md`,
|
||||
/// Unicode 17.0): every space (general category Zs: the no-break, thin and
|
||||
/// ideographic spaces), every dash (property Dash: the hyphens, en and em
|
||||
/// dashes, minus sign, Hebrew maqaf, fullwidth and small hyphen-minus),
|
||||
/// and the characters whose compatibility form is a dot, slash or
|
||||
/// underscore (the fullwidth and small full stops, the one dot leader,
|
||||
/// the fullwidth solidus, the fullwidth and presentation low lines).
|
||||
fn tin_separator(c: char) -> bool {
|
||||
match c {
|
||||
' ' | '-' | '.' | '/' | '_' => true,
|
||||
c if c.is_ascii() => false,
|
||||
// White_Space less the line breaks (NEL, LS, PS): the Zs spaces.
|
||||
c if c.is_whitespace() => !matches!(c, '\u{85}' | '\u{2028}' | '\u{2029}'),
|
||||
c => matches!(
|
||||
c,
|
||||
// Dash (PropList.txt), less U+002D above.
|
||||
'\u{58A}'
|
||||
| '\u{5BE}'
|
||||
| '\u{1400}'
|
||||
| '\u{1806}'
|
||||
| '\u{2010}'..='\u{2015}'
|
||||
| '\u{2053}'
|
||||
| '\u{207B}'
|
||||
| '\u{208B}'
|
||||
| '\u{2212}'
|
||||
| '\u{2E17}'
|
||||
| '\u{2E1A}'
|
||||
| '\u{2E3A}'..='\u{2E3B}'
|
||||
| '\u{2E40}'
|
||||
| '\u{2E5D}'
|
||||
| '\u{301C}'
|
||||
| '\u{3030}'
|
||||
| '\u{30A0}'
|
||||
| '\u{FE31}'..='\u{FE32}'
|
||||
| '\u{FE58}'
|
||||
| '\u{FE63}'
|
||||
| '\u{FF0D}'
|
||||
| '\u{10D6E}'
|
||||
| '\u{10EAD}'
|
||||
// NFKC '.', '/', '_' (UnicodeData.txt decompositions).
|
||||
| '\u{2024}'
|
||||
| '\u{FE52}'
|
||||
| '\u{FF0E}'
|
||||
| '\u{FF0F}'
|
||||
| '\u{FE33}'..='\u{FE34}'
|
||||
| '\u{FE4D}'..='\u{FE4F}'
|
||||
| '\u{FF3F}'
|
||||
),
|
||||
}
|
||||
}
|
||||
|
||||
/// Whether `c` is invisible, so a TIN's digits read as neighbors across it
|
||||
/// however many there are (`1\u{200B}2` shows as `12`): Unicode's
|
||||
/// Default_Ignorable_Code_Point (DerivedCoreProperties.txt, 17.0): the
|
||||
/// soft hyphen, the zero-width space and joiners, the left-to-right and
|
||||
/// right-to-left marks and the other directional formatting characters,
|
||||
/// the word joiner, the byte-order mark, the variation selectors, the tags
|
||||
/// and the fillers. None is ASCII.
|
||||
fn tin_invisible(c: char) -> bool {
|
||||
matches!(
|
||||
c,
|
||||
' ' | '-'
|
||||
| '.'
|
||||
| '/'
|
||||
| '_'
|
||||
| '\u{a0}'
|
||||
| '\u{2010}'
|
||||
| '\u{2011}'
|
||||
| '\u{2012}'
|
||||
| '\u{2013}'
|
||||
| '\u{2014}'
|
||||
| '\u{2212}'
|
||||
'\u{AD}'
|
||||
| '\u{34F}'
|
||||
| '\u{61C}'
|
||||
| '\u{115F}'..='\u{1160}'
|
||||
| '\u{17B4}'..='\u{17B5}'
|
||||
| '\u{180B}'..='\u{180F}'
|
||||
| '\u{200B}'..='\u{200F}'
|
||||
| '\u{202A}'..='\u{202E}'
|
||||
| '\u{2060}'..='\u{206F}'
|
||||
| '\u{3164}'
|
||||
| '\u{FE00}'..='\u{FE0F}'
|
||||
| '\u{FEFF}'
|
||||
| '\u{FFA0}'
|
||||
| '\u{FFF0}'..='\u{FFF8}'
|
||||
| '\u{1BCA0}'..='\u{1BCA3}'
|
||||
| '\u{1D173}'..='\u{1D17A}'
|
||||
| '\u{E0000}'..='\u{E0FFF}'
|
||||
)
|
||||
}
|
||||
|
||||
/// The most separators that may sit between two of a TIN's digits
|
||||
/// (`" - "` is three).
|
||||
/// (`" - "` is three); invisible characters ([`tin_invisible`]) don't
|
||||
/// count.
|
||||
const MAX_TIN_GAP: usize = 3;
|
||||
|
||||
/// Each run in `chars` that holds nine digits or more, as (first, last)
|
||||
/// char indices of its digits: digits with at most [`MAX_TIN_GAP`]
|
||||
/// separators ([`tin_separator`]) between neighbors.
|
||||
/// separators ([`tin_separator`]) between neighbors, and any number of
|
||||
/// invisible characters.
|
||||
fn full_tin_runs(chars: &[char]) -> Vec<(usize, usize)> {
|
||||
let mut runs = Vec::new();
|
||||
let mut i = 0;
|
||||
@@ -315,11 +384,19 @@ fn full_tin_runs(chars: &[char]) -> Vec<(usize, usize)> {
|
||||
j += 1;
|
||||
continue;
|
||||
}
|
||||
let gap = chars[j..].iter().take_while(|c| tin_separator(**c)).count();
|
||||
if (1..=MAX_TIN_GAP).contains(&gap)
|
||||
&& chars.get(j + gap).is_some_and(|c| c.is_numeric())
|
||||
let between = chars[j..]
|
||||
.iter()
|
||||
.take_while(|c| tin_separator(**c) || tin_invisible(**c))
|
||||
.count();
|
||||
let gap = chars[j..j + between]
|
||||
.iter()
|
||||
.filter(|c| !tin_invisible(**c))
|
||||
.count();
|
||||
if between > 0
|
||||
&& gap <= MAX_TIN_GAP
|
||||
&& chars.get(j + between).is_some_and(|c| c.is_numeric())
|
||||
{
|
||||
j += gap;
|
||||
j += between;
|
||||
} else {
|
||||
break;
|
||||
}
|
||||
@@ -333,11 +410,13 @@ fn full_tin_runs(chars: &[char]) -> Vec<(usize, usize)> {
|
||||
}
|
||||
|
||||
/// Whether `text` holds a run of nine digits, with at most three
|
||||
/// separators (space, hyphen, dot, slash, underscore, a Unicode dash or
|
||||
/// no-break space) between neighbors: an SSN, EIN, or ITIN however it's
|
||||
/// written (`291-12-2846`, `291.12.2846`, `291 - 12 - 2846`). Any Unicode
|
||||
/// numeral counts as a digit (full-width, Arabic-Indic, ...), so a number
|
||||
/// can't slip through in another script.
|
||||
/// separators (a space, hyphen, dot, slash or underscore, in any of
|
||||
/// their Unicode forms) between neighbors, invisible
|
||||
/// characters not counted: an SSN, EIN, or ITIN however it's written
|
||||
/// (`291-12-2846`, `291.12.2846`, `291 - 12 - 2846`, `２９１－１２－２８４６`).
|
||||
/// Any Unicode numeral counts as a digit (full-width, Arabic-Indic, ...),
|
||||
/// so a number can't slip through in another script. In ASCII, exactly
|
||||
/// the space, hyphen, dot, slash and underscore separate.
|
||||
pub fn has_full_tin(text: &str) -> bool {
|
||||
let chars: Vec<char> = text.chars().collect();
|
||||
!full_tin_runs(&chars).is_empty()
|
||||
|
||||
@@ -365,6 +365,7 @@ fn a_shared_payment_counts_the_vendors_share() {
|
||||
&s.l,
|
||||
2026,
|
||||
None,
|
||||
None,
|
||||
&ledger_core::PaymentAccounts {
|
||||
cash: vec![bank],
|
||||
financing: vec![],
|
||||
|
||||
@@ -177,7 +177,7 @@ fn has_full_tin_truth_table() {
|
||||
("291\u{2013}12\u{2013}2846", true), // en dashes
|
||||
("291\u{a0}12\u{a0}2846", true), // no-break spaces
|
||||
("12 3456789", false), // four separators break the run: 2 + 7
|
||||
("571, 93, 2846", false), // a comma is not a separator
|
||||
("291, 12, 2846", false), // a comma is not a separator
|
||||
("12\t3456789", false), // a tab is not a space
|
||||
("12345678", false), // eight digits
|
||||
("123-45-678", false),
|
||||
@@ -241,6 +241,145 @@ fn has_full_tin_unicode_digits() {
|
||||
assert!(has_full_tin("١٢٣٤٥٦٧٨٩")); // Arabic-Indic digits
|
||||
}
|
||||
|
||||
/// Oracle: `has_full_tin` doc (P3 review round 2): past ASCII, every form
|
||||
/// of the five separators Unicode has separates (every Zs space, every
|
||||
/// Dash character, the compatibility forms of the dot, slash and
|
||||
/// underscore; `docs/spec/sources/unicode-tin-text.md`, Unicode 17.0),
|
||||
/// and invisible characters (Default_Ignorable_Code_Point) don't count
|
||||
/// toward the gap. A TIN typed in fullwidth forms, Arabic-Indic digits,
|
||||
/// with an en dash, a non-breaking hyphen, a Hebrew maqaf, any space, or
|
||||
/// with right-to-left marks around its hyphens is still a full TIN; line
|
||||
/// breaks, commas and a fourth visible separator still end a run.
|
||||
/// (Before, "１２３－４５－６７８９" passed: the fullwidth hyphen-minus wasn't
|
||||
/// a separator, nor the ideographic or thin space.)
|
||||
#[test]
|
||||
fn has_full_tin_reads_every_script_and_separator() {
|
||||
let tins = [
|
||||
// Fullwidth digits and hyphen-minus.
|
||||
"\u{FF11}\u{FF12}\u{FF13}\u{FF0D}\u{FF14}\u{FF15}\u{FF0D}\u{FF16}\u{FF17}\u{FF18}\u{FF19}",
|
||||
"291\u{FF0D}12\u{FF0D}2846", // fullwidth hyphen-minus
|
||||
"291\u{FE63}12\u{FE63}2846", // small hyphen-minus
|
||||
// Arabic-Indic, Extended Arabic-Indic, Devanagari (en dashes).
|
||||
"\u{661}\u{662}\u{663}-\u{664}\u{665}-\u{666}\u{667}\u{668}\u{669}",
|
||||
"\u{6F1}\u{6F2}\u{6F3}\u{6F4}\u{6F5}\u{6F6}\u{6F7}\u{6F8}\u{6F9}",
|
||||
"\u{967}\u{968}\u{969}\u{2013}\u{96A}\u{96B}\u{2013}\u{96C}\u{96D}\u{96E}\u{96F}",
|
||||
"291\u{2013}12\u{2013}2846", // en dash
|
||||
"291\u{2011}12\u{2011}2846", // non-breaking hyphen
|
||||
"291\u{2012}12\u{2012}2846", // figure dash
|
||||
"291\u{2015}12\u{2015}2846", // horizontal bar
|
||||
"291\u{5BE}12\u{5BE}2846", // Hebrew maqaf
|
||||
"291\u{301C}12\u{301C}2846", // wave dash
|
||||
"291\u{A0}12\u{A0}2846", // no-break space
|
||||
"291\u{3000}12\u{3000}2846", // ideographic space
|
||||
"291\u{2009}12\u{2009}2846", // thin space
|
||||
"291\u{202F}12\u{202F}2846", // narrow no-break space
|
||||
"291 \u{2014} 12 \u{2014} 2846", // three: space, em dash, space
|
||||
"291\u{FF0E}12\u{FF0E}2846", // fullwidth full stop
|
||||
"291\u{2024}12\u{2024}2846", // one dot leader
|
||||
"291\u{FF0F}12\u{FF0F}2846", // fullwidth solidus
|
||||
"291\u{FF3F}12\u{FF3F}2846", // fullwidth low line
|
||||
// Zero-width spaces between every digit.
|
||||
"2\u{200B}9\u{200B}1\u{200B}1\u{200B}2\u{200B}2\u{200B}8\u{200B}4\u{200B}6",
|
||||
"291\u{200F}-\u{200F}12\u{200F}-\u{200F}2846", // right-to-left marks around hyphens
|
||||
"291\u{200E}\u{200E}\u{200E}\u{200E} 12 2846", // invisible marks don't count
|
||||
"291\u{AD}12\u{AD}2846", // soft hyphens
|
||||
"291\u{2060}122846", // word joiner
|
||||
"\u{FEFF}291122846",
|
||||
];
|
||||
for t in tins {
|
||||
assert!(has_full_tin(t), "has_full_tin({t:?})");
|
||||
let masked = redact_full_tins(t);
|
||||
assert!(!has_full_tin(&masked), "{t:?} masked: {masked:?}");
|
||||
assert_eq!(
|
||||
masked.chars().count(),
|
||||
t.chars().count(),
|
||||
"only digits change: {masked:?}"
|
||||
);
|
||||
}
|
||||
assert_eq!(
|
||||
redact_full_tins(
|
||||
"ref \u{FF11}\u{FF12}\u{FF13}\u{FF0D}\u{FF14}\u{FF15}\u{FF0D}\u{FF16}\u{FF17}\u{FF18}\u{FF19}"
|
||||
),
|
||||
"ref •••\u{FF0D}••\u{FF0D}••••"
|
||||
);
|
||||
assert_eq!(
|
||||
redact_full_tins("291\u{200F}-\u{200F}12-2846 ok"),
|
||||
"•••\u{200F}-\u{200F}••-•••• ok"
|
||||
);
|
||||
let not = [
|
||||
"12\u{3000}\u{3000}\u{3000}\u{3000}3456789", // four visible separators
|
||||
"12 \u{2014} \u{200B}-3456789", // four, an invisible one among them
|
||||
"291\u{2028}12\u{2028}2846", // line separators end a run
|
||||
"291\u{2029}12\u{2029}2846", // paragraph separators
|
||||
"291\u{85}12\u{85}2846", // next line
|
||||
"291\u{FF0C}12\u{FF0C}2846", // fullwidth commas
|
||||
"291\u{3001}12\u{3001}2846", // ideographic commas
|
||||
"\u{FF11}\u{FF12}\u{FF13}\u{FF14}\u{FF15}\u{FF16}\u{FF17}\u{FF18}", // eight digits
|
||||
"291\u{200B}12284", // eight, invisibly joined
|
||||
];
|
||||
for t in not {
|
||||
assert!(!has_full_tin(t), "has_full_tin({t:?})");
|
||||
}
|
||||
}
|
||||
|
||||
/// The reading before P3 review round 2, the oracle for ASCII: past it,
|
||||
/// only the no-break space and a few dashes separated, and every
|
||||
/// separator counted.
|
||||
fn has_full_tin_before(text: &str) -> bool {
|
||||
let sep = |c: char| {
|
||||
matches!(
|
||||
c,
|
||||
' ' | '-' | '.' | '/' | '_' | '\u{a0}' | '\u{2010}'..='\u{2014}' | '\u{2212}'
|
||||
)
|
||||
};
|
||||
let chars: Vec<char> = text.chars().collect();
|
||||
let mut i = 0;
|
||||
while i < chars.len() {
|
||||
if !chars[i].is_numeric() {
|
||||
i += 1;
|
||||
continue;
|
||||
}
|
||||
let (mut digits, mut last, mut j) = (1, i, i + 1);
|
||||
while j < chars.len() {
|
||||
if chars[j].is_numeric() {
|
||||
digits += 1;
|
||||
last = j;
|
||||
j += 1;
|
||||
continue;
|
||||
}
|
||||
let gap = chars[j..].iter().take_while(|c| sep(**c)).count();
|
||||
if (1..=3).contains(&gap) && chars.get(j + gap).is_some_and(|c| c.is_numeric()) {
|
||||
j += gap;
|
||||
} else {
|
||||
break;
|
||||
}
|
||||
}
|
||||
if digits >= 9 {
|
||||
return true;
|
||||
}
|
||||
i = last + 1;
|
||||
}
|
||||
false
|
||||
}
|
||||
|
||||
proptest! {
|
||||
#![proptest_config(ProptestConfig::with_cases(2000))]
|
||||
|
||||
/// `has_full_tin` reads ASCII exactly as it did (the fix only widens
|
||||
/// what separates past ASCII), and what it found before in any text
|
||||
/// it still finds: it only ever finds more.
|
||||
#[test]
|
||||
fn has_full_tin_keeps_its_ascii_reading(
|
||||
ascii in "[0-9 ./_\t\n\r,:a-]{0,40}",
|
||||
any in "[0-9 ./_\t\n,a\u{A0}\u{2010}-\u{2015}\u{2212}\u{FF0D}\u{3000}\u{200B}\u{200F}\u{FF10}-\u{FF19}\u{660}-\u{669}\u{2028}-]{0,40}",
|
||||
) {
|
||||
prop_assert_eq!(has_full_tin(&ascii), has_full_tin_before(&ascii), "{:?}", ascii);
|
||||
if has_full_tin_before(&any) {
|
||||
prop_assert!(has_full_tin(&any), "{:?}", any);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------- check_text
|
||||
|
||||
/// Oracle: `check_text` doc: not a full TIN, at most 500 characters, no
|
||||
|
||||
Reference in new issue
Block a user