From fc7ba81f1214593fa9fc3bd75a2439d6c17be9bd Mon Sep 17 00:00:00 2001 From: Stephen Waits Date: Thu, 29 Oct 2026 01:28:50 -0600 Subject: [PATCH] ci: mutation testing is local only (D-59); the weekly run keeps proofs, oracle and fuzz The owner, 2026-10-29: mutation testing is for local runs, not CI. The weekly workflow's mutation shards (core, store, import, config, MCP) are removed; they failed every week on the survivors and timeouts D-33 deferred to P12, and their red hid the run's real findings (a fuzz_store_replay crash and a fuzz_settings_replay build break, both being fixed). just mutants-* stays for the desktop; P12 runs the pass whole. Recorded as D-59; the docs that said the shards keep running are corrected. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01HS8WZziDnyxunC23MUxrLp --- .github/workflows/scheduled.yml | 85 ++------------------------------- README.md | 6 +-- docs/oracle.md | 2 +- docs/testing.md | 3 +- plan/served.md | 1 + 5 files changed, 11 insertions(+), 86 deletions(-) diff --git a/.github/workflows/scheduled.yml b/.github/workflows/scheduled.yml index b47083c..cfeadd2 100644 --- a/.github/workflows/scheduled.yml +++ b/.github/workflows/scheduled.yml @@ -1,8 +1,7 @@ -# Weekly, and on demand: the slow layers (docs/testing.md). -# -# Mutation testing is split into shards (`just mutants-* k/n`), each its -# own job, so every job ends well inside GitHub's six hours. The viewer has -# no mutation job (owner, 2026-10-18). +# Weekly, and on demand: the slow layers (docs/testing.md): the proofs, +# the GnuCash oracle and the fuzz targets. Mutation testing is not run in +# CI (owner, 2026-10-29, D-59): it is a local run (`just mutants-*`), and +# P12 runs it whole on the owner's desktop. name: scheduled on: schedule: @@ -39,79 +38,3 @@ jobs: - uses: actions/checkout@v4 - uses: jdx/mise-action@v2 - run: just fuzz ${{ matrix.target }} 600 - mutants: - runs-on: ubuntu-24.04 - timeout-minutes: 350 - strategy: - fail-fast: false - matrix: - shard: [0, 1, 2, 3, 4, 5, 6, 7] - steps: - - uses: actions/checkout@v4 - - uses: jdx/mise-action@v2 - - run: just mutants ${{ matrix.shard }}/8 - - if: always() - uses: actions/upload-artifact@v4 - with: - name: mutants-core-${{ matrix.shard }} - path: mutants.out - mutants-store: - runs-on: ubuntu-24.04 - timeout-minutes: 350 - strategy: - fail-fast: false - matrix: - shard: [0, 1, 2, 3] - steps: - - uses: actions/checkout@v4 - - uses: jdx/mise-action@v2 - - run: just mutants-store ${{ matrix.shard }}/4 - - if: always() - uses: actions/upload-artifact@v4 - with: - name: mutants-store-${{ matrix.shard }} - path: mutants-store/mutants.out - mutants-import: - runs-on: ubuntu-24.04 - timeout-minutes: 350 - strategy: - fail-fast: false - matrix: - shard: [0, 1] - steps: - - uses: actions/checkout@v4 - - uses: jdx/mise-action@v2 - - run: just mutants-import ${{ matrix.shard }}/2 - - if: always() - uses: actions/upload-artifact@v4 - with: - name: mutants-import-${{ matrix.shard }} - path: mutants-import/mutants.out - mutants-config: - runs-on: ubuntu-24.04 - timeout-minutes: 350 - steps: - - uses: actions/checkout@v4 - - uses: jdx/mise-action@v2 - - run: just mutants-config - - if: always() - uses: actions/upload-artifact@v4 - with: - name: mutants-config - path: mutants-config/mutants.out - mutants-mcp: - runs-on: ubuntu-24.04 - timeout-minutes: 350 - strategy: - fail-fast: false - matrix: - shard: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] - steps: - - uses: actions/checkout@v4 - - uses: jdx/mise-action@v2 - - run: just mutants-mcp ${{ matrix.shard }}/16 - - if: always() - uses: actions/upload-artifact@v4 - with: - name: mutants-mcp-${{ matrix.shard }} - path: mutants-mcp/mutants.out diff --git a/README.md b/README.md index 27321fd..2e2dda9 100644 --- a/README.md +++ b/README.md @@ -51,9 +51,9 @@ oracles, never ship what they haven't demonstrated. pass found three engine bugs that every test, fuzzer, and mutant had missed: negative amounts accepted, and two crashes on oversized amounts. It also caught a soundness bug in the model checker itself. -- Mutation testing via cargo-mutants (paused for development until the - P12 max-testing pass, D-33; CI's weekly shards still run); fuzzing via - cargo-fuzz (11.5M+ executions, zero crashes). +- Mutation testing via cargo-mutants, a local run (paused for development + until the P12 max-testing pass, D-33; not a CI job, D-59); fuzzing via + cargo-fuzz (11.5M+ executions, zero crashes), weekly in CI. - GnuCash is *a* behavioral cross-check, never the authority and never a code source (clean room, MIT OR Apache-2.0). Accounting principles, proved, are the spec. Where any other program disagrees, we work out diff --git a/docs/oracle.md b/docs/oracle.md index 2948971..6e412b9 100644 --- a/docs/oracle.md +++ b/docs/oracle.md @@ -53,7 +53,7 @@ case: partial first/final months in different-length calendar months don't complement, so the lifetime total drifted by cents (fixed with the final-month true-up in `depreciation_schedule`). -**Layer 3 — Mutation testing (paused for development until P12, D-33, 2026-10-20; CI's weekly shards still run).** `cargo mutants -p ledger-core` +**Layer 3 — Mutation testing (a local run, never a CI job, D-59; paused for development until P12, D-33).** `cargo mutants -p ledger-core` (228 mutants as of 2026-10-10: 159 caught, 28 unviable, 2 timeouts from induced infinite loops, 39 missed). Every *missed* mutant is triaged: real gaps get new tests (this pass added parent-account validation, diff --git a/docs/testing.md b/docs/testing.md index 3eba438..e5bbee3 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -37,7 +37,8 @@ promise. | Mutation (GUI) | can the tests fail? | `ledger-gui` view model and formatting (2026-10-14: 140 caught of 176 viable on the full run, then 92 of 99 on a targeted rerun of the money paths after new tests; of its 7 survivors, one then killed by a test (burn over exactly one month, confirmed by a rerun of `compute_kpis`'s 48 mutants), two dead arms removed (`parties`' bill arms: the ledger tags bill entries with their vendor, which wins), four equivalent: an entry's debits vs its credits (they balance), the three-month burn boundary (both paths give 3), burn over zero covered months (only when the first entry is after the window, which then holds no loss), and `compact`'s sign test at zero (zero prints "$0" unsigned either way)) | retired 2026-10-18 (owner: it crushed the machine; for the viewer, test its two musts directly: it never writes the books, `tests/read_only.rs`, and every figure it shows is the engine's, `tests/view_model.rs`) | | never | CI: `.github/workflows/check.yml` runs `just check` on every push; -`scheduled.yml` runs proofs, the oracle, fuzzing, and mutation weekly. Mutation is split into shards, one job each (`just mutants-mcp 3/16` runs the fourth of sixteen), because GitHub stops a job at six hours and a whole crate takes longer on its runners (2026-10-19, the first scheduled run: the core and MCP runs were stopped at the limit). Test timeouts are three times each run's own baseline, never under the recipe's floor, so a slower runner doesn't report false timeouts. The viewer has no mutation job. Development mutation runs are paused until the P12 max-testing pass (D-33, 2026-10-20): each unit records its targets in a **For P12** paragraph instead; the weekly shards keep running in CI. +`two-user.yml` the self-host deployment's two-account check (P9a); +`scheduled.yml` runs the proofs, the oracle and the fuzz targets weekly. Mutation ran there in shards until 2026-10-29 (`just mutants-mcp 3/16` runs the fourth of sixteen; GitHub stops a job at six hours), and is now local only (D-59). Test timeouts are three times each run's own baseline, never under the recipe's floor, so a slower runner doesn't report false timeouts. The viewer has no mutation job. Development mutation runs are paused until the P12 max-testing pass (D-33, 2026-10-20): each unit records its targets in a **For P12** paragraph instead. Mutation testing is a local run, not a CI job (the owner, 2026-10-29, D-59): the weekly shards were removed from `scheduled.yml`, and the "weekly" in the mutation rows above is history (the 2026-10-26 run: real survivors and timeouts in every crate, the debt P12 clears). Seeds and failure databases: proptest's `*.proptest-regressions` files are checked in and replay on every run. The GnuCash oracle's books come diff --git a/plan/served.md b/plan/served.md index 8894284..bb5eaed 100644 --- a/plan/served.md +++ b/plan/served.md @@ -2055,6 +2055,7 @@ The owner decisions of 2026-10-16 (at the top of this document) replace the form | D-55 | Party contact details (owner, 2026-10-27: asked which fields, chose "Full contact block") | A party carries, all optional and journaled on `party.updated` behind `documents_v2`: `phone`, `contact` (a named person), `billing_address`, `shipping_address`, `billing_email`. The existing `email` stays the general address and the existing `address` the general/business one; when `billing_email` or `billing_address` is set, an invoice's bill-to uses it, else the general one. `PartySnapshot` carries the same at issue; the issued PDF prints the contact person, phone, billing address and billing email, and prints a Ship-to line when a shipping address is set. `books.update_party` and `books.create_invoice`'s `bill_to` override take all six; `""` clears a text field. Both emails go through `parties::check_email` (one `@`, no spaces, the full-TIN scan). S4.4 builds the core, store, journal and MCP parts; S4.5 the PDF. Folded into `documents_v2` because the owner's upgrade has not happened (D-52), so no second go-ahead. No projection columns: there is no parties table in the SQLite projection, and replay plus `differs_from` carry the facts. Terms per side are already D-52/S4.2's and are unchanged. | | D-56 | The destination, and when P11a starts (owner, 2026-10-28: the work is "ultimately taking it to a single bundled server+GUI installation for Mac/Windows/Linux and then beyond") | **The installed app is this plan's destination** (D-19, D-22 and D-24 to D-27 unchanged), and the hosted service of section 11 is what comes after it. So P11a is no longer "any time": it starts as soon as an engineer is free, beside the P4b finish, with its new step 0 first (the macOS and Windows keyring stores, the Windows data root and pipes, a macOS and Windows CI build), because today the executable's agent path works on Linux alone. The P10 chain (P4, P5, P8) keeps its order and runs beside it. | | D-57 | Q-5 answered (owner, 2026-10-28, the question tool: "Once, after everything") | **One format upgrade of the live books, after S7 and P4 have landed**, declaring `documents_v2` and `drafts_v2` together (D-52 confirmed). Until then the new writes are refused on the owner's books with the upgrade remedy; a few days without numbered invoices is the accepted cost. A-14 records it. | +| D-59 | Mutation testing in CI (owner, 2026-10-29, asked how the always-red weekly mutation shards should run until P12: "Don't think we need to run the long ass shit like mutation testing in CI. It's more for local test runs.") | **Mutation testing is a local run, never a CI job.** The mutation shards are removed from `scheduled.yml`, which keeps the proofs, the GnuCash oracle and the fuzz targets weekly; `just mutants-*` stays for the desktop, and P12 runs the whole pass there (D-33). The 2026-10-26 weekly run's survivors (real ones in every crate, and timeouts) are P12's list. | | D-58 | Q-8 answered (owner, 2026-10-28, the question tool: "Yes, all stand as built") | **D-47 to D-54 are the owner's decisions**, no longer the review's recommendations: reversals stay posted (D-47), the vault stays v1 (D-48), the P4b deferrals (D-49), the preview digest with no key (D-50), the P9 split (D-51), one P4b upgrade (D-52), no jobs and no new crates (D-53), the P11 split (D-54). The "the owner may revisit" clauses on those rows are closed. | ### 10.2 Deferred