ci: mutation testing is local only (D-59); the weekly run keeps proofs, oracle and fuzz

The owner, 2026-10-29: mutation testing is for local runs, not CI. The
weekly workflow's mutation shards (core, store, import, config, MCP) are
removed; they failed every week on the survivors and timeouts D-33
deferred to P12, and their red hid the run's real findings (a
fuzz_store_replay crash and a fuzz_settings_replay build break, both
being fixed). just mutants-* stays for the desktop; P12 runs the pass
whole. Recorded as D-59; the docs that said the shards keep running are
corrected.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HS8WZziDnyxunC23MUxrLp
This commit is contained in:
swaitsandClaude Fable 5.1 committed 2026-10-29 01:28:50 -06:00
1 parent 79fb7ff2fd
commit fc7ba81f12
5 files changed
+11 -86

No files matched your search

+4 -81
View File
@@ -1,8 +1,7 @@
# Weekly, and on demand: the slow layers (docs/testing.md). # Weekly, and on demand: the slow layers (docs/testing.md): the proofs,
# # the GnuCash oracle and the fuzz targets. Mutation testing is not run in
# Mutation testing is split into shards (`just mutants-* k/n`), each its # CI (owner, 2026-10-29, D-59): it is a local run (`just mutants-*`), and
# own job, so every job ends well inside GitHub's six hours. The viewer has # P12 runs it whole on the owner's desktop.
# no mutation job (owner, 2026-10-18).
name: scheduled name: scheduled
on: on:
schedule: schedule:
@@ -39,79 +38,3 @@ jobs:
- uses: actions/checkout@v4 - uses: actions/checkout@v4
- uses: jdx/mise-action@v2 - uses: jdx/mise-action@v2
- run: just fuzz ${{ matrix.target }} 600 - run: just fuzz ${{ matrix.target }} 600
mutants:
runs-on: ubuntu-24.04
timeout-minutes: 350
strategy:
fail-fast: false
matrix:
shard: [0, 1, 2, 3, 4, 5, 6, 7]
steps:
- uses: actions/checkout@v4
- uses: jdx/mise-action@v2
- run: just mutants ${{ matrix.shard }}/8
- if: always()
uses: actions/upload-artifact@v4
with:
name: mutants-core-${{ matrix.shard }}
path: mutants.out
mutants-store:
runs-on: ubuntu-24.04
timeout-minutes: 350
strategy:
fail-fast: false
matrix:
shard: [0, 1, 2, 3]
steps:
- uses: actions/checkout@v4
- uses: jdx/mise-action@v2
- run: just mutants-store ${{ matrix.shard }}/4
- if: always()
uses: actions/upload-artifact@v4
with:
name: mutants-store-${{ matrix.shard }}
path: mutants-store/mutants.out
mutants-import:
runs-on: ubuntu-24.04
timeout-minutes: 350
strategy:
fail-fast: false
matrix:
shard: [0, 1]
steps:
- uses: actions/checkout@v4
- uses: jdx/mise-action@v2
- run: just mutants-import ${{ matrix.shard }}/2
- if: always()
uses: actions/upload-artifact@v4
with:
name: mutants-import-${{ matrix.shard }}
path: mutants-import/mutants.out
mutants-config:
runs-on: ubuntu-24.04
timeout-minutes: 350
steps:
- uses: actions/checkout@v4
- uses: jdx/mise-action@v2
- run: just mutants-config
- if: always()
uses: actions/upload-artifact@v4
with:
name: mutants-config
path: mutants-config/mutants.out
mutants-mcp:
runs-on: ubuntu-24.04
timeout-minutes: 350
strategy:
fail-fast: false
matrix:
shard: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]
steps:
- uses: actions/checkout@v4
- uses: jdx/mise-action@v2
- run: just mutants-mcp ${{ matrix.shard }}/16
- if: always()
uses: actions/upload-artifact@v4
with:
name: mutants-mcp-${{ matrix.shard }}
path: mutants-mcp/mutants.out
+3 -3
View File
@@ -51,9 +51,9 @@ oracles, never ship what they haven't demonstrated.
pass found three engine bugs that every test, fuzzer, and mutant had pass found three engine bugs that every test, fuzzer, and mutant had
missed: negative amounts accepted, and two crashes on oversized missed: negative amounts accepted, and two crashes on oversized
amounts. It also caught a soundness bug in the model checker itself. amounts. It also caught a soundness bug in the model checker itself.
- Mutation testing via cargo-mutants (paused for development until the - Mutation testing via cargo-mutants, a local run (paused for development
P12 max-testing pass, D-33; CI's weekly shards still run); fuzzing via until the P12 max-testing pass, D-33; not a CI job, D-59); fuzzing via
cargo-fuzz (11.5M+ executions, zero crashes). cargo-fuzz (11.5M+ executions, zero crashes), weekly in CI.
- GnuCash is *a* behavioral cross-check, never the authority and never - GnuCash is *a* behavioral cross-check, never the authority and never
a code source (clean room, MIT OR Apache-2.0). Accounting principles, a code source (clean room, MIT OR Apache-2.0). Accounting principles,
proved, are the spec. Where any other program disagrees, we work out proved, are the spec. Where any other program disagrees, we work out
+1 -1
View File
@@ -53,7 +53,7 @@ case: partial first/final months in different-length calendar months don't
complement, so the lifetime total drifted by cents (fixed with the complement, so the lifetime total drifted by cents (fixed with the
final-month true-up in `depreciation_schedule`). final-month true-up in `depreciation_schedule`).
**Layer 3 — Mutation testing (paused for development until P12, D-33, 2026-10-20; CI's weekly shards still run).** `cargo mutants -p ledger-core` **Layer 3 — Mutation testing (a local run, never a CI job, D-59; paused for development until P12, D-33).** `cargo mutants -p ledger-core`
(228 mutants as of 2026-10-10: 159 caught, 28 unviable, 2 timeouts from (228 mutants as of 2026-10-10: 159 caught, 28 unviable, 2 timeouts from
induced infinite loops, 39 missed). Every *missed* mutant is triaged: induced infinite loops, 39 missed). Every *missed* mutant is triaged:
real gaps get new tests (this pass added parent-account validation, real gaps get new tests (this pass added parent-account validation,
+2 -1
View File
@@ -37,7 +37,8 @@ promise.
| Mutation (GUI) | can the tests fail? | `ledger-gui` view model and formatting (2026-10-14: 140 caught of 176 viable on the full run, then 92 of 99 on a targeted rerun of the money paths after new tests; of its 7 survivors, one then killed by a test (burn over exactly one month, confirmed by a rerun of `compute_kpis`'s 48 mutants), two dead arms removed (`parties`' bill arms: the ledger tags bill entries with their vendor, which wins), four equivalent: an entry's debits vs its credits (they balance), the three-month burn boundary (both paths give 3), burn over zero covered months (only when the first entry is after the window, which then holds no loss), and `compact`'s sign test at zero (zero prints "$0" unsigned either way)) | retired 2026-10-18 (owner: it crushed the machine; for the viewer, test its two musts directly: it never writes the books, `tests/read_only.rs`, and every figure it shows is the engine's, `tests/view_model.rs`) | | never | | Mutation (GUI) | can the tests fail? | `ledger-gui` view model and formatting (2026-10-14: 140 caught of 176 viable on the full run, then 92 of 99 on a targeted rerun of the money paths after new tests; of its 7 survivors, one then killed by a test (burn over exactly one month, confirmed by a rerun of `compute_kpis`'s 48 mutants), two dead arms removed (`parties`' bill arms: the ledger tags bill entries with their vendor, which wins), four equivalent: an entry's debits vs its credits (they balance), the three-month burn boundary (both paths give 3), burn over zero covered months (only when the first entry is after the window, which then holds no loss), and `compact`'s sign test at zero (zero prints "$0" unsigned either way)) | retired 2026-10-18 (owner: it crushed the machine; for the viewer, test its two musts directly: it never writes the books, `tests/read_only.rs`, and every figure it shows is the engine's, `tests/view_model.rs`) | | never |
CI: `.github/workflows/check.yml` runs `just check` on every push; CI: `.github/workflows/check.yml` runs `just check` on every push;
`scheduled.yml` runs proofs, the oracle, fuzzing, and mutation weekly. Mutation is split into shards, one job each (`just mutants-mcp 3/16` runs the fourth of sixteen), because GitHub stops a job at six hours and a whole crate takes longer on its runners (2026-10-19, the first scheduled run: the core and MCP runs were stopped at the limit). Test timeouts are three times each run's own baseline, never under the recipe's floor, so a slower runner doesn't report false timeouts. The viewer has no mutation job. Development mutation runs are paused until the P12 max-testing pass (D-33, 2026-10-20): each unit records its targets in a **For P12** paragraph instead; the weekly shards keep running in CI. `two-user.yml` the self-host deployment's two-account check (P9a);
`scheduled.yml` runs the proofs, the oracle and the fuzz targets weekly. Mutation ran there in shards until 2026-10-29 (`just mutants-mcp 3/16` runs the fourth of sixteen; GitHub stops a job at six hours), and is now local only (D-59). Test timeouts are three times each run's own baseline, never under the recipe's floor, so a slower runner doesn't report false timeouts. The viewer has no mutation job. Development mutation runs are paused until the P12 max-testing pass (D-33, 2026-10-20): each unit records its targets in a **For P12** paragraph instead. Mutation testing is a local run, not a CI job (the owner, 2026-10-29, D-59): the weekly shards were removed from `scheduled.yml`, and the "weekly" in the mutation rows above is history (the 2026-10-26 run: real survivors and timeouts in every crate, the debt P12 clears).
Seeds and failure databases: proptest's `*.proptest-regressions` files Seeds and failure databases: proptest's `*.proptest-regressions` files
are checked in and replay on every run. The GnuCash oracle's books come are checked in and replay on every run. The GnuCash oracle's books come
+1
View File
@@ -2055,6 +2055,7 @@ The owner decisions of 2026-10-16 (at the top of this document) replace the form
| D-55 | Party contact details (owner, 2026-10-27: asked which fields, chose "Full contact block") | A party carries, all optional and journaled on `party.updated` behind `documents_v2`: `phone`, `contact` (a named person), `billing_address`, `shipping_address`, `billing_email`. The existing `email` stays the general address and the existing `address` the general/business one; when `billing_email` or `billing_address` is set, an invoice's bill-to uses it, else the general one. `PartySnapshot` carries the same at issue; the issued PDF prints the contact person, phone, billing address and billing email, and prints a Ship-to line when a shipping address is set. `books.update_party` and `books.create_invoice`'s `bill_to` override take all six; `""` clears a text field. Both emails go through `parties::check_email` (one `@`, no spaces, the full-TIN scan). S4.4 builds the core, store, journal and MCP parts; S4.5 the PDF. Folded into `documents_v2` because the owner's upgrade has not happened (D-52), so no second go-ahead. No projection columns: there is no parties table in the SQLite projection, and replay plus `differs_from` carry the facts. Terms per side are already D-52/S4.2's and are unchanged. | | D-55 | Party contact details (owner, 2026-10-27: asked which fields, chose "Full contact block") | A party carries, all optional and journaled on `party.updated` behind `documents_v2`: `phone`, `contact` (a named person), `billing_address`, `shipping_address`, `billing_email`. The existing `email` stays the general address and the existing `address` the general/business one; when `billing_email` or `billing_address` is set, an invoice's bill-to uses it, else the general one. `PartySnapshot` carries the same at issue; the issued PDF prints the contact person, phone, billing address and billing email, and prints a Ship-to line when a shipping address is set. `books.update_party` and `books.create_invoice`'s `bill_to` override take all six; `""` clears a text field. Both emails go through `parties::check_email` (one `@`, no spaces, the full-TIN scan). S4.4 builds the core, store, journal and MCP parts; S4.5 the PDF. Folded into `documents_v2` because the owner's upgrade has not happened (D-52), so no second go-ahead. No projection columns: there is no parties table in the SQLite projection, and replay plus `differs_from` carry the facts. Terms per side are already D-52/S4.2's and are unchanged. |
| D-56 | The destination, and when P11a starts (owner, 2026-10-28: the work is "ultimately taking it to a single bundled server+GUI installation for Mac/Windows/Linux and then beyond") | **The installed app is this plan's destination** (D-19, D-22 and D-24 to D-27 unchanged), and the hosted service of section 11 is what comes after it. So P11a is no longer "any time": it starts as soon as an engineer is free, beside the P4b finish, with its new step 0 first (the macOS and Windows keyring stores, the Windows data root and pipes, a macOS and Windows CI build), because today the executable's agent path works on Linux alone. The P10 chain (P4, P5, P8) keeps its order and runs beside it. | | D-56 | The destination, and when P11a starts (owner, 2026-10-28: the work is "ultimately taking it to a single bundled server+GUI installation for Mac/Windows/Linux and then beyond") | **The installed app is this plan's destination** (D-19, D-22 and D-24 to D-27 unchanged), and the hosted service of section 11 is what comes after it. So P11a is no longer "any time": it starts as soon as an engineer is free, beside the P4b finish, with its new step 0 first (the macOS and Windows keyring stores, the Windows data root and pipes, a macOS and Windows CI build), because today the executable's agent path works on Linux alone. The P10 chain (P4, P5, P8) keeps its order and runs beside it. |
| D-57 | Q-5 answered (owner, 2026-10-28, the question tool: "Once, after everything") | **One format upgrade of the live books, after S7 and P4 have landed**, declaring `documents_v2` and `drafts_v2` together (D-52 confirmed). Until then the new writes are refused on the owner's books with the upgrade remedy; a few days without numbered invoices is the accepted cost. A-14 records it. | | D-57 | Q-5 answered (owner, 2026-10-28, the question tool: "Once, after everything") | **One format upgrade of the live books, after S7 and P4 have landed**, declaring `documents_v2` and `drafts_v2` together (D-52 confirmed). Until then the new writes are refused on the owner's books with the upgrade remedy; a few days without numbered invoices is the accepted cost. A-14 records it. |
| D-59 | Mutation testing in CI (owner, 2026-10-29, asked how the always-red weekly mutation shards should run until P12: "Don't think we need to run the long ass shit like mutation testing in CI. It's more for local test runs.") | **Mutation testing is a local run, never a CI job.** The mutation shards are removed from `scheduled.yml`, which keeps the proofs, the GnuCash oracle and the fuzz targets weekly; `just mutants-*` stays for the desktop, and P12 runs the whole pass there (D-33). The 2026-10-26 weekly run's survivors (real ones in every crate, and timeouts) are P12's list. |
| D-58 | Q-8 answered (owner, 2026-10-28, the question tool: "Yes, all stand as built") | **D-47 to D-54 are the owner's decisions**, no longer the review's recommendations: reversals stay posted (D-47), the vault stays v1 (D-48), the P4b deferrals (D-49), the preview digest with no key (D-50), the P9 split (D-51), one P4b upgrade (D-52), no jobs and no new crates (D-53), the P11 split (D-54). The "the owner may revisit" clauses on those rows are closed. | | D-58 | Q-8 answered (owner, 2026-10-28, the question tool: "Yes, all stand as built") | **D-47 to D-54 are the owner's decisions**, no longer the review's recommendations: reversals stay posted (D-47), the vault stays v1 (D-48), the P4b deferrals (D-49), the preview digest with no key (D-50), the P9 split (D-51), one P4b upgrade (D-52), no jobs and no new crates (D-53), the P11 split (D-54). The "the owner may revisit" clauses on those rows are closed. |
### 10.2 Deferred ### 10.2 Deferred