docs(mutants): document the mutation-testing campaign in mutants.toml
Records the outcome of the 2026-05 mutation-testing campaign and the gotchas for future runs: - always pass --all-features so the async runners and explorer module are compiled (otherwise their mutants are unviable/missed noise); - --test-tool nextest needs --no-config because the libtest-style --test-threads=1 arg isn't accepted there; - where the new invariant tests live, and what the residual MISSED / TIMEOUT categories actually represent.
This commit is contained in:
+26
-2
@@ -5,12 +5,36 @@
|
|||||||
# cargo mutants # full sweep (slow)
|
# cargo mutants # full sweep (slow)
|
||||||
# cargo mutants --in-diff HEAD~1 # only mutate recently-changed lines
|
# cargo mutants --in-diff HEAD~1 # only mutate recently-changed lines
|
||||||
#
|
#
|
||||||
|
# IMPORTANT: pass `--all-features` (or at least `--features async,serde`).
|
||||||
|
# Without them the async `run_async` paths and the `explorer` module are
|
||||||
|
# not compiled, so their mutants come back as unviable/missed noise rather
|
||||||
|
# than being exercised by the test suite.
|
||||||
|
#
|
||||||
|
# Note: `--test-tool nextest` does not accept the libtest-style
|
||||||
|
# `--test-threads=1` set below; for a nextest run pass `--no-config` (and
|
||||||
|
# re-add `--all-features` / the `--file` filters you need on the CLI).
|
||||||
|
#
|
||||||
# A *surviving* mutation = the test suite passed despite a code change,
|
# A *surviving* mutation = the test suite passed despite a code change,
|
||||||
# which usually means a missing test or a missing invariant.
|
# which usually means a missing test or a missing invariant.
|
||||||
#
|
#
|
||||||
# This isn't gated CI; it's an advisory tool. The property tests in
|
# This isn't gated CI; it's an advisory tool. The property tests in
|
||||||
# tests/properties.rs are the natural place to land new invariants
|
# tests/properties.rs and the per-algorithm exact-output snapshot tests
|
||||||
# discovered via mutation runs.
|
# in tests/algorithm_properties.rs are the natural places to land new
|
||||||
|
# invariants discovered via mutation runs.
|
||||||
|
#
|
||||||
|
# Mutation-coverage notes (2026-05 campaign — catch rate ~74% -> ~85%):
|
||||||
|
# - tests/algorithm_properties.rs pins an exact final-population
|
||||||
|
# snapshot for every algorithm at a fixed seed. Those snapshots use
|
||||||
|
# deliberately *hard* fixtures (3-D Rosenbrock, an 8-city scattered
|
||||||
|
# TSP, a budget-sensitive multi-fidelity problem): on convex /
|
||||||
|
# trivially-solved problems the optimizers converge to the same
|
||||||
|
# answer regardless of arithmetic mutations, which hides them.
|
||||||
|
# - The residual MISSED mutants are dominated by (a) equivalent
|
||||||
|
# mutants — e.g. `<` vs `<=` at a boundary the inputs never hit —
|
||||||
|
# and (b) arithmetic the optimizers are mathematically robust to.
|
||||||
|
# - TIMEOUT mutants here are loop-bound mutations that make an
|
||||||
|
# offspring-collection loop non-terminating; cargo-mutants reports
|
||||||
|
# those *as detected*, in their own category separate from MISSED.
|
||||||
|
|
||||||
# Files to skip mutating. We skip:
|
# Files to skip mutating. We skip:
|
||||||
# - examples (illustrative, not core algorithm correctness)
|
# - examples (illustrative, not core algorithm correctness)
|
||||||
|
|||||||
Reference in New Issue
Block a user