swaits
54f9db5dc9
Records the outcome of the 2026-05 mutation-testing campaign and the gotchas for future runs: - always pass --all-features so the async runners and explorer module are compiled (otherwise their mutants are unviable/missed noise); - --test-tool nextest needs --no-config because the libtest-style --test-threads=1 arg isn't accepted there; - where the new invariant tests live, and what the residual MISSED / TIMEOUT categories actually represent.
58 lines
2.7 KiB
TOML
58 lines
2.7 KiB
TOML
# cargo-mutants configuration for heuropt.
|
|
#
|
|
# Run with:
|
|
# cargo install cargo-mutants
|
|
# cargo mutants # full sweep (slow)
|
|
# cargo mutants --in-diff HEAD~1 # only mutate recently-changed lines
|
|
#
|
|
# IMPORTANT: pass `--all-features` (or at least `--features async,serde`).
|
|
# Without them the async `run_async` paths and the `explorer` module are
|
|
# not compiled, so their mutants come back as unviable/missed noise rather
|
|
# than being exercised by the test suite.
|
|
#
|
|
# Note: `--test-tool nextest` does not accept the libtest-style
|
|
# `--test-threads=1` set below; for a nextest run pass `--no-config` (and
|
|
# re-add `--all-features` / the `--file` filters you need on the CLI).
|
|
#
|
|
# A *surviving* mutation = the test suite passed despite a code change,
|
|
# which usually means a missing test or a missing invariant.
|
|
#
|
|
# This isn't gated CI; it's an advisory tool. The property tests in
|
|
# tests/properties.rs and the per-algorithm exact-output snapshot tests
|
|
# in tests/algorithm_properties.rs are the natural places to land new
|
|
# invariants discovered via mutation runs.
|
|
#
|
|
# Mutation-coverage notes (2026-05 campaign — catch rate ~74% -> ~85%):
|
|
# - tests/algorithm_properties.rs pins an exact final-population
|
|
# snapshot for every algorithm at a fixed seed. Those snapshots use
|
|
# deliberately *hard* fixtures (3-D Rosenbrock, an 8-city scattered
|
|
# TSP, a budget-sensitive multi-fidelity problem): on convex /
|
|
# trivially-solved problems the optimizers converge to the same
|
|
# answer regardless of arithmetic mutations, which hides them.
|
|
# - The residual MISSED mutants are dominated by (a) equivalent
|
|
# mutants — e.g. `<` vs `<=` at a boundary the inputs never hit —
|
|
# and (b) arithmetic the optimizers are mathematically robust to.
|
|
# - TIMEOUT mutants here are loop-bound mutations that make an
|
|
# offspring-collection loop non-terminating; cargo-mutants reports
|
|
# those *as detected*, in their own category separate from MISSED.
|
|
|
|
# Files to skip mutating. We skip:
|
|
# - examples (illustrative, not core algorithm correctness)
|
|
# - benches (microbench harness, not behavior)
|
|
# - the docs/* spec markdown
|
|
# - tests_support (test helpers; mutating them changes test inputs,
|
|
# not behavior under test)
|
|
exclude_globs = [
|
|
"examples/**/*.rs",
|
|
"benches/**/*.rs",
|
|
"src/tests_support/**/*.rs",
|
|
]
|
|
|
|
# `cargo-mutants` defaults to `cargo test` for the suite. Keep that.
|
|
# `--no-shuffle` makes failure attribution deterministic.
|
|
additional_cargo_test_args = ["--", "--test-threads=1"]
|
|
|
|
# Time-out per mutated build+test cycle. Big enough for a slow test
|
|
# (proptest can take ~10s) but short enough to detect infinite loops.
|
|
timeout_multiplier = 5.0
|