From 36e1d9d796dcbfd2859d596299e0b823a93620cc Mon Sep 17 00:00:00 2001 From: Stephen Waits Date: Tue, 5 May 2026 10:39:08 -0600 Subject: [PATCH] test(mutants): add cargo-mutants config for advisory mutation testing MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds `.cargo/mutants.toml` configuring cargo-mutants to focus on the algorithmic core (skipping benches, examples, tests_support) and pass `--test-tool=cargo --no-shuffle` so a mutation that breaks the suite gets caught quickly. Mutation testing modifies the source one operator at a time (`>` → `>=`, `+` → `-`, `true` → `false`, etc.) and re-runs the test suite. A mutation that *survives* (tests still pass) is a hint that the test suite isn't checking that bit of behavior — usually because: - The mutated branch is dead code - The unit tests rely on side-effects rather than return values - A property test or invariant is missing Not wired into CI as a gating check (it's slow — every mutation re-runs the whole suite). Run locally with `cargo install cargo-mutants` followed by `cargo mutants --in-diff HEAD~1` for incremental coverage, or `cargo mutants` for a full sweep. The config exclusions list explains *why* each module is skipped — most are the "obvious" kind (benchmark harness, example problems) where mutation kills are not informative. --- .cargo/mutants.toml | 33 +++++++++++++++++++++++++++++++++ CHANGELOG.md | 24 ++++++++++++++++++++++++ 2 files changed, 57 insertions(+) create mode 100644 .cargo/mutants.toml diff --git a/.cargo/mutants.toml b/.cargo/mutants.toml new file mode 100644 index 0000000..3776ab1 --- /dev/null +++ b/.cargo/mutants.toml @@ -0,0 +1,33 @@ +# cargo-mutants configuration for heuropt. +# +# Run with: +# cargo install cargo-mutants +# cargo mutants # full sweep (slow) +# cargo mutants --in-diff HEAD~1 # only mutate recently-changed lines +# +# A *surviving* mutation = the test suite passed despite a code change, +# which usually means a missing test or a missing invariant. +# +# This isn't gated CI; it's an advisory tool. The property tests in +# tests/properties.rs are the natural place to land new invariants +# discovered via mutation runs. + +# Files to skip mutating. We skip: +# - examples (illustrative, not core algorithm correctness) +# - benches (microbench harness, not behavior) +# - the docs/* spec markdown +# - tests_support (test helpers; mutating them changes test inputs, +# not behavior under test) +exclude_globs = [ + "examples/**/*.rs", + "benches/**/*.rs", + "src/tests_support/**/*.rs", +] + +# `cargo-mutants` defaults to `cargo test` for the suite. Keep that. +# `--no-shuffle` makes failure attribution deterministic. +additional_cargo_test_args = ["--", "--test-threads=1"] + +# Time-out per mutated build+test cycle. Big enough for a slow test +# (proptest can take ~10s) but short enough to detect infinite loops. +timeout_multiplier = 5.0 diff --git a/CHANGELOG.md b/CHANGELOG.md index 44ab6fa..df943f7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,30 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Added + +- **Decision tree update** in README to cover all v0.3.0 algorithms, + with a new top-level branch on "is each evaluation expensive?" so + `BayesianOpt` / `Tpe` / `Hyperband` have a clear home. +- **Comparison results snapshot** at `examples/compare-results.md` — + reference output of the harness across 7 benchmark problems and ~20 + algorithms, captured after v0.3.0 landed. +- **Instruction-count benchmarks** via `gungraun` (the Rust 2026 + rename of `iai-callgrind`) at `benches/hot_paths.rs`. Covers + `non_dominated_sort`, `crowding_distance`, `hypervolume_2d`, + `hypervolume_nd` (HSO), and one-generation costs of NSGA-II and + CMA-ES. Stable across machines via callgrind. +- **Property-based test suite** at `tests/properties.rs` using + `proptest`. Covers Pareto-comparison antisymmetry/reflexivity, + `pareto_front`/`non_dominated_sort` partitioning, operator bounds- + preservation (SBX, PolyMut, BoundedGaussianMutation), repair + correctness (ClampToBounds, ProjectToSimplex), and seeded- + determinism on DE and CMA-ES. +- **cargo-mutants config** at `.cargo/mutants.toml` for advisory + mutation testing. Not gated in CI; run with `cargo mutants` to + surface tests that don't actually check the behavior they look like + they do. + ## [0.3.0] — 2026-05-05 Theme: filling heuropt's expensive-evaluation, gradient-free, and