Phase 1 tests for src/algorithms/cma_es.rs.
- compare_so: feasibility-first ordering, minimize/maximize inversion,
infeasibility-violation comparison.
- better_than_so: matches compare_so == Less; equal evaluations are
not strictly better.
- A 30-generation Sphere1D run pins that CMA-ES actually decreases
the objective (catches mutants that collapse the update rules).
Phase 1 tests for src/algorithms/bayesian_opt.rs. Adds 15 tests
pinning the GP regression and EI acquisition machinery:
- rbf_kernel: signal-variance return at zero distance, exp(-0.5) at
unit distance, monotone in length scale, decays to 0 for far points.
- normal_pdf: symmetric about zero, value at zero equals 1/sqrt(2π).
- normal_cdf: 0.5 at z=0, symmetric tail sums to 1.
- erf: odd function and erf(0) ≈ 0 within the rational approximation's
~1e-7 accuracy.
- expected_improvement: zero at sigma=0, monotone in sigma, positive
when mu < f_best.
- oriented_target: sign flips under direction, infeasible adds 1e6
penalty.
- better: feasibility-first then objective ordering under both
directions.
Phase 1 for src/algorithms/ant_colony_tsp.rs. Targets the ~30 remaining
mutants after Phase 0 sweeps — mostly arithmetic flips in build_tour
(pheromone × heuristic weighting) and the feasibility-comparison logic
in better_than_so.
Added:
- Four branch tests for better_than_so covering the full feasibility
cross-product: feasible-vs-infeasible (both orders), two-infeasible
(smaller violation wins), and two-feasible under both directions.
Plus an equal-objectives test pinning the strict-less-than semantics.
- build_tour-is-a-permutation invariant across 20 seeds × 6 start cities.
- A strong-heuristic test: with eta favoring the next-city by 1000x and
beta=5, build_tour walks the preferred path. Pins the .powf(beta)
arithmetic.
- Zero-alpha/zero-beta degenerate-case test: uniform random fallback
still returns a permutation.
Phase 1 for src/algorithms/age_moea.rs. Targets the ~50 algorithm-internal
mutants surviving after the Phase 0 sweeps:
Pure helper-fn pins (lp_norm / lp_distance / nearest_neighbor_distance /
estimate_p):
- L_p norm at p ∈ {1, 2} on canonical vectors (unit, all-ones,
Pythagorean 3-4-5, signed-via-abs).
- L_p distance: zero-to-itself = 0, symmetry, L_1/L_2 sanity values.
- nearest_neighbor_distance: empty selected → ∞, self-only → ∞, picks
the closest of mixed-distance candidates.
- estimate_p: empty-front fallback to p=2; axis-aligned extremes (CV=0
for all p, returns first candidate 0.25); corner-vs-diagonal extremes
(CV minimized at large p).
Full-run pins:
- A 10-generation Schaffer-N1 run with seed 7 verifies the pareto front
is non-empty and finite/nonneg (catches body collapse mutants).
- Population size after run matches config across three pop sizes
(catches size mutants).
- Evaluation count falls in [pop, pop*(gens+1)] (catches comparison
flips in the offspring-collection loop).
Phase 1 of the mutation-testing campaign for src/operators/real.rs (the
file with the largest mutant surface — 128 missed mutants spread across
GaussianMutation, BoundedGaussianMutation, SBX, PolynomialMutation,
LevyMutation, and the Mantegna gamma/sigma helpers).
Added:
- Seed-pinned numerical snapshots for each operator's vary() output on
a fixed parent and seed. Any arithmetic flip in the operator's math
changes one of the snapshot values and fails the assertion. The
snapshot tolerance is 1e-12 so even subtle FP drift is caught.
- An algebraic-identity test for SBX: c1 + c2 = p1 + p2 per dimension
before clamping. This identity holds for any β and pins the
(1+β)·p1 + (1-β)·p2 formula cleanly across 20 seeds.
- A scale-coupling test for PolynomialMutation: a 10× wider bound range
produces a 10× larger perturbation step at the same seed. Catches any
mutation that breaks the δ·(hi-lo) coupling.
- Three direct pin tests for mantegna_sigma_u (alpha = 1.5, 1.0, 2.0)
exercising the gamma() Lanczos series and the formula's edge cases
(alpha = 1.0 → Cauchy, alpha = 2.0 → Normal-limit where sin(π) ≈ 0).
- A monotonicity property test (sigma_u changes with alpha) to catch
structural mutants that collapse the formula to a constant.
Phase 1 of the mutation-testing campaign for src/operators/permutation.rs.
Adds 13 tests targeting the 30 surviving mutants in the new permutation
toolkit:
Mutation operators (Inversion / Insertion / Scramble):
- Previously only checked that the output was a valid permutation, which
passes trivially when the mutant 'replace >= 2 with < 2' skips the
guard entirely (no mutation = identity output = still a permutation).
New tests run 30 seeds on an 8-element parent and assert at least one
seed produces a non-identity output. Kills the >= ↔ < flips.
ShuffledMultisetPermutation::initialize:
- Tightened to assert pop.len() == size up-front, killing the 'replace
with vec![]' mutant.
Crossover operators (OX / PMX / CX / ERX):
- 'Recombines for n >= 3' tests: with 5-element distinct parents, some
seed must produce a child differing from both parents. Kills the
< ↔ > / == / <= guard flips that would early-return parents at n >= 3.
- CX-specific pinned tests: the single-cycle case (children = parents)
and the two-cycle case (exactly known output). Pins the cycle-detection
arithmetic and the parent-alternation logic — kills the ==↔!= and
+= ↔ *= mutants inside cx_child.
- ERX: 'distinct starts can yield distinct children' across 30 seeds —
kills the prev/next-index arithmetic mutants in the adjacency table.
Some residual mutants in this file are equivalent (e.g., < ↔ <= when
n=2 still produces the same OX result because for length-2 inputs the
segment-and-fill recombination converges to the parents anyway).
Documented in test comments.
The degenerate-magnitude shortcut in ProjectToSimplex::repair scans
`decision` for the argmax and concentrates all mass there. cargo
mutants found that the strict-greater scan was unpinned: replacing
`>` with `>=` (which would shift the argmax to the last tied
index) and `>` with `==` (which would silently skip larger
values further along) both survived.
Two tests:
- A 3-element vector with two tied maxima at the front pins that the
scan keeps the first index on a tie.
- A 3-element vector whose argmax is at index 1 pins that the scan
actually walks past the start when later values are larger.
Other mutants in this file (the `>` ↔ `>=` threshold check at line
106, the `*` ↔ `+` in the threshold constant, the `-` ↔ `+` /
`/` in the tau-fallback initializer, and the `>` ↔ `>=` in the
projection loop's rho update) are equivalent mutants for non-pathological
inputs: the normal-path and shortcut-path math converge to the same
projection result for any input the operator is documented to handle.
Leaving them in the residue.
Phase 0.3 of the mutation-testing campaign. Extends the inline tests in
src/explorer/mod.rs with 24 new tests covering the gaps cargo-mutants
identified — about 30 surviving mutants in this one file.
Coverage added:
- Exact-output tests for ToDecisionValues impls on Vec<f64>, Vec<i64>,
Vec<bool>, Vec<usize> (the previous tests only asserted lengths or
spot-checked individual entries).
- from_result propagates evaluations / generations from the
OptimizationResult into RunMeta.
- with_problem_name / with_wall_clock / with_timestamp each set their
field and preserve the rest of the export.
- to_json emits a JSON containing schema_version, candidates, problem
name, and algorithm name strings.
- to_writer and to_file round-trip the same bytes.
- Free top-level to_json / to_writer / to_file convenience functions
exercised end-to-end (round-trip through tmp file).
- pad_decision_schema at all three boundaries (< target / == target /
> target) to pin the < comparison.
- candidate_to_export's in_pareto_front toggles at front_rank == 0.
- candidate_to_export's feasible toggles at constraint_violation <= 0.
- candidate_to_export pads short objective vectors and truncates long
ones (the defensive branch).
Adds eight operators for permutation (Vec<usize>) decisions:
Initializers
- ShuffledPermutation { n } — random shuffles of [0..n)
- ShuffledMultisetPermutation { repeats_per_id } — random shuffles
of an arbitrary multiset (e.g., JSS operation strings)
Crossovers (strict permutations only)
- OrderCrossover (OX)
- PartiallyMappedCrossover (PMX)
- CycleCrossover (CX)
- EdgeRecombinationCrossover (ERX)
Mutations (preserve both strict permutations and multisets)
- InversionMutation
- InsertionMutation
- ScrambleMutation
All are re-exported from the prelude. Comprehensive unit tests + doctests
included; the pre-existing SwapMutation is untouched.
Companion to the feat(explorer) commit. Bumps the version and
brings every cross-referencing doc up to v0.9 currency.
- Cargo.toml: version 0.8.0 -> 0.9.0.
- CHANGELOG: 0.9.0 entry covering the explorer export, the
Problem-side metadata additions, the AlgorithmInfo trait, the
pick_a_car example, and the new cookbook recipe.
- README: closing paragraph of the PickACar example points users
at the explorer with a one-call snippet
(`ExplorerExport::from_result(...).with_algorithm_info(...)
.to_file(...)?`). Version snippets bumped 0.8 -> 0.9.
- New cookbook recipe at docs/book/src/cookbook/explorer.md
covering: enabling the serde feature, enriching Problem with
labels/units/decision-schema, the export call, the JSON schema,
and custom decision-type handling.
- SUMMARY.md and cookbook.md link the new recipe.
- migration.md: new "To 0.9" section documenting the additive
changes (purely backwards-compatible upgrade from 0.8.x).
- introduction.md, comparison.md, choosing-an-algorithm.md,
stability.md: version refs bumped 0.8 -> 0.9.
- cookbook/parallel.md, cookbook/async.md: version refs bumped
0.8 -> 0.9.
- getting-started.md: version refs bumped, serde feature
description expanded to mention the explorer module.
- SECURITY.md: supported-versions table moves to 0.9.x.
Adds a tiny additive surface that turns any OptimizationResult into
a self-describing JSON file the heuropt-explorer webapp can load.
Real Pareto fronts have 50–200+ candidates spanning 2–7+ objectives;
reading them as numbers in a terminal scales badly. This commit
ships the heuropt-side of the explorer — the schema and the export
API. The webapp itself lives in a separate repo on its own cadence.
Three trait/type extensions, all with working defaults so existing
impls compile untouched:
- Objective gains optional `label: Option<String>` and
`unit: Option<String>` fields, plus fluent builders
`.with_label("Price").with_unit(\"\$k\")`. Existing
`Objective::minimize(name)` / `Objective::maximize(name)` are
unchanged. Both fields are #[serde(default,
skip_serializing_if = \"Option::is_none\")] so existing JSON
round-trips cleanly.
- Problem trait gains an optional
`fn decision_schema(&self) -> Vec<DecisionVariable>` with default
empty impl. Override it to provide pretty names / labels / units /
bounds for the explorer; the default produces fallback x[0],
x[1], … names. New DecisionVariable type at
`heuropt::core::DecisionVariable` with builder methods.
- New `heuropt::traits::AlgorithmInfo` trait with `name()`
(required) and `seed()` (default None). Every built-in algorithm
— all 33 — implements it. Separate from Optimizer<P> so
multi-fidelity Hyperband (which uses PartialProblem) implements
it uniformly.
The new explorer module:
- `heuropt::explorer::ExplorerExport` envelope with versioned
schema (SCHEMA_VERSION = 1).
- ExplorerCandidate per row, with front_rank from
non_dominated_sort attached at export time so downstream tools
don't re-derive it.
- ToDecisionValues adapter trait with provided impls for Vec<f64>,
Vec<bool>, Vec<usize>, Vec<i64>; custom decision types implement
one method.
- Free functions to_json / to_writer / to_file plus a builder API
(with_algorithm_info, with_problem_name, with_wall_clock,
with_timestamp).
- Gated on the existing `serde` feature, which now also pulls in
`serde_json` as a dep.
The example:
- `examples/pick_a_car.rs` — promotes the README's PickACar to a
real example, fully enriched with Objective labels/units and a
decision_schema. Runs NSGA-III for 200 generations, prints a
sample slice, writes pick_a_car.json. Gated on `serde`.
10 new explorer unit tests cover round-trip serde, fallback
decision-variable names, enriched export, AlgorithmInfo flow,
front-rank correctness, and the ToDecisionValues impls. Lib test
count went from 229 to 242.
Companion to the feat(async) commit. Brings every cross-referencing
doc up to v0.8 currency, replaces marketing-flavored copy with plain
prose, and replaces toy benchmark problems with relatable ones that
include actual run output and interpretive narrative.
- README: collapses the four-bullet "Read the user guide / API
reference / Tested with N tests / Hot paths optimized" list into
a single Docs links line.
- README: replaces the Schaffer-N1 toy problem with a PickACar
multi-objective design problem — three decision variables
(displacement, weight, drag), four objectives (price, 0-60,
fuel, noise), and *nonlinear* cost relationships so the Pareto
front is a real surface, not a 1D sweep. Includes actual NSGA-III
run output (representative slice across the 100-car front) and
a narrative explaining what each row tells you and why hand-
picking would miss the interesting tradeoffs.
- README: removes rustdoc-style hidden `#` setup lines from code
blocks. The README is rendered as plain markdown on GitHub /
crates.io, where those lines are visible garbage instead of
hidden setup. Code blocks are now self-contained.
- Guide quickstart (getting-started.md): replaces Sphere ( Σ x² )
with a least-squares LineFit example. Same shape (single-
objective continuous), but recognizable framing. Includes
actual CMA-ES output, residual table, and narrative comparing
the answer to standard regression.
- Algorithm count audit: stale "35 algorithms" claim corrected to
the actual 33 across README, src/lib.rs, introduction.md, and
the comparison.md table cell.
- Async feature flag listed in the optional-features sections of
README, src/lib.rs, getting-started.md.
- introduction.md, choosing-an-algorithm.md, comparison.md,
stability.md, migration.md, cookbook/parallel.md,
cookbook/custom-optimizer.md: cross-references updated to
describe full async coverage and link the new cookbook recipe.
- stability.md: removes the speculative "Observer / Snapshot /
Checkpoint planned" bullet (those didn't ship); documents the
AsyncProblem / AsyncPartialProblem trait stability.
- migration.md: new "To 0.8" section with paths from 0.5.x and 0.7.x.
- CHANGELOG: 0.8.0 entry capturing the async feature plus the
documentation / governance / CI catch-up.
- SECURITY.md: supported versions table reflects 0.8.x.
Async coverage was incomplete in 0.7 (only RandomSearch and
DifferentialEvolution had run_async). 0.8 closes the gap: every one
of the 33 algorithms now exposes
run_async(&problem, concurrency).await, gated on the async feature.
- Population-based algorithms fan out per-generation evaluations
through evaluate_batch_async with concurrency-bounded
FuturesOrdered chunks.
- Steady-state algorithms (HillClimber, SimulatedAnnealing,
OnePlusOneEs, Paes, NelderMead) await each step sequentially;
they accept the concurrency parameter for API uniformity.
- TabuSearch fans out the K-neighbor batch each step.
- Surrogate algorithms (BayesianOpt, Tpe) batch the initial design
and await per-iteration acquisitions sequentially so the surrogate
can update between picks.
- Hyperband uses a new AsyncPartialProblem trait (mirroring
PartialProblem for multi-fidelity workloads) and a parallel
evaluate_batch_at_budget_async helper; each Successive-Halving
rung fans out its budgeted evaluations.
All paths preserve seeded determinism: RNG draws happen on the main
task in the same order as the sync path, and only the evaluations
are concurrent.
Adds a dedicated cookbook recipe at docs/book/src/cookbook/async.md
with a worked example (DifferentialEvolution under tokio) and
guidance on picking concurrency. Cross-references in SUMMARY.md
and cookbook.md are updated to surface the new recipe.
The follow-up docs commit reconciles the rest of the user guide
and README to describe the new feature; this commit is the bare
async surface.
Completes the rustdoc audit — every public item now has at least one
```rust example block in its docstring, exercised by
`cargo test --doc` (55 doctests, all passing).
- Operators: BitFlipMutation, SwapMutation, RealBounds,
GaussianMutation, BoundedGaussianMutation,
SimulatedBinaryCrossover, PolynomialMutation, LevyMutation,
ClampToBounds, ProjectToSimplex.
- Metrics: hypervolume_2d, hypervolume_nd, spacing.
- Pareto utilities: pareto_compare, pareto_front, best_candidate,
non_dominated_sort, crowding_distance, das_dennis,
ParetoArchive.
Each example is short (5-15 lines) and self-contained — copy-paste
into a fresh project and it runs.
Adds heuropt-plot, a tiny SVG-only plotter that takes heuropt
results and emits scatter plots (pareto_front_svg) and line plots
(convergence_svg). No heavy 'plotters' or 'tiny-skia' dep — hand-
rolled SVG so the crate adds <100 KB to a build.
Workspace setup: root Cargo.toml gains [workspace] with members =
['.', 'heuropt-plot']. heuropt-plot has its own version (0.1.0) and
publishes independently against heuropt 0.8+.
Adds examples/visualize.rs that wires it up: NSGA-II on Schaffer
N.1, plain run() (no observer plumbing), final-front SVG written to
disk.
Adds the headline async/await capability for IO-bound evaluations
(HTTP services, RPC clients, spawned subprocesses) — the
differentiator vs pymoo / hyperopt / MOEA Framework.
No public-API breaks for synchronous users. The new surface is
gated behind a new `async` feature flag.
- core::async_problem::AsyncProblem trait (async fn evaluate_async).
- algorithms::parallel_eval_async::evaluate_batch_async helper using
futures::stream::FuturesOrdered with concurrency-bounded chunks;
preserves input order so seeded determinism holds when evaluations
are themselves deterministic.
- run_async on RandomSearch and DifferentialEvolution.
- examples/async_eval.rs: simulated 20 ms remote service. concurrency=1
→ 4.2 s, concurrency=4 → 2.1 s (2× speedup).
Bumps Cargo.toml to 0.8.0; CHANGELOG entry covers the above plus a
note that 0.6.0/0.7.0 on crates.io are yanked experimentals and 0.8
picks up cleanly from 0.5.
Theme: documentation and project polish. No public-API changes; this
is the v0.5 release that elevates heuropt's docs/onboarding/governance
to bar-setting status.
Adds:
- mdbook user guide at docs/book/ with intro, getting-started,
defining-problems, choosing-an-algorithm, cookbook (7 recipes),
comparison vs other libraries, stability/SemVer, migration guides.
Deploys to https://swaits.github.io/heuropt/ via .github/workflows/
docs.yml.
- Runnable rustdoc examples on every algorithm (35 of them), all
exercised by cargo test --doc.
- Three real-world examples: portfolio.rs (multi-obj with budget
constraint), hyperparam_tuning.rs (BO + TPE), scheduling.rs
(permutation via SA + SwapMutation against Smith's-rule oracle).
- Governance: CONTRIBUTING.md, SECURITY.md, CODE_OF_CONDUCT.md
(adopting builderscode.org's Builder's Code of Conduct), GitHub
issue templates, PR template.
Polishes:
- README hero with badges + user-guide link.
- lib.rs crate-level docs.
- CHANGELOG entry for 0.5.0.
Bumps Cargo.toml to 0.5.0.
`ParetoArchive::insert` calls `pareto_compare` twice per existing
member (once per pass), and each call re-allocates two Vec<f64>s
via `as_minimization` — 4N allocations per insert. Cache the
candidate's oriented + feasibility/violation once, build each
member's oriented vector once for the call, then inline the
dominance test against those cached arrays.
Used by PESA-II (per offspring per generation), PAES (per child),
ε-MOEA, and any user code working through the archive directly.
Wall-clock (compare harness, 10-seed mean):
- PESA-II / DTLZ2: 498 → 426 ms (-14 %)
- PESA-II / ZDT1: 87 → 75 ms (-14 %)
Smaller wins on PAES / MOPSO / IBEA / HypE / ε-MOEA where the
archive isn't the dominant per-generation cost.
Bit-identical via the compare harness.
The M≥3 branch of `hso_recursive` cloned every input point into
`sorted: Vec<Vec<f64>>` solely so it could sort. Each clone is M
f64s allocated; with N points per call and ~30 HV calls per SMS-EMOA
generation × 30 k generations, that's millions of small Vec<f64>
allocations.
Sort indices into a `Vec<usize>` instead, then iterate the original
points by index. The pre-projection step still produces a
Vec<Vec<f64>> (which the active-prefix slicing requires), but we
save the outer N inner-Vec clones per call.
gungraun (instructions):
- hypervolume_nd_3d n=30: 87 969 → 70 334 (-20 %, 1.25×)
- hypervolume_nd_3d n=100: 422 767 → 367 767 (-13 %, 1.15×)
Cumulative vs the v0.3.0 baseline:
- hypervolume_nd_3d n=30: 676 902 → 70 334 (9.6×)
- hypervolume_nd_3d n=100: 13 523 760 → 367 767 (37×)
Wall-clock impact is in the noise on the compare harness because the
SMS-EMOA worst-front HV calls operate on small fronts (5–10 points
once converged). The win is most visible in synthetic dense-front
HV benchmarks.
Two independent wins in SPEA2's per-generation hot path. Both
bit-identical against the compare harness.
# 1. compute_fitness — cache oriented + distance matrix
`compute_fitness` is called twice per generation. The strength-graph
loop calls `pareto_compare` in an N² loop, allocating two Vec<f64>s
per call via `as_minimization`. Inline the dominance test against
cached oriented arrays. The density loop's per-row euclidean recompute
is replaced by a symmetric N×N distance matrix built once.
# 2. build_archive — incremental sort maintenance in truncation
The archive-truncation loop was O(K³ log K) — each pruning iteration
recomputed every alive member's pairwise distances and re-sorted them,
when the only change since the prior iteration was that one specific
neighbor (the just-removed victim) became dead. Compute the distance
matrix and sorted neighbor vectors once, then on victim removal use
binary-search-remove on every survivor's still-sorted vector. Total
truncation cost drops from O(K³ log K) to O(K² log K). Victim choice
is bit-identical.
gungraun (instructions):
- spea2_short: 179 113 → 133 783 (-25 %, 1.34×)
Wall-clock (compare harness, 10-seed mean):
- SPEA2 / ZDT1: 458 → 241 ms (1.9×, cumulative)
- SPEA2 / DTLZ2: 4304 → 513 ms (8.4×, cumulative)
The splitting-front survival selection in AGE-MOEA recomputed two
expensive things per while-iteration:
* `lp_norm(translated[i], p)` for every remaining i — even though the
value is constant across iterations.
* `nearest_neighbor_distance(i, …, &keep, p)` — a fresh full scan
over the keep list, even though only one new candidate was added
since the last scan.
Both are `powf`-heavy in the L_p frame.
Compute lp_norm once per candidate at function entry. Maintain a
`nearest[]` array seeded from the initial keep set and updated on
every pick by a single `min(nearest[i], lp_distance(i, pick, p))`
per remaining i. That cuts the score loop from O(R · K · M) to
O(R · M) per iteration, with the dominant powf calls in
lp_distance counted once per (remaining, pick) pair instead of per
(remaining, full-keep).
Wall-clock (compare harness, 10-seed mean):
- AGE-MOEA / DTLZ1: 2266 → 430 ms on top of v0.3.0 baseline (5.3×)
- AGE-MOEA / ZDT3: 935 → 376 ms (2.5×)
The Deb fast non-dominated sort calls `pareto_compare` twice for
every (i, j) pair, and each `pareto_compare` call invokes
`ObjectiveSpace::as_minimization` twice — so for an N-point
population that's 4·N·(N-1) fresh `Vec<f64>` allocations per sort.
At N=100 with thousands of generations across the compare harness,
this dominated the per-generation cost of every Pareto-based MOEA.
Cache `as_minimization`/feasibility/violation once per individual
up front, then inline the dominance test against those cached
arrays. The output (per-pair dominance outcome and the per-i
`dominates` lists) is bit-identical to `pareto_compare`.
gungraun (instructions):
- non_dominated_sort_2d n=50: 852 317 → 198 574 (-77 %, 4.3×)
- non_dominated_sort_2d n=200: 13 513 271 → 2 601 813 (-81 %, 5.2×)
Wall-clock (compare harness, 10-seed mean):
- NSGA-II / ZDT1: 268 → 65 ms (4.1×)
- NSGA-II / ZDT3: 267 → 65 ms (4.1×)
- NSGA-II / DTLZ2: 344 → 106 ms (3.2×)
- NSGA-II / Rastrigin: 260 → 71 ms (3.7×)
- NSGA-III / DTLZ2: 318 → 122 ms (2.6×)
- NSGA-III / DTLZ1: 303 → 122 ms (2.5×)
- SMS-EMOA / DTLZ2: 1413 → 1369 ms (small additional win on top of HV)
- AGE-MOEA / DTLZ1: 430 → 229 ms (1.9×, on top of the AGE-MOEA caching)
- HypE / DTLZ2: 80 → 44 ms (1.8×)
The HSO recursion in `hypervolume_nd` had three overheads that
dominated SMS-EMOA's per-generation cost on DTLZ2 (5.6 s baseline,
~30 k generations × ~40 HV calls per generation = ~1.2 M HV calls
per run):
1. `active = sorted.clone()` plus `active.iter().position(...)`
linear scan to remove the just-processed point each band — O(N)
per band, total O(N²) per HV call.
2. Per-band re-projection
`active.iter().map(|q| q[..last].to_vec())` — full
Vec<Vec<f64>> rebuild for every band, O(N·M) allocations per HV
call.
3. `non_dominated_projection` called even when recursing into the
M=2 base case, whose sweep already filters dominated points
internally.
Replace (1) with prefix-slicing `projected_all[..=k]` (sort points
ascending by last axis once; the active set at each band is just a
prefix). Pre-project once outside the loop (2). Skip the explicit
non-dominance filter when the inner recursion is M=2 (3).
Bit-identical output verified by re-running the compare harness and
diffing against the v0.3.0 snapshot — every quality metric matches
to the last decimal.
gungraun (instructions):
- hypervolume_nd_3d n=30: 676 902 → 87 969 (-87 %, 7.7×)
- hypervolume_nd_3d n=100: 13 523 760 → 422 767 (-97 %, 32×)
Wall-clock (compare harness, 10-seed mean):
- SMS-EMOA / DTLZ2: 5643 ms → 1413 ms (-4230 ms, -75 %)
Multi-fidelity optimization. Hyperband (Li et al. 2017) and its
foundation Successive Halving (Karnin et al. 2013) tune
hyperparameters by allocating *uneven* compute across configurations:
sample many cheap-to-evaluate-at-low-budget configs, then promote
the survivors to higher budgets. Crucial for ML hyperparameter
tuning where each evaluation is a partial training run.
This requires a new trait — `Problem::evaluate` is a single-shot
black box, but Hyperband needs to evaluate the SAME decision at
different fidelity budgets:
pub trait PartialProblem {
type Decision: Clone;
fn objectives(&self) -> ObjectiveSpace;
fn evaluate_at_budget(&self, decision: &Self::Decision,
budget: f64) -> Evaluation;
}
`PartialProblem` is intentionally NOT a sub-trait of `Problem`.
Implementors who already have a `Problem` and want their
`evaluate_at_budget` to ignore budget can write a one-line wrapper.
`Hyperband` is the optimizer:
pub struct HyperbandConfig {
max_budget: f64, eta: f64, max_brackets: usize, seed: u64,
}
pub struct Hyperband<I> { config, initializer, ... }
Single-objective only. The decision sampler is an `Initializer<D>` so
it works the same way as every other heuropt algorithm. Generic over
decision type.
Spec §22 Round 4-D listed bounded mutation / repair operators as future
work; this is the second piece of that. A `Repair<D>` trait that nudges
infeasible decisions back to feasibility, intended to be called from a
user's Variation operator (or a CompositeVariation pipeline) when
projection-style constraint handling is preferred over the
penalty-style `constraint_violation` approach.
Trait:
pub trait Repair<D> {
fn repair(&mut self, decision: &mut D);
}
Provided impls:
- `ClampToBounds` — clamps each variable of a Vec<f64> to per-axis bounds
- `ProjectToSimplex` — projects a Vec<f64> onto the (clipped) probability
simplex (Σ x_i = total, x_i ≥ 0), useful for portfolio-style problems
and reference-direction normalization
Both stay in the existing `operators` module (alongside Variation
operators) since they share the same "transforms decisions" theme. Re-
exported from the prelude.
Runarsson & Yao 2000 stochastic ranking: a probabilistic alternative
to feasibility-first tournament selection. Each pairwise comparison
during a bubble-sort pass uses the *objective* value with probability
`pf` even when one or both candidates are infeasible. The classic
recommendation `pf = 0.45` reliably outperforms strict
feasibility-first on heavily-constrained problems where occasionally
exploring the infeasible region helps cross narrow feasible corridors.
New helper: `stochastic_ranking_select` lives next to
`tournament_select_single_objective` in `selection::tournament`.
Single-objective only; same signature pattern (population, objectives,
count, rng, plus the new `pf` knob).
Bergstra et al. 2011: sample-efficient sequential optimizer that's the
workhorse of Hyperopt and Optuna. Different surrogate from BO's
Gaussian process — TPE models p(x | y < y*) with one KDE and
p(x | y >= y*) with another, then samples candidates from the 'good'
KDE and ranks by the ratio l(x) / g(x). The acquisition is implicit
in the ratio (a closed-form analog of Expected Improvement).
Implementation:
- 1-D Gaussian KDE per axis, with bandwidth chosen by Scott's rule
- Per-step:
- Evaluate observations into 'good' (top γ fraction by target) and
'bad'
- Sample n_candidates from the good distribution (independent per
axis) and pick the one with the largest l(x)/g(x)
- Evaluate it, append to history
Vec<f64> only, single-objective only. Compared with BayesianOpt:
- Cheaper per-step (no GP factorization)
- Doesn't need kernel hyperparameter tuning to work well
- Naturally extends to mixed/categorical decision types (future work)
- Generally less sample-efficient than well-tuned BO on smooth
continuous problems, but more robust out of the box
Tests cover convergence on 1-D Sphere within a tight budget,
deterministic reruns, panic on multi-objective.
Wierstra et al. 2008/2014 NES with the diagonal-covariance "separable"
variant (sNES). Different theoretical foundation from CMA-ES: rather
than tracking a full covariance matrix and adapting it through
evolution paths, sNES updates the sampling distribution's parameters
by following the natural gradient of expected fitness.
Each generation:
- Sample λ offspring from N(μ, diag(σ²))
- Rank-shape the fitnesses (utility weights from the standard NES table)
- Update μ along the natural gradient: μ ← μ + η_μ · σ · sum(u_i · z_i)
- Update σ multiplicatively: σ_j ← σ_j · exp(η_σ/2 · sum(u_i · (z_i,j² - 1)))
Vec<f64> decisions only, single-objective only. The diagonal covariance
makes per-step cost O(λ·n) instead of CMA-ES's O(λ·n²) — much faster on
high-dimensional problems where full-covariance tracking is expensive
or numerically fragile, at the cost of being unable to handle strongly
rotated landscapes.
The first sample-efficient algorithm in heuropt. Bayesian optimization
maintains a Gaussian-process surrogate of the objective and at each
step picks the next decision by maximizing an acquisition function on
that surrogate, so the evaluation budget is used surgically.
Implementation:
- **Kernel**: anisotropic RBF (squared-exponential) with per-axis
length scales, signal variance, and a small noise/jitter floor.
Hyperparameters are exposed in the config; a future version can add
marginal-likelihood maximization.
- **Posterior**: standard formulation. Cholesky factorizes K (using
the new internal helper); mean and variance predictions follow.
- **Acquisition**: Expected Improvement against the best observed
feasible point. Optimized by best-of-N random sampling — simple,
predictable cost, no inner-optimizer footgun.
- **Initial design**: `initial_samples` uniform-random points in
bounds before the BO loop starts.
- **Constraints**: feasibility-aware EI — best observed value uses
only feasible points; infeasible candidates are penalized.
Vec<f64> decisions, single-objective only. Targets the regime no
existing heuropt algorithm covers: 50–500 evaluations on an
expensive black-box function (CFD sim, ML training run, real-world
measurement).
Tests cover convergence on the 1-D sphere within a tight evaluation
budget (~30 evals get to f < 1e-6 — vs population-based methods
needing thousands), deterministic reruns, and panic on
multi-objective + dim mismatches.
Hand-rolled `A = L · L^T` factorization plus forward/backward triangular
solves, used by the upcoming Bayesian Optimization implementation for
the GP posterior. Same f64 row-major Vec<Vec<f64>> interface as the
existing Jacobi eigen helper so we don't pull in nalgebra for one
algorithm.
Returns Err on non-positive-definite input (a small jitter is the
typical caller-side fix). Tested against the standard 2x2 case, the
3x3 known-result case, A·x = b round-trip, and the SPD-failure case.
Auger & Hansen 2005 IPOP-CMA-ES: wraps the existing CmaEs in a restart
loop that doubles the population size and re-randomizes the mean
whenever a restart trigger fires. Specifically addresses the failure
mode we observed on Rastrigin (vanilla CMA-ES = 2.3 vs DE = 0).
Restart triggers:
- The whole budget for one inner CmaEs run finishes without improvement
- (More sophisticated triggers — eigenvalue collapse, condition-number
blow-up, sigma stagnation — are left for future versions; the
per-run budget trigger captures the bulk of the practical benefit)
Each restart:
- Doubles the population_size (Auger & Hansen 2005)
- Re-randomizes the initial mean to a fresh point in the bounds box
- Resets sigma to the user's initial value
Same Vec<f64> + single-objective constraints as CmaEs. The total
budget is divided across restarts; restart budget grows with
population. Tests verify it beats vanilla CMA-ES on Rastrigin.
Nelder & Mead 1965: gradient-free local optimizer that maintains a
simplex of n+1 points in n-D and at each iteration replaces the worst
vertex by one of {reflect, expand, outside-contract, inside-contract,
shrink} relative to the centroid of the rest. The five standard
coefficients (reflection α=1, expansion γ=2, contraction ρ=0.5,
shrinkage σ=0.5) are exposed in the config but default to canonical
values so users can leave them alone.
Single-objective only, Vec<f64> only, bounds enforced by clamping
each new vertex. Termination is purely iteration-count for v0.2;
"vertices have collapsed" stopping is a future enhancement.
Filling a real gap: heuropt had population-based local search
(SimulatedAnnealing, HillClimber) but no classical direct-search
algorithm. Excellent for low-dim smooth-ish problems where a
population is overkill.
Rechenberg 1973's elemental evolution strategy: one parent, one child
each generation, accept the child if it is no worse, and adapt the
mutation step size by tracking the success rate. If more than 1/5 of
recent moves were accepted the search is too cautious — multiply σ by
`step_increase` (typical 1.22). Below 1/5 — divide by the same factor.
At 1/5 — leave it alone. The success window has length `adaptation_period`.
Single-objective only. Vec<f64> only. Generic Gaussian step bounded by
the embedded `RealBounds`.
Why ship it: it's the smallest possible self-adapting evolution strategy
and a useful pedagogical / baseline endpoint. Pairs well as the budget
floor ("give me anything cheaper than CMA-ES").
Zhang, Tian & Jin 2015 KnEA: many-objective MOEA that biases survival
selection toward 'knee points' on the Pareto front — points where a
small improvement in one objective costs a large degradation in
another.
Each generation:
- NSGA-II-like loop with offspring + non_dominated_sort
- For the splitting front, identify knee points by perpendicular
distance from the hyperplane connecting the front's extreme points.
Members further from the hyperplane (= more 'kneeness') are preferred.
- Survival keeps every knee-tagged member; if room remains, fill from
remaining members by largest perpendicular distance.
Knee points are intuitively the most attractive points on a Pareto
front when no preference information is available. KnEA pushes the
search toward them at the cost of less uniform front coverage.
Yang, Li, Liu & Zheng 2013 GrEA: many-objective MOEA whose secondary
ranking is a grid-based diversity score instead of crowding distance
or reference vectors.
Each generation:
- NSGA-II-like loop with offspring + non_dominated_sort
- For the splitting front:
- Translate by ideal/nadir; partition objective space into a
(`grid_divisions` per axis) grid
- For every member compute three grid scores:
- GR (grid rank) = sum of grid coordinates (closer to ideal = lower)
- GCD (grid crowding distance) = #neighbors within 1 grid unit (in any axis)
- GCPD (grid coordinate point distance) = max coord - min coord
- Sort F_l ascending by GR, then by GCD, then by GCPD
- Take the top `n - already_selected` survivors
GrEA's grid-based niching is a different lens from NSGA-III's reference
points and RVEA's reference vectors — particularly effective on
non-convex fronts where reference-vector approaches struggle.
Panichella 2019 AGE-MOEA: a many-objective MOEA that *infers* the
front's geometry (its L_p shape, where p = 1 is linear, p = 2 is
spherical, p < 1 is convex etc.) from the current non-dominated set
and uses that estimate to drive both proximity and diversity in
survival selection.
Each generation:
- NSGA-II-like loop: random parent selection + variation + evaluation
- Combine + non_dominated_sort
- Fill front-by-front; for the splitting front:
- Translate by ideal point z*
- Find extreme points by ASF (same as NSGA-III) and intercepts
- Estimate the geometry parameter p by minimizing
\|f − ideal\|_p constancy on the extreme points
- Score every member by survival_score = (proximity_to_ideal) +
(1 / nearest-neighbor distance in the same L_p frame)
- Keep the top scorers
The geometry estimation is the novel contribution; with 3+ objectives
it produces fronts whose spread better matches the true shape than
NSGA-III's reference points (which assume a known geometry).
Rao 2011 TLBO: parameter-free single-objective optimizer for Vec<f64>.
The selling point — uniquely among the metaheuristics we ship — is that
it has NO algorithm-specific hyperparameters: no F, CR, w, c1, c2, σ,
mutation rate, etc. Just population_size and generations.
Each generation has two phases:
- **Teacher phase**: identify the best individual (the 'teacher'). For
every learner, compute a 'mean' learner and try replacing it with a
candidate moved toward the teacher by a random fraction, scaled by
the gap between teacher and (TF · mean), where TF ∈ {1, 2}.
- **Learner phase**: each learner picks a random partner and tries
moving toward the better one of the pair. Only successful moves are
kept.
Single-objective only, Vec<f64> only, bounds enforced via clamping.
Tests cover Sphere1D convergence, deterministic reruns, and panic on
multi-objective.
Lévy-flight perturbation: each variable receives a step drawn from a
heavy-tailed Lévy(α) distribution rather than a Normal. The result is
"mostly small steps with rare big jumps," which gives a more
exploratory mutation than Gaussian without abandoning local search.
Decision type: Vec<f64>, with optional bounds (clamped per-axis if
`bounds` is non-empty). The step is sampled via Mantegna's algorithm
which generates Lévy(α) by combining two Normal samples and taking
the right power, controlled by the tail exponent `alpha` (typical
1.5; 1 is heavy, 2 collapses to Normal).
This is the only genuinely-different mutation kernel from Cuckoo
Search and other Lévy-flight metaheuristics; ship it as a Variation
operator usable from any algorithm rather than as a separate
algorithm.
Replaces strict Pareto dominance with ε-dominance: A ε-dominates B when
`floor(A_i / ε) ≤ floor(B_i / ε)` for every objective and strictly
less in at least one (minimization frame). The result is a regular
discretization of objective space — at most one archive member per
ε-box — so the front spreads out automatically and the archive size
self-limits without truncation tricks.
Steady-state design: each generation samples one parent from the main
population and one from the ε-archive, applies variation, evaluates
the child, and offers it to both archives. Every member's
ε-coordinates and the box-tie rules are precomputed each insertion.
Tests: produces a front on Schaffer N.1 with reasonable spread,
deterministic reruns, panic on `epsilon[i] <= 0.0` and on
`epsilon.len() != objectives.len()`.
Corne, Jerram, Knowles & Oates 2001: divides objective space into a
hyperbox grid and uses per-box population counts to drive selection
toward sparsely-populated regions.
Each generation:
- Maintain an external archive of non-dominated members
- Build a hyperbox grid (`grid_divisions` per axis on the archive's
current axis ranges); count members per box
- Selection picks two parents by region-based tournament: choose two
random non-empty boxes and take a uniform-random member from the
one with fewer occupants
- Variation produces an offspring; insert into archive, dropping
dominated members and (if archive overflows) the most-crowded
occupant of the most-occupied box
Tests cover non-empty front on Schaffer N.1, deterministic reruns,
and panic on `archive_size == 0`.
Cheng, Jin, Olhofer & Sendhoff 2016 RVEA: many-objective MOEA built
around a fixed set of Das–Dennis reference vectors. Each generation:
- Generate offspring via random parent selection + variation +
evaluation
- Combine population + offspring; translate by ideal point z*
- Associate every member with the reference vector whose angle to
the translated objective vector is smallest
- For each occupied vector, keep the member with the smallest
Angle-Penalized Distance (APD) score; the rest are dropped
- APD = (1 + α(t)·θ_max·γ) · |f − z*| where γ is the angle to the
associated reference and α(t) = (t / t_max)^2 anneals the angle
penalty over the run
This produces well-spread fronts at high objective counts where
Pareto-rank methods (NSGA-II, SPEA2) lose discrimination.
Bader & Zitzler 2011: HypE estimates hypervolume contributions via
Monte Carlo sampling instead of computing them exactly. The point of
the trick is that exact hypervolume becomes prohibitively expensive
beyond ~5 objectives, while MC sampling stays cheap and accurate
enough at any dimension.
Each generation:
- Generate offspring via parent selection + variation + evaluation
- Combine, run non_dominated_sort, fill front-by-front
- For the splitting front, estimate each member's HV contribution
by drawing `n_samples` uniform points in the box [ideal, reference]
and counting how many points are dominated by *exactly* one front
member — that count, divided by n_samples and multiplied by the
box volume, is the member's expected unique HV contribution.
- Drop members one at a time from the splitting front by smallest
estimated contribution.
Public API matches the rest of the MO algorithms (Config + Optimizer).
The reference point is supplied in the config so the user controls
the integration domain. Tests cover non-empty front, deterministic
reruns, and panic on dim-mismatched reference.
Beume, Naujoks & Emmerich 2007: a steady-state MOEA that uses
hypervolume contribution as the secondary survival selection criterion.
Each generation:
- Generate ONE child via parent selection + variation + evaluation.
- Combine population + child, run non_dominated_sort.
- The discarded individual is the worst-front member with the
smallest hypervolume contribution (computed via the new
hypervolume_nd_from_evaluations helper).
Selection-quality is excellent at moderate objective counts (2–4) at
the cost of higher per-step compute (each survival selection requires
N+1 hypervolume evaluations of size ≤ N each). Best paired with a
tightly-bounded objective space — the user supplies a fixed reference
point in the config.
Tests: produces a non-empty front on Schaffer N.1, deterministic
reruns, panic on `population_size == 0`, panic on
`reference_point.len() != objectives.len()`.
Generalizes the existing 2-D hypervolume to arbitrary M ≥ 1 dimensions
using the standard recursive Hypervolume-by-Slicing-Objectives (HSO)
algorithm from While et al. 2006:
- For M = 1: return reference[0] - min(points[0])
- For M = 2: sort by axis 0, sweep accumulating rectangles (matches
hypervolume_2d's existing exact behavior)
- For M ≥ 3: sort by the last axis, peel off slices of increasing
thickness and recursively compute the (M−1)-dimensional HV of each
slice's projected non-dominated subset
Direction-aware: minimization-oriented input is the entry point, so
maximize objectives are negated by the caller via
`ObjectiveSpace::as_minimization` before the recursion runs.
Tested against:
- the existing 2-D analytical case (3 points → area 6)
- a known 3-D unit-cube case (1 point at origin, ref [1,1,1] → 1)
- empty front → 0
- agreement with hypervolume_2d on random 2-D fronts
Mühlenbein 1997 UMDA: simplest Estimation-of-Distribution Algorithm for
`Vec<bool>` problems. Each generation:
- Evaluate the current population
- Select the top μ members by fitness
- Estimate per-bit marginal probability p_i = (count of 1s at bit i in
the μ-best) / μ
- Sample population_size new individuals from the resulting product-of-
Bernoullis distribution
Single-objective only. Bit-wise probabilities are clamped to
`[1 / (2 · μ), 1 - 1 / (2 · μ)]` to keep the population from collapsing
to a deterministic single string before convergence is meaningful
(standard Laplace-style smoothing for UMDA).
Tests: solves OneMax (maximize Σ bits) on a 20-bit instance,
deterministic reruns, panic on multi-objective.
Dorigo-style Ant System for permutation problems on a complete graph:
each generation, every ant constructs a tour by probabilistically
picking the next node from those it has not yet visited, weighted by
`τ_ij^α · η_ij^β` where τ is the pheromone level on edge (i, j) and
η is the heuristic desirability (1 / distance, here). After all ants
finish, pheromone evaporates by a factor `(1 - ρ)` and is reinforced
on each ant's tour proportional to that tour's quality.
Decision type is `Vec<usize>` (a permutation of 0..n_cities). The user
supplies a distance matrix and the n_cities is inferred. Single-objective
only (the cost is total tour length, which the Problem evaluates).
Tests build a 5-city ring and verify ACO finds a near-optimal tour,
plus deterministic reruns and panic on multi-objective.
swaits