Phase 1 tests for src/algorithms/cma_es.rs.
- compare_so: feasibility-first ordering, minimize/maximize inversion,
infeasibility-violation comparison.
- better_than_so: matches compare_so == Less; equal evaluations are
not strictly better.
- A 30-generation Sphere1D run pins that CMA-ES actually decreases
the objective (catches mutants that collapse the update rules).
Phase 1 tests for src/algorithms/bayesian_opt.rs. Adds 15 tests
pinning the GP regression and EI acquisition machinery:
- rbf_kernel: signal-variance return at zero distance, exp(-0.5) at
unit distance, monotone in length scale, decays to 0 for far points.
- normal_pdf: symmetric about zero, value at zero equals 1/sqrt(2π).
- normal_cdf: 0.5 at z=0, symmetric tail sums to 1.
- erf: odd function and erf(0) ≈ 0 within the rational approximation's
~1e-7 accuracy.
- expected_improvement: zero at sigma=0, monotone in sigma, positive
when mu < f_best.
- oriented_target: sign flips under direction, infeasible adds 1e6
penalty.
- better: feasibility-first then objective ordering under both
directions.
Phase 1 for src/algorithms/ant_colony_tsp.rs. Targets the ~30 remaining
mutants after Phase 0 sweeps — mostly arithmetic flips in build_tour
(pheromone × heuristic weighting) and the feasibility-comparison logic
in better_than_so.
Added:
- Four branch tests for better_than_so covering the full feasibility
cross-product: feasible-vs-infeasible (both orders), two-infeasible
(smaller violation wins), and two-feasible under both directions.
Plus an equal-objectives test pinning the strict-less-than semantics.
- build_tour-is-a-permutation invariant across 20 seeds × 6 start cities.
- A strong-heuristic test: with eta favoring the next-city by 1000x and
beta=5, build_tour walks the preferred path. Pins the .powf(beta)
arithmetic.
- Zero-alpha/zero-beta degenerate-case test: uniform random fallback
still returns a permutation.
Phase 1 for src/algorithms/age_moea.rs. Targets the ~50 algorithm-internal
mutants surviving after the Phase 0 sweeps:
Pure helper-fn pins (lp_norm / lp_distance / nearest_neighbor_distance /
estimate_p):
- L_p norm at p ∈ {1, 2} on canonical vectors (unit, all-ones,
Pythagorean 3-4-5, signed-via-abs).
- L_p distance: zero-to-itself = 0, symmetry, L_1/L_2 sanity values.
- nearest_neighbor_distance: empty selected → ∞, self-only → ∞, picks
the closest of mixed-distance candidates.
- estimate_p: empty-front fallback to p=2; axis-aligned extremes (CV=0
for all p, returns first candidate 0.25); corner-vs-diagonal extremes
(CV minimized at large p).
Full-run pins:
- A 10-generation Schaffer-N1 run with seed 7 verifies the pareto front
is non-empty and finite/nonneg (catches body collapse mutants).
- Population size after run matches config across three pop sizes
(catches size mutants).
- Evaluation count falls in [pop, pop*(gens+1)] (catches comparison
flips in the offspring-collection loop).
Companion to the feat(explorer) commit. Bumps the version and
brings every cross-referencing doc up to v0.9 currency.
- Cargo.toml: version 0.8.0 -> 0.9.0.
- CHANGELOG: 0.9.0 entry covering the explorer export, the
Problem-side metadata additions, the AlgorithmInfo trait, the
pick_a_car example, and the new cookbook recipe.
- README: closing paragraph of the PickACar example points users
at the explorer with a one-call snippet
(`ExplorerExport::from_result(...).with_algorithm_info(...)
.to_file(...)?`). Version snippets bumped 0.8 -> 0.9.
- New cookbook recipe at docs/book/src/cookbook/explorer.md
covering: enabling the serde feature, enriching Problem with
labels/units/decision-schema, the export call, the JSON schema,
and custom decision-type handling.
- SUMMARY.md and cookbook.md link the new recipe.
- migration.md: new "To 0.9" section documenting the additive
changes (purely backwards-compatible upgrade from 0.8.x).
- introduction.md, comparison.md, choosing-an-algorithm.md,
stability.md: version refs bumped 0.8 -> 0.9.
- cookbook/parallel.md, cookbook/async.md: version refs bumped
0.8 -> 0.9.
- getting-started.md: version refs bumped, serde feature
description expanded to mention the explorer module.
- SECURITY.md: supported-versions table moves to 0.9.x.
Adds a tiny additive surface that turns any OptimizationResult into
a self-describing JSON file the heuropt-explorer webapp can load.
Real Pareto fronts have 50–200+ candidates spanning 2–7+ objectives;
reading them as numbers in a terminal scales badly. This commit
ships the heuropt-side of the explorer — the schema and the export
API. The webapp itself lives in a separate repo on its own cadence.
Three trait/type extensions, all with working defaults so existing
impls compile untouched:
- Objective gains optional `label: Option<String>` and
`unit: Option<String>` fields, plus fluent builders
`.with_label("Price").with_unit(\"\$k\")`. Existing
`Objective::minimize(name)` / `Objective::maximize(name)` are
unchanged. Both fields are #[serde(default,
skip_serializing_if = \"Option::is_none\")] so existing JSON
round-trips cleanly.
- Problem trait gains an optional
`fn decision_schema(&self) -> Vec<DecisionVariable>` with default
empty impl. Override it to provide pretty names / labels / units /
bounds for the explorer; the default produces fallback x[0],
x[1], … names. New DecisionVariable type at
`heuropt::core::DecisionVariable` with builder methods.
- New `heuropt::traits::AlgorithmInfo` trait with `name()`
(required) and `seed()` (default None). Every built-in algorithm
— all 33 — implements it. Separate from Optimizer<P> so
multi-fidelity Hyperband (which uses PartialProblem) implements
it uniformly.
The new explorer module:
- `heuropt::explorer::ExplorerExport` envelope with versioned
schema (SCHEMA_VERSION = 1).
- ExplorerCandidate per row, with front_rank from
non_dominated_sort attached at export time so downstream tools
don't re-derive it.
- ToDecisionValues adapter trait with provided impls for Vec<f64>,
Vec<bool>, Vec<usize>, Vec<i64>; custom decision types implement
one method.
- Free functions to_json / to_writer / to_file plus a builder API
(with_algorithm_info, with_problem_name, with_wall_clock,
with_timestamp).
- Gated on the existing `serde` feature, which now also pulls in
`serde_json` as a dep.
The example:
- `examples/pick_a_car.rs` — promotes the README's PickACar to a
real example, fully enriched with Objective labels/units and a
decision_schema. Runs NSGA-III for 200 generations, prints a
sample slice, writes pick_a_car.json. Gated on `serde`.
10 new explorer unit tests cover round-trip serde, fallback
decision-variable names, enriched export, AlgorithmInfo flow,
front-rank correctness, and the ToDecisionValues impls. Lib test
count went from 229 to 242.
Async coverage was incomplete in 0.7 (only RandomSearch and
DifferentialEvolution had run_async). 0.8 closes the gap: every one
of the 33 algorithms now exposes
run_async(&problem, concurrency).await, gated on the async feature.
- Population-based algorithms fan out per-generation evaluations
through evaluate_batch_async with concurrency-bounded
FuturesOrdered chunks.
- Steady-state algorithms (HillClimber, SimulatedAnnealing,
OnePlusOneEs, Paes, NelderMead) await each step sequentially;
they accept the concurrency parameter for API uniformity.
- TabuSearch fans out the K-neighbor batch each step.
- Surrogate algorithms (BayesianOpt, Tpe) batch the initial design
and await per-iteration acquisitions sequentially so the surrogate
can update between picks.
- Hyperband uses a new AsyncPartialProblem trait (mirroring
PartialProblem for multi-fidelity workloads) and a parallel
evaluate_batch_at_budget_async helper; each Successive-Halving
rung fans out its budgeted evaluations.
All paths preserve seeded determinism: RNG draws happen on the main
task in the same order as the sync path, and only the evaluations
are concurrent.
Adds a dedicated cookbook recipe at docs/book/src/cookbook/async.md
with a worked example (DifferentialEvolution under tokio) and
guidance on picking concurrency. Cross-references in SUMMARY.md
and cookbook.md are updated to surface the new recipe.
The follow-up docs commit reconciles the rest of the user guide
and README to describe the new feature; this commit is the bare
async surface.
Adds the headline async/await capability for IO-bound evaluations
(HTTP services, RPC clients, spawned subprocesses) — the
differentiator vs pymoo / hyperopt / MOEA Framework.
No public-API breaks for synchronous users. The new surface is
gated behind a new `async` feature flag.
- core::async_problem::AsyncProblem trait (async fn evaluate_async).
- algorithms::parallel_eval_async::evaluate_batch_async helper using
futures::stream::FuturesOrdered with concurrency-bounded chunks;
preserves input order so seeded determinism holds when evaluations
are themselves deterministic.
- run_async on RandomSearch and DifferentialEvolution.
- examples/async_eval.rs: simulated 20 ms remote service. concurrency=1
→ 4.2 s, concurrency=4 → 2.1 s (2× speedup).
Bumps Cargo.toml to 0.8.0; CHANGELOG entry covers the above plus a
note that 0.6.0/0.7.0 on crates.io are yanked experimentals and 0.8
picks up cleanly from 0.5.
Theme: documentation and project polish. No public-API changes; this
is the v0.5 release that elevates heuropt's docs/onboarding/governance
to bar-setting status.
Adds:
- mdbook user guide at docs/book/ with intro, getting-started,
defining-problems, choosing-an-algorithm, cookbook (7 recipes),
comparison vs other libraries, stability/SemVer, migration guides.
Deploys to https://swaits.github.io/heuropt/ via .github/workflows/
docs.yml.
- Runnable rustdoc examples on every algorithm (35 of them), all
exercised by cargo test --doc.
- Three real-world examples: portfolio.rs (multi-obj with budget
constraint), hyperparam_tuning.rs (BO + TPE), scheduling.rs
(permutation via SA + SwapMutation against Smith's-rule oracle).
- Governance: CONTRIBUTING.md, SECURITY.md, CODE_OF_CONDUCT.md
(adopting builderscode.org's Builder's Code of Conduct), GitHub
issue templates, PR template.
Polishes:
- README hero with badges + user-guide link.
- lib.rs crate-level docs.
- CHANGELOG entry for 0.5.0.
Bumps Cargo.toml to 0.5.0.
Two independent wins in SPEA2's per-generation hot path. Both
bit-identical against the compare harness.
# 1. compute_fitness — cache oriented + distance matrix
`compute_fitness` is called twice per generation. The strength-graph
loop calls `pareto_compare` in an N² loop, allocating two Vec<f64>s
per call via `as_minimization`. Inline the dominance test against
cached oriented arrays. The density loop's per-row euclidean recompute
is replaced by a symmetric N×N distance matrix built once.
# 2. build_archive — incremental sort maintenance in truncation
The archive-truncation loop was O(K³ log K) — each pruning iteration
recomputed every alive member's pairwise distances and re-sorted them,
when the only change since the prior iteration was that one specific
neighbor (the just-removed victim) became dead. Compute the distance
matrix and sorted neighbor vectors once, then on victim removal use
binary-search-remove on every survivor's still-sorted vector. Total
truncation cost drops from O(K³ log K) to O(K² log K). Victim choice
is bit-identical.
gungraun (instructions):
- spea2_short: 179 113 → 133 783 (-25 %, 1.34×)
Wall-clock (compare harness, 10-seed mean):
- SPEA2 / ZDT1: 458 → 241 ms (1.9×, cumulative)
- SPEA2 / DTLZ2: 4304 → 513 ms (8.4×, cumulative)
The splitting-front survival selection in AGE-MOEA recomputed two
expensive things per while-iteration:
* `lp_norm(translated[i], p)` for every remaining i — even though the
value is constant across iterations.
* `nearest_neighbor_distance(i, …, &keep, p)` — a fresh full scan
over the keep list, even though only one new candidate was added
since the last scan.
Both are `powf`-heavy in the L_p frame.
Compute lp_norm once per candidate at function entry. Maintain a
`nearest[]` array seeded from the initial keep set and updated on
every pick by a single `min(nearest[i], lp_distance(i, pick, p))`
per remaining i. That cuts the score loop from O(R · K · M) to
O(R · M) per iteration, with the dominant powf calls in
lp_distance counted once per (remaining, pick) pair instead of per
(remaining, full-keep).
Wall-clock (compare harness, 10-seed mean):
- AGE-MOEA / DTLZ1: 2266 → 430 ms on top of v0.3.0 baseline (5.3×)
- AGE-MOEA / ZDT3: 935 → 376 ms (2.5×)
Multi-fidelity optimization. Hyperband (Li et al. 2017) and its
foundation Successive Halving (Karnin et al. 2013) tune
hyperparameters by allocating *uneven* compute across configurations:
sample many cheap-to-evaluate-at-low-budget configs, then promote
the survivors to higher budgets. Crucial for ML hyperparameter
tuning where each evaluation is a partial training run.
This requires a new trait — `Problem::evaluate` is a single-shot
black box, but Hyperband needs to evaluate the SAME decision at
different fidelity budgets:
pub trait PartialProblem {
type Decision: Clone;
fn objectives(&self) -> ObjectiveSpace;
fn evaluate_at_budget(&self, decision: &Self::Decision,
budget: f64) -> Evaluation;
}
`PartialProblem` is intentionally NOT a sub-trait of `Problem`.
Implementors who already have a `Problem` and want their
`evaluate_at_budget` to ignore budget can write a one-line wrapper.
`Hyperband` is the optimizer:
pub struct HyperbandConfig {
max_budget: f64, eta: f64, max_brackets: usize, seed: u64,
}
pub struct Hyperband<I> { config, initializer, ... }
Single-objective only. The decision sampler is an `Initializer<D>` so
it works the same way as every other heuropt algorithm. Generic over
decision type.
Bergstra et al. 2011: sample-efficient sequential optimizer that's the
workhorse of Hyperopt and Optuna. Different surrogate from BO's
Gaussian process — TPE models p(x | y < y*) with one KDE and
p(x | y >= y*) with another, then samples candidates from the 'good'
KDE and ranks by the ratio l(x) / g(x). The acquisition is implicit
in the ratio (a closed-form analog of Expected Improvement).
Implementation:
- 1-D Gaussian KDE per axis, with bandwidth chosen by Scott's rule
- Per-step:
- Evaluate observations into 'good' (top γ fraction by target) and
'bad'
- Sample n_candidates from the good distribution (independent per
axis) and pick the one with the largest l(x)/g(x)
- Evaluate it, append to history
Vec<f64> only, single-objective only. Compared with BayesianOpt:
- Cheaper per-step (no GP factorization)
- Doesn't need kernel hyperparameter tuning to work well
- Naturally extends to mixed/categorical decision types (future work)
- Generally less sample-efficient than well-tuned BO on smooth
continuous problems, but more robust out of the box
Tests cover convergence on 1-D Sphere within a tight budget,
deterministic reruns, panic on multi-objective.
Wierstra et al. 2008/2014 NES with the diagonal-covariance "separable"
variant (sNES). Different theoretical foundation from CMA-ES: rather
than tracking a full covariance matrix and adapting it through
evolution paths, sNES updates the sampling distribution's parameters
by following the natural gradient of expected fitness.
Each generation:
- Sample λ offspring from N(μ, diag(σ²))
- Rank-shape the fitnesses (utility weights from the standard NES table)
- Update μ along the natural gradient: μ ← μ + η_μ · σ · sum(u_i · z_i)
- Update σ multiplicatively: σ_j ← σ_j · exp(η_σ/2 · sum(u_i · (z_i,j² - 1)))
Vec<f64> decisions only, single-objective only. The diagonal covariance
makes per-step cost O(λ·n) instead of CMA-ES's O(λ·n²) — much faster on
high-dimensional problems where full-covariance tracking is expensive
or numerically fragile, at the cost of being unable to handle strongly
rotated landscapes.
The first sample-efficient algorithm in heuropt. Bayesian optimization
maintains a Gaussian-process surrogate of the objective and at each
step picks the next decision by maximizing an acquisition function on
that surrogate, so the evaluation budget is used surgically.
Implementation:
- **Kernel**: anisotropic RBF (squared-exponential) with per-axis
length scales, signal variance, and a small noise/jitter floor.
Hyperparameters are exposed in the config; a future version can add
marginal-likelihood maximization.
- **Posterior**: standard formulation. Cholesky factorizes K (using
the new internal helper); mean and variance predictions follow.
- **Acquisition**: Expected Improvement against the best observed
feasible point. Optimized by best-of-N random sampling — simple,
predictable cost, no inner-optimizer footgun.
- **Initial design**: `initial_samples` uniform-random points in
bounds before the BO loop starts.
- **Constraints**: feasibility-aware EI — best observed value uses
only feasible points; infeasible candidates are penalized.
Vec<f64> decisions, single-objective only. Targets the regime no
existing heuropt algorithm covers: 50–500 evaluations on an
expensive black-box function (CFD sim, ML training run, real-world
measurement).
Tests cover convergence on the 1-D sphere within a tight evaluation
budget (~30 evals get to f < 1e-6 — vs population-based methods
needing thousands), deterministic reruns, and panic on
multi-objective + dim mismatches.
Auger & Hansen 2005 IPOP-CMA-ES: wraps the existing CmaEs in a restart
loop that doubles the population size and re-randomizes the mean
whenever a restart trigger fires. Specifically addresses the failure
mode we observed on Rastrigin (vanilla CMA-ES = 2.3 vs DE = 0).
Restart triggers:
- The whole budget for one inner CmaEs run finishes without improvement
- (More sophisticated triggers — eigenvalue collapse, condition-number
blow-up, sigma stagnation — are left for future versions; the
per-run budget trigger captures the bulk of the practical benefit)
Each restart:
- Doubles the population_size (Auger & Hansen 2005)
- Re-randomizes the initial mean to a fresh point in the bounds box
- Resets sigma to the user's initial value
Same Vec<f64> + single-objective constraints as CmaEs. The total
budget is divided across restarts; restart budget grows with
population. Tests verify it beats vanilla CMA-ES on Rastrigin.
Nelder & Mead 1965: gradient-free local optimizer that maintains a
simplex of n+1 points in n-D and at each iteration replaces the worst
vertex by one of {reflect, expand, outside-contract, inside-contract,
shrink} relative to the centroid of the rest. The five standard
coefficients (reflection α=1, expansion γ=2, contraction ρ=0.5,
shrinkage σ=0.5) are exposed in the config but default to canonical
values so users can leave them alone.
Single-objective only, Vec<f64> only, bounds enforced by clamping
each new vertex. Termination is purely iteration-count for v0.2;
"vertices have collapsed" stopping is a future enhancement.
Filling a real gap: heuropt had population-based local search
(SimulatedAnnealing, HillClimber) but no classical direct-search
algorithm. Excellent for low-dim smooth-ish problems where a
population is overkill.
Rechenberg 1973's elemental evolution strategy: one parent, one child
each generation, accept the child if it is no worse, and adapt the
mutation step size by tracking the success rate. If more than 1/5 of
recent moves were accepted the search is too cautious — multiply σ by
`step_increase` (typical 1.22). Below 1/5 — divide by the same factor.
At 1/5 — leave it alone. The success window has length `adaptation_period`.
Single-objective only. Vec<f64> only. Generic Gaussian step bounded by
the embedded `RealBounds`.
Why ship it: it's the smallest possible self-adapting evolution strategy
and a useful pedagogical / baseline endpoint. Pairs well as the budget
floor ("give me anything cheaper than CMA-ES").
Zhang, Tian & Jin 2015 KnEA: many-objective MOEA that biases survival
selection toward 'knee points' on the Pareto front — points where a
small improvement in one objective costs a large degradation in
another.
Each generation:
- NSGA-II-like loop with offspring + non_dominated_sort
- For the splitting front, identify knee points by perpendicular
distance from the hyperplane connecting the front's extreme points.
Members further from the hyperplane (= more 'kneeness') are preferred.
- Survival keeps every knee-tagged member; if room remains, fill from
remaining members by largest perpendicular distance.
Knee points are intuitively the most attractive points on a Pareto
front when no preference information is available. KnEA pushes the
search toward them at the cost of less uniform front coverage.
Yang, Li, Liu & Zheng 2013 GrEA: many-objective MOEA whose secondary
ranking is a grid-based diversity score instead of crowding distance
or reference vectors.
Each generation:
- NSGA-II-like loop with offspring + non_dominated_sort
- For the splitting front:
- Translate by ideal/nadir; partition objective space into a
(`grid_divisions` per axis) grid
- For every member compute three grid scores:
- GR (grid rank) = sum of grid coordinates (closer to ideal = lower)
- GCD (grid crowding distance) = #neighbors within 1 grid unit (in any axis)
- GCPD (grid coordinate point distance) = max coord - min coord
- Sort F_l ascending by GR, then by GCD, then by GCPD
- Take the top `n - already_selected` survivors
GrEA's grid-based niching is a different lens from NSGA-III's reference
points and RVEA's reference vectors — particularly effective on
non-convex fronts where reference-vector approaches struggle.
Panichella 2019 AGE-MOEA: a many-objective MOEA that *infers* the
front's geometry (its L_p shape, where p = 1 is linear, p = 2 is
spherical, p < 1 is convex etc.) from the current non-dominated set
and uses that estimate to drive both proximity and diversity in
survival selection.
Each generation:
- NSGA-II-like loop: random parent selection + variation + evaluation
- Combine + non_dominated_sort
- Fill front-by-front; for the splitting front:
- Translate by ideal point z*
- Find extreme points by ASF (same as NSGA-III) and intercepts
- Estimate the geometry parameter p by minimizing
\|f − ideal\|_p constancy on the extreme points
- Score every member by survival_score = (proximity_to_ideal) +
(1 / nearest-neighbor distance in the same L_p frame)
- Keep the top scorers
The geometry estimation is the novel contribution; with 3+ objectives
it produces fronts whose spread better matches the true shape than
NSGA-III's reference points (which assume a known geometry).
Rao 2011 TLBO: parameter-free single-objective optimizer for Vec<f64>.
The selling point — uniquely among the metaheuristics we ship — is that
it has NO algorithm-specific hyperparameters: no F, CR, w, c1, c2, σ,
mutation rate, etc. Just population_size and generations.
Each generation has two phases:
- **Teacher phase**: identify the best individual (the 'teacher'). For
every learner, compute a 'mean' learner and try replacing it with a
candidate moved toward the teacher by a random fraction, scaled by
the gap between teacher and (TF · mean), where TF ∈ {1, 2}.
- **Learner phase**: each learner picks a random partner and tries
moving toward the better one of the pair. Only successful moves are
kept.
Single-objective only, Vec<f64> only, bounds enforced via clamping.
Tests cover Sphere1D convergence, deterministic reruns, and panic on
multi-objective.
Replaces strict Pareto dominance with ε-dominance: A ε-dominates B when
`floor(A_i / ε) ≤ floor(B_i / ε)` for every objective and strictly
less in at least one (minimization frame). The result is a regular
discretization of objective space — at most one archive member per
ε-box — so the front spreads out automatically and the archive size
self-limits without truncation tricks.
Steady-state design: each generation samples one parent from the main
population and one from the ε-archive, applies variation, evaluates
the child, and offers it to both archives. Every member's
ε-coordinates and the box-tie rules are precomputed each insertion.
Tests: produces a front on Schaffer N.1 with reasonable spread,
deterministic reruns, panic on `epsilon[i] <= 0.0` and on
`epsilon.len() != objectives.len()`.
Corne, Jerram, Knowles & Oates 2001: divides objective space into a
hyperbox grid and uses per-box population counts to drive selection
toward sparsely-populated regions.
Each generation:
- Maintain an external archive of non-dominated members
- Build a hyperbox grid (`grid_divisions` per axis on the archive's
current axis ranges); count members per box
- Selection picks two parents by region-based tournament: choose two
random non-empty boxes and take a uniform-random member from the
one with fewer occupants
- Variation produces an offspring; insert into archive, dropping
dominated members and (if archive overflows) the most-crowded
occupant of the most-occupied box
Tests cover non-empty front on Schaffer N.1, deterministic reruns,
and panic on `archive_size == 0`.
Cheng, Jin, Olhofer & Sendhoff 2016 RVEA: many-objective MOEA built
around a fixed set of Das–Dennis reference vectors. Each generation:
- Generate offspring via random parent selection + variation +
evaluation
- Combine population + offspring; translate by ideal point z*
- Associate every member with the reference vector whose angle to
the translated objective vector is smallest
- For each occupied vector, keep the member with the smallest
Angle-Penalized Distance (APD) score; the rest are dropped
- APD = (1 + α(t)·θ_max·γ) · |f − z*| where γ is the angle to the
associated reference and α(t) = (t / t_max)^2 anneals the angle
penalty over the run
This produces well-spread fronts at high objective counts where
Pareto-rank methods (NSGA-II, SPEA2) lose discrimination.
Bader & Zitzler 2011: HypE estimates hypervolume contributions via
Monte Carlo sampling instead of computing them exactly. The point of
the trick is that exact hypervolume becomes prohibitively expensive
beyond ~5 objectives, while MC sampling stays cheap and accurate
enough at any dimension.
Each generation:
- Generate offspring via parent selection + variation + evaluation
- Combine, run non_dominated_sort, fill front-by-front
- For the splitting front, estimate each member's HV contribution
by drawing `n_samples` uniform points in the box [ideal, reference]
and counting how many points are dominated by *exactly* one front
member — that count, divided by n_samples and multiplied by the
box volume, is the member's expected unique HV contribution.
- Drop members one at a time from the splitting front by smallest
estimated contribution.
Public API matches the rest of the MO algorithms (Config + Optimizer).
The reference point is supplied in the config so the user controls
the integration domain. Tests cover non-empty front, deterministic
reruns, and panic on dim-mismatched reference.
Beume, Naujoks & Emmerich 2007: a steady-state MOEA that uses
hypervolume contribution as the secondary survival selection criterion.
Each generation:
- Generate ONE child via parent selection + variation + evaluation.
- Combine population + child, run non_dominated_sort.
- The discarded individual is the worst-front member with the
smallest hypervolume contribution (computed via the new
hypervolume_nd_from_evaluations helper).
Selection-quality is excellent at moderate objective counts (2–4) at
the cost of higher per-step compute (each survival selection requires
N+1 hypervolume evaluations of size ≤ N each). Best paired with a
tightly-bounded objective space — the user supplies a fixed reference
point in the config.
Tests: produces a non-empty front on Schaffer N.1, deterministic
reruns, panic on `population_size == 0`, panic on
`reference_point.len() != objectives.len()`.
Mühlenbein 1997 UMDA: simplest Estimation-of-Distribution Algorithm for
`Vec<bool>` problems. Each generation:
- Evaluate the current population
- Select the top μ members by fitness
- Estimate per-bit marginal probability p_i = (count of 1s at bit i in
the μ-best) / μ
- Sample population_size new individuals from the resulting product-of-
Bernoullis distribution
Single-objective only. Bit-wise probabilities are clamped to
`[1 / (2 · μ), 1 - 1 / (2 · μ)]` to keep the population from collapsing
to a deterministic single string before convergence is meaningful
(standard Laplace-style smoothing for UMDA).
Tests: solves OneMax (maximize Σ bits) on a 20-bit instance,
deterministic reruns, panic on multi-objective.
Dorigo-style Ant System for permutation problems on a complete graph:
each generation, every ant constructs a tour by probabilistically
picking the next node from those it has not yet visited, weighted by
`τ_ij^α · η_ij^β` where τ is the pheromone level on edge (i, j) and
η is the heuristic desirability (1 / distance, here). After all ants
finish, pheromone evaporates by a factor `(1 - ρ)` and is reinforced
on each ant's tour proportional to that tour's quality.
Decision type is `Vec<usize>` (a permutation of 0..n_cities). The user
supplies a distance matrix and the n_cities is inferred. Single-objective
only (the cost is total tour length, which the Problem evaluates).
Tests build a 5-city ring and verify ACO finds a near-optimal tour,
plus deterministic reruns and panic on multi-objective.
Zitzler & Künzli 2004 IBEA: replaces Pareto-rank + crowding fitness
with a single scalar fitness derived from a binary quality indicator
(here, the additive ε-indicator). Loses no information at three or
more objectives the way crowding distance does.
Algorithm:
- For every (i, j) pair compute I(i, j) = max_k (f_k(i) - f_k(j)) on
minimization-oriented objectives.
- Fitness F(i) = -Σ_{j≠i} exp(-I(j, i) / κ).
- Each generation: combine parents + offspring, iteratively remove the
lowest-F member (cleanly recomputing the contribution of the dropped
member from each surviving member's fitness) until population_size
remain.
- Parent selection: binary tournament on F (higher wins).
Bounds-aware operators recommended (SBX + PolyMut).
Tests: produces a non-empty front on Schaffer N.1, deterministic
reruns, panic on `population_size == 0`.
Coello, Pulido & Lechuga 2004 MOPSO: PSO adapted for multi-objective
optimization via an external Pareto archive used as the source of
swarm leaders.
Each generation:
- Evaluate every particle's current position
- Insert non-dominated members into the archive (using ParetoArchive)
- For each particle, pick a leader from the archive (uniform random
among archive members)
- Update velocity using inertia + cognitive (toward pbest) + social
(toward leader)
- Update positions, clamp to bounds
- Refresh personal bests using Pareto comparison: pbest is replaced
only when the new position dominates it; on non-dominated, keep
with 50/50 random tiebreak
Vec<f64> decisions only. Truncates the archive to `archive_size` via
the existing simple-tail truncation. Tests: produces a non-empty
front on Schaffer N.1, deterministic reruns, panic on
single-objective.
Hansen & Ostermeier 2001 CMA-ES, the canonical real-valued
single-objective stochastic optimizer. Implements the full (μ/μ_w, λ)
update with rank-μ + rank-1 covariance updates and cumulative step-size
adaptation:
- Sample λ offspring from N(mean, σ² · C)
- Select the μ best, weight them, recompute mean
- Update evolution paths p_σ (step size) and p_c (covariance)
- Rank-1 update of C from p_c, plus rank-μ update from selected offspring
- Adapt σ via |p_σ| / E‖N(0,I)‖
Eigendecomposition (used to convert C into its B·D form for sampling
N(0, σ²·C)) goes through the new internal Jacobi helper, recomputed
every `eigen_decomposition_period` generations to amortize cost.
Vec<f64> decisions only. Bounds taken from a `RealBounds` field; mean
and offspring are clamped per dimension. Single-objective only.
Hyperparameters use the standard CMA-ES defaults (μ=λ/2, weights from
Hansen's tutorial, c_σ, c_c, c_1, c_μ, d_σ all formulae from §7.1).
Tests cover: convergence on Sphere1D and 5-D Rosenbrock, deterministic
reruns, panic on multi-objective, panic on `population_size < 4`.
Glover 1986 tabu search for single-objective problems. Generic over
decision type — the user supplies a neighbor-generator closure that
produces a finite list of candidate moves from the current incumbent
(e.g. all 2-swaps for a permutation, or N Gaussian-perturbed copies of
a real vector). Each iteration picks the best non-tabu neighbor (with
an aspiration override that lets a tabu move through if it beats the
best-seen-ever incumbent) and adds the chosen move's decision to a
fixed-size FIFO tabu list.
Single-objective only. Tracks the best-seen-ever incumbent across the
run, returned as the result. Generic over the decision `D: Hash + Eq`
so the tabu list can match by full decision (simple and correct;
move-based tabu is left for users to implement themselves via a
custom decision wrapper).
Eberhart & Kennedy 1995 PSO with the standard inertia-weight update:
v[i,t+1] = w·v[i,t] + c1·r1·(pbest[i] - x[i,t]) + c2·r2·(gbest - x[i,t])
x[i,t+1] = clamp(x[i,t] + v[i,t+1], bounds)
Single-objective only, `Vec<f64>` decisions only (PSO's velocity vector
needs a Euclidean structure that doesn't generalize cleanly to bool/perm).
Velocities are clamped to ±(hi - lo) per dim to keep particles from
exploding off into space.
Config exposes the four standard knobs — swarm size, generations,
inertia w, cognitive c1, social c2 — plus a seed. Tests cover
convergence on Sphere1D, deterministic reruns, and panic on
multi-objective.
Canonical generational GA with elitism: each generation runs binary
tournament selection (using `tournament_select_single_objective`) on
the current population, applies the variation operator pair-wise to
produce offspring, evaluates them, then replaces the population while
preserving the top `elitism` members from the previous generation
(elitism prevents fitness regression on a single seed).
Single-objective only. Generic over decision type — pair with
`SimulatedBinaryCrossover + PolynomialMutation` for real-valued,
single-point crossover + bit-flip for binary, etc.
Tests: convergence on Sphere1D, deterministic reruns, panic on
multi-objective, panic on `population_size < 2`, panic on
`elitism > population_size`.
Classic Kirkpatrick et al. 1983 SA: hill climber that also accepts
worse moves with probability `exp(-Δ/T)` where T anneals geometrically
from `initial_temperature` to `final_temperature` over the iteration
count.
Single-objective only. Generic over decision type — works on real
vectors, bool vectors, permutations, anything. Tracks the best-seen
incumbent across the run (not just the last accepted move) so the
result reflects the actual best ever visited, not where the random
walk happened to end.
Tests cover: convergence on Sphere1D under reasonable hyperparameters,
deterministic reruns, panic on multi-objective, panic on
non-positive temperatures.
The simplest possible local search: start from one initializer-sampled
decision, repeatedly mutate it via the variation operator, and keep the
child only when it is strictly better than the current incumbent (with
the standard feasible-beats-infeasible / lower-violation tiebreaks
when relevant).
Single-objective only — panics with a clear message if the problem
exposes more than one objective. Deterministic under a seed. Returns
a population/front of size one (the current incumbent) so it slots
into the comparison harness like any other optimizer.
Implementation of Zhang & Li 2007 MOEA/D — the canonical
decomposition-based MOEA. Different paradigm from Pareto-dominance
algorithms: each subproblem is a scalarized single-objective problem
defined by a Das–Dennis weight vector, and subproblems with similar
weight vectors form neighborhoods that share genetic material.
Each generation iterates over every weight vector `i`:
1. Pick two parents uniformly from the T-nearest neighbors of weight i
(T = neighborhood_size).
2. Apply variation, evaluate the child.
3. Update the ideal point z* with the child's objectives.
4. Walk the entire neighborhood: for each j, if the child's
Tchebycheff value g(child | w_j, z*) <= g(current[j] | w_j, z*),
replace current[j] with the child.
Tchebycheff scalarization:
g(f | w, z*) = max_k w_k · |f_k - z*_k|
(With the standard `w_k = 1e-6` floor when a weight is zero, so the
max well-defined.)
Public API:
MoeadConfig {
generations,
reference_divisions, // Das-Dennis H, also fixes population size
neighborhood_size, // T
seed,
}
Moead { config, initializer, variation }
impl<P, I, V> Optimizer<P> for Moead<I, V>
Population size equals the number of weight vectors generated by
das_dennis(num_objectives, reference_divisions). Re-exported from the
prelude. Tests cover non-empty Pareto front, deterministic reruns,
and panic on `reference_divisions` that would yield zero weights.
- nsga3: drop redundant `.into_iter()` in extend call; use
`#[allow(clippy::needless_range_loop)]` on the back-substitution
loop where `j` indexes into the matrix; remove an unneeded
`return` keyword in a closure.
- spea2: switch `pool.extend(x.drain(..))` to `pool.append(&mut x)`.
- examples/compare.rs DTLZ2 evaluator: same `needless_range_loop`
silencer on the inner cosine product loop.
Implementation of Deb & Jain 2014 NSGA-III — the canonical
many-objective MOEA. Replaces NSGA-II's crowding-distance niching
with a structured reference-point niching procedure that scales to
3+ objectives where crowding distance loses its diversity signal.
Each generation:
1. Random parent selection + variation + offspring evaluation, same as
NSGA-II.
2. Combine + non_dominated_sort, fill the next population front-by-
front until the next front would overflow (the splitting front F_l).
3. Survival on F_l uses reference-point niching:
- Translate by the ideal point z* (per-axis min in oriented space).
- Compute extreme points by ASF and intercepts; normalize by
intercepts (with a robust fallback to per-axis range if extreme
points are degenerate).
- Associate every member of the working pool with the closest
reference direction by perpendicular distance.
- Iteratively pick from F_l: prefer the niche with the smallest
count among references that have F_l candidates; if the niche is
empty in the already-selected set, take the closest associated
member by perpendicular distance, otherwise pick uniformly from
the niche.
Public API:
Nsga3Config { population_size, generations, reference_divisions, seed }
Nsga3 { config, initializer, variation }
impl<P, I, V> Optimizer<P> for Nsga3<I, V>
Re-exported from the prelude. Tests cover non-empty Pareto front,
exact final population size, deterministic reruns, and panic on
`population_size == 0`. Uses the existing tests_support problems.
Implementation of Zitzler, Laumanns, Thiele 2001 SPEA2 — the classic
Pareto MOEA built around an explicit external archive of fixed size.
Each generation:
1. Combine the current population and the archive into one pool.
2. For every member, compute strength S(i) = number of others that
member dominates, then raw fitness R(i) = sum of S(j) over members
j that dominate i.
3. Add a density estimator D(i) = 1/(σ_k + 2) where σ_k is the distance
to the k-th nearest neighbor (k = floor(sqrt(|pool|))) in
minimization-oriented objective space.
4. Final fitness F(i) = R(i) + D(i); lower is better.
5. Build the next archive by taking every non-dominated member
(R(i) == 0). If too many, prune by repeatedly removing the member
with the smallest k-th-nearest-neighbor distance. If too few, fill
from the rest sorted by F ascending.
6. Generate the next population by binary tournament on F (lower wins),
then variation, then evaluation.
Public API mirrors the other algorithms:
Spea2Config { population_size, archive_size, generations, seed }
Spea2 { config, initializer, variation }
impl<P, I, V> Optimizer<P> for Spea2<I, V>
Re-exported from the prelude. Tests cover archive size invariants,
non-empty Pareto front on Schaffer N.1, deterministic reruns under
the same seed, and panic on population_size == 0.
Adds a `parallel` Cargo feature that pulls in rayon and parallelizes
the only step that's actually expensive in practice — calls to
`Problem::evaluate` — across the population. RNG-driven steps (parent
and donor selection, variation, replacement decisions) stay serial, so
seeded runs remain deterministic regardless of feature state, and the
default and `--features parallel` builds produce bit-identical
results.
Wiring:
- New `algorithms::parallel_eval::evaluate_batch` helper with two
cfg-gated implementations (rayon's `into_par_iter` when the feature
is on, plain `into_iter` otherwise). Both preserve input order, so
pareto_front and crowding-distance decisions remain reproducible.
- `RandomSearch`, `Nsga2`, and `DifferentialEvolution` now route
population/offspring evaluation through the helper. NSGA-II's main
loop is restructured into a serial selection-and-variation phase
followed by a parallel-friendly batch evaluation phase.
- DE's per-target loop is restructured into three phases (serial trial
construction → batch evaluation → serial replacement). Side effect
of the restructuring: DE is now the canonical synchronous DE/rand/1/bin
rather than the asynchronous variant where target `i+1` sees `i`'s
in-flight update. Synchronous is the textbook formulation, so this
is a small correctness improvement on top of the parallelism enable.
- PAES stays serial — its main loop has a sequential dependency on the
current candidate and would gain nothing from rayon.
Cost: algorithm impls now require `P: Sync` and `P::Decision: Send`
unconditionally so a single impl serves both feature modes. This is a
small bound tightening that any plain-data Problem already satisfies; in
return the public `Problem` trait itself stays unchanged and the
default build picks up no new dependencies.
Verified:
- `cargo test` and `cargo test --features parallel` both pass; the
Nsga2 `deterministic_with_same_seed` test confirms reproducibility.
- `cargo run --release --example benchmarks` and the same with
`--features parallel` produce bit-identical ZDT1 / Rastrigin
results.
Optional v1 algorithm requested by the user (spec §12.4):
- Vec<f64> decisions only.
- Single-objective only — panics with a clear message otherwise.
- Standard DE/rand/1/bin: for each target i, sample distinct r1, r2, r3;
mutant = x[r1] + F * (x[r2] - x[r3]); apply binomial crossover with at
least one forced index; greedy replacement on direction-correct
comparison.
- Bounds taken from the embedded RealBounds (mutants are clamped to the
per-variable range so the trial vector stays feasible).
- Seed-deterministic; tests verify reproducibility, that DE improves on
the initial random population for a sphere problem, and that
multi-objective use panics.
Standard (μ+λ) NSGA-II with binary tournament parent selection on
(rank, crowding distance) and elitist survival selection on the combined
parent + offspring population (spec §12.3):
1. Initialize population_size random decisions.
2. Each generation: select parents by binary tournament (rank ↑ then
crowding ↓ then random), apply variation, evaluate offspring,
combine, non_dominated_sort, fill the next population front-by-front
trimming the partial last front by crowding distance descending.
3. Return final population, Pareto front, best (None for >1 objective),
evaluation count, and generation count.
Internal Nsga2Entry { candidate, rank, crowding_distance } stays
private. Panics with clear messages on `population_size == 0` or
empty `vary` output. Tests cover population length, evaluation count,
non-empty front, and full determinism with the same seed (spec §18.4).
A readable v1 PAES (spec §12.2):
- Single starting decision from the initializer.
- Each iteration mutates the current decision via the Variation operator,
evaluates the child, and pareto_compares to the current.
- Dominating children become current; for non-dominated comparisons we
move to the child (acceptable v1 behavior per spec).
- Both current and child are inserted into a ParetoArchive truncated
to `archive_size` (simple tail-truncation in v1).
The final result returns the archive as both `population` and
`pareto_front`. Tests verify the archive never exceeds
`archive_size`.
The reference baseline and the spec's recommended starting example. Per
iteration it asks the initializer for `batch_size` decisions, evaluates
each, and accumulates them. At the end it returns the full population
plus the Pareto front and (if single-objective) the best feasible
candidate. `generations` equals `iterations`; `evaluations` equals
`iterations * batch_size` (spec §12.1).
Includes a tiny single-objective sphere test problem under
`tests_support` that later algorithm tests will reuse.
swaits