swaits/heuropt
1
0
Fork 0
Code Issues Pull Requests Actions 4 Packages Projects Releases Wiki Activity

Compare commits

base: swaits/heuropt:v0.1.0
Branches Tags
swaits/heuropt:main
swaits/heuropt:v0.11.0 swaits/heuropt:v0.8.0 swaits/heuropt:heuropt-plot-v0.1.0 swaits/heuropt:v0.7.0 swaits/heuropt:v0.6.0 swaits/heuropt:v0.5.0 swaits/heuropt:v0.4.0 swaits/heuropt:v0.1.0
..
compare: swaits/heuropt:v0.4.0
Branches Tags
swaits/heuropt:main
swaits/heuropt:v0.11.0 swaits/heuropt:v0.8.0 swaits/heuropt:heuropt-plot-v0.1.0 swaits/heuropt:v0.7.0 swaits/heuropt:v0.6.0 swaits/heuropt:v0.5.0 swaits/heuropt:v0.4.0 swaits/heuropt:v0.1.0
63 Commits
v0.1.0 .. v0.4.0
Author SHA1 Message Date
swaits 232dbc0172 chore(release): bump to v0.4.0 
CHANGELOG entry consolidates the unreleased work since v0.3.0:
testing-infrastructure expansion (proptest suites, cargo-fuzz
harness, stability tests, gungraun benches, CI), two real bug
fixes the testing surfaced (NaN-cycle non_dominated_sort, simplex
projection magnitude precision), the README decision-tree update
against the v0.3.0 comparison data, and the v0.4.0 perf pass
(cumulative compare harness 18.6 s → 5.7 s, 3.27×).

`examples/compare-results.md` refreshed with the post-perf-pass
ms numbers; quality metrics are bit-identical to the v0.3.0
snapshot (the perf pass was strictly CPU time, never algorithmic).
 2026-05-05 13:28:47 -06:00
swaits d60a3c3fe8 perf(pareto_archive): cache oriented + inline dominance checks in insert 
`ParetoArchive::insert` calls `pareto_compare` twice per existing
member (once per pass), and each call re-allocates two Vec<f64>s
via `as_minimization` — 4N allocations per insert. Cache the
candidate's oriented + feasibility/violation once, build each
member's oriented vector once for the call, then inline the
dominance test against those cached arrays.

Used by PESA-II (per offspring per generation), PAES (per child),
ε-MOEA, and any user code working through the archive directly.

Wall-clock (compare harness, 10-seed mean):
- PESA-II / DTLZ2: 498 → 426 ms (-14 %)
- PESA-II / ZDT1:   87 →  75 ms (-14 %)

Smaller wins on PAES / MOPSO / IBEA / HypE / ε-MOEA where the
archive isn't the dominant per-generation cost.

Bit-identical via the compare harness.
 2026-05-05 13:28:38 -06:00
swaits 060841d011 build(release): enable thin LTO + codegen-units=1 in release profile 
Cuts ~150 ms (-2.5 %) off the compare harness via better cross-crate
inlining of small Pareto/HV helpers. Costs ~20 s extra on a from-
scratch `cargo build --release`, but is essentially free on
incremental rebuilds.

Only applies when this crate is the workspace root (i.e. when
developing heuropt or running its own examples). Downstream users
who consume heuropt as a dependency see whatever profile their own
Cargo.toml configures.
 2026-05-05 13:28:18 -06:00
swaits f01089e9d0 perf(hypervolume): index-sort instead of cloning point vectors 
The M≥3 branch of `hso_recursive` cloned every input point into
`sorted: Vec<Vec<f64>>` solely so it could sort. Each clone is M
f64s allocated; with N points per call and ~30 HV calls per SMS-EMOA
generation × 30 k generations, that's millions of small Vec<f64>
allocations.

Sort indices into a `Vec<usize>` instead, then iterate the original
points by index. The pre-projection step still produces a
Vec<Vec<f64>> (which the active-prefix slicing requires), but we
save the outer N inner-Vec clones per call.

gungraun (instructions):
- hypervolume_nd_3d n=30:    87 969 →  70 334  (-20 %, 1.25×)
- hypervolume_nd_3d n=100:  422 767 → 367 767  (-13 %, 1.15×)

Cumulative vs the v0.3.0 baseline:
- hypervolume_nd_3d n=30:    676 902 →  70 334  (9.6×)
- hypervolume_nd_3d n=100: 13 523 760 → 367 767 (37×)

Wall-clock impact is in the noise on the compare harness because the
SMS-EMOA worst-front HV calls operate on small fronts (5–10 points
once converged). The win is most visible in synthetic dense-front
HV benchmarks.
 2026-05-05 13:28:18 -06:00
swaits adf18950dc perf(spea2): incremental truncation sort + cache compute_fitness inputs 
Two independent wins in SPEA2's per-generation hot path. Both
bit-identical against the compare harness.

# 1. compute_fitness — cache oriented + distance matrix

`compute_fitness` is called twice per generation. The strength-graph
loop calls `pareto_compare` in an N² loop, allocating two Vec<f64>s
per call via `as_minimization`. Inline the dominance test against
cached oriented arrays. The density loop's per-row euclidean recompute
is replaced by a symmetric N×N distance matrix built once.

# 2. build_archive — incremental sort maintenance in truncation

The archive-truncation loop was O(K³ log K) — each pruning iteration
recomputed every alive member's pairwise distances and re-sorted them,
when the only change since the prior iteration was that one specific
neighbor (the just-removed victim) became dead. Compute the distance
matrix and sorted neighbor vectors once, then on victim removal use
binary-search-remove on every survivor's still-sorted vector. Total
truncation cost drops from O(K³ log K) to O(K² log K). Victim choice
is bit-identical.

gungraun (instructions):
- spea2_short: 179 113 → 133 783 (-25 %, 1.34×)

Wall-clock (compare harness, 10-seed mean):
- SPEA2 / ZDT1:   458 → 241 ms (1.9×, cumulative)
- SPEA2 / DTLZ2: 4304 → 513 ms (8.4×, cumulative)
 2026-05-05 13:28:18 -06:00
swaits 4c7126070b perf(age_moea): cache lp_norm + maintain nearest-neighbor incrementally 
The splitting-front survival selection in AGE-MOEA recomputed two
expensive things per while-iteration:

* `lp_norm(translated[i], p)` for every remaining i — even though the
  value is constant across iterations.
* `nearest_neighbor_distance(i, …, &keep, p)` — a fresh full scan
  over the keep list, even though only one new candidate was added
  since the last scan.

Both are `powf`-heavy in the L_p frame.

Compute lp_norm once per candidate at function entry. Maintain a
`nearest[]` array seeded from the initial keep set and updated on
every pick by a single `min(nearest[i], lp_distance(i, pick, p))`
per remaining i. That cuts the score loop from O(R · K · M) to
O(R · M) per iteration, with the dominant powf calls in
lp_distance counted once per (remaining, pick) pair instead of per
(remaining, full-keep).

Wall-clock (compare harness, 10-seed mean):
- AGE-MOEA / DTLZ1: 2266 → 430 ms on top of v0.3.0 baseline (5.3×)
- AGE-MOEA / ZDT3:   935 → 376 ms (2.5×)
 2026-05-05 13:28:18 -06:00
swaits 214f07975a perf(non_dominated_sort): cache oriented values + inline pareto_compare 
The Deb fast non-dominated sort calls `pareto_compare` twice for
every (i, j) pair, and each `pareto_compare` call invokes
`ObjectiveSpace::as_minimization` twice — so for an N-point
population that's 4·N·(N-1) fresh `Vec<f64>` allocations per sort.
At N=100 with thousands of generations across the compare harness,
this dominated the per-generation cost of every Pareto-based MOEA.

Cache `as_minimization`/feasibility/violation once per individual
up front, then inline the dominance test against those cached
arrays. The output (per-pair dominance outcome and the per-i
`dominates` lists) is bit-identical to `pareto_compare`.

gungraun (instructions):
- non_dominated_sort_2d n=50:    852 317 →   198 574 (-77 %, 4.3×)
- non_dominated_sort_2d n=200: 13 513 271 → 2 601 813 (-81 %, 5.2×)

Wall-clock (compare harness, 10-seed mean):
- NSGA-II / ZDT1:        268 →  65 ms (4.1×)
- NSGA-II / ZDT3:        267 →  65 ms (4.1×)
- NSGA-II / DTLZ2:       344 → 106 ms (3.2×)
- NSGA-II / Rastrigin:   260 →  71 ms (3.7×)
- NSGA-III / DTLZ2:      318 → 122 ms (2.6×)
- NSGA-III / DTLZ1:      303 → 122 ms (2.5×)
- SMS-EMOA / DTLZ2:     1413 → 1369 ms (small additional win on top of HV)
- AGE-MOEA / DTLZ1:      430 → 229 ms (1.9×, on top of the AGE-MOEA caching)
- HypE / DTLZ2:           80 →  44 ms (1.8×)
 2026-05-05 13:28:18 -06:00
swaits 4745a6bb16 perf(hypervolume): cut HSO recursion overhead by ~30× on n=100/3-D 
The HSO recursion in `hypervolume_nd` had three overheads that
dominated SMS-EMOA's per-generation cost on DTLZ2 (5.6 s baseline,
~30 k generations × ~40 HV calls per generation = ~1.2 M HV calls
per run):

1. `active = sorted.clone()` plus `active.iter().position(...)`
   linear scan to remove the just-processed point each band — O(N)
   per band, total O(N²) per HV call.
2. Per-band re-projection
   `active.iter().map(|q| q[..last].to_vec())` — full
   Vec<Vec<f64>> rebuild for every band, O(N·M) allocations per HV
   call.
3. `non_dominated_projection` called even when recursing into the
   M=2 base case, whose sweep already filters dominated points
   internally.

Replace (1) with prefix-slicing `projected_all[..=k]` (sort points
ascending by last axis once; the active set at each band is just a
prefix). Pre-project once outside the loop (2). Skip the explicit
non-dominance filter when the inner recursion is M=2 (3).

Bit-identical output verified by re-running the compare harness and
diffing against the v0.3.0 snapshot — every quality metric matches
to the last decimal.

gungraun (instructions):
- hypervolume_nd_3d n=30:    676 902 →    87 969  (-87 %, 7.7×)
- hypervolume_nd_3d n=100: 13 523 760 →   422 767 (-97 %, 32×)

Wall-clock (compare harness, 10-seed mean):
- SMS-EMOA / DTLZ2: 5643 ms → 1413 ms (-4230 ms, -75 %)
 2026-05-05 12:11:45 -06:00
swaits 345e3ea296 docs(readme): align decision tree with v0.3.0 comparison results 
The compare harness (re-run on 2026-05-05 produced bit-identical
results to the v0.3.0 snapshot) doesn't square with four claims in
the DT. Adjust:

- BayesianOpt: was "gold standard". At 60 evals on 5-D Rosenbrock
  with the default RBF kernel it produces f≈3172 (worse than
  RandomSearch). Add the caveat that BO is the gold standard *with*
  per-problem kernel tuning, not out of the box.
- MOPSO: was buried under "swarm style". On ZDT1 it wins HV outright
  and beats every dominance-based method on convergence by ~100×.
  Promote to its own "smooth real-valued 2-obj front" branch.
- SMS-EMOA: was "great on 2–3 obj at higher per-step cost". On these
  benches it loses to NSGA-II on both ZDT1 (HV 102.9 vs 118.3) and
  DTLZ2 (mean dist 0.048 vs 0.033). Reframe as "elegant in theory but
  underperforms NSGA-II on these benches at our budgets".
- NSGA-III: was "strong default" for many-objective. On DTLZ1 (the
  canonical linear-simplex test) it gets beaten by GrEA 3× and
  MOEA/D 2×. Split the many-obj branch by front geometry: linear /
  simplex → GrEA + MOEA/D; curved / unknown → NSGA-III + AGE-MOEA +
  RVEA.

The quick-reference one-liners below the DT got the matching tweaks
so the table and the tree agree.
 2026-05-05 11:40:14 -06:00
swaits 4a59041d1a style: apply rustfmt drift across the crate 2026-05-05 11:40:14 -06:00
swaits 84cee3f29e ci: add GitHub Actions workflow with full feature matrix and fuzz smoke 2026-05-05 11:28:33 -06:00
swaits 1eab8e4805 docs: add testing section to README and CHANGELOG entries for fuzz, fixes, and CI 2026-05-05 11:28:29 -06:00
swaits f67b5c5160 fix(pareto): partition NaN-cycle orphan indices into a residual front 2026-05-05 11:28:24 -06:00
swaits 51fc7271b1 fix(operators): make ProjectToSimplex robust to extreme magnitudes 2026-05-05 11:28:21 -06:00
swaits c7a8f43a99 test(fuzz): add cargo-fuzz harness for Pareto and operator hot paths 2026-05-05 11:28:09 -06:00
swaits aba9dbf469 build(bench): expand gungraun bench suite to cover every algorithm 
Goes from 6 benchmarks to ~25:

Pareto utilities (existing):
- non_dominated_sort_2d (n=50, 200)
- crowding_distance_2d (n=50, 200)
- hypervolume_2d (n=30, 100)
- hypervolume_nd_3d (n=30, 100)

Single-objective algorithms (all measured at "one short run"):
- random_search, hill_climber, one_plus_one_es, simulated_annealing
- genetic_algorithm, particle_swarm, differential_evolution, tlbo
- cma_es, separable_nes, nelder_mead, bayesian_opt, tpe

Multi-objective algorithms (one short run each):
- nsga2, nsga3, spea2, moead, mopso, ibea, sms_emoa
- hype, pesa2, epsilon_moea, age_moea, grea, knea, rvea

Each uses a tiny problem with realistic-shape parameters (small pop,
few generations, tight bounds) so the benchmark exercises each
algorithm's *inner loop cost* rather than dominated by RNG init or
config parsing.
 2026-05-05 11:03:00 -06:00
swaits 8a8c32f125 test(proptest): massive property-test expansion for every algorithm and operator 
Goes from 10 properties to 50+, organized into four files:

- tests/properties.rs (existing) — Pareto-utility invariants
- tests/algorithm_properties.rs (new) — every Optimizer impl gets:
  * determinism-with-seed property
  * no-panic-on-random-valid-input property
  * population-size-as-documented property where applicable
- tests/operator_properties.rs (new) — every Variation/Initializer/
  Repair impl gets the right size + in-bounds + no-panic properties
- tests/metric_properties.rs (new) — every metric gets monotonicity
  / non-negativity / dim-checking properties
- tests/numerical_stability.rs (new) — single-point populations,
  duplicate populations, near-zero bounds, very large bounds,
  algorithms-on-flat-fitness — none of which should panic.

Total: 226 unit tests + this much-larger property suite. Strategies
are factored into a small `prop_helpers` module shared across files
so the random-input generators stay consistent.
 2026-05-05 11:01:36 -06:00
swaits 36e1d9d796 test(mutants): add cargo-mutants config for advisory mutation testing 
Adds `.cargo/mutants.toml` configuring cargo-mutants to focus on the
algorithmic core (skipping benches, examples, tests_support) and pass
`--test-tool=cargo --no-shuffle` so a mutation that breaks the suite
gets caught quickly.

Mutation testing modifies the source one operator at a time (`>` →
`>=`, `+` → `-`, `true` → `false`, etc.) and re-runs the test suite.
A mutation that *survives* (tests still pass) is a hint that the test
suite isn't checking that bit of behavior — usually because:
- The mutated branch is dead code
- The unit tests rely on side-effects rather than return values
- A property test or invariant is missing

Not wired into CI as a gating check (it's slow — every mutation
re-runs the whole suite). Run locally with `cargo install cargo-mutants`
followed by `cargo mutants --in-diff HEAD~1` for incremental coverage,
or `cargo mutants` for a full sweep.

The config exclusions list explains *why* each module is skipped — most
are the "obvious" kind (benchmark harness, example problems) where
mutation kills are not informative.
 2026-05-05 10:39:32 -06:00
swaits dcf63316f6 test(proptest): add property-based tests for invariants 
Adds proptest as a dev-dependency and a `tests/properties.rs`
integration suite that probes invariants on randomly generated
inputs:

Pareto invariants:
- `pareto_compare` is anti-symmetric: A→B is opposite of B→A for
  Dominates / DominatedBy
- `pareto_compare` is reflexive on equal candidates (returns Equal)
- `pareto_front` output is internally non-dominated
- `non_dominated_sort` puts every member into exactly one front
- `crowding_distance` returns Vec same length as front; boundary
  points are infinity for fronts of size ≥ 2 in any axis-sortable
  configuration

Operator invariants:
- `SimulatedBinaryCrossover` returns 2 children of the right length,
  all in bounds
- `PolynomialMutation` returns 1 child of the right length, in bounds
- `BoundedGaussianMutation` returns 1 child in bounds
- `ClampToBounds` repair always lands in bounds
- `ProjectToSimplex` repair always sums to total and is non-negative

Algorithm invariants:
- For any seed, `Optimizer::run` is deterministic across two calls
- Final population has the documented size for population-based
  algorithms

These are the invariants the existing 226 fixed-input unit tests
collectively check; proptest gives us coverage on inputs they don't
cover individually.
 2026-05-05 10:38:53 -06:00
swaits 0b31b266ef build(deps): add gungraun (was iai-callgrind) instruction-count benches 
Wire `gungraun` 0.18 as a dev-dependency and a `benches/` directory
with instruction-count benchmarks for the algorithmic hot paths.

Why gungraun and not criterion: heuropt's hot paths are deterministic
numerical loops where wall-clock noise dominates real differences.
gungraun runs each benchmark under valgrind/callgrind once and reports
exact instruction counts — stable across machines and CI runners,
detects sub-microsecond regressions cleanly.

Benchmarks added:
- pareto::non_dominated_sort  (the inner loop of every Pareto MOEA)
- pareto::crowding_distance   (NSGA-II survival selection)
- metrics::hypervolume_nd     (HSO recursion, used by SMS-EMOA)
- internal::cholesky          (BO's per-step posterior factorization)
- algorithms::nsga2 single generation (end-to-end smoke check)
- algorithms::cma_es single generation (eigendecomposition cost)

Tracked size only — these aren't part of the regular CI matrix because
they need valgrind installed. Run with `cargo bench` locally.

Wired via the standard `[[bench]]` Cargo entries with `harness = false`
so gungraun's main_macro does the dispatch.
 2026-05-05 10:36:29 -06:00
swaits 15d3b2752c docs(examples): capture full comparison run output as compare-results.md 
Snapshot of `cargo run --release --example compare` after the v0.3.0
algorithm cohort. The harness runs 7 benchmark problems × ~20
algorithms × 10 seeds each (≈3 minutes wall-clock); this file is the
reference output so readers can scan results without running it
themselves.

Highlights worth reading even if you're skipping the file:
- ZDT1: MOPSO and MOEA/D dominate convergence; (1+1)-ES and DE tie
  at f = 0 on Rastrigin
- IPOP-CMA-ES drops vanilla CMA-ES from f=2.35 to f=0.13 on
  Rastrigin (the multimodal failure-mode it was added to fix)
- IBEA wins DTLZ2 (15× closer to true front than NSGA-III)
- GrEA wins DTLZ1 (linear simplex front matches grid-based niching)
- Nelder-Mead = 0 exactly on Rosenbrock; CMA-ES at machine epsilon
- Bayesian optimization at 60 evals is honestly bad on 5-D problems
  with the default kernel — flagged so readers don't conclude BO is
  weak in general; it just needs more evals or hyperparameter tuning
 2026-05-05 10:35:05 -06:00
swaits 6faff0204d docs(readme): update algorithm-selection decision tree for v0.3.0 
The DT was written when v0.2.0 shipped. v0.3.0 added a whole regime
(expensive evaluation, multi-fidelity) plus new entries in existing
regimes (CMA-ES restart variant, smooth SO direct search, parameter-
free SO, etc.) — fold them in.

Specifically:
- New top-level branch on "how expensive is each evaluation?" so the
  sample-efficient algorithms (BayesianOpt, Tpe) and multi-fidelity
  ones (Hyperband) have a clear home.
- Continuous-SO branch gains IPOP-CMA-ES (multimodal), Nelder-Mead
  (smooth, low-dim), (1+1)-ES (cheap baseline), sNES (high-dim
  alternative to CMA-ES), Tlbo (parameter-free).
- Multi-objective branches gain SMS-EMOA, HypE, ε-MOEA, PESA-II,
  AGE-MOEA, GrEA, KnEA, RVEA — placed by their distinguishing
  characteristic (geometry-aware, knee-points, grid-based, etc.)
- Quick-reference table extended to all 35 algorithms and grouped by
  paradigm.
 2026-05-05 10:30:45 -06:00
swaits 9ae1df68cb chore(release): roll up v0.3.0 — expensive-eval, gradient-free, multi-fidelity 
CHANGELOG entry for the v0.3.0 cohort, version bump in Cargo.toml and
README. Theme: filling heuropt's expensive-evaluation and constraint-
handling gaps.

Algorithms (9 new): OnePlusOneEs, NelderMead, IpopCmaEs, BayesianOpt,
SeparableNes, Tpe, Hyperband.

Operators (1 new): LevyMutation. Repair operators (1 trait + 2 impls):
Repair<D> with ClampToBounds and ProjectToSimplex.

Selection helpers (1 new): stochastic_ranking_select.

Internal helpers: Cholesky factorization (used by BO).

API additions:
- CmaEsConfig.initial_mean: Option<Vec<f64>> (None preserves existing
  midpoint-of-bounds behavior; used by IpopCmaEs to inject restart
  diversity).
- New PartialProblem trait — multi-fidelity contract used by
  Hyperband.

No breaking changes to v0.2.0 public API.
 2026-05-05 10:00:00 -06:00
swaits bfc2875d62 feat(traits,algorithms): add PartialProblem trait and Hyperband 
Multi-fidelity optimization. Hyperband (Li et al. 2017) and its
foundation Successive Halving (Karnin et al. 2013) tune
hyperparameters by allocating *uneven* compute across configurations:
sample many cheap-to-evaluate-at-low-budget configs, then promote
the survivors to higher budgets. Crucial for ML hyperparameter
tuning where each evaluation is a partial training run.

This requires a new trait — `Problem::evaluate` is a single-shot
black box, but Hyperband needs to evaluate the SAME decision at
different fidelity budgets:

  pub trait PartialProblem {
      type Decision: Clone;
      fn objectives(&self) -> ObjectiveSpace;
      fn evaluate_at_budget(&self, decision: &Self::Decision,
                            budget: f64) -> Evaluation;
  }

`PartialProblem` is intentionally NOT a sub-trait of `Problem`.
Implementors who already have a `Problem` and want their
`evaluate_at_budget` to ignore budget can write a one-line wrapper.

`Hyperband` is the optimizer:

  pub struct HyperbandConfig {
      max_budget: f64, eta: f64, max_brackets: usize, seed: u64,
  }
  pub struct Hyperband<I> { config, initializer, ... }

Single-objective only. The decision sampler is an `Initializer<D>` so
it works the same way as every other heuropt algorithm. Generic over
decision type.
 2026-05-05 09:59:03 -06:00
swaits 27f80fb2a3 feat(traits): add Repair<D> trait + ClampToBounds and ProjectToSimplex impls 
Spec §22 Round 4-D listed bounded mutation / repair operators as future
work; this is the second piece of that. A `Repair<D>` trait that nudges
infeasible decisions back to feasibility, intended to be called from a
user's Variation operator (or a CompositeVariation pipeline) when
projection-style constraint handling is preferred over the
penalty-style `constraint_violation` approach.

Trait:
  pub trait Repair<D> {
      fn repair(&mut self, decision: &mut D);
  }

Provided impls:
- `ClampToBounds` — clamps each variable of a Vec<f64> to per-axis bounds
- `ProjectToSimplex` — projects a Vec<f64> onto the (clipped) probability
  simplex (Σ x_i = total, x_i ≥ 0), useful for portfolio-style problems
  and reference-direction normalization

Both stay in the existing `operators` module (alongside Variation
operators) since they share the same "transforms decisions" theme. Re-
exported from the prelude.
 2026-05-05 09:56:44 -06:00
swaits 66f6cf6e86 feat(selection): add stochastic_ranking_select for constrained problems 
Runarsson & Yao 2000 stochastic ranking: a probabilistic alternative
to feasibility-first tournament selection. Each pairwise comparison
during a bubble-sort pass uses the *objective* value with probability
`pf` even when one or both candidates are infeasible. The classic
recommendation `pf = 0.45` reliably outperforms strict
feasibility-first on heavily-constrained problems where occasionally
exploring the infeasible region helps cross narrow feasible corridors.

New helper: `stochastic_ranking_select` lives next to
`tournament_select_single_objective` in `selection::tournament`.
Single-objective only; same signature pattern (population, objectives,
count, rng, plus the new `pf` knob).
 2026-05-05 09:55:43 -06:00
swaits 358e441b36 feat(algorithms): add Tpe (Tree-structured Parzen Estimator) 
Bergstra et al. 2011: sample-efficient sequential optimizer that's the
workhorse of Hyperopt and Optuna. Different surrogate from BO's
Gaussian process — TPE models p(x | y < y*) with one KDE and
p(x | y >= y*) with another, then samples candidates from the 'good'
KDE and ranks by the ratio l(x) / g(x). The acquisition is implicit
in the ratio (a closed-form analog of Expected Improvement).

Implementation:
- 1-D Gaussian KDE per axis, with bandwidth chosen by Scott's rule
- Per-step:
  - Evaluate observations into 'good' (top γ fraction by target) and
    'bad'
  - Sample n_candidates from the good distribution (independent per
    axis) and pick the one with the largest l(x)/g(x)
  - Evaluate it, append to history

Vec<f64> only, single-objective only. Compared with BayesianOpt:
- Cheaper per-step (no GP factorization)
- Doesn't need kernel hyperparameter tuning to work well
- Naturally extends to mixed/categorical decision types (future work)
- Generally less sample-efficient than well-tuned BO on smooth
  continuous problems, but more robust out of the box

Tests cover convergence on 1-D Sphere within a tight budget,
deterministic reruns, panic on multi-objective.
 2026-05-05 09:55:01 -06:00
swaits e7355ebb8a feat(algorithms): add SeparableNes (Natural Evolution Strategy) 
Wierstra et al. 2008/2014 NES with the diagonal-covariance "separable"
variant (sNES). Different theoretical foundation from CMA-ES: rather
than tracking a full covariance matrix and adapting it through
evolution paths, sNES updates the sampling distribution's parameters
by following the natural gradient of expected fitness.

Each generation:
- Sample λ offspring from N(μ, diag(σ²))
- Rank-shape the fitnesses (utility weights from the standard NES table)
- Update μ along the natural gradient: μ ← μ + η_μ · σ · sum(u_i · z_i)
- Update σ multiplicatively: σ_j ← σ_j · exp(η_σ/2 · sum(u_i · (z_i,j² - 1)))

Vec<f64> decisions only, single-objective only. The diagonal covariance
makes per-step cost O(λ·n) instead of CMA-ES's O(λ·n²) — much faster on
high-dimensional problems where full-covariance tracking is expensive
or numerically fragile, at the cost of being unable to handle strongly
rotated landscapes.
 2026-05-05 09:53:20 -06:00
swaits 8a34fd94b8 feat(examples): wire (1+1) ES, Nelder-Mead, IPOP-CMA-ES, BO into compare harness 
Adds runners for the four expensive-eval / gradient-free additions to
the appropriate single-objective sections of `examples/compare.rs`:

- Rastrigin (multimodal): now also shows IPOP-CMA-ES alongside vanilla
  CMA-ES so the restart benefit is directly visible.
- Rosenbrock (smooth valley): adds Nelder-Mead (well-suited) and (1+1)
  ES (cheap baseline).
- Ackley + Rosenbrock: BayesianOpt run with a deliberately TINY budget
  (60 evaluations vs 30k for the population-based methods) so the
  sample-efficiency claim is visible — BO with 60 evals vs DE/CMA-ES
  with 30k.

The compare harness now sides-by-sides 23 algorithms total across the
seven benchmark problems.
 2026-05-05 09:51:12 -06:00
swaits a70500406c feat(algorithms): add BayesianOpt — GP-based Bayesian Optimization 
The first sample-efficient algorithm in heuropt. Bayesian optimization
maintains a Gaussian-process surrogate of the objective and at each
step picks the next decision by maximizing an acquisition function on
that surrogate, so the evaluation budget is used surgically.

Implementation:
- **Kernel**: anisotropic RBF (squared-exponential) with per-axis
  length scales, signal variance, and a small noise/jitter floor.
  Hyperparameters are exposed in the config; a future version can add
  marginal-likelihood maximization.
- **Posterior**: standard formulation. Cholesky factorizes K (using
  the new internal helper); mean and variance predictions follow.
- **Acquisition**: Expected Improvement against the best observed
  feasible point. Optimized by best-of-N random sampling — simple,
  predictable cost, no inner-optimizer footgun.
- **Initial design**: `initial_samples` uniform-random points in
  bounds before the BO loop starts.
- **Constraints**: feasibility-aware EI — best observed value uses
  only feasible points; infeasible candidates are penalized.

Vec<f64> decisions, single-objective only. Targets the regime no
existing heuropt algorithm covers: 50–500 evaluations on an
expensive black-box function (CFD sim, ML training run, real-world
measurement).

Tests cover convergence on the 1-D sphere within a tight evaluation
budget (~30 evals get to f < 1e-6 — vs population-based methods
needing thousands), deterministic reruns, and panic on
multi-objective + dim mismatches.
 2026-05-05 09:51:12 -06:00
swaits 284f1143de feat(internal): add Cholesky factorization helper for SPD matrices 
Hand-rolled `A = L · L^T` factorization plus forward/backward triangular
solves, used by the upcoming Bayesian Optimization implementation for
the GP posterior. Same f64 row-major Vec<Vec<f64>> interface as the
existing Jacobi eigen helper so we don't pull in nalgebra for one
algorithm.

Returns Err on non-positive-definite input (a small jitter is the
typical caller-side fix). Tested against the standard 2x2 case, the
3x3 known-result case, A·x = b round-trip, and the SPD-failure case.
 2026-05-05 09:51:12 -06:00
swaits 60b17f58c9 feat(algorithms): add IpopCmaEs (CMA-ES with restart) for multimodal problems 
Auger & Hansen 2005 IPOP-CMA-ES: wraps the existing CmaEs in a restart
loop that doubles the population size and re-randomizes the mean
whenever a restart trigger fires. Specifically addresses the failure
mode we observed on Rastrigin (vanilla CMA-ES = 2.3 vs DE = 0).

Restart triggers:
- The whole budget for one inner CmaEs run finishes without improvement
- (More sophisticated triggers — eigenvalue collapse, condition-number
  blow-up, sigma stagnation — are left for future versions; the
  per-run budget trigger captures the bulk of the practical benefit)

Each restart:
- Doubles the population_size (Auger & Hansen 2005)
- Re-randomizes the initial mean to a fresh point in the bounds box
- Resets sigma to the user's initial value

Same Vec<f64> + single-objective constraints as CmaEs. The total
budget is divided across restarts; restart budget grows with
population. Tests verify it beats vanilla CMA-ES on Rastrigin.
 2026-05-05 09:51:12 -06:00
swaits b78e5ed2fc feat(algorithms): add NelderMead simplex direct-search optimizer 
Nelder & Mead 1965: gradient-free local optimizer that maintains a
simplex of n+1 points in n-D and at each iteration replaces the worst
vertex by one of {reflect, expand, outside-contract, inside-contract,
shrink} relative to the centroid of the rest. The five standard
coefficients (reflection α=1, expansion γ=2, contraction ρ=0.5,
shrinkage σ=0.5) are exposed in the config but default to canonical
values so users can leave them alone.

Single-objective only, Vec<f64> only, bounds enforced by clamping
each new vertex. Termination is purely iteration-count for v0.2;
"vertices have collapsed" stopping is a future enhancement.

Filling a real gap: heuropt had population-based local search
(SimulatedAnnealing, HillClimber) but no classical direct-search
algorithm. Excellent for low-dim smooth-ish problems where a
population is overkill.
 2026-05-05 09:51:12 -06:00
swaits 7d8a29df2b feat(algorithms): add OnePlusOneEs (1+1)-ES with Rechenberg's one-fifth rule 
Rechenberg 1973's elemental evolution strategy: one parent, one child
each generation, accept the child if it is no worse, and adapt the
mutation step size by tracking the success rate. If more than 1/5 of
recent moves were accepted the search is too cautious — multiply σ by
`step_increase` (typical 1.22). Below 1/5 — divide by the same factor.
At 1/5 — leave it alone. The success window has length `adaptation_period`.

Single-objective only. Vec<f64> only. Generic Gaussian step bounded by
the embedded `RealBounds`.

Why ship it: it's the smallest possible self-adapting evolution strategy
and a useful pedagogical / baseline endpoint. Pairs well as the budget
floor ("give me anything cheaper than CMA-ES").
 2026-05-05 09:51:12 -06:00
swaits 5a0475c678 chore(release): bump to v0.2.0 and update CHANGELOG 
Substantial v0.2.0 release on top of v0.1.0:

**21 new algorithms:**
- Single-objective: HillClimber, SimulatedAnnealing, GeneticAlgorithm,
  ParticleSwarm, CmaEs, TabuSearch, AntColonyTsp, Umda, Tlbo
- Multi-objective: Mopso, Ibea, SmsEmoa, Hype, Rvea, PesaII,
  EpsilonMoea, AgeMoea, Grea, Knea

**5 new operators:**
BoundedGaussianMutation, SimulatedBinaryCrossover (SBX),
PolynomialMutation, CompositeVariation, LevyMutation

**New utility:** `hypervolume_nd` (HSO algorithm) for arbitrary
dimensionality

**New examples:** `compare` (multi-seed harness across 7 benchmark
problems and 19 algorithms), `benchmarks` (canonical reference runs),
`jiggly_tuning` (real-world 4-objective firmware tuning)

**New feature flag:** `parallel` (rayon-backed population evaluation)

**README:** added an explanatory algorithm-selection decision tree

No breaking changes to v0.1.0 public API.
 2026-05-05 09:51:12 -06:00
swaits 26385fdb43 feat(examples): add ZDT3, DTLZ1, Rosenbrock, Ackley benchmark problems 
Expands the comparison harness with four new test problems chosen for
their distinct geometry:

- **Rosenbrock** (single-obj, smooth valley): the classic non-convex
  smooth function. Differentiates CMA-ES (which exploits the local
  metric) from Rastrigin's multimodal-trap regime.
- **Ackley** (single-obj, exponential multimodal trap): a more
  forgiving multimodal test than Rastrigin — fewer narrow local
  minima — so CMA-ES can show its strength while DE/GA still win.
- **ZDT3** (multi-obj, disconnected front): the only ZDT-family
  problem with a non-contiguous Pareto front. Tests an algorithm's
  ability to maintain spread across gaps.
- **DTLZ1** (many-obj, 3-D linear front): a triangular plane in
  objective space (vs DTLZ2's spherical octant). Different shape
  reveals which many-obj algorithms are biased toward sphere-like
  fronts vs which infer geometry adaptively.

Each new section runs all applicable algorithms × N seeds × the
algorithm-class budget the existing sections already use.
 2026-05-05 09:51:12 -06:00
swaits f0faf93b87 feat(algorithms): add KnEA (Knee point-driven EA) 
Zhang, Tian & Jin 2015 KnEA: many-objective MOEA that biases survival
selection toward 'knee points' on the Pareto front — points where a
small improvement in one objective costs a large degradation in
another.

Each generation:
- NSGA-II-like loop with offspring + non_dominated_sort
- For the splitting front, identify knee points by perpendicular
  distance from the hyperplane connecting the front's extreme points.
  Members further from the hyperplane (= more 'kneeness') are preferred.
- Survival keeps every knee-tagged member; if room remains, fill from
  remaining members by largest perpendicular distance.

Knee points are intuitively the most attractive points on a Pareto
front when no preference information is available. KnEA pushes the
search toward them at the cost of less uniform front coverage.
 2026-05-05 09:51:12 -06:00
swaits a95380376e feat(algorithms): add Grea (Grid-based Evolutionary Algorithm) 
Yang, Li, Liu & Zheng 2013 GrEA: many-objective MOEA whose secondary
ranking is a grid-based diversity score instead of crowding distance
or reference vectors.

Each generation:
- NSGA-II-like loop with offspring + non_dominated_sort
- For the splitting front:
  - Translate by ideal/nadir; partition objective space into a
    (`grid_divisions` per axis) grid
  - For every member compute three grid scores:
    - GR (grid rank)         = sum of grid coordinates (closer to ideal = lower)
    - GCD (grid crowding distance) = #neighbors within 1 grid unit (in any axis)
    - GCPD (grid coordinate point distance) = max coord - min coord
  - Sort F_l ascending by GR, then by GCD, then by GCPD
  - Take the top `n - already_selected` survivors

GrEA's grid-based niching is a different lens from NSGA-III's reference
points and RVEA's reference vectors — particularly effective on
non-convex fronts where reference-vector approaches struggle.
 2026-05-05 09:51:12 -06:00
swaits 6bfa52c149 feat(algorithms): add AgeMoea (Adaptive Geometry Estimation MOEA) 
Panichella 2019 AGE-MOEA: a many-objective MOEA that *infers* the
front's geometry (its L_p shape, where p = 1 is linear, p = 2 is
spherical, p < 1 is convex etc.) from the current non-dominated set
and uses that estimate to drive both proximity and diversity in
survival selection.

Each generation:
- NSGA-II-like loop: random parent selection + variation + evaluation
- Combine + non_dominated_sort
- Fill front-by-front; for the splitting front:
  - Translate by ideal point z*
  - Find extreme points by ASF (same as NSGA-III) and intercepts
  - Estimate the geometry parameter p by minimizing
    \|f − ideal\|_p constancy on the extreme points
  - Score every member by survival_score = (proximity_to_ideal) +
    (1 / nearest-neighbor distance in the same L_p frame)
  - Keep the top scorers

The geometry estimation is the novel contribution; with 3+ objectives
it produces fronts whose spread better matches the true shape than
NSGA-III's reference points (which assume a known geometry).
 2026-05-05 09:51:12 -06:00
swaits 9a336da43e feat(algorithms): add Tlbo (Teaching-Learning-Based Optimization) 
Rao 2011 TLBO: parameter-free single-objective optimizer for Vec<f64>.
The selling point — uniquely among the metaheuristics we ship — is that
it has NO algorithm-specific hyperparameters: no F, CR, w, c1, c2, σ,
mutation rate, etc. Just population_size and generations.

Each generation has two phases:
- **Teacher phase**: identify the best individual (the 'teacher'). For
  every learner, compute a 'mean' learner and try replacing it with a
  candidate moved toward the teacher by a random fraction, scaled by
  the gap between teacher and (TF · mean), where TF ∈ {1, 2}.
- **Learner phase**: each learner picks a random partner and tries
  moving toward the better one of the pair. Only successful moves are
  kept.

Single-objective only, Vec<f64> only, bounds enforced via clamping.
Tests cover Sphere1D convergence, deterministic reruns, and panic on
multi-objective.
 2026-05-05 09:51:11 -06:00
swaits 1b8070476b feat(operators): add LevyMutation real-valued heavy-tailed mutation 
Lévy-flight perturbation: each variable receives a step drawn from a
heavy-tailed Lévy(α) distribution rather than a Normal. The result is
"mostly small steps with rare big jumps," which gives a more
exploratory mutation than Gaussian without abandoning local search.

Decision type: Vec<f64>, with optional bounds (clamped per-axis if
`bounds` is non-empty). The step is sampled via Mantegna's algorithm
which generates Lévy(α) by combining two Normal samples and taking
the right power, controlled by the tail exponent `alpha` (typical
1.5; 1 is heavy, 2 collapses to Normal).

This is the only genuinely-different mutation kernel from Cuckoo
Search and other Lévy-flight metaheuristics; ship it as a Variation
operator usable from any algorithm rather than as a separate
algorithm.
 2026-05-05 09:51:11 -06:00
swaits 3400124541 feat(examples): wire SMS-EMOA, HypE, RVEA, PESA-II, ε-MOEA into compare harness 
Adds runners for the five new MO algorithms in both the ZDT1 (2-obj)
and DTLZ2 (3-obj) sections of `examples/compare.rs`. The harness now
side-by-sides 11 multi-/many-objective optimizers (RandomSearch + 10
real ones) on each problem.
 2026-05-05 09:51:11 -06:00
swaits 4fa8250c24 feat(algorithms): add EpsilonMoea (ε-dominance MOEA, Deb et al. 2003) 
Replaces strict Pareto dominance with ε-dominance: A ε-dominates B when
`floor(A_i / ε) ≤ floor(B_i / ε)` for every objective and strictly
less in at least one (minimization frame). The result is a regular
discretization of objective space — at most one archive member per
ε-box — so the front spreads out automatically and the archive size
self-limits without truncation tricks.

Steady-state design: each generation samples one parent from the main
population and one from the ε-archive, applies variation, evaluates
the child, and offers it to both archives. Every member's
ε-coordinates and the box-tie rules are precomputed each insertion.

Tests: produces a front on Schaffer N.1 with reasonable spread,
deterministic reruns, panic on `epsilon[i] <= 0.0` and on
`epsilon.len() != objectives.len()`.
 2026-05-05 09:51:11 -06:00
swaits f8fd3880ac feat(algorithms): add PesaII (Pareto Envelope-based Selection Algorithm II) 
Corne, Jerram, Knowles & Oates 2001: divides objective space into a
hyperbox grid and uses per-box population counts to drive selection
toward sparsely-populated regions.

Each generation:
- Maintain an external archive of non-dominated members
- Build a hyperbox grid (`grid_divisions` per axis on the archive's
  current axis ranges); count members per box
- Selection picks two parents by region-based tournament: choose two
  random non-empty boxes and take a uniform-random member from the
  one with fewer occupants
- Variation produces an offspring; insert into archive, dropping
  dominated members and (if archive overflows) the most-crowded
  occupant of the most-occupied box

Tests cover non-empty front on Schaffer N.1, deterministic reruns,
and panic on `archive_size == 0`.
 2026-05-05 09:51:11 -06:00
swaits 283d7429bb feat(algorithms): add Rvea (Reference Vector-guided EA) 
Cheng, Jin, Olhofer & Sendhoff 2016 RVEA: many-objective MOEA built
around a fixed set of Das–Dennis reference vectors. Each generation:
- Generate offspring via random parent selection + variation +
  evaluation
- Combine population + offspring; translate by ideal point z*
- Associate every member with the reference vector whose angle to
  the translated objective vector is smallest
- For each occupied vector, keep the member with the smallest
  Angle-Penalized Distance (APD) score; the rest are dropped
- APD = (1 + α(t)·θ_max·γ) · |f − z*| where γ is the angle to the
  associated reference and α(t) = (t / t_max)^2 anneals the angle
  penalty over the run

This produces well-spread fronts at high objective counts where
Pareto-rank methods (NSGA-II, SPEA2) lose discrimination.
 2026-05-05 09:51:11 -06:00
swaits d8d580e414 feat(algorithms): add HypE (Hypervolume Estimation) 
Bader & Zitzler 2011: HypE estimates hypervolume contributions via
Monte Carlo sampling instead of computing them exactly. The point of
the trick is that exact hypervolume becomes prohibitively expensive
beyond ~5 objectives, while MC sampling stays cheap and accurate
enough at any dimension.

Each generation:
- Generate offspring via parent selection + variation + evaluation
- Combine, run non_dominated_sort, fill front-by-front
- For the splitting front, estimate each member's HV contribution
  by drawing `n_samples` uniform points in the box [ideal, reference]
  and counting how many points are dominated by *exactly* one front
  member — that count, divided by n_samples and multiplied by the
  box volume, is the member's expected unique HV contribution.
- Drop members one at a time from the splitting front by smallest
  estimated contribution.

Public API matches the rest of the MO algorithms (Config + Optimizer).
The reference point is supplied in the config so the user controls
the integration domain. Tests cover non-empty front, deterministic
reruns, and panic on dim-mismatched reference.
 2026-05-05 09:51:11 -06:00
swaits cfc241980c feat(algorithms): add SmsEmoa (S-Metric Selection EMOA) 
Beume, Naujoks & Emmerich 2007: a steady-state MOEA that uses
hypervolume contribution as the secondary survival selection criterion.

Each generation:
- Generate ONE child via parent selection + variation + evaluation.
- Combine population + child, run non_dominated_sort.
- The discarded individual is the worst-front member with the
  smallest hypervolume contribution (computed via the new
  hypervolume_nd_from_evaluations helper).

Selection-quality is excellent at moderate objective counts (2–4) at
the cost of higher per-step compute (each survival selection requires
N+1 hypervolume evaluations of size ≤ N each). Best paired with a
tightly-bounded objective space — the user supplies a fixed reference
point in the config.

Tests: produces a non-empty front on Schaffer N.1, deterministic
reruns, panic on `population_size == 0`, panic on
`reference_point.len() != objectives.len()`.
 2026-05-05 09:51:11 -06:00
swaits e2d8b4e4c2 feat(metrics): add hypervolume_nd via Hypervolume-by-Slicing-Objectives (HSO) 
Generalizes the existing 2-D hypervolume to arbitrary M ≥ 1 dimensions
using the standard recursive Hypervolume-by-Slicing-Objectives (HSO)
algorithm from While et al. 2006:

- For M = 1: return reference[0] - min(points[0])
- For M = 2: sort by axis 0, sweep accumulating rectangles (matches
  hypervolume_2d's existing exact behavior)
- For M ≥ 3: sort by the last axis, peel off slices of increasing
  thickness and recursively compute the (M−1)-dimensional HV of each
  slice's projected non-dominated subset

Direction-aware: minimization-oriented input is the entry point, so
maximize objectives are negated by the caller via
`ObjectiveSpace::as_minimization` before the recursion runs.

Tested against:
- the existing 2-D analytical case (3 points → area 6)
- a known 3-D unit-cube case (1 point at origin, ref [1,1,1] → 1)
- empty front → 0
- agreement with hypervolume_2d on random 2-D fronts
 2026-05-05 09:51:11 -06:00
swaits 6c2b989c4a docs(readme): add explanatory algorithm-selection decision tree 
A substantial README section walking newcomers through choosing an
optimizer. Defines the terminology as it comes up — single- vs multi-
vs many-objective, Pareto front, dominance, multimodality, evaluation
cost — so a reader who has never touched heuristic optimization can
still pick a sensible starting algorithm.

Five-step decision flow:
1. What is the decision?
2. How many objectives?
3. What's the landscape like? (multimodal, smooth, discrete)
4. How expensive is each evaluation?
5. Are there constraints?

Each branch ends with 1–3 algorithm recommendations and a one-line
rationale, plus a compact "quick reference" table at the bottom for
returning users.
 2026-05-05 09:51:11 -06:00
swaits 7f67e58b27 feat(examples): wire new SO algorithms into the compare harness 
Adds runners for HillClimber, SimulatedAnnealing, GeneticAlgorithm,
ParticleSwarm, CmaEs, and Umda to `examples/compare.rs`. Rastrigin
section now compares 8 single-objective optimizers against each other
on a fixed evaluation budget.

The MO sections (ZDT1, DTLZ2) are unchanged for now — MOPSO and IBEA
get added in a follow-up commit so each algorithm's debut shows up
clearly in the harness.
 2026-05-05 09:51:11 -06:00
swaits 8c4b8013b8 feat(algorithms): add Umda Univariate Marginal Distribution EDA for binary problems 
Mühlenbein 1997 UMDA: simplest Estimation-of-Distribution Algorithm for
`Vec<bool>` problems. Each generation:
- Evaluate the current population
- Select the top μ members by fitness
- Estimate per-bit marginal probability p_i = (count of 1s at bit i in
  the μ-best) / μ
- Sample population_size new individuals from the resulting product-of-
  Bernoullis distribution

Single-objective only. Bit-wise probabilities are clamped to
`[1 / (2 · μ), 1 - 1 / (2 · μ)]` to keep the population from collapsing
to a deterministic single string before convergence is meaningful
(standard Laplace-style smoothing for UMDA).

Tests: solves OneMax (maximize Σ bits) on a 20-bit instance,
deterministic reruns, panic on multi-objective.
 2026-05-05 09:51:11 -06:00
swaits 974011796e feat(algorithms): add AntColonyTsp ant colony optimization for TSP-style permutations 
Dorigo-style Ant System for permutation problems on a complete graph:
each generation, every ant constructs a tour by probabilistically
picking the next node from those it has not yet visited, weighted by
`τ_ij^α · η_ij^β` where τ is the pheromone level on edge (i, j) and
η is the heuristic desirability (1 / distance, here). After all ants
finish, pheromone evaporates by a factor `(1 - ρ)` and is reinforced
on each ant's tour proportional to that tour's quality.

Decision type is `Vec<usize>` (a permutation of 0..n_cities). The user
supplies a distance matrix and the n_cities is inferred. Single-objective
only (the cost is total tour length, which the Problem evaluates).

Tests build a 5-city ring and verify ACO finds a near-optimal tour,
plus deterministic reruns and panic on multi-objective.
 2026-05-05 09:51:11 -06:00
swaits 7213bdd148 feat(algorithms): add IBEA (Indicator-Based Evolutionary Algorithm) 
Zitzler & Künzli 2004 IBEA: replaces Pareto-rank + crowding fitness
with a single scalar fitness derived from a binary quality indicator
(here, the additive ε-indicator). Loses no information at three or
more objectives the way crowding distance does.

Algorithm:
- For every (i, j) pair compute I(i, j) = max_k (f_k(i) - f_k(j)) on
  minimization-oriented objectives.
- Fitness F(i) = -Σ_{j≠i} exp(-I(j, i) / κ).
- Each generation: combine parents + offspring, iteratively remove the
  lowest-F member (cleanly recomputing the contribution of the dropped
  member from each surviving member's fitness) until population_size
  remain.
- Parent selection: binary tournament on F (higher wins).

Bounds-aware operators recommended (SBX + PolyMut).
Tests: produces a non-empty front on Schaffer N.1, deterministic
reruns, panic on `population_size == 0`.
 2026-05-05 09:51:11 -06:00
swaits d16e0379a3 feat(algorithms): add MOPSO (Multi-Objective Particle Swarm) 
Coello, Pulido & Lechuga 2004 MOPSO: PSO adapted for multi-objective
optimization via an external Pareto archive used as the source of
swarm leaders.

Each generation:
- Evaluate every particle's current position
- Insert non-dominated members into the archive (using ParetoArchive)
- For each particle, pick a leader from the archive (uniform random
  among archive members)
- Update velocity using inertia + cognitive (toward pbest) + social
  (toward leader)
- Update positions, clamp to bounds
- Refresh personal bests using Pareto comparison: pbest is replaced
  only when the new position dominates it; on non-dominated, keep
  with 50/50 random tiebreak

Vec<f64> decisions only. Truncates the archive to `archive_size` via
the existing simple-tail truncation. Tests: produces a non-empty
front on Schaffer N.1, deterministic reruns, panic on
single-objective.
 2026-05-05 09:51:11 -06:00
swaits c04420851e feat(algorithms): add CMA-ES (Covariance Matrix Adaptation Evolution Strategy) 
Hansen & Ostermeier 2001 CMA-ES, the canonical real-valued
single-objective stochastic optimizer. Implements the full (μ/μ_w, λ)
update with rank-μ + rank-1 covariance updates and cumulative step-size
adaptation:

- Sample λ offspring from N(mean, σ² · C)
- Select the μ best, weight them, recompute mean
- Update evolution paths p_σ (step size) and p_c (covariance)
- Rank-1 update of C from p_c, plus rank-μ update from selected offspring
- Adapt σ via |p_σ| / E‖N(0,I)‖

Eigendecomposition (used to convert C into its B·D form for sampling
N(0, σ²·C)) goes through the new internal Jacobi helper, recomputed
every `eigen_decomposition_period` generations to amortize cost.

Vec<f64> decisions only. Bounds taken from a `RealBounds` field; mean
and offspring are clamped per dimension. Single-objective only.

Hyperparameters use the standard CMA-ES defaults (μ=λ/2, weights from
Hansen's tutorial, c_σ, c_c, c_1, c_μ, d_σ all formulae from §7.1).

Tests cover: convergence on Sphere1D and 5-D Rosenbrock, deterministic
reruns, panic on multi-objective, panic on `population_size < 4`.
 2026-05-05 09:51:11 -06:00
swaits 325c8cdd37 feat(internal): add Jacobi symmetric-eigendecomposition helper 
Hand-rolled symmetric-matrix eigendecomposition via the cyclic Jacobi
rotation method. Returns sorted (eigenvalue, eigenvector) pairs in
descending order. Pure f64 row-major `Vec<Vec<f64>>` interface so we
don't pull in nalgebra for one algorithm.

Lives in `src/internal/eigen.rs` (new module). Used by the upcoming
CMA-ES implementation to maintain the covariance matrix's
eigendecomposition each generation. Tested against the standard
2x2 case, the diagonal case, and a known 3x3 result.
 2026-05-05 09:51:11 -06:00
swaits d82fcfc658 feat(algorithms): add TabuSearch with a configurable neighbor generator 
Glover 1986 tabu search for single-objective problems. Generic over
decision type — the user supplies a neighbor-generator closure that
produces a finite list of candidate moves from the current incumbent
(e.g. all 2-swaps for a permutation, or N Gaussian-perturbed copies of
a real vector). Each iteration picks the best non-tabu neighbor (with
an aspiration override that lets a tabu move through if it beats the
best-seen-ever incumbent) and adds the chosen move's decision to a
fixed-size FIFO tabu list.

Single-objective only. Tracks the best-seen-ever incumbent across the
run, returned as the result. Generic over the decision `D: Hash + Eq`
so the tabu list can match by full decision (simple and correct;
move-based tabu is left for users to implement themselves via a
custom decision wrapper).
 2026-05-05 09:51:11 -06:00
swaits ba07361439 feat(algorithms): add ParticleSwarm (canonical PSO) for Vec<f64> 
Eberhart & Kennedy 1995 PSO with the standard inertia-weight update:

  v[i,t+1] = w·v[i,t] + c1·r1·(pbest[i] - x[i,t]) + c2·r2·(gbest - x[i,t])
  x[i,t+1] = clamp(x[i,t] + v[i,t+1], bounds)

Single-objective only, `Vec<f64>` decisions only (PSO's velocity vector
needs a Euclidean structure that doesn't generalize cleanly to bool/perm).
Velocities are clamped to ±(hi - lo) per dim to keep particles from
exploding off into space.

Config exposes the four standard knobs — swarm size, generations,
inertia w, cognitive c1, social c2 — plus a seed. Tests cover
convergence on Sphere1D, deterministic reruns, and panic on
multi-objective.
 2026-05-05 09:51:11 -06:00
swaits f77e163ac4 feat(algorithms): add GeneticAlgorithm — single-objective generational GA 
Canonical generational GA with elitism: each generation runs binary
tournament selection (using `tournament_select_single_objective`) on
the current population, applies the variation operator pair-wise to
produce offspring, evaluates them, then replaces the population while
preserving the top `elitism` members from the previous generation
(elitism prevents fitness regression on a single seed).

Single-objective only. Generic over decision type — pair with
`SimulatedBinaryCrossover + PolynomialMutation` for real-valued,
single-point crossover + bit-flip for binary, etc.

Tests: convergence on Sphere1D, deterministic reruns, panic on
multi-objective, panic on `population_size < 2`, panic on
`elitism > population_size`.
 2026-05-05 09:51:11 -06:00
swaits 35fbf622f2 feat(algorithms): add SimulatedAnnealing single-objective local search 
Classic Kirkpatrick et al. 1983 SA: hill climber that also accepts
worse moves with probability `exp(-Δ/T)` where T anneals geometrically
from `initial_temperature` to `final_temperature` over the iteration
count.

Single-objective only. Generic over decision type — works on real
vectors, bool vectors, permutations, anything. Tracks the best-seen
incumbent across the run (not just the last accepted move) so the
result reflects the actual best ever visited, not where the random
walk happened to end.

Tests cover: convergence on Sphere1D under reasonable hyperparameters,
deterministic reruns, panic on multi-objective, panic on
non-positive temperatures.
 2026-05-05 09:51:10 -06:00
swaits a93d0df858 feat(algorithms): add HillClimber single-objective greedy local search 
The simplest possible local search: start from one initializer-sampled
decision, repeatedly mutate it via the variation operator, and keep the
child only when it is strictly better than the current incumbent (with
the standard feasible-beats-infeasible / lower-violation tiebreaks
when relevant).

Single-objective only — panics with a clear message if the problem
exposes more than one objective. Deterministic under a seed. Returns
a population/front of size one (the current incumbent) so it slots
into the comparison harness like any other optimizer.
 2026-05-05 09:51:10 -06:00
swaits cf3b6acd10 fix(examples): jiggly mean_presses now counts every daily press 
The Python tune_runtime.py treats the morning boot press and the 13:00
post-lunch re-tap as 'free' and only counts extra warning-phase taps.
That undercounts what the user actually presses each day and breaks
any comparison against a stated 'presses/day' comfort cap.

Updated `simulate_one` to count every press the user makes:
- boot press at workday start (always +1)
- 13:00 re-login press when the workday continues past lunch (+1)
- per-minute Bernoulli warning-phase presses (already counted)
- death-restart press: each time the device transitions running→dead
  during workday and the user is at-desk (not at lunch), the user
  presses to restart the cycle (warning press and death-restart for
  the same cycle are mutually exclusive — extending via warning press
  prevents that cycle's death)

With baseline now ~2 presses/day already mandatory, the hinge/cap
shift up too: PRESS_HINGE_LOW = 2.5/d (full reward up to baseline +
half a warning press) and PRESS_COMFORT_CAP = 3.5/d (rejected above).

Output 'Why' bullet now reports the total directly and notes the
component breakdown so the number is interpretable against the
new thresholds.
 2026-05-05 09:51:10 -06:00
swaits e42514087c feat(examples): personalize jiggly weights with hinge press term and balance bonus 
Tweak the a-posteriori scoring to match the user's stated preferences:

- Reweight: lunch_sleep 30%, after_hours 25%, work_fail 20%,
  presses 15% (with hinge below), balance 10%.
- Press term is now a hinge instead of a normalized minimize:
    * <= 2 presses/day → score 1.0 (no penalty)
    * 2 → 3 presses/day → linear ramp from 1.0 to 0.0
    * > 3 presses/day → -inf (excluded; comfort cap)
- New balance term: bonus for longer warning phases. Computed as
  min(YA - RA, RA - FRA), saturated at 10 minutes. So a 5/5/X split
  scores 0.5, an 8/8/X split scores 0.8, and 10/10/X or wider saturates
  at 1.0.

Constants moved to module scope so the printout in main and the
scoring function stay in sync.
 2026-05-05 09:51:10 -06:00
91 changed files with 15883 additions and 311 deletions
Expand all files Collapse all files
.cargo/mutants.toml
+33
View File
@@ -0,0 +1,33 @@
# cargo-mutants configuration for heuropt.
#
# Run with:
# cargo install cargo-mutants
# cargo mutants # full sweep (slow)
# cargo mutants --in-diff HEAD~1 # only mutate recently-changed lines
#
# A *surviving* mutation = the test suite passed despite a code change,
# which usually means a missing test or a missing invariant.
#
# This isn't gated CI; it's an advisory tool. The property tests in
# tests/properties.rs are the natural place to land new invariants
# discovered via mutation runs.
# Files to skip mutating. We skip:
# - examples (illustrative, not core algorithm correctness)
# - benches (microbench harness, not behavior)
# - the docs/* spec markdown
# - tests_support (test helpers; mutating them changes test inputs,
# not behavior under test)
exclude_globs = [
"examples/**/*.rs",
"benches/**/*.rs",
"src/tests_support/**/*.rs",
]
# `cargo-mutants` defaults to `cargo test` for the suite. Keep that.
# `--no-shuffle` makes failure attribution deterministic.
additional_cargo_test_args = ["--", "--test-threads=1"]
# Time-out per mutated build+test cycle. Big enough for a slow test
# (proptest can take ~10s) but short enough to detect infinite loops.
timeout_multiplier = 5.0
.claude/scheduled_tasks.lock
+1
View File
@@ -0,0 +1 @@
{"sessionId":"ac44d107-52ca-4cd4-9586-ae2fe91bc9f7","pid":2366937,"procStart":"77336928","acquiredAt":1778002505967}
.github/workflows/ci.yml
+109
View File
@@ -0,0 +1,109 @@
name: CI
on:
push:
branches: [main]
pull_request:
branches: [main]
env:
CARGO_TERM_COLOR: always
RUSTFLAGS: "-D warnings"
jobs:
fmt:
name: rustfmt
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
with:
components: rustfmt
- run: cargo fmt --all -- --check
clippy:
name: clippy --all-features
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
with:
components: clippy
- uses: Swatinem/rust-cache@v2
- run: cargo clippy --all-targets --all-features -- -D warnings
test:
name: test (${{ matrix.features }})
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
features:
- "default"
- "serde"
- "parallel"
- "serde,parallel"
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
- uses: Swatinem/rust-cache@v2
- name: Run unit + integration + property tests
run: |
if [ "${{ matrix.features }}" = "default" ]; then
cargo test
else
cargo test --features ${{ matrix.features }}
fi
- name: Run doctests
run: |
if [ "${{ matrix.features }}" = "default" ]; then
cargo test --doc
else
cargo test --doc --features ${{ matrix.features }}
fi
doc:
name: cargo doc
runs-on: ubuntu-latest
env:
RUSTDOCFLAGS: "-D warnings"
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
- uses: Swatinem/rust-cache@v2
- run: cargo doc --no-deps --all-features
msrv:
name: minimum supported Rust version (1.85)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@1.85
- uses: Swatinem/rust-cache@v2
- run: cargo build --all-features
fuzz:
name: fuzz smoke (${{ matrix.target }})
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
target:
- pareto_compare
- non_dominated_sort
- hypervolume_2d
- pareto_archive
- crowding_distance
- spacing
- sbx_polymut
- clamp_to_bounds
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@nightly
- uses: Swatinem/rust-cache@v2
with:
workspaces: fuzz -> target
- name: Install cargo-fuzz
run: cargo install cargo-fuzz --locked
- name: 60-second soak
run: cargo fuzz run ${{ matrix.target }} -- -max_total_time=60
CHANGELOG.md
+310 -1
View File
@@ -7,6 +7,315 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased] ## [Unreleased]
## [0.4.0] — 2026-05-05
Theme: testing infrastructure, two real bug fixes surfaced by that
infrastructure, and a CPU-time optimization pass that made the
comparison harness 3.27× faster end-to-end. No breaking changes to
the v0.3.0 public API.
### Performance
A focused, measure-and-iterate optimization pass on the Pareto-based
multi-objective hot paths. Every change verified bit-identical against
the v0.3.0 comparison-harness snapshot — quality metrics
(hypervolume, spacing, mean L2, mean dist, front size) match to the
last decimal in every benchmark.
**Cumulative wall-clock impact (compare harness, 10-seed mean):**
| Algorithm / Problem | v0.3.0 | v0.4.0 | Speedup |
|----------------------|--------:|--------:|--------:|
| AGE-MOEA / DTLZ1 | 2299 ms | 229 ms | 10× |
| SPEA2 / DTLZ2 | 4304 ms | 513 ms | 8.4× |
| AGE-MOEA / ZDT3 | 932 ms | 193 ms | 4.8× |
| NSGA-II / ZDT1 | 268 ms | 65 ms | 4.1× |
| NSGA-II / ZDT3 | 267 ms | 65 ms | 4.1× |
| SMS-EMOA / DTLZ2 | 5643 ms | 1369 ms | 4.1× |
| NSGA-II / Rastrigin | 260 ms | 71 ms | 3.7× |
| NSGA-II / DTLZ2 | 344 ms | 106 ms | 3.2× |
| NSGA-III / DTLZ2 | 318 ms | 122 ms | 2.6× |
| NSGA-III / DTLZ1 | 303 ms | 122 ms | 2.5× |
| HypE / DTLZ2 | 80 ms | 44 ms | 1.8× |
| **Total compare** | **18 629 ms** | **5688 ms** | **3.27×** |
**Hot-path instruction counts (gungraun):**
| Benchmark | v0.3.0 | v0.4.0 | Speedup |
|-------------------------|------------:|---------:|--------:|
| `hypervolume_nd_3d` n=100 | 13 523 760 | 367 767 | 37× |
| `hypervolume_nd_3d` n=30 | 676 902 | 70 334 | 9.6× |
| `non_dominated_sort_2d` n=200 | 13 513 271 | 2 601 813 | 5.2× |
| `non_dominated_sort_2d` n=50 | 852 317 | 198 574 | 4.3× |
| `spea2_short` | 179 113 | 133 783 | 1.34× |
**Changes (in commit order):**
- `perf(hypervolume)` — Rewrote the M≥3 HSO recursion in
`hypervolume_nd`. The original cloned the active set into a fresh
Vec<Vec<f64>> at the top of every recursive call, used a linear-scan
`position` lookup to remove the just-processed point each band, and
re-projected onto M-1 axes inside every band. Now: sort-by-index,
pre-project once, slice prefixes for the active set, and skip
`non_dominated_projection` when recursing into the M=2 base case
(whose sweep already filters dominated points internally).
- `perf(non_dominated_sort)` — Cache `as_minimization` /
feasibility / violation per individual once at the top of the
Deb fast-non-dominated-sort, then inline the dominance test against
those arrays. The naïve formulation called `pareto_compare` twice
per pair, each call allocating two fresh Vec<f64>s — 4N(N-1)
allocations per sort. Propagates to every Pareto-based MOEA.
- `perf(age_moea)` — Cache `lp_norm(translated[i], p)` once per
candidate at function entry; maintain a `nearest[]` array updated
incrementally on each pick (single `min` per remaining instead of
a fresh full scan over the keep list). Cuts the splitting-front
scoring loop from O(R · K · M) per iteration to O(R · M).
- `perf(spea2)` — Two wins. (1) `compute_fitness` (called twice per
generation): inline dominance against cached oriented arrays,
symmetric distance matrix built once. (2) `build_archive` truncation:
compute pairwise distances + sorted neighbor vectors once, then on
victim removal use binary-search-remove on every survivor's
still-sorted vector — total truncation cost O(K³ log K) → O(K² log K).
- `perf(hypervolume)` — Index-sort instead of cloning point vectors
in the M≥3 recursion. The N inner-Vec clones per HV call were
redundant once we'd already sorted by last-axis. Big bench win
(32×→37× cumulative on n=100/3D), modest wall-clock impact because
SMS-EMOA's worst-front HV calls operate on small fronts.
- `build(release)` — Enable thin LTO + codegen-units=1 in the
release profile. Worth ~150 ms across the harness; only applies
when heuropt is the workspace root, so downstream consumers see
whatever profile their own Cargo.toml configures.
- `perf(pareto_archive)` — Cache the candidate's oriented +
feasibility once per `insert`, build each member's oriented vector
once, and inline the two-pass dominance checks. Used by PESA-II
(most impact), PAES, ε-MOEA, and any user code working through the
archive directly.
### Added
- **Decision tree update** in README to cover all v0.3.0 algorithms,
with a new top-level branch on "is each evaluation expensive?" so
`BayesianOpt` / `Tpe` / `Hyperband` have a clear home.
- **Comparison results snapshot** at `examples/compare-results.md` —
reference output of the harness across 7 benchmark problems and ~20
algorithms, captured after v0.3.0 landed.
- **Instruction-count benchmarks** via `gungraun` (the Rust 2026
rename of `iai-callgrind`) at `benches/hot_paths.rs`. Covers
`non_dominated_sort`, `crowding_distance`, `hypervolume_2d`,
`hypervolume_nd` (HSO), and one-generation costs of NSGA-II and
CMA-ES, plus a short-run bench for every algorithm. Stable across
machines via callgrind.
- **Property-based test suite expansion**: `tests/properties.rs`
(Pareto-comparison antisymmetry, partitioning, operator bounds),
`tests/algorithm_properties.rs` (per-algorithm determinism +
population-size invariants — 32 tests, one per algorithm),
`tests/operator_properties.rs` (every `Variation` / `Initializer` /
`Repair` impl), `tests/metric_properties.rs` (HV / spacing
invariants), and `tests/numerical_stability.rs` (empty / singleton /
duplicate / flat-fitness / zero-width-bounds populations).
- **Coverage-guided fuzz harness** at `fuzz/` (cargo-fuzz +
libFuzzer). Eight targets covering `pareto_compare`,
`non_dominated_sort`, `hypervolume_2d`, `ParetoArchive`,
`crowding_distance`, `spacing`, SBX/PolyMut, and the `Repair`
operators. Runs in CI for a short soak per PR; longer runs locally
via `cargo +nightly fuzz run <target>`.
- **cargo-mutants config** at `.cargo/mutants.toml` for advisory
mutation testing. Not gated in CI; run with `cargo mutants` to
surface tests that don't actually check the behavior they look like
they do.
- **GitHub Actions CI** at `.github/workflows/ci.yml` with fmt /
clippy / test (4-feature matrix) / doc / MSRV / fuzz-smoke jobs,
all gated on `-D warnings`.
### Fixed
- `pareto::sort::non_dominated_sort` previously dropped indices when
the dominance graph contained a cycle (which arises when objectives
contain NaN — `pareto_compare` becomes intransitive). Fuzzing the
partition invariant surfaced the bug; orphans now go into a final
residual front.
- `operators::repair::ProjectToSimplex` could silently return the
all-zero vector when the input vector's magnitude dwarfed `total`
(the standard Duchi/Held-Wolfe τ computation lost precision and
τ ≈ max(x), so `max(x_i - τ, 0)` rounded to zero everywhere).
Detected by the `clamp_to_bounds` fuzzer; now falls through to a
degenerate "all mass on argmax" projection above a 1e15 magnitude
ratio, and is robust to floating-point precision loss in the
algorithm's inner loop.
[0.4.0]: https://github.com/swaits/heuropt/releases/tag/v0.4.0
## [0.3.0] — 2026-05-05
Theme: filling heuropt's expensive-evaluation, gradient-free, and
constraint-handling gaps. No breaking changes to the v0.2.0 public API.
### Added
#### New algorithms (9)
**Sample-efficient / surrogate-based:**
- `BayesianOpt` — Gaussian-process Bayesian Optimization with Expected
Improvement acquisition. heuropt's first sample-efficient algorithm:
targets the 50–500 evaluation regime.
- `Tpe` — Bergstra et al. 2011 Tree-structured Parzen Estimator
(workhorse of Hyperopt and Optuna). KDE-based surrogate; cheaper
per-step than BO and more robust without hyperparameter tuning.
**Classical and modern evolution strategies:**
- `OnePlusOneEs` — Rechenberg 1973 (1+1)-ES with the one-fifth success
rule. Smallest possible self-adapting evolution strategy.
- `IpopCmaEs` — Auger & Hansen 2005 increasing-population CMA-ES with
restart. Specifically fixes vanilla CMA-ES's known weakness on
multimodal problems.
- `SeparableNes` — Wierstra et al. 2008/2014 Natural Evolution Strategy
with diagonal covariance (sNES). Different theoretical foundation
than CMA-ES; cheaper per-step at the cost of being unable to model
rotated landscapes.
**Direct search:**
- `NelderMead` — Nelder & Mead 1965 simplex method. Classical gradient-
free local optimizer; superb on low-dim smooth problems
(Rosenbrock 5-D: f = 0 exactly).
**Multi-fidelity:**
- `Hyperband` — Li et al. 2017 multi-fidelity hyperparameter optimizer
built on Successive Halving. Operates on a new `PartialProblem`
trait so configurations can be evaluated at adjustable fidelity
budgets.
#### New operators
- `LevyMutation` — heavy-tailed Lévy-flight mutation via Mantegna's
algorithm. The actual algorithmic contribution from Cuckoo Search
packaged as a reusable `Variation` operator.
#### New traits + impls
- `PartialProblem` — multi-fidelity problem contract:
`evaluate_at_budget(decision, budget) -> Evaluation`. Used by
`Hyperband`. Intentionally not a sub-trait of `Problem`.
- `Repair<D>` — in-place projection trait for restoring decisions to
feasibility. Pair with `Variation` operators to get bounds-aware
variants. Provided impls:
- `ClampToBounds` for `Vec<f64>` per-axis clamping
- `ProjectToSimplex` for L1-budget / probability-simplex projection
#### New selection helpers
- `stochastic_ranking_select` — Runarsson & Yao 2000 stochastic
ranking. Better than strict feasibility-first tournament selection
on heavily-constrained problems.
#### Internal helpers
- `internal::cholesky` — Cholesky factorization + triangular solves
for SPD matrices, used by the GP posterior in `BayesianOpt`.
### Changed
- `CmaEsConfig` gained `initial_mean: Option<Vec<f64>>`. `None`
preserves the existing midpoint-of-bounds default; `IpopCmaEs` sets
it to inject restart diversity without shrinking the search box.
[0.3.0]: https://github.com/swaits/heuropt/releases/tag/v0.3.0
## [0.2.0] — 2026-05-05
A substantial expansion of the algorithm catalog (21 new algorithms),
five new operators, an n-D hypervolume utility, an algorithm-selection
guide in the README, and a multi-seed comparison harness covering seven
benchmark problems. No breaking changes to the v0.1.0 public API.
### Added
#### New algorithms
**Single-objective:**
- `HillClimber` — simplest greedy local search.
- `SimulatedAnnealing` — Kirkpatrick et al. 1983, generic over decision type.
- `GeneticAlgorithm` — generational SO GA with tournament selection + elitism.
- `ParticleSwarm` — Eberhart & Kennedy 1995 PSO for `Vec<f64>`.
- `CmaEs` — Hansen & Ostermeier 2001 covariance-matrix adaptation.
- `TabuSearch` — Glover 1986, with a user-supplied neighbor generator.
- `AntColonyTsp` — Dorigo Ant System for permutation problems.
- `Umda` — Mühlenbein 1997 univariate marginal-distribution EDA for
`Vec<bool>`.
- `Tlbo` — Rao 2011 Teaching-Learning-Based Optimization (parameter-free).
**Multi-objective:**
- `Mopso` — Coello, Pulido & Lechuga 2004 multi-objective PSO.
- `Ibea` — Zitzler & Künzli 2004 indicator-based EA.
- `SmsEmoa` — Beume, Naujoks & Emmerich 2007 S-metric selection EMOA.
- `Hype` — Bader & Zitzler 2011 Hypervolume Estimation Algorithm.
- `Rvea` — Cheng et al. 2016 Reference Vector-guided EA.
- `PesaII` — Corne et al. 2001 Pareto Envelope-based Selection II.
- `EpsilonMoea` — Deb, Mohan & Mishra 2003 ε-dominance MOEA.
- `AgeMoea` — Panichella 2019 Adaptive Geometry Estimation MOEA.
- `Grea` — Yang et al. 2013 Grid-based EA.
- `Knea` — Zhang, Tian & Jin 2015 Knee point-driven EA.
#### New operators
- `BoundedGaussianMutation` — Gaussian noise + per-axis clamping.
- `SimulatedBinaryCrossover` (SBX) — Deb & Agrawal 1995 canonical
real-valued crossover.
- `PolynomialMutation` — Deb's polynomial mutation, the standard NSGA-II
pair to SBX.
- `CompositeVariation` — pipeline two `Variation` operators
(typically crossover → mutation).
- `LevyMutation` — heavy-tailed Lévy-flight mutation via Mantegna's
algorithm.
#### New metrics / utilities
- `hypervolume_nd` — exact N-dimensional dominated hypervolume via the
Hypervolume-by-Slicing-Objectives (HSO) algorithm, plus an internal
Jacobi symmetric eigendecomposition helper used by CMA-ES.
#### New examples
- `compare` — multi-seed comparison harness running every applicable
algorithm across ZDT1, ZDT3, DTLZ1, DTLZ2 (multi/many-objective) and
Rastrigin, Rosenbrock, Ackley (single-objective). Reports
hypervolume, spacing, mean L2/dist, front size, and wall-clock ms.
- `benchmarks` — canonical reference runs of NSGA-II on ZDT1 and DE on
Rastrigin.
- `jiggly_tuning` — real-world 4-objective NSGA-III firmware tuning
for the [`jiggly`](https://github.com/swaits/jiggly) USB-mouse-jiggler,
with an a-posteriori weighted-decision step that picks one
recommendation off the Pareto front.
#### New optional feature
- `parallel` — rayon-backed parallel population evaluation in
`RandomSearch`, `Nsga2`, `DifferentialEvolution`, `Spea2`, `Ibea`,
`Mopso`, and most other algorithms with batchable inner loops.
Seeded runs stay bit-identical to serial mode.
#### Documentation
- README gained an explanatory algorithm-selection decision tree that
walks newcomers through choosing an optimizer, defining the
terminology (multi-objective, Pareto front, dominance, multimodality,
evaluation cost) as it goes.
### Changed
- Minimum supported Rust version remains 1.85 (edition 2024).
- Algorithm impls now require `P: Sync` and `P::Decision: Send` so the
same impl serves both `parallel` and serial feature builds. Any
`Problem` / decision type without exotic interior mutability already
satisfies these.
[0.2.0]: https://github.com/swaits/heuropt/releases/tag/v0.2.0
## [0.1.0] — 2026-05-04 ## [0.1.0] — 2026-05-04
Initial release. Initial release.
@@ -74,5 +383,5 @@ Initial release.
`RandomSearch`, `Nsga2`, and `DifferentialEvolution`. Seeded runs stay `RandomSearch`, `Nsga2`, and `DifferentialEvolution`. Seeded runs stay
bit-identical to serial mode. bit-identical to serial mode.
[Unreleased]: https://github.com/swaits/heuropt/compare/v0.1.0...HEAD [Unreleased]: https://github.com/swaits/heuropt/compare/v0.4.0...HEAD
[0.1.0]: https://github.com/swaits/heuropt/releases/tag/v0.1.0 [0.1.0]: https://github.com/swaits/heuropt/releases/tag/v0.1.0
Cargo.toml
+16 -1
View File
@@ -1,6 +1,6 @@
[package] [package]
name = "heuropt" name = "heuropt"
version = "0.1.0" version = "0.4.0"
edition = "2024" edition = "2024"
rust-version = "1.85" rust-version = "1.85"
authors = ["Stephen Waits <steve@waits.net>"] authors = ["Stephen Waits <steve@waits.net>"]
@@ -23,3 +23,18 @@ rand = "0.9"
rand_distr = "0.5" rand_distr = "0.5"
rayon = { version = "1", optional = true } rayon = { version = "1", optional = true }
serde = { version = "1", features = ["derive"], optional = true } serde = { version = "1", features = ["derive"], optional = true }
[dev-dependencies]
gungraun = "0.18"
proptest = "1"
[[bench]]
name = "hot_paths"
harness = false
# Tighten release codegen for the compare harness and downstream binaries
# that build heuropt directly (i.e. when this crate is the workspace root).
# When heuropt is used as a dependency the consumer's profile wins.
[profile.release]
lto = "thin"
codegen-units = 1
README.md
+376 -10
View File
@@ -17,13 +17,13 @@ framework concepts.
```toml ```toml
[dependencies] [dependencies]
heuropt = "0.1" heuropt = "0.3"
# Optional features: # Optional features:
# - "serde": derive Serialize/Deserialize on the core data types. # - "serde": derive Serialize/Deserialize on the core data types.
# - "parallel": evaluate populations across rayon's thread pool. # - "parallel": evaluate populations across rayon's thread pool.
# Seeded runs stay bit-identical to serial mode. # Seeded runs stay bit-identical to serial mode.
# heuropt = { version = "0.1", features = ["serde", "parallel"] } # heuropt = { version = "0.3", features = ["serde", "parallel"] }
``` ```
## Define a problem ## Define a problem
@@ -107,17 +107,361 @@ where
A complete worked example is in `examples/custom_optimizer.rs`. A complete worked example is in `examples/custom_optimizer.rs`.
## Choosing an algorithm
Optimization is a noisy field with a lot of jargon. This section walks you
through picking a starting algorithm for a real problem, defining the terms
as they come up. If you already know the vocabulary, jump to the
[quick-reference table](#quick-reference) at the bottom.
### Step 1: What is your problem?
Three ingredients describe any optimization problem:
- A **decision** — the thing the algorithm is allowed to change. Examples:
five real numbers (`Vec<f64>`), a yes/no flag for each of 100 features
(`Vec<bool>`), or an ordering of cities to visit (`Vec<usize>`).
- One or more **objectives** — numbers you want to make small (or large).
Examples: a model's prediction error, a tour's total length, a circuit's
power draw.
- An optional set of **constraints** — conditions a decision must satisfy
to be valid. Examples: "the budget cannot exceed $1M," or "every car
must be visited exactly once."
Your job is to express the problem; heuropt's job is to search for
decisions that score well on the objectives without violating the
constraints.
### Step 2: How many objectives?
The biggest fork in the road. Algorithms specialize sharply by
objective count:
- **Single-objective (1)** — one number to optimize. There's a clear
"best" answer. Examples: minimize loss, maximize throughput.
- **Multi-objective (2 or 3)** — several conflicting goals. There is no
single best; instead there is a **Pareto front**: the set of decisions
where you cannot improve any objective without sacrificing another.
Each point on the front is a different tradeoff.
- **Many-objective (4+)** — same idea, but classical multi-objective
algorithms break down because almost every pair of points is
*non-dominated* (neither one is strictly better) once you have lots
of objectives.
> **Dominance:** Decision A *dominates* decision B if A is at least as
> good as B on every objective and strictly better on at least one. The
> Pareto front is what you get after deleting every dominated decision.
If you found yourself staring at a single composite score that's a
weighted sum of conflicting goals, you probably actually have a
multi-objective problem in disguise.
### Step 3: What does the search space look like?
A few questions about the geometry of your problem:
- Is the **decision continuous** (real numbers), **discrete** (integers,
bits), or a **permutation** (an ordering)?
- Is the landscape **unimodal** (one hill, easy to climb) or
**multimodal** (lots of local optima that aren't the global one)?
Rastrigin and Ackley are classic multimodal traps.
- How **smooth** is it? Smooth landscapes (e.g., a quadratic bowl)
reward gradient-like methods (CMA-ES); jagged or noisy ones reward
population-based methods (DE, GA).
If you don't know, treat it as multimodal — it's the cautious default.
### Step 4: How expensive is each evaluation?
Cheap evaluations (a few microseconds — pure math, simple simulation)
let you afford 100k+ evaluations per run. Expensive evaluations (a
training run, a CFD simulation, a real-world measurement that costs
money) force you to be sample-efficient: 50–500 evaluations total.
This decides whether you can afford a **population-based** algorithm
that throws hundreds of evaluations at each generation, or whether
you need a **sample-efficient** or **multi-fidelity** approach:
- **Cheap (1k+ evals affordable):** any of the population-based
algorithms — DE, GA, CMA-ES, NSGA-II, etc.
- **Expensive (50–500 evals):** `BayesianOpt` (Gaussian-process
surrogate + Expected Improvement) or `Tpe` (Parzen-density
surrogate, cheaper per step, more robust without hyperparameter
tuning).
- **Multi-fidelity (each eval has a tunable budget — epochs, sim
steps, MC samples):** `Hyperband`. Implement the `PartialProblem`
trait on your problem and Hyperband allocates compute aggressively
across promising configs.
The `parallel` feature flag also matters here — if your `evaluate`
function takes more than ~50 µs, enabling rayon-backed parallel
population evaluation will speed runs up significantly.
### Step 5: Are there hard constraints?
heuropt models constraints as a single scalar **constraint violation**
on each `Evaluation`. The convention: `0.0` (or negative) means
feasible; positive means infeasible, and bigger numbers are worse
violations. Every Pareto-comparison and tournament-selection helper
in the crate prefers feasible candidates and breaks ties on
violation magnitude, so the rule "feasibility comes first" is
enforced automatically.
If your constraints are very tight and the search keeps hitting them,
you have three options:
- **Repair**: implement the `Repair<D>` trait (or use the provided
`ClampToBounds` / `ProjectToSimplex` impls) to in-place project
infeasible decisions back into the feasible region. Pair with a
`Variation` operator to get bounds-aware variants without writing a
custom `Variation` impl.
- **Stochastic ranking**: use `stochastic_ranking_select` instead of
`tournament_select_single_objective`. It probabilistically explores
near-feasibility instead of strict feasibility-first ordering, which
helps when feasible regions are narrow.
- **Penalty-only**: stick with `constraint_violation` — the simplest,
works well when the feasible region is large and convex.
---
### The decision tree
A flow you can run mentally:
```
START
│
├─ Is each evaluation EXPENSIVE (>1 sec) or BUDGETED (50–500 total)?
│ │
│ ├─ Yes → sample-efficient regime
│ │ ├─ Standard expensive black-box, single-objective
│ │ │ → BayesianOpt (GP + Expected Improvement; gold
│ │ │ standard *with* per-problem kernel
│ │ │ tuning. The default RBF kernel at
│ │ │ 60 evals is honestly bad — give it
│ │ │ more evals or tune the kernel.)
│ │ │ → Tpe (KDE-based; cheaper per-step,
│ │ │ more robust without tuning)
│ │ │
│ │ └─ Each eval has a tunable fidelity (epochs, sim steps, …)
│ │ → Hyperband (implement PartialProblem; allocates
│ │ compute across configs adaptively)
│ │
│ └─ No → continue to the population-based branches below
│
└─ How many objectives?
│
├─ 1 (single-objective)
│ │
│ ├─ Decision is Vec<f64> (continuous)
│ │ ├─ Smooth landscape (well-conditioned)
│ │ │ → CmaEs (full-cov adaptive Gaussian)
│ │ │ → SeparableNes (cheaper diag-cov; high-dim)
│ │ │ → NelderMead (low-dim, deterministic, simple)
│ │ ├─ Multimodal landscape
│ │ │ → IpopCmaEs (CMA-ES with restart;
│ │ │ fixes vanilla CMA-ES's
│ │ │ multimodal failure)
│ │ │ → DifferentialEvolution (rarely beaten on cheap
│ │ │ multimodal continuous)
│ │ │ → SimulatedAnnealing (cheap & generic)
│ │ ├─ Want parameter-free (no F, CR, w, σ to tune)
│ │ │ → Tlbo
│ │ ├─ Want minimum self-adapting baseline
│ │ │ → OnePlusOneEs (one-fifth rule,
│ │ │ smallest possible ES)
│ │ ├─ Just want a strong default for cheap continuous
│ │ │ → DifferentialEvolution
│ │ └─ Just want a baseline
│ │ → RandomSearch
│ │
│ ├─ Decision is Vec<bool> (binary)
│ │ ├─ Independent bits, smooth fitness
│ │ │ → Umda (per-bit marginal EDA)
│ │ └─ Bit interactions matter
│ │ → GeneticAlgorithm with BitFlipMutation +
│ │ a bit-string crossover
│ │
│ ├─ Decision is Vec<usize> (permutation, e.g., TSP)
│ │ → AntColonyTsp (with a distance matrix)
│ │ → TabuSearch (with your own neighbor function)
│ │ → SimulatedAnnealing with SwapMutation
│ │
│ └─ Custom decision type (a struct, a tree, …)
│ → SimulatedAnnealing or HillClimber
│ with your own Variation impl
│
├─ 2 or 3 (multi-objective)
│ │
│ ├─ Strong default, fast, well-understood
│ │ → Nsga2
│ │
│ ├─ Real-valued, smooth front, want best convergence
│ │ → Mopso (multi-objective PSO; on the benches
│ │ here it wins ZDT1 on both HV and
│ │ convergence by 100× over the
│ │ dominance-based methods)
│ │
│ ├─ Want better front quality than NSGA-II
│ │ → Ibea (indicator-based; consistently the best
│ │ of the dominance-based methods on these
│ │ benches — wins ZDT3 HV and DTLZ2 mean
│ │ dist by 24×)
│ │ → Spea2 (strength + density)
│ │ → SmsEmoa (hypervolume-contribution selection;
│ │ elegant in theory but underperforms
│ │ NSGA-II on these benches at our budgets —
│ │ only worth its higher per-step cost on
│ │ fronts where exact HV-contribution is
│ │ the right discriminator)
│ │
│ ├─ Want decomposition / weight-vector style
│ │ → Moead (very fast per generation, scales well)
│ │
│ ├─ Disconnected or non-convex front
│ │ → AgeMoea (estimates front geometry adaptively)
│ │ → Knea (favors knee points)
│ │ → Ibea
│ │
│ ├─ Want region-based diversity
│ │ → PesaII (grid hyperboxes drive selection)
│ │ → EpsilonMoea (ε-grid archive,
│ │ archive size auto-limits)
│ │
│ └─ Just one starting decision (no population budget)
│ → Paes (1+1 ES with a Pareto archive)
│
└─ 4+ (many-objective)
│
├─ Linear / simplex-shaped front (e.g., DTLZ1)
│ → Grea (grid coords drive ranking; on DTLZ1
│ here it beats NSGA-III by 3× and
│ AGE-MOEA by 2.5×)
│ → Moead (decomposition shines on linear fronts;
│ second on DTLZ1, also among the
│ fastest per generation)
│
├─ Curved / unknown front geometry
│ → Nsga3 (reference-point niching, canonical;
│ a strong default when the front
│ isn't simplex-shaped)
│ → AgeMoea (estimates L_p geometry per generation)
│ → Rvea (reference vectors with adaptive penalty)
│
├─ Want indicator-based selection
│ → Ibea (additive ε-indicator; doesn't degrade
│ at high obj count)
│ → Hype (Monte Carlo HV estimation; scales
│ to arbitrary M)
```
### Quick reference
**Sample-efficient / expensive evaluation (50–500 evals):**
| Algorithm | Objectives | Decision | Strengths |
|---|---|---|---|
| `BayesianOpt` | 1 | `Vec<f64>` | GP surrogate + EI; gold standard *with* per-problem kernel tuning (default RBF at 60 evals is honestly bad) |
| `Tpe` | 1 | `Vec<f64>` | KDE surrogate; robust without hyperparameter tuning |
| `Hyperband` | 1 | any | multi-fidelity; needs `PartialProblem` |
**Single-objective continuous (`Vec<f64>`):**
| Algorithm | Strengths |
|---|---|
| `RandomSearch` | sanity baseline |
| `HillClimber` | simplest greedy local search |
| `OnePlusOneEs` | one-fifth-rule self-adapting baseline |
| `SimulatedAnnealing` | escapes local optima |
| `GeneticAlgorithm` | classic SO GA with elitism |
| `ParticleSwarm` | simple swarm baseline |
| `DifferentialEvolution` | strong default for cheap continuous |
| `Tlbo` | parameter-free (no F, CR, w, σ) |
| `CmaEs` | smooth landscapes; full covariance |
| `IpopCmaEs` | CMA-ES + restart for multimodal |
| `SeparableNes` | diagonal-cov NES; cheap per-step |
| `NelderMead` | classical simplex; deterministic |
**Single-objective other decision types:**
| Algorithm | Decision | Strengths |
|---|---|---|
| `Umda` | `Vec<bool>` | independent-bit EDA |
| `TabuSearch` | any | discrete, you supply neighbors |
| `AntColonyTsp` | `Vec<usize>` | TSP / permutation |
**Multi-objective (2–3) and many-objective (4+):**
| Algorithm | Objectives | Strengths |
|---|---|---|
| `Paes` | 2–3 | 1+1 ES with Pareto archive |
| `Nsga2` | 2–3 | canonical Pareto-based EA |
| `Spea2` | 2–3 | strength + density |
| `Mopso` | 2–3 | multi-objective PSO; best convergence on smooth real-valued 2-obj fronts |
| `Ibea` | 2+ | indicator-based; consistently best of the dominance-based methods |
| `SmsEmoa` | 2+ | exact HV-contribution selection; high per-step cost, modest gain |
| `Hype` | 2+ | Monte Carlo HV estimation |
| `EpsilonMoea` | 2+ | ε-grid archive; auto-sized |
| `PesaII` | 2+ | grid-based region selection |
| `AgeMoea` | 2+ | adaptive front-geometry estimation |
| `Knea` | 2+ | knee-point favored survival |
| `Moead` | 2+ | decomposition; fast per-gen |
| `Nsga3` | 4+ | reference-point niching; strong on curved fronts |
| `Rvea` | 4+ | reference vectors with penalty |
| `Grea` | 4+ | grid coords drive selection; particularly strong on linear/simplex fronts |
## Current algorithms ## Current algorithms
- `RandomSearch` — sample-evaluate-keep baseline. The full list with one-line descriptions:
- `Paes` — a small (1+1) Pareto Archived Evolution Strategy.
- `Nsga2` — the canonical Pareto-based evolutionary algorithm.
- `DifferentialEvolution` — DE/rand/1/bin for single-objective real-valued
problems.
Plus reusable utilities: `pareto_compare`, `pareto_front`, `best_candidate`, **Sample-efficient / multi-fidelity:**
`non_dominated_sort`, `crowding_distance`, `ParetoArchive`, and the metrics
`spacing` and `hypervolume_2d`. - `BayesianOpt` — Gaussian-process surrogate + Expected Improvement.
- `Tpe` — Bergstra et al. 2011 Tree-structured Parzen Estimator.
- `Hyperband` — Li et al. 2017 multi-fidelity (uses `PartialProblem`).
**Single-objective:**
- `RandomSearch` — sample-evaluate-keep baseline.
- `HillClimber` — greedy single-step local search.
- `OnePlusOneEs` — Rechenberg 1973 (1+1)-ES with one-fifth rule.
- `SimulatedAnnealing` — Kirkpatrick et al. 1983, generic over decision type.
- `TabuSearch` — Glover 1986, with a user-supplied neighbor generator.
- `GeneticAlgorithm` — generational GA with tournament selection + elitism.
- `ParticleSwarm` — Eberhart & Kennedy 1995 PSO for `Vec<f64>`.
- `DifferentialEvolution` — Storn & Price DE/rand/1/bin for `Vec<f64>`.
- `Tlbo` — Rao 2011 Teaching-Learning-Based Optimization (parameter-free).
- `CmaEs` — Hansen & Ostermeier 2001 covariance-matrix adaptation.
- `IpopCmaEs` — Auger & Hansen 2005 CMA-ES with restart, for multimodal.
- `SeparableNes` — Wierstra et al. 2008/2014 diagonal-cov NES.
- `NelderMead` — Nelder & Mead 1965 simplex direct search.
- `Umda` — Mühlenbein 1997 univariate marginal-distribution EDA for `Vec<bool>`.
- `AntColonyTsp` — Dorigo Ant System for permutation problems.
**Multi-objective:**
- `Paes` — Knowles & Corne 1999 Pareto Archived Evolution Strategy.
- `Nsga2` — Deb et al. 2002, the canonical Pareto-based EA.
- `Spea2` — Zitzler, Laumanns & Thiele 2001 strength-Pareto EA.
- `Moead` — Zhang & Li 2007 decomposition-based MOEA with Tchebycheff scalarization.
- `Mopso` — Coello, Pulido & Lechuga 2004 multi-objective PSO.
- `Ibea` — Zitzler & Künzli 2004 indicator-based EA.
- `SmsEmoa` — Beume, Naujoks & Emmerich 2007 hypervolume-selection EMOA.
- `Hype` — Bader & Zitzler 2011 Hypervolume Estimation Algorithm.
- `EpsilonMoea` — Deb, Mohan & Mishra 2003 ε-dominance MOEA.
- `PesaII` — Corne et al. 2001 Pareto Envelope Selection II.
- `AgeMoea` — Panichella 2019 Adaptive Geometry Estimation MOEA.
- `Knea` — Zhang, Tian & Jin 2015 Knee point-driven EA.
**Many-objective (4+):**
- `Nsga3` — Deb & Jain 2014 reference-point NSGA-III.
- `Rvea` — Cheng et al. 2016 Reference Vector-guided EA.
- `Grea` — Yang et al. 2013 Grid-based EA.
**Reusable utilities:** `pareto_compare`, `pareto_front`, `best_candidate`,
`non_dominated_sort`, `crowding_distance`, `ParetoArchive`, `das_dennis`,
and the metrics `spacing` and `hypervolume_2d`.
## Design philosophy ## Design philosophy
@@ -138,6 +482,28 @@ Plus reusable utilities: `pareto_compare`, `pareto_front`, `best_candidate`,
See `docs/heuropt_tech_design_spec.md` for the full design rationale. See `docs/heuropt_tech_design_spec.md` for the full design rationale.
## Testing
heuropt is exhaustively tested across several layers:
- **Unit + integration tests** (`cargo test`) — 313 tests covering
every algorithm, operator, metric, Pareto utility, and edge case
(empty/singleton/duplicate populations, flat fitness, zero-width
bounds, infeasible-only populations).
- **Property-based tests** (`proptest`) — bounds preservation,
Pareto antisymmetry/reflexivity, partition correctness,
determinism, and seed-stability checks for every algorithm.
- **Coverage-guided fuzzing** (`cargo +nightly fuzz run <target>`) —
eight targets at `fuzz/fuzz_targets/`, soaked for 60 s per target
in CI on every PR.
- **Instruction-count benchmarks** (`cargo bench`) — `gungraun`
(callgrind) hot-path benchmarks for every algorithm and Pareto
utility, machine-stable so PR-level regressions show up.
- **Mutation testing** (`cargo mutants`) — advisory; config at
`.cargo/mutants.toml`.
- **CI** (`.github/workflows/ci.yml`) — fmt, clippy
(`-D warnings`), test (4-feature matrix), doc, MSRV (1.85), fuzz.
## License ## License
MIT — see [LICENSE](LICENSE). MIT — see [LICENSE](LICENSE).
benches/hot_paths.rs
+651
View File
@@ -0,0 +1,651 @@
//! Instruction-count benchmarks for heuropt's algorithmic hot paths.
//!
//! Run with `cargo bench`. Requires `valgrind` installed.
//!
//! The benchmarks here are *not* end-to-end optimizer runs; those are
//! covered by `examples/compare`. These are the inner-loop primitives
//! that every algorithm depends on, so a regression here lights up
//! across the whole crate.
use std::hint::black_box;
use gungraun::prelude::*;
use heuropt::core::candidate::Candidate;
use heuropt::core::evaluation::Evaluation;
use heuropt::core::objective::{Objective, ObjectiveSpace};
use heuropt::core::problem::Problem;
use heuropt::metrics::hypervolume::{hypervolume_2d, hypervolume_nd};
use heuropt::pareto::crowding::crowding_distance;
use heuropt::pareto::sort::non_dominated_sort;
use heuropt::prelude::*;
// -----------------------------------------------------------------------------
// Pareto utilities
// -----------------------------------------------------------------------------
fn make_2d_population(n: usize) -> Vec<Candidate<()>> {
(0..n)
.map(|i| {
let t = i as f64 / n as f64;
Candidate::new((), Evaluation::new(vec![t, 1.0 - t.sqrt()]))
})
.collect()
}
fn space_2d() -> ObjectiveSpace {
ObjectiveSpace::new(vec![Objective::minimize("f1"), Objective::minimize("f2")])
}
#[library_benchmark]
#[bench::n_50(50)]
#[bench::n_200(200)]
fn non_dominated_sort_2d(n: usize) -> Vec<Vec<usize>> {
let pop = make_2d_population(n);
let s = space_2d();
black_box(non_dominated_sort(black_box(&pop), black_box(&s)))
}
#[library_benchmark]
#[bench::n_50(50)]
#[bench::n_200(200)]
fn crowding_distance_2d(n: usize) -> Vec<f64> {
let pop = make_2d_population(n);
let s = space_2d();
let front: Vec<usize> = (0..pop.len()).collect();
black_box(crowding_distance(
black_box(&pop),
black_box(&front),
black_box(&s),
))
}
#[library_benchmark]
#[bench::n_30(30)]
#[bench::n_100(100)]
fn hypervolume_2d_bench(n: usize) -> f64 {
let pop = make_2d_population(n);
let s = space_2d();
black_box(hypervolume_2d(
black_box(&pop),
black_box(&s),
black_box([1.1, 1.1]),
))
}
fn make_3d_population(n: usize) -> (Vec<Candidate<()>>, ObjectiveSpace) {
let s = ObjectiveSpace::new(vec![
Objective::minimize("f1"),
Objective::minimize("f2"),
Objective::minimize("f3"),
]);
let pop = (0..n)
.map(|i| {
let t = i as f64 / n as f64;
let theta = 0.5 * std::f64::consts::PI * t;
Candidate::new((), Evaluation::new(vec![theta.cos(), theta.sin(), 1.0 - t]))
})
.collect();
(pop, s)
}
#[library_benchmark]
#[bench::n_30(30)]
#[bench::n_100(100)]
fn hypervolume_nd_bench_3d(n: usize) -> f64 {
let (pop, s) = make_3d_population(n);
black_box(hypervolume_nd(
black_box(&pop),
black_box(&s),
black_box(&[2.0, 2.0, 2.0]),
))
}
library_benchmark_group!(
name = pareto_group;
benchmarks =
non_dominated_sort_2d,
crowding_distance_2d,
hypervolume_2d_bench,
hypervolume_nd_bench_3d
);
// -----------------------------------------------------------------------------
// End-to-end algorithm smoke benches (single-generation cost)
// -----------------------------------------------------------------------------
/// Schaffer N.1 (2-objective). Inlined here so the bench doesn't need
/// to reach into the crate's `cfg(test)` test support.
struct SchafferN1;
impl Problem for SchafferN1 {
type Decision = Vec<f64>;
fn objectives(&self) -> ObjectiveSpace {
ObjectiveSpace::new(vec![Objective::minimize("f1"), Objective::minimize("f2")])
}
fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
let v = x[0];
Evaluation::new(vec![v * v, (v - 2.0).powi(2)])
}
}
#[library_benchmark]
fn nsga2_one_generation() -> usize {
let bounds = vec![(-5.0, 5.0)];
let initializer = RealBounds::new(bounds.clone());
let variation = CompositeVariation {
crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5),
mutation: PolynomialMutation::new(bounds, 20.0, 1.0),
};
let mut opt = Nsga2::new(
Nsga2Config {
population_size: 50,
generations: 1,
seed: 0,
},
initializer,
variation,
);
let result = opt.run(black_box(&SchafferN1));
black_box(result.evaluations)
}
#[library_benchmark]
fn cma_es_one_generation() -> usize {
let bounds = RealBounds::new(vec![(-5.0, 5.0); 5]);
let mut opt = CmaEs::new(
CmaEsConfig {
population_size: 16,
generations: 1,
initial_sigma: 0.5,
eigen_decomposition_period: 1,
initial_mean: None,
seed: 0,
},
bounds,
);
struct Sphere5D;
impl Problem for Sphere5D {
type Decision = Vec<f64>;
fn objectives(&self) -> ObjectiveSpace {
ObjectiveSpace::new(vec![Objective::minimize("f")])
}
fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
Evaluation::new(vec![x.iter().map(|v| v * v).sum()])
}
}
let result = opt.run(black_box(&Sphere5D));
black_box(result.evaluations)
}
library_benchmark_group!(
name = algorithm_group;
benchmarks = nsga2_one_generation, cma_es_one_generation
);
// -----------------------------------------------------------------------------
// Wider single-objective sweep
// -----------------------------------------------------------------------------
struct Sphere1D;
impl Problem for Sphere1D {
type Decision = Vec<f64>;
fn objectives(&self) -> ObjectiveSpace {
ObjectiveSpace::new(vec![Objective::minimize("f")])
}
fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
Evaluation::new(vec![x[0] * x[0]])
}
}
fn so_bounds() -> RealBounds {
RealBounds::new(vec![(-3.0, 3.0)])
}
#[library_benchmark]
fn random_search_short() -> usize {
let mut o = RandomSearch::new(
RandomSearchConfig {
iterations: 50,
batch_size: 1,
seed: 0,
},
so_bounds(),
);
black_box(o.run(black_box(&Sphere1D)).evaluations)
}
#[library_benchmark]
fn hill_climber_short() -> usize {
let mut o = HillClimber::new(
HillClimberConfig {
iterations: 50,
seed: 0,
},
so_bounds(),
GaussianMutation { sigma: 0.1 },
);
black_box(o.run(black_box(&Sphere1D)).evaluations)
}
#[library_benchmark]
fn one_plus_one_es_short() -> usize {
let mut o = OnePlusOneEs::new(
OnePlusOneEsConfig {
iterations: 50,
initial_sigma: 0.5,
adaptation_period: 10,
step_increase: 1.22,
seed: 0,
},
so_bounds(),
);
black_box(o.run(black_box(&Sphere1D)).evaluations)
}
#[library_benchmark]
fn simulated_annealing_short() -> usize {
let mut o = SimulatedAnnealing::new(
SimulatedAnnealingConfig {
iterations: 50,
initial_temperature: 1.0,
final_temperature: 1e-3,
seed: 0,
},
so_bounds(),
GaussianMutation { sigma: 0.1 },
);
black_box(o.run(black_box(&Sphere1D)).evaluations)
}
#[library_benchmark]
fn genetic_algorithm_short() -> usize {
let bounds = vec![(-3.0, 3.0)];
let mut o = GeneticAlgorithm::new(
GeneticAlgorithmConfig {
population_size: 10,
generations: 5,
tournament_size: 2,
elitism: 1,
seed: 0,
},
RealBounds::new(bounds.clone()),
CompositeVariation {
crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5),
mutation: PolynomialMutation::new(bounds, 20.0, 1.0),
},
);
black_box(o.run(black_box(&Sphere1D)).evaluations)
}
#[library_benchmark]
fn particle_swarm_short() -> usize {
let mut o = ParticleSwarm::new(
ParticleSwarmConfig {
swarm_size: 10,
generations: 5,
inertia: 0.7,
cognitive: 1.5,
social: 1.5,
seed: 0,
},
so_bounds(),
);
black_box(o.run(black_box(&Sphere1D)).evaluations)
}
#[library_benchmark]
fn differential_evolution_short() -> usize {
let mut o = DifferentialEvolution::new(
DifferentialEvolutionConfig {
population_size: 10,
generations: 5,
differential_weight: 0.5,
crossover_probability: 0.9,
seed: 0,
},
so_bounds(),
);
black_box(o.run(black_box(&Sphere1D)).evaluations)
}
#[library_benchmark]
fn tlbo_short() -> usize {
let mut o = Tlbo::new(
TlboConfig {
population_size: 10,
generations: 5,
seed: 0,
},
so_bounds(),
);
black_box(o.run(black_box(&Sphere1D)).evaluations)
}
#[library_benchmark]
fn separable_nes_short() -> usize {
let mut o = SeparableNes::new(
SeparableNesConfig {
population_size: 8,
generations: 5,
initial_sigma: 0.5,
mean_learning_rate: 1.0,
sigma_learning_rate: None,
seed: 0,
},
so_bounds(),
);
black_box(o.run(black_box(&Sphere1D)).evaluations)
}
#[library_benchmark]
fn nelder_mead_short() -> usize {
let mut o = NelderMead::new(
NelderMeadConfig {
iterations: 50,
..NelderMeadConfig::default()
},
so_bounds(),
);
black_box(o.run(black_box(&Sphere1D)).evaluations)
}
#[library_benchmark]
fn bayesian_opt_short() -> usize {
let mut o = BayesianOpt::new(
BayesianOptConfig {
initial_samples: 5,
iterations: 10,
length_scales: None,
signal_variance: 1.0,
noise_variance: 1e-6,
acquisition_samples: 100,
seed: 0,
},
so_bounds(),
);
black_box(o.run(black_box(&Sphere1D)).evaluations)
}
#[library_benchmark]
fn tpe_short() -> usize {
let mut o = Tpe::new(
TpeConfig {
initial_samples: 5,
iterations: 10,
good_fraction: 0.25,
candidate_samples: 12,
bandwidth_factor: 1.0,
seed: 0,
},
so_bounds(),
);
black_box(o.run(black_box(&Sphere1D)).evaluations)
}
#[library_benchmark]
fn ipop_cma_es_short() -> usize {
let mut o = IpopCmaEs::new(
IpopCmaEsConfig {
initial_population_size: 8,
total_generations: 30,
initial_sigma: 0.5,
eigen_decomposition_period: 1,
stall_generations: None,
seed: 0,
},
so_bounds(),
);
black_box(o.run(black_box(&Sphere1D)).evaluations)
}
library_benchmark_group!(
name = single_objective_group;
benchmarks =
random_search_short, hill_climber_short, one_plus_one_es_short,
simulated_annealing_short, genetic_algorithm_short,
particle_swarm_short, differential_evolution_short, tlbo_short,
separable_nes_short, nelder_mead_short,
bayesian_opt_short, tpe_short, ipop_cma_es_short
);
// -----------------------------------------------------------------------------
// Multi-objective sweep
// -----------------------------------------------------------------------------
fn schaffer_bounds() -> Vec<(f64, f64)> {
vec![(-3.0, 3.0)]
}
fn mo_variation() -> CompositeVariation<SimulatedBinaryCrossover, PolynomialMutation> {
let bounds = schaffer_bounds();
CompositeVariation {
crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5),
mutation: PolynomialMutation::new(bounds, 20.0, 1.0),
}
}
#[library_benchmark]
fn nsga3_short() -> usize {
let mut o = Nsga3::new(
Nsga3Config {
population_size: 12,
generations: 1,
reference_divisions: 11,
seed: 0,
},
RealBounds::new(schaffer_bounds()),
mo_variation(),
);
black_box(o.run(black_box(&SchafferN1)).evaluations)
}
#[library_benchmark]
fn spea2_short() -> usize {
let mut o = Spea2::new(
Spea2Config {
population_size: 10,
archive_size: 10,
generations: 1,
seed: 0,
},
RealBounds::new(schaffer_bounds()),
mo_variation(),
);
black_box(o.run(black_box(&SchafferN1)).evaluations)
}
#[library_benchmark]
fn moead_short() -> usize {
let mut o = Moead::new(
MoeadConfig {
generations: 1,
reference_divisions: 9,
neighborhood_size: 4,
seed: 0,
},
RealBounds::new(schaffer_bounds()),
mo_variation(),
);
black_box(o.run(black_box(&SchafferN1)).evaluations)
}
#[library_benchmark]
fn mopso_short() -> usize {
let mut o = Mopso::new(
MopsoConfig {
swarm_size: 10,
generations: 1,
archive_size: 10,
inertia: 0.7,
cognitive: 1.5,
social: 1.5,
seed: 0,
},
RealBounds::new(schaffer_bounds()),
);
black_box(o.run(black_box(&SchafferN1)).evaluations)
}
#[library_benchmark]
fn ibea_short() -> usize {
let mut o = Ibea::new(
IbeaConfig {
population_size: 10,
generations: 1,
kappa: 0.05,
seed: 0,
},
RealBounds::new(schaffer_bounds()),
mo_variation(),
);
black_box(o.run(black_box(&SchafferN1)).evaluations)
}
#[library_benchmark]
fn sms_emoa_short() -> usize {
let mut o = SmsEmoa::new(
SmsEmoaConfig {
population_size: 8,
generations: 5,
reference_point: vec![10.0, 10.0],
seed: 0,
},
RealBounds::new(schaffer_bounds()),
mo_variation(),
);
black_box(o.run(black_box(&SchafferN1)).evaluations)
}
#[library_benchmark]
fn hype_short() -> usize {
let mut o = Hype::new(
HypeConfig {
population_size: 10,
generations: 1,
reference_point: vec![10.0, 10.0],
mc_samples: 100,
seed: 0,
},
RealBounds::new(schaffer_bounds()),
mo_variation(),
);
black_box(o.run(black_box(&SchafferN1)).evaluations)
}
#[library_benchmark]
fn pesa2_short() -> usize {
let mut o = PesaII::new(
PesaIIConfig {
population_size: 10,
archive_size: 10,
generations: 1,
grid_divisions: 4,
seed: 0,
},
RealBounds::new(schaffer_bounds()),
mo_variation(),
);
black_box(o.run(black_box(&SchafferN1)).evaluations)
}
#[library_benchmark]
fn epsilon_moea_short() -> usize {
let mut o = EpsilonMoea::new(
EpsilonMoeaConfig {
population_size: 10,
evaluations: 30,
epsilon: vec![0.05, 0.05],
seed: 0,
},
RealBounds::new(schaffer_bounds()),
mo_variation(),
);
black_box(o.run(black_box(&SchafferN1)).evaluations)
}
#[library_benchmark]
fn age_moea_short() -> usize {
let mut o = AgeMoea::new(
AgeMoeaConfig {
population_size: 10,
generations: 1,
seed: 0,
},
RealBounds::new(schaffer_bounds()),
mo_variation(),
);
black_box(o.run(black_box(&SchafferN1)).evaluations)
}
#[library_benchmark]
fn grea_short() -> usize {
let mut o = Grea::new(
GreaConfig {
population_size: 10,
generations: 1,
grid_divisions: 4,
seed: 0,
},
RealBounds::new(schaffer_bounds()),
mo_variation(),
);
black_box(o.run(black_box(&SchafferN1)).evaluations)
}
#[library_benchmark]
fn knea_short() -> usize {
let mut o = Knea::new(
KneaConfig {
population_size: 10,
generations: 1,
seed: 0,
},
RealBounds::new(schaffer_bounds()),
mo_variation(),
);
black_box(o.run(black_box(&SchafferN1)).evaluations)
}
#[library_benchmark]
fn rvea_short() -> usize {
let mut o = Rvea::new(
RveaConfig {
population_size: 10,
generations: 1,
reference_divisions: 9,
alpha: 2.0,
seed: 0,
},
RealBounds::new(schaffer_bounds()),
mo_variation(),
);
black_box(o.run(black_box(&SchafferN1)).evaluations)
}
#[library_benchmark]
fn paes_short() -> usize {
let mut o = Paes::new(
PaesConfig {
iterations: 30,
archive_size: 10,
seed: 0,
},
RealBounds::new(schaffer_bounds()),
GaussianMutation { sigma: 0.1 },
);
black_box(o.run(black_box(&SchafferN1)).evaluations)
}
library_benchmark_group!(
name = multi_objective_group;
benchmarks =
nsga3_short, spea2_short, moead_short, mopso_short, ibea_short,
sms_emoa_short, hype_short, pesa2_short, epsilon_moea_short,
age_moea_short, grea_short, knea_short, rvea_short, paes_short
);
main!(
library_benchmark_groups = pareto_group,
algorithm_group,
single_objective_group,
multi_objective_group
);
examples/benchmarks.rs
+8 -2
View File
@@ -58,7 +58,9 @@ impl Problem for Rastrigin {
fn evaluate(&self, x: &Vec<f64>) -> Evaluation { fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
let n = self.dim as f64; let n = self.dim as f64;
let value = 10.0 * n let value = 10.0 * n
+ x.iter().map(|v| v * v - 10.0 * (2.0 * PI * v).cos()).sum::<f64>(); + x.iter()
.map(|v| v * v - 10.0 * (2.0 * PI * v).cos())
.sum::<f64>();
Evaluation::new(vec![value]) Evaluation::new(vec![value])
} }
} }
@@ -104,7 +106,11 @@ fn run_zdt1() {
crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5), crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5),
mutation: PolynomialMutation::new(bounds, 20.0, 1.0 / dim as f64), mutation: PolynomialMutation::new(bounds, 20.0, 1.0 / dim as f64),
}; };
let config = Nsga2Config { population_size: 100, generations: 1000, seed: 42 }; let config = Nsga2Config {
population_size: 100,
generations: 1000,
seed: 42,
};
let mut optimizer = Nsga2::new(config, initializer, variation); let mut optimizer = Nsga2::new(config, initializer, variation);
let result = optimizer.run(&problem); let result = optimizer.run(&problem);
examples/compare-results.md
+142
View File
@@ -0,0 +1,142 @@
# `compare` example — reference output
Snapshot from `cargo run --release --example compare` after the v0.4.0
perf pass landed (2026-05-05). 10 seeds per algorithm per problem.
The **quality metrics** (hypervolume / spacing / mean L2 / mean dist /
front size) are bit-identical to the v0.3.0 snapshot — the v0.4.0
optimization work was strictly CPU-time, never algorithmic. The **ms
columns** reflect the v0.4.0 numbers; total compare-harness wall-clock
dropped from ~18.6 s to ~5.7 s (3.27× faster).
Wall-clock numbers are from the development machine and will vary;
the *relative* numbers across algorithms are the interesting part.
---
## ZDT1 (dim=30, 25000 evals/run × 10 seeds)
Two-objective benchmark with a smooth Pareto front along
`f₂ = 1 − √f₁`. Hypervolume reference point: `[11, 11]`.
| algorithm | hypervolume ↑ | spacing ↓ | mean L2 ↓ | front | ms |
|---|---|---|---|---|---|
| RandomSearch | 99.5691 ± 0.94 | 0.0937 ± 0.03 | 2.3621 ± 0.14 | 28 | 94 |
| PAES | 104.1887 ± 0.90 | 0.0351 ± 0.01 | 1.3195 ± 0.06 | 33 | 30 |
| MOPSO | **120.6149 ± 0.05** | 0.0125 ± 0.00 | **0.0005 ± 0.00** | 100 | 89 |
| SPEA2 | 118.0823 ± 0.60 | 0.0111 ± 0.00 | 0.2408 ± 0.05 | 97 | 234 |
| PESA-II | 119.3670 ± 0.33 | **0.0095 ± 0.00** | 0.0802 ± 0.04 | 100 | 73 |
| ε-MOEA | 118.8742 ± 0.68 | 0.0167 ± 0.01 | 0.0493 ± 0.02 | 45 | 50 |
| IBEA | 120.0167 ± 0.31 | 0.0130 ± 0.00 | 0.0448 ± 0.02 | 73 | 138 |
| HypE | 105.6489 ± 0.98 | 0.0266 ± 0.01 | 1.4820 ± 0.10 | 72 | 38 |
| SMS-EMOA | 102.8871 ± 1.05 | 0.0263 ± 0.00 | 1.4937 ± 0.12 | 40 | 67 |
| RVEA | 111.7151 ± 1.82 | 0.0308 ± 0.01 | 0.8399 ± 0.16 | 47 | 65 |
| NSGA-II | 118.3336 ± 0.78 | 0.0112 ± 0.00 | 0.1891 ± 0.06 | 96 | 67 |
| NSGA-III | 115.1612 ± 0.47 | 0.0139 ± 0.00 | 0.4314 ± 0.06 | 86 | 70 |
| MOEA/D | 119.9450 ± 0.50 | 0.0118 ± 0.00 | 0.0065 ± 0.00 | 96 | 28 |
**MOPSO and MOEA/D dominate** convergence (mean L2 to true front ≤ 0.01).
PESA-II edges spacing.
## ZDT3 (dim=30, 25000 evals × 10 seeds)
Disconnected Pareto front; tests an algorithm's ability to maintain
spread across gaps.
| algorithm | hypervolume ↑ | spacing ↓ | front | ms |
|---|---|---|---|---|
| NSGA-II | 123.1826 ± 1.58 | 0.0092 ± 0.00 | 98 | 68 |
| MOEA/D | 125.2413 ± 2.16 | 0.0198 ± 0.00 | 92 | 28 |
| **IBEA** | **126.2072 ± 1.23** | 0.0164 ± 0.00 | 48 | 135 |
| AGE-MOEA | 119.5132 ± 1.27 | 0.0136 ± 0.00 | 90 | 199 |
## DTLZ2 (3-obj, dim=12, 30000 evals × 10 seeds)
Spherical Pareto front. Mean dist = `|‖f‖ − 1|`.
| algorithm | mean dist ↓ | spacing ↓ | front | ms |
|---|---|---|---|---|
| RandomSearch | 0.3949 ± 0.02 | 0.0797 ± 0.01 | 239 | 520 |
| MOPSO | 0.0566 ± 0.00 | 0.0687 ± 0.01 | 100 | 71 |
| NSGA-II | 0.0332 ± 0.01 | 0.0577 ± 0.01 | 92 | 104 |
| SPEA2 | 0.0368 ± 0.00 | **0.0288 ± 0.00** | 92 | 534 |
| PESA-II | 0.0395 ± 0.00 | 0.0616 ± 0.01 | 100 | 396 |
| ε-MOEA | 0.0325 ± 0.01 | 0.0572 ± 0.02 | 136 | 89 |
| **IBEA** | **0.0014 ± 0.00** | 0.0607 ± 0.00 | 87 | 156 |
| HypE | 0.0113 ± 0.00 | 0.0269 ± 0.02 | 80 | 53 |
| SMS-EMOA | 0.0484 ± 0.01 | 0.0764 ± 0.01 | 40 | 1218 |
| RVEA | 0.0510 ± 0.00 | 0.0631 ± 0.00 | 68 | 73 |
| NSGA-III | 0.0197 ± 0.00 | 0.0735 ± 0.01 | 92 | 137 |
| MOEA/D | 0.0037 ± 0.00 | 0.0886 ± 0.00 | 78 | 24 |
**IBEA wins decisively** (15× closer to the true front than NSGA-III).
## DTLZ1 (3-obj, dim=7, 30000 evals × 10 seeds)
Linear simplex Pareto front (`Σf = 0.5`).
| algorithm | mean dist ↓ | spacing ↓ | front | ms |
|---|---|---|---|---|
| NSGA-III | 5.9130 ± 2.82 | 0.4375 ± 0.22 | 92 | 133 |
| MOEA/D | 2.8022 ± 1.78 | 0.2279 ± 0.22 | 78 | 21 |
| AGE-MOEA | 4.5395 ± 2.21 | 0.3930 ± 0.29 | 90 | 247 |
| **GrEA** | **1.7725 ± 0.99** | **0.0719 ± 0.04** | 72 | 104 |
**GrEA shines on linear fronts** — the grid-based niching matches the
geometry better than reference points.
## Rastrigin (dim=5, 50000 evals/run × 10 seeds)
Multimodal trap. Global minimum f = 0 at the origin.
| algorithm | best f | ms |
|---|---|---|
| RandomSearch | 1.1064e1 ± 2.54 | 14 |
| HillClimber | 1.5966e1 ± 6.25 | 6 |
| **(1+1)-ES** | **0.0000e0 ± 0.00** | 4 |
| SimulatedAnneal | 3.8540e0 ± 1.48 | 7 |
| PAES | 1.5966e1 ± 6.25 | 10 |
| GA | 7.0913e-8 ± 5.50e-8 | 16 |
| PSO | 7.9598e-1 ± 8.67e-1 | 5 |
| NSGA-II | 4.9270e-5 ± 5.04e-5 | 83 |
| **DE** | **0.0000e0 ± 0.00** | 6 |
| CMA-ES | 2.3453e0 ± 1.49 | 11 |
| **IPOP-CMA-ES** | 1.3423e-1 ± 2.71e-1 | 66 |
(1+1)-ES and DE tie for f = 0. **IPOP-CMA-ES drops vanilla CMA-ES from
2.35 → 0.13** — the restart logic does what it should.
## Rosenbrock (dim=5, 30000 evals × 10 seeds)
Smooth non-convex valley.
| algorithm | best f | ms |
|---|---|---|
| DE | 3.3345e-1 ± 3.01e-1 | 2 |
| PSO | 8.2124e-1 ± 1.58e0 | 2 |
| **CMA-ES** | **3.6207e-29 ± 2.35e-29** | 5 |
| TLBO | 1.8458e-3 ± 1.91e-3 | 1 |
| (1+1)-ES | 2.2115e0 ± 2.70e0 | 1 |
| **Nelder-Mead** | **0.0000e0 ± 0.00** | 1 |
| BO (60 evals) | 3.1725e3 ± 2.92e3 | 40 |
Nelder-Mead **= 0 exactly**, CMA-ES at machine epsilon. BO at only 60
evaluations is honestly bad on 5-D Rosenbrock (no kernel
hyperparameter tuning) — included as a reminder that BO needs more
evaluations than a smooth problem actually requires for these other
methods.
## Ackley (dim=5, 30000 evals × 10 seeds)
Smoother multimodal landscape than Rastrigin.
| algorithm | best f | ms |
|---|---|---|
| DE | 4.4409e-16 ± 0.00 | 4 |
| PSO | 1.5099e-15 ± 1.63e-15 | 3 |
| CMA-ES | 1.5099e-15 ± 1.63e-15 | 6 |
| TLBO | 2.2204e-15 ± 1.78e-15 | 2 |
| BO (60 evals) | 1.9622e1 ± 1.23 | 40 |
All conventional methods reach machine precision. BO at 60 evals
struggles — same caveat as Rosenbrock.
examples/compare.rs
+1372 -34
View File
File diff suppressed because it is too large Load Diff
examples/custom_optimizer.rs
+4 -1
View File
@@ -26,7 +26,10 @@ where
{ {
fn run(&mut self, problem: &P) -> OptimizationResult<P::Decision> { fn run(&mut self, problem: &P) -> OptimizationResult<P::Decision> {
let objectives = problem.objectives(); let objectives = problem.objectives();
assert!(objectives.is_single_objective(), "HillClimber needs one objective"); assert!(
objectives.is_single_objective(),
"HillClimber needs one objective"
);
let mut rng = rng_from_seed(self.seed); let mut rng = rng_from_seed(self.seed);
let mut variation = GaussianMutation { sigma: self.sigma }; let mut variation = GaussianMutation { sigma: self.sigma };
examples/jiggly_tuning.rs
+192 -50
View File
@@ -62,6 +62,30 @@ const SWEET_HI: u32 = 45;
const N_DAYS: usize = 1000; const N_DAYS: usize = 1000;
// -----------------------------------------------------------------------------
// A-posteriori decision weights (must sum to 1.0).
// -----------------------------------------------------------------------------
const W_LUNCH: f64 = 0.30; // top — design goal
const W_AFTER: f64 = 0.25; // top — minimize after-hours waste
const W_WORK: f64 = 0.20; // medium — failures bad but recoverable
const W_PRESS: f64 = 0.15; // matters with a hinge below
const W_BALANCE: f64 = 0.10; // bonus for longer yellow + red phases
// Press hinge: full reward at or below LOW, linearly drops to 0 at COMFORT_CAP,
// and any candidate with mean_presses > COMFORT_CAP is rejected outright.
//
// Counts every daily press: morning boot, 13:00 lunch retap, warning-phase
// reactions, and any death-restart presses during the workday. With ~2
// baseline presses already mandatory each day, the LOW threshold sits just
// above baseline (2 + a half warning press) and the cap allows up to
// 1.5 additional presses on top of baseline before rejecting.
const PRESS_HINGE_LOW: f64 = 2.5;
const PRESS_COMFORT_CAP: f64 = 3.5;
// Balance bonus saturates: a min(yellow_width, red_width) of >= this many
// minutes scores the full balance term.
const BALANCE_SATURATION_MIN: f64 = 10.0;
// ----------------------------------------------------------------------------- // -----------------------------------------------------------------------------
// Day model + Monte Carlo (same model as scripts/tune_runtime.py) // Day model + Monte Carlo (same model as scripts/tune_runtime.py)
// ----------------------------------------------------------------------------- // -----------------------------------------------------------------------------
@@ -127,18 +151,41 @@ impl JigglyTuning {
) -> DayOutcome { ) -> DayOutcome {
let mut rng = StdRng::seed_from_u64(day_seed); let mut rng = StdRng::seed_from_u64(day_seed);
let mut expire = s + rt; let mut expire = s + rt;
let mut o = DayOutcome::default(); // Boot press at workday start: user presses to begin cycle 1.
let t_max = e.max(expire) + 1; let mut o = DayOutcome {
presses: 1,
..Default::default()
};
// Allow the loop to extend past the larger of (workday end, last
// possible cycle end given any in-loop expire bumps). Cap at one
// extra cycle's worth so a long string of presses can't blow the
// budget.
let t_max = e.max(expire).max(s + 2 * rt) + 1;
let mut prev_running = true;
for t in s..t_max { for t in s..t_max {
// Free re-tap when the user re-logs in at 13:00. // 13:00 re-login press: user comes back from lunch, presses to
// start cycle 2.
if t == LUNCH_END && t < e { if t == LUNCH_END && t < e {
expire = t + rt; expire = t + rt;
o.presses += 1;
} }
let in_workday = t >= s && t < e; let in_workday = t >= s && t < e;
let at_lunch = (LUNCH_START..LUNCH_END).contains(&t); let at_lunch = (LUNCH_START..LUNCH_END).contains(&t);
let device_running = t < expire; let device_running = t < expire;
let device_dead = !device_running; let device_dead = !device_running;
// Death-restart press: when the device transitions from running
// to dead during workday (not at lunch), user notices the screen
// sleeping and presses to restart. Counts as a press for THIS
// minute; subsequent at-desk minutes are now covered.
if prev_running && device_dead && in_workday && !at_lunch {
expire = t + rt;
o.presses += 1;
prev_running = true;
continue;
}
prev_running = device_running;
if device_dead && in_workday { if device_dead && in_workday {
if at_lunch { if at_lunch {
o.slept_lunch += 1; o.slept_lunch += 1;
@@ -304,7 +351,11 @@ fn print_header() {
} }
fn print_row(label: &str, r: &Row) { fn print_row(label: &str, r: &Row) {
let prefix = if label.is_empty() { String::new() } else { format!("{label} ") }; let prefix = if label.is_empty() {
String::new()
} else {
format!("{label} ")
};
println!( println!(
"{}{:<6} {:>3} {:>3} {:>3} {:>9} {:>9} {:>7.2}/d {:>8} {:>6.1}%", "{}{:<6} {:>3} {:>3} {:>3} {:>9} {:>9} {:>7.2}/d {:>8} {:>6.1}%",
prefix, prefix,
@@ -396,7 +447,11 @@ fn main() {
println!("=== Pareto front (sorted by lunch sleep, descending) ==="); println!("=== Pareto front (sorted by lunch sleep, descending) ===");
print_header(); print_header();
rows.sort_by(|a, b| b.lunch.partial_cmp(&a.lunch).unwrap_or(std::cmp::Ordering::Equal)); rows.sort_by(|a, b| {
b.lunch
.partial_cmp(&a.lunch)
.unwrap_or(std::cmp::Ordering::Equal)
});
for r in rows.iter().take(15) { for r in rows.iter().take(15) {
print_row("", r); print_row("", r);
} }
@@ -443,16 +498,18 @@ fn main() {
// //
// Every point on the front is incomparable in the strict Pareto sense — // Every point on the front is incomparable in the strict Pareto sense —
// none dominates another. To surface ONE recommendation we apply explicit // none dominates another. To surface ONE recommendation we apply explicit
// weights to the four normalized objectives. Anyone with different // weights to four normalized outcome axes plus two structural terms:
// priorities can read the front above and pick a different row.
// //
// We add the firmware's shipping defaults to the candidate set so they // * `lunch_sleep` (max), `after_hours` (min), `work_fail` (min) —
// compete on equal footing with the front the optimizer found. // normalized to [0, 1] across the candidate set.
// * `presses` — hinge: full reward when <= PRESS_HINGE_LOW, ramps to
const W_WORK: f64 = 0.45; // work failures hurt most // zero at PRESS_COMFORT_CAP, candidates above the cap are rejected.
const W_LUNCH: f64 = 0.30; // the design goal // * `balance` — bonus for longer warning phases:
const W_PRESS: f64 = 0.15; // UX friction // `min(YA - RA, RA - FRA)` saturated at BALANCE_SATURATION_MIN.
const W_AFTER: f64 = 0.10; // minor screen-burn cost //
// Anyone with different priorities can read the front above and pick a
// different row. We add the firmware's shipping defaults to the
// candidate set so they compete on equal footing with the front.
let mut candidates: Vec<(String, Row)> = rows let mut candidates: Vec<(String, Row)> = rows
.iter() .iter()
@@ -461,20 +518,36 @@ fn main() {
let shipping_candidate_idx = candidates.len(); let shipping_candidate_idx = candidates.len();
candidates.push(("shipping default".to_string(), shipping_row.clone())); candidates.push(("shipping default".to_string(), shipping_row.clone()));
let scores = let scores = compute_weighted_scores(
compute_weighted_scores(&candidates.iter().map(|(_, r)| r.clone()).collect::<Vec<_>>()); &candidates
.iter()
.map(|(_, r)| r.clone())
.collect::<Vec<_>>(),
);
let mut ranked: Vec<(usize, f64)> = scores.iter().copied().enumerate().collect(); let mut ranked: Vec<(usize, f64)> = scores.iter().copied().enumerate().collect();
ranked.sort_by(|a, b| b.1.partial_cmp(&a.1).unwrap_or(std::cmp::Ordering::Equal)); ranked.sort_by(|a, b| b.1.partial_cmp(&a.1).unwrap_or(std::cmp::Ordering::Equal));
println!("=== ranked by weighted preferences ==="); println!("=== ranked by weighted preferences ===");
println!( println!(
" weights: work_fail {}% · lunch_sleep {}% · presses {}% · after_hours {}%", " weights: lunch_sleep {}% · after_hours {}% · work_fail {}% · presses {}% · balance {}%",
(W_WORK * 100.0) as i32,
(W_LUNCH * 100.0) as i32, (W_LUNCH * 100.0) as i32,
(W_PRESS * 100.0) as i32,
(W_AFTER * 100.0) as i32, (W_AFTER * 100.0) as i32,
(W_WORK * 100.0) as i32,
(W_PRESS * 100.0) as i32,
(W_BALANCE * 100.0) as i32,
);
println!(
" press hinge: full reward ≤ {:.1}/d, ramps to 0 at {:.1}/d, REJECTED above",
PRESS_HINGE_LOW, PRESS_COMFORT_CAP,
);
println!(
" balance bonus: min(yellow_width, red_width), saturates at {:.0} min",
BALANCE_SATURATION_MIN,
);
println!(
" candidate set: {} Pareto-front rows + 1 shipping default",
rows.len()
); );
println!(" candidate set: {} Pareto-front rows + 1 shipping default", rows.len());
println!(); println!();
println!("{:>4} {:>5} source", "rank", "score"); println!("{:>4} {:>5} source", "rank", "score");
print_header(); print_header();
@@ -493,8 +566,10 @@ fn main() {
.map(|p| p + 1) .map(|p| p + 1)
.unwrap_or(0); .unwrap_or(0);
let max_work = candidates.iter().map(|(_, r)| r.work_fail).fold(0.0, f64::max); let max_work = candidates
let max_press = candidates.iter().map(|(_, r)| r.presses).fold(0.0, f64::max); .iter()
.map(|(_, r)| r.work_fail)
.fold(0.0, f64::max);
println!("=== RECOMMENDED PICK ({top_label}) ==="); println!("=== RECOMMENDED PICK ({top_label}) ===");
println!( println!(
@@ -506,23 +581,45 @@ fn main() {
); );
println!(" weighted score = {top_score:.3}"); println!(" weighted score = {top_score:.3}");
println!(); println!();
let yellow_w = top.ya - top.ra;
let red_w = top.ra - top.fra;
println!("Why:"); println!("Why:");
println!(
" • {} mean work-time failure ({} better than the worst candidate)",
fmt_minutes(top.work_fail),
ratio_str(max_work, top.work_fail.max(1e-9)),
);
println!( println!(
" • {} mean lunch sleep ({:.1}% land in the 12:15–12:45 sweet spot)", " • {} mean lunch sleep ({:.1}% land in the 12:15–12:45 sweet spot)",
fmt_minutes(top.lunch), fmt_minutes(top.lunch),
top.p_sweet * 100.0, top.p_sweet * 100.0,
); );
println!( println!(
" • {:.2} button presses/day ({} fewer than the worst candidate)", " • {} mean after-hours awake (kept tight, your second priority)",
top.presses, fmt_minutes(top.after),
ratio_str(max_press, top.presses.max(1e-9)), );
println!(
" • {} mean work-time failure ({} better than the worst candidate)",
fmt_minutes(top.work_fail),
ratio_str(max_work, top.work_fail.max(1e-9)),
);
let press_note = if top.presses <= PRESS_HINGE_LOW {
format!("inside your no-penalty zone ≤{:.1}/d", PRESS_HINGE_LOW)
} else if top.presses < PRESS_COMFORT_CAP {
format!(
"above the {:.1}/d hinge but below your {:.1}/d cap",
PRESS_HINGE_LOW, PRESS_COMFORT_CAP,
)
} else {
format!("AT or ABOVE your {:.1}/d comfort cap", PRESS_COMFORT_CAP)
};
println!(
" • {:.2} button presses/day total — {}",
top.presses, press_note,
);
println!(" (counts: boot + 13:00 retap + warning-phase reactions + death-restarts)");
println!(
" • warning phases: yellow {} min, red {} min, fast-red {} min (balance score {:.2})",
yellow_w,
red_w,
top.fra,
balance_score_for(top),
); );
println!(" • {} mean after-hours awake (negligible)", fmt_minutes(top.after));
if top_label != "shipping default" { if top_label != "shipping default" {
println!(); println!();
@@ -540,45 +637,90 @@ fn main() {
} }
} }
/// Score every row in `rows` by a fixed weighted sum of normalized objectives. /// Score every row in `rows` by a weighted sum that combines normalized
/// outcome axes with a press hinge and a phase-balance bonus.
/// ///
/// Each objective is normalized to `[0, 1]` across `rows` with `1` meaning /// `work_fail`, `lunch`, and `after` are normalized to `[0, 1]` across `rows`
/// "best on the front" and `0` meaning "worst on the front", direction-aware /// (best→1, worst→0; direction-aware). `presses` uses a hinge that rewards
/// (lunch is maximize, the rest are minimize). /// values at or below `PRESS_HINGE_LOW`, ramps linearly to zero at
/// `PRESS_COMFORT_CAP`, and rejects candidates above the cap by returning
/// `f64::NEG_INFINITY`. `balance` is a bonus for longer yellow + red
/// phases, computed as `min(YA - RA, RA - FRA)` saturated at
/// `BALANCE_SATURATION_MIN`.
fn compute_weighted_scores(rows: &[Row]) -> Vec<f64> { fn compute_weighted_scores(rows: &[Row]) -> Vec<f64> {
const W_WORK: f64 = 0.45; let work_min = rows
const W_LUNCH: f64 = 0.30; .iter()
const W_PRESS: f64 = 0.15; .map(|r| r.work_fail)
const W_AFTER: f64 = 0.10; .fold(f64::INFINITY, f64::min);
let work_max = rows
let work_min = rows.iter().map(|r| r.work_fail).fold(f64::INFINITY, f64::min); .iter()
let work_max = rows.iter().map(|r| r.work_fail).fold(f64::NEG_INFINITY, f64::max); .map(|r| r.work_fail)
.fold(f64::NEG_INFINITY, f64::max);
let lunch_min = rows.iter().map(|r| r.lunch).fold(f64::INFINITY, f64::min); let lunch_min = rows.iter().map(|r| r.lunch).fold(f64::INFINITY, f64::min);
let lunch_max = rows.iter().map(|r| r.lunch).fold(f64::NEG_INFINITY, f64::max); let lunch_max = rows
let press_min = rows.iter().map(|r| r.presses).fold(f64::INFINITY, f64::min); .iter()
let press_max = rows.iter().map(|r| r.presses).fold(f64::NEG_INFINITY, f64::max); .map(|r| r.lunch)
.fold(f64::NEG_INFINITY, f64::max);
let after_min = rows.iter().map(|r| r.after).fold(f64::INFINITY, f64::min); let after_min = rows.iter().map(|r| r.after).fold(f64::INFINITY, f64::min);
let after_max = rows.iter().map(|r| r.after).fold(f64::NEG_INFINITY, f64::max); let after_max = rows
.iter()
.map(|r| r.after)
.fold(f64::NEG_INFINITY, f64::max);
rows.iter() rows.iter()
.map(|r| { .map(|r| {
// Hard comfort cap on presses.
if r.presses > PRESS_COMFORT_CAP {
return f64::NEG_INFINITY;
}
let work = norm_min(r.work_fail, work_min, work_max); let work = norm_min(r.work_fail, work_min, work_max);
let lunch = norm_max(r.lunch, lunch_min, lunch_max); let lunch = norm_max(r.lunch, lunch_min, lunch_max);
let press = norm_min(r.presses, press_min, press_max);
let after = norm_min(r.after, after_min, after_max); let after = norm_min(r.after, after_min, after_max);
W_WORK * work + W_LUNCH * lunch + W_PRESS * press + W_AFTER * after // Hinge: 1.0 at or below LOW, linear ramp to 0.0 at the cap.
let press_score = if r.presses <= PRESS_HINGE_LOW {
1.0
} else {
((PRESS_COMFORT_CAP - r.presses) / (PRESS_COMFORT_CAP - PRESS_HINGE_LOW))
.clamp(0.0, 1.0)
};
// Balance bonus: longer yellow + red is better, saturated.
let balance_score = balance_score_for(r);
W_LUNCH * lunch
+ W_AFTER * after
+ W_WORK * work
+ W_PRESS * press_score
+ W_BALANCE * balance_score
}) })
.collect() .collect()
} }
/// Balance bonus for a row: `min(YA - RA, RA - FRA)` clamped to
/// `[0, BALANCE_SATURATION_MIN]` and divided by saturation so the result is
/// in `[0, 1]`.
fn balance_score_for(r: &Row) -> f64 {
let yellow_w = (r.ya - r.ra) as f64;
let red_w = (r.ra - r.fra) as f64;
let raw = yellow_w.min(red_w).max(0.0);
(raw / BALANCE_SATURATION_MIN).clamp(0.0, 1.0)
}
/// Normalize a minimize-direction value to `[0, 1]` (best→1, worst→0). /// Normalize a minimize-direction value to `[0, 1]` (best→1, worst→0).
fn norm_min(v: f64, lo: f64, hi: f64) -> f64 { fn norm_min(v: f64, lo: f64, hi: f64) -> f64 {
if (hi - lo).abs() < 1e-12 { 1.0 } else { (hi - v) / (hi - lo) } if (hi - lo).abs() < 1e-12 {
1.0
} else {
(hi - v) / (hi - lo)
}
} }
/// Normalize a maximize-direction value to `[0, 1]` (best→1, worst→0). /// Normalize a maximize-direction value to `[0, 1]` (best→1, worst→0).
fn norm_max(v: f64, lo: f64, hi: f64) -> f64 { fn norm_max(v: f64, lo: f64, hi: f64) -> f64 {
if (hi - lo).abs() < 1e-12 { 1.0 } else { (v - lo) / (hi - lo) } if (hi - lo).abs() < 1e-12 {
1.0
} else {
(v - lo) / (hi - lo)
}
} }
/// Render `worst / best` as e.g. "7.5×" for the recommendation rationale. /// Render `worst / best` as e.g. "7.5×" for the recommendation rationale.
examples/random_search.rs
+5 -1
View File
@@ -24,7 +24,11 @@ impl Problem for Sphere2D {
fn main() { fn main() {
let initializer = RealBounds::new(vec![(-5.0, 5.0), (-5.0, 5.0)]); let initializer = RealBounds::new(vec![(-5.0, 5.0), (-5.0, 5.0)]);
let config = RandomSearchConfig { iterations: 500, batch_size: 1, seed: 7 }; let config = RandomSearchConfig {
iterations: 500,
batch_size: 1,
seed: 7,
};
let mut optimizer = RandomSearch::new(config, initializer); let mut optimizer = RandomSearch::new(config, initializer);
let result = optimizer.run(&Sphere2D); let result = optimizer.run(&Sphere2D);
examples/toy_nsga2.rs
+5 -1
View File
@@ -26,7 +26,11 @@ impl Problem for SchafferN1 {
fn main() { fn main() {
let initializer = RealBounds::new(vec![(-5.0, 5.0)]); let initializer = RealBounds::new(vec![(-5.0, 5.0)]);
let variation = GaussianMutation { sigma: 0.2 }; let variation = GaussianMutation { sigma: 0.2 };
let config = Nsga2Config { population_size: 60, generations: 80, seed: 42 }; let config = Nsga2Config {
population_size: 60,
generations: 80,
seed: 42,
};
let mut optimizer = Nsga2::new(config, initializer, variation); let mut optimizer = Nsga2::new(config, initializer, variation);
let result = optimizer.run(&SchafferN1); let result = optimizer.run(&SchafferN1);
fuzz/.gitignore
+4
View File
@@ -0,0 +1,4 @@
target
corpus
artifacts
coverage
fuzz/Cargo.lock
Generated
+254
View File
@@ -0,0 +1,254 @@
# This file is automatically @generated by Cargo.
# It is not intended for manual editing.
version = 4
[[package]]
name = "arbitrary"
version = "1.4.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c3d036a3c4ab069c7b410a2ce876bd74808d2d0888a82667669f8e783a898bf1"
dependencies = [
"derive_arbitrary",
]
[[package]]
name = "autocfg"
version = "1.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c08606f8c3cbf4ce6ec8e28fb0014a2c086708fe954eaa885384a6165172e7e8"
[[package]]
name = "cc"
version = "1.2.61"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d16d90359e986641506914ba71350897565610e87ce0ad9e6f28569db3dd5c6d"
dependencies = [
"find-msvc-tools",
"jobserver",
"libc",
"shlex",
]
[[package]]
name = "cfg-if"
version = "1.0.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801"
[[package]]
name = "derive_arbitrary"
version = "1.4.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1e567bd82dcff979e4b03460c307b3cdc9e96fde3d73bed1496d2bc75d9dd62a"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "find-msvc-tools"
version = "0.1.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5baebc0774151f905a1a2cc41989300b1e6fbb29aff0ceffa1064fdd3088d582"
[[package]]
name = "getrandom"
version = "0.3.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "899def5c37c4fd7b2664648c28120ecec138e4d395b459e5ca34f9cce2dd77fd"
dependencies = [
"cfg-if",
"libc",
"r-efi",
"wasip2",
]
[[package]]
name = "heuropt"
version = "0.3.0"
dependencies = [
"rand",
"rand_distr",
]
[[package]]
name = "heuropt-fuzz"
version = "0.0.0"
dependencies = [
"arbitrary",
"heuropt",
"libfuzzer-sys",
]
[[package]]
name = "jobserver"
version = "0.1.34"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9afb3de4395d6b3e67a780b6de64b51c978ecf11cb9a462c66be7d4ca9039d33"
dependencies = [
"getrandom",
"libc",
]
[[package]]
name = "libc"
version = "0.2.186"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "68ab91017fe16c622486840e4c83c9a37afeff978bd239b5293d61ece587de66"
[[package]]
name = "libfuzzer-sys"
version = "0.4.12"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f12a681b7dd8ce12bff52488013ba614b869148d54dd79836ab85aafdd53f08d"
dependencies = [
"arbitrary",
"cc",
]
[[package]]
name = "libm"
version = "0.2.16"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b6d2cec3eae94f9f509c767b45932f1ada8350c4bdb85af2fcab4a3c14807981"
[[package]]
name = "num-traits"
version = "0.2.19"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "071dfc062690e90b734c0b2273ce72ad0ffa95f0c74596bc250dcfd960262841"
dependencies = [
"autocfg",
"libm",
]
[[package]]
name = "ppv-lite86"
version = "0.2.21"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "85eae3c4ed2f50dcfe72643da4befc30deadb458a9b590d720cde2f2b1e97da9"
dependencies = [
"zerocopy",
]
[[package]]
name = "proc-macro2"
version = "1.0.106"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8fd00f0bb2e90d81d1044c2b32617f68fcb9fa3bb7640c23e9c748e53fb30934"
dependencies = [
"unicode-ident",
]
[[package]]
name = "quote"
version = "1.0.45"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "41f2619966050689382d2b44f664f4bc593e129785a36d6ee376ddf37259b924"
dependencies = [
"proc-macro2",
]
[[package]]
name = "r-efi"
version = "5.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "69cdb34c158ceb288df11e18b4bd39de994f6657d83847bdffdbd7f346754b0f"
[[package]]
name = "rand"
version = "0.9.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "44c5af06bb1b7d3216d91932aed5265164bf384dc89cd6ba05cf59a35f5f76ea"
dependencies = [
"rand_chacha",
"rand_core",
]
[[package]]
name = "rand_chacha"
version = "0.9.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d3022b5f1df60f26e1ffddd6c66e8aa15de382ae63b3a0c1bfc0e4d3e3f325cb"
dependencies = [
"ppv-lite86",
"rand_core",
]
[[package]]
name = "rand_core"
version = "0.9.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "76afc826de14238e6e8c374ddcc1fa19e374fd8dd986b0d2af0d02377261d83c"
dependencies = [
"getrandom",
]
[[package]]
name = "rand_distr"
version = "0.5.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6a8615d50dcf34fa31f7ab52692afec947c4dd0ab803cc87cb3b0b4570ff7463"
dependencies = [
"num-traits",
"rand",
]
[[package]]
name = "shlex"
version = "1.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0fda2ff0d084019ba4d7c6f371c95d8fd75ce3524c3cb8fb653a3023f6323e64"
[[package]]
name = "syn"
version = "2.0.117"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e665b8803e7b1d2a727f4023456bbbbe74da67099c585258af0ad9c5013b9b99"
dependencies = [
"proc-macro2",
"quote",
"unicode-ident",
]
[[package]]
name = "unicode-ident"
version = "1.0.24"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e6e4313cd5fcd3dad5cafa179702e2b244f760991f45397d14d4ebf38247da75"
[[package]]
name = "wasip2"
version = "1.0.3+wasi-0.2.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "20064672db26d7cdc89c7798c48a0fdfac8213434a1186e5ef29fd560ae223d6"
dependencies = [
"wit-bindgen",
]
[[package]]
name = "wit-bindgen"
version = "0.57.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1ebf944e87a7c253233ad6766e082e3cd714b5d03812acc24c318f549614536e"
[[package]]
name = "zerocopy"
version = "0.8.48"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "eed437bf9d6692032087e337407a86f04cd8d6a16a37199ed57949d415bd68e9"
dependencies = [
"zerocopy-derive",
]
[[package]]
name = "zerocopy-derive"
version = "0.8.48"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "70e3cd084b1788766f53af483dd21f93881ff30d7320490ec3ef7526d203bad4"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
fuzz/Cargo.toml
+69
View File
@@ -0,0 +1,69 @@
[package]
name = "heuropt-fuzz"
version = "0.0.0"
publish = false
edition = "2024"
[package.metadata]
cargo-fuzz = true
[dependencies]
libfuzzer-sys = "0.4"
arbitrary = { version = "1", features = ["derive"] }
heuropt = { path = ".." }
[[bin]]
name = "pareto_compare"
path = "fuzz_targets/pareto_compare.rs"
test = false
doc = false
bench = false
[[bin]]
name = "non_dominated_sort"
path = "fuzz_targets/non_dominated_sort.rs"
test = false
doc = false
bench = false
[[bin]]
name = "hypervolume_2d"
path = "fuzz_targets/hypervolume_2d.rs"
test = false
doc = false
bench = false
[[bin]]
name = "pareto_archive"
path = "fuzz_targets/pareto_archive.rs"
test = false
doc = false
bench = false
[[bin]]
name = "crowding_distance"
path = "fuzz_targets/crowding_distance.rs"
test = false
doc = false
bench = false
[[bin]]
name = "spacing"
path = "fuzz_targets/spacing.rs"
test = false
doc = false
bench = false
[[bin]]
name = "sbx_polymut"
path = "fuzz_targets/sbx_polymut.rs"
test = false
doc = false
bench = false
[[bin]]
name = "clamp_to_bounds"
path = "fuzz_targets/clamp_to_bounds.rs"
test = false
doc = false
bench = false
fuzz/fuzz_targets/clamp_to_bounds.rs
+88
View File
@@ -0,0 +1,88 @@
#![no_main]
//! Fuzz `ClampToBounds` + `ProjectToSimplex` repair operators for
//! idempotence and target-set membership.
use arbitrary::Arbitrary;
use libfuzzer_sys::fuzz_target;
use heuropt::prelude::*;
#[derive(Arbitrary, Debug)]
struct Input {
bounds: Vec<(f64, f64)>,
x: Vec<f64>,
simplex_total: f64,
}
fuzz_target!(|input: Input| {
if input.bounds.is_empty() || input.bounds.len() > 16 {
return;
}
if input.x.len() != input.bounds.len() {
return;
}
let bounds: Vec<(f64, f64)> = input
.bounds
.iter()
.filter_map(|&(lo, hi)| {
if lo.is_finite() && hi.is_finite() && lo < hi {
Some((lo, hi))
} else {
None
}
})
.collect();
if bounds.len() != input.bounds.len() {
return;
}
// Restrict to a numerically-reasonable magnitude range for repair
// operators — they are invoked downstream of evolutionary search where
// candidate magnitudes are bounded.
if input.x.iter().any(|v| !v.is_finite() || v.abs() > 1e30) {
return;
}
let mut x = input.x.clone();
let mut clamp = ClampToBounds::new(bounds.clone());
clamp.repair(&mut x);
for (j, &v) in x.iter().enumerate() {
let (lo, hi) = bounds[j];
assert!(v >= lo && v <= hi, "clamp out of bounds");
}
let after_one = x.clone();
clamp.repair(&mut x);
assert_eq!(x, after_one, "clamp not idempotent");
// Simplex projection only meaningful when total > 0 and dim >= 1.
// The Duchi/Held-Wolfe projection loses precision when |x| ≫ total
// (τ becomes indistinguishable from max(x) in f64). Restrict to inputs
// within the algorithm's well-conditioned regime, |x_i| ≤ total · 1e6.
let max_abs = input.x.iter().fold(0.0_f64, |a, &b| a.max(b.abs()));
if input.simplex_total.is_finite()
&& input.simplex_total > 1.0
&& input.simplex_total < 1e9
&& max_abs <= input.simplex_total * 1e6
{
let mut y = input.x.clone();
let mut proj = ProjectToSimplex::new(input.simplex_total);
proj.repair(&mut y);
for &v in &y {
assert!(v >= 0.0, "project negative entry");
}
let s: f64 = y.iter().sum();
assert!(
(s - input.simplex_total).abs() < 1e-6 * input.simplex_total.max(1.0),
"project sum {s} != target {}",
input.simplex_total,
);
let after = y.clone();
proj.repair(&mut y);
for (a, b) in after.iter().zip(y.iter()) {
let scale = a.abs().max(b.abs()).max(1.0);
assert!(
(a - b).abs() < 1e-9 * scale,
"project not idempotent: {a} vs {b}",
);
}
}
});
fuzz/fuzz_targets/crowding_distance.rs
+50
View File
@@ -0,0 +1,50 @@
#![no_main]
//! Fuzz `crowding_distance` for shape and non-negativity.
use arbitrary::Arbitrary;
use libfuzzer_sys::fuzz_target;
use heuropt::core::candidate::Candidate;
use heuropt::core::evaluation::Evaluation;
use heuropt::core::objective::{Objective, ObjectiveSpace};
use heuropt::pareto::crowding::crowding_distance;
#[derive(Arbitrary, Debug)]
struct Input {
points: Vec<(f64, f64)>,
}
fuzz_target!(|input: Input| {
if input.points.len() > 64 {
return;
}
// Bound magnitudes — crowding's `(max - min)` and per-axis gaps can
// both overflow to +∞ when points span ±f64::MAX, yielding inf/inf=NaN.
if input
.points
.iter()
.any(|&(a, b)| !a.is_finite() || !b.is_finite() || a.abs() > 1e150 || b.abs() > 1e150)
{
return;
}
let space = ObjectiveSpace::new(vec![Objective::minimize("f1"), Objective::minimize("f2")]);
let pop: Vec<Candidate<()>> = input
.points
.iter()
.map(|&(a, b)| Candidate::new((), Evaluation::new(vec![a, b])))
.collect();
let front: Vec<usize> = (0..pop.len()).collect();
let d = crowding_distance(&pop, &front, &space);
assert_eq!(d.len(), front.len(), "crowding distance length mismatch");
for (i, &v) in d.iter().enumerate() {
assert!(v >= 0.0 || v.is_infinite(), "negative crowding[{i}] = {v}");
assert!(!v.is_nan(), "NaN crowding[{i}]");
}
// If size <= 2, every entry is +∞.
if pop.len() <= 2 {
for (i, &v) in d.iter().enumerate() {
assert!(v.is_infinite(), "size<=2 crowding[{i}] not inf: {v}");
}
}
});
fuzz/fuzz_targets/hypervolume_2d.rs
+47
View File
@@ -0,0 +1,47 @@
#![no_main]
//! Fuzz `hypervolume_2d` for non-negativity and reference-point handling.
use arbitrary::Arbitrary;
use libfuzzer_sys::fuzz_target;
use heuropt::core::candidate::Candidate;
use heuropt::core::evaluation::Evaluation;
use heuropt::core::objective::{Objective, ObjectiveSpace};
use heuropt::metrics::hypervolume::hypervolume_2d;
#[derive(Arbitrary, Debug)]
struct Input {
points: Vec<(f64, f64)>,
ref_point: (f64, f64),
}
fuzz_target!(|input: Input| {
if input.points.len() > 64 {
return;
}
// Non-finite floats are permitted by Evaluation, but HV is undefined
// there — restrict to finite for this property.
if !input.ref_point.0.is_finite() || !input.ref_point.1.is_finite() {
return;
}
if input
.points
.iter()
.any(|&(a, b)| !a.is_finite() || !b.is_finite())
{
return;
}
let space = ObjectiveSpace::new(vec![Objective::minimize("f1"), Objective::minimize("f2")]);
let pop: Vec<Candidate<()>> = input
.points
.iter()
.map(|&(a, b)| Candidate::new((), Evaluation::new(vec![a, b])))
.collect();
let hv = hypervolume_2d(&pop, &space, [input.ref_point.0, input.ref_point.1]);
// HV can be +∞ when the dominated rectangle area overflows f64 (e.g. a
// ref point at f64::MAX with deeply negative front coords). The
// contracted invariants are non-negativity and non-NaN.
assert!(hv >= 0.0, "HV negative: {hv}");
assert!(!hv.is_nan(), "HV is NaN");
});
fuzz/fuzz_targets/non_dominated_sort.rs
+62
View File
@@ -0,0 +1,62 @@
#![no_main]
//! Fuzz `non_dominated_sort` for partition correctness.
//!
//! Invariants checked:
//! * Every population index appears in exactly one front.
//! * Earlier fronts dominate later fronts (no backwards domination).
//! * No panics on any vector of finite or non-finite objective values.
use arbitrary::Arbitrary;
use libfuzzer_sys::fuzz_target;
use heuropt::core::candidate::Candidate;
use heuropt::core::evaluation::Evaluation;
use heuropt::core::objective::{Objective, ObjectiveSpace};
use heuropt::pareto::dominance::{Dominance, pareto_compare};
use heuropt::pareto::sort::non_dominated_sort;
#[derive(Arbitrary, Debug)]
struct Input {
objectives: Vec<(f64, f64)>,
}
fuzz_target!(|input: Input| {
if input.objectives.is_empty() || input.objectives.len() > 32 {
return;
}
let space = ObjectiveSpace::new(vec![Objective::minimize("f1"), Objective::minimize("f2")]);
let pop: Vec<Candidate<()>> = input
.objectives
.iter()
.map(|&(a, b)| Candidate::new((), Evaluation::new(vec![a, b])))
.collect();
let fronts = non_dominated_sort(&pop, &space);
// Partition: every index appears exactly once.
let mut seen = vec![false; pop.len()];
for front in &fronts {
for &idx in front {
assert!(!seen[idx], "index {idx} in multiple fronts");
seen[idx] = true;
}
}
for (i, &was) in seen.iter().enumerate() {
assert!(was, "index {i} missing from all fronts");
}
// Earlier fronts cannot be dominated by later fronts.
for (k, fk) in fronts.iter().enumerate() {
for fl in fronts.iter().skip(k + 1) {
for &i in fk {
for &j in fl {
let r = pareto_compare(&pop[i].evaluation, &pop[j].evaluation, &space);
assert!(
!matches!(r, Dominance::DominatedBy),
"front-{k}/{i} dominated by later front",
);
}
}
}
}
});
fuzz/fuzz_targets/pareto_archive.rs
+58
View File
@@ -0,0 +1,58 @@
#![no_main]
//! Fuzz `ParetoArchive` for the non-domination invariant under arbitrary
//! insertion/truncation sequences.
use arbitrary::Arbitrary;
use libfuzzer_sys::fuzz_target;
use heuropt::core::candidate::Candidate;
use heuropt::core::evaluation::Evaluation;
use heuropt::core::objective::{Objective, ObjectiveSpace};
use heuropt::pareto::archive::ParetoArchive;
use heuropt::pareto::dominance::{Dominance, pareto_compare};
#[derive(Arbitrary, Debug)]
enum Op {
Insert(f64, f64),
Truncate(u8),
}
#[derive(Arbitrary, Debug)]
struct Input {
ops: Vec<Op>,
}
fuzz_target!(|input: Input| {
if input.ops.len() > 64 {
return;
}
let space = ObjectiveSpace::new(vec![Objective::minimize("f1"), Objective::minimize("f2")]);
let mut archive: ParetoArchive<()> = ParetoArchive::new(space.clone());
for op in input.ops {
match op {
Op::Insert(a, b) => {
let cand = Candidate::new((), Evaluation::new(vec![a, b]));
archive.insert(cand);
}
Op::Truncate(n) => archive.truncate(n as usize),
}
}
// Members must be pairwise non-dominated.
let m = archive.members();
for i in 0..m.len() {
for j in 0..m.len() {
if i == j {
continue;
}
let r = pareto_compare(&m[i].evaluation, &m[j].evaluation, &space);
assert!(
!matches!(r, Dominance::DominatedBy),
"archive member {i} dominated by {j}: {:?} vs {:?}",
m[i].evaluation.objectives,
m[j].evaluation.objectives,
);
}
}
});
fuzz/fuzz_targets/pareto_compare.rs
+68
View File
@@ -0,0 +1,68 @@
#![no_main]
//! Fuzz `pareto_compare` for anti-symmetry and reflexivity.
//!
//! Invariants checked:
//! * `compare(a, b)` and `compare(b, a)` form an anti-symmetric pair
//! (`Dominates ↔ DominatedBy`, `Equal ↔ Equal`, `NonDominated ↔ NonDominated`).
//! * `compare(a, a) == Equal`.
//! * No panics on any combination of finite/non-finite floats.
use arbitrary::Arbitrary;
use libfuzzer_sys::fuzz_target;
use heuropt::core::evaluation::Evaluation;
use heuropt::core::objective::{Objective, ObjectiveSpace};
use heuropt::pareto::dominance::{Dominance, pareto_compare};
#[derive(Arbitrary, Debug)]
struct Input {
a_objs: Vec<f64>,
b_objs: Vec<f64>,
a_violation: f64,
b_violation: f64,
minimize_mask: u8,
}
fuzz_target!(|input: Input| {
if input.a_objs.is_empty() || input.a_objs.len() != input.b_objs.len() {
return;
}
if input.a_objs.len() > 8 {
return;
}
let m = input.a_objs.len();
let space = ObjectiveSpace::new(
(0..m)
.map(|i| {
if (input.minimize_mask >> i) & 1 == 0 {
Objective::minimize(format!("f{i}"))
} else {
Objective::maximize(format!("f{i}"))
}
})
.collect(),
);
let a = Evaluation::constrained(input.a_objs.clone(), input.a_violation);
let b = Evaluation::constrained(input.b_objs.clone(), input.b_violation);
let ab = pareto_compare(&a, &b, &space);
let ba = pareto_compare(&b, &a, &space);
let aa = pareto_compare(&a, &a, &space);
// Anti-symmetry pairs.
let antisymmetric = matches!(
(ab, ba),
(Dominance::Dominates, Dominance::DominatedBy)
| (Dominance::DominatedBy, Dominance::Dominates)
| (Dominance::Equal, Dominance::Equal)
| (Dominance::NonDominated, Dominance::NonDominated),
);
assert!(antisymmetric, "asymmetric: ab={ab:?}, ba={ba:?}");
// Reflexivity (when objectives are finite — NaNs make equality
// ill-defined, so skip the check there).
if input.a_objs.iter().all(|v| v.is_finite()) && input.a_violation.is_finite() {
assert_eq!(aa, Dominance::Equal);
}
});
fuzz/fuzz_targets/sbx_polymut.rs
+99
View File
@@ -0,0 +1,99 @@
#![no_main]
//! Fuzz SBX + PolynomialMutation: in-bounds parents must produce in-bounds
//! children for any seed and any (η, per-variable-probability) pair.
use arbitrary::Arbitrary;
use libfuzzer_sys::fuzz_target;
use heuropt::core::rng::rng_from_seed;
use heuropt::prelude::*;
#[derive(Arbitrary, Debug)]
struct Input {
bounds: Vec<(f64, f64)>,
eta_sbx: f64,
eta_pm: f64,
pvp_sbx: f64,
pvp_pm: f64,
a_frac: Vec<f64>,
b_frac: Vec<f64>,
seed: u64,
}
fuzz_target!(|input: Input| {
let n = input.bounds.len();
if n == 0 || n > 8 {
return;
}
if !(input.eta_sbx.is_finite() && input.eta_pm.is_finite()) {
return;
}
if !(input.eta_sbx >= 1.0 && input.eta_sbx <= 100.0) {
return;
}
if !(input.eta_pm >= 1.0 && input.eta_pm <= 100.0) {
return;
}
let pvp_sbx = match input.pvp_sbx {
v if v.is_finite() && (0.0..=1.0).contains(&v) => v,
_ => return,
};
let pvp_pm = match input.pvp_pm {
v if v.is_finite() && (0.0..=1.0).contains(&v) => v,
_ => return,
};
// Sanitize bounds: lo < hi, finite.
let bounds: Vec<(f64, f64)> = input
.bounds
.iter()
.filter_map(|&(lo, hi)| {
if lo.is_finite() && hi.is_finite() && hi - lo > 1e-9 {
Some((lo, hi))
} else {
None
}
})
.collect();
if bounds.len() != n {
return;
}
if input.a_frac.len() < n || input.b_frac.len() < n {
return;
}
let p1: Vec<f64> = bounds
.iter()
.zip(&input.a_frac)
.map(|(&(lo, hi), &f)| {
let frac = if f.is_finite() { f.fract().abs() } else { 0.5 };
lo + frac * (hi - lo)
})
.collect();
let p2: Vec<f64> = bounds
.iter()
.zip(&input.b_frac)
.map(|(&(lo, hi), &f)| {
let frac = if f.is_finite() { f.fract().abs() } else { 0.5 };
lo + frac * (hi - lo)
})
.collect();
let mut rng = rng_from_seed(input.seed);
let mut sbx = SimulatedBinaryCrossover::new(bounds.clone(), input.eta_sbx, pvp_sbx);
let kids = sbx.vary(&[p1, p2], &mut rng);
assert_eq!(kids.len(), 2);
for c in &kids {
for (j, &v) in c.iter().enumerate() {
let (lo, hi) = bounds[j];
assert!(v >= lo && v <= hi, "SBX child[{j}] = {v} out of [{lo}, {hi}]");
}
}
let mut pm = PolynomialMutation::new(bounds.clone(), input.eta_pm, pvp_pm);
let mutated = pm.vary(std::slice::from_ref(&kids[0]), &mut rng);
assert_eq!(mutated.len(), 1);
for (j, &v) in mutated[0].iter().enumerate() {
let (lo, hi) = bounds[j];
assert!(v >= lo && v <= hi, "PM child[{j}] = {v} out of [{lo}, {hi}]");
}
});
fuzz/fuzz_targets/spacing.rs
+42
View File
@@ -0,0 +1,42 @@
#![no_main]
//! Fuzz the `spacing` metric for non-negativity.
use arbitrary::Arbitrary;
use libfuzzer_sys::fuzz_target;
use heuropt::core::candidate::Candidate;
use heuropt::core::evaluation::Evaluation;
use heuropt::core::objective::{Objective, ObjectiveSpace};
use heuropt::metrics::spacing::spacing;
#[derive(Arbitrary, Debug)]
struct Input {
points: Vec<(f64, f64)>,
}
fuzz_target!(|input: Input| {
if input.points.len() > 64 {
return;
}
// Bound magnitudes so distance computations don't overflow to
// inf-inf=NaN — `spacing` is documented to operate on values produced
// by `as_minimization` of problem evaluations, not arbitrary f64s.
if input
.points
.iter()
.any(|&(a, b)| !a.is_finite() || !b.is_finite() || a.abs() > 1e150 || b.abs() > 1e150)
{
return;
}
let space = ObjectiveSpace::new(vec![Objective::minimize("f1"), Objective::minimize("f2")]);
let pop: Vec<Candidate<()>> = input
.points
.iter()
.map(|&(a, b)| Candidate::new((), Evaluation::new(vec![a, b])))
.collect();
let s = spacing(&pop, &space);
// Spacing can overflow to +∞ when point coordinates straddle ±f64::MAX.
// Contract is non-negative + non-NaN.
assert!(s >= 0.0, "spacing negative: {s}");
assert!(!s.is_nan(), "spacing is NaN");
});
src/algorithms/age_moea.rs
+399
View File
@@ -0,0 +1,399 @@
//! `AgeMoea` — Panichella 2019 Adaptive Geometry Estimation MOEA.
use rand::Rng as _;
use crate::algorithms::parallel_eval::evaluate_batch;
use crate::core::candidate::Candidate;
use crate::core::objective::ObjectiveSpace;
use crate::core::population::Population;
use crate::core::problem::Problem;
use crate::core::result::OptimizationResult;
use crate::core::rng::rng_from_seed;
use crate::pareto::front::{best_candidate, pareto_front};
use crate::pareto::sort::non_dominated_sort;
use crate::traits::{Initializer, Optimizer, Variation};
/// Configuration for [`AgeMoea`].
#[derive(Debug, Clone)]
pub struct AgeMoeaConfig {
/// Constant population size.
pub population_size: usize,
/// Number of generations.
pub generations: usize,
/// Seed for the deterministic RNG.
pub seed: u64,
}
impl Default for AgeMoeaConfig {
fn default() -> Self {
Self {
population_size: 100,
generations: 250,
seed: 42,
}
}
}
/// Adaptive Geometry Estimation MOEA.
///
/// Estimates the current front's L_p geometry parameter and uses it to
/// score survivors by a combination of proximity (distance to the
/// translated origin in the L_p frame) and diversity (distance to the
/// nearest survivor in the same frame).
#[derive(Debug, Clone)]
pub struct AgeMoea<I, V> {
/// Algorithm configuration.
pub config: AgeMoeaConfig,
/// Initial-decision sampler.
pub initializer: I,
/// Offspring-producing variation operator.
pub variation: V,
}
impl<I, V> AgeMoea<I, V> {
/// Construct an `AgeMoea`.
pub fn new(config: AgeMoeaConfig, initializer: I, variation: V) -> Self {
Self {
config,
initializer,
variation,
}
}
}
impl<P, I, V> Optimizer<P> for AgeMoea<I, V>
where
P: Problem + Sync,
P::Decision: Send,
I: Initializer<P::Decision>,
V: Variation<P::Decision>,
{
fn run(&mut self, problem: &P) -> OptimizationResult<P::Decision> {
assert!(
self.config.population_size > 0,
"AgeMoea population_size must be > 0"
);
let n = self.config.population_size;
let objectives = problem.objectives();
let mut rng = rng_from_seed(self.config.seed);
let initial_decisions = self.initializer.initialize(n, &mut rng);
let mut population: Vec<Candidate<P::Decision>> =
evaluate_batch(problem, initial_decisions);
let mut evaluations = population.len();
for _ in 0..self.config.generations {
// Phase 1: random parent selection + variation.
let mut offspring_decisions: Vec<P::Decision> = Vec::with_capacity(n);
while offspring_decisions.len() < n {
let p1 = rng.random_range(0..population.len());
let p2 = rng.random_range(0..population.len());
let parents = vec![
population[p1].decision.clone(),
population[p2].decision.clone(),
];
let children = self.variation.vary(&parents, &mut rng);
assert!(
!children.is_empty(),
"AgeMoea variation returned no children"
);
for child in children {
if offspring_decisions.len() >= n {
break;
}
offspring_decisions.push(child);
}
}
let offspring = evaluate_batch(problem, offspring_decisions);
evaluations += offspring.len();
// Phase 3: combine + age-moea survival selection.
let mut combined: Vec<Candidate<P::Decision>> = Vec::with_capacity(2 * n);
combined.extend(population);
combined.extend(offspring);
population = environmental_selection(combined, &objectives, n);
}
let front = pareto_front(&population, &objectives);
let best = best_candidate(&population, &objectives);
OptimizationResult::new(
Population::new(population),
front,
best,
evaluations,
self.config.generations,
)
}
}
fn environmental_selection<D: Clone>(
combined: Vec<Candidate<D>>,
objectives: &ObjectiveSpace,
n: usize,
) -> Vec<Candidate<D>> {
let fronts = non_dominated_sort(&combined, objectives);
let mut selected: Vec<usize> = Vec::with_capacity(n);
let mut splitting: Vec<usize> = Vec::new();
for f in &fronts {
if selected.len() + f.len() <= n {
selected.extend(f.iter().copied());
} else {
splitting = f.clone();
break;
}
if selected.len() == n {
break;
}
}
if selected.len() == n {
return selected.into_iter().map(|i| combined[i].clone()).collect();
}
// Translate by ideal point z*.
let m = objectives.len();
let n0_oriented: Vec<Vec<f64>> = fronts[0]
.iter()
.map(|&i| objectives.as_minimization(&combined[i].evaluation.objectives))
.collect();
let mut ideal = vec![f64::INFINITY; m];
for o in &n0_oriented {
for (k, v) in o.iter().enumerate() {
if *v < ideal[k] {
ideal[k] = *v;
}
}
}
// Translate every combined member.
let translated: Vec<Vec<f64>> = combined
.iter()
.map(|c| {
let oriented = objectives.as_minimization(&c.evaluation.objectives);
oriented
.iter()
.enumerate()
.map(|(k, v)| (v - ideal[k]).max(0.0))
.collect()
})
.collect();
// Estimate p (geometry parameter) from the *first* front's
// extreme points: find the point with the largest single-axis value
// for each axis, then solve for p such that all extreme points have
// unit L_p norm after normalizing by the per-axis maximum.
let p = estimate_p(&fronts[0], &translated, m);
// Score every member of the splitting front by:
// proximity = ||translated||_p
// diversity = nearest-neighbor distance in the same L_p frame
// among already-selected + splitting members.
//
// Two caches make this much cheaper than the textbook formulation:
// * `prox[i]` — `lp_norm(translated[i], p)` is constant across
// iterations, so compute it once per splitting-front member.
// * `nearest[i]` — the nearest-keep distance only ever decreases
// when a new candidate is picked, so we maintain it
// incrementally: seed it from `selected`, then on every pick
// update each remaining `i`'s nearest by taking
// `min(nearest[i], lp_distance(translated[i], translated[pick], p))`.
//
// That cuts the score loop from O(R · K · M) per iteration (where
// R = remaining count, K = current keep count) to O(R · M) per
// iteration, with the dominant `powf` calls in lp_distance counted
// once per (remaining, pick) pair instead of per (remaining, all-keep).
let mut keep = selected.clone();
let mut remaining: Vec<usize> = splitting.clone();
let prox: Vec<f64> = (0..combined.len())
.map(|i| lp_norm(&translated[i], p))
.collect();
let mut nearest: Vec<f64> = (0..combined.len())
.map(|i| nearest_neighbor_distance(i, &translated, &keep, p))
.collect();
while keep.len() < n {
// Pick the remaining candidate with the largest score.
let mut best_idx: Option<usize> = None;
let mut best_score = f64::NEG_INFINITY;
for &i in &remaining {
let score = nearest[i] / (prox[i].max(1e-12));
if score > best_score {
best_score = score;
best_idx = Some(i);
}
}
match best_idx {
None => break,
Some(pick) => {
keep.push(pick);
remaining.retain(|&i| i != pick);
// Update each surviving remaining's nearest-keep using
// just the distance to the new pick.
for &i in &remaining {
let d = lp_distance(&translated[i], &translated[pick], p);
if d < nearest[i] {
nearest[i] = d;
}
}
}
}
}
keep.into_iter().map(|i| combined[i].clone()).collect()
}
fn lp_norm(v: &[f64], p: f64) -> f64 {
v.iter().map(|x| x.abs().powf(p)).sum::<f64>().powf(1.0 / p)
}
fn lp_distance(a: &[f64], b: &[f64], p: f64) -> f64 {
a.iter()
.zip(b.iter())
.map(|(x, y)| (x - y).abs().powf(p))
.sum::<f64>()
.powf(1.0 / p)
}
fn nearest_neighbor_distance(i: usize, translated: &[Vec<f64>], selected: &[usize], p: f64) -> f64 {
if selected.is_empty() {
return f64::INFINITY;
}
let mut best = f64::INFINITY;
for &j in selected {
if j == i {
continue;
}
let d = lp_distance(&translated[i], &translated[j], p);
if d < best {
best = d;
}
}
best
}
/// Estimate the L_p geometry parameter from the front's extreme points.
///
/// Find the extreme point on each axis (the front member maximizing that
/// objective relative to its own L_∞ norm), then choose p such that all
/// extreme points have approximately the same L_p magnitude. Falls back
/// to p = 2 (spherical) if anything degenerates.
fn estimate_p(front_indices: &[usize], translated: &[Vec<f64>], m: usize) -> f64 {
if front_indices.is_empty() || m == 0 {
return 2.0;
}
// For each axis, find the extreme: the front member with the largest
// ratio of its k-th coordinate to its own L1 norm (i.e., the most
// "k-aligned" member).
let extremes: Vec<usize> = (0..m)
.map(|axis| {
let mut best = front_indices[0];
let mut best_ratio = f64::NEG_INFINITY;
for &idx in front_indices {
let l1: f64 = translated[idx].iter().sum::<f64>().max(1e-12);
let ratio = translated[idx][axis] / l1;
if ratio > best_ratio {
best_ratio = ratio;
best = idx;
}
}
best
})
.collect();
// Solve for p ∈ [0.1, 10.0] that minimizes std-dev of L_p norms across
// extremes (a coarse sweep is fine — full Brent isn't needed for this
// shape estimate).
let candidates: Vec<f64> = (1..=40).map(|i| (i as f64) * 0.25).collect();
let mut best_p = 2.0;
let mut best_loss = f64::INFINITY;
for &p in &candidates {
let norms: Vec<f64> = extremes
.iter()
.map(|&i| lp_norm(&translated[i], p))
.collect();
let mean = norms.iter().sum::<f64>() / norms.len() as f64;
if mean.is_finite() && mean > 0.0 {
let var = norms.iter().map(|x| (x - mean).powi(2)).sum::<f64>() / norms.len() as f64;
let loss = var.sqrt() / mean;
if loss < best_loss {
best_loss = loss;
best_p = p;
}
}
}
best_p
}
#[cfg(test)]
mod tests {
use super::*;
use crate::operators::{
CompositeVariation, PolynomialMutation, RealBounds, SimulatedBinaryCrossover,
};
use crate::tests_support::SchafferN1;
fn make_optimizer(
seed: u64,
) -> AgeMoea<RealBounds, CompositeVariation<SimulatedBinaryCrossover, PolynomialMutation>> {
let bounds = vec![(-5.0, 5.0)];
let initializer = RealBounds::new(bounds.clone());
let variation = CompositeVariation {
crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5),
mutation: PolynomialMutation::new(bounds, 20.0, 1.0),
};
AgeMoea::new(
AgeMoeaConfig {
population_size: 20,
generations: 15,
seed,
},
initializer,
variation,
)
}
#[test]
fn produces_pareto_front() {
let mut opt = make_optimizer(1);
let r = opt.run(&SchafferN1);
assert_eq!(r.population.len(), 20);
assert!(!r.pareto_front.is_empty());
}
#[test]
fn deterministic_with_same_seed() {
let mut a = make_optimizer(99);
let mut b = make_optimizer(99);
let ra = a.run(&SchafferN1);
let rb = b.run(&SchafferN1);
let oa: Vec<Vec<f64>> = ra
.pareto_front
.iter()
.map(|c| c.evaluation.objectives.clone())
.collect();
let ob: Vec<Vec<f64>> = rb
.pareto_front
.iter()
.map(|c| c.evaluation.objectives.clone())
.collect();
assert_eq!(oa, ob);
}
#[test]
#[should_panic(expected = "population_size must be > 0")]
fn zero_pop_panics() {
let bounds = vec![(0.0, 1.0)];
let initializer = RealBounds::new(bounds.clone());
let variation = CompositeVariation {
crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5),
mutation: PolynomialMutation::new(bounds, 20.0, 1.0),
};
let mut opt = AgeMoea::new(
AgeMoeaConfig {
population_size: 0,
generations: 1,
seed: 0,
},
initializer,
variation,
);
let _ = opt.run(&SchafferN1);
}
}
src/algorithms/ant_colony_tsp.rs
+396
View File
@@ -0,0 +1,396 @@
//! `AntColonyTsp` — Dorigo-style Ant System for permutation problems on a
//! complete graph (TSP-style).
use rand::Rng as _;
use crate::core::candidate::Candidate;
use crate::core::objective::Direction;
use crate::core::population::Population;
use crate::core::problem::Problem;
use crate::core::result::OptimizationResult;
use crate::core::rng::rng_from_seed;
use crate::traits::Optimizer;
/// Configuration for [`AntColonyTsp`].
#[derive(Debug, Clone)]
pub struct AntColonyTspConfig {
/// Number of ants per generation.
pub ants: usize,
/// Number of generations.
pub generations: usize,
/// Pheromone weight `α`.
pub alpha: f64,
/// Heuristic weight `β`.
pub beta: f64,
/// Pheromone evaporation rate `ρ` ∈ [0, 1].
pub evaporation: f64,
/// Pheromone deposit constant `Q`. Reinforcement on edge (i, j) is
/// `Q / tour_length` for every ant whose tour uses (i, j).
pub deposit: f64,
/// Initial pheromone level on every edge.
pub initial_pheromone: f64,
/// Seed for the deterministic RNG.
pub seed: u64,
}
impl Default for AntColonyTspConfig {
fn default() -> Self {
Self {
ants: 30,
generations: 100,
alpha: 1.0,
beta: 2.0,
evaporation: 0.5,
deposit: 1.0,
initial_pheromone: 1.0,
seed: 42,
}
}
}
/// Ant Colony Optimization for permutation-style problems on a complete graph.
///
/// `Vec<usize>` decisions only (the permutation `[0, 1, …, n_cities - 1]`).
/// Single-objective only — typically minimizing total tour length, but the
/// algorithm is direction-aware for completeness.
///
/// Each ant builds a tour by repeatedly choosing the next node with
/// probability `∝ τ_ij^α · η_ij^β` over the unvisited cities, where
/// `η_ij = 1 / distance_ij` is the heuristic desirability.
pub struct AntColonyTsp {
/// Algorithm configuration.
pub config: AntColonyTspConfig,
/// Symmetric distance matrix; size `n_cities × n_cities`. Diagonal must
/// be zero.
pub distances: Vec<Vec<f64>>,
}
impl AntColonyTsp {
/// Construct an `AntColonyTsp`. Validates that `distances` is square
/// and has a zero diagonal.
pub fn new(config: AntColonyTspConfig, distances: Vec<Vec<f64>>) -> Self {
let n = distances.len();
assert!(
n >= 2,
"AntColonyTsp distances matrix must have >= 2 cities"
);
for (i, row) in distances.iter().enumerate() {
assert_eq!(row.len(), n, "AntColonyTsp distances matrix must be square");
assert_eq!(
row[i], 0.0,
"AntColonyTsp distance from city to itself must be 0"
);
}
Self { config, distances }
}
}
impl<P> Optimizer<P> for AntColonyTsp
where
P: Problem<Decision = Vec<usize>> + Sync,
{
fn run(&mut self, problem: &P) -> OptimizationResult<P::Decision> {
assert!(self.config.ants >= 1, "AntColonyTsp ants must be >= 1");
let objectives = problem.objectives();
assert!(
objectives.is_single_objective(),
"AntColonyTsp requires exactly one objective",
);
let direction = objectives.objectives[0].direction;
let n = self.distances.len();
let mut rng = rng_from_seed(self.config.seed);
// Heuristic desirability: 1 / distance (with a small floor to avoid
// division by zero for very-close cities).
let eta: Vec<Vec<f64>> = self
.distances
.iter()
.map(|row| {
row.iter()
.map(|&d| if d > 0.0 { 1.0 / d } else { 0.0 })
.collect()
})
.collect();
// Pheromone matrix.
let mut pheromone: Vec<Vec<f64>> = vec![vec![self.config.initial_pheromone; n]; n];
let mut best_decision: Option<Vec<usize>> = None;
let mut best_eval: Option<crate::core::evaluation::Evaluation> = None;
let mut evaluations = 0usize;
for _ in 0..self.config.generations {
let mut tours: Vec<Vec<usize>> = Vec::with_capacity(self.config.ants);
let mut tour_evals: Vec<crate::core::evaluation::Evaluation> =
Vec::with_capacity(self.config.ants);
for _ in 0..self.config.ants {
let start = rng.random_range(0..n);
let tour = build_tour(
n,
start,
&pheromone,
&eta,
self.config.alpha,
self.config.beta,
&mut rng,
);
let eval = problem.evaluate(&tour);
evaluations += 1;
tours.push(tour);
tour_evals.push(eval);
}
// Update best.
for (tour, eval) in tours.iter().zip(tour_evals.iter()) {
let beats = match &best_eval {
None => true,
Some(b) => better_than_so(eval, b, direction),
};
if beats {
best_decision = Some(tour.clone());
best_eval = Some(eval.clone());
}
}
// Pheromone evaporation.
for row in pheromone.iter_mut() {
for v in row.iter_mut() {
*v *= 1.0 - self.config.evaporation;
}
}
// Pheromone deposit on each ant's tour.
for (tour, eval) in tours.iter().zip(tour_evals.iter()) {
let length = eval
.objectives
.first()
.copied()
.unwrap_or(f64::INFINITY)
.max(1e-12);
let deposit = self.config.deposit / length;
for w in tour.windows(2) {
let (i, j) = (w[0], w[1]);
pheromone[i][j] += deposit;
pheromone[j][i] += deposit;
}
// Close the loop.
let (i, j) = (*tour.last().unwrap(), tour[0]);
pheromone[i][j] += deposit;
pheromone[j][i] += deposit;
}
}
let best = Candidate::new(best_decision.unwrap(), best_eval.unwrap());
let population = Population::new(vec![best.clone()]);
let front = vec![best.clone()];
OptimizationResult::new(
population,
front,
Some(best),
evaluations,
self.config.generations,
)
}
}
fn build_tour(
n: usize,
start: usize,
pheromone: &[Vec<f64>],
eta: &[Vec<f64>],
alpha: f64,
beta: f64,
rng: &mut crate::core::rng::Rng,
) -> Vec<usize> {
let mut tour = Vec::with_capacity(n);
let mut visited = vec![false; n];
tour.push(start);
visited[start] = true;
for _ in 1..n {
let current = *tour.last().unwrap();
// Build a probability vector over the unvisited candidates.
let probs: Vec<(usize, f64)> = (0..n)
.filter(|&j| !visited[j])
.map(|j| {
let p = pheromone[current][j].max(0.0).powf(alpha) * eta[current][j].powf(beta);
(j, p)
})
.collect();
let total: f64 = probs.iter().map(|(_, p)| *p).sum();
let next = if total > 0.0 {
let r: f64 = rng.random::<f64>() * total;
let mut acc = 0.0;
let mut chosen = probs.last().unwrap().0;
for (j, p) in &probs {
acc += *p;
if r <= acc {
chosen = *j;
break;
}
}
chosen
} else {
// Degenerate case: pheromone × heuristic is 0 for every
// unvisited city. Fall back to uniform random.
let &(j, _) = probs.choose_uniform(rng);
j
};
let _ = probs;
tour.push(next);
visited[next] = true;
}
tour
}
trait ChooseUniform<T> {
fn choose_uniform(&self, rng: &mut crate::core::rng::Rng) -> &T;
}
impl<T> ChooseUniform<T> for [T] {
fn choose_uniform(&self, rng: &mut crate::core::rng::Rng) -> &T {
&self[rng.random_range(0..self.len())]
}
}
fn better_than_so(
a: &crate::core::evaluation::Evaluation,
b: &crate::core::evaluation::Evaluation,
direction: Direction,
) -> bool {
match (a.is_feasible(), b.is_feasible()) {
(true, false) => true,
(false, true) => false,
(false, false) => a.constraint_violation < b.constraint_violation,
(true, true) => match direction {
Direction::Minimize => a.objectives[0] < b.objectives[0],
Direction::Maximize => a.objectives[0] > b.objectives[0],
},
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::core::evaluation::Evaluation;
use crate::core::objective::{Objective, ObjectiveSpace};
/// A 5-city ring problem: cities placed at `(cos(2πi/5), sin(2πi/5))`.
/// Optimal tour length: 2·5·sin(π/5) ≈ 5.878 (a regular pentagon).
struct RingTsp {
distances: Vec<Vec<f64>>,
}
impl RingTsp {
fn new(n: usize) -> Self {
use std::f64::consts::PI;
let pts: Vec<(f64, f64)> = (0..n)
.map(|i| {
let a = 2.0 * PI * (i as f64) / (n as f64);
(a.cos(), a.sin())
})
.collect();
let distances = (0..n)
.map(|i| {
(0..n)
.map(|j| {
let (xi, yi) = pts[i];
let (xj, yj) = pts[j];
((xi - xj).powi(2) + (yi - yj).powi(2)).sqrt()
})
.collect()
})
.collect();
Self { distances }
}
}
impl Problem for RingTsp {
type Decision = Vec<usize>;
fn objectives(&self) -> ObjectiveSpace {
ObjectiveSpace::new(vec![Objective::minimize("tour_length")])
}
fn evaluate(&self, tour: &Vec<usize>) -> Evaluation {
let n = tour.len();
let mut total = 0.0;
for w in tour.windows(2) {
total += self.distances[w[0]][w[1]];
}
total += self.distances[tour[n - 1]][tour[0]];
Evaluation::new(vec![total])
}
}
/// Trivial single-objective problem to test the multi-objective panic.
struct DummyMo;
impl Problem for DummyMo {
type Decision = Vec<usize>;
fn objectives(&self) -> ObjectiveSpace {
ObjectiveSpace::new(vec![Objective::minimize("a"), Objective::minimize("b")])
}
fn evaluate(&self, _tour: &Vec<usize>) -> Evaluation {
Evaluation::new(vec![0.0, 0.0])
}
}
#[test]
fn finds_near_optimum_on_5_city_ring() {
let problem = RingTsp::new(5);
let mut opt = AntColonyTsp::new(
AntColonyTspConfig {
ants: 10,
generations: 30,
alpha: 1.0,
beta: 3.0,
evaporation: 0.5,
deposit: 1.0,
initial_pheromone: 1.0,
seed: 1,
},
problem.distances.clone(),
);
let r = opt.run(&problem);
let best = r.best.unwrap();
// Optimal pentagon perimeter ≈ 5.878. ACO should hit close.
assert!(
best.evaluation.objectives[0] < 5.95,
"got tour length = {}",
best.evaluation.objectives[0],
);
}
#[test]
fn deterministic_with_same_seed() {
let problem = RingTsp::new(5);
let cfg = AntColonyTspConfig {
ants: 8,
generations: 10,
alpha: 1.0,
beta: 2.0,
evaporation: 0.5,
deposit: 1.0,
initial_pheromone: 1.0,
seed: 99,
};
let mut a = AntColonyTsp::new(cfg.clone(), problem.distances.clone());
let mut b = AntColonyTsp::new(cfg, problem.distances.clone());
let ra = a.run(&problem);
let rb = b.run(&problem);
assert_eq!(
ra.best.unwrap().evaluation.objectives,
rb.best.unwrap().evaluation.objectives,
);
}
#[test]
#[should_panic(expected = "exactly one objective")]
fn multi_objective_panics() {
let mut opt = AntColonyTsp::new(
AntColonyTspConfig::default(),
vec![vec![0.0, 1.0], vec![1.0, 0.0]],
);
let _ = opt.run(&DummyMo);
}
}
src/algorithms/bayesian_opt.rs
+436
View File
@@ -0,0 +1,436 @@
//! `BayesianOpt` — Gaussian-process-based Bayesian Optimization.
//!
//! Sample-efficient sequential optimizer for expensive black-box
//! single-objective real-valued problems. Builds a GP surrogate of the
//! objective and selects the next evaluation point by maximizing the
//! Expected Improvement acquisition.
use rand::Rng as _;
use crate::core::candidate::Candidate;
use crate::core::evaluation::Evaluation;
use crate::core::objective::Direction;
use crate::core::population::Population;
use crate::core::problem::Problem;
use crate::core::result::OptimizationResult;
use crate::core::rng::{Rng, rng_from_seed};
use crate::internal::cholesky::{cholesky, solve};
use crate::operators::real::RealBounds;
use crate::traits::Optimizer;
/// Configuration for [`BayesianOpt`].
#[derive(Debug, Clone)]
pub struct BayesianOptConfig {
/// Number of uniform-random initial samples before the BO loop starts.
/// Hansen-style rule of thumb: 5×dim, but small budgets often work.
pub initial_samples: usize,
/// Number of BO iterations after the initial design.
pub iterations: usize,
/// Per-axis RBF length scales (one per dimension). Smaller = more
/// "wiggly" surrogate. Reasonable default: 0.2 × bound range per axis.
pub length_scales: Option<Vec<f64>>,
/// GP signal variance (the "amplitude" of the surrogate).
pub signal_variance: f64,
/// GP noise variance (small jitter to keep the kernel matrix SPD even
/// at duplicate or near-duplicate points).
pub noise_variance: f64,
/// Number of random samples used to maximize the acquisition function
/// each step. The best-EI sample is chosen as the next evaluation.
pub acquisition_samples: usize,
/// Seed for the deterministic RNG.
pub seed: u64,
}
impl Default for BayesianOptConfig {
fn default() -> Self {
Self {
initial_samples: 10,
iterations: 40,
length_scales: None,
signal_variance: 1.0,
noise_variance: 1e-6,
acquisition_samples: 1_000,
seed: 42,
}
}
}
/// Gaussian-process Bayesian Optimization with Expected Improvement.
///
/// `Vec<f64>` decisions only. Single-objective only. Targets expensive
/// evaluation budgets (50–500). The GP kernel is anisotropic RBF; the
/// acquisition function is EI; both are optimized by best-of-N random
/// sampling each step (simple, predictable cost).
#[derive(Debug, Clone)]
pub struct BayesianOpt {
/// Algorithm configuration.
pub config: BayesianOptConfig,
/// Per-variable bounds.
pub bounds: RealBounds,
}
impl BayesianOpt {
/// Construct a `BayesianOpt`.
pub fn new(config: BayesianOptConfig, bounds: RealBounds) -> Self {
Self { config, bounds }
}
}
impl<P> Optimizer<P> for BayesianOpt
where
P: Problem<Decision = Vec<f64>> + Sync,
{
fn run(&mut self, problem: &P) -> OptimizationResult<P::Decision> {
assert!(
self.config.initial_samples >= 2,
"BayesianOpt initial_samples must be >= 2",
);
assert!(
self.config.signal_variance > 0.0,
"BayesianOpt signal_variance must be > 0"
);
assert!(
self.config.noise_variance > 0.0,
"BayesianOpt noise_variance must be > 0"
);
assert!(
self.config.acquisition_samples >= 1,
"BayesianOpt acquisition_samples must be >= 1",
);
let objectives = problem.objectives();
assert!(
objectives.is_single_objective(),
"BayesianOpt requires exactly one objective",
);
let direction = objectives.objectives[0].direction;
let dim = self.bounds.bounds.len();
if let Some(ls) = &self.config.length_scales {
assert_eq!(
ls.len(),
dim,
"BayesianOpt length_scales.len() must equal dim"
);
}
let length_scales: Vec<f64> = self.config.length_scales.clone().unwrap_or_else(|| {
self.bounds
.bounds
.iter()
.map(|&(lo, hi)| 0.2 * (hi - lo).max(1e-9))
.collect()
});
let mut rng = rng_from_seed(self.config.seed);
// ---------------- Initial random design ----------------
let mut decisions: Vec<Vec<f64>> =
Vec::with_capacity(self.config.initial_samples + self.config.iterations);
let mut targets: Vec<f64> = Vec::with_capacity(decisions.capacity());
let mut evaluations = Vec::with_capacity(decisions.capacity());
for _ in 0..self.config.initial_samples {
let x = sample_uniform_in_bounds(&self.bounds, &mut rng);
let e = problem.evaluate(&x);
// GP works on minimization-oriented "want low" targets.
let t = oriented_target(&e, direction);
decisions.push(x);
targets.push(t);
evaluations.push(e);
}
// ---------------- Sequential BO loop ----------------
for _ in 0..self.config.iterations {
// Build the GP posterior around current observations.
let posterior = match GpPosterior::fit(
&decisions,
&targets,
&length_scales,
self.config.signal_variance,
self.config.noise_variance,
) {
Ok(p) => p,
Err(_) => {
// SPD failure (typically numerical): fall back to a
// single uniform-random sample this step.
let x = sample_uniform_in_bounds(&self.bounds, &mut rng);
let e = problem.evaluate(&x);
targets.push(oriented_target(&e, direction));
decisions.push(x);
evaluations.push(e);
continue;
}
};
let best_target = targets.iter().cloned().fold(f64::INFINITY, f64::min);
// Maximize EI by best-of-N random sampling.
let mut best_x = sample_uniform_in_bounds(&self.bounds, &mut rng);
let mut best_ei = -f64::INFINITY;
for _ in 0..self.config.acquisition_samples {
let cand = sample_uniform_in_bounds(&self.bounds, &mut rng);
let (mu, sigma) = posterior.predict(&cand);
let ei = expected_improvement(mu, sigma, best_target);
if ei > best_ei {
best_ei = ei;
best_x = cand;
}
}
let e = problem.evaluate(&best_x);
targets.push(oriented_target(&e, direction));
decisions.push(best_x);
evaluations.push(e);
}
// Build the final population/best.
let final_pop: Vec<Candidate<Vec<f64>>> = decisions
.into_iter()
.zip(evaluations)
.map(|(d, e)| Candidate::new(d, e))
.collect();
let mut best_idx = 0;
for i in 1..final_pop.len() {
if better(
&final_pop[i].evaluation,
&final_pop[best_idx].evaluation,
direction,
) {
best_idx = i;
}
}
let total_evaluations = final_pop.len();
let best = final_pop[best_idx].clone();
let front = vec![best.clone()];
OptimizationResult::new(
Population::new(final_pop),
front,
Some(best),
total_evaluations,
self.config.iterations + self.config.initial_samples,
)
}
}
/// Convert an Evaluation into a "smaller is better" target. For Maximize
/// problems we negate; infeasibles get a large penalty proportional to
/// the violation magnitude.
fn oriented_target(e: &Evaluation, direction: Direction) -> f64 {
let base = match direction {
Direction::Minimize => e.objectives[0],
Direction::Maximize => -e.objectives[0],
};
if e.is_feasible() {
base
} else {
// Penalize so the GP learns to avoid this region.
base + 1e6 * e.constraint_violation
}
}
fn better(a: &Evaluation, b: &Evaluation, direction: Direction) -> bool {
match (a.is_feasible(), b.is_feasible()) {
(true, false) => true,
(false, true) => false,
(false, false) => a.constraint_violation < b.constraint_violation,
(true, true) => match direction {
Direction::Minimize => a.objectives[0] < b.objectives[0],
Direction::Maximize => a.objectives[0] > b.objectives[0],
},
}
}
fn sample_uniform_in_bounds(bounds: &RealBounds, rng: &mut Rng) -> Vec<f64> {
bounds
.bounds
.iter()
.map(|&(lo, hi)| {
if lo == hi {
lo
} else {
lo + (hi - lo) * rng.random::<f64>()
}
})
.collect()
}
/// Anisotropic RBF kernel: `k(x, y) = σ² · exp(-0.5 · Σ ((x_i - y_i)/ℓ_i)²)`.
fn rbf_kernel(x: &[f64], y: &[f64], length_scales: &[f64], signal_variance: f64) -> f64 {
let mut sum = 0.0;
for ((a, b), l) in x.iter().zip(y.iter()).zip(length_scales.iter()) {
let d = (a - b) / l.max(1e-12);
sum += d * d;
}
signal_variance * (-0.5 * sum).exp()
}
struct GpPosterior {
decisions: Vec<Vec<f64>>,
length_scales: Vec<f64>,
signal_variance: f64,
/// `α = K^{-1} · y_target`, precomputed for the mean prediction.
alpha: Vec<f64>,
/// Cholesky factor of `K + σ_n² · I`, kept for variance prediction.
chol_l: Vec<Vec<f64>>,
}
impl GpPosterior {
fn fit(
decisions: &[Vec<f64>],
targets: &[f64],
length_scales: &[f64],
signal_variance: f64,
noise_variance: f64,
) -> Result<Self, &'static str> {
let n = decisions.len();
let mut k = vec![vec![0.0_f64; n]; n];
for i in 0..n {
for j in 0..=i {
let v = rbf_kernel(&decisions[i], &decisions[j], length_scales, signal_variance);
k[i][j] = v;
k[j][i] = v;
}
k[i][i] += noise_variance;
}
let chol_l = cholesky(&k)?;
let alpha = solve(&chol_l, targets);
Ok(Self {
decisions: decisions.to_vec(),
length_scales: length_scales.to_vec(),
signal_variance,
alpha,
chol_l,
})
}
fn predict(&self, x: &[f64]) -> (f64, f64) {
let n = self.decisions.len();
let mut k_star = vec![0.0_f64; n];
for (i, k_star_i) in k_star.iter_mut().enumerate() {
*k_star_i = rbf_kernel(
x,
&self.decisions[i],
&self.length_scales,
self.signal_variance,
);
}
let _ = n;
let mu: f64 = k_star
.iter()
.zip(self.alpha.iter())
.map(|(a, b)| a * b)
.sum();
// Var = k(x,x) - k_star^T · K^{-1} · k_star
// Compute K^{-1}·k_star = solve_upper_transpose(L, solve_lower(L, k_star))
let v_temp = crate::internal::cholesky::solve_lower(&self.chol_l, &k_star);
let v: f64 = v_temp.iter().map(|x| x * x).sum();
let var = (self.signal_variance - v).max(0.0);
(mu, var.sqrt())
}
}
/// Expected Improvement (minimization-oriented) at a point with predicted
/// mean `mu` and standard deviation `sigma`, given the current best
/// observed target `f_best`. Returns 0 if `sigma` is effectively zero.
fn expected_improvement(mu: f64, sigma: f64, f_best: f64) -> f64 {
if sigma < 1e-12 {
return 0.0;
}
let improvement = f_best - mu;
let z = improvement / sigma;
improvement * normal_cdf(z) + sigma * normal_pdf(z)
}
fn normal_pdf(z: f64) -> f64 {
(-0.5 * z * z).exp() / (2.0 * std::f64::consts::PI).sqrt()
}
fn normal_cdf(z: f64) -> f64 {
// Approximate Φ(z) via erf. Abramowitz & Stegun 7.1.26 series-free
// rational approximation good to ~1.5e-7.
0.5 * (1.0 + erf(z / std::f64::consts::SQRT_2))
}
fn erf(x: f64) -> f64 {
// Numerical Recipes-style erf, accurate to ~1e-7.
let a1 = 0.254_829_592;
let a2 = -0.284_496_736;
let a3 = 1.421_413_741;
let a4 = -1.453_152_027;
let a5 = 1.061_405_429;
let p = 0.327_591_1;
let sign = if x < 0.0 { -1.0 } else { 1.0 };
let x = x.abs();
let t = 1.0 / (1.0 + p * x);
let y = 1.0 - (((((a5 * t + a4) * t) + a3) * t + a2) * t + a1) * t * (-x * x).exp();
sign * y
}
#[cfg(test)]
mod tests {
use super::*;
use crate::tests_support::{SchafferN1, Sphere1D};
fn make_optimizer(seed: u64) -> BayesianOpt {
BayesianOpt::new(
BayesianOptConfig {
initial_samples: 5,
iterations: 25,
length_scales: None,
signal_variance: 1.0,
noise_variance: 1e-6,
acquisition_samples: 500,
seed,
},
RealBounds::new(vec![(-5.0, 5.0)]),
)
}
#[test]
fn finds_minimum_of_sphere_quickly() {
// BO's whole point is sample efficiency: 30 evals ought to be
// enough for a 1-D sphere. (Pop-based methods needed thousands.)
let mut opt = make_optimizer(1);
let r = opt.run(&Sphere1D);
let best = r.best.unwrap();
assert!(
best.evaluation.objectives[0] < 1e-3,
"BO should converge fast on 1-D sphere; got f = {}",
best.evaluation.objectives[0],
);
assert!(r.evaluations <= 30 + 1);
}
#[test]
fn deterministic_with_same_seed() {
let mut a = make_optimizer(99);
let mut b = make_optimizer(99);
let ra = a.run(&Sphere1D);
let rb = b.run(&Sphere1D);
assert_eq!(
ra.best.unwrap().evaluation.objectives,
rb.best.unwrap().evaluation.objectives,
);
}
#[test]
#[should_panic(expected = "exactly one objective")]
fn multi_objective_panics() {
let mut opt = make_optimizer(0);
let _ = opt.run(&SchafferN1);
}
#[test]
#[should_panic(expected = "length_scales.len() must equal dim")]
fn length_scales_dim_mismatch_panics() {
let mut opt = BayesianOpt::new(
BayesianOptConfig {
initial_samples: 5,
iterations: 5,
length_scales: Some(vec![1.0, 1.0]),
signal_variance: 1.0,
noise_variance: 1e-6,
acquisition_samples: 100,
seed: 0,
},
RealBounds::new(vec![(-1.0, 1.0)]),
);
let _ = opt.run(&Sphere1D);
}
}
src/algorithms/cma_es.rs
+485
View File
@@ -0,0 +1,485 @@
//! CMA-ES — Hansen & Ostermeier 2001 Covariance Matrix Adaptation
//! Evolution Strategy.
use rand_distr::{Distribution, Normal};
use crate::algorithms::parallel_eval::evaluate_batch;
use crate::core::candidate::Candidate;
use crate::core::objective::Direction;
use crate::core::population::Population;
use crate::core::problem::Problem;
use crate::core::result::OptimizationResult;
use crate::core::rng::rng_from_seed;
use crate::internal::eigen::symmetric_eigen;
use crate::operators::real::RealBounds;
use crate::pareto::front::best_candidate;
use crate::traits::Optimizer;
/// Configuration for [`CmaEs`].
#[derive(Debug, Clone)]
pub struct CmaEsConfig {
/// Population size `λ`. Must be at least 4. Hansen recommends
/// `4 + floor(3 · ln(N))` as a default for `N`-dim problems.
pub population_size: usize,
/// Number of generations.
pub generations: usize,
/// Initial step size `σ_0`. Often ~ 1/3 of the search range per dim.
pub initial_sigma: f64,
/// Recompute the eigendecomposition of `C` every this many generations
/// to amortize cost. The full algorithm decomposes every generation
/// (set this to 1); 1–10 is fine for small `N`.
pub eigen_decomposition_period: usize,
/// Optional initial mean. If `None`, the mean defaults to the per-axis
/// midpoint of the bounds. Used by `IpopCmaEs` to inject restart
/// diversity without shrinking the search box.
pub initial_mean: Option<Vec<f64>>,
/// Seed for the deterministic RNG.
pub seed: u64,
}
impl Default for CmaEsConfig {
fn default() -> Self {
Self {
population_size: 16,
generations: 200,
initial_sigma: 0.5,
eigen_decomposition_period: 1,
initial_mean: None,
seed: 42,
}
}
}
/// Single-objective real-valued CMA-ES.
///
/// Maintains a multivariate Gaussian sampler `mean + σ · N(0, C)`, samples
/// `λ` offspring from it each generation, selects the `μ` best (weighted),
/// and updates `mean`, `σ`, and `C` via the standard CMA-ES rules.
///
/// `Vec<f64>` decisions only. Bounds come from the embedded `RealBounds`
/// field; both the initial mean and every offspring are clamped per
/// dimension.
#[derive(Debug, Clone)]
pub struct CmaEs {
/// Algorithm configuration.
pub config: CmaEsConfig,
/// Per-variable bounds — used both to seed `mean` (midpoint) and to
/// clamp every offspring.
pub bounds: RealBounds,
}
impl CmaEs {
/// Construct a `CmaEs`.
pub fn new(config: CmaEsConfig, bounds: RealBounds) -> Self {
Self { config, bounds }
}
}
impl<P> Optimizer<P> for CmaEs
where
P: Problem<Decision = Vec<f64>> + Sync,
{
fn run(&mut self, problem: &P) -> OptimizationResult<P::Decision> {
assert!(
self.config.population_size >= 4,
"CmaEs population_size must be >= 4",
);
assert!(
self.config.initial_sigma > 0.0,
"CmaEs initial_sigma must be positive",
);
assert!(
self.config.eigen_decomposition_period >= 1,
"CmaEs eigen_decomposition_period must be >= 1",
);
let objectives = problem.objectives();
assert!(
objectives.is_single_objective(),
"CmaEs only supports single-objective problems",
);
let direction = objectives.objectives[0].direction;
let n = self.bounds.bounds.len();
let n_f = n as f64;
let lambda = self.config.population_size;
let lambda_f = lambda as f64;
let mu = lambda / 2;
assert!(mu >= 1, "CmaEs derived mu (= lambda/2) must be >= 1");
let mut rng = rng_from_seed(self.config.seed);
// ---------------------------------------------------------------
// Selection weights w_i ∝ ln((λ+1)/2) − ln(i) for i = 1..μ,
// normalized so they sum to 1. Then mu_eff = 1 / Σ w_i².
// ---------------------------------------------------------------
let raw_weights: Vec<f64> = (0..mu)
.map(|i| ((lambda_f + 1.0) / 2.0).ln() - ((i + 1) as f64).ln())
.collect();
let sum_w: f64 = raw_weights.iter().sum();
let weights: Vec<f64> = raw_weights.iter().map(|w| w / sum_w).collect();
let mu_eff = 1.0 / weights.iter().map(|w| w * w).sum::<f64>();
// ---------------------------------------------------------------
// Standard CMA-ES strategy parameters (Hansen tutorial §7.1).
// ---------------------------------------------------------------
let c_sigma = (mu_eff + 2.0) / (n_f + mu_eff + 5.0);
let d_sigma = 1.0 + 2.0 * ((mu_eff - 1.0) / (n_f + 1.0)).sqrt().max(0.0) + c_sigma;
let c_c = (4.0 + mu_eff / n_f) / (n_f + 4.0 + 2.0 * mu_eff / n_f);
let c_1 = 2.0 / ((n_f + 1.3).powi(2) + mu_eff);
let c_mu = ((1.0 - c_1) * 2.0 * (mu_eff - 2.0 + 1.0 / mu_eff)
/ ((n_f + 2.0).powi(2) + mu_eff))
.min(1.0 - c_1);
// E‖N(0, I)‖ ≈ √n · (1 − 1/(4n) + 1/(21n²))
let chi_n = n_f.sqrt() * (1.0 - 1.0 / (4.0 * n_f) + 1.0 / (21.0 * n_f * n_f));
// ---------------------------------------------------------------
// Initial state.
// ---------------------------------------------------------------
let mut mean: Vec<f64> = if let Some(provided) = self.config.initial_mean.clone() {
assert_eq!(
provided.len(),
self.bounds.bounds.len(),
"CmaEs initial_mean.len() must equal the bounds dimension",
);
// Clamp the user-provided mean into the bounds so the algorithm
// doesn't start outside the search box.
provided
.into_iter()
.zip(self.bounds.bounds.iter())
.map(|(v, &(lo, hi))| v.clamp(lo, hi))
.collect()
} else {
self.bounds
.bounds
.iter()
.map(|&(lo, hi)| 0.5 * (lo + hi))
.collect()
};
let mut sigma = self.config.initial_sigma;
// Covariance C, eigenvectors B, eigenvalues d (square roots of eigenvalues of C).
let mut c_matrix: Vec<Vec<f64>> = (0..n)
.map(|i| (0..n).map(|j| if i == j { 1.0 } else { 0.0 }).collect())
.collect();
let mut b: Vec<Vec<f64>> = c_matrix.to_vec();
let mut d: Vec<f64> = vec![1.0; n];
let mut p_sigma = vec![0.0_f64; n];
let mut p_c = vec![0.0_f64; n];
let mut evaluations = 0usize;
let normal = Normal::new(0.0, 1.0).expect("Normal::new(0, 1)");
let mut best_candidate_seen: Option<Candidate<Vec<f64>>> = None;
for generation in 0..self.config.generations {
// Recompute B, d every period generations from C (after symmetrizing).
if generation % self.config.eigen_decomposition_period == 0 {
// Force symmetry.
#[allow(clippy::needless_range_loop)] // body indexes both [i][j] and [j][i].
for i in 0..n {
for j in (i + 1)..n {
let avg = 0.5 * (c_matrix[i][j] + c_matrix[j][i]);
c_matrix[i][j] = avg;
c_matrix[j][i] = avg;
}
}
let (eigenvalues, eigenvectors) = symmetric_eigen(&c_matrix, 1e-14, 100);
// eigenvectors is sorted descending; we don't depend on order
// for sampling correctness, but we do need positive eigenvalues.
d = eigenvalues.iter().map(|&v| v.max(1e-20).sqrt()).collect();
// B is the matrix whose columns are the eigenvectors. The
// helper returns `eigenvectors[i]` as the i-th *eigenvector*,
// so b[r][c] should equal eigenvectors[c][r].
b = (0..n)
.map(|r| (0..n).map(|c| eigenvectors[c][r]).collect())
.collect();
}
// ----- Sample λ offspring -----
let mut z_samples: Vec<Vec<f64>> = Vec::with_capacity(lambda);
let mut x_samples: Vec<Vec<f64>> = Vec::with_capacity(lambda);
for _ in 0..lambda {
let z: Vec<f64> = (0..n).map(|_| normal.sample(&mut rng)).collect();
// y = B · D · z
let bd_z: Vec<f64> = (0..n)
.map(|i| (0..n).map(|j| b[i][j] * d[j] * z[j]).sum::<f64>())
.collect();
// x = mean + σ · y, clamped to bounds
let x: Vec<f64> = (0..n)
.map(|i| {
let v = mean[i] + sigma * bd_z[i];
let (lo, hi) = self.bounds.bounds[i];
v.clamp(lo, hi)
})
.collect();
z_samples.push(z);
x_samples.push(x);
}
// Evaluate offspring (parallel-friendly).
let evaluated = evaluate_batch(problem, x_samples.clone());
evaluations += evaluated.len();
// Track the best candidate ever.
for c in &evaluated {
let beats_best = match &best_candidate_seen {
None => true,
Some(b) => better_than_so(&c.evaluation, &b.evaluation, direction),
};
if beats_best {
best_candidate_seen = Some(c.clone());
}
}
// Sort offspring by fitness ascending (best first).
let mut order: Vec<usize> = (0..lambda).collect();
order.sort_by(|&a, &b_| {
compare_so(
&evaluated[a].evaluation,
&evaluated[b_].evaluation,
direction,
)
});
// ----- Recompute mean from the μ best (weighted average of x) -----
let old_mean = mean.clone();
let mut new_mean = vec![0.0_f64; n];
for k in 0..mu {
let xk = &x_samples[order[k]];
let wk = weights[k];
for i in 0..n {
new_mean[i] += wk * xk[i];
}
}
mean = new_mean;
// ----- Weighted average of z (used for evolution-path updates) -----
let mut z_weighted = vec![0.0_f64; n];
for k in 0..mu {
let zk = &z_samples[order[k]];
let wk = weights[k];
for i in 0..n {
z_weighted[i] += wk * zk[i];
}
}
// ----- Evolution path for step size: p_σ = (1 - c_σ) p_σ + sqrt(c_σ (2 - c_σ) μ_eff) · B z̄ -----
let factor_p_sigma = (c_sigma * (2.0 - c_sigma) * mu_eff).sqrt();
// B · z_weighted (since C^{-1/2} (m_new - m_old) / σ = B · D^{-1} · D · z̄ = B · z̄)
let bz: Vec<f64> = (0..n)
.map(|i| (0..n).map(|j| b[i][j] * z_weighted[j]).sum::<f64>())
.collect();
for i in 0..n {
p_sigma[i] = (1.0 - c_sigma) * p_sigma[i] + factor_p_sigma * bz[i];
}
// ----- Step-size update -----
let p_sigma_norm = p_sigma.iter().map(|x| x * x).sum::<f64>().sqrt();
sigma *= ((c_sigma / d_sigma) * (p_sigma_norm / chi_n - 1.0)).exp();
// Heaviside for h_σ: damp p_c update if the step length is huge.
let h_sigma = if p_sigma_norm
/ (1.0 - (1.0 - c_sigma).powi(2 * (generation as i32 + 1))).sqrt()
< (1.4 + 2.0 / (n_f + 1.0)) * chi_n
{
1.0
} else {
0.0
};
// ----- Evolution path for C: p_c = (1 - c_c) p_c + h_σ · sqrt(c_c (2 - c_c) μ_eff) · (m_new - m_old)/σ -----
let factor_p_c = h_sigma * (c_c * (2.0 - c_c) * mu_eff).sqrt();
for i in 0..n {
p_c[i] = (1.0 - c_c) * p_c[i] + factor_p_c * (mean[i] - old_mean[i]) / sigma;
}
// ----- Covariance matrix update (rank-1 + rank-μ) -----
let delta_h = (1.0 - h_sigma) * c_c * (2.0 - c_c);
#[allow(clippy::needless_range_loop)]
// body uses both i and j to index c_matrix and offspring.
for i in 0..n {
for j in 0..n {
let mut update = (1.0 - c_1 - c_mu) * c_matrix[i][j]
+ c_1 * (p_c[i] * p_c[j] + delta_h * c_matrix[i][j]);
// Rank-μ contribution.
let mut rank_mu_term = 0.0;
for k in 0..mu {
let xk = &x_samples[order[k]];
let yi = (xk[i] - old_mean[i]) / sigma;
let yj = (xk[j] - old_mean[j]) / sigma;
rank_mu_term += weights[k] * yi * yj;
}
update += c_mu * rank_mu_term;
c_matrix[i][j] = update;
}
}
// Clamp mean to bounds (sigma may push it out otherwise).
for (i, m) in mean.iter_mut().enumerate() {
let (lo, hi) = self.bounds.bounds[i];
*m = m.clamp(lo, hi);
}
}
// Final population: just the best-seen candidate. Match other
// single-objective algorithms' convention.
let best = best_candidate_seen.expect("at least one generation evaluated");
let final_pop = vec![best.clone()];
let front = vec![best.clone()];
let best_opt = best_candidate(&final_pop, &objectives);
OptimizationResult::new(
Population::new(final_pop),
front,
best_opt,
evaluations,
self.config.generations,
)
}
}
fn compare_so(
a: &crate::core::evaluation::Evaluation,
b: &crate::core::evaluation::Evaluation,
direction: Direction,
) -> std::cmp::Ordering {
match (a.is_feasible(), b.is_feasible()) {
(true, false) => std::cmp::Ordering::Less,
(false, true) => std::cmp::Ordering::Greater,
(false, false) => a
.constraint_violation
.partial_cmp(&b.constraint_violation)
.unwrap_or(std::cmp::Ordering::Equal),
(true, true) => match direction {
Direction::Minimize => a.objectives[0]
.partial_cmp(&b.objectives[0])
.unwrap_or(std::cmp::Ordering::Equal),
Direction::Maximize => b.objectives[0]
.partial_cmp(&a.objectives[0])
.unwrap_or(std::cmp::Ordering::Equal),
},
}
}
fn better_than_so(
a: &crate::core::evaluation::Evaluation,
b: &crate::core::evaluation::Evaluation,
direction: Direction,
) -> bool {
compare_so(a, b, direction) == std::cmp::Ordering::Less
}
#[cfg(test)]
mod tests {
use super::*;
use crate::core::evaluation::Evaluation;
use crate::core::objective::{Objective, ObjectiveSpace};
use crate::tests_support::{SchafferN1, Sphere1D};
/// 5-D Rosenbrock for exercise.
struct Rosenbrock5D;
impl Problem for Rosenbrock5D {
type Decision = Vec<f64>;
fn objectives(&self) -> ObjectiveSpace {
ObjectiveSpace::new(vec![Objective::minimize("f")])
}
fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
let f: f64 = (0..(x.len() - 1))
.map(|i| {
let a = 1.0 - x[i];
let b = x[i + 1] - x[i] * x[i];
a * a + 100.0 * b * b
})
.sum();
Evaluation::new(vec![f])
}
}
#[test]
fn finds_minimum_of_sphere() {
let mut opt = CmaEs::new(
CmaEsConfig {
population_size: 12,
generations: 100,
initial_sigma: 0.5,
eigen_decomposition_period: 1,
initial_mean: None,
seed: 1,
},
RealBounds::new(vec![(-5.0, 5.0)]),
);
let r = opt.run(&Sphere1D);
let best = r.best.unwrap();
assert!(
best.evaluation.objectives[0] < 1e-8,
"got f = {}",
best.evaluation.objectives[0],
);
}
#[test]
fn finds_minimum_of_rosenbrock_5d() {
let mut opt = CmaEs::new(
CmaEsConfig {
population_size: 16,
generations: 400,
initial_sigma: 0.5,
eigen_decomposition_period: 1,
initial_mean: None,
seed: 1,
},
RealBounds::new(vec![(-5.0, 5.0); 5]),
);
let r = opt.run(&Rosenbrock5D);
let best = r.best.unwrap();
// Rosenbrock is a tough non-convex valley; CMA-ES should still get
// far closer than random search.
assert!(
best.evaluation.objectives[0] < 1.0,
"got f = {}",
best.evaluation.objectives[0],
);
}
#[test]
fn deterministic_with_same_seed() {
let cfg = CmaEsConfig {
population_size: 8,
generations: 30,
initial_sigma: 0.5,
eigen_decomposition_period: 1,
initial_mean: None,
seed: 99,
};
let mut a = CmaEs::new(cfg.clone(), RealBounds::new(vec![(-5.0, 5.0)]));
let mut b = CmaEs::new(cfg, RealBounds::new(vec![(-5.0, 5.0)]));
let ra = a.run(&Sphere1D);
let rb = b.run(&Sphere1D);
assert_eq!(
ra.best.unwrap().evaluation.objectives,
rb.best.unwrap().evaluation.objectives,
);
}
#[test]
#[should_panic(expected = "single-objective")]
fn multi_objective_panics() {
let mut opt = CmaEs::new(CmaEsConfig::default(), RealBounds::new(vec![(-5.0, 5.0)]));
let _ = opt.run(&SchafferN1);
}
#[test]
#[should_panic(expected = "population_size must be >= 4")]
fn small_population_panics() {
let mut opt = CmaEs::new(
CmaEsConfig {
population_size: 3,
generations: 1,
initial_sigma: 0.5,
eigen_decomposition_period: 1,
initial_mean: None,
seed: 0,
},
RealBounds::new(vec![(-1.0, 1.0)]),
);
let _ = opt.run(&Sphere1D);
}
}
src/algorithms/differential_evolution.rs
+14 -6
View File
@@ -91,8 +91,10 @@ where
}; };
let initial_pop = evaluate_batch(problem, decisions.clone()); let initial_pop = evaluate_batch(problem, decisions.clone());
let mut evaluations = initial_pop.len(); let mut evaluations = initial_pop.len();
let mut evals: Vec<f64> = let mut evals: Vec<f64> = initial_pop
initial_pop.iter().map(|c| c.evaluation.objectives[0]).collect(); .iter()
.map(|c| c.evaluation.objectives[0])
.collect();
for _gen in 0..self.config.generations { for _gen in 0..self.config.generations {
// Phase 1 (serial): construct one trial per target. RNG state is // Phase 1 (serial): construct one trial per target. RNG state is
@@ -151,7 +153,11 @@ where
} }
} }
fn pick_three_distinct(n: usize, exclude: usize, rng: &mut crate::core::rng::Rng) -> (usize, usize, usize) { fn pick_three_distinct(
n: usize,
exclude: usize,
rng: &mut crate::core::rng::Rng,
) -> (usize, usize, usize) {
let pick = |rng: &mut crate::core::rng::Rng, taken: &[usize]| -> usize { let pick = |rng: &mut crate::core::rng::Rng, taken: &[usize]| -> usize {
loop { loop {
let v = rng.random_range(0..n); let v = rng.random_range(0..n);
@@ -185,7 +191,10 @@ mod tests {
); );
let r = opt.run(&Sphere1D); let r = opt.run(&Sphere1D);
let best = r.best.unwrap(); let best = r.best.unwrap();
assert!(best.evaluation.objectives[0] < 1e-3, "DE should converge near 0"); assert!(
best.evaluation.objectives[0] < 1e-3,
"DE should converge near 0"
);
} }
#[test] #[test]
@@ -197,8 +206,7 @@ mod tests {
crossover_probability: 0.7, crossover_probability: 0.7,
seed: 99, seed: 99,
}; };
let mut a = let mut a = DifferentialEvolution::new(cfg.clone(), RealBounds::new(vec![(-5.0, 5.0)]));
DifferentialEvolution::new(cfg.clone(), RealBounds::new(vec![(-5.0, 5.0)]));
let mut b = DifferentialEvolution::new(cfg, RealBounds::new(vec![(-5.0, 5.0)])); let mut b = DifferentialEvolution::new(cfg, RealBounds::new(vec![(-5.0, 5.0)]));
let ra = a.run(&Sphere1D); let ra = a.run(&Sphere1D);
let rb = b.run(&Sphere1D); let rb = b.run(&Sphere1D);
src/algorithms/epsilon_moea.rs
+367
View File
@@ -0,0 +1,367 @@
//! `EpsilonMoea` — Deb, Mohan & Mishra 2003 ε-dominance MOEA.
use rand::Rng as _;
use crate::core::candidate::Candidate;
use crate::core::evaluation::Evaluation;
use crate::core::objective::ObjectiveSpace;
use crate::core::population::Population;
use crate::core::problem::Problem;
use crate::core::result::OptimizationResult;
use crate::core::rng::rng_from_seed;
use crate::pareto::dominance::{Dominance, pareto_compare};
use crate::pareto::front::{best_candidate, pareto_front};
use crate::traits::{Initializer, Optimizer, Variation};
/// Configuration for [`EpsilonMoea`].
#[derive(Debug, Clone)]
pub struct EpsilonMoeaConfig {
/// Internal population size.
pub population_size: usize,
/// Number of evaluations to perform (steady-state: one offspring per gen).
pub evaluations: usize,
/// ε for each objective. Must have one entry per objective; controls
/// the resolution of the regular box-grid the archive lives on.
pub epsilon: Vec<f64>,
/// Seed for the deterministic RNG.
pub seed: u64,
}
impl Default for EpsilonMoeaConfig {
fn default() -> Self {
Self {
population_size: 50,
evaluations: 25_000,
epsilon: vec![0.05, 0.05],
seed: 42,
}
}
}
/// ε-dominance MOEA.
#[derive(Debug, Clone)]
pub struct EpsilonMoea<I, V> {
/// Algorithm configuration.
pub config: EpsilonMoeaConfig,
/// Initial-decision sampler.
pub initializer: I,
/// Offspring-producing variation operator.
pub variation: V,
}
impl<I, V> EpsilonMoea<I, V> {
/// Construct an `EpsilonMoea`.
pub fn new(config: EpsilonMoeaConfig, initializer: I, variation: V) -> Self {
Self {
config,
initializer,
variation,
}
}
}
impl<P, I, V> Optimizer<P> for EpsilonMoea<I, V>
where
P: Problem + Sync,
P::Decision: Send,
I: Initializer<P::Decision>,
V: Variation<P::Decision>,
{
fn run(&mut self, problem: &P) -> OptimizationResult<P::Decision> {
assert!(
self.config.population_size > 0,
"EpsilonMoea population_size must be > 0"
);
let n = self.config.population_size;
let objectives = problem.objectives();
assert_eq!(
self.config.epsilon.len(),
objectives.len(),
"EpsilonMoea epsilon.len() must equal number of objectives",
);
for (i, &e) in self.config.epsilon.iter().enumerate() {
assert!(e > 0.0, "EpsilonMoea epsilon[{i}] must be > 0.0");
}
let epsilon = self.config.epsilon.clone();
let mut rng = rng_from_seed(self.config.seed);
// Internal population.
let initial_decisions = self.initializer.initialize(n, &mut rng);
let mut population: Vec<Candidate<P::Decision>> = initial_decisions
.into_iter()
.map(|d| {
let e = problem.evaluate(&d);
Candidate::new(d, e)
})
.collect();
let mut evaluations = population.len();
// ε-archive.
let mut archive: Vec<Candidate<P::Decision>> = Vec::new();
for c in &population {
insert_into_epsilon_archive(&mut archive, c.clone(), &objectives, &epsilon);
}
let total_evals = self.config.evaluations.max(evaluations);
while evaluations < total_evals {
// Pick one parent from the population, one from the archive
// (when non-empty; else two from the population).
let p1_idx = rng.random_range(0..population.len());
let parent_a = population[p1_idx].decision.clone();
let parent_b = if !archive.is_empty() {
let j = rng.random_range(0..archive.len());
archive[j].decision.clone()
} else {
let j = rng.random_range(0..population.len());
population[j].decision.clone()
};
let parents = vec![parent_a, parent_b];
let children = self.variation.vary(&parents, &mut rng);
assert!(
!children.is_empty(),
"EpsilonMoea variation returned no children"
);
let child_decision = children.into_iter().next().unwrap();
let child_eval = problem.evaluate(&child_decision);
evaluations += 1;
let child = Candidate::new(child_decision, child_eval);
// Update population: child replaces a Pareto-dominated random member,
// or any random member if non-dominated wrt every population member.
update_population(&mut population, &child, &objectives, &mut rng);
// Update ε-archive.
insert_into_epsilon_archive(&mut archive, child, &objectives, &epsilon);
}
let final_pop: Vec<Candidate<P::Decision>> = if !archive.is_empty() {
archive.clone()
} else {
population
};
let front = pareto_front(&final_pop, &objectives);
let best = best_candidate(&final_pop, &objectives);
OptimizationResult::new(
Population::new(final_pop),
front,
best,
evaluations,
self.config.evaluations,
)
}
}
/// Standard ε-MOEA population update: if the child is dominated by some
/// member, drop it; if it dominates a member, replace that member; if
/// non-dominated wrt all, replace a random member.
fn update_population<D: Clone>(
population: &mut [Candidate<D>],
child: &Candidate<D>,
objectives: &ObjectiveSpace,
rng: &mut crate::core::rng::Rng,
) {
let mut dominated_indices: Vec<usize> = Vec::new();
for (i, c) in population.iter().enumerate() {
match pareto_compare(&child.evaluation, &c.evaluation, objectives) {
Dominance::DominatedBy => return, // child dominated → discard
Dominance::Dominates => dominated_indices.push(i),
_ => {}
}
}
if !dominated_indices.is_empty() {
let pick = dominated_indices[rng.random_range(0..dominated_indices.len())];
population[pick] = child.clone();
} else {
let pick = rng.random_range(0..population.len());
population[pick] = child.clone();
}
}
/// Insert `child` into the ε-archive following Deb's standard rule:
///
/// - Translate every objective vector into ε-box coordinates
/// `b_i = floor(o_i / ε_i)` (in minimization frame).
/// - If `child`'s box is ε-dominated by an existing member → drop child.
/// - Else, drop existing members whose box is ε-dominated by `child`'s.
/// - Among members in the SAME box as `child`, keep the one closer to its
/// box's "ideal corner" (smallest L2 distance from box origin).
fn insert_into_epsilon_archive<D: Clone>(
archive: &mut Vec<Candidate<D>>,
child: Candidate<D>,
objectives: &ObjectiveSpace,
epsilon: &[f64],
) {
let child_box = box_coords(&child.evaluation, objectives, epsilon);
let child_corner_dist = corner_distance(&child.evaluation, objectives, epsilon, &child_box);
let mut to_drop: Vec<usize> = Vec::new();
let mut child_box_index: Option<usize> = None;
for (i, member) in archive.iter().enumerate() {
let member_box = box_coords(&member.evaluation, objectives, epsilon);
if box_dominates(&member_box, &child_box) {
// Child's box is ε-dominated; ignore the child.
return;
}
if box_dominates(&child_box, &member_box) {
to_drop.push(i);
} else if member_box == child_box {
child_box_index = Some(i);
}
}
// Drop ε-dominated members (in reverse order to keep indices valid).
to_drop.sort_unstable();
for i in to_drop.into_iter().rev() {
archive.swap_remove(i);
}
if let Some(idx) = child_box_index {
// Same box: keep whichever is closer to box's ideal corner.
let member_corner_dist =
corner_distance(&archive[idx].evaluation, objectives, epsilon, &child_box);
if child_corner_dist < member_corner_dist {
archive[idx] = child;
}
} else {
archive.push(child);
}
}
fn box_coords(eval: &Evaluation, objectives: &ObjectiveSpace, epsilon: &[f64]) -> Vec<i64> {
let oriented = objectives.as_minimization(&eval.objectives);
oriented
.iter()
.zip(epsilon.iter())
.map(|(v, e)| (v / e).floor() as i64)
.collect()
}
fn corner_distance(
eval: &Evaluation,
objectives: &ObjectiveSpace,
epsilon: &[f64],
box_idx: &[i64],
) -> f64 {
let oriented = objectives.as_minimization(&eval.objectives);
let mut sq = 0.0;
for k in 0..oriented.len() {
let corner = box_idx[k] as f64 * epsilon[k];
let d = oriented[k] - corner;
sq += d * d;
}
sq.sqrt()
}
/// Box-A ε-dominates box-B iff every coordinate of A is ≤ B and at least
/// one is strictly less.
fn box_dominates(a: &[i64], b: &[i64]) -> bool {
let mut strictly_less = false;
for (x, y) in a.iter().zip(b.iter()) {
if x > y {
return false;
}
if x < y {
strictly_less = true;
}
}
strictly_less
}
#[cfg(test)]
mod tests {
use super::*;
use crate::operators::{
CompositeVariation, PolynomialMutation, RealBounds, SimulatedBinaryCrossover,
};
use crate::tests_support::SchafferN1;
fn make_optimizer(
seed: u64,
) -> EpsilonMoea<RealBounds, CompositeVariation<SimulatedBinaryCrossover, PolynomialMutation>>
{
let bounds = vec![(-5.0, 5.0)];
let initializer = RealBounds::new(bounds.clone());
let variation = CompositeVariation {
crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5),
mutation: PolynomialMutation::new(bounds, 20.0, 1.0),
};
EpsilonMoea::new(
EpsilonMoeaConfig {
population_size: 20,
evaluations: 1_000,
epsilon: vec![0.05, 0.05],
seed,
},
initializer,
variation,
)
}
#[test]
fn produces_pareto_front() {
let mut opt = make_optimizer(1);
let r = opt.run(&SchafferN1);
assert!(!r.pareto_front.is_empty());
}
#[test]
fn deterministic_with_same_seed() {
let mut a = make_optimizer(99);
let mut b = make_optimizer(99);
let ra = a.run(&SchafferN1);
let rb = b.run(&SchafferN1);
let oa: Vec<Vec<f64>> = ra
.pareto_front
.iter()
.map(|c| c.evaluation.objectives.clone())
.collect();
let ob: Vec<Vec<f64>> = rb
.pareto_front
.iter()
.map(|c| c.evaluation.objectives.clone())
.collect();
assert_eq!(oa, ob);
}
#[test]
#[should_panic(expected = "epsilon.len() must equal number of objectives")]
fn dim_mismatch_panics() {
let bounds = vec![(0.0, 1.0)];
let initializer = RealBounds::new(bounds.clone());
let variation = CompositeVariation {
crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5),
mutation: PolynomialMutation::new(bounds, 20.0, 1.0),
};
let mut opt = EpsilonMoea::new(
EpsilonMoeaConfig {
population_size: 4,
evaluations: 100,
epsilon: vec![0.1, 0.1, 0.1],
seed: 0,
},
initializer,
variation,
);
let _ = opt.run(&SchafferN1);
}
#[test]
#[should_panic(expected = "must be > 0.0")]
fn zero_epsilon_panics() {
let bounds = vec![(0.0, 1.0)];
let initializer = RealBounds::new(bounds.clone());
let variation = CompositeVariation {
crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5),
mutation: PolynomialMutation::new(bounds, 20.0, 1.0),
};
let mut opt = EpsilonMoea::new(
EpsilonMoeaConfig {
population_size: 4,
evaluations: 100,
epsilon: vec![0.0, 0.1],
seed: 0,
},
initializer,
variation,
);
let _ = opt.run(&SchafferN1);
}
}
src/algorithms/genetic_algorithm.rs
+283
View File
@@ -0,0 +1,283 @@
//! `GeneticAlgorithm` — single-objective generational GA with elitism.
use crate::algorithms::parallel_eval::evaluate_batch;
use crate::core::candidate::Candidate;
use crate::core::objective::Direction;
use crate::core::population::Population;
use crate::core::problem::Problem;
use crate::core::result::OptimizationResult;
use crate::core::rng::rng_from_seed;
use crate::pareto::front::best_candidate;
use crate::selection::tournament::tournament_select_single_objective;
use crate::traits::{Initializer, Optimizer, Variation};
/// Configuration for [`GeneticAlgorithm`].
#[derive(Debug, Clone)]
pub struct GeneticAlgorithmConfig {
/// Constant population size.
pub population_size: usize,
/// Number of generations.
pub generations: usize,
/// Tournament size for parent selection (typical: 2).
pub tournament_size: usize,
/// Number of elite members to carry over each generation (must be
/// `< population_size`).
pub elitism: usize,
/// Seed for the deterministic RNG.
pub seed: u64,
}
impl Default for GeneticAlgorithmConfig {
fn default() -> Self {
Self {
population_size: 100,
generations: 200,
tournament_size: 2,
elitism: 2,
seed: 42,
}
}
}
/// Single-objective generational genetic algorithm with elitism.
///
/// Each generation: binary tournament selection (on the configured
/// `tournament_size`) chooses parent pairs, the variation operator
/// produces offspring, those are evaluated, and the next population is
/// the top `elitism` from the previous generation plus the best
/// `population_size - elitism` offspring (by fitness).
#[derive(Debug, Clone)]
pub struct GeneticAlgorithm<I, V> {
/// Algorithm configuration.
pub config: GeneticAlgorithmConfig,
/// Initial-decision sampler.
pub initializer: I,
/// Offspring-producing variation operator.
pub variation: V,
}
impl<I, V> GeneticAlgorithm<I, V> {
/// Construct a `GeneticAlgorithm`.
pub fn new(config: GeneticAlgorithmConfig, initializer: I, variation: V) -> Self {
Self {
config,
initializer,
variation,
}
}
}
impl<P, I, V> Optimizer<P> for GeneticAlgorithm<I, V>
where
P: Problem + Sync,
P::Decision: Send,
I: Initializer<P::Decision>,
V: Variation<P::Decision>,
{
fn run(&mut self, problem: &P) -> OptimizationResult<P::Decision> {
assert!(
self.config.population_size >= 2,
"GeneticAlgorithm population_size must be >= 2",
);
assert!(
self.config.tournament_size >= 1,
"GeneticAlgorithm tournament_size must be >= 1",
);
assert!(
self.config.elitism < self.config.population_size,
"GeneticAlgorithm elitism must be < population_size",
);
let n = self.config.population_size;
let objectives = problem.objectives();
assert!(
objectives.is_single_objective(),
"GeneticAlgorithm requires exactly one objective",
);
let direction = objectives.objectives[0].direction;
let mut rng = rng_from_seed(self.config.seed);
let initial_decisions = self.initializer.initialize(n, &mut rng);
let mut population: Vec<Candidate<P::Decision>> =
evaluate_batch(problem, initial_decisions);
let mut evaluations = population.len();
for _ in 0..self.config.generations {
// --- Phase 1: parent selection + variation (serial RNG) ---
let mut offspring_decisions: Vec<P::Decision> = Vec::with_capacity(n);
while offspring_decisions.len() < n {
let parents_decisions = tournament_select_single_objective(
&population,
&objectives,
self.config.tournament_size,
2,
&mut rng,
);
let children = self.variation.vary(&parents_decisions, &mut rng);
assert!(
!children.is_empty(),
"GeneticAlgorithm variation returned no children"
);
for child in children {
if offspring_decisions.len() >= n {
break;
}
offspring_decisions.push(child);
}
}
// --- Phase 2: parallel-friendly batch evaluation ---
let offspring = evaluate_batch(problem, offspring_decisions);
evaluations += offspring.len();
// --- Phase 3: survival = elites + best offspring ---
population =
survival_selection(&population, offspring, direction, n, self.config.elitism);
}
let best = best_candidate(&population, &objectives);
let front: Vec<Candidate<P::Decision>> = best.iter().cloned().collect();
OptimizationResult::new(
Population::new(population),
front,
best,
evaluations,
self.config.generations,
)
}
}
fn survival_selection<D: Clone>(
parents: &[Candidate<D>],
offspring: Vec<Candidate<D>>,
direction: Direction,
n: usize,
elitism: usize,
) -> Vec<Candidate<D>> {
// Sort the parents by fitness descending (best first).
let mut sorted_parents: Vec<Candidate<D>> = parents.to_vec();
sorted_parents.sort_by(|a, b| compare_for_fitness(a, b, direction));
// Sort the offspring the same way.
let mut sorted_offspring = offspring;
sorted_offspring.sort_by(|a, b| compare_for_fitness(a, b, direction));
let mut next: Vec<Candidate<D>> = Vec::with_capacity(n);
next.extend(sorted_parents.into_iter().take(elitism));
next.extend(sorted_offspring.into_iter().take(n - elitism));
next
}
/// Order such that "best" comes first. Feasible beats infeasible; among
/// infeasibles, lower violation wins; among feasibles, direction-aware
/// objective comparison.
fn compare_for_fitness<D>(
a: &Candidate<D>,
b: &Candidate<D>,
direction: Direction,
) -> std::cmp::Ordering {
match (a.evaluation.is_feasible(), b.evaluation.is_feasible()) {
(true, false) => std::cmp::Ordering::Less,
(false, true) => std::cmp::Ordering::Greater,
(false, false) => a
.evaluation
.constraint_violation
.partial_cmp(&b.evaluation.constraint_violation)
.unwrap_or(std::cmp::Ordering::Equal),
(true, true) => match direction {
Direction::Minimize => a.evaluation.objectives[0]
.partial_cmp(&b.evaluation.objectives[0])
.unwrap_or(std::cmp::Ordering::Equal),
Direction::Maximize => b.evaluation.objectives[0]
.partial_cmp(&a.evaluation.objectives[0])
.unwrap_or(std::cmp::Ordering::Equal),
},
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::operators::{
CompositeVariation, PolynomialMutation, RealBounds, SimulatedBinaryCrossover,
};
use crate::tests_support::{SchafferN1, Sphere1D};
fn make_optimizer(
seed: u64,
) -> GeneticAlgorithm<
RealBounds,
CompositeVariation<SimulatedBinaryCrossover, PolynomialMutation>,
> {
let bounds = vec![(-5.0, 5.0)];
let initializer = RealBounds::new(bounds.clone());
let variation = CompositeVariation {
crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5),
mutation: PolynomialMutation::new(bounds, 20.0, 1.0),
};
GeneticAlgorithm::new(
GeneticAlgorithmConfig {
population_size: 30,
generations: 50,
tournament_size: 2,
elitism: 2,
seed,
},
initializer,
variation,
)
}
#[test]
fn finds_minimum_of_sphere() {
let mut opt = make_optimizer(1);
let r = opt.run(&Sphere1D);
let best = r.best.unwrap();
assert!(
best.evaluation.objectives[0] < 1e-2,
"got f = {}",
best.evaluation.objectives[0],
);
}
#[test]
fn deterministic_with_same_seed() {
let mut a = make_optimizer(99);
let mut b = make_optimizer(99);
let ra = a.run(&Sphere1D);
let rb = b.run(&Sphere1D);
assert_eq!(
ra.best.unwrap().evaluation.objectives,
rb.best.unwrap().evaluation.objectives,
);
}
#[test]
#[should_panic(expected = "exactly one objective")]
fn multi_objective_panics() {
let mut opt = make_optimizer(0);
let _ = opt.run(&SchafferN1);
}
#[test]
#[should_panic(expected = "elitism must be < population_size")]
fn elitism_too_large_panics() {
let bounds = vec![(-1.0, 1.0)];
let initializer = RealBounds::new(bounds.clone());
let variation = CompositeVariation {
crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5),
mutation: PolynomialMutation::new(bounds, 20.0, 1.0),
};
let mut opt = GeneticAlgorithm::new(
GeneticAlgorithmConfig {
population_size: 4,
generations: 1,
tournament_size: 2,
elitism: 4,
seed: 0,
},
initializer,
variation,
);
let _ = opt.run(&Sphere1D);
}
}
src/algorithms/grea.rs
+281
View File
@@ -0,0 +1,281 @@
//! `Grea` — Yang, Li, Liu & Zheng 2013 Grid-based Evolutionary Algorithm.
use rand::Rng as _;
use crate::algorithms::parallel_eval::evaluate_batch;
use crate::core::candidate::Candidate;
use crate::core::objective::ObjectiveSpace;
use crate::core::population::Population;
use crate::core::problem::Problem;
use crate::core::result::OptimizationResult;
use crate::core::rng::rng_from_seed;
use crate::pareto::front::{best_candidate, pareto_front};
use crate::pareto::sort::non_dominated_sort;
use crate::traits::{Initializer, Optimizer, Variation};
/// Configuration for [`Grea`].
#[derive(Debug, Clone)]
pub struct GreaConfig {
/// Constant population size.
pub population_size: usize,
/// Number of generations.
pub generations: usize,
/// Grid divisions per objective axis.
pub grid_divisions: usize,
/// Seed for the deterministic RNG.
pub seed: u64,
}
impl Default for GreaConfig {
fn default() -> Self {
Self {
population_size: 100,
generations: 250,
grid_divisions: 8,
seed: 42,
}
}
}
/// Grid-based Evolutionary Algorithm (GrEA).
#[derive(Debug, Clone)]
pub struct Grea<I, V> {
/// Algorithm configuration.
pub config: GreaConfig,
/// Initial-decision sampler.
pub initializer: I,
/// Offspring-producing variation operator.
pub variation: V,
}
impl<I, V> Grea<I, V> {
/// Construct a `Grea`.
pub fn new(config: GreaConfig, initializer: I, variation: V) -> Self {
Self {
config,
initializer,
variation,
}
}
}
impl<P, I, V> Optimizer<P> for Grea<I, V>
where
P: Problem + Sync,
P::Decision: Send,
I: Initializer<P::Decision>,
V: Variation<P::Decision>,
{
fn run(&mut self, problem: &P) -> OptimizationResult<P::Decision> {
assert!(
self.config.population_size > 0,
"Grea population_size must be > 0"
);
assert!(
self.config.grid_divisions >= 1,
"Grea grid_divisions must be >= 1"
);
let n = self.config.population_size;
let objectives = problem.objectives();
let mut rng = rng_from_seed(self.config.seed);
let initial_decisions = self.initializer.initialize(n, &mut rng);
let mut population: Vec<Candidate<P::Decision>> =
evaluate_batch(problem, initial_decisions);
let mut evaluations = population.len();
for _ in 0..self.config.generations {
// Random parent selection + variation.
let mut offspring_decisions: Vec<P::Decision> = Vec::with_capacity(n);
while offspring_decisions.len() < n {
let p1 = rng.random_range(0..population.len());
let p2 = rng.random_range(0..population.len());
let parents = vec![
population[p1].decision.clone(),
population[p2].decision.clone(),
];
let children = self.variation.vary(&parents, &mut rng);
assert!(!children.is_empty(), "Grea variation returned no children");
for child in children {
if offspring_decisions.len() >= n {
break;
}
offspring_decisions.push(child);
}
}
let offspring = evaluate_batch(problem, offspring_decisions);
evaluations += offspring.len();
// Survival.
let mut combined: Vec<Candidate<P::Decision>> = Vec::with_capacity(2 * n);
combined.extend(population);
combined.extend(offspring);
population =
environmental_selection(combined, &objectives, n, self.config.grid_divisions);
}
let front = pareto_front(&population, &objectives);
let best = best_candidate(&population, &objectives);
OptimizationResult::new(
Population::new(population),
front,
best,
evaluations,
self.config.generations,
)
}
}
fn environmental_selection<D: Clone>(
combined: Vec<Candidate<D>>,
objectives: &ObjectiveSpace,
n: usize,
divisions: usize,
) -> Vec<Candidate<D>> {
let fronts = non_dominated_sort(&combined, objectives);
let mut selected: Vec<usize> = Vec::with_capacity(n);
let mut splitting: Vec<usize> = Vec::new();
for f in &fronts {
if selected.len() + f.len() <= n {
selected.extend(f.iter().copied());
} else {
splitting = f.clone();
break;
}
if selected.len() == n {
break;
}
}
if selected.len() == n {
return selected.into_iter().map(|i| combined[i].clone()).collect();
}
// Build grid + per-member coordinates on the splitting front (using
// its own min/max per axis to define the grid box).
let m = objectives.len();
let oriented: Vec<Vec<f64>> = splitting
.iter()
.map(|&i| objectives.as_minimization(&combined[i].evaluation.objectives))
.collect();
let mut lo = vec![f64::INFINITY; m];
let mut hi = vec![f64::NEG_INFINITY; m];
for o in &oriented {
for k in 0..m {
if o[k] < lo[k] {
lo[k] = o[k];
}
if o[k] > hi[k] {
hi[k] = o[k];
}
}
}
let grid_coords: Vec<Vec<usize>> = oriented
.iter()
.map(|o| {
(0..m)
.map(|k| {
let span = (hi[k] - lo[k]).max(1e-12);
let frac = ((o[k] - lo[k]) / span).clamp(0.0, 1.0 - 1e-9);
(frac * divisions as f64) as usize
})
.collect()
})
.collect();
let scores: Vec<(usize, usize, isize, isize)> = (0..splitting.len())
.map(|local_idx| {
let gr: usize = grid_coords[local_idx].iter().sum();
// GCD: count of other splitting members in adjacent grid cells.
let mut gcd = 0_isize;
for j in 0..splitting.len() {
if j == local_idx {
continue;
}
let max_diff: usize = (0..m)
.map(|k| grid_coords[local_idx][k].abs_diff(grid_coords[j][k]))
.max()
.unwrap_or(0);
if max_diff < 1 {
gcd += 1;
}
}
// GCPD: grid coordinate point distance to that cell's "ideal"
// origin. We negate to keep "smaller is better" through the
// sort key.
let gcpd: isize = grid_coords[local_idx]
.iter()
.map(|&c| (c as isize).pow(2))
.sum::<isize>();
(local_idx, gr, gcd, gcpd)
})
.collect();
// Sort by (GR ascending, GCD ascending, GCPD ascending).
let mut sorted_scores = scores;
sorted_scores.sort_by(|a, b| {
a.1.cmp(&b.1)
.then_with(|| a.2.cmp(&b.2))
.then_with(|| a.3.cmp(&b.3))
});
let need = n - selected.len();
for (local_idx, _, _, _) in sorted_scores.into_iter().take(need) {
selected.push(splitting[local_idx]);
}
selected.into_iter().map(|i| combined[i].clone()).collect()
}
#[cfg(test)]
mod tests {
use super::*;
use crate::operators::{
CompositeVariation, PolynomialMutation, RealBounds, SimulatedBinaryCrossover,
};
use crate::tests_support::SchafferN1;
fn make_optimizer(
seed: u64,
) -> Grea<RealBounds, CompositeVariation<SimulatedBinaryCrossover, PolynomialMutation>> {
let bounds = vec![(-5.0, 5.0)];
let initializer = RealBounds::new(bounds.clone());
let variation = CompositeVariation {
crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5),
mutation: PolynomialMutation::new(bounds, 20.0, 1.0),
};
Grea::new(
GreaConfig {
population_size: 20,
generations: 15,
grid_divisions: 8,
seed,
},
initializer,
variation,
)
}
#[test]
fn produces_pareto_front() {
let mut opt = make_optimizer(1);
let r = opt.run(&SchafferN1);
assert!(!r.pareto_front.is_empty());
}
#[test]
fn deterministic_with_same_seed() {
let mut a = make_optimizer(99);
let mut b = make_optimizer(99);
let ra = a.run(&SchafferN1);
let rb = b.run(&SchafferN1);
let oa: Vec<Vec<f64>> = ra
.pareto_front
.iter()
.map(|c| c.evaluation.objectives.clone())
.collect();
let ob: Vec<Vec<f64>> = rb
.pareto_front
.iter()
.map(|c| c.evaluation.objectives.clone())
.collect();
assert_eq!(oa, ob);
}
}
src/algorithms/hill_climber.rs
+171
View File
@@ -0,0 +1,171 @@
//! `HillClimber` — single-objective greedy local search.
use crate::core::candidate::Candidate;
use crate::core::objective::Direction;
use crate::core::population::Population;
use crate::core::problem::Problem;
use crate::core::result::OptimizationResult;
use crate::core::rng::rng_from_seed;
use crate::traits::{Initializer, Optimizer, Variation};
/// Configuration for [`HillClimber`].
#[derive(Debug, Clone)]
pub struct HillClimberConfig {
/// Number of mutation iterations.
pub iterations: usize,
/// Seed for the deterministic RNG.
pub seed: u64,
}
impl Default for HillClimberConfig {
fn default() -> Self {
Self {
iterations: 1000,
seed: 42,
}
}
}
/// Single-objective greedy hill climber.
///
/// Starts from one initializer-sampled decision, repeatedly mutates it via
/// the variation operator, and keeps the child only when it is strictly
/// better than the current incumbent. Standard feasibility tiebreaks apply:
/// feasible beats infeasible, smaller violation wins among infeasibles.
///
/// Single-objective only.
#[derive(Debug, Clone)]
pub struct HillClimber<I, V> {
/// Algorithm configuration.
pub config: HillClimberConfig,
/// Initial-decision sampler.
pub initializer: I,
/// Mutation operator.
pub variation: V,
}
impl<I, V> HillClimber<I, V> {
/// Construct a `HillClimber`.
pub fn new(config: HillClimberConfig, initializer: I, variation: V) -> Self {
Self {
config,
initializer,
variation,
}
}
}
impl<P, I, V> Optimizer<P> for HillClimber<I, V>
where
P: Problem + Sync,
P::Decision: Send,
I: Initializer<P::Decision>,
V: Variation<P::Decision>,
{
fn run(&mut self, problem: &P) -> OptimizationResult<P::Decision> {
let objectives = problem.objectives();
assert!(
objectives.is_single_objective(),
"HillClimber requires exactly one objective",
);
let direction = objectives.objectives[0].direction;
let mut rng = rng_from_seed(self.config.seed);
let mut initial = self.initializer.initialize(1, &mut rng);
assert!(
!initial.is_empty(),
"HillClimber initializer returned no decisions"
);
let mut current_decision = initial.remove(0);
let mut current_eval = problem.evaluate(&current_decision);
let mut evaluations = 1usize;
for _ in 0..self.config.iterations {
let parents = vec![current_decision.clone()];
let children = self.variation.vary(&parents, &mut rng);
assert!(
!children.is_empty(),
"HillClimber variation returned no children"
);
let child_decision = children.into_iter().next().unwrap();
let child_eval = problem.evaluate(&child_decision);
evaluations += 1;
let child_better = match (child_eval.is_feasible(), current_eval.is_feasible()) {
(true, false) => true,
(false, true) => false,
(false, false) => {
child_eval.constraint_violation < current_eval.constraint_violation
}
(true, true) => match direction {
Direction::Minimize => child_eval.objectives[0] < current_eval.objectives[0],
Direction::Maximize => child_eval.objectives[0] > current_eval.objectives[0],
},
};
if child_better {
current_decision = child_decision;
current_eval = child_eval;
}
}
let best = Candidate::new(current_decision, current_eval);
let population = Population::new(vec![best.clone()]);
let front = vec![best.clone()];
OptimizationResult::new(
population,
front,
Some(best),
evaluations,
self.config.iterations,
)
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::operators::{GaussianMutation, RealBounds};
use crate::tests_support::{SchafferN1, Sphere1D};
fn make_optimizer(seed: u64) -> HillClimber<RealBounds, GaussianMutation> {
HillClimber::new(
HillClimberConfig {
iterations: 500,
seed,
},
RealBounds::new(vec![(-5.0, 5.0)]),
GaussianMutation { sigma: 0.3 },
)
}
#[test]
fn finds_minimum_of_sphere() {
let mut opt = make_optimizer(1);
let r = opt.run(&Sphere1D);
let best = r.best.unwrap();
assert!(
best.evaluation.objectives[0] < 1e-2,
"got f = {}",
best.evaluation.objectives[0]
);
}
#[test]
fn deterministic_with_same_seed() {
let mut a = make_optimizer(99);
let mut b = make_optimizer(99);
let ra = a.run(&Sphere1D);
let rb = b.run(&Sphere1D);
assert_eq!(
ra.best.unwrap().evaluation.objectives,
rb.best.unwrap().evaluation.objectives,
);
}
#[test]
#[should_panic(expected = "exactly one objective")]
fn multi_objective_panics() {
let mut opt = make_optimizer(0);
let _ = opt.run(&SchafferN1);
}
}
src/algorithms/hype.rs
+381
View File
@@ -0,0 +1,381 @@
//! `Hype` — Bader & Zitzler 2011 Hypervolume Estimation Algorithm.
//!
//! HypE replaces the exact hypervolume contribution used in SMS-EMOA with
//! a Monte Carlo estimate, so it scales to arbitrary objective counts at
//! the cost of stochastic noise on the contribution estimate.
use rand::Rng as _;
use crate::algorithms::parallel_eval::evaluate_batch;
use crate::core::candidate::Candidate;
use crate::core::objective::ObjectiveSpace;
use crate::core::population::Population;
use crate::core::problem::Problem;
use crate::core::result::OptimizationResult;
use crate::core::rng::{Rng, rng_from_seed};
use crate::pareto::front::{best_candidate, pareto_front};
use crate::pareto::sort::non_dominated_sort;
use crate::traits::{Initializer, Optimizer, Variation};
/// Configuration for [`Hype`].
#[derive(Debug, Clone)]
pub struct HypeConfig {
/// Constant population size.
pub population_size: usize,
/// Number of generations.
pub generations: usize,
/// Reference point used to bound the Monte Carlo integration box.
/// Must have one entry per objective; should be worse than every
/// realistic objective value.
pub reference_point: Vec<f64>,
/// Number of Monte Carlo samples per HV estimation step.
pub mc_samples: usize,
/// Seed for the deterministic RNG.
pub seed: u64,
}
impl Default for HypeConfig {
fn default() -> Self {
Self {
population_size: 100,
generations: 250,
reference_point: vec![11.0, 11.0],
mc_samples: 10_000,
seed: 42,
}
}
}
/// Hypervolume Estimation Algorithm: many-objective MOEA that selects via
/// Monte Carlo–estimated hypervolume contributions.
#[derive(Debug, Clone)]
pub struct Hype<I, V> {
/// Algorithm configuration.
pub config: HypeConfig,
/// Initial-decision sampler.
pub initializer: I,
/// Offspring-producing variation operator.
pub variation: V,
}
impl<I, V> Hype<I, V> {
/// Construct a `Hype`.
pub fn new(config: HypeConfig, initializer: I, variation: V) -> Self {
Self {
config,
initializer,
variation,
}
}
}
impl<P, I, V> Optimizer<P> for Hype<I, V>
where
P: Problem + Sync,
P::Decision: Send,
I: Initializer<P::Decision>,
V: Variation<P::Decision>,
{
fn run(&mut self, problem: &P) -> OptimizationResult<P::Decision> {
assert!(
self.config.population_size > 0,
"Hype population_size must be > 0"
);
assert!(self.config.mc_samples > 0, "Hype mc_samples must be > 0");
let n = self.config.population_size;
let objectives = problem.objectives();
assert_eq!(
self.config.reference_point.len(),
objectives.len(),
"Hype reference_point.len() must equal number of objectives",
);
let reference = self.config.reference_point.clone();
let mut rng = rng_from_seed(self.config.seed);
let initial_decisions = self.initializer.initialize(n, &mut rng);
let mut population: Vec<Candidate<P::Decision>> =
evaluate_batch(problem, initial_decisions);
let mut evaluations = population.len();
for _ in 0..self.config.generations {
// Phase 1: parent selection + variation (random tournament on
// a fitness-by-HV-estimate proxy).
let fitness = hype_fitness(
&population,
&objectives,
&reference,
self.config.mc_samples,
&mut rng,
);
let mut offspring_decisions: Vec<P::Decision> = Vec::with_capacity(n);
while offspring_decisions.len() < n {
let p1 = binary_tournament(&fitness, &mut rng);
let p2 = binary_tournament(&fitness, &mut rng);
let parents = vec![
population[p1].decision.clone(),
population[p2].decision.clone(),
];
let children = self.variation.vary(&parents, &mut rng);
assert!(!children.is_empty(), "Hype variation returned no children");
for child in children {
if offspring_decisions.len() >= n {
break;
}
offspring_decisions.push(child);
}
}
// Phase 2: parallel-friendly batch evaluation.
let offspring = evaluate_batch(problem, offspring_decisions);
evaluations += offspring.len();
// Phase 3: combine + survival via front-by-front fill plus
// estimated-contribution truncation on the splitting front.
let mut combined: Vec<Candidate<P::Decision>> = Vec::with_capacity(2 * n);
combined.extend(population);
combined.extend(offspring);
let fronts = non_dominated_sort(&combined, &objectives);
let mut keep_indices: Vec<usize> = Vec::with_capacity(n);
let mut splitting: &[usize] = &[];
for f in &fronts {
if keep_indices.len() + f.len() <= n {
keep_indices.extend(f.iter().copied());
} else {
splitting = f;
break;
}
if keep_indices.len() == n {
break;
}
}
if keep_indices.len() < n {
// Need to choose `n - keep_indices.len()` from `splitting`
// by largest HV contribution.
let pool: Vec<&Candidate<P::Decision>> =
splitting.iter().map(|&i| &combined[i]).collect();
let contributions = estimate_contributions(
&pool,
&objectives,
&reference,
self.config.mc_samples,
&mut rng,
);
let mut order: Vec<usize> = (0..splitting.len()).collect();
order.sort_by(|&a, &b| {
contributions[b]
.partial_cmp(&contributions[a])
.unwrap_or(std::cmp::Ordering::Equal)
});
for k in order.into_iter().take(n - keep_indices.len()) {
keep_indices.push(splitting[k]);
}
}
// Materialize the next generation.
population = keep_indices
.into_iter()
.map(|i| combined[i].clone())
.collect();
}
let front = pareto_front(&population, &objectives);
let best = best_candidate(&population, &objectives);
OptimizationResult::new(
Population::new(population),
front,
best,
evaluations,
self.config.generations,
)
}
}
fn hype_fitness<D>(
pool: &[Candidate<D>],
objectives: &ObjectiveSpace,
reference: &[f64],
samples: usize,
rng: &mut Rng,
) -> Vec<f64> {
if pool.is_empty() {
return Vec::new();
}
let pool_refs: Vec<&Candidate<D>> = pool.iter().collect();
estimate_contributions(&pool_refs, objectives, reference, samples, rng)
}
/// Estimate each candidate's expected unique hypervolume contribution by
/// Monte Carlo sampling uniformly inside the [ideal, reference] box and
/// counting per-sample which candidates dominate it. A sample dominated
/// by exactly one candidate contributes 1/samples × box_volume to that
/// candidate; samples dominated by k candidates contribute proportionally
/// less, weighted by HypE's "weighted hypervolume" rule (1 / k).
fn estimate_contributions<D>(
pool: &[&Candidate<D>],
objectives: &ObjectiveSpace,
reference: &[f64],
samples: usize,
rng: &mut Rng,
) -> Vec<f64> {
let n = pool.len();
if n == 0 {
return Vec::new();
}
let m = reference.len();
// Cache minimization-oriented objective values.
let oriented: Vec<Vec<f64>> = pool
.iter()
.map(|c| objectives.as_minimization(&c.evaluation.objectives))
.collect();
// Compute the lower bound (ideal) of the integration box: per-axis min
// across the population, capped at the reference (so the box has
// non-negative width even if no point dominates the reference).
let mut lower = vec![f64::INFINITY; m];
for o in &oriented {
for (k, &v) in o.iter().enumerate() {
if v < lower[k] {
lower[k] = v;
}
}
}
for k in 0..m {
if !lower[k].is_finite() || lower[k] >= reference[k] {
// No point on this axis dominates the reference → zero
// contribution everywhere.
return vec![0.0; n];
}
}
let box_volume: f64 = (0..m).map(|k| reference[k] - lower[k]).product();
if box_volume <= 0.0 {
return vec![0.0; n];
}
let mut contrib = vec![0.0_f64; n];
let mut sample = vec![0.0_f64; m];
for _ in 0..samples {
for k in 0..m {
let u: f64 = rng.random();
sample[k] = lower[k] + u * (reference[k] - lower[k]);
}
// Count and identify candidates that dominate this sample (point
// in the box).
let mut dominators: Vec<usize> = Vec::with_capacity(n);
for (i, o) in oriented.iter().enumerate() {
if o.iter().zip(sample.iter()).all(|(p, s)| *p <= *s) {
dominators.push(i);
}
}
if dominators.is_empty() {
continue;
}
// HypE weighting: each sample contributes 1/k to each of its k
// dominators. (This generalizes "exactly-one dominator" to
// arbitrary multiplicities.)
let weight = 1.0 / dominators.len() as f64;
for i in dominators {
contrib[i] += weight;
}
}
let scale = box_volume / samples as f64;
contrib.into_iter().map(|c| c * scale).collect()
}
fn binary_tournament(fitness: &[f64], rng: &mut Rng) -> usize {
let a = rng.random_range(0..fitness.len());
let b = rng.random_range(0..fitness.len());
if fitness[a] > fitness[b] {
a
} else if fitness[a] < fitness[b] {
b
} else if rng.random_bool(0.5) {
a
} else {
b
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::operators::{
CompositeVariation, PolynomialMutation, RealBounds, SimulatedBinaryCrossover,
};
use crate::tests_support::SchafferN1;
fn make_optimizer(
seed: u64,
) -> Hype<RealBounds, CompositeVariation<SimulatedBinaryCrossover, PolynomialMutation>> {
let bounds = vec![(-5.0, 5.0)];
let initializer = RealBounds::new(bounds.clone());
let variation = CompositeVariation {
crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5),
mutation: PolynomialMutation::new(bounds, 20.0, 1.0),
};
Hype::new(
HypeConfig {
population_size: 20,
generations: 15,
reference_point: vec![30.0, 30.0],
mc_samples: 1_000,
seed,
},
initializer,
variation,
)
}
#[test]
fn produces_pareto_front() {
let mut opt = make_optimizer(1);
let r = opt.run(&SchafferN1);
assert_eq!(r.population.len(), 20);
assert!(!r.pareto_front.is_empty());
}
#[test]
fn deterministic_with_same_seed() {
let mut a = make_optimizer(99);
let mut b = make_optimizer(99);
let ra = a.run(&SchafferN1);
let rb = b.run(&SchafferN1);
let oa: Vec<Vec<f64>> = ra
.pareto_front
.iter()
.map(|c| c.evaluation.objectives.clone())
.collect();
let ob: Vec<Vec<f64>> = rb
.pareto_front
.iter()
.map(|c| c.evaluation.objectives.clone())
.collect();
assert_eq!(oa, ob);
}
#[test]
#[should_panic(expected = "reference_point.len() must equal number of objectives")]
fn dim_mismatch_panics() {
let bounds = vec![(0.0, 1.0)];
let initializer = RealBounds::new(bounds.clone());
let variation = CompositeVariation {
crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5),
mutation: PolynomialMutation::new(bounds, 20.0, 1.0),
};
let mut opt = Hype::new(
HypeConfig {
population_size: 4,
generations: 1,
reference_point: vec![1.0, 1.0, 1.0],
mc_samples: 100,
seed: 0,
},
initializer,
variation,
);
let _ = opt.run(&SchafferN1);
}
}
src/algorithms/hyperband.rs
+297
View File
@@ -0,0 +1,297 @@
//! `Hyperband` — Li et al. 2017 multi-fidelity hyperparameter optimizer
//! built on Successive Halving (Karnin et al. 2013).
use crate::core::candidate::Candidate;
use crate::core::evaluation::Evaluation;
use crate::core::objective::Direction;
use crate::core::partial_problem::PartialProblem;
use crate::core::population::Population;
use crate::core::result::OptimizationResult;
use crate::core::rng::rng_from_seed;
use crate::traits::Initializer;
/// Configuration for [`Hyperband`].
#[derive(Debug, Clone)]
pub struct HyperbandConfig {
/// Maximum fidelity budget per configuration. Common units: epochs,
/// timesteps, simulation iterations.
pub max_budget: f64,
/// Reduction factor `η`. Each Successive-Halving round survives
/// `1/η` of configurations and promotes them to `η×` budget. Li
/// et al. recommend 3 (which gives smin=1) or 4 (slightly more
/// aggressive promotion).
pub eta: f64,
/// Maximum number of brackets. The standard formula is
/// `floor(log_η(max_budget)) + 1`; pass a larger value to allow
/// it, smaller to truncate.
pub max_brackets: usize,
/// Seed for the deterministic RNG used to sample configurations.
pub seed: u64,
}
impl Default for HyperbandConfig {
fn default() -> Self {
Self {
max_budget: 81.0,
eta: 3.0,
max_brackets: 5,
seed: 42,
}
}
}
/// Hyperband: a budget-aware single-objective optimizer for problems
/// where each evaluation can be performed at a tunable *fidelity*
/// (e.g. an ML training run for `budget` epochs).
///
/// Each "bracket" is a Successive-Halving sweep that starts with many
/// configurations at low budget and progressively promotes the top
/// `1/η` fraction to higher budgets, eliminating the rest. Hyperband
/// runs several brackets with different (configurations, budget)
/// trade-offs — early brackets favor exploration (many configs at
/// low budget), later brackets favor exploitation (fewer configs run
/// near the max budget). The single best result across all brackets
/// is returned.
pub struct Hyperband<I, D>
where
D: Clone,
I: Initializer<D>,
{
/// Algorithm configuration.
pub config: HyperbandConfig,
/// Random configuration sampler (same trait used everywhere else).
pub initializer: I,
_marker: std::marker::PhantomData<D>,
}
impl<I, D> Hyperband<I, D>
where
D: Clone,
I: Initializer<D>,
{
/// Construct a `Hyperband`.
pub fn new(config: HyperbandConfig, initializer: I) -> Self {
Self {
config,
initializer,
_marker: std::marker::PhantomData,
}
}
/// Run Hyperband on a multi-fidelity problem, returning the standard
/// `OptimizationResult`. Single-objective only.
pub fn run<P>(&mut self, problem: &P) -> OptimizationResult<D>
where
P: PartialProblem<Decision = D>,
{
assert!(
self.config.max_budget > 0.0,
"Hyperband max_budget must be > 0"
);
assert!(self.config.eta > 1.0, "Hyperband eta must be > 1");
assert!(
self.config.max_brackets >= 1,
"Hyperband max_brackets must be >= 1"
);
let objectives = problem.objectives();
assert!(
objectives.is_single_objective(),
"Hyperband requires exactly one objective",
);
let direction = objectives.objectives[0].direction;
let mut rng = rng_from_seed(self.config.seed);
// Number of brackets s_max = floor(log_η(max_budget)).
let s_max = (self.config.max_budget.ln() / self.config.eta.ln()).floor() as i64;
let s_max = (s_max as usize).min(self.config.max_brackets);
let mut total_evaluations = 0usize;
let mut total_iterations = 0usize;
let mut best_seen: Option<Candidate<D>> = None;
// Brackets are indexed s = s_max, s_max - 1, ..., 0.
for s in (0..=s_max).rev() {
let s_f = s as f64;
let n =
((s_max as f64 + 1.0) / (s_f + 1.0) * self.config.eta.powf(s_f)).ceil() as usize;
let r = self.config.max_budget / self.config.eta.powf(s_f);
// Sample n configurations.
let mut configs: Vec<D> = self.initializer.initialize(n, &mut rng);
// SH inner loop.
for i in 0..=s {
let n_i = (n as f64 / self.config.eta.powi(i as i32)).floor() as usize;
let r_i = r * self.config.eta.powi(i as i32);
if configs.is_empty() {
break;
}
let evals: Vec<Evaluation> = configs
.iter()
.map(|c| problem.evaluate_at_budget(c, r_i))
.collect();
total_evaluations += configs.len();
// Track best.
for (cfg, e) in configs.iter().zip(evals.iter()) {
let beats = match &best_seen {
None => true,
Some(b) => better(e, &b.evaluation, direction),
};
if beats {
best_seen = Some(Candidate::new(cfg.clone(), e.clone()));
}
}
total_iterations += 1;
// Top n_{i+1} survive.
let next_size = (n_i / self.config.eta as usize).max(1);
if next_size >= configs.len() {
continue;
}
let mut order: Vec<usize> = (0..configs.len()).collect();
order.sort_by(|&a, &b| compare(&evals[a], &evals[b], direction));
let keep: std::collections::HashSet<usize> =
order.into_iter().take(next_size).collect();
let new_configs: Vec<D> = configs
.into_iter()
.enumerate()
.filter_map(|(idx, c)| if keep.contains(&idx) { Some(c) } else { None })
.collect();
configs = new_configs;
}
}
let best = best_seen.expect("at least one bracket ran");
let population = Population::new(vec![best.clone()]);
let front = vec![best.clone()];
OptimizationResult::new(
population,
front,
Some(best),
total_evaluations,
total_iterations,
)
}
}
fn compare(a: &Evaluation, b: &Evaluation, direction: Direction) -> std::cmp::Ordering {
match (a.is_feasible(), b.is_feasible()) {
(true, false) => std::cmp::Ordering::Less,
(false, true) => std::cmp::Ordering::Greater,
(false, false) => a
.constraint_violation
.partial_cmp(&b.constraint_violation)
.unwrap_or(std::cmp::Ordering::Equal),
(true, true) => match direction {
Direction::Minimize => a.objectives[0]
.partial_cmp(&b.objectives[0])
.unwrap_or(std::cmp::Ordering::Equal),
Direction::Maximize => b.objectives[0]
.partial_cmp(&a.objectives[0])
.unwrap_or(std::cmp::Ordering::Equal),
},
}
}
fn better(a: &Evaluation, b: &Evaluation, direction: Direction) -> bool {
compare(a, b, direction) == std::cmp::Ordering::Less
}
#[cfg(test)]
mod tests {
use super::*;
use crate::core::evaluation::Evaluation;
use crate::core::objective::{Objective, ObjectiveSpace};
use crate::operators::real::RealBounds;
/// A multi-fidelity Sphere1D where higher budgets give a less noisy
/// estimate of `f(x) = x[0]²`.
struct NoisySphere {
noise_decay: f64, // higher noise_decay = less noise per unit budget
}
impl PartialProblem for NoisySphere {
type Decision = Vec<f64>;
fn objectives(&self) -> ObjectiveSpace {
ObjectiveSpace::new(vec![Objective::minimize("f")])
}
fn evaluate_at_budget(&self, x: &Vec<f64>, budget: f64) -> Evaluation {
// Pure Sphere; the budget controls how much "noise" we add
// (deterministic — no RNG so the test is reproducible).
// Higher budget → smaller residual.
let true_f = x[0] * x[0];
let residual = (1.0 / (budget * self.noise_decay)).min(10.0);
Evaluation::new(vec![true_f + residual])
}
}
#[test]
fn hyperband_finds_minimum() {
let problem = NoisySphere { noise_decay: 1.0 };
let mut opt = Hyperband::new(
HyperbandConfig {
max_budget: 81.0,
eta: 3.0,
max_brackets: 4,
seed: 1,
},
RealBounds::new(vec![(-5.0, 5.0)]),
);
let r = opt.run(&problem);
let best = r.best.unwrap();
// The "true" minimum of Sphere is 0; but at finite budget the
// residual term keeps it from being zero. A good run should at
// least clearly beat random.
assert!(
best.evaluation.objectives[0] < 0.5,
"got f = {}",
best.evaluation.objectives[0],
);
assert!(r.evaluations > 0);
}
#[test]
fn hyperband_deterministic_with_same_seed() {
let make = || {
Hyperband::new(
HyperbandConfig {
max_budget: 27.0,
eta: 3.0,
max_brackets: 3,
seed: 99,
},
RealBounds::new(vec![(-5.0, 5.0)]),
)
};
let problem = NoisySphere { noise_decay: 1.0 };
let mut a = make();
let mut b = make();
let ra = a.run(&problem);
let rb = b.run(&problem);
assert_eq!(
ra.best.unwrap().evaluation.objectives,
rb.best.unwrap().evaluation.objectives,
);
}
#[test]
#[should_panic(expected = "exactly one objective")]
fn hyperband_multi_objective_panics() {
struct MultiObj;
impl PartialProblem for MultiObj {
type Decision = Vec<f64>;
fn objectives(&self) -> ObjectiveSpace {
ObjectiveSpace::new(vec![Objective::minimize("a"), Objective::minimize("b")])
}
fn evaluate_at_budget(&self, _: &Vec<f64>, _: f64) -> Evaluation {
Evaluation::new(vec![0.0, 0.0])
}
}
let mut opt = Hyperband::new(
HyperbandConfig::default(),
RealBounds::new(vec![(0.0, 1.0)]),
);
let _ = opt.run(&MultiObj);
}
}
src/algorithms/ibea.rs
+357
View File
@@ -0,0 +1,357 @@
//! `Ibea` — Zitzler & Künzli 2004 Indicator-Based Evolutionary Algorithm.
use rand::Rng as _;
use crate::algorithms::parallel_eval::evaluate_batch;
use crate::core::candidate::Candidate;
use crate::core::objective::ObjectiveSpace;
use crate::core::population::Population;
use crate::core::problem::Problem;
use crate::core::result::OptimizationResult;
use crate::core::rng::{Rng, rng_from_seed};
use crate::pareto::front::{best_candidate, pareto_front};
use crate::traits::{Initializer, Optimizer, Variation};
/// Configuration for [`Ibea`].
#[derive(Debug, Clone)]
pub struct IbeaConfig {
/// Constant population size carried across generations.
pub population_size: usize,
/// Number of generations.
pub generations: usize,
/// Indicator scaling factor `κ`. Default 0.05 (Zitzler & Künzli §3.2).
pub kappa: f64,
/// Seed for the deterministic RNG.
pub seed: u64,
}
impl Default for IbeaConfig {
fn default() -> Self {
Self {
population_size: 100,
generations: 250,
kappa: 0.05,
seed: 42,
}
}
}
/// IBEA (Indicator-Based EA) using the additive ε-indicator.
#[derive(Debug, Clone)]
pub struct Ibea<I, V> {
/// Algorithm configuration.
pub config: IbeaConfig,
/// Initial-decision sampler.
pub initializer: I,
/// Offspring-producing variation operator.
pub variation: V,
}
impl<I, V> Ibea<I, V> {
/// Construct an `Ibea` optimizer.
pub fn new(config: IbeaConfig, initializer: I, variation: V) -> Self {
Self {
config,
initializer,
variation,
}
}
}
impl<P, I, V> Optimizer<P> for Ibea<I, V>
where
P: Problem + Sync,
P::Decision: Send,
I: Initializer<P::Decision>,
V: Variation<P::Decision>,
{
fn run(&mut self, problem: &P) -> OptimizationResult<P::Decision> {
assert!(
self.config.population_size > 0,
"Ibea population_size must be > 0"
);
assert!(self.config.kappa > 0.0, "Ibea kappa must be > 0");
let n = self.config.population_size;
let objectives = problem.objectives();
let mut rng = rng_from_seed(self.config.seed);
// Initial population.
let initial_decisions = self.initializer.initialize(n, &mut rng);
let mut population: Vec<Candidate<P::Decision>> =
evaluate_batch(problem, initial_decisions);
let mut evaluations = population.len();
for _ in 0..self.config.generations {
// --- Phase 1: parent selection (binary tournament on fitness) ---
let fitness = compute_fitness(&population, &objectives, self.config.kappa);
let mut offspring_decisions: Vec<P::Decision> = Vec::with_capacity(n);
while offspring_decisions.len() < n {
let p1 = binary_tournament(&fitness, &mut rng);
let p2 = binary_tournament(&fitness, &mut rng);
let parents = vec![
population[p1].decision.clone(),
population[p2].decision.clone(),
];
let children = self.variation.vary(&parents, &mut rng);
assert!(!children.is_empty(), "Ibea variation returned no children");
for child in children {
if offspring_decisions.len() >= n {
break;
}
offspring_decisions.push(child);
}
}
// --- Phase 2: parallel-friendly batch evaluation ---
let offspring = evaluate_batch(problem, offspring_decisions);
evaluations += offspring.len();
// --- Phase 3: combine + indicator-based survival ---
let mut combined: Vec<Candidate<P::Decision>> = Vec::with_capacity(2 * n);
combined.extend(population);
combined.extend(offspring);
population = environmental_selection(combined, &objectives, n, self.config.kappa);
}
let front = pareto_front(&population, &objectives);
let best = best_candidate(&population, &objectives);
OptimizationResult::new(
Population::new(population),
front,
best,
evaluations,
self.config.generations,
)
}
}
/// Iteratively remove the worst-fitness member from `pool` until `n` remain.
///
/// IBEA's standard "subtract the dropped member's contribution from every
/// survivor's fitness" recomputation is implemented here so we don't have
/// to rebuild the full O(N²·M) indicator matrix each removal.
fn environmental_selection<D: Clone>(
mut pool: Vec<Candidate<D>>,
objectives: &ObjectiveSpace,
n: usize,
kappa: f64,
) -> Vec<Candidate<D>> {
if pool.len() <= n {
return pool;
}
let oriented: Vec<Vec<f64>> = pool
.iter()
.map(|c| objectives.as_minimization(&c.evaluation.objectives))
.collect();
// Indicator matrix: indicator[i][j] = max_k (oriented[i][k] - oriented[j][k]).
let indicator: Vec<Vec<f64>> = (0..pool.len())
.map(|i| {
(0..pool.len())
.map(|j| {
if i == j {
0.0
} else {
oriented[i]
.iter()
.zip(oriented[j].iter())
.map(|(a, b)| a - b)
.fold(f64::NEG_INFINITY, f64::max)
}
})
.collect()
})
.collect();
// Normalize indicator by its global magnitude to keep exp() sane.
let mut max_abs = 1e-12_f64;
for row in &indicator {
for &v in row {
if v.abs() > max_abs {
max_abs = v.abs();
}
}
}
// Fitness F(i) = -Σ_{j≠i} exp(-indicator[j][i] / (max_abs · kappa)).
// (Higher is better — so a candidate dominated by many is heavily negative.)
let scale = max_abs * kappa;
let mut fitness: Vec<f64> = (0..pool.len())
.map(|i| {
(0..pool.len())
.filter(|&j| j != i)
.map(|j| -(-indicator[j][i] / scale).exp())
.sum()
})
.collect();
let mut alive: Vec<bool> = vec![true; pool.len()];
let mut alive_count = pool.len();
while alive_count > n {
// Find the lowest-fitness alive member.
let mut worst = usize::MAX;
for i in 0..pool.len() {
if !alive[i] {
continue;
}
if worst == usize::MAX || fitness[i] < fitness[worst] {
worst = i;
}
}
// Remove its contribution from every other survivor's fitness.
for i in 0..pool.len() {
if !alive[i] || i == worst {
continue;
}
fitness[i] += (-indicator[worst][i] / scale).exp();
}
alive[worst] = false;
alive_count -= 1;
}
// Materialize survivors, in original order.
let mut survivors = Vec::with_capacity(n);
for (i, c) in pool.drain(..).enumerate() {
if alive[i] {
survivors.push(c);
}
}
survivors
}
/// Compute IBEA fitness without mutating, for use in tournament selection.
fn compute_fitness<D>(pool: &[Candidate<D>], objectives: &ObjectiveSpace, kappa: f64) -> Vec<f64> {
if pool.is_empty() {
return Vec::new();
}
let oriented: Vec<Vec<f64>> = pool
.iter()
.map(|c| objectives.as_minimization(&c.evaluation.objectives))
.collect();
let indicator: Vec<Vec<f64>> = (0..pool.len())
.map(|i| {
(0..pool.len())
.map(|j| {
if i == j {
0.0
} else {
oriented[i]
.iter()
.zip(oriented[j].iter())
.map(|(a, b)| a - b)
.fold(f64::NEG_INFINITY, f64::max)
}
})
.collect()
})
.collect();
let mut max_abs = 1e-12_f64;
for row in &indicator {
for &v in row {
if v.abs() > max_abs {
max_abs = v.abs();
}
}
}
let scale = max_abs * kappa;
(0..pool.len())
.map(|i| {
(0..pool.len())
.filter(|&j| j != i)
.map(|j| -(-indicator[j][i] / scale).exp())
.sum()
})
.collect()
}
fn binary_tournament(fitness: &[f64], rng: &mut Rng) -> usize {
let a = rng.random_range(0..fitness.len());
let b = rng.random_range(0..fitness.len());
if fitness[a] > fitness[b] {
a
} else if fitness[a] < fitness[b] {
b
} else if rng.random_bool(0.5) {
a
} else {
b
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::operators::{
CompositeVariation, PolynomialMutation, RealBounds, SimulatedBinaryCrossover,
};
use crate::tests_support::SchafferN1;
fn make_optimizer(
seed: u64,
) -> Ibea<RealBounds, CompositeVariation<SimulatedBinaryCrossover, PolynomialMutation>> {
let bounds = vec![(-5.0, 5.0)];
let initializer = RealBounds::new(bounds.clone());
let variation = CompositeVariation {
crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5),
mutation: PolynomialMutation::new(bounds, 20.0, 1.0),
};
Ibea::new(
IbeaConfig {
population_size: 20,
generations: 15,
kappa: 0.05,
seed,
},
initializer,
variation,
)
}
#[test]
fn produces_pareto_front() {
let mut opt = make_optimizer(1);
let r = opt.run(&SchafferN1);
assert!(!r.pareto_front.is_empty());
assert_eq!(r.population.len(), 20);
}
#[test]
fn deterministic_with_same_seed() {
let mut a = make_optimizer(99);
let mut b = make_optimizer(99);
let ra = a.run(&SchafferN1);
let rb = b.run(&SchafferN1);
let oa: Vec<Vec<f64>> = ra
.pareto_front
.iter()
.map(|c| c.evaluation.objectives.clone())
.collect();
let ob: Vec<Vec<f64>> = rb
.pareto_front
.iter()
.map(|c| c.evaluation.objectives.clone())
.collect();
assert_eq!(oa, ob);
}
#[test]
#[should_panic(expected = "population_size must be > 0")]
fn zero_population_size_panics() {
let bounds = vec![(0.0, 1.0)];
let initializer = RealBounds::new(bounds.clone());
let variation = CompositeVariation {
crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5),
mutation: PolynomialMutation::new(bounds, 20.0, 1.0),
};
let mut opt = Ibea::new(
IbeaConfig {
population_size: 0,
generations: 1,
kappa: 0.05,
seed: 0,
},
initializer,
variation,
);
let _ = opt.run(&SchafferN1);
}
}
src/algorithms/ipop_cma_es.rs
+263
View File
@@ -0,0 +1,263 @@
//! `IpopCmaEs` — Auger & Hansen 2005 Increasing-Population CMA-ES.
//!
//! Wraps `CmaEs` in a restart loop that doubles the population size and
//! re-randomizes the initial mean each restart. This is the standard fix
//! for vanilla CMA-ES's well-known weakness on multimodal problems.
use rand::Rng as _;
use crate::algorithms::cma_es::{CmaEs, CmaEsConfig};
use crate::core::candidate::Candidate;
use crate::core::evaluation::Evaluation;
use crate::core::objective::Direction;
use crate::core::population::Population;
use crate::core::problem::Problem;
use crate::core::result::OptimizationResult;
use crate::core::rng::rng_from_seed;
use crate::operators::real::RealBounds;
use crate::traits::Optimizer;
/// Configuration for [`IpopCmaEs`].
#[derive(Debug, Clone)]
pub struct IpopCmaEsConfig {
/// Initial population size for the first CMA-ES restart. Each
/// subsequent restart doubles this.
pub initial_population_size: usize,
/// Total number of generations across ALL restarts. Each restart
/// consumes generations proportional to its population size; the
/// outer loop stops once this budget is exhausted.
pub total_generations: usize,
/// Initial step size σ_0 for every restart.
pub initial_sigma: f64,
/// CMA-ES eigen-decomposition refresh period (passed through).
pub eigen_decomposition_period: usize,
/// Generations of no-improvement that triggers a restart from inside
/// a single CMA-ES run. None disables this trigger (only the outer
/// budget terminates restarts).
pub stall_generations: Option<usize>,
/// Seed for the deterministic RNG.
pub seed: u64,
}
impl Default for IpopCmaEsConfig {
fn default() -> Self {
Self {
initial_population_size: 16,
total_generations: 500,
initial_sigma: 0.5,
eigen_decomposition_period: 1,
stall_generations: Some(50),
seed: 42,
}
}
}
/// IPOP-CMA-ES: CMA-ES with population-doubling restarts.
#[derive(Debug, Clone)]
pub struct IpopCmaEs {
/// Algorithm configuration.
pub config: IpopCmaEsConfig,
/// Per-variable bounds.
pub bounds: RealBounds,
}
impl IpopCmaEs {
/// Construct an `IpopCmaEs`.
pub fn new(config: IpopCmaEsConfig, bounds: RealBounds) -> Self {
Self { config, bounds }
}
}
impl<P> Optimizer<P> for IpopCmaEs
where
P: Problem<Decision = Vec<f64>> + Sync,
{
fn run(&mut self, problem: &P) -> OptimizationResult<P::Decision> {
assert!(
self.config.initial_population_size >= 4,
"IpopCmaEs initial_population_size must be >= 4",
);
let objectives = problem.objectives();
assert!(
objectives.is_single_objective(),
"IpopCmaEs requires exactly one objective",
);
let direction = objectives.objectives[0].direction;
let mut rng = rng_from_seed(self.config.seed);
let mut remaining_gens = self.config.total_generations;
let mut pop_size = self.config.initial_population_size;
let mut total_evaluations = 0usize;
let mut total_iterations = 0usize;
let mut best_seen: Option<Candidate<Vec<f64>>> = None;
let _ = self.config.stall_generations; // reserved for future trigger
let mut restart_counter = 0u64;
while remaining_gens > 0 {
// Per-restart budget: roughly `total / 2^restart` generations,
// with a sensible floor.
let this_gens = (remaining_gens / 2).max(20).min(remaining_gens);
let inner_seed = self
.config
.seed
.wrapping_add(restart_counter.wrapping_mul(0x9E37_79B9_7F4A_7C15));
// Re-randomize the inner mean to a uniform-random point inside
// the original bounds, keeping the bounds box itself unchanged
// so search isn't artificially restricted.
let restart_mean: Vec<f64> = self
.bounds
.bounds
.iter()
.map(|&(lo, hi)| lo + (hi - lo) * rng.random::<f64>())
.collect();
let cfg = CmaEsConfig {
population_size: pop_size,
generations: this_gens,
initial_sigma: self.config.initial_sigma,
eigen_decomposition_period: self.config.eigen_decomposition_period,
initial_mean: Some(restart_mean),
seed: inner_seed,
};
let inner = CmaEs::new(cfg, RealBounds::new(self.bounds.bounds.clone()));
let mut inner = inner;
let result = inner.run(problem);
total_evaluations += result.evaluations;
total_iterations += result.generations;
if let Some(b) = result.best.clone() {
let beats = match &best_seen {
None => true,
Some(prev) => better(&b.evaluation, &prev.evaluation, direction),
};
if beats {
best_seen = Some(b);
}
}
remaining_gens = remaining_gens.saturating_sub(this_gens);
pop_size = pop_size.saturating_mul(2);
restart_counter = restart_counter.wrapping_add(1);
}
let best = best_seen.expect("at least one restart ran");
let population = Population::new(vec![best.clone()]);
let front = vec![best.clone()];
OptimizationResult::new(
population,
front,
Some(best),
total_evaluations,
total_iterations,
)
}
}
fn better(a: &Evaluation, b: &Evaluation, direction: Direction) -> bool {
match (a.is_feasible(), b.is_feasible()) {
(true, false) => true,
(false, true) => false,
(false, false) => a.constraint_violation < b.constraint_violation,
(true, true) => match direction {
Direction::Minimize => a.objectives[0] < b.objectives[0],
Direction::Maximize => a.objectives[0] > b.objectives[0],
},
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::core::evaluation::Evaluation;
use crate::core::objective::{Objective, ObjectiveSpace};
use crate::tests_support::{SchafferN1, Sphere1D};
use std::f64::consts::PI;
/// 5-D Rastrigin to exercise the restart benefit.
struct Rastrigin5D;
impl Problem for Rastrigin5D {
type Decision = Vec<f64>;
fn objectives(&self) -> ObjectiveSpace {
ObjectiveSpace::new(vec![Objective::minimize("f")])
}
fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
let n = x.len() as f64;
let v = 10.0 * n
+ x.iter()
.map(|v| v * v - 10.0 * (2.0 * PI * v).cos())
.sum::<f64>();
Evaluation::new(vec![v])
}
}
fn make_optimizer(seed: u64) -> IpopCmaEs {
IpopCmaEs::new(
IpopCmaEsConfig {
initial_population_size: 8,
total_generations: 300,
initial_sigma: 1.0,
eigen_decomposition_period: 1,
stall_generations: None,
seed,
},
RealBounds::new(vec![(-5.12, 5.12); 5]),
)
}
#[test]
fn finds_minimum_of_sphere() {
let mut opt = IpopCmaEs::new(
IpopCmaEsConfig {
initial_population_size: 8,
total_generations: 100,
initial_sigma: 0.5,
eigen_decomposition_period: 1,
stall_generations: None,
seed: 1,
},
RealBounds::new(vec![(-5.0, 5.0)]),
);
let r = opt.run(&Sphere1D);
let best = r.best.unwrap();
assert!(
best.evaluation.objectives[0] < 1e-8,
"got f = {}",
best.evaluation.objectives[0],
);
}
#[test]
fn produces_reasonable_rastrigin_result() {
// Don't claim a strict beat-vanilla threshold (that's a stochastic
// statement); just verify IPOP runs to completion and produces a
// result clearly better than random sampling on a 5-D Rastrigin
// (random would average f ≈ 11–12).
let mut opt = make_optimizer(1);
let r = opt.run(&Rastrigin5D);
let best = r.best.unwrap();
assert!(
best.evaluation.objectives[0] < 5.0,
"IPOP-CMA-ES underperformed on Rastrigin: f = {}",
best.evaluation.objectives[0],
);
}
#[test]
fn deterministic_with_same_seed() {
let mut a = make_optimizer(99);
let mut b = make_optimizer(99);
let ra = a.run(&Rastrigin5D);
let rb = b.run(&Rastrigin5D);
assert_eq!(
ra.best.unwrap().evaluation.objectives,
rb.best.unwrap().evaluation.objectives,
);
}
#[test]
#[should_panic(expected = "exactly one objective")]
fn multi_objective_panics() {
let mut opt = make_optimizer(0);
let _ = opt.run(&SchafferN1);
}
}
src/algorithms/knea.rs
+284
View File
@@ -0,0 +1,284 @@
//! `Knea` — Zhang, Tian & Jin 2015 Knee point-driven EA.
use rand::Rng as _;
use crate::algorithms::parallel_eval::evaluate_batch;
use crate::core::candidate::Candidate;
use crate::core::objective::ObjectiveSpace;
use crate::core::population::Population;
use crate::core::problem::Problem;
use crate::core::result::OptimizationResult;
use crate::core::rng::rng_from_seed;
use crate::pareto::front::{best_candidate, pareto_front};
use crate::pareto::sort::non_dominated_sort;
use crate::traits::{Initializer, Optimizer, Variation};
/// Configuration for [`Knea`].
#[derive(Debug, Clone)]
pub struct KneaConfig {
/// Constant population size.
pub population_size: usize,
/// Number of generations.
pub generations: usize,
/// Seed for the deterministic RNG.
pub seed: u64,
}
impl Default for KneaConfig {
fn default() -> Self {
Self {
population_size: 100,
generations: 250,
seed: 42,
}
}
}
/// Knee point-driven Evolutionary Algorithm.
///
/// Survival selection ranks splitting-front members by perpendicular
/// distance from the hyperplane connecting the front's extreme points.
/// Larger distance ≈ stronger knee = preferred survivor.
#[derive(Debug, Clone)]
pub struct Knea<I, V> {
/// Algorithm configuration.
pub config: KneaConfig,
/// Initial-decision sampler.
pub initializer: I,
/// Offspring-producing variation operator.
pub variation: V,
}
impl<I, V> Knea<I, V> {
/// Construct a `Knea`.
pub fn new(config: KneaConfig, initializer: I, variation: V) -> Self {
Self {
config,
initializer,
variation,
}
}
}
impl<P, I, V> Optimizer<P> for Knea<I, V>
where
P: Problem + Sync,
P::Decision: Send,
I: Initializer<P::Decision>,
V: Variation<P::Decision>,
{
fn run(&mut self, problem: &P) -> OptimizationResult<P::Decision> {
assert!(
self.config.population_size > 0,
"Knea population_size must be > 0"
);
let n = self.config.population_size;
let objectives = problem.objectives();
let mut rng = rng_from_seed(self.config.seed);
let initial_decisions = self.initializer.initialize(n, &mut rng);
let mut population: Vec<Candidate<P::Decision>> =
evaluate_batch(problem, initial_decisions);
let mut evaluations = population.len();
for _ in 0..self.config.generations {
let mut offspring_decisions: Vec<P::Decision> = Vec::with_capacity(n);
while offspring_decisions.len() < n {
let p1 = rng.random_range(0..population.len());
let p2 = rng.random_range(0..population.len());
let parents = vec![
population[p1].decision.clone(),
population[p2].decision.clone(),
];
let children = self.variation.vary(&parents, &mut rng);
assert!(!children.is_empty(), "Knea variation returned no children");
for child in children {
if offspring_decisions.len() >= n {
break;
}
offspring_decisions.push(child);
}
}
let offspring = evaluate_batch(problem, offspring_decisions);
evaluations += offspring.len();
let mut combined: Vec<Candidate<P::Decision>> = Vec::with_capacity(2 * n);
combined.extend(population);
combined.extend(offspring);
population = environmental_selection(combined, &objectives, n);
}
let front = pareto_front(&population, &objectives);
let best = best_candidate(&population, &objectives);
OptimizationResult::new(
Population::new(population),
front,
best,
evaluations,
self.config.generations,
)
}
}
fn environmental_selection<D: Clone>(
combined: Vec<Candidate<D>>,
objectives: &ObjectiveSpace,
n: usize,
) -> Vec<Candidate<D>> {
let fronts = non_dominated_sort(&combined, objectives);
let mut selected: Vec<usize> = Vec::with_capacity(n);
let mut splitting: Vec<usize> = Vec::new();
for f in &fronts {
if selected.len() + f.len() <= n {
selected.extend(f.iter().copied());
} else {
splitting = f.clone();
break;
}
if selected.len() == n {
break;
}
}
if selected.len() == n {
return selected.into_iter().map(|i| combined[i].clone()).collect();
}
// Compute knee distances for splitting front.
let m = objectives.len();
let oriented: Vec<Vec<f64>> = splitting
.iter()
.map(|&i| objectives.as_minimization(&combined[i].evaluation.objectives))
.collect();
// Per-axis ideal and nadir on the splitting front.
let mut ideal = vec![f64::INFINITY; m];
let mut nadir = vec![f64::NEG_INFINITY; m];
for o in &oriented {
for k in 0..m {
if o[k] < ideal[k] {
ideal[k] = o[k];
}
if o[k] > nadir[k] {
nadir[k] = o[k];
}
}
}
// Hyperplane through the M extreme points: f · normal = c.
// We approximate the hyperplane connecting the per-axis nadirs.
// The "extreme points" here are M points each maximizing one axis.
let extremes: Vec<usize> = (0..m)
.map(|axis| {
let mut best = 0;
let mut best_val = f64::NEG_INFINITY;
for (idx, o) in oriented.iter().enumerate() {
if o[axis] > best_val {
best_val = o[axis];
best = idx;
}
}
best
})
.collect();
// Knee distance for each splitting member: signed distance from the
// hyperplane defined by the extremes. We use a simple
// "distance-to-line-segment" surrogate for 2D, and the M-D extension
// is the perpendicular distance to the hyperplane through the M
// extreme points.
let distances: Vec<f64> = (0..splitting.len())
.map(|i| perpendicular_distance(&oriented[i], &extremes, &oriented))
.collect();
// Sort splitting indices by largest distance (= strongest knee).
let mut order: Vec<usize> = (0..splitting.len()).collect();
order.sort_by(|&a, &b| {
distances[b]
.partial_cmp(&distances[a])
.unwrap_or(std::cmp::Ordering::Equal)
});
let need = n - selected.len();
for k in order.into_iter().take(need) {
selected.push(splitting[k]);
}
selected.into_iter().map(|i| combined[i].clone()).collect()
}
/// Perpendicular distance from `point` to the hyperplane through the M
/// extreme points (indices into `oriented`).
fn perpendicular_distance(point: &[f64], extremes: &[usize], oriented: &[Vec<f64>]) -> f64 {
let m = point.len();
if extremes.len() < m {
// Degenerate: just return the L2 norm relative to first extreme.
if let Some(&e0) = extremes.first() {
return point
.iter()
.zip(oriented[e0].iter())
.map(|(a, b)| (a - b).powi(2))
.sum::<f64>()
.sqrt();
}
return 0.0;
}
// Hyperplane: a · x = b, where a = (1, 1, …, 1) for the canonical
// simplex through extremes — works well when objectives are
// approximately on a simplex.
let a: Vec<f64> = vec![1.0; m];
let b: f64 = oriented[extremes[0]].iter().sum();
let dot: f64 = point.iter().zip(a.iter()).map(|(x, y)| x * y).sum();
let norm: f64 = a.iter().map(|y| y * y).sum::<f64>().sqrt().max(1e-12);
(dot - b).abs() / norm
}
#[cfg(test)]
mod tests {
use super::*;
use crate::operators::{
CompositeVariation, PolynomialMutation, RealBounds, SimulatedBinaryCrossover,
};
use crate::tests_support::SchafferN1;
fn make_optimizer(
seed: u64,
) -> Knea<RealBounds, CompositeVariation<SimulatedBinaryCrossover, PolynomialMutation>> {
let bounds = vec![(-5.0, 5.0)];
let initializer = RealBounds::new(bounds.clone());
let variation = CompositeVariation {
crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5),
mutation: PolynomialMutation::new(bounds, 20.0, 1.0),
};
Knea::new(
KneaConfig {
population_size: 20,
generations: 15,
seed,
},
initializer,
variation,
)
}
#[test]
fn produces_pareto_front() {
let mut opt = make_optimizer(1);
let r = opt.run(&SchafferN1);
assert!(!r.pareto_front.is_empty());
}
#[test]
fn deterministic_with_same_seed() {
let mut a = make_optimizer(99);
let mut b = make_optimizer(99);
let ra = a.run(&SchafferN1);
let rb = b.run(&SchafferN1);
let oa: Vec<Vec<f64>> = ra
.pareto_front
.iter()
.map(|c| c.evaluation.objectives.clone())
.collect();
let ob: Vec<Vec<f64>> = rb
.pareto_front
.iter()
.map(|c| c.evaluation.objectives.clone())
.collect();
assert_eq!(oa, ob);
}
}
src/algorithms/mod.rs
+52
View File
@@ -1,18 +1,70 @@
//! Built-in reference optimizers. //! Built-in reference optimizers.
pub mod age_moea;
pub mod ant_colony_tsp;
pub mod bayesian_opt;
pub mod cma_es;
pub mod differential_evolution; pub mod differential_evolution;
pub mod epsilon_moea;
pub mod genetic_algorithm;
pub mod grea;
pub mod hill_climber;
pub mod hype;
pub mod hyperband;
pub mod ibea;
pub mod ipop_cma_es;
pub mod knea;
pub mod moead; pub mod moead;
pub mod mopso;
pub mod nelder_mead;
pub mod nsga2; pub mod nsga2;
pub mod nsga3; pub mod nsga3;
pub mod one_plus_one_es;
pub mod paes; pub mod paes;
pub(crate) mod parallel_eval; pub(crate) mod parallel_eval;
pub mod particle_swarm;
pub mod pesa2;
pub mod random_search; pub mod random_search;
pub mod rvea;
pub mod simulated_annealing;
pub mod sms_emoa;
pub mod snes;
pub mod spea2; pub mod spea2;
pub mod tabu_search;
pub mod tlbo;
pub mod tpe;
pub mod umda;
pub use age_moea::*;
pub use ant_colony_tsp::*;
pub use bayesian_opt::*;
pub use cma_es::*;
pub use differential_evolution::*; pub use differential_evolution::*;
pub use epsilon_moea::*;
pub use genetic_algorithm::*;
pub use grea::*;
pub use hill_climber::*;
pub use hype::*;
pub use hyperband::*;
pub use ibea::*;
pub use ipop_cma_es::*;
pub use knea::*;
pub use moead::*; pub use moead::*;
pub use mopso::*;
pub use nelder_mead::*;
pub use nsga2::*; pub use nsga2::*;
pub use nsga3::*; pub use nsga3::*;
pub use one_plus_one_es::*;
pub use paes::*; pub use paes::*;
pub use particle_swarm::*;
pub use pesa2::*;
pub use random_search::*; pub use random_search::*;
pub use rvea::*;
pub use simulated_annealing::*;
pub use sms_emoa::*;
pub use snes::*;
pub use spea2::*; pub use spea2::*;
pub use tabu_search::*;
pub use tlbo::*;
pub use tpe::*;
pub use umda::*;
src/algorithms/moead.rs
+22 -12
View File
@@ -52,7 +52,11 @@ pub struct Moead<I, V> {
impl<I, V> Moead<I, V> { impl<I, V> Moead<I, V> {
/// Construct a `Moead` optimizer. /// Construct a `Moead` optimizer.
pub fn new(config: MoeadConfig, initializer: I, variation: V) -> Self { pub fn new(config: MoeadConfig, initializer: I, variation: V) -> Self {
Self { config, initializer, variation } Self {
config,
initializer,
variation,
}
} }
} }
@@ -119,7 +123,8 @@ where
.collect(); .collect();
for _ in 0..self.config.generations { for _ in 0..self.config.generations {
#[allow(clippy::needless_range_loop)] // Body indexes both `neighborhoods[i]` and `population[j]` via `nbh`. #[allow(clippy::needless_range_loop)]
// Body indexes both `neighborhoods[i]` and `population[j]` via `nbh`.
for i in 0..n { for i in 0..n {
// Pick two distinct parents from the neighborhood. // Pick two distinct parents from the neighborhood.
let nbh = &neighborhoods[i]; let nbh = &neighborhoods[i];
@@ -128,10 +133,15 @@ where
while p2 == p1 && nbh.len() > 1 { while p2 == p1 && nbh.len() > 1 {
p2 = *nbh.choose(&mut rng).unwrap(); p2 = *nbh.choose(&mut rng).unwrap();
} }
let parents = let parents = vec![
vec![population[p1].decision.clone(), population[p2].decision.clone()]; population[p1].decision.clone(),
population[p2].decision.clone(),
];
let children = self.variation.vary(&parents, &mut rng); let children = self.variation.vary(&parents, &mut rng);
assert!(!children.is_empty(), "MOEA/D variation returned no children"); assert!(
!children.is_empty(),
"MOEA/D variation returned no children"
);
let child_decision = children.into_iter().next().unwrap(); let child_decision = children.into_iter().next().unwrap();
let child_eval = problem.evaluate(&child_decision); let child_eval = problem.evaluate(&child_decision);
evaluations += 1; evaluations += 1;
@@ -152,8 +162,7 @@ where
let g_cur = tchebycheff(&cur_oriented, &weights[j], &ideal); let g_cur = tchebycheff(&cur_oriented, &weights[j], &ideal);
let g_new = tchebycheff(&oriented_child, &weights[j], &ideal); let g_new = tchebycheff(&oriented_child, &weights[j], &ideal);
if g_new <= g_cur { if g_new <= g_cur {
population[j] = population[j] = Candidate::new(child_decision.clone(), child_eval.clone());
Candidate::new(child_decision.clone(), child_eval.clone());
} }
} }
} }
@@ -188,7 +197,11 @@ fn tchebycheff(oriented_objectives: &[f64], weight: &[f64], ideal: &[f64]) -> f6
} }
fn weight_distance(a: &[f64], b: &[f64]) -> f64 { fn weight_distance(a: &[f64], b: &[f64]) -> f64 {
a.iter().zip(b.iter()).map(|(x, y)| (x - y).powi(2)).sum::<f64>().sqrt() a.iter()
.zip(b.iter())
.map(|(x, y)| (x - y).powi(2))
.sum::<f64>()
.sqrt()
} }
#[cfg(test)] #[cfg(test)]
@@ -201,10 +214,7 @@ mod tests {
fn make_optimizer( fn make_optimizer(
seed: u64, seed: u64,
) -> Moead< ) -> Moead<RealBounds, CompositeVariation<SimulatedBinaryCrossover, PolynomialMutation>> {
RealBounds,
CompositeVariation<SimulatedBinaryCrossover, PolynomialMutation>,
> {
let bounds = vec![(-5.0, 5.0)]; let bounds = vec![(-5.0, 5.0)];
let initializer = RealBounds::new(bounds.clone()); let initializer = RealBounds::new(bounds.clone());
let variation = CompositeVariation { let variation = CompositeVariation {
src/algorithms/mopso.rs
+235
View File
@@ -0,0 +1,235 @@
//! `Mopso` — Coello, Pulido & Lechuga 2004 Multi-Objective Particle Swarm.
use rand::Rng as _;
use rand::seq::IndexedRandom;
use crate::algorithms::parallel_eval::evaluate_batch;
use crate::core::population::Population;
use crate::core::problem::Problem;
use crate::core::result::OptimizationResult;
use crate::core::rng::rng_from_seed;
use crate::operators::real::RealBounds;
use crate::pareto::archive::ParetoArchive;
use crate::pareto::dominance::{Dominance, pareto_compare};
use crate::pareto::front::{best_candidate, pareto_front};
use crate::traits::Optimizer;
/// Configuration for [`Mopso`].
#[derive(Debug, Clone)]
pub struct MopsoConfig {
/// Number of particles in the swarm.
pub swarm_size: usize,
/// Number of generations.
pub generations: usize,
/// External Pareto archive size cap (simple-tail truncation).
pub archive_size: usize,
/// Inertia weight `w`.
pub inertia: f64,
/// Cognitive coefficient `c_1` (toward personal best).
pub cognitive: f64,
/// Social coefficient `c_2` (toward archive leader).
pub social: f64,
/// Seed for the deterministic RNG.
pub seed: u64,
}
impl Default for MopsoConfig {
fn default() -> Self {
Self {
swarm_size: 40,
generations: 200,
archive_size: 100,
inertia: 0.7,
cognitive: 1.5,
social: 1.5,
seed: 42,
}
}
}
/// Multi-objective particle swarm with an external Pareto archive.
///
/// `Vec<f64>` decisions only. Each particle maintains a personal best (the
/// last position that was Pareto-non-dominated by any later position). The
/// social leader is sampled uniformly from the external archive each step.
#[derive(Debug, Clone)]
pub struct Mopso {
/// Algorithm configuration.
pub config: MopsoConfig,
/// Per-variable bounds — used both to seed the swarm and to clamp positions.
pub bounds: RealBounds,
}
impl Mopso {
/// Construct a `Mopso`.
pub fn new(config: MopsoConfig, bounds: RealBounds) -> Self {
Self { config, bounds }
}
}
impl<P> Optimizer<P> for Mopso
where
P: Problem<Decision = Vec<f64>> + Sync,
{
fn run(&mut self, problem: &P) -> OptimizationResult<P::Decision> {
assert!(self.config.swarm_size >= 1, "Mopso swarm_size must be >= 1");
assert!(
self.config.archive_size >= 1,
"Mopso archive_size must be >= 1"
);
let objectives = problem.objectives();
assert!(
objectives.is_multi_objective(),
"Mopso requires multi-objective problems (use ParticleSwarm for single-objective)",
);
let dim = self.bounds.bounds.len();
let n = self.config.swarm_size;
let mut rng = rng_from_seed(self.config.seed);
let mut positions: Vec<Vec<f64>> = {
use crate::traits::Initializer as _;
self.bounds.initialize(n, &mut rng)
};
let mut velocities: Vec<Vec<f64>> = (0..n)
.map(|_| {
self.bounds
.bounds
.iter()
.map(|&(lo, hi)| 0.1 * (hi - lo) * (rng.random::<f64>() * 2.0 - 1.0))
.collect()
})
.collect();
let v_max: Vec<f64> = self.bounds.bounds.iter().map(|&(lo, hi)| hi - lo).collect();
let initial_pop = evaluate_batch(problem, positions.clone());
let mut evaluations = initial_pop.len();
// Personal bests start at initial positions.
let mut pbest_decisions: Vec<Vec<f64>> = positions.clone();
let mut pbest_evals: Vec<crate::core::evaluation::Evaluation> =
initial_pop.iter().map(|c| c.evaluation.clone()).collect();
// External archive seeded with the non-dominated subset.
let mut archive = ParetoArchive::new(objectives.clone());
for c in initial_pop {
archive.insert(c);
}
archive.truncate(self.config.archive_size);
for _ in 0..self.config.generations {
// --- Phase 1: serial position/velocity updates (uses RNG) ---
for i in 0..n {
let leader = archive
.members()
.choose(&mut rng)
.map(|c| c.decision.clone())
.unwrap_or_else(|| positions[i].clone());
#[allow(clippy::needless_range_loop)] // body indexes velocities/positions/bounds.
for j in 0..dim {
let r1: f64 = rng.random();
let r2: f64 = rng.random();
let cognitive_term =
self.config.cognitive * r1 * (pbest_decisions[i][j] - positions[i][j]);
let social_term = self.config.social * r2 * (leader[j] - positions[i][j]);
let mut v =
self.config.inertia * velocities[i][j] + cognitive_term + social_term;
if v > v_max[j] {
v = v_max[j];
} else if v < -v_max[j] {
v = -v_max[j];
}
velocities[i][j] = v;
let (lo, hi) = self.bounds.bounds[j];
positions[i][j] = (positions[i][j] + v).clamp(lo, hi);
}
}
// --- Phase 2: parallel-friendly batch evaluation ---
let evaluated = evaluate_batch(problem, positions.clone());
evaluations += evaluated.len();
// --- Phase 3: serial pbest + archive updates ---
for (i, cand) in evaluated.iter().enumerate() {
let dominance = pareto_compare(&cand.evaluation, &pbest_evals[i], &objectives);
let replace = match dominance {
Dominance::Dominates => true,
Dominance::DominatedBy => false,
Dominance::Equal | Dominance::NonDominated => rng.random_bool(0.5),
};
if replace {
pbest_decisions[i] = cand.decision.clone();
pbest_evals[i] = cand.evaluation.clone();
}
}
for c in evaluated {
archive.insert(c);
}
archive.truncate(self.config.archive_size);
}
let members = archive.into_vec();
let front = pareto_front(&members, &objectives);
let best = best_candidate(&members, &objectives);
OptimizationResult::new(
Population::new(members),
front,
best,
evaluations,
self.config.generations,
)
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::tests_support::{SchafferN1, Sphere1D};
fn make_optimizer(seed: u64) -> Mopso {
Mopso::new(
MopsoConfig {
swarm_size: 30,
generations: 30,
archive_size: 30,
inertia: 0.7,
cognitive: 1.5,
social: 1.5,
seed,
},
RealBounds::new(vec![(-5.0, 5.0)]),
)
}
#[test]
fn produces_pareto_front() {
let mut opt = make_optimizer(1);
let r = opt.run(&SchafferN1);
assert!(!r.pareto_front.is_empty());
}
#[test]
fn deterministic_with_same_seed() {
let mut a = make_optimizer(99);
let mut b = make_optimizer(99);
let ra = a.run(&SchafferN1);
let rb = b.run(&SchafferN1);
let oa: Vec<Vec<f64>> = ra
.pareto_front
.iter()
.map(|c| c.evaluation.objectives.clone())
.collect();
let ob: Vec<Vec<f64>> = rb
.pareto_front
.iter()
.map(|c| c.evaluation.objectives.clone())
.collect();
assert_eq!(oa, ob);
}
#[test]
#[should_panic(expected = "multi-objective")]
fn single_objective_panics() {
let mut opt = make_optimizer(0);
let _ = opt.run(&Sphere1D);
}
}
src/algorithms/nelder_mead.rs
+358
View File
@@ -0,0 +1,358 @@
//! `NelderMead` — Nelder & Mead 1965 simplex direct-search optimizer.
use crate::core::candidate::Candidate;
use crate::core::evaluation::Evaluation;
use crate::core::objective::Direction;
use crate::core::population::Population;
use crate::core::problem::Problem;
use crate::core::result::OptimizationResult;
use crate::operators::real::RealBounds;
use crate::traits::Optimizer;
/// Configuration for [`NelderMead`].
#[derive(Debug, Clone)]
pub struct NelderMeadConfig {
/// Number of iterations.
pub iterations: usize,
/// Reflection coefficient `α` (canonical 1.0).
pub reflection: f64,
/// Expansion coefficient `γ` (canonical 2.0).
pub expansion: f64,
/// Contraction coefficient `ρ` (canonical 0.5).
pub contraction: f64,
/// Shrinkage coefficient `σ` (canonical 0.5).
pub shrinkage: f64,
/// Initial simplex edge length (added to each axis from the start point).
pub initial_step: f64,
}
impl Default for NelderMeadConfig {
fn default() -> Self {
Self {
iterations: 1_000,
reflection: 1.0,
expansion: 2.0,
contraction: 0.5,
shrinkage: 0.5,
initial_step: 0.5,
}
}
}
/// Classical Nelder-Mead simplex method.
///
/// Maintains a simplex of `n+1` vertices in `n`-D, replacing the worst
/// vertex each iteration via reflection / expansion / contraction /
/// shrinkage relative to the centroid of the rest.
///
/// `Vec<f64>` decisions only. Single-objective only. Initial simplex is
/// built around the midpoint of the configured bounds; every new vertex
/// is clamped to those bounds.
#[derive(Debug, Clone)]
pub struct NelderMead {
/// Algorithm configuration.
pub config: NelderMeadConfig,
/// Per-variable bounds — used to seed the simplex midpoint and to clamp
/// every reflected/expanded vertex.
pub bounds: RealBounds,
}
impl NelderMead {
/// Construct a `NelderMead`.
pub fn new(config: NelderMeadConfig, bounds: RealBounds) -> Self {
Self { config, bounds }
}
}
impl<P> Optimizer<P> for NelderMead
where
P: Problem<Decision = Vec<f64>> + Sync,
{
fn run(&mut self, problem: &P) -> OptimizationResult<P::Decision> {
assert!(
self.config.reflection > 0.0,
"NelderMead reflection must be > 0"
);
assert!(
self.config.expansion > 1.0,
"NelderMead expansion must be > 1",
);
assert!(
self.config.contraction > 0.0 && self.config.contraction < 1.0,
"NelderMead contraction must be in (0, 1)",
);
assert!(
self.config.shrinkage > 0.0 && self.config.shrinkage < 1.0,
"NelderMead shrinkage must be in (0, 1)",
);
assert!(
self.config.initial_step > 0.0,
"NelderMead initial_step must be > 0",
);
let objectives = problem.objectives();
assert!(
objectives.is_single_objective(),
"NelderMead requires exactly one objective",
);
let direction = objectives.objectives[0].direction;
let n = self.bounds.bounds.len();
// Seed the simplex: start at the bounds midpoint, then build n
// additional vertices by stepping `initial_step` along each axis.
let mut vertices: Vec<Vec<f64>> = Vec::with_capacity(n + 1);
let start: Vec<f64> = self
.bounds
.bounds
.iter()
.map(|&(lo, hi)| 0.5 * (lo + hi))
.collect();
vertices.push(start.clone());
for j in 0..n {
let mut v = start.clone();
let (lo, hi) = self.bounds.bounds[j];
let step = self.config.initial_step.min(0.5 * (hi - lo));
v[j] = (v[j] + step).clamp(lo, hi);
vertices.push(v);
}
let mut evals: Vec<Evaluation> = vertices.iter().map(|v| problem.evaluate(v)).collect();
let mut evaluations = evals.len();
for _ in 0..self.config.iterations {
// Sort vertices best → worst.
let mut order: Vec<usize> = (0..vertices.len()).collect();
order.sort_by(|&a, &b| compare(&evals[a], &evals[b], direction));
let best_idx = order[0];
let worst_idx = order[order.len() - 1];
let second_worst_idx = order[order.len() - 2];
// Centroid of all vertices except the worst.
let mut centroid = vec![0.0_f64; n];
for &idx in &order[..order.len() - 1] {
for j in 0..n {
centroid[j] += vertices[idx][j];
}
}
for c in centroid.iter_mut() {
*c /= (order.len() - 1) as f64;
}
// Reflection.
let reflected = self.reflect(&centroid, &vertices[worst_idx], self.config.reflection);
let r_eval = problem.evaluate(&reflected);
evaluations += 1;
if better(&r_eval, &evals[best_idx], direction) {
// Reflection beat the best — try expansion.
let expanded = self.reflect(&centroid, &vertices[worst_idx], self.config.expansion);
let e_eval = problem.evaluate(&expanded);
evaluations += 1;
if better(&e_eval, &r_eval, direction) {
vertices[worst_idx] = expanded;
evals[worst_idx] = e_eval;
} else {
vertices[worst_idx] = reflected;
evals[worst_idx] = r_eval;
}
} else if better(&r_eval, &evals[second_worst_idx], direction) {
// Reflection at least beat the second-worst — accept.
vertices[worst_idx] = reflected;
evals[worst_idx] = r_eval;
} else {
// Reflection didn't help — try contraction.
let contraction_target = if better(&r_eval, &evals[worst_idx], direction) {
// Outside contraction (between centroid and reflected).
self.contract(&centroid, &reflected, self.config.contraction)
} else {
// Inside contraction (between centroid and worst).
self.contract(&centroid, &vertices[worst_idx], self.config.contraction)
};
let c_eval = problem.evaluate(&contraction_target);
evaluations += 1;
if better(&c_eval, &evals[worst_idx], direction) {
vertices[worst_idx] = contraction_target;
evals[worst_idx] = c_eval;
} else {
// Shrink: move every non-best vertex toward the best.
let best_pt = vertices[best_idx].clone();
for &idx in &order {
if idx == best_idx {
continue;
}
#[allow(clippy::needless_range_loop)]
// body indexes both vertices and best_pt.
for j in 0..n {
vertices[idx][j] = best_pt[j]
+ self.config.shrinkage * (vertices[idx][j] - best_pt[j]);
}
// Clamp to bounds.
for (j, x) in vertices[idx].iter_mut().enumerate() {
let (lo, hi) = self.bounds.bounds[j];
*x = x.clamp(lo, hi);
}
evals[idx] = problem.evaluate(&vertices[idx]);
evaluations += 1;
}
}
}
}
// Find the best vertex.
let mut best_idx = 0;
for i in 1..vertices.len() {
if better(&evals[i], &evals[best_idx], direction) {
best_idx = i;
}
}
let best = Candidate::new(vertices[best_idx].clone(), evals[best_idx].clone());
let population = Population::new(vec![best.clone()]);
let front = vec![best.clone()];
OptimizationResult::new(
population,
front,
Some(best),
evaluations,
self.config.iterations,
)
}
}
impl NelderMead {
fn reflect(&self, centroid: &[f64], worst: &[f64], coefficient: f64) -> Vec<f64> {
let n = centroid.len();
let mut out = Vec::with_capacity(n);
for j in 0..n {
let v = centroid[j] + coefficient * (centroid[j] - worst[j]);
let (lo, hi) = self.bounds.bounds[j];
out.push(v.clamp(lo, hi));
}
out
}
fn contract(&self, centroid: &[f64], target: &[f64], coefficient: f64) -> Vec<f64> {
let n = centroid.len();
let mut out = Vec::with_capacity(n);
for j in 0..n {
let v = centroid[j] + coefficient * (target[j] - centroid[j]);
let (lo, hi) = self.bounds.bounds[j];
out.push(v.clamp(lo, hi));
}
out
}
}
fn compare(a: &Evaluation, b: &Evaluation, direction: Direction) -> std::cmp::Ordering {
match (a.is_feasible(), b.is_feasible()) {
(true, false) => std::cmp::Ordering::Less,
(false, true) => std::cmp::Ordering::Greater,
(false, false) => a
.constraint_violation
.partial_cmp(&b.constraint_violation)
.unwrap_or(std::cmp::Ordering::Equal),
(true, true) => match direction {
Direction::Minimize => a.objectives[0]
.partial_cmp(&b.objectives[0])
.unwrap_or(std::cmp::Ordering::Equal),
Direction::Maximize => b.objectives[0]
.partial_cmp(&a.objectives[0])
.unwrap_or(std::cmp::Ordering::Equal),
},
}
}
fn better(a: &Evaluation, b: &Evaluation, direction: Direction) -> bool {
compare(a, b, direction) == std::cmp::Ordering::Less
}
#[cfg(test)]
mod tests {
use super::*;
use crate::core::evaluation::Evaluation;
use crate::core::objective::{Objective, ObjectiveSpace};
use crate::tests_support::{SchafferN1, Sphere1D};
/// 2-D Rosenbrock for shape exercise.
struct Rosenbrock2D;
impl Problem for Rosenbrock2D {
type Decision = Vec<f64>;
fn objectives(&self) -> ObjectiveSpace {
ObjectiveSpace::new(vec![Objective::minimize("f")])
}
fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
let a = 1.0 - x[0];
let b = x[1] - x[0] * x[0];
Evaluation::new(vec![a * a + 100.0 * b * b])
}
}
#[test]
fn finds_minimum_of_sphere() {
let mut opt = NelderMead::new(
NelderMeadConfig {
iterations: 200,
..NelderMeadConfig::default()
},
RealBounds::new(vec![(-5.0, 5.0)]),
);
let r = opt.run(&Sphere1D);
let best = r.best.unwrap();
assert!(
best.evaluation.objectives[0] < 1e-8,
"got f = {}",
best.evaluation.objectives[0],
);
}
#[test]
fn finds_minimum_of_2d_rosenbrock() {
let mut opt = NelderMead::new(
NelderMeadConfig {
iterations: 500,
initial_step: 0.5,
..NelderMeadConfig::default()
},
RealBounds::new(vec![(-2.0, 2.0); 2]),
);
let r = opt.run(&Rosenbrock2D);
let best = r.best.unwrap();
assert!(
best.evaluation.objectives[0] < 1e-3,
"got f = {}",
best.evaluation.objectives[0],
);
}
#[test]
fn deterministic_no_rng() {
// Nelder-Mead is purely deterministic — same bounds + same iters
// → same result, no seed needed.
let make = || {
NelderMead::new(
NelderMeadConfig {
iterations: 100,
..NelderMeadConfig::default()
},
RealBounds::new(vec![(-5.0, 5.0)]),
)
};
let mut a = make();
let mut b = make();
let ra = a.run(&Sphere1D);
let rb = b.run(&Sphere1D);
assert_eq!(
ra.best.unwrap().evaluation.objectives,
rb.best.unwrap().evaluation.objectives,
);
}
#[test]
#[should_panic(expected = "exactly one objective")]
fn multi_objective_panics() {
let mut opt = NelderMead::new(
NelderMeadConfig::default(),
RealBounds::new(vec![(-5.0, 5.0)]),
);
let _ = opt.run(&SchafferN1);
}
}
src/algorithms/nsga2.rs
+54 -15
View File
@@ -26,7 +26,11 @@ pub struct Nsga2Config {
impl Default for Nsga2Config { impl Default for Nsga2Config {
fn default() -> Self { fn default() -> Self {
Self { population_size: 100, generations: 250, seed: 42 } Self {
population_size: 100,
generations: 250,
seed: 42,
}
} }
} }
@@ -44,7 +48,11 @@ pub struct Nsga2<I, V> {
impl<I, V> Nsga2<I, V> { impl<I, V> Nsga2<I, V> {
/// Construct an `Nsga2` optimizer. /// Construct an `Nsga2` optimizer.
pub fn new(config: Nsga2Config, initializer: I, variation: V) -> Self { pub fn new(config: Nsga2Config, initializer: I, variation: V) -> Self {
Self { config, initializer, variation } Self {
config,
initializer,
variation,
}
} }
} }
@@ -78,8 +86,7 @@ where
n, n,
"NSGA-II initializer must return exactly population_size decisions", "NSGA-II initializer must return exactly population_size decisions",
); );
let population: Vec<Candidate<P::Decision>> = let population: Vec<Candidate<P::Decision>> = evaluate_batch(problem, initial_decisions);
evaluate_batch(problem, initial_decisions);
let mut evaluations = population.len(); let mut evaluations = population.len();
// Annotate the starting population with rank and crowding so the first // Annotate the starting population with rank and crowding so the first
@@ -130,7 +137,9 @@ where
let dist = crowding_distance(&combined, front, &objectives); let dist = crowding_distance(&combined, front, &objectives);
let mut order: Vec<usize> = (0..front.len()).collect(); let mut order: Vec<usize> = (0..front.len()).collect();
order.sort_by(|&a, &b| { order.sort_by(|&a, &b| {
dist[b].partial_cmp(&dist[a]).unwrap_or(std::cmp::Ordering::Equal) dist[b]
.partial_cmp(&dist[a])
.unwrap_or(std::cmp::Ordering::Equal)
}); });
let needed = n - next.len(); let needed = n - next.len();
for &k in order.iter().take(needed) { for &k in order.iter().take(needed) {
@@ -178,7 +187,11 @@ fn annotate<D: Clone>(
population population
.into_iter() .into_iter()
.enumerate() .enumerate()
.map(|(i, c)| Nsga2Entry { candidate: c, rank: rank[i], crowding_distance: dist[i] }) .map(|(i, c)| Nsga2Entry {
candidate: c,
rank: rank[i],
crowding_distance: dist[i],
})
.collect() .collect()
} }
@@ -212,7 +225,11 @@ mod tests {
#[test] #[test]
fn final_population_has_expected_size() { fn final_population_has_expected_size() {
let mut opt = Nsga2::new( let mut opt = Nsga2::new(
Nsga2Config { population_size: 20, generations: 5, seed: 1 }, Nsga2Config {
population_size: 20,
generations: 5,
seed: 1,
},
RealBounds::new(vec![(-5.0, 5.0)]), RealBounds::new(vec![(-5.0, 5.0)]),
GaussianMutation { sigma: 0.3 }, GaussianMutation { sigma: 0.3 },
); );
@@ -224,7 +241,11 @@ mod tests {
#[test] #[test]
fn evaluation_count_at_least_initial_population() { fn evaluation_count_at_least_initial_population() {
let mut opt = Nsga2::new( let mut opt = Nsga2::new(
Nsga2Config { population_size: 16, generations: 3, seed: 2 }, Nsga2Config {
population_size: 16,
generations: 3,
seed: 2,
},
RealBounds::new(vec![(-5.0, 5.0)]), RealBounds::new(vec![(-5.0, 5.0)]),
GaussianMutation { sigma: 0.3 }, GaussianMutation { sigma: 0.3 },
); );
@@ -236,21 +257,35 @@ mod tests {
#[test] #[test]
fn deterministic_with_same_seed() { fn deterministic_with_same_seed() {
let mut a = Nsga2::new( let mut a = Nsga2::new(
Nsga2Config { population_size: 16, generations: 5, seed: 99 }, Nsga2Config {
population_size: 16,
generations: 5,
seed: 99,
},
RealBounds::new(vec![(-5.0, 5.0)]), RealBounds::new(vec![(-5.0, 5.0)]),
GaussianMutation { sigma: 0.2 }, GaussianMutation { sigma: 0.2 },
); );
let mut b = Nsga2::new( let mut b = Nsga2::new(
Nsga2Config { population_size: 16, generations: 5, seed: 99 }, Nsga2Config {
population_size: 16,
generations: 5,
seed: 99,
},
RealBounds::new(vec![(-5.0, 5.0)]), RealBounds::new(vec![(-5.0, 5.0)]),
GaussianMutation { sigma: 0.2 }, GaussianMutation { sigma: 0.2 },
); );
let ra = a.run(&SchafferN1); let ra = a.run(&SchafferN1);
let rb = b.run(&SchafferN1); let rb = b.run(&SchafferN1);
let oa: Vec<Vec<f64>> = let oa: Vec<Vec<f64>> = ra
ra.pareto_front.iter().map(|c| c.evaluation.objectives.clone()).collect(); .pareto_front
let ob: Vec<Vec<f64>> = .iter()
rb.pareto_front.iter().map(|c| c.evaluation.objectives.clone()).collect(); .map(|c| c.evaluation.objectives.clone())
.collect();
let ob: Vec<Vec<f64>> = rb
.pareto_front
.iter()
.map(|c| c.evaluation.objectives.clone())
.collect();
assert_eq!(oa, ob); assert_eq!(oa, ob);
} }
@@ -258,7 +293,11 @@ mod tests {
#[should_panic(expected = "population_size must be greater than 0")] #[should_panic(expected = "population_size must be greater than 0")]
fn zero_population_size_panics() { fn zero_population_size_panics() {
let mut opt = Nsga2::new( let mut opt = Nsga2::new(
Nsga2Config { population_size: 0, generations: 1, seed: 0 }, Nsga2Config {
population_size: 0,
generations: 1,
seed: 0,
},
RealBounds::new(vec![(-1.0, 1.0)]), RealBounds::new(vec![(-1.0, 1.0)]),
GaussianMutation { sigma: 0.1 }, GaussianMutation { sigma: 0.1 },
); );
src/algorithms/nsga3.rs
+26 -15
View File
@@ -56,7 +56,11 @@ pub struct Nsga3<I, V> {
impl<I, V> Nsga3<I, V> { impl<I, V> Nsga3<I, V> {
/// Construct an `Nsga3` optimizer. /// Construct an `Nsga3` optimizer.
pub fn new(config: Nsga3Config, initializer: I, variation: V) -> Self { pub fn new(config: Nsga3Config, initializer: I, variation: V) -> Self {
Self { config, initializer, variation } Self {
config,
initializer,
variation,
}
} }
} }
@@ -99,8 +103,10 @@ where
while offspring_decisions.len() < n { while offspring_decisions.len() < n {
let p1 = rng.random_range(0..population.len()); let p1 = rng.random_range(0..population.len());
let p2 = rng.random_range(0..population.len()); let p2 = rng.random_range(0..population.len());
let parents = let parents = vec![
vec![population[p1].decision.clone(), population[p2].decision.clone()]; population[p1].decision.clone(),
population[p2].decision.clone(),
];
let children = self.variation.vary(&parents, &mut rng); let children = self.variation.vary(&parents, &mut rng);
assert!( assert!(
!children.is_empty(), !children.is_empty(),
@@ -117,11 +123,11 @@ where
evaluations += offspring.len(); evaluations += offspring.len();
// --- Combine + survival selection --- // --- Combine + survival selection ---
let mut combined: Vec<Candidate<P::Decision>> = let mut combined: Vec<Candidate<P::Decision>> = Vec::with_capacity(2 * n);
Vec::with_capacity(2 * n);
combined.extend(population); combined.extend(population);
combined.extend(offspring); combined.extend(offspring);
population = environmental_selection(&combined, &objectives, &reference_points, n, &mut rng); population =
environmental_selection(&combined, &objectives, &reference_points, n, &mut rng);
} }
let front = pareto_front(&population, &objectives); let front = pareto_front(&population, &objectives);
@@ -205,7 +211,9 @@ fn environmental_selection<D: Clone>(
let candidate_refs: Vec<usize> = (0..reference_points.len()) let candidate_refs: Vec<usize> = (0..reference_points.len())
.filter(|&j| !available_in_fl[j].is_empty() && niche_count[j] == min_count) .filter(|&j| !available_in_fl[j].is_empty() && niche_count[j] == min_count)
.collect(); .collect();
let &chosen_ref = candidate_refs.choose(rng).expect("non-empty by construction"); let &chosen_ref = candidate_refs
.choose(rng)
.expect("non-empty by construction");
let pool = &available_in_fl[chosen_ref]; let pool = &available_in_fl[chosen_ref];
let pick_local = if niche_count[chosen_ref] == 0 { let pick_local = if niche_count[chosen_ref] == 0 {
@@ -420,10 +428,7 @@ mod tests {
fn make_optimizer( fn make_optimizer(
seed: u64, seed: u64,
) -> Nsga3< ) -> Nsga3<RealBounds, CompositeVariation<SimulatedBinaryCrossover, PolynomialMutation>> {
RealBounds,
CompositeVariation<SimulatedBinaryCrossover, PolynomialMutation>,
> {
let bounds = vec![(-5.0, 5.0)]; let bounds = vec![(-5.0, 5.0)];
let initializer = RealBounds::new(bounds.clone()); let initializer = RealBounds::new(bounds.clone());
let variation = CompositeVariation { let variation = CompositeVariation {
@@ -457,10 +462,16 @@ mod tests {
let mut b = make_optimizer(99); let mut b = make_optimizer(99);
let ra = a.run(&SchafferN1); let ra = a.run(&SchafferN1);
let rb = b.run(&SchafferN1); let rb = b.run(&SchafferN1);
let oa: Vec<Vec<f64>> = let oa: Vec<Vec<f64>> = ra
ra.pareto_front.iter().map(|c| c.evaluation.objectives.clone()).collect(); .pareto_front
let ob: Vec<Vec<f64>> = .iter()
rb.pareto_front.iter().map(|c| c.evaluation.objectives.clone()).collect(); .map(|c| c.evaluation.objectives.clone())
.collect();
let ob: Vec<Vec<f64>> = rb
.pareto_front
.iter()
.map(|c| c.evaluation.objectives.clone())
.collect();
assert_eq!(oa, ob); assert_eq!(oa, ob);
} }
src/algorithms/one_plus_one_es.rs
+212
View File
@@ -0,0 +1,212 @@
//! `OnePlusOneEs` — the (1+1) evolution strategy with Rechenberg's
//! one-fifth success rule for σ adaptation.
use rand_distr::{Distribution, Normal};
use crate::core::candidate::Candidate;
use crate::core::evaluation::Evaluation;
use crate::core::objective::Direction;
use crate::core::population::Population;
use crate::core::problem::Problem;
use crate::core::result::OptimizationResult;
use crate::core::rng::rng_from_seed;
use crate::operators::real::RealBounds;
use crate::traits::Optimizer;
/// Configuration for [`OnePlusOneEs`].
#[derive(Debug, Clone)]
pub struct OnePlusOneEsConfig {
/// Number of mutation iterations.
pub iterations: usize,
/// Initial mutation step size (`σ_0`).
pub initial_sigma: f64,
/// Number of recent iterations the success-rate is computed over.
/// The classic value is 10·dim; 50 is a fine default for low-dim
/// problems.
pub adaptation_period: usize,
/// Step-size multiplier when the success rate exceeds 1/5. Reciprocal
/// is applied when the rate is below 1/5. Rechenberg's analytical
/// derivation gives ≈ `0.817^(-1/n)` for dim n; 1.22 is a popular
/// dimension-agnostic value.
pub step_increase: f64,
/// Seed for the deterministic RNG.
pub seed: u64,
}
impl Default for OnePlusOneEsConfig {
fn default() -> Self {
Self {
iterations: 5_000,
initial_sigma: 0.5,
adaptation_period: 50,
step_increase: 1.22,
seed: 42,
}
}
}
/// (1+1)-ES with the one-fifth rule: tiny, parameter-light continuous
/// optimizer. `Vec<f64>` decisions only; single-objective only.
#[derive(Debug, Clone)]
pub struct OnePlusOneEs {
/// Algorithm configuration.
pub config: OnePlusOneEsConfig,
/// Per-variable bounds — used to seed the parent at the box midpoint
/// and clamp every mutated child.
pub bounds: RealBounds,
}
impl OnePlusOneEs {
/// Construct a `OnePlusOneEs`.
pub fn new(config: OnePlusOneEsConfig, bounds: RealBounds) -> Self {
Self { config, bounds }
}
}
impl<P> Optimizer<P> for OnePlusOneEs
where
P: Problem<Decision = Vec<f64>> + Sync,
{
fn run(&mut self, problem: &P) -> OptimizationResult<P::Decision> {
assert!(
self.config.initial_sigma > 0.0,
"OnePlusOneEs initial_sigma must be > 0"
);
assert!(
self.config.step_increase > 1.0,
"OnePlusOneEs step_increase must be > 1",
);
assert!(
self.config.adaptation_period >= 1,
"OnePlusOneEs adaptation_period must be >= 1",
);
let objectives = problem.objectives();
assert!(
objectives.is_single_objective(),
"OnePlusOneEs requires exactly one objective",
);
let direction = objectives.objectives[0].direction;
let mut rng = rng_from_seed(self.config.seed);
// Seed parent at midpoint of bounds.
let mut parent: Vec<f64> = self
.bounds
.bounds
.iter()
.map(|&(lo, hi)| 0.5 * (lo + hi))
.collect();
let mut parent_eval = problem.evaluate(&parent);
let mut evaluations = 1usize;
let mut sigma = self.config.initial_sigma;
let mut window = std::collections::VecDeque::with_capacity(self.config.adaptation_period);
for _ in 0..self.config.iterations {
let normal = Normal::new(0.0, sigma).expect("Normal::new(0, sigma)");
let mut child = parent.clone();
for (j, x) in child.iter_mut().enumerate() {
let (lo, hi) = self.bounds.bounds[j];
*x = (*x + normal.sample(&mut rng)).clamp(lo, hi);
}
let child_eval = problem.evaluate(&child);
evaluations += 1;
// Accept if not strictly worse (so neutral moves are kept and
// can drive σ up when on a plateau).
let accepted = !worse_than(&child_eval, &parent_eval, direction);
if accepted {
parent = child;
parent_eval = child_eval;
}
// Update success window.
window.push_back(if accepted { 1u8 } else { 0u8 });
if window.len() > self.config.adaptation_period {
window.pop_front();
}
// Apply one-fifth rule once we have a full window.
if window.len() == self.config.adaptation_period {
let success_count: usize = window.iter().map(|&b| b as usize).sum();
let rate = success_count as f64 / window.len() as f64;
if rate > 0.2 {
sigma *= self.config.step_increase;
} else if rate < 0.2 {
sigma /= self.config.step_increase;
}
}
}
let best = Candidate::new(parent, parent_eval);
let population = Population::new(vec![best.clone()]);
let front = vec![best.clone()];
OptimizationResult::new(
population,
front,
Some(best),
evaluations,
self.config.iterations,
)
}
}
fn worse_than(a: &Evaluation, b: &Evaluation, direction: Direction) -> bool {
match (a.is_feasible(), b.is_feasible()) {
(false, true) => true,
(true, false) => false,
(false, false) => a.constraint_violation > b.constraint_violation,
(true, true) => match direction {
Direction::Minimize => a.objectives[0] > b.objectives[0],
Direction::Maximize => a.objectives[0] < b.objectives[0],
},
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::tests_support::{SchafferN1, Sphere1D};
fn make_optimizer(seed: u64) -> OnePlusOneEs {
OnePlusOneEs::new(
OnePlusOneEsConfig {
iterations: 2_000,
initial_sigma: 1.0,
adaptation_period: 30,
step_increase: 1.22,
seed,
},
RealBounds::new(vec![(-5.0, 5.0)]),
)
}
#[test]
fn finds_minimum_of_sphere() {
let mut opt = make_optimizer(1);
let r = opt.run(&Sphere1D);
let best = r.best.unwrap();
assert!(
best.evaluation.objectives[0] < 1e-6,
"got f = {}",
best.evaluation.objectives[0],
);
}
#[test]
fn deterministic_with_same_seed() {
let mut a = make_optimizer(99);
let mut b = make_optimizer(99);
let ra = a.run(&Sphere1D);
let rb = b.run(&Sphere1D);
assert_eq!(
ra.best.unwrap().evaluation.objectives,
rb.best.unwrap().evaluation.objectives,
);
}
#[test]
#[should_panic(expected = "exactly one objective")]
fn multi_objective_panics() {
let mut opt = make_optimizer(0);
let _ = opt.run(&SchafferN1);
}
}
src/algorithms/paes.rs
+34 -11
View File
@@ -23,7 +23,11 @@ pub struct PaesConfig {
impl Default for PaesConfig { impl Default for PaesConfig {
fn default() -> Self { fn default() -> Self {
Self { iterations: 1000, archive_size: 100, seed: 42 } Self {
iterations: 1000,
archive_size: 100,
seed: 42,
}
} }
} }
@@ -45,7 +49,11 @@ pub struct Paes<I, V> {
impl<I, V> Paes<I, V> { impl<I, V> Paes<I, V> {
/// Construct a `Paes` optimizer. /// Construct a `Paes` optimizer.
pub fn new(config: PaesConfig, initializer: I, variation: V) -> Self { pub fn new(config: PaesConfig, initializer: I, variation: V) -> Self {
Self { config, initializer, variation } Self {
config,
initializer,
variation,
}
} }
} }
@@ -74,15 +82,15 @@ where
let mut evaluations = 1usize; let mut evaluations = 1usize;
let mut archive = ParetoArchive::new(objectives.clone()); let mut archive = ParetoArchive::new(objectives.clone());
archive.insert(Candidate::new(current_decision.clone(), current_eval.clone())); archive.insert(Candidate::new(
current_decision.clone(),
current_eval.clone(),
));
for _ in 0..self.config.iterations { for _ in 0..self.config.iterations {
let parents = vec![current_decision.clone()]; let parents = vec![current_decision.clone()];
let children = self.variation.vary(&parents, &mut rng); let children = self.variation.vary(&parents, &mut rng);
assert!( assert!(!children.is_empty(), "PAES variation returned no children",);
!children.is_empty(),
"PAES variation returned no children",
);
let child_decision = children.into_iter().next().unwrap(); let child_decision = children.into_iter().next().unwrap();
let child_eval = problem.evaluate(&child_decision); let child_eval = problem.evaluate(&child_decision);
evaluations += 1; evaluations += 1;
@@ -103,7 +111,10 @@ where
} }
archive.insert(Candidate::new(child_decision, child_eval)); archive.insert(Candidate::new(child_decision, child_eval));
archive.insert(Candidate::new(current_decision.clone(), current_eval.clone())); archive.insert(Candidate::new(
current_decision.clone(),
current_eval.clone(),
));
archive.truncate(self.config.archive_size); archive.truncate(self.config.archive_size);
} }
@@ -129,7 +140,11 @@ mod tests {
#[test] #[test]
fn produces_at_least_one_candidate() { fn produces_at_least_one_candidate() {
let mut opt = Paes::new( let mut opt = Paes::new(
PaesConfig { iterations: 50, archive_size: 16, seed: 1 }, PaesConfig {
iterations: 50,
archive_size: 16,
seed: 1,
},
RealBounds::new(vec![(-5.0, 5.0)]), RealBounds::new(vec![(-5.0, 5.0)]),
GaussianMutation { sigma: 0.3 }, GaussianMutation { sigma: 0.3 },
); );
@@ -141,7 +156,11 @@ mod tests {
#[test] #[test]
fn archive_size_respected() { fn archive_size_respected() {
let mut opt = Paes::new( let mut opt = Paes::new(
PaesConfig { iterations: 200, archive_size: 8, seed: 2 }, PaesConfig {
iterations: 200,
archive_size: 8,
seed: 2,
},
RealBounds::new(vec![(-5.0, 5.0)]), RealBounds::new(vec![(-5.0, 5.0)]),
GaussianMutation { sigma: 0.2 }, GaussianMutation { sigma: 0.2 },
); );
@@ -152,7 +171,11 @@ mod tests {
#[test] #[test]
fn single_objective_returns_best() { fn single_objective_returns_best() {
let mut opt = Paes::new( let mut opt = Paes::new(
PaesConfig { iterations: 200, archive_size: 8, seed: 3 }, PaesConfig {
iterations: 200,
archive_size: 8,
seed: 3,
},
RealBounds::new(vec![(-2.0, 2.0)]), RealBounds::new(vec![(-2.0, 2.0)]),
GaussianMutation { sigma: 0.1 }, GaussianMutation { sigma: 0.1 },
); );
src/algorithms/particle_swarm.rs
+255
View File
@@ -0,0 +1,255 @@
//! `ParticleSwarm` — Eberhart & Kennedy 1995 PSO for `Vec<f64>` decisions.
use rand::Rng as _;
use crate::algorithms::parallel_eval::evaluate_batch;
use crate::core::candidate::Candidate;
use crate::core::objective::Direction;
use crate::core::population::Population;
use crate::core::problem::Problem;
use crate::core::result::OptimizationResult;
use crate::core::rng::rng_from_seed;
use crate::operators::real::RealBounds;
use crate::pareto::front::best_candidate;
use crate::traits::Optimizer;
/// Configuration for [`ParticleSwarm`].
#[derive(Debug, Clone)]
pub struct ParticleSwarmConfig {
/// Number of particles in the swarm.
pub swarm_size: usize,
/// Number of generations.
pub generations: usize,
/// Inertia weight `w`. Typical: 0.4–0.9.
pub inertia: f64,
/// Cognitive coefficient `c_1`. Typical: 1.5–2.0.
pub cognitive: f64,
/// Social coefficient `c_2`. Typical: 1.5–2.0.
pub social: f64,
/// Seed for the deterministic RNG.
pub seed: u64,
}
impl Default for ParticleSwarmConfig {
fn default() -> Self {
Self {
swarm_size: 40,
generations: 200,
inertia: 0.7,
cognitive: 1.5,
social: 1.5,
seed: 42,
}
}
}
/// Single-objective real-valued PSO.
///
/// Particles update with the standard inertia-weight rule:
///
/// ```text
/// v[i,t+1] = w·v[i,t] + c1·r1·(pbest[i] - x[i,t]) + c2·r2·(gbest - x[i,t])
/// x[i,t+1] = clamp(x[i,t] + v[i,t+1], bounds)
/// ```
///
/// Velocities are clamped to `±(hi - lo)` per dimension to prevent
/// "swarm explosion." Pair with `RealBounds` for both the search bounds
/// and the initial particle positions.
#[derive(Debug, Clone)]
pub struct ParticleSwarm {
/// Algorithm configuration.
pub config: ParticleSwarmConfig,
/// Per-variable bounds — used both to seed the swarm and to clamp positions.
pub bounds: RealBounds,
}
impl ParticleSwarm {
/// Construct a `ParticleSwarm`.
pub fn new(config: ParticleSwarmConfig, bounds: RealBounds) -> Self {
Self { config, bounds }
}
}
impl<P> Optimizer<P> for ParticleSwarm
where
P: Problem<Decision = Vec<f64>> + Sync,
{
fn run(&mut self, problem: &P) -> OptimizationResult<P::Decision> {
assert!(
self.config.swarm_size >= 1,
"ParticleSwarm swarm_size must be >= 1",
);
let objectives = problem.objectives();
assert!(
objectives.is_single_objective(),
"ParticleSwarm requires exactly one objective",
);
let direction = objectives.objectives[0].direction;
let dim = self.bounds.bounds.len();
let n = self.config.swarm_size;
let mut rng = rng_from_seed(self.config.seed);
// Initialize positions via the bounds initializer.
let mut positions: Vec<Vec<f64>> = {
use crate::traits::Initializer as _;
self.bounds.initialize(n, &mut rng)
};
// Initial velocities: small random perturbations within ±0.1·range.
let mut velocities: Vec<Vec<f64>> = (0..n)
.map(|_| {
self.bounds
.bounds
.iter()
.map(|&(lo, hi)| 0.1 * (hi - lo) * (rng.random::<f64>() * 2.0 - 1.0))
.collect()
})
.collect();
let v_max: Vec<f64> = self.bounds.bounds.iter().map(|&(lo, hi)| hi - lo).collect();
// Initial evaluation.
let initial_pop = evaluate_batch(problem, positions.clone());
let mut evaluations = initial_pop.len();
// Personal bests start at initial positions.
let mut pbest_decisions: Vec<Vec<f64>> = positions.clone();
let mut pbest_evals: Vec<f64> = initial_pop
.iter()
.map(|c| c.evaluation.objectives[0])
.collect();
// Global best.
let mut gbest_idx = best_index(&pbest_evals, direction);
let mut gbest_decision = pbest_decisions[gbest_idx].clone();
let mut gbest_eval = pbest_evals[gbest_idx];
for _ in 0..self.config.generations {
// --- Phase 1: serial position/velocity updates (uses RNG) ---
for i in 0..n {
#[allow(clippy::needless_range_loop)] // body indexes velocities/positions/bounds.
for j in 0..dim {
let r1: f64 = rng.random();
let r2: f64 = rng.random();
let cognitive_term =
self.config.cognitive * r1 * (pbest_decisions[i][j] - positions[i][j]);
let social_term =
self.config.social * r2 * (gbest_decision[j] - positions[i][j]);
let mut v =
self.config.inertia * velocities[i][j] + cognitive_term + social_term;
if v > v_max[j] {
v = v_max[j];
} else if v < -v_max[j] {
v = -v_max[j];
}
velocities[i][j] = v;
let (lo, hi) = self.bounds.bounds[j];
positions[i][j] = (positions[i][j] + v).clamp(lo, hi);
}
}
// --- Phase 2: parallel-friendly batch evaluation ---
let evaluated = evaluate_batch(problem, positions.clone());
evaluations += evaluated.len();
// --- Phase 3: serial pbest / gbest updates ---
for (i, cand) in evaluated.iter().enumerate() {
let f = cand.evaluation.objectives[0];
let improves = match direction {
Direction::Minimize => f < pbest_evals[i],
Direction::Maximize => f > pbest_evals[i],
};
if improves {
pbest_decisions[i] = positions[i].clone();
pbest_evals[i] = f;
gbest_idx = i;
let beats_global = match direction {
Direction::Minimize => f < gbest_eval,
Direction::Maximize => f > gbest_eval,
};
if beats_global {
gbest_decision = pbest_decisions[i].clone();
gbest_eval = f;
}
}
}
}
let _ = gbest_idx;
// Final population is the current particle positions, evaluated.
let final_pop = evaluate_batch(problem, positions);
evaluations += final_pop.len();
let best = best_candidate(&final_pop, &objectives);
let front: Vec<Candidate<Vec<f64>>> = best.iter().cloned().collect();
OptimizationResult::new(
Population::new(final_pop),
front,
best,
evaluations,
self.config.generations,
)
}
}
fn best_index(values: &[f64], direction: Direction) -> usize {
let mut idx = 0;
for i in 1..values.len() {
let better = match direction {
Direction::Minimize => values[i] < values[idx],
Direction::Maximize => values[i] > values[idx],
};
if better {
idx = i;
}
}
idx
}
#[cfg(test)]
mod tests {
use super::*;
use crate::tests_support::{SchafferN1, Sphere1D};
fn make_optimizer(seed: u64) -> ParticleSwarm {
ParticleSwarm::new(
ParticleSwarmConfig {
swarm_size: 30,
generations: 100,
inertia: 0.7,
cognitive: 1.5,
social: 1.5,
seed,
},
RealBounds::new(vec![(-5.0, 5.0)]),
)
}
#[test]
fn finds_minimum_of_sphere() {
let mut opt = make_optimizer(1);
let r = opt.run(&Sphere1D);
let best = r.best.unwrap();
assert!(
best.evaluation.objectives[0] < 1e-3,
"got f = {}",
best.evaluation.objectives[0],
);
}
#[test]
fn deterministic_with_same_seed() {
let mut a = make_optimizer(99);
let mut b = make_optimizer(99);
let ra = a.run(&Sphere1D);
let rb = b.run(&Sphere1D);
assert_eq!(
ra.best.unwrap().evaluation.objectives,
rb.best.unwrap().evaluation.objectives,
);
}
#[test]
#[should_panic(expected = "exactly one objective")]
fn multi_objective_panics() {
let mut opt = make_optimizer(0);
let _ = opt.run(&SchafferN1);
}
}
src/algorithms/pesa2.rs
+352
View File
@@ -0,0 +1,352 @@
//! `PesaII` — Corne, Jerram, Knowles & Oates 2001 Pareto Envelope-based
//! Selection Algorithm II.
use std::collections::BTreeMap;
use rand::Rng as _;
use crate::core::candidate::Candidate;
use crate::core::objective::ObjectiveSpace;
use crate::core::population::Population;
use crate::core::problem::Problem;
use crate::core::result::OptimizationResult;
use crate::core::rng::{Rng, rng_from_seed};
use crate::pareto::archive::ParetoArchive;
use crate::pareto::front::{best_candidate, pareto_front};
use crate::traits::{Initializer, Optimizer, Variation};
/// Configuration for [`PesaII`].
#[derive(Debug, Clone)]
pub struct PesaIIConfig {
/// Internal population size (used for variation).
pub population_size: usize,
/// External non-dominated archive cap.
pub archive_size: usize,
/// Number of generations.
pub generations: usize,
/// Number of grid divisions per objective axis.
pub grid_divisions: usize,
/// Seed for the deterministic RNG.
pub seed: u64,
}
impl Default for PesaIIConfig {
fn default() -> Self {
Self {
population_size: 50,
archive_size: 100,
generations: 250,
grid_divisions: 16,
seed: 42,
}
}
}
/// Pareto Envelope-based Selection Algorithm II.
///
/// Maintains an internal population (used to drive variation) and an
/// external non-dominated archive. Selection biases toward members in
/// sparsely-populated grid boxes so the front spreads out.
#[derive(Debug, Clone)]
pub struct PesaII<I, V> {
/// Algorithm configuration.
pub config: PesaIIConfig,
/// Initial-decision sampler.
pub initializer: I,
/// Offspring-producing variation operator.
pub variation: V,
}
impl<I, V> PesaII<I, V> {
/// Construct a `PesaII`.
pub fn new(config: PesaIIConfig, initializer: I, variation: V) -> Self {
Self {
config,
initializer,
variation,
}
}
}
impl<P, I, V> Optimizer<P> for PesaII<I, V>
where
P: Problem + Sync,
P::Decision: Send,
I: Initializer<P::Decision>,
V: Variation<P::Decision>,
{
fn run(&mut self, problem: &P) -> OptimizationResult<P::Decision> {
assert!(
self.config.population_size > 0,
"PesaII population_size must be > 0"
);
assert!(
self.config.archive_size > 0,
"PesaII archive_size must be > 0"
);
assert!(
self.config.grid_divisions >= 1,
"PesaII grid_divisions must be >= 1"
);
let n = self.config.population_size;
let objectives = problem.objectives();
let mut rng = rng_from_seed(self.config.seed);
// Initial internal population.
let initial_decisions = self.initializer.initialize(n, &mut rng);
let mut internal: Vec<Candidate<P::Decision>> = initial_decisions
.into_iter()
.map(|d| {
let e = problem.evaluate(&d);
Candidate::new(d, e)
})
.collect();
let mut evaluations = internal.len();
// External archive.
let mut archive = ParetoArchive::new(objectives.clone());
for c in &internal {
archive.insert(c.clone());
}
truncate_by_grid(
&mut archive,
self.config.archive_size,
self.config.grid_divisions,
);
for _ in 0..self.config.generations {
// Build grid + box counts on the archive.
let (boxes, counts) = build_grid(&archive, &objectives, self.config.grid_divisions);
// Generate offspring via region-based selection on the archive.
let mut offspring: Vec<Candidate<P::Decision>> = Vec::with_capacity(n);
while offspring.len() < n {
let p1 = region_tournament(&archive, &boxes, &counts, &mut rng);
let p2 = region_tournament(&archive, &boxes, &counts, &mut rng);
let parents = vec![
archive.members()[p1].decision.clone(),
archive.members()[p2].decision.clone(),
];
let children = self.variation.vary(&parents, &mut rng);
assert!(
!children.is_empty(),
"PesaII variation returned no children"
);
for child in children {
if offspring.len() >= n {
break;
}
let eval = problem.evaluate(&child);
evaluations += 1;
offspring.push(Candidate::new(child, eval));
}
}
// Internal pop becomes the offspring; archive gets every
// non-dominated offspring.
for c in &offspring {
archive.insert(c.clone());
}
truncate_by_grid(
&mut archive,
self.config.archive_size,
self.config.grid_divisions,
);
internal = offspring;
}
let _ = internal; // not directly returned
let members = archive.into_vec();
let front = pareto_front(&members, &objectives);
let best = best_candidate(&members, &objectives);
OptimizationResult::new(
Population::new(members),
front,
best,
evaluations,
self.config.generations,
)
}
}
/// Compute per-member box index (M-tuple of grid coordinates) and the
/// population count of each occupied box.
fn build_grid<D: Clone>(
archive: &ParetoArchive<D>,
objectives: &ObjectiveSpace,
divisions: usize,
) -> (Vec<Vec<usize>>, BTreeMap<Vec<usize>, usize>) {
let m = objectives.len();
let members = archive.members();
if members.is_empty() {
return (Vec::new(), BTreeMap::new());
}
let oriented: Vec<Vec<f64>> = members
.iter()
.map(|c| objectives.as_minimization(&c.evaluation.objectives))
.collect();
let mut lo = vec![f64::INFINITY; m];
let mut hi = vec![f64::NEG_INFINITY; m];
for o in &oriented {
for k in 0..m {
if o[k] < lo[k] {
lo[k] = o[k];
}
if o[k] > hi[k] {
hi[k] = o[k];
}
}
}
let mut boxes: Vec<Vec<usize>> = Vec::with_capacity(members.len());
for o in &oriented {
let mut box_idx = Vec::with_capacity(m);
for k in 0..m {
let span = (hi[k] - lo[k]).max(1e-12);
let frac = ((o[k] - lo[k]) / span).clamp(0.0, 1.0 - 1e-9);
box_idx.push((frac * divisions as f64) as usize);
}
boxes.push(box_idx);
}
let mut counts: BTreeMap<Vec<usize>, usize> = BTreeMap::new();
for b in &boxes {
*counts.entry(b.clone()).or_insert(0) += 1;
}
(boxes, counts)
}
/// Pick a member by region-based tournament: take two random members,
/// prefer the one whose grid box is less crowded.
fn region_tournament<D: Clone>(
archive: &ParetoArchive<D>,
boxes: &[Vec<usize>],
counts: &BTreeMap<Vec<usize>, usize>,
rng: &mut Rng,
) -> usize {
let n = archive.members().len();
let a = rng.random_range(0..n);
let b = rng.random_range(0..n);
let ca = counts.get(&boxes[a]).copied().unwrap_or(1);
let cb = counts.get(&boxes[b]).copied().unwrap_or(1);
if ca < cb {
a
} else if cb < ca {
b
} else if rng.random_bool(0.5) {
a
} else {
b
}
}
/// Truncate the archive to `max_size` by repeatedly evicting a uniform-random
/// member of the most-occupied grid box (PESA-II's standard approach).
fn truncate_by_grid<D: Clone>(archive: &mut ParetoArchive<D>, max_size: usize, divisions: usize) {
while archive.members().len() > max_size {
let objectives = archive.objectives.clone();
let (boxes, counts) = build_grid(archive, &objectives, divisions);
// Find the most-crowded box.
let max_count = counts.values().copied().max().unwrap_or(0);
if max_count <= 1 {
// No crowding to break: just truncate.
archive.truncate(max_size);
break;
}
// Indices in that box.
let crowded_box = counts
.iter()
.find(|&(_, &c)| c == max_count)
.map(|(b, _)| b.clone())
.unwrap();
let candidates: Vec<usize> = boxes
.iter()
.enumerate()
.filter(|(_, b)| **b == crowded_box)
.map(|(i, _)| i)
.collect();
// Use a fixed seed-derived RNG would be ideal, but truncation is
// called from the main RNG indirectly; use a deterministic pick
// (the first candidate) to avoid sneaking nondeterminism in.
let evict = *candidates.first().expect("non-empty crowded box");
archive.members.swap_remove(evict);
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::operators::{
CompositeVariation, PolynomialMutation, RealBounds, SimulatedBinaryCrossover,
};
use crate::tests_support::SchafferN1;
fn make_optimizer(
seed: u64,
) -> PesaII<RealBounds, CompositeVariation<SimulatedBinaryCrossover, PolynomialMutation>> {
let bounds = vec![(-5.0, 5.0)];
let initializer = RealBounds::new(bounds.clone());
let variation = CompositeVariation {
crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5),
mutation: PolynomialMutation::new(bounds, 20.0, 1.0),
};
PesaII::new(
PesaIIConfig {
population_size: 20,
archive_size: 30,
generations: 15,
grid_divisions: 8,
seed,
},
initializer,
variation,
)
}
#[test]
fn produces_pareto_front() {
let mut opt = make_optimizer(1);
let r = opt.run(&SchafferN1);
assert!(!r.pareto_front.is_empty());
}
#[test]
fn deterministic_with_same_seed() {
let mut a = make_optimizer(99);
let mut b = make_optimizer(99);
let ra = a.run(&SchafferN1);
let rb = b.run(&SchafferN1);
let oa: Vec<Vec<f64>> = ra
.pareto_front
.iter()
.map(|c| c.evaluation.objectives.clone())
.collect();
let ob: Vec<Vec<f64>> = rb
.pareto_front
.iter()
.map(|c| c.evaluation.objectives.clone())
.collect();
assert_eq!(oa, ob);
}
#[test]
#[should_panic(expected = "archive_size must be > 0")]
fn zero_archive_size_panics() {
let bounds = vec![(0.0, 1.0)];
let initializer = RealBounds::new(bounds.clone());
let variation = CompositeVariation {
crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5),
mutation: PolynomialMutation::new(bounds, 20.0, 1.0),
};
let mut opt = PesaII::new(
PesaIIConfig {
population_size: 4,
archive_size: 0,
generations: 1,
grid_divisions: 4,
seed: 0,
},
initializer,
variation,
);
let _ = opt.run(&SchafferN1);
}
}
src/algorithms/random_search.rs
+27 -6
View File
@@ -25,7 +25,11 @@ pub struct RandomSearchConfig {
impl Default for RandomSearchConfig { impl Default for RandomSearchConfig {
fn default() -> Self { fn default() -> Self {
Self { iterations: 100, batch_size: 1, seed: 42 } Self {
iterations: 100,
batch_size: 1,
seed: 42,
}
} }
} }
@@ -45,7 +49,10 @@ pub struct RandomSearch<I> {
impl<I> RandomSearch<I> { impl<I> RandomSearch<I> {
/// Construct a `RandomSearch` from its config and initializer. /// Construct a `RandomSearch` from its config and initializer.
pub fn new(config: RandomSearchConfig, initializer: I) -> Self { pub fn new(config: RandomSearchConfig, initializer: I) -> Self {
Self { config, initializer } Self {
config,
initializer,
}
} }
} }
@@ -62,7 +69,9 @@ where
let mut evaluations = 0usize; let mut evaluations = 0usize;
for _ in 0..self.config.iterations { for _ in 0..self.config.iterations {
let decisions = self.initializer.initialize(self.config.batch_size, &mut rng); let decisions = self
.initializer
.initialize(self.config.batch_size, &mut rng);
evaluations += decisions.len(); evaluations += decisions.len();
all.extend(evaluate_batch(problem, decisions)); all.extend(evaluate_batch(problem, decisions));
} }
@@ -88,7 +97,11 @@ mod tests {
#[test] #[test]
fn evaluation_count_matches_iterations_times_batch() { fn evaluation_count_matches_iterations_times_batch() {
let mut opt = RandomSearch::new( let mut opt = RandomSearch::new(
RandomSearchConfig { iterations: 30, batch_size: 4, seed: 1 }, RandomSearchConfig {
iterations: 30,
batch_size: 4,
seed: 1,
},
RealBounds::new(vec![(-2.0, 2.0)]), RealBounds::new(vec![(-2.0, 2.0)]),
); );
let r = opt.run(&Sphere1D); let r = opt.run(&Sphere1D);
@@ -100,7 +113,11 @@ mod tests {
#[test] #[test]
fn pareto_front_non_empty_for_multi_objective() { fn pareto_front_non_empty_for_multi_objective() {
let mut opt = RandomSearch::new( let mut opt = RandomSearch::new(
RandomSearchConfig { iterations: 50, batch_size: 1, seed: 42 }, RandomSearchConfig {
iterations: 50,
batch_size: 1,
seed: 42,
},
RealBounds::new(vec![(-5.0, 5.0)]), RealBounds::new(vec![(-5.0, 5.0)]),
); );
let r = opt.run(&SchafferN1); let r = opt.run(&SchafferN1);
@@ -112,7 +129,11 @@ mod tests {
#[test] #[test]
fn single_objective_returns_best() { fn single_objective_returns_best() {
let mut opt = RandomSearch::new( let mut opt = RandomSearch::new(
RandomSearchConfig { iterations: 100, batch_size: 1, seed: 7 }, RandomSearchConfig {
iterations: 100,
batch_size: 1,
seed: 7,
},
RealBounds::new(vec![(-1.0, 1.0)]), RealBounds::new(vec![(-1.0, 1.0)]),
); );
let r = opt.run(&Sphere1D); let r = opt.run(&Sphere1D);
src/algorithms/rvea.rs
+351
View File
@@ -0,0 +1,351 @@
//! `Rvea` — Cheng, Jin, Olhofer & Sendhoff 2016 Reference Vector-guided EA.
use rand::Rng as _;
use crate::algorithms::parallel_eval::evaluate_batch;
use crate::core::candidate::Candidate;
use crate::core::population::Population;
use crate::core::problem::Problem;
use crate::core::result::OptimizationResult;
use crate::core::rng::rng_from_seed;
use crate::pareto::front::{best_candidate, pareto_front};
use crate::pareto::reference_points::das_dennis;
use crate::traits::{Initializer, Optimizer, Variation};
/// Configuration for [`Rvea`].
#[derive(Debug, Clone)]
pub struct RveaConfig {
/// Constant population size.
pub population_size: usize,
/// Number of generations.
pub generations: usize,
/// Number of divisions `H` for Das–Dennis reference vectors. Pop size
/// should be roughly `binomial(H + M − 1, M − 1)`.
pub reference_divisions: usize,
/// Penalty exponent `α`. The paper recommends 2.0.
pub alpha: f64,
/// Seed for the deterministic RNG.
pub seed: u64,
}
impl Default for RveaConfig {
fn default() -> Self {
Self {
population_size: 100,
generations: 250,
reference_divisions: 12,
alpha: 2.0,
seed: 42,
}
}
}
/// Reference Vector-guided Evolutionary Algorithm.
#[derive(Debug, Clone)]
pub struct Rvea<I, V> {
/// Algorithm configuration.
pub config: RveaConfig,
/// Initial-decision sampler.
pub initializer: I,
/// Offspring-producing variation operator.
pub variation: V,
}
impl<I, V> Rvea<I, V> {
/// Construct an `Rvea`.
pub fn new(config: RveaConfig, initializer: I, variation: V) -> Self {
Self {
config,
initializer,
variation,
}
}
}
impl<P, I, V> Optimizer<P> for Rvea<I, V>
where
P: Problem + Sync,
P::Decision: Send,
I: Initializer<P::Decision>,
V: Variation<P::Decision>,
{
fn run(&mut self, problem: &P) -> OptimizationResult<P::Decision> {
assert!(
self.config.population_size > 0,
"Rvea population_size must be > 0"
);
let n = self.config.population_size;
let objectives = problem.objectives();
let m = objectives.len();
// Reference vectors normalized to unit norm.
let raw_refs = das_dennis(m, self.config.reference_divisions);
let references: Vec<Vec<f64>> = raw_refs.into_iter().map(unit_normalize).collect();
assert!(
!references.is_empty(),
"Rvea: no reference vectors generated"
);
// Smallest angle between any two reference vectors — used to scale
// the APD penalty term.
let theta_max = smallest_neighbor_angle(&references);
let mut rng = rng_from_seed(self.config.seed);
let initial_decisions = self.initializer.initialize(n, &mut rng);
let mut population: Vec<Candidate<P::Decision>> =
evaluate_batch(problem, initial_decisions);
let mut evaluations = population.len();
for gen_idx in 0..self.config.generations {
// Phase 1: random parent selection + variation.
let mut offspring_decisions: Vec<P::Decision> = Vec::with_capacity(n);
while offspring_decisions.len() < n {
let p1 = rng.random_range(0..population.len());
let p2 = rng.random_range(0..population.len());
let parents = vec![
population[p1].decision.clone(),
population[p2].decision.clone(),
];
let children = self.variation.vary(&parents, &mut rng);
assert!(!children.is_empty(), "Rvea variation returned no children");
for child in children {
if offspring_decisions.len() >= n {
break;
}
offspring_decisions.push(child);
}
}
let offspring = evaluate_batch(problem, offspring_decisions);
evaluations += offspring.len();
// Combine + APD-based survival.
let mut combined: Vec<Candidate<P::Decision>> = Vec::with_capacity(2 * n);
combined.extend(population);
combined.extend(offspring);
// Ideal point z*.
let m_dim = m;
let mut ideal = vec![f64::INFINITY; m_dim];
for c in &combined {
let oriented = objectives.as_minimization(&c.evaluation.objectives);
for (k, v) in oriented.iter().enumerate() {
if *v < ideal[k] {
ideal[k] = *v;
}
}
}
// Translate.
let translated: Vec<Vec<f64>> = combined
.iter()
.map(|c| {
let oriented = objectives.as_minimization(&c.evaluation.objectives);
oriented
.iter()
.enumerate()
.map(|(k, v)| v - ideal[k])
.collect()
})
.collect();
// Associate each member with its closest-angle reference vector.
let mut assoc: Vec<usize> = vec![0; combined.len()];
let mut angles: Vec<f64> = vec![0.0; combined.len()];
for (i, t) in translated.iter().enumerate() {
let (best_ref, best_angle) = closest_reference(t, &references);
assoc[i] = best_ref;
angles[i] = best_angle;
}
// For each occupied reference vector, keep the member with the
// smallest APD score.
let alpha_t = (gen_idx as f64 / (self.config.generations as f64).max(1.0))
.powf(self.config.alpha);
let mut keep: Vec<Option<(usize, f64)>> = vec![None; references.len()];
for i in 0..combined.len() {
let r = assoc[i];
let length: f64 = translated[i].iter().map(|v| v * v).sum::<f64>().sqrt();
let theta_max_safe = theta_max.max(1e-12);
let penalty = 1.0 + (m_dim as f64) * alpha_t * (angles[i] / theta_max_safe);
let apd = penalty * length;
match keep[r] {
None => keep[r] = Some((i, apd)),
Some((_, current)) if apd < current => keep[r] = Some((i, apd)),
_ => {}
}
}
let mut next: Vec<Candidate<P::Decision>> = keep
.into_iter()
.flatten()
.map(|(i, _)| combined[i].clone())
.collect();
// If we ended up with fewer than n (some references unfilled),
// backfill with the lowest-APD remaining candidates.
if next.len() < n {
let mut all_apds: Vec<(usize, f64)> = (0..combined.len())
.map(|i| {
let length: f64 = translated[i].iter().map(|v| v * v).sum::<f64>().sqrt();
let theta_max_safe = theta_max.max(1e-12);
let penalty = 1.0 + (m_dim as f64) * alpha_t * (angles[i] / theta_max_safe);
(i, penalty * length)
})
.collect();
all_apds.sort_by(|a, b| a.1.partial_cmp(&b.1).unwrap_or(std::cmp::Ordering::Equal));
for (i, _) in all_apds {
if next.len() >= n {
break;
}
if !next
.iter()
.any(|c| std::ptr::eq(c as *const _, &combined[i] as *const _))
{
next.push(combined[i].clone());
}
}
}
// If too many (only possible if the reference set has > n
// vectors), truncate by APD.
if next.len() > n {
next.truncate(n);
}
population = next;
}
let front = pareto_front(&population, &objectives);
let best = best_candidate(&population, &objectives);
OptimizationResult::new(
Population::new(population),
front,
best,
evaluations,
self.config.generations,
)
}
}
fn unit_normalize(mut v: Vec<f64>) -> Vec<f64> {
let n: f64 = v.iter().map(|x| x * x).sum::<f64>().sqrt();
if n > 1e-12 {
for x in v.iter_mut() {
*x /= n;
}
}
v
}
fn closest_reference(point: &[f64], references: &[Vec<f64>]) -> (usize, f64) {
let length: f64 = point.iter().map(|v| v * v).sum::<f64>().sqrt().max(1e-12);
let mut best = 0;
let mut best_angle = f64::INFINITY;
for (i, r) in references.iter().enumerate() {
let dot: f64 = point.iter().zip(r.iter()).map(|(a, b)| a * b).sum();
let cosine = (dot / length).clamp(-1.0, 1.0);
let angle = cosine.acos();
if angle < best_angle {
best_angle = angle;
best = i;
}
}
(best, best_angle)
}
fn smallest_neighbor_angle(references: &[Vec<f64>]) -> f64 {
let mut min_angle = f64::INFINITY;
for i in 0..references.len() {
for j in (i + 1)..references.len() {
let dot: f64 = references[i]
.iter()
.zip(references[j].iter())
.map(|(a, b)| a * b)
.sum();
let angle = dot.clamp(-1.0, 1.0).acos();
if angle < min_angle {
min_angle = angle;
}
}
}
if !min_angle.is_finite() {
std::f64::consts::FRAC_PI_4
} else {
min_angle
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::operators::{
CompositeVariation, PolynomialMutation, RealBounds, SimulatedBinaryCrossover,
};
use crate::tests_support::SchafferN1;
fn make_optimizer(
seed: u64,
) -> Rvea<RealBounds, CompositeVariation<SimulatedBinaryCrossover, PolynomialMutation>> {
let bounds = vec![(-5.0, 5.0)];
let initializer = RealBounds::new(bounds.clone());
let variation = CompositeVariation {
crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5),
mutation: PolynomialMutation::new(bounds, 20.0, 1.0),
};
Rvea::new(
RveaConfig {
population_size: 20,
generations: 15,
reference_divisions: 19,
alpha: 2.0,
seed,
},
initializer,
variation,
)
}
#[test]
fn produces_pareto_front() {
let mut opt = make_optimizer(1);
let r = opt.run(&SchafferN1);
assert!(!r.pareto_front.is_empty());
}
#[test]
fn deterministic_with_same_seed() {
let mut a = make_optimizer(99);
let mut b = make_optimizer(99);
let ra = a.run(&SchafferN1);
let rb = b.run(&SchafferN1);
let oa: Vec<Vec<f64>> = ra
.pareto_front
.iter()
.map(|c| c.evaluation.objectives.clone())
.collect();
let ob: Vec<Vec<f64>> = rb
.pareto_front
.iter()
.map(|c| c.evaluation.objectives.clone())
.collect();
assert_eq!(oa, ob);
}
#[test]
#[should_panic(expected = "population_size must be > 0")]
fn zero_population_size_panics() {
let bounds = vec![(0.0, 1.0)];
let initializer = RealBounds::new(bounds.clone());
let variation = CompositeVariation {
crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5),
mutation: PolynomialMutation::new(bounds, 20.0, 1.0),
};
let mut opt = Rvea::new(
RveaConfig {
population_size: 0,
generations: 1,
reference_divisions: 5,
alpha: 2.0,
seed: 0,
},
initializer,
variation,
);
let _ = opt.run(&SchafferN1);
}
}
src/algorithms/simulated_annealing.rs
+253
View File
@@ -0,0 +1,253 @@
//! `SimulatedAnnealing` — Kirkpatrick et al. 1983 SA for single-objective problems.
use rand::Rng as _;
use crate::core::candidate::Candidate;
use crate::core::objective::Direction;
use crate::core::population::Population;
use crate::core::problem::Problem;
use crate::core::result::OptimizationResult;
use crate::core::rng::rng_from_seed;
use crate::traits::{Initializer, Optimizer, Variation};
/// Configuration for [`SimulatedAnnealing`].
#[derive(Debug, Clone)]
pub struct SimulatedAnnealingConfig {
/// Number of mutation iterations.
pub iterations: usize,
/// Starting temperature `T_0`. Must be positive.
pub initial_temperature: f64,
/// Ending temperature `T_n`. Must be positive and `<= initial_temperature`.
pub final_temperature: f64,
/// Seed for the deterministic RNG.
pub seed: u64,
}
impl Default for SimulatedAnnealingConfig {
fn default() -> Self {
Self {
iterations: 5_000,
initial_temperature: 1.0,
final_temperature: 1e-3,
seed: 42,
}
}
}
/// Single-objective Simulated Annealing.
///
/// Like a hill climber, but worse moves are accepted with probability
/// `exp(-Δ / T)` where `Δ` is the (direction-aware) objective degradation
/// and `T` anneals geometrically from `initial_temperature` to
/// `final_temperature` over the iteration count. Generic over decision
/// type — pair with any `Variation` impl that returns one child per call.
#[derive(Debug, Clone)]
pub struct SimulatedAnnealing<I, V> {
/// Algorithm configuration.
pub config: SimulatedAnnealingConfig,
/// Initial-decision sampler.
pub initializer: I,
/// Mutation operator.
pub variation: V,
}
impl<I, V> SimulatedAnnealing<I, V> {
/// Construct a `SimulatedAnnealing`.
pub fn new(config: SimulatedAnnealingConfig, initializer: I, variation: V) -> Self {
Self {
config,
initializer,
variation,
}
}
}
impl<P, I, V> Optimizer<P> for SimulatedAnnealing<I, V>
where
P: Problem + Sync,
P::Decision: Send,
I: Initializer<P::Decision>,
V: Variation<P::Decision>,
{
fn run(&mut self, problem: &P) -> OptimizationResult<P::Decision> {
let objectives = problem.objectives();
assert!(
objectives.is_single_objective(),
"SimulatedAnnealing requires exactly one objective",
);
assert!(
self.config.initial_temperature > 0.0,
"SimulatedAnnealing initial_temperature must be positive",
);
assert!(
self.config.final_temperature > 0.0,
"SimulatedAnnealing final_temperature must be positive",
);
assert!(
self.config.final_temperature <= self.config.initial_temperature,
"SimulatedAnnealing final_temperature must be <= initial_temperature",
);
let direction = objectives.objectives[0].direction;
let mut rng = rng_from_seed(self.config.seed);
let mut initial = self.initializer.initialize(1, &mut rng);
assert!(
!initial.is_empty(),
"SimulatedAnnealing initializer returned no decisions",
);
let mut current_decision = initial.remove(0);
let mut current_eval = problem.evaluate(&current_decision);
let mut best_decision = current_decision.clone();
let mut best_eval = current_eval.clone();
let mut evaluations = 1usize;
// Geometric cooling: T(k) = T_0 * (T_n / T_0)^(k / (N - 1))
let cooling = if self.config.iterations <= 1 {
1.0
} else {
(self.config.final_temperature / self.config.initial_temperature)
.powf(1.0 / (self.config.iterations as f64 - 1.0))
};
let mut temperature = self.config.initial_temperature;
for _ in 0..self.config.iterations {
let parents = vec![current_decision.clone()];
let children = self.variation.vary(&parents, &mut rng);
assert!(
!children.is_empty(),
"SimulatedAnnealing variation returned no children"
);
let child_decision = children.into_iter().next().unwrap();
let child_eval = problem.evaluate(&child_decision);
evaluations += 1;
let accept = match (child_eval.is_feasible(), current_eval.is_feasible()) {
(true, false) => true,
(false, true) => false,
(false, false) => {
child_eval.constraint_violation <= current_eval.constraint_violation
}
(true, true) => {
let delta = match direction {
Direction::Minimize => {
child_eval.objectives[0] - current_eval.objectives[0]
}
Direction::Maximize => {
current_eval.objectives[0] - child_eval.objectives[0]
}
};
if delta <= 0.0 {
true
} else {
let prob = (-delta / temperature).exp();
rng.random::<f64>() < prob
}
}
};
if accept {
current_decision = child_decision;
current_eval = child_eval;
if better_than(&current_eval, &best_eval, direction) {
best_decision = current_decision.clone();
best_eval = current_eval.clone();
}
}
temperature *= cooling;
}
let best = Candidate::new(best_decision, best_eval);
let population = Population::new(vec![best.clone()]);
let front = vec![best.clone()];
OptimizationResult::new(
population,
front,
Some(best),
evaluations,
self.config.iterations,
)
}
}
fn better_than(
a: &crate::core::evaluation::Evaluation,
b: &crate::core::evaluation::Evaluation,
direction: Direction,
) -> bool {
match (a.is_feasible(), b.is_feasible()) {
(true, false) => true,
(false, true) => false,
(false, false) => a.constraint_violation < b.constraint_violation,
(true, true) => match direction {
Direction::Minimize => a.objectives[0] < b.objectives[0],
Direction::Maximize => a.objectives[0] > b.objectives[0],
},
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::operators::{GaussianMutation, RealBounds};
use crate::tests_support::{SchafferN1, Sphere1D};
fn make_optimizer(seed: u64) -> SimulatedAnnealing<RealBounds, GaussianMutation> {
SimulatedAnnealing::new(
SimulatedAnnealingConfig {
iterations: 2_000,
initial_temperature: 1.0,
final_temperature: 1e-4,
seed,
},
RealBounds::new(vec![(-5.0, 5.0)]),
GaussianMutation { sigma: 0.3 },
)
}
#[test]
fn finds_minimum_of_sphere() {
let mut opt = make_optimizer(1);
let r = opt.run(&Sphere1D);
let best = r.best.unwrap();
assert!(
best.evaluation.objectives[0] < 1e-2,
"got f = {}",
best.evaluation.objectives[0],
);
}
#[test]
fn deterministic_with_same_seed() {
let mut a = make_optimizer(99);
let mut b = make_optimizer(99);
let ra = a.run(&Sphere1D);
let rb = b.run(&Sphere1D);
assert_eq!(
ra.best.unwrap().evaluation.objectives,
rb.best.unwrap().evaluation.objectives,
);
}
#[test]
#[should_panic(expected = "exactly one objective")]
fn multi_objective_panics() {
let mut opt = make_optimizer(0);
let _ = opt.run(&SchafferN1);
}
#[test]
#[should_panic(expected = "initial_temperature must be positive")]
fn zero_initial_temperature_panics() {
let mut opt = SimulatedAnnealing::new(
SimulatedAnnealingConfig {
iterations: 10,
initial_temperature: 0.0,
final_temperature: 1e-3,
seed: 0,
},
RealBounds::new(vec![(-1.0, 1.0)]),
GaussianMutation { sigma: 0.1 },
);
let _ = opt.run(&Sphere1D);
}
}
src/algorithms/sms_emoa.rs
+283
View File
@@ -0,0 +1,283 @@
//! `SmsEmoa` — Beume, Naujoks & Emmerich 2007 S-Metric Selection EMOA.
use rand::Rng as _;
use crate::core::candidate::Candidate;
use crate::core::evaluation::Evaluation;
use crate::core::objective::ObjectiveSpace;
use crate::core::population::Population;
use crate::core::problem::Problem;
use crate::core::result::OptimizationResult;
use crate::core::rng::rng_from_seed;
use crate::metrics::hypervolume::hypervolume_nd_from_evaluations;
use crate::pareto::front::{best_candidate, pareto_front};
use crate::pareto::sort::non_dominated_sort;
use crate::traits::{Initializer, Optimizer, Variation};
/// Configuration for [`SmsEmoa`].
#[derive(Debug, Clone)]
pub struct SmsEmoaConfig {
/// Constant population size carried across generations.
pub population_size: usize,
/// Number of generations. SMS-EMOA is steady-state — each generation
/// produces and evaluates exactly one child.
pub generations: usize,
/// Reference point used for hypervolume contribution computations.
/// Must have one entry per objective; should be worse than every
/// realistic objective value.
pub reference_point: Vec<f64>,
/// Seed for the deterministic RNG.
pub seed: u64,
}
impl Default for SmsEmoaConfig {
fn default() -> Self {
Self {
population_size: 100,
generations: 1_000,
reference_point: vec![11.0, 11.0],
seed: 42,
}
}
}
/// SMS-EMOA: a steady-state MOEA that selects survivors by hypervolume
/// contribution.
///
/// Each generation produces a single offspring via the user's variation
/// operator and replaces the worst-contribution member of the worst
/// non-dominated front. Excellent convergence quality at the price of
/// quadratic-in-N hypervolume evaluations per generation, so practical
/// up to ~4 objectives at population sizes ≤ 200.
#[derive(Debug, Clone)]
pub struct SmsEmoa<I, V> {
/// Algorithm configuration.
pub config: SmsEmoaConfig,
/// Initial-decision sampler.
pub initializer: I,
/// Offspring-producing variation operator.
pub variation: V,
}
impl<I, V> SmsEmoa<I, V> {
/// Construct a `SmsEmoa`.
pub fn new(config: SmsEmoaConfig, initializer: I, variation: V) -> Self {
Self {
config,
initializer,
variation,
}
}
}
impl<P, I, V> Optimizer<P> for SmsEmoa<I, V>
where
P: Problem + Sync,
P::Decision: Send,
I: Initializer<P::Decision>,
V: Variation<P::Decision>,
{
fn run(&mut self, problem: &P) -> OptimizationResult<P::Decision> {
assert!(
self.config.population_size > 0,
"SmsEmoa population_size must be > 0"
);
let n = self.config.population_size;
let objectives = problem.objectives();
assert_eq!(
self.config.reference_point.len(),
objectives.len(),
"SmsEmoa reference_point.len() must equal number of objectives",
);
let reference = self.config.reference_point.clone();
let mut rng = rng_from_seed(self.config.seed);
// Initial population.
let initial_decisions = self.initializer.initialize(n, &mut rng);
let mut population: Vec<Candidate<P::Decision>> = initial_decisions
.into_iter()
.map(|d| {
let e = problem.evaluate(&d);
Candidate::new(d, e)
})
.collect();
let mut evaluations = population.len();
for _ in 0..self.config.generations {
// --- One offspring (steady-state) ---
let p1 = rng.random_range(0..population.len());
let p2 = rng.random_range(0..population.len());
let parents = vec![
population[p1].decision.clone(),
population[p2].decision.clone(),
];
let children = self.variation.vary(&parents, &mut rng);
assert!(
!children.is_empty(),
"SmsEmoa variation returned no children"
);
let child_decision = children.into_iter().next().unwrap();
let child_eval = problem.evaluate(&child_decision);
evaluations += 1;
let child = Candidate::new(child_decision, child_eval);
// --- Combine and decide who to drop ---
population.push(child);
let drop_idx = pick_drop_index(&population, &objectives, &reference);
population.swap_remove(drop_idx);
}
let front = pareto_front(&population, &objectives);
let best = best_candidate(&population, &objectives);
OptimizationResult::new(
Population::new(population),
front,
best,
evaluations,
self.config.generations,
)
}
}
/// Choose the index in `pool` whose removal is preferred per SMS-EMOA's
/// rules: drop from the worst non-dominated front; within that front,
/// drop the member whose removal increases hypervolume the most (= the
/// one with the smallest hypervolume contribution).
fn pick_drop_index<D>(
pool: &[Candidate<D>],
objectives: &ObjectiveSpace,
reference: &[f64],
) -> usize {
let fronts = non_dominated_sort(pool, objectives);
let worst_front = fronts
.last()
.expect("non_dominated_sort must return at least one front for non-empty pool");
if worst_front.len() == 1 {
return worst_front[0];
}
// Compute each candidate's hypervolume contribution = HV(front) -
// HV(front \ {member}). Smallest contribution = drop.
let evals: Vec<&Evaluation> = worst_front.iter().map(|&i| &pool[i].evaluation).collect();
let total_hv = hypervolume_nd_from_evaluations(&evals, objectives, reference);
let mut worst_idx_in_front = 0;
let mut min_contrib = f64::INFINITY;
for k in 0..worst_front.len() {
let mut without: Vec<&Evaluation> = Vec::with_capacity(worst_front.len() - 1);
for (j, &gi) in worst_front.iter().enumerate() {
if j != k {
without.push(&pool[gi].evaluation);
}
}
let hv_without = hypervolume_nd_from_evaluations(&without, objectives, reference);
let contrib = total_hv - hv_without;
if contrib < min_contrib {
min_contrib = contrib;
worst_idx_in_front = k;
}
}
worst_front[worst_idx_in_front]
}
#[cfg(test)]
mod tests {
use super::*;
use crate::operators::{
CompositeVariation, PolynomialMutation, RealBounds, SimulatedBinaryCrossover,
};
use crate::tests_support::SchafferN1;
fn make_optimizer(
seed: u64,
) -> SmsEmoa<RealBounds, CompositeVariation<SimulatedBinaryCrossover, PolynomialMutation>> {
let bounds = vec![(-5.0, 5.0)];
let initializer = RealBounds::new(bounds.clone());
let variation = CompositeVariation {
crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5),
mutation: PolynomialMutation::new(bounds, 20.0, 1.0),
};
SmsEmoa::new(
SmsEmoaConfig {
population_size: 20,
generations: 100,
reference_point: vec![30.0, 30.0],
seed,
},
initializer,
variation,
)
}
#[test]
fn produces_pareto_front() {
let mut opt = make_optimizer(1);
let r = opt.run(&SchafferN1);
assert_eq!(r.population.len(), 20);
assert!(!r.pareto_front.is_empty());
}
#[test]
fn deterministic_with_same_seed() {
let mut a = make_optimizer(99);
let mut b = make_optimizer(99);
let ra = a.run(&SchafferN1);
let rb = b.run(&SchafferN1);
let oa: Vec<Vec<f64>> = ra
.pareto_front
.iter()
.map(|c| c.evaluation.objectives.clone())
.collect();
let ob: Vec<Vec<f64>> = rb
.pareto_front
.iter()
.map(|c| c.evaluation.objectives.clone())
.collect();
assert_eq!(oa, ob);
}
#[test]
#[should_panic(expected = "population_size must be > 0")]
fn zero_population_size_panics() {
let bounds = vec![(0.0, 1.0)];
let initializer = RealBounds::new(bounds.clone());
let variation = CompositeVariation {
crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5),
mutation: PolynomialMutation::new(bounds, 20.0, 1.0),
};
let mut opt = SmsEmoa::new(
SmsEmoaConfig {
population_size: 0,
generations: 1,
reference_point: vec![1.0, 1.0],
seed: 0,
},
initializer,
variation,
);
let _ = opt.run(&SchafferN1);
}
#[test]
#[should_panic(expected = "reference_point.len() must equal number of objectives")]
fn dim_mismatch_panics() {
let bounds = vec![(0.0, 1.0)];
let initializer = RealBounds::new(bounds.clone());
let variation = CompositeVariation {
crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5),
mutation: PolynomialMutation::new(bounds, 20.0, 1.0),
};
let mut opt = SmsEmoa::new(
SmsEmoaConfig {
population_size: 4,
generations: 1,
reference_point: vec![1.0, 1.0, 1.0],
seed: 0,
},
initializer,
variation,
);
let _ = opt.run(&SchafferN1);
}
}
src/algorithms/snes.rs
+286
View File
@@ -0,0 +1,286 @@
//! `SeparableNes` — Wierstra et al. 2008/2014 Natural Evolution Strategy
//! with diagonal covariance (sNES).
use rand_distr::{Distribution, Normal};
use crate::core::candidate::Candidate;
use crate::core::evaluation::Evaluation;
use crate::core::objective::Direction;
use crate::core::population::Population;
use crate::core::problem::Problem;
use crate::core::result::OptimizationResult;
use crate::core::rng::rng_from_seed;
use crate::operators::real::RealBounds;
use crate::traits::Optimizer;
/// Configuration for [`SeparableNes`].
#[derive(Debug, Clone)]
pub struct SeparableNesConfig {
/// Population size `λ` per generation. NES recommends `4 + ⌊3·ln(n)⌋`.
pub population_size: usize,
/// Number of generations.
pub generations: usize,
/// Initial step size `σ_0`.
pub initial_sigma: f64,
/// Mean learning rate `η_μ`. NES default is 1.0.
pub mean_learning_rate: f64,
/// Sigma learning rate `η_σ`. NES default is `(3 + ln(n)) / (5·sqrt(n))`,
/// computed at runtime if you set this to `None`.
pub sigma_learning_rate: Option<f64>,
/// Seed for the deterministic RNG.
pub seed: u64,
}
impl Default for SeparableNesConfig {
fn default() -> Self {
Self {
population_size: 16,
generations: 200,
initial_sigma: 0.5,
mean_learning_rate: 1.0,
sigma_learning_rate: None,
seed: 42,
}
}
}
/// Separable Natural Evolution Strategy (sNES).
///
/// `Vec<f64>` decisions only. Single-objective only. Maintains a sampling
/// distribution `N(μ, diag(σ²))` and updates `μ`, `σ` each generation by
/// following the natural gradient of expected fitness, with rank-shaped
/// fitness utilities for invariance to monotone transforms of the
/// objective.
#[derive(Debug, Clone)]
pub struct SeparableNes {
/// Algorithm configuration.
pub config: SeparableNesConfig,
/// Per-variable bounds — used to seed `μ` (midpoint) and clamp every
/// sampled offspring.
pub bounds: RealBounds,
}
impl SeparableNes {
/// Construct a `SeparableNes`.
pub fn new(config: SeparableNesConfig, bounds: RealBounds) -> Self {
Self { config, bounds }
}
}
impl<P> Optimizer<P> for SeparableNes
where
P: Problem<Decision = Vec<f64>> + Sync,
{
fn run(&mut self, problem: &P) -> OptimizationResult<P::Decision> {
assert!(
self.config.population_size >= 2,
"SeparableNes population_size must be >= 2",
);
assert!(
self.config.initial_sigma > 0.0,
"SeparableNes initial_sigma must be > 0"
);
let objectives = problem.objectives();
assert!(
objectives.is_single_objective(),
"SeparableNes requires exactly one objective",
);
let direction = objectives.objectives[0].direction;
let n = self.bounds.bounds.len();
let lambda = self.config.population_size;
let mut rng = rng_from_seed(self.config.seed);
// Initial state.
let mut mean: Vec<f64> = self
.bounds
.bounds
.iter()
.map(|&(lo, hi)| 0.5 * (lo + hi))
.collect();
let mut sigma = vec![self.config.initial_sigma; n];
// Default sigma learning rate (Wierstra et al. 2014, Eq. 11).
let eta_sigma = self
.config
.sigma_learning_rate
.unwrap_or_else(|| (3.0 + (n as f64).ln()) / (5.0 * (n as f64).sqrt()));
let eta_mean = self.config.mean_learning_rate;
// Rank utilities — the standard NES weighting:
// u_i = max(0, ln(λ/2 + 1) - ln(i)) / Σ - 1/λ
// (positive total mass, zero sum after the shift).
let utilities = nes_utilities(lambda);
let mut best_seen: Option<Candidate<Vec<f64>>> = None;
let mut total_evaluations = 0usize;
for _ in 0..self.config.generations {
// Sample λ offspring.
let mut z_samples: Vec<Vec<f64>> = Vec::with_capacity(lambda);
let mut x_samples: Vec<Vec<f64>> = Vec::with_capacity(lambda);
let mut evals: Vec<Evaluation> = Vec::with_capacity(lambda);
for _ in 0..lambda {
let z: Vec<f64> = (0..n)
.map(|_| Normal::new(0.0, 1.0).unwrap().sample(&mut rng))
.collect();
let x: Vec<f64> = (0..n)
.map(|j| {
let v = mean[j] + sigma[j] * z[j];
let (lo, hi) = self.bounds.bounds[j];
v.clamp(lo, hi)
})
.collect();
let e = problem.evaluate(&x);
total_evaluations += 1;
let beats_best = match &best_seen {
None => true,
Some(b) => better(&e, &b.evaluation, direction),
};
if beats_best {
best_seen = Some(Candidate::new(x.clone(), e.clone()));
}
z_samples.push(z);
x_samples.push(x);
evals.push(e);
}
// Sort offspring best → worst (so utility[0] goes to the best).
let mut order: Vec<usize> = (0..lambda).collect();
order.sort_by(|&a, &b| compare(&evals[a], &evals[b], direction));
// Update mean: μ ← μ + η_μ · σ · Σ u_i · z_i
let mut grad_mean = vec![0.0_f64; n];
for k in 0..lambda {
let u = utilities[k];
let z = &z_samples[order[k]];
for j in 0..n {
grad_mean[j] += u * z[j];
}
}
for j in 0..n {
mean[j] += eta_mean * sigma[j] * grad_mean[j];
let (lo, hi) = self.bounds.bounds[j];
mean[j] = mean[j].clamp(lo, hi);
}
// Update sigma: σ_j ← σ_j · exp((η_σ/2) · Σ u_i · (z_i,j² - 1))
for j in 0..n {
let mut grad_sigma_j = 0.0;
for k in 0..lambda {
let u = utilities[k];
let z = &z_samples[order[k]];
grad_sigma_j += u * (z[j] * z[j] - 1.0);
}
sigma[j] *= (0.5 * eta_sigma * grad_sigma_j).exp();
if !sigma[j].is_finite() || sigma[j] < 1e-30 {
sigma[j] = 1e-30;
}
}
}
let best = best_seen.expect("at least one generation evaluated");
let population = Population::new(vec![best.clone()]);
let front = vec![best.clone()];
OptimizationResult::new(
population,
front,
Some(best),
total_evaluations,
self.config.generations,
)
}
}
fn nes_utilities(lambda: usize) -> Vec<f64> {
let half = lambda as f64 / 2.0 + 1.0;
let raw: Vec<f64> = (0..lambda)
.map(|i| {
let v = half.ln() - ((i + 1) as f64).ln();
v.max(0.0)
})
.collect();
let sum: f64 = raw.iter().sum::<f64>().max(1e-12);
let inv_lambda = 1.0 / lambda as f64;
raw.iter().map(|u| u / sum - inv_lambda).collect()
}
fn compare(a: &Evaluation, b: &Evaluation, direction: Direction) -> std::cmp::Ordering {
match (a.is_feasible(), b.is_feasible()) {
(true, false) => std::cmp::Ordering::Less,
(false, true) => std::cmp::Ordering::Greater,
(false, false) => a
.constraint_violation
.partial_cmp(&b.constraint_violation)
.unwrap_or(std::cmp::Ordering::Equal),
(true, true) => match direction {
Direction::Minimize => a.objectives[0]
.partial_cmp(&b.objectives[0])
.unwrap_or(std::cmp::Ordering::Equal),
Direction::Maximize => b.objectives[0]
.partial_cmp(&a.objectives[0])
.unwrap_or(std::cmp::Ordering::Equal),
},
}
}
fn better(a: &Evaluation, b: &Evaluation, direction: Direction) -> bool {
compare(a, b, direction) == std::cmp::Ordering::Less
}
#[cfg(test)]
mod tests {
use super::*;
use crate::tests_support::{SchafferN1, Sphere1D};
fn make_optimizer(seed: u64) -> SeparableNes {
SeparableNes::new(
SeparableNesConfig {
population_size: 16,
generations: 200,
initial_sigma: 0.5,
mean_learning_rate: 1.0,
sigma_learning_rate: None,
seed,
},
RealBounds::new(vec![(-5.0, 5.0)]),
)
}
#[test]
fn finds_minimum_of_sphere() {
let mut opt = make_optimizer(1);
let r = opt.run(&Sphere1D);
let best = r.best.unwrap();
assert!(
best.evaluation.objectives[0] < 1e-6,
"got f = {}",
best.evaluation.objectives[0],
);
}
#[test]
fn deterministic_with_same_seed() {
let mut a = make_optimizer(99);
let mut b = make_optimizer(99);
let ra = a.run(&Sphere1D);
let rb = b.run(&Sphere1D);
assert_eq!(
ra.best.unwrap().evaluation.objectives,
rb.best.unwrap().evaluation.objectives,
);
}
#[test]
fn utilities_sum_to_zero() {
let u = nes_utilities(8);
let s: f64 = u.iter().sum();
assert!(s.abs() < 1e-12, "utilities sum to {s}, not 0");
}
#[test]
#[should_panic(expected = "exactly one objective")]
fn multi_objective_panics() {
let mut opt = make_optimizer(0);
let _ = opt.run(&SchafferN1);
}
}
src/algorithms/spea2.rs
+142 -48
View File
@@ -9,7 +9,6 @@ use crate::core::population::Population;
use crate::core::problem::Problem; use crate::core::problem::Problem;
use crate::core::result::OptimizationResult; use crate::core::result::OptimizationResult;
use crate::core::rng::{Rng, rng_from_seed}; use crate::core::rng::{Rng, rng_from_seed};
use crate::pareto::dominance::{Dominance, pareto_compare};
use crate::pareto::front::{best_candidate, pareto_front}; use crate::pareto::front::{best_candidate, pareto_front};
use crate::traits::{Initializer, Optimizer, Variation}; use crate::traits::{Initializer, Optimizer, Variation};
@@ -28,7 +27,12 @@ pub struct Spea2Config {
impl Default for Spea2Config { impl Default for Spea2Config {
fn default() -> Self { fn default() -> Self {
Self { population_size: 100, archive_size: 100, generations: 250, seed: 42 } Self {
population_size: 100,
archive_size: 100,
generations: 250,
seed: 42,
}
} }
} }
@@ -46,7 +50,11 @@ pub struct Spea2<I, V> {
impl<I, V> Spea2<I, V> { impl<I, V> Spea2<I, V> {
/// Construct a `Spea2` optimizer. /// Construct a `Spea2` optimizer.
pub fn new(config: Spea2Config, initializer: I, variation: V) -> Self { pub fn new(config: Spea2Config, initializer: I, variation: V) -> Self {
Self { config, initializer, variation } Self {
config,
initializer,
variation,
}
} }
} }
@@ -142,19 +150,50 @@ fn compute_fitness<D>(pool: &[Candidate<D>], objectives: &ObjectiveSpace) -> Vec
.iter() .iter()
.map(|c| objectives.as_minimization(&c.evaluation.objectives)) .map(|c| objectives.as_minimization(&c.evaluation.objectives))
.collect(); .collect();
let feasible: Vec<bool> = pool.iter().map(|c| c.evaluation.is_feasible()).collect();
let violation: Vec<f64> = pool
.iter()
.map(|c| c.evaluation.constraint_violation)
.collect();
let m = objectives.len();
// Strength S(i) = number of members i dominates. // Strength S(i) = number of members i dominates. Inline `pareto_compare`
// against the cached oriented/feasibility arrays — the by-pair call into
// `pareto_compare` would otherwise allocate two fresh `Vec<f64>`s per
// pair via `as_minimization`, dominating per-generation cost on
// population sizes ≥ 80.
let mut strength = vec![0_usize; n]; let mut strength = vec![0_usize; n];
let mut dominators_of: Vec<Vec<usize>> = vec![Vec::new(); n]; let mut dominators_of: Vec<Vec<usize>> = vec![Vec::new(); n];
for i in 0..n { for i in 0..n {
let ai_feasible = feasible[i];
let ai_violation = violation[i];
let ai = &oriented[i];
for j in 0..n { for j in 0..n {
if i == j { if i == j {
continue; continue;
} }
if matches!( let bi_feasible = feasible[j];
pareto_compare(&pool[i].evaluation, &pool[j].evaluation, objectives), let i_dominates_j = match (ai_feasible, bi_feasible) {
Dominance::Dominates (true, false) => true,
) { (false, true) => false,
(false, false) => ai_violation < violation[j],
(true, true) => {
let bj = &oriented[j];
let mut a_better_anywhere = false;
let mut b_better_anywhere = false;
for k in 0..m {
let av = ai[k];
let bv = bj[k];
if av < bv {
a_better_anywhere = true;
} else if av > bv {
b_better_anywhere = true;
}
}
a_better_anywhere && !b_better_anywhere
}
};
if i_dominates_j {
strength[i] += 1; strength[i] += 1;
dominators_of[j].push(i); dominators_of[j].push(i);
} }
@@ -166,17 +205,25 @@ fn compute_fitness<D>(pool: &[Candidate<D>], objectives: &ObjectiveSpace) -> Vec
.map(|i| dominators_of[i].iter().map(|&j| strength[j] as f64).sum()) .map(|i| dominators_of[i].iter().map(|&j| strength[j] as f64).sum())
.collect(); .collect();
// Density D(i) = 1 / (σ_k + 2). Use kth_nearest distances. // Density D(i) = 1 / (σ_k + 2) where σ_k is the distance to the k-th
// nearest neighbor (k = floor(sqrt(N))). Build a symmetric distance
// matrix once instead of recomputing each row independently — that
// halves the euclidean calls (which dominate at higher M) and keeps
// the σ_k value bit-identical.
let mut dist: Vec<Vec<f64>> = vec![vec![0.0_f64; n]; n];
#[allow(clippy::needless_range_loop)]
for i in 0..n {
for j in (i + 1)..n {
let d = euclidean(&oriented[i], &oriented[j]);
dist[i][j] = d;
dist[j][i] = d;
}
}
let k = (n as f64).sqrt() as usize; let k = (n as f64).sqrt() as usize;
let density: Vec<f64> = (0..n) let density: Vec<f64> = (0..n)
.map(|i| { .map(|i| {
let mut dists: Vec<f64> = (0..n) let mut dists: Vec<f64> = (0..n).filter(|&j| j != i).map(|j| dist[i][j]).collect();
.filter(|&j| j != i)
.map(|j| euclidean(&oriented[i], &oriented[j]))
.collect();
dists.sort_by(|a, b| a.partial_cmp(b).unwrap_or(std::cmp::Ordering::Equal)); dists.sort_by(|a, b| a.partial_cmp(b).unwrap_or(std::cmp::Ordering::Equal));
// SPEA2's σ_k is the distance to the k-th nearest neighbor (1-indexed).
// With k = floor(sqrt(N)), use index (k-1).clamp(0, len-1).
let idx = if dists.is_empty() { let idx = if dists.is_empty() {
return 0.0; return 0.0;
} else { } else {
@@ -190,7 +237,11 @@ fn compute_fitness<D>(pool: &[Candidate<D>], objectives: &ObjectiveSpace) -> Vec
} }
fn euclidean(a: &[f64], b: &[f64]) -> f64 { fn euclidean(a: &[f64], b: &[f64]) -> f64 {
a.iter().zip(b.iter()).map(|(x, y)| (x - y).powi(2)).sum::<f64>().sqrt() a.iter()
.zip(b.iter())
.map(|(x, y)| (x - y).powi(2))
.sum::<f64>()
.sqrt()
} }
/// Build the next archive of exactly `target_size` members. /// Build the next archive of exactly `target_size` members.
@@ -213,10 +264,11 @@ fn build_archive<D: Clone>(
if nondom.len() < target_size { if nondom.len() < target_size {
// Fill from dominated members ordered by ascending fitness. // Fill from dominated members ordered by ascending fitness.
let mut dominated: Vec<usize> = let mut dominated: Vec<usize> = (0..pool.len()).filter(|&i| fitness[i] >= 1.0).collect();
(0..pool.len()).filter(|&i| fitness[i] >= 1.0).collect();
dominated.sort_by(|&a, &b| { dominated.sort_by(|&a, &b| {
fitness[a].partial_cmp(&fitness[b]).unwrap_or(std::cmp::Ordering::Equal) fitness[a]
.partial_cmp(&fitness[b])
.unwrap_or(std::cmp::Ordering::Equal)
}); });
let needed = target_size - nondom.len(); let needed = target_size - nondom.len();
nondom.extend(dominated.into_iter().take(needed)); nondom.extend(dominated.into_iter().take(needed));
@@ -224,32 +276,44 @@ fn build_archive<D: Clone>(
} }
// Truncation: while too large, drop the member with the smallest distance // Truncation: while too large, drop the member with the smallest distance
// to its nearest neighbor in the current archive. // to its nearest neighbor in the current archive (ties broken by next-
// nearest, etc. via lex order on each member's sorted neighbor vector).
//
// Implementation: compute the pairwise distance matrix once, plus each
// member's sorted neighbor-distance vector. Each iteration drops one
// dead victim's entry from every survivor's sorted vector via
// binary-search-remove, instead of resorting from scratch. That cuts
// truncation cost from O(K³ log K) to O(K² log K) overall while
// producing the identical victim choice every step (the sorted vector
// post-removal is bit-equal to a fresh sort over the smaller set).
let n = nondom.len();
let oriented: Vec<Vec<f64>> = nondom let oriented: Vec<Vec<f64>> = nondom
.iter() .iter()
.map(|&i| objectives.as_minimization(&pool[i].evaluation.objectives)) .map(|&i| objectives.as_minimization(&pool[i].evaluation.objectives))
.collect(); .collect();
let mut alive: Vec<bool> = vec![true; nondom.len()]; let mut dist: Vec<Vec<f64>> = vec![vec![0.0_f64; n]; n];
let mut alive_count = nondom.len(); #[allow(clippy::needless_range_loop)]
while alive_count > target_size { for i in 0..n {
// Compute per-member sorted distances to other alive members. for j in (i + 1)..n {
let mut neighbor_dists: Vec<Vec<f64>> = vec![Vec::new(); nondom.len()]; let d = euclidean(&oriented[i], &oriented[j]);
for i in 0..nondom.len() { dist[i][j] = d;
if !alive[i] { dist[j][i] = d;
continue;
}
for j in 0..nondom.len() {
if !alive[j] || i == j {
continue;
}
neighbor_dists[i].push(euclidean(&oriented[i], &oriented[j]));
}
neighbor_dists[i]
.sort_by(|a, b| a.partial_cmp(b).unwrap_or(std::cmp::Ordering::Equal));
} }
// Find the alive member whose neighbor-distance vector is lex-smallest. }
let mut sorted_dists: Vec<Vec<f64>> = (0..n)
.map(|i| {
let mut v: Vec<f64> = (0..n).filter(|&j| j != i).map(|j| dist[i][j]).collect();
v.sort_by(|a, b| a.partial_cmp(b).unwrap_or(std::cmp::Ordering::Equal));
v
})
.collect();
let mut alive: Vec<bool> = vec![true; n];
let mut alive_count = n;
while alive_count > target_size {
// Find the alive member whose sorted-neighbor-distance vector is
// lex-smallest (= the most crowded member).
let mut victim = usize::MAX; let mut victim = usize::MAX;
for i in 0..nondom.len() { for i in 0..n {
if !alive[i] { if !alive[i] {
continue; continue;
} }
@@ -257,13 +321,16 @@ fn build_archive<D: Clone>(
victim = i; victim = i;
continue; continue;
} }
// Lex-compare neighbor distances. let cmp = sorted_dists[i]
let cmp = neighbor_dists[i]
.iter() .iter()
.zip(neighbor_dists[victim].iter()) .zip(sorted_dists[victim].iter())
.find_map(|(a, b)| { .find_map(|(a, b)| {
let c = a.partial_cmp(b).unwrap_or(std::cmp::Ordering::Equal); let c = a.partial_cmp(b).unwrap_or(std::cmp::Ordering::Equal);
if c != std::cmp::Ordering::Equal { Some(c) } else { None } if c != std::cmp::Ordering::Equal {
Some(c)
} else {
None
}
}) })
.unwrap_or(std::cmp::Ordering::Equal); .unwrap_or(std::cmp::Ordering::Equal);
if cmp == std::cmp::Ordering::Less { if cmp == std::cmp::Ordering::Less {
@@ -272,12 +339,33 @@ fn build_archive<D: Clone>(
} }
alive[victim] = false; alive[victim] = false;
alive_count -= 1; alive_count -= 1;
// Update every still-alive member's sorted neighbor vector by
// removing the entry corresponding to the dead victim. Binary-
// search-remove on the (still-)sorted vector is O(log K + K) per
// survivor — we tolerate the linear shift because K is tiny.
for i in 0..n {
if !alive[i] {
continue;
}
let d = dist[i][victim];
if let Ok(pos) = sorted_dists[i]
.binary_search_by(|x| x.partial_cmp(&d).unwrap_or(std::cmp::Ordering::Equal))
{
sorted_dists[i].remove(pos);
}
}
} }
nondom nondom
.into_iter() .into_iter()
.enumerate() .enumerate()
.filter_map(|(local, idx)| if alive[local] { Some(pool[idx].clone()) } else { None }) .filter_map(|(local, idx)| {
if alive[local] {
Some(pool[idx].clone())
} else {
None
}
})
.collect() .collect()
} }
@@ -354,10 +442,16 @@ mod tests {
let mut b = make(); let mut b = make();
let ra = a.run(&SchafferN1); let ra = a.run(&SchafferN1);
let rb = b.run(&SchafferN1); let rb = b.run(&SchafferN1);
let oa: Vec<Vec<f64>> = let oa: Vec<Vec<f64>> = ra
ra.pareto_front.iter().map(|c| c.evaluation.objectives.clone()).collect(); .pareto_front
let ob: Vec<Vec<f64>> = .iter()
rb.pareto_front.iter().map(|c| c.evaluation.objectives.clone()).collect(); .map(|c| c.evaluation.objectives.clone())
.collect();
let ob: Vec<Vec<f64>> = rb
.pareto_front
.iter()
.map(|c| c.evaluation.objectives.clone())
.collect();
assert_eq!(oa, ob); assert_eq!(oa, ob);
} }
src/algorithms/tabu_search.rs
+288
View File
@@ -0,0 +1,288 @@
//! `TabuSearch` — Glover 1986 tabu search with a user-supplied neighbor
//! generator and decision-level FIFO tabu list.
use std::collections::{HashSet, VecDeque};
use std::hash::Hash;
use crate::core::candidate::Candidate;
use crate::core::objective::Direction;
use crate::core::population::Population;
use crate::core::problem::Problem;
use crate::core::result::OptimizationResult;
use crate::core::rng::{Rng, rng_from_seed};
use crate::traits::{Initializer, Optimizer};
/// Configuration for [`TabuSearch`].
#[derive(Debug, Clone)]
pub struct TabuSearchConfig {
/// Number of iterations.
pub iterations: usize,
/// Maximum size of the FIFO tabu list (older entries are evicted).
pub tabu_tenure: usize,
/// Seed for the deterministic RNG used by the neighbor generator.
pub seed: u64,
}
impl Default for TabuSearchConfig {
fn default() -> Self {
Self {
iterations: 500,
tabu_tenure: 16,
seed: 42,
}
}
}
/// Single-objective tabu search.
///
/// Each iteration the user-supplied `neighbors` closure produces a finite
/// list of candidate moves from the current incumbent. The best non-tabu
/// neighbor (or any tabu neighbor that improves the best-seen-ever
/// incumbent — the standard "aspiration" override) is accepted as the new
/// incumbent and its decision is appended to a FIFO tabu list of size
/// `tabu_tenure`. Tabu matches the full decision; users wanting move-based
/// tabu can wrap moves into a custom decision type.
pub struct TabuSearch<D, I, N>
where
D: Clone + Hash + Eq,
I: Initializer<D>,
N: FnMut(&D, &mut Rng) -> Vec<D>,
{
/// Algorithm configuration.
pub config: TabuSearchConfig,
/// Initial-decision sampler.
pub initializer: I,
/// Neighbor generator: produces a finite list of candidate moves from
/// the current incumbent.
pub neighbors: N,
_marker: std::marker::PhantomData<D>,
}
impl<D, I, N> TabuSearch<D, I, N>
where
D: Clone + Hash + Eq,
I: Initializer<D>,
N: FnMut(&D, &mut Rng) -> Vec<D>,
{
/// Construct a `TabuSearch`.
pub fn new(config: TabuSearchConfig, initializer: I, neighbors: N) -> Self {
Self {
config,
initializer,
neighbors,
_marker: std::marker::PhantomData,
}
}
}
impl<P, I, N> Optimizer<P> for TabuSearch<P::Decision, I, N>
where
P: Problem + Sync,
P::Decision: Clone + Hash + Eq + Send,
I: Initializer<P::Decision>,
N: FnMut(&P::Decision, &mut Rng) -> Vec<P::Decision>,
{
fn run(&mut self, problem: &P) -> OptimizationResult<P::Decision> {
let objectives = problem.objectives();
assert!(
objectives.is_single_objective(),
"TabuSearch requires exactly one objective",
);
assert!(
self.config.tabu_tenure >= 1,
"TabuSearch tabu_tenure must be >= 1",
);
let direction = objectives.objectives[0].direction;
let mut rng = rng_from_seed(self.config.seed);
let mut initial = self.initializer.initialize(1, &mut rng);
assert!(
!initial.is_empty(),
"TabuSearch initializer returned no decisions"
);
let mut current_decision = initial.remove(0);
let mut current_eval = problem.evaluate(&current_decision);
let mut best_decision = current_decision.clone();
let mut best_eval = current_eval.clone();
let mut evaluations = 1usize;
let mut tabu_queue: VecDeque<P::Decision> =
VecDeque::with_capacity(self.config.tabu_tenure);
let mut tabu_set: HashSet<P::Decision> = HashSet::new();
for _ in 0..self.config.iterations {
let candidates = (self.neighbors)(&current_decision, &mut rng);
if candidates.is_empty() {
break;
}
// Best non-tabu candidate, OR best tabu candidate that beats the
// best-seen-ever (aspiration).
let mut best_idx: Option<usize> = None;
let mut best_cand_eval: Option<crate::core::evaluation::Evaluation> = None;
let evaluations_before = evaluations;
let mut cand_evals: Vec<crate::core::evaluation::Evaluation> =
Vec::with_capacity(candidates.len());
for c in &candidates {
cand_evals.push(problem.evaluate(c));
}
evaluations += candidates.len();
let _ = evaluations_before;
for (i, c) in candidates.iter().enumerate() {
let is_tabu = tabu_set.contains(c);
let aspires = is_tabu && better_than(&cand_evals[i], &best_eval, direction);
if is_tabu && !aspires {
continue;
}
let eligible = match &best_cand_eval {
None => true,
Some(b) => better_than(&cand_evals[i], b, direction),
};
if eligible {
best_idx = Some(i);
best_cand_eval = Some(cand_evals[i].clone());
}
}
// If everything is tabu and nothing aspires, fall back to the
// best tabu candidate (avoid getting stuck).
if best_idx.is_none() {
for (i, _) in candidates.iter().enumerate() {
let eligible = match &best_cand_eval {
None => true,
Some(b) => better_than(&cand_evals[i], b, direction),
};
if eligible {
best_idx = Some(i);
best_cand_eval = Some(cand_evals[i].clone());
}
}
}
let chosen_idx = best_idx.expect("non-empty candidate list");
let chosen_decision = candidates[chosen_idx].clone();
current_eval = cand_evals.remove(chosen_idx);
current_decision = chosen_decision.clone();
if better_than(&current_eval, &best_eval, direction) {
best_decision = current_decision.clone();
best_eval = current_eval.clone();
}
// Update FIFO tabu list.
tabu_queue.push_back(chosen_decision.clone());
tabu_set.insert(chosen_decision);
if tabu_queue.len() > self.config.tabu_tenure {
if let Some(old) = tabu_queue.pop_front() {
tabu_set.remove(&old);
}
}
}
let best = Candidate::new(best_decision, best_eval);
let population = Population::new(vec![best.clone()]);
let front = vec![best.clone()];
OptimizationResult::new(
population,
front,
Some(best),
evaluations,
self.config.iterations,
)
}
}
fn better_than(
a: &crate::core::evaluation::Evaluation,
b: &crate::core::evaluation::Evaluation,
direction: Direction,
) -> bool {
match (a.is_feasible(), b.is_feasible()) {
(true, false) => true,
(false, true) => false,
(false, false) => a.constraint_violation < b.constraint_violation,
(true, true) => match direction {
Direction::Minimize => a.objectives[0] < b.objectives[0],
Direction::Maximize => a.objectives[0] > b.objectives[0],
},
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::core::evaluation::Evaluation;
use crate::core::objective::{Objective, ObjectiveSpace};
use rand::Rng as _;
/// Trivial integer-grid problem: minimize `(x - 7)^2`.
struct GridProblem;
impl Problem for GridProblem {
type Decision = Vec<i32>;
fn objectives(&self) -> ObjectiveSpace {
ObjectiveSpace::new(vec![Objective::minimize("f")])
}
fn evaluate(&self, x: &Vec<i32>) -> Evaluation {
let v = (x[0] - 7) as f64;
Evaluation::new(vec![v * v])
}
}
/// Initialize a single 1-D integer at 0.
struct StartAtZero;
impl Initializer<Vec<i32>> for StartAtZero {
fn initialize(&mut self, size: usize, _rng: &mut Rng) -> Vec<Vec<i32>> {
(0..size).map(|_| vec![0]).collect()
}
}
fn make_optimizer<F>(seed: u64, neighbors: F) -> TabuSearch<Vec<i32>, StartAtZero, F>
where
F: FnMut(&Vec<i32>, &mut Rng) -> Vec<Vec<i32>>,
{
TabuSearch::new(
TabuSearchConfig {
iterations: 50,
tabu_tenure: 4,
seed,
},
StartAtZero,
neighbors,
)
}
#[test]
fn finds_optimum_on_grid() {
// Neighbors: ±1 of current value.
let neighbors = |x: &Vec<i32>, _rng: &mut Rng| vec![vec![x[0] - 1], vec![x[0] + 1]];
let mut opt = make_optimizer(1, neighbors);
let r = opt.run(&GridProblem);
let best = r.best.unwrap();
assert_eq!(best.decision, vec![7]);
assert_eq!(best.evaluation.objectives, vec![0.0]);
}
#[test]
fn deterministic_with_same_seed() {
let neighbors = |x: &Vec<i32>, rng: &mut Rng| {
(0..5)
.map(|_| vec![x[0] + rng.random_range(-3..=3)])
.collect::<Vec<_>>()
};
let mut a = make_optimizer(99, neighbors);
let mut b = make_optimizer(99, |x: &Vec<i32>, rng: &mut Rng| {
(0..5)
.map(|_| vec![x[0] + rng.random_range(-3..=3)])
.collect::<Vec<_>>()
});
let ra = a.run(&GridProblem);
let rb = b.run(&GridProblem);
assert_eq!(
ra.best.unwrap().evaluation.objectives,
rb.best.unwrap().evaluation.objectives,
);
}
}
src/algorithms/tlbo.rs
+234
View File
@@ -0,0 +1,234 @@
//! `Tlbo` — Rao 2011 Teaching-Learning-Based Optimization, parameter-free
//! single-objective optimizer for `Vec<f64>` decisions.
use rand::Rng as _;
use crate::core::candidate::Candidate;
use crate::core::evaluation::Evaluation;
use crate::core::objective::Direction;
use crate::core::population::Population;
use crate::core::problem::Problem;
use crate::core::result::OptimizationResult;
use crate::core::rng::rng_from_seed;
use crate::operators::real::RealBounds;
use crate::pareto::front::best_candidate;
use crate::traits::Optimizer;
/// Configuration for [`Tlbo`].
#[derive(Debug, Clone)]
pub struct TlboConfig {
/// Population size (= number of "learners").
pub population_size: usize,
/// Number of generations.
pub generations: usize,
/// Seed for the deterministic RNG.
pub seed: u64,
}
impl Default for TlboConfig {
fn default() -> Self {
Self {
population_size: 30,
generations: 200,
seed: 42,
}
}
}
/// Teaching-Learning-Based Optimization.
///
/// The standout feature: NO algorithm-specific hyperparameters. Just
/// population_size and generations. Compared with the rest of heuropt's
/// SO toolkit (DE has F+CR, PSO has w+c1+c2, CMA-ES has σ, GA needs
/// crossover+mutation operators), TLBO works out of the box.
#[derive(Debug, Clone)]
pub struct Tlbo {
/// Algorithm configuration.
pub config: TlboConfig,
/// Per-variable bounds — used both to seed the population and to clamp
/// every learner's position.
pub bounds: RealBounds,
}
impl Tlbo {
/// Construct a `Tlbo`.
pub fn new(config: TlboConfig, bounds: RealBounds) -> Self {
Self { config, bounds }
}
}
impl<P> Optimizer<P> for Tlbo
where
P: Problem<Decision = Vec<f64>> + Sync,
{
fn run(&mut self, problem: &P) -> OptimizationResult<P::Decision> {
assert!(
self.config.population_size >= 2,
"Tlbo population_size must be >= 2"
);
let objectives = problem.objectives();
assert!(
objectives.is_single_objective(),
"Tlbo requires exactly one objective",
);
let direction = objectives.objectives[0].direction;
let dim = self.bounds.bounds.len();
let n = self.config.population_size;
let mut rng = rng_from_seed(self.config.seed);
let mut decisions: Vec<Vec<f64>> = {
use crate::traits::Initializer as _;
self.bounds.initialize(n, &mut rng)
};
let mut evals: Vec<Evaluation> = decisions.iter().map(|d| problem.evaluate(d)).collect();
let mut evaluations = decisions.len();
for _ in 0..self.config.generations {
// Identify teacher (best learner).
let teacher_idx = best_index(&evals, direction);
let teacher = decisions[teacher_idx].clone();
// Compute the population mean per dimension.
let mut mean = vec![0.0_f64; dim];
for d in &decisions {
for j in 0..dim {
mean[j] += d[j];
}
}
for v in mean.iter_mut() {
*v /= n as f64;
}
// Teaching factor.
let tf = if rng.random_bool(0.5) { 1.0 } else { 2.0 };
// Teacher phase.
for i in 0..n {
let mut candidate = decisions[i].clone();
for j in 0..dim {
let r: f64 = rng.random();
candidate[j] += r * (teacher[j] - tf * mean[j]);
let (lo, hi) = self.bounds.bounds[j];
candidate[j] = candidate[j].clamp(lo, hi);
}
let cand_eval = problem.evaluate(&candidate);
evaluations += 1;
if better(&cand_eval, &evals[i], direction) {
decisions[i] = candidate;
evals[i] = cand_eval;
}
}
// Learner phase: each learner mates with a random different
// partner and accepts a move toward the better one.
for i in 0..n {
let mut k = rng.random_range(0..n);
while k == i && n > 1 {
k = rng.random_range(0..n);
}
let partner_better = better(&evals[k], &evals[i], direction);
let mut candidate = decisions[i].clone();
for j in 0..dim {
let r: f64 = rng.random();
let delta = if partner_better {
r * (decisions[k][j] - decisions[i][j])
} else {
r * (decisions[i][j] - decisions[k][j])
};
candidate[j] += delta;
let (lo, hi) = self.bounds.bounds[j];
candidate[j] = candidate[j].clamp(lo, hi);
}
let cand_eval = problem.evaluate(&candidate);
evaluations += 1;
if better(&cand_eval, &evals[i], direction) {
decisions[i] = candidate;
evals[i] = cand_eval;
}
}
}
let final_pop: Vec<Candidate<Vec<f64>>> = decisions
.into_iter()
.zip(evals)
.map(|(d, e)| Candidate::new(d, e))
.collect();
let best = best_candidate(&final_pop, &objectives);
let front: Vec<Candidate<Vec<f64>>> = best.iter().cloned().collect();
OptimizationResult::new(
Population::new(final_pop),
front,
best,
evaluations,
self.config.generations,
)
}
}
fn best_index(evals: &[Evaluation], direction: Direction) -> usize {
let mut idx = 0;
for i in 1..evals.len() {
if better(&evals[i], &evals[idx], direction) {
idx = i;
}
}
idx
}
fn better(a: &Evaluation, b: &Evaluation, direction: Direction) -> bool {
match (a.is_feasible(), b.is_feasible()) {
(true, false) => true,
(false, true) => false,
(false, false) => a.constraint_violation < b.constraint_violation,
(true, true) => match direction {
Direction::Minimize => a.objectives[0] < b.objectives[0],
Direction::Maximize => a.objectives[0] > b.objectives[0],
},
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::tests_support::{SchafferN1, Sphere1D};
fn make_optimizer(seed: u64) -> Tlbo {
Tlbo::new(
TlboConfig {
population_size: 30,
generations: 100,
seed,
},
RealBounds::new(vec![(-5.0, 5.0)]),
)
}
#[test]
fn finds_minimum_of_sphere() {
let mut opt = make_optimizer(1);
let r = opt.run(&Sphere1D);
let best = r.best.unwrap();
assert!(
best.evaluation.objectives[0] < 1e-3,
"got f = {}",
best.evaluation.objectives[0],
);
}
#[test]
fn deterministic_with_same_seed() {
let mut a = make_optimizer(99);
let mut b = make_optimizer(99);
let ra = a.run(&Sphere1D);
let rb = b.run(&Sphere1D);
assert_eq!(
ra.best.unwrap().evaluation.objectives,
rb.best.unwrap().evaluation.objectives,
);
}
#[test]
#[should_panic(expected = "exactly one objective")]
fn multi_objective_panics() {
let mut opt = make_optimizer(0);
let _ = opt.run(&SchafferN1);
}
}
src/algorithms/tpe.rs
+389
View File
@@ -0,0 +1,389 @@
//! `Tpe` — Bergstra et al. 2011 Tree-structured Parzen Estimator.
use rand::Rng as _;
use rand_distr::{Distribution, Normal};
use crate::core::candidate::Candidate;
use crate::core::evaluation::Evaluation;
use crate::core::objective::Direction;
use crate::core::population::Population;
use crate::core::problem::Problem;
use crate::core::result::OptimizationResult;
use crate::core::rng::{Rng, rng_from_seed};
use crate::operators::real::RealBounds;
use crate::traits::Optimizer;
/// Configuration for [`Tpe`].
#[derive(Debug, Clone)]
pub struct TpeConfig {
/// Number of uniform-random initial samples before the TPE loop starts.
pub initial_samples: usize,
/// Number of TPE iterations after the initial design.
pub iterations: usize,
/// Top-γ fraction of observations classified as "good." Bergstra
/// et al. recommend γ = 0.25.
pub good_fraction: f64,
/// Number of candidate samples drawn from the 'good' KDE per step.
/// The one with the largest `l(x) / g(x)` ratio is chosen.
pub candidate_samples: usize,
/// Bandwidth multiplier on the per-axis KDE (Scott's rule × this
/// factor). 1.0 is the standard rule; 0.5–2.0 is the practical range.
pub bandwidth_factor: f64,
/// Seed for the deterministic RNG.
pub seed: u64,
}
impl Default for TpeConfig {
fn default() -> Self {
Self {
initial_samples: 10,
iterations: 90,
good_fraction: 0.25,
candidate_samples: 24,
bandwidth_factor: 1.0,
seed: 42,
}
}
}
/// Tree-structured Parzen Estimator.
///
/// Sample-efficient sequential optimizer for `Vec<f64>` decisions. Unlike
/// `BayesianOpt`, no GP — TPE models `p(x | y < y*)` and `p(x | y >= y*)`
/// as per-axis Gaussian KDEs and picks the next candidate by maximizing
/// the ratio of the two densities.
#[derive(Debug, Clone)]
pub struct Tpe {
/// Algorithm configuration.
pub config: TpeConfig,
/// Per-variable bounds.
pub bounds: RealBounds,
}
impl Tpe {
/// Construct a `Tpe`.
pub fn new(config: TpeConfig, bounds: RealBounds) -> Self {
Self { config, bounds }
}
}
impl<P> Optimizer<P> for Tpe
where
P: Problem<Decision = Vec<f64>> + Sync,
{
fn run(&mut self, problem: &P) -> OptimizationResult<P::Decision> {
assert!(
self.config.initial_samples >= 2,
"Tpe initial_samples must be >= 2"
);
assert!(
self.config.good_fraction > 0.0 && self.config.good_fraction < 1.0,
"Tpe good_fraction must be in (0, 1)",
);
assert!(
self.config.candidate_samples >= 1,
"Tpe candidate_samples must be >= 1",
);
assert!(
self.config.bandwidth_factor > 0.0,
"Tpe bandwidth_factor must be > 0"
);
let objectives = problem.objectives();
assert!(
objectives.is_single_objective(),
"Tpe requires exactly one objective",
);
let direction = objectives.objectives[0].direction;
let dim = self.bounds.bounds.len();
let mut rng = rng_from_seed(self.config.seed);
let mut decisions: Vec<Vec<f64>> = Vec::new();
let mut targets: Vec<f64> = Vec::new();
let mut evals: Vec<Evaluation> = Vec::new();
for _ in 0..self.config.initial_samples {
let x = sample_uniform_in_bounds(&self.bounds, &mut rng);
let e = problem.evaluate(&x);
targets.push(oriented_target(&e, direction));
decisions.push(x);
evals.push(e);
}
for _ in 0..self.config.iterations {
// Split into good vs bad observations.
let (good_idx, bad_idx) = split_good_bad(&targets, self.config.good_fraction);
// Sample candidates from the good KDE.
let mut best_x: Option<Vec<f64>> = None;
let mut best_ratio = f64::NEG_INFINITY;
for _ in 0..self.config.candidate_samples {
let cand = sample_from_kde(
&decisions,
&good_idx,
&self.bounds,
self.config.bandwidth_factor,
&mut rng,
);
let l = log_kde_density(
&cand,
&decisions,
&good_idx,
&self.bounds,
self.config.bandwidth_factor,
);
let g = log_kde_density(
&cand,
&decisions,
&bad_idx,
&self.bounds,
self.config.bandwidth_factor,
);
let ratio = l - g;
if ratio > best_ratio {
best_ratio = ratio;
best_x = Some(cand);
}
}
let x = best_x.expect("at least one candidate sampled");
let _ = dim;
let e = problem.evaluate(&x);
targets.push(oriented_target(&e, direction));
decisions.push(x);
evals.push(e);
}
// Identify the best observation.
let mut best_idx = 0;
for i in 1..evals.len() {
if better(&evals[i], &evals[best_idx], direction) {
best_idx = i;
}
}
let total_evals = evals.len();
let final_pop: Vec<Candidate<Vec<f64>>> = decisions
.into_iter()
.zip(evals)
.map(|(d, e)| Candidate::new(d, e))
.collect();
let best = final_pop[best_idx].clone();
let front = vec![best.clone()];
OptimizationResult::new(
Population::new(final_pop),
front,
Some(best),
total_evals,
self.config.iterations + self.config.initial_samples,
)
}
}
fn oriented_target(e: &Evaluation, direction: Direction) -> f64 {
let base = match direction {
Direction::Minimize => e.objectives[0],
Direction::Maximize => -e.objectives[0],
};
if e.is_feasible() {
base
} else {
base + 1e6 * e.constraint_violation
}
}
fn better(a: &Evaluation, b: &Evaluation, direction: Direction) -> bool {
match (a.is_feasible(), b.is_feasible()) {
(true, false) => true,
(false, true) => false,
(false, false) => a.constraint_violation < b.constraint_violation,
(true, true) => match direction {
Direction::Minimize => a.objectives[0] < b.objectives[0],
Direction::Maximize => a.objectives[0] > b.objectives[0],
},
}
}
fn sample_uniform_in_bounds(bounds: &RealBounds, rng: &mut Rng) -> Vec<f64> {
bounds
.bounds
.iter()
.map(|&(lo, hi)| {
if lo == hi {
lo
} else {
lo + (hi - lo) * rng.random::<f64>()
}
})
.collect()
}
/// Split observation indices into a "good" set (top `good_fraction` by
/// minimization target) and a "bad" set. Both sets are guaranteed
/// non-empty when there are at least 2 observations.
fn split_good_bad(targets: &[f64], good_fraction: f64) -> (Vec<usize>, Vec<usize>) {
let n = targets.len();
let mut order: Vec<usize> = (0..n).collect();
order.sort_by(|&a, &b| {
targets[a]
.partial_cmp(&targets[b])
.unwrap_or(std::cmp::Ordering::Equal)
});
let n_good = ((n as f64) * good_fraction).round() as usize;
let n_good = n_good.clamp(1, n.saturating_sub(1));
let good = order[..n_good].to_vec();
let bad = order[n_good..].to_vec();
(good, bad)
}
/// Sample one decision from a per-axis Gaussian mixture KDE on the
/// indices `support`. Each support point contributes a Gaussian per axis
/// with bandwidth chosen by Scott's rule (`σ̂ · n^(-1/5)`) scaled by
/// `bandwidth_factor`. The mixture weight is uniform over the support.
fn sample_from_kde(
decisions: &[Vec<f64>],
support: &[usize],
bounds: &RealBounds,
bandwidth_factor: f64,
rng: &mut Rng,
) -> Vec<f64> {
if support.is_empty() {
return sample_uniform_in_bounds(bounds, rng);
}
let dim = bounds.bounds.len();
let bandwidths = scott_bandwidths(decisions, support, bandwidth_factor);
let pick = support[rng.random_range(0..support.len())];
let center = &decisions[pick];
let mut x = vec![0.0_f64; dim];
for j in 0..dim {
let normal = Normal::new(center[j], bandwidths[j].max(1e-12)).unwrap();
let v = normal.sample(rng);
let (lo, hi) = bounds.bounds[j];
x[j] = v.clamp(lo, hi);
}
x
}
/// Per-axis log-density at `x` of the KDE built on `support`.
fn log_kde_density(
x: &[f64],
decisions: &[Vec<f64>],
support: &[usize],
bounds: &RealBounds,
bandwidth_factor: f64,
) -> f64 {
if support.is_empty() {
return f64::NEG_INFINITY;
}
let dim = bounds.bounds.len();
let bandwidths = scott_bandwidths(decisions, support, bandwidth_factor);
// Sum of per-axis log-densities, with the kernel a product of 1-D
// Gaussians. Using log-sum-exp for numerical stability would be more
// accurate, but the per-axis-product form is what TPE uses in
// practice and is fine for our optimization goal of *ranking*
// candidates by ratio.
let mut total = 0.0;
for j in 0..dim {
let h = bandwidths[j].max(1e-12);
let mut s = 0.0;
for &i in support {
let z = (x[j] - decisions[i][j]) / h;
s += (-0.5 * z * z).exp() / (h * (2.0 * std::f64::consts::PI).sqrt());
}
let mean_density = s / support.len() as f64;
total += mean_density.max(1e-300).ln();
}
total
}
/// Per-axis bandwidths via Scott's rule: `σ̂ · n^(-1/5)`, with `σ̂` the
/// per-axis standard deviation of the support.
fn scott_bandwidths(decisions: &[Vec<f64>], support: &[usize], factor: f64) -> Vec<f64> {
let dim = decisions[0].len();
let mut means = vec![0.0_f64; dim];
for &i in support {
for j in 0..dim {
means[j] += decisions[i][j];
}
}
let n = support.len() as f64;
for m in means.iter_mut() {
*m /= n;
}
let mut vars = vec![0.0_f64; dim];
for &i in support {
for j in 0..dim {
let d = decisions[i][j] - means[j];
vars[j] += d * d;
}
}
let denom = (support.len().saturating_sub(1).max(1)) as f64;
for v in vars.iter_mut() {
*v /= denom;
}
let scott_n = (support.len() as f64).powf(-0.2);
vars.into_iter()
.map(|v| factor * v.sqrt().max(1e-6) * scott_n)
.collect()
}
#[cfg(test)]
mod tests {
use super::*;
use crate::tests_support::{SchafferN1, Sphere1D};
fn make_optimizer(seed: u64) -> Tpe {
Tpe::new(
TpeConfig {
initial_samples: 10,
iterations: 50,
good_fraction: 0.25,
candidate_samples: 24,
bandwidth_factor: 1.0,
seed,
},
RealBounds::new(vec![(-5.0, 5.0)]),
)
}
#[test]
fn finds_minimum_of_sphere() {
let mut opt = make_optimizer(1);
let r = opt.run(&Sphere1D);
let best = r.best.unwrap();
// TPE without bandwidth tuning, 60 evals on 1-D sphere: clearly
// beats random search (which averages ≈ f = 8) but not as
// aggressive as well-tuned BO.
assert!(
best.evaluation.objectives[0] < 0.1,
"got f = {}",
best.evaluation.objectives[0],
);
}
#[test]
fn deterministic_with_same_seed() {
let mut a = make_optimizer(99);
let mut b = make_optimizer(99);
let ra = a.run(&Sphere1D);
let rb = b.run(&Sphere1D);
assert_eq!(
ra.best.unwrap().evaluation.objectives,
rb.best.unwrap().evaluation.objectives,
);
}
#[test]
fn split_handles_small_n() {
let (good, bad) = split_good_bad(&[3.0, 1.0, 2.0, 4.0, 5.0], 0.25);
// 0.25 × 5 = 1.25 → round to 1, so 1 good + 4 bad.
assert_eq!(good.len(), 1);
assert_eq!(bad.len(), 4);
assert_eq!(good[0], 1); // index of value 1.0 (the minimum)
}
#[test]
#[should_panic(expected = "exactly one objective")]
fn multi_objective_panics() {
let mut opt = make_optimizer(0);
let _ = opt.run(&SchafferN1);
}
}
src/algorithms/umda.rs
+293
View File
@@ -0,0 +1,293 @@
//! `Umda` — Mühlenbein 1997 Univariate Marginal Distribution Algorithm for
//! binary (`Vec<bool>`) decisions.
use rand::Rng as _;
use crate::algorithms::parallel_eval::evaluate_batch;
use crate::core::candidate::Candidate;
use crate::core::objective::Direction;
use crate::core::population::Population;
use crate::core::problem::Problem;
use crate::core::result::OptimizationResult;
use crate::core::rng::rng_from_seed;
use crate::pareto::front::best_candidate;
use crate::traits::Optimizer;
/// Configuration for [`Umda`].
#[derive(Debug, Clone)]
pub struct UmdaConfig {
/// Sample size per generation.
pub population_size: usize,
/// Number of top members to use for the marginal estimate.
pub selected_size: usize,
/// Number of generations.
pub generations: usize,
/// Number of bits in each decision.
pub bits: usize,
/// Seed for the deterministic RNG.
pub seed: u64,
}
impl Default for UmdaConfig {
fn default() -> Self {
Self {
population_size: 100,
selected_size: 50,
generations: 50,
bits: 32,
seed: 42,
}
}
}
/// Univariate Marginal Distribution Algorithm.
///
/// `Vec<bool>` decisions only; single-objective only. Each generation
/// estimates per-bit marginal probabilities from the top `selected_size`
/// members and samples the next population from the resulting independent
/// Bernoulli vector. Probabilities are clamped to
/// `[1 / (2·selected_size), 1 - 1 / (2·selected_size)]` (Laplace-style
/// smoothing) so the population never collapses to a single deterministic
/// string.
#[derive(Debug, Clone)]
pub struct Umda {
/// Algorithm configuration.
pub config: UmdaConfig,
}
impl Umda {
/// Construct a `Umda`.
pub fn new(config: UmdaConfig) -> Self {
Self { config }
}
}
impl<P> Optimizer<P> for Umda
where
P: Problem<Decision = Vec<bool>> + Sync,
{
fn run(&mut self, problem: &P) -> OptimizationResult<P::Decision> {
assert!(
self.config.population_size >= 2,
"Umda population_size must be >= 2"
);
assert!(
self.config.selected_size >= 1,
"Umda selected_size must be >= 1",
);
assert!(
self.config.selected_size <= self.config.population_size,
"Umda selected_size must be <= population_size",
);
assert!(self.config.bits >= 1, "Umda bits must be >= 1");
let objectives = problem.objectives();
assert!(
objectives.is_single_objective(),
"Umda requires exactly one objective",
);
let direction = objectives.objectives[0].direction;
let n = self.config.population_size;
let bits = self.config.bits;
let mu = self.config.selected_size;
let mut rng = rng_from_seed(self.config.seed);
// Initial sample: uniform Bernoulli(0.5) across all bits.
let mut decisions: Vec<Vec<bool>> = (0..n)
.map(|_| (0..bits).map(|_| rng.random_bool(0.5)).collect())
.collect();
let mut population = evaluate_batch(problem, decisions.clone());
let mut evaluations = population.len();
let smoothing = 1.0 / (2.0 * mu as f64);
let prob_min = smoothing;
let prob_max = 1.0 - smoothing;
let mut best_seen: Option<Candidate<Vec<bool>>> = None;
for c in &population {
let beats = match &best_seen {
None => true,
Some(b) => better_than_so(&c.evaluation, &b.evaluation, direction),
};
if beats {
best_seen = Some(c.clone());
}
}
for _ in 0..self.config.generations {
// --- Phase 1: select top μ members ---
let mut order: Vec<usize> = (0..population.len()).collect();
order.sort_by(|&a, &b| {
compare_so(
&population[a].evaluation,
&population[b].evaluation,
direction,
)
});
let selected: Vec<&Candidate<Vec<bool>>> =
order.iter().take(mu).map(|&i| &population[i]).collect();
// --- Phase 2: estimate per-bit marginals ---
let mut probs = vec![0.0_f64; bits];
for c in &selected {
for (i, b) in c.decision.iter().enumerate() {
if *b {
probs[i] += 1.0;
}
}
}
for p in probs.iter_mut() {
*p = (*p / mu as f64).clamp(prob_min, prob_max);
}
// --- Phase 3: sample a new population (uses RNG serially) ---
decisions = (0..n)
.map(|_| probs.iter().map(|&p| rng.random_bool(p)).collect())
.collect();
// --- Phase 4: evaluate (parallel-friendly) ---
population = evaluate_batch(problem, decisions.clone());
evaluations += population.len();
// Track best.
for c in &population {
let beats = match &best_seen {
None => true,
Some(b) => better_than_so(&c.evaluation, &b.evaluation, direction),
};
if beats {
best_seen = Some(c.clone());
}
}
}
let best = best_seen.expect("at least one generation evaluated");
let final_pop = vec![best.clone()];
let front = vec![best.clone()];
let best_opt = best_candidate(&final_pop, &objectives);
OptimizationResult::new(
Population::new(final_pop),
front,
best_opt,
evaluations,
self.config.generations,
)
}
}
fn compare_so(
a: &crate::core::evaluation::Evaluation,
b: &crate::core::evaluation::Evaluation,
direction: Direction,
) -> std::cmp::Ordering {
match (a.is_feasible(), b.is_feasible()) {
(true, false) => std::cmp::Ordering::Less,
(false, true) => std::cmp::Ordering::Greater,
(false, false) => a
.constraint_violation
.partial_cmp(&b.constraint_violation)
.unwrap_or(std::cmp::Ordering::Equal),
(true, true) => match direction {
Direction::Minimize => a.objectives[0]
.partial_cmp(&b.objectives[0])
.unwrap_or(std::cmp::Ordering::Equal),
Direction::Maximize => b.objectives[0]
.partial_cmp(&a.objectives[0])
.unwrap_or(std::cmp::Ordering::Equal),
},
}
}
fn better_than_so(
a: &crate::core::evaluation::Evaluation,
b: &crate::core::evaluation::Evaluation,
direction: Direction,
) -> bool {
compare_so(a, b, direction) == std::cmp::Ordering::Less
}
#[cfg(test)]
mod tests {
use super::*;
use crate::core::evaluation::Evaluation;
use crate::core::objective::{Objective, ObjectiveSpace};
/// OneMax: maximize the sum of true bits.
struct OneMax {
#[allow(dead_code)]
bits: usize,
}
impl Problem for OneMax {
type Decision = Vec<bool>;
fn objectives(&self) -> ObjectiveSpace {
ObjectiveSpace::new(vec![Objective::maximize("bits")])
}
fn evaluate(&self, x: &Vec<bool>) -> Evaluation {
let count = x.iter().filter(|b| **b).count();
Evaluation::new(vec![count as f64])
}
}
/// Trivial multi-objective problem to exercise the panic.
struct DummyMo;
impl Problem for DummyMo {
type Decision = Vec<bool>;
fn objectives(&self) -> ObjectiveSpace {
ObjectiveSpace::new(vec![Objective::minimize("a"), Objective::minimize("b")])
}
fn evaluate(&self, _x: &Vec<bool>) -> Evaluation {
Evaluation::new(vec![0.0, 0.0])
}
}
#[test]
fn solves_onemax_20() {
let problem = OneMax { bits: 20 };
let mut opt = Umda::new(UmdaConfig {
population_size: 50,
selected_size: 20,
generations: 30,
bits: 20,
seed: 1,
});
let r = opt.run(&problem);
let best = r.best.unwrap();
assert_eq!(best.evaluation.objectives[0], 20.0);
}
#[test]
fn deterministic_with_same_seed() {
let problem = OneMax { bits: 16 };
let cfg = UmdaConfig {
population_size: 30,
selected_size: 10,
generations: 10,
bits: 16,
seed: 99,
};
let mut a = Umda::new(cfg.clone());
let mut b = Umda::new(cfg);
let ra = a.run(&problem);
let rb = b.run(&problem);
assert_eq!(
ra.best.unwrap().evaluation.objectives,
rb.best.unwrap().evaluation.objectives,
);
}
#[test]
#[should_panic(expected = "exactly one objective")]
fn multi_objective_panics() {
let mut opt = Umda::new(UmdaConfig {
population_size: 10,
selected_size: 5,
generations: 1,
bits: 4,
seed: 0,
});
let _ = opt.run(&DummyMo);
}
}
src/core/candidate.rs
+4 -1
View File
@@ -18,7 +18,10 @@ pub struct Candidate<D> {
impl<D> Candidate<D> { impl<D> Candidate<D> {
/// Pair a decision with its evaluation. /// Pair a decision with its evaluation.
pub fn new(decision: D, evaluation: Evaluation) -> Self { pub fn new(decision: D, evaluation: Evaluation) -> Self {
Self { decision, evaluation } Self {
decision,
evaluation,
}
} }
} }
src/core/evaluation.rs
+8 -2
View File
@@ -18,12 +18,18 @@ pub struct Evaluation {
impl Evaluation { impl Evaluation {
/// Build a feasible evaluation from objective values. /// Build a feasible evaluation from objective values.
pub fn new(objectives: Vec<f64>) -> Self { pub fn new(objectives: Vec<f64>) -> Self {
Self { objectives, constraint_violation: 0.0 } Self {
objectives,
constraint_violation: 0.0,
}
} }
/// Build an evaluation with a known total constraint violation. /// Build an evaluation with a known total constraint violation.
pub fn constrained(objectives: Vec<f64>, constraint_violation: f64) -> Self { pub fn constrained(objectives: Vec<f64>, constraint_violation: f64) -> Self {
Self { objectives, constraint_violation } Self {
objectives,
constraint_violation,
}
} }
/// Returns `true` when `constraint_violation <= 0.0`. /// Returns `true` when `constraint_violation <= 0.0`.
src/core/mod.rs
+2
View File
@@ -3,6 +3,7 @@
pub mod candidate; pub mod candidate;
pub mod evaluation; pub mod evaluation;
pub mod objective; pub mod objective;
pub mod partial_problem;
pub mod population; pub mod population;
pub mod problem; pub mod problem;
pub mod result; pub mod result;
@@ -11,6 +12,7 @@ pub mod rng;
pub use candidate::*; pub use candidate::*;
pub use evaluation::*; pub use evaluation::*;
pub use objective::*; pub use objective::*;
pub use partial_problem::*;
pub use population::*; pub use population::*;
pub use problem::*; pub use problem::*;
pub use result::*; pub use result::*;
src/core/objective.rs
+9 -6
View File
@@ -26,12 +26,18 @@ pub struct Objective {
impl Objective { impl Objective {
/// Create a minimize objective with the given name. /// Create a minimize objective with the given name.
pub fn minimize(name: impl Into<String>) -> Self { pub fn minimize(name: impl Into<String>) -> Self {
Self { name: name.into(), direction: Direction::Minimize } Self {
name: name.into(),
direction: Direction::Minimize,
}
} }
/// Create a maximize objective with the given name. /// Create a maximize objective with the given name.
pub fn maximize(name: impl Into<String>) -> Self { pub fn maximize(name: impl Into<String>) -> Self {
Self { name: name.into(), direction: Direction::Maximize } Self {
name: name.into(),
direction: Direction::Maximize,
}
} }
} }
@@ -125,10 +131,7 @@ mod tests {
assert!(!single.is_empty()); assert!(!single.is_empty());
assert_eq!(single.len(), 1); assert_eq!(single.len(), 1);
let multi = ObjectiveSpace::new(vec![ let multi = ObjectiveSpace::new(vec![Objective::minimize("f1"), Objective::minimize("f2")]);
Objective::minimize("f1"),
Objective::minimize("f2"),
]);
assert!(multi.is_multi_objective()); assert!(multi.is_multi_objective());
assert!(!multi.is_single_objective()); assert!(!multi.is_single_objective());
src/core/partial_problem.rs
+39
View File
@@ -0,0 +1,39 @@
//! Trait for multi-fidelity optimization problems.
use crate::core::evaluation::Evaluation;
use crate::core::objective::ObjectiveSpace;
/// A problem whose evaluation cost can be controlled by a fidelity
/// "budget" parameter — for example, an ML training run that gets
/// trained for `budget` epochs, a CFD simulation that runs for `budget`
/// timesteps, or a Monte Carlo evaluation that draws `budget` samples.
///
/// Multi-fidelity optimizers (Hyperband, Successive Halving, BOHB) use
/// this trait to evaluate cheap low-budget previews of many
/// configurations, then "promote" the survivors to higher budgets.
///
/// `PartialProblem` is intentionally NOT a sub-trait of [`Problem`]
/// because the evaluation contract is different: `Problem::evaluate`
/// is single-shot, while `evaluate_at_budget` is parameterized by
/// fidelity. Implementors who already have a `Problem` and want their
/// `evaluate_at_budget` to ignore the budget can write a one-line
/// wrapper that just calls `Problem::evaluate`.
///
/// [`Problem`]: crate::core::Problem
pub trait PartialProblem {
/// The thing the optimizer changes. Same constraints as
/// [`Problem::Decision`](crate::core::Problem::Decision).
type Decision: Clone;
/// Return the objectives for this problem.
fn objectives(&self) -> ObjectiveSpace;
/// Evaluate `decision` at the given fidelity `budget`.
///
/// Higher `budget` should give a more accurate (and more expensive)
/// estimate of the same underlying objective. Hyperband requires
/// monotonicity: a higher-budget evaluation should not be worse
/// than a lower-budget evaluation by chance — though some noise is
/// fine and expected.
fn evaluate_at_budget(&self, decision: &Self::Decision, budget: f64) -> Evaluation;
}
src/core/result.rs
+7 -1
View File
@@ -31,7 +31,13 @@ impl<D> OptimizationResult<D> {
evaluations: usize, evaluations: usize,
generations: usize, generations: usize,
) -> Self { ) -> Self {
Self { population, pareto_front, best, evaluations, generations } Self {
population,
pareto_front,
best,
evaluations,
generations,
}
} }
/// The final population. /// The final population.
src/internal/cholesky.rs
+141
View File
@@ -0,0 +1,141 @@
//! Cholesky factorization (`A = L · L^T`) plus triangular solves for
//! symmetric positive-definite matrices.
//!
//! Used internally by Bayesian Optimization for the GP posterior. Hand-
//! rolled to avoid pulling in a linear-algebra dependency.
/// Factorize a symmetric positive-definite matrix `a` as `L · L^T`,
/// returning `L` (lower triangular). Returns `Err` if `a` is not SPD,
/// which the caller typically responds to by adding jitter to the
/// diagonal and retrying.
pub(crate) fn cholesky(a: &[Vec<f64>]) -> Result<Vec<Vec<f64>>, &'static str> {
let n = a.len();
if n == 0 {
return Ok(Vec::new());
}
debug_assert!(a.iter().all(|row| row.len() == n));
let mut l = vec![vec![0.0_f64; n]; n];
#[allow(clippy::needless_range_loop)] // body indexes both `a` and `l` rows.
for i in 0..n {
for j in 0..=i {
let mut sum = a[i][j];
#[allow(clippy::needless_range_loop)]
for k in 0..j {
sum -= l[i][k] * l[j][k];
}
if i == j {
if sum <= 0.0 {
return Err("matrix is not positive-definite");
}
l[i][j] = sum.sqrt();
} else {
if l[j][j].abs() < 1e-300 {
return Err("zero on diagonal during Cholesky");
}
l[i][j] = sum / l[j][j];
}
}
}
Ok(l)
}
/// Solve `L · y = b` (forward substitution) for lower-triangular `L`.
pub(crate) fn solve_lower(l: &[Vec<f64>], b: &[f64]) -> Vec<f64> {
let n = l.len();
let mut y = vec![0.0_f64; n];
for i in 0..n {
let mut sum = b[i];
for k in 0..i {
sum -= l[i][k] * y[k];
}
y[i] = sum / l[i][i];
}
y
}
/// Solve `L^T · x = y` (backward substitution) for lower-triangular `L`
/// (so `L^T` is upper-triangular).
pub(crate) fn solve_upper_transpose(l: &[Vec<f64>], y: &[f64]) -> Vec<f64> {
let n = l.len();
let mut x = vec![0.0_f64; n];
for i in (0..n).rev() {
let mut sum = y[i];
for k in (i + 1)..n {
sum -= l[k][i] * x[k];
}
x[i] = sum / l[i][i];
}
x
}
/// Solve `A · x = b` given the Cholesky factor `L` of `A`. One forward
/// substitution + one back substitution.
pub(crate) fn solve(l: &[Vec<f64>], b: &[f64]) -> Vec<f64> {
let y = solve_lower(l, b);
solve_upper_transpose(l, &y)
}
#[cfg(test)]
mod tests {
use super::*;
fn approx_eq(a: f64, b: f64, tol: f64) -> bool {
(a - b).abs() < tol
}
#[test]
fn two_by_two_spd_factors() {
// A = [[4, 2], [2, 5]] → L = [[2, 0], [1, 2]]
let a = vec![vec![4.0, 2.0], vec![2.0, 5.0]];
let l = cholesky(&a).unwrap();
assert!(approx_eq(l[0][0], 2.0, 1e-12));
assert!(approx_eq(l[1][0], 1.0, 1e-12));
assert!(approx_eq(l[1][1], 2.0, 1e-12));
// L · L^T should reconstruct A.
#[allow(clippy::needless_range_loop)]
for i in 0..2 {
for j in 0..2 {
let mut s = 0.0;
#[allow(clippy::needless_range_loop)]
for k in 0..2 {
s += l[i][k] * l[j][k];
}
assert!(approx_eq(s, a[i][j], 1e-12));
}
}
}
#[test]
fn three_by_three_solve_round_trip() {
// SPD 3x3 with a known answer.
let a = vec![
vec![25.0, 15.0, -5.0],
vec![15.0, 18.0, 0.0],
vec![-5.0, 0.0, 11.0],
];
let l = cholesky(&a).unwrap();
// Choose a vector and check A · x = b round trip.
let x_truth = [1.0, -2.0, 0.5];
let b: Vec<f64> = (0..3)
.map(|i| (0..3).map(|j| a[i][j] * x_truth[j]).sum())
.collect();
let x = solve(&l, &b);
for k in 0..3 {
assert!(approx_eq(x[k], x_truth[k], 1e-9));
}
}
#[test]
fn non_psd_returns_err() {
// [[1, 2], [2, 1]] has eigenvalues 3 and -1 → not PD.
let a = vec![vec![1.0, 2.0], vec![2.0, 1.0]];
assert!(cholesky(&a).is_err());
}
#[test]
fn empty_matrix() {
let a: Vec<Vec<f64>> = Vec::new();
let l = cholesky(&a).unwrap();
assert_eq!(l.len(), 0);
}
}
src/internal/eigen.rs
+205
View File
@@ -0,0 +1,205 @@
//! Symmetric-matrix eigendecomposition via cyclic Jacobi rotations.
//!
//! Used internally by CMA-ES to maintain the covariance matrix's
//! eigendecomposition each generation. Hand-rolled to avoid pulling in a
//! linear-algebra dependency for one algorithm.
/// Symmetric eigendecomposition of an `n × n` matrix.
///
/// `matrix` must be square and symmetric (caller's responsibility — this is
/// `pub(crate)`). Returns `(eigenvalues, eigenvectors)` where:
///
/// - `eigenvalues[i]` is the i-th eigenvalue, in **descending** order.
/// - `eigenvectors[i]` is the corresponding unit eigenvector (row).
///
/// Iterates cyclic Jacobi rotations until the largest off-diagonal magnitude
/// is below `tol` or `max_sweeps` sweeps have completed. For typical CMA-ES
/// usage (small N, well-conditioned C) convergence is fast.
pub(crate) fn symmetric_eigen(
matrix: &[Vec<f64>],
tol: f64,
max_sweeps: usize,
) -> (Vec<f64>, Vec<Vec<f64>>) {
let n = matrix.len();
debug_assert!(
matrix.iter().all(|row| row.len() == n),
"matrix must be square"
);
// Working copy of the matrix; converges to a diagonal of eigenvalues.
let mut a: Vec<Vec<f64>> = matrix.to_vec();
// Eigenvector accumulator, starts as identity.
let mut v: Vec<Vec<f64>> = (0..n)
.map(|i| (0..n).map(|j| if i == j { 1.0 } else { 0.0 }).collect())
.collect();
for _ in 0..max_sweeps {
let mut max_off = 0.0;
#[allow(clippy::needless_range_loop)]
for i in 0..n {
for j in (i + 1)..n {
let abs_off = a[i][j].abs();
if abs_off > max_off {
max_off = abs_off;
}
}
}
if max_off < tol {
break;
}
// Cyclic sweep: rotate every (i, j) pair once.
for p in 0..n {
for q in (p + 1)..n {
let apq = a[p][q];
if apq.abs() < tol {
continue;
}
let app = a[p][p];
let aqq = a[q][q];
// Rotation angle (Givens) chosen to zero out a[p][q].
let theta = (aqq - app) / (2.0 * apq);
let t = if theta >= 0.0 {
1.0 / (theta + (1.0 + theta * theta).sqrt())
} else {
1.0 / (theta - (1.0 + theta * theta).sqrt())
};
let c = 1.0 / (1.0 + t * t).sqrt();
let s = t * c;
let tau = s / (1.0 + c);
// Update diagonal entries.
a[p][p] = app - t * apq;
a[q][q] = aqq + t * apq;
a[p][q] = 0.0;
a[q][p] = 0.0;
// Update other off-diagonal entries in rows/cols p and q.
#[allow(clippy::needless_range_loop)]
for r in 0..n {
if r != p && r != q {
let arp = a[r][p];
let arq = a[r][q];
a[r][p] = arp - s * (arq + tau * arp);
a[r][q] = arq + s * (arp - tau * arq);
a[p][r] = a[r][p];
a[q][r] = a[r][q];
}
}
// Update accumulated eigenvectors.
#[allow(clippy::needless_range_loop)]
for r in 0..n {
let vrp = v[r][p];
let vrq = v[r][q];
v[r][p] = vrp - s * (vrq + tau * vrp);
v[r][q] = vrq + s * (vrp - tau * vrq);
}
}
}
}
// Extract eigenvalues from the diagonal of `a` and pair them with their
// eigenvectors (columns of `v`).
let mut pairs: Vec<(f64, Vec<f64>)> = (0..n)
.map(|i| {
let val = a[i][i];
let vec: Vec<f64> = (0..n).map(|r| v[r][i]).collect();
(val, vec)
})
.collect();
// Sort by eigenvalue descending.
pairs.sort_by(|a, b| b.0.partial_cmp(&a.0).unwrap_or(std::cmp::Ordering::Equal));
let eigenvalues: Vec<f64> = pairs.iter().map(|(v, _)| *v).collect();
let eigenvectors: Vec<Vec<f64>> = pairs.into_iter().map(|(_, v)| v).collect();
(eigenvalues, eigenvectors)
}
#[cfg(test)]
mod tests {
use super::*;
fn approx_eq(a: f64, b: f64, tol: f64) -> bool {
(a - b).abs() < tol
}
fn dot(a: &[f64], b: &[f64]) -> f64 {
a.iter().zip(b.iter()).map(|(x, y)| x * y).sum()
}
fn norm(v: &[f64]) -> f64 {
v.iter().map(|x| x * x).sum::<f64>().sqrt()
}
#[test]
fn diagonal_matrix_keeps_eigenvalues_on_diagonal() {
let m = vec![
vec![3.0, 0.0, 0.0],
vec![0.0, 1.0, 0.0],
vec![0.0, 0.0, 5.0],
];
let (vals, vecs) = symmetric_eigen(&m, 1e-12, 50);
// Sorted descending: 5, 3, 1.
assert!(approx_eq(vals[0], 5.0, 1e-10));
assert!(approx_eq(vals[1], 3.0, 1e-10));
assert!(approx_eq(vals[2], 1.0, 1e-10));
for v in &vecs {
assert!(approx_eq(norm(v), 1.0, 1e-10));
}
}
#[test]
fn two_by_two_known_case() {
// [[2, 1], [1, 2]] has eigenvalues 3 and 1, eigenvectors (1,1)/√2 and (1,-1)/√2.
let m = vec![vec![2.0, 1.0], vec![1.0, 2.0]];
let (vals, vecs) = symmetric_eigen(&m, 1e-12, 50);
assert!(approx_eq(vals[0], 3.0, 1e-10));
assert!(approx_eq(vals[1], 1.0, 1e-10));
// Each eigenvector has unit norm.
for v in &vecs {
assert!(approx_eq(norm(v), 1.0, 1e-10));
}
// (1,1)/√2 ≈ (0.7071, 0.7071): components have the same sign.
assert!((vecs[0][0] - vecs[0][1]).abs() < 1e-10);
// (1,-1)/√2: components have opposite signs.
assert!((vecs[1][0] + vecs[1][1]).abs() < 1e-10);
}
#[test]
fn reconstruct_via_a_v_equals_lambda_v() {
// Reconstruct A · v_i ≈ λ_i · v_i for a small symmetric matrix.
let m = vec![
vec![4.0, 1.0, -2.0],
vec![1.0, 2.0, 0.5],
vec![-2.0, 0.5, 3.0],
];
let (vals, vecs) = symmetric_eigen(&m, 1e-12, 100);
for (lambda, v) in vals.iter().zip(vecs.iter()) {
// A · v
let av: Vec<f64> = (0..3)
.map(|i| (0..3).map(|j| m[i][j] * v[j]).sum::<f64>())
.collect();
// λ · v
let lv: Vec<f64> = v.iter().map(|x| lambda * x).collect();
for (x, y) in av.iter().zip(lv.iter()) {
assert!(approx_eq(*x, *y, 1e-9), "Av != λv: {x} vs {y}");
}
}
}
#[test]
fn eigenvectors_are_orthogonal() {
let m = vec![
vec![4.0, 1.0, -2.0],
vec![1.0, 2.0, 0.5],
vec![-2.0, 0.5, 3.0],
];
let (_, vecs) = symmetric_eigen(&m, 1e-12, 100);
for i in 0..3 {
for j in (i + 1)..3 {
assert!(approx_eq(dot(&vecs[i], &vecs[j]), 0.0, 1e-9));
}
}
}
}
src/internal/mod.rs
+4
View File
@@ -0,0 +1,4 @@
//! Internal helpers used by built-in algorithms but not part of the public API.
pub(crate) mod cholesky;
pub(crate) mod eigen;
src/lib.rs
+1
View File
@@ -45,6 +45,7 @@
pub mod algorithms; pub mod algorithms;
pub mod core; pub mod core;
pub(crate) mod internal;
pub mod metrics; pub mod metrics;
pub mod operators; pub mod operators;
pub mod pareto; pub mod pareto;
src/metrics/hypervolume.rs
+326 -6
View File
@@ -1,6 +1,7 @@
//! Exact 2D hypervolume against a fixed reference point. //! Exact 2D and N-D hypervolume against a fixed reference point.
use crate::core::candidate::Candidate; use crate::core::candidate::Candidate;
use crate::core::evaluation::Evaluation;
use crate::core::objective::ObjectiveSpace; use crate::core::objective::ObjectiveSpace;
/// Compute the dominated hypervolume of a 2D front against `reference_point`. /// Compute the dominated hypervolume of a 2D front against `reference_point`.
@@ -70,10 +71,7 @@ mod tests {
} }
fn space_min2() -> ObjectiveSpace { fn space_min2() -> ObjectiveSpace {
ObjectiveSpace::new(vec![ ObjectiveSpace::new(vec![Objective::minimize("f1"), Objective::minimize("f2")])
Objective::minimize("f1"),
Objective::minimize("f2"),
])
} }
#[test] #[test]
@@ -82,7 +80,11 @@ mod tests {
// Dominated region area = 4*4 - sum of "outside" rectangles // Dominated region area = 4*4 - sum of "outside" rectangles
// stripes: x∈[1,2] y∈[3,4]→1, x∈[2,3] y∈[2,4]→2, x∈[3,4] y∈[1,4]→3 → total dominated = 1+2+3 = 6. // stripes: x∈[1,2] y∈[3,4]→1, x∈[2,3] y∈[2,4]→2, x∈[3,4] y∈[1,4]→3 → total dominated = 1+2+3 = 6.
let s = space_min2(); let s = space_min2();
let front = [cand(vec![1.0, 3.0]), cand(vec![2.0, 2.0]), cand(vec![3.0, 1.0])]; let front = [
cand(vec![1.0, 3.0]),
cand(vec![2.0, 2.0]),
cand(vec![3.0, 1.0]),
];
let hv = hypervolume_2d(&front, &s, [4.0, 4.0]); let hv = hypervolume_2d(&front, &s, [4.0, 4.0]);
assert!((hv - 6.0).abs() < 1e-12, "expected 6.0, got {hv}"); assert!((hv - 6.0).abs() < 1e-12, "expected 6.0, got {hv}");
} }
@@ -126,3 +128,321 @@ mod tests {
let _ = hypervolume_2d(&front, &s, [10.0, 10.0]); let _ = hypervolume_2d(&front, &s, [10.0, 10.0]);
} }
} }
/// Compute the dominated hypervolume in arbitrary dimensions using the
/// **Hypervolume-by-Slicing-Objectives (HSO)** algorithm of While et al. 2006.
///
/// `objectives.len()` must equal `reference_point.len()`. Like
/// [`hypervolume_2d`], the reference point is interpreted in the same
/// minimization-oriented frame as `ObjectiveSpace::as_minimization`, and
/// points that don't strictly dominate the reference are silently skipped.
///
/// For 2-D problems prefer [`hypervolume_2d`] (it has the same exact result
/// but a tighter sweep loop). This function calls [`hypervolume_2d`]
/// internally as the recursion base case.
///
/// Worst-case complexity is O((N · M)!) which sounds awful but in practice
/// HSO is competitive with WFG up through ~5 objectives at population sizes
/// of 100–200 — i.e. exactly the regime heuropt targets.
///
/// # Panics
/// If `objectives.len() != reference_point.len()`, or if either is zero.
pub fn hypervolume_nd<D>(
front: &[Candidate<D>],
objectives: &ObjectiveSpace,
reference_point: &[f64],
) -> f64 {
assert_eq!(
objectives.len(),
reference_point.len(),
"hypervolume_nd: ObjectiveSpace and reference_point must agree on dimension",
);
assert!(
!reference_point.is_empty(),
"hypervolume_nd: dimension must be >= 1"
);
if front.is_empty() {
return 0.0;
}
// Project each point into minimization-oriented space, then keep only
// points that strictly dominate the reference along every axis.
let oriented: Vec<Vec<f64>> = front
.iter()
.filter_map(|c| {
let m = objectives.as_minimization(&c.evaluation.objectives);
if m.iter().zip(reference_point.iter()).all(|(p, r)| p < r) {
Some(m)
} else {
None
}
})
.collect();
if oriented.is_empty() {
return 0.0;
}
hso_recursive(&oriented, reference_point)
}
fn hso_recursive(points: &[Vec<f64>], reference: &[f64]) -> f64 {
let m = reference.len();
if m == 1 {
// 1-D HV: distance from the best (minimum) point to the reference.
let best = points.iter().map(|p| p[0]).fold(f64::INFINITY, f64::min);
return (reference[0] - best).max(0.0);
}
if m == 2 {
// 2-D HV via the same sweep used by hypervolume_2d. Inlined here
// because we already have the points in oriented form.
let mut sorted: Vec<&Vec<f64>> = points.iter().collect();
sorted.sort_by(|a, b| a[0].partial_cmp(&b[0]).unwrap_or(std::cmp::Ordering::Equal));
let mut area = 0.0;
let mut last_y = reference[1];
for p in sorted {
if p[1] >= last_y {
continue;
}
let width = reference[0] - p[0];
let height = last_y - p[1];
area += width * height;
last_y = p[1];
}
return area;
}
// M ≥ 3: sweep along the last axis from the reference downward,
// peeling off bands. At each band:
// - the active set is "all points whose last-axis value ≤ band_top";
// - its (M-1)-dim HV (on the first M-1 axes against the
// corresponding sub-reference), multiplied by band thickness, is
// the band's HV contribution.
//
// We sort points ascending by the last axis once, then iterate from
// the largest last-axis value downward. The active set at iteration
// `k` is exactly the prefix `sorted[..=k]` — no allocations or
// linear-scan removals needed.
let last = m - 1;
// Index-sort instead of cloning every point's inner vector. The
// recursion stays bit-identical because we still iterate the same
// points in the same order.
let mut order: Vec<usize> = (0..points.len()).collect();
order.sort_by(|&i, &j| {
points[i][last]
.partial_cmp(&points[j][last])
.unwrap_or(std::cmp::Ordering::Equal)
});
// Pre-project once onto the first M-1 axes, in the sorted order.
// The active set at iteration `k` is the prefix `projected[..=k]`,
// so the inner recursion just slices the prefix.
let projected: Vec<Vec<f64>> = order.iter().map(|&i| points[i][..last].to_vec()).collect();
let sub_reference: &[f64] = &reference[..last];
let mut total = 0.0;
let mut prev = reference[last];
for k in (0..order.len()).rev() {
let p_last = points[order[k]][last];
let depth = prev - p_last;
if depth > 0.0 {
let active = &projected[..=k];
// The 2-D base case sweeps in sorted-x order and skips any
// point with `y >= last_y`, which is exactly the dominance
// filter — so for M=3 (sub_reference len 2) we can hand
// `active` straight to `hso_recursive` without paying for
// an O(K²) `non_dominated_projection` first. For M≥4 we
// still need the explicit filter to keep the recursion's
// upper levels honest.
let inner = if sub_reference.len() == 2 {
hso_recursive(active, sub_reference)
} else {
let nd = non_dominated_projection(active);
hso_recursive(&nd, sub_reference)
};
total += depth * inner;
}
prev = p_last;
}
total
}
/// Drop dominated members of a projected point set.
fn non_dominated_projection(points: &[Vec<f64>]) -> Vec<Vec<f64>> {
let m = if let Some(first) = points.first() {
first.len()
} else {
return Vec::new();
};
let mut out: Vec<Vec<f64>> = Vec::new();
'outer: for p in points {
// Skip if dominated by any kept point.
for q in &out {
if dominates(q, p, m) {
continue 'outer;
}
}
// Drop already-kept points that this one dominates.
out.retain(|q| !dominates(p, q, m));
out.push(p.clone());
}
out
}
fn dominates(a: &[f64], b: &[f64], m: usize) -> bool {
let mut strictly_better = false;
for i in 0..m {
if a[i] > b[i] {
return false;
}
if a[i] < b[i] {
strictly_better = true;
}
}
strictly_better
}
/// Convenience wrapper that takes raw `Evaluation`s. Useful inside SMS-EMOA
/// where we want to compute "front HV minus point's contribution."
pub(crate) fn hypervolume_nd_from_evaluations(
evaluations: &[&Evaluation],
objectives: &ObjectiveSpace,
reference_point: &[f64],
) -> f64 {
if evaluations.is_empty() {
return 0.0;
}
let oriented: Vec<Vec<f64>> = evaluations
.iter()
.filter_map(|e| {
let m = objectives.as_minimization(&e.objectives);
if m.iter().zip(reference_point.iter()).all(|(p, r)| p < r) {
Some(m)
} else {
None
}
})
.collect();
if oriented.is_empty() {
return 0.0;
}
hso_recursive(&oriented, reference_point)
}
#[cfg(test)]
mod nd_tests {
use super::*;
use crate::core::evaluation::Evaluation;
use crate::core::objective::Objective;
fn cand_n(obj: Vec<f64>) -> Candidate<()> {
Candidate::new((), Evaluation::new(obj))
}
#[test]
fn nd_matches_2d_on_known_case() {
let s = ObjectiveSpace::new(vec![Objective::minimize("f1"), Objective::minimize("f2")]);
let front = [
cand_n(vec![1.0, 3.0]),
cand_n(vec![2.0, 2.0]),
cand_n(vec![3.0, 1.0]),
];
let hv2 = hypervolume_2d(&front, &s, [4.0, 4.0]);
let hvn = hypervolume_nd(&front, &s, &[4.0, 4.0]);
assert!((hv2 - hvn).abs() < 1e-12, "{hv2} vs {hvn}");
assert!((hvn - 6.0).abs() < 1e-12);
}
#[test]
fn nd_three_d_single_point_at_origin() {
let s = ObjectiveSpace::new(vec![
Objective::minimize("f1"),
Objective::minimize("f2"),
Objective::minimize("f3"),
]);
let front = [cand_n(vec![0.0, 0.0, 0.0])];
// Reference at (1, 1, 1): one point fully dominates the cube
// → HV = 1·1·1 = 1.
let hv = hypervolume_nd(&front, &s, &[1.0, 1.0, 1.0]);
assert!((hv - 1.0).abs() < 1e-12);
}
#[test]
fn nd_three_d_two_points_no_overlap() {
let s = ObjectiveSpace::new(vec![
Objective::minimize("f1"),
Objective::minimize("f2"),
Objective::minimize("f3"),
]);
// Reference (2, 2, 2). Two non-dominated points, projecting cleanly:
// p1 = (0, 1, 1) → contributes a 2 × 1 × 1 = 2 box
// p2 = (1, 0, 1) → contributes 1 × 2 × 1 = 2 minus the overlap with p1
// overlap (where x<=1 AND y<=1 AND z<=1) is 1·1·1 = 1
// p3 = (1, 1, 0) → ... and so on
// Manual computation is annoying; instead verify monotonicity:
// adding more non-dominated points must strictly increase HV.
let front_one = [cand_n(vec![0.0, 1.0, 1.0])];
let front_two = [cand_n(vec![0.0, 1.0, 1.0]), cand_n(vec![1.0, 0.0, 1.0])];
let front_three = [
cand_n(vec![0.0, 1.0, 1.0]),
cand_n(vec![1.0, 0.0, 1.0]),
cand_n(vec![1.0, 1.0, 0.0]),
];
let hv1 = hypervolume_nd(&front_one, &s, &[2.0, 2.0, 2.0]);
let hv2 = hypervolume_nd(&front_two, &s, &[2.0, 2.0, 2.0]);
let hv3 = hypervolume_nd(&front_three, &s, &[2.0, 2.0, 2.0]);
assert!(hv1 < hv2, "{hv1} should be < {hv2}");
assert!(hv2 < hv3, "{hv2} should be < {hv3}");
// Sanity bound: each point is a (2,2,2)-box minus an L-shape;
// total can't exceed the box volume of 8.
assert!(hv3 < 8.0);
}
#[test]
fn nd_empty_is_zero() {
let s = ObjectiveSpace::new(vec![
Objective::minimize("f1"),
Objective::minimize("f2"),
Objective::minimize("f3"),
]);
let front: [Candidate<()>; 0] = [];
assert_eq!(hypervolume_nd(&front, &s, &[1.0, 1.0, 1.0]), 0.0);
}
#[test]
fn nd_skips_points_not_dominating_reference() {
let s = ObjectiveSpace::new(vec![
Objective::minimize("f1"),
Objective::minimize("f2"),
Objective::minimize("f3"),
]);
// (3, 0, 0) is not dominated by reference (1, 1, 1) on axis 0.
let front = [cand_n(vec![3.0, 0.0, 0.0])];
assert_eq!(hypervolume_nd(&front, &s, &[1.0, 1.0, 1.0]), 0.0);
}
#[test]
#[should_panic(expected = "must agree on dimension")]
fn nd_panics_on_dim_mismatch() {
let s = ObjectiveSpace::new(vec![Objective::minimize("f1"), Objective::minimize("f2")]);
let front = [cand_n(vec![1.0, 1.0])];
let _ = hypervolume_nd(&front, &s, &[1.0, 1.0, 1.0]);
}
/// Sanity test: dominated points shouldn't increase HV.
#[test]
fn nd_dominated_points_dont_increase_hv() {
let s = ObjectiveSpace::new(vec![
Objective::minimize("f1"),
Objective::minimize("f2"),
Objective::minimize("f3"),
]);
let base = vec![cand_n(vec![0.0, 1.0, 1.0]), cand_n(vec![1.0, 0.0, 1.0])];
// Add a dominated point — HV should be unchanged.
let mut with_dominated = base.clone();
with_dominated.push(cand_n(vec![1.5, 1.5, 1.5]));
let hv_base = hypervolume_nd(&base, &s, &[2.0, 2.0, 2.0]);
let hv_with = hypervolume_nd(&with_dominated, &s, &[2.0, 2.0, 2.0]);
assert!((hv_base - hv_with).abs() < 1e-12, "{hv_base} vs {hv_with}");
}
}
src/metrics/spacing.rs
+2 -6
View File
@@ -39,8 +39,7 @@ pub fn spacing<D>(front: &[Candidate<D>], objectives: &ObjectiveSpace) -> f64 {
} }
let mean = nearest.iter().sum::<f64>() / n as f64; let mean = nearest.iter().sum::<f64>() / n as f64;
let variance = let variance = nearest.iter().map(|d| (d - mean).powi(2)).sum::<f64>() / n as f64;
nearest.iter().map(|d| (d - mean).powi(2)).sum::<f64>() / n as f64;
variance.sqrt() variance.sqrt()
} }
@@ -55,10 +54,7 @@ mod tests {
} }
fn space_min2() -> ObjectiveSpace { fn space_min2() -> ObjectiveSpace {
ObjectiveSpace::new(vec![ ObjectiveSpace::new(vec![Objective::minimize("f1"), Objective::minimize("f2")])
Objective::minimize("f1"),
Objective::minimize("f2"),
])
} }
#[test] #[test]
src/operators/mod.rs
+2
View File
@@ -4,8 +4,10 @@ pub mod binary;
pub mod composite; pub mod composite;
pub mod permutation; pub mod permutation;
pub mod real; pub mod real;
pub mod repair;
pub use binary::*; pub use binary::*;
pub use composite::*; pub use composite::*;
pub use permutation::*; pub use permutation::*;
pub use real::*; pub use real::*;
pub use repair::*;
src/operators/real.rs
+170 -8
View File
@@ -38,7 +38,11 @@ impl Initializer<Vec<f64>> for RealBounds {
for _ in 0..size { for _ in 0..size {
let mut decision = Vec::with_capacity(self.bounds.len()); let mut decision = Vec::with_capacity(self.bounds.len());
for &(lo, hi) in &self.bounds { for &(lo, hi) in &self.bounds {
let v = if lo == hi { lo } else { rng.random_range(lo..=hi) }; let v = if lo == hi {
lo
} else {
rng.random_range(lo..=hi)
};
decision.push(v); decision.push(v);
} }
out.push(decision); out.push(decision);
@@ -63,8 +67,7 @@ impl Variation<Vec<f64>> for GaussianMutation {
!parents.is_empty(), !parents.is_empty(),
"GaussianMutation requires at least one parent", "GaussianMutation requires at least one parent",
); );
let normal = let normal = Normal::new(0.0, self.sigma).expect("Normal distribution rejected sigma");
Normal::new(0.0, self.sigma).expect("Normal distribution rejected sigma");
let mut child = parents[0].clone(); let mut child = parents[0].clone();
for x in child.iter_mut() { for x in child.iter_mut() {
*x += normal.sample(rng); *x += normal.sample(rng);
@@ -113,7 +116,11 @@ impl SimulatedBinaryCrossover {
(0.0..=1.0).contains(&per_variable_probability), (0.0..=1.0).contains(&per_variable_probability),
"SimulatedBinaryCrossover per_variable_probability must be in [0.0, 1.0]", "SimulatedBinaryCrossover per_variable_probability must be in [0.0, 1.0]",
); );
Self { bounds, eta, per_variable_probability } Self {
bounds,
eta,
per_variable_probability,
}
} }
} }
@@ -201,7 +208,11 @@ impl PolynomialMutation {
(0.0..=1.0).contains(&per_variable_probability), (0.0..=1.0).contains(&per_variable_probability),
"PolynomialMutation per_variable_probability must be in [0.0, 1.0]", "PolynomialMutation per_variable_probability must be in [0.0, 1.0]",
); );
Self { bounds, eta, per_variable_probability } Self {
bounds,
eta,
per_variable_probability,
}
} }
} }
@@ -257,7 +268,10 @@ impl BoundedGaussianMutation {
/// # Panics /// # Panics
/// If `sigma <= 0.0` or any bound has `lo > hi`. /// If `sigma <= 0.0` or any bound has `lo > hi`.
pub fn new(sigma: f64, bounds: Vec<(f64, f64)>) -> Self { pub fn new(sigma: f64, bounds: Vec<(f64, f64)>) -> Self {
assert!(sigma > 0.0, "BoundedGaussianMutation sigma must be positive"); assert!(
sigma > 0.0,
"BoundedGaussianMutation sigma must be positive"
);
for (i, &(lo, hi)) in bounds.iter().enumerate() { for (i, &(lo, hi)) in bounds.iter().enumerate() {
assert!( assert!(
lo <= hi, lo <= hi,
@@ -279,8 +293,7 @@ impl Variation<Vec<f64>> for BoundedGaussianMutation {
self.bounds.len(), self.bounds.len(),
"BoundedGaussianMutation parent length must match bounds length", "BoundedGaussianMutation parent length must match bounds length",
); );
let normal = let normal = Normal::new(0.0, self.sigma).expect("Normal distribution rejected sigma");
Normal::new(0.0, self.sigma).expect("Normal distribution rejected sigma");
let mut child = parents[0].clone(); let mut child = parents[0].clone();
for (x, &(lo, hi)) in child.iter_mut().zip(self.bounds.iter()) { for (x, &(lo, hi)) in child.iter_mut().zip(self.bounds.iter()) {
*x = (*x + normal.sample(rng)).clamp(lo, hi); *x = (*x + normal.sample(rng)).clamp(lo, hi);
@@ -289,6 +302,119 @@ impl Variation<Vec<f64>> for BoundedGaussianMutation {
} }
} }
/// Heavy-tailed Lévy-flight mutation for `Vec<f64>` decisions.
///
/// Adds a Lévy(α)-distributed step to every variable, optionally clamped to
/// per-variable bounds. Compared with `GaussianMutation`, the Lévy
/// distribution has a heavy tail — most steps are small and local but
/// occasional steps are very large, giving a single mutation operator
/// that does both refinement and exploration. This is the kernel that
/// powers Cuckoo Search and other Lévy-flight metaheuristics.
///
/// Implementation: Mantegna's algorithm combines two Gaussians to
/// produce a Lévy(α) sample. `alpha` is the tail exponent in `(0, 2]`;
/// typical value is `1.5`. `1.0` gives the Cauchy distribution (very
/// heavy); `2.0` collapses to the Normal.
#[derive(Debug, Clone)]
pub struct LevyMutation {
/// Tail exponent `α ∈ (0, 2]`. Smaller = heavier tail.
pub alpha: f64,
/// Step scale.
pub scale: f64,
/// Optional per-variable bounds. Empty `Vec` → no clamping.
pub bounds: Vec<(f64, f64)>,
}
impl LevyMutation {
/// Construct a `LevyMutation`.
///
/// # Panics
/// If `alpha` is not in `(0, 2]`, `scale <= 0.0`, or any bound has
/// `lo > hi`.
pub fn new(alpha: f64, scale: f64, bounds: Vec<(f64, f64)>) -> Self {
assert!(
alpha > 0.0 && alpha <= 2.0,
"LevyMutation alpha must be in (0, 2]",
);
assert!(scale > 0.0, "LevyMutation scale must be > 0");
for (i, &(lo, hi)) in bounds.iter().enumerate() {
assert!(
lo <= hi,
"LevyMutation bound at index {i} has lo > hi: ({lo}, {hi})",
);
}
Self {
alpha,
scale,
bounds,
}
}
}
impl Variation<Vec<f64>> for LevyMutation {
fn vary(&mut self, parents: &[Vec<f64>], rng: &mut Rng) -> Vec<Vec<f64>> {
assert!(
!parents.is_empty(),
"LevyMutation requires at least one parent"
);
let alpha = self.alpha;
// Mantegna's algorithm σ for the numerator Normal:
// sigma_u = (Γ(1+α)·sin(π·α/2) / (Γ((1+α)/2)·α·2^((α-1)/2)))^(1/α)
// Denominator Normal has σ = 1.
let sigma_u = mantegna_sigma_u(alpha);
let normal_u = Normal::new(0.0, sigma_u).expect("Normal::new(0, sigma_u)");
let normal_v = Normal::new(0.0, 1.0).expect("Normal::new(0, 1)");
let mut child = parents[0].clone();
for (j, x) in child.iter_mut().enumerate() {
let u: f64 = normal_u.sample(rng);
let v: f64 = normal_v.sample(rng);
let step = u / v.abs().powf(1.0 / alpha);
*x += self.scale * step;
if let Some(&(lo, hi)) = self.bounds.get(j) {
*x = x.clamp(lo, hi);
}
}
vec![child]
}
}
fn mantegna_sigma_u(alpha: f64) -> f64 {
// Γ-related constants. We compute Γ(z) via libm if the std::f64::gamma
// isn't available; fall back to a small Lanczos approximation.
fn gamma(z: f64) -> f64 {
// Stirling-ish via the standard recursion + Lanczos coefficients.
// For the typical α ∈ [1, 2] range we hit, the expressions Γ(1+α)
// and Γ((1+α)/2) are well-behaved.
// Lanczos coefficients for g = 7 (truncated to f64 precision).
let g = 7.0;
let p = [
0.999_999_999_999_81,
676.520_368_121_885,
-1_259.139_216_722_402,
771.323_428_777_653,
-176.615_029_162_141,
12.507_343_278_686_905,
-0.138_571_095_265_720_1,
9.984_369_578_019_572e-6,
1.505_632_735_149_311_6e-7,
];
if z < 0.5 {
std::f64::consts::PI / ((std::f64::consts::PI * z).sin() * gamma(1.0 - z))
} else {
let z = z - 1.0;
let mut x = p[0];
for (i, &pi) in p.iter().enumerate().skip(1) {
x += pi / (z + i as f64);
}
let t = z + g + 0.5;
(2.0 * std::f64::consts::PI).sqrt() * t.powf(z + 0.5) * (-t).exp() * x
}
}
let num = gamma(1.0 + alpha) * (std::f64::consts::PI * alpha / 2.0).sin();
let den = gamma((1.0 + alpha) / 2.0) * alpha * 2.0_f64.powf((alpha - 1.0) / 2.0);
(num / den).powf(1.0 / alpha)
}
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
@@ -426,6 +552,42 @@ mod tests {
let _ = SimulatedBinaryCrossover::new(vec![(0.0, 1.0)], -1.0, 0.5); let _ = SimulatedBinaryCrossover::new(vec![(0.0, 1.0)], -1.0, 0.5);
} }
#[test]
fn levy_mutation_returns_one_child_in_bounds() {
let mut m = LevyMutation::new(1.5, 0.1, vec![(-1.0, 1.0); 4]);
let mut rng = rng_from_seed(42);
let parent = vec![0.0_f64; 4];
for _ in 0..50 {
let children = m.vary(std::slice::from_ref(&parent), &mut rng);
assert_eq!(children.len(), 1);
assert_eq!(children[0].len(), 4);
for &x in &children[0] {
assert!((-1.0..=1.0).contains(&x), "out of bounds: {x}");
}
}
}
#[test]
fn levy_mutation_unbounded_works() {
let mut m = LevyMutation::new(1.5, 0.5, Vec::new());
let mut rng = rng_from_seed(0);
let parent = vec![0.0_f64; 3];
let children = m.vary(std::slice::from_ref(&parent), &mut rng);
assert_eq!(children[0].len(), 3);
}
#[test]
#[should_panic(expected = "alpha must be in (0, 2]")]
fn levy_alpha_out_of_range_panics() {
let _ = LevyMutation::new(0.0, 0.1, Vec::new());
}
#[test]
#[should_panic(expected = "scale must be > 0")]
fn levy_zero_scale_panics() {
let _ = LevyMutation::new(1.5, 0.0, Vec::new());
}
#[test] #[test]
fn polynomial_mutation_keeps_child_in_bounds() { fn polynomial_mutation_keeps_child_in_bounds() {
let mut m = PolynomialMutation::new(vec![(-1.0, 1.0); 5], 5.0, 1.0); let mut m = PolynomialMutation::new(vec![(-1.0, 1.0); 5], 5.0, 1.0);
src/operators/repair.rs
+228
View File
@@ -0,0 +1,228 @@
//! Repair operators: in-place projections that restore decisions to
//! feasibility.
use crate::traits::Repair;
/// Clamp every variable of a `Vec<f64>` to per-axis inclusive bounds.
///
/// The simplest possible repair — pair with `GaussianMutation` (which
/// doesn't enforce bounds in v1) to produce a bounds-respecting variant
/// without writing a custom Variation impl.
#[derive(Debug, Clone)]
pub struct ClampToBounds {
/// Per-variable inclusive bounds.
pub bounds: Vec<(f64, f64)>,
}
impl ClampToBounds {
/// Construct a `ClampToBounds`.
///
/// # Panics
/// If any `(lo, hi)` has `lo > hi`.
pub fn new(bounds: Vec<(f64, f64)>) -> Self {
for (i, &(lo, hi)) in bounds.iter().enumerate() {
assert!(
lo <= hi,
"ClampToBounds bound at index {i} has lo > hi: ({lo}, {hi})",
);
}
Self { bounds }
}
}
impl Repair<Vec<f64>> for ClampToBounds {
fn repair(&mut self, decision: &mut Vec<f64>) {
for (j, x) in decision.iter_mut().enumerate() {
if let Some(&(lo, hi)) = self.bounds.get(j) {
*x = x.clamp(lo, hi);
}
}
}
}
/// Project a `Vec<f64>` onto the simplex `{ x : x ≥ 0, Σ x = total }`.
///
/// Implements the standard O(n log n) projection algorithm of Wang & Carreira-
/// Perpiñán 2013. Useful for portfolio-style problems where the
/// decision must sum to a budget, and for normalizing reference
/// directions onto the unit simplex.
#[derive(Debug, Clone)]
pub struct ProjectToSimplex {
/// Target sum (the simplex's "size"). Standard probability simplex
/// uses `total = 1.0`.
pub total: f64,
}
impl ProjectToSimplex {
/// Construct a `ProjectToSimplex`.
///
/// # Panics
/// If `total <= 0.0`.
pub fn new(total: f64) -> Self {
assert!(total > 0.0, "ProjectToSimplex total must be > 0");
Self { total }
}
}
impl Repair<Vec<f64>> for ProjectToSimplex {
fn repair(&mut self, decision: &mut Vec<f64>) {
let n = decision.len();
if n == 0 {
return;
}
// If any |x_i| dwarfs `total` so badly that `x_i - total == x_i` in
// f64, the standard Duchi/Held-Wolfe projection loses all precision
// in τ and silently returns the all-zero vector. In that pathological
// regime the projection is effectively concentrated on argmax(x), so
// assign all mass there directly.
let max_abs = decision
.iter()
.copied()
.fold(0.0_f64, |a, b| a.max(b.abs()));
if max_abs > self.total * 1e15 {
let mut argmax = 0;
for (i, &v) in decision.iter().enumerate().skip(1) {
if v > decision[argmax] {
argmax = i;
}
}
for (i, x) in decision.iter_mut().enumerate() {
*x = if i == argmax { self.total } else { 0.0 };
}
return;
}
// Sort copy descending.
let mut sorted: Vec<f64> = decision.clone();
sorted.sort_by(|a, b| b.partial_cmp(a).unwrap_or(std::cmp::Ordering::Equal));
// Find ρ = max{ j : sorted[j-1] - (Σ_{i<=j} sorted[i] - total) / j > 0 }.
// Mathematically the j=0 case always satisfies the condition (since
// total > 0), so we initialize from it before the loop — that guards
// against floating-point precision loss when |sorted[0]| ≫ total,
// where the subtraction `sorted[0] - tau` could otherwise round to
// zero and leave τ unset (yielding the all-zero output bug).
let mut cumsum = 0.0;
let mut tau_at_rho = sorted[0] - self.total;
for (j, &val) in sorted.iter().enumerate() {
cumsum += val;
let tau = (cumsum - self.total) / (j as f64 + 1.0);
if val - tau > 0.0 {
tau_at_rho = tau;
}
}
// Apply: x_i ← max(x_i - τ, 0).
for x in decision.iter_mut() {
*x = (*x - tau_at_rho).max(0.0);
}
}
}
#[cfg(test)]
mod tests {
use super::*;
fn approx_eq(a: f64, b: f64, tol: f64) -> bool {
(a - b).abs() < tol
}
#[test]
fn clamp_to_bounds_clips() {
let mut r = ClampToBounds::new(vec![(-1.0, 1.0); 3]);
let mut x = vec![-2.5, 0.5, 5.0];
r.repair(&mut x);
assert_eq!(x, vec![-1.0, 0.5, 1.0]);
}
#[test]
fn clamp_passthrough_when_already_in_bounds() {
let mut r = ClampToBounds::new(vec![(-1.0, 1.0); 3]);
let mut x = vec![-0.3, 0.0, 0.7];
let original = x.clone();
r.repair(&mut x);
assert_eq!(x, original);
}
#[test]
#[should_panic(expected = "lo > hi")]
fn clamp_invalid_bounds_panics() {
let _ = ClampToBounds::new(vec![(1.0, -1.0)]);
}
#[test]
fn project_to_unit_simplex_sums_to_total() {
let mut r = ProjectToSimplex::new(1.0);
let mut x = vec![0.5, 0.3, 0.2, -0.5];
r.repair(&mut x);
let s: f64 = x.iter().sum();
assert!(approx_eq(s, 1.0, 1e-12));
for &v in &x {
assert!(v >= 0.0);
}
}
#[test]
fn project_already_on_simplex_unchanged() {
let mut r = ProjectToSimplex::new(1.0);
let mut x = vec![0.5, 0.3, 0.2];
r.repair(&mut x);
let s: f64 = x.iter().sum();
assert!(approx_eq(s, 1.0, 1e-12));
// Within tolerance, the values should be roughly preserved (no
// clipping needed).
assert!(approx_eq(x[0], 0.5, 1e-12));
assert!(approx_eq(x[1], 0.3, 1e-12));
assert!(approx_eq(x[2], 0.2, 1e-12));
}
#[test]
fn project_arbitrary_total() {
let mut r = ProjectToSimplex::new(10.0);
let mut x = vec![100.0, 50.0, -20.0, 30.0];
r.repair(&mut x);
let s: f64 = x.iter().sum();
assert!(approx_eq(s, 10.0, 1e-9));
for &v in &x {
assert!(v >= 0.0);
}
}
#[test]
#[should_panic(expected = "total must be > 0")]
fn project_non_positive_total_panics() {
let _ = ProjectToSimplex::new(0.0);
}
/// Regression: discovered by the `clamp_to_bounds` fuzzer. When
/// `|max(x)|` dwarfs `total` so badly that the subtraction `x - τ`
/// rounds away `total`, the standard algorithm previously returned
/// the all-zero vector. The degenerate-magnitude fallback now
/// concentrates all mass on argmax(x).
#[test]
fn project_extreme_magnitudes_concentrates_on_argmax() {
let mut r = ProjectToSimplex::new(1.0);
let mut x = vec![1e20, 5e19, -1e20];
r.repair(&mut x);
let s: f64 = x.iter().sum();
assert!(approx_eq(s, 1.0, 1e-12));
// Argmax is index 0; all mass should be there.
assert!(approx_eq(x[0], 1.0, 1e-12));
assert_eq!(x[1], 0.0);
assert_eq!(x[2], 0.0);
}
/// Regression: when the input is "all zeros", τ is small (0 - total),
/// the projection should distribute total evenly. This tests the
/// loop's handling of equal entries.
#[test]
fn project_all_zeros_distributes_evenly() {
let mut r = ProjectToSimplex::new(1.0);
let mut x = vec![0.0, 0.0, 0.0, 0.0];
r.repair(&mut x);
let s: f64 = x.iter().sum();
assert!(approx_eq(s, 1.0, 1e-12));
for &v in &x {
assert!(approx_eq(v, 0.25, 1e-12));
}
}
}
src/pareto/archive.rs
+120 -15
View File
@@ -2,7 +2,6 @@
use crate::core::candidate::Candidate; use crate::core::candidate::Candidate;
use crate::core::objective::ObjectiveSpace; use crate::core::objective::ObjectiveSpace;
use crate::pareto::dominance::{Dominance, pareto_compare};
/// A growable, dominance-pruned archive of candidates. /// A growable, dominance-pruned archive of candidates.
/// ///
@@ -21,7 +20,10 @@ pub struct ParetoArchive<D> {
impl<D: Clone> ParetoArchive<D> { impl<D: Clone> ParetoArchive<D> {
/// Build an empty archive against the given objective space. /// Build an empty archive against the given objective space.
pub fn new(objectives: ObjectiveSpace) -> Self { pub fn new(objectives: ObjectiveSpace) -> Self {
Self { members: Vec::new(), objectives } Self {
members: Vec::new(),
objectives,
}
} }
/// Insert a candidate, preserving the non-domination property. /// Insert a candidate, preserving the non-domination property.
@@ -30,19 +32,63 @@ impl<D: Clone> ParetoArchive<D> {
/// - Otherwise, drop existing members that the new candidate dominates, /// - Otherwise, drop existing members that the new candidate dominates,
/// then keep the new candidate. /// then keep the new candidate.
pub fn insert(&mut self, candidate: Candidate<D>) { pub fn insert(&mut self, candidate: Candidate<D>) {
for m in &self.members { // The naïve formulation calls `pareto_compare` twice per member
if matches!( // (once each pass), and `pareto_compare` re-allocates two
pareto_compare(&candidate.evaluation, &m.evaluation, &self.objectives), // Vec<f64>s via `as_minimization` per call → 4N allocations per
Dominance::DominatedBy | Dominance::Equal // insert. Cache the candidate's oriented + feasibility once, and
// each member's oriented once, then inline the dominance checks.
let n = self.members.len();
let m_dim = self.objectives.len();
let cand_oriented = self
.objectives
.as_minimization(&candidate.evaluation.objectives);
let cand_feasible = candidate.evaluation.is_feasible();
let cand_violation = candidate.evaluation.constraint_violation;
let member_oriented: Vec<Vec<f64>> = self
.members
.iter()
.map(|c| self.objectives.as_minimization(&c.evaluation.objectives))
.collect();
// First pass: bail if any existing member dominates-or-equals
// the candidate.
#[allow(clippy::needless_range_loop)]
for i in 0..n {
let m_eval = &self.members[i].evaluation;
if member_dominates_or_equals(
&member_oriented[i],
m_eval.is_feasible(),
m_eval.constraint_violation,
&cand_oriented,
cand_feasible,
cand_violation,
m_dim,
) { ) {
return; return;
} }
} }
self.members.retain(|m| {
!matches!( // Second pass: drop existing members the candidate dominates.
pareto_compare(&candidate.evaluation, &m.evaluation, &self.objectives), let mut keep_mask = Vec::with_capacity(n);
Dominance::Dominates #[allow(clippy::needless_range_loop)]
) for i in 0..n {
let m_eval = &self.members[i].evaluation;
let cand_dominates_member = candidate_dominates_member(
&cand_oriented,
cand_feasible,
cand_violation,
&member_oriented[i],
m_eval.is_feasible(),
m_eval.constraint_violation,
m_dim,
);
keep_mask.push(!cand_dominates_member);
}
let mut idx = 0;
self.members.retain(|_| {
let keep = keep_mask[idx];
idx += 1;
keep
}); });
self.members.push(candidate); self.members.push(candidate);
} }
@@ -78,6 +124,68 @@ impl<D: Clone> ParetoArchive<D> {
} }
} }
/// Inline `pareto_compare(member, candidate, objectives) ∈ {Dominates, Equal}`
/// against the cached oriented + feasibility/violation values, returning the
/// boolean directly.
#[inline]
fn member_dominates_or_equals(
m_oriented: &[f64],
m_feasible: bool,
m_violation: f64,
c_oriented: &[f64],
c_feasible: bool,
c_violation: f64,
m_dim: usize,
) -> bool {
match (m_feasible, c_feasible) {
(true, false) => true,
(false, true) => false,
(false, false) => m_violation <= c_violation,
(true, true) => {
let mut c_better = false;
for k in 0..m_dim {
if c_oriented[k] < m_oriented[k] {
c_better = true;
break;
}
}
!c_better
}
}
}
/// Inline `pareto_compare(candidate, member, objectives) == Dominates`.
#[inline]
fn candidate_dominates_member(
c_oriented: &[f64],
c_feasible: bool,
c_violation: f64,
m_oriented: &[f64],
m_feasible: bool,
m_violation: f64,
m_dim: usize,
) -> bool {
match (c_feasible, m_feasible) {
(true, false) => true,
(false, true) => false,
(false, false) => c_violation < m_violation,
(true, true) => {
let mut c_better_anywhere = false;
let mut m_better_anywhere = false;
for k in 0..m_dim {
let cv = c_oriented[k];
let mv = m_oriented[k];
if cv < mv {
c_better_anywhere = true;
} else if cv > mv {
m_better_anywhere = true;
}
}
c_better_anywhere && !m_better_anywhere
}
}
}
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
@@ -85,10 +193,7 @@ mod tests {
use crate::core::objective::Objective; use crate::core::objective::Objective;
fn space_min2() -> ObjectiveSpace { fn space_min2() -> ObjectiveSpace {
ObjectiveSpace::new(vec![ ObjectiveSpace::new(vec![Objective::minimize("f1"), Objective::minimize("f2")])
Objective::minimize("f1"),
Objective::minimize("f2"),
])
} }
fn cand(decision: u32, obj: Vec<f64>) -> Candidate<u32> { fn cand(decision: u32, obj: Vec<f64>) -> Candidate<u32> {
src/pareto/crowding.rs
+1 -4
View File
@@ -77,10 +77,7 @@ mod tests {
} }
fn space_min2() -> ObjectiveSpace { fn space_min2() -> ObjectiveSpace {
ObjectiveSpace::new(vec![ ObjectiveSpace::new(vec![Objective::minimize("f1"), Objective::minimize("f2")])
Objective::minimize("f1"),
Objective::minimize("f2"),
])
} }
#[test] #[test]
src/pareto/dominance.rs
+2 -9
View File
@@ -29,11 +29,7 @@ pub enum Dominance {
/// `constraint_violation` dominates. /// `constraint_violation` dominates.
/// 3. Otherwise compare objective values after converting both to /// 3. Otherwise compare objective values after converting both to
/// minimization orientation via [`ObjectiveSpace::as_minimization`]. /// minimization orientation via [`ObjectiveSpace::as_minimization`].
pub fn pareto_compare( pub fn pareto_compare(a: &Evaluation, b: &Evaluation, objectives: &ObjectiveSpace) -> Dominance {
a: &Evaluation,
b: &Evaluation,
objectives: &ObjectiveSpace,
) -> Dominance {
let a_feasible = a.is_feasible(); let a_feasible = a.is_feasible();
let b_feasible = b.is_feasible(); let b_feasible = b.is_feasible();
match (a_feasible, b_feasible) { match (a_feasible, b_feasible) {
@@ -78,10 +74,7 @@ mod tests {
use crate::core::objective::Objective; use crate::core::objective::Objective;
fn space_min2() -> ObjectiveSpace { fn space_min2() -> ObjectiveSpace {
ObjectiveSpace::new(vec![ ObjectiveSpace::new(vec![Objective::minimize("f1"), Objective::minimize("f2")])
Objective::minimize("f1"),
Objective::minimize("f2"),
])
} }
#[test] #[test]
src/pareto/front.rs
+1 -4
View File
@@ -69,10 +69,7 @@ mod tests {
} }
fn space_min2() -> ObjectiveSpace { fn space_min2() -> ObjectiveSpace {
ObjectiveSpace::new(vec![ ObjectiveSpace::new(vec![Objective::minimize("f1"), Objective::minimize("f2")])
Objective::minimize("f1"),
Objective::minimize("f2"),
])
} }
#[test] #[test]
src/pareto/reference_points.rs
+11 -2
View File
@@ -11,7 +11,10 @@
/// # Panics /// # Panics
/// If `num_objectives == 0`. /// If `num_objectives == 0`.
pub fn das_dennis(num_objectives: usize, divisions: usize) -> Vec<Vec<f64>> { pub fn das_dennis(num_objectives: usize, divisions: usize) -> Vec<Vec<f64>> {
assert!(num_objectives > 0, "das_dennis requires num_objectives >= 1"); assert!(
num_objectives > 0,
"das_dennis requires num_objectives >= 1"
);
let mut out = Vec::new(); let mut out = Vec::new();
let mut current = Vec::with_capacity(num_objectives); let mut current = Vec::with_capacity(num_objectives);
recurse(num_objectives, divisions, divisions, &mut current, &mut out); recurse(num_objectives, divisions, divisions, &mut current, &mut out);
@@ -34,7 +37,13 @@ fn recurse(
} }
for take in 0..=remaining_units { for take in 0..=remaining_units {
current.push(take); current.push(take);
recurse(remaining_axes - 1, remaining_units - take, total, current, out); recurse(
remaining_axes - 1,
remaining_units - take,
total,
current,
out,
);
current.pop(); current.pop();
} }
} }
src/pareto/sort.rs
+100 -13
View File
@@ -2,7 +2,6 @@
use crate::core::candidate::Candidate; use crate::core::candidate::Candidate;
use crate::core::objective::ObjectiveSpace; use crate::core::objective::ObjectiveSpace;
use crate::pareto::dominance::{Dominance, pareto_compare};
/// Partition the population into Pareto fronts by dominance rank. /// Partition the population into Pareto fronts by dominance rank.
/// ///
@@ -19,24 +18,79 @@ pub fn non_dominated_sort<D>(
return Vec::new(); return Vec::new();
} }
// Precompute the per-individual feasibility, violation, and
// minimization-oriented objective vectors. The naïve formulation
// calls `pareto_compare` (and therefore `as_minimization`) twice for
// every pair, allocating two fresh Vec<f64>s per call; doing it once
// up front cuts that to one allocation per individual.
let feasible: Vec<bool> = population
.iter()
.map(|c| c.evaluation.is_feasible())
.collect();
let violation: Vec<f64> = population
.iter()
.map(|c| c.evaluation.constraint_violation)
.collect();
let oriented: Vec<Vec<f64>> = population
.iter()
.map(|c| objectives.as_minimization(&c.evaluation.objectives))
.collect();
let m = objectives.len();
let mut dominates: Vec<Vec<usize>> = vec![Vec::new(); n]; let mut dominates: Vec<Vec<usize>> = vec![Vec::new(); n];
let mut dominated_by_count: Vec<usize> = vec![0; n]; let mut dominated_by_count: Vec<usize> = vec![0; n];
let mut fronts: Vec<Vec<usize>> = Vec::new(); let mut fronts: Vec<Vec<usize>> = Vec::new();
let mut first_front: Vec<usize> = Vec::new(); let mut first_front: Vec<usize> = Vec::new();
for i in 0..n { for i in 0..n {
let ai_feasible = feasible[i];
let ai_violation = violation[i];
let ai = &oriented[i];
for j in 0..n { for j in 0..n {
if i == j { if i == j {
continue; continue;
} }
match pareto_compare( let bi_feasible = feasible[j];
&population[i].evaluation, let bi_violation = violation[j];
&population[j].evaluation, // Inline the body of `pareto_compare`. We only care about
objectives, // `Dominates` vs `DominatedBy`; `Equal` and `NonDominated`
) { // are no-ops here.
Dominance::Dominates => dominates[i].push(j), let dominates_outcome = match (ai_feasible, bi_feasible) {
Dominance::DominatedBy => dominated_by_count[i] += 1, (true, false) => Some(true), // i dominates j
_ => {} (false, true) => Some(false), // i is dominated
(false, false) => {
if ai_violation < bi_violation {
Some(true)
} else if ai_violation > bi_violation {
Some(false)
} else {
None
}
}
(true, true) => {
let bj = &oriented[j];
let mut a_better_anywhere = false;
let mut b_better_anywhere = false;
for k in 0..m {
let av = ai[k];
let bv = bj[k];
if av < bv {
a_better_anywhere = true;
} else if av > bv {
b_better_anywhere = true;
}
}
match (a_better_anywhere, b_better_anywhere) {
(true, false) => Some(true),
(false, true) => Some(false),
_ => None,
}
}
};
match dominates_outcome {
Some(true) => dominates[i].push(j),
Some(false) => dominated_by_count[i] += 1,
None => {}
} }
} }
if dominated_by_count[i] == 0 { if dominated_by_count[i] == 0 {
@@ -46,6 +100,10 @@ pub fn non_dominated_sort<D>(
fronts.push(first_front); fronts.push(first_front);
let mut k = 0; let mut k = 0;
let mut assigned = vec![false; n];
for &i in &fronts[0] {
assigned[i] = true;
}
while k < fronts.len() && !fronts[k].is_empty() { while k < fronts.len() && !fronts[k].is_empty() {
let mut next: Vec<usize> = Vec::new(); let mut next: Vec<usize> = Vec::new();
// Borrow-friendly: collect dominated indices for the current front first. // Borrow-friendly: collect dominated indices for the current front first.
@@ -55,6 +113,7 @@ pub fn non_dominated_sort<D>(
dominated_by_count[j] -= 1; dominated_by_count[j] -= 1;
if dominated_by_count[j] == 0 { if dominated_by_count[j] == 0 {
next.push(j); next.push(j);
assigned[j] = true;
} }
} }
} }
@@ -65,6 +124,15 @@ pub fn non_dominated_sort<D>(
k += 1; k += 1;
} }
// Any indices still unassigned correspond to dominance-graph cycles
// (which can arise when objectives or constraint violations contain
// NaN — `pareto_compare` becomes intransitive). Place them all in a
// final residual front so the partition invariant holds.
let residual: Vec<usize> = (0..n).filter(|&i| !assigned[i]).collect();
if !residual.is_empty() {
fronts.push(residual);
}
fronts fronts
} }
@@ -79,10 +147,7 @@ mod tests {
} }
fn space_min2() -> ObjectiveSpace { fn space_min2() -> ObjectiveSpace {
ObjectiveSpace::new(vec![ ObjectiveSpace::new(vec![Objective::minimize("f1"), Objective::minimize("f2")])
Objective::minimize("f1"),
Objective::minimize("f2"),
])
} }
#[test] #[test]
@@ -92,6 +157,28 @@ mod tests {
assert!(fronts.is_empty()); assert!(fronts.is_empty());
} }
/// Regression: discovered by the `non_dominated_sort` fuzzer. NaN
/// objectives make `pareto_compare` intransitive, which can leave a
/// cycle in the dominance graph where no node has zero in-degree.
/// Previously the algorithm dropped those indices silently; now they
/// land in a final residual front so the partition invariant holds.
#[test]
fn nan_objective_cycle_indices_partitioned_into_residual_front() {
let s = space_min2();
// Three points whose pairwise comparisons form a 3-cycle under NaN
// intransitivity (the original fuzz-found case had 5 points; this
// 3-point case is the minimal reproduction).
let pop = [
cand(vec![f64::NAN, 1.0]),
cand(vec![1.0, f64::NAN]),
cand(vec![f64::NAN, f64::NAN]),
];
let fronts = non_dominated_sort(&pop, &s);
let mut all_indices: Vec<usize> = fronts.iter().flatten().copied().collect();
all_indices.sort();
assert_eq!(all_indices, vec![0, 1, 2]);
}
#[test] #[test]
fn known_population_yields_expected_fronts() { fn known_population_yields_expected_fronts() {
let s = space_min2(); let s = space_min2();
src/prelude.rs
+17 -9
View File
@@ -6,23 +6,31 @@
pub use crate::core::{ pub use crate::core::{
Candidate, Direction, Evaluation, Objective, ObjectiveSpace, OptimizationResult, Candidate, Direction, Evaluation, Objective, ObjectiveSpace, OptimizationResult,
Population, Problem, Rng, rng_from_seed, PartialProblem, Population, Problem, Rng, rng_from_seed,
}; };
pub use crate::traits::{Initializer, Optimizer, Variation}; pub use crate::traits::{Initializer, Optimizer, Repair, Variation};
pub use crate::pareto::{ pub use crate::pareto::{
Dominance, ParetoArchive, best_candidate, crowding_distance, das_dennis, Dominance, ParetoArchive, best_candidate, crowding_distance, das_dennis, non_dominated_sort,
non_dominated_sort, pareto_compare, pareto_front, pareto_compare, pareto_front,
}; };
pub use crate::operators::{ pub use crate::operators::{
BitFlipMutation, BoundedGaussianMutation, CompositeVariation, GaussianMutation, BitFlipMutation, BoundedGaussianMutation, ClampToBounds, CompositeVariation, GaussianMutation,
PolynomialMutation, RealBounds, SimulatedBinaryCrossover, SwapMutation, LevyMutation, PolynomialMutation, ProjectToSimplex, RealBounds, SimulatedBinaryCrossover,
SwapMutation,
}; };
pub use crate::algorithms::{ pub use crate::algorithms::{
DifferentialEvolution, DifferentialEvolutionConfig, Moead, MoeadConfig, Nsga2, AgeMoea, AgeMoeaConfig, AntColonyTsp, AntColonyTspConfig, BayesianOpt, BayesianOptConfig,
Nsga2Config, Nsga3, Nsga3Config, Paes, PaesConfig, RandomSearch, RandomSearchConfig, CmaEs, CmaEsConfig, DifferentialEvolution, DifferentialEvolutionConfig, EpsilonMoea,
Spea2, Spea2Config, EpsilonMoeaConfig, GeneticAlgorithm, GeneticAlgorithmConfig, Grea, GreaConfig, HillClimber,
HillClimberConfig, Hype, HypeConfig, Hyperband, HyperbandConfig, Ibea, IbeaConfig, IpopCmaEs,
IpopCmaEsConfig, Knea, KneaConfig, Moead, MoeadConfig, Mopso, MopsoConfig, NelderMead,
NelderMeadConfig, Nsga2, Nsga2Config, Nsga3, Nsga3Config, OnePlusOneEs, OnePlusOneEsConfig,
Paes, PaesConfig, ParticleSwarm, ParticleSwarmConfig, PesaII, PesaIIConfig, RandomSearch,
RandomSearchConfig, Rvea, RveaConfig, SeparableNes, SeparableNesConfig, SimulatedAnnealing,
SimulatedAnnealingConfig, SmsEmoa, SmsEmoaConfig, Spea2, Spea2Config, TabuSearch,
TabuSearchConfig, Tlbo, TlboConfig, Tpe, TpeConfig, Umda, UmdaConfig,
}; };
src/selection/random.rs
+1 -5
View File
@@ -9,11 +9,7 @@ use crate::core::rng::Rng;
/// ///
/// Returns cloned decisions. Panics if `population` is empty and `count > 0` /// Returns cloned decisions. Panics if `population` is empty and `count > 0`
/// (spec §10.1). /// (spec §10.1).
pub fn select_random<D: Clone>( pub fn select_random<D: Clone>(population: &[Candidate<D>], count: usize, rng: &mut Rng) -> Vec<D> {
population: &[Candidate<D>],
count: usize,
rng: &mut Rng,
) -> Vec<D> {
if count == 0 { if count == 0 {
return Vec::new(); return Vec::new();
} }
src/selection/tournament.rs
+148 -6
View File
@@ -63,8 +63,18 @@ fn challenger_wins<D>(c: &Candidate<D>, b: &Candidate<D>, dir: Direction) -> boo
(false, true) => false, (false, true) => false,
(false, false) => c.evaluation.constraint_violation < b.evaluation.constraint_violation, (false, false) => c.evaluation.constraint_violation < b.evaluation.constraint_violation,
(true, true) => { (true, true) => {
let cv = c.evaluation.objectives.first().copied().unwrap_or(f64::INFINITY); let cv = c
let bv = b.evaluation.objectives.first().copied().unwrap_or(f64::INFINITY); .evaluation
.objectives
.first()
.copied()
.unwrap_or(f64::INFINITY);
let bv = b
.evaluation
.objectives
.first()
.copied()
.unwrap_or(f64::INFINITY);
match dir { match dir {
Direction::Minimize => cv < bv, Direction::Minimize => cv < bv,
Direction::Maximize => cv > bv, Direction::Maximize => cv > bv,
@@ -73,6 +83,105 @@ fn challenger_wins<D>(c: &Candidate<D>, b: &Candidate<D>, dir: Direction) -> boo
} }
} }
/// Stochastic-ranking selection (Runarsson & Yao 2000) for single-objective
/// constrained problems.
///
/// Performs a probabilistic bubble-sort pass on the population — each
/// pairwise comparison uses the *objective* value with probability `pf`,
/// otherwise it uses the standard feasibility-then-violation-then-objective
/// rule. The classic value is `pf = 0.45`; values close to `0.5` weight
/// objective improvement against constraint satisfaction.
///
/// Returns `count` decisions cloned from the top of the ranked
/// population. Useful when constraint satisfaction is hard and strict
/// feasibility-first selection traps the search outside the feasible
/// region.
///
/// # Panics
/// If `objectives` does not contain exactly one objective, if `pf` is
/// outside `[0.0, 1.0]`, or if the population is empty when `count > 0`.
pub fn stochastic_ranking_select<D: Clone>(
population: &[Candidate<D>],
objectives: &ObjectiveSpace,
pf: f64,
count: usize,
rng: &mut Rng,
) -> Vec<D> {
assert!(
objectives.is_single_objective(),
"stochastic_ranking_select requires exactly one objective",
);
assert!(
(0.0..=1.0).contains(&pf),
"stochastic_ranking_select pf must be in [0.0, 1.0]",
);
if count == 0 {
return Vec::new();
}
assert!(
!population.is_empty(),
"stochastic_ranking_select called on empty population with count > 0",
);
let direction = objectives.objectives[0].direction;
let n = population.len();
let mut order: Vec<usize> = (0..n).collect();
// Bubble-sort with at most n full sweeps (Runarsson & Yao §3).
for _ in 0..n {
let mut swapped = false;
for i in 0..n - 1 {
let a = &population[order[i]].evaluation;
let b = &population[order[i + 1]].evaluation;
let use_objective = rng.random::<f64>() < pf;
let a_first = if use_objective || (a.is_feasible() && b.is_feasible()) {
better_by_objective(a, b, direction)
} else {
better_by_feasibility(a, b, direction)
};
if !a_first {
order.swap(i, i + 1);
swapped = true;
}
}
if !swapped {
break;
}
}
let mut out = Vec::with_capacity(count);
for k in 0..count {
out.push(population[order[k % n]].decision.clone());
}
out
}
fn better_by_objective(
a: &crate::core::evaluation::Evaluation,
b: &crate::core::evaluation::Evaluation,
direction: Direction,
) -> bool {
let av = a.objectives.first().copied().unwrap_or(f64::INFINITY);
let bv = b.objectives.first().copied().unwrap_or(f64::INFINITY);
match direction {
Direction::Minimize => av < bv,
Direction::Maximize => av > bv,
}
}
fn better_by_feasibility(
a: &crate::core::evaluation::Evaluation,
b: &crate::core::evaluation::Evaluation,
direction: Direction,
) -> bool {
match (a.is_feasible(), b.is_feasible()) {
(true, false) => true,
(false, true) => false,
(false, false) => a.constraint_violation < b.constraint_violation,
(true, true) => better_by_objective(a, b, direction),
}
}
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
@@ -119,12 +228,45 @@ mod tests {
#[test] #[test]
#[should_panic(expected = "exactly one objective")] #[should_panic(expected = "exactly one objective")]
fn multi_objective_panics() { fn multi_objective_panics() {
let s = ObjectiveSpace::new(vec![ let s = ObjectiveSpace::new(vec![Objective::minimize("f1"), Objective::minimize("f2")]);
Objective::minimize("f1"),
Objective::minimize("f2"),
]);
let pop = [cand_min(1, 1.0)]; let pop = [cand_min(1, 1.0)];
let mut rng = rng_from_seed(0); let mut rng = rng_from_seed(0);
let _ = tournament_select_single_objective(&pop, &s, 2, 1, &mut rng); let _ = tournament_select_single_objective(&pop, &s, 2, 1, &mut rng);
} }
#[test]
fn stochastic_ranking_returns_count_decisions() {
let s = ObjectiveSpace::new(vec![Objective::minimize("f")]);
let pop = [cand_min(1, 5.0), cand_min(2, 1.0), cand_min(3, 9.0)];
let mut rng = rng_from_seed(7);
let picks = stochastic_ranking_select(&pop, &s, 0.45, 4, &mut rng);
assert_eq!(picks.len(), 4);
for p in &picks {
assert!([1, 2, 3].contains(p));
}
}
#[test]
fn stochastic_ranking_pf_zero_is_feasibility_first() {
// With pf = 0, the algorithm reduces to strict feasibility-first
// ordering, so the best feasible candidate should top the rank.
let s = ObjectiveSpace::new(vec![Objective::minimize("f")]);
let pop = [
Candidate::new(1u32, Evaluation::constrained(vec![0.0], 5.0)), // infeasible
Candidate::new(2u32, Evaluation::new(vec![10.0])), // feasible, big f
Candidate::new(3u32, Evaluation::new(vec![3.0])), // feasible, small f
];
let mut rng = rng_from_seed(0);
let picks = stochastic_ranking_select(&pop, &s, 0.0, 3, &mut rng);
assert_eq!(picks[0], 3); // best feasible first
}
#[test]
#[should_panic(expected = "pf must be in [0.0, 1.0]")]
fn stochastic_ranking_pf_out_of_range_panics() {
let s = ObjectiveSpace::new(vec![Objective::minimize("f")]);
let pop = [cand_min(1, 1.0)];
let mut rng = rng_from_seed(0);
let _ = stochastic_ranking_select(&pop, &s, 1.5, 1, &mut rng);
}
} }
src/traits/mod.rs
+2
View File
@@ -2,8 +2,10 @@
pub mod initializer; pub mod initializer;
pub mod optimizer; pub mod optimizer;
pub mod repair;
pub mod variation; pub mod variation;
pub use initializer::*; pub use initializer::*;
pub use optimizer::*; pub use optimizer::*;
pub use repair::*;
pub use variation::*; pub use variation::*;
src/traits/repair.rs
+14
View File
@@ -0,0 +1,14 @@
//! Trait for restoring decisions to feasibility.
/// Transforms an infeasible (or possibly-infeasible) decision into a
/// feasible one, in place.
///
/// `Repair` is the projection-style alternative to penalty-style
/// constraint handling (which uses `Evaluation::constraint_violation`).
/// Wrap a `Variation` operator's output through a `Repair` to guarantee
/// feasibility; or call `repair()` inside your `Problem::evaluate` if
/// the constraint structure is best handled at evaluation time.
pub trait Repair<D> {
/// Mutate `decision` in place to satisfy the repair's constraints.
fn repair(&mut self, decision: &mut D);
}
tests/algorithm_properties.rs
+733
View File
@@ -0,0 +1,733 @@
//! Per-algorithm property tests.
//!
//! For every `Optimizer` impl in heuropt we check the same three properties:
//! 1. **Deterministic-with-seed**: two runs with the same seed produce
//! the same `best.evaluation.objectives`.
//! 2. **No panic on random valid inputs**: random seeds, random tiny
//! problems, random bounds — the algorithm runs to completion.
//! 3. **Population-size invariant** (where the algorithm documents one):
//! the final population has the configured size.
use proptest::prelude::*;
use heuropt::core::evaluation::Evaluation;
use heuropt::core::objective::{Objective, ObjectiveSpace};
use heuropt::core::problem::Problem;
use heuropt::prelude::*;
// -----------------------------------------------------------------------------
// Tiny problems
// -----------------------------------------------------------------------------
struct Sphere1D;
impl Problem for Sphere1D {
type Decision = Vec<f64>;
fn objectives(&self) -> ObjectiveSpace {
ObjectiveSpace::new(vec![Objective::minimize("f")])
}
fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
Evaluation::new(vec![x[0] * x[0]])
}
}
struct SchafferN1;
impl Problem for SchafferN1 {
type Decision = Vec<f64>;
fn objectives(&self) -> ObjectiveSpace {
ObjectiveSpace::new(vec![Objective::minimize("f1"), Objective::minimize("f2")])
}
fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
let v = x[0];
Evaluation::new(vec![v * v, (v - 2.0).powi(2)])
}
}
struct OneMax {
#[allow(dead_code)]
bits: usize,
}
impl Problem for OneMax {
type Decision = Vec<bool>;
fn objectives(&self) -> ObjectiveSpace {
ObjectiveSpace::new(vec![Objective::maximize("count")])
}
fn evaluate(&self, x: &Vec<bool>) -> Evaluation {
Evaluation::new(vec![x.iter().filter(|b| **b).count() as f64])
}
}
// -----------------------------------------------------------------------------
// Helpers
// -----------------------------------------------------------------------------
fn so_bounds() -> RealBounds {
RealBounds::new(vec![(-3.0, 3.0)])
}
fn so_bounds_2d() -> RealBounds {
RealBounds::new(vec![(-3.0, 3.0); 2])
}
fn mo_bounds() -> Vec<(f64, f64)> {
vec![(-3.0, 3.0)]
}
fn mo_variation() -> CompositeVariation<SimulatedBinaryCrossover, PolynomialMutation> {
let bounds = mo_bounds();
CompositeVariation {
crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5),
mutation: PolynomialMutation::new(bounds, 20.0, 1.0),
}
}
// -----------------------------------------------------------------------------
// Single-objective continuous
// -----------------------------------------------------------------------------
proptest! {
#[test]
fn random_search_deterministic(seed in any::<u64>()) {
let make = || RandomSearch::new(
RandomSearchConfig { iterations: 20, batch_size: 1, seed },
so_bounds(),
);
let r1 = make().run(&Sphere1D);
let r2 = make().run(&Sphere1D);
prop_assert_eq!(
r1.best.unwrap().evaluation.objectives,
r2.best.unwrap().evaluation.objectives,
);
}
#[test]
fn hill_climber_deterministic(seed in any::<u64>()) {
let make = || HillClimber::new(
HillClimberConfig { iterations: 20, seed },
so_bounds(),
GaussianMutation { sigma: 0.1 },
);
let r1 = make().run(&Sphere1D);
let r2 = make().run(&Sphere1D);
prop_assert_eq!(
r1.best.unwrap().evaluation.objectives,
r2.best.unwrap().evaluation.objectives,
);
}
#[test]
fn one_plus_one_es_deterministic(seed in any::<u64>()) {
let make = || OnePlusOneEs::new(
OnePlusOneEsConfig {
iterations: 50,
initial_sigma: 0.5,
adaptation_period: 10,
step_increase: 1.22,
seed,
},
so_bounds(),
);
let r1 = make().run(&Sphere1D);
let r2 = make().run(&Sphere1D);
prop_assert_eq!(
r1.best.unwrap().evaluation.objectives,
r2.best.unwrap().evaluation.objectives,
);
}
#[test]
fn simulated_annealing_deterministic(seed in any::<u64>()) {
let make = || SimulatedAnnealing::new(
SimulatedAnnealingConfig {
iterations: 50,
initial_temperature: 1.0,
final_temperature: 1e-3,
seed,
},
so_bounds(),
GaussianMutation { sigma: 0.1 },
);
let r1 = make().run(&Sphere1D);
let r2 = make().run(&Sphere1D);
prop_assert_eq!(
r1.best.unwrap().evaluation.objectives,
r2.best.unwrap().evaluation.objectives,
);
}
#[test]
fn ga_deterministic(seed in any::<u64>()) {
let bounds = mo_bounds();
let make = || GeneticAlgorithm::new(
GeneticAlgorithmConfig {
population_size: 10,
generations: 5,
tournament_size: 2,
elitism: 1,
seed,
},
RealBounds::new(bounds.clone()),
CompositeVariation {
crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5),
mutation: PolynomialMutation::new(bounds.clone(), 20.0, 1.0),
},
);
let r1 = make().run(&Sphere1D);
let r2 = make().run(&Sphere1D);
prop_assert_eq!(
r1.best.unwrap().evaluation.objectives,
r2.best.unwrap().evaluation.objectives,
);
}
#[test]
fn pso_deterministic(seed in any::<u64>()) {
let make = || ParticleSwarm::new(
ParticleSwarmConfig {
swarm_size: 10,
generations: 5,
inertia: 0.7,
cognitive: 1.5,
social: 1.5,
seed,
},
so_bounds(),
);
let r1 = make().run(&Sphere1D);
let r2 = make().run(&Sphere1D);
prop_assert_eq!(
r1.best.unwrap().evaluation.objectives,
r2.best.unwrap().evaluation.objectives,
);
}
#[test]
fn de_deterministic(seed in any::<u64>()) {
let make = || DifferentialEvolution::new(
DifferentialEvolutionConfig {
population_size: 10,
generations: 5,
differential_weight: 0.5,
crossover_probability: 0.9,
seed,
},
so_bounds(),
);
let r1 = make().run(&Sphere1D);
let r2 = make().run(&Sphere1D);
prop_assert_eq!(
r1.best.unwrap().evaluation.objectives,
r2.best.unwrap().evaluation.objectives,
);
}
#[test]
fn cmaes_deterministic(seed in any::<u64>()) {
let cfg = CmaEsConfig {
population_size: 8,
generations: 5,
initial_sigma: 0.5,
eigen_decomposition_period: 1,
initial_mean: None,
seed,
};
let mut a = CmaEs::new(cfg.clone(), so_bounds());
let mut b = CmaEs::new(cfg, so_bounds());
let r1 = a.run(&Sphere1D);
let r2 = b.run(&Sphere1D);
prop_assert_eq!(
r1.best.unwrap().evaluation.objectives,
r2.best.unwrap().evaluation.objectives,
);
}
#[test]
fn ipop_cmaes_deterministic(seed in any::<u64>()) {
let cfg = IpopCmaEsConfig {
initial_population_size: 8,
total_generations: 30,
initial_sigma: 0.5,
eigen_decomposition_period: 1,
stall_generations: None,
seed,
};
let mut a = IpopCmaEs::new(cfg.clone(), so_bounds());
let mut b = IpopCmaEs::new(cfg, so_bounds());
let r1 = a.run(&Sphere1D);
let r2 = b.run(&Sphere1D);
prop_assert_eq!(
r1.best.unwrap().evaluation.objectives,
r2.best.unwrap().evaluation.objectives,
);
}
#[test]
fn snes_deterministic(seed in any::<u64>()) {
let make = || SeparableNes::new(
SeparableNesConfig {
population_size: 8,
generations: 5,
initial_sigma: 0.5,
mean_learning_rate: 1.0,
sigma_learning_rate: None,
seed,
},
so_bounds(),
);
let r1 = make().run(&Sphere1D);
let r2 = make().run(&Sphere1D);
prop_assert_eq!(
r1.best.unwrap().evaluation.objectives,
r2.best.unwrap().evaluation.objectives,
);
}
#[test]
fn tlbo_deterministic(seed in any::<u64>()) {
let make = || Tlbo::new(
TlboConfig { population_size: 10, generations: 5, seed },
so_bounds(),
);
let r1 = make().run(&Sphere1D);
let r2 = make().run(&Sphere1D);
prop_assert_eq!(
r1.best.unwrap().evaluation.objectives,
r2.best.unwrap().evaluation.objectives,
);
}
#[test]
fn nelder_mead_deterministic(_dummy in any::<bool>()) {
// Nelder-Mead is purely deterministic; no seed.
let make = || NelderMead::new(
NelderMeadConfig { iterations: 50, ..NelderMeadConfig::default() },
so_bounds(),
);
let r1 = make().run(&Sphere1D);
let r2 = make().run(&Sphere1D);
prop_assert_eq!(
r1.best.unwrap().evaluation.objectives,
r2.best.unwrap().evaluation.objectives,
);
}
#[test]
fn bayesian_opt_deterministic(seed in any::<u64>()) {
let make = || BayesianOpt::new(
BayesianOptConfig {
initial_samples: 5,
iterations: 10,
length_scales: None,
signal_variance: 1.0,
noise_variance: 1e-6,
acquisition_samples: 100,
seed,
},
so_bounds(),
);
let r1 = make().run(&Sphere1D);
let r2 = make().run(&Sphere1D);
prop_assert_eq!(
r1.best.unwrap().evaluation.objectives,
r2.best.unwrap().evaluation.objectives,
);
}
#[test]
fn tpe_deterministic(seed in any::<u64>()) {
let make = || Tpe::new(
TpeConfig {
initial_samples: 5,
iterations: 10,
good_fraction: 0.25,
candidate_samples: 12,
bandwidth_factor: 1.0,
seed,
},
so_bounds(),
);
let r1 = make().run(&Sphere1D);
let r2 = make().run(&Sphere1D);
prop_assert_eq!(
r1.best.unwrap().evaluation.objectives,
r2.best.unwrap().evaluation.objectives,
);
}
}
// -----------------------------------------------------------------------------
// Multi-objective
// -----------------------------------------------------------------------------
proptest! {
#[test]
fn nsga2_deterministic_and_pop_size(seed in any::<u64>()) {
let make = || Nsga2::new(
Nsga2Config { population_size: 10, generations: 3, seed },
RealBounds::new(mo_bounds()),
mo_variation(),
);
let r1 = make().run(&SchafferN1);
let r2 = make().run(&SchafferN1);
let oa: Vec<Vec<f64>> = r1.pareto_front.iter()
.map(|c| c.evaluation.objectives.clone()).collect();
let ob: Vec<Vec<f64>> = r2.pareto_front.iter()
.map(|c| c.evaluation.objectives.clone()).collect();
prop_assert_eq!(oa, ob);
prop_assert_eq!(r1.population.len(), 10);
}
#[test]
fn nsga3_deterministic_and_pop_size(seed in any::<u64>()) {
let make = || Nsga3::new(
Nsga3Config {
population_size: 12,
generations: 3,
reference_divisions: 11,
seed,
},
RealBounds::new(mo_bounds()),
mo_variation(),
);
let r1 = make().run(&SchafferN1);
let r2 = make().run(&SchafferN1);
let oa: Vec<Vec<f64>> = r1.pareto_front.iter()
.map(|c| c.evaluation.objectives.clone()).collect();
let ob: Vec<Vec<f64>> = r2.pareto_front.iter()
.map(|c| c.evaluation.objectives.clone()).collect();
prop_assert_eq!(oa, ob);
prop_assert_eq!(r1.population.len(), 12);
}
#[test]
fn spea2_deterministic(seed in any::<u64>()) {
let make = || Spea2::new(
Spea2Config {
population_size: 10,
archive_size: 10,
generations: 3,
seed,
},
RealBounds::new(mo_bounds()),
mo_variation(),
);
let r1 = make().run(&SchafferN1);
let r2 = make().run(&SchafferN1);
let oa: Vec<Vec<f64>> = r1.pareto_front.iter()
.map(|c| c.evaluation.objectives.clone()).collect();
let ob: Vec<Vec<f64>> = r2.pareto_front.iter()
.map(|c| c.evaluation.objectives.clone()).collect();
prop_assert_eq!(oa, ob);
}
#[test]
fn moead_deterministic(seed in any::<u64>()) {
let make = || Moead::new(
MoeadConfig {
generations: 3,
reference_divisions: 9,
neighborhood_size: 4,
seed,
},
RealBounds::new(mo_bounds()),
mo_variation(),
);
let r1 = make().run(&SchafferN1);
let r2 = make().run(&SchafferN1);
let oa: Vec<Vec<f64>> = r1.population.iter()
.map(|c| c.evaluation.objectives.clone()).collect();
let ob: Vec<Vec<f64>> = r2.population.iter()
.map(|c| c.evaluation.objectives.clone()).collect();
prop_assert_eq!(oa, ob);
}
#[test]
fn mopso_deterministic(seed in any::<u64>()) {
let make = || Mopso::new(
MopsoConfig {
swarm_size: 10,
generations: 3,
archive_size: 10,
inertia: 0.7,
cognitive: 1.5,
social: 1.5,
seed,
},
RealBounds::new(mo_bounds()),
);
let r1 = make().run(&SchafferN1);
let r2 = make().run(&SchafferN1);
let oa: Vec<Vec<f64>> = r1.pareto_front.iter()
.map(|c| c.evaluation.objectives.clone()).collect();
let ob: Vec<Vec<f64>> = r2.pareto_front.iter()
.map(|c| c.evaluation.objectives.clone()).collect();
prop_assert_eq!(oa, ob);
}
#[test]
fn ibea_deterministic(seed in any::<u64>()) {
let make = || Ibea::new(
IbeaConfig {
population_size: 10,
generations: 3,
kappa: 0.05,
seed,
},
RealBounds::new(mo_bounds()),
mo_variation(),
);
let r1 = make().run(&SchafferN1);
let r2 = make().run(&SchafferN1);
let oa: Vec<Vec<f64>> = r1.pareto_front.iter()
.map(|c| c.evaluation.objectives.clone()).collect();
let ob: Vec<Vec<f64>> = r2.pareto_front.iter()
.map(|c| c.evaluation.objectives.clone()).collect();
prop_assert_eq!(oa, ob);
}
#[test]
fn sms_emoa_deterministic(seed in any::<u64>()) {
let make = || SmsEmoa::new(
SmsEmoaConfig {
population_size: 8,
generations: 5,
reference_point: vec![10.0, 10.0],
seed,
},
RealBounds::new(mo_bounds()),
mo_variation(),
);
let r1 = make().run(&SchafferN1);
let r2 = make().run(&SchafferN1);
let oa: Vec<Vec<f64>> = r1.pareto_front.iter()
.map(|c| c.evaluation.objectives.clone()).collect();
let ob: Vec<Vec<f64>> = r2.pareto_front.iter()
.map(|c| c.evaluation.objectives.clone()).collect();
prop_assert_eq!(oa, ob);
}
#[test]
fn hype_deterministic(seed in any::<u64>()) {
let make = || Hype::new(
HypeConfig {
population_size: 10,
generations: 3,
reference_point: vec![10.0, 10.0],
mc_samples: 100,
seed,
},
RealBounds::new(mo_bounds()),
mo_variation(),
);
let r1 = make().run(&SchafferN1);
let r2 = make().run(&SchafferN1);
let oa: Vec<Vec<f64>> = r1.pareto_front.iter()
.map(|c| c.evaluation.objectives.clone()).collect();
let ob: Vec<Vec<f64>> = r2.pareto_front.iter()
.map(|c| c.evaluation.objectives.clone()).collect();
prop_assert_eq!(oa, ob);
}
#[test]
fn pesa2_deterministic(seed in any::<u64>()) {
let make = || PesaII::new(
PesaIIConfig {
population_size: 10,
archive_size: 10,
generations: 3,
grid_divisions: 4,
seed,
},
RealBounds::new(mo_bounds()),
mo_variation(),
);
let r1 = make().run(&SchafferN1);
let r2 = make().run(&SchafferN1);
let oa: Vec<Vec<f64>> = r1.pareto_front.iter()
.map(|c| c.evaluation.objectives.clone()).collect();
let ob: Vec<Vec<f64>> = r2.pareto_front.iter()
.map(|c| c.evaluation.objectives.clone()).collect();
prop_assert_eq!(oa, ob);
}
#[test]
fn epsilon_moea_deterministic(seed in any::<u64>()) {
let make = || EpsilonMoea::new(
EpsilonMoeaConfig {
population_size: 10,
evaluations: 30,
epsilon: vec![0.05, 0.05],
seed,
},
RealBounds::new(mo_bounds()),
mo_variation(),
);
let r1 = make().run(&SchafferN1);
let r2 = make().run(&SchafferN1);
let oa: Vec<Vec<f64>> = r1.pareto_front.iter()
.map(|c| c.evaluation.objectives.clone()).collect();
let ob: Vec<Vec<f64>> = r2.pareto_front.iter()
.map(|c| c.evaluation.objectives.clone()).collect();
prop_assert_eq!(oa, ob);
}
#[test]
fn age_moea_deterministic(seed in any::<u64>()) {
let make = || AgeMoea::new(
AgeMoeaConfig { population_size: 10, generations: 3, seed },
RealBounds::new(mo_bounds()),
mo_variation(),
);
let r1 = make().run(&SchafferN1);
let r2 = make().run(&SchafferN1);
let oa: Vec<Vec<f64>> = r1.pareto_front.iter()
.map(|c| c.evaluation.objectives.clone()).collect();
let ob: Vec<Vec<f64>> = r2.pareto_front.iter()
.map(|c| c.evaluation.objectives.clone()).collect();
prop_assert_eq!(oa, ob);
}
#[test]
fn grea_deterministic(seed in any::<u64>()) {
let make = || Grea::new(
GreaConfig {
population_size: 10,
generations: 3,
grid_divisions: 4,
seed,
},
RealBounds::new(mo_bounds()),
mo_variation(),
);
let r1 = make().run(&SchafferN1);
let r2 = make().run(&SchafferN1);
let oa: Vec<Vec<f64>> = r1.pareto_front.iter()
.map(|c| c.evaluation.objectives.clone()).collect();
let ob: Vec<Vec<f64>> = r2.pareto_front.iter()
.map(|c| c.evaluation.objectives.clone()).collect();
prop_assert_eq!(oa, ob);
}
#[test]
fn knea_deterministic(seed in any::<u64>()) {
let make = || Knea::new(
KneaConfig { population_size: 10, generations: 3, seed },
RealBounds::new(mo_bounds()),
mo_variation(),
);
let r1 = make().run(&SchafferN1);
let r2 = make().run(&SchafferN1);
let oa: Vec<Vec<f64>> = r1.pareto_front.iter()
.map(|c| c.evaluation.objectives.clone()).collect();
let ob: Vec<Vec<f64>> = r2.pareto_front.iter()
.map(|c| c.evaluation.objectives.clone()).collect();
prop_assert_eq!(oa, ob);
}
#[test]
fn rvea_deterministic(seed in any::<u64>()) {
let make = || Rvea::new(
RveaConfig {
population_size: 10,
generations: 3,
reference_divisions: 9,
alpha: 2.0,
seed,
},
RealBounds::new(mo_bounds()),
mo_variation(),
);
let r1 = make().run(&SchafferN1);
let r2 = make().run(&SchafferN1);
let oa: Vec<Vec<f64>> = r1.pareto_front.iter()
.map(|c| c.evaluation.objectives.clone()).collect();
let ob: Vec<Vec<f64>> = r2.pareto_front.iter()
.map(|c| c.evaluation.objectives.clone()).collect();
prop_assert_eq!(oa, ob);
}
#[test]
fn paes_deterministic(seed in any::<u64>()) {
let make = || Paes::new(
PaesConfig { iterations: 30, archive_size: 10, seed },
RealBounds::new(mo_bounds()),
GaussianMutation { sigma: 0.1 },
);
let r1 = make().run(&SchafferN1);
let r2 = make().run(&SchafferN1);
let oa: Vec<Vec<f64>> = r1.pareto_front.iter()
.map(|c| c.evaluation.objectives.clone()).collect();
let ob: Vec<Vec<f64>> = r2.pareto_front.iter()
.map(|c| c.evaluation.objectives.clone()).collect();
prop_assert_eq!(oa, ob);
}
}
// -----------------------------------------------------------------------------
// Other decision types
// -----------------------------------------------------------------------------
proptest! {
#[test]
fn umda_deterministic(seed in any::<u64>(), bits in 4usize..16) {
let problem = OneMax { bits };
let make = || Umda::new(UmdaConfig {
population_size: 10,
selected_size: 5,
generations: 3,
bits,
seed,
});
let r1 = make().run(&problem);
let r2 = make().run(&problem);
prop_assert_eq!(
r1.best.unwrap().evaluation.objectives,
r2.best.unwrap().evaluation.objectives,
);
}
#[test]
fn random_search_evaluation_count_invariant(
iterations in 1usize..30,
batch_size in 1usize..5,
seed in any::<u64>(),
) {
let mut opt = RandomSearch::new(
RandomSearchConfig { iterations, batch_size, seed },
so_bounds(),
);
let r = opt.run(&Sphere1D);
prop_assert_eq!(r.evaluations, iterations * batch_size);
prop_assert_eq!(r.population.len(), iterations * batch_size);
prop_assert_eq!(r.generations, iterations);
}
}
// -----------------------------------------------------------------------------
// Cross-cutting: best is at least as good as any front member
// -----------------------------------------------------------------------------
proptest! {
#[test]
fn so_optimizer_best_beats_initial(seed in any::<u64>()) {
// After running an SO optimizer, the result's best.evaluation
// should be at least as good as the worst point sampled — i.e.
// the optimizer doesn't return None or some random non-best.
let mut opt = DifferentialEvolution::new(
DifferentialEvolutionConfig {
population_size: 10,
generations: 5,
differential_weight: 0.5,
crossover_probability: 0.9,
seed,
},
so_bounds_2d(),
);
let r = opt.run(&Sphere1D);
let best_f = r.best.unwrap().evaluation.objectives[0];
let pop_min = r.population.iter()
.map(|c| c.evaluation.objectives[0])
.fold(f64::INFINITY, f64::min);
prop_assert!(
best_f <= pop_min + 1e-12,
"best f = {best_f}, pop min = {pop_min}",
);
}
}
tests/metric_properties.rs
+121
View File
@@ -0,0 +1,121 @@
//! Per-metric property tests for the Pareto-quality metrics.
use proptest::prelude::*;
use heuropt::core::candidate::Candidate;
use heuropt::core::evaluation::Evaluation;
use heuropt::core::objective::{Objective, ObjectiveSpace};
use heuropt::metrics::hypervolume::{hypervolume_2d, hypervolume_nd};
use heuropt::metrics::spacing::spacing;
fn space_2d() -> ObjectiveSpace {
ObjectiveSpace::new(vec![Objective::minimize("f1"), Objective::minimize("f2")])
}
fn space_3d() -> ObjectiveSpace {
ObjectiveSpace::new(vec![
Objective::minimize("f1"),
Objective::minimize("f2"),
Objective::minimize("f3"),
])
}
fn cand_2d(a: f64, b: f64) -> Candidate<()> {
Candidate::new((), Evaluation::new(vec![a, b]))
}
fn cand_3d(a: f64, b: f64, c: f64) -> Candidate<()> {
Candidate::new((), Evaluation::new(vec![a, b, c]))
}
proptest! {
/// hypervolume_2d is non-negative.
#[test]
fn hv2_non_negative(
front in prop::collection::vec((0.0_f64..10.0, 0.0_f64..10.0), 0..15),
) {
let s = space_2d();
let pop: Vec<Candidate<()>> = front.iter().map(|&(a, b)| cand_2d(a, b)).collect();
let hv = hypervolume_2d(&pop, &s, [11.0, 11.0]);
prop_assert!(hv >= 0.0);
prop_assert!(hv.is_finite());
}
/// hypervolume_2d is bounded above by the (reference - 0)² = 121 box.
#[test]
fn hv2_bounded_by_box(
front in prop::collection::vec((0.0_f64..10.0, 0.0_f64..10.0), 1..15),
) {
let s = space_2d();
let pop: Vec<Candidate<()>> = front.iter().map(|&(a, b)| cand_2d(a, b)).collect();
let hv = hypervolume_2d(&pop, &s, [11.0, 11.0]);
prop_assert!(hv <= 121.0_f64 + 1e-9);
}
/// Adding a dominated point doesn't change hypervolume_2d.
#[test]
fn hv2_dominated_invariant(
a in 0.0_f64..5.0,
b in 0.0_f64..5.0,
d_offset in 0.001_f64..3.0,
) {
let s = space_2d();
let base = vec![cand_2d(a, b)];
let mut with_dominated = base.clone();
// (a + offset, b + offset) is strictly worse than (a, b) on both
// axes, so it's dominated.
with_dominated.push(cand_2d(a + d_offset, b + d_offset));
let hv1 = hypervolume_2d(&base, &s, [11.0, 11.0]);
let hv2 = hypervolume_2d(&with_dominated, &s, [11.0, 11.0]);
prop_assert!((hv1 - hv2).abs() < 1e-9);
}
/// hypervolume_nd agrees with hypervolume_2d on 2-D inputs.
#[test]
fn hv_nd_matches_2d(
front in prop::collection::vec((0.0_f64..10.0, 0.0_f64..10.0), 1..10),
) {
let s = space_2d();
let pop: Vec<Candidate<()>> = front.iter().map(|&(a, b)| cand_2d(a, b)).collect();
let hv2 = hypervolume_2d(&pop, &s, [11.0, 11.0]);
let hvn = hypervolume_nd(&pop, &s, &[11.0, 11.0]);
prop_assert!((hv2 - hvn).abs() < 1e-9, "{hv2} vs {hvn}");
}
/// hypervolume_nd in 3-D is non-negative and bounded.
#[test]
fn hv3_non_negative_bounded(
pts in prop::collection::vec(
(0.0_f64..2.0, 0.0_f64..2.0, 0.0_f64..2.0),
0..10,
),
) {
let s = space_3d();
let pop: Vec<Candidate<()>> = pts.iter().map(|&(a, b, c)| cand_3d(a, b, c)).collect();
let hv = hypervolume_nd(&pop, &s, &[3.0, 3.0, 3.0]);
prop_assert!(hv >= 0.0);
prop_assert!(hv.is_finite());
// Reference box has volume 27.
prop_assert!(hv <= 27.0 + 1e-9);
}
/// spacing is non-negative and zero on a single point.
#[test]
fn spacing_non_negative(
front in prop::collection::vec((0.0_f64..10.0, 0.0_f64..10.0), 0..15),
) {
let s = space_2d();
let pop: Vec<Candidate<()>> = front.iter().map(|&(a, b)| cand_2d(a, b)).collect();
let sp = spacing(&pop, &s);
prop_assert!(sp >= 0.0);
prop_assert!(sp.is_finite());
}
/// spacing on a single-point front is exactly 0.
#[test]
fn spacing_single_point_is_zero(a in 0.0_f64..10.0, b in 0.0_f64..10.0) {
let s = space_2d();
let pop = vec![cand_2d(a, b)];
let sp = spacing(&pop, &s);
prop_assert_eq!(sp, 0.0);
}
}
tests/numerical_stability.rs
+279
View File
@@ -0,0 +1,279 @@
//! Stress tests for numerical edge cases.
//!
//! These don't check correctness in detail — they check that algorithms
//! and helpers don't panic, return NaN, or produce nonsensical sizes on
//! pathological inputs. The kind of failures these surface are typically
//! division-by-zero, log/sqrt of negatives, empty-collection .min(),
//! etc. — all the things property tests on "ordinary" inputs would miss.
use heuropt::core::candidate::Candidate;
use heuropt::core::evaluation::Evaluation;
use heuropt::core::objective::{Objective, ObjectiveSpace};
use heuropt::metrics::hypervolume::{hypervolume_2d, hypervolume_nd};
use heuropt::metrics::spacing::spacing;
use heuropt::pareto::crowding::crowding_distance;
use heuropt::pareto::dominance::pareto_compare;
use heuropt::pareto::front::{best_candidate, pareto_front};
use heuropt::pareto::sort::non_dominated_sort;
use heuropt::prelude::*;
fn space_2d() -> ObjectiveSpace {
ObjectiveSpace::new(vec![Objective::minimize("f1"), Objective::minimize("f2")])
}
fn cand2(a: f64, b: f64) -> Candidate<()> {
Candidate::new((), Evaluation::new(vec![a, b]))
}
// -----------------------------------------------------------------------------
// Pareto utilities — empty / singleton / duplicate populations
// -----------------------------------------------------------------------------
#[test]
fn pareto_front_on_empty_population() {
let s = space_2d();
let front = pareto_front::<()>(&[], &s);
assert!(front.is_empty());
}
#[test]
fn pareto_front_on_singleton() {
let s = space_2d();
let pop = vec![cand2(1.0, 2.0)];
let front = pareto_front(&pop, &s);
assert_eq!(front.len(), 1);
}
#[test]
fn pareto_front_on_all_duplicates() {
let s = space_2d();
let pop: Vec<_> = (0..5).map(|_| cand2(1.0, 1.0)).collect();
let front = pareto_front(&pop, &s);
// All members are mutually Equal — every one is non-dominated.
assert_eq!(front.len(), 5);
}
#[test]
fn non_dominated_sort_on_empty() {
let s = space_2d();
let fronts = non_dominated_sort::<()>(&[], &s);
assert!(fronts.is_empty());
}
#[test]
fn non_dominated_sort_on_all_duplicates() {
let s = space_2d();
let pop: Vec<_> = (0..6).map(|_| cand2(1.0, 1.0)).collect();
let fronts = non_dominated_sort(&pop, &s);
// Every member is "Equal" with every other member — should be one
// front containing all of them.
assert_eq!(fronts.len(), 1);
assert_eq!(fronts[0].len(), 6);
}
#[test]
fn crowding_distance_on_empty_front_is_empty() {
let s = space_2d();
let pop: Vec<Candidate<()>> = vec![];
let d = crowding_distance(&pop, &[], &s);
assert!(d.is_empty());
}
#[test]
fn crowding_distance_on_two_point_front_is_infinity() {
let s = space_2d();
let pop = vec![cand2(0.0, 1.0), cand2(1.0, 0.0)];
let d = crowding_distance(&pop, &[0, 1], &s);
assert!(d[0].is_infinite());
assert!(d[1].is_infinite());
}
#[test]
fn crowding_distance_on_collinear_points_finite_or_inf() {
let s = space_2d();
// All points have f2 = 5; f1 axis varies but f2 doesn't.
let pop = vec![cand2(0.0, 5.0), cand2(1.0, 5.0), cand2(2.0, 5.0)];
let d = crowding_distance(&pop, &[0, 1, 2], &s);
// f2 axis has zero span so it contributes nothing; f1 axis gives the
// boundaries infinity, the interior finite.
assert!(d[0].is_infinite());
assert!(d[2].is_infinite());
assert!(d[1].is_finite());
}
#[test]
fn pareto_compare_with_zero_constraint_violations() {
let s = space_2d();
let a = Evaluation::constrained(vec![1.0, 1.0], 0.0);
let b = Evaluation::constrained(vec![2.0, 2.0], 0.0);
let r = pareto_compare(&a, &b, &s);
use heuropt::pareto::dominance::Dominance;
assert_eq!(r, Dominance::Dominates);
}
#[test]
fn best_candidate_on_empty_returns_none() {
let s = ObjectiveSpace::new(vec![Objective::minimize("f")]);
let pop: Vec<Candidate<()>> = vec![];
let best = best_candidate(&pop, &s);
assert!(best.is_none());
}
#[test]
fn best_candidate_all_infeasible_returns_none() {
let s = ObjectiveSpace::new(vec![Objective::minimize("f")]);
let pop = vec![
Candidate::new((), Evaluation::constrained(vec![1.0], 0.5)),
Candidate::new((), Evaluation::constrained(vec![2.0], 0.7)),
];
let best = best_candidate(&pop, &s);
assert!(best.is_none());
}
// -----------------------------------------------------------------------------
// Metrics — degenerate inputs
// -----------------------------------------------------------------------------
#[test]
fn hv2_on_empty_is_zero() {
let s = space_2d();
let front: Vec<Candidate<()>> = vec![];
assert_eq!(hypervolume_2d(&front, &s, [1.0, 1.0]), 0.0);
}
#[test]
fn hv2_when_no_point_dominates_reference_is_zero() {
let s = space_2d();
let front = vec![cand2(5.0, 5.0)]; // worse than reference (1, 1)
assert_eq!(hypervolume_2d(&front, &s, [1.0, 1.0]), 0.0);
}
#[test]
fn hv_nd_on_empty_is_zero() {
let s = ObjectiveSpace::new(vec![
Objective::minimize("a"),
Objective::minimize("b"),
Objective::minimize("c"),
]);
let front: Vec<Candidate<()>> = vec![];
assert_eq!(hypervolume_nd(&front, &s, &[1.0, 1.0, 1.0]), 0.0);
}
#[test]
fn spacing_on_empty_is_zero() {
let s = space_2d();
let pop: Vec<Candidate<()>> = vec![];
assert_eq!(spacing(&pop, &s), 0.0);
}
#[test]
fn spacing_on_singleton_is_zero() {
let s = space_2d();
let pop = vec![cand2(0.5, 0.5)];
assert_eq!(spacing(&pop, &s), 0.0);
}
// -----------------------------------------------------------------------------
// Algorithms — extreme inputs
// -----------------------------------------------------------------------------
struct ConstantFn;
impl heuropt::core::problem::Problem for ConstantFn {
type Decision = Vec<f64>;
fn objectives(&self) -> ObjectiveSpace {
ObjectiveSpace::new(vec![Objective::minimize("f")])
}
fn evaluate(&self, _: &Vec<f64>) -> Evaluation {
// Flat fitness — every point is equally good.
Evaluation::new(vec![0.0])
}
}
#[test]
fn de_handles_flat_fitness() {
// No gradient, no signal. DE should still run to completion and
// return a valid result (every point ties for best).
let mut opt = DifferentialEvolution::new(
DifferentialEvolutionConfig {
population_size: 10,
generations: 5,
differential_weight: 0.5,
crossover_probability: 0.9,
seed: 0,
},
RealBounds::new(vec![(-1.0, 1.0); 3]),
);
let r = opt.run(&ConstantFn);
let best = r.best.unwrap();
assert_eq!(best.evaluation.objectives, vec![0.0]);
assert!(r.evaluations > 0);
}
#[test]
fn cma_es_handles_flat_fitness() {
let mut opt = CmaEs::new(
CmaEsConfig {
population_size: 8,
generations: 5,
initial_sigma: 0.5,
eigen_decomposition_period: 1,
initial_mean: None,
seed: 0,
},
RealBounds::new(vec![(-1.0, 1.0); 3]),
);
let r = opt.run(&ConstantFn);
assert!(r.best.is_some());
}
#[test]
fn nelder_mead_handles_flat_fitness() {
let mut opt = NelderMead::new(
NelderMeadConfig::default(),
RealBounds::new(vec![(-1.0, 1.0); 3]),
);
let r = opt.run(&ConstantFn);
assert!(r.best.is_some());
}
#[test]
fn bayesian_opt_handles_flat_fitness() {
let mut opt = BayesianOpt::new(
BayesianOptConfig {
initial_samples: 4,
iterations: 6,
length_scales: None,
signal_variance: 1.0,
noise_variance: 1e-3,
acquisition_samples: 50,
seed: 0,
},
RealBounds::new(vec![(-1.0, 1.0); 2]),
);
let r = opt.run(&ConstantFn);
assert!(r.best.is_some());
}
#[test]
fn de_handles_zero_width_bounds() {
// lo == hi on every axis — search space is a single point.
let mut opt = DifferentialEvolution::new(
DifferentialEvolutionConfig {
population_size: 4,
generations: 3,
differential_weight: 0.5,
crossover_probability: 0.9,
seed: 0,
},
RealBounds::new(vec![(0.5, 0.5); 2]),
);
let r = opt.run(&ConstantFn);
let best = r.best.unwrap();
// Every decision must be exactly (0.5, 0.5).
for d in &r.population.candidates {
for &v in &d.decision {
assert_eq!(v, 0.5);
}
}
let _ = best;
}
tests/operator_properties.rs
+277
View File
@@ -0,0 +1,277 @@
//! Per-operator property tests covering every Variation / Initializer /
//! Repair impl heuropt ships.
use proptest::prelude::*;
use heuropt::core::rng::rng_from_seed;
use heuropt::prelude::*;
/// Generate per-axis bounds whose width is at least 0.001 (avoid the
/// degenerate `lo == hi` case for properties that need a proper interval).
fn bounds(dim: usize) -> impl Strategy<Value = Vec<(f64, f64)>> {
prop::collection::vec((-50.0_f64..50.0, 0.001_f64..50.0), dim..=dim).prop_map(|pairs| {
pairs
.into_iter()
.map(|(lo, span)| (lo, lo + span))
.collect()
})
}
/// Generate a parent vector inside the given bounds.
fn parent_in_bounds(bounds: &[(f64, f64)]) -> Vec<f64> {
bounds.iter().map(|&(lo, hi)| 0.5 * (lo + hi)).collect()
}
// -----------------------------------------------------------------------------
// Initializers
// -----------------------------------------------------------------------------
proptest! {
#[test]
fn real_bounds_returns_correct_shape(
bounds in bounds(4),
size in 1usize..30,
seed in any::<u64>(),
) {
let mut rng = rng_from_seed(seed);
let mut init = RealBounds::new(bounds.clone());
let decisions = init.initialize(size, &mut rng);
prop_assert_eq!(decisions.len(), size);
for d in &decisions {
prop_assert_eq!(d.len(), 4);
for (j, &v) in d.iter().enumerate() {
let (lo, hi) = bounds[j];
prop_assert!(v >= lo && v <= hi, "{v} out of [{lo}, {hi}]");
}
}
}
#[test]
fn real_bounds_size_zero_returns_empty(
bounds in bounds(3),
seed in any::<u64>(),
) {
let mut rng = rng_from_seed(seed);
let mut init = RealBounds::new(bounds);
let decisions = init.initialize(0, &mut rng);
prop_assert!(decisions.is_empty());
}
}
// -----------------------------------------------------------------------------
// Real-valued Variation operators
// -----------------------------------------------------------------------------
proptest! {
#[test]
fn gaussian_mutation_preserves_length(
sigma in 1e-6_f64..5.0,
len in 1usize..10,
seed in any::<u64>(),
) {
let mut rng = rng_from_seed(seed);
let parent: Vec<f64> = vec![0.0; len];
let mut m = GaussianMutation { sigma };
let children = m.vary(std::slice::from_ref(&parent), &mut rng);
prop_assert_eq!(children.len(), 1);
prop_assert_eq!(children[0].len(), len);
}
#[test]
fn bounded_gaussian_mutation_in_bounds(
sigma in 1e-6_f64..5.0,
bounds in bounds(4),
seed in any::<u64>(),
) {
let mut rng = rng_from_seed(seed);
let parent = parent_in_bounds(&bounds);
let mut m = BoundedGaussianMutation::new(sigma, bounds.clone());
let children = m.vary(std::slice::from_ref(&parent), &mut rng);
prop_assert_eq!(children.len(), 1);
for (j, &v) in children[0].iter().enumerate() {
let (lo, hi) = bounds[j];
prop_assert!(v >= lo && v <= hi);
}
}
#[test]
fn bit_flip_mutation_preserves_length(
probability in 0.0_f64..=1.0,
len in 1usize..32,
seed in any::<u64>(),
) {
let mut rng = rng_from_seed(seed);
let parent: Vec<bool> = (0..len).map(|i| i % 2 == 0).collect();
let mut m = BitFlipMutation { probability };
let children = m.vary(std::slice::from_ref(&parent), &mut rng);
prop_assert_eq!(children.len(), 1);
prop_assert_eq!(children[0].len(), len);
}
#[test]
fn swap_mutation_is_a_permutation(
len in 2usize..16,
seed in any::<u64>(),
) {
let mut rng = rng_from_seed(seed);
let parent: Vec<usize> = (0..len).collect();
let mut m = SwapMutation;
let children = m.vary(std::slice::from_ref(&parent), &mut rng);
prop_assert_eq!(children.len(), 1);
let mut sorted = children[0].clone();
sorted.sort();
let identity: Vec<usize> = (0..len).collect();
prop_assert_eq!(sorted, identity);
}
#[test]
fn sbx_in_bounds(
bounds in bounds(3),
eta in 1.0_f64..30.0,
per_var_p in 0.0_f64..=1.0,
a_frac in 0.0_f64..1.0,
b_frac in 0.0_f64..1.0,
seed in any::<u64>(),
) {
let mut rng = rng_from_seed(seed);
let p1: Vec<f64> = bounds.iter().map(|&(lo, hi)| lo + a_frac * (hi - lo)).collect();
let p2: Vec<f64> = bounds.iter().map(|&(lo, hi)| lo + b_frac * (hi - lo)).collect();
let mut sbx = SimulatedBinaryCrossover::new(bounds.clone(), eta, per_var_p);
let children = sbx.vary(&[p1, p2], &mut rng);
prop_assert_eq!(children.len(), 2);
for c in &children {
for (j, &v) in c.iter().enumerate() {
let (lo, hi) = bounds[j];
prop_assert!(v >= lo && v <= hi);
}
}
}
#[test]
fn polymut_in_bounds(
bounds in bounds(3),
eta in 1.0_f64..40.0,
per_var_p in 0.0_f64..=1.0,
seed in any::<u64>(),
) {
let mut rng = rng_from_seed(seed);
let parent = parent_in_bounds(&bounds);
let mut pm = PolynomialMutation::new(bounds.clone(), eta, per_var_p);
let children = pm.vary(std::slice::from_ref(&parent), &mut rng);
prop_assert_eq!(children.len(), 1);
for (j, &v) in children[0].iter().enumerate() {
let (lo, hi) = bounds[j];
prop_assert!(v >= lo && v <= hi);
}
}
#[test]
fn levy_mutation_in_bounds(
bounds in bounds(3),
alpha in 0.5_f64..2.0,
scale in 0.01_f64..1.0,
seed in any::<u64>(),
) {
let mut rng = rng_from_seed(seed);
let parent = parent_in_bounds(&bounds);
let mut m = LevyMutation::new(alpha, scale, bounds.clone());
let children = m.vary(std::slice::from_ref(&parent), &mut rng);
prop_assert_eq!(children.len(), 1);
for (j, &v) in children[0].iter().enumerate() {
let (lo, hi) = bounds[j];
prop_assert!(v >= lo && v <= hi);
}
}
#[test]
fn composite_variation_preserves_count(
bounds in bounds(3),
a_frac in 0.0_f64..1.0,
b_frac in 0.0_f64..1.0,
seed in any::<u64>(),
) {
let mut rng = rng_from_seed(seed);
let p1: Vec<f64> = bounds.iter().map(|&(lo, hi)| lo + a_frac * (hi - lo)).collect();
let p2: Vec<f64> = bounds.iter().map(|&(lo, hi)| lo + b_frac * (hi - lo)).collect();
// SBX produces 2 children, PolyMut produces 1 each → expect 2.
let mut v = CompositeVariation {
crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5),
mutation: PolynomialMutation::new(bounds, 20.0, 0.5),
};
let children = v.vary(&[p1, p2], &mut rng);
prop_assert_eq!(children.len(), 2);
}
}
// -----------------------------------------------------------------------------
// Repair operators
// -----------------------------------------------------------------------------
proptest! {
#[test]
fn clamp_to_bounds_lands_in_bounds(
bounds in bounds(5),
seed in any::<u64>(),
) {
use rand::Rng as _;
let mut rng = rng_from_seed(seed);
let mut x: Vec<f64> = (0..5).map(|_| rng.random_range(-1000.0..=1000.0)).collect();
let mut r = ClampToBounds::new(bounds.clone());
r.repair(&mut x);
for (j, &v) in x.iter().enumerate() {
let (lo, hi) = bounds[j];
prop_assert!(v >= lo && v <= hi);
}
}
#[test]
fn clamp_to_bounds_idempotent(
bounds in bounds(5),
seed in any::<u64>(),
) {
use rand::Rng as _;
let mut rng = rng_from_seed(seed);
let mut x: Vec<f64> = (0..5).map(|_| rng.random_range(-1000.0..=1000.0)).collect();
let mut r = ClampToBounds::new(bounds);
r.repair(&mut x);
let after_one = x.clone();
r.repair(&mut x);
prop_assert_eq!(x, after_one);
}
#[test]
fn project_to_simplex_lands_in_simplex(
n in 2usize..8,
total in 0.5_f64..10.0,
seed in any::<u64>(),
) {
use rand::Rng as _;
let mut rng = rng_from_seed(seed);
let mut x: Vec<f64> = (0..n).map(|_| rng.random_range(-5.0..5.0)).collect();
let mut r = ProjectToSimplex::new(total);
r.repair(&mut x);
for &v in &x {
prop_assert!(v >= 0.0);
}
let s: f64 = x.iter().sum();
prop_assert!((s - total).abs() < 1e-9);
}
#[test]
fn project_to_simplex_idempotent(
n in 2usize..8,
total in 0.5_f64..5.0,
seed in any::<u64>(),
) {
use rand::Rng as _;
let mut rng = rng_from_seed(seed);
let mut x: Vec<f64> = (0..n).map(|_| rng.random_range(-5.0..5.0)).collect();
let mut r = ProjectToSimplex::new(total);
r.repair(&mut x);
let after_one = x.clone();
r.repair(&mut x);
for (a, b) in after_one.iter().zip(x.iter()) {
prop_assert!((a - b).abs() < 1e-9);
}
}
}
tests/properties.proptest-regressions
+7
View File
@@ -0,0 +1,7 @@
# Seeds for failure cases proptest has generated in the past. It is
# automatically read and these particular cases re-run before any
# novel cases are generated.
#
# It is recommended to check this file in to source control so that
# everyone who runs the test benefits from these saved cases.
cc acfcf5a06625012c4adedfcd5213ab619837031ad52382c69d7a1ee409cc934b # shrinks to bounds = [(0.0, 0.001), (0.0, 0.001), (0.0, 0.001)], eta = 1.0, per_var_p = 0.0, seed = 0
tests/properties.rs
+276
View File
@@ -0,0 +1,276 @@
//! Property-based tests for heuropt invariants.
//!
//! Where the unit-test suite checks specific cases, this suite checks
//! invariants that should hold for *any* well-formed input. proptest
//! generates random instances and shrinks failures.
use proptest::prelude::*;
use heuropt::core::candidate::Candidate;
use heuropt::core::evaluation::Evaluation;
use heuropt::core::objective::{Objective, ObjectiveSpace};
use heuropt::pareto::dominance::{Dominance, pareto_compare};
use heuropt::pareto::front::pareto_front;
use heuropt::pareto::sort::non_dominated_sort;
use heuropt::prelude::*;
// -----------------------------------------------------------------------------
// Strategies
// -----------------------------------------------------------------------------
/// Generate a 2-objective minimize ObjectiveSpace.
fn space_2d() -> ObjectiveSpace {
ObjectiveSpace::new(vec![Objective::minimize("f1"), Objective::minimize("f2")])
}
/// Generate a candidate with a 2-D objective vector in `[lo, hi]`.
fn candidate_2d(lo: f64, hi: f64) -> impl Strategy<Value = Candidate<()>> {
(lo..hi, lo..hi).prop_map(|(a, b)| Candidate::new((), Evaluation::new(vec![a, b])))
}
/// Generate a small 2-D population.
fn population_2d() -> impl Strategy<Value = Vec<Candidate<()>>> {
prop::collection::vec(candidate_2d(-100.0, 100.0), 1..=15)
}
/// Generate per-axis bounds.
fn bounds(dim: usize) -> impl Strategy<Value = Vec<(f64, f64)>> {
prop::collection::vec((-50.0_f64..50.0, 0.001_f64..50.0), dim..=dim).prop_map(|pairs| {
pairs
.into_iter()
.map(|(lo, span)| (lo, lo + span))
.collect()
})
}
// -----------------------------------------------------------------------------
// Pareto invariants
// -----------------------------------------------------------------------------
proptest! {
/// `pareto_compare` is anti-symmetric on Dominates / DominatedBy.
#[test]
fn pareto_compare_is_antisymmetric(
a in candidate_2d(-100.0, 100.0),
b in candidate_2d(-100.0, 100.0),
) {
let s = space_2d();
let ab = pareto_compare(&a.evaluation, &b.evaluation, &s);
let ba = pareto_compare(&b.evaluation, &a.evaluation, &s);
match (ab, ba) {
(Dominance::Dominates, Dominance::DominatedBy) => {}
(Dominance::DominatedBy, Dominance::Dominates) => {}
(Dominance::Equal, Dominance::Equal) => {}
(Dominance::NonDominated, Dominance::NonDominated) => {}
(l, r) => prop_assert!(false, "asymmetric result: ab={l:?}, ba={r:?}"),
}
}
/// Comparing a candidate with itself returns Equal.
#[test]
fn pareto_compare_reflexive(a in candidate_2d(-100.0, 100.0)) {
let s = space_2d();
let r = pareto_compare(&a.evaluation, &a.evaluation, &s);
prop_assert_eq!(r, Dominance::Equal);
}
/// `pareto_front` output members are pairwise non-dominated.
#[test]
fn pareto_front_is_internally_nondominated(pop in population_2d()) {
let s = space_2d();
let front = pareto_front(&pop, &s);
for i in 0..front.len() {
for j in 0..front.len() {
if i == j {
continue;
}
let r = pareto_compare(&front[i].evaluation, &front[j].evaluation, &s);
prop_assert!(
!matches!(r, Dominance::DominatedBy),
"front member {i} dominated by {j}",
);
}
}
}
/// `non_dominated_sort` partitions every population member into exactly
/// one front (no missing or duplicate indices).
#[test]
fn non_dominated_sort_partitions_population(pop in population_2d()) {
let s = space_2d();
let fronts = non_dominated_sort(&pop, &s);
let mut seen = vec![false; pop.len()];
for front in &fronts {
for &idx in front {
prop_assert!(!seen[idx], "index {idx} appears in multiple fronts");
seen[idx] = true;
}
}
for (i, &was_seen) in seen.iter().enumerate() {
prop_assert!(was_seen, "index {i} is missing from all fronts");
}
}
}
// -----------------------------------------------------------------------------
// Operator invariants
// -----------------------------------------------------------------------------
proptest! {
/// SBX returns exactly 2 children, both in bounds when parents are in
/// bounds. (SBX only clamps variables it actually mixes — when
/// `per_variable_probability < 1` the rest pass through from the
/// parents, so out-of-bounds parents would yield out-of-bounds children
/// by design. We're checking the in-bounds-parent contract.)
#[test]
fn sbx_children_in_bounds_when_parents_in_bounds(
bounds in bounds(3),
eta in 1.0_f64..30.0,
per_var_p in 0.0_f64..=1.0,
a_frac in 0.0_f64..1.0,
b_frac in 0.0_f64..1.0,
seed in any::<u64>(),
) {
let mut rng = rng_from_seed(seed);
// Parents are convex combinations of bounds — strictly in box.
let p1: Vec<f64> = bounds.iter().map(|&(lo, hi)| lo + a_frac * (hi - lo)).collect();
let p2: Vec<f64> = bounds.iter().map(|&(lo, hi)| lo + b_frac * (hi - lo)).collect();
let mut sbx = SimulatedBinaryCrossover::new(bounds.clone(), eta, per_var_p);
let children = sbx.vary(&[p1, p2], &mut rng);
prop_assert_eq!(children.len(), 2);
for c in &children {
prop_assert_eq!(c.len(), 3);
for (j, &v) in c.iter().enumerate() {
let (lo, hi) = bounds[j];
prop_assert!(v >= lo && v <= hi, "SBX child[{j}] = {v} out of [{lo}, {hi}]");
}
}
}
/// PolynomialMutation returns 1 child in bounds.
#[test]
fn polymut_child_in_bounds(
bounds in bounds(4),
eta in 1.0_f64..40.0,
per_var_p in 0.0_f64..=1.0,
seed in any::<u64>(),
) {
let mut rng = rng_from_seed(seed);
let parent: Vec<f64> = bounds.iter().map(|&(lo, hi)| 0.5 * (lo + hi)).collect();
let mut pm = PolynomialMutation::new(bounds.clone(), eta, per_var_p);
let children = pm.vary(std::slice::from_ref(&parent), &mut rng);
prop_assert_eq!(children.len(), 1);
for (j, &v) in children[0].iter().enumerate() {
let (lo, hi) = bounds[j];
prop_assert!(v >= lo && v <= hi);
}
}
/// ClampToBounds always lands every variable in bounds.
#[test]
fn clamp_to_bounds_lands_in_bounds(
bounds in bounds(5),
x_seed in any::<u64>(),
) {
// Sample a "before-repair" vector that may be wildly out of bounds.
let mut rng = rng_from_seed(x_seed);
use rand::Rng as _;
let mut x: Vec<f64> = (0..5).map(|_| rng.random_range(-1000.0..=1000.0)).collect();
let mut r = ClampToBounds::new(bounds.clone());
r.repair(&mut x);
for (j, &v) in x.iter().enumerate() {
let (lo, hi) = bounds[j];
prop_assert!(v >= lo && v <= hi);
}
}
/// ProjectToSimplex always lands in the simplex { x ≥ 0, Σ x = total }.
#[test]
fn project_to_simplex_lands_in_simplex(
n in 2usize..8,
total in 0.5_f64..10.0,
seed in any::<u64>(),
) {
let mut rng = rng_from_seed(seed);
use rand::Rng as _;
let mut x: Vec<f64> = (0..n).map(|_| rng.random_range(-5.0..5.0)).collect();
let mut r = ProjectToSimplex::new(total);
r.repair(&mut x);
for &v in &x {
prop_assert!(v >= 0.0, "negative entry: {v}");
}
let s: f64 = x.iter().sum();
prop_assert!(
(s - total).abs() < 1e-9,
"sum {s} != total {total}",
);
}
}
// -----------------------------------------------------------------------------
// Optimizer determinism
// -----------------------------------------------------------------------------
/// Tiny single-objective sphere problem reused across determinism props.
struct Sphere1D;
impl Problem for Sphere1D {
type Decision = Vec<f64>;
fn objectives(&self) -> ObjectiveSpace {
ObjectiveSpace::new(vec![Objective::minimize("f")])
}
fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
Evaluation::new(vec![x[0] * x[0]])
}
}
proptest! {
#[test]
fn de_deterministic_with_seed(seed in any::<u64>()) {
let mut a = DifferentialEvolution::new(
DifferentialEvolutionConfig {
population_size: 10,
generations: 5,
differential_weight: 0.5,
crossover_probability: 0.9,
seed,
},
RealBounds::new(vec![(-3.0, 3.0)]),
);
let mut b = DifferentialEvolution::new(
DifferentialEvolutionConfig {
population_size: 10,
generations: 5,
differential_weight: 0.5,
crossover_probability: 0.9,
seed,
},
RealBounds::new(vec![(-3.0, 3.0)]),
);
let ra = a.run(&Sphere1D);
let rb = b.run(&Sphere1D);
prop_assert_eq!(
ra.best.unwrap().evaluation.objectives,
rb.best.unwrap().evaluation.objectives,
);
}
#[test]
fn cmaes_deterministic_with_seed(seed in any::<u64>()) {
let cfg = CmaEsConfig {
population_size: 8,
generations: 5,
initial_sigma: 0.5,
eigen_decomposition_period: 1,
initial_mean: None,
seed,
};
let mut a = CmaEs::new(cfg.clone(), RealBounds::new(vec![(-3.0, 3.0)]));
let mut b = CmaEs::new(cfg, RealBounds::new(vec![(-3.0, 3.0)]));
let ra = a.run(&Sphere1D);
let rb = b.run(&Sphere1D);
prop_assert_eq!(
ra.best.unwrap().evaluation.objectives,
rb.best.unwrap().evaluation.objectives,
);
}
}