feat: v0.5.0 — comprehensive documentation release
Theme: documentation and project polish. No public-API changes; this is the v0.5 release that elevates heuropt's docs/onboarding/governance to bar-setting status. Adds: - mdbook user guide at docs/book/ with intro, getting-started, defining-problems, choosing-an-algorithm, cookbook (7 recipes), comparison vs other libraries, stability/SemVer, migration guides. Deploys to https://swaits.github.io/heuropt/ via .github/workflows/ docs.yml. - Runnable rustdoc examples on every algorithm (35 of them), all exercised by cargo test --doc. - Three real-world examples: portfolio.rs (multi-obj with budget constraint), hyperparam_tuning.rs (BO + TPE), scheduling.rs (permutation via SA + SwapMutation against Smith's-rule oracle). - Governance: CONTRIBUTING.md, SECURITY.md, CODE_OF_CONDUCT.md (adopting builderscode.org's Builder's Code of Conduct), GitHub issue templates, PR template. Polishes: - README hero with badges + user-guide link. - lib.rs crate-level docs. - CHANGELOG entry for 0.5.0. Bumps Cargo.toml to 0.5.0.
This commit is contained in:
@@ -0,0 +1,34 @@
|
||||
[book]
|
||||
title = "heuropt — the user guide"
|
||||
description = "A practical Rust toolkit for heuristic single-, multi-, and many-objective optimization."
|
||||
authors = ["Stephen Waits"]
|
||||
language = "en"
|
||||
src = "src"
|
||||
|
||||
[build]
|
||||
build-dir = "../../target/book"
|
||||
create-missing = false
|
||||
|
||||
[output.html]
|
||||
default-theme = "rust"
|
||||
preferred-dark-theme = "navy"
|
||||
git-repository-url = "https://github.com/swaits/heuropt"
|
||||
edit-url-template = "https://github.com/swaits/heuropt/edit/main/docs/book/{path}"
|
||||
site-url = "/heuropt/"
|
||||
no-section-label = true
|
||||
|
||||
[output.html.fold]
|
||||
enable = true
|
||||
level = 1
|
||||
|
||||
[output.html.search]
|
||||
enable = true
|
||||
limit-results = 30
|
||||
teaser-word-count = 30
|
||||
use-boolean-and = true
|
||||
|
||||
[output.html.print]
|
||||
enable = true
|
||||
|
||||
[rust]
|
||||
edition = "2024"
|
||||
@@ -0,0 +1,26 @@
|
||||
# Summary
|
||||
|
||||
[Introduction](./introduction.md)
|
||||
|
||||
# Getting started
|
||||
|
||||
- [Five-minute walkthrough](./getting-started.md)
|
||||
- [Defining a problem](./defining-problems.md)
|
||||
- [Choosing an algorithm](./choosing-an-algorithm.md)
|
||||
|
||||
# Cookbook
|
||||
|
||||
- [Recipes](./cookbook.md)
|
||||
- [Parallelize evaluation with rayon](./cookbook/parallel.md)
|
||||
- [Tune a model with expensive evaluations](./cookbook/expensive-evaluations.md)
|
||||
- [Compare two algorithms on your problem](./cookbook/compare.md)
|
||||
- [Optimize a permutation (TSP-style)](./cookbook/permutation.md)
|
||||
- [Constrain your search with `Repair`](./cookbook/constraints.md)
|
||||
- [Pick one answer off a Pareto front](./cookbook/pick-one.md)
|
||||
- [Write your own algorithm](./cookbook/custom-optimizer.md)
|
||||
|
||||
# Reference
|
||||
|
||||
- [Comparison with other libraries](./comparison.md)
|
||||
- [Stability and SemVer](./stability.md)
|
||||
- [Migration guides](./migration.md)
|
||||
@@ -0,0 +1,285 @@
|
||||
# Choosing an algorithm
|
||||
|
||||
The README has a compact decision tree. This chapter expands it with
|
||||
the *reasoning* behind each branch.
|
||||
|
||||
## Step 0: How expensive is one evaluation?
|
||||
|
||||
This is the first fork because it changes everything that comes
|
||||
after it.
|
||||
|
||||
| Eval cost | Budget you can afford | Algorithm family |
|
||||
|----------------------------|---------------------------|-----------------------------|
|
||||
| Microseconds (pure math) | 10 000 – 1 000 000 evals | Population-based |
|
||||
| Milliseconds (sim, IO) | 1 000 – 10 000 evals | Population-based |
|
||||
| Seconds (small training) | 100 – 1 000 evals | Sample-efficient (BO, TPE) |
|
||||
| Minutes+ (full training) | 50 – 500 evals | Sample-efficient + multi-fidelity |
|
||||
|
||||
For the cheap-eval branch, you have the run of the catalog. For the
|
||||
expensive branch, classical evolutionary methods waste your evaluation
|
||||
budget — go to [`BayesianOpt`] or [`Tpe`]. For the *very* expensive
|
||||
branch where each eval has a tunable budget (epochs, MC samples, sim
|
||||
steps), [`Hyperband`] over the [`PartialProblem`] trait is the move.
|
||||
|
||||
## Step 1: How many objectives?
|
||||
|
||||
The biggest fork.
|
||||
|
||||
- **One** — there's a single best answer. Pick from the
|
||||
single-objective branch.
|
||||
- **Two or three** — a Pareto front. Pick from the multi-objective
|
||||
branch.
|
||||
- **Four or more** — a many-objective Pareto front; classical
|
||||
multi-objective methods break down here because almost every pair
|
||||
of points is non-dominated. Pick from the many-objective branch.
|
||||
|
||||
> **Pareto front:** the set of decisions where you cannot improve any
|
||||
> objective without sacrificing another. In a 2-objective minimize
|
||||
> problem, plot every solution; the Pareto front is the lower-left
|
||||
> envelope.
|
||||
|
||||
If you found yourself staring at a single composite score that's a
|
||||
weighted sum of conflicting goals, you probably have a multi-objective
|
||||
problem in disguise. A weighted sum bakes in your preferences before
|
||||
you've seen the trade-off; running a multi-objective optimizer first
|
||||
and picking off the front later is almost always a better workflow
|
||||
(see [Pick one answer off a Pareto front](./cookbook/pick-one.md)).
|
||||
|
||||
## Step 2 — single-objective continuous
|
||||
|
||||
These all take `Vec<f64>` decisions.
|
||||
|
||||
### Smooth, low-to-moderate dimension
|
||||
|
||||
[`CmaEs`] is the strong default. It adapts the search distribution's
|
||||
covariance to the local landscape. On the comparison harness it
|
||||
hits machine epsilon on Rosenbrock at 30 000 evaluations.
|
||||
|
||||
For very low-dimensional smooth problems (≤ 5 dim), [`NelderMead`] is
|
||||
deterministic and converges to f = 0 exactly on Rosenbrock.
|
||||
|
||||
### High dimension, smooth
|
||||
|
||||
[`SeparableNes`] uses a diagonal covariance — cheaper per step than
|
||||
CmaEs at the cost of being unable to model rotated landscapes. Worth
|
||||
trying when CmaEs's `O(d²)` per-step cost hurts.
|
||||
|
||||
### Multimodal landscapes
|
||||
|
||||
Multimodal = many local minima that aren't the global one. Rastrigin
|
||||
and Ackley are classic traps.
|
||||
|
||||
[`IpopCmaEs`] is CmaEs with an increasing-population restart strategy
|
||||
specifically designed for this. On the harness it drops vanilla CmaEs's
|
||||
Rastrigin score from f = 2.35 to f = 0.13.
|
||||
|
||||
[`DifferentialEvolution`] is rarely beaten on cheap multimodal
|
||||
continuous problems. On Rastrigin it ties with `(1+1)-ES` at f = 0.
|
||||
|
||||
[`SimulatedAnnealing`] is a cheap, generic baseline that escapes local
|
||||
optima via temperature decay.
|
||||
|
||||
### Want parameter-free
|
||||
|
||||
[`Tlbo`] (Teaching-Learning-Based Optimization) has no `F`, `CR`, `w`,
|
||||
or `σ` to tune. Often a respectable middle-of-the-pack performer.
|
||||
|
||||
### Smallest possible self-adapting baseline
|
||||
|
||||
[`OnePlusOneEs`] — Rechenberg's 1973 `(1+1)`-ES with the one-fifth
|
||||
success rule. On the harness it hits f = 0 on Rastrigin in 50 000
|
||||
evaluations.
|
||||
|
||||
### Just want a baseline
|
||||
|
||||
[`RandomSearch`]. Useful as a sanity check: if your fancy optimizer
|
||||
can't beat random search, something is wrong (with the fancy
|
||||
optimizer or with the problem).
|
||||
|
||||
## Step 2 — single-objective other types
|
||||
|
||||
| Decision type | Algorithm | Notes |
|
||||
|---|---|---|
|
||||
| `Vec<bool>` | [`Umda`] | Per-bit marginal EDA. Independent-bit assumption. |
|
||||
| `Vec<bool>` | [`GeneticAlgorithm`] + [`BitFlipMutation`] | When bit interactions matter. |
|
||||
| `Vec<usize>` (permutation) | [`AntColonyTsp`] | TSP-style with a distance matrix. |
|
||||
| `Vec<usize>` (permutation) | [`SimulatedAnnealing`] + [`SwapMutation`] | Generic discrete baseline. |
|
||||
| `Vec<usize>` or custom | [`TabuSearch`] | You supply the neighbor function. |
|
||||
| Custom struct | [`SimulatedAnnealing`] / [`HillClimber`] | With your own `Variation` impl. |
|
||||
|
||||
## Step 2 — multi-objective (2 or 3)
|
||||
|
||||
### Strong default
|
||||
|
||||
[`Nsga2`] is the canonical Pareto-based EA. Fast, well-understood,
|
||||
maintains diversity via crowding distance. On the harness it lands
|
||||
on the Pareto front of every test problem.
|
||||
|
||||
### Real-valued, smooth front, want best convergence
|
||||
|
||||
[`Mopso`] (multi-objective PSO with archive). On ZDT1 it wins
|
||||
hypervolume outright and converges 100× tighter than the
|
||||
dominance-based methods.
|
||||
|
||||
### Better front quality than NSGA-II
|
||||
|
||||
[`Ibea`] (indicator-based) is consistently the best of the
|
||||
dominance-based methods on the harness — wins ZDT3 hypervolume and
|
||||
DTLZ2 mean distance by 24×. It uses an additive ε-indicator for
|
||||
selection rather than dominance + crowding.
|
||||
|
||||
[`Spea2`] (strength + density) — solid alternative; explicit external
|
||||
archive separate from the population.
|
||||
|
||||
[`SmsEmoa`] uses exact hypervolume contribution for selection. Elegant
|
||||
in theory; in practice on the harness budgets here it underperforms
|
||||
NSGA-II. Worth the higher per-step cost only when exact HV
|
||||
contribution is the right discriminator.
|
||||
|
||||
### Decomposition / weight-vector style
|
||||
|
||||
[`Moead`] decomposes the multi-objective problem into many scalar
|
||||
sub-problems (Tchebycheff or weighted sum) and solves them in
|
||||
parallel. Very fast per generation; scales naturally to many
|
||||
objectives.
|
||||
|
||||
### Disconnected or non-convex front
|
||||
|
||||
[`AgeMoea`] estimates the front geometry adaptively (the L_p
|
||||
parameter `p` is fit from data each generation).
|
||||
|
||||
[`Knea`] favors knee points — the regions of the front where small
|
||||
gains in one objective cost large losses in another.
|
||||
|
||||
[`Ibea`] also handles disconnected fronts well.
|
||||
|
||||
### Region-based diversity
|
||||
|
||||
[`PesaII`] uses grid hyperboxes to drive selection — divide the
|
||||
objective space into a grid, pick from the least-crowded boxes.
|
||||
|
||||
[`EpsilonMoea`] uses an ε-grid archive that auto-limits its size.
|
||||
|
||||
### Just one starting decision (no population budget)
|
||||
|
||||
[`Paes`] — `(1+1)`-ES with a Pareto archive. Cheap, simple, useful
|
||||
when your evaluations are expensive enough that you can't afford a
|
||||
population.
|
||||
|
||||
## Step 2 — many-objective (4+)
|
||||
|
||||
### Linear / simplex-shaped front (e.g., DTLZ1)
|
||||
|
||||
[`Grea`] — grid coords drive ranking. On DTLZ1 it beats NSGA-III by
|
||||
3× and AGE-MOEA by 2.5×.
|
||||
|
||||
[`Moead`] — decomposition shines on linear fronts; second on DTLZ1
|
||||
and among the fastest per generation.
|
||||
|
||||
### Curved / unknown front geometry
|
||||
|
||||
[`Nsga3`] — reference-point niching; canonical many-objective method;
|
||||
strong default when the front isn't simplex-shaped.
|
||||
|
||||
[`AgeMoea`] — estimates L_p geometry per generation.
|
||||
|
||||
[`Rvea`] — reference vectors with adaptive penalty.
|
||||
|
||||
### Indicator-based selection
|
||||
|
||||
[`Ibea`] — additive ε-indicator; doesn't degrade at high obj count.
|
||||
|
||||
[`HypE`] — Monte Carlo hypervolume estimation; scales to arbitrary
|
||||
objective count where exact HV is too expensive.
|
||||
|
||||
## Step 3: Are there hard constraints?
|
||||
|
||||
heuropt models constraints as a single scalar `constraint_violation`
|
||||
on each `Evaluation`. Three escalations when the feasibility region
|
||||
is hard to find:
|
||||
|
||||
1. **Penalty-only.** Just set `constraint_violation > 0` for
|
||||
infeasible decisions. The default tournament/Pareto comparisons
|
||||
prefer feasibles automatically.
|
||||
2. **Repair.** Implement [`Repair<D>`] (or use the provided
|
||||
[`ClampToBounds`] / [`ProjectToSimplex`]) to project infeasible
|
||||
decisions back into the feasible region. Pair with a `Variation`
|
||||
in a [`CompositeVariation`] for bounds-aware variants.
|
||||
3. **Stochastic ranking.** Use [`stochastic_ranking_select`] instead
|
||||
of `tournament_select_single_objective`. It probabilistically
|
||||
explores near-feasibility instead of strict feasibility-first
|
||||
ordering, which helps when feasible regions are narrow.
|
||||
|
||||
See [Constrain your search with `Repair`](./cookbook/constraints.md)
|
||||
for worked examples.
|
||||
|
||||
## Step 4: Should you parallelize?
|
||||
|
||||
Enable the `parallel` feature flag if your `evaluate` takes more
|
||||
than ~50 µs. Population-based algorithms ([`RandomSearch`], [`Nsga2`],
|
||||
[`DifferentialEvolution`], [`Spea2`], [`Ibea`], [`Mopso`], …) batch-
|
||||
evaluate via rayon when the feature is on. **Seeded runs stay
|
||||
bit-identical** to serial mode.
|
||||
|
||||
```toml
|
||||
heuropt = { version = "0.5", features = ["parallel"] }
|
||||
```
|
||||
|
||||
## TL;DR table
|
||||
|
||||
| Situation | Pick |
|
||||
|---|---|
|
||||
| Smooth single-objective continuous | [`CmaEs`] |
|
||||
| Multimodal single-objective continuous | [`IpopCmaEs`] or [`DifferentialEvolution`] |
|
||||
| Expensive single-objective | [`BayesianOpt`] or [`Tpe`] |
|
||||
| Multi-fidelity single-objective | [`Hyperband`] |
|
||||
| 2- or 3-objective default | [`Nsga2`] |
|
||||
| 2-objective real-valued smooth front | [`Mopso`] |
|
||||
| Disconnected / non-convex front | [`Ibea`] |
|
||||
| Many-objective default (curved front) | [`Nsga3`] |
|
||||
| Many-objective linear / simplex front | [`Grea`] |
|
||||
| Permutation problem | [`AntColonyTsp`] |
|
||||
| Binary problem | [`Umda`] |
|
||||
| Custom decision type | [`SimulatedAnnealing`] + your `Variation` |
|
||||
| Sanity baseline | [`RandomSearch`] |
|
||||
|
||||
[`CmaEs`]: https://docs.rs/heuropt/latest/heuropt/algorithms/cma_es/struct.CmaEs.html
|
||||
[`IpopCmaEs`]: https://docs.rs/heuropt/latest/heuropt/algorithms/ipop_cma_es/struct.IpopCmaEs.html
|
||||
[`SeparableNes`]: https://docs.rs/heuropt/latest/heuropt/algorithms/snes/struct.SeparableNes.html
|
||||
[`NelderMead`]: https://docs.rs/heuropt/latest/heuropt/algorithms/nelder_mead/struct.NelderMead.html
|
||||
[`DifferentialEvolution`]: https://docs.rs/heuropt/latest/heuropt/algorithms/differential_evolution/struct.DifferentialEvolution.html
|
||||
[`SimulatedAnnealing`]: https://docs.rs/heuropt/latest/heuropt/algorithms/simulated_annealing/struct.SimulatedAnnealing.html
|
||||
[`Tlbo`]: https://docs.rs/heuropt/latest/heuropt/algorithms/tlbo/struct.Tlbo.html
|
||||
[`OnePlusOneEs`]: https://docs.rs/heuropt/latest/heuropt/algorithms/one_plus_one_es/struct.OnePlusOneEs.html
|
||||
[`RandomSearch`]: https://docs.rs/heuropt/latest/heuropt/algorithms/random_search/struct.RandomSearch.html
|
||||
[`HillClimber`]: https://docs.rs/heuropt/latest/heuropt/algorithms/hill_climber/struct.HillClimber.html
|
||||
[`BayesianOpt`]: https://docs.rs/heuropt/latest/heuropt/algorithms/bayesian_opt/struct.BayesianOpt.html
|
||||
[`Tpe`]: https://docs.rs/heuropt/latest/heuropt/algorithms/tpe/struct.Tpe.html
|
||||
[`Hyperband`]: https://docs.rs/heuropt/latest/heuropt/algorithms/hyperband/struct.Hyperband.html
|
||||
[`PartialProblem`]: https://docs.rs/heuropt/latest/heuropt/core/partial_problem/trait.PartialProblem.html
|
||||
[`Umda`]: https://docs.rs/heuropt/latest/heuropt/algorithms/umda/struct.Umda.html
|
||||
[`GeneticAlgorithm`]: https://docs.rs/heuropt/latest/heuropt/algorithms/genetic_algorithm/struct.GeneticAlgorithm.html
|
||||
[`BitFlipMutation`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.BitFlipMutation.html
|
||||
[`AntColonyTsp`]: https://docs.rs/heuropt/latest/heuropt/algorithms/ant_colony_tsp/struct.AntColonyTsp.html
|
||||
[`SwapMutation`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.SwapMutation.html
|
||||
[`TabuSearch`]: https://docs.rs/heuropt/latest/heuropt/algorithms/tabu_search/struct.TabuSearch.html
|
||||
[`Nsga2`]: https://docs.rs/heuropt/latest/heuropt/algorithms/nsga2/struct.Nsga2.html
|
||||
[`Nsga3`]: https://docs.rs/heuropt/latest/heuropt/algorithms/nsga3/struct.Nsga3.html
|
||||
[`Mopso`]: https://docs.rs/heuropt/latest/heuropt/algorithms/mopso/struct.Mopso.html
|
||||
[`Ibea`]: https://docs.rs/heuropt/latest/heuropt/algorithms/ibea/struct.Ibea.html
|
||||
[`Spea2`]: https://docs.rs/heuropt/latest/heuropt/algorithms/spea2/struct.Spea2.html
|
||||
[`SmsEmoa`]: https://docs.rs/heuropt/latest/heuropt/algorithms/sms_emoa/struct.SmsEmoa.html
|
||||
[`Moead`]: https://docs.rs/heuropt/latest/heuropt/algorithms/moead/struct.Moead.html
|
||||
[`AgeMoea`]: https://docs.rs/heuropt/latest/heuropt/algorithms/age_moea/struct.AgeMoea.html
|
||||
[`Knea`]: https://docs.rs/heuropt/latest/heuropt/algorithms/knea/struct.Knea.html
|
||||
[`PesaII`]: https://docs.rs/heuropt/latest/heuropt/algorithms/pesa2/struct.PesaII.html
|
||||
[`EpsilonMoea`]: https://docs.rs/heuropt/latest/heuropt/algorithms/epsilon_moea/struct.EpsilonMoea.html
|
||||
[`Paes`]: https://docs.rs/heuropt/latest/heuropt/algorithms/paes/struct.Paes.html
|
||||
[`Grea`]: https://docs.rs/heuropt/latest/heuropt/algorithms/grea/struct.Grea.html
|
||||
[`Rvea`]: https://docs.rs/heuropt/latest/heuropt/algorithms/rvea/struct.Rvea.html
|
||||
[`HypE`]: https://docs.rs/heuropt/latest/heuropt/algorithms/hype/struct.Hype.html
|
||||
[`Repair<D>`]: https://docs.rs/heuropt/latest/heuropt/traits/trait.Repair.html
|
||||
[`ClampToBounds`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.ClampToBounds.html
|
||||
[`ProjectToSimplex`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.ProjectToSimplex.html
|
||||
[`stochastic_ranking_select`]: https://docs.rs/heuropt/latest/heuropt/selection/tournament/fn.stochastic_ranking_select.html
|
||||
[`CompositeVariation`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.CompositeVariation.html
|
||||
@@ -0,0 +1,108 @@
|
||||
# Comparison with other libraries
|
||||
|
||||
heuropt is one of many heuristic-optimization libraries. This chapter
|
||||
is an honest, opinionated comparison to help you choose.
|
||||
|
||||
The columns:
|
||||
|
||||
- **Lang** — primary implementation language.
|
||||
- **Algorithms** — rough catalog count.
|
||||
- **Multi-obj** — built-in support for Pareto-based multi-objective
|
||||
optimization.
|
||||
- **Surrogates** — built-in Bayesian / TPE / multi-fidelity.
|
||||
- **Determinism** — seeded reproducibility as a first-class property.
|
||||
- **Async / async-eval** — first-class async runtime support.
|
||||
|
||||
| Library | Lang | Algorithms | Multi-obj | Surrogates | Determinism | Async |
|
||||
|---|---|---|---|---|---|---|
|
||||
| **heuropt 0.5** | Rust | 35 | ✅ NSGA-II/III, SPEA2, IBEA, MOEA/D, MOPSO, SMS-EMOA, HypE, AGE-MOEA, GrEA, KnEA, RVEA, PESA-II, ε-MOEA, PAES | ✅ BO, TPE, Hyperband | ✅ bit-identical seeded | ⏳ planned |
|
||||
| pymoo | Python | ~25 | ✅ extensive | partial (BO via plug-ins) | ✅ | ❌ |
|
||||
| DEAP | Python | flexible toolbox | ✅ | ❌ | ✅ | ❌ |
|
||||
| hyperopt | Python | TPE-focused | ❌ | ✅ TPE | partial | partial |
|
||||
| optuna | Python | TPE / CMA-ES / NSGA-II | ✅ | ✅ TPE, BoTorch via plug-in | ✅ | ✅ |
|
||||
| MOEA Framework | Java | ~40 | ✅ very extensive | ❌ | ✅ | ❌ |
|
||||
| metaheuristics-rs | Rust | ~10 | partial | ❌ | ✅ | ❌ |
|
||||
| argmin | Rust | line-search / quasi-Newton | ❌ | ❌ | ✅ | ❌ |
|
||||
|
||||
## When to pick heuropt
|
||||
|
||||
- You're working in **Rust** and want a single, dependency-light crate
|
||||
for evolutionary / metaheuristic optimization.
|
||||
- You need **multi-objective or many-objective** algorithms (12+
|
||||
Pareto-aware methods in the catalog) AND you don't want to glue
|
||||
Python into your Rust pipeline.
|
||||
- You want **bit-identical determinism**: same seed produces same
|
||||
output, on every machine, across releases unless explicitly noted
|
||||
otherwise.
|
||||
- You want a **small, readable codebase** — every algorithm is
|
||||
written for clarity, no trait-object plumbing, no GATs in user-
|
||||
facing APIs. Reading `RandomSearch` should be enough to write a
|
||||
new optimizer.
|
||||
|
||||
## When *not* to pick heuropt
|
||||
|
||||
- You need **first-class async / await** for evaluations that talk to
|
||||
HTTP services or spawn subprocesses. heuropt is sync; that's on
|
||||
the roadmap but not shipping yet.
|
||||
- You need **gradient-based** optimization. Use `argmin` (Rust) or
|
||||
`scipy.optimize` (Python) — heuropt is gradient-free by design.
|
||||
- You need **GPU-accelerated** evaluations. heuropt's `evaluate`
|
||||
function runs on CPU; use Python (jax/torch) or roll your own
|
||||
GPU pipeline.
|
||||
- You need **distributed multi-machine** evaluation. heuropt
|
||||
parallelizes within one process via rayon. Distribution is up to
|
||||
you (split the seeds across machines, aggregate).
|
||||
- You're comfortable in Python and pymoo / optuna already cover
|
||||
your problem. heuropt's value-add over pymoo is mostly that it's
|
||||
Rust — if that doesn't matter to you, the Python ecosystem has more
|
||||
battle-tested integrations.
|
||||
|
||||
## Algorithm coverage at a glance
|
||||
|
||||
heuropt covers the same major Pareto MOEAs as pymoo and MOEA Framework:
|
||||
NSGA-II/III, SPEA2, IBEA, MOEA/D, MOPSO, SMS-EMOA, HypE, AGE-MOEA,
|
||||
GrEA, KnEA, RVEA, PESA-II, ε-MOEA, PAES.
|
||||
|
||||
The expensive-evaluation regime: BayesianOpt + TPE + Hyperband. This
|
||||
is comparable to optuna's coverage but in pure Rust.
|
||||
|
||||
The single-objective continuous catalog (CMA-ES, IPOP-CMA-ES, sNES,
|
||||
DE, PSO, GA, TLBO, (1+1)-ES, NelderMead, RandomSearch, HillClimber,
|
||||
SimulatedAnnealing) covers the canonical baselines and several modern
|
||||
variants.
|
||||
|
||||
What heuropt does **not** ship that some libraries do:
|
||||
|
||||
- **Re-themed metaphor metaheuristics** (Whale Optimization, Grey
|
||||
Wolf, Bat, Firefly, Harris Hawks, etc.). These are cut from the
|
||||
catalog deliberately — they are mostly DE/PSO with new names. If
|
||||
you specifically need one, please open an issue with citations.
|
||||
- **Non-evolutionary global optimizers** like dual annealing or
|
||||
basin-hopping (use `scipy.optimize` for those).
|
||||
- **A web UI / dashboard** like optuna's. heuropt is library-only.
|
||||
|
||||
## Speed
|
||||
|
||||
heuropt's hot paths (Pareto utilities, hypervolume, key inner loops)
|
||||
are heavily optimized — see the perf entry in the v0.4.0 CHANGELOG.
|
||||
On the comparison harness in `examples/compare.rs` (10-seed mean,
|
||||
30 000 evaluations on DTLZ2), the total wall-clock time across 12
|
||||
algorithms is ~5 seconds. Per-algorithm timings are in
|
||||
[`examples/compare-results.md`](https://github.com/swaits/heuropt/blob/main/examples/compare-results.md).
|
||||
|
||||
For comparison-shopping speed against Python libraries, the gap is
|
||||
typically 10×–100× in heuropt's favor for compute-bound
|
||||
`evaluate` functions, because Rust skips the Python-loop overhead. If
|
||||
your `evaluate` calls into NumPy/PyTorch and those are the bottleneck,
|
||||
the gap shrinks substantially.
|
||||
|
||||
## Honest weakness: ecosystem
|
||||
|
||||
The biggest thing pymoo / optuna / DEAP have that heuropt doesn't:
|
||||
**community + plug-ins + tutorials**. They've been around longer and
|
||||
have rich third-party integrations (visualization, MLflow,
|
||||
Hyperband+BO hybrids, distributed runners). heuropt is younger; the
|
||||
core is solid but the ecosystem is small.
|
||||
|
||||
If you adopt heuropt and miss a thing, the project is small enough
|
||||
that contributions land fast. See [CONTRIBUTING.md](https://github.com/swaits/heuropt/blob/main/CONTRIBUTING.md).
|
||||
@@ -0,0 +1,26 @@
|
||||
# Cookbook
|
||||
|
||||
Short, focused recipes for the patterns that come up in practice.
|
||||
Each recipe is self-contained and small enough to copy into your own
|
||||
project.
|
||||
|
||||
## Recipes
|
||||
|
||||
- [Parallelize evaluation with rayon](./cookbook/parallel.md) — when
|
||||
your `evaluate` is non-trivial, the `parallel` feature pays for
|
||||
itself almost immediately.
|
||||
- [Tune a model with expensive evaluations](./cookbook/expensive-evaluations.md)
|
||||
— `BayesianOpt`, `Tpe`, and `Hyperband` for the 50–500-eval
|
||||
regime.
|
||||
- [Compare two algorithms on your problem](./cookbook/compare.md) —
|
||||
multi-seed harness pattern straight from `examples/compare.rs`.
|
||||
- [Optimize a permutation (TSP-style)](./cookbook/permutation.md) —
|
||||
`AntColonyTsp` with a distance matrix.
|
||||
- [Constrain your search with `Repair`](./cookbook/constraints.md) —
|
||||
bounds, simplex projection, custom repair.
|
||||
- [Pick one answer off a Pareto front](./cookbook/pick-one.md) — the
|
||||
a-posteriori weighted-decision pattern from the `jiggly_tuning`
|
||||
example.
|
||||
- [Write your own algorithm](./cookbook/custom-optimizer.md) —
|
||||
implement `Optimizer<P>` from scratch, à la the
|
||||
`examples/custom_optimizer.rs` walkthrough.
|
||||
@@ -0,0 +1,148 @@
|
||||
# Compare two algorithms on your problem
|
||||
|
||||
The harness in `examples/compare.rs` runs every applicable algorithm
|
||||
against every test problem with N seeds and reports mean ± std.
|
||||
You can lift the same pattern for your own problem in ~30 lines.
|
||||
|
||||
## The pattern
|
||||
|
||||
1. Wrap your problem in a struct that implements [`Problem`].
|
||||
2. Pick a few candidate algorithms.
|
||||
3. For each algorithm × seed, run and record the metric you care about.
|
||||
4. Print mean ± std.
|
||||
|
||||
## Worked example
|
||||
|
||||
```rust,no_run
|
||||
use heuropt::prelude::*;
|
||||
use std::time::Instant;
|
||||
|
||||
struct MyProblem;
|
||||
impl Problem for MyProblem {
|
||||
type Decision = Vec<f64>;
|
||||
fn objectives(&self) -> ObjectiveSpace {
|
||||
ObjectiveSpace::new(vec![Objective::minimize("f")])
|
||||
}
|
||||
fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
|
||||
// your problem here
|
||||
Evaluation::new(vec![x.iter().map(|v| v * v).sum::<f64>()])
|
||||
}
|
||||
}
|
||||
|
||||
const SEEDS: u64 = 10;
|
||||
const DIM: usize = 5;
|
||||
const BUDGET: usize = 30_000;
|
||||
|
||||
fn main() {
|
||||
let bounds: Vec<(f64, f64)> = vec![(-5.0, 5.0); DIM];
|
||||
|
||||
let mut best_de = vec![];
|
||||
let mut best_cmaes = vec![];
|
||||
let mut best_ipop = vec![];
|
||||
let mut t_de = vec![];
|
||||
let mut t_cmaes = vec![];
|
||||
let mut t_ipop = vec![];
|
||||
|
||||
for seed in 0..SEEDS {
|
||||
// Differential Evolution
|
||||
let t = Instant::now();
|
||||
let mut de = DifferentialEvolution::new(
|
||||
DifferentialEvolutionConfig {
|
||||
population_size: 30,
|
||||
generations: BUDGET / 30,
|
||||
differential_weight: 0.5,
|
||||
crossover_probability: 0.9,
|
||||
seed,
|
||||
},
|
||||
RealBounds::new(bounds.clone()),
|
||||
);
|
||||
let r = de.run(&MyProblem);
|
||||
t_de.push(t.elapsed().as_millis() as f64);
|
||||
best_de.push(r.best.unwrap().evaluation.objectives[0]);
|
||||
|
||||
// CMA-ES
|
||||
let t = Instant::now();
|
||||
let mut cma = CmaEs::new(
|
||||
CmaEsConfig {
|
||||
population_size: 12,
|
||||
generations: BUDGET / 12,
|
||||
initial_sigma: 1.0,
|
||||
eigen_decomposition_period: 1,
|
||||
initial_mean: None,
|
||||
seed,
|
||||
},
|
||||
RealBounds::new(bounds.clone()),
|
||||
);
|
||||
let r = cma.run(&MyProblem);
|
||||
t_cmaes.push(t.elapsed().as_millis() as f64);
|
||||
best_cmaes.push(r.best.unwrap().evaluation.objectives[0]);
|
||||
|
||||
// IPOP-CMA-ES
|
||||
let t = Instant::now();
|
||||
let mut ipop = IpopCmaEs::new(
|
||||
IpopCmaEsConfig {
|
||||
base: CmaEsConfig {
|
||||
population_size: 12,
|
||||
generations: BUDGET / 12 / 4,
|
||||
initial_sigma: 1.0,
|
||||
eigen_decomposition_period: 1,
|
||||
initial_mean: None,
|
||||
seed,
|
||||
},
|
||||
max_restarts: 3,
|
||||
population_factor: 2.0,
|
||||
seed,
|
||||
},
|
||||
RealBounds::new(bounds.clone()),
|
||||
);
|
||||
let r = ipop.run(&MyProblem);
|
||||
t_ipop.push(t.elapsed().as_millis() as f64);
|
||||
best_ipop.push(r.best.unwrap().evaluation.objectives[0]);
|
||||
}
|
||||
|
||||
println!("{:<12} {:>14} {:>10}", "algorithm", "best f (mean±std)", "ms");
|
||||
print_row("DE", &best_de, &t_de);
|
||||
print_row("CMA-ES", &best_cmaes, &t_cmaes);
|
||||
print_row("IPOP-CMA-ES", &best_ipop, &t_ipop);
|
||||
}
|
||||
|
||||
fn print_row(name: &str, values: &[f64], times: &[f64]) {
|
||||
let (m, s) = mean_std(values);
|
||||
let (t, _) = mean_std(times);
|
||||
println!("{:<12} {:>10.3e} ± {:>5.2e} {:>6.0}", name, m, s, t);
|
||||
}
|
||||
|
||||
fn mean_std(xs: &[f64]) -> (f64, f64) {
|
||||
let n = xs.len() as f64;
|
||||
let m = xs.iter().sum::<f64>() / n;
|
||||
let v = xs.iter().map(|x| (x - m).powi(2)).sum::<f64>() / n;
|
||||
(m, v.sqrt())
|
||||
}
|
||||
```
|
||||
|
||||
## What to record
|
||||
|
||||
- **`best.evaluation.objectives[0]`** for single-objective.
|
||||
- **`hypervolume_2d(&result.pareto_front, &space, ref_point)`** for
|
||||
2-objective.
|
||||
- **`spacing(&result.pareto_front, &space)`** for front uniformity.
|
||||
- **`result.evaluations`** to cross-check that every algorithm got
|
||||
the same evaluation budget.
|
||||
- Wall-clock `Instant::now()` deltas for runtime comparison.
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- **Population size matters.** Different algorithms have very
|
||||
different sweet spots. Don't just give them all the same
|
||||
population — the README's algorithm pages note typical defaults.
|
||||
- **Different algorithms count "generations" differently.** What
|
||||
matters is the total `evaluations` count. Set
|
||||
`generations = BUDGET / population_size` to match across
|
||||
algorithms (with caveats for steady-state algorithms like SMS-EMOA
|
||||
that evaluate one offspring per generation).
|
||||
- **One seed is not a comparison.** Always run ≥ 5 seeds; ≥ 10 is
|
||||
better. Single-seed comparisons are noise.
|
||||
- **The harness in `examples/compare.rs` is the canonical version.**
|
||||
When in doubt, copy from there.
|
||||
|
||||
[`Problem`]: https://docs.rs/heuropt/latest/heuropt/core/problem/trait.Problem.html
|
||||
@@ -0,0 +1,126 @@
|
||||
# Constrain your search with `Repair`
|
||||
|
||||
heuropt models constraints with a single `constraint_violation` scalar
|
||||
on each `Evaluation`. That works for soft penalties. When constraints
|
||||
are *hard* and the search keeps generating infeasible decisions, the
|
||||
better pattern is **repair**: project each candidate back into the
|
||||
feasible region every time it leaves.
|
||||
|
||||
The [`Repair<D>`] trait is the abstraction. Two impls ship in the box;
|
||||
you can write your own for arbitrary geometry.
|
||||
|
||||
## Built-in: `ClampToBounds`
|
||||
|
||||
For per-axis box constraints (`lo ≤ xᵢ ≤ hi`), pair `ClampToBounds`
|
||||
with any `Variation` to get a bounds-aware variant for free.
|
||||
|
||||
```rust,no_run
|
||||
use heuropt::prelude::*;
|
||||
|
||||
let bounds = vec![(-5.0, 5.0); 3];
|
||||
|
||||
// Without bounds, GaussianMutation can step outside the search box.
|
||||
// ClampToBounds projects each variable back in.
|
||||
let mut sigma = GaussianMutation { sigma: 0.5 };
|
||||
let mut clamp = ClampToBounds::new(bounds.clone());
|
||||
|
||||
let mut rng = rng_from_seed(42);
|
||||
let parent = vec![4.9, -4.9, 0.0];
|
||||
let mut child = sigma.vary(std::slice::from_ref(&parent), &mut rng).pop().unwrap();
|
||||
clamp.repair(&mut child);
|
||||
// every entry of `child` is now within [-5, 5].
|
||||
```
|
||||
|
||||
`ClampToBounds` is idempotent: applying it twice is the same as
|
||||
applying it once.
|
||||
|
||||
For most real problems you'd just use [`BoundedGaussianMutation`]
|
||||
which combines both in one operator.
|
||||
|
||||
## Built-in: `ProjectToSimplex`
|
||||
|
||||
For *budget* constraints — "the components must sum to a fixed
|
||||
total and be non-negative" — `ProjectToSimplex` projects onto the
|
||||
probability simplex (or any scaled simplex).
|
||||
|
||||
```rust,no_run
|
||||
use heuropt::prelude::*;
|
||||
|
||||
let mut proj = ProjectToSimplex::new(1.0); // probability simplex
|
||||
let mut x = vec![0.6, 0.5, -0.1, 0.3]; // sum 1.3, one negative
|
||||
proj.repair(&mut x);
|
||||
// x now sums to 1.0 and every entry is ≥ 0.
|
||||
let s: f64 = x.iter().sum();
|
||||
debug_assert!((s - 1.0).abs() < 1e-12);
|
||||
debug_assert!(x.iter().all(|&v| v >= 0.0));
|
||||
```
|
||||
|
||||
Use this for portfolio / resource-allocation problems where the
|
||||
decision is a vector of weights that must sum to a budget.
|
||||
|
||||
## Custom repair
|
||||
|
||||
Anything that takes a `&mut Vec<f64>` (or any `&mut D` for your
|
||||
custom decision type) and returns a feasible version is a valid
|
||||
`Repair`. Implement the trait directly:
|
||||
|
||||
```rust,no_run
|
||||
use heuropt::prelude::*;
|
||||
|
||||
/// Force the largest variable to be at least `min_largest`.
|
||||
struct AtLeastOneActive { min_largest: f64 }
|
||||
|
||||
impl Repair<Vec<f64>> for AtLeastOneActive {
|
||||
fn repair(&mut self, x: &mut Vec<f64>) {
|
||||
let max_idx = x.iter()
|
||||
.enumerate()
|
||||
.fold(0, |best, (i, &v)| {
|
||||
if v > x[best] { i } else { best }
|
||||
});
|
||||
if x[max_idx] < self.min_largest {
|
||||
x[max_idx] = self.min_largest;
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Stochastic-ranking selection
|
||||
|
||||
When the feasible region is *narrow* — most of the search space is
|
||||
infeasible — the strict "feasibles always beat infeasibles" rule
|
||||
traps the search outside it. Runarsson & Yao's stochastic ranking
|
||||
breaks the trap by, on each pairwise comparison, using a probabilistic
|
||||
"compare by objective" instead of "compare by feasibility" with a
|
||||
small probability `pf`:
|
||||
|
||||
```rust,ignore
|
||||
use heuropt::selection::tournament::stochastic_ranking_select;
|
||||
|
||||
let picks = stochastic_ranking_select(
|
||||
&population,
|
||||
&objectives,
|
||||
0.45, // pf — Runarsson & Yao's canonical value
|
||||
count,
|
||||
&mut rng,
|
||||
);
|
||||
```
|
||||
|
||||
This is a drop-in replacement for `tournament_select_single_objective`
|
||||
in your custom optimizer or in a forked algorithm.
|
||||
|
||||
## When to use which
|
||||
|
||||
| Situation | Use |
|
||||
|---|---|
|
||||
| Box constraints | [`BoundedGaussianMutation`] (built-in mutation) |
|
||||
| Manual repair after any mutation | [`ClampToBounds`] |
|
||||
| Budget / probability-simplex constraints | [`ProjectToSimplex`] |
|
||||
| Custom geometric constraints | Your own `Repair` impl |
|
||||
| Narrow feasible region, frequent infeasibility | [`stochastic_ranking_select`] |
|
||||
| Soft penalty, mostly feasible search | Set `constraint_violation` and let default tournament handle it |
|
||||
|
||||
[`Repair<D>`]: https://docs.rs/heuropt/latest/heuropt/traits/trait.Repair.html
|
||||
[`ClampToBounds`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.ClampToBounds.html
|
||||
[`ProjectToSimplex`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.ProjectToSimplex.html
|
||||
[`BoundedGaussianMutation`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.BoundedGaussianMutation.html
|
||||
[`stochastic_ranking_select`]: https://docs.rs/heuropt/latest/heuropt/selection/tournament/fn.stochastic_ranking_select.html
|
||||
@@ -0,0 +1,146 @@
|
||||
# Write your own algorithm
|
||||
|
||||
Implement [`Optimizer<P>`] and you're done. There are no other traits
|
||||
to think about, no internal hooks to register. The example walks
|
||||
through a tiny hill-climber that reads almost identically to the
|
||||
canonical pseudocode.
|
||||
|
||||
## The trait
|
||||
|
||||
```rust,ignore
|
||||
pub trait Optimizer<P>
|
||||
where
|
||||
P: Problem,
|
||||
{
|
||||
fn run(&mut self, problem: &P) -> OptimizationResult<P::Decision>;
|
||||
}
|
||||
```
|
||||
|
||||
That's it. You own your config, your RNG, your main loop, and your
|
||||
`OptimizationResult` construction.
|
||||
|
||||
## A minimal hill-climber
|
||||
|
||||
```rust,no_run
|
||||
use heuropt::prelude::*;
|
||||
|
||||
pub struct MyHillClimber<I, V> {
|
||||
pub iterations: usize,
|
||||
pub seed: u64,
|
||||
pub initializer: I,
|
||||
pub variation: V,
|
||||
}
|
||||
|
||||
impl<P, I, V> Optimizer<P> for MyHillClimber<I, V>
|
||||
where
|
||||
P: Problem,
|
||||
P::Decision: Clone,
|
||||
I: Initializer<P::Decision>,
|
||||
V: Variation<P::Decision>,
|
||||
{
|
||||
fn run(&mut self, problem: &P) -> OptimizationResult<P::Decision> {
|
||||
let mut rng = rng_from_seed(self.seed);
|
||||
let objectives = problem.objectives();
|
||||
assert!(objectives.is_single_objective(), "MyHillClimber is single-objective only");
|
||||
|
||||
// Start with one initial decision.
|
||||
let init_decisions = self.initializer.initialize(1, &mut rng);
|
||||
let init = init_decisions.into_iter().next().unwrap();
|
||||
let mut current = Candidate::new(init.clone(), problem.evaluate(&init));
|
||||
let mut evaluations: usize = 1;
|
||||
|
||||
for _ in 0..self.iterations {
|
||||
let children = self.variation.vary(std::slice::from_ref(¤t.decision), &mut rng);
|
||||
for child_decision in children {
|
||||
let child_eval = problem.evaluate(&child_decision);
|
||||
evaluations += 1;
|
||||
let child = Candidate::new(child_decision, child_eval);
|
||||
if better(&child.evaluation, ¤t.evaluation, &objectives) {
|
||||
current = child;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
let pareto_front = vec![current.clone()];
|
||||
let best = Some(current.clone());
|
||||
OptimizationResult::new(
|
||||
Population::new(vec![current]),
|
||||
pareto_front,
|
||||
best,
|
||||
evaluations,
|
||||
self.iterations,
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
fn better(a: &Evaluation, b: &Evaluation, objectives: &ObjectiveSpace) -> bool {
|
||||
let am = objectives.as_minimization(&a.objectives);
|
||||
let bm = objectives.as_minimization(&b.objectives);
|
||||
am[0] < bm[0]
|
||||
}
|
||||
```
|
||||
|
||||
## Things to notice
|
||||
|
||||
- **`Rng` is one concrete type.** No generics — call
|
||||
[`rng_from_seed`] and pass `&mut rng` everywhere it's needed.
|
||||
- **`Initializer<D>`** sources the starting point(s).
|
||||
- **`Variation<D>`** generates children from parents. For the
|
||||
hill-climber it's called with one parent.
|
||||
- **`OptimizationResult`** carries the final population, the Pareto
|
||||
front (just the best for single-objective), the best candidate,
|
||||
the total evaluations, and the iteration count.
|
||||
- **`as_minimization`** flips maximize-axis values so your
|
||||
comparison logic only ever needs to deal with "lower is better."
|
||||
|
||||
## Adding parallel evaluation
|
||||
|
||||
If your algorithm batch-evaluates candidates per generation, use the
|
||||
crate's internal helper. From inside heuropt source you can call
|
||||
`evaluate_batch(problem, decisions)`; from outside you'd use rayon
|
||||
directly behind a feature flag, the same way the built-in algorithms
|
||||
do.
|
||||
|
||||
```rust,ignore
|
||||
#[cfg(feature = "parallel")]
|
||||
fn batch_eval<P>(problem: &P, decisions: Vec<P::Decision>) -> Vec<Candidate<P::Decision>>
|
||||
where P: Problem + Sync, P::Decision: Send,
|
||||
{
|
||||
use rayon::prelude::*;
|
||||
decisions.into_par_iter()
|
||||
.map(|d| Candidate::new(d.clone(), problem.evaluate(&d)))
|
||||
.collect()
|
||||
}
|
||||
|
||||
#[cfg(not(feature = "parallel"))]
|
||||
fn batch_eval<P>(problem: &P, decisions: Vec<P::Decision>) -> Vec<Candidate<P::Decision>>
|
||||
where P: Problem,
|
||||
{
|
||||
decisions.into_iter()
|
||||
.map(|d| Candidate::new(d.clone(), problem.evaluate(&d)))
|
||||
.collect()
|
||||
}
|
||||
```
|
||||
|
||||
To stay bit-identical between serial and parallel modes, keep the
|
||||
RNG and selection on the main thread; only the *evaluations* run in
|
||||
parallel.
|
||||
|
||||
## What's *not* in the trait
|
||||
|
||||
- **No iteration / step API.** The optimizer owns its loop.
|
||||
- **No callbacks.** A future minor release may add an observer hook;
|
||||
for now you'd run the algorithm to completion and process the
|
||||
result.
|
||||
- **No error type.** Invalid configuration panics with a clear
|
||||
message; this matches the style of the built-in algorithms.
|
||||
- **No async.** `evaluate` is synchronous; for async work, drive it
|
||||
on a tokio runtime around the optimizer loop yourself.
|
||||
|
||||
The smallness is the point: you should be able to read a built-in
|
||||
algorithm and write your own in an afternoon. See
|
||||
`examples/custom_optimizer.rs` for a slightly more polished version
|
||||
of the hill-climber above.
|
||||
|
||||
[`Optimizer<P>`]: https://docs.rs/heuropt/latest/heuropt/traits/trait.Optimizer.html
|
||||
[`rng_from_seed`]: https://docs.rs/heuropt/latest/heuropt/core/rng/fn.rng_from_seed.html
|
||||
@@ -0,0 +1,164 @@
|
||||
# Tune a model with expensive evaluations
|
||||
|
||||
Population-based EAs throw thousands of evaluations at a problem. If
|
||||
each evaluation costs a minute (a model training run, a CFD solve, a
|
||||
real-world measurement) you can't afford that. heuropt has three
|
||||
algorithms aimed at this regime.
|
||||
|
||||
| Algorithm | Surrogate | Best for |
|
||||
|---|---|---|
|
||||
| [`BayesianOpt`] | Gaussian process + Expected Improvement | The textbook choice; needs kernel tuning to shine |
|
||||
| [`Tpe`] | Kernel-density estimate of good vs bad points | Cheaper per step; more robust without tuning |
|
||||
| [`Hyperband`] | (none — it's a multi-fidelity scheduler) | When each eval has a tunable budget (epochs, MC samples) |
|
||||
|
||||
## When each is right
|
||||
|
||||
- **Black-box, fixed cost per eval, smooth-ish landscape** → BO.
|
||||
- **Black-box, fixed cost per eval, no time to tune the surrogate** → TPE.
|
||||
- **Each eval has a tunable fidelity** → Hyperband.
|
||||
|
||||
## Bayesian Optimization
|
||||
|
||||
A worked example with a synthetic 5-D problem and a 60-evaluation
|
||||
budget — same configuration the `compare` harness uses.
|
||||
|
||||
```rust,no_run
|
||||
use heuropt::prelude::*;
|
||||
|
||||
struct Rosenbrock5D;
|
||||
impl Problem for Rosenbrock5D {
|
||||
type Decision = Vec<f64>;
|
||||
fn objectives(&self) -> ObjectiveSpace {
|
||||
ObjectiveSpace::new(vec![Objective::minimize("f")])
|
||||
}
|
||||
fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
|
||||
let f: f64 = x.windows(2).map(|w|
|
||||
100.0 * (w[1] - w[0].powi(2)).powi(2) + (1.0 - w[0]).powi(2)
|
||||
).sum();
|
||||
Evaluation::new(vec![f])
|
||||
}
|
||||
}
|
||||
|
||||
let bounds = vec![(-2.048_f64, 2.048_f64); 5];
|
||||
let mut opt = BayesianOpt::new(
|
||||
BayesianOptConfig {
|
||||
evaluations: 60,
|
||||
initial_samples: 10,
|
||||
length_scale: 1.0,
|
||||
signal_variance: 1.0,
|
||||
noise_variance: 1e-6,
|
||||
seed: 42,
|
||||
},
|
||||
RealBounds::new(bounds),
|
||||
);
|
||||
let r = opt.run(&Rosenbrock5D);
|
||||
println!("best f after 60 evals: {}", r.best.unwrap().evaluation.objectives[0]);
|
||||
```
|
||||
|
||||
> **Honest disclosure.** On the comparison harness this default
|
||||
> configuration produces **f ≈ 3170 ± 2920** on Rosenbrock 5-D — well
|
||||
> below what a tuned BO can do. The default RBF kernel without
|
||||
> per-problem hyperparameter tuning is the limitation. For real
|
||||
> workloads, consider:
|
||||
>
|
||||
> - More evaluations (200+ instead of 60).
|
||||
> - Tuning `length_scale` to a known scale of your problem
|
||||
> (lower for high-frequency landscapes, higher for smooth ones).
|
||||
> - TPE instead of BO if you don't want to tune the kernel.
|
||||
|
||||
## Tree-structured Parzen Estimator
|
||||
|
||||
TPE keeps two density estimates — `l(x)` over historical good points
|
||||
and `g(x)` over the rest — and picks new candidates that maximize the
|
||||
ratio. Cheaper per step than a GP and famously robust without
|
||||
hand-tuning.
|
||||
|
||||
```rust,no_run
|
||||
use heuropt::prelude::*;
|
||||
# struct Rosenbrock5D;
|
||||
# impl Problem for Rosenbrock5D {
|
||||
# type Decision = Vec<f64>;
|
||||
# fn objectives(&self) -> ObjectiveSpace { ObjectiveSpace::new(vec![Objective::minimize("f")]) }
|
||||
# fn evaluate(&self, _x: &Vec<f64>) -> Evaluation { Evaluation::new(vec![0.0]) }
|
||||
# }
|
||||
let bounds = vec![(-2.048_f64, 2.048_f64); 5];
|
||||
let mut opt = Tpe::new(
|
||||
TpeConfig {
|
||||
evaluations: 60,
|
||||
initial_samples: 10,
|
||||
gamma: 0.25,
|
||||
candidates_per_step: 24,
|
||||
bandwidth_factor: 1.06,
|
||||
seed: 42,
|
||||
},
|
||||
RealBounds::new(bounds),
|
||||
);
|
||||
let _r = opt.run(&Rosenbrock5D);
|
||||
```
|
||||
|
||||
`gamma` is the fraction of best points used as `l(x)`; `0.25` is the
|
||||
canonical Bergstra value.
|
||||
|
||||
## Hyperband
|
||||
|
||||
[`Hyperband`] needs your problem to implement [`PartialProblem`] —
|
||||
that is, you can evaluate at a tunable fidelity (e.g. number of
|
||||
training epochs). The algorithm schedules many cheap-fidelity runs
|
||||
and promotes only the survivors to higher fidelity.
|
||||
|
||||
```rust,no_run
|
||||
use heuropt::prelude::*;
|
||||
use heuropt::core::partial_problem::PartialProblem;
|
||||
|
||||
struct ModelTuning;
|
||||
impl Problem for ModelTuning {
|
||||
type Decision = Vec<f64>;
|
||||
fn objectives(&self) -> ObjectiveSpace {
|
||||
ObjectiveSpace::new(vec![Objective::minimize("val_loss")])
|
||||
}
|
||||
fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
|
||||
// Full-fidelity eval = train at max_epochs.
|
||||
self.evaluate_at_budget(x, 100.0)
|
||||
}
|
||||
}
|
||||
impl PartialProblem for ModelTuning {
|
||||
fn evaluate_at_budget(&self, x: &Vec<f64>, budget: f64) -> Evaluation {
|
||||
// Replace with: train your model for `budget` epochs, return val_loss.
|
||||
// For demo, pretend more budget = lower noisy loss.
|
||||
let lr = x[0];
|
||||
let wd = x[1];
|
||||
let loss = (lr - 0.001).powi(2) + (wd - 1e-4).powi(2)
|
||||
+ 1.0 / (budget + 1.0);
|
||||
Evaluation::new(vec![loss])
|
||||
}
|
||||
}
|
||||
|
||||
let bounds = vec![(1e-5_f64, 1e-1), (1e-6_f64, 1e-2)];
|
||||
let mut hyperband = Hyperband::new(
|
||||
HyperbandConfig {
|
||||
max_budget: 100.0,
|
||||
eta: 3.0,
|
||||
seed: 42,
|
||||
},
|
||||
RealBounds::new(bounds),
|
||||
);
|
||||
let _r = hyperband.run(&ModelTuning);
|
||||
```
|
||||
|
||||
`max_budget` is the most epochs (or whatever your fidelity unit is)
|
||||
you'd ever spend on a single config. `eta` controls how aggressive
|
||||
the elimination is — `3.0` is the classic value; higher means more
|
||||
aggressive culling.
|
||||
|
||||
## Strategy: combining surrogate + multi-fidelity
|
||||
|
||||
The state of the art (BOHB) combines BO with Hyperband: TPE picks the
|
||||
configurations Hyperband then evaluates at increasing fidelity.
|
||||
heuropt doesn't ship a unified BOHB but the building blocks are
|
||||
there — wrap your `PartialProblem` with a TPE-driven sampler and
|
||||
feed the picks into `Hyperband`. PRs welcome.
|
||||
|
||||
[`BayesianOpt`]: https://docs.rs/heuropt/latest/heuropt/algorithms/bayesian_opt/struct.BayesianOpt.html
|
||||
[`Tpe`]: https://docs.rs/heuropt/latest/heuropt/algorithms/tpe/struct.Tpe.html
|
||||
[`Hyperband`]: https://docs.rs/heuropt/latest/heuropt/algorithms/hyperband/struct.Hyperband.html
|
||||
[`PartialProblem`]: https://docs.rs/heuropt/latest/heuropt/core/partial_problem/trait.PartialProblem.html
|
||||
@@ -0,0 +1,127 @@
|
||||
# Parallelize evaluation with rayon
|
||||
|
||||
If a single call to your `evaluate` takes more than ~50 µs, enabling
|
||||
the `parallel` feature usually pays for itself immediately on
|
||||
population-based algorithms. Each generation evaluates an entire
|
||||
population, and rayon parallelizes that batch.
|
||||
|
||||
## Enable the feature
|
||||
|
||||
```toml
|
||||
[dependencies]
|
||||
heuropt = { version = "0.5", features = ["parallel"] }
|
||||
```
|
||||
|
||||
There's nothing else to opt into in your code. The
|
||||
population-evaluation helper is feature-gated; with `parallel` on it
|
||||
uses `rayon::into_par_iter` internally, with `parallel` off it falls
|
||||
back to plain `into_iter`.
|
||||
|
||||
## Determinism still holds
|
||||
|
||||
Seeded runs are bit-identical between the serial and parallel modes.
|
||||
The trick is that population members are evaluated in parallel but
|
||||
*assembled* back into the same order. Variation, selection, and the
|
||||
RNG are all driven by the main thread, so seed-stability tests still
|
||||
pass.
|
||||
|
||||
## Which algorithms benefit
|
||||
|
||||
Algorithms with a per-generation `evaluate_batch`:
|
||||
|
||||
- [`RandomSearch`], [`Nsga2`], [`Nsga3`], [`Spea2`], [`Moead`],
|
||||
[`Mopso`], [`Ibea`], [`SmsEmoa`], [`HypE`], [`PesaII`],
|
||||
[`EpsilonMoea`], [`AgeMoea`], [`Knea`], [`Grea`], [`Rvea`].
|
||||
- [`DifferentialEvolution`] and [`GeneticAlgorithm`] benefit on the
|
||||
initial population and offspring batches.
|
||||
|
||||
Steady-state algorithms ([`Paes`], [`SimulatedAnnealing`],
|
||||
[`HillClimber`], [`OnePlusOneEs`]) only evaluate one or a few
|
||||
candidates per iteration, so the parallel feature gives them
|
||||
nothing — leave it off if those are your primary optimizers.
|
||||
|
||||
## Worked example
|
||||
|
||||
The Sphere problem is too cheap to actually benefit from parallelism
|
||||
— this example just shows the shape. In real workloads `evaluate` is
|
||||
the expensive bit (a simulation, a model fit, an HTTP call).
|
||||
|
||||
```rust,no_run
|
||||
use heuropt::prelude::*;
|
||||
|
||||
struct ExpensiveSphere;
|
||||
impl Problem for ExpensiveSphere {
|
||||
type Decision = Vec<f64>;
|
||||
fn objectives(&self) -> ObjectiveSpace {
|
||||
ObjectiveSpace::new(vec![Objective::minimize("f")])
|
||||
}
|
||||
fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
|
||||
// Pretend this is a 5 ms simulation.
|
||||
std::thread::sleep(std::time::Duration::from_millis(5));
|
||||
Evaluation::new(vec![x.iter().map(|v| v * v).sum::<f64>()])
|
||||
}
|
||||
}
|
||||
|
||||
fn main() {
|
||||
let bounds = vec![(-1.0_f64, 1.0_f64); 5];
|
||||
let mut opt = DifferentialEvolution::new(
|
||||
DifferentialEvolutionConfig {
|
||||
population_size: 16,
|
||||
generations: 50,
|
||||
differential_weight: 0.5,
|
||||
crossover_probability: 0.9,
|
||||
seed: 42,
|
||||
},
|
||||
RealBounds::new(bounds),
|
||||
);
|
||||
let r = opt.run(&ExpensiveSphere);
|
||||
println!("best f = {}", r.best.unwrap().evaluation.objectives[0]);
|
||||
}
|
||||
```
|
||||
|
||||
With the `parallel` feature on, each generation's 16 evaluations run
|
||||
across rayon's worker threads. On a 16-core machine the wall-clock
|
||||
cost per generation drops from `16 × 5 ms = 80 ms` to roughly
|
||||
`5 ms + scheduling overhead`.
|
||||
|
||||
## Sizing your thread pool
|
||||
|
||||
heuropt uses rayon's global thread pool. Override the size with:
|
||||
|
||||
```rust,ignore
|
||||
rayon::ThreadPoolBuilder::new().num_threads(8).build_global().unwrap();
|
||||
```
|
||||
|
||||
Run this **before** any heuropt call, or use rayon's `install` API
|
||||
to scope it.
|
||||
|
||||
## When parallelism *doesn't* help
|
||||
|
||||
- Your `evaluate` is sub-microsecond (Sphere, Rastrigin, Ackley
|
||||
unweighted) — the rayon scheduling overhead exceeds the work.
|
||||
- You're already running multiple seeds in parallel at the harness
|
||||
level (see [Compare two algorithms](./compare.md)). Stacking
|
||||
parallelism rarely helps.
|
||||
- The algorithm is steady-state (Paes, SA, hill climber).
|
||||
|
||||
[`RandomSearch`]: https://docs.rs/heuropt/latest/heuropt/algorithms/random_search/struct.RandomSearch.html
|
||||
[`Nsga2`]: https://docs.rs/heuropt/latest/heuropt/algorithms/nsga2/struct.Nsga2.html
|
||||
[`Nsga3`]: https://docs.rs/heuropt/latest/heuropt/algorithms/nsga3/struct.Nsga3.html
|
||||
[`Spea2`]: https://docs.rs/heuropt/latest/heuropt/algorithms/spea2/struct.Spea2.html
|
||||
[`Moead`]: https://docs.rs/heuropt/latest/heuropt/algorithms/moead/struct.Moead.html
|
||||
[`Mopso`]: https://docs.rs/heuropt/latest/heuropt/algorithms/mopso/struct.Mopso.html
|
||||
[`Ibea`]: https://docs.rs/heuropt/latest/heuropt/algorithms/ibea/struct.Ibea.html
|
||||
[`SmsEmoa`]: https://docs.rs/heuropt/latest/heuropt/algorithms/sms_emoa/struct.SmsEmoa.html
|
||||
[`HypE`]: https://docs.rs/heuropt/latest/heuropt/algorithms/hype/struct.Hype.html
|
||||
[`PesaII`]: https://docs.rs/heuropt/latest/heuropt/algorithms/pesa2/struct.PesaII.html
|
||||
[`EpsilonMoea`]: https://docs.rs/heuropt/latest/heuropt/algorithms/epsilon_moea/struct.EpsilonMoea.html
|
||||
[`AgeMoea`]: https://docs.rs/heuropt/latest/heuropt/algorithms/age_moea/struct.AgeMoea.html
|
||||
[`Knea`]: https://docs.rs/heuropt/latest/heuropt/algorithms/knea/struct.Knea.html
|
||||
[`Grea`]: https://docs.rs/heuropt/latest/heuropt/algorithms/grea/struct.Grea.html
|
||||
[`Rvea`]: https://docs.rs/heuropt/latest/heuropt/algorithms/rvea/struct.Rvea.html
|
||||
[`DifferentialEvolution`]: https://docs.rs/heuropt/latest/heuropt/algorithms/differential_evolution/struct.DifferentialEvolution.html
|
||||
[`GeneticAlgorithm`]: https://docs.rs/heuropt/latest/heuropt/algorithms/genetic_algorithm/struct.GeneticAlgorithm.html
|
||||
[`Paes`]: https://docs.rs/heuropt/latest/heuropt/algorithms/paes/struct.Paes.html
|
||||
[`SimulatedAnnealing`]: https://docs.rs/heuropt/latest/heuropt/algorithms/simulated_annealing/struct.SimulatedAnnealing.html
|
||||
[`HillClimber`]: https://docs.rs/heuropt/latest/heuropt/algorithms/hill_climber/struct.HillClimber.html
|
||||
[`OnePlusOneEs`]: https://docs.rs/heuropt/latest/heuropt/algorithms/one_plus_one_es/struct.OnePlusOneEs.html
|
||||
@@ -0,0 +1,167 @@
|
||||
# Optimize a permutation (TSP-style)
|
||||
|
||||
When your decision is "an ordering" — visiting cities, scheduling
|
||||
jobs, routing — the natural representation is `Vec<usize>` and the
|
||||
specialized algorithm is [`AntColonyTsp`]. Generic alternatives are
|
||||
[`SimulatedAnnealing`] + [`SwapMutation`] for any permutation, and
|
||||
[`TabuSearch`] when you have a custom neighbor function.
|
||||
|
||||
## TSP with `AntColonyTsp`
|
||||
|
||||
```rust,no_run
|
||||
use heuropt::prelude::*;
|
||||
|
||||
struct Tsp {
|
||||
distances: Vec<Vec<f64>>,
|
||||
}
|
||||
|
||||
impl Problem for Tsp {
|
||||
type Decision = Vec<usize>;
|
||||
|
||||
fn objectives(&self) -> ObjectiveSpace {
|
||||
ObjectiveSpace::new(vec![Objective::minimize("length")])
|
||||
}
|
||||
|
||||
fn evaluate(&self, tour: &Vec<usize>) -> Evaluation {
|
||||
let mut len = 0.0;
|
||||
for w in tour.windows(2) {
|
||||
len += self.distances[w[0]][w[1]];
|
||||
}
|
||||
len += self.distances[*tour.last().unwrap()][tour[0]];
|
||||
Evaluation::new(vec![len])
|
||||
}
|
||||
}
|
||||
|
||||
fn main() {
|
||||
// 5-city Euclidean instance
|
||||
let cities = vec![
|
||||
(0.0, 0.0),
|
||||
(1.0, 5.0),
|
||||
(5.0, 2.0),
|
||||
(6.0, 6.0),
|
||||
(8.0, 3.0),
|
||||
];
|
||||
let n = cities.len();
|
||||
let mut distances = vec![vec![0.0; n]; n];
|
||||
for i in 0..n {
|
||||
for j in 0..n {
|
||||
let dx = cities[i].0 - cities[j].0;
|
||||
let dy = cities[i].1 - cities[j].1;
|
||||
distances[i][j] = (dx * dx + dy * dy).sqrt();
|
||||
}
|
||||
}
|
||||
let problem = Tsp { distances: distances.clone() };
|
||||
|
||||
let mut opt = AntColonyTsp::new(AntColonyTspConfig {
|
||||
ants: 20,
|
||||
iterations: 200,
|
||||
alpha: 1.0,
|
||||
beta: 5.0,
|
||||
evaporation: 0.5,
|
||||
deposit: 1.0,
|
||||
distances,
|
||||
seed: 42,
|
||||
});
|
||||
|
||||
let r = opt.run(&problem);
|
||||
let best = r.best.unwrap();
|
||||
println!("best tour length: {:.3}", best.evaluation.objectives[0]);
|
||||
println!("tour: {:?}", best.decision);
|
||||
}
|
||||
```
|
||||
|
||||
`alpha` weights pheromone influence and `beta` weights the
|
||||
heuristic (1 / distance). `evaporation` is the per-iteration decay
|
||||
of pheromone trails. The classic Dorigo paper uses `alpha = 1`,
|
||||
`beta = 2..5`, `evaporation = 0.1..0.5`.
|
||||
|
||||
## Generic permutation: SA + SwapMutation
|
||||
|
||||
Use this when your problem isn't TSP-shaped (no distance matrix
|
||||
makes sense) but you still want to optimize an ordering.
|
||||
|
||||
```rust,no_run
|
||||
use heuropt::prelude::*;
|
||||
|
||||
struct JobShop {
|
||||
process_times: Vec<f64>,
|
||||
}
|
||||
impl Problem for JobShop {
|
||||
type Decision = Vec<usize>;
|
||||
fn objectives(&self) -> ObjectiveSpace {
|
||||
ObjectiveSpace::new(vec![Objective::minimize("makespan")])
|
||||
}
|
||||
fn evaluate(&self, schedule: &Vec<usize>) -> Evaluation {
|
||||
// Pretend cumulative weighted-completion-time. Replace with your real cost.
|
||||
let cost: f64 = schedule.iter().enumerate()
|
||||
.map(|(i, &job)| (i as f64 + 1.0) * self.process_times[job])
|
||||
.sum();
|
||||
Evaluation::new(vec![cost])
|
||||
}
|
||||
}
|
||||
|
||||
fn make_initial_perm(n: usize, seed: u64) -> Vec<usize> {
|
||||
use rand::seq::SliceRandom;
|
||||
let mut rng = rng_from_seed(seed);
|
||||
let mut perm: Vec<usize> = (0..n).collect();
|
||||
perm.shuffle(&mut rng);
|
||||
perm
|
||||
}
|
||||
|
||||
let times = vec![3.0, 1.5, 4.2, 2.7, 5.1];
|
||||
let problem = JobShop { process_times: times.clone() };
|
||||
|
||||
// SimulatedAnnealing needs a starting decision; pass a custom Initializer.
|
||||
struct OnePerm(Vec<usize>);
|
||||
impl Initializer<Vec<usize>> for OnePerm {
|
||||
fn initialize(&mut self, _size: usize, _rng: &mut Rng) -> Vec<Vec<usize>> {
|
||||
vec![self.0.clone()]
|
||||
}
|
||||
}
|
||||
|
||||
let mut opt = SimulatedAnnealing::new(
|
||||
SimulatedAnnealingConfig {
|
||||
iterations: 2000,
|
||||
initial_temperature: 5.0,
|
||||
final_temperature: 1e-3,
|
||||
seed: 7,
|
||||
},
|
||||
OnePerm(make_initial_perm(times.len(), 7)),
|
||||
SwapMutation,
|
||||
);
|
||||
let r = opt.run(&problem);
|
||||
let best = r.best.unwrap();
|
||||
println!("best makespan: {:.3}", best.evaluation.objectives[0]);
|
||||
println!("schedule: {:?}", best.decision);
|
||||
```
|
||||
|
||||
`SwapMutation` swaps two random indices in the permutation —
|
||||
preserves the "every element appears once" invariant for free.
|
||||
|
||||
## Custom neighborhoods: `TabuSearch`
|
||||
|
||||
When swap isn't the right move set (e.g., 2-opt for TSP, insert /
|
||||
shift for scheduling), use [`TabuSearch`] with your own neighbor
|
||||
function.
|
||||
|
||||
```rust,ignore
|
||||
use heuropt::prelude::*;
|
||||
let neighbors = |x: &Vec<usize>, _rng: &mut Rng| -> Vec<Vec<usize>> {
|
||||
// Generate all 2-opt neighbors of x.
|
||||
let mut out = Vec::new();
|
||||
for i in 0..x.len() {
|
||||
for j in (i + 2)..x.len() {
|
||||
let mut child = x.clone();
|
||||
child[i + 1..=j].reverse();
|
||||
out.push(child);
|
||||
}
|
||||
}
|
||||
out
|
||||
};
|
||||
// Pass `neighbors` to TabuSearch::new(...).
|
||||
```
|
||||
|
||||
[`AntColonyTsp`]: https://docs.rs/heuropt/latest/heuropt/algorithms/ant_colony_tsp/struct.AntColonyTsp.html
|
||||
[`SimulatedAnnealing`]: https://docs.rs/heuropt/latest/heuropt/algorithms/simulated_annealing/struct.SimulatedAnnealing.html
|
||||
[`SwapMutation`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.SwapMutation.html
|
||||
[`TabuSearch`]: https://docs.rs/heuropt/latest/heuropt/algorithms/tabu_search/struct.TabuSearch.html
|
||||
@@ -0,0 +1,127 @@
|
||||
# Pick one answer off a Pareto front
|
||||
|
||||
A multi-objective optimizer hands you a *front* — a Pareto-optimal
|
||||
trade-off curve — not a single answer. Eventually you have to pick
|
||||
*one* point off it. There are several principled ways to do that;
|
||||
this recipe covers the most common: the **a-posteriori weighted
|
||||
decision rule**.
|
||||
|
||||
The pattern: optimize *without* baking your preferences into the
|
||||
search, then apply your preferences as a scoring function over the
|
||||
front.
|
||||
|
||||
This is exactly the pattern from `examples/jiggly_tuning.rs` (the
|
||||
USB-jiggler firmware tuning example).
|
||||
|
||||
## The shape
|
||||
|
||||
```rust,no_run
|
||||
use heuropt::prelude::*;
|
||||
|
||||
# struct Cost;
|
||||
# impl Problem for Cost {
|
||||
# type Decision = Vec<f64>;
|
||||
# fn objectives(&self) -> ObjectiveSpace {
|
||||
# ObjectiveSpace::new(vec![Objective::minimize("a"), Objective::minimize("b"), Objective::minimize("c")])
|
||||
# }
|
||||
# fn evaluate(&self, _x: &Vec<f64>) -> Evaluation { Evaluation::new(vec![0.0,0.0,0.0]) }
|
||||
# }
|
||||
|
||||
let problem = Cost;
|
||||
let mut opt = Nsga2::new(
|
||||
Nsga2Config { population_size: 100, generations: 200, seed: 42 },
|
||||
RealBounds::new(vec![(-1.0, 1.0); 4]),
|
||||
CompositeVariation {
|
||||
crossover: SimulatedBinaryCrossover::new(vec![(-1.0, 1.0); 4], 15.0, 0.5),
|
||||
mutation: PolynomialMutation::new(vec![(-1.0, 1.0); 4], 20.0, 1.0),
|
||||
},
|
||||
);
|
||||
let result = opt.run(&problem);
|
||||
|
||||
// 1. Get the Pareto front.
|
||||
let front = &result.pareto_front;
|
||||
|
||||
// 2. Define your preferences as a scoring function over (oriented)
|
||||
// objective values. Lower score = preferred.
|
||||
let space = problem.objectives();
|
||||
let weights = [1.0, 2.0, 0.5];
|
||||
|
||||
let scored: Vec<(f64, &Candidate<Vec<f64>>)> = front.iter()
|
||||
.map(|c| {
|
||||
let oriented = space.as_minimization(&c.evaluation.objectives);
|
||||
let score: f64 = oriented.iter().zip(&weights)
|
||||
.map(|(v, w)| v * w)
|
||||
.sum();
|
||||
(score, c)
|
||||
})
|
||||
.collect();
|
||||
|
||||
// 3. Pick the lowest-scoring point.
|
||||
let best = scored.iter()
|
||||
.min_by(|a, b| a.0.partial_cmp(&b.0).unwrap())
|
||||
.unwrap();
|
||||
|
||||
println!("picked: {:?} with weighted score {:.3}",
|
||||
best.1.evaluation.objectives, best.0);
|
||||
```
|
||||
|
||||
`as_minimization` returns the objective vector with maximized axes
|
||||
flipped to negative — so a single set of *positive* weights does
|
||||
the right thing whether each axis is min or max.
|
||||
|
||||
## Why a-posteriori vs a-priori weighting
|
||||
|
||||
If you know your weights up front, you could just optimize the
|
||||
weighted sum directly with a single-objective algorithm. Why bother
|
||||
with the multi-objective dance?
|
||||
|
||||
Two reasons:
|
||||
|
||||
1. **Weighted sum can't reach concave parts of the Pareto front.**
|
||||
Any single-objective optimization with a linear scalarization
|
||||
converges to a point at the boundary of the convex hull. Concave
|
||||
front segments are unreachable. The multi-objective optimizer
|
||||
finds them.
|
||||
2. **Weights are usually wrong on the first try.** Optimizing the
|
||||
front first lets you see what's actually possible before deciding
|
||||
how much each axis is worth. Run once, look at the trade-offs,
|
||||
adjust weights.
|
||||
|
||||
## Penalty terms beyond linear weights
|
||||
|
||||
The jiggly example also adds a *hinge penalty* — a term that's zero
|
||||
inside an acceptable region and grows quadratically once you exceed
|
||||
some hard cap. Useful when one axis is "soft up to X, hard cap at Y":
|
||||
|
||||
```rust,no_run
|
||||
fn hinge(x: f64, soft_cap: f64, hard_cap: f64) -> f64 {
|
||||
if x <= soft_cap { 0.0 }
|
||||
else if x >= hard_cap { f64::INFINITY }
|
||||
else {
|
||||
let t = (x - soft_cap) / (hard_cap - soft_cap);
|
||||
100.0 * t * t
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Compose linear weights + hinge penalties and you have a flexible
|
||||
scoring function over the front without re-running the optimizer.
|
||||
|
||||
## Other strategies
|
||||
|
||||
- **Knee point.** Pick the point where small gains in one axis cost
|
||||
large losses in another — the "elbow" of the trade-off curve.
|
||||
[`Knea`] explicitly biases the search toward knees during the run.
|
||||
- **Reference-direction.** Pick the point closest to a desired
|
||||
trade-off direction (a unit vector in objective space).
|
||||
[`Moead`] / [`Nsga3`] use this internally during search; you can
|
||||
apply it post-hoc the same way.
|
||||
- **Random / interactive selection.** Show the front to a user
|
||||
(perhaps via a plotting library), let them pick.
|
||||
|
||||
The right pick depends on the problem; the front itself doesn't
|
||||
prescribe one.
|
||||
|
||||
[`Knea`]: https://docs.rs/heuropt/latest/heuropt/algorithms/knea/struct.Knea.html
|
||||
[`Moead`]: https://docs.rs/heuropt/latest/heuropt/algorithms/moead/struct.Moead.html
|
||||
[`Nsga3`]: https://docs.rs/heuropt/latest/heuropt/algorithms/nsga3/struct.Nsga3.html
|
||||
@@ -0,0 +1,244 @@
|
||||
# Defining a problem
|
||||
|
||||
Everything in heuropt starts with the [`Problem`] trait. This chapter
|
||||
walks through every shape it can take.
|
||||
|
||||
## The trait
|
||||
|
||||
```rust,ignore
|
||||
pub trait Problem {
|
||||
type Decision: Clone;
|
||||
fn objectives(&self) -> ObjectiveSpace;
|
||||
fn evaluate(&self, decision: &Self::Decision) -> Evaluation;
|
||||
}
|
||||
```
|
||||
|
||||
Three things you decide:
|
||||
|
||||
1. **`Decision`** — the type of the thing you're optimizing.
|
||||
`Vec<f64>` is by far the most common; `Vec<bool>` for binary
|
||||
search, `Vec<usize>` for permutations, your own struct for
|
||||
anything else.
|
||||
2. **`objectives`** — how many objectives you have, what they're
|
||||
called, and whether each is minimized or maximized. Returned as
|
||||
an [`ObjectiveSpace`].
|
||||
3. **`evaluate`** — given one decision, score it. Returns an
|
||||
[`Evaluation`] with a vector of objective values (and optionally
|
||||
a constraint-violation scalar).
|
||||
|
||||
`evaluate` takes `&self`, so caches and lookup tables are easy. It
|
||||
is called many thousands of times during a typical run, so keep it
|
||||
fast.
|
||||
|
||||
## Single-objective continuous
|
||||
|
||||
The Rosenbrock banana — minimize a smooth non-convex valley.
|
||||
|
||||
```rust,no_run
|
||||
use heuropt::prelude::*;
|
||||
|
||||
struct Rosenbrock;
|
||||
|
||||
impl Problem for Rosenbrock {
|
||||
type Decision = Vec<f64>;
|
||||
|
||||
fn objectives(&self) -> ObjectiveSpace {
|
||||
ObjectiveSpace::new(vec![Objective::minimize("f")])
|
||||
}
|
||||
|
||||
fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
|
||||
let f: f64 = x.windows(2)
|
||||
.map(|w| 100.0 * (w[1] - w[0].powi(2)).powi(2) + (1.0 - w[0]).powi(2))
|
||||
.sum();
|
||||
Evaluation::new(vec![f])
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Multi-objective
|
||||
|
||||
ZDT1 — two objectives that conflict. The Pareto front is the set of
|
||||
non-dominated trade-offs.
|
||||
|
||||
```rust,no_run
|
||||
use heuropt::prelude::*;
|
||||
|
||||
struct Zdt1 { dim: usize }
|
||||
|
||||
impl Problem for Zdt1 {
|
||||
type Decision = Vec<f64>;
|
||||
|
||||
fn objectives(&self) -> ObjectiveSpace {
|
||||
ObjectiveSpace::new(vec![
|
||||
Objective::minimize("f1"),
|
||||
Objective::minimize("f2"),
|
||||
])
|
||||
}
|
||||
|
||||
fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
|
||||
let n = x.len() as f64;
|
||||
let f1 = x[0];
|
||||
let g = 1.0 + 9.0 * x[1..].iter().sum::<f64>() / (n - 1.0);
|
||||
let h = 1.0 - (f1 / g).sqrt();
|
||||
let f2 = g * h;
|
||||
Evaluation::new(vec![f1, f2])
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
For multi-objective problems, pick a Pareto-aware optimizer:
|
||||
[`Nsga2`] is the canonical default; [`Mopso`] often wins on
|
||||
smooth-front 2-objective problems; [`Ibea`] often wins on
|
||||
disconnected fronts. See [choosing-an-algorithm](./choosing-an-algorithm.md).
|
||||
|
||||
## Maximizing instead of minimizing
|
||||
|
||||
heuropt's internals normalize everything to minimization, but you
|
||||
declare your objective with the orientation that's natural for your
|
||||
problem. A scoring problem might want to maximize:
|
||||
|
||||
```rust,no_run
|
||||
use heuropt::prelude::*;
|
||||
let space = ObjectiveSpace::new(vec![
|
||||
Objective::minimize("cost"),
|
||||
Objective::maximize("accuracy"),
|
||||
]);
|
||||
```
|
||||
|
||||
`Objective::maximize` is a convenience for `Direction::Maximize`. Mix
|
||||
freely; the Pareto-comparison machinery handles the orientation.
|
||||
|
||||
## Constraints
|
||||
|
||||
heuropt models constraints as a single non-negative scalar
|
||||
**`constraint_violation`** on each `Evaluation`. The convention:
|
||||
|
||||
- `0.0` (or negative) means **feasible**.
|
||||
- Any positive value means **infeasible**, and bigger numbers are
|
||||
worse violations.
|
||||
|
||||
Pareto-comparison and tournament-selection helpers prefer feasible
|
||||
candidates and break ties on the violation magnitude — so the rule
|
||||
"feasibility comes first" is enforced automatically.
|
||||
|
||||
```rust,no_run
|
||||
use heuropt::prelude::*;
|
||||
|
||||
struct Constrained;
|
||||
impl Problem for Constrained {
|
||||
type Decision = Vec<f64>;
|
||||
|
||||
fn objectives(&self) -> ObjectiveSpace {
|
||||
ObjectiveSpace::new(vec![Objective::minimize("f")])
|
||||
}
|
||||
|
||||
fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
|
||||
let f: f64 = x.iter().map(|v| v * v).sum();
|
||||
|
||||
// Constraint: x[0] + x[1] >= 1. Violation = how much we miss it by.
|
||||
let g1 = (1.0 - (x[0] + x[1])).max(0.0);
|
||||
let total_violation: f64 = g1; // sum of max(0, gᵢ) for each constraint
|
||||
|
||||
Evaluation::constrained(vec![f], total_violation)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
If your constraints are very tight and the search keeps hitting them,
|
||||
see [Constrain your search with `Repair`](./cookbook/constraints.md).
|
||||
|
||||
## Decision types beyond `Vec<f64>`
|
||||
|
||||
### Binary (`Vec<bool>`)
|
||||
|
||||
```rust,no_run
|
||||
use heuropt::prelude::*;
|
||||
|
||||
struct OneMax { bits: usize }
|
||||
impl Problem for OneMax {
|
||||
type Decision = Vec<bool>;
|
||||
fn objectives(&self) -> ObjectiveSpace {
|
||||
ObjectiveSpace::new(vec![Objective::maximize("ones")])
|
||||
}
|
||||
fn evaluate(&self, x: &Vec<bool>) -> Evaluation {
|
||||
Evaluation::new(vec![x.iter().filter(|b| **b).count() as f64])
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
For `Vec<bool>` problems, [`Umda`] is a parameter-free EDA;
|
||||
[`GeneticAlgorithm`] with [`BitFlipMutation`] is the GA route.
|
||||
|
||||
### Permutations (`Vec<usize>`)
|
||||
|
||||
```rust,no_run
|
||||
use heuropt::prelude::*;
|
||||
|
||||
struct Tsp { distances: Vec<Vec<f64>> }
|
||||
impl Problem for Tsp {
|
||||
type Decision = Vec<usize>;
|
||||
fn objectives(&self) -> ObjectiveSpace {
|
||||
ObjectiveSpace::new(vec![Objective::minimize("length")])
|
||||
}
|
||||
fn evaluate(&self, tour: &Vec<usize>) -> Evaluation {
|
||||
let mut len = 0.0;
|
||||
for w in tour.windows(2) {
|
||||
len += self.distances[w[0]][w[1]];
|
||||
}
|
||||
len += self.distances[*tour.last().unwrap()][tour[0]];
|
||||
Evaluation::new(vec![len])
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
For permutations, [`AntColonyTsp`] specializes on TSP-style problems;
|
||||
[`TabuSearch`] takes a user-supplied neighbor function for arbitrary
|
||||
discrete neighborhoods; [`SimulatedAnnealing`] with [`SwapMutation`]
|
||||
is the simplest baseline.
|
||||
|
||||
### Custom decision types
|
||||
|
||||
Any `Clone` type works. If you have a struct, just implement `Clone`
|
||||
and you can use it. You'll need to write your own `Variation` impl
|
||||
to mutate it; see [Write your own algorithm](./cookbook/custom-optimizer.md).
|
||||
|
||||
## What `Evaluation` carries
|
||||
|
||||
```rust,ignore
|
||||
pub struct Evaluation {
|
||||
pub objectives: Vec<f64>, // one entry per objective
|
||||
pub constraint_violation: f64, // 0.0 = feasible
|
||||
}
|
||||
```
|
||||
|
||||
That's it. Construct with [`Evaluation::new`] for unconstrained
|
||||
problems or [`Evaluation::constrained`] when you have a violation.
|
||||
|
||||
## Summary
|
||||
|
||||
- Implement [`Problem`] with your decision type.
|
||||
- Declare objectives via [`ObjectiveSpace`] (mix minimize/maximize
|
||||
freely).
|
||||
- Return an [`Evaluation`] from `evaluate`.
|
||||
- For constraints, set `constraint_violation > 0` for infeasible
|
||||
decisions; heuropt's selection helpers prefer feasibles
|
||||
automatically.
|
||||
|
||||
Next: [Choosing an algorithm](./choosing-an-algorithm.md) walks
|
||||
through the decision tree.
|
||||
|
||||
[`Problem`]: https://docs.rs/heuropt/latest/heuropt/core/problem/trait.Problem.html
|
||||
[`ObjectiveSpace`]: https://docs.rs/heuropt/latest/heuropt/core/objective/struct.ObjectiveSpace.html
|
||||
[`Evaluation`]: https://docs.rs/heuropt/latest/heuropt/core/evaluation/struct.Evaluation.html
|
||||
[`Evaluation::new`]: https://docs.rs/heuropt/latest/heuropt/core/evaluation/struct.Evaluation.html#method.new
|
||||
[`Evaluation::constrained`]: https://docs.rs/heuropt/latest/heuropt/core/evaluation/struct.Evaluation.html#method.constrained
|
||||
[`Nsga2`]: https://docs.rs/heuropt/latest/heuropt/algorithms/nsga2/struct.Nsga2.html
|
||||
[`Mopso`]: https://docs.rs/heuropt/latest/heuropt/algorithms/mopso/struct.Mopso.html
|
||||
[`Ibea`]: https://docs.rs/heuropt/latest/heuropt/algorithms/ibea/struct.Ibea.html
|
||||
[`Umda`]: https://docs.rs/heuropt/latest/heuropt/algorithms/umda/struct.Umda.html
|
||||
[`GeneticAlgorithm`]: https://docs.rs/heuropt/latest/heuropt/algorithms/genetic_algorithm/struct.GeneticAlgorithm.html
|
||||
[`BitFlipMutation`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.BitFlipMutation.html
|
||||
[`AntColonyTsp`]: https://docs.rs/heuropt/latest/heuropt/algorithms/ant_colony_tsp/struct.AntColonyTsp.html
|
||||
[`TabuSearch`]: https://docs.rs/heuropt/latest/heuropt/algorithms/tabu_search/struct.TabuSearch.html
|
||||
[`SimulatedAnnealing`]: https://docs.rs/heuropt/latest/heuropt/algorithms/simulated_annealing/struct.SimulatedAnnealing.html
|
||||
[`SwapMutation`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.SwapMutation.html
|
||||
@@ -0,0 +1,127 @@
|
||||
# Five-minute walkthrough
|
||||
|
||||
The shortest path from a fresh project to a working optimizer.
|
||||
|
||||
## 1. Add heuropt to your `Cargo.toml`
|
||||
|
||||
```toml
|
||||
[dependencies]
|
||||
heuropt = "0.5"
|
||||
```
|
||||
|
||||
The default feature set is small. Optional features:
|
||||
|
||||
- `parallel` — rayon-backed parallel population evaluation.
|
||||
- `serde` — `Serialize` / `Deserialize` derives on the core data
|
||||
types.
|
||||
|
||||
```toml
|
||||
heuropt = { version = "0.5", features = ["parallel"] }
|
||||
```
|
||||
|
||||
## 2. Define a problem
|
||||
|
||||
A problem is a struct that implements the [`Problem`] trait. You tell
|
||||
heuropt what kind of decision your problem takes (`Vec<f64>`,
|
||||
`Vec<bool>`, …), what objectives it has (minimize or maximize), and
|
||||
how to score one decision.
|
||||
|
||||
```rust,no_run
|
||||
use heuropt::prelude::*;
|
||||
|
||||
struct Sphere;
|
||||
|
||||
impl Problem for Sphere {
|
||||
type Decision = Vec<f64>;
|
||||
|
||||
fn objectives(&self) -> ObjectiveSpace {
|
||||
ObjectiveSpace::new(vec![Objective::minimize("f")])
|
||||
}
|
||||
|
||||
fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
|
||||
let f: f64 = x.iter().map(|v| v * v).sum();
|
||||
Evaluation::new(vec![f])
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The Sphere function is a single-objective continuous problem: minimize
|
||||
`f(x) = Σ xᵢ²`. The optimum is `x = 0`, `f = 0`.
|
||||
|
||||
## 3. Pick an algorithm and run it
|
||||
|
||||
For a smooth single-objective continuous problem, [`CmaEs`] is a
|
||||
strong default. Configure it, build it, run it.
|
||||
|
||||
```rust,no_run
|
||||
# use heuropt::prelude::*;
|
||||
# struct Sphere;
|
||||
# impl Problem for Sphere {
|
||||
# type Decision = Vec<f64>;
|
||||
# fn objectives(&self) -> ObjectiveSpace {
|
||||
# ObjectiveSpace::new(vec![Objective::minimize("f")])
|
||||
# }
|
||||
# fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
|
||||
# Evaluation::new(vec![x.iter().map(|v| v * v).sum::<f64>()])
|
||||
# }
|
||||
# }
|
||||
let bounds = RealBounds::new(vec![(-5.0, 5.0); 5]); // 5-dim search box
|
||||
|
||||
let mut opt = CmaEs::new(
|
||||
CmaEsConfig {
|
||||
population_size: 12,
|
||||
generations: 80,
|
||||
initial_sigma: 1.0,
|
||||
eigen_decomposition_period: 1,
|
||||
initial_mean: None,
|
||||
seed: 42,
|
||||
},
|
||||
bounds,
|
||||
);
|
||||
|
||||
let result = opt.run(&Sphere);
|
||||
|
||||
let best = result.best.expect("at least one feasible candidate");
|
||||
println!("best f = {:.3e} at x = {:?}", best.evaluation.objectives[0], best.decision);
|
||||
```
|
||||
|
||||
Run with `cargo run --release` — heuristic optimization is allergic
|
||||
to debug builds. Expect output like:
|
||||
|
||||
```text
|
||||
best f = 1.4e-29 at x = [-1.6e-15, 4.5e-16, ...]
|
||||
```
|
||||
|
||||
CMA-ES drops to machine epsilon on the Sphere in well under 80
|
||||
generations.
|
||||
|
||||
## 4. What just happened
|
||||
|
||||
- [`Problem`] is the **what** you're optimizing.
|
||||
- [`CmaEs`] (or any other optimizer) is the **how**.
|
||||
- [`CmaEsConfig`] is a plain public-field struct: there are no
|
||||
builders, no chained setters, just public fields you set
|
||||
directly.
|
||||
- [`Optimizer::run`] returns an [`OptimizationResult`] containing the
|
||||
full final `population`, the `pareto_front` (just the best for
|
||||
single-objective), the `best` candidate, the total `evaluations`,
|
||||
and the number of `generations`.
|
||||
|
||||
## 5. Where to go next
|
||||
|
||||
- **Multi-objective:** see [Defining a problem](./defining-problems.md)
|
||||
for how to express two or more objectives, and
|
||||
[Choosing an algorithm](./choosing-an-algorithm.md) for which
|
||||
optimizer fits.
|
||||
- **Want to know which algorithm to pick:** read the README's
|
||||
decision tree, or jump straight to the [choosing-an-algorithm](./choosing-an-algorithm.md)
|
||||
chapter for the long form.
|
||||
- **Production patterns:** the [cookbook](./cookbook.md) has recipes
|
||||
for parallelism, expensive evaluations, comparing algorithms, and
|
||||
more.
|
||||
|
||||
[`Problem`]: https://docs.rs/heuropt/latest/heuropt/core/problem/trait.Problem.html
|
||||
[`Optimizer::run`]: https://docs.rs/heuropt/latest/heuropt/traits/trait.Optimizer.html
|
||||
[`OptimizationResult`]: https://docs.rs/heuropt/latest/heuropt/core/result/struct.OptimizationResult.html
|
||||
[`CmaEs`]: https://docs.rs/heuropt/latest/heuropt/algorithms/cma_es/struct.CmaEs.html
|
||||
[`CmaEsConfig`]: https://docs.rs/heuropt/latest/heuropt/algorithms/cma_es/struct.CmaEsConfig.html
|
||||
@@ -0,0 +1,82 @@
|
||||
# Introduction
|
||||
|
||||
heuropt is a practical Rust toolkit for **heuristic optimization** — the
|
||||
art of searching for good answers when the problem is too gnarly to
|
||||
solve analytically.
|
||||
|
||||
The kinds of problems heuropt is built for:
|
||||
|
||||
- **Single-objective:** "find the parameters that minimize the loss of
|
||||
this model." Hyperparameter tuning. Curve fitting. Calibration.
|
||||
- **Multi-objective:** "find the trade-off curve between cost and
|
||||
accuracy." Engineering design. Portfolio optimization. Fleet
|
||||
scheduling.
|
||||
- **Many-objective (4+):** the same idea but with enough objectives
|
||||
that classical Pareto methods break down. Power-grid planning.
|
||||
Airfoil design. Multi-criteria recommendation.
|
||||
|
||||
If your problem is differentiable and convex, you don't need this
|
||||
crate — use a gradient solver. heuropt is for the *messy* problems:
|
||||
landscapes with lots of local minima, decisions that aren't continuous
|
||||
(permutations, bit vectors), or evaluations that are noisy / expensive
|
||||
/ black-box.
|
||||
|
||||
## Why heuropt
|
||||
|
||||
There are other Rust optimization crates and many more in Python (pymoo,
|
||||
hyperopt, optuna, DEAP). heuropt's design priorities:
|
||||
|
||||
1. **Approachable code.** No trait objects in the public API. No
|
||||
GATs, HRTBs, generic-RNG plumbing. A junior Rust engineer should
|
||||
be able to read `RandomSearch` and write a new optimizer by
|
||||
implementing only the `Optimizer<P>` trait.
|
||||
2. **One concrete RNG type.** Seeded determinism is a property tested
|
||||
across the crate; identical inputs always produce identical
|
||||
outputs.
|
||||
3. **Algorithms that work.** Every algorithm is benchmarked against
|
||||
the canonical test problems (ZDT, DTLZ, Rastrigin, Rosenbrock,
|
||||
Ackley) and the results are checked into [examples/compare-results.md](https://github.com/swaits/heuropt/blob/main/examples/compare-results.md)
|
||||
so you can see what each algorithm's strengths actually are.
|
||||
4. **Testing as a first-class concern.** 316+ unit / integration /
|
||||
property tests, eight cargo-fuzz targets in CI, gungraun
|
||||
instruction-count benchmarks. The fuzzers find real bugs and the
|
||||
property tests check actual invariants.
|
||||
|
||||
## What's in the box
|
||||
|
||||
heuropt v0.5 ships **35 algorithms** spanning:
|
||||
|
||||
- Single-objective continuous: `RandomSearch`, `HillClimber`,
|
||||
`OnePlusOneEs`, `SimulatedAnnealing`, `GeneticAlgorithm`,
|
||||
`ParticleSwarm`, `DifferentialEvolution`, `Tlbo`, `CmaEs`,
|
||||
`IpopCmaEs`, `SeparableNes`, `NelderMead`.
|
||||
- Single-objective other types: `Umda` (binary), `TabuSearch`
|
||||
(any), `AntColonyTsp` (permutation).
|
||||
- Multi-objective (2–3): `Paes`, `Nsga2`, `Spea2`, `Mopso`, `Ibea`,
|
||||
`SmsEmoa`, `HypE`, `EpsilonMoea`, `PesaII`, `AgeMoea`, `Knea`,
|
||||
`Moead`.
|
||||
- Many-objective (4+): `Nsga3`, `Rvea`, `Grea`.
|
||||
- Sample-efficient / multi-fidelity: `BayesianOpt`, `Tpe`,
|
||||
`Hyperband`.
|
||||
|
||||
Plus the operators (SBX, PolynomialMutation, BoundedGaussianMutation,
|
||||
LevyMutation, BitFlipMutation, SwapMutation, ClampToBounds,
|
||||
ProjectToSimplex), the metrics (hypervolume, spacing), and the Pareto
|
||||
utilities (dominance, fronts, crowding distance, Das–Dennis reference
|
||||
points, the `ParetoArchive`) that you'd expect.
|
||||
|
||||
## How to use this guide
|
||||
|
||||
If you're new to heuropt, read it linearly:
|
||||
|
||||
1. [Five-minute walkthrough](./getting-started.md) — install, define
|
||||
a problem, run an optimizer, look at the result.
|
||||
2. [Defining a problem](./defining-problems.md) — the `Problem`
|
||||
trait in depth: single- vs multi-objective, constraints, custom
|
||||
decision types.
|
||||
3. [Choosing an algorithm](./choosing-an-algorithm.md) — the
|
||||
decision tree, expanded with the reasoning behind each branch.
|
||||
|
||||
If you're already up and running, jump into the [cookbook](./cookbook.md)
|
||||
for recipes, or [comparison](./comparison.md) for how heuropt stacks
|
||||
up against other libraries.
|
||||
@@ -0,0 +1,69 @@
|
||||
# Migration guides
|
||||
|
||||
Per-release notes for upgrading between heuropt versions. Skip the
|
||||
sections that don't apply to your starting version.
|
||||
|
||||
## To 0.5
|
||||
|
||||
### From 0.4.x
|
||||
|
||||
**No public-API changes.** v0.5 is a documentation-and-polish release.
|
||||
Bumping `heuropt = "0.5"` in your Cargo.toml is enough.
|
||||
|
||||
What changed:
|
||||
|
||||
- Added a comprehensive mdbook user guide (this book).
|
||||
- Added runnable rustdoc examples on every public algorithm,
|
||||
operator, metric, and Pareto utility.
|
||||
- Added real-world `examples/portfolio.rs`,
|
||||
`examples/hyperparam_tuning.rs`, and `examples/scheduling.rs`.
|
||||
- Added `CONTRIBUTING.md`, `SECURITY.md`, `CODE_OF_CONDUCT.md`
|
||||
(Builder's Code of Conduct), GitHub issue templates, and PR
|
||||
template.
|
||||
|
||||
The full list is in CHANGELOG.md.
|
||||
|
||||
### From earlier than 0.4
|
||||
|
||||
If you're coming from 0.3.x or earlier, also read the older sections
|
||||
below.
|
||||
|
||||
## To 0.4
|
||||
|
||||
### From 0.3.x
|
||||
|
||||
**No public-API changes.** v0.4 was a testing-infrastructure
|
||||
expansion + perf pass. Same `cargo update` story.
|
||||
|
||||
The compare-harness wall-clock got 3.27× faster on v0.4 with
|
||||
bit-identical quality metrics, so any benchmark numbers you have
|
||||
from v0.3 are still numerically accurate but will run faster.
|
||||
|
||||
## To 0.3
|
||||
|
||||
### From 0.2.x
|
||||
|
||||
**Additive only.** New algorithms (`BayesianOpt`, `Tpe`,
|
||||
`OnePlusOneEs`, `IpopCmaEs`, `SeparableNes`, `NelderMead`,
|
||||
`Hyperband`), new operators (`LevyMutation`, `ClampToBounds`,
|
||||
`ProjectToSimplex`), new traits (`PartialProblem`, `Repair<D>`).
|
||||
|
||||
`CmaEsConfig` gained an `initial_mean: Option<Vec<f64>>` field;
|
||||
existing call sites need a `.. CmaEsConfig { initial_mean: None,
|
||||
.. }` update.
|
||||
|
||||
## To 0.2
|
||||
|
||||
### From 0.1.x
|
||||
|
||||
**Additive.** New algorithms across the catalog (HillClimber, SA,
|
||||
GA, PSO, CMA-ES, TabuSearch, AntColonyTsp, Umda, TLBO, MOPSO, IBEA,
|
||||
SMS-EMOA, HypE, RVEA, PESA-II, ε-MOEA, AGE-MOEA, GrEA, KnEA), new
|
||||
operators (`SimulatedBinaryCrossover`, `PolynomialMutation`,
|
||||
`CompositeVariation`, `BoundedGaussianMutation`), and the
|
||||
`hypervolume_nd` metric.
|
||||
|
||||
`Optimizer<P>` impls now require `P: Sync` and `P::Decision: Send`
|
||||
(this enables the `parallel` feature without changing the public
|
||||
trait surface). Any normal `Problem` you've written satisfies these
|
||||
bounds automatically.
|
||||
@@ -0,0 +1,96 @@
|
||||
# Stability and SemVer
|
||||
|
||||
heuropt is pre-1.0. The public API may change between minor versions.
|
||||
This page sets explicit expectations.
|
||||
|
||||
## What "public API" means in heuropt
|
||||
|
||||
The crate's public surface is everything re-exported from
|
||||
[`heuropt::prelude`] plus the items reachable from `heuropt::core`,
|
||||
`heuropt::traits`, `heuropt::operators`, `heuropt::algorithms`,
|
||||
`heuropt::pareto`, `heuropt::metrics`, and `heuropt::selection`.
|
||||
|
||||
Items in `heuropt::internal` (e.g. the Cholesky / eigendecomposition
|
||||
helpers) are **not** public API. They may change between any two
|
||||
versions — use them at your own risk.
|
||||
|
||||
## SemVer in heuropt 0.x
|
||||
|
||||
While we are pre-1.0:
|
||||
|
||||
- **Minor bumps (`0.5 → 0.6`) may break the public API.** The
|
||||
CHANGELOG calls out everything that changed, and a **migration
|
||||
guide** in this book documents the move.
|
||||
- **Patch bumps (`0.5.0 → 0.5.1`) only contain bug fixes,
|
||||
performance improvements, and additive non-breaking features.**
|
||||
No deprecations, no removals.
|
||||
|
||||
## What's actually likely to change before 1.0
|
||||
|
||||
In rough order of likelihood:
|
||||
|
||||
1. **`Optimizer<P>` may grow new optional methods** for callbacks,
|
||||
stop conditions, and save/resume support. These will land as
|
||||
methods with default implementations so existing trait impls
|
||||
keep compiling, but the trait shape will be different.
|
||||
2. **Algorithm config structs may gain fields.** All current configs
|
||||
are public-field structs; adding a non-`Default` field is a
|
||||
breaking change. We may switch to builder patterns to avoid this
|
||||
class of break, or we may add `#[non_exhaustive]`.
|
||||
3. **The `Snapshot`, `Observer`, and `Checkpoint` types** (planned
|
||||
for a future release) will land as new public surfaces.
|
||||
4. **Some operators may move between `operators` and `pareto`** as
|
||||
the boundary between "things that produce candidates" and "Pareto
|
||||
utilities" gets clearer.
|
||||
|
||||
What is **not** likely to change:
|
||||
|
||||
- The `Problem` trait shape.
|
||||
- The `Variation` / `Initializer` / `Repair` traits.
|
||||
- The `Evaluation` / `Candidate` / `Population` / `OptimizationResult`
|
||||
data types.
|
||||
- The seeded determinism property.
|
||||
|
||||
## What "bit-identical" means for stability
|
||||
|
||||
heuropt promises that a given algorithm + seed + config produces the
|
||||
same numeric output on the same minor version of heuropt.
|
||||
|
||||
Across minor versions, output may change if an algorithm's
|
||||
implementation changes (e.g. a perf rewrite that reorders
|
||||
floating-point operations, or a new feature that changes the
|
||||
RNG-consumption pattern). The CHANGELOG calls this out explicitly
|
||||
when it happens. As of v0.5, the entire history of perf optimizations
|
||||
has been bit-identical against the v0.3.0 reference.
|
||||
|
||||
## MSRV (minimum supported Rust version)
|
||||
|
||||
heuropt's MSRV is **1.85** as of v0.5. This is tested in CI against
|
||||
every PR.
|
||||
|
||||
MSRV bumps are treated as patch-bump-eligible (they don't break the
|
||||
public API). When the MSRV is bumped, the CHANGELOG entry for that
|
||||
release will note the new MSRV.
|
||||
|
||||
## Feature-flag stability
|
||||
|
||||
The current optional features:
|
||||
|
||||
- `serde` — adds `Serialize` / `Deserialize` derives on the core data
|
||||
types.
|
||||
- `parallel` — rayon-backed parallel population evaluation.
|
||||
|
||||
Features added in 0.x can be renamed or removed in any minor bump
|
||||
that documents the change. Removing a feature is treated like a
|
||||
breaking API change.
|
||||
|
||||
## How to track changes
|
||||
|
||||
- **CHANGELOG.md** — the canonical record of changes per release.
|
||||
- **Migration guides** — per-release, in this book at
|
||||
[migration](./migration.md).
|
||||
- **GitHub releases** — each tag has release notes.
|
||||
- **Watch the repo** — https://github.com/swaits/heuropt — to be
|
||||
notified of new releases.
|
||||
|
||||
[`heuropt::prelude`]: https://docs.rs/heuropt/latest/heuropt/prelude/index.html
|
||||
Reference in New Issue
Block a user