12 Commits
Author SHA1 Message Date
swaits 57a43c260e docs: 0.8.0 release polish — README, guide, changelog
Companion to the feat(async) commit. Brings every cross-referencing
doc up to v0.8 currency, replaces marketing-flavored copy with plain
prose, and replaces toy benchmark problems with relatable ones that
include actual run output and interpretive narrative.

- README: collapses the four-bullet "Read the user guide / API
  reference / Tested with N tests / Hot paths optimized" list into
  a single Docs links line.
- README: replaces the Schaffer-N1 toy problem with a PickACar
  multi-objective design problem — three decision variables
  (displacement, weight, drag), four objectives (price, 0-60,
  fuel, noise), and *nonlinear* cost relationships so the Pareto
  front is a real surface, not a 1D sweep. Includes actual NSGA-III
  run output (representative slice across the 100-car front) and
  a narrative explaining what each row tells you and why hand-
  picking would miss the interesting tradeoffs.
- README: removes rustdoc-style hidden `#` setup lines from code
  blocks. The README is rendered as plain markdown on GitHub /
  crates.io, where those lines are visible garbage instead of
  hidden setup. Code blocks are now self-contained.
- Guide quickstart (getting-started.md): replaces Sphere ( Σ x² )
  with a least-squares LineFit example. Same shape (single-
  objective continuous), but recognizable framing. Includes
  actual CMA-ES output, residual table, and narrative comparing
  the answer to standard regression.
- Algorithm count audit: stale "35 algorithms" claim corrected to
  the actual 33 across README, src/lib.rs, introduction.md, and
  the comparison.md table cell.
- Async feature flag listed in the optional-features sections of
  README, src/lib.rs, getting-started.md.
- introduction.md, choosing-an-algorithm.md, comparison.md,
  stability.md, migration.md, cookbook/parallel.md,
  cookbook/custom-optimizer.md: cross-references updated to
  describe full async coverage and link the new cookbook recipe.
- stability.md: removes the speculative "Observer / Snapshot /
  Checkpoint planned" bullet (those didn't ship); documents the
  AsyncProblem / AsyncPartialProblem trait stability.
- migration.md: new "To 0.8" section with paths from 0.5.x and 0.7.x.
- CHANGELOG: 0.8.0 entry capturing the async feature plus the
  documentation / governance / CI catch-up.
- SECURITY.md: supported versions table reflects 0.8.x.
2026-05-06 11:51:13 -06:00
swaits cbfedd85fa feat(async): add run_async to every algorithm in the catalog
Async coverage was incomplete in 0.7 (only RandomSearch and
DifferentialEvolution had run_async). 0.8 closes the gap: every one
of the 33 algorithms now exposes
run_async(&problem, concurrency).await, gated on the async feature.

- Population-based algorithms fan out per-generation evaluations
  through evaluate_batch_async with concurrency-bounded
  FuturesOrdered chunks.
- Steady-state algorithms (HillClimber, SimulatedAnnealing,
  OnePlusOneEs, Paes, NelderMead) await each step sequentially;
  they accept the concurrency parameter for API uniformity.
- TabuSearch fans out the K-neighbor batch each step.
- Surrogate algorithms (BayesianOpt, Tpe) batch the initial design
  and await per-iteration acquisitions sequentially so the surrogate
  can update between picks.
- Hyperband uses a new AsyncPartialProblem trait (mirroring
  PartialProblem for multi-fidelity workloads) and a parallel
  evaluate_batch_at_budget_async helper; each Successive-Halving
  rung fans out its budgeted evaluations.

All paths preserve seeded determinism: RNG draws happen on the main
task in the same order as the sync path, and only the evaluations
are concurrent.

Adds a dedicated cookbook recipe at docs/book/src/cookbook/async.md
with a worked example (DifferentialEvolution under tokio) and
guidance on picking concurrency. Cross-references in SUMMARY.md
and cookbook.md are updated to surface the new recipe.

The follow-up docs commit reconciles the rest of the user guide
and README to describe the new feature; this commit is the bare
async surface.
2026-05-06 11:51:13 -06:00
swaits d1288aa623 ci(docs): re-enable GitHub Pages deploy
Pages is now enabled on the repo (Settings → Pages → 'Build and
deployment: GitHub Actions'), so the workflow can use the standard
configure-pages → upload-pages-artifact → deploy-pages chain
without needing the GITHUB_TOKEN to enable Pages itself.

PR builds run the build job (catches mdbook breakage) but skip the
deploy job, so PRs don't republish the live site.
2026-05-06 09:04:13 -06:00
swaits af226e3d3b feat: drop heuropt-plot companion crate
Removes the heuropt-plot subcrate, the visualize example that used
it, and the related workspace plumbing (root [workspace] table, the
[workspace] override added to fuzz/Cargo.toml to detach from it,
heuropt-plot dev-dep, CHANGELOG mention).

The visualization concern is better served as an independent third-
party project than as a companion crate in this repo. No effect on
heuropt's public API or the async work in 0.8.0.
2026-05-06 09:04:13 -06:00
swaits cfd5207fb6 ci: drop Pages deploy + loosen simplex-projection fuzz tolerance
Two CI fixes; the previous `enablement: true` attempt didn't work
because the default GITHUB_TOKEN can write to Pages but can't enable
it on a repo that doesn't yet have it configured.

1. .github/workflows/docs.yml: drop the Pages deploy job entirely.
   Build mdbook on every push and upload it as a CI artifact. When
   Pages is enabled manually (Settings → Pages → 'Build and
   deployment: GitHub Actions'), this file can grow back a deploy
   job using actions/configure-pages + actions/deploy-pages.

2. fuzz/fuzz_targets/clamp_to_bounds.rs: the simplex projection's τ
   computation operates on values up to `simplex_total · 1e6` per
   the input filter, so its FP precision floor is ~1e-4 of the
   input scale. Outputs near the `max(x_i − τ, 0)` clamp boundary
   can flip between 0 and a small positive value across
   re-applications without that being a correctness bug. The fuzz
   target is meant to catch *gross* non-idempotence (the all-zeros
   bug that the v0.4 cleanup fixed), not ULP-level slop. Loosen the
   per-element tolerance to `1e-4 · max(simplex_total, max|x_i|, 1)`.
   Verified clean over a 10 M-run soak.
2026-05-06 08:32:22 -06:00
swaits ae1daf687d ci(docs): auto-enable GitHub Pages on first run
The Docs workflow was failing on `actions/configure-pages@v5` with
"Get Pages site failed" because Pages isn't enabled on the repo
yet. Setting `enablement: true` lets the action auto-enable it so
the deploy can proceed without a manual Settings → Pages click.
2026-05-06 08:18:23 -06:00
swaits c1bc3b0528 docs(rustdoc): add runnable examples across operators, metrics, and Pareto utilities
Completes the rustdoc audit — every public item now has at least one
```rust example block in its docstring, exercised by
`cargo test --doc` (55 doctests, all passing).

- Operators: BitFlipMutation, SwapMutation, RealBounds,
  GaussianMutation, BoundedGaussianMutation,
  SimulatedBinaryCrossover, PolynomialMutation, LevyMutation,
  ClampToBounds, ProjectToSimplex.
- Metrics: hypervolume_2d, hypervolume_nd, spacing.
- Pareto utilities: pareto_compare, pareto_front, best_candidate,
  non_dominated_sort, crowding_distance, das_dennis,
  ParetoArchive.

Each example is short (5-15 lines) and self-contained — copy-paste
into a fresh project and it runs.
2026-05-06 08:16:04 -06:00
swaits d564f862d7 ci: fix mdbook edition + isolate fuzz crate from workspace
mdbook 0.4.40 (the version pinned in .github/workflows/docs.yml)
doesn't recognize edition = '2024' under [rust], failing the docs
build. Drop to '2021' for the in-book code blocks; the heuropt
crate itself stays on Rust 2024.

Adding [workspace] to the root Cargo.toml made fuzz/Cargo.toml
inherit it, but fuzz isn't in the members list — every fuzz-smoke
job failed with 'current package believes it's in a workspace when
it's not'. Add an empty [workspace] table at the top of
fuzz/Cargo.toml so cargo treats fuzz as the root of its own
workspace and stops walking up.
2026-05-06 08:15:55 -06:00
swaits 5b5fe50df3 feat(heuropt-plot): v0.1.0 — SVG visualization companion crate
Adds heuropt-plot, a tiny SVG-only plotter that takes heuropt
results and emits scatter plots (pareto_front_svg) and line plots
(convergence_svg). No heavy 'plotters' or 'tiny-skia' dep — hand-
rolled SVG so the crate adds <100 KB to a build.

Workspace setup: root Cargo.toml gains [workspace] with members =
['.', 'heuropt-plot']. heuropt-plot has its own version (0.1.0) and
publishes independently against heuropt 0.8+.

Adds examples/visualize.rs that wires it up: NSGA-II on Schaffer
N.1, plain run() (no observer plumbing), final-front SVG written to
disk.
2026-05-06 07:58:08 -06:00
swaits 6368ca5f3d feat(async): AsyncProblem trait + run_async on RandomSearch and DifferentialEvolution
Adds the headline async/await capability for IO-bound evaluations
(HTTP services, RPC clients, spawned subprocesses) — the
differentiator vs pymoo / hyperopt / MOEA Framework.

No public-API breaks for synchronous users. The new surface is
gated behind a new `async` feature flag.

- core::async_problem::AsyncProblem trait (async fn evaluate_async).
- algorithms::parallel_eval_async::evaluate_batch_async helper using
  futures::stream::FuturesOrdered with concurrency-bounded chunks;
  preserves input order so seeded determinism holds when evaluations
  are themselves deterministic.
- run_async on RandomSearch and DifferentialEvolution.
- examples/async_eval.rs: simulated 20 ms remote service. concurrency=1
  → 4.2 s, concurrency=4 → 2.1 s (2× speedup).

Bumps Cargo.toml to 0.8.0; CHANGELOG entry covers the above plus a
note that 0.6.0/0.7.0 on crates.io are yanked experimentals and 0.8
picks up cleanly from 0.5.
2026-05-06 07:55:56 -06:00
swaits fa3f2e8fb0 feat: v0.5.0 — comprehensive documentation release
Theme: documentation and project polish. No public-API changes; this
is the v0.5 release that elevates heuropt's docs/onboarding/governance
to bar-setting status.

Adds:
- mdbook user guide at docs/book/ with intro, getting-started,
  defining-problems, choosing-an-algorithm, cookbook (7 recipes),
  comparison vs other libraries, stability/SemVer, migration guides.
  Deploys to https://swaits.github.io/heuropt/ via .github/workflows/
  docs.yml.
- Runnable rustdoc examples on every algorithm (35 of them), all
  exercised by cargo test --doc.
- Three real-world examples: portfolio.rs (multi-obj with budget
  constraint), hyperparam_tuning.rs (BO + TPE), scheduling.rs
  (permutation via SA + SwapMutation against Smith's-rule oracle).
- Governance: CONTRIBUTING.md, SECURITY.md, CODE_OF_CONDUCT.md
  (adopting builderscode.org's Builder's Code of Conduct), GitHub
  issue templates, PR template.

Polishes:
- README hero with badges + user-guide link.
- lib.rs crate-level docs.
- CHANGELOG entry for 0.5.0.

Bumps Cargo.toml to 0.5.0.
2026-05-05 14:33:12 -06:00
swaits a9edb0916f ci(fuzz): drop --locked on cargo install cargo-fuzz
cargo-fuzz's bundled Cargo.lock pinned rustix=0.36.5, which used the
now-removed `rustc_attrs` cfg name and broke the install step on
current nightly toolchain (the only toolchain that can build the
fuzzers via libfuzzer-sys). Letting cargo resolve fresh picks a
recent rustix that builds cleanly.

Fixes the fuzz-smoke matrix on the v0.4.0 push CI run.
2026-05-05 13:35:58 -06:00
88 changed files with 8859 additions and 61 deletions
+40
View File
@@ -0,0 +1,40 @@
---
name: Bug report
about: A correctness, performance, or panic bug in heuropt
title: "bug: <one-line summary>"
labels: bug
---
## What happened
<Concise description of the bug.>
## Reproducer
```rust
// Smallest example that demonstrates the bug. Ideally <30 lines and
// runnable as a fresh `examples/repro.rs`. Include the Cargo.toml
// `[features]` you used.
```
Command used:
```sh
cargo run --release --example repro
```
## Expected vs observed
- **Expected:** <what should happen>
- **Observed:** <what actually happens>
## Environment
- heuropt version:
- `rustc --version`:
- OS / arch:
- Feature flags enabled:
## Additional context
<Anything else — fuzz artifact path, screenshots, profiler output.>
+8
View File
@@ -0,0 +1,8 @@
blank_issues_enabled: false
contact_links:
- name: Security vulnerability
url: https://github.com/swaits/heuropt/security/advisories/new
about: Please use private vulnerability reporting — do not open a public issue. See SECURITY.md.
- name: Question / discussion
url: https://github.com/swaits/heuropt/discussions
about: For open-ended questions or design discussions.
+24
View File
@@ -0,0 +1,24 @@
---
name: Docs issue
about: Something in the README, mdbook guide, or rustdoc is wrong, missing, or unclear
title: "docs: <one-line summary>"
labels: documentation
---
## Where
- [ ] `README.md`
- [ ] mdbook user guide (chapter / section: ____ )
- [ ] rustdoc on a specific item (path: ____ )
- [ ] Examples (`examples/____.rs`)
- [ ] CHANGELOG / migration guide
- [ ] Other: ____
## What's wrong
<Concrete description: typo, broken link, outdated code sample,
missing topic, unclear explanation, etc.>
## What it should say (if you know)
<Optional: proposed wording or correct content. Even a sketch helps.>
+39
View File
@@ -0,0 +1,39 @@
---
name: Feature request
about: Propose a new algorithm, operator, metric, or API addition
title: "feat: <one-line summary>"
labels: enhancement
---
## What and why
<What you want, and the problem it solves. If this is a new algorithm
or operator, cite the paper or canonical reference.>
## Proposed API sketch
```rust
// What the public surface would look like — config struct fields,
// trait impl, etc. Doesn't need to be final, just enough to discuss.
```
## Alternatives considered
<Other approaches you thought about and why this one wins. If a
similar feature already exists in heuropt or another Rust crate,
explain how this differs.>
## Scope
- [ ] New trait (will need API discussion)
- [ ] New algorithm
- [ ] New operator
- [ ] New metric / Pareto utility
- [ ] New optional feature flag
- [ ] Change to existing public API (potentially breaking)
## Willing to implement?
- [ ] Yes, I'll send a PR.
- [ ] Yes, but I'd like guidance on the design first.
- [ ] No, I'm reporting the need.
+32
View File
@@ -0,0 +1,32 @@
<!--
Thanks for the contribution! Please skim CONTRIBUTING.md if you
haven't yet — it has the local-test checklist and the conventional-
commits requirement.
-->
## What
<One- or two-sentence summary. Focus on the *what* and *why*, not
the *how*.>
## Why
<Motivation. Link the issue this resolves with `Closes #N` if
applicable.>
## Checklist
- [ ] `cargo fmt --all`
- [ ] `cargo clippy --all-targets --all-features -- -D warnings`
- [ ] `cargo test` and `cargo test --all-features`
- [ ] `cargo doc --no-deps --all-features` (with `-D warnings`)
- [ ] Conventional-commit subject(s) (`<type>(<scope>): <summary>`)
- [ ] If touching algorithm output: confirmed bit-identical results
via `cargo run --release --example compare`
- [ ] If perf change: included gungraun before/after numbers in the
commit message
- [ ] Updated CHANGELOG.md under `[Unreleased]` if user-visible
## Anything else
<Caveats, follow-ups, screenshots, perf numbers, etc.>
+5 -1
View File
@@ -104,6 +104,10 @@ jobs:
with:
workspaces: fuzz -> target
- name: Install cargo-fuzz
run: cargo install cargo-fuzz --locked
# No `--locked`: cargo-fuzz's bundled Cargo.lock pins
# rustix=0.36.5, which uses the now-removed `rustc_attrs` cfg
# name and fails to build on current nightly. Letting cargo
# resolve fresh picks a recent rustix that builds cleanly.
run: cargo install cargo-fuzz
- name: 60-second soak
run: cargo fuzz run ${{ matrix.target }} -- -max_total_time=60
+55
View File
@@ -0,0 +1,55 @@
name: Docs
on:
push:
branches: [main]
tags: ["v*.*.*"]
pull_request:
branches: [main]
workflow_dispatch:
permissions:
contents: read
pages: write
id-token: write
# Only one Pages deploy at a time. Don't cancel a running deploy
# (otherwise we can leave the Pages site partially updated).
concurrency:
group: pages
cancel-in-progress: false
jobs:
build:
name: Build mdbook
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Install mdbook
run: |
mkdir -p ~/.local/bin
curl -sSL "https://github.com/rust-lang/mdBook/releases/download/v0.4.40/mdbook-v0.4.40-x86_64-unknown-linux-musl.tar.gz" \
| tar -xz -C ~/.local/bin
echo "$HOME/.local/bin" >> "$GITHUB_PATH"
- name: Build
run: |
cd docs/book
mdbook build
- uses: actions/configure-pages@v5
- uses: actions/upload-pages-artifact@v3
with:
path: target/book
deploy:
name: Deploy to GitHub Pages
# Only deploy on pushes to main / tag pushes / manual runs.
# PR builds get the build-and-upload step but no deploy.
if: github.event_name != 'pull_request'
needs: build
runs-on: ubuntu-latest
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
steps:
- id: deployment
uses: actions/deploy-pages@v4
+170 -1
View File
@@ -7,6 +7,175 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
## [0.8.0] — 2026-05-06
Theme: async evaluation, plus the docs / governance / CI catch-up
that came with finalizing the release.
heuropt now supports problems where each evaluation is a
`.await`-able operation — HTTP services, RPC clients, spawned
subprocesses. This is the differentiating capability vs.
pymoo / hyperopt / optuna / DEAP / MOEA Framework, none of which
ship first-class async support at the *evaluation* level.
No public-API breaks for synchronous users. The new surface is
gated behind a new `async` feature flag.
### Added
#### Async evaluation (the headline feature)
- New optional feature `async`, gated on
[`futures`](https://crates.io/crates/futures).
- `core::async_problem::AsyncProblem` trait — mirrors `Problem` but
with `async fn evaluate_async(&self, decision)`. Adapt an
existing sync `Problem` with a one-line wrapper.
- `core::async_problem::AsyncPartialProblem` trait — mirrors
`PartialProblem` for multi-fidelity (Hyperband) workloads with
`async fn evaluate_at_budget_async(decision, budget)`.
- Per-algorithm `run_async(&problem, concurrency).await` methods on
**every** algorithm in the catalog — all 33 of them — driving
evaluations through whichever async runtime the caller is using
(typically tokio). `concurrency` bounds in-flight evaluations.
Population-based algorithms (NSGA-II, NSGA-III, SPEA2, MOEA/D,
CMA-ES, DE, GA, PSO, IBEA, SMS-EMOA, HypE, ε-MOEA, PESA-II,
AGE-MOEA, KnEA, GrEA, RVEA, MOPSO, TLBO, IPOP-CMA-ES, sNES, UMDA,
Ant Colony, GA, Random Search) fan out per generation. Steady-state
algorithms (Hill Climber, SA, (1+1)-ES, PAES, Nelder-Mead, Tabu
Search) await each step sequentially. Surrogate algorithms (BO,
TPE) batch the initial design and then await per-iteration
acquisitions. Hyperband fans out each Successive-Halving rung
through `AsyncPartialProblem`.
- Internal `algorithms::parallel_eval_async::evaluate_batch_async`
and `evaluate_batch_at_budget_async` helpers — use
`futures::stream::FuturesOrdered` with concurrency-bounded chunks,
preserve input order so seeded determinism is preserved when
evaluations are themselves deterministic.
- `examples/async_eval.rs` — worked example with a simulated 20 ms
remote service. At concurrency = 1 it's serial; at concurrency = 4
it's 2× faster; demonstrates `DifferentialEvolution` under tokio.
#### Documentation
- New cookbook recipe **[Async evaluation](docs/book/src/cookbook/async.md)**
— implementing `AsyncProblem`, picking concurrency, determinism
guarantees, async vs. `parallel`.
- Comparison-with-other-libraries chapter updated: `heuropt 0.8`
row, `Async ✅ AsyncProblem + run_async` column, "When to pick
heuropt" gains an explicit IO-bound bullet.
- Stability chapter rewritten: removes the speculative "Observer /
Checkpoint planned" bullet (those didn't ship), documents the new
`async` feature flag.
- Migration guide: new "To 0.8" section covering both
`0.5.x → 0.8` (feature-additive — opt in by enabling the `async`
feature) and `0.7 → 0.8` (the partial async surface from 0.7 is
superseded by complete coverage; existing `run_async` callers
keep working).
- Runnable `cargo test --doc` examples added to every public
operator (10), metric (3), and Pareto utility (7) — every
public item across the crate now ships with at least one
example. 55 doctests in total (was 45).
#### CI / build
- `.github/workflows/docs.yml` builds the mdbook user guide on
every push and deploys to GitHub Pages on `main` /
tag pushes.
- `mdbook` book now uses `[rust] edition = "2021"` to satisfy
`mdbook 0.4.40`.
- `clamp_to_bounds` cargo-fuzz target tolerance loosened to
`1e-4 · max(simplex_total, max_abs_x, 1)` so the fuzzer doesn't
flag ULP-level slop in the simplex projection's
`max(x_i τ, 0)` clamp boundary.
[0.8.0]: https://github.com/swaits/heuropt/releases/tag/v0.8.0
## [0.5.0] — 2026-05-05
Theme: comprehensive documentation and project polish. No public-API
changes — bumping `heuropt = "0.5"` in your `Cargo.toml` is enough.
### Added
#### User guide (mdbook)
A new mdbook user guide at `docs/book/`, deployed to
<https://swaits.github.io/heuropt/> via a CI workflow on tag pushes.
Chapters:
- **Introduction** — what heuropt is, who it's for, what's in the box.
- **Five-minute walkthrough** — install, define a problem, run an
optimizer, look at the result.
- **Defining a problem** — the `Problem` trait in depth: single- vs
multi-objective, constraints, custom decision types
(`Vec<f64>`, `Vec<bool>`, `Vec<usize>`, custom structs).
- **Choosing an algorithm** — the README's decision tree, expanded
to a full chapter with the reasoning behind every branch.
- **Cookbook** — seven recipes covering parallelism, expensive
evaluations, comparison harnesses, permutation problems,
constraint repair, picking one answer off a Pareto front, and
writing your own optimizer.
- **Comparison with other libraries** — heuropt vs pymoo, hyperopt,
optuna, MOEA Framework, metaheuristics-rs, argmin. Honest about
when *not* to pick heuropt.
- **Stability and SemVer** — explicit guarantees about which surfaces
are stable; what's likely to change before 1.0; bit-identical
determinism contract.
- **Migration guides** — per-release upgrade notes.
#### Runnable rustdoc examples
Every algorithm now has a runnable ` ```rust ` example block in its
rustdoc — 35 algorithms, all exercised by `cargo test --doc`. Plus
the existing crate-level example in `lib.rs` and the
`CompositeVariation` operator example.
#### Real-world examples
Three new polished examples covering distinct domains:
- `examples/portfolio.rs` — multi-objective portfolio optimization
with budget constraint via `ProjectToSimplex`. Pareto front of
return-vs-risk trade-offs, plus a-posteriori weighted decision.
- `examples/hyperparam_tuning.rs` — sample-efficient hyperparameter
tuning with `BayesianOpt` and `Tpe`, demonstrating mixed-scale
decoding (log-uniform learning rate, integer depth) and a 60-eval
budget.
- `examples/scheduling.rs` — single-machine weighted-completion-time
scheduling: permutation decisions optimized via
`SimulatedAnnealing` + `SwapMutation`, comparing against the
Smith's-rule oracle.
#### Governance docs
- `CONTRIBUTING.md` — local-test checklist, conventional-commits
requirement, contribution areas that land easily vs. those that
need prior discussion.
- `SECURITY.md` — disclosure policy, supported versions, what counts
as a security issue.
- `CODE_OF_CONDUCT.md` — adopts the
[Builder's Code of Conduct](https://builderscode.org/) (CC0).
- `.github/ISSUE_TEMPLATE/` — bug, feature, docs templates plus a
`config.yml` that points security reports to the private
vulnerability-disclosure flow.
- `.github/PULL_REQUEST_TEMPLATE.md` — short, opinionated PR
template.
#### CI / tooling
- `.github/workflows/docs.yml` — builds the mdbook user guide and
deploys it to GitHub Pages on `main` pushes and tag pushes.
### Changed
- README hero block expanded with badges and a punchier opening;
added explicit links to the user guide, the docs.rs API reference,
and the testing-coverage breakdown.
- `lib.rs` crate-level docs polished — better intro, points readers
at the user guide and the design spec.
[0.5.0]: https://github.com/swaits/heuropt/releases/tag/v0.5.0
## [0.4.0] — 2026-05-05
Theme: testing infrastructure, two real bug fixes surfaced by that
@@ -383,5 +552,5 @@ Initial release.
`RandomSearch`, `Nsga2`, and `DifferentialEvolution`. Seeded runs stay
bit-identical to serial mode.
[Unreleased]: https://github.com/swaits/heuropt/compare/v0.4.0...HEAD
[Unreleased]: https://github.com/swaits/heuropt/compare/v0.8.0...HEAD
[0.1.0]: https://github.com/swaits/heuropt/releases/tag/v0.1.0
+43
View File
@@ -0,0 +1,43 @@
# Code of Conduct
heuropt adopts the [Builder's Code of Conduct](https://builderscode.org/),
version 1.0.
A Code of Conduct for people who build things.
## The Rule
> "Stay professional. Stay technical."
## Expected
- Contribute constructively.
- Respect others' time and work.
- Focus on the work and its technical merit.
## Not Welcome
- Harassment, name-calling, or personal attacks.
- Trolling, spamming, or derailing discussions.
- Discussions about contributors rather than their contributions.
## Enforcement
Violations result in:
1. **Warning** — first offense.
2. **Temporary suspension** — repeated or serious violations.
3. **Permanent ban** — continued violations.
Maintainers can remove, block, or ban anyone who disrupts the project.
## Reporting
Email **steve@waits.net** with `[heuropt CoC]` in the subject line.
Reports are handled confidentially.
---
The Builder's Code of Conduct is dedicated to the public domain under
CC0 1.0 Universal. You may use, modify, and distribute it freely
without attribution.
+117
View File
@@ -0,0 +1,117 @@
# Contributing to heuropt
Thanks for considering a contribution. heuropt is a small, opinionated
crate, but careful additions are welcome.
## Quick checklist
Before opening a pull request:
- [ ] `cargo fmt --all`
- [ ] `cargo clippy --all-targets --all-features -- -D warnings`
- [ ] `cargo test` (default features) and `cargo test --all-features`
- [ ] `cargo doc --no-deps --all-features` with `RUSTDOCFLAGS="-D warnings"`
- [ ] If you touched algorithm output: re-run `cargo run --release --example compare`
and confirm the quality metrics did not change. Speed-only changes
are required to be **bit-identical** against the prior snapshot.
CI runs all of the above on every PR; the matrix covers MSRV (1.85),
the default / serde / parallel / serde+parallel feature combinations,
and a 60-second fuzz soak per target.
## Commit style
Conventional Commits (https://www.conventionalcommits.org/) are
required. The first line follows `<type>(<scope>): <summary>` where
`<type>` is one of `feat`, `fix`, `perf`, `refactor`, `docs`, `test`,
`chore`, `ci`, `build`, `style`. `<scope>` is the most specific module
the change touches (e.g. `nsga2`, `hypervolume`, `pareto_archive`).
Bad: `Phase 1.1: Add core data types`
Good: `feat(core): add data types and Rng alias`
Multiple logical changes in a single PR should be split into multiple
commits, each on a single concern.
## What kinds of contributions land easily
- **Bug fixes** with a regression test that fails on `main` and passes
on the fix.
- **Performance wins** that preserve bit-identical output and include
a `cargo bench` (gungraun) before/after, plus a `cargo run --release
--example compare` diff confirming no quality regression.
- **Documentation improvements** — missing rustdoc examples, README
clarifications, mdbook chapters.
- **New algorithms** that fit the established `Optimizer<P>` shape and
ship with: a unit test, a property test (determinism + invariants),
a comparison-harness entry, and rustdoc.
- **New operators / metrics / Pareto utilities** with the same
hygiene.
## What needs prior discussion
Open an issue before starting on:
- New traits or breaking changes to the public API surface.
- A new optional feature flag.
- Anything that depends on a heavy new dependency.
- Restructuring of `src/algorithms/` or `src/pareto/`.
The crate intentionally keeps the trait surface small (`Problem`,
`Optimizer`, `Initializer`, `Variation`, `Repair`); changes there
are not refused but they need a clear motivation.
## Running the test suites locally
```sh
# unit + integration + property tests
cargo test
# all feature combinations
cargo test --features serde
cargo test --features parallel
cargo test --all-features
# instruction-count benchmarks (needs valgrind installed)
cargo bench
# coverage-guided fuzzing (needs nightly + cargo-fuzz)
cd fuzz
cargo +nightly fuzz run pareto_compare -- -max_total_time=60
# mutation testing (slow, optional)
cargo install cargo-mutants
cargo mutants
```
## Reporting bugs
Please include:
1. The smallest reproducing input you can produce — ideally a 20-line
`examples/repro.rs`.
2. The exact command (`cargo run --release --example repro` etc.) and
the observed vs expected output.
3. The Rust toolchain (`rustc --version`) and feature flags.
4. The heuropt version you saw the bug on.
Bugs that surface fuzz-target panics are particularly welcome; please
attach the failing artifact (`fuzz/artifacts/<target>/crash-...`) so
we can add it to the regression-test corpus.
## Security
For security concerns please follow the disclosure policy in
[SECURITY.md](SECURITY.md). Don't open public issues for security
bugs.
## Code of conduct
This project follows the [Builder's Code of Conduct](CODE_OF_CONDUCT.md).
The short version: stay professional, stay technical, focus on the
work and its merit.
## License
By submitting a contribution, you agree that your work is licensed
under the same MIT license as the rest of heuropt.
+8 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "heuropt"
version = "0.4.0"
version = "0.8.0"
edition = "2024"
rust-version = "1.85"
authors = ["Stephen Waits <steve@waits.net>"]
@@ -17,8 +17,10 @@ categories = ["algorithms", "science", "mathematics", "simulation"]
default = []
serde = ["dep:serde"]
parallel = ["dep:rayon"]
async = ["dep:futures"]
[dependencies]
futures = { version = "0.3", optional = true, default-features = false, features = ["std", "async-await"] }
rand = "0.9"
rand_distr = "0.5"
rayon = { version = "1", optional = true }
@@ -27,11 +29,16 @@ serde = { version = "1", features = ["derive"], optional = true }
[dev-dependencies]
gungraun = "0.18"
proptest = "1"
tokio = { version = "1", features = ["rt-multi-thread", "macros", "time"] }
[[bench]]
name = "hot_paths"
harness = false
[[example]]
name = "async_eval"
required-features = ["async"]
# Tighten release codegen for the compare harness and downstream binaries
# that build heuropt directly (i.e. when this crate is the workspace root).
# When heuropt is used as a dependency the consumer's profile wins.
+160 -46
View File
@@ -2,79 +2,189 @@
[![Crates.io](https://img.shields.io/crates/v/heuropt.svg)](https://crates.io/crates/heuropt)
[![Documentation](https://docs.rs/heuropt/badge.svg)](https://docs.rs/heuropt)
[![Book](https://img.shields.io/badge/book-online-blue.svg)](https://swaits.github.io/heuropt/)
[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
[![CI](https://github.com/swaits/heuropt/actions/workflows/ci.yml/badge.svg)](https://github.com/swaits/heuropt/actions/workflows/ci.yml)
A practical Rust toolkit for implementing heuristic single-objective,
multi-objective, and many-objective optimization algorithms.
**A practical Rust toolkit for heuristic optimization.** Single-objective.
Multi-objective. Many-objective. 33 algorithms — every one of them with a
sync `run` and an async `run_async`. One small set of traits. Bit-identical
seeded determinism. No trait objects, no GATs, no generic-RNG plumbing in
the public API.
`heuropt` is **not** a research framework full of abstract machinery — it is a
small set of concrete types, a handful of simple traits, and a few reference
algorithms. The goal: an entry-level Rust engineer can define a problem, run a
built-in optimizer, or implement a new optimizer without learning any
framework concepts.
If you can write a `Problem` impl and read `RandomSearch`, you can write your
own optimizer. That's the whole pitch.
Docs: [user guide](https://swaits.github.io/heuropt/) · [API reference](https://docs.rs/heuropt).
## Installation
```toml
[dependencies]
heuropt = "0.3"
heuropt = "0.8"
# Optional features:
# - "serde": derive Serialize/Deserialize on the core data types.
# - "parallel": evaluate populations across rayon's thread pool.
# Seeded runs stay bit-identical to serial mode.
# heuropt = { version = "0.3", features = ["serde", "parallel"] }
# - "async": AsyncProblem / AsyncPartialProblem traits and a
# run_async(&problem, concurrency).await method on
# every algorithm — for IO-bound evaluations.
# heuropt = { version = "0.8", features = ["serde", "parallel", "async"] }
```
## Define a problem
## Define a problem and run an optimizer
You're designing a car. Three things you can pick: **engine
displacement** (1.06.0 L), **curb weight** (11002200 kg, where
going lighter requires aluminum/carbon and costs money), and
**aerodynamic drag** (Cd from 0.20 to 0.40, where slipperier needs
expensive aero R&D). Four things you want to optimize: **price**,
**0-60 acceleration**, **fuel consumption**, **idle noise** — all
in tension.
The relationships between decisions and objectives are nonlinear
and coupled: engine cost grows superlinearly with displacement,
weight reduction below 1500 kg costs a quadratic premium, drag
reduction below 0.35 Cd costs a 1.5-power premium, and 0-60 depends
on weight × engine in a non-trivial way. You can't just sweep one
slider — the Pareto front is a genuine surface in 3D decision space,
and finding it by hand is hopeless.
NSGA-III is the canonical many-objective (4+) optimizer; it uses
DasDennis reference points to keep the front well-spread.
```rust
use heuropt::prelude::*;
struct SchafferN1;
struct PickACar;
impl Problem for SchafferN1 {
type Decision = Vec<f64>;
impl Problem for PickACar {
type Decision = Vec<f64>; // [engine_liters, weight_kg, drag_cd]
fn objectives(&self) -> ObjectiveSpace {
ObjectiveSpace::new(vec![
Objective::minimize("f1"),
Objective::minimize("f2"),
Objective::minimize("price_thousand_dollars"),
Objective::minimize("seconds_to_60mph"),
Objective::minimize("fuel_gallons_per_100mi"),
Objective::minimize("noise_db_at_idle"),
])
}
fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
let v = x[0];
Evaluation::new(vec![v * v, (v - 2.0).powi(2)])
let displacement = x[0]; // liters
let weight = x[1]; // kg
let drag = x[2]; // dimensionless Cd
// Price ($k): engine cost grows superlinearly; weight reduction
// below 1500 kg and drag reduction below 0.35 Cd both cost extra.
let engine_cost = 3.0 * displacement.powf(1.6);
let weight_cost = ((1500.0 - weight).max(0.0) / 100.0).powi(2) * 2.0;
let aero_cost = ((0.35 - drag).max(0.0) * 100.0).powf(1.5) * 0.4;
let price = 10.0 + engine_cost + weight_cost + aero_cost;
// 0-60 (s): heavier = slower; bigger engine = quicker but with
// diminishing returns.
let weight_factor = (weight - 1100.0) / 1000.0;
let engine_factor = ((displacement - 1.0) / 5.0).max(0.0).powf(0.7);
let zero_to_sixty = 5.0 + 5.0 * weight_factor - 4.0 * engine_factor;
// Fuel consumption (gal/100 mi): all three matter.
let fuel = 0.5 + 0.5 * displacement + 0.5 * weight / 1000.0 + 4.0 * drag;
// Idle noise (dB): engine dominates, mildly nonlinear.
let noise = 60.0 + 3.0 * displacement.powf(1.2);
Evaluation::new(vec![price, zero_to_sixty, fuel, noise])
}
}
fn main() {
let bounds = vec![
(1.0_f64, 6.0_f64), // engine
(1100.0_f64, 2200.0_f64), // weight
(0.20_f64, 0.40_f64), // drag
];
let mut optimizer = Nsga3::new(
Nsga3Config {
population_size: 100,
generations: 200,
reference_divisions: 5,
seed: 42,
},
RealBounds::new(bounds.clone()),
CompositeVariation {
crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.9),
mutation: PolynomialMutation::new(bounds, 20.0, 1.0 / 3.0),
},
);
let result = optimizer.run(&PickACar);
let mut front: Vec<_> = result.pareto_front.iter().collect();
front.sort_by(|a, b| {
a.evaluation.objectives[0]
.partial_cmp(&b.evaluation.objectives[0]).unwrap()
});
println!("{:>5} {:>5} {:>4} {:>6} {:>5} {:>5} {:>5}",
"L", "kg", "Cd", "$k", "0-60", "fuel", "dB");
for c in &front {
let d = &c.decision;
let o = &c.evaluation.objectives;
println!("{:>5.2} {:>5.0} {:>4.2} {:>6.1} {:>5.1} {:>5.2} {:>5.1}",
d[0], d[1], d[2], o[0], o[1], o[2], o[3]);
}
}
```
## Run NSGA-II
Run it (`cargo run --release`) and you get 100 cars on the front.
A representative slice from the actual output, hand-picked across
the spectrum:
```rust
use heuropt::prelude::*;
# struct SchafferN1;
# impl Problem for SchafferN1 {
# type Decision = Vec<f64>;
# fn objectives(&self) -> ObjectiveSpace {
# ObjectiveSpace::new(vec![Objective::minimize("f1"), Objective::minimize("f2")])
# }
# fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
# Evaluation::new(vec![x[0] * x[0], (x[0] - 2.0).powi(2)])
# }
# }
let initializer = RealBounds::new(vec![(-5.0, 5.0)]);
let variation = GaussianMutation { sigma: 0.2 };
let config = Nsga2Config { population_size: 60, generations: 80, seed: 42 };
let mut optimizer = Nsga2::new(config, initializer, variation);
let result = optimizer.run(&SchafferN1);
println!("Pareto front size: {}", result.pareto_front.len());
```text
L kg Cd $k 0-60 fuel dB ← role
1.00 1505 0.35 13.0 7.0 3.17 63.0 cheap baseline
2.00 1370 0.35 22.4 5.1 3.56 66.7 sensible sport sedan
2.45 1330 0.38 28.5 4.5 3.92 68.8 quicker midprice
1.00 1430 0.21 35.8 6.6 2.54 63.0 fuel-saver (small + slippery)
3.50 1300 0.25 52.9 3.5 3.88 73.3 genuine sports car
5.27 1100 0.20 108.1 1.4 4.48 82.0 hypercar corner
```
See `examples/toy_nsga2.rs` for the full version.
### Reading the result
Every row is **non-dominated** — no row is strictly better than
another on every metric. The interesting part is what each one does
*differently*:
- The **cheap baseline** ($13k) takes the path of least resistance:
smallest engine, no weight reduction, average drag. Slow but
affordable.
- The **sensible sedan** ($22k) trades $9k for **2 seconds off
0-60** by running a 2.0L engine with mild weight reduction.
- The **fuel-saver** is interesting: it's a 1.0L econobox engine,
but it spends $22k *just on aero* (0.21 Cd) to push fuel
consumption down to **2.54 gal/100mi**. The optimizer figured
out that aero matters more than displacement at this fuel point.
No human would pick this combo by intuition.
- The **sports car** ($53k) doesn't blow money on the lightest
possible weight — it picks 1300 kg, because dropping further
costs disproportionately and the 3.5L engine is doing most of
the acceleration work.
- The **hypercar corner** ($108k) is the optimizer pushing every
decision to its ceiling: minimum weight (1100 kg), minimum
drag (0.20 Cd), big engine (5.3L). Sub-1.5 second 0-60, but
you pay for it on every other axis except fuel (because the
weight + aero savings partly cancel the V8's thirst).
That last point is the kind of insight a Pareto front gives you
that no single-objective optimizer would: **the cheapest fuel-
efficient car is not the smallest engine alone**, it's a small
engine + aggressive aero. **The lightest sports car is not the
lightest possible**, it's the point where weight cost stops paying
back in 0-60. The optimizer doesn't tell you what to buy — it
hands you the frontier of *every defensible compromise* and lets
you pick by your own priorities.
## Implement a custom optimizer
@@ -94,13 +204,7 @@ where
// Evaluate them with `problem.evaluate(...)`.
// Keep the best, or maintain a Pareto archive.
// Return an OptimizationResult.
# OptimizationResult::new(
# Population::new(Vec::new()),
# Vec::new(),
# None,
# 0,
# 0,
# )
todo!()
}
}
```
@@ -504,6 +608,16 @@ heuropt is exhaustively tested across several layers:
- **CI** (`.github/workflows/ci.yml`) — fmt, clippy
(`-D warnings`), test (4-feature matrix), doc, MSRV (1.85), fuzz.
## Contributing
See [CONTRIBUTING.md](CONTRIBUTING.md) for the local-test checklist,
conventional-commits requirement, and project-governance docs.
This project follows the [Builder's Code of Conduct](CODE_OF_CONDUCT.md):
stay professional, stay technical, focus on the work and its merit.
For security disclosures, see [SECURITY.md](SECURITY.md).
## License
MIT — see [LICENSE](LICENSE).
+62
View File
@@ -0,0 +1,62 @@
# Security policy
## Supported versions
Security fixes are applied to the latest released minor version on
crates.io. Patch-level releases (`0.x.y``0.x.y+1`) are issued as
needed.
| Version | Supported |
|---------|--------------------|
| 0.8.x | ✅ |
| ≤ 0.7.x | ❌ (please upgrade) |
heuropt is pre-1.0; the public API may change between minor versions.
Once 1.0.0 ships, the support window will be at least the latest two
minor versions.
## Reporting a vulnerability
Please **do not** open a public GitHub issue for a security bug.
Instead use one of these channels:
- GitHub's [private vulnerability reporting](https://github.com/swaits/heuropt/security/advisories/new)
on the repository.
- Email **steve@waits.net** with subject line `[heuropt security]
<short summary>`.
Please include:
1. A description of the vulnerability and the affected versions.
2. The smallest reproducer you can produce — a `cargo run --example
repro` is ideal.
3. Your assessment of impact and exploitability.
4. Any suggested mitigation if you have one.
## What I will do
- Acknowledge the report within **72 hours**.
- Confirm or refute reproducibility within **7 days**.
- Issue a fix in a patch release within **30 days** for confirmed
high-severity issues; less urgent issues may roll into the next
minor release.
- Credit the reporter in the CHANGELOG entry unless you ask
otherwise.
## What counts as a security issue
heuropt is a numerical library, not a network service or sandbox. The
realistic security-relevant categories are:
- **Memory safety**: any unsafe-code-related UB or unwinds-across-FFI
bug. heuropt itself uses no `unsafe`; this category covers
dependencies it transitively pulls in.
- **Denial of service**: an input to a public API that causes
unbounded memory growth, infinite loop, or panic outside its
documented panic conditions. (Documented panics for invalid config
are not bugs.)
- **Supply-chain compromise**: a published heuropt crate that doesn't
match the source on the tagged commit.
Functional correctness bugs (an algorithm produces wrong
hypervolumes, etc.) are tracked as ordinary issues, not security.
+34
View File
@@ -0,0 +1,34 @@
[book]
title = "heuropt — the user guide"
description = "A practical Rust toolkit for heuristic single-, multi-, and many-objective optimization."
authors = ["Stephen Waits"]
language = "en"
src = "src"
[build]
build-dir = "../../target/book"
create-missing = false
[output.html]
default-theme = "rust"
preferred-dark-theme = "navy"
git-repository-url = "https://github.com/swaits/heuropt"
edit-url-template = "https://github.com/swaits/heuropt/edit/main/docs/book/{path}"
site-url = "/heuropt/"
no-section-label = true
[output.html.fold]
enable = true
level = 1
[output.html.search]
enable = true
limit-results = 30
teaser-word-count = 30
use-boolean-and = true
[output.html.print]
enable = true
[rust]
edition = "2021"
+27
View File
@@ -0,0 +1,27 @@
# Summary
[Introduction](./introduction.md)
# Getting started
- [Five-minute walkthrough](./getting-started.md)
- [Defining a problem](./defining-problems.md)
- [Choosing an algorithm](./choosing-an-algorithm.md)
# Cookbook
- [Recipes](./cookbook.md)
- [Parallelize evaluation with rayon](./cookbook/parallel.md)
- [Async evaluation (HTTP / RPC / subprocess)](./cookbook/async.md)
- [Tune a model with expensive evaluations](./cookbook/expensive-evaluations.md)
- [Compare two algorithms on your problem](./cookbook/compare.md)
- [Optimize a permutation (TSP-style)](./cookbook/permutation.md)
- [Constrain your search with `Repair`](./cookbook/constraints.md)
- [Pick one answer off a Pareto front](./cookbook/pick-one.md)
- [Write your own algorithm](./cookbook/custom-optimizer.md)
# Reference
- [Comparison with other libraries](./comparison.md)
- [Stability and SemVer](./stability.md)
- [Migration guides](./migration.md)
+291
View File
@@ -0,0 +1,291 @@
# Choosing an algorithm
The README has a compact decision tree. This chapter expands it with
the *reasoning* behind each branch.
## Step 0: How expensive is one evaluation?
This is the first fork because it changes everything that comes
after it.
| Eval cost | Budget you can afford | Algorithm family |
|----------------------------|---------------------------|-----------------------------|
| Microseconds (pure math) | 10 000 1 000 000 evals | Population-based |
| Milliseconds (sim, IO) | 1 000 10 000 evals | Population-based |
| Seconds (small training) | 100 1 000 evals | Sample-efficient (BO, TPE) |
| Minutes+ (full training) | 50 500 evals | Sample-efficient + multi-fidelity |
For the cheap-eval branch, you have the run of the catalog. For the
expensive branch, classical evolutionary methods waste your evaluation
budget — go to [`BayesianOpt`] or [`Tpe`]. For the *very* expensive
branch where each eval has a tunable budget (epochs, MC samples, sim
steps), [`Hyperband`] over the [`PartialProblem`] trait is the move.
## Step 1: How many objectives?
The biggest fork.
- **One** — there's a single best answer. Pick from the
single-objective branch.
- **Two or three** — a Pareto front. Pick from the multi-objective
branch.
- **Four or more** — a many-objective Pareto front; classical
multi-objective methods break down here because almost every pair
of points is non-dominated. Pick from the many-objective branch.
> **Pareto front:** the set of decisions where you cannot improve any
> objective without sacrificing another. In a 2-objective minimize
> problem, plot every solution; the Pareto front is the lower-left
> envelope.
If you found yourself staring at a single composite score that's a
weighted sum of conflicting goals, you probably have a multi-objective
problem in disguise. A weighted sum bakes in your preferences before
you've seen the trade-off; running a multi-objective optimizer first
and picking off the front later is almost always a better workflow
(see [Pick one answer off a Pareto front](./cookbook/pick-one.md)).
## Step 2 — single-objective continuous
These all take `Vec<f64>` decisions.
### Smooth, low-to-moderate dimension
[`CmaEs`] is the strong default. It adapts the search distribution's
covariance to the local landscape. On the comparison harness it
hits machine epsilon on Rosenbrock at 30 000 evaluations.
For very low-dimensional smooth problems (≤ 5 dim), [`NelderMead`] is
deterministic and converges to f = 0 exactly on Rosenbrock.
### High dimension, smooth
[`SeparableNes`] uses a diagonal covariance — cheaper per step than
CmaEs at the cost of being unable to model rotated landscapes. Worth
trying when CmaEs's `O(d²)` per-step cost hurts.
### Multimodal landscapes
Multimodal = many local minima that aren't the global one. Rastrigin
and Ackley are classic traps.
[`IpopCmaEs`] is CmaEs with an increasing-population restart strategy
specifically designed for this. On the harness it drops vanilla CmaEs's
Rastrigin score from f = 2.35 to f = 0.13.
[`DifferentialEvolution`] is rarely beaten on cheap multimodal
continuous problems. On Rastrigin it ties with `(1+1)-ES` at f = 0.
[`SimulatedAnnealing`] is a cheap, generic baseline that escapes local
optima via temperature decay.
### Want parameter-free
[`Tlbo`] (Teaching-Learning-Based Optimization) has no `F`, `CR`, `w`,
or `σ` to tune. Often a respectable middle-of-the-pack performer.
### Smallest possible self-adapting baseline
[`OnePlusOneEs`] — Rechenberg's 1973 `(1+1)`-ES with the one-fifth
success rule. On the harness it hits f = 0 on Rastrigin in 50 000
evaluations.
### Just want a baseline
[`RandomSearch`]. Useful as a sanity check: if your fancy optimizer
can't beat random search, something is wrong (with the fancy
optimizer or with the problem).
## Step 2 — single-objective other types
| Decision type | Algorithm | Notes |
|---|---|---|
| `Vec<bool>` | [`Umda`] | Per-bit marginal EDA. Independent-bit assumption. |
| `Vec<bool>` | [`GeneticAlgorithm`] + [`BitFlipMutation`] | When bit interactions matter. |
| `Vec<usize>` (permutation) | [`AntColonyTsp`] | TSP-style with a distance matrix. |
| `Vec<usize>` (permutation) | [`SimulatedAnnealing`] + [`SwapMutation`] | Generic discrete baseline. |
| `Vec<usize>` or custom | [`TabuSearch`] | You supply the neighbor function. |
| Custom struct | [`SimulatedAnnealing`] / [`HillClimber`] | With your own `Variation` impl. |
## Step 2 — multi-objective (2 or 3)
### Strong default
[`Nsga2`] is the canonical Pareto-based EA. Fast, well-understood,
maintains diversity via crowding distance. On the harness it lands
on the Pareto front of every test problem.
### Real-valued, smooth front, want best convergence
[`Mopso`] (multi-objective PSO with archive). On ZDT1 it wins
hypervolume outright and converges 100× tighter than the
dominance-based methods.
### Better front quality than NSGA-II
[`Ibea`] (indicator-based) is consistently the best of the
dominance-based methods on the harness — wins ZDT3 hypervolume and
DTLZ2 mean distance by 24×. It uses an additive ε-indicator for
selection rather than dominance + crowding.
[`Spea2`] (strength + density) — solid alternative; explicit external
archive separate from the population.
[`SmsEmoa`] uses exact hypervolume contribution for selection. Elegant
in theory; in practice on the harness budgets here it underperforms
NSGA-II. Worth the higher per-step cost only when exact HV
contribution is the right discriminator.
### Decomposition / weight-vector style
[`Moead`] decomposes the multi-objective problem into many scalar
sub-problems (Tchebycheff or weighted sum) and solves them in
parallel. Very fast per generation; scales naturally to many
objectives.
### Disconnected or non-convex front
[`AgeMoea`] estimates the front geometry adaptively (the L_p
parameter `p` is fit from data each generation).
[`Knea`] favors knee points — the regions of the front where small
gains in one objective cost large losses in another.
[`Ibea`] also handles disconnected fronts well.
### Region-based diversity
[`PesaII`] uses grid hyperboxes to drive selection — divide the
objective space into a grid, pick from the least-crowded boxes.
[`EpsilonMoea`] uses an ε-grid archive that auto-limits its size.
### Just one starting decision (no population budget)
[`Paes`] — `(1+1)`-ES with a Pareto archive. Cheap, simple, useful
when your evaluations are expensive enough that you can't afford a
population.
## Step 2 — many-objective (4+)
### Linear / simplex-shaped front (e.g., DTLZ1)
[`Grea`] — grid coords drive ranking. On DTLZ1 it beats NSGA-III by
3× and AGE-MOEA by 2.5×.
[`Moead`] — decomposition shines on linear fronts; second on DTLZ1
and among the fastest per generation.
### Curved / unknown front geometry
[`Nsga3`] — reference-point niching; canonical many-objective method;
strong default when the front isn't simplex-shaped.
[`AgeMoea`] — estimates L_p geometry per generation.
[`Rvea`] — reference vectors with adaptive penalty.
### Indicator-based selection
[`Ibea`] — additive ε-indicator; doesn't degrade at high obj count.
[`HypE`] — Monte Carlo hypervolume estimation; scales to arbitrary
objective count where exact HV is too expensive.
## Step 3: Are there hard constraints?
heuropt models constraints as a single scalar `constraint_violation`
on each `Evaluation`. Three escalations when the feasibility region
is hard to find:
1. **Penalty-only.** Just set `constraint_violation > 0` for
infeasible decisions. The default tournament/Pareto comparisons
prefer feasibles automatically.
2. **Repair.** Implement [`Repair<D>`] (or use the provided
[`ClampToBounds`] / [`ProjectToSimplex`]) to project infeasible
decisions back into the feasible region. Pair with a `Variation`
in a [`CompositeVariation`] for bounds-aware variants.
3. **Stochastic ranking.** Use [`stochastic_ranking_select`] instead
of `tournament_select_single_objective`. It probabilistically
explores near-feasibility instead of strict feasibility-first
ordering, which helps when feasible regions are narrow.
See [Constrain your search with `Repair`](./cookbook/constraints.md)
for worked examples.
## Step 4: Should you parallelize?
Enable the `parallel` feature flag if your `evaluate` takes more
than ~50 µs. Population-based algorithms ([`RandomSearch`], [`Nsga2`],
[`DifferentialEvolution`], [`Spea2`], [`Ibea`], [`Mopso`], …) batch-
evaluate via rayon when the feature is on. **Seeded runs stay
bit-identical** to serial mode.
```toml
heuropt = { version = "0.8", features = ["parallel"] }
```
If your evaluation is **IO-bound** (HTTP request, RPC, subprocess)
rather than CPU-bound, use the `async` feature instead — it gives
you `AsyncProblem` and a `run_async(&problem, concurrency).await`
method on every algorithm in the catalog. See the
[Async evaluation cookbook recipe](./cookbook/async.md).
## TL;DR table
| Situation | Pick |
|---|---|
| Smooth single-objective continuous | [`CmaEs`] |
| Multimodal single-objective continuous | [`IpopCmaEs`] or [`DifferentialEvolution`] |
| Expensive single-objective | [`BayesianOpt`] or [`Tpe`] |
| Multi-fidelity single-objective | [`Hyperband`] |
| 2- or 3-objective default | [`Nsga2`] |
| 2-objective real-valued smooth front | [`Mopso`] |
| Disconnected / non-convex front | [`Ibea`] |
| Many-objective default (curved front) | [`Nsga3`] |
| Many-objective linear / simplex front | [`Grea`] |
| Permutation problem | [`AntColonyTsp`] |
| Binary problem | [`Umda`] |
| Custom decision type | [`SimulatedAnnealing`] + your `Variation` |
| Sanity baseline | [`RandomSearch`] |
[`CmaEs`]: https://docs.rs/heuropt/latest/heuropt/algorithms/cma_es/struct.CmaEs.html
[`IpopCmaEs`]: https://docs.rs/heuropt/latest/heuropt/algorithms/ipop_cma_es/struct.IpopCmaEs.html
[`SeparableNes`]: https://docs.rs/heuropt/latest/heuropt/algorithms/snes/struct.SeparableNes.html
[`NelderMead`]: https://docs.rs/heuropt/latest/heuropt/algorithms/nelder_mead/struct.NelderMead.html
[`DifferentialEvolution`]: https://docs.rs/heuropt/latest/heuropt/algorithms/differential_evolution/struct.DifferentialEvolution.html
[`SimulatedAnnealing`]: https://docs.rs/heuropt/latest/heuropt/algorithms/simulated_annealing/struct.SimulatedAnnealing.html
[`Tlbo`]: https://docs.rs/heuropt/latest/heuropt/algorithms/tlbo/struct.Tlbo.html
[`OnePlusOneEs`]: https://docs.rs/heuropt/latest/heuropt/algorithms/one_plus_one_es/struct.OnePlusOneEs.html
[`RandomSearch`]: https://docs.rs/heuropt/latest/heuropt/algorithms/random_search/struct.RandomSearch.html
[`HillClimber`]: https://docs.rs/heuropt/latest/heuropt/algorithms/hill_climber/struct.HillClimber.html
[`BayesianOpt`]: https://docs.rs/heuropt/latest/heuropt/algorithms/bayesian_opt/struct.BayesianOpt.html
[`Tpe`]: https://docs.rs/heuropt/latest/heuropt/algorithms/tpe/struct.Tpe.html
[`Hyperband`]: https://docs.rs/heuropt/latest/heuropt/algorithms/hyperband/struct.Hyperband.html
[`PartialProblem`]: https://docs.rs/heuropt/latest/heuropt/core/partial_problem/trait.PartialProblem.html
[`Umda`]: https://docs.rs/heuropt/latest/heuropt/algorithms/umda/struct.Umda.html
[`GeneticAlgorithm`]: https://docs.rs/heuropt/latest/heuropt/algorithms/genetic_algorithm/struct.GeneticAlgorithm.html
[`BitFlipMutation`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.BitFlipMutation.html
[`AntColonyTsp`]: https://docs.rs/heuropt/latest/heuropt/algorithms/ant_colony_tsp/struct.AntColonyTsp.html
[`SwapMutation`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.SwapMutation.html
[`TabuSearch`]: https://docs.rs/heuropt/latest/heuropt/algorithms/tabu_search/struct.TabuSearch.html
[`Nsga2`]: https://docs.rs/heuropt/latest/heuropt/algorithms/nsga2/struct.Nsga2.html
[`Nsga3`]: https://docs.rs/heuropt/latest/heuropt/algorithms/nsga3/struct.Nsga3.html
[`Mopso`]: https://docs.rs/heuropt/latest/heuropt/algorithms/mopso/struct.Mopso.html
[`Ibea`]: https://docs.rs/heuropt/latest/heuropt/algorithms/ibea/struct.Ibea.html
[`Spea2`]: https://docs.rs/heuropt/latest/heuropt/algorithms/spea2/struct.Spea2.html
[`SmsEmoa`]: https://docs.rs/heuropt/latest/heuropt/algorithms/sms_emoa/struct.SmsEmoa.html
[`Moead`]: https://docs.rs/heuropt/latest/heuropt/algorithms/moead/struct.Moead.html
[`AgeMoea`]: https://docs.rs/heuropt/latest/heuropt/algorithms/age_moea/struct.AgeMoea.html
[`Knea`]: https://docs.rs/heuropt/latest/heuropt/algorithms/knea/struct.Knea.html
[`PesaII`]: https://docs.rs/heuropt/latest/heuropt/algorithms/pesa2/struct.PesaII.html
[`EpsilonMoea`]: https://docs.rs/heuropt/latest/heuropt/algorithms/epsilon_moea/struct.EpsilonMoea.html
[`Paes`]: https://docs.rs/heuropt/latest/heuropt/algorithms/paes/struct.Paes.html
[`Grea`]: https://docs.rs/heuropt/latest/heuropt/algorithms/grea/struct.Grea.html
[`Rvea`]: https://docs.rs/heuropt/latest/heuropt/algorithms/rvea/struct.Rvea.html
[`HypE`]: https://docs.rs/heuropt/latest/heuropt/algorithms/hype/struct.Hype.html
[`Repair<D>`]: https://docs.rs/heuropt/latest/heuropt/traits/trait.Repair.html
[`ClampToBounds`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.ClampToBounds.html
[`ProjectToSimplex`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.ProjectToSimplex.html
[`stochastic_ranking_select`]: https://docs.rs/heuropt/latest/heuropt/selection/tournament/fn.stochastic_ranking_select.html
[`CompositeVariation`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.CompositeVariation.html
+109
View File
@@ -0,0 +1,109 @@
# Comparison with other libraries
heuropt is one of many heuristic-optimization libraries. This chapter
is an honest, opinionated comparison to help you choose.
The columns:
- **Lang** — primary implementation language.
- **Algorithms** — rough catalog count.
- **Multi-obj** — built-in support for Pareto-based multi-objective
optimization.
- **Surrogates** — built-in Bayesian / TPE / multi-fidelity.
- **Determinism** — seeded reproducibility as a first-class property.
- **Async / async-eval** — first-class async runtime support.
| Library | Lang | Algorithms | Multi-obj | Surrogates | Determinism | Async |
|---|---|---|---|---|---|---|
| **heuropt 0.8** | Rust | 33 | ✅ NSGA-II/III, SPEA2, IBEA, MOEA/D, MOPSO, SMS-EMOA, HypE, AGE-MOEA, GrEA, KnEA, RVEA, PESA-II, ε-MOEA, PAES | ✅ BO, TPE, Hyperband | ✅ bit-identical seeded | ✅ `AsyncProblem` + `run_async` on every algorithm |
| pymoo | Python | ~25 | ✅ extensive | partial (BO via plug-ins) | ✅ | ❌ |
| DEAP | Python | flexible toolbox | ✅ | ❌ | ✅ | ❌ |
| hyperopt | Python | TPE-focused | ❌ | ✅ TPE | partial | partial |
| optuna | Python | TPE / CMA-ES / NSGA-II | ✅ | ✅ TPE, BoTorch via plug-in | ✅ | partial (study-level, not eval-level) |
| MOEA Framework | Java | ~40 | ✅ very extensive | ❌ | ✅ | ❌ |
| metaheuristics-rs | Rust | ~10 | partial | ❌ | ✅ | ❌ |
| argmin | Rust | line-search / quasi-Newton | ❌ | ❌ | ✅ | ❌ |
## When to pick heuropt
- You're working in **Rust** and want a single, dependency-light crate
for evolutionary / metaheuristic optimization.
- You need **multi-objective or many-objective** algorithms (12+
Pareto-aware methods in the catalog) AND you don't want to glue
Python into your Rust pipeline.
- You want **bit-identical determinism**: same seed produces same
output, on every machine, across releases unless explicitly noted
otherwise.
- You want a **small, readable codebase** — every algorithm is
written for clarity, no trait-object plumbing, no GATs in user-
facing APIs. Reading `RandomSearch` should be enough to write a
new optimizer.
- You have **IO-bound evaluations** — calling an HTTP service, an
RPC, or a subprocess — and want first-class `async fn evaluate`
support. heuropt is the only mainstream optimization library that
ships this (see [Async evaluation](./cookbook/async.md)).
## When *not* to pick heuropt
- You need **gradient-based** optimization. Use `argmin` (Rust) or
`scipy.optimize` (Python) — heuropt is gradient-free by design.
- You need **GPU-accelerated** evaluations. heuropt's `evaluate`
function runs on CPU; use Python (jax/torch) or roll your own
GPU pipeline.
- You need **distributed multi-machine** evaluation. heuropt
parallelizes within one process via rayon. Distribution is up to
you (split the seeds across machines, aggregate).
- You're comfortable in Python and pymoo / optuna already cover
your problem. heuropt's value-add over pymoo is mostly that it's
Rust — if that doesn't matter to you, the Python ecosystem has more
battle-tested integrations.
## Algorithm coverage at a glance
heuropt covers the same major Pareto MOEAs as pymoo and MOEA Framework:
NSGA-II/III, SPEA2, IBEA, MOEA/D, MOPSO, SMS-EMOA, HypE, AGE-MOEA,
GrEA, KnEA, RVEA, PESA-II, ε-MOEA, PAES.
The expensive-evaluation regime: BayesianOpt + TPE + Hyperband. This
is comparable to optuna's coverage but in pure Rust.
The single-objective continuous catalog (CMA-ES, IPOP-CMA-ES, sNES,
DE, PSO, GA, TLBO, (1+1)-ES, NelderMead, RandomSearch, HillClimber,
SimulatedAnnealing) covers the canonical baselines and several modern
variants.
What heuropt does **not** ship that some libraries do:
- **Re-themed metaphor metaheuristics** (Whale Optimization, Grey
Wolf, Bat, Firefly, Harris Hawks, etc.). These are cut from the
catalog deliberately — they are mostly DE/PSO with new names. If
you specifically need one, please open an issue with citations.
- **Non-evolutionary global optimizers** like dual annealing or
basin-hopping (use `scipy.optimize` for those).
- **A web UI / dashboard** like optuna's. heuropt is library-only.
## Speed
heuropt's hot paths (Pareto utilities, hypervolume, key inner loops)
are heavily optimized — see the perf entry in the v0.4.0 CHANGELOG.
On the comparison harness in `examples/compare.rs` (10-seed mean,
30 000 evaluations on DTLZ2), the total wall-clock time across 12
algorithms is ~5 seconds. Per-algorithm timings are in
[`examples/compare-results.md`](https://github.com/swaits/heuropt/blob/main/examples/compare-results.md).
For comparison-shopping speed against Python libraries, the gap is
typically 10×–100× in heuropt's favor for compute-bound
`evaluate` functions, because Rust skips the Python-loop overhead. If
your `evaluate` calls into NumPy/PyTorch and those are the bottleneck,
the gap shrinks substantially.
## Honest weakness: ecosystem
The biggest thing pymoo / optuna / DEAP have that heuropt doesn't:
**community + plug-ins + tutorials**. They've been around longer and
have rich third-party integrations (visualization, MLflow,
Hyperband+BO hybrids, distributed runners). heuropt is younger; the
core is solid but the ecosystem is small.
If you adopt heuropt and miss a thing, the project is small enough
that contributions land fast. See [CONTRIBUTING.md](https://github.com/swaits/heuropt/blob/main/CONTRIBUTING.md).
+30
View File
@@ -0,0 +1,30 @@
# Cookbook
Short, focused recipes for the patterns that come up in practice.
Each recipe is self-contained and small enough to copy into your own
project.
## Recipes
- [Parallelize evaluation with rayon](./cookbook/parallel.md) — when
your `evaluate` is non-trivial CPU work, the `parallel` feature
pays for itself almost immediately.
- [Async evaluation](./cookbook/async.md) — when your `evaluate` is
IO-bound (HTTP / RPC / subprocess), the `async` feature lets the
optimizer await many evaluations concurrently. The differentiating
feature vs other optimization libraries.
- [Tune a model with expensive evaluations](./cookbook/expensive-evaluations.md)
`BayesianOpt`, `Tpe`, and `Hyperband` for the 50500-eval
regime.
- [Compare two algorithms on your problem](./cookbook/compare.md) —
multi-seed harness pattern straight from `examples/compare.rs`.
- [Optimize a permutation (TSP-style)](./cookbook/permutation.md) —
`AntColonyTsp` with a distance matrix.
- [Constrain your search with `Repair`](./cookbook/constraints.md) —
bounds, simplex projection, custom repair.
- [Pick one answer off a Pareto front](./cookbook/pick-one.md) — the
a-posteriori weighted-decision pattern from the `jiggly_tuning`
example.
- [Write your own algorithm](./cookbook/custom-optimizer.md) —
implement `Optimizer<P>` from scratch, à la the
`examples/custom_optimizer.rs` walkthrough.
+165
View File
@@ -0,0 +1,165 @@
# Async evaluation
When your `evaluate` does **IO** — calls an HTTP service, sends an
RPC, spawns a subprocess — `await`-ing it from the optimizer is
much more efficient than blocking a thread per evaluation. heuropt
ships first-class async support behind the `async` feature flag.
This is the differentiating capability vs pymoo / hyperopt /
optuna / DEAP / MOEA Framework — none of those have a native async
evaluation path.
## Enable the feature
```toml
[dependencies]
heuropt = { version = "0.8", features = ["async"] }
# Pick whatever async runtime you want; heuropt itself depends only on
# `futures`. The example below uses tokio.
tokio = { version = "1", features = ["rt-multi-thread", "macros", "time"] }
```
## Implement `AsyncProblem`
It mirrors the regular [`Problem`] trait one-for-one — same
`Decision` type, same `objectives()`, but `evaluate` is replaced
with `evaluate_async` returning a future.
```rust,no_run
use heuropt::core::async_problem::AsyncProblem;
use heuropt::prelude::*;
struct RemoteService;
impl AsyncProblem for RemoteService {
type Decision = Vec<f64>;
fn objectives(&self) -> ObjectiveSpace {
ObjectiveSpace::new(vec![Objective::minimize("loss")])
}
async fn evaluate_async(&self, x: &Vec<f64>) -> Evaluation {
// Real workload: HTTP call to a model-scoring service, an RPC,
// a subprocess. Here we just sleep to model 20 ms latency.
tokio::time::sleep(std::time::Duration::from_millis(20)).await;
let loss: f64 = x.iter().map(|v| v * v).sum();
Evaluation::new(vec![loss])
}
}
```
## Run the optimizer with `run_async`
`run_async(&problem, concurrency).await` is provided by **every**
algorithm in the catalog as of v0.8. `concurrency` caps how many
evaluations are in-flight at once.
```rust,no_run
# use heuropt::core::async_problem::AsyncProblem;
# use heuropt::prelude::*;
# struct RemoteService;
# impl AsyncProblem for RemoteService {
# type Decision = Vec<f64>;
# fn objectives(&self) -> ObjectiveSpace {
# ObjectiveSpace::new(vec![Objective::minimize("loss")])
# }
# async fn evaluate_async(&self, x: &Vec<f64>) -> Evaluation {
# Evaluation::new(vec![x.iter().map(|v| v * v).sum::<f64>()])
# }
# }
#[tokio::main]
async fn main() {
let bounds = vec![(-1.0_f64, 1.0_f64); 4];
let mut opt = DifferentialEvolution::new(
DifferentialEvolutionConfig {
population_size: 16,
generations: 50,
differential_weight: 0.5,
crossover_probability: 0.9,
seed: 42,
},
RealBounds::new(bounds),
);
let r = opt.run_async(&RemoteService, /* concurrency */ 8).await;
println!("best: {}", r.best.unwrap().evaluation.objectives[0]);
}
```
## Picking `concurrency`
Concurrency is the maximum in-flight evaluation count. Tradeoffs:
| Setting | Effect |
|---|---|
| `1` | Sequential; equivalent to a sync run with extra overhead |
| `pop_size` | Full per-generation parallelism; fastest if your service tolerates it |
| `< pop_size` | Bounded — useful if your downstream service has a rate limit or finite worker pool |
The bigger you go, the more memory the in-flight futures hold and
the more load you put on the downstream service. A reasonable
starting point is `min(pop_size, 16)` and increase only if the
downstream service is comfortable.
## Determinism
Same seed produces the same final result whether you use `run` or
`run_async`, **provided your async `evaluate_async` is itself
deterministic**. heuropt drives the RNG and selection on the main
task; only the evaluations are concurrent, and the
`evaluate_batch_async` helper preserves input order before feeding
results back to the algorithm.
## What the worked example shows
`examples/async_eval.rs` runs `RandomSearch` (200 evaluations × 20 ms
each) at `concurrency = 1, 4, 16` and `DifferentialEvolution` at
`concurrency = 8`. On a recent machine:
```text
RandomSearch with 200 evaluations (20 ms each)
concurrency = 1 elapsed ≈ 4250 ms (sequential 200 × 20 ms)
concurrency = 4 elapsed ≈ 2100 ms (2× speedup, batch_size=2 caps it)
concurrency = 16 elapsed ≈ 2100 ms (same — batch_size dominates)
DifferentialEvolution at concurrency=8
elapsed ≈ 230 ms (8 ants run in parallel each generation)
```
Run it yourself: `cargo run --release --features async --example async_eval`.
## Which algorithms support `run_async`?
**All 33** algorithms in the catalog. The shape of the async path
depends on the algorithm:
- **Population-based / batch-evaluating** — NSGA-II, NSGA-III, SPEA2,
MOEA/D, IBEA, SMS-EMOA, HypE, ε-MOEA, PESA-II, AGE-MOEA, KnEA,
GrEA, RVEA, MOPSO, GA, DE, PSO, CMA-ES, IPOP-CMA-ES, sNES, TLBO,
UMDA, Ant Colony, Random Search. Each generation's offspring
evaluations are fanned out concurrently up to `concurrency`.
- **Steady-state (one-eval-per-step)** — Hill Climber, Simulated
Annealing, (1+1)-ES, PAES, Nelder-Mead. The `concurrency`
parameter is accepted for API uniformity but evaluation order is
inherently sequential.
- **Tabu Search** — fans out the K-neighbor batch each step.
- **Surrogate (BO, TPE)** — fans out the initial design batch, then
awaits per-iteration acquisitions sequentially (the surrogate
must update before the next point is chosen).
- **Hyperband** — uses the separate
[`AsyncPartialProblem`](https://docs.rs/heuropt/latest/heuropt/core/async_problem/trait.AsyncPartialProblem.html)
trait (multi-fidelity); each Successive-Halving rung's evaluations
fan out concurrently.
## Async vs `parallel`
| If your `evaluate` is… | Use |
|---|---|
| CPU-bound (math, simulation) | `parallel` feature → see [Parallelize evaluation](./parallel.md) |
| IO-bound (HTTP, RPC, subprocess) | `async` feature (this recipe) |
Both can be on at once if your evaluation does *both* substantial
CPU work *and* IO. The two features are independent.
[`Problem`]: https://docs.rs/heuropt/latest/heuropt/core/problem/trait.Problem.html
+148
View File
@@ -0,0 +1,148 @@
# Compare two algorithms on your problem
The harness in `examples/compare.rs` runs every applicable algorithm
against every test problem with N seeds and reports mean ± std.
You can lift the same pattern for your own problem in ~30 lines.
## The pattern
1. Wrap your problem in a struct that implements [`Problem`].
2. Pick a few candidate algorithms.
3. For each algorithm × seed, run and record the metric you care about.
4. Print mean ± std.
## Worked example
```rust,no_run
use heuropt::prelude::*;
use std::time::Instant;
struct MyProblem;
impl Problem for MyProblem {
type Decision = Vec<f64>;
fn objectives(&self) -> ObjectiveSpace {
ObjectiveSpace::new(vec![Objective::minimize("f")])
}
fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
// your problem here
Evaluation::new(vec![x.iter().map(|v| v * v).sum::<f64>()])
}
}
const SEEDS: u64 = 10;
const DIM: usize = 5;
const BUDGET: usize = 30_000;
fn main() {
let bounds: Vec<(f64, f64)> = vec![(-5.0, 5.0); DIM];
let mut best_de = vec![];
let mut best_cmaes = vec![];
let mut best_ipop = vec![];
let mut t_de = vec![];
let mut t_cmaes = vec![];
let mut t_ipop = vec![];
for seed in 0..SEEDS {
// Differential Evolution
let t = Instant::now();
let mut de = DifferentialEvolution::new(
DifferentialEvolutionConfig {
population_size: 30,
generations: BUDGET / 30,
differential_weight: 0.5,
crossover_probability: 0.9,
seed,
},
RealBounds::new(bounds.clone()),
);
let r = de.run(&MyProblem);
t_de.push(t.elapsed().as_millis() as f64);
best_de.push(r.best.unwrap().evaluation.objectives[0]);
// CMA-ES
let t = Instant::now();
let mut cma = CmaEs::new(
CmaEsConfig {
population_size: 12,
generations: BUDGET / 12,
initial_sigma: 1.0,
eigen_decomposition_period: 1,
initial_mean: None,
seed,
},
RealBounds::new(bounds.clone()),
);
let r = cma.run(&MyProblem);
t_cmaes.push(t.elapsed().as_millis() as f64);
best_cmaes.push(r.best.unwrap().evaluation.objectives[0]);
// IPOP-CMA-ES
let t = Instant::now();
let mut ipop = IpopCmaEs::new(
IpopCmaEsConfig {
base: CmaEsConfig {
population_size: 12,
generations: BUDGET / 12 / 4,
initial_sigma: 1.0,
eigen_decomposition_period: 1,
initial_mean: None,
seed,
},
max_restarts: 3,
population_factor: 2.0,
seed,
},
RealBounds::new(bounds.clone()),
);
let r = ipop.run(&MyProblem);
t_ipop.push(t.elapsed().as_millis() as f64);
best_ipop.push(r.best.unwrap().evaluation.objectives[0]);
}
println!("{:<12} {:>14} {:>10}", "algorithm", "best f (mean±std)", "ms");
print_row("DE", &best_de, &t_de);
print_row("CMA-ES", &best_cmaes, &t_cmaes);
print_row("IPOP-CMA-ES", &best_ipop, &t_ipop);
}
fn print_row(name: &str, values: &[f64], times: &[f64]) {
let (m, s) = mean_std(values);
let (t, _) = mean_std(times);
println!("{:<12} {:>10.3e} ± {:>5.2e} {:>6.0}", name, m, s, t);
}
fn mean_std(xs: &[f64]) -> (f64, f64) {
let n = xs.len() as f64;
let m = xs.iter().sum::<f64>() / n;
let v = xs.iter().map(|x| (x - m).powi(2)).sum::<f64>() / n;
(m, v.sqrt())
}
```
## What to record
- **`best.evaluation.objectives[0]`** for single-objective.
- **`hypervolume_2d(&result.pareto_front, &space, ref_point)`** for
2-objective.
- **`spacing(&result.pareto_front, &space)`** for front uniformity.
- **`result.evaluations`** to cross-check that every algorithm got
the same evaluation budget.
- Wall-clock `Instant::now()` deltas for runtime comparison.
## Pitfalls
- **Population size matters.** Different algorithms have very
different sweet spots. Don't just give them all the same
population — the README's algorithm pages note typical defaults.
- **Different algorithms count "generations" differently.** What
matters is the total `evaluations` count. Set
`generations = BUDGET / population_size` to match across
algorithms (with caveats for steady-state algorithms like SMS-EMOA
that evaluate one offspring per generation).
- **One seed is not a comparison.** Always run ≥ 5 seeds; ≥ 10 is
better. Single-seed comparisons are noise.
- **The harness in `examples/compare.rs` is the canonical version.**
When in doubt, copy from there.
[`Problem`]: https://docs.rs/heuropt/latest/heuropt/core/problem/trait.Problem.html
+126
View File
@@ -0,0 +1,126 @@
# Constrain your search with `Repair`
heuropt models constraints with a single `constraint_violation` scalar
on each `Evaluation`. That works for soft penalties. When constraints
are *hard* and the search keeps generating infeasible decisions, the
better pattern is **repair**: project each candidate back into the
feasible region every time it leaves.
The [`Repair<D>`] trait is the abstraction. Two impls ship in the box;
you can write your own for arbitrary geometry.
## Built-in: `ClampToBounds`
For per-axis box constraints (`lo ≤ xᵢ ≤ hi`), pair `ClampToBounds`
with any `Variation` to get a bounds-aware variant for free.
```rust,no_run
use heuropt::prelude::*;
let bounds = vec![(-5.0, 5.0); 3];
// Without bounds, GaussianMutation can step outside the search box.
// ClampToBounds projects each variable back in.
let mut sigma = GaussianMutation { sigma: 0.5 };
let mut clamp = ClampToBounds::new(bounds.clone());
let mut rng = rng_from_seed(42);
let parent = vec![4.9, -4.9, 0.0];
let mut child = sigma.vary(std::slice::from_ref(&parent), &mut rng).pop().unwrap();
clamp.repair(&mut child);
// every entry of `child` is now within [-5, 5].
```
`ClampToBounds` is idempotent: applying it twice is the same as
applying it once.
For most real problems you'd just use [`BoundedGaussianMutation`]
which combines both in one operator.
## Built-in: `ProjectToSimplex`
For *budget* constraints — "the components must sum to a fixed
total and be non-negative" — `ProjectToSimplex` projects onto the
probability simplex (or any scaled simplex).
```rust,no_run
use heuropt::prelude::*;
let mut proj = ProjectToSimplex::new(1.0); // probability simplex
let mut x = vec![0.6, 0.5, -0.1, 0.3]; // sum 1.3, one negative
proj.repair(&mut x);
// x now sums to 1.0 and every entry is ≥ 0.
let s: f64 = x.iter().sum();
debug_assert!((s - 1.0).abs() < 1e-12);
debug_assert!(x.iter().all(|&v| v >= 0.0));
```
Use this for portfolio / resource-allocation problems where the
decision is a vector of weights that must sum to a budget.
## Custom repair
Anything that takes a `&mut Vec<f64>` (or any `&mut D` for your
custom decision type) and returns a feasible version is a valid
`Repair`. Implement the trait directly:
```rust,no_run
use heuropt::prelude::*;
/// Force the largest variable to be at least `min_largest`.
struct AtLeastOneActive { min_largest: f64 }
impl Repair<Vec<f64>> for AtLeastOneActive {
fn repair(&mut self, x: &mut Vec<f64>) {
let max_idx = x.iter()
.enumerate()
.fold(0, |best, (i, &v)| {
if v > x[best] { i } else { best }
});
if x[max_idx] < self.min_largest {
x[max_idx] = self.min_largest;
}
}
}
```
## Stochastic-ranking selection
When the feasible region is *narrow* — most of the search space is
infeasible — the strict "feasibles always beat infeasibles" rule
traps the search outside it. Runarsson & Yao's stochastic ranking
breaks the trap by, on each pairwise comparison, using a probabilistic
"compare by objective" instead of "compare by feasibility" with a
small probability `pf`:
```rust,ignore
use heuropt::selection::tournament::stochastic_ranking_select;
let picks = stochastic_ranking_select(
&population,
&objectives,
0.45, // pf — Runarsson & Yao's canonical value
count,
&mut rng,
);
```
This is a drop-in replacement for `tournament_select_single_objective`
in your custom optimizer or in a forked algorithm.
## When to use which
| Situation | Use |
|---|---|
| Box constraints | [`BoundedGaussianMutation`] (built-in mutation) |
| Manual repair after any mutation | [`ClampToBounds`] |
| Budget / probability-simplex constraints | [`ProjectToSimplex`] |
| Custom geometric constraints | Your own `Repair` impl |
| Narrow feasible region, frequent infeasibility | [`stochastic_ranking_select`] |
| Soft penalty, mostly feasible search | Set `constraint_violation` and let default tournament handle it |
[`Repair<D>`]: https://docs.rs/heuropt/latest/heuropt/traits/trait.Repair.html
[`ClampToBounds`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.ClampToBounds.html
[`ProjectToSimplex`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.ProjectToSimplex.html
[`BoundedGaussianMutation`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.BoundedGaussianMutation.html
[`stochastic_ranking_select`]: https://docs.rs/heuropt/latest/heuropt/selection/tournament/fn.stochastic_ranking_select.html
+149
View File
@@ -0,0 +1,149 @@
# Write your own algorithm
Implement [`Optimizer<P>`] and you're done. There are no other traits
to think about, no internal hooks to register. The example walks
through a tiny hill-climber that reads almost identically to the
canonical pseudocode.
## The trait
```rust,ignore
pub trait Optimizer<P>
where
P: Problem,
{
fn run(&mut self, problem: &P) -> OptimizationResult<P::Decision>;
}
```
That's it. You own your config, your RNG, your main loop, and your
`OptimizationResult` construction.
## A minimal hill-climber
```rust,no_run
use heuropt::prelude::*;
pub struct MyHillClimber<I, V> {
pub iterations: usize,
pub seed: u64,
pub initializer: I,
pub variation: V,
}
impl<P, I, V> Optimizer<P> for MyHillClimber<I, V>
where
P: Problem,
P::Decision: Clone,
I: Initializer<P::Decision>,
V: Variation<P::Decision>,
{
fn run(&mut self, problem: &P) -> OptimizationResult<P::Decision> {
let mut rng = rng_from_seed(self.seed);
let objectives = problem.objectives();
assert!(objectives.is_single_objective(), "MyHillClimber is single-objective only");
// Start with one initial decision.
let init_decisions = self.initializer.initialize(1, &mut rng);
let init = init_decisions.into_iter().next().unwrap();
let mut current = Candidate::new(init.clone(), problem.evaluate(&init));
let mut evaluations: usize = 1;
for _ in 0..self.iterations {
let children = self.variation.vary(std::slice::from_ref(&current.decision), &mut rng);
for child_decision in children {
let child_eval = problem.evaluate(&child_decision);
evaluations += 1;
let child = Candidate::new(child_decision, child_eval);
if better(&child.evaluation, &current.evaluation, &objectives) {
current = child;
}
}
}
let pareto_front = vec![current.clone()];
let best = Some(current.clone());
OptimizationResult::new(
Population::new(vec![current]),
pareto_front,
best,
evaluations,
self.iterations,
)
}
}
fn better(a: &Evaluation, b: &Evaluation, objectives: &ObjectiveSpace) -> bool {
let am = objectives.as_minimization(&a.objectives);
let bm = objectives.as_minimization(&b.objectives);
am[0] < bm[0]
}
```
## Things to notice
- **`Rng` is one concrete type.** No generics — call
[`rng_from_seed`] and pass `&mut rng` everywhere it's needed.
- **`Initializer<D>`** sources the starting point(s).
- **`Variation<D>`** generates children from parents. For the
hill-climber it's called with one parent.
- **`OptimizationResult`** carries the final population, the Pareto
front (just the best for single-objective), the best candidate,
the total evaluations, and the iteration count.
- **`as_minimization`** flips maximize-axis values so your
comparison logic only ever needs to deal with "lower is better."
## Adding parallel evaluation
If your algorithm batch-evaluates candidates per generation, use the
crate's internal helper. From inside heuropt source you can call
`evaluate_batch(problem, decisions)`; from outside you'd use rayon
directly behind a feature flag, the same way the built-in algorithms
do.
```rust,ignore
#[cfg(feature = "parallel")]
fn batch_eval<P>(problem: &P, decisions: Vec<P::Decision>) -> Vec<Candidate<P::Decision>>
where P: Problem + Sync, P::Decision: Send,
{
use rayon::prelude::*;
decisions.into_par_iter()
.map(|d| Candidate::new(d.clone(), problem.evaluate(&d)))
.collect()
}
#[cfg(not(feature = "parallel"))]
fn batch_eval<P>(problem: &P, decisions: Vec<P::Decision>) -> Vec<Candidate<P::Decision>>
where P: Problem,
{
decisions.into_iter()
.map(|d| Candidate::new(d.clone(), problem.evaluate(&d)))
.collect()
}
```
To stay bit-identical between serial and parallel modes, keep the
RNG and selection on the main thread; only the *evaluations* run in
parallel.
## What's *not* in the trait
- **No iteration / step API.** The optimizer owns its loop.
- **No callbacks.** A future minor release may add an observer hook;
for now you'd run the algorithm to completion and process the
result.
- **No error type.** Invalid configuration panics with a clear
message; this matches the style of the built-in algorithms.
- **No async on the trait.** `Optimizer<P>` is synchronous. For
async evaluation, implement [`AsyncProblem`](https://docs.rs/heuropt/latest/heuropt/core/async_problem/trait.AsyncProblem.html)
on your problem and use the `run_async(&problem, concurrency)`
method that comes with the `async` feature. See the
[Async evaluation cookbook recipe](./async.md).
The smallness is the point: you should be able to read a built-in
algorithm and write your own in an afternoon. See
`examples/custom_optimizer.rs` for a slightly more polished version
of the hill-climber above.
[`Optimizer<P>`]: https://docs.rs/heuropt/latest/heuropt/traits/trait.Optimizer.html
[`rng_from_seed`]: https://docs.rs/heuropt/latest/heuropt/core/rng/fn.rng_from_seed.html
@@ -0,0 +1,164 @@
# Tune a model with expensive evaluations
Population-based EAs throw thousands of evaluations at a problem. If
each evaluation costs a minute (a model training run, a CFD solve, a
real-world measurement) you can't afford that. heuropt has three
algorithms aimed at this regime.
| Algorithm | Surrogate | Best for |
|---|---|---|
| [`BayesianOpt`] | Gaussian process + Expected Improvement | The textbook choice; needs kernel tuning to shine |
| [`Tpe`] | Kernel-density estimate of good vs bad points | Cheaper per step; more robust without tuning |
| [`Hyperband`] | (none — it's a multi-fidelity scheduler) | When each eval has a tunable budget (epochs, MC samples) |
## When each is right
- **Black-box, fixed cost per eval, smooth-ish landscape** → BO.
- **Black-box, fixed cost per eval, no time to tune the surrogate** → TPE.
- **Each eval has a tunable fidelity** → Hyperband.
## Bayesian Optimization
A worked example with a synthetic 5-D problem and a 60-evaluation
budget — same configuration the `compare` harness uses.
```rust,no_run
use heuropt::prelude::*;
struct Rosenbrock5D;
impl Problem for Rosenbrock5D {
type Decision = Vec<f64>;
fn objectives(&self) -> ObjectiveSpace {
ObjectiveSpace::new(vec![Objective::minimize("f")])
}
fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
let f: f64 = x.windows(2).map(|w|
100.0 * (w[1] - w[0].powi(2)).powi(2) + (1.0 - w[0]).powi(2)
).sum();
Evaluation::new(vec![f])
}
}
let bounds = vec![(-2.048_f64, 2.048_f64); 5];
let mut opt = BayesianOpt::new(
BayesianOptConfig {
evaluations: 60,
initial_samples: 10,
length_scale: 1.0,
signal_variance: 1.0,
noise_variance: 1e-6,
seed: 42,
},
RealBounds::new(bounds),
);
let r = opt.run(&Rosenbrock5D);
println!("best f after 60 evals: {}", r.best.unwrap().evaluation.objectives[0]);
```
> **Honest disclosure.** On the comparison harness this default
> configuration produces **f ≈ 3170 ± 2920** on Rosenbrock 5-D — well
> below what a tuned BO can do. The default RBF kernel without
> per-problem hyperparameter tuning is the limitation. For real
> workloads, consider:
>
> - More evaluations (200+ instead of 60).
> - Tuning `length_scale` to a known scale of your problem
> (lower for high-frequency landscapes, higher for smooth ones).
> - TPE instead of BO if you don't want to tune the kernel.
## Tree-structured Parzen Estimator
TPE keeps two density estimates — `l(x)` over historical good points
and `g(x)` over the rest — and picks new candidates that maximize the
ratio. Cheaper per step than a GP and famously robust without
hand-tuning.
```rust,no_run
use heuropt::prelude::*;
# struct Rosenbrock5D;
# impl Problem for Rosenbrock5D {
# type Decision = Vec<f64>;
# fn objectives(&self) -> ObjectiveSpace { ObjectiveSpace::new(vec![Objective::minimize("f")]) }
# fn evaluate(&self, _x: &Vec<f64>) -> Evaluation { Evaluation::new(vec![0.0]) }
# }
let bounds = vec![(-2.048_f64, 2.048_f64); 5];
let mut opt = Tpe::new(
TpeConfig {
evaluations: 60,
initial_samples: 10,
gamma: 0.25,
candidates_per_step: 24,
bandwidth_factor: 1.06,
seed: 42,
},
RealBounds::new(bounds),
);
let _r = opt.run(&Rosenbrock5D);
```
`gamma` is the fraction of best points used as `l(x)`; `0.25` is the
canonical Bergstra value.
## Hyperband
[`Hyperband`] needs your problem to implement [`PartialProblem`] —
that is, you can evaluate at a tunable fidelity (e.g. number of
training epochs). The algorithm schedules many cheap-fidelity runs
and promotes only the survivors to higher fidelity.
```rust,no_run
use heuropt::prelude::*;
use heuropt::core::partial_problem::PartialProblem;
struct ModelTuning;
impl Problem for ModelTuning {
type Decision = Vec<f64>;
fn objectives(&self) -> ObjectiveSpace {
ObjectiveSpace::new(vec![Objective::minimize("val_loss")])
}
fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
// Full-fidelity eval = train at max_epochs.
self.evaluate_at_budget(x, 100.0)
}
}
impl PartialProblem for ModelTuning {
fn evaluate_at_budget(&self, x: &Vec<f64>, budget: f64) -> Evaluation {
// Replace with: train your model for `budget` epochs, return val_loss.
// For demo, pretend more budget = lower noisy loss.
let lr = x[0];
let wd = x[1];
let loss = (lr - 0.001).powi(2) + (wd - 1e-4).powi(2)
+ 1.0 / (budget + 1.0);
Evaluation::new(vec![loss])
}
}
let bounds = vec![(1e-5_f64, 1e-1), (1e-6_f64, 1e-2)];
let mut hyperband = Hyperband::new(
HyperbandConfig {
max_budget: 100.0,
eta: 3.0,
seed: 42,
},
RealBounds::new(bounds),
);
let _r = hyperband.run(&ModelTuning);
```
`max_budget` is the most epochs (or whatever your fidelity unit is)
you'd ever spend on a single config. `eta` controls how aggressive
the elimination is — `3.0` is the classic value; higher means more
aggressive culling.
## Strategy: combining surrogate + multi-fidelity
The state of the art (BOHB) combines BO with Hyperband: TPE picks the
configurations Hyperband then evaluates at increasing fidelity.
heuropt doesn't ship a unified BOHB but the building blocks are
there — wrap your `PartialProblem` with a TPE-driven sampler and
feed the picks into `Hyperband`. PRs welcome.
[`BayesianOpt`]: https://docs.rs/heuropt/latest/heuropt/algorithms/bayesian_opt/struct.BayesianOpt.html
[`Tpe`]: https://docs.rs/heuropt/latest/heuropt/algorithms/tpe/struct.Tpe.html
[`Hyperband`]: https://docs.rs/heuropt/latest/heuropt/algorithms/hyperband/struct.Hyperband.html
[`PartialProblem`]: https://docs.rs/heuropt/latest/heuropt/core/partial_problem/trait.PartialProblem.html
+137
View File
@@ -0,0 +1,137 @@
# Parallelize evaluation with rayon
If a single call to your `evaluate` takes more than ~50 µs, enabling
the `parallel` feature usually pays for itself immediately on
population-based algorithms. Each generation evaluates an entire
population, and rayon parallelizes that batch.
## Enable the feature
```toml
[dependencies]
heuropt = { version = "0.8", features = ["parallel"] }
```
There's nothing else to opt into in your code. The
population-evaluation helper is feature-gated; with `parallel` on it
uses `rayon::into_par_iter` internally, with `parallel` off it falls
back to plain `into_iter`.
## Determinism still holds
Seeded runs are bit-identical between the serial and parallel modes.
The trick is that population members are evaluated in parallel but
*assembled* back into the same order. Variation, selection, and the
RNG are all driven by the main thread, so seed-stability tests still
pass.
## Which algorithms benefit
Algorithms with a per-generation `evaluate_batch`:
- [`RandomSearch`], [`Nsga2`], [`Nsga3`], [`Spea2`], [`Moead`],
[`Mopso`], [`Ibea`], [`SmsEmoa`], [`HypE`], [`PesaII`],
[`EpsilonMoea`], [`AgeMoea`], [`Knea`], [`Grea`], [`Rvea`].
- [`DifferentialEvolution`] and [`GeneticAlgorithm`] benefit on the
initial population and offspring batches.
Steady-state algorithms ([`Paes`], [`SimulatedAnnealing`],
[`HillClimber`], [`OnePlusOneEs`]) only evaluate one or a few
candidates per iteration, so the parallel feature gives them
nothing — leave it off if those are your primary optimizers.
## Worked example
The Sphere problem is too cheap to actually benefit from parallelism
— this example just shows the shape. In real workloads `evaluate` is
the expensive bit (a simulation, a model fit, an HTTP call).
```rust,no_run
use heuropt::prelude::*;
struct ExpensiveSphere;
impl Problem for ExpensiveSphere {
type Decision = Vec<f64>;
fn objectives(&self) -> ObjectiveSpace {
ObjectiveSpace::new(vec![Objective::minimize("f")])
}
fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
// Pretend this is a 5 ms simulation.
std::thread::sleep(std::time::Duration::from_millis(5));
Evaluation::new(vec![x.iter().map(|v| v * v).sum::<f64>()])
}
}
fn main() {
let bounds = vec![(-1.0_f64, 1.0_f64); 5];
let mut opt = DifferentialEvolution::new(
DifferentialEvolutionConfig {
population_size: 16,
generations: 50,
differential_weight: 0.5,
crossover_probability: 0.9,
seed: 42,
},
RealBounds::new(bounds),
);
let r = opt.run(&ExpensiveSphere);
println!("best f = {}", r.best.unwrap().evaluation.objectives[0]);
}
```
With the `parallel` feature on, each generation's 16 evaluations run
across rayon's worker threads. On a 16-core machine the wall-clock
cost per generation drops from `16 × 5 ms = 80 ms` to roughly
`5 ms + scheduling overhead`.
## Sizing your thread pool
heuropt uses rayon's global thread pool. Override the size with:
```rust,ignore
rayon::ThreadPoolBuilder::new().num_threads(8).build_global().unwrap();
```
Run this **before** any heuropt call, or use rayon's `install` API
to scope it.
## When parallelism *doesn't* help
- Your `evaluate` is sub-microsecond (Sphere, Rastrigin, Ackley
unweighted) — the rayon scheduling overhead exceeds the work.
- You're already running multiple seeds in parallel at the harness
level (see [Compare two algorithms](./compare.md)). Stacking
parallelism rarely helps.
- The algorithm is steady-state (Paes, SA, hill climber).
## `parallel` vs `async`
| If your `evaluate` is… | Use |
|---|---|
| CPU-bound (math, simulation) | `parallel` feature (this recipe) |
| IO-bound (HTTP, RPC, subprocess) | `async` feature → see [Async evaluation](./async.md) |
Both can be on at once if your evaluation does *both* substantial
CPU work *and* IO. The two features are independent.
[`RandomSearch`]: https://docs.rs/heuropt/latest/heuropt/algorithms/random_search/struct.RandomSearch.html
[`Nsga2`]: https://docs.rs/heuropt/latest/heuropt/algorithms/nsga2/struct.Nsga2.html
[`Nsga3`]: https://docs.rs/heuropt/latest/heuropt/algorithms/nsga3/struct.Nsga3.html
[`Spea2`]: https://docs.rs/heuropt/latest/heuropt/algorithms/spea2/struct.Spea2.html
[`Moead`]: https://docs.rs/heuropt/latest/heuropt/algorithms/moead/struct.Moead.html
[`Mopso`]: https://docs.rs/heuropt/latest/heuropt/algorithms/mopso/struct.Mopso.html
[`Ibea`]: https://docs.rs/heuropt/latest/heuropt/algorithms/ibea/struct.Ibea.html
[`SmsEmoa`]: https://docs.rs/heuropt/latest/heuropt/algorithms/sms_emoa/struct.SmsEmoa.html
[`HypE`]: https://docs.rs/heuropt/latest/heuropt/algorithms/hype/struct.Hype.html
[`PesaII`]: https://docs.rs/heuropt/latest/heuropt/algorithms/pesa2/struct.PesaII.html
[`EpsilonMoea`]: https://docs.rs/heuropt/latest/heuropt/algorithms/epsilon_moea/struct.EpsilonMoea.html
[`AgeMoea`]: https://docs.rs/heuropt/latest/heuropt/algorithms/age_moea/struct.AgeMoea.html
[`Knea`]: https://docs.rs/heuropt/latest/heuropt/algorithms/knea/struct.Knea.html
[`Grea`]: https://docs.rs/heuropt/latest/heuropt/algorithms/grea/struct.Grea.html
[`Rvea`]: https://docs.rs/heuropt/latest/heuropt/algorithms/rvea/struct.Rvea.html
[`DifferentialEvolution`]: https://docs.rs/heuropt/latest/heuropt/algorithms/differential_evolution/struct.DifferentialEvolution.html
[`GeneticAlgorithm`]: https://docs.rs/heuropt/latest/heuropt/algorithms/genetic_algorithm/struct.GeneticAlgorithm.html
[`Paes`]: https://docs.rs/heuropt/latest/heuropt/algorithms/paes/struct.Paes.html
[`SimulatedAnnealing`]: https://docs.rs/heuropt/latest/heuropt/algorithms/simulated_annealing/struct.SimulatedAnnealing.html
[`HillClimber`]: https://docs.rs/heuropt/latest/heuropt/algorithms/hill_climber/struct.HillClimber.html
[`OnePlusOneEs`]: https://docs.rs/heuropt/latest/heuropt/algorithms/one_plus_one_es/struct.OnePlusOneEs.html
+167
View File
@@ -0,0 +1,167 @@
# Optimize a permutation (TSP-style)
When your decision is "an ordering" — visiting cities, scheduling
jobs, routing — the natural representation is `Vec<usize>` and the
specialized algorithm is [`AntColonyTsp`]. Generic alternatives are
[`SimulatedAnnealing`] + [`SwapMutation`] for any permutation, and
[`TabuSearch`] when you have a custom neighbor function.
## TSP with `AntColonyTsp`
```rust,no_run
use heuropt::prelude::*;
struct Tsp {
distances: Vec<Vec<f64>>,
}
impl Problem for Tsp {
type Decision = Vec<usize>;
fn objectives(&self) -> ObjectiveSpace {
ObjectiveSpace::new(vec![Objective::minimize("length")])
}
fn evaluate(&self, tour: &Vec<usize>) -> Evaluation {
let mut len = 0.0;
for w in tour.windows(2) {
len += self.distances[w[0]][w[1]];
}
len += self.distances[*tour.last().unwrap()][tour[0]];
Evaluation::new(vec![len])
}
}
fn main() {
// 5-city Euclidean instance
let cities = vec![
(0.0, 0.0),
(1.0, 5.0),
(5.0, 2.0),
(6.0, 6.0),
(8.0, 3.0),
];
let n = cities.len();
let mut distances = vec![vec![0.0; n]; n];
for i in 0..n {
for j in 0..n {
let dx = cities[i].0 - cities[j].0;
let dy = cities[i].1 - cities[j].1;
distances[i][j] = (dx * dx + dy * dy).sqrt();
}
}
let problem = Tsp { distances: distances.clone() };
let mut opt = AntColonyTsp::new(AntColonyTspConfig {
ants: 20,
iterations: 200,
alpha: 1.0,
beta: 5.0,
evaporation: 0.5,
deposit: 1.0,
distances,
seed: 42,
});
let r = opt.run(&problem);
let best = r.best.unwrap();
println!("best tour length: {:.3}", best.evaluation.objectives[0]);
println!("tour: {:?}", best.decision);
}
```
`alpha` weights pheromone influence and `beta` weights the
heuristic (1 / distance). `evaporation` is the per-iteration decay
of pheromone trails. The classic Dorigo paper uses `alpha = 1`,
`beta = 2..5`, `evaporation = 0.1..0.5`.
## Generic permutation: SA + SwapMutation
Use this when your problem isn't TSP-shaped (no distance matrix
makes sense) but you still want to optimize an ordering.
```rust,no_run
use heuropt::prelude::*;
struct JobShop {
process_times: Vec<f64>,
}
impl Problem for JobShop {
type Decision = Vec<usize>;
fn objectives(&self) -> ObjectiveSpace {
ObjectiveSpace::new(vec![Objective::minimize("makespan")])
}
fn evaluate(&self, schedule: &Vec<usize>) -> Evaluation {
// Pretend cumulative weighted-completion-time. Replace with your real cost.
let cost: f64 = schedule.iter().enumerate()
.map(|(i, &job)| (i as f64 + 1.0) * self.process_times[job])
.sum();
Evaluation::new(vec![cost])
}
}
fn make_initial_perm(n: usize, seed: u64) -> Vec<usize> {
use rand::seq::SliceRandom;
let mut rng = rng_from_seed(seed);
let mut perm: Vec<usize> = (0..n).collect();
perm.shuffle(&mut rng);
perm
}
let times = vec![3.0, 1.5, 4.2, 2.7, 5.1];
let problem = JobShop { process_times: times.clone() };
// SimulatedAnnealing needs a starting decision; pass a custom Initializer.
struct OnePerm(Vec<usize>);
impl Initializer<Vec<usize>> for OnePerm {
fn initialize(&mut self, _size: usize, _rng: &mut Rng) -> Vec<Vec<usize>> {
vec![self.0.clone()]
}
}
let mut opt = SimulatedAnnealing::new(
SimulatedAnnealingConfig {
iterations: 2000,
initial_temperature: 5.0,
final_temperature: 1e-3,
seed: 7,
},
OnePerm(make_initial_perm(times.len(), 7)),
SwapMutation,
);
let r = opt.run(&problem);
let best = r.best.unwrap();
println!("best makespan: {:.3}", best.evaluation.objectives[0]);
println!("schedule: {:?}", best.decision);
```
`SwapMutation` swaps two random indices in the permutation —
preserves the "every element appears once" invariant for free.
## Custom neighborhoods: `TabuSearch`
When swap isn't the right move set (e.g., 2-opt for TSP, insert /
shift for scheduling), use [`TabuSearch`] with your own neighbor
function.
```rust,ignore
use heuropt::prelude::*;
let neighbors = |x: &Vec<usize>, _rng: &mut Rng| -> Vec<Vec<usize>> {
// Generate all 2-opt neighbors of x.
let mut out = Vec::new();
for i in 0..x.len() {
for j in (i + 2)..x.len() {
let mut child = x.clone();
child[i + 1..=j].reverse();
out.push(child);
}
}
out
};
// Pass `neighbors` to TabuSearch::new(...).
```
[`AntColonyTsp`]: https://docs.rs/heuropt/latest/heuropt/algorithms/ant_colony_tsp/struct.AntColonyTsp.html
[`SimulatedAnnealing`]: https://docs.rs/heuropt/latest/heuropt/algorithms/simulated_annealing/struct.SimulatedAnnealing.html
[`SwapMutation`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.SwapMutation.html
[`TabuSearch`]: https://docs.rs/heuropt/latest/heuropt/algorithms/tabu_search/struct.TabuSearch.html
+127
View File
@@ -0,0 +1,127 @@
# Pick one answer off a Pareto front
A multi-objective optimizer hands you a *front* — a Pareto-optimal
trade-off curve — not a single answer. Eventually you have to pick
*one* point off it. There are several principled ways to do that;
this recipe covers the most common: the **a-posteriori weighted
decision rule**.
The pattern: optimize *without* baking your preferences into the
search, then apply your preferences as a scoring function over the
front.
This is exactly the pattern from `examples/jiggly_tuning.rs` (the
USB-jiggler firmware tuning example).
## The shape
```rust,no_run
use heuropt::prelude::*;
# struct Cost;
# impl Problem for Cost {
# type Decision = Vec<f64>;
# fn objectives(&self) -> ObjectiveSpace {
# ObjectiveSpace::new(vec![Objective::minimize("a"), Objective::minimize("b"), Objective::minimize("c")])
# }
# fn evaluate(&self, _x: &Vec<f64>) -> Evaluation { Evaluation::new(vec![0.0,0.0,0.0]) }
# }
let problem = Cost;
let mut opt = Nsga2::new(
Nsga2Config { population_size: 100, generations: 200, seed: 42 },
RealBounds::new(vec![(-1.0, 1.0); 4]),
CompositeVariation {
crossover: SimulatedBinaryCrossover::new(vec![(-1.0, 1.0); 4], 15.0, 0.5),
mutation: PolynomialMutation::new(vec![(-1.0, 1.0); 4], 20.0, 1.0),
},
);
let result = opt.run(&problem);
// 1. Get the Pareto front.
let front = &result.pareto_front;
// 2. Define your preferences as a scoring function over (oriented)
// objective values. Lower score = preferred.
let space = problem.objectives();
let weights = [1.0, 2.0, 0.5];
let scored: Vec<(f64, &Candidate<Vec<f64>>)> = front.iter()
.map(|c| {
let oriented = space.as_minimization(&c.evaluation.objectives);
let score: f64 = oriented.iter().zip(&weights)
.map(|(v, w)| v * w)
.sum();
(score, c)
})
.collect();
// 3. Pick the lowest-scoring point.
let best = scored.iter()
.min_by(|a, b| a.0.partial_cmp(&b.0).unwrap())
.unwrap();
println!("picked: {:?} with weighted score {:.3}",
best.1.evaluation.objectives, best.0);
```
`as_minimization` returns the objective vector with maximized axes
flipped to negative — so a single set of *positive* weights does
the right thing whether each axis is min or max.
## Why a-posteriori vs a-priori weighting
If you know your weights up front, you could just optimize the
weighted sum directly with a single-objective algorithm. Why bother
with the multi-objective dance?
Two reasons:
1. **Weighted sum can't reach concave parts of the Pareto front.**
Any single-objective optimization with a linear scalarization
converges to a point at the boundary of the convex hull. Concave
front segments are unreachable. The multi-objective optimizer
finds them.
2. **Weights are usually wrong on the first try.** Optimizing the
front first lets you see what's actually possible before deciding
how much each axis is worth. Run once, look at the trade-offs,
adjust weights.
## Penalty terms beyond linear weights
The jiggly example also adds a *hinge penalty* — a term that's zero
inside an acceptable region and grows quadratically once you exceed
some hard cap. Useful when one axis is "soft up to X, hard cap at Y":
```rust,no_run
fn hinge(x: f64, soft_cap: f64, hard_cap: f64) -> f64 {
if x <= soft_cap { 0.0 }
else if x >= hard_cap { f64::INFINITY }
else {
let t = (x - soft_cap) / (hard_cap - soft_cap);
100.0 * t * t
}
}
```
Compose linear weights + hinge penalties and you have a flexible
scoring function over the front without re-running the optimizer.
## Other strategies
- **Knee point.** Pick the point where small gains in one axis cost
large losses in another — the "elbow" of the trade-off curve.
[`Knea`] explicitly biases the search toward knees during the run.
- **Reference-direction.** Pick the point closest to a desired
trade-off direction (a unit vector in objective space).
[`Moead`] / [`Nsga3`] use this internally during search; you can
apply it post-hoc the same way.
- **Random / interactive selection.** Show the front to a user
(perhaps via a plotting library), let them pick.
The right pick depends on the problem; the front itself doesn't
prescribe one.
[`Knea`]: https://docs.rs/heuropt/latest/heuropt/algorithms/knea/struct.Knea.html
[`Moead`]: https://docs.rs/heuropt/latest/heuropt/algorithms/moead/struct.Moead.html
[`Nsga3`]: https://docs.rs/heuropt/latest/heuropt/algorithms/nsga3/struct.Nsga3.html
+244
View File
@@ -0,0 +1,244 @@
# Defining a problem
Everything in heuropt starts with the [`Problem`] trait. This chapter
walks through every shape it can take.
## The trait
```rust,ignore
pub trait Problem {
type Decision: Clone;
fn objectives(&self) -> ObjectiveSpace;
fn evaluate(&self, decision: &Self::Decision) -> Evaluation;
}
```
Three things you decide:
1. **`Decision`** — the type of the thing you're optimizing.
`Vec<f64>` is by far the most common; `Vec<bool>` for binary
search, `Vec<usize>` for permutations, your own struct for
anything else.
2. **`objectives`** — how many objectives you have, what they're
called, and whether each is minimized or maximized. Returned as
an [`ObjectiveSpace`].
3. **`evaluate`** — given one decision, score it. Returns an
[`Evaluation`] with a vector of objective values (and optionally
a constraint-violation scalar).
`evaluate` takes `&self`, so caches and lookup tables are easy. It
is called many thousands of times during a typical run, so keep it
fast.
## Single-objective continuous
The Rosenbrock banana — minimize a smooth non-convex valley.
```rust,no_run
use heuropt::prelude::*;
struct Rosenbrock;
impl Problem for Rosenbrock {
type Decision = Vec<f64>;
fn objectives(&self) -> ObjectiveSpace {
ObjectiveSpace::new(vec![Objective::minimize("f")])
}
fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
let f: f64 = x.windows(2)
.map(|w| 100.0 * (w[1] - w[0].powi(2)).powi(2) + (1.0 - w[0]).powi(2))
.sum();
Evaluation::new(vec![f])
}
}
```
## Multi-objective
ZDT1 — two objectives that conflict. The Pareto front is the set of
non-dominated trade-offs.
```rust,no_run
use heuropt::prelude::*;
struct Zdt1 { dim: usize }
impl Problem for Zdt1 {
type Decision = Vec<f64>;
fn objectives(&self) -> ObjectiveSpace {
ObjectiveSpace::new(vec![
Objective::minimize("f1"),
Objective::minimize("f2"),
])
}
fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
let n = x.len() as f64;
let f1 = x[0];
let g = 1.0 + 9.0 * x[1..].iter().sum::<f64>() / (n - 1.0);
let h = 1.0 - (f1 / g).sqrt();
let f2 = g * h;
Evaluation::new(vec![f1, f2])
}
}
```
For multi-objective problems, pick a Pareto-aware optimizer:
[`Nsga2`] is the canonical default; [`Mopso`] often wins on
smooth-front 2-objective problems; [`Ibea`] often wins on
disconnected fronts. See [choosing-an-algorithm](./choosing-an-algorithm.md).
## Maximizing instead of minimizing
heuropt's internals normalize everything to minimization, but you
declare your objective with the orientation that's natural for your
problem. A scoring problem might want to maximize:
```rust,no_run
use heuropt::prelude::*;
let space = ObjectiveSpace::new(vec![
Objective::minimize("cost"),
Objective::maximize("accuracy"),
]);
```
`Objective::maximize` is a convenience for `Direction::Maximize`. Mix
freely; the Pareto-comparison machinery handles the orientation.
## Constraints
heuropt models constraints as a single non-negative scalar
**`constraint_violation`** on each `Evaluation`. The convention:
- `0.0` (or negative) means **feasible**.
- Any positive value means **infeasible**, and bigger numbers are
worse violations.
Pareto-comparison and tournament-selection helpers prefer feasible
candidates and break ties on the violation magnitude — so the rule
"feasibility comes first" is enforced automatically.
```rust,no_run
use heuropt::prelude::*;
struct Constrained;
impl Problem for Constrained {
type Decision = Vec<f64>;
fn objectives(&self) -> ObjectiveSpace {
ObjectiveSpace::new(vec![Objective::minimize("f")])
}
fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
let f: f64 = x.iter().map(|v| v * v).sum();
// Constraint: x[0] + x[1] >= 1. Violation = how much we miss it by.
let g1 = (1.0 - (x[0] + x[1])).max(0.0);
let total_violation: f64 = g1; // sum of max(0, gᵢ) for each constraint
Evaluation::constrained(vec![f], total_violation)
}
}
```
If your constraints are very tight and the search keeps hitting them,
see [Constrain your search with `Repair`](./cookbook/constraints.md).
## Decision types beyond `Vec<f64>`
### Binary (`Vec<bool>`)
```rust,no_run
use heuropt::prelude::*;
struct OneMax { bits: usize }
impl Problem for OneMax {
type Decision = Vec<bool>;
fn objectives(&self) -> ObjectiveSpace {
ObjectiveSpace::new(vec![Objective::maximize("ones")])
}
fn evaluate(&self, x: &Vec<bool>) -> Evaluation {
Evaluation::new(vec![x.iter().filter(|b| **b).count() as f64])
}
}
```
For `Vec<bool>` problems, [`Umda`] is a parameter-free EDA;
[`GeneticAlgorithm`] with [`BitFlipMutation`] is the GA route.
### Permutations (`Vec<usize>`)
```rust,no_run
use heuropt::prelude::*;
struct Tsp { distances: Vec<Vec<f64>> }
impl Problem for Tsp {
type Decision = Vec<usize>;
fn objectives(&self) -> ObjectiveSpace {
ObjectiveSpace::new(vec![Objective::minimize("length")])
}
fn evaluate(&self, tour: &Vec<usize>) -> Evaluation {
let mut len = 0.0;
for w in tour.windows(2) {
len += self.distances[w[0]][w[1]];
}
len += self.distances[*tour.last().unwrap()][tour[0]];
Evaluation::new(vec![len])
}
}
```
For permutations, [`AntColonyTsp`] specializes on TSP-style problems;
[`TabuSearch`] takes a user-supplied neighbor function for arbitrary
discrete neighborhoods; [`SimulatedAnnealing`] with [`SwapMutation`]
is the simplest baseline.
### Custom decision types
Any `Clone` type works. If you have a struct, just implement `Clone`
and you can use it. You'll need to write your own `Variation` impl
to mutate it; see [Write your own algorithm](./cookbook/custom-optimizer.md).
## What `Evaluation` carries
```rust,ignore
pub struct Evaluation {
pub objectives: Vec<f64>, // one entry per objective
pub constraint_violation: f64, // 0.0 = feasible
}
```
That's it. Construct with [`Evaluation::new`] for unconstrained
problems or [`Evaluation::constrained`] when you have a violation.
## Summary
- Implement [`Problem`] with your decision type.
- Declare objectives via [`ObjectiveSpace`] (mix minimize/maximize
freely).
- Return an [`Evaluation`] from `evaluate`.
- For constraints, set `constraint_violation > 0` for infeasible
decisions; heuropt's selection helpers prefer feasibles
automatically.
Next: [Choosing an algorithm](./choosing-an-algorithm.md) walks
through the decision tree.
[`Problem`]: https://docs.rs/heuropt/latest/heuropt/core/problem/trait.Problem.html
[`ObjectiveSpace`]: https://docs.rs/heuropt/latest/heuropt/core/objective/struct.ObjectiveSpace.html
[`Evaluation`]: https://docs.rs/heuropt/latest/heuropt/core/evaluation/struct.Evaluation.html
[`Evaluation::new`]: https://docs.rs/heuropt/latest/heuropt/core/evaluation/struct.Evaluation.html#method.new
[`Evaluation::constrained`]: https://docs.rs/heuropt/latest/heuropt/core/evaluation/struct.Evaluation.html#method.constrained
[`Nsga2`]: https://docs.rs/heuropt/latest/heuropt/algorithms/nsga2/struct.Nsga2.html
[`Mopso`]: https://docs.rs/heuropt/latest/heuropt/algorithms/mopso/struct.Mopso.html
[`Ibea`]: https://docs.rs/heuropt/latest/heuropt/algorithms/ibea/struct.Ibea.html
[`Umda`]: https://docs.rs/heuropt/latest/heuropt/algorithms/umda/struct.Umda.html
[`GeneticAlgorithm`]: https://docs.rs/heuropt/latest/heuropt/algorithms/genetic_algorithm/struct.GeneticAlgorithm.html
[`BitFlipMutation`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.BitFlipMutation.html
[`AntColonyTsp`]: https://docs.rs/heuropt/latest/heuropt/algorithms/ant_colony_tsp/struct.AntColonyTsp.html
[`TabuSearch`]: https://docs.rs/heuropt/latest/heuropt/algorithms/tabu_search/struct.TabuSearch.html
[`SimulatedAnnealing`]: https://docs.rs/heuropt/latest/heuropt/algorithms/simulated_annealing/struct.SimulatedAnnealing.html
[`SwapMutation`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.SwapMutation.html
+167
View File
@@ -0,0 +1,167 @@
# Five-minute walkthrough
The shortest path from a fresh project to a working optimizer.
## 1. Add heuropt to your `Cargo.toml`
```toml
[dependencies]
heuropt = "0.8"
```
The default feature set is small. Optional features:
- `parallel` — rayon-backed parallel population evaluation.
- `serde``Serialize` / `Deserialize` derives on the core data
types.
- `async``AsyncProblem` trait + per-algorithm `run_async` for
IO-bound evaluations.
```toml
heuropt = { version = "0.8", features = ["parallel"] }
```
## 2. Define a problem and run an optimizer
A problem is a struct that implements the [`Problem`] trait. You tell
heuropt what kind of decision your problem takes (`Vec<f64>`,
`Vec<bool>`, …), what objectives it has (minimize or maximize), and
how to score one decision.
We'll fit a straight line to a handful of `(x, y)` data points by
finding the slope and intercept that minimize the sum of squared
errors — same objective as least-squares regression. For a smooth
single-objective continuous problem like this, [`CmaEs`] is a strong
default.
```rust,no_run
use heuropt::prelude::*;
struct LineFit {
points: Vec<(f64, f64)>,
}
impl Problem for LineFit {
type Decision = Vec<f64>; // [slope, intercept]
fn objectives(&self) -> ObjectiveSpace {
ObjectiveSpace::new(vec![Objective::minimize("sum_squared_error")])
}
fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
let (slope, intercept) = (x[0], x[1]);
let sse: f64 = self
.points
.iter()
.map(|(px, py)| (py - (slope * px + intercept)).powi(2))
.sum();
Evaluation::new(vec![sse])
}
}
fn main() {
// Five noisy points roughly on the line y = 2x + 1.
let problem = LineFit {
points: vec![(0.0, 1.1), (1.0, 2.9), (2.0, 5.1), (3.0, 6.8), (4.0, 9.2)],
};
// Search box: slope and intercept each in [-10, 10].
let bounds = RealBounds::new(vec![(-10.0, 10.0); 2]);
let mut opt = CmaEs::new(
CmaEsConfig {
population_size: 12,
generations: 80,
initial_sigma: 1.0,
eigen_decomposition_period: 1,
initial_mean: None,
seed: 42,
},
bounds,
);
let result = opt.run(&problem);
let best = result.best.expect("at least one feasible candidate");
let (slope, intercept) = (best.decision[0], best.decision[1]);
println!(
"best fit: y = {:.4} x + {:.4} (sse = {:.4e}, evaluations = {})",
slope, intercept, best.evaluation.objectives[0], result.evaluations,
);
println!();
println!("predictions vs actual:");
for (px, py) in &problem.points {
let pred = slope * px + intercept;
println!(
" x = {:.1} actual = {:.2} predicted = {:.4} residual = {:+.4}",
px, py, pred, py - pred,
);
}
}
```
Run with `cargo run --release` — heuristic optimization is allergic
to debug builds. The actual output:
```text
best fit: y = 2.0100 x + 1.0000 (sse = 1.0700e-1, evaluations = 960)
predictions vs actual:
x = 0.0 actual = 1.10 predicted = 1.0000 residual = +0.1000
x = 1.0 actual = 2.90 predicted = 3.0100 residual = -0.1100
x = 2.0 actual = 5.10 predicted = 5.0200 residual = +0.0800
x = 3.0 actual = 6.80 predicted = 7.0300 residual = -0.2300
x = 4.0 actual = 9.20 predicted = 9.0400 residual = +0.1600
```
### Reading the result
CMA-ES recovered **slope ≈ 2.01, intercept ≈ 1.00** — within
hundredths of the underlying line `y = 2x + 1` that the data was
sampled from. The residuals are evenly distributed in sign (3
positive, 2 negative) and small in magnitude (the largest is 0.23
at `x = 3`), which means the fit is balancing the noise rather than
chasing any single point.
The total **sum of squared errors is 0.107** — that is the value
the optimizer was actually minimizing, and it matches the answer
you'd get from running `numpy.polyfit` or solving the normal
equations directly. CMA-ES is overkill for a two-parameter problem
(closed-form least-squares does it in one step), but the **same
code shape** scales straight up to nonlinear models, robust loss
functions, or constrained variants where there is no closed form.
It used 960 evaluations to get there. That's `population_size × generations`
= 12 × 80 = 960, and CMA-ES converges to machine epsilon on
problems this clean in well under that budget.
## 4. What just happened
- [`Problem`] is the **what** you're optimizing.
- [`CmaEs`] (or any other optimizer) is the **how**.
- [`CmaEsConfig`] is a plain public-field struct: there are no
builders, no chained setters, just public fields you set
directly.
- [`Optimizer::run`] returns an [`OptimizationResult`] containing the
full final `population`, the `pareto_front` (just the best for
single-objective), the `best` candidate, the total `evaluations`,
and the number of `generations`.
## 5. Where to go next
- **Multi-objective:** see [Defining a problem](./defining-problems.md)
for how to express two or more objectives, and
[Choosing an algorithm](./choosing-an-algorithm.md) for which
optimizer fits.
- **Want to know which algorithm to pick:** read the README's
decision tree, or jump straight to the [choosing-an-algorithm](./choosing-an-algorithm.md)
chapter for the long form.
- **Production patterns:** the [cookbook](./cookbook.md) has recipes
for parallelism, expensive evaluations, comparing algorithms, and
more.
[`Problem`]: https://docs.rs/heuropt/latest/heuropt/core/problem/trait.Problem.html
[`Optimizer::run`]: https://docs.rs/heuropt/latest/heuropt/traits/trait.Optimizer.html
[`OptimizationResult`]: https://docs.rs/heuropt/latest/heuropt/core/result/struct.OptimizationResult.html
[`CmaEs`]: https://docs.rs/heuropt/latest/heuropt/algorithms/cma_es/struct.CmaEs.html
[`CmaEsConfig`]: https://docs.rs/heuropt/latest/heuropt/algorithms/cma_es/struct.CmaEsConfig.html
+91
View File
@@ -0,0 +1,91 @@
# Introduction
heuropt is a practical Rust toolkit for **heuristic optimization** — the
art of searching for good answers when the problem is too gnarly to
solve analytically.
The kinds of problems heuropt is built for:
- **Single-objective:** "find the parameters that minimize the loss of
this model." Hyperparameter tuning. Curve fitting. Calibration.
- **Multi-objective:** "find the trade-off curve between cost and
accuracy." Engineering design. Portfolio optimization. Fleet
scheduling.
- **Many-objective (4+):** the same idea but with enough objectives
that classical Pareto methods break down. Power-grid planning.
Airfoil design. Multi-criteria recommendation.
If your problem is differentiable and convex, you don't need this
crate — use a gradient solver. heuropt is for the *messy* problems:
landscapes with lots of local minima, decisions that aren't continuous
(permutations, bit vectors), or evaluations that are noisy / expensive
/ black-box.
## Why heuropt
There are other Rust optimization crates and many more in Python (pymoo,
hyperopt, optuna, DEAP). heuropt's design priorities:
1. **Approachable code.** No trait objects in the public API. No
GATs, HRTBs, generic-RNG plumbing. A junior Rust engineer should
be able to read `RandomSearch` and write a new optimizer by
implementing only the `Optimizer<P>` trait.
2. **One concrete RNG type.** Seeded determinism is a property tested
across the crate; identical inputs always produce identical
outputs.
3. **Algorithms that work.** Every algorithm is benchmarked against
the canonical test problems (ZDT, DTLZ, Rastrigin, Rosenbrock,
Ackley) and the results are checked into [examples/compare-results.md](https://github.com/swaits/heuropt/blob/main/examples/compare-results.md)
so you can see what each algorithm's strengths actually are.
4. **Testing as a first-class concern.** 316+ unit / integration /
property tests, eight cargo-fuzz targets in CI, gungraun
instruction-count benchmarks. The fuzzers find real bugs and the
property tests check actual invariants.
## What's in the box
heuropt v0.8 ships **33 algorithms** spanning:
- Single-objective continuous: `RandomSearch`, `HillClimber`,
`OnePlusOneEs`, `SimulatedAnnealing`, `GeneticAlgorithm`,
`ParticleSwarm`, `DifferentialEvolution`, `Tlbo`, `CmaEs`,
`IpopCmaEs`, `SeparableNes`, `NelderMead`.
- Single-objective other types: `Umda` (binary), `TabuSearch`
(any), `AntColonyTsp` (permutation).
- Multi-objective (23): `Paes`, `Nsga2`, `Spea2`, `Mopso`, `Ibea`,
`SmsEmoa`, `HypE`, `EpsilonMoea`, `PesaII`, `AgeMoea`, `Knea`,
`Moead`.
- Many-objective (4+): `Nsga3`, `Rvea`, `Grea`.
- Sample-efficient / multi-fidelity: `BayesianOpt`, `Tpe`,
`Hyperband`.
Plus the operators (SBX, PolynomialMutation, BoundedGaussianMutation,
LevyMutation, BitFlipMutation, SwapMutation, ClampToBounds,
ProjectToSimplex), the metrics (hypervolume, spacing), and the Pareto
utilities (dominance, fronts, crowding distance, DasDennis reference
points, the `ParetoArchive`) that you'd expect.
**Async evaluation** (since v0.8, behind the `async` feature flag):
when your `evaluate` function is IO-bound — calling an HTTP service,
an RPC, or a subprocess — implement [`AsyncProblem`] and use
`run_async(&problem, concurrency).await` on any algorithm in the
catalog. heuropt is the only mainstream optimization library with
first-class async support across its entire algorithm set.
[`AsyncProblem`]: https://docs.rs/heuropt/latest/heuropt/core/async_problem/trait.AsyncProblem.html
## How to use this guide
If you're new to heuropt, read it linearly:
1. [Five-minute walkthrough](./getting-started.md) — install, define
a problem, run an optimizer, look at the result.
2. [Defining a problem](./defining-problems.md) — the `Problem`
trait in depth: single- vs multi-objective, constraints, custom
decision types.
3. [Choosing an algorithm](./choosing-an-algorithm.md) — the
decision tree, expanded with the reasoning behind each branch.
If you're already up and running, jump into the [cookbook](./cookbook.md)
for recipes, or [comparison](./comparison.md) for how heuropt stacks
up against other libraries.
+127
View File
@@ -0,0 +1,127 @@
# Migration guides
Per-release notes for upgrading between heuropt versions. Skip the
sections that don't apply to your starting version.
## To 0.8
### From 0.5.x
**Additive feature only.** Bumping `heuropt = "0.8"` is enough for
any code that doesn't need async evaluation. To opt into async,
enable the new feature flag:
```toml
heuropt = { version = "0.8", features = ["async"] }
```
What changed:
- New `async` feature flag, gated on the
[`futures`](https://crates.io/crates/futures) crate.
- New `core::async_problem::AsyncProblem` trait — mirrors `Problem`
but with `async fn evaluate_async`.
- New `core::async_problem::AsyncPartialProblem` trait — mirrors
`PartialProblem` for multi-fidelity (Hyperband) workloads.
- `run_async(&problem, concurrency).await` on **every** algorithm in
the catalog (33 of them) for IO-bound evaluations.
- New cookbook recipe: [Async evaluation](./cookbook/async.md).
### From 0.7.x
`0.7.0` introduced an experimental observability layer (`Snapshot`,
`Observer`, `run_with`, `MaxTime`, `TargetFitness`, `Stagnation`,
`Periodic`, `AnyOf`, `AllOf`, `TracingObserver`) and three
additional Pareto metrics (`igd`, `igd_plus`, `r2`). All of those
were rolled back in `0.8.0` — the design didn't bake long enough
and they shipped half-wired (`run_with` was overridden on only 3 of
35 algorithms). The `tracing` feature flag is also gone.
If your code uses any of those APIs, the migration is:
- Remove all `run_with(&problem, &mut observer)` calls and replace
with `run(&problem)`.
- Remove all uses of `Observer`, `Snapshot`, `ControlFlow`,
`MaxTime`, `MaxIterations`, `TargetFitness`, `Stagnation`,
`Periodic`, `AnyOf`, `AllOf`, `TracingObserver`.
- Remove all uses of `metrics::igd::igd`, `metrics::igd::igd_plus`,
`metrics::r2::r2`.
- Remove `Population::as_slice()` calls (the method is gone).
- Drop the `tracing` feature from your `Cargo.toml` if you had it.
Stop conditions can still be implemented by wrapping `run` in a
loop with a custom RNG-driven termination, or by wrapping
the algorithm yourself; observers may return as a public API in a
future release once the design has settled.
The async work introduced in 0.7.0 (`AsyncProblem` + `run_async`)
**survived** and is broadened in 0.8: every algorithm in the catalog
now has a `run_async` (0.7.0 only had it on three of them), and
multi-fidelity problems get a parallel `AsyncPartialProblem` trait
that Hyperband's `run_async` consumes. Existing call sites continue
to work unchanged.
## To 0.5
### From 0.4.x
**No public-API changes.** v0.5 is a documentation-and-polish release.
Bumping `heuropt = "0.5"` in your Cargo.toml is enough.
What changed:
- Added a comprehensive mdbook user guide (this book).
- Added runnable rustdoc examples on every public algorithm,
operator, metric, and Pareto utility.
- Added real-world `examples/portfolio.rs`,
`examples/hyperparam_tuning.rs`, and `examples/scheduling.rs`.
- Added `CONTRIBUTING.md`, `SECURITY.md`, `CODE_OF_CONDUCT.md`
(Builder's Code of Conduct), GitHub issue templates, and PR
template.
The full list is in CHANGELOG.md.
### From earlier than 0.4
If you're coming from 0.3.x or earlier, also read the older sections
below.
## To 0.4
### From 0.3.x
**No public-API changes.** v0.4 was a testing-infrastructure
expansion + perf pass. Same `cargo update` story.
The compare-harness wall-clock got 3.27× faster on v0.4 with
bit-identical quality metrics, so any benchmark numbers you have
from v0.3 are still numerically accurate but will run faster.
## To 0.3
### From 0.2.x
**Additive only.** New algorithms (`BayesianOpt`, `Tpe`,
`OnePlusOneEs`, `IpopCmaEs`, `SeparableNes`, `NelderMead`,
`Hyperband`), new operators (`LevyMutation`, `ClampToBounds`,
`ProjectToSimplex`), new traits (`PartialProblem`, `Repair<D>`).
`CmaEsConfig` gained an `initial_mean: Option<Vec<f64>>` field;
existing call sites need a `.. CmaEsConfig { initial_mean: None,
.. }` update.
## To 0.2
### From 0.1.x
**Additive.** New algorithms across the catalog (HillClimber, SA,
GA, PSO, CMA-ES, TabuSearch, AntColonyTsp, Umda, TLBO, MOPSO, IBEA,
SMS-EMOA, HypE, RVEA, PESA-II, ε-MOEA, AGE-MOEA, GrEA, KnEA), new
operators (`SimulatedBinaryCrossover`, `PolynomialMutation`,
`CompositeVariation`, `BoundedGaussianMutation`), and the
`hypervolume_nd` metric.
`Optimizer<P>` impls now require `P: Sync` and `P::Decision: Send`
(this enables the `parallel` feature without changing the public
trait surface). Any normal `Problem` you've written satisfies these
bounds automatically.
+95
View File
@@ -0,0 +1,95 @@
# Stability and SemVer
heuropt is pre-1.0. The public API may change between minor versions.
This page sets explicit expectations.
## What "public API" means in heuropt
The crate's public surface is everything re-exported from
[`heuropt::prelude`] plus the items reachable from `heuropt::core`,
`heuropt::traits`, `heuropt::operators`, `heuropt::algorithms`,
`heuropt::pareto`, `heuropt::metrics`, and `heuropt::selection`.
Items in `heuropt::internal` (e.g. the Cholesky / eigendecomposition
helpers) are **not** public API. They may change between any two
versions — use them at your own risk.
## SemVer in heuropt 0.x
While we are pre-1.0:
- **Minor bumps (`0.8 → 0.9`) may break the public API.** The
CHANGELOG calls out everything that changed, and a **migration
guide** in this book documents the move.
- **Patch bumps (`0.8.0 → 0.8.1`) only contain bug fixes,
performance improvements, and additive non-breaking features.**
No deprecations, no removals.
## What's actually likely to change before 1.0
In rough order of likelihood:
1. **Algorithm config structs may gain fields.** All current configs
are public-field structs; adding a non-`Default` field is a
breaking change. We may switch to builder patterns to avoid this
class of break, or we may add `#[non_exhaustive]`.
2. **Some operators may move between `operators` and `pareto`** as
the boundary between "things that produce candidates" and "Pareto
utilities" gets clearer.
What is **not** likely to change:
- The `Problem` trait shape.
- The `AsyncProblem` / `AsyncPartialProblem` trait shapes.
- The `Variation` / `Initializer` / `Repair` traits.
- The `Optimizer<P>` trait — single `run` method, no callbacks.
- The `Evaluation` / `Candidate` / `Population` / `OptimizationResult`
data types.
- The seeded determinism property.
## What "bit-identical" means for stability
heuropt promises that a given algorithm + seed + config produces the
same numeric output on the same minor version of heuropt.
Across minor versions, output may change if an algorithm's
implementation changes (e.g. a perf rewrite that reorders
floating-point operations, or a new feature that changes the
RNG-consumption pattern). The CHANGELOG calls this out explicitly
when it happens. As of v0.8, the entire history of perf
optimizations has been bit-identical against the v0.3.0 reference.
## MSRV (minimum supported Rust version)
heuropt's MSRV is **1.85** as of v0.8. This is tested in CI against
every PR.
MSRV bumps are treated as patch-bump-eligible (they don't break the
public API). When the MSRV is bumped, the CHANGELOG entry for that
release will note the new MSRV.
## Feature-flag stability
The current optional features:
- `serde` — adds `Serialize` / `Deserialize` derives on the core data
types.
- `parallel` — rayon-backed parallel population evaluation.
- `async``AsyncProblem` + `AsyncPartialProblem` traits, plus a
`run_async` method on every algorithm in the catalog, for
IO-bound evaluations.
Features added in 0.x can be renamed or removed in any minor bump
that documents the change. Removing a feature is treated like a
breaking API change.
## How to track changes
- **CHANGELOG.md** — the canonical record of changes per release.
- **Migration guides** — per-release, in this book at
[migration](./migration.md).
- **GitHub releases** — each tag has release notes.
- **Watch the repo** — https://github.com/swaits/heuropt — to be
notified of new releases.
[`heuropt::prelude`]: https://docs.rs/heuropt/latest/heuropt/prelude/index.html
+84
View File
@@ -0,0 +1,84 @@
//! Async evaluation example: optimize hyperparameters where each
//! evaluation is an awaitable (simulated HTTP) call.
//!
//! Demonstrates:
//! - Implementing [`AsyncProblem`].
//! - Driving the optimizer through `tokio` with bounded concurrency.
//! - Comparing wall-clock time at concurrency = 1 vs 8.
//!
//! Run with: `cargo run --release --features async --example async_eval`
use std::time::Instant;
use heuropt::core::async_problem::AsyncProblem;
use heuropt::prelude::*;
struct RemoteService;
impl AsyncProblem for RemoteService {
type Decision = Vec<f64>;
fn objectives(&self) -> ObjectiveSpace {
ObjectiveSpace::new(vec![Objective::minimize("loss")])
}
async fn evaluate_async(&self, x: &Vec<f64>) -> Evaluation {
// Simulate a 20 ms remote-service round-trip per evaluation.
// The compute itself is ~free; the latency is the bottleneck.
tokio::time::sleep(std::time::Duration::from_millis(20)).await;
let loss: f64 = x.iter().map(|v| v * v).sum();
Evaluation::new(vec![loss])
}
}
#[tokio::main]
async fn main() {
let bounds = vec![(-1.0_f64, 1.0_f64); 4];
let problem = RemoteService;
println!("RandomSearch with 200 evaluations (20 ms each)");
println!();
for &concurrency in &[1_usize, 4, 16] {
let mut opt = RandomSearch::new(
RandomSearchConfig {
iterations: 100,
batch_size: 2,
seed: 42,
},
RealBounds::new(bounds.clone()),
);
let started = Instant::now();
let result = opt.run_async(&problem, concurrency).await;
let elapsed = started.elapsed();
println!(
"concurrency = {:>2} elapsed = {:>5} ms best loss = {:>8.5} evaluations = {}",
concurrency,
elapsed.as_millis(),
result.best.unwrap().evaluation.objectives[0],
result.evaluations,
);
}
println!();
println!("DifferentialEvolution at concurrency=8");
let started = Instant::now();
let mut de = DifferentialEvolution::new(
DifferentialEvolutionConfig {
population_size: 8,
generations: 10,
differential_weight: 0.5,
crossover_probability: 0.9,
seed: 42,
},
RealBounds::new(bounds.clone()),
);
let result = de.run_async(&problem, 8).await;
let elapsed = started.elapsed();
println!(
"elapsed = {:>5} ms best loss = {:>8.5} evaluations = {}",
elapsed.as_millis(),
result.best.unwrap().evaluation.objectives[0],
result.evaluations,
);
}
+124
View File
@@ -0,0 +1,124 @@
//! Tune a synthetic ML model's hyperparameters with Bayesian Optimization
//! and (separately) Tree-structured Parzen Estimator.
//!
//! The "model" here is a deterministic function over `(learning_rate,
//! weight_decay, depth)` that mimics the shape of a real validation-loss
//! surface — a noisy minimum near sensible hyperparameters with sharp
//! penalties as you stray. It's compute-cheap so the example runs in
//! seconds, but the *workflow* is exactly what you'd use on a real
//! 30-second-per-eval model.
//!
//! Demonstrates:
//! - Sample-efficient optimization: 60 evaluations total, not 60,000.
//! - Comparing BO vs TPE on the same problem with the same budget.
//! - Decoding decision vectors with mixed scales (log-uniform learning
//! rate, integer-valued depth) using transforms inside `evaluate`.
//!
//! Run with: `cargo run --release --example hyperparam_tuning`
use heuropt::prelude::*;
/// A pretend deep-learning model whose validation loss is a
/// reproducible analytic function of three hyperparameters.
struct ModelTuning;
impl Problem for ModelTuning {
type Decision = Vec<f64>;
fn objectives(&self) -> ObjectiveSpace {
ObjectiveSpace::new(vec![Objective::minimize("val_loss")])
}
fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
// The decision vector is in [0, 1] per dim; we decode each axis
// into the "real" hyperparameter space.
let lr = log_uniform(x[0], 1e-5, 1e-1); // learning rate
let wd = log_uniform(x[1], 1e-6, 1e-2); // weight decay
let depth = scale_to_int(x[2], 2, 12); // num layers
// Synthetic validation loss surface:
// * minimum at lr ≈ 1e-3, wd ≈ 1e-4, depth = 6
// * log-quadratic in lr / wd (typical hyperparameter shape)
// * mild penalty for depth far from 6
// * tiny deterministic "noise" so flat regions don't all tie
let lr_term = (lr.log10() - (-3.0)).powi(2);
let wd_term = (wd.log10() - (-4.0)).powi(2);
let depth_term = 0.05 * ((depth as f64 - 6.0).abs());
let noise = 0.02 * ((10.0 * x[0] + 17.0 * x[1] + 23.0 * x[2]).sin());
let val_loss = 0.05 + 0.3 * lr_term + 0.2 * wd_term + depth_term + noise;
Evaluation::new(vec![val_loss])
}
}
fn log_uniform(unit: f64, lo: f64, hi: f64) -> f64 {
let log_lo = lo.ln();
let log_hi = hi.ln();
(log_lo + unit * (log_hi - log_lo)).exp()
}
fn scale_to_int(unit: f64, lo: i32, hi: i32) -> i32 {
let span = (hi - lo + 1) as f64;
let i = (unit * span).floor() as i32;
(lo + i).min(hi)
}
fn run_bo(seed: u64) -> OptimizationResult<Vec<f64>> {
let mut opt = BayesianOpt::new(
BayesianOptConfig {
initial_samples: 10,
iterations: 50, // 60 total evals
length_scales: None,
signal_variance: 1.0,
noise_variance: 1e-6,
acquisition_samples: 200,
seed,
},
RealBounds::new(vec![(0.0, 1.0); 3]),
);
opt.run(&ModelTuning)
}
fn run_tpe(seed: u64) -> OptimizationResult<Vec<f64>> {
let mut opt = Tpe::new(
TpeConfig {
initial_samples: 10,
iterations: 50, // 60 total evals
good_fraction: 0.25,
candidate_samples: 64,
bandwidth_factor: 1.0,
seed,
},
RealBounds::new(vec![(0.0, 1.0); 3]),
);
opt.run(&ModelTuning)
}
fn report(name: &str, r: &OptimizationResult<Vec<f64>>) {
let best = r.best.as_ref().expect("at least one feasible candidate");
let lr = log_uniform(best.decision[0], 1e-5, 1e-1);
let wd = log_uniform(best.decision[1], 1e-6, 1e-2);
let depth = scale_to_int(best.decision[2], 2, 12);
println!(
"{:<8} val_loss = {:>7.4} | lr = {:>10.2e} wd = {:>10.2e} depth = {} | evals = {}",
name, best.evaluation.objectives[0], lr, wd, depth, r.evaluations,
);
}
fn main() {
println!("Tuning ModelTuning (synthetic 3-D loss surface)");
println!("Optimum: lr ≈ 1e-3, wd ≈ 1e-4, depth = 6, val_loss ≈ 0.03");
println!();
println!(
"{:<8} {:<26} {:<24} {:<24}",
"alg", "best", "(decoded hyperparams)", "(eval budget)"
);
for seed in 0..5 {
println!();
println!("seed {}:", seed);
let bo = run_bo(seed);
let tpe = run_tpe(seed);
report("BO", &bo);
report("TPE", &tpe);
}
}
+210
View File
@@ -0,0 +1,210 @@
//! Multi-objective portfolio optimization with a budget constraint.
//!
//! Real-world flavor: pick a portfolio over five synthetic assets that
//! trades off **return** (maximize) against **risk** (minimize). Weights
//! must be non-negative and sum to 1.0 (the standard probability-simplex
//! budget constraint).
//!
//! Demonstrates:
//! - Multi-objective formulation with a maximize axis (return) and a
//! minimize axis (variance-based risk).
//! - The `ProjectToSimplex` repair operator wired into a `Repair`-aware
//! variation pipeline so every offspring respects the budget.
//! - NSGA-II producing a Pareto front of trade-offs.
//! - Picking one answer off the front via a-posteriori weighting (see
//! `docs/book/src/cookbook/pick-one.md`).
//!
//! Run with: `cargo run --release --example portfolio`
use heuropt::prelude::*;
/// Five-asset toy market. Means and a covariance matrix you'd estimate
/// from real returns; here they're synthetic but realistic-shape.
struct Portfolio {
/// Expected per-period returns (one per asset).
expected_returns: [f64; 5],
/// Symmetric 5×5 covariance matrix.
covariance: [[f64; 5]; 5],
}
impl Problem for Portfolio {
type Decision = Vec<f64>;
fn objectives(&self) -> ObjectiveSpace {
ObjectiveSpace::new(vec![
Objective::maximize("return"),
Objective::minimize("risk"),
])
}
fn evaluate(&self, weights: &Vec<f64>) -> Evaluation {
// Expected return: w · μ
let r: f64 = weights
.iter()
.zip(self.expected_returns.iter())
.map(|(w, m)| w * m)
.sum();
// Risk (portfolio variance): w · Σ · w
let mut risk = 0.0;
for i in 0..5 {
for j in 0..5 {
risk += weights[i] * self.covariance[i][j] * weights[j];
}
}
Evaluation::new(vec![r, risk])
}
}
/// Variation pipeline that respects the simplex constraint: SBX +
/// PolyMut produce real-valued children, then `ProjectToSimplex` projects
/// them back onto `{ w : w ≥ 0, Σw = 1 }`.
struct SimplexVariation {
crossover: SimulatedBinaryCrossover,
mutation: PolynomialMutation,
repair: ProjectToSimplex,
}
impl Variation<Vec<f64>> for SimplexVariation {
fn vary(&mut self, parents: &[Vec<f64>], rng: &mut Rng) -> Vec<Vec<f64>> {
let crossed = self.crossover.vary(parents, rng);
let mut out = Vec::with_capacity(crossed.len());
for child in crossed {
let mut mutated = self
.mutation
.vary(std::slice::from_ref(&child), rng)
.pop()
.expect("PolynomialMutation returned no child");
self.repair.repair(&mut mutated);
out.push(mutated);
}
out
}
}
/// `Initializer` that uniformly samples points on the simplex via the
/// standard "log-and-normalize" trick. Every initial member is feasible
/// by construction.
struct SimplexInit {
dim: usize,
}
impl Initializer<Vec<f64>> for SimplexInit {
fn initialize(&mut self, size: usize, rng: &mut Rng) -> Vec<Vec<f64>> {
use rand::Rng as _;
let mut out = Vec::with_capacity(size);
for _ in 0..size {
// Sample exponentials, normalize → uniform on simplex.
let mut e: Vec<f64> = (0..self.dim)
.map(|_| -(1.0_f64 - rng.random::<f64>()).ln())
.collect();
let s: f64 = e.iter().sum();
for v in e.iter_mut() {
*v /= s;
}
out.push(e);
}
out
}
}
fn main() {
let problem = Portfolio {
// Synthetic but plausible: 8% / 12% / 5% / 15% / 3% expected
// returns. The two "stocks" (B, D) have higher expected return
// and higher variance than the bonds / cash equivalents.
expected_returns: [0.08, 0.12, 0.05, 0.15, 0.03],
covariance: [
[0.04, 0.02, 0.01, 0.03, 0.005],
[0.02, 0.10, 0.01, 0.05, 0.005],
[0.01, 0.01, 0.02, 0.01, 0.005],
[0.03, 0.05, 0.01, 0.16, 0.005],
[0.005, 0.005, 0.005, 0.005, 0.001],
],
};
let bounds = vec![(0.0_f64, 1.0_f64); 5];
let variation = SimplexVariation {
crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 1.0),
mutation: PolynomialMutation::new(bounds.clone(), 20.0, 1.0 / 5.0),
repair: ProjectToSimplex::new(1.0),
};
let mut opt = Nsga2::new(
Nsga2Config {
population_size: 100,
generations: 200,
seed: 42,
},
SimplexInit { dim: 5 },
variation,
);
let result = opt.run(&problem);
println!("Pareto front size: {}", result.pareto_front.len());
println!("Total evaluations: {}", result.evaluations);
// Pick one: a-posteriori weighted decision favoring return slightly.
// Lower score = preferred. We compare in oriented space (maximize
// axis already flipped to negative by `as_minimization`).
let space = problem.objectives();
let weights = [1.0, 1.5]; // weight risk a bit more than -return
let chosen = result
.pareto_front
.iter()
.min_by(|a, b| {
let ax: f64 = space
.as_minimization(&a.evaluation.objectives)
.iter()
.zip(&weights)
.map(|(v, w)| v * w)
.sum();
let bx: f64 = space
.as_minimization(&b.evaluation.objectives)
.iter()
.zip(&weights)
.map(|(v, w)| v * w)
.sum();
ax.partial_cmp(&bx).unwrap_or(std::cmp::Ordering::Equal)
})
.expect("non-empty front");
println!();
println!(
"Picked portfolio: weights = [{:.3}, {:.3}, {:.3}, {:.3}, {:.3}]",
chosen.decision[0],
chosen.decision[1],
chosen.decision[2],
chosen.decision[3],
chosen.decision[4],
);
println!(
" expected return: {:>6.4}",
chosen.evaluation.objectives[0]
);
println!(
" risk (variance): {:>6.4}",
chosen.evaluation.objectives[1]
);
// Print 5 representative points across the front.
println!();
println!("Sample of the front (return, risk):");
let mut sorted = result.pareto_front.clone();
sorted.sort_by(|a, b| {
a.evaluation.objectives[0]
.partial_cmp(&b.evaluation.objectives[0])
.unwrap_or(std::cmp::Ordering::Equal)
});
let n = sorted.len();
for k in (0..n).step_by((n / 5).max(1)) {
let c = &sorted[k];
println!(
" return = {:.4}, risk = {:.4}",
c.evaluation.objectives[0], c.evaluation.objectives[1],
);
}
}
+132
View File
@@ -0,0 +1,132 @@
//! Single-machine job-shop scheduling: minimize total weighted
//! completion time given per-job processing times and due-date weights.
//!
//! The decision is a permutation `Vec<usize>` — the order in which
//! jobs are processed. We use `SimulatedAnnealing` paired with
//! `SwapMutation` (the standard generic-permutation pair).
//!
//! Demonstrates:
//! - Permutation decisions (`Vec<usize>`).
//! - Simulated annealing with a custom `Initializer` that produces a
//! randomly shuffled identity permutation.
//! - `SwapMutation` preserving the permutation invariant for free.
//!
//! Run with: `cargo run --release --example scheduling`
use heuropt::prelude::*;
/// Single-machine weighted-completion-time problem (1 || Σwᵢ Cᵢ).
struct Scheduling {
/// Processing time for each job.
process_times: Vec<f64>,
/// Importance weight for each job. Higher weight = more
/// punishing if the job finishes late.
weights: Vec<f64>,
}
impl Problem for Scheduling {
type Decision = Vec<usize>;
fn objectives(&self) -> ObjectiveSpace {
ObjectiveSpace::new(vec![Objective::minimize("total_wct")])
}
fn evaluate(&self, schedule: &Vec<usize>) -> Evaluation {
// Compute each job's completion time as the running sum of
// processing times in the chosen order.
let mut clock = 0.0_f64;
let mut total_wct = 0.0_f64;
for &job in schedule {
clock += self.process_times[job];
total_wct += self.weights[job] * clock;
}
Evaluation::new(vec![total_wct])
}
}
/// Initializer that produces a single randomly-shuffled permutation
/// `[0, 1, …, n-1]`. Simulated annealing only needs one initial decision.
struct ShuffledPerm {
n: usize,
}
impl Initializer<Vec<usize>> for ShuffledPerm {
fn initialize(&mut self, _size: usize, rng: &mut Rng) -> Vec<Vec<usize>> {
use rand::seq::SliceRandom;
let mut perm: Vec<usize> = (0..self.n).collect();
perm.shuffle(rng);
vec![perm]
}
}
fn main() {
// 12 jobs. The optimal policy is the Smith's-rule order: sort by
// p_i / w_i ascending (shortest weighted processing time first).
// We can compute that directly to compare against the search result.
let jobs = [
(3.0_f64, 2.0_f64),
(5.0, 1.0),
(2.0, 4.0),
(8.0, 3.0),
(4.0, 5.0),
(1.0, 2.0),
(7.0, 6.0),
(6.0, 1.0),
(3.0, 3.0),
(5.0, 4.0),
(2.0, 2.0),
(4.0, 1.0),
];
let process_times: Vec<f64> = jobs.iter().map(|j| j.0).collect();
let weights: Vec<f64> = jobs.iter().map(|j| j.1).collect();
let n = jobs.len();
let problem = Scheduling {
process_times: process_times.clone(),
weights: weights.clone(),
};
// Smith's rule oracle: sort jobs by p / w ascending.
let mut smith_order: Vec<usize> = (0..n).collect();
smith_order.sort_by(|&a, &b| {
let ra = process_times[a] / weights[a];
let rb = process_times[b] / weights[b];
ra.partial_cmp(&rb).unwrap_or(std::cmp::Ordering::Equal)
});
let smith_score = problem.evaluate(&smith_order).objectives[0];
// Search via simulated annealing with swap mutation.
let mut opt = SimulatedAnnealing::new(
SimulatedAnnealingConfig {
iterations: 5_000,
initial_temperature: 50.0,
final_temperature: 1e-3,
seed: 42,
},
ShuffledPerm { n },
SwapMutation,
);
let result = opt.run(&problem);
let best = result.best.unwrap();
println!("Single-machine weighted completion time, {} jobs", n);
println!();
println!(
"Smith's-rule oracle: {:>8.2} order = {:?}",
smith_score, smith_order
);
println!(
"Simulated annealing best: {:>8.2} order = {:?}",
best.evaluation.objectives[0], best.decision,
);
println!(
"Random initial schedule: {:>8.2} order = {:?}",
problem.evaluate(&(0..n).collect()).objectives[0],
(0..n).collect::<Vec<usize>>(),
);
println!();
println!(
"SA reached optimum (Smith): {}",
(best.evaluation.objectives[0] - smith_score).abs() < 1e-9
);
}
+1 -1
View File
@@ -66,7 +66,7 @@ dependencies = [
[[package]]
name = "heuropt"
version = "0.3.0"
version = "0.8.0"
dependencies = [
"rand",
"rand_distr",
+9 -2
View File
@@ -77,10 +77,17 @@ fuzz_target!(|input: Input| {
);
let after = y.clone();
proj.repair(&mut y);
// The simplex projection's `τ` computation operates on values
// up to `simplex_total · 1e6` (per the filter above), so its FP
// precision floor is ~1e-4 of the input scale. Outputs near the
// `max(x_i τ, 0)` clamp boundary can flip between 0 and a
// small positive value across re-applications. The fuzzer is
// checking for *gross* non-idempotence (all-zeros vs valid),
// not ULP-level slop.
let scale = input.simplex_total.max(max_abs).max(1.0);
for (a, b) in after.iter().zip(y.iter()) {
let scale = a.abs().max(b.abs()).max(1.0);
assert!(
(a - b).abs() < 1e-9 * scale,
(a - b).abs() < 1e-4 * scale,
"project not idempotent: {a} vs {b}",
);
}
+103
View File
@@ -40,6 +40,35 @@ impl Default for AgeMoeaConfig {
/// score survivors by a combination of proximity (distance to the
/// translated origin in the L_p frame) and diversity (distance to the
/// nearest survivor in the same frame).
///
/// # Example
///
/// ```
/// use heuropt::prelude::*;
///
/// struct Schaffer;
/// impl Problem for Schaffer {
/// type Decision = Vec<f64>;
/// fn objectives(&self) -> ObjectiveSpace {
/// ObjectiveSpace::new(vec![Objective::minimize("f1"), Objective::minimize("f2")])
/// }
/// fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
/// Evaluation::new(vec![x[0] * x[0], (x[0] - 2.0).powi(2)])
/// }
/// }
///
/// let bounds = vec![(-5.0_f64, 5.0_f64)];
/// let mut opt = AgeMoea::new(
/// AgeMoeaConfig { population_size: 30, generations: 20, seed: 42 },
/// RealBounds::new(bounds.clone()),
/// CompositeVariation {
/// crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5),
/// mutation: PolynomialMutation::new(bounds, 20.0, 1.0),
/// },
/// );
/// let r = opt.run(&Schaffer);
/// assert!(!r.pareto_front.is_empty());
/// ```
#[derive(Debug, Clone)]
pub struct AgeMoea<I, V> {
/// Algorithm configuration.
@@ -126,6 +155,80 @@ where
}
}
#[cfg(feature = "async")]
impl<I, V> AgeMoea<I, V> {
/// Async version of [`Optimizer::run`] — drives evaluations through
/// the user-chosen async runtime. Available only with the `async`
/// feature.
///
/// `concurrency` bounds in-flight evaluations per batch.
pub async fn run_async<P>(
&mut self,
problem: &P,
concurrency: usize,
) -> OptimizationResult<P::Decision>
where
P: crate::core::async_problem::AsyncProblem,
I: Initializer<P::Decision>,
V: Variation<P::Decision>,
{
use crate::algorithms::parallel_eval_async::evaluate_batch_async;
assert!(
self.config.population_size > 0,
"AgeMoea population_size must be > 0"
);
let n = self.config.population_size;
let objectives = problem.objectives();
let mut rng = rng_from_seed(self.config.seed);
let initial_decisions = self.initializer.initialize(n, &mut rng);
let mut population: Vec<Candidate<P::Decision>> =
evaluate_batch_async(problem, initial_decisions, concurrency).await;
let mut evaluations = population.len();
for _ in 0..self.config.generations {
let mut offspring_decisions: Vec<P::Decision> = Vec::with_capacity(n);
while offspring_decisions.len() < n {
let p1 = rng.random_range(0..population.len());
let p2 = rng.random_range(0..population.len());
let parents = vec![
population[p1].decision.clone(),
population[p2].decision.clone(),
];
let children = self.variation.vary(&parents, &mut rng);
assert!(
!children.is_empty(),
"AgeMoea variation returned no children"
);
for child in children {
if offspring_decisions.len() >= n {
break;
}
offspring_decisions.push(child);
}
}
let offspring = evaluate_batch_async(problem, offspring_decisions, concurrency).await;
evaluations += offspring.len();
let mut combined: Vec<Candidate<P::Decision>> = Vec::with_capacity(2 * n);
combined.extend(population);
combined.extend(offspring);
population = environmental_selection(combined, &objectives, n);
}
let front = pareto_front(&population, &objectives);
let best = best_candidate(&population, &objectives);
OptimizationResult::new(
Population::new(population),
front,
best,
evaluations,
self.config.generations,
)
}
}
fn environmental_selection<D: Clone>(
combined: Vec<Candidate<D>>,
objectives: &ObjectiveSpace,
+160
View File
@@ -57,6 +57,53 @@ impl Default for AntColonyTspConfig {
/// Each ant builds a tour by repeatedly choosing the next node with
/// probability `∝ τ_ij^α · η_ij^β` over the unvisited cities, where
/// `η_ij = 1 / distance_ij` is the heuristic desirability.
///
/// # Example
///
/// ```
/// use heuropt::prelude::*;
///
/// struct Tsp { distances: Vec<Vec<f64>> }
/// impl Problem for Tsp {
/// type Decision = Vec<usize>;
/// fn objectives(&self) -> ObjectiveSpace {
/// ObjectiveSpace::new(vec![Objective::minimize("length")])
/// }
/// fn evaluate(&self, tour: &Vec<usize>) -> Evaluation {
/// let mut len = 0.0;
/// for w in tour.windows(2) { len += self.distances[w[0]][w[1]]; }
/// len += self.distances[*tour.last().unwrap()][tour[0]];
/// Evaluation::new(vec![len])
/// }
/// }
///
/// // 5 cities laid out in a small square + center. The optimal tour
/// // is the perimeter; the diagonal is suboptimal.
/// let cities = [(0.0_f64, 0.0), (3.0, 0.0), (3.0, 3.0), (0.0, 3.0), (1.5, 1.5)];
/// let n = cities.len();
/// let mut d = vec![vec![0.0; n]; n];
/// for i in 0..n {
/// for j in 0..n {
/// let dx = cities[i].0 - cities[j].0;
/// let dy = cities[i].1 - cities[j].1;
/// d[i][j] = (dx * dx + dy * dy).sqrt();
/// }
/// }
/// let problem = Tsp { distances: d.clone() };
///
/// let mut opt = AntColonyTsp::new(AntColonyTspConfig {
/// ants: 10,
/// generations: 50,
/// alpha: 1.0,
/// beta: 5.0,
/// evaporation: 0.5,
/// deposit: 1.0,
/// initial_pheromone: 0.1,
/// seed: 42,
/// }, d);
/// let r = opt.run(&problem);
/// assert!(r.best.is_some());
/// ```
pub struct AntColonyTsp {
/// Algorithm configuration.
pub config: AntColonyTspConfig,
@@ -194,6 +241,119 @@ where
}
}
#[cfg(feature = "async")]
impl AntColonyTsp {
/// Async version of [`Optimizer::run`] — drives evaluations through
/// the user-chosen async runtime. Available only with the `async`
/// feature.
///
/// `concurrency` bounds in-flight evaluations per generation.
pub async fn run_async<P>(
&mut self,
problem: &P,
concurrency: usize,
) -> OptimizationResult<Vec<usize>>
where
P: crate::core::async_problem::AsyncProblem<Decision = Vec<usize>>,
{
use crate::algorithms::parallel_eval_async::evaluate_batch_async;
assert!(self.config.ants >= 1, "AntColonyTsp ants must be >= 1");
let objectives = problem.objectives();
assert!(
objectives.is_single_objective(),
"AntColonyTsp requires exactly one objective",
);
let direction = objectives.objectives[0].direction;
let n = self.distances.len();
let mut rng = rng_from_seed(self.config.seed);
let eta: Vec<Vec<f64>> = self
.distances
.iter()
.map(|row| {
row.iter()
.map(|&d| if d > 0.0 { 1.0 / d } else { 0.0 })
.collect()
})
.collect();
let mut pheromone: Vec<Vec<f64>> = vec![vec![self.config.initial_pheromone; n]; n];
let mut best_decision: Option<Vec<usize>> = None;
let mut best_eval: Option<crate::core::evaluation::Evaluation> = None;
let mut evaluations = 0usize;
for _ in 0..self.config.generations {
let mut tours: Vec<Vec<usize>> = Vec::with_capacity(self.config.ants);
for _ in 0..self.config.ants {
let start = rng.random_range(0..n);
let tour = build_tour(
n,
start,
&pheromone,
&eta,
self.config.alpha,
self.config.beta,
&mut rng,
);
tours.push(tour);
}
let cands = evaluate_batch_async(problem, tours.clone(), concurrency).await;
evaluations += cands.len();
let tour_evals: Vec<crate::core::evaluation::Evaluation> =
cands.into_iter().map(|c| c.evaluation).collect();
for (tour, eval) in tours.iter().zip(tour_evals.iter()) {
let beats = match &best_eval {
None => true,
Some(b) => better_than_so(eval, b, direction),
};
if beats {
best_decision = Some(tour.clone());
best_eval = Some(eval.clone());
}
}
for row in pheromone.iter_mut() {
for v in row.iter_mut() {
*v *= 1.0 - self.config.evaporation;
}
}
for (tour, eval) in tours.iter().zip(tour_evals.iter()) {
let length = eval
.objectives
.first()
.copied()
.unwrap_or(f64::INFINITY)
.max(1e-12);
let deposit = self.config.deposit / length;
for w in tour.windows(2) {
let (i, j) = (w[0], w[1]);
pheromone[i][j] += deposit;
pheromone[j][i] += deposit;
}
let (i, j) = (*tour.last().unwrap(), tour[0]);
pheromone[i][j] += deposit;
pheromone[j][i] += deposit;
}
}
let best = Candidate::new(best_decision.unwrap(), best_eval.unwrap());
let population = Population::new(vec![best.clone()]);
let front = vec![best.clone()];
OptimizationResult::new(
population,
front,
Some(best),
evaluations,
self.config.generations,
)
}
}
fn build_tour(
n: usize,
start: usize,
+178
View File
@@ -61,6 +61,40 @@ impl Default for BayesianOptConfig {
/// evaluation budgets (50500). The GP kernel is anisotropic RBF; the
/// acquisition function is EI; both are optimized by best-of-N random
/// sampling each step (simple, predictable cost).
///
/// # Example
///
/// ```
/// use heuropt::prelude::*;
///
/// struct Sphere;
/// impl Problem for Sphere {
/// type Decision = Vec<f64>;
/// fn objectives(&self) -> ObjectiveSpace {
/// ObjectiveSpace::new(vec![Objective::minimize("f")])
/// }
/// fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
/// Evaluation::new(vec![x.iter().map(|v| v * v).sum::<f64>()])
/// }
/// }
///
/// let mut opt = BayesianOpt::new(
/// BayesianOptConfig {
/// initial_samples: 10,
/// iterations: 30,
/// length_scales: None, // default per-axis length scales
/// signal_variance: 1.0,
/// noise_variance: 1e-6,
/// acquisition_samples: 200,
/// seed: 42,
/// },
/// RealBounds::new(vec![(-3.0, 3.0); 3]),
/// );
/// let r = opt.run(&Sphere);
/// // 10 random + 30 BO steps = 40 total evaluations.
/// assert_eq!(r.evaluations, 40);
/// assert!(r.best.is_some());
/// ```
#[derive(Debug, Clone)]
pub struct BayesianOpt {
/// Algorithm configuration.
@@ -362,6 +396,150 @@ fn erf(x: f64) -> f64 {
sign * y
}
#[cfg(feature = "async")]
impl BayesianOpt {
/// Async version of [`Optimizer::run`] — drives evaluations through
/// the user-chosen async runtime. Available only with the `async`
/// feature.
///
/// `concurrency` bounds in-flight evaluations during the initial
/// uniform-sample design; the sequential BO loop runs one
/// evaluation per iteration regardless.
pub async fn run_async<P>(
&mut self,
problem: &P,
concurrency: usize,
) -> OptimizationResult<Vec<f64>>
where
P: crate::core::async_problem::AsyncProblem<Decision = Vec<f64>>,
{
use crate::algorithms::parallel_eval_async::evaluate_batch_async;
assert!(
self.config.initial_samples >= 2,
"BayesianOpt initial_samples must be >= 2",
);
assert!(
self.config.signal_variance > 0.0,
"BayesianOpt signal_variance must be > 0"
);
assert!(
self.config.noise_variance > 0.0,
"BayesianOpt noise_variance must be > 0"
);
assert!(
self.config.acquisition_samples >= 1,
"BayesianOpt acquisition_samples must be >= 1",
);
let objectives = problem.objectives();
assert!(
objectives.is_single_objective(),
"BayesianOpt requires exactly one objective",
);
let direction = objectives.objectives[0].direction;
let dim = self.bounds.bounds.len();
if let Some(ls) = &self.config.length_scales {
assert_eq!(
ls.len(),
dim,
"BayesianOpt length_scales.len() must equal dim"
);
}
let length_scales: Vec<f64> = self.config.length_scales.clone().unwrap_or_else(|| {
self.bounds
.bounds
.iter()
.map(|&(lo, hi)| 0.2 * (hi - lo).max(1e-9))
.collect()
});
let mut rng = rng_from_seed(self.config.seed);
// Initial random design: sample all decisions first (consuming
// RNG in the same order as the sync `run`), then evaluate
// concurrently.
let mut decisions: Vec<Vec<f64>> =
Vec::with_capacity(self.config.initial_samples + self.config.iterations);
let mut targets: Vec<f64> = Vec::with_capacity(decisions.capacity());
let mut evaluations: Vec<Evaluation> = Vec::with_capacity(decisions.capacity());
let initial_decisions: Vec<Vec<f64>> = (0..self.config.initial_samples)
.map(|_| sample_uniform_in_bounds(&self.bounds, &mut rng))
.collect();
let initial_cands = evaluate_batch_async(problem, initial_decisions, concurrency).await;
for c in initial_cands {
let t = oriented_target(&c.evaluation, direction);
decisions.push(c.decision);
targets.push(t);
evaluations.push(c.evaluation);
}
for _ in 0..self.config.iterations {
let posterior = match GpPosterior::fit(
&decisions,
&targets,
&length_scales,
self.config.signal_variance,
self.config.noise_variance,
) {
Ok(p) => p,
Err(_) => {
let x = sample_uniform_in_bounds(&self.bounds, &mut rng);
let e = problem.evaluate_async(&x).await;
targets.push(oriented_target(&e, direction));
decisions.push(x);
evaluations.push(e);
continue;
}
};
let best_target = targets.iter().cloned().fold(f64::INFINITY, f64::min);
let mut best_x = sample_uniform_in_bounds(&self.bounds, &mut rng);
let mut best_ei = -f64::INFINITY;
for _ in 0..self.config.acquisition_samples {
let cand = sample_uniform_in_bounds(&self.bounds, &mut rng);
let (mu, sigma) = posterior.predict(&cand);
let ei = expected_improvement(mu, sigma, best_target);
if ei > best_ei {
best_ei = ei;
best_x = cand;
}
}
let e = problem.evaluate_async(&best_x).await;
targets.push(oriented_target(&e, direction));
decisions.push(best_x);
evaluations.push(e);
}
let final_pop: Vec<Candidate<Vec<f64>>> = decisions
.into_iter()
.zip(evaluations)
.map(|(d, e)| Candidate::new(d, e))
.collect();
let mut best_idx = 0;
for i in 1..final_pop.len() {
if better(
&final_pop[i].evaluation,
&final_pop[best_idx].evaluation,
direction,
) {
best_idx = i;
}
}
let total_evaluations = final_pop.len();
let best = final_pop[best_idx].clone();
let front = vec![best.clone()];
OptimizationResult::new(
Population::new(final_pop),
front,
Some(best),
total_evaluations,
self.config.iterations + self.config.initial_samples,
)
}
}
#[cfg(test)]
mod tests {
use super::*;
+263
View File
@@ -59,6 +59,38 @@ impl Default for CmaEsConfig {
/// `Vec<f64>` decisions only. Bounds come from the embedded `RealBounds`
/// field; both the initial mean and every offspring are clamped per
/// dimension.
///
/// # Example
///
/// ```
/// use heuropt::prelude::*;
///
/// struct Sphere;
/// impl Problem for Sphere {
/// type Decision = Vec<f64>;
/// fn objectives(&self) -> ObjectiveSpace {
/// ObjectiveSpace::new(vec![Objective::minimize("f")])
/// }
/// fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
/// Evaluation::new(vec![x.iter().map(|v| v * v).sum::<f64>()])
/// }
/// }
///
/// let mut opt = CmaEs::new(
/// CmaEsConfig {
/// population_size: 12,
/// generations: 100,
/// initial_sigma: 1.0,
/// eigen_decomposition_period: 1,
/// initial_mean: None,
/// seed: 42,
/// },
/// RealBounds::new(vec![(-5.0, 5.0); 5]),
/// );
/// let r = opt.run(&Sphere);
/// // CMA-ES converges aggressively on Sphere.
/// assert!(r.best.unwrap().evaluation.objectives[0] < 1e-3);
/// ```
#[derive(Debug, Clone)]
pub struct CmaEs {
/// Algorithm configuration.
@@ -334,6 +366,237 @@ where
}
}
#[cfg(feature = "async")]
impl CmaEs {
/// Async version of [`Optimizer::run`] — drives evaluations through
/// the user-chosen async runtime. Available only with the `async`
/// feature.
///
/// `concurrency` bounds in-flight evaluations per generation.
pub async fn run_async<P>(
&mut self,
problem: &P,
concurrency: usize,
) -> OptimizationResult<Vec<f64>>
where
P: crate::core::async_problem::AsyncProblem<Decision = Vec<f64>>,
{
use crate::algorithms::parallel_eval_async::evaluate_batch_async;
assert!(
self.config.population_size >= 4,
"CmaEs population_size must be >= 4",
);
assert!(
self.config.initial_sigma > 0.0,
"CmaEs initial_sigma must be positive",
);
assert!(
self.config.eigen_decomposition_period >= 1,
"CmaEs eigen_decomposition_period must be >= 1",
);
let objectives = problem.objectives();
assert!(
objectives.is_single_objective(),
"CmaEs only supports single-objective problems",
);
let direction = objectives.objectives[0].direction;
let n = self.bounds.bounds.len();
let n_f = n as f64;
let lambda = self.config.population_size;
let lambda_f = lambda as f64;
let mu = lambda / 2;
assert!(mu >= 1, "CmaEs derived mu (= lambda/2) must be >= 1");
let mut rng = rng_from_seed(self.config.seed);
let raw_weights: Vec<f64> = (0..mu)
.map(|i| ((lambda_f + 1.0) / 2.0).ln() - ((i + 1) as f64).ln())
.collect();
let sum_w: f64 = raw_weights.iter().sum();
let weights: Vec<f64> = raw_weights.iter().map(|w| w / sum_w).collect();
let mu_eff = 1.0 / weights.iter().map(|w| w * w).sum::<f64>();
let c_sigma = (mu_eff + 2.0) / (n_f + mu_eff + 5.0);
let d_sigma = 1.0 + 2.0 * ((mu_eff - 1.0) / (n_f + 1.0)).sqrt().max(0.0) + c_sigma;
let c_c = (4.0 + mu_eff / n_f) / (n_f + 4.0 + 2.0 * mu_eff / n_f);
let c_1 = 2.0 / ((n_f + 1.3).powi(2) + mu_eff);
let c_mu = ((1.0 - c_1) * 2.0 * (mu_eff - 2.0 + 1.0 / mu_eff)
/ ((n_f + 2.0).powi(2) + mu_eff))
.min(1.0 - c_1);
let chi_n = n_f.sqrt() * (1.0 - 1.0 / (4.0 * n_f) + 1.0 / (21.0 * n_f * n_f));
let mut mean: Vec<f64> = if let Some(provided) = self.config.initial_mean.clone() {
assert_eq!(
provided.len(),
self.bounds.bounds.len(),
"CmaEs initial_mean.len() must equal the bounds dimension",
);
provided
.into_iter()
.zip(self.bounds.bounds.iter())
.map(|(v, &(lo, hi))| v.clamp(lo, hi))
.collect()
} else {
self.bounds
.bounds
.iter()
.map(|&(lo, hi)| 0.5 * (lo + hi))
.collect()
};
let mut sigma = self.config.initial_sigma;
let mut c_matrix: Vec<Vec<f64>> = (0..n)
.map(|i| (0..n).map(|j| if i == j { 1.0 } else { 0.0 }).collect())
.collect();
let mut b: Vec<Vec<f64>> = c_matrix.to_vec();
let mut d: Vec<f64> = vec![1.0; n];
let mut p_sigma = vec![0.0_f64; n];
let mut p_c = vec![0.0_f64; n];
let mut evaluations = 0usize;
let normal = Normal::new(0.0, 1.0).expect("Normal::new(0, 1)");
let mut best_candidate_seen: Option<Candidate<Vec<f64>>> = None;
for generation in 0..self.config.generations {
if generation % self.config.eigen_decomposition_period == 0 {
#[allow(clippy::needless_range_loop)]
for i in 0..n {
for j in (i + 1)..n {
let avg = 0.5 * (c_matrix[i][j] + c_matrix[j][i]);
c_matrix[i][j] = avg;
c_matrix[j][i] = avg;
}
}
let (eigenvalues, eigenvectors) = symmetric_eigen(&c_matrix, 1e-14, 100);
d = eigenvalues.iter().map(|&v| v.max(1e-20).sqrt()).collect();
b = (0..n)
.map(|r| (0..n).map(|c| eigenvectors[c][r]).collect())
.collect();
}
let mut z_samples: Vec<Vec<f64>> = Vec::with_capacity(lambda);
let mut x_samples: Vec<Vec<f64>> = Vec::with_capacity(lambda);
for _ in 0..lambda {
let z: Vec<f64> = (0..n).map(|_| normal.sample(&mut rng)).collect();
let bd_z: Vec<f64> = (0..n)
.map(|i| (0..n).map(|j| b[i][j] * d[j] * z[j]).sum::<f64>())
.collect();
let x: Vec<f64> = (0..n)
.map(|i| {
let v = mean[i] + sigma * bd_z[i];
let (lo, hi) = self.bounds.bounds[i];
v.clamp(lo, hi)
})
.collect();
z_samples.push(z);
x_samples.push(x);
}
let evaluated = evaluate_batch_async(problem, x_samples.clone(), concurrency).await;
evaluations += evaluated.len();
for c in &evaluated {
let beats_best = match &best_candidate_seen {
None => true,
Some(b) => better_than_so(&c.evaluation, &b.evaluation, direction),
};
if beats_best {
best_candidate_seen = Some(c.clone());
}
}
let mut order: Vec<usize> = (0..lambda).collect();
order.sort_by(|&a, &b_| {
compare_so(
&evaluated[a].evaluation,
&evaluated[b_].evaluation,
direction,
)
});
let old_mean = mean.clone();
let mut new_mean = vec![0.0_f64; n];
for k in 0..mu {
let xk = &x_samples[order[k]];
let wk = weights[k];
for i in 0..n {
new_mean[i] += wk * xk[i];
}
}
mean = new_mean;
let mut z_weighted = vec![0.0_f64; n];
for k in 0..mu {
let zk = &z_samples[order[k]];
let wk = weights[k];
for i in 0..n {
z_weighted[i] += wk * zk[i];
}
}
let factor_p_sigma = (c_sigma * (2.0 - c_sigma) * mu_eff).sqrt();
let bz: Vec<f64> = (0..n)
.map(|i| (0..n).map(|j| b[i][j] * z_weighted[j]).sum::<f64>())
.collect();
for i in 0..n {
p_sigma[i] = (1.0 - c_sigma) * p_sigma[i] + factor_p_sigma * bz[i];
}
let p_sigma_norm = p_sigma.iter().map(|x| x * x).sum::<f64>().sqrt();
sigma *= ((c_sigma / d_sigma) * (p_sigma_norm / chi_n - 1.0)).exp();
let h_sigma = if p_sigma_norm
/ (1.0 - (1.0 - c_sigma).powi(2 * (generation as i32 + 1))).sqrt()
< (1.4 + 2.0 / (n_f + 1.0)) * chi_n
{
1.0
} else {
0.0
};
let factor_p_c = h_sigma * (c_c * (2.0 - c_c) * mu_eff).sqrt();
for i in 0..n {
p_c[i] = (1.0 - c_c) * p_c[i] + factor_p_c * (mean[i] - old_mean[i]) / sigma;
}
let delta_h = (1.0 - h_sigma) * c_c * (2.0 - c_c);
#[allow(clippy::needless_range_loop)]
for i in 0..n {
for j in 0..n {
let mut update = (1.0 - c_1 - c_mu) * c_matrix[i][j]
+ c_1 * (p_c[i] * p_c[j] + delta_h * c_matrix[i][j]);
let mut rank_mu_term = 0.0;
for k in 0..mu {
let xk = &x_samples[order[k]];
let yi = (xk[i] - old_mean[i]) / sigma;
let yj = (xk[j] - old_mean[j]) / sigma;
rank_mu_term += weights[k] * yi * yj;
}
update += c_mu * rank_mu_term;
c_matrix[i][j] = update;
}
}
for (i, m) in mean.iter_mut().enumerate() {
let (lo, hi) = self.bounds.bounds[i];
*m = m.clamp(lo, hi);
}
}
let best = best_candidate_seen.expect("at least one generation evaluated");
let final_pop = vec![best.clone()];
let front = vec![best.clone()];
let best_opt = best_candidate(&final_pop, &objectives);
OptimizationResult::new(
Population::new(final_pop),
front,
best_opt,
evaluations,
self.config.generations,
)
}
}
fn compare_so(
a: &crate::core::evaluation::Evaluation,
b: &crate::core::evaluation::Evaluation,
+131
View File
@@ -44,6 +44,37 @@ impl Default for DifferentialEvolutionConfig {
///
/// `Vec<f64>` decisions only; single-objective problems only. Bounds come from
/// the embedded `RealBounds`, and mutant vectors are clamped to those bounds.
///
/// # Example
///
/// ```
/// use heuropt::prelude::*;
///
/// struct Sphere;
/// impl Problem for Sphere {
/// type Decision = Vec<f64>;
/// fn objectives(&self) -> ObjectiveSpace {
/// ObjectiveSpace::new(vec![Objective::minimize("f")])
/// }
/// fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
/// Evaluation::new(vec![x.iter().map(|v| v * v).sum::<f64>()])
/// }
/// }
///
/// let mut opt = DifferentialEvolution::new(
/// DifferentialEvolutionConfig {
/// population_size: 20,
/// generations: 50,
/// differential_weight: 0.5,
/// crossover_probability: 0.9,
/// seed: 42,
/// },
/// RealBounds::new(vec![(-5.0, 5.0); 5]),
/// );
/// let r = opt.run(&Sphere);
/// // DE crushes Sphere; expect very small objective.
/// assert!(r.best.unwrap().evaluation.objectives[0] < 1e-3);
/// ```
#[derive(Debug, Clone)]
pub struct DifferentialEvolution {
/// Algorithm configuration.
@@ -153,6 +184,106 @@ where
}
}
#[cfg(feature = "async")]
impl DifferentialEvolution {
/// Async version of [`Optimizer::run`] — drives evaluations through
/// the user-chosen async runtime. Available only with the `async`
/// feature.
///
/// `concurrency` bounds in-flight evaluations per batch (initial
/// population and per-generation trials).
pub async fn run_async<P>(
&mut self,
problem: &P,
concurrency: usize,
) -> OptimizationResult<Vec<f64>>
where
P: crate::core::async_problem::AsyncProblem<Decision = Vec<f64>>,
{
use rand::Rng as _;
use crate::algorithms::parallel_eval_async::evaluate_batch_async;
use crate::traits::Initializer as _;
assert!(
self.config.population_size >= 4,
"DifferentialEvolution requires population_size >= 4",
);
assert!(
(0.0..=1.0).contains(&self.config.crossover_probability),
"DifferentialEvolution crossover_probability must be in [0.0, 1.0]",
);
let objectives = problem.objectives();
assert!(
objectives.is_single_objective(),
"DifferentialEvolution only supports single-objective problems",
);
let direction = objectives.objectives[0].direction;
let dim = self.bounds.bounds.len();
let n = self.config.population_size;
let mut rng = rng_from_seed(self.config.seed);
let mut decisions: Vec<Vec<f64>> = self.bounds.initialize(n, &mut rng);
let initial_pop = evaluate_batch_async(problem, decisions.clone(), concurrency).await;
let mut evaluations = initial_pop.len();
let mut current_pop = initial_pop;
let mut evals: Vec<f64> = current_pop
.iter()
.map(|c| c.evaluation.objectives[0])
.collect();
for _generation in 0..self.config.generations {
let trials: Vec<Vec<f64>> = (0..n)
.map(|i| {
let (r1, r2, r3) = pick_three_distinct(n, i, &mut rng);
let j_rand = rng.random_range(0..dim);
let mut trial = decisions[i].clone();
for j in 0..dim {
let take_donor =
rng.random_bool(self.config.crossover_probability) || j == j_rand;
if take_donor {
let mutant = decisions[r1][j]
+ self.config.differential_weight
* (decisions[r2][j] - decisions[r3][j]);
let (lo, hi) = self.bounds.bounds[j];
trial[j] = mutant.clamp(lo, hi);
}
}
trial
})
.collect();
let trial_cands: Vec<Candidate<Vec<f64>>> =
evaluate_batch_async(problem, trials, concurrency).await;
evaluations += trial_cands.len();
for (i, trial_cand) in trial_cands.into_iter().enumerate() {
let trial_obj = trial_cand.evaluation.objectives[0];
let target_obj = evals[i];
let trial_better = match direction {
Direction::Minimize => trial_obj <= target_obj,
Direction::Maximize => trial_obj >= target_obj,
};
if trial_better {
decisions[i] = trial_cand.decision.clone();
evals[i] = trial_obj;
current_pop[i] = trial_cand;
}
}
}
let front = pareto_front(&current_pop, &objectives);
let best = best_candidate(&current_pop, &objectives);
OptimizationResult::new(
Population::new(current_pop),
front,
best,
evaluations,
self.config.generations,
)
}
}
fn pick_three_distinct(
n: usize,
exclude: usize,
+131
View File
@@ -39,6 +39,45 @@ impl Default for EpsilonMoeaConfig {
}
/// ε-dominance MOEA.
///
/// Steady-state EA with an ε-grid archive: every member that lands in
/// the same ε-box as an existing one is replaced by the closer point
/// to the box's grid corner. Auto-bounds the front size by the choice
/// of `epsilon`.
///
/// # Example
///
/// ```
/// use heuropt::prelude::*;
///
/// struct Schaffer;
/// impl Problem for Schaffer {
/// type Decision = Vec<f64>;
/// fn objectives(&self) -> ObjectiveSpace {
/// ObjectiveSpace::new(vec![Objective::minimize("f1"), Objective::minimize("f2")])
/// }
/// fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
/// Evaluation::new(vec![x[0] * x[0], (x[0] - 2.0).powi(2)])
/// }
/// }
///
/// let bounds = vec![(-5.0_f64, 5.0_f64)];
/// let mut opt = EpsilonMoea::new(
/// EpsilonMoeaConfig {
/// population_size: 20,
/// evaluations: 1_000,
/// epsilon: vec![0.1, 0.1],
/// seed: 42,
/// },
/// RealBounds::new(bounds.clone()),
/// CompositeVariation {
/// crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5),
/// mutation: PolynomialMutation::new(bounds, 20.0, 1.0),
/// },
/// );
/// let r = opt.run(&Schaffer);
/// assert!(!r.pareto_front.is_empty());
/// ```
#[derive(Debug, Clone)]
pub struct EpsilonMoea<I, V> {
/// Algorithm configuration.
@@ -151,6 +190,98 @@ where
}
}
#[cfg(feature = "async")]
impl<I, V> EpsilonMoea<I, V> {
/// Async version of [`Optimizer::run`] — drives evaluations through
/// the user-chosen async runtime. Available only with the `async`
/// feature.
///
/// `concurrency` bounds in-flight evaluations of the initial
/// population. Per-step evaluations are sequential because the
/// algorithm is steady-state (one offspring per step).
pub async fn run_async<P>(
&mut self,
problem: &P,
concurrency: usize,
) -> OptimizationResult<P::Decision>
where
P: crate::core::async_problem::AsyncProblem,
I: Initializer<P::Decision>,
V: Variation<P::Decision>,
{
use crate::algorithms::parallel_eval_async::evaluate_batch_async;
assert!(
self.config.population_size > 0,
"EpsilonMoea population_size must be > 0"
);
let n = self.config.population_size;
let objectives = problem.objectives();
assert_eq!(
self.config.epsilon.len(),
objectives.len(),
"EpsilonMoea epsilon.len() must equal number of objectives",
);
for (i, &e) in self.config.epsilon.iter().enumerate() {
assert!(e > 0.0, "EpsilonMoea epsilon[{i}] must be > 0.0");
}
let epsilon = self.config.epsilon.clone();
let mut rng = rng_from_seed(self.config.seed);
let initial_decisions = self.initializer.initialize(n, &mut rng);
let mut population: Vec<Candidate<P::Decision>> =
evaluate_batch_async(problem, initial_decisions, concurrency).await;
let mut evaluations = population.len();
let mut archive: Vec<Candidate<P::Decision>> = Vec::new();
for c in &population {
insert_into_epsilon_archive(&mut archive, c.clone(), &objectives, &epsilon);
}
let total_evals = self.config.evaluations.max(evaluations);
while evaluations < total_evals {
let p1_idx = rng.random_range(0..population.len());
let parent_a = population[p1_idx].decision.clone();
let parent_b = if !archive.is_empty() {
let j = rng.random_range(0..archive.len());
archive[j].decision.clone()
} else {
let j = rng.random_range(0..population.len());
population[j].decision.clone()
};
let parents = vec![parent_a, parent_b];
let children = self.variation.vary(&parents, &mut rng);
assert!(
!children.is_empty(),
"EpsilonMoea variation returned no children"
);
let child_decision = children.into_iter().next().unwrap();
let child_eval = problem.evaluate_async(&child_decision).await;
evaluations += 1;
let child = Candidate::new(child_decision, child_eval);
update_population(&mut population, &child, &objectives, &mut rng);
insert_into_epsilon_archive(&mut archive, child, &objectives, &epsilon);
}
let final_pop: Vec<Candidate<P::Decision>> = if !archive.is_empty() {
archive.clone()
} else {
population
};
let front = pareto_front(&final_pop, &objectives);
let best = best_candidate(&final_pop, &objectives);
OptimizationResult::new(
Population::new(final_pop),
front,
best,
evaluations,
self.config.evaluations,
)
}
}
/// Standard ε-MOEA population update: if the child is dominated by some
/// member, drop it; if it dominates a member, replace that member; if
/// non-dominated wrt all, replace a random member.
+123
View File
@@ -46,6 +46,41 @@ impl Default for GeneticAlgorithmConfig {
/// produces offspring, those are evaluated, and the next population is
/// the top `elitism` from the previous generation plus the best
/// `population_size - elitism` offspring (by fitness).
///
/// # Example
///
/// ```
/// use heuropt::prelude::*;
///
/// struct Sphere;
/// impl Problem for Sphere {
/// type Decision = Vec<f64>;
/// fn objectives(&self) -> ObjectiveSpace {
/// ObjectiveSpace::new(vec![Objective::minimize("f")])
/// }
/// fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
/// Evaluation::new(vec![x.iter().map(|v| v * v).sum::<f64>()])
/// }
/// }
///
/// let bounds = vec![(-5.0_f64, 5.0_f64); 3];
/// let mut opt = GeneticAlgorithm::new(
/// GeneticAlgorithmConfig {
/// population_size: 30,
/// generations: 50,
/// tournament_size: 2,
/// elitism: 2,
/// seed: 42,
/// },
/// RealBounds::new(bounds.clone()),
/// CompositeVariation {
/// crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5),
/// mutation: PolynomialMutation::new(bounds, 20.0, 1.0),
/// },
/// );
/// let r = opt.run(&Sphere);
/// assert!(r.best.is_some());
/// ```
#[derive(Debug, Clone)]
pub struct GeneticAlgorithm<I, V> {
/// Algorithm configuration.
@@ -146,6 +181,94 @@ where
}
}
#[cfg(feature = "async")]
impl<I, V> GeneticAlgorithm<I, V> {
/// Async version of [`Optimizer::run`] — drives evaluations through
/// the user-chosen async runtime. Available only with the `async`
/// feature.
///
/// `concurrency` bounds in-flight evaluations per batch (initial
/// population and per-generation offspring).
pub async fn run_async<P>(
&mut self,
problem: &P,
concurrency: usize,
) -> OptimizationResult<P::Decision>
where
P: crate::core::async_problem::AsyncProblem,
I: Initializer<P::Decision>,
V: Variation<P::Decision>,
{
use crate::algorithms::parallel_eval_async::evaluate_batch_async;
assert!(
self.config.population_size >= 2,
"GeneticAlgorithm population_size must be >= 2",
);
assert!(
self.config.tournament_size >= 1,
"GeneticAlgorithm tournament_size must be >= 1",
);
assert!(
self.config.elitism < self.config.population_size,
"GeneticAlgorithm elitism must be < population_size",
);
let n = self.config.population_size;
let objectives = problem.objectives();
assert!(
objectives.is_single_objective(),
"GeneticAlgorithm requires exactly one objective",
);
let direction = objectives.objectives[0].direction;
let mut rng = rng_from_seed(self.config.seed);
let initial_decisions = self.initializer.initialize(n, &mut rng);
let mut population: Vec<Candidate<P::Decision>> =
evaluate_batch_async(problem, initial_decisions, concurrency).await;
let mut evaluations = population.len();
for _ in 0..self.config.generations {
let mut offspring_decisions: Vec<P::Decision> = Vec::with_capacity(n);
while offspring_decisions.len() < n {
let parents_decisions = tournament_select_single_objective(
&population,
&objectives,
self.config.tournament_size,
2,
&mut rng,
);
let children = self.variation.vary(&parents_decisions, &mut rng);
assert!(
!children.is_empty(),
"GeneticAlgorithm variation returned no children"
);
for child in children {
if offspring_decisions.len() >= n {
break;
}
offspring_decisions.push(child);
}
}
let offspring = evaluate_batch_async(problem, offspring_decisions, concurrency).await;
evaluations += offspring.len();
population =
survival_selection(&population, offspring, direction, n, self.config.elitism);
}
let best = best_candidate(&population, &objectives);
let front: Vec<Candidate<P::Decision>> = best.iter().cloned().collect();
OptimizationResult::new(
Population::new(population),
front,
best,
evaluations,
self.config.generations,
)
}
}
fn survival_selection<D: Clone>(
parents: &[Candidate<D>],
offspring: Vec<Candidate<D>>,
+110
View File
@@ -38,6 +38,40 @@ impl Default for GreaConfig {
}
/// Grid-based Evolutionary Algorithm (GrEA).
///
/// Many-objective EA that uses three grid-based metrics — grid rank,
/// grid crowding distance, and grid coordinate point distance — to
/// select survivors. Particularly strong on linear / simplex-shaped
/// fronts (e.g. DTLZ1).
///
/// # Example
///
/// ```
/// use heuropt::prelude::*;
///
/// struct Schaffer;
/// impl Problem for Schaffer {
/// type Decision = Vec<f64>;
/// fn objectives(&self) -> ObjectiveSpace {
/// ObjectiveSpace::new(vec![Objective::minimize("f1"), Objective::minimize("f2")])
/// }
/// fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
/// Evaluation::new(vec![x[0] * x[0], (x[0] - 2.0).powi(2)])
/// }
/// }
///
/// let bounds = vec![(-5.0_f64, 5.0_f64)];
/// let mut opt = Grea::new(
/// GreaConfig { population_size: 30, generations: 20, grid_divisions: 8, seed: 42 },
/// RealBounds::new(bounds.clone()),
/// CompositeVariation {
/// crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5),
/// mutation: PolynomialMutation::new(bounds, 20.0, 1.0),
/// },
/// );
/// let r = opt.run(&Schaffer);
/// assert!(!r.pareto_front.is_empty());
/// ```
#[derive(Debug, Clone)]
pub struct Grea<I, V> {
/// Algorithm configuration.
@@ -126,6 +160,82 @@ where
}
}
#[cfg(feature = "async")]
impl<I, V> Grea<I, V> {
/// Async version of [`Optimizer::run`] — drives evaluations through
/// the user-chosen async runtime. Available only with the `async`
/// feature.
///
/// `concurrency` bounds in-flight evaluations per batch.
pub async fn run_async<P>(
&mut self,
problem: &P,
concurrency: usize,
) -> OptimizationResult<P::Decision>
where
P: crate::core::async_problem::AsyncProblem,
I: Initializer<P::Decision>,
V: Variation<P::Decision>,
{
use crate::algorithms::parallel_eval_async::evaluate_batch_async;
assert!(
self.config.population_size > 0,
"Grea population_size must be > 0"
);
assert!(
self.config.grid_divisions >= 1,
"Grea grid_divisions must be >= 1"
);
let n = self.config.population_size;
let objectives = problem.objectives();
let mut rng = rng_from_seed(self.config.seed);
let initial_decisions = self.initializer.initialize(n, &mut rng);
let mut population: Vec<Candidate<P::Decision>> =
evaluate_batch_async(problem, initial_decisions, concurrency).await;
let mut evaluations = population.len();
for _ in 0..self.config.generations {
let mut offspring_decisions: Vec<P::Decision> = Vec::with_capacity(n);
while offspring_decisions.len() < n {
let p1 = rng.random_range(0..population.len());
let p2 = rng.random_range(0..population.len());
let parents = vec![
population[p1].decision.clone(),
population[p2].decision.clone(),
];
let children = self.variation.vary(&parents, &mut rng);
assert!(!children.is_empty(), "Grea variation returned no children");
for child in children {
if offspring_decisions.len() >= n {
break;
}
offspring_decisions.push(child);
}
}
let offspring = evaluate_batch_async(problem, offspring_decisions, concurrency).await;
evaluations += offspring.len();
let mut combined: Vec<Candidate<P::Decision>> = Vec::with_capacity(2 * n);
combined.extend(population);
combined.extend(offspring);
population =
environmental_selection(combined, &objectives, n, self.config.grid_divisions);
}
let front = pareto_front(&population, &objectives);
let best = best_candidate(&population, &objectives);
OptimizationResult::new(
Population::new(population),
front,
best,
evaluations,
self.config.generations,
)
}
}
fn environmental_selection<D: Clone>(
combined: Vec<Candidate<D>>,
objectives: &ObjectiveSpace,
+103
View File
@@ -34,6 +34,31 @@ impl Default for HillClimberConfig {
/// feasible beats infeasible, smaller violation wins among infeasibles.
///
/// Single-objective only.
///
/// # Example
///
/// ```
/// use heuropt::prelude::*;
///
/// struct Sphere;
/// impl Problem for Sphere {
/// type Decision = Vec<f64>;
/// fn objectives(&self) -> ObjectiveSpace {
/// ObjectiveSpace::new(vec![Objective::minimize("f")])
/// }
/// fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
/// Evaluation::new(vec![x.iter().map(|v| v * v).sum::<f64>()])
/// }
/// }
///
/// let mut opt = HillClimber::new(
/// HillClimberConfig { iterations: 500, seed: 42 },
/// RealBounds::new(vec![(-5.0, 5.0); 3]),
/// GaussianMutation { sigma: 0.3 },
/// );
/// let r = opt.run(&Sphere);
/// assert!(r.best.is_some());
/// ```
#[derive(Debug, Clone)]
pub struct HillClimber<I, V> {
/// Algorithm configuration.
@@ -121,6 +146,84 @@ where
}
}
#[cfg(feature = "async")]
impl<I, V> HillClimber<I, V> {
/// Async version of [`Optimizer::run`] — drives evaluations through
/// the user-chosen async runtime. Available only with the `async`
/// feature.
///
/// `concurrency` is mostly inert here because HillClimber evaluates
/// one child per iteration; it's accepted for API parity with other
/// algorithms.
pub async fn run_async<P>(
&mut self,
problem: &P,
concurrency: usize,
) -> OptimizationResult<P::Decision>
where
P: crate::core::async_problem::AsyncProblem,
I: Initializer<P::Decision>,
V: Variation<P::Decision>,
{
let _ = concurrency;
let objectives = problem.objectives();
assert!(
objectives.is_single_objective(),
"HillClimber requires exactly one objective",
);
let direction = objectives.objectives[0].direction;
let mut rng = rng_from_seed(self.config.seed);
let mut initial = self.initializer.initialize(1, &mut rng);
assert!(
!initial.is_empty(),
"HillClimber initializer returned no decisions"
);
let mut current_decision = initial.remove(0);
let mut current_eval = problem.evaluate_async(&current_decision).await;
let mut evaluations = 1usize;
for _ in 0..self.config.iterations {
let parents = vec![current_decision.clone()];
let children = self.variation.vary(&parents, &mut rng);
assert!(
!children.is_empty(),
"HillClimber variation returned no children"
);
let child_decision = children.into_iter().next().unwrap();
let child_eval = problem.evaluate_async(&child_decision).await;
evaluations += 1;
let child_better = match (child_eval.is_feasible(), current_eval.is_feasible()) {
(true, false) => true,
(false, true) => false,
(false, false) => {
child_eval.constraint_violation < current_eval.constraint_violation
}
(true, true) => match direction {
Direction::Minimize => child_eval.objectives[0] < current_eval.objectives[0],
Direction::Maximize => child_eval.objectives[0] > current_eval.objectives[0],
},
};
if child_better {
current_decision = child_decision;
current_eval = child_eval;
}
}
let best = Candidate::new(current_decision, current_eval);
let population = Population::new(vec![best.clone()]);
let front = vec![best.clone()];
OptimizationResult::new(
population,
front,
Some(best),
evaluations,
self.config.iterations,
)
}
}
#[cfg(test)]
mod tests {
use super::*;
+160
View File
@@ -48,6 +48,41 @@ impl Default for HypeConfig {
/// Hypervolume Estimation Algorithm: many-objective MOEA that selects via
/// Monte Carloestimated hypervolume contributions.
///
/// # Example
///
/// ```
/// use heuropt::prelude::*;
///
/// struct Schaffer;
/// impl Problem for Schaffer {
/// type Decision = Vec<f64>;
/// fn objectives(&self) -> ObjectiveSpace {
/// ObjectiveSpace::new(vec![Objective::minimize("f1"), Objective::minimize("f2")])
/// }
/// fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
/// Evaluation::new(vec![x[0] * x[0], (x[0] - 2.0).powi(2)])
/// }
/// }
///
/// let bounds = vec![(-5.0_f64, 5.0_f64)];
/// let mut opt = Hype::new(
/// HypeConfig {
/// population_size: 20,
/// generations: 20,
/// reference_point: vec![30.0, 30.0],
/// mc_samples: 100,
/// seed: 42,
/// },
/// RealBounds::new(bounds.clone()),
/// CompositeVariation {
/// crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5),
/// mutation: PolynomialMutation::new(bounds, 20.0, 1.0),
/// },
/// );
/// let r = opt.run(&Schaffer);
/// assert!(!r.pareto_front.is_empty());
/// ```
#[derive(Debug, Clone)]
pub struct Hype<I, V> {
/// Algorithm configuration.
@@ -191,6 +226,131 @@ where
}
}
#[cfg(feature = "async")]
impl<I, V> Hype<I, V> {
/// Async version of [`Optimizer::run`] — drives evaluations through
/// the user-chosen async runtime. Available only with the `async`
/// feature.
///
/// `concurrency` bounds in-flight evaluations per batch.
pub async fn run_async<P>(
&mut self,
problem: &P,
concurrency: usize,
) -> OptimizationResult<P::Decision>
where
P: crate::core::async_problem::AsyncProblem,
I: Initializer<P::Decision>,
V: Variation<P::Decision>,
{
use crate::algorithms::parallel_eval_async::evaluate_batch_async;
assert!(
self.config.population_size > 0,
"Hype population_size must be > 0"
);
assert!(self.config.mc_samples > 0, "Hype mc_samples must be > 0");
let n = self.config.population_size;
let objectives = problem.objectives();
assert_eq!(
self.config.reference_point.len(),
objectives.len(),
"Hype reference_point.len() must equal number of objectives",
);
let reference = self.config.reference_point.clone();
let mut rng = rng_from_seed(self.config.seed);
let initial_decisions = self.initializer.initialize(n, &mut rng);
let mut population: Vec<Candidate<P::Decision>> =
evaluate_batch_async(problem, initial_decisions, concurrency).await;
let mut evaluations = population.len();
for _ in 0..self.config.generations {
let fitness = hype_fitness(
&population,
&objectives,
&reference,
self.config.mc_samples,
&mut rng,
);
let mut offspring_decisions: Vec<P::Decision> = Vec::with_capacity(n);
while offspring_decisions.len() < n {
let p1 = binary_tournament(&fitness, &mut rng);
let p2 = binary_tournament(&fitness, &mut rng);
let parents = vec![
population[p1].decision.clone(),
population[p2].decision.clone(),
];
let children = self.variation.vary(&parents, &mut rng);
assert!(!children.is_empty(), "Hype variation returned no children");
for child in children {
if offspring_decisions.len() >= n {
break;
}
offspring_decisions.push(child);
}
}
let offspring = evaluate_batch_async(problem, offspring_decisions, concurrency).await;
evaluations += offspring.len();
let mut combined: Vec<Candidate<P::Decision>> = Vec::with_capacity(2 * n);
combined.extend(population);
combined.extend(offspring);
let fronts = non_dominated_sort(&combined, &objectives);
let mut keep_indices: Vec<usize> = Vec::with_capacity(n);
let mut splitting: &[usize] = &[];
for f in &fronts {
if keep_indices.len() + f.len() <= n {
keep_indices.extend(f.iter().copied());
} else {
splitting = f;
break;
}
if keep_indices.len() == n {
break;
}
}
if keep_indices.len() < n {
let pool: Vec<&Candidate<P::Decision>> =
splitting.iter().map(|&i| &combined[i]).collect();
let contributions = estimate_contributions(
&pool,
&objectives,
&reference,
self.config.mc_samples,
&mut rng,
);
let mut order: Vec<usize> = (0..splitting.len()).collect();
order.sort_by(|&a, &b| {
contributions[b]
.partial_cmp(&contributions[a])
.unwrap_or(std::cmp::Ordering::Equal)
});
for k in order.into_iter().take(n - keep_indices.len()) {
keep_indices.push(splitting[k]);
}
}
population = keep_indices
.into_iter()
.map(|i| combined[i].clone())
.collect();
}
let front = pareto_front(&population, &objectives);
let best = best_candidate(&population, &objectives);
OptimizationResult::new(
Population::new(population),
front,
best,
evaluations,
self.config.generations,
)
}
}
fn hype_fitness<D>(
pool: &[Candidate<D>],
objectives: &ObjectiveSpace,
+132
View File
@@ -52,6 +52,38 @@ impl Default for HyperbandConfig {
/// low budget), later brackets favor exploitation (fewer configs run
/// near the max budget). The single best result across all brackets
/// is returned.
///
/// # Example
///
/// ```
/// use heuropt::prelude::*;
/// use heuropt::core::partial_problem::PartialProblem;
///
/// struct Tuning;
/// impl PartialProblem for Tuning {
/// type Decision = Vec<f64>;
/// fn objectives(&self) -> ObjectiveSpace {
/// ObjectiveSpace::new(vec![Objective::minimize("loss")])
/// }
/// fn evaluate_at_budget(&self, x: &Vec<f64>, budget: f64) -> Evaluation {
/// // Pretend a model where more budget = lower loss.
/// let loss = x[0].powi(2) + x[1].powi(2) + 1.0 / (budget + 1.0);
/// Evaluation::new(vec![loss])
/// }
/// }
///
/// let mut opt = Hyperband::new(
/// HyperbandConfig {
/// max_budget: 27.0,
/// eta: 3.0,
/// max_brackets: 4,
/// seed: 42,
/// },
/// RealBounds::new(vec![(-1.0, 1.0); 2]),
/// );
/// let r = opt.run(&Tuning);
/// assert!(r.best.is_some());
/// ```
pub struct Hyperband<I, D>
where
D: Clone,
@@ -174,6 +206,106 @@ where
}
}
#[cfg(feature = "async")]
impl<I, D> Hyperband<I, D>
where
D: Clone,
I: Initializer<D>,
{
/// Async version of [`Hyperband::run`] — evaluates each
/// Successive-Halving rung's configurations concurrently through the
/// caller's async runtime. Available only with the `async` feature.
///
/// `concurrency` bounds in-flight evaluations per rung.
pub async fn run_async<P>(&mut self, problem: &P, concurrency: usize) -> OptimizationResult<D>
where
P: crate::core::async_problem::AsyncPartialProblem<Decision = D>,
D: Send + Sync,
{
use crate::algorithms::parallel_eval_async::evaluate_batch_at_budget_async;
assert!(
self.config.max_budget > 0.0,
"Hyperband max_budget must be > 0"
);
assert!(self.config.eta > 1.0, "Hyperband eta must be > 1");
assert!(
self.config.max_brackets >= 1,
"Hyperband max_brackets must be >= 1"
);
let objectives = problem.objectives();
assert!(
objectives.is_single_objective(),
"Hyperband requires exactly one objective",
);
let direction = objectives.objectives[0].direction;
let mut rng = rng_from_seed(self.config.seed);
let s_max = (self.config.max_budget.ln() / self.config.eta.ln()).floor() as i64;
let s_max = (s_max as usize).min(self.config.max_brackets);
let mut total_evaluations = 0usize;
let mut total_iterations = 0usize;
let mut best_seen: Option<Candidate<D>> = None;
for s in (0..=s_max).rev() {
let s_f = s as f64;
let n =
((s_max as f64 + 1.0) / (s_f + 1.0) * self.config.eta.powf(s_f)).ceil() as usize;
let r = self.config.max_budget / self.config.eta.powf(s_f);
let mut configs: Vec<D> = self.initializer.initialize(n, &mut rng);
for i in 0..=s {
let n_i = (n as f64 / self.config.eta.powi(i as i32)).floor() as usize;
let r_i = r * self.config.eta.powi(i as i32);
if configs.is_empty() {
break;
}
let evals: Vec<Evaluation> =
evaluate_batch_at_budget_async(problem, &configs, r_i, concurrency).await;
total_evaluations += configs.len();
for (cfg, e) in configs.iter().zip(evals.iter()) {
let beats = match &best_seen {
None => true,
Some(b) => better(e, &b.evaluation, direction),
};
if beats {
best_seen = Some(Candidate::new(cfg.clone(), e.clone()));
}
}
total_iterations += 1;
let next_size = (n_i / self.config.eta as usize).max(1);
if next_size >= configs.len() {
continue;
}
let mut order: Vec<usize> = (0..configs.len()).collect();
order.sort_by(|&a, &b| compare(&evals[a], &evals[b], direction));
let keep: std::collections::HashSet<usize> =
order.into_iter().take(next_size).collect();
let new_configs: Vec<D> = configs
.into_iter()
.enumerate()
.filter_map(|(idx, c)| if keep.contains(&idx) { Some(c) } else { None })
.collect();
configs = new_configs;
}
}
let best = best_seen.expect("at least one bracket ran");
let population = Population::new(vec![best.clone()]);
let front = vec![best.clone()];
OptimizationResult::new(
population,
front,
Some(best),
total_evaluations,
total_iterations,
)
}
}
fn compare(a: &Evaluation, b: &Evaluation, direction: Direction) -> std::cmp::Ordering {
match (a.is_feasible(), b.is_feasible()) {
(true, false) => std::cmp::Ordering::Less,
+108
View File
@@ -37,6 +37,40 @@ impl Default for IbeaConfig {
}
/// IBEA (Indicator-Based EA) using the additive ε-indicator.
///
/// Selects survivors by their contribution to a quality indicator
/// (additive ε) rather than by dominance + crowding. On the comparison
/// harness it consistently produces the best convergence of the dominance-
/// alternative methods on smooth and disconnected fronts alike.
///
/// # Example
///
/// ```
/// use heuropt::prelude::*;
///
/// struct Schaffer;
/// impl Problem for Schaffer {
/// type Decision = Vec<f64>;
/// fn objectives(&self) -> ObjectiveSpace {
/// ObjectiveSpace::new(vec![Objective::minimize("f1"), Objective::minimize("f2")])
/// }
/// fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
/// Evaluation::new(vec![x[0] * x[0], (x[0] - 2.0).powi(2)])
/// }
/// }
///
/// let bounds = vec![(-5.0_f64, 5.0_f64)];
/// let mut opt = Ibea::new(
/// IbeaConfig { population_size: 30, generations: 20, kappa: 0.05, seed: 42 },
/// RealBounds::new(bounds.clone()),
/// CompositeVariation {
/// crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5),
/// mutation: PolynomialMutation::new(bounds, 20.0, 1.0),
/// },
/// );
/// let r = opt.run(&Schaffer);
/// assert!(!r.pareto_front.is_empty());
/// ```
#[derive(Debug, Clone)]
pub struct Ibea<I, V> {
/// Algorithm configuration.
@@ -125,6 +159,80 @@ where
}
}
#[cfg(feature = "async")]
impl<I, V> Ibea<I, V> {
/// Async version of [`Optimizer::run`] — drives evaluations through
/// the user-chosen async runtime. Available only with the `async`
/// feature.
///
/// `concurrency` bounds in-flight evaluations per batch.
pub async fn run_async<P>(
&mut self,
problem: &P,
concurrency: usize,
) -> OptimizationResult<P::Decision>
where
P: crate::core::async_problem::AsyncProblem,
I: Initializer<P::Decision>,
V: Variation<P::Decision>,
{
use crate::algorithms::parallel_eval_async::evaluate_batch_async;
assert!(
self.config.population_size > 0,
"Ibea population_size must be > 0"
);
assert!(self.config.kappa > 0.0, "Ibea kappa must be > 0");
let n = self.config.population_size;
let objectives = problem.objectives();
let mut rng = rng_from_seed(self.config.seed);
let initial_decisions = self.initializer.initialize(n, &mut rng);
let mut population: Vec<Candidate<P::Decision>> =
evaluate_batch_async(problem, initial_decisions, concurrency).await;
let mut evaluations = population.len();
for _ in 0..self.config.generations {
let fitness = compute_fitness(&population, &objectives, self.config.kappa);
let mut offspring_decisions: Vec<P::Decision> = Vec::with_capacity(n);
while offspring_decisions.len() < n {
let p1 = binary_tournament(&fitness, &mut rng);
let p2 = binary_tournament(&fitness, &mut rng);
let parents = vec![
population[p1].decision.clone(),
population[p2].decision.clone(),
];
let children = self.variation.vary(&parents, &mut rng);
assert!(!children.is_empty(), "Ibea variation returned no children");
for child in children {
if offspring_decisions.len() >= n {
break;
}
offspring_decisions.push(child);
}
}
let offspring = evaluate_batch_async(problem, offspring_decisions, concurrency).await;
evaluations += offspring.len();
let mut combined: Vec<Candidate<P::Decision>> = Vec::with_capacity(2 * n);
combined.extend(population);
combined.extend(offspring);
population = environmental_selection(combined, &objectives, n, self.config.kappa);
}
let front = pareto_front(&population, &objectives);
let best = best_candidate(&population, &objectives);
OptimizationResult::new(
Population::new(population),
front,
best,
evaluations,
self.config.generations,
)
}
}
/// Iteratively remove the worst-fitness member from `pool` until `n` remain.
///
/// IBEA's standard "subtract the dropped member's contribution from every
+124
View File
@@ -53,6 +53,42 @@ impl Default for IpopCmaEsConfig {
}
/// IPOP-CMA-ES: CMA-ES with population-doubling restarts.
///
/// Specifically designed to fix vanilla CMA-ES's weakness on multimodal
/// landscapes — each restart doubles the population and randomizes the
/// initial mean to escape from local basins. On the comparison harness
/// it drops vanilla CMA-ES's Rastrigin score from f = 2.35 to f = 0.13.
///
/// # Example
///
/// ```
/// use heuropt::prelude::*;
///
/// struct Sphere;
/// impl Problem for Sphere {
/// type Decision = Vec<f64>;
/// fn objectives(&self) -> ObjectiveSpace {
/// ObjectiveSpace::new(vec![Objective::minimize("f")])
/// }
/// fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
/// Evaluation::new(vec![x.iter().map(|v| v * v).sum::<f64>()])
/// }
/// }
///
/// let mut opt = IpopCmaEs::new(
/// IpopCmaEsConfig {
/// initial_population_size: 8,
/// total_generations: 100,
/// initial_sigma: 1.0,
/// eigen_decomposition_period: 1,
/// stall_generations: Some(20),
/// seed: 42,
/// },
/// RealBounds::new(vec![(-5.0, 5.0); 3]),
/// );
/// let r = opt.run(&Sphere);
/// assert!(r.best.unwrap().evaluation.objectives[0] < 1.0);
/// ```
#[derive(Debug, Clone)]
pub struct IpopCmaEs {
/// Algorithm configuration.
@@ -151,6 +187,94 @@ where
}
}
#[cfg(feature = "async")]
impl IpopCmaEs {
/// Async version of [`Optimizer::run`] — drives evaluations through
/// the user-chosen async runtime. Available only with the `async`
/// feature.
///
/// `concurrency` bounds in-flight evaluations within each restart's
/// CMA-ES generation.
pub async fn run_async<P>(
&mut self,
problem: &P,
concurrency: usize,
) -> OptimizationResult<Vec<f64>>
where
P: crate::core::async_problem::AsyncProblem<Decision = Vec<f64>>,
{
assert!(
self.config.initial_population_size >= 4,
"IpopCmaEs initial_population_size must be >= 4",
);
let objectives = problem.objectives();
assert!(
objectives.is_single_objective(),
"IpopCmaEs requires exactly one objective",
);
let direction = objectives.objectives[0].direction;
let mut rng = rng_from_seed(self.config.seed);
let mut remaining_gens = self.config.total_generations;
let mut pop_size = self.config.initial_population_size;
let mut total_evaluations = 0usize;
let mut total_iterations = 0usize;
let mut best_seen: Option<Candidate<Vec<f64>>> = None;
let _ = self.config.stall_generations;
let mut restart_counter = 0u64;
while remaining_gens > 0 {
let this_gens = (remaining_gens / 2).max(20).min(remaining_gens);
let inner_seed = self
.config
.seed
.wrapping_add(restart_counter.wrapping_mul(0x9E37_79B9_7F4A_7C15));
let restart_mean: Vec<f64> = self
.bounds
.bounds
.iter()
.map(|&(lo, hi)| lo + (hi - lo) * rng.random::<f64>())
.collect();
let cfg = CmaEsConfig {
population_size: pop_size,
generations: this_gens,
initial_sigma: self.config.initial_sigma,
eigen_decomposition_period: self.config.eigen_decomposition_period,
initial_mean: Some(restart_mean),
seed: inner_seed,
};
let mut inner = CmaEs::new(cfg, RealBounds::new(self.bounds.bounds.clone()));
let result = inner.run_async(problem, concurrency).await;
total_evaluations += result.evaluations;
total_iterations += result.generations;
if let Some(b) = result.best.clone() {
let beats = match &best_seen {
None => true,
Some(prev) => better(&b.evaluation, &prev.evaluation, direction),
};
if beats {
best_seen = Some(b);
}
}
remaining_gens = remaining_gens.saturating_sub(this_gens);
pop_size = pop_size.saturating_mul(2);
restart_counter = restart_counter.wrapping_add(1);
}
let best = best_seen.expect("at least one restart ran");
let population = Population::new(vec![best.clone()]);
let front = vec![best.clone()];
OptimizationResult::new(
population,
front,
Some(best),
total_evaluations,
total_iterations,
)
}
}
fn better(a: &Evaluation, b: &Evaluation, direction: Direction) -> bool {
match (a.is_feasible(), b.is_feasible()) {
(true, false) => true,
+100
View File
@@ -39,6 +39,35 @@ impl Default for KneaConfig {
/// Survival selection ranks splitting-front members by perpendicular
/// distance from the hyperplane connecting the front's extreme points.
/// Larger distance ≈ stronger knee = preferred survivor.
///
/// # Example
///
/// ```
/// use heuropt::prelude::*;
///
/// struct Schaffer;
/// impl Problem for Schaffer {
/// type Decision = Vec<f64>;
/// fn objectives(&self) -> ObjectiveSpace {
/// ObjectiveSpace::new(vec![Objective::minimize("f1"), Objective::minimize("f2")])
/// }
/// fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
/// Evaluation::new(vec![x[0] * x[0], (x[0] - 2.0).powi(2)])
/// }
/// }
///
/// let bounds = vec![(-5.0_f64, 5.0_f64)];
/// let mut opt = Knea::new(
/// KneaConfig { population_size: 30, generations: 20, seed: 42 },
/// RealBounds::new(bounds.clone()),
/// CompositeVariation {
/// crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5),
/// mutation: PolynomialMutation::new(bounds, 20.0, 1.0),
/// },
/// );
/// let r = opt.run(&Schaffer);
/// assert!(!r.pareto_front.is_empty());
/// ```
#[derive(Debug, Clone)]
pub struct Knea<I, V> {
/// Algorithm configuration.
@@ -120,6 +149,77 @@ where
}
}
#[cfg(feature = "async")]
impl<I, V> Knea<I, V> {
/// Async version of [`Optimizer::run`] — drives evaluations through
/// the user-chosen async runtime. Available only with the `async`
/// feature.
///
/// `concurrency` bounds in-flight evaluations per batch.
pub async fn run_async<P>(
&mut self,
problem: &P,
concurrency: usize,
) -> OptimizationResult<P::Decision>
where
P: crate::core::async_problem::AsyncProblem,
I: Initializer<P::Decision>,
V: Variation<P::Decision>,
{
use crate::algorithms::parallel_eval_async::evaluate_batch_async;
assert!(
self.config.population_size > 0,
"Knea population_size must be > 0"
);
let n = self.config.population_size;
let objectives = problem.objectives();
let mut rng = rng_from_seed(self.config.seed);
let initial_decisions = self.initializer.initialize(n, &mut rng);
let mut population: Vec<Candidate<P::Decision>> =
evaluate_batch_async(problem, initial_decisions, concurrency).await;
let mut evaluations = population.len();
for _ in 0..self.config.generations {
let mut offspring_decisions: Vec<P::Decision> = Vec::with_capacity(n);
while offspring_decisions.len() < n {
let p1 = rng.random_range(0..population.len());
let p2 = rng.random_range(0..population.len());
let parents = vec![
population[p1].decision.clone(),
population[p2].decision.clone(),
];
let children = self.variation.vary(&parents, &mut rng);
assert!(!children.is_empty(), "Knea variation returned no children");
for child in children {
if offspring_decisions.len() >= n {
break;
}
offspring_decisions.push(child);
}
}
let offspring = evaluate_batch_async(problem, offspring_decisions, concurrency).await;
evaluations += offspring.len();
let mut combined: Vec<Candidate<P::Decision>> = Vec::with_capacity(2 * n);
combined.extend(population);
combined.extend(offspring);
population = environmental_selection(combined, &objectives, n);
}
let front = pareto_front(&population, &objectives);
let best = best_candidate(&population, &objectives);
OptimizationResult::new(
Population::new(population),
front,
best,
evaluations,
self.config.generations,
)
}
}
fn environmental_selection<D: Clone>(
combined: Vec<Candidate<D>>,
objectives: &ObjectiveSpace,
+2
View File
@@ -22,6 +22,8 @@ pub mod nsga3;
pub mod one_plus_one_es;
pub mod paes;
pub(crate) mod parallel_eval;
#[cfg(feature = "async")]
pub(crate) mod parallel_eval_async;
pub mod particle_swarm;
pub mod pesa2;
pub mod random_search;
+159
View File
@@ -39,6 +39,45 @@ impl Default for MoeadConfig {
}
/// MOEA/D optimizer using the Tchebycheff scalarizing function.
///
/// Decomposes the multi-objective problem into many single-objective
/// scalarizations along DasDennis weight vectors and solves them
/// in parallel with neighborhood-based mating. Very fast per generation;
/// scales naturally to many objectives.
///
/// # Example
///
/// ```
/// use heuropt::prelude::*;
///
/// struct Schaffer;
/// impl Problem for Schaffer {
/// type Decision = Vec<f64>;
/// fn objectives(&self) -> ObjectiveSpace {
/// ObjectiveSpace::new(vec![Objective::minimize("f1"), Objective::minimize("f2")])
/// }
/// fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
/// Evaluation::new(vec![x[0] * x[0], (x[0] - 2.0).powi(2)])
/// }
/// }
///
/// let bounds = vec![(-5.0_f64, 5.0_f64)];
/// let mut opt = Moead::new(
/// MoeadConfig {
/// generations: 30,
/// reference_divisions: 19, // 20 weights for 2 objectives
/// neighborhood_size: 5,
/// seed: 42,
/// },
/// RealBounds::new(bounds.clone()),
/// CompositeVariation {
/// crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5),
/// mutation: PolynomialMutation::new(bounds, 20.0, 1.0),
/// },
/// );
/// let r = opt.run(&Schaffer);
/// assert!(!r.pareto_front.is_empty());
/// ```
#[derive(Debug, Clone)]
pub struct Moead<I, V> {
/// Algorithm configuration.
@@ -180,6 +219,126 @@ where
}
}
#[cfg(feature = "async")]
impl<I, V> Moead<I, V> {
/// Async version of [`Optimizer::run`] — drives evaluations through
/// the user-chosen async runtime. Available only with the `async`
/// feature.
///
/// `concurrency` bounds in-flight evaluations of the initial
/// population. Per-generation evaluations are sequential because
/// each child's outcome feeds back into the same generation's
/// neighborhood updates.
pub async fn run_async<P>(
&mut self,
problem: &P,
concurrency: usize,
) -> OptimizationResult<P::Decision>
where
P: crate::core::async_problem::AsyncProblem,
I: Initializer<P::Decision>,
V: Variation<P::Decision>,
{
use crate::algorithms::parallel_eval_async::evaluate_batch_async;
let objectives = problem.objectives();
let m = objectives.len();
let weights = das_dennis(m, self.config.reference_divisions);
assert!(
!weights.is_empty(),
"Moead weight set is empty — increase reference_divisions",
);
let n = weights.len();
let t = self.config.neighborhood_size.min(n);
assert!(t >= 2, "Moead neighborhood_size must be >= 2");
let mut rng = rng_from_seed(self.config.seed);
let initial_decisions = self.initializer.initialize(n, &mut rng);
assert_eq!(
initial_decisions.len(),
n,
"MOEA/D initializer must return exactly {n} decisions",
);
let mut population: Vec<Candidate<P::Decision>> =
evaluate_batch_async(problem, initial_decisions, concurrency).await;
let mut evaluations = population.len();
let mut ideal = vec![f64::INFINITY; m];
for c in &population {
let oriented = objectives.as_minimization(&c.evaluation.objectives);
for (k, v) in oriented.iter().enumerate() {
if *v < ideal[k] {
ideal[k] = *v;
}
}
}
let neighborhoods: Vec<Vec<usize>> = (0..n)
.map(|i| {
let mut idx: Vec<usize> = (0..n).collect();
idx.sort_by(|&a, &b| {
let da = weight_distance(&weights[i], &weights[a]);
let db = weight_distance(&weights[i], &weights[b]);
da.partial_cmp(&db).unwrap_or(std::cmp::Ordering::Equal)
});
idx.into_iter().take(t).collect()
})
.collect();
for _ in 0..self.config.generations {
#[allow(clippy::needless_range_loop)]
for i in 0..n {
let nbh = &neighborhoods[i];
let p1 = *nbh.choose(&mut rng).unwrap();
let mut p2 = *nbh.choose(&mut rng).unwrap();
while p2 == p1 && nbh.len() > 1 {
p2 = *nbh.choose(&mut rng).unwrap();
}
let parents = vec![
population[p1].decision.clone(),
population[p2].decision.clone(),
];
let children = self.variation.vary(&parents, &mut rng);
assert!(
!children.is_empty(),
"MOEA/D variation returned no children"
);
let child_decision = children.into_iter().next().unwrap();
let child_eval = problem.evaluate_async(&child_decision).await;
evaluations += 1;
let oriented_child = objectives.as_minimization(&child_eval.objectives);
for (k, v) in oriented_child.iter().enumerate() {
if *v < ideal[k] {
ideal[k] = *v;
}
}
for &j in nbh {
let cur_oriented =
objectives.as_minimization(&population[j].evaluation.objectives);
let g_cur = tchebycheff(&cur_oriented, &weights[j], &ideal);
let g_new = tchebycheff(&oriented_child, &weights[j], &ideal);
if g_new <= g_cur {
population[j] = Candidate::new(child_decision.clone(), child_eval.clone());
}
}
}
}
let front = pareto_front(&population, &objectives);
let best = best_candidate(&population, &objectives);
OptimizationResult::new(
Population::new(population),
front,
best,
evaluations,
self.config.generations,
)
}
}
/// Tchebycheff scalarization: `max_k w_k * |f_k - z*_k|`.
///
/// `weight` components that are zero are floored to `1e-6` so every axis
+150
View File
@@ -52,6 +52,38 @@ impl Default for MopsoConfig {
/// `Vec<f64>` decisions only. Each particle maintains a personal best (the
/// last position that was Pareto-non-dominated by any later position). The
/// social leader is sampled uniformly from the external archive each step.
///
/// # Example
///
/// ```
/// use heuropt::prelude::*;
///
/// struct Schaffer;
/// impl Problem for Schaffer {
/// type Decision = Vec<f64>;
/// fn objectives(&self) -> ObjectiveSpace {
/// ObjectiveSpace::new(vec![Objective::minimize("f1"), Objective::minimize("f2")])
/// }
/// fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
/// Evaluation::new(vec![x[0] * x[0], (x[0] - 2.0).powi(2)])
/// }
/// }
///
/// let mut opt = Mopso::new(
/// MopsoConfig {
/// swarm_size: 30,
/// generations: 50,
/// archive_size: 30,
/// inertia: 0.4,
/// cognitive: 1.5,
/// social: 1.5,
/// seed: 42,
/// },
/// RealBounds::new(vec![(-5.0, 5.0)]),
/// );
/// let r = opt.run(&Schaffer);
/// assert!(!r.pareto_front.is_empty());
/// ```
#[derive(Debug, Clone)]
pub struct Mopso {
/// Algorithm configuration.
@@ -180,6 +212,124 @@ where
}
}
#[cfg(feature = "async")]
impl Mopso {
/// Async version of [`Optimizer::run`] — drives evaluations through
/// the user-chosen async runtime. Available only with the `async`
/// feature.
///
/// `concurrency` bounds in-flight evaluations per batch.
pub async fn run_async<P>(
&mut self,
problem: &P,
concurrency: usize,
) -> OptimizationResult<Vec<f64>>
where
P: crate::core::async_problem::AsyncProblem<Decision = Vec<f64>>,
{
use crate::algorithms::parallel_eval_async::evaluate_batch_async;
use crate::traits::Initializer as _;
assert!(self.config.swarm_size >= 1, "Mopso swarm_size must be >= 1");
assert!(
self.config.archive_size >= 1,
"Mopso archive_size must be >= 1"
);
let objectives = problem.objectives();
assert!(
objectives.is_multi_objective(),
"Mopso requires multi-objective problems (use ParticleSwarm for single-objective)",
);
let dim = self.bounds.bounds.len();
let n = self.config.swarm_size;
let mut rng = rng_from_seed(self.config.seed);
let mut positions: Vec<Vec<f64>> = self.bounds.initialize(n, &mut rng);
let mut velocities: Vec<Vec<f64>> = (0..n)
.map(|_| {
self.bounds
.bounds
.iter()
.map(|&(lo, hi)| 0.1 * (hi - lo) * (rng.random::<f64>() * 2.0 - 1.0))
.collect()
})
.collect();
let v_max: Vec<f64> = self.bounds.bounds.iter().map(|&(lo, hi)| hi - lo).collect();
let initial_pop = evaluate_batch_async(problem, positions.clone(), concurrency).await;
let mut evaluations = initial_pop.len();
let mut pbest_decisions: Vec<Vec<f64>> = positions.clone();
let mut pbest_evals: Vec<crate::core::evaluation::Evaluation> =
initial_pop.iter().map(|c| c.evaluation.clone()).collect();
let mut archive = ParetoArchive::new(objectives.clone());
for c in initial_pop {
archive.insert(c);
}
archive.truncate(self.config.archive_size);
for _ in 0..self.config.generations {
for i in 0..n {
let leader = archive
.members()
.choose(&mut rng)
.map(|c| c.decision.clone())
.unwrap_or_else(|| positions[i].clone());
#[allow(clippy::needless_range_loop)]
for j in 0..dim {
let r1: f64 = rng.random();
let r2: f64 = rng.random();
let cognitive_term =
self.config.cognitive * r1 * (pbest_decisions[i][j] - positions[i][j]);
let social_term = self.config.social * r2 * (leader[j] - positions[i][j]);
let mut v =
self.config.inertia * velocities[i][j] + cognitive_term + social_term;
if v > v_max[j] {
v = v_max[j];
} else if v < -v_max[j] {
v = -v_max[j];
}
velocities[i][j] = v;
let (lo, hi) = self.bounds.bounds[j];
positions[i][j] = (positions[i][j] + v).clamp(lo, hi);
}
}
let evaluated = evaluate_batch_async(problem, positions.clone(), concurrency).await;
evaluations += evaluated.len();
for (i, cand) in evaluated.iter().enumerate() {
let dominance = pareto_compare(&cand.evaluation, &pbest_evals[i], &objectives);
let replace = match dominance {
Dominance::Dominates => true,
Dominance::DominatedBy => false,
Dominance::Equal | Dominance::NonDominated => rng.random_bool(0.5),
};
if replace {
pbest_decisions[i] = cand.decision.clone();
pbest_evals[i] = cand.evaluation.clone();
}
}
for c in evaluated {
archive.insert(c);
}
archive.truncate(self.config.archive_size);
}
let members = archive.into_vec();
let front = pareto_front(&members, &objectives);
let best = best_candidate(&members, &objectives);
OptimizationResult::new(
Population::new(members),
front,
best,
evaluations,
self.config.generations,
)
}
}
#[cfg(test)]
mod tests {
use super::*;
+187
View File
@@ -48,6 +48,38 @@ impl Default for NelderMeadConfig {
/// `Vec<f64>` decisions only. Single-objective only. Initial simplex is
/// built around the midpoint of the configured bounds; every new vertex
/// is clamped to those bounds.
///
/// # Example
///
/// ```
/// use heuropt::prelude::*;
///
/// struct Sphere;
/// impl Problem for Sphere {
/// type Decision = Vec<f64>;
/// fn objectives(&self) -> ObjectiveSpace {
/// ObjectiveSpace::new(vec![Objective::minimize("f")])
/// }
/// fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
/// Evaluation::new(vec![x.iter().map(|v| v * v).sum::<f64>()])
/// }
/// }
///
/// let mut opt = NelderMead::new(
/// NelderMeadConfig {
/// iterations: 200,
/// reflection: 1.0,
/// expansion: 2.0,
/// contraction: 0.5,
/// shrinkage: 0.5,
/// initial_step: 1.0,
/// },
/// RealBounds::new(vec![(-5.0, 5.0); 3]),
/// );
/// let r = opt.run(&Sphere);
/// // Nelder-Mead reaches machine precision on Sphere.
/// assert!(r.best.unwrap().evaluation.objectives[0] < 1e-10);
/// ```
#[derive(Debug, Clone)]
pub struct NelderMead {
/// Algorithm configuration.
@@ -263,6 +295,161 @@ fn better(a: &Evaluation, b: &Evaluation, direction: Direction) -> bool {
compare(a, b, direction) == std::cmp::Ordering::Less
}
#[cfg(feature = "async")]
impl NelderMead {
/// Async version of [`Optimizer::run`] — drives evaluations through
/// the user-chosen async runtime. Available only with the `async`
/// feature.
///
/// `concurrency` is largely inert here because Nelder-Mead
/// evaluates one or two new vertices per iteration sequentially
/// (the next decision depends on the previous evaluation); it's
/// accepted for API parity with other algorithms.
pub async fn run_async<P>(
&mut self,
problem: &P,
concurrency: usize,
) -> OptimizationResult<Vec<f64>>
where
P: crate::core::async_problem::AsyncProblem<Decision = Vec<f64>>,
{
let _ = concurrency;
assert!(
self.config.reflection > 0.0,
"NelderMead reflection must be > 0"
);
assert!(
self.config.expansion > 1.0,
"NelderMead expansion must be > 1",
);
assert!(
self.config.contraction > 0.0 && self.config.contraction < 1.0,
"NelderMead contraction must be in (0, 1)",
);
assert!(
self.config.shrinkage > 0.0 && self.config.shrinkage < 1.0,
"NelderMead shrinkage must be in (0, 1)",
);
assert!(
self.config.initial_step > 0.0,
"NelderMead initial_step must be > 0",
);
let objectives = problem.objectives();
assert!(
objectives.is_single_objective(),
"NelderMead requires exactly one objective",
);
let direction = objectives.objectives[0].direction;
let n = self.bounds.bounds.len();
let mut vertices: Vec<Vec<f64>> = Vec::with_capacity(n + 1);
let start: Vec<f64> = self
.bounds
.bounds
.iter()
.map(|&(lo, hi)| 0.5 * (lo + hi))
.collect();
vertices.push(start.clone());
for j in 0..n {
let mut v = start.clone();
let (lo, hi) = self.bounds.bounds[j];
let step = self.config.initial_step.min(0.5 * (hi - lo));
v[j] = (v[j] + step).clamp(lo, hi);
vertices.push(v);
}
let mut evals: Vec<Evaluation> = Vec::with_capacity(vertices.len());
for v in &vertices {
evals.push(problem.evaluate_async(v).await);
}
let mut evaluations = evals.len();
for _ in 0..self.config.iterations {
let mut order: Vec<usize> = (0..vertices.len()).collect();
order.sort_by(|&a, &b| compare(&evals[a], &evals[b], direction));
let best_idx = order[0];
let worst_idx = order[order.len() - 1];
let second_worst_idx = order[order.len() - 2];
let mut centroid = vec![0.0_f64; n];
for &idx in &order[..order.len() - 1] {
for j in 0..n {
centroid[j] += vertices[idx][j];
}
}
for c in centroid.iter_mut() {
*c /= (order.len() - 1) as f64;
}
let reflected = self.reflect(&centroid, &vertices[worst_idx], self.config.reflection);
let r_eval = problem.evaluate_async(&reflected).await;
evaluations += 1;
if better(&r_eval, &evals[best_idx], direction) {
let expanded = self.reflect(&centroid, &vertices[worst_idx], self.config.expansion);
let e_eval = problem.evaluate_async(&expanded).await;
evaluations += 1;
if better(&e_eval, &r_eval, direction) {
vertices[worst_idx] = expanded;
evals[worst_idx] = e_eval;
} else {
vertices[worst_idx] = reflected;
evals[worst_idx] = r_eval;
}
} else if better(&r_eval, &evals[second_worst_idx], direction) {
vertices[worst_idx] = reflected;
evals[worst_idx] = r_eval;
} else {
let contraction_target = if better(&r_eval, &evals[worst_idx], direction) {
self.contract(&centroid, &reflected, self.config.contraction)
} else {
self.contract(&centroid, &vertices[worst_idx], self.config.contraction)
};
let c_eval = problem.evaluate_async(&contraction_target).await;
evaluations += 1;
if better(&c_eval, &evals[worst_idx], direction) {
vertices[worst_idx] = contraction_target;
evals[worst_idx] = c_eval;
} else {
let best_pt = vertices[best_idx].clone();
for &idx in &order {
if idx == best_idx {
continue;
}
#[allow(clippy::needless_range_loop)]
for j in 0..n {
vertices[idx][j] = best_pt[j]
+ self.config.shrinkage * (vertices[idx][j] - best_pt[j]);
}
for (j, x) in vertices[idx].iter_mut().enumerate() {
let (lo, hi) = self.bounds.bounds[j];
*x = x.clamp(lo, hi);
}
evals[idx] = problem.evaluate_async(&vertices[idx]).await;
evaluations += 1;
}
}
}
}
let mut best_idx = 0;
for i in 1..vertices.len() {
if better(&evals[i], &evals[best_idx], direction) {
best_idx = i;
}
}
let best = Candidate::new(vertices[best_idx].clone(), evals[best_idx].clone());
let population = Population::new(vec![best.clone()]);
let front = vec![best.clone()];
OptimizationResult::new(
population,
front,
Some(best),
evaluations,
self.config.iterations,
)
}
}
#[cfg(test)]
mod tests {
use super::*;
+148
View File
@@ -35,6 +35,43 @@ impl Default for Nsga2Config {
}
/// NSGA-II optimizer (spec §12.3).
///
/// The canonical Pareto-based EA: combines non-dominated sorting with
/// crowding-distance secondary ranking. A strong default for 2- or
/// 3-objective problems.
///
/// # Example
///
/// ```
/// use heuropt::prelude::*;
///
/// struct Schaffer;
/// impl Problem for Schaffer {
/// type Decision = Vec<f64>;
/// fn objectives(&self) -> ObjectiveSpace {
/// ObjectiveSpace::new(vec![
/// Objective::minimize("f1"),
/// Objective::minimize("f2"),
/// ])
/// }
/// fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
/// Evaluation::new(vec![x[0] * x[0], (x[0] - 2.0).powi(2)])
/// }
/// }
///
/// let bounds = vec![(-5.0_f64, 5.0_f64)];
/// let mut opt = Nsga2::new(
/// Nsga2Config { population_size: 30, generations: 20, seed: 42 },
/// RealBounds::new(bounds.clone()),
/// CompositeVariation {
/// crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5),
/// mutation: PolynomialMutation::new(bounds, 20.0, 1.0),
/// },
/// );
/// let r = opt.run(&Schaffer);
/// assert_eq!(r.population.len(), 30);
/// assert!(!r.pareto_front.is_empty());
/// ```
#[derive(Debug, Clone)]
pub struct Nsga2<I, V> {
/// Algorithm configuration.
@@ -195,6 +232,117 @@ fn annotate<D: Clone>(
.collect()
}
#[cfg(feature = "async")]
impl<I, V> Nsga2<I, V> {
/// Async version of [`Optimizer::run`] — drives evaluations through
/// the user-chosen async runtime. Available only with the `async`
/// feature.
///
/// `concurrency` bounds in-flight evaluations per batch (initial
/// population and per-generation offspring).
pub async fn run_async<P>(
&mut self,
problem: &P,
concurrency: usize,
) -> OptimizationResult<P::Decision>
where
P: crate::core::async_problem::AsyncProblem,
I: Initializer<P::Decision>,
V: Variation<P::Decision>,
{
use crate::algorithms::parallel_eval_async::evaluate_batch_async;
assert!(
self.config.population_size > 0,
"Nsga2 population_size must be greater than 0",
);
let n = self.config.population_size;
let objectives = problem.objectives();
let mut rng = rng_from_seed(self.config.seed);
let initial_decisions = self.initializer.initialize(n, &mut rng);
assert_eq!(
initial_decisions.len(),
n,
"NSGA-II initializer must return exactly population_size decisions",
);
let population: Vec<Candidate<P::Decision>> =
evaluate_batch_async(problem, initial_decisions, concurrency).await;
let mut evaluations = population.len();
let mut annotated = annotate(population, &objectives);
for _ in 0..self.config.generations {
let mut offspring_decisions: Vec<P::Decision> = Vec::with_capacity(n);
while offspring_decisions.len() < n {
let p1 = binary_tournament(&annotated, &mut rng);
let p2 = binary_tournament(&annotated, &mut rng);
let parents = vec![
annotated[p1].candidate.decision.clone(),
annotated[p2].candidate.decision.clone(),
];
let children = self.variation.vary(&parents, &mut rng);
assert!(
!children.is_empty(),
"NSGA-II variation returned no children",
);
for child_decision in children {
if offspring_decisions.len() >= n {
break;
}
offspring_decisions.push(child_decision);
}
}
let offspring: Vec<Candidate<P::Decision>> =
evaluate_batch_async(problem, offspring_decisions, concurrency).await;
evaluations += offspring.len();
let mut combined: Vec<Candidate<P::Decision>> = Vec::with_capacity(2 * n);
combined.extend(annotated.into_iter().map(|e| e.candidate));
combined.extend(offspring);
let fronts = non_dominated_sort(&combined, &objectives);
let mut next: Vec<Candidate<P::Decision>> = Vec::with_capacity(n);
for front in &fronts {
if next.len() + front.len() <= n {
for &idx in front {
next.push(combined[idx].clone());
}
} else {
let dist = crowding_distance(&combined, front, &objectives);
let mut order: Vec<usize> = (0..front.len()).collect();
order.sort_by(|&a, &b| {
dist[b]
.partial_cmp(&dist[a])
.unwrap_or(std::cmp::Ordering::Equal)
});
let needed = n - next.len();
for &k in order.iter().take(needed) {
next.push(combined[front[k]].clone());
}
break;
}
if next.len() == n {
break;
}
}
annotated = annotate(next, &objectives);
}
let final_pop: Vec<Candidate<P::Decision>> =
annotated.into_iter().map(|e| e.candidate).collect();
let front = pareto_front(&final_pop, &objectives);
let best = best_candidate(&final_pop, &objectives);
OptimizationResult::new(
Population::new(final_pop),
front,
best,
evaluations,
self.config.generations,
)
}
}
fn binary_tournament<D>(entries: &[Nsga2Entry<D>], rng: &mut Rng) -> usize {
let n = entries.len();
let a = rng.random_range(0..n);
+124
View File
@@ -43,6 +43,44 @@ impl Default for Nsga3Config {
}
/// NSGA-III optimizer.
///
/// NSGA-II's many-objective successor: replaces crowding distance with
/// reference-point niching over DasDennis points in the normalized
/// objective space. The canonical default for 4+ objectives.
///
/// # Example
///
/// ```
/// use heuropt::prelude::*;
///
/// struct Schaffer;
/// impl Problem for Schaffer {
/// type Decision = Vec<f64>;
/// fn objectives(&self) -> ObjectiveSpace {
/// ObjectiveSpace::new(vec![Objective::minimize("f1"), Objective::minimize("f2")])
/// }
/// fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
/// Evaluation::new(vec![x[0] * x[0], (x[0] - 2.0).powi(2)])
/// }
/// }
///
/// let bounds = vec![(-5.0_f64, 5.0_f64)];
/// let mut opt = Nsga3::new(
/// Nsga3Config {
/// population_size: 30,
/// generations: 20,
/// reference_divisions: 12,
/// seed: 42,
/// },
/// RealBounds::new(bounds.clone()),
/// CompositeVariation {
/// crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5),
/// mutation: PolynomialMutation::new(bounds, 20.0, 1.0),
/// },
/// );
/// let r = opt.run(&Schaffer);
/// assert!(!r.pareto_front.is_empty());
/// ```
#[derive(Debug, Clone)]
pub struct Nsga3<I, V> {
/// Algorithm configuration.
@@ -142,6 +180,92 @@ where
}
}
#[cfg(feature = "async")]
impl<I, V> Nsga3<I, V> {
/// Async version of [`Optimizer::run`] — drives evaluations through
/// the user-chosen async runtime. Available only with the `async`
/// feature.
///
/// `concurrency` bounds in-flight evaluations per batch.
pub async fn run_async<P>(
&mut self,
problem: &P,
concurrency: usize,
) -> OptimizationResult<P::Decision>
where
P: crate::core::async_problem::AsyncProblem,
I: Initializer<P::Decision>,
V: Variation<P::Decision>,
{
use crate::algorithms::parallel_eval_async::evaluate_batch_async;
assert!(
self.config.population_size > 0,
"Nsga3 population_size must be greater than 0",
);
let n = self.config.population_size;
let objectives = problem.objectives();
let m = objectives.len();
let reference_points = das_dennis(m, self.config.reference_divisions);
assert!(
!reference_points.is_empty(),
"Nsga3 reference set is empty — check reference_divisions",
);
let mut rng = rng_from_seed(self.config.seed);
let initial_decisions = self.initializer.initialize(n, &mut rng);
assert_eq!(
initial_decisions.len(),
n,
"NSGA-III initializer must return exactly population_size decisions",
);
let mut population: Vec<Candidate<P::Decision>> =
evaluate_batch_async(problem, initial_decisions, concurrency).await;
let mut evaluations = population.len();
for _ in 0..self.config.generations {
let mut offspring_decisions: Vec<P::Decision> = Vec::with_capacity(n);
while offspring_decisions.len() < n {
let p1 = rng.random_range(0..population.len());
let p2 = rng.random_range(0..population.len());
let parents = vec![
population[p1].decision.clone(),
population[p2].decision.clone(),
];
let children = self.variation.vary(&parents, &mut rng);
assert!(
!children.is_empty(),
"NSGA-III variation returned no children",
);
for child_decision in children {
if offspring_decisions.len() >= n {
break;
}
offspring_decisions.push(child_decision);
}
}
let offspring = evaluate_batch_async(problem, offspring_decisions, concurrency).await;
evaluations += offspring.len();
let mut combined: Vec<Candidate<P::Decision>> = Vec::with_capacity(2 * n);
combined.extend(population);
combined.extend(offspring);
population =
environmental_selection(&combined, &objectives, &reference_points, n, &mut rng);
}
let front = pareto_front(&population, &objectives);
let best = best_candidate(&population, &objectives);
OptimizationResult::new(
Population::new(population),
front,
best,
evaluations,
self.config.generations,
)
}
}
/// NSGA-III environmental selection: front-by-front + reference-point niching
/// on the splitting front.
fn environmental_selection<D: Clone>(
+124
View File
@@ -47,6 +47,36 @@ impl Default for OnePlusOneEsConfig {
/// (1+1)-ES with the one-fifth rule: tiny, parameter-light continuous
/// optimizer. `Vec<f64>` decisions only; single-objective only.
///
/// # Example
///
/// ```
/// use heuropt::prelude::*;
///
/// struct Sphere;
/// impl Problem for Sphere {
/// type Decision = Vec<f64>;
/// fn objectives(&self) -> ObjectiveSpace {
/// ObjectiveSpace::new(vec![Objective::minimize("f")])
/// }
/// fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
/// Evaluation::new(vec![x.iter().map(|v| v * v).sum::<f64>()])
/// }
/// }
///
/// let mut opt = OnePlusOneEs::new(
/// OnePlusOneEsConfig {
/// iterations: 1_000,
/// initial_sigma: 0.5,
/// adaptation_period: 50,
/// step_increase: 1.22,
/// seed: 42,
/// },
/// RealBounds::new(vec![(-5.0, 5.0); 3]),
/// );
/// let r = opt.run(&Sphere);
/// assert!(r.best.unwrap().evaluation.objectives[0] < 1e-3);
/// ```
#[derive(Debug, Clone)]
pub struct OnePlusOneEs {
/// Algorithm configuration.
@@ -161,6 +191,100 @@ fn worse_than(a: &Evaluation, b: &Evaluation, direction: Direction) -> bool {
}
}
#[cfg(feature = "async")]
impl OnePlusOneEs {
/// Async version of [`Optimizer::run`] — drives evaluations through
/// the user-chosen async runtime. Available only with the `async`
/// feature.
///
/// `concurrency` is mostly inert here because (1+1)-ES evaluates
/// one child per iteration; it's accepted for API parity with
/// other algorithms.
pub async fn run_async<P>(
&mut self,
problem: &P,
concurrency: usize,
) -> OptimizationResult<Vec<f64>>
where
P: crate::core::async_problem::AsyncProblem<Decision = Vec<f64>>,
{
let _ = concurrency;
assert!(
self.config.initial_sigma > 0.0,
"OnePlusOneEs initial_sigma must be > 0"
);
assert!(
self.config.step_increase > 1.0,
"OnePlusOneEs step_increase must be > 1",
);
assert!(
self.config.adaptation_period >= 1,
"OnePlusOneEs adaptation_period must be >= 1",
);
let objectives = problem.objectives();
assert!(
objectives.is_single_objective(),
"OnePlusOneEs requires exactly one objective",
);
let direction = objectives.objectives[0].direction;
let mut rng = rng_from_seed(self.config.seed);
let mut parent: Vec<f64> = self
.bounds
.bounds
.iter()
.map(|&(lo, hi)| 0.5 * (lo + hi))
.collect();
let mut parent_eval = problem.evaluate_async(&parent).await;
let mut evaluations = 1usize;
let mut sigma = self.config.initial_sigma;
let mut window = std::collections::VecDeque::with_capacity(self.config.adaptation_period);
for _ in 0..self.config.iterations {
let normal = Normal::new(0.0, sigma).expect("Normal::new(0, sigma)");
let mut child = parent.clone();
for (j, x) in child.iter_mut().enumerate() {
let (lo, hi) = self.bounds.bounds[j];
*x = (*x + normal.sample(&mut rng)).clamp(lo, hi);
}
let child_eval = problem.evaluate_async(&child).await;
evaluations += 1;
let accepted = !worse_than(&child_eval, &parent_eval, direction);
if accepted {
parent = child;
parent_eval = child_eval;
}
window.push_back(if accepted { 1u8 } else { 0u8 });
if window.len() > self.config.adaptation_period {
window.pop_front();
}
if window.len() == self.config.adaptation_period {
let success_count: usize = window.iter().map(|&b| b as usize).sum();
let rate = success_count as f64 / window.len() as f64;
if rate > 0.2 {
sigma *= self.config.step_increase;
} else if rate < 0.2 {
sigma /= self.config.step_increase;
}
}
}
let best = Candidate::new(parent, parent_eval);
let population = Population::new(vec![best.clone()]);
let front = vec![best.clone()];
OptimizationResult::new(
population,
front,
Some(best),
evaluations,
self.config.iterations,
)
}
}
#[cfg(test)]
mod tests {
use super::*;
+111
View File
@@ -36,6 +36,31 @@ impl Default for PaesConfig {
/// One current candidate, one mutation per iteration, one bounded archive.
/// Intentionally a readable baseline rather than a research-perfect PAES
/// (spec §12.2).
///
/// # Example
///
/// ```
/// use heuropt::prelude::*;
///
/// struct Schaffer;
/// impl Problem for Schaffer {
/// type Decision = Vec<f64>;
/// fn objectives(&self) -> ObjectiveSpace {
/// ObjectiveSpace::new(vec![Objective::minimize("f1"), Objective::minimize("f2")])
/// }
/// fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
/// Evaluation::new(vec![x[0] * x[0], (x[0] - 2.0).powi(2)])
/// }
/// }
///
/// let mut opt = Paes::new(
/// PaesConfig { iterations: 200, archive_size: 30, seed: 42 },
/// RealBounds::new(vec![(-5.0, 5.0)]),
/// GaussianMutation { sigma: 0.3 },
/// );
/// let r = opt.run(&Schaffer);
/// assert!(!r.pareto_front.is_empty());
/// ```
#[derive(Debug, Clone)]
pub struct Paes<I, V> {
/// Algorithm configuration.
@@ -131,6 +156,92 @@ where
}
}
#[cfg(feature = "async")]
impl<I, V> Paes<I, V> {
/// Async version of [`Optimizer::run`] — drives evaluations through
/// the user-chosen async runtime. Available only with the `async`
/// feature.
///
/// `concurrency` is mostly inert here because PAES evaluates one
/// child per iteration; it's accepted for API parity with other
/// algorithms.
pub async fn run_async<P>(
&mut self,
problem: &P,
concurrency: usize,
) -> OptimizationResult<P::Decision>
where
P: crate::core::async_problem::AsyncProblem,
I: Initializer<P::Decision>,
V: Variation<P::Decision>,
{
let _ = concurrency;
assert!(
self.config.archive_size > 0,
"PAES archive_size must be greater than 0",
);
let objectives = problem.objectives();
let mut rng = rng_from_seed(self.config.seed);
let mut initial = self.initializer.initialize(1, &mut rng);
assert!(
!initial.is_empty(),
"PAES initializer returned no decisions",
);
let mut current_decision = initial.remove(0);
let mut current_eval = problem.evaluate_async(&current_decision).await;
let mut evaluations = 1usize;
let mut archive = ParetoArchive::new(objectives.clone());
archive.insert(Candidate::new(
current_decision.clone(),
current_eval.clone(),
));
for _ in 0..self.config.iterations {
let parents = vec![current_decision.clone()];
let children = self.variation.vary(&parents, &mut rng);
assert!(!children.is_empty(), "PAES variation returned no children",);
let child_decision = children.into_iter().next().unwrap();
let child_eval = problem.evaluate_async(&child_decision).await;
evaluations += 1;
match pareto_compare(&child_eval, &current_eval, &objectives) {
Dominance::Dominates => {
current_decision = child_decision.clone();
current_eval = child_eval.clone();
}
Dominance::DominatedBy => {
// Stay at current.
}
Dominance::NonDominated | Dominance::Equal => {
current_decision = child_decision.clone();
current_eval = child_eval.clone();
}
}
archive.insert(Candidate::new(child_decision, child_eval));
archive.insert(Candidate::new(
current_decision.clone(),
current_eval.clone(),
));
archive.truncate(self.config.archive_size);
}
let members = archive.into_vec();
let front = pareto_front(&members, &objectives);
let best = best_candidate(&members, &objectives);
OptimizationResult::new(
Population::new(members),
front,
best,
evaluations,
self.config.iterations,
)
}
}
#[cfg(test)]
mod tests {
use super::*;
+91
View File
@@ -0,0 +1,91 @@
//! Async population evaluator.
//!
//! Available only with the `async` feature. Used by the `run_async`
//! method on algorithms that support async problems.
use futures::stream::{FuturesOrdered, StreamExt};
use crate::core::async_problem::{AsyncPartialProblem, AsyncProblem};
use crate::core::candidate::Candidate;
use crate::core::evaluation::Evaluation;
/// Evaluate every decision concurrently against `problem`, preserving
/// input order in the returned vector. Concurrency is bounded by
/// `concurrency` (≥ 1) — too high a value wastes memory and may
/// overload downstream services; too low forfeits parallelism.
///
/// Returns a future that the caller drives via their preferred
/// runtime (typically tokio).
pub async fn evaluate_batch_async<P>(
problem: &P,
decisions: Vec<P::Decision>,
concurrency: usize,
) -> Vec<Candidate<P::Decision>>
where
P: AsyncProblem,
{
assert!(
concurrency >= 1,
"evaluate_batch_async concurrency must be >= 1"
);
let mut out: Vec<Candidate<P::Decision>> = Vec::with_capacity(decisions.len());
// Process in concurrency-bounded chunks to keep peak memory low
// and avoid blasting downstream services. Each chunk uses
// FuturesOrdered to preserve per-chunk order, and chunks are
// emitted in their natural order.
let mut iter = decisions.into_iter();
loop {
let mut futs = FuturesOrdered::new();
for _ in 0..concurrency {
match iter.next() {
Some(d) => {
futs.push_back(async move {
let e = problem.evaluate_async(&d).await;
Candidate::new(d, e)
});
}
None => break,
}
}
if futs.is_empty() {
break;
}
while let Some(c) = futs.next().await {
out.push(c);
}
}
out
}
/// Evaluate every decision at the given `budget` concurrently against a
/// multi-fidelity `problem`, preserving input order. Hyperband's async
/// path uses this for each Successive-Halving rung.
pub async fn evaluate_batch_at_budget_async<P>(
problem: &P,
decisions: &[P::Decision],
budget: f64,
concurrency: usize,
) -> Vec<Evaluation>
where
P: AsyncPartialProblem,
{
assert!(
concurrency >= 1,
"evaluate_batch_at_budget_async concurrency must be >= 1"
);
let mut out: Vec<Evaluation> = Vec::with_capacity(decisions.len());
let mut idx = 0usize;
while idx < decisions.len() {
let mut futs = FuturesOrdered::new();
let end = (idx + concurrency).min(decisions.len());
for d in &decisions[idx..end] {
futs.push_back(async move { problem.evaluate_at_budget_async(d, budget).await });
}
while let Some(e) = futs.next().await {
out.push(e);
}
idx = end;
}
out
}
+154
View File
@@ -55,6 +55,37 @@ impl Default for ParticleSwarmConfig {
/// Velocities are clamped to `±(hi - lo)` per dimension to prevent
/// "swarm explosion." Pair with `RealBounds` for both the search bounds
/// and the initial particle positions.
///
/// # Example
///
/// ```
/// use heuropt::prelude::*;
///
/// struct Sphere;
/// impl Problem for Sphere {
/// type Decision = Vec<f64>;
/// fn objectives(&self) -> ObjectiveSpace {
/// ObjectiveSpace::new(vec![Objective::minimize("f")])
/// }
/// fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
/// Evaluation::new(vec![x.iter().map(|v| v * v).sum::<f64>()])
/// }
/// }
///
/// let mut opt = ParticleSwarm::new(
/// ParticleSwarmConfig {
/// swarm_size: 20,
/// generations: 50,
/// inertia: 0.7,
/// cognitive: 1.5,
/// social: 1.5,
/// seed: 42,
/// },
/// RealBounds::new(vec![(-5.0, 5.0); 3]),
/// );
/// let r = opt.run(&Sphere);
/// assert!(r.best.is_some());
/// ```
#[derive(Debug, Clone)]
pub struct ParticleSwarm {
/// Algorithm configuration.
@@ -189,6 +220,129 @@ where
}
}
#[cfg(feature = "async")]
impl ParticleSwarm {
/// Async version of [`Optimizer::run`] — drives evaluations through
/// the user-chosen async runtime. Available only with the `async`
/// feature.
///
/// `concurrency` bounds in-flight evaluations per batch (initial
/// swarm, per-generation positions, and the final evaluation pass).
pub async fn run_async<P>(
&mut self,
problem: &P,
concurrency: usize,
) -> OptimizationResult<Vec<f64>>
where
P: crate::core::async_problem::AsyncProblem<Decision = Vec<f64>>,
{
use crate::algorithms::parallel_eval_async::evaluate_batch_async;
assert!(
self.config.swarm_size >= 1,
"ParticleSwarm swarm_size must be >= 1",
);
let objectives = problem.objectives();
assert!(
objectives.is_single_objective(),
"ParticleSwarm requires exactly one objective",
);
let direction = objectives.objectives[0].direction;
let dim = self.bounds.bounds.len();
let n = self.config.swarm_size;
let mut rng = rng_from_seed(self.config.seed);
let mut positions: Vec<Vec<f64>> = {
use crate::traits::Initializer as _;
self.bounds.initialize(n, &mut rng)
};
let mut velocities: Vec<Vec<f64>> = (0..n)
.map(|_| {
self.bounds
.bounds
.iter()
.map(|&(lo, hi)| 0.1 * (hi - lo) * (rng.random::<f64>() * 2.0 - 1.0))
.collect()
})
.collect();
let v_max: Vec<f64> = self.bounds.bounds.iter().map(|&(lo, hi)| hi - lo).collect();
let initial_pop = evaluate_batch_async(problem, positions.clone(), concurrency).await;
let mut evaluations = initial_pop.len();
let mut pbest_decisions: Vec<Vec<f64>> = positions.clone();
let mut pbest_evals: Vec<f64> = initial_pop
.iter()
.map(|c| c.evaluation.objectives[0])
.collect();
let mut gbest_idx = best_index(&pbest_evals, direction);
let mut gbest_decision = pbest_decisions[gbest_idx].clone();
let mut gbest_eval = pbest_evals[gbest_idx];
for _ in 0..self.config.generations {
for i in 0..n {
#[allow(clippy::needless_range_loop)]
for j in 0..dim {
let r1: f64 = rng.random();
let r2: f64 = rng.random();
let cognitive_term =
self.config.cognitive * r1 * (pbest_decisions[i][j] - positions[i][j]);
let social_term =
self.config.social * r2 * (gbest_decision[j] - positions[i][j]);
let mut v =
self.config.inertia * velocities[i][j] + cognitive_term + social_term;
if v > v_max[j] {
v = v_max[j];
} else if v < -v_max[j] {
v = -v_max[j];
}
velocities[i][j] = v;
let (lo, hi) = self.bounds.bounds[j];
positions[i][j] = (positions[i][j] + v).clamp(lo, hi);
}
}
let evaluated = evaluate_batch_async(problem, positions.clone(), concurrency).await;
evaluations += evaluated.len();
for (i, cand) in evaluated.iter().enumerate() {
let f = cand.evaluation.objectives[0];
let improves = match direction {
Direction::Minimize => f < pbest_evals[i],
Direction::Maximize => f > pbest_evals[i],
};
if improves {
pbest_decisions[i] = positions[i].clone();
pbest_evals[i] = f;
gbest_idx = i;
let beats_global = match direction {
Direction::Minimize => f < gbest_eval,
Direction::Maximize => f > gbest_eval,
};
if beats_global {
gbest_decision = pbest_decisions[i].clone();
gbest_eval = f;
}
}
}
}
let _ = gbest_idx;
let final_pop = evaluate_batch_async(problem, positions, concurrency).await;
evaluations += final_pop.len();
let best = best_candidate(&final_pop, &objectives);
let front: Vec<Candidate<Vec<f64>>> = best.iter().cloned().collect();
OptimizationResult::new(
Population::new(final_pop),
front,
best,
evaluations,
self.config.generations,
)
}
}
fn best_index(values: &[f64], direction: Direction) -> usize {
let mut idx = 0;
for i in 1..values.len() {
+138
View File
@@ -47,6 +47,41 @@ impl Default for PesaIIConfig {
/// Maintains an internal population (used to drive variation) and an
/// external non-dominated archive. Selection biases toward members in
/// sparsely-populated grid boxes so the front spreads out.
///
/// # Example
///
/// ```
/// use heuropt::prelude::*;
///
/// struct Schaffer;
/// impl Problem for Schaffer {
/// type Decision = Vec<f64>;
/// fn objectives(&self) -> ObjectiveSpace {
/// ObjectiveSpace::new(vec![Objective::minimize("f1"), Objective::minimize("f2")])
/// }
/// fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
/// Evaluation::new(vec![x[0] * x[0], (x[0] - 2.0).powi(2)])
/// }
/// }
///
/// let bounds = vec![(-5.0_f64, 5.0_f64)];
/// let mut opt = PesaII::new(
/// PesaIIConfig {
/// population_size: 20,
/// archive_size: 30,
/// generations: 20,
/// grid_divisions: 8,
/// seed: 42,
/// },
/// RealBounds::new(bounds.clone()),
/// CompositeVariation {
/// crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5),
/// mutation: PolynomialMutation::new(bounds, 20.0, 1.0),
/// },
/// );
/// let r = opt.run(&Schaffer);
/// assert!(!r.pareto_front.is_empty());
/// ```
#[derive(Debug, Clone)]
pub struct PesaII<I, V> {
/// Algorithm configuration.
@@ -169,6 +204,109 @@ where
}
}
#[cfg(feature = "async")]
impl<I, V> PesaII<I, V> {
/// Async version of [`Optimizer::run`] — drives evaluations through
/// the user-chosen async runtime. Available only with the `async`
/// feature.
///
/// `concurrency` bounds in-flight evaluations of the initial
/// population. Per-step evaluations are sequential to preserve the
/// algorithm's exact RNG sequencing.
pub async fn run_async<P>(
&mut self,
problem: &P,
concurrency: usize,
) -> OptimizationResult<P::Decision>
where
P: crate::core::async_problem::AsyncProblem,
I: Initializer<P::Decision>,
V: Variation<P::Decision>,
{
use crate::algorithms::parallel_eval_async::evaluate_batch_async;
assert!(
self.config.population_size > 0,
"PesaII population_size must be > 0"
);
assert!(
self.config.archive_size > 0,
"PesaII archive_size must be > 0"
);
assert!(
self.config.grid_divisions >= 1,
"PesaII grid_divisions must be >= 1"
);
let n = self.config.population_size;
let objectives = problem.objectives();
let mut rng = rng_from_seed(self.config.seed);
let initial_decisions = self.initializer.initialize(n, &mut rng);
let mut internal: Vec<Candidate<P::Decision>> =
evaluate_batch_async(problem, initial_decisions, concurrency).await;
let mut evaluations = internal.len();
let mut archive = ParetoArchive::new(objectives.clone());
for c in &internal {
archive.insert(c.clone());
}
truncate_by_grid(
&mut archive,
self.config.archive_size,
self.config.grid_divisions,
);
for _ in 0..self.config.generations {
let (boxes, counts) = build_grid(&archive, &objectives, self.config.grid_divisions);
let mut offspring: Vec<Candidate<P::Decision>> = Vec::with_capacity(n);
while offspring.len() < n {
let p1 = region_tournament(&archive, &boxes, &counts, &mut rng);
let p2 = region_tournament(&archive, &boxes, &counts, &mut rng);
let parents = vec![
archive.members()[p1].decision.clone(),
archive.members()[p2].decision.clone(),
];
let children = self.variation.vary(&parents, &mut rng);
assert!(
!children.is_empty(),
"PesaII variation returned no children"
);
for child in children {
if offspring.len() >= n {
break;
}
let eval = problem.evaluate_async(&child).await;
evaluations += 1;
offspring.push(Candidate::new(child, eval));
}
}
for c in &offspring {
archive.insert(c.clone());
}
truncate_by_grid(
&mut archive,
self.config.archive_size,
self.config.grid_divisions,
);
internal = offspring;
}
let _ = internal;
let members = archive.into_vec();
let front = pareto_front(&members, &objectives);
let best = best_candidate(&members, &objectives);
OptimizationResult::new(
Population::new(members),
front,
best,
evaluations,
self.config.generations,
)
}
}
/// Compute per-member box index (M-tuple of grid coordinates) and the
/// population count of each occupied box.
fn build_grid<D: Clone>(
+70
View File
@@ -38,6 +38,31 @@ impl Default for RandomSearchConfig {
/// Each iteration the configured `Initializer` produces `batch_size` decisions
/// which are evaluated and pushed into the population. Cheap, parallelism-free,
/// and useful as a sanity-check baseline.
///
/// # Example
///
/// ```
/// use heuropt::prelude::*;
///
/// struct Sphere;
/// impl Problem for Sphere {
/// type Decision = Vec<f64>;
/// fn objectives(&self) -> ObjectiveSpace {
/// ObjectiveSpace::new(vec![Objective::minimize("f")])
/// }
/// fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
/// Evaluation::new(vec![x.iter().map(|v| v * v).sum::<f64>()])
/// }
/// }
///
/// let mut opt = RandomSearch::new(
/// RandomSearchConfig { iterations: 200, batch_size: 10, seed: 42 },
/// RealBounds::new(vec![(-5.0, 5.0); 3]),
/// );
/// let r = opt.run(&Sphere);
/// assert_eq!(r.evaluations, 200 * 10);
/// assert!(r.best.is_some());
/// ```
#[derive(Debug, Clone)]
pub struct RandomSearch<I> {
/// Algorithm configuration.
@@ -88,6 +113,51 @@ where
}
}
#[cfg(feature = "async")]
impl<I> RandomSearch<I> {
/// Async version of [`Optimizer::run`] — drives evaluations through
/// the user-chosen async runtime (typically tokio). Useful when
/// `evaluate` is IO-bound (HTTP, RPC, subprocess).
///
/// `concurrency` bounds how many evaluations are in-flight at once;
/// `1` is sequential, larger values push more load to the
/// downstream service.
///
/// Available only with the `async` feature.
pub async fn run_async<P>(
&mut self,
problem: &P,
concurrency: usize,
) -> OptimizationResult<P::Decision>
where
P: crate::core::async_problem::AsyncProblem,
I: Initializer<P::Decision>,
{
use crate::algorithms::parallel_eval_async::evaluate_batch_async;
let objectives = problem.objectives();
let mut rng = rng_from_seed(self.config.seed);
let mut all: Vec<Candidate<P::Decision>> = Vec::new();
let mut evaluations = 0usize;
for _ in 0..self.config.iterations {
let decisions = self
.initializer
.initialize(self.config.batch_size, &mut rng);
evaluations += decisions.len();
let cands = evaluate_batch_async(problem, decisions, concurrency).await;
all.extend(cands);
}
let front = pareto_front(&all, &objectives);
let best = best_candidate(&all, &objectives);
OptimizationResult::new(
Population::new(all),
front,
best,
evaluations,
self.config.iterations,
)
}
}
#[cfg(test)]
mod tests {
use super::*;
+196
View File
@@ -41,6 +41,45 @@ impl Default for RveaConfig {
}
/// Reference Vector-guided Evolutionary Algorithm.
///
/// Many-objective EA that uses DasDennis reference vectors with an
/// adaptive penalty term to balance convergence and diversity as
/// generations progress.
///
/// # Example
///
/// ```
/// use heuropt::prelude::*;
///
/// struct Schaffer;
/// impl Problem for Schaffer {
/// type Decision = Vec<f64>;
/// fn objectives(&self) -> ObjectiveSpace {
/// ObjectiveSpace::new(vec![Objective::minimize("f1"), Objective::minimize("f2")])
/// }
/// fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
/// Evaluation::new(vec![x[0] * x[0], (x[0] - 2.0).powi(2)])
/// }
/// }
///
/// let bounds = vec![(-5.0_f64, 5.0_f64)];
/// let mut opt = Rvea::new(
/// RveaConfig {
/// population_size: 30,
/// generations: 20,
/// reference_divisions: 19,
/// alpha: 2.0,
/// seed: 42,
/// },
/// RealBounds::new(bounds.clone()),
/// CompositeVariation {
/// crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5),
/// mutation: PolynomialMutation::new(bounds, 20.0, 1.0),
/// },
/// );
/// let r = opt.run(&Schaffer);
/// assert!(!r.pareto_front.is_empty());
/// ```
#[derive(Debug, Clone)]
pub struct Rvea<I, V> {
/// Algorithm configuration.
@@ -222,6 +261,163 @@ where
}
}
#[cfg(feature = "async")]
impl<I, V> Rvea<I, V> {
/// Async version of [`Optimizer::run`] — drives evaluations through
/// the user-chosen async runtime. Available only with the `async`
/// feature.
///
/// `concurrency` bounds in-flight evaluations per batch.
pub async fn run_async<P>(
&mut self,
problem: &P,
concurrency: usize,
) -> OptimizationResult<P::Decision>
where
P: crate::core::async_problem::AsyncProblem,
I: Initializer<P::Decision>,
V: Variation<P::Decision>,
{
use crate::algorithms::parallel_eval_async::evaluate_batch_async;
assert!(
self.config.population_size > 0,
"Rvea population_size must be > 0"
);
let n = self.config.population_size;
let objectives = problem.objectives();
let m = objectives.len();
let raw_refs = das_dennis(m, self.config.reference_divisions);
let references: Vec<Vec<f64>> = raw_refs.into_iter().map(unit_normalize).collect();
assert!(
!references.is_empty(),
"Rvea: no reference vectors generated"
);
let theta_max = smallest_neighbor_angle(&references);
let mut rng = rng_from_seed(self.config.seed);
let initial_decisions = self.initializer.initialize(n, &mut rng);
let mut population: Vec<Candidate<P::Decision>> =
evaluate_batch_async(problem, initial_decisions, concurrency).await;
let mut evaluations = population.len();
for gen_idx in 0..self.config.generations {
let mut offspring_decisions: Vec<P::Decision> = Vec::with_capacity(n);
while offspring_decisions.len() < n {
let p1 = rng.random_range(0..population.len());
let p2 = rng.random_range(0..population.len());
let parents = vec![
population[p1].decision.clone(),
population[p2].decision.clone(),
];
let children = self.variation.vary(&parents, &mut rng);
assert!(!children.is_empty(), "Rvea variation returned no children");
for child in children {
if offspring_decisions.len() >= n {
break;
}
offspring_decisions.push(child);
}
}
let offspring = evaluate_batch_async(problem, offspring_decisions, concurrency).await;
evaluations += offspring.len();
let mut combined: Vec<Candidate<P::Decision>> = Vec::with_capacity(2 * n);
combined.extend(population);
combined.extend(offspring);
let m_dim = m;
let mut ideal = vec![f64::INFINITY; m_dim];
for c in &combined {
let oriented = objectives.as_minimization(&c.evaluation.objectives);
for (k, v) in oriented.iter().enumerate() {
if *v < ideal[k] {
ideal[k] = *v;
}
}
}
let translated: Vec<Vec<f64>> = combined
.iter()
.map(|c| {
let oriented = objectives.as_minimization(&c.evaluation.objectives);
oriented
.iter()
.enumerate()
.map(|(k, v)| v - ideal[k])
.collect()
})
.collect();
let mut assoc: Vec<usize> = vec![0; combined.len()];
let mut angles: Vec<f64> = vec![0.0; combined.len()];
for (i, t) in translated.iter().enumerate() {
let (best_ref, best_angle) = closest_reference(t, &references);
assoc[i] = best_ref;
angles[i] = best_angle;
}
let alpha_t = (gen_idx as f64 / (self.config.generations as f64).max(1.0))
.powf(self.config.alpha);
let mut keep: Vec<Option<(usize, f64)>> = vec![None; references.len()];
for i in 0..combined.len() {
let r = assoc[i];
let length: f64 = translated[i].iter().map(|v| v * v).sum::<f64>().sqrt();
let theta_max_safe = theta_max.max(1e-12);
let penalty = 1.0 + (m_dim as f64) * alpha_t * (angles[i] / theta_max_safe);
let apd = penalty * length;
match keep[r] {
None => keep[r] = Some((i, apd)),
Some((_, current)) if apd < current => keep[r] = Some((i, apd)),
_ => {}
}
}
let mut next: Vec<Candidate<P::Decision>> = keep
.into_iter()
.flatten()
.map(|(i, _)| combined[i].clone())
.collect();
if next.len() < n {
let mut all_apds: Vec<(usize, f64)> = (0..combined.len())
.map(|i| {
let length: f64 = translated[i].iter().map(|v| v * v).sum::<f64>().sqrt();
let theta_max_safe = theta_max.max(1e-12);
let penalty = 1.0 + (m_dim as f64) * alpha_t * (angles[i] / theta_max_safe);
(i, penalty * length)
})
.collect();
all_apds.sort_by(|a, b| a.1.partial_cmp(&b.1).unwrap_or(std::cmp::Ordering::Equal));
for (i, _) in all_apds {
if next.len() >= n {
break;
}
if !next
.iter()
.any(|c| std::ptr::eq(c as *const _, &combined[i] as *const _))
{
next.push(combined[i].clone());
}
}
}
if next.len() > n {
next.truncate(n);
}
population = next;
}
let front = pareto_front(&population, &objectives);
let best = best_candidate(&population, &objectives);
OptimizationResult::new(
Population::new(population),
front,
best,
evaluations,
self.config.generations,
)
}
}
fn unit_normalize(mut v: Vec<f64>) -> Vec<f64> {
let n: f64 = v.iter().map(|x| x * x).sum::<f64>().sqrt();
if n > 1e-12 {
+148
View File
@@ -41,6 +41,36 @@ impl Default for SimulatedAnnealingConfig {
/// and `T` anneals geometrically from `initial_temperature` to
/// `final_temperature` over the iteration count. Generic over decision
/// type — pair with any `Variation` impl that returns one child per call.
///
/// # Example
///
/// ```
/// use heuropt::prelude::*;
///
/// struct Sphere;
/// impl Problem for Sphere {
/// type Decision = Vec<f64>;
/// fn objectives(&self) -> ObjectiveSpace {
/// ObjectiveSpace::new(vec![Objective::minimize("f")])
/// }
/// fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
/// Evaluation::new(vec![x.iter().map(|v| v * v).sum::<f64>()])
/// }
/// }
///
/// let mut opt = SimulatedAnnealing::new(
/// SimulatedAnnealingConfig {
/// iterations: 2_000,
/// initial_temperature: 1.0,
/// final_temperature: 1e-3,
/// seed: 42,
/// },
/// RealBounds::new(vec![(-5.0, 5.0); 3]),
/// GaussianMutation { sigma: 0.3 },
/// );
/// let r = opt.run(&Sphere);
/// assert!(r.best.is_some());
/// ```
#[derive(Debug, Clone)]
pub struct SimulatedAnnealing<I, V> {
/// Algorithm configuration.
@@ -185,6 +215,124 @@ fn better_than(
}
}
#[cfg(feature = "async")]
impl<I, V> SimulatedAnnealing<I, V> {
/// Async version of [`Optimizer::run`] — drives evaluations through
/// the user-chosen async runtime. Available only with the `async`
/// feature.
///
/// `concurrency` is mostly inert here because SA evaluates one
/// child per iteration; it's accepted for API parity with other
/// algorithms.
pub async fn run_async<P>(
&mut self,
problem: &P,
concurrency: usize,
) -> OptimizationResult<P::Decision>
where
P: crate::core::async_problem::AsyncProblem,
I: Initializer<P::Decision>,
V: Variation<P::Decision>,
{
let _ = concurrency;
let objectives = problem.objectives();
assert!(
objectives.is_single_objective(),
"SimulatedAnnealing requires exactly one objective",
);
assert!(
self.config.initial_temperature > 0.0,
"SimulatedAnnealing initial_temperature must be positive",
);
assert!(
self.config.final_temperature > 0.0,
"SimulatedAnnealing final_temperature must be positive",
);
assert!(
self.config.final_temperature <= self.config.initial_temperature,
"SimulatedAnnealing final_temperature must be <= initial_temperature",
);
let direction = objectives.objectives[0].direction;
let mut rng = rng_from_seed(self.config.seed);
let mut initial = self.initializer.initialize(1, &mut rng);
assert!(
!initial.is_empty(),
"SimulatedAnnealing initializer returned no decisions",
);
let mut current_decision = initial.remove(0);
let mut current_eval = problem.evaluate_async(&current_decision).await;
let mut best_decision = current_decision.clone();
let mut best_eval = current_eval.clone();
let mut evaluations = 1usize;
let cooling = if self.config.iterations <= 1 {
1.0
} else {
(self.config.final_temperature / self.config.initial_temperature)
.powf(1.0 / (self.config.iterations as f64 - 1.0))
};
let mut temperature = self.config.initial_temperature;
for _ in 0..self.config.iterations {
let parents = vec![current_decision.clone()];
let children = self.variation.vary(&parents, &mut rng);
assert!(
!children.is_empty(),
"SimulatedAnnealing variation returned no children"
);
let child_decision = children.into_iter().next().unwrap();
let child_eval = problem.evaluate_async(&child_decision).await;
evaluations += 1;
let accept = match (child_eval.is_feasible(), current_eval.is_feasible()) {
(true, false) => true,
(false, true) => false,
(false, false) => {
child_eval.constraint_violation <= current_eval.constraint_violation
}
(true, true) => {
let delta = match direction {
Direction::Minimize => {
child_eval.objectives[0] - current_eval.objectives[0]
}
Direction::Maximize => {
current_eval.objectives[0] - child_eval.objectives[0]
}
};
if delta <= 0.0 {
true
} else {
let prob = (-delta / temperature).exp();
rng.random::<f64>() < prob
}
}
};
if accept {
current_decision = child_decision;
current_eval = child_eval;
if better_than(&current_eval, &best_eval, direction) {
best_decision = current_decision.clone();
best_eval = current_eval.clone();
}
}
temperature *= cooling;
}
let best = Candidate::new(best_decision, best_eval);
let population = Population::new(vec![best.clone()]);
let front = vec![best.clone()];
OptimizationResult::new(
population,
front,
Some(best),
evaluations,
self.config.iterations,
)
}
}
#[cfg(test)]
mod tests {
use super::*;
+108
View File
@@ -49,6 +49,40 @@ impl Default for SmsEmoaConfig {
/// non-dominated front. Excellent convergence quality at the price of
/// quadratic-in-N hypervolume evaluations per generation, so practical
/// up to ~4 objectives at population sizes ≤ 200.
///
/// # Example
///
/// ```
/// use heuropt::prelude::*;
///
/// struct Schaffer;
/// impl Problem for Schaffer {
/// type Decision = Vec<f64>;
/// fn objectives(&self) -> ObjectiveSpace {
/// ObjectiveSpace::new(vec![Objective::minimize("f1"), Objective::minimize("f2")])
/// }
/// fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
/// Evaluation::new(vec![x[0] * x[0], (x[0] - 2.0).powi(2)])
/// }
/// }
///
/// let bounds = vec![(-5.0_f64, 5.0_f64)];
/// let mut opt = SmsEmoa::new(
/// SmsEmoaConfig {
/// population_size: 20,
/// generations: 100,
/// reference_point: vec![30.0, 30.0],
/// seed: 42,
/// },
/// RealBounds::new(bounds.clone()),
/// CompositeVariation {
/// crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5),
/// mutation: PolynomialMutation::new(bounds, 20.0, 1.0),
/// },
/// );
/// let r = opt.run(&Schaffer);
/// assert!(!r.pareto_front.is_empty());
/// ```
#[derive(Debug, Clone)]
pub struct SmsEmoa<I, V> {
/// Algorithm configuration.
@@ -139,6 +173,80 @@ where
}
}
#[cfg(feature = "async")]
impl<I, V> SmsEmoa<I, V> {
/// Async version of [`Optimizer::run`] — drives evaluations through
/// the user-chosen async runtime. Available only with the `async`
/// feature.
///
/// `concurrency` bounds in-flight evaluations of the initial
/// population. Per-generation evaluations are sequential because
/// SMS-EMOA is a steady-state algorithm (one child per generation).
pub async fn run_async<P>(
&mut self,
problem: &P,
concurrency: usize,
) -> OptimizationResult<P::Decision>
where
P: crate::core::async_problem::AsyncProblem,
I: Initializer<P::Decision>,
V: Variation<P::Decision>,
{
use crate::algorithms::parallel_eval_async::evaluate_batch_async;
assert!(
self.config.population_size > 0,
"SmsEmoa population_size must be > 0"
);
let n = self.config.population_size;
let objectives = problem.objectives();
assert_eq!(
self.config.reference_point.len(),
objectives.len(),
"SmsEmoa reference_point.len() must equal number of objectives",
);
let reference = self.config.reference_point.clone();
let mut rng = rng_from_seed(self.config.seed);
let initial_decisions = self.initializer.initialize(n, &mut rng);
let mut population: Vec<Candidate<P::Decision>> =
evaluate_batch_async(problem, initial_decisions, concurrency).await;
let mut evaluations = population.len();
for _ in 0..self.config.generations {
let p1 = rng.random_range(0..population.len());
let p2 = rng.random_range(0..population.len());
let parents = vec![
population[p1].decision.clone(),
population[p2].decision.clone(),
];
let children = self.variation.vary(&parents, &mut rng);
assert!(
!children.is_empty(),
"SmsEmoa variation returned no children"
);
let child_decision = children.into_iter().next().unwrap();
let child_eval = problem.evaluate_async(&child_decision).await;
evaluations += 1;
let child = Candidate::new(child_decision, child_eval);
population.push(child);
let drop_idx = pick_drop_index(&population, &objectives, &reference);
population.swap_remove(drop_idx);
}
let front = pareto_front(&population, &objectives);
let best = best_candidate(&population, &objectives);
OptimizationResult::new(
Population::new(population),
front,
best,
evaluations,
self.config.generations,
)
}
}
/// Choose the index in `pool` whose removal is preferred per SMS-EMOA's
/// rules: drop from the worst non-dominated front; within that front,
/// drop the member whose removal increases hypervolume the most (= the
+162
View File
@@ -51,6 +51,37 @@ impl Default for SeparableNesConfig {
/// following the natural gradient of expected fitness, with rank-shaped
/// fitness utilities for invariance to monotone transforms of the
/// objective.
///
/// # Example
///
/// ```
/// use heuropt::prelude::*;
///
/// struct Sphere;
/// impl Problem for Sphere {
/// type Decision = Vec<f64>;
/// fn objectives(&self) -> ObjectiveSpace {
/// ObjectiveSpace::new(vec![Objective::minimize("f")])
/// }
/// fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
/// Evaluation::new(vec![x.iter().map(|v| v * v).sum::<f64>()])
/// }
/// }
///
/// let mut opt = SeparableNes::new(
/// SeparableNesConfig {
/// population_size: 16,
/// generations: 80,
/// initial_sigma: 1.0,
/// mean_learning_rate: 1.0,
/// sigma_learning_rate: None, // use NES default
/// seed: 42,
/// },
/// RealBounds::new(vec![(-5.0, 5.0); 3]),
/// );
/// let r = opt.run(&Sphere);
/// assert!(r.best.unwrap().evaluation.objectives[0] < 1e-3);
/// ```
#[derive(Debug, Clone)]
pub struct SeparableNes {
/// Algorithm configuration.
@@ -191,6 +222,137 @@ where
}
}
#[cfg(feature = "async")]
impl SeparableNes {
/// Async version of [`Optimizer::run`] — drives evaluations through
/// the user-chosen async runtime. Available only with the `async`
/// feature.
///
/// `concurrency` bounds in-flight evaluations per generation.
pub async fn run_async<P>(
&mut self,
problem: &P,
concurrency: usize,
) -> OptimizationResult<Vec<f64>>
where
P: crate::core::async_problem::AsyncProblem<Decision = Vec<f64>>,
{
use crate::algorithms::parallel_eval_async::evaluate_batch_async;
assert!(
self.config.population_size >= 2,
"SeparableNes population_size must be >= 2",
);
assert!(
self.config.initial_sigma > 0.0,
"SeparableNes initial_sigma must be > 0"
);
let objectives = problem.objectives();
assert!(
objectives.is_single_objective(),
"SeparableNes requires exactly one objective",
);
let direction = objectives.objectives[0].direction;
let n = self.bounds.bounds.len();
let lambda = self.config.population_size;
let mut rng = rng_from_seed(self.config.seed);
let mut mean: Vec<f64> = self
.bounds
.bounds
.iter()
.map(|&(lo, hi)| 0.5 * (lo + hi))
.collect();
let mut sigma = vec![self.config.initial_sigma; n];
let eta_sigma = self
.config
.sigma_learning_rate
.unwrap_or_else(|| (3.0 + (n as f64).ln()) / (5.0 * (n as f64).sqrt()));
let eta_mean = self.config.mean_learning_rate;
let utilities = nes_utilities(lambda);
let mut best_seen: Option<Candidate<Vec<f64>>> = None;
let mut total_evaluations = 0usize;
for _ in 0..self.config.generations {
// Sample λ offspring; matches the sync RNG draw order so seeded
// runs reproduce exactly.
let mut z_samples: Vec<Vec<f64>> = Vec::with_capacity(lambda);
let mut x_samples: Vec<Vec<f64>> = Vec::with_capacity(lambda);
for _ in 0..lambda {
let z: Vec<f64> = (0..n)
.map(|_| Normal::new(0.0, 1.0).unwrap().sample(&mut rng))
.collect();
let x: Vec<f64> = (0..n)
.map(|j| {
let v = mean[j] + sigma[j] * z[j];
let (lo, hi) = self.bounds.bounds[j];
v.clamp(lo, hi)
})
.collect();
z_samples.push(z);
x_samples.push(x);
}
let cands = evaluate_batch_async(problem, x_samples.clone(), concurrency).await;
total_evaluations += cands.len();
let evals: Vec<Evaluation> = cands.iter().map(|c| c.evaluation.clone()).collect();
for c in &cands {
let beats_best = match &best_seen {
None => true,
Some(b) => better(&c.evaluation, &b.evaluation, direction),
};
if beats_best {
best_seen = Some(c.clone());
}
}
let mut order: Vec<usize> = (0..lambda).collect();
order.sort_by(|&a, &b| compare(&evals[a], &evals[b], direction));
let mut grad_mean = vec![0.0_f64; n];
for k in 0..lambda {
let u = utilities[k];
let z = &z_samples[order[k]];
for j in 0..n {
grad_mean[j] += u * z[j];
}
}
for j in 0..n {
mean[j] += eta_mean * sigma[j] * grad_mean[j];
let (lo, hi) = self.bounds.bounds[j];
mean[j] = mean[j].clamp(lo, hi);
}
for j in 0..n {
let mut grad_sigma_j = 0.0;
for k in 0..lambda {
let u = utilities[k];
let z = &z_samples[order[k]];
grad_sigma_j += u * (z[j] * z[j] - 1.0);
}
sigma[j] *= (0.5 * eta_sigma * grad_sigma_j).exp();
if !sigma[j].is_finite() || sigma[j] < 1e-30 {
sigma[j] = 1e-30;
}
}
}
let best = best_seen.expect("at least one generation evaluated");
let population = Population::new(vec![best.clone()]);
let front = vec![best.clone()];
OptimizationResult::new(
population,
front,
Some(best),
total_evaluations,
self.config.generations,
)
}
}
fn nes_utilities(lambda: usize) -> Vec<f64> {
let half = lambda as f64 / 2.0 + 1.0;
let raw: Vec<f64> = (0..lambda)
+119
View File
@@ -37,6 +37,40 @@ impl Default for Spea2Config {
}
/// SPEA2 optimizer.
///
/// Strength Pareto Evolutionary Algorithm 2: combines a strength-based
/// dominance score with a k-th nearest-neighbor density estimate. Maintains
/// an external archive separate from the working population.
///
/// # Example
///
/// ```
/// use heuropt::prelude::*;
///
/// struct Schaffer;
/// impl Problem for Schaffer {
/// type Decision = Vec<f64>;
/// fn objectives(&self) -> ObjectiveSpace {
/// ObjectiveSpace::new(vec![Objective::minimize("f1"), Objective::minimize("f2")])
/// }
/// fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
/// Evaluation::new(vec![x[0] * x[0], (x[0] - 2.0).powi(2)])
/// }
/// }
///
/// let bounds = vec![(-5.0_f64, 5.0_f64)];
/// let mut opt = Spea2::new(
/// Spea2Config { population_size: 30, archive_size: 30, generations: 20, seed: 42 },
/// RealBounds::new(bounds.clone()),
/// CompositeVariation {
/// crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5),
/// mutation: PolynomialMutation::new(bounds, 20.0, 1.0),
/// },
/// );
/// let r = opt.run(&Schaffer);
/// assert_eq!(r.population.len(), 30);
/// assert!(!r.pareto_front.is_empty());
/// ```
#[derive(Debug, Clone)]
pub struct Spea2<I, V> {
/// Algorithm configuration.
@@ -135,6 +169,91 @@ where
}
}
#[cfg(feature = "async")]
impl<I, V> Spea2<I, V> {
/// Async version of [`Optimizer::run`] — drives evaluations through
/// the user-chosen async runtime. Available only with the `async`
/// feature.
///
/// `concurrency` bounds in-flight evaluations per batch.
pub async fn run_async<P>(
&mut self,
problem: &P,
concurrency: usize,
) -> OptimizationResult<P::Decision>
where
P: crate::core::async_problem::AsyncProblem,
I: Initializer<P::Decision>,
V: Variation<P::Decision>,
{
use crate::algorithms::parallel_eval_async::evaluate_batch_async;
assert!(
self.config.population_size > 0,
"Spea2 population_size must be greater than 0",
);
assert!(
self.config.archive_size > 0,
"Spea2 archive_size must be greater than 0",
);
let n_pop = self.config.population_size;
let n_arc = self.config.archive_size;
let objectives = problem.objectives();
let mut rng = rng_from_seed(self.config.seed);
let initial_decisions = self.initializer.initialize(n_pop, &mut rng);
assert_eq!(
initial_decisions.len(),
n_pop,
"SPEA2 initializer must return exactly population_size decisions",
);
let mut population: Vec<Candidate<P::Decision>> =
evaluate_batch_async(problem, initial_decisions, concurrency).await;
let mut evaluations = population.len();
let mut archive: Vec<Candidate<P::Decision>> = Vec::new();
for _ in 0..self.config.generations {
let mut pool: Vec<Candidate<P::Decision>> =
Vec::with_capacity(population.len() + archive.len());
pool.append(&mut population);
pool.append(&mut archive);
let fitness = compute_fitness(&pool, &objectives);
archive = build_archive(&pool, &fitness, &objectives, n_arc);
let archive_fitness = compute_fitness(&archive, &objectives);
let mut offspring_decisions: Vec<P::Decision> = Vec::with_capacity(n_pop);
while offspring_decisions.len() < n_pop {
let p1 = binary_tournament(&archive_fitness, &mut rng);
let p2 = binary_tournament(&archive_fitness, &mut rng);
let parents = vec![archive[p1].decision.clone(), archive[p2].decision.clone()];
let children = self.variation.vary(&parents, &mut rng);
assert!(!children.is_empty(), "SPEA2 variation returned no children");
for child_decision in children {
if offspring_decisions.len() >= n_pop {
break;
}
offspring_decisions.push(child_decision);
}
}
let new_population =
evaluate_batch_async(problem, offspring_decisions, concurrency).await;
evaluations += new_population.len();
population = new_population;
}
let front = pareto_front(&archive, &objectives);
let best = best_candidate(&archive, &objectives);
OptimizationResult::new(
Population::new(archive),
front,
best,
evaluations,
self.config.generations,
)
}
}
/// SPEA2 fitness: `R(i) + D(i)`, where lower is better.
///
/// `R(i)` is the sum of `S(j)` over all `j` that dominate `i`. `S(j)` is the
+122
View File
@@ -209,6 +209,128 @@ fn better_than(
}
}
#[cfg(feature = "async")]
impl<D, I, N> TabuSearch<D, I, N>
where
D: Clone + Hash + Eq,
I: Initializer<D>,
N: FnMut(&D, &mut Rng) -> Vec<D>,
{
/// Async version of [`Optimizer::run`] — drives evaluations through
/// the user-chosen async runtime. Available only with the `async`
/// feature.
///
/// Each iteration evaluates the K neighbors of the current
/// incumbent concurrently (bounded by `concurrency`), then picks
/// the best non-tabu (or aspiration-passing) move.
pub async fn run_async<P>(&mut self, problem: &P, concurrency: usize) -> OptimizationResult<D>
where
P: crate::core::async_problem::AsyncProblem<Decision = D>,
D: Send + Sync,
{
use crate::algorithms::parallel_eval_async::evaluate_batch_async;
let objectives = problem.objectives();
assert!(
objectives.is_single_objective(),
"TabuSearch requires exactly one objective",
);
assert!(
self.config.tabu_tenure >= 1,
"TabuSearch tabu_tenure must be >= 1",
);
let direction = objectives.objectives[0].direction;
let mut rng = rng_from_seed(self.config.seed);
let mut initial = self.initializer.initialize(1, &mut rng);
assert!(
!initial.is_empty(),
"TabuSearch initializer returned no decisions"
);
let mut current_decision = initial.remove(0);
let mut current_eval = problem.evaluate_async(&current_decision).await;
let mut best_decision = current_decision.clone();
let mut best_eval = current_eval.clone();
let mut evaluations = 1usize;
let mut tabu_queue: VecDeque<D> = VecDeque::with_capacity(self.config.tabu_tenure);
let mut tabu_set: HashSet<D> = HashSet::new();
for _ in 0..self.config.iterations {
let candidates = (self.neighbors)(&current_decision, &mut rng);
if candidates.is_empty() {
break;
}
let cand_results = evaluate_batch_async(problem, candidates.clone(), concurrency).await;
let mut cand_evals: Vec<crate::core::evaluation::Evaluation> =
cand_results.into_iter().map(|c| c.evaluation).collect();
evaluations += candidates.len();
let mut best_idx: Option<usize> = None;
let mut best_cand_eval: Option<crate::core::evaluation::Evaluation> = None;
for (i, c) in candidates.iter().enumerate() {
let is_tabu = tabu_set.contains(c);
let aspires = is_tabu && better_than(&cand_evals[i], &best_eval, direction);
if is_tabu && !aspires {
continue;
}
let eligible = match &best_cand_eval {
None => true,
Some(b) => better_than(&cand_evals[i], b, direction),
};
if eligible {
best_idx = Some(i);
best_cand_eval = Some(cand_evals[i].clone());
}
}
if best_idx.is_none() {
for (i, _) in candidates.iter().enumerate() {
let eligible = match &best_cand_eval {
None => true,
Some(b) => better_than(&cand_evals[i], b, direction),
};
if eligible {
best_idx = Some(i);
best_cand_eval = Some(cand_evals[i].clone());
}
}
}
let chosen_idx = best_idx.expect("non-empty candidate list");
let chosen_decision = candidates[chosen_idx].clone();
current_eval = cand_evals.remove(chosen_idx);
current_decision = chosen_decision.clone();
if better_than(&current_eval, &best_eval, direction) {
best_decision = current_decision.clone();
best_eval = current_eval.clone();
}
tabu_queue.push_back(chosen_decision.clone());
tabu_set.insert(chosen_decision);
if tabu_queue.len() > self.config.tabu_tenure {
if let Some(old) = tabu_queue.pop_front() {
tabu_set.remove(&old);
}
}
}
let best = Candidate::new(best_decision, best_eval);
let population = Population::new(vec![best.clone()]);
let front = vec![best.clone()];
OptimizationResult::new(
population,
front,
Some(best),
evaluations,
self.config.iterations,
)
}
}
#[cfg(test)]
mod tests {
use super::*;
+140
View File
@@ -41,6 +41,30 @@ impl Default for TlboConfig {
/// population_size and generations. Compared with the rest of heuropt's
/// SO toolkit (DE has F+CR, PSO has w+c1+c2, CMA-ES has σ, GA needs
/// crossover+mutation operators), TLBO works out of the box.
///
/// # Example
///
/// ```
/// use heuropt::prelude::*;
///
/// struct Sphere;
/// impl Problem for Sphere {
/// type Decision = Vec<f64>;
/// fn objectives(&self) -> ObjectiveSpace {
/// ObjectiveSpace::new(vec![Objective::minimize("f")])
/// }
/// fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
/// Evaluation::new(vec![x.iter().map(|v| v * v).sum::<f64>()])
/// }
/// }
///
/// let mut opt = Tlbo::new(
/// TlboConfig { population_size: 20, generations: 50, seed: 42 },
/// RealBounds::new(vec![(-5.0, 5.0); 3]),
/// );
/// let r = opt.run(&Sphere);
/// assert!(r.best.unwrap().evaluation.objectives[0] < 1e-3);
/// ```
#[derive(Debug, Clone)]
pub struct Tlbo {
/// Algorithm configuration.
@@ -163,6 +187,122 @@ where
}
}
#[cfg(feature = "async")]
impl Tlbo {
/// Async version of [`Optimizer::run`] — drives evaluations through
/// the user-chosen async runtime. Available only with the `async`
/// feature.
///
/// `concurrency` bounds in-flight evaluations within batched phases
/// (only the initial population uses a batch; the teacher and learner
/// phases evaluate sequentially because each accept/reject step
/// depends on the previous one).
pub async fn run_async<P>(
&mut self,
problem: &P,
concurrency: usize,
) -> OptimizationResult<Vec<f64>>
where
P: crate::core::async_problem::AsyncProblem<Decision = Vec<f64>>,
{
use crate::algorithms::parallel_eval_async::evaluate_batch_async;
assert!(
self.config.population_size >= 2,
"Tlbo population_size must be >= 2"
);
let objectives = problem.objectives();
assert!(
objectives.is_single_objective(),
"Tlbo requires exactly one objective",
);
let direction = objectives.objectives[0].direction;
let dim = self.bounds.bounds.len();
let n = self.config.population_size;
let mut rng = rng_from_seed(self.config.seed);
let mut decisions: Vec<Vec<f64>> = {
use crate::traits::Initializer as _;
self.bounds.initialize(n, &mut rng)
};
let initial = evaluate_batch_async(problem, decisions.clone(), concurrency).await;
let mut evals: Vec<Evaluation> = initial.iter().map(|c| c.evaluation.clone()).collect();
let mut evaluations = initial.len();
for _ in 0..self.config.generations {
let teacher_idx = best_index(&evals, direction);
let teacher = decisions[teacher_idx].clone();
let mut mean = vec![0.0_f64; dim];
for d in &decisions {
for j in 0..dim {
mean[j] += d[j];
}
}
for v in mean.iter_mut() {
*v /= n as f64;
}
let tf = if rng.random_bool(0.5) { 1.0 } else { 2.0 };
for i in 0..n {
let mut candidate = decisions[i].clone();
for j in 0..dim {
let r: f64 = rng.random();
candidate[j] += r * (teacher[j] - tf * mean[j]);
let (lo, hi) = self.bounds.bounds[j];
candidate[j] = candidate[j].clamp(lo, hi);
}
let cand_eval = problem.evaluate_async(&candidate).await;
evaluations += 1;
if better(&cand_eval, &evals[i], direction) {
decisions[i] = candidate;
evals[i] = cand_eval;
}
}
for i in 0..n {
let mut k = rng.random_range(0..n);
while k == i && n > 1 {
k = rng.random_range(0..n);
}
let partner_better = better(&evals[k], &evals[i], direction);
let mut candidate = decisions[i].clone();
for j in 0..dim {
let r: f64 = rng.random();
let delta = if partner_better {
r * (decisions[k][j] - decisions[i][j])
} else {
r * (decisions[i][j] - decisions[k][j])
};
candidate[j] += delta;
let (lo, hi) = self.bounds.bounds[j];
candidate[j] = candidate[j].clamp(lo, hi);
}
let cand_eval = problem.evaluate_async(&candidate).await;
evaluations += 1;
if better(&cand_eval, &evals[i], direction) {
decisions[i] = candidate;
evals[i] = cand_eval;
}
}
}
let final_pop: Vec<Candidate<Vec<f64>>> = decisions
.into_iter()
.zip(evals)
.map(|(d, e)| Candidate::new(d, e))
.collect();
let best = best_candidate(&final_pop, &objectives);
let front: Vec<Candidate<Vec<f64>>> = best.iter().cloned().collect();
OptimizationResult::new(
Population::new(final_pop),
front,
best,
evaluations,
self.config.generations,
)
}
}
fn best_index(evals: &[Evaluation], direction: Direction) -> usize {
let mut idx = 0;
for i in 1..evals.len() {
+154
View File
@@ -52,6 +52,37 @@ impl Default for TpeConfig {
/// `BayesianOpt`, no GP — TPE models `p(x | y < y*)` and `p(x | y >= y*)`
/// as per-axis Gaussian KDEs and picks the next candidate by maximizing
/// the ratio of the two densities.
///
/// # Example
///
/// ```
/// use heuropt::prelude::*;
///
/// struct Sphere;
/// impl Problem for Sphere {
/// type Decision = Vec<f64>;
/// fn objectives(&self) -> ObjectiveSpace {
/// ObjectiveSpace::new(vec![Objective::minimize("f")])
/// }
/// fn evaluate(&self, x: &Vec<f64>) -> Evaluation {
/// Evaluation::new(vec![x.iter().map(|v| v * v).sum::<f64>()])
/// }
/// }
///
/// let mut opt = Tpe::new(
/// TpeConfig {
/// initial_samples: 10,
/// iterations: 50,
/// good_fraction: 0.25,
/// candidate_samples: 24,
/// bandwidth_factor: 1.0,
/// seed: 42,
/// },
/// RealBounds::new(vec![(-3.0, 3.0); 3]),
/// );
/// let r = opt.run(&Sphere);
/// assert_eq!(r.evaluations, 60);
/// ```
#[derive(Debug, Clone)]
pub struct Tpe {
/// Algorithm configuration.
@@ -325,6 +356,129 @@ fn scott_bandwidths(decisions: &[Vec<f64>], support: &[usize], factor: f64) -> V
.collect()
}
#[cfg(feature = "async")]
impl Tpe {
/// Async version of [`Optimizer::run`] — drives evaluations through
/// the user-chosen async runtime. Available only with the `async`
/// feature.
///
/// `concurrency` bounds in-flight evaluations during the initial
/// uniform-sample design; the sequential TPE loop runs one
/// evaluation per iteration regardless.
pub async fn run_async<P>(
&mut self,
problem: &P,
concurrency: usize,
) -> OptimizationResult<Vec<f64>>
where
P: crate::core::async_problem::AsyncProblem<Decision = Vec<f64>>,
{
use crate::algorithms::parallel_eval_async::evaluate_batch_async;
assert!(
self.config.initial_samples >= 2,
"Tpe initial_samples must be >= 2"
);
assert!(
self.config.good_fraction > 0.0 && self.config.good_fraction < 1.0,
"Tpe good_fraction must be in (0, 1)",
);
assert!(
self.config.candidate_samples >= 1,
"Tpe candidate_samples must be >= 1",
);
assert!(
self.config.bandwidth_factor > 0.0,
"Tpe bandwidth_factor must be > 0"
);
let objectives = problem.objectives();
assert!(
objectives.is_single_objective(),
"Tpe requires exactly one objective",
);
let direction = objectives.objectives[0].direction;
let dim = self.bounds.bounds.len();
let mut rng = rng_from_seed(self.config.seed);
let mut decisions: Vec<Vec<f64>> = Vec::new();
let mut targets: Vec<f64> = Vec::new();
let mut evals: Vec<Evaluation> = Vec::new();
let initial_decisions: Vec<Vec<f64>> = (0..self.config.initial_samples)
.map(|_| sample_uniform_in_bounds(&self.bounds, &mut rng))
.collect();
let initial_cands = evaluate_batch_async(problem, initial_decisions, concurrency).await;
for c in initial_cands {
targets.push(oriented_target(&c.evaluation, direction));
decisions.push(c.decision);
evals.push(c.evaluation);
}
for _ in 0..self.config.iterations {
let (good_idx, bad_idx) = split_good_bad(&targets, self.config.good_fraction);
let mut best_x: Option<Vec<f64>> = None;
let mut best_ratio = f64::NEG_INFINITY;
for _ in 0..self.config.candidate_samples {
let cand = sample_from_kde(
&decisions,
&good_idx,
&self.bounds,
self.config.bandwidth_factor,
&mut rng,
);
let l = log_kde_density(
&cand,
&decisions,
&good_idx,
&self.bounds,
self.config.bandwidth_factor,
);
let g = log_kde_density(
&cand,
&decisions,
&bad_idx,
&self.bounds,
self.config.bandwidth_factor,
);
let ratio = l - g;
if ratio > best_ratio {
best_ratio = ratio;
best_x = Some(cand);
}
}
let x = best_x.expect("at least one candidate sampled");
let _ = dim;
let e = problem.evaluate_async(&x).await;
targets.push(oriented_target(&e, direction));
decisions.push(x);
evals.push(e);
}
let mut best_idx = 0;
for i in 1..evals.len() {
if better(&evals[i], &evals[best_idx], direction) {
best_idx = i;
}
}
let total_evals = evals.len();
let final_pop: Vec<Candidate<Vec<f64>>> = decisions
.into_iter()
.zip(evals)
.map(|(d, e)| Candidate::new(d, e))
.collect();
let best = final_pop[best_idx].clone();
let front = vec![best.clone()];
OptimizationResult::new(
Population::new(final_pop),
front,
Some(best),
total_evals,
self.config.iterations + self.config.initial_samples,
)
}
}
#[cfg(test)]
mod tests {
use super::*;
+147
View File
@@ -49,6 +49,34 @@ impl Default for UmdaConfig {
/// `[1 / (2·selected_size), 1 - 1 / (2·selected_size)]` (Laplace-style
/// smoothing) so the population never collapses to a single deterministic
/// string.
///
/// # Example
///
/// ```
/// use heuropt::prelude::*;
///
/// struct OneMax;
/// impl Problem for OneMax {
/// type Decision = Vec<bool>;
/// fn objectives(&self) -> ObjectiveSpace {
/// ObjectiveSpace::new(vec![Objective::maximize("ones")])
/// }
/// fn evaluate(&self, x: &Vec<bool>) -> Evaluation {
/// Evaluation::new(vec![x.iter().filter(|b| **b).count() as f64])
/// }
/// }
///
/// let mut opt = Umda::new(UmdaConfig {
/// population_size: 50,
/// selected_size: 20,
/// generations: 30,
/// bits: 16,
/// seed: 42,
/// });
/// let r = opt.run(&OneMax);
/// // OneMax with 16 bits: optimum is 16. UMDA should be very close.
/// assert!(r.best.unwrap().evaluation.objectives[0] >= 14.0);
/// ```
#[derive(Debug, Clone)]
pub struct Umda {
/// Algorithm configuration.
@@ -174,6 +202,125 @@ where
}
}
#[cfg(feature = "async")]
impl Umda {
/// Async version of [`Optimizer::run`] — drives evaluations through
/// the user-chosen async runtime. Available only with the `async`
/// feature.
///
/// `concurrency` bounds in-flight evaluations per batch (initial
/// population and per-generation samples).
pub async fn run_async<P>(
&mut self,
problem: &P,
concurrency: usize,
) -> OptimizationResult<Vec<bool>>
where
P: crate::core::async_problem::AsyncProblem<Decision = Vec<bool>>,
{
use crate::algorithms::parallel_eval_async::evaluate_batch_async;
assert!(
self.config.population_size >= 2,
"Umda population_size must be >= 2"
);
assert!(
self.config.selected_size >= 1,
"Umda selected_size must be >= 1",
);
assert!(
self.config.selected_size <= self.config.population_size,
"Umda selected_size must be <= population_size",
);
assert!(self.config.bits >= 1, "Umda bits must be >= 1");
let objectives = problem.objectives();
assert!(
objectives.is_single_objective(),
"Umda requires exactly one objective",
);
let direction = objectives.objectives[0].direction;
let n = self.config.population_size;
let bits = self.config.bits;
let mu = self.config.selected_size;
let mut rng = rng_from_seed(self.config.seed);
let mut decisions: Vec<Vec<bool>> = (0..n)
.map(|_| (0..bits).map(|_| rng.random_bool(0.5)).collect())
.collect();
let mut population = evaluate_batch_async(problem, decisions.clone(), concurrency).await;
let mut evaluations = population.len();
let smoothing = 1.0 / (2.0 * mu as f64);
let prob_min = smoothing;
let prob_max = 1.0 - smoothing;
let mut best_seen: Option<Candidate<Vec<bool>>> = None;
for c in &population {
let beats = match &best_seen {
None => true,
Some(b) => better_than_so(&c.evaluation, &b.evaluation, direction),
};
if beats {
best_seen = Some(c.clone());
}
}
for _ in 0..self.config.generations {
let mut order: Vec<usize> = (0..population.len()).collect();
order.sort_by(|&a, &b| {
compare_so(
&population[a].evaluation,
&population[b].evaluation,
direction,
)
});
let selected: Vec<&Candidate<Vec<bool>>> =
order.iter().take(mu).map(|&i| &population[i]).collect();
let mut probs = vec![0.0_f64; bits];
for c in &selected {
for (i, b) in c.decision.iter().enumerate() {
if *b {
probs[i] += 1.0;
}
}
}
for p in probs.iter_mut() {
*p = (*p / mu as f64).clamp(prob_min, prob_max);
}
decisions = (0..n)
.map(|_| probs.iter().map(|&p| rng.random_bool(p)).collect())
.collect();
population = evaluate_batch_async(problem, decisions.clone(), concurrency).await;
evaluations += population.len();
for c in &population {
let beats = match &best_seen {
None => true,
Some(b) => better_than_so(&c.evaluation, &b.evaluation, direction),
};
if beats {
best_seen = Some(c.clone());
}
}
}
let best = best_seen.expect("at least one generation evaluated");
let final_pop = vec![best.clone()];
let front = vec![best.clone()];
let best_opt = best_candidate(&final_pop, &objectives);
OptimizationResult::new(
Population::new(final_pop),
front,
best_opt,
evaluations,
self.config.generations,
)
}
}
fn compare_so(
a: &crate::core::evaluation::Evaluation,
b: &crate::core::evaluation::Evaluation,
+81
View File
@@ -0,0 +1,81 @@
//! Async-evaluable problems for IO-bound workloads.
//!
//! Most heuropt algorithms operate synchronously: their `Problem::evaluate`
//! returns immediately. For workloads where evaluation is *IO-bound* — calling
//! an HTTP service, querying a remote model, spawning a subprocess —
//! awaiting an async fn is much more efficient than blocking a worker
//! thread.
//!
//! [`AsyncProblem`] mirrors [`Problem`](crate::core::Problem) but its
//! `evaluate_async` returns a future. Every algorithm in heuropt exposes
//! a `run_async` method that drives evaluations through a user-chosen
//! async runtime (typically tokio). Hyperband uses
//! [`AsyncPartialProblem`] instead, which mirrors
//! [`PartialProblem`](crate::core::partial_problem::PartialProblem) for
//! multi-fidelity workloads.
//!
//! Available only with the `async` feature.
use std::future::Future;
use crate::core::evaluation::Evaluation;
use crate::core::objective::ObjectiveSpace;
/// A problem whose evaluation is async — useful when `evaluate` does
/// IO (HTTP, RPC, subprocess) rather than pure CPU work.
///
/// Mirrors [`Problem`](crate::core::Problem) one-for-one except that
/// `evaluate_async` returns a future. The returned future must be
/// `Send` so the algorithm can run many evaluations concurrently
/// across a runtime's worker pool.
///
/// Implementors who already have a synchronous `Problem` can adapt
/// to `AsyncProblem` with a one-line wrapper:
///
/// ```ignore
/// impl AsyncProblem for MyProblem {
/// type Decision = <Self as Problem>::Decision;
/// fn objectives(&self) -> ObjectiveSpace { Problem::objectives(self) }
/// async fn evaluate_async(&self, x: &Self::Decision) -> Evaluation {
/// Problem::evaluate(self, x)
/// }
/// }
/// ```
pub trait AsyncProblem: Sync {
/// The thing the optimizer changes. Same constraints as
/// [`Problem::Decision`](crate::core::Problem::Decision).
type Decision: Clone + Send + Sync;
/// Return the objectives for this problem.
fn objectives(&self) -> ObjectiveSpace;
/// Evaluate `decision` asynchronously. The returned future is
/// driven by whichever runtime the algorithm's `run_async` is
/// invoked from.
fn evaluate_async(&self, decision: &Self::Decision) -> impl Future<Output = Evaluation> + Send;
}
/// Async equivalent of [`PartialProblem`](crate::core::partial_problem::PartialProblem)
/// for multi-fidelity workloads — used by Hyperband's `run_async`.
///
/// Like [`AsyncProblem`], `evaluate_at_budget_async` returns a future
/// so callers can fan out budgeted evaluations across an async runtime.
pub trait AsyncPartialProblem: Sync {
/// The thing the optimizer changes. Same constraints as
/// [`PartialProblem::Decision`](crate::core::partial_problem::PartialProblem::Decision).
type Decision: Clone + Send + Sync;
/// Return the objectives for this problem.
fn objectives(&self) -> ObjectiveSpace;
/// Evaluate `decision` at the given fidelity `budget` asynchronously.
///
/// Same monotonicity contract as
/// [`PartialProblem::evaluate_at_budget`](crate::core::partial_problem::PartialProblem::evaluate_at_budget):
/// higher budget should give a more accurate estimate.
fn evaluate_at_budget_async(
&self,
decision: &Self::Decision,
budget: f64,
) -> impl Future<Output = Evaluation> + Send;
}
+4
View File
@@ -1,5 +1,7 @@
//! Concrete data types and the `Problem` trait that the rest of the crate is built on.
#[cfg(feature = "async")]
pub mod async_problem;
pub mod candidate;
pub mod evaluation;
pub mod objective;
@@ -9,6 +11,8 @@ pub mod problem;
pub mod result;
pub mod rng;
#[cfg(feature = "async")]
pub use async_problem::AsyncProblem;
pub use candidate::*;
pub use evaluation::*;
pub use objective::*;
+36 -9
View File
@@ -1,16 +1,43 @@
//! `heuropt` — a practical Rust toolkit for implementing heuristic
//! single-objective, multi-objective, and many-objective optimization
//! algorithms.
//! `heuropt` — a practical Rust toolkit for heuristic single-,
//! multi-, and many-objective optimization.
//!
//! The crate aims to make three things obvious:
//!
//! 1. Define an optimization problem by implementing [`Problem`](crate::core::Problem).
//! 2. Run a built-in optimizer such as [`Nsga2`](crate::algorithms::Nsga2) or
//! [`RandomSearch`](crate::algorithms::RandomSearch).
//! 3. Implement a new optimizer by implementing
//! [`Optimizer`](crate::traits::Optimizer).
//! 1. **Define a problem** by implementing [`Problem`](crate::core::Problem).
//! 2. **Run a built-in optimizer** — pick from 33 algorithms in
//! [`algorithms`] covering single-objective continuous (CMA-ES,
//! Differential Evolution, Nelder-Mead, …), multi-objective
//! (NSGA-II, MOPSO, IBEA, MOEA/D, …), many-objective (NSGA-III,
//! GrEA, RVEA, …), and sample-efficient regimes (Bayesian
//! Optimization, TPE, Hyperband).
//! 3. **Or implement your own** by implementing
//! [`Optimizer`](crate::traits::Optimizer). The trait is one
//! method long.
//!
//! See `docs/heuropt_tech_design_spec.md` for the full design rationale.
//! ## Where to read more
//!
//! - **User guide / cookbook / comparison vs pymoo & friends:**
//! <https://swaits.github.io/heuropt/>.
//! - **Algorithm selection:** the README's decision tree, or the
//! "Choosing an algorithm" book chapter.
//! - **Design rationale:** `docs/heuropt_tech_design_spec.md` in the
//! repository.
//!
//! ## Optional features
//!
//! - `serde` — derives `Serialize` / `Deserialize` on the core data
//! types ([`Candidate`](crate::core::Candidate),
//! [`Population`](crate::core::Population),
//! [`Evaluation`](crate::core::Evaluation), …).
//! - `parallel` — rayon-backed parallel population evaluation in
//! every population-based algorithm. Seeded runs stay bit-
//! identical to serial mode.
//! - `async` — adds the
//! [`AsyncProblem`](crate::core::async_problem::AsyncProblem) and
//! [`AsyncPartialProblem`](crate::core::async_problem::AsyncPartialProblem)
//! traits and a `run_async(&problem, concurrency).await` method on
//! every algorithm. Use this when your `evaluate` does IO (HTTP,
//! RPC, subprocess) — see the [Async evaluation cookbook recipe](https://swaits.github.io/heuropt/cookbook/async.html).
//!
//! # Quick example
//!
+38
View File
@@ -14,6 +14,26 @@ use crate::core::objective::ObjectiveSpace;
///
/// # Panics
/// If `objectives` does not have exactly two objectives.
///
/// # Example
///
/// ```
/// use heuropt::prelude::*;
/// use heuropt::metrics::hypervolume_2d;
///
/// let space = ObjectiveSpace::new(vec![
/// Objective::minimize("f1"),
/// Objective::minimize("f2"),
/// ]);
/// // Reference (4, 4); front at (1,3), (2,2), (3,1) → dominated area = 6.
/// let front = [
/// Candidate::new((), Evaluation::new(vec![1.0, 3.0])),
/// Candidate::new((), Evaluation::new(vec![2.0, 2.0])),
/// Candidate::new((), Evaluation::new(vec![3.0, 1.0])),
/// ];
/// let hv = hypervolume_2d(&front, &space, [4.0, 4.0]);
/// assert!((hv - 6.0).abs() < 1e-12);
/// ```
pub fn hypervolume_2d<D>(
front: &[Candidate<D>],
objectives: &ObjectiveSpace,
@@ -147,6 +167,24 @@ mod tests {
///
/// # Panics
/// If `objectives.len() != reference_point.len()`, or if either is zero.
///
/// # Example
///
/// ```
/// use heuropt::prelude::*;
/// use heuropt::metrics::hypervolume_nd;
///
/// let space = ObjectiveSpace::new(vec![
/// Objective::minimize("f1"),
/// Objective::minimize("f2"),
/// Objective::minimize("f3"),
/// ]);
/// // Single corner point at the origin against a unit-cube reference:
/// // dominated volume = 1.
/// let front = [Candidate::new((), Evaluation::new(vec![0.0, 0.0, 0.0]))];
/// let hv = hypervolume_nd(&front, &space, &[1.0, 1.0, 1.0]);
/// assert!((hv - 1.0).abs() < 1e-12);
/// ```
pub fn hypervolume_nd<D>(
front: &[Candidate<D>],
objectives: &ObjectiveSpace,
+20
View File
@@ -11,6 +11,26 @@ use crate::core::objective::ObjectiveSpace;
/// uniform front has spacing 0.
///
/// Returns `0.0` for empty or single-point fronts (spec §14.1).
///
/// # Example
///
/// ```
/// use heuropt::prelude::*;
/// use heuropt::metrics::spacing;
///
/// let space = ObjectiveSpace::new(vec![
/// Objective::minimize("f1"),
/// Objective::minimize("f2"),
/// ]);
/// // Five points evenly spaced on a line — spacing should be 0.
/// let front: Vec<Candidate<()>> = (0..5)
/// .map(|i| {
/// let t = i as f64;
/// Candidate::new((), Evaluation::new(vec![t, 4.0 - t]))
/// })
/// .collect();
/// assert!(spacing(&front, &space) < 1e-12);
/// ```
pub fn spacing<D>(front: &[Candidate<D>], objectives: &ObjectiveSpace) -> f64 {
let n = front.len();
if n < 2 {
+13
View File
@@ -9,6 +9,19 @@ use crate::traits::Variation;
///
/// Always returns exactly one child (spec §11.3). Panics if `probability` is
/// outside `[0.0, 1.0]` or if no parents are provided.
///
/// # Example
///
/// ```
/// use heuropt::prelude::*;
///
/// let mut rng = rng_from_seed(42);
/// let mut m = BitFlipMutation { probability: 0.5 };
/// let parent = vec![true, false, true, false];
/// let children = m.vary(std::slice::from_ref(&parent), &mut rng);
/// assert_eq!(children.len(), 1);
/// assert_eq!(children[0].len(), parent.len());
/// ```
#[derive(Debug, Clone)]
pub struct BitFlipMutation {
/// Per-bit flip probability. Must lie in `[0.0, 1.0]`.
+16
View File
@@ -8,6 +8,22 @@ use crate::traits::Variation;
/// Swap two distinct random indices in the first parent (spec §11.4).
///
/// If the parent has length `< 2` the child is returned unchanged.
///
/// # Example
///
/// ```
/// use heuropt::prelude::*;
///
/// let mut rng = rng_from_seed(42);
/// let mut m = SwapMutation;
/// let parent: Vec<usize> = (0..6).collect();
/// let children = m.vary(std::slice::from_ref(&parent), &mut rng);
/// assert_eq!(children.len(), 1);
/// // Still a permutation of [0, 1, 2, 3, 4, 5]:
/// let mut sorted = children[0].clone();
/// sorted.sort();
/// assert_eq!(sorted, vec![0, 1, 2, 3, 4, 5]);
/// ```
#[derive(Debug, Clone, Copy, Default)]
pub struct SwapMutation;
+101
View File
@@ -10,6 +10,23 @@ use crate::traits::{Initializer, Variation};
///
/// Bounds are inclusive `(lo, hi)` ranges per dimension. Panics if any bound
/// has `lo > hi` (spec §11.1).
///
/// # Example
///
/// ```
/// use heuropt::prelude::*;
///
/// let mut rng = rng_from_seed(42);
/// let mut init = RealBounds::new(vec![(-1.0, 1.0); 3]);
/// let decisions = init.initialize(5, &mut rng);
/// assert_eq!(decisions.len(), 5);
/// for d in &decisions {
/// assert_eq!(d.len(), 3);
/// for &v in d {
/// assert!(v >= -1.0 && v <= 1.0);
/// }
/// }
/// ```
#[derive(Debug, Clone)]
pub struct RealBounds {
/// Per-variable inclusive bounds in decision order.
@@ -54,6 +71,19 @@ impl Initializer<Vec<f64>> for RealBounds {
/// Add `Normal(0, sigma)` noise to every variable of the first parent.
///
/// Always returns exactly one child. Does not enforce bounds in v1 (spec §11.2).
///
/// # Example
///
/// ```
/// use heuropt::prelude::*;
///
/// let mut rng = rng_from_seed(42);
/// let mut m = GaussianMutation { sigma: 0.1 };
/// let parent = vec![0.0; 4];
/// let children = m.vary(std::slice::from_ref(&parent), &mut rng);
/// assert_eq!(children.len(), 1);
/// assert_eq!(children[0].len(), parent.len());
/// ```
#[derive(Debug, Clone)]
pub struct GaussianMutation {
/// Standard deviation of the Gaussian noise. Must be positive.
@@ -88,6 +118,26 @@ impl Variation<Vec<f64>> for GaussianMutation {
///
/// Panics on construction if any bound has `lo > hi`, or at run time if
/// `parents.len() < 2` or any parent length differs from `bounds.len()`.
///
/// # Example
///
/// ```
/// use heuropt::prelude::*;
///
/// let bounds = vec![(-1.0, 1.0); 3];
/// let mut sbx = SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5);
/// let mut rng = rng_from_seed(42);
/// let parents = [vec![-0.5, 0.0, 0.5], vec![0.5, 0.5, -0.5]];
/// let children = sbx.vary(&parents, &mut rng);
/// assert_eq!(children.len(), 2);
/// // Children stay in bounds.
/// for c in &children {
/// for (j, &v) in c.iter().enumerate() {
/// let (lo, hi) = bounds[j];
/// assert!(v >= lo && v <= hi);
/// }
/// }
/// ```
#[derive(Debug, Clone)]
pub struct SimulatedBinaryCrossover {
/// Per-variable inclusive bounds. Length must match the parent decisions.
@@ -180,6 +230,23 @@ impl Variation<Vec<f64>> for SimulatedBinaryCrossover {
///
/// This is the simple bound-rescale form; the bound-aware `δ_q` variant from
/// the full paper is left as a future refinement.
///
/// # Example
///
/// ```
/// use heuropt::prelude::*;
///
/// let bounds = vec![(-1.0, 1.0); 3];
/// let mut pm = PolynomialMutation::new(bounds.clone(), 20.0, 1.0 / 3.0);
/// let mut rng = rng_from_seed(42);
/// let parent = vec![0.0, 0.5, -0.5];
/// let children = pm.vary(std::slice::from_ref(&parent), &mut rng);
/// assert_eq!(children.len(), 1);
/// for (j, &v) in children[0].iter().enumerate() {
/// let (lo, hi) = bounds[j];
/// assert!(v >= lo && v <= hi);
/// }
/// ```
#[derive(Debug, Clone)]
pub struct PolynomialMutation {
/// Per-variable inclusive bounds. Length must match the parent decision.
@@ -254,6 +321,23 @@ impl Variation<Vec<f64>> for PolynomialMutation {
/// Always returns exactly one child. Use this when you want feasibility
/// maintained across generations without leaning on
/// clamp-inside-`Problem::evaluate`.
///
/// # Example
///
/// ```
/// use heuropt::prelude::*;
///
/// let bounds = vec![(-1.0, 1.0); 3];
/// let mut m = BoundedGaussianMutation::new(0.3, bounds.clone());
/// let mut rng = rng_from_seed(42);
/// let parent = vec![0.0; 3];
/// let children = m.vary(std::slice::from_ref(&parent), &mut rng);
/// assert_eq!(children.len(), 1);
/// for (j, &v) in children[0].iter().enumerate() {
/// let (lo, hi) = bounds[j];
/// assert!(v >= lo && v <= hi);
/// }
/// ```
#[derive(Debug, Clone)]
pub struct BoundedGaussianMutation {
/// Standard deviation of the Gaussian noise. Must be positive.
@@ -315,6 +399,23 @@ impl Variation<Vec<f64>> for BoundedGaussianMutation {
/// produce a Lévy(α) sample. `alpha` is the tail exponent in `(0, 2]`;
/// typical value is `1.5`. `1.0` gives the Cauchy distribution (very
/// heavy); `2.0` collapses to the Normal.
///
/// # Example
///
/// ```
/// use heuropt::prelude::*;
///
/// let bounds = vec![(-1.0, 1.0); 3];
/// let mut m = LevyMutation::new(1.5, 0.1, bounds.clone());
/// let mut rng = rng_from_seed(42);
/// let parent = vec![0.0; 3];
/// let children = m.vary(std::slice::from_ref(&parent), &mut rng);
/// assert_eq!(children.len(), 1);
/// for (j, &v) in children[0].iter().enumerate() {
/// let (lo, hi) = bounds[j];
/// assert!(v >= lo && v <= hi);
/// }
/// ```
#[derive(Debug, Clone)]
pub struct LevyMutation {
/// Tail exponent `α ∈ (0, 2]`. Smaller = heavier tail.
+24
View File
@@ -8,6 +8,17 @@ use crate::traits::Repair;
/// The simplest possible repair — pair with `GaussianMutation` (which
/// doesn't enforce bounds in v1) to produce a bounds-respecting variant
/// without writing a custom Variation impl.
///
/// # Example
///
/// ```
/// use heuropt::prelude::*;
///
/// let mut r = ClampToBounds::new(vec![(-1.0, 1.0); 3]);
/// let mut x = vec![-2.0, 0.5, 5.0];
/// r.repair(&mut x);
/// assert_eq!(x, vec![-1.0, 0.5, 1.0]);
/// ```
#[derive(Debug, Clone)]
pub struct ClampToBounds {
/// Per-variable inclusive bounds.
@@ -46,6 +57,19 @@ impl Repair<Vec<f64>> for ClampToBounds {
/// Perpiñán 2013. Useful for portfolio-style problems where the
/// decision must sum to a budget, and for normalizing reference
/// directions onto the unit simplex.
///
/// # Example
///
/// ```
/// use heuropt::prelude::*;
///
/// let mut r = ProjectToSimplex::new(1.0);
/// let mut x = vec![0.6, 0.5, -0.1, 0.3];
/// r.repair(&mut x);
/// let sum: f64 = x.iter().sum();
/// assert!((sum - 1.0).abs() < 1e-12);
/// assert!(x.iter().all(|&v| v >= 0.0));
/// ```
#[derive(Debug, Clone)]
pub struct ProjectToSimplex {
/// Target sum (the simplex's "size"). Standard probability simplex
+17
View File
@@ -9,6 +9,23 @@ use crate::core::objective::ObjectiveSpace;
/// archive insert/extend operations maintain the non-domination property among
/// members; `truncate` enforces a maximum size by simple tail-truncation in
/// v1.
///
/// # Example
///
/// ```
/// use heuropt::prelude::*;
///
/// let s = ObjectiveSpace::new(vec![
/// Objective::minimize("f1"),
/// Objective::minimize("f2"),
/// ]);
/// let mut a: ParetoArchive<u32> = ParetoArchive::new(s);
/// a.insert(Candidate::new(1, Evaluation::new(vec![1.0, 4.0])));
/// a.insert(Candidate::new(2, Evaluation::new(vec![3.0, 2.0])));
/// // Dominated by both — should be discarded:
/// a.insert(Candidate::new(3, Evaluation::new(vec![5.0, 5.0])));
/// assert_eq!(a.members().len(), 2);
/// ```
#[derive(Debug, Clone)]
pub struct ParetoArchive<D> {
/// The current approximate non-dominated set.
+22
View File
@@ -11,6 +11,28 @@ use crate::core::objective::ObjectiveSpace;
/// `f64::INFINITY`. If the front has 0 entries an empty vector is returned;
/// 1 or 2 entries return all `f64::INFINITY`. All comparisons happen on
/// minimization-oriented objective values (spec §9.6).
///
/// # Example
///
/// ```
/// use heuropt::prelude::*;
///
/// let s = ObjectiveSpace::new(vec![
/// Objective::minimize("f1"),
/// Objective::minimize("f2"),
/// ]);
/// // Three points along a Pareto-like trade-off; the interior point gets
/// // a finite crowding distance, the boundaries get +∞.
/// let pop = [
/// Candidate::new((), Evaluation::new(vec![0.0, 4.0])),
/// Candidate::new((), Evaluation::new(vec![2.0, 2.0])),
/// Candidate::new((), Evaluation::new(vec![4.0, 0.0])),
/// ];
/// let d = crowding_distance(&pop, &[0, 1, 2], &s);
/// assert!(d[0].is_infinite());
/// assert!(d[1].is_finite() && d[1] > 0.0);
/// assert!(d[2].is_infinite());
/// ```
pub fn crowding_distance<D>(
population: &[Candidate<D>],
front: &[usize],
+15
View File
@@ -29,6 +29,21 @@ pub enum Dominance {
/// `constraint_violation` dominates.
/// 3. Otherwise compare objective values after converting both to
/// minimization orientation via [`ObjectiveSpace::as_minimization`].
///
/// # Example
///
/// ```
/// use heuropt::prelude::*;
///
/// let s = ObjectiveSpace::new(vec![
/// Objective::minimize("f1"),
/// Objective::minimize("f2"),
/// ]);
/// let a = Evaluation::new(vec![1.0, 1.0]);
/// let b = Evaluation::new(vec![2.0, 2.0]);
/// assert_eq!(pareto_compare(&a, &b, &s), Dominance::Dominates);
/// assert_eq!(pareto_compare(&b, &a, &s), Dominance::DominatedBy);
/// ```
pub fn pareto_compare(a: &Evaluation, b: &Evaluation, objectives: &ObjectiveSpace) -> Dominance {
let a_feasible = a.is_feasible();
let b_feasible = b.is_feasible();
+34
View File
@@ -8,6 +8,25 @@ use crate::pareto::dominance::{Dominance, pareto_compare};
///
/// O(N²·M) in v1 (spec §9.3). Input order is preserved among returned
/// candidates.
///
/// # Example
///
/// ```
/// use heuropt::prelude::*;
///
/// let s = ObjectiveSpace::new(vec![
/// Objective::minimize("f1"),
/// Objective::minimize("f2"),
/// ]);
/// let pop = [
/// Candidate::new(1u32, Evaluation::new(vec![1.0, 4.0])), // non-dominated
/// Candidate::new(2u32, Evaluation::new(vec![3.0, 2.0])), // non-dominated
/// Candidate::new(3u32, Evaluation::new(vec![5.0, 5.0])), // dominated
/// ];
/// let front = pareto_front(&pop, &s);
/// let kept: Vec<u32> = front.iter().map(|c| c.decision).collect();
/// assert_eq!(kept, vec![1, 2]);
/// ```
pub fn pareto_front<D: Clone>(
population: &[Candidate<D>],
objectives: &ObjectiveSpace,
@@ -34,6 +53,21 @@ pub fn pareto_front<D: Clone>(
///
/// Returns `None` if there is not exactly one objective, if the population is
/// empty, or if every candidate is infeasible (spec §9.4).
///
/// # Example
///
/// ```
/// use heuropt::prelude::*;
///
/// let s = ObjectiveSpace::new(vec![Objective::minimize("f")]);
/// let pop = [
/// Candidate::new(1u32, Evaluation::new(vec![3.0])),
/// Candidate::new(2u32, Evaluation::new(vec![1.0])),
/// Candidate::new(3u32, Evaluation::new(vec![2.0])),
/// ];
/// let best = best_candidate(&pop, &s).unwrap();
/// assert_eq!(best.decision, 2);
/// ```
pub fn best_candidate<D: Clone>(
population: &[Candidate<D>],
objectives: &ObjectiveSpace,
+15
View File
@@ -10,6 +10,21 @@
///
/// # Panics
/// If `num_objectives == 0`.
///
/// # Example
///
/// ```
/// use heuropt::prelude::*;
///
/// // 3 objectives, 4 divisions → binomial(6, 2) = 15 points.
/// let pts = das_dennis(3, 4);
/// assert_eq!(pts.len(), 15);
/// for w in &pts {
/// assert_eq!(w.len(), 3);
/// let sum: f64 = w.iter().sum();
/// assert!((sum - 1.0).abs() < 1e-12);
/// }
/// ```
pub fn das_dennis(num_objectives: usize, divisions: usize) -> Vec<Vec<f64>> {
assert!(
num_objectives > 0,
+20
View File
@@ -9,6 +9,26 @@ use crate::core::objective::ObjectiveSpace;
/// non-dominated after removing `fronts[0]`, and so on. Each entry is an index
/// into the input population. Equal-objective candidates land on the same
/// front. O(N²·M) is acceptable for v1 (spec §9.5).
///
/// # Example
///
/// ```
/// use heuropt::prelude::*;
///
/// let s = ObjectiveSpace::new(vec![
/// Objective::minimize("f1"),
/// Objective::minimize("f2"),
/// ]);
/// let pop = [
/// Candidate::new((), Evaluation::new(vec![1.0, 5.0])), // front 0
/// Candidate::new((), Evaluation::new(vec![2.0, 3.0])), // front 0
/// Candidate::new((), Evaluation::new(vec![4.0, 1.0])), // front 0
/// Candidate::new((), Evaluation::new(vec![3.0, 4.0])), // front 1
/// Candidate::new((), Evaluation::new(vec![5.0, 6.0])), // front 2
/// ];
/// let fronts = non_dominated_sort(&pop, &s);
/// assert_eq!(fronts.len(), 3);
/// ```
pub fn non_dominated_sort<D>(
population: &[Candidate<D>],
objectives: &ObjectiveSpace,
+2
View File
@@ -4,6 +4,8 @@
//! use heuropt::prelude::*;
//! ```
#[cfg(feature = "async")]
pub use crate::core::async_problem::AsyncProblem;
pub use crate::core::{
Candidate, Direction, Evaluation, Objective, ObjectiveSpace, OptimizationResult,
PartialProblem, Population, Problem, Rng, rng_from_seed,