diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md new file mode 100644 index 0000000..e704ccf --- /dev/null +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -0,0 +1,40 @@ +--- +name: Bug report +about: A correctness, performance, or panic bug in heuropt +title: "bug: " +labels: bug +--- + +## What happened + + + +## Reproducer + +```rust +// Smallest example that demonstrates the bug. Ideally <30 lines and +// runnable as a fresh `examples/repro.rs`. Include the Cargo.toml +// `[features]` you used. +``` + +Command used: + +```sh +cargo run --release --example repro +``` + +## Expected vs observed + +- **Expected:** +- **Observed:** + +## Environment + +- heuropt version: +- `rustc --version`: +- OS / arch: +- Feature flags enabled: + +## Additional context + + diff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.yml new file mode 100644 index 0000000..927a026 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/config.yml @@ -0,0 +1,8 @@ +blank_issues_enabled: false +contact_links: + - name: Security vulnerability + url: https://github.com/swaits/heuropt/security/advisories/new + about: Please use private vulnerability reporting — do not open a public issue. See SECURITY.md. + - name: Question / discussion + url: https://github.com/swaits/heuropt/discussions + about: For open-ended questions or design discussions. diff --git a/.github/ISSUE_TEMPLATE/docs.md b/.github/ISSUE_TEMPLATE/docs.md new file mode 100644 index 0000000..c0d1889 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/docs.md @@ -0,0 +1,24 @@ +--- +name: Docs issue +about: Something in the README, mdbook guide, or rustdoc is wrong, missing, or unclear +title: "docs: " +labels: documentation +--- + +## Where + +- [ ] `README.md` +- [ ] mdbook user guide (chapter / section: ____ ) +- [ ] rustdoc on a specific item (path: ____ ) +- [ ] Examples (`examples/____.rs`) +- [ ] CHANGELOG / migration guide +- [ ] Other: ____ + +## What's wrong + + + +## What it should say (if you know) + + diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md new file mode 100644 index 0000000..eddd16b --- /dev/null +++ b/.github/ISSUE_TEMPLATE/feature_request.md @@ -0,0 +1,39 @@ +--- +name: Feature request +about: Propose a new algorithm, operator, metric, or API addition +title: "feat: " +labels: enhancement +--- + +## What and why + + + +## Proposed API sketch + +```rust +// What the public surface would look like — config struct fields, +// trait impl, etc. Doesn't need to be final, just enough to discuss. +``` + +## Alternatives considered + + + +## Scope + +- [ ] New trait (will need API discussion) +- [ ] New algorithm +- [ ] New operator +- [ ] New metric / Pareto utility +- [ ] New optional feature flag +- [ ] Change to existing public API (potentially breaking) + +## Willing to implement? + +- [ ] Yes, I'll send a PR. +- [ ] Yes, but I'd like guidance on the design first. +- [ ] No, I'm reporting the need. diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md new file mode 100644 index 0000000..3157d57 --- /dev/null +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -0,0 +1,32 @@ + + +## What + + + +## Why + + + +## Checklist + +- [ ] `cargo fmt --all` +- [ ] `cargo clippy --all-targets --all-features -- -D warnings` +- [ ] `cargo test` and `cargo test --all-features` +- [ ] `cargo doc --no-deps --all-features` (with `-D warnings`) +- [ ] Conventional-commit subject(s) (`(): `) +- [ ] If touching algorithm output: confirmed bit-identical results + via `cargo run --release --example compare` +- [ ] If perf change: included gungraun before/after numbers in the + commit message +- [ ] Updated CHANGELOG.md under `[Unreleased]` if user-visible + +## Anything else + + diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml new file mode 100644 index 0000000..a40bf08 --- /dev/null +++ b/.github/workflows/docs.yml @@ -0,0 +1,48 @@ +name: Docs + +on: + push: + branches: [main] + tags: ["v*.*.*"] + workflow_dispatch: + +permissions: + contents: read + pages: write + id-token: write + +concurrency: + group: pages + cancel-in-progress: false + +jobs: + build: + name: Build mdbook + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - name: Install mdbook + run: | + mkdir -p ~/.local/bin + curl -sSL "https://github.com/rust-lang/mdBook/releases/download/v0.4.40/mdbook-v0.4.40-x86_64-unknown-linux-musl.tar.gz" \ + | tar -xz -C ~/.local/bin + echo "$HOME/.local/bin" >> "$GITHUB_PATH" + - name: Build + run: | + cd docs/book + mdbook build + - uses: actions/configure-pages@v5 + - uses: actions/upload-pages-artifact@v3 + with: + path: target/book + + deploy: + name: Deploy to GitHub Pages + needs: build + runs-on: ubuntu-latest + environment: + name: github-pages + url: ${{ steps.deployment.outputs.page_url }} + steps: + - id: deployment + uses: actions/deploy-pages@v4 diff --git a/CHANGELOG.md b/CHANGELOG.md index 378ac2b..f292979 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,92 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +## [0.5.0] — 2026-05-05 + +Theme: comprehensive documentation and project polish. No public-API +changes — bumping `heuropt = "0.5"` in your `Cargo.toml` is enough. + +### Added + +#### User guide (mdbook) + +A new mdbook user guide at `docs/book/`, deployed to + via a CI workflow on tag pushes. +Chapters: + +- **Introduction** — what heuropt is, who it's for, what's in the box. +- **Five-minute walkthrough** — install, define a problem, run an + optimizer, look at the result. +- **Defining a problem** — the `Problem` trait in depth: single- vs + multi-objective, constraints, custom decision types + (`Vec`, `Vec`, `Vec`, custom structs). +- **Choosing an algorithm** — the README's decision tree, expanded + to a full chapter with the reasoning behind every branch. +- **Cookbook** — seven recipes covering parallelism, expensive + evaluations, comparison harnesses, permutation problems, + constraint repair, picking one answer off a Pareto front, and + writing your own optimizer. +- **Comparison with other libraries** — heuropt vs pymoo, hyperopt, + optuna, MOEA Framework, metaheuristics-rs, argmin. Honest about + when *not* to pick heuropt. +- **Stability and SemVer** — explicit guarantees about which surfaces + are stable; what's likely to change before 1.0; bit-identical + determinism contract. +- **Migration guides** — per-release upgrade notes. + +#### Runnable rustdoc examples + +Every algorithm now has a runnable ` ```rust ` example block in its +rustdoc — 35 algorithms, all exercised by `cargo test --doc`. Plus +the existing crate-level example in `lib.rs` and the +`CompositeVariation` operator example. + +#### Real-world examples + +Three new polished examples covering distinct domains: + +- `examples/portfolio.rs` — multi-objective portfolio optimization + with budget constraint via `ProjectToSimplex`. Pareto front of + return-vs-risk trade-offs, plus a-posteriori weighted decision. +- `examples/hyperparam_tuning.rs` — sample-efficient hyperparameter + tuning with `BayesianOpt` and `Tpe`, demonstrating mixed-scale + decoding (log-uniform learning rate, integer depth) and a 60-eval + budget. +- `examples/scheduling.rs` — single-machine weighted-completion-time + scheduling: permutation decisions optimized via + `SimulatedAnnealing` + `SwapMutation`, comparing against the + Smith's-rule oracle. + +#### Governance docs + +- `CONTRIBUTING.md` — local-test checklist, conventional-commits + requirement, contribution areas that land easily vs. those that + need prior discussion. +- `SECURITY.md` — disclosure policy, supported versions, what counts + as a security issue. +- `CODE_OF_CONDUCT.md` — adopts the + [Builder's Code of Conduct](https://builderscode.org/) (CC0). +- `.github/ISSUE_TEMPLATE/` — bug, feature, docs templates plus a + `config.yml` that points security reports to the private + vulnerability-disclosure flow. +- `.github/PULL_REQUEST_TEMPLATE.md` — short, opinionated PR + template. + +#### CI / tooling + +- `.github/workflows/docs.yml` — builds the mdbook user guide and + deploys it to GitHub Pages on `main` pushes and tag pushes. + +### Changed + +- README hero block expanded with badges and a punchier opening; + added explicit links to the user guide, the docs.rs API reference, + and the testing-coverage breakdown. +- `lib.rs` crate-level docs polished — better intro, points readers + at the user guide and the design spec. + +[0.5.0]: https://github.com/swaits/heuropt/releases/tag/v0.5.0 + ## [0.4.0] — 2026-05-05 Theme: testing infrastructure, two real bug fixes surfaced by that @@ -383,5 +469,5 @@ Initial release. `RandomSearch`, `Nsga2`, and `DifferentialEvolution`. Seeded runs stay bit-identical to serial mode. -[Unreleased]: https://github.com/swaits/heuropt/compare/v0.4.0...HEAD +[Unreleased]: https://github.com/swaits/heuropt/compare/v0.5.0...HEAD [0.1.0]: https://github.com/swaits/heuropt/releases/tag/v0.1.0 diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md new file mode 100644 index 0000000..2ca1028 --- /dev/null +++ b/CODE_OF_CONDUCT.md @@ -0,0 +1,43 @@ +# Code of Conduct + +heuropt adopts the [Builder's Code of Conduct](https://builderscode.org/), +version 1.0. + +A Code of Conduct for people who build things. + +## The Rule + +> "Stay professional. Stay technical." + +## Expected + +- Contribute constructively. +- Respect others' time and work. +- Focus on the work and its technical merit. + +## Not Welcome + +- Harassment, name-calling, or personal attacks. +- Trolling, spamming, or derailing discussions. +- Discussions about contributors rather than their contributions. + +## Enforcement + +Violations result in: + +1. **Warning** — first offense. +2. **Temporary suspension** — repeated or serious violations. +3. **Permanent ban** — continued violations. + +Maintainers can remove, block, or ban anyone who disrupts the project. + +## Reporting + +Email **steve@waits.net** with `[heuropt CoC]` in the subject line. +Reports are handled confidentially. + +--- + +The Builder's Code of Conduct is dedicated to the public domain under +CC0 1.0 Universal. You may use, modify, and distribute it freely +without attribution. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000..5964d14 --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,117 @@ +# Contributing to heuropt + +Thanks for considering a contribution. heuropt is a small, opinionated +crate, but careful additions are welcome. + +## Quick checklist + +Before opening a pull request: + +- [ ] `cargo fmt --all` +- [ ] `cargo clippy --all-targets --all-features -- -D warnings` +- [ ] `cargo test` (default features) and `cargo test --all-features` +- [ ] `cargo doc --no-deps --all-features` with `RUSTDOCFLAGS="-D warnings"` +- [ ] If you touched algorithm output: re-run `cargo run --release --example compare` + and confirm the quality metrics did not change. Speed-only changes + are required to be **bit-identical** against the prior snapshot. + +CI runs all of the above on every PR; the matrix covers MSRV (1.85), +the default / serde / parallel / serde+parallel feature combinations, +and a 60-second fuzz soak per target. + +## Commit style + +Conventional Commits (https://www.conventionalcommits.org/) are +required. The first line follows `(): ` where +`` is one of `feat`, `fix`, `perf`, `refactor`, `docs`, `test`, +`chore`, `ci`, `build`, `style`. `` is the most specific module +the change touches (e.g. `nsga2`, `hypervolume`, `pareto_archive`). + +Bad: `Phase 1.1: Add core data types` +Good: `feat(core): add data types and Rng alias` + +Multiple logical changes in a single PR should be split into multiple +commits, each on a single concern. + +## What kinds of contributions land easily + +- **Bug fixes** with a regression test that fails on `main` and passes + on the fix. +- **Performance wins** that preserve bit-identical output and include + a `cargo bench` (gungraun) before/after, plus a `cargo run --release + --example compare` diff confirming no quality regression. +- **Documentation improvements** — missing rustdoc examples, README + clarifications, mdbook chapters. +- **New algorithms** that fit the established `Optimizer

` shape and + ship with: a unit test, a property test (determinism + invariants), + a comparison-harness entry, and rustdoc. +- **New operators / metrics / Pareto utilities** with the same + hygiene. + +## What needs prior discussion + +Open an issue before starting on: + +- New traits or breaking changes to the public API surface. +- A new optional feature flag. +- Anything that depends on a heavy new dependency. +- Restructuring of `src/algorithms/` or `src/pareto/`. + +The crate intentionally keeps the trait surface small (`Problem`, +`Optimizer`, `Initializer`, `Variation`, `Repair`); changes there +are not refused but they need a clear motivation. + +## Running the test suites locally + +```sh +# unit + integration + property tests +cargo test + +# all feature combinations +cargo test --features serde +cargo test --features parallel +cargo test --all-features + +# instruction-count benchmarks (needs valgrind installed) +cargo bench + +# coverage-guided fuzzing (needs nightly + cargo-fuzz) +cd fuzz +cargo +nightly fuzz run pareto_compare -- -max_total_time=60 + +# mutation testing (slow, optional) +cargo install cargo-mutants +cargo mutants +``` + +## Reporting bugs + +Please include: + +1. The smallest reproducing input you can produce — ideally a 20-line + `examples/repro.rs`. +2. The exact command (`cargo run --release --example repro` etc.) and + the observed vs expected output. +3. The Rust toolchain (`rustc --version`) and feature flags. +4. The heuropt version you saw the bug on. + +Bugs that surface fuzz-target panics are particularly welcome; please +attach the failing artifact (`fuzz/artifacts//crash-...`) so +we can add it to the regression-test corpus. + +## Security + +For security concerns please follow the disclosure policy in +[SECURITY.md](SECURITY.md). Don't open public issues for security +bugs. + +## Code of conduct + +This project follows the [Builder's Code of Conduct](CODE_OF_CONDUCT.md). +The short version: stay professional, stay technical, focus on the +work and its merit. + +## License + +By submitting a contribution, you agree that your work is licensed +under the same MIT license as the rest of heuropt. diff --git a/Cargo.toml b/Cargo.toml index 3b28b1c..7ef1879 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "heuropt" -version = "0.4.0" +version = "0.5.0" edition = "2024" rust-version = "1.85" authors = ["Stephen Waits "] diff --git a/README.md b/README.md index 33be31d..6ebd745 100644 --- a/README.md +++ b/README.md @@ -2,28 +2,39 @@ [![Crates.io](https://img.shields.io/crates/v/heuropt.svg)](https://crates.io/crates/heuropt) [![Documentation](https://docs.rs/heuropt/badge.svg)](https://docs.rs/heuropt) +[![Book](https://img.shields.io/badge/book-online-blue.svg)](https://swaits.github.io/heuropt/) [![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE) +[![CI](https://github.com/swaits/heuropt/actions/workflows/ci.yml/badge.svg)](https://github.com/swaits/heuropt/actions/workflows/ci.yml) -A practical Rust toolkit for implementing heuristic single-objective, -multi-objective, and many-objective optimization algorithms. +**A practical Rust toolkit for heuristic optimization.** Single-objective. +Multi-objective. Many-objective. 35 algorithms. One small set of traits. +Bit-identical seeded determinism. No trait objects, no GATs, no generic-RNG +plumbing in the public API. -`heuropt` is **not** a research framework full of abstract machinery — it is a -small set of concrete types, a handful of simple traits, and a few reference -algorithms. The goal: an entry-level Rust engineer can define a problem, run a -built-in optimizer, or implement a new optimizer without learning any -framework concepts. +If you can write a `Problem` impl and read `RandomSearch`, you can write your +own optimizer. That's the whole pitch. + +- 📖 **Read the [user guide](https://swaits.github.io/heuropt/)** for tutorials, + cookbook recipes, comparison with pymoo / hyperopt / MOEA Framework, and + stability policy. +- 🔧 **[API reference on docs.rs](https://docs.rs/heuropt)** has runnable + ` ```rust ` examples on every algorithm. +- 🧪 Tested with **316+ unit / integration / property tests** plus 8 + cargo-fuzz targets running on every PR. +- ⚡ Hot paths heavily optimized — comparison harness 3.27× faster as of + v0.4.0, all bit-identical to the reference output. ## Installation ```toml [dependencies] -heuropt = "0.3" +heuropt = "0.5" # Optional features: # - "serde": derive Serialize/Deserialize on the core data types. # - "parallel": evaluate populations across rayon's thread pool. # Seeded runs stay bit-identical to serial mode. -# heuropt = { version = "0.3", features = ["serde", "parallel"] } +# heuropt = { version = "0.5", features = ["serde", "parallel"] } ``` ## Define a problem @@ -504,6 +515,16 @@ heuropt is exhaustively tested across several layers: - **CI** (`.github/workflows/ci.yml`) — fmt, clippy (`-D warnings`), test (4-feature matrix), doc, MSRV (1.85), fuzz. +## Contributing + +See [CONTRIBUTING.md](CONTRIBUTING.md) for the local-test checklist, +conventional-commits requirement, and project-governance docs. + +This project follows the [Builder's Code of Conduct](CODE_OF_CONDUCT.md): +stay professional, stay technical, focus on the work and its merit. + +For security disclosures, see [SECURITY.md](SECURITY.md). + ## License MIT — see [LICENSE](LICENSE). diff --git a/SECURITY.md b/SECURITY.md new file mode 100644 index 0000000..55b5072 --- /dev/null +++ b/SECURITY.md @@ -0,0 +1,62 @@ +# Security policy + +## Supported versions + +Security fixes are applied to the latest released minor version on +crates.io. Patch-level releases (`0.x.y` → `0.x.y+1`) are issued as +needed. + +| Version | Supported | +|---------|--------------------| +| 0.5.x | ✅ | +| ≤ 0.4.x | ❌ (please upgrade) | + +heuropt is pre-1.0; the public API may change between minor versions. +Once 1.0.0 ships, the support window will be at least the latest two +minor versions. + +## Reporting a vulnerability + +Please **do not** open a public GitHub issue for a security bug. +Instead use one of these channels: + +- GitHub's [private vulnerability reporting](https://github.com/swaits/heuropt/security/advisories/new) + on the repository. +- Email **steve@waits.net** with subject line `[heuropt security] + `. + +Please include: + +1. A description of the vulnerability and the affected versions. +2. The smallest reproducer you can produce — a `cargo run --example + repro` is ideal. +3. Your assessment of impact and exploitability. +4. Any suggested mitigation if you have one. + +## What I will do + +- Acknowledge the report within **72 hours**. +- Confirm or refute reproducibility within **7 days**. +- Issue a fix in a patch release within **30 days** for confirmed + high-severity issues; less urgent issues may roll into the next + minor release. +- Credit the reporter in the CHANGELOG entry unless you ask + otherwise. + +## What counts as a security issue + +heuropt is a numerical library, not a network service or sandbox. The +realistic security-relevant categories are: + +- **Memory safety**: any unsafe-code-related UB or unwinds-across-FFI + bug. heuropt itself uses no `unsafe`; this category covers + dependencies it transitively pulls in. +- **Denial of service**: an input to a public API that causes + unbounded memory growth, infinite loop, or panic outside its + documented panic conditions. (Documented panics for invalid config + are not bugs.) +- **Supply-chain compromise**: a published heuropt crate that doesn't + match the source on the tagged commit. + +Functional correctness bugs (an algorithm produces wrong +hypervolumes, etc.) are tracked as ordinary issues, not security. diff --git a/docs/book/book.toml b/docs/book/book.toml new file mode 100644 index 0000000..8c81c09 --- /dev/null +++ b/docs/book/book.toml @@ -0,0 +1,34 @@ +[book] +title = "heuropt — the user guide" +description = "A practical Rust toolkit for heuristic single-, multi-, and many-objective optimization." +authors = ["Stephen Waits"] +language = "en" +src = "src" + +[build] +build-dir = "../../target/book" +create-missing = false + +[output.html] +default-theme = "rust" +preferred-dark-theme = "navy" +git-repository-url = "https://github.com/swaits/heuropt" +edit-url-template = "https://github.com/swaits/heuropt/edit/main/docs/book/{path}" +site-url = "/heuropt/" +no-section-label = true + +[output.html.fold] +enable = true +level = 1 + +[output.html.search] +enable = true +limit-results = 30 +teaser-word-count = 30 +use-boolean-and = true + +[output.html.print] +enable = true + +[rust] +edition = "2024" diff --git a/docs/book/src/SUMMARY.md b/docs/book/src/SUMMARY.md new file mode 100644 index 0000000..8ccd16d --- /dev/null +++ b/docs/book/src/SUMMARY.md @@ -0,0 +1,26 @@ +# Summary + +[Introduction](./introduction.md) + +# Getting started + +- [Five-minute walkthrough](./getting-started.md) +- [Defining a problem](./defining-problems.md) +- [Choosing an algorithm](./choosing-an-algorithm.md) + +# Cookbook + +- [Recipes](./cookbook.md) + - [Parallelize evaluation with rayon](./cookbook/parallel.md) + - [Tune a model with expensive evaluations](./cookbook/expensive-evaluations.md) + - [Compare two algorithms on your problem](./cookbook/compare.md) + - [Optimize a permutation (TSP-style)](./cookbook/permutation.md) + - [Constrain your search with `Repair`](./cookbook/constraints.md) + - [Pick one answer off a Pareto front](./cookbook/pick-one.md) + - [Write your own algorithm](./cookbook/custom-optimizer.md) + +# Reference + +- [Comparison with other libraries](./comparison.md) +- [Stability and SemVer](./stability.md) +- [Migration guides](./migration.md) diff --git a/docs/book/src/choosing-an-algorithm.md b/docs/book/src/choosing-an-algorithm.md new file mode 100644 index 0000000..0437320 --- /dev/null +++ b/docs/book/src/choosing-an-algorithm.md @@ -0,0 +1,285 @@ +# Choosing an algorithm + +The README has a compact decision tree. This chapter expands it with +the *reasoning* behind each branch. + +## Step 0: How expensive is one evaluation? + +This is the first fork because it changes everything that comes +after it. + +| Eval cost | Budget you can afford | Algorithm family | +|----------------------------|---------------------------|-----------------------------| +| Microseconds (pure math) | 10 000 – 1 000 000 evals | Population-based | +| Milliseconds (sim, IO) | 1 000 – 10 000 evals | Population-based | +| Seconds (small training) | 100 – 1 000 evals | Sample-efficient (BO, TPE) | +| Minutes+ (full training) | 50 – 500 evals | Sample-efficient + multi-fidelity | + +For the cheap-eval branch, you have the run of the catalog. For the +expensive branch, classical evolutionary methods waste your evaluation +budget — go to [`BayesianOpt`] or [`Tpe`]. For the *very* expensive +branch where each eval has a tunable budget (epochs, MC samples, sim +steps), [`Hyperband`] over the [`PartialProblem`] trait is the move. + +## Step 1: How many objectives? + +The biggest fork. + +- **One** — there's a single best answer. Pick from the + single-objective branch. +- **Two or three** — a Pareto front. Pick from the multi-objective + branch. +- **Four or more** — a many-objective Pareto front; classical + multi-objective methods break down here because almost every pair + of points is non-dominated. Pick from the many-objective branch. + +> **Pareto front:** the set of decisions where you cannot improve any +> objective without sacrificing another. In a 2-objective minimize +> problem, plot every solution; the Pareto front is the lower-left +> envelope. + +If you found yourself staring at a single composite score that's a +weighted sum of conflicting goals, you probably have a multi-objective +problem in disguise. A weighted sum bakes in your preferences before +you've seen the trade-off; running a multi-objective optimizer first +and picking off the front later is almost always a better workflow +(see [Pick one answer off a Pareto front](./cookbook/pick-one.md)). + +## Step 2 — single-objective continuous + +These all take `Vec` decisions. + +### Smooth, low-to-moderate dimension + +[`CmaEs`] is the strong default. It adapts the search distribution's +covariance to the local landscape. On the comparison harness it +hits machine epsilon on Rosenbrock at 30 000 evaluations. + +For very low-dimensional smooth problems (≤ 5 dim), [`NelderMead`] is +deterministic and converges to f = 0 exactly on Rosenbrock. + +### High dimension, smooth + +[`SeparableNes`] uses a diagonal covariance — cheaper per step than +CmaEs at the cost of being unable to model rotated landscapes. Worth +trying when CmaEs's `O(d²)` per-step cost hurts. + +### Multimodal landscapes + +Multimodal = many local minima that aren't the global one. Rastrigin +and Ackley are classic traps. + +[`IpopCmaEs`] is CmaEs with an increasing-population restart strategy +specifically designed for this. On the harness it drops vanilla CmaEs's +Rastrigin score from f = 2.35 to f = 0.13. + +[`DifferentialEvolution`] is rarely beaten on cheap multimodal +continuous problems. On Rastrigin it ties with `(1+1)-ES` at f = 0. + +[`SimulatedAnnealing`] is a cheap, generic baseline that escapes local +optima via temperature decay. + +### Want parameter-free + +[`Tlbo`] (Teaching-Learning-Based Optimization) has no `F`, `CR`, `w`, +or `σ` to tune. Often a respectable middle-of-the-pack performer. + +### Smallest possible self-adapting baseline + +[`OnePlusOneEs`] — Rechenberg's 1973 `(1+1)`-ES with the one-fifth +success rule. On the harness it hits f = 0 on Rastrigin in 50 000 +evaluations. + +### Just want a baseline + +[`RandomSearch`]. Useful as a sanity check: if your fancy optimizer +can't beat random search, something is wrong (with the fancy +optimizer or with the problem). + +## Step 2 — single-objective other types + +| Decision type | Algorithm | Notes | +|---|---|---| +| `Vec` | [`Umda`] | Per-bit marginal EDA. Independent-bit assumption. | +| `Vec` | [`GeneticAlgorithm`] + [`BitFlipMutation`] | When bit interactions matter. | +| `Vec` (permutation) | [`AntColonyTsp`] | TSP-style with a distance matrix. | +| `Vec` (permutation) | [`SimulatedAnnealing`] + [`SwapMutation`] | Generic discrete baseline. | +| `Vec` or custom | [`TabuSearch`] | You supply the neighbor function. | +| Custom struct | [`SimulatedAnnealing`] / [`HillClimber`] | With your own `Variation` impl. | + +## Step 2 — multi-objective (2 or 3) + +### Strong default + +[`Nsga2`] is the canonical Pareto-based EA. Fast, well-understood, +maintains diversity via crowding distance. On the harness it lands +on the Pareto front of every test problem. + +### Real-valued, smooth front, want best convergence + +[`Mopso`] (multi-objective PSO with archive). On ZDT1 it wins +hypervolume outright and converges 100× tighter than the +dominance-based methods. + +### Better front quality than NSGA-II + +[`Ibea`] (indicator-based) is consistently the best of the +dominance-based methods on the harness — wins ZDT3 hypervolume and +DTLZ2 mean distance by 24×. It uses an additive ε-indicator for +selection rather than dominance + crowding. + +[`Spea2`] (strength + density) — solid alternative; explicit external +archive separate from the population. + +[`SmsEmoa`] uses exact hypervolume contribution for selection. Elegant +in theory; in practice on the harness budgets here it underperforms +NSGA-II. Worth the higher per-step cost only when exact HV +contribution is the right discriminator. + +### Decomposition / weight-vector style + +[`Moead`] decomposes the multi-objective problem into many scalar +sub-problems (Tchebycheff or weighted sum) and solves them in +parallel. Very fast per generation; scales naturally to many +objectives. + +### Disconnected or non-convex front + +[`AgeMoea`] estimates the front geometry adaptively (the L_p +parameter `p` is fit from data each generation). + +[`Knea`] favors knee points — the regions of the front where small +gains in one objective cost large losses in another. + +[`Ibea`] also handles disconnected fronts well. + +### Region-based diversity + +[`PesaII`] uses grid hyperboxes to drive selection — divide the +objective space into a grid, pick from the least-crowded boxes. + +[`EpsilonMoea`] uses an ε-grid archive that auto-limits its size. + +### Just one starting decision (no population budget) + +[`Paes`] — `(1+1)`-ES with a Pareto archive. Cheap, simple, useful +when your evaluations are expensive enough that you can't afford a +population. + +## Step 2 — many-objective (4+) + +### Linear / simplex-shaped front (e.g., DTLZ1) + +[`Grea`] — grid coords drive ranking. On DTLZ1 it beats NSGA-III by +3× and AGE-MOEA by 2.5×. + +[`Moead`] — decomposition shines on linear fronts; second on DTLZ1 +and among the fastest per generation. + +### Curved / unknown front geometry + +[`Nsga3`] — reference-point niching; canonical many-objective method; +strong default when the front isn't simplex-shaped. + +[`AgeMoea`] — estimates L_p geometry per generation. + +[`Rvea`] — reference vectors with adaptive penalty. + +### Indicator-based selection + +[`Ibea`] — additive ε-indicator; doesn't degrade at high obj count. + +[`HypE`] — Monte Carlo hypervolume estimation; scales to arbitrary +objective count where exact HV is too expensive. + +## Step 3: Are there hard constraints? + +heuropt models constraints as a single scalar `constraint_violation` +on each `Evaluation`. Three escalations when the feasibility region +is hard to find: + +1. **Penalty-only.** Just set `constraint_violation > 0` for + infeasible decisions. The default tournament/Pareto comparisons + prefer feasibles automatically. +2. **Repair.** Implement [`Repair`] (or use the provided + [`ClampToBounds`] / [`ProjectToSimplex`]) to project infeasible + decisions back into the feasible region. Pair with a `Variation` + in a [`CompositeVariation`] for bounds-aware variants. +3. **Stochastic ranking.** Use [`stochastic_ranking_select`] instead + of `tournament_select_single_objective`. It probabilistically + explores near-feasibility instead of strict feasibility-first + ordering, which helps when feasible regions are narrow. + +See [Constrain your search with `Repair`](./cookbook/constraints.md) +for worked examples. + +## Step 4: Should you parallelize? + +Enable the `parallel` feature flag if your `evaluate` takes more +than ~50 µs. Population-based algorithms ([`RandomSearch`], [`Nsga2`], +[`DifferentialEvolution`], [`Spea2`], [`Ibea`], [`Mopso`], …) batch- +evaluate via rayon when the feature is on. **Seeded runs stay +bit-identical** to serial mode. + +```toml +heuropt = { version = "0.5", features = ["parallel"] } +``` + +## TL;DR table + +| Situation | Pick | +|---|---| +| Smooth single-objective continuous | [`CmaEs`] | +| Multimodal single-objective continuous | [`IpopCmaEs`] or [`DifferentialEvolution`] | +| Expensive single-objective | [`BayesianOpt`] or [`Tpe`] | +| Multi-fidelity single-objective | [`Hyperband`] | +| 2- or 3-objective default | [`Nsga2`] | +| 2-objective real-valued smooth front | [`Mopso`] | +| Disconnected / non-convex front | [`Ibea`] | +| Many-objective default (curved front) | [`Nsga3`] | +| Many-objective linear / simplex front | [`Grea`] | +| Permutation problem | [`AntColonyTsp`] | +| Binary problem | [`Umda`] | +| Custom decision type | [`SimulatedAnnealing`] + your `Variation` | +| Sanity baseline | [`RandomSearch`] | + +[`CmaEs`]: https://docs.rs/heuropt/latest/heuropt/algorithms/cma_es/struct.CmaEs.html +[`IpopCmaEs`]: https://docs.rs/heuropt/latest/heuropt/algorithms/ipop_cma_es/struct.IpopCmaEs.html +[`SeparableNes`]: https://docs.rs/heuropt/latest/heuropt/algorithms/snes/struct.SeparableNes.html +[`NelderMead`]: https://docs.rs/heuropt/latest/heuropt/algorithms/nelder_mead/struct.NelderMead.html +[`DifferentialEvolution`]: https://docs.rs/heuropt/latest/heuropt/algorithms/differential_evolution/struct.DifferentialEvolution.html +[`SimulatedAnnealing`]: https://docs.rs/heuropt/latest/heuropt/algorithms/simulated_annealing/struct.SimulatedAnnealing.html +[`Tlbo`]: https://docs.rs/heuropt/latest/heuropt/algorithms/tlbo/struct.Tlbo.html +[`OnePlusOneEs`]: https://docs.rs/heuropt/latest/heuropt/algorithms/one_plus_one_es/struct.OnePlusOneEs.html +[`RandomSearch`]: https://docs.rs/heuropt/latest/heuropt/algorithms/random_search/struct.RandomSearch.html +[`HillClimber`]: https://docs.rs/heuropt/latest/heuropt/algorithms/hill_climber/struct.HillClimber.html +[`BayesianOpt`]: https://docs.rs/heuropt/latest/heuropt/algorithms/bayesian_opt/struct.BayesianOpt.html +[`Tpe`]: https://docs.rs/heuropt/latest/heuropt/algorithms/tpe/struct.Tpe.html +[`Hyperband`]: https://docs.rs/heuropt/latest/heuropt/algorithms/hyperband/struct.Hyperband.html +[`PartialProblem`]: https://docs.rs/heuropt/latest/heuropt/core/partial_problem/trait.PartialProblem.html +[`Umda`]: https://docs.rs/heuropt/latest/heuropt/algorithms/umda/struct.Umda.html +[`GeneticAlgorithm`]: https://docs.rs/heuropt/latest/heuropt/algorithms/genetic_algorithm/struct.GeneticAlgorithm.html +[`BitFlipMutation`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.BitFlipMutation.html +[`AntColonyTsp`]: https://docs.rs/heuropt/latest/heuropt/algorithms/ant_colony_tsp/struct.AntColonyTsp.html +[`SwapMutation`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.SwapMutation.html +[`TabuSearch`]: https://docs.rs/heuropt/latest/heuropt/algorithms/tabu_search/struct.TabuSearch.html +[`Nsga2`]: https://docs.rs/heuropt/latest/heuropt/algorithms/nsga2/struct.Nsga2.html +[`Nsga3`]: https://docs.rs/heuropt/latest/heuropt/algorithms/nsga3/struct.Nsga3.html +[`Mopso`]: https://docs.rs/heuropt/latest/heuropt/algorithms/mopso/struct.Mopso.html +[`Ibea`]: https://docs.rs/heuropt/latest/heuropt/algorithms/ibea/struct.Ibea.html +[`Spea2`]: https://docs.rs/heuropt/latest/heuropt/algorithms/spea2/struct.Spea2.html +[`SmsEmoa`]: https://docs.rs/heuropt/latest/heuropt/algorithms/sms_emoa/struct.SmsEmoa.html +[`Moead`]: https://docs.rs/heuropt/latest/heuropt/algorithms/moead/struct.Moead.html +[`AgeMoea`]: https://docs.rs/heuropt/latest/heuropt/algorithms/age_moea/struct.AgeMoea.html +[`Knea`]: https://docs.rs/heuropt/latest/heuropt/algorithms/knea/struct.Knea.html +[`PesaII`]: https://docs.rs/heuropt/latest/heuropt/algorithms/pesa2/struct.PesaII.html +[`EpsilonMoea`]: https://docs.rs/heuropt/latest/heuropt/algorithms/epsilon_moea/struct.EpsilonMoea.html +[`Paes`]: https://docs.rs/heuropt/latest/heuropt/algorithms/paes/struct.Paes.html +[`Grea`]: https://docs.rs/heuropt/latest/heuropt/algorithms/grea/struct.Grea.html +[`Rvea`]: https://docs.rs/heuropt/latest/heuropt/algorithms/rvea/struct.Rvea.html +[`HypE`]: https://docs.rs/heuropt/latest/heuropt/algorithms/hype/struct.Hype.html +[`Repair`]: https://docs.rs/heuropt/latest/heuropt/traits/trait.Repair.html +[`ClampToBounds`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.ClampToBounds.html +[`ProjectToSimplex`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.ProjectToSimplex.html +[`stochastic_ranking_select`]: https://docs.rs/heuropt/latest/heuropt/selection/tournament/fn.stochastic_ranking_select.html +[`CompositeVariation`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.CompositeVariation.html diff --git a/docs/book/src/comparison.md b/docs/book/src/comparison.md new file mode 100644 index 0000000..9e79263 --- /dev/null +++ b/docs/book/src/comparison.md @@ -0,0 +1,108 @@ +# Comparison with other libraries + +heuropt is one of many heuristic-optimization libraries. This chapter +is an honest, opinionated comparison to help you choose. + +The columns: + +- **Lang** — primary implementation language. +- **Algorithms** — rough catalog count. +- **Multi-obj** — built-in support for Pareto-based multi-objective + optimization. +- **Surrogates** — built-in Bayesian / TPE / multi-fidelity. +- **Determinism** — seeded reproducibility as a first-class property. +- **Async / async-eval** — first-class async runtime support. + +| Library | Lang | Algorithms | Multi-obj | Surrogates | Determinism | Async | +|---|---|---|---|---|---|---| +| **heuropt 0.5** | Rust | 35 | ✅ NSGA-II/III, SPEA2, IBEA, MOEA/D, MOPSO, SMS-EMOA, HypE, AGE-MOEA, GrEA, KnEA, RVEA, PESA-II, ε-MOEA, PAES | ✅ BO, TPE, Hyperband | ✅ bit-identical seeded | ⏳ planned | +| pymoo | Python | ~25 | ✅ extensive | partial (BO via plug-ins) | ✅ | ❌ | +| DEAP | Python | flexible toolbox | ✅ | ❌ | ✅ | ❌ | +| hyperopt | Python | TPE-focused | ❌ | ✅ TPE | partial | partial | +| optuna | Python | TPE / CMA-ES / NSGA-II | ✅ | ✅ TPE, BoTorch via plug-in | ✅ | ✅ | +| MOEA Framework | Java | ~40 | ✅ very extensive | ❌ | ✅ | ❌ | +| metaheuristics-rs | Rust | ~10 | partial | ❌ | ✅ | ❌ | +| argmin | Rust | line-search / quasi-Newton | ❌ | ❌ | ✅ | ❌ | + +## When to pick heuropt + +- You're working in **Rust** and want a single, dependency-light crate + for evolutionary / metaheuristic optimization. +- You need **multi-objective or many-objective** algorithms (12+ + Pareto-aware methods in the catalog) AND you don't want to glue + Python into your Rust pipeline. +- You want **bit-identical determinism**: same seed produces same + output, on every machine, across releases unless explicitly noted + otherwise. +- You want a **small, readable codebase** — every algorithm is + written for clarity, no trait-object plumbing, no GATs in user- + facing APIs. Reading `RandomSearch` should be enough to write a + new optimizer. + +## When *not* to pick heuropt + +- You need **first-class async / await** for evaluations that talk to + HTTP services or spawn subprocesses. heuropt is sync; that's on + the roadmap but not shipping yet. +- You need **gradient-based** optimization. Use `argmin` (Rust) or + `scipy.optimize` (Python) — heuropt is gradient-free by design. +- You need **GPU-accelerated** evaluations. heuropt's `evaluate` + function runs on CPU; use Python (jax/torch) or roll your own + GPU pipeline. +- You need **distributed multi-machine** evaluation. heuropt + parallelizes within one process via rayon. Distribution is up to + you (split the seeds across machines, aggregate). +- You're comfortable in Python and pymoo / optuna already cover + your problem. heuropt's value-add over pymoo is mostly that it's + Rust — if that doesn't matter to you, the Python ecosystem has more + battle-tested integrations. + +## Algorithm coverage at a glance + +heuropt covers the same major Pareto MOEAs as pymoo and MOEA Framework: +NSGA-II/III, SPEA2, IBEA, MOEA/D, MOPSO, SMS-EMOA, HypE, AGE-MOEA, +GrEA, KnEA, RVEA, PESA-II, ε-MOEA, PAES. + +The expensive-evaluation regime: BayesianOpt + TPE + Hyperband. This +is comparable to optuna's coverage but in pure Rust. + +The single-objective continuous catalog (CMA-ES, IPOP-CMA-ES, sNES, +DE, PSO, GA, TLBO, (1+1)-ES, NelderMead, RandomSearch, HillClimber, +SimulatedAnnealing) covers the canonical baselines and several modern +variants. + +What heuropt does **not** ship that some libraries do: + +- **Re-themed metaphor metaheuristics** (Whale Optimization, Grey + Wolf, Bat, Firefly, Harris Hawks, etc.). These are cut from the + catalog deliberately — they are mostly DE/PSO with new names. If + you specifically need one, please open an issue with citations. +- **Non-evolutionary global optimizers** like dual annealing or + basin-hopping (use `scipy.optimize` for those). +- **A web UI / dashboard** like optuna's. heuropt is library-only. + +## Speed + +heuropt's hot paths (Pareto utilities, hypervolume, key inner loops) +are heavily optimized — see the perf entry in the v0.4.0 CHANGELOG. +On the comparison harness in `examples/compare.rs` (10-seed mean, +30 000 evaluations on DTLZ2), the total wall-clock time across 12 +algorithms is ~5 seconds. Per-algorithm timings are in +[`examples/compare-results.md`](https://github.com/swaits/heuropt/blob/main/examples/compare-results.md). + +For comparison-shopping speed against Python libraries, the gap is +typically 10×–100× in heuropt's favor for compute-bound +`evaluate` functions, because Rust skips the Python-loop overhead. If +your `evaluate` calls into NumPy/PyTorch and those are the bottleneck, +the gap shrinks substantially. + +## Honest weakness: ecosystem + +The biggest thing pymoo / optuna / DEAP have that heuropt doesn't: +**community + plug-ins + tutorials**. They've been around longer and +have rich third-party integrations (visualization, MLflow, +Hyperband+BO hybrids, distributed runners). heuropt is younger; the +core is solid but the ecosystem is small. + +If you adopt heuropt and miss a thing, the project is small enough +that contributions land fast. See [CONTRIBUTING.md](https://github.com/swaits/heuropt/blob/main/CONTRIBUTING.md). diff --git a/docs/book/src/cookbook.md b/docs/book/src/cookbook.md new file mode 100644 index 0000000..bdc6c7f --- /dev/null +++ b/docs/book/src/cookbook.md @@ -0,0 +1,26 @@ +# Cookbook + +Short, focused recipes for the patterns that come up in practice. +Each recipe is self-contained and small enough to copy into your own +project. + +## Recipes + +- [Parallelize evaluation with rayon](./cookbook/parallel.md) — when + your `evaluate` is non-trivial, the `parallel` feature pays for + itself almost immediately. +- [Tune a model with expensive evaluations](./cookbook/expensive-evaluations.md) + — `BayesianOpt`, `Tpe`, and `Hyperband` for the 50–500-eval + regime. +- [Compare two algorithms on your problem](./cookbook/compare.md) — + multi-seed harness pattern straight from `examples/compare.rs`. +- [Optimize a permutation (TSP-style)](./cookbook/permutation.md) — + `AntColonyTsp` with a distance matrix. +- [Constrain your search with `Repair`](./cookbook/constraints.md) — + bounds, simplex projection, custom repair. +- [Pick one answer off a Pareto front](./cookbook/pick-one.md) — the + a-posteriori weighted-decision pattern from the `jiggly_tuning` + example. +- [Write your own algorithm](./cookbook/custom-optimizer.md) — + implement `Optimizer

` from scratch, à la the + `examples/custom_optimizer.rs` walkthrough. diff --git a/docs/book/src/cookbook/compare.md b/docs/book/src/cookbook/compare.md new file mode 100644 index 0000000..9eefb6f --- /dev/null +++ b/docs/book/src/cookbook/compare.md @@ -0,0 +1,148 @@ +# Compare two algorithms on your problem + +The harness in `examples/compare.rs` runs every applicable algorithm +against every test problem with N seeds and reports mean ± std. +You can lift the same pattern for your own problem in ~30 lines. + +## The pattern + +1. Wrap your problem in a struct that implements [`Problem`]. +2. Pick a few candidate algorithms. +3. For each algorithm × seed, run and record the metric you care about. +4. Print mean ± std. + +## Worked example + +```rust,no_run +use heuropt::prelude::*; +use std::time::Instant; + +struct MyProblem; +impl Problem for MyProblem { + type Decision = Vec; + fn objectives(&self) -> ObjectiveSpace { + ObjectiveSpace::new(vec![Objective::minimize("f")]) + } + fn evaluate(&self, x: &Vec) -> Evaluation { + // your problem here + Evaluation::new(vec![x.iter().map(|v| v * v).sum::()]) + } +} + +const SEEDS: u64 = 10; +const DIM: usize = 5; +const BUDGET: usize = 30_000; + +fn main() { + let bounds: Vec<(f64, f64)> = vec![(-5.0, 5.0); DIM]; + + let mut best_de = vec![]; + let mut best_cmaes = vec![]; + let mut best_ipop = vec![]; + let mut t_de = vec![]; + let mut t_cmaes = vec![]; + let mut t_ipop = vec![]; + + for seed in 0..SEEDS { + // Differential Evolution + let t = Instant::now(); + let mut de = DifferentialEvolution::new( + DifferentialEvolutionConfig { + population_size: 30, + generations: BUDGET / 30, + differential_weight: 0.5, + crossover_probability: 0.9, + seed, + }, + RealBounds::new(bounds.clone()), + ); + let r = de.run(&MyProblem); + t_de.push(t.elapsed().as_millis() as f64); + best_de.push(r.best.unwrap().evaluation.objectives[0]); + + // CMA-ES + let t = Instant::now(); + let mut cma = CmaEs::new( + CmaEsConfig { + population_size: 12, + generations: BUDGET / 12, + initial_sigma: 1.0, + eigen_decomposition_period: 1, + initial_mean: None, + seed, + }, + RealBounds::new(bounds.clone()), + ); + let r = cma.run(&MyProblem); + t_cmaes.push(t.elapsed().as_millis() as f64); + best_cmaes.push(r.best.unwrap().evaluation.objectives[0]); + + // IPOP-CMA-ES + let t = Instant::now(); + let mut ipop = IpopCmaEs::new( + IpopCmaEsConfig { + base: CmaEsConfig { + population_size: 12, + generations: BUDGET / 12 / 4, + initial_sigma: 1.0, + eigen_decomposition_period: 1, + initial_mean: None, + seed, + }, + max_restarts: 3, + population_factor: 2.0, + seed, + }, + RealBounds::new(bounds.clone()), + ); + let r = ipop.run(&MyProblem); + t_ipop.push(t.elapsed().as_millis() as f64); + best_ipop.push(r.best.unwrap().evaluation.objectives[0]); + } + + println!("{:<12} {:>14} {:>10}", "algorithm", "best f (mean±std)", "ms"); + print_row("DE", &best_de, &t_de); + print_row("CMA-ES", &best_cmaes, &t_cmaes); + print_row("IPOP-CMA-ES", &best_ipop, &t_ipop); +} + +fn print_row(name: &str, values: &[f64], times: &[f64]) { + let (m, s) = mean_std(values); + let (t, _) = mean_std(times); + println!("{:<12} {:>10.3e} ± {:>5.2e} {:>6.0}", name, m, s, t); +} + +fn mean_std(xs: &[f64]) -> (f64, f64) { + let n = xs.len() as f64; + let m = xs.iter().sum::() / n; + let v = xs.iter().map(|x| (x - m).powi(2)).sum::() / n; + (m, v.sqrt()) +} +``` + +## What to record + +- **`best.evaluation.objectives[0]`** for single-objective. +- **`hypervolume_2d(&result.pareto_front, &space, ref_point)`** for + 2-objective. +- **`spacing(&result.pareto_front, &space)`** for front uniformity. +- **`result.evaluations`** to cross-check that every algorithm got + the same evaluation budget. +- Wall-clock `Instant::now()` deltas for runtime comparison. + +## Pitfalls + +- **Population size matters.** Different algorithms have very + different sweet spots. Don't just give them all the same + population — the README's algorithm pages note typical defaults. +- **Different algorithms count "generations" differently.** What + matters is the total `evaluations` count. Set + `generations = BUDGET / population_size` to match across + algorithms (with caveats for steady-state algorithms like SMS-EMOA + that evaluate one offspring per generation). +- **One seed is not a comparison.** Always run ≥ 5 seeds; ≥ 10 is + better. Single-seed comparisons are noise. +- **The harness in `examples/compare.rs` is the canonical version.** + When in doubt, copy from there. + +[`Problem`]: https://docs.rs/heuropt/latest/heuropt/core/problem/trait.Problem.html diff --git a/docs/book/src/cookbook/constraints.md b/docs/book/src/cookbook/constraints.md new file mode 100644 index 0000000..eac6e77 --- /dev/null +++ b/docs/book/src/cookbook/constraints.md @@ -0,0 +1,126 @@ +# Constrain your search with `Repair` + +heuropt models constraints with a single `constraint_violation` scalar +on each `Evaluation`. That works for soft penalties. When constraints +are *hard* and the search keeps generating infeasible decisions, the +better pattern is **repair**: project each candidate back into the +feasible region every time it leaves. + +The [`Repair`] trait is the abstraction. Two impls ship in the box; +you can write your own for arbitrary geometry. + +## Built-in: `ClampToBounds` + +For per-axis box constraints (`lo ≤ xᵢ ≤ hi`), pair `ClampToBounds` +with any `Variation` to get a bounds-aware variant for free. + +```rust,no_run +use heuropt::prelude::*; + +let bounds = vec![(-5.0, 5.0); 3]; + +// Without bounds, GaussianMutation can step outside the search box. +// ClampToBounds projects each variable back in. +let mut sigma = GaussianMutation { sigma: 0.5 }; +let mut clamp = ClampToBounds::new(bounds.clone()); + +let mut rng = rng_from_seed(42); +let parent = vec![4.9, -4.9, 0.0]; +let mut child = sigma.vary(std::slice::from_ref(&parent), &mut rng).pop().unwrap(); +clamp.repair(&mut child); +// every entry of `child` is now within [-5, 5]. +``` + +`ClampToBounds` is idempotent: applying it twice is the same as +applying it once. + +For most real problems you'd just use [`BoundedGaussianMutation`] +which combines both in one operator. + +## Built-in: `ProjectToSimplex` + +For *budget* constraints — "the components must sum to a fixed +total and be non-negative" — `ProjectToSimplex` projects onto the +probability simplex (or any scaled simplex). + +```rust,no_run +use heuropt::prelude::*; + +let mut proj = ProjectToSimplex::new(1.0); // probability simplex +let mut x = vec![0.6, 0.5, -0.1, 0.3]; // sum 1.3, one negative +proj.repair(&mut x); +// x now sums to 1.0 and every entry is ≥ 0. +let s: f64 = x.iter().sum(); +debug_assert!((s - 1.0).abs() < 1e-12); +debug_assert!(x.iter().all(|&v| v >= 0.0)); +``` + +Use this for portfolio / resource-allocation problems where the +decision is a vector of weights that must sum to a budget. + +## Custom repair + +Anything that takes a `&mut Vec` (or any `&mut D` for your +custom decision type) and returns a feasible version is a valid +`Repair`. Implement the trait directly: + +```rust,no_run +use heuropt::prelude::*; + +/// Force the largest variable to be at least `min_largest`. +struct AtLeastOneActive { min_largest: f64 } + +impl Repair> for AtLeastOneActive { + fn repair(&mut self, x: &mut Vec) { + let max_idx = x.iter() + .enumerate() + .fold(0, |best, (i, &v)| { + if v > x[best] { i } else { best } + }); + if x[max_idx] < self.min_largest { + x[max_idx] = self.min_largest; + } + } +} +``` + +## Stochastic-ranking selection + +When the feasible region is *narrow* — most of the search space is +infeasible — the strict "feasibles always beat infeasibles" rule +traps the search outside it. Runarsson & Yao's stochastic ranking +breaks the trap by, on each pairwise comparison, using a probabilistic +"compare by objective" instead of "compare by feasibility" with a +small probability `pf`: + +```rust,ignore +use heuropt::selection::tournament::stochastic_ranking_select; + +let picks = stochastic_ranking_select( + &population, + &objectives, + 0.45, // pf — Runarsson & Yao's canonical value + count, + &mut rng, +); +``` + +This is a drop-in replacement for `tournament_select_single_objective` +in your custom optimizer or in a forked algorithm. + +## When to use which + +| Situation | Use | +|---|---| +| Box constraints | [`BoundedGaussianMutation`] (built-in mutation) | +| Manual repair after any mutation | [`ClampToBounds`] | +| Budget / probability-simplex constraints | [`ProjectToSimplex`] | +| Custom geometric constraints | Your own `Repair` impl | +| Narrow feasible region, frequent infeasibility | [`stochastic_ranking_select`] | +| Soft penalty, mostly feasible search | Set `constraint_violation` and let default tournament handle it | + +[`Repair`]: https://docs.rs/heuropt/latest/heuropt/traits/trait.Repair.html +[`ClampToBounds`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.ClampToBounds.html +[`ProjectToSimplex`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.ProjectToSimplex.html +[`BoundedGaussianMutation`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.BoundedGaussianMutation.html +[`stochastic_ranking_select`]: https://docs.rs/heuropt/latest/heuropt/selection/tournament/fn.stochastic_ranking_select.html diff --git a/docs/book/src/cookbook/custom-optimizer.md b/docs/book/src/cookbook/custom-optimizer.md new file mode 100644 index 0000000..73d224d --- /dev/null +++ b/docs/book/src/cookbook/custom-optimizer.md @@ -0,0 +1,146 @@ +# Write your own algorithm + +Implement [`Optimizer

`] and you're done. There are no other traits +to think about, no internal hooks to register. The example walks +through a tiny hill-climber that reads almost identically to the +canonical pseudocode. + +## The trait + +```rust,ignore +pub trait Optimizer

+where + P: Problem, +{ + fn run(&mut self, problem: &P) -> OptimizationResult; +} +``` + +That's it. You own your config, your RNG, your main loop, and your +`OptimizationResult` construction. + +## A minimal hill-climber + +```rust,no_run +use heuropt::prelude::*; + +pub struct MyHillClimber { + pub iterations: usize, + pub seed: u64, + pub initializer: I, + pub variation: V, +} + +impl Optimizer

for MyHillClimber +where + P: Problem, + P::Decision: Clone, + I: Initializer, + V: Variation, +{ + fn run(&mut self, problem: &P) -> OptimizationResult { + let mut rng = rng_from_seed(self.seed); + let objectives = problem.objectives(); + assert!(objectives.is_single_objective(), "MyHillClimber is single-objective only"); + + // Start with one initial decision. + let init_decisions = self.initializer.initialize(1, &mut rng); + let init = init_decisions.into_iter().next().unwrap(); + let mut current = Candidate::new(init.clone(), problem.evaluate(&init)); + let mut evaluations: usize = 1; + + for _ in 0..self.iterations { + let children = self.variation.vary(std::slice::from_ref(¤t.decision), &mut rng); + for child_decision in children { + let child_eval = problem.evaluate(&child_decision); + evaluations += 1; + let child = Candidate::new(child_decision, child_eval); + if better(&child.evaluation, ¤t.evaluation, &objectives) { + current = child; + } + } + } + + let pareto_front = vec![current.clone()]; + let best = Some(current.clone()); + OptimizationResult::new( + Population::new(vec![current]), + pareto_front, + best, + evaluations, + self.iterations, + ) + } +} + +fn better(a: &Evaluation, b: &Evaluation, objectives: &ObjectiveSpace) -> bool { + let am = objectives.as_minimization(&a.objectives); + let bm = objectives.as_minimization(&b.objectives); + am[0] < bm[0] +} +``` + +## Things to notice + +- **`Rng` is one concrete type.** No generics — call + [`rng_from_seed`] and pass `&mut rng` everywhere it's needed. +- **`Initializer`** sources the starting point(s). +- **`Variation`** generates children from parents. For the + hill-climber it's called with one parent. +- **`OptimizationResult`** carries the final population, the Pareto + front (just the best for single-objective), the best candidate, + the total evaluations, and the iteration count. +- **`as_minimization`** flips maximize-axis values so your + comparison logic only ever needs to deal with "lower is better." + +## Adding parallel evaluation + +If your algorithm batch-evaluates candidates per generation, use the +crate's internal helper. From inside heuropt source you can call +`evaluate_batch(problem, decisions)`; from outside you'd use rayon +directly behind a feature flag, the same way the built-in algorithms +do. + +```rust,ignore +#[cfg(feature = "parallel")] +fn batch_eval

(problem: &P, decisions: Vec) -> Vec> +where P: Problem + Sync, P::Decision: Send, +{ + use rayon::prelude::*; + decisions.into_par_iter() + .map(|d| Candidate::new(d.clone(), problem.evaluate(&d))) + .collect() +} + +#[cfg(not(feature = "parallel"))] +fn batch_eval

(problem: &P, decisions: Vec) -> Vec> +where P: Problem, +{ + decisions.into_iter() + .map(|d| Candidate::new(d.clone(), problem.evaluate(&d))) + .collect() +} +``` + +To stay bit-identical between serial and parallel modes, keep the +RNG and selection on the main thread; only the *evaluations* run in +parallel. + +## What's *not* in the trait + +- **No iteration / step API.** The optimizer owns its loop. +- **No callbacks.** A future minor release may add an observer hook; + for now you'd run the algorithm to completion and process the + result. +- **No error type.** Invalid configuration panics with a clear + message; this matches the style of the built-in algorithms. +- **No async.** `evaluate` is synchronous; for async work, drive it + on a tokio runtime around the optimizer loop yourself. + +The smallness is the point: you should be able to read a built-in +algorithm and write your own in an afternoon. See +`examples/custom_optimizer.rs` for a slightly more polished version +of the hill-climber above. + +[`Optimizer

`]: https://docs.rs/heuropt/latest/heuropt/traits/trait.Optimizer.html +[`rng_from_seed`]: https://docs.rs/heuropt/latest/heuropt/core/rng/fn.rng_from_seed.html diff --git a/docs/book/src/cookbook/expensive-evaluations.md b/docs/book/src/cookbook/expensive-evaluations.md new file mode 100644 index 0000000..c4fc212 --- /dev/null +++ b/docs/book/src/cookbook/expensive-evaluations.md @@ -0,0 +1,164 @@ +# Tune a model with expensive evaluations + +Population-based EAs throw thousands of evaluations at a problem. If +each evaluation costs a minute (a model training run, a CFD solve, a +real-world measurement) you can't afford that. heuropt has three +algorithms aimed at this regime. + +| Algorithm | Surrogate | Best for | +|---|---|---| +| [`BayesianOpt`] | Gaussian process + Expected Improvement | The textbook choice; needs kernel tuning to shine | +| [`Tpe`] | Kernel-density estimate of good vs bad points | Cheaper per step; more robust without tuning | +| [`Hyperband`] | (none — it's a multi-fidelity scheduler) | When each eval has a tunable budget (epochs, MC samples) | + +## When each is right + +- **Black-box, fixed cost per eval, smooth-ish landscape** → BO. +- **Black-box, fixed cost per eval, no time to tune the surrogate** → TPE. +- **Each eval has a tunable fidelity** → Hyperband. + +## Bayesian Optimization + +A worked example with a synthetic 5-D problem and a 60-evaluation +budget — same configuration the `compare` harness uses. + +```rust,no_run +use heuropt::prelude::*; + +struct Rosenbrock5D; +impl Problem for Rosenbrock5D { + type Decision = Vec; + fn objectives(&self) -> ObjectiveSpace { + ObjectiveSpace::new(vec![Objective::minimize("f")]) + } + fn evaluate(&self, x: &Vec) -> Evaluation { + let f: f64 = x.windows(2).map(|w| + 100.0 * (w[1] - w[0].powi(2)).powi(2) + (1.0 - w[0]).powi(2) + ).sum(); + Evaluation::new(vec![f]) + } +} + +let bounds = vec![(-2.048_f64, 2.048_f64); 5]; +let mut opt = BayesianOpt::new( + BayesianOptConfig { + evaluations: 60, + initial_samples: 10, + length_scale: 1.0, + signal_variance: 1.0, + noise_variance: 1e-6, + seed: 42, + }, + RealBounds::new(bounds), +); +let r = opt.run(&Rosenbrock5D); +println!("best f after 60 evals: {}", r.best.unwrap().evaluation.objectives[0]); +``` + +> **Honest disclosure.** On the comparison harness this default +> configuration produces **f ≈ 3170 ± 2920** on Rosenbrock 5-D — well +> below what a tuned BO can do. The default RBF kernel without +> per-problem hyperparameter tuning is the limitation. For real +> workloads, consider: +> +> - More evaluations (200+ instead of 60). +> - Tuning `length_scale` to a known scale of your problem +> (lower for high-frequency landscapes, higher for smooth ones). +> - TPE instead of BO if you don't want to tune the kernel. + +## Tree-structured Parzen Estimator + +TPE keeps two density estimates — `l(x)` over historical good points +and `g(x)` over the rest — and picks new candidates that maximize the +ratio. Cheaper per step than a GP and famously robust without +hand-tuning. + +```rust,no_run +use heuropt::prelude::*; +# struct Rosenbrock5D; +# impl Problem for Rosenbrock5D { +# type Decision = Vec; +# fn objectives(&self) -> ObjectiveSpace { ObjectiveSpace::new(vec![Objective::minimize("f")]) } +# fn evaluate(&self, _x: &Vec) -> Evaluation { Evaluation::new(vec![0.0]) } +# } +let bounds = vec![(-2.048_f64, 2.048_f64); 5]; +let mut opt = Tpe::new( + TpeConfig { + evaluations: 60, + initial_samples: 10, + gamma: 0.25, + candidates_per_step: 24, + bandwidth_factor: 1.06, + seed: 42, + }, + RealBounds::new(bounds), +); +let _r = opt.run(&Rosenbrock5D); +``` + +`gamma` is the fraction of best points used as `l(x)`; `0.25` is the +canonical Bergstra value. + +## Hyperband + +[`Hyperband`] needs your problem to implement [`PartialProblem`] — +that is, you can evaluate at a tunable fidelity (e.g. number of +training epochs). The algorithm schedules many cheap-fidelity runs +and promotes only the survivors to higher fidelity. + +```rust,no_run +use heuropt::prelude::*; +use heuropt::core::partial_problem::PartialProblem; + +struct ModelTuning; +impl Problem for ModelTuning { + type Decision = Vec; + fn objectives(&self) -> ObjectiveSpace { + ObjectiveSpace::new(vec![Objective::minimize("val_loss")]) + } + fn evaluate(&self, x: &Vec) -> Evaluation { + // Full-fidelity eval = train at max_epochs. + self.evaluate_at_budget(x, 100.0) + } +} +impl PartialProblem for ModelTuning { + fn evaluate_at_budget(&self, x: &Vec, budget: f64) -> Evaluation { + // Replace with: train your model for `budget` epochs, return val_loss. + // For demo, pretend more budget = lower noisy loss. + let lr = x[0]; + let wd = x[1]; + let loss = (lr - 0.001).powi(2) + (wd - 1e-4).powi(2) + + 1.0 / (budget + 1.0); + Evaluation::new(vec![loss]) + } +} + +let bounds = vec![(1e-5_f64, 1e-1), (1e-6_f64, 1e-2)]; +let mut hyperband = Hyperband::new( + HyperbandConfig { + max_budget: 100.0, + eta: 3.0, + seed: 42, + }, + RealBounds::new(bounds), +); +let _r = hyperband.run(&ModelTuning); +``` + +`max_budget` is the most epochs (or whatever your fidelity unit is) +you'd ever spend on a single config. `eta` controls how aggressive +the elimination is — `3.0` is the classic value; higher means more +aggressive culling. + +## Strategy: combining surrogate + multi-fidelity + +The state of the art (BOHB) combines BO with Hyperband: TPE picks the +configurations Hyperband then evaluates at increasing fidelity. +heuropt doesn't ship a unified BOHB but the building blocks are +there — wrap your `PartialProblem` with a TPE-driven sampler and +feed the picks into `Hyperband`. PRs welcome. + +[`BayesianOpt`]: https://docs.rs/heuropt/latest/heuropt/algorithms/bayesian_opt/struct.BayesianOpt.html +[`Tpe`]: https://docs.rs/heuropt/latest/heuropt/algorithms/tpe/struct.Tpe.html +[`Hyperband`]: https://docs.rs/heuropt/latest/heuropt/algorithms/hyperband/struct.Hyperband.html +[`PartialProblem`]: https://docs.rs/heuropt/latest/heuropt/core/partial_problem/trait.PartialProblem.html diff --git a/docs/book/src/cookbook/parallel.md b/docs/book/src/cookbook/parallel.md new file mode 100644 index 0000000..b8b16d0 --- /dev/null +++ b/docs/book/src/cookbook/parallel.md @@ -0,0 +1,127 @@ +# Parallelize evaluation with rayon + +If a single call to your `evaluate` takes more than ~50 µs, enabling +the `parallel` feature usually pays for itself immediately on +population-based algorithms. Each generation evaluates an entire +population, and rayon parallelizes that batch. + +## Enable the feature + +```toml +[dependencies] +heuropt = { version = "0.5", features = ["parallel"] } +``` + +There's nothing else to opt into in your code. The +population-evaluation helper is feature-gated; with `parallel` on it +uses `rayon::into_par_iter` internally, with `parallel` off it falls +back to plain `into_iter`. + +## Determinism still holds + +Seeded runs are bit-identical between the serial and parallel modes. +The trick is that population members are evaluated in parallel but +*assembled* back into the same order. Variation, selection, and the +RNG are all driven by the main thread, so seed-stability tests still +pass. + +## Which algorithms benefit + +Algorithms with a per-generation `evaluate_batch`: + +- [`RandomSearch`], [`Nsga2`], [`Nsga3`], [`Spea2`], [`Moead`], + [`Mopso`], [`Ibea`], [`SmsEmoa`], [`HypE`], [`PesaII`], + [`EpsilonMoea`], [`AgeMoea`], [`Knea`], [`Grea`], [`Rvea`]. +- [`DifferentialEvolution`] and [`GeneticAlgorithm`] benefit on the + initial population and offspring batches. + +Steady-state algorithms ([`Paes`], [`SimulatedAnnealing`], +[`HillClimber`], [`OnePlusOneEs`]) only evaluate one or a few +candidates per iteration, so the parallel feature gives them +nothing — leave it off if those are your primary optimizers. + +## Worked example + +The Sphere problem is too cheap to actually benefit from parallelism +— this example just shows the shape. In real workloads `evaluate` is +the expensive bit (a simulation, a model fit, an HTTP call). + +```rust,no_run +use heuropt::prelude::*; + +struct ExpensiveSphere; +impl Problem for ExpensiveSphere { + type Decision = Vec; + fn objectives(&self) -> ObjectiveSpace { + ObjectiveSpace::new(vec![Objective::minimize("f")]) + } + fn evaluate(&self, x: &Vec) -> Evaluation { + // Pretend this is a 5 ms simulation. + std::thread::sleep(std::time::Duration::from_millis(5)); + Evaluation::new(vec![x.iter().map(|v| v * v).sum::()]) + } +} + +fn main() { + let bounds = vec![(-1.0_f64, 1.0_f64); 5]; + let mut opt = DifferentialEvolution::new( + DifferentialEvolutionConfig { + population_size: 16, + generations: 50, + differential_weight: 0.5, + crossover_probability: 0.9, + seed: 42, + }, + RealBounds::new(bounds), + ); + let r = opt.run(&ExpensiveSphere); + println!("best f = {}", r.best.unwrap().evaluation.objectives[0]); +} +``` + +With the `parallel` feature on, each generation's 16 evaluations run +across rayon's worker threads. On a 16-core machine the wall-clock +cost per generation drops from `16 × 5 ms = 80 ms` to roughly +`5 ms + scheduling overhead`. + +## Sizing your thread pool + +heuropt uses rayon's global thread pool. Override the size with: + +```rust,ignore +rayon::ThreadPoolBuilder::new().num_threads(8).build_global().unwrap(); +``` + +Run this **before** any heuropt call, or use rayon's `install` API +to scope it. + +## When parallelism *doesn't* help + +- Your `evaluate` is sub-microsecond (Sphere, Rastrigin, Ackley + unweighted) — the rayon scheduling overhead exceeds the work. +- You're already running multiple seeds in parallel at the harness + level (see [Compare two algorithms](./compare.md)). Stacking + parallelism rarely helps. +- The algorithm is steady-state (Paes, SA, hill climber). + +[`RandomSearch`]: https://docs.rs/heuropt/latest/heuropt/algorithms/random_search/struct.RandomSearch.html +[`Nsga2`]: https://docs.rs/heuropt/latest/heuropt/algorithms/nsga2/struct.Nsga2.html +[`Nsga3`]: https://docs.rs/heuropt/latest/heuropt/algorithms/nsga3/struct.Nsga3.html +[`Spea2`]: https://docs.rs/heuropt/latest/heuropt/algorithms/spea2/struct.Spea2.html +[`Moead`]: https://docs.rs/heuropt/latest/heuropt/algorithms/moead/struct.Moead.html +[`Mopso`]: https://docs.rs/heuropt/latest/heuropt/algorithms/mopso/struct.Mopso.html +[`Ibea`]: https://docs.rs/heuropt/latest/heuropt/algorithms/ibea/struct.Ibea.html +[`SmsEmoa`]: https://docs.rs/heuropt/latest/heuropt/algorithms/sms_emoa/struct.SmsEmoa.html +[`HypE`]: https://docs.rs/heuropt/latest/heuropt/algorithms/hype/struct.Hype.html +[`PesaII`]: https://docs.rs/heuropt/latest/heuropt/algorithms/pesa2/struct.PesaII.html +[`EpsilonMoea`]: https://docs.rs/heuropt/latest/heuropt/algorithms/epsilon_moea/struct.EpsilonMoea.html +[`AgeMoea`]: https://docs.rs/heuropt/latest/heuropt/algorithms/age_moea/struct.AgeMoea.html +[`Knea`]: https://docs.rs/heuropt/latest/heuropt/algorithms/knea/struct.Knea.html +[`Grea`]: https://docs.rs/heuropt/latest/heuropt/algorithms/grea/struct.Grea.html +[`Rvea`]: https://docs.rs/heuropt/latest/heuropt/algorithms/rvea/struct.Rvea.html +[`DifferentialEvolution`]: https://docs.rs/heuropt/latest/heuropt/algorithms/differential_evolution/struct.DifferentialEvolution.html +[`GeneticAlgorithm`]: https://docs.rs/heuropt/latest/heuropt/algorithms/genetic_algorithm/struct.GeneticAlgorithm.html +[`Paes`]: https://docs.rs/heuropt/latest/heuropt/algorithms/paes/struct.Paes.html +[`SimulatedAnnealing`]: https://docs.rs/heuropt/latest/heuropt/algorithms/simulated_annealing/struct.SimulatedAnnealing.html +[`HillClimber`]: https://docs.rs/heuropt/latest/heuropt/algorithms/hill_climber/struct.HillClimber.html +[`OnePlusOneEs`]: https://docs.rs/heuropt/latest/heuropt/algorithms/one_plus_one_es/struct.OnePlusOneEs.html diff --git a/docs/book/src/cookbook/permutation.md b/docs/book/src/cookbook/permutation.md new file mode 100644 index 0000000..3ff51a3 --- /dev/null +++ b/docs/book/src/cookbook/permutation.md @@ -0,0 +1,167 @@ +# Optimize a permutation (TSP-style) + +When your decision is "an ordering" — visiting cities, scheduling +jobs, routing — the natural representation is `Vec` and the +specialized algorithm is [`AntColonyTsp`]. Generic alternatives are +[`SimulatedAnnealing`] + [`SwapMutation`] for any permutation, and +[`TabuSearch`] when you have a custom neighbor function. + +## TSP with `AntColonyTsp` + +```rust,no_run +use heuropt::prelude::*; + +struct Tsp { + distances: Vec>, +} + +impl Problem for Tsp { + type Decision = Vec; + + fn objectives(&self) -> ObjectiveSpace { + ObjectiveSpace::new(vec![Objective::minimize("length")]) + } + + fn evaluate(&self, tour: &Vec) -> Evaluation { + let mut len = 0.0; + for w in tour.windows(2) { + len += self.distances[w[0]][w[1]]; + } + len += self.distances[*tour.last().unwrap()][tour[0]]; + Evaluation::new(vec![len]) + } +} + +fn main() { + // 5-city Euclidean instance + let cities = vec![ + (0.0, 0.0), + (1.0, 5.0), + (5.0, 2.0), + (6.0, 6.0), + (8.0, 3.0), + ]; + let n = cities.len(); + let mut distances = vec![vec![0.0; n]; n]; + for i in 0..n { + for j in 0..n { + let dx = cities[i].0 - cities[j].0; + let dy = cities[i].1 - cities[j].1; + distances[i][j] = (dx * dx + dy * dy).sqrt(); + } + } + let problem = Tsp { distances: distances.clone() }; + + let mut opt = AntColonyTsp::new(AntColonyTspConfig { + ants: 20, + iterations: 200, + alpha: 1.0, + beta: 5.0, + evaporation: 0.5, + deposit: 1.0, + distances, + seed: 42, + }); + + let r = opt.run(&problem); + let best = r.best.unwrap(); + println!("best tour length: {:.3}", best.evaluation.objectives[0]); + println!("tour: {:?}", best.decision); +} +``` + +`alpha` weights pheromone influence and `beta` weights the +heuristic (1 / distance). `evaporation` is the per-iteration decay +of pheromone trails. The classic Dorigo paper uses `alpha = 1`, +`beta = 2..5`, `evaporation = 0.1..0.5`. + +## Generic permutation: SA + SwapMutation + +Use this when your problem isn't TSP-shaped (no distance matrix +makes sense) but you still want to optimize an ordering. + +```rust,no_run +use heuropt::prelude::*; + +struct JobShop { + process_times: Vec, +} +impl Problem for JobShop { + type Decision = Vec; + fn objectives(&self) -> ObjectiveSpace { + ObjectiveSpace::new(vec![Objective::minimize("makespan")]) + } + fn evaluate(&self, schedule: &Vec) -> Evaluation { + // Pretend cumulative weighted-completion-time. Replace with your real cost. + let cost: f64 = schedule.iter().enumerate() + .map(|(i, &job)| (i as f64 + 1.0) * self.process_times[job]) + .sum(); + Evaluation::new(vec![cost]) + } +} + +fn make_initial_perm(n: usize, seed: u64) -> Vec { + use rand::seq::SliceRandom; + let mut rng = rng_from_seed(seed); + let mut perm: Vec = (0..n).collect(); + perm.shuffle(&mut rng); + perm +} + +let times = vec![3.0, 1.5, 4.2, 2.7, 5.1]; +let problem = JobShop { process_times: times.clone() }; + +// SimulatedAnnealing needs a starting decision; pass a custom Initializer. +struct OnePerm(Vec); +impl Initializer> for OnePerm { + fn initialize(&mut self, _size: usize, _rng: &mut Rng) -> Vec> { + vec![self.0.clone()] + } +} + +let mut opt = SimulatedAnnealing::new( + SimulatedAnnealingConfig { + iterations: 2000, + initial_temperature: 5.0, + final_temperature: 1e-3, + seed: 7, + }, + OnePerm(make_initial_perm(times.len(), 7)), + SwapMutation, +); +let r = opt.run(&problem); +let best = r.best.unwrap(); +println!("best makespan: {:.3}", best.evaluation.objectives[0]); +println!("schedule: {:?}", best.decision); +``` + +`SwapMutation` swaps two random indices in the permutation — +preserves the "every element appears once" invariant for free. + +## Custom neighborhoods: `TabuSearch` + +When swap isn't the right move set (e.g., 2-opt for TSP, insert / +shift for scheduling), use [`TabuSearch`] with your own neighbor +function. + +```rust,ignore +use heuropt::prelude::*; +let neighbors = |x: &Vec, _rng: &mut Rng| -> Vec> { + // Generate all 2-opt neighbors of x. + let mut out = Vec::new(); + for i in 0..x.len() { + for j in (i + 2)..x.len() { + let mut child = x.clone(); + child[i + 1..=j].reverse(); + out.push(child); + } + } + out +}; +// Pass `neighbors` to TabuSearch::new(...). +``` + +[`AntColonyTsp`]: https://docs.rs/heuropt/latest/heuropt/algorithms/ant_colony_tsp/struct.AntColonyTsp.html +[`SimulatedAnnealing`]: https://docs.rs/heuropt/latest/heuropt/algorithms/simulated_annealing/struct.SimulatedAnnealing.html +[`SwapMutation`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.SwapMutation.html +[`TabuSearch`]: https://docs.rs/heuropt/latest/heuropt/algorithms/tabu_search/struct.TabuSearch.html diff --git a/docs/book/src/cookbook/pick-one.md b/docs/book/src/cookbook/pick-one.md new file mode 100644 index 0000000..e9d9ea5 --- /dev/null +++ b/docs/book/src/cookbook/pick-one.md @@ -0,0 +1,127 @@ +# Pick one answer off a Pareto front + +A multi-objective optimizer hands you a *front* — a Pareto-optimal +trade-off curve — not a single answer. Eventually you have to pick +*one* point off it. There are several principled ways to do that; +this recipe covers the most common: the **a-posteriori weighted +decision rule**. + +The pattern: optimize *without* baking your preferences into the +search, then apply your preferences as a scoring function over the +front. + +This is exactly the pattern from `examples/jiggly_tuning.rs` (the +USB-jiggler firmware tuning example). + +## The shape + +```rust,no_run +use heuropt::prelude::*; + +# struct Cost; +# impl Problem for Cost { +# type Decision = Vec; +# fn objectives(&self) -> ObjectiveSpace { +# ObjectiveSpace::new(vec![Objective::minimize("a"), Objective::minimize("b"), Objective::minimize("c")]) +# } +# fn evaluate(&self, _x: &Vec) -> Evaluation { Evaluation::new(vec![0.0,0.0,0.0]) } +# } + +let problem = Cost; +let mut opt = Nsga2::new( + Nsga2Config { population_size: 100, generations: 200, seed: 42 }, + RealBounds::new(vec![(-1.0, 1.0); 4]), + CompositeVariation { + crossover: SimulatedBinaryCrossover::new(vec![(-1.0, 1.0); 4], 15.0, 0.5), + mutation: PolynomialMutation::new(vec![(-1.0, 1.0); 4], 20.0, 1.0), + }, +); +let result = opt.run(&problem); + +// 1. Get the Pareto front. +let front = &result.pareto_front; + +// 2. Define your preferences as a scoring function over (oriented) +// objective values. Lower score = preferred. +let space = problem.objectives(); +let weights = [1.0, 2.0, 0.5]; + +let scored: Vec<(f64, &Candidate>)> = front.iter() + .map(|c| { + let oriented = space.as_minimization(&c.evaluation.objectives); + let score: f64 = oriented.iter().zip(&weights) + .map(|(v, w)| v * w) + .sum(); + (score, c) + }) + .collect(); + +// 3. Pick the lowest-scoring point. +let best = scored.iter() + .min_by(|a, b| a.0.partial_cmp(&b.0).unwrap()) + .unwrap(); + +println!("picked: {:?} with weighted score {:.3}", + best.1.evaluation.objectives, best.0); +``` + +`as_minimization` returns the objective vector with maximized axes +flipped to negative — so a single set of *positive* weights does +the right thing whether each axis is min or max. + +## Why a-posteriori vs a-priori weighting + +If you know your weights up front, you could just optimize the +weighted sum directly with a single-objective algorithm. Why bother +with the multi-objective dance? + +Two reasons: + +1. **Weighted sum can't reach concave parts of the Pareto front.** + Any single-objective optimization with a linear scalarization + converges to a point at the boundary of the convex hull. Concave + front segments are unreachable. The multi-objective optimizer + finds them. +2. **Weights are usually wrong on the first try.** Optimizing the + front first lets you see what's actually possible before deciding + how much each axis is worth. Run once, look at the trade-offs, + adjust weights. + +## Penalty terms beyond linear weights + +The jiggly example also adds a *hinge penalty* — a term that's zero +inside an acceptable region and grows quadratically once you exceed +some hard cap. Useful when one axis is "soft up to X, hard cap at Y": + +```rust,no_run +fn hinge(x: f64, soft_cap: f64, hard_cap: f64) -> f64 { + if x <= soft_cap { 0.0 } + else if x >= hard_cap { f64::INFINITY } + else { + let t = (x - soft_cap) / (hard_cap - soft_cap); + 100.0 * t * t + } +} +``` + +Compose linear weights + hinge penalties and you have a flexible +scoring function over the front without re-running the optimizer. + +## Other strategies + +- **Knee point.** Pick the point where small gains in one axis cost + large losses in another — the "elbow" of the trade-off curve. + [`Knea`] explicitly biases the search toward knees during the run. +- **Reference-direction.** Pick the point closest to a desired + trade-off direction (a unit vector in objective space). + [`Moead`] / [`Nsga3`] use this internally during search; you can + apply it post-hoc the same way. +- **Random / interactive selection.** Show the front to a user + (perhaps via a plotting library), let them pick. + +The right pick depends on the problem; the front itself doesn't +prescribe one. + +[`Knea`]: https://docs.rs/heuropt/latest/heuropt/algorithms/knea/struct.Knea.html +[`Moead`]: https://docs.rs/heuropt/latest/heuropt/algorithms/moead/struct.Moead.html +[`Nsga3`]: https://docs.rs/heuropt/latest/heuropt/algorithms/nsga3/struct.Nsga3.html diff --git a/docs/book/src/defining-problems.md b/docs/book/src/defining-problems.md new file mode 100644 index 0000000..febc8f8 --- /dev/null +++ b/docs/book/src/defining-problems.md @@ -0,0 +1,244 @@ +# Defining a problem + +Everything in heuropt starts with the [`Problem`] trait. This chapter +walks through every shape it can take. + +## The trait + +```rust,ignore +pub trait Problem { + type Decision: Clone; + fn objectives(&self) -> ObjectiveSpace; + fn evaluate(&self, decision: &Self::Decision) -> Evaluation; +} +``` + +Three things you decide: + +1. **`Decision`** — the type of the thing you're optimizing. + `Vec` is by far the most common; `Vec` for binary + search, `Vec` for permutations, your own struct for + anything else. +2. **`objectives`** — how many objectives you have, what they're + called, and whether each is minimized or maximized. Returned as + an [`ObjectiveSpace`]. +3. **`evaluate`** — given one decision, score it. Returns an + [`Evaluation`] with a vector of objective values (and optionally + a constraint-violation scalar). + +`evaluate` takes `&self`, so caches and lookup tables are easy. It +is called many thousands of times during a typical run, so keep it +fast. + +## Single-objective continuous + +The Rosenbrock banana — minimize a smooth non-convex valley. + +```rust,no_run +use heuropt::prelude::*; + +struct Rosenbrock; + +impl Problem for Rosenbrock { + type Decision = Vec; + + fn objectives(&self) -> ObjectiveSpace { + ObjectiveSpace::new(vec![Objective::minimize("f")]) + } + + fn evaluate(&self, x: &Vec) -> Evaluation { + let f: f64 = x.windows(2) + .map(|w| 100.0 * (w[1] - w[0].powi(2)).powi(2) + (1.0 - w[0]).powi(2)) + .sum(); + Evaluation::new(vec![f]) + } +} +``` + +## Multi-objective + +ZDT1 — two objectives that conflict. The Pareto front is the set of +non-dominated trade-offs. + +```rust,no_run +use heuropt::prelude::*; + +struct Zdt1 { dim: usize } + +impl Problem for Zdt1 { + type Decision = Vec; + + fn objectives(&self) -> ObjectiveSpace { + ObjectiveSpace::new(vec![ + Objective::minimize("f1"), + Objective::minimize("f2"), + ]) + } + + fn evaluate(&self, x: &Vec) -> Evaluation { + let n = x.len() as f64; + let f1 = x[0]; + let g = 1.0 + 9.0 * x[1..].iter().sum::() / (n - 1.0); + let h = 1.0 - (f1 / g).sqrt(); + let f2 = g * h; + Evaluation::new(vec![f1, f2]) + } +} +``` + +For multi-objective problems, pick a Pareto-aware optimizer: +[`Nsga2`] is the canonical default; [`Mopso`] often wins on +smooth-front 2-objective problems; [`Ibea`] often wins on +disconnected fronts. See [choosing-an-algorithm](./choosing-an-algorithm.md). + +## Maximizing instead of minimizing + +heuropt's internals normalize everything to minimization, but you +declare your objective with the orientation that's natural for your +problem. A scoring problem might want to maximize: + +```rust,no_run +use heuropt::prelude::*; +let space = ObjectiveSpace::new(vec![ + Objective::minimize("cost"), + Objective::maximize("accuracy"), +]); +``` + +`Objective::maximize` is a convenience for `Direction::Maximize`. Mix +freely; the Pareto-comparison machinery handles the orientation. + +## Constraints + +heuropt models constraints as a single non-negative scalar +**`constraint_violation`** on each `Evaluation`. The convention: + +- `0.0` (or negative) means **feasible**. +- Any positive value means **infeasible**, and bigger numbers are + worse violations. + +Pareto-comparison and tournament-selection helpers prefer feasible +candidates and break ties on the violation magnitude — so the rule +"feasibility comes first" is enforced automatically. + +```rust,no_run +use heuropt::prelude::*; + +struct Constrained; +impl Problem for Constrained { + type Decision = Vec; + + fn objectives(&self) -> ObjectiveSpace { + ObjectiveSpace::new(vec![Objective::minimize("f")]) + } + + fn evaluate(&self, x: &Vec) -> Evaluation { + let f: f64 = x.iter().map(|v| v * v).sum(); + + // Constraint: x[0] + x[1] >= 1. Violation = how much we miss it by. + let g1 = (1.0 - (x[0] + x[1])).max(0.0); + let total_violation: f64 = g1; // sum of max(0, gᵢ) for each constraint + + Evaluation::constrained(vec![f], total_violation) + } +} +``` + +If your constraints are very tight and the search keeps hitting them, +see [Constrain your search with `Repair`](./cookbook/constraints.md). + +## Decision types beyond `Vec` + +### Binary (`Vec`) + +```rust,no_run +use heuropt::prelude::*; + +struct OneMax { bits: usize } +impl Problem for OneMax { + type Decision = Vec; + fn objectives(&self) -> ObjectiveSpace { + ObjectiveSpace::new(vec![Objective::maximize("ones")]) + } + fn evaluate(&self, x: &Vec) -> Evaluation { + Evaluation::new(vec![x.iter().filter(|b| **b).count() as f64]) + } +} +``` + +For `Vec` problems, [`Umda`] is a parameter-free EDA; +[`GeneticAlgorithm`] with [`BitFlipMutation`] is the GA route. + +### Permutations (`Vec`) + +```rust,no_run +use heuropt::prelude::*; + +struct Tsp { distances: Vec> } +impl Problem for Tsp { + type Decision = Vec; + fn objectives(&self) -> ObjectiveSpace { + ObjectiveSpace::new(vec![Objective::minimize("length")]) + } + fn evaluate(&self, tour: &Vec) -> Evaluation { + let mut len = 0.0; + for w in tour.windows(2) { + len += self.distances[w[0]][w[1]]; + } + len += self.distances[*tour.last().unwrap()][tour[0]]; + Evaluation::new(vec![len]) + } +} +``` + +For permutations, [`AntColonyTsp`] specializes on TSP-style problems; +[`TabuSearch`] takes a user-supplied neighbor function for arbitrary +discrete neighborhoods; [`SimulatedAnnealing`] with [`SwapMutation`] +is the simplest baseline. + +### Custom decision types + +Any `Clone` type works. If you have a struct, just implement `Clone` +and you can use it. You'll need to write your own `Variation` impl +to mutate it; see [Write your own algorithm](./cookbook/custom-optimizer.md). + +## What `Evaluation` carries + +```rust,ignore +pub struct Evaluation { + pub objectives: Vec, // one entry per objective + pub constraint_violation: f64, // 0.0 = feasible +} +``` + +That's it. Construct with [`Evaluation::new`] for unconstrained +problems or [`Evaluation::constrained`] when you have a violation. + +## Summary + +- Implement [`Problem`] with your decision type. +- Declare objectives via [`ObjectiveSpace`] (mix minimize/maximize + freely). +- Return an [`Evaluation`] from `evaluate`. +- For constraints, set `constraint_violation > 0` for infeasible + decisions; heuropt's selection helpers prefer feasibles + automatically. + +Next: [Choosing an algorithm](./choosing-an-algorithm.md) walks +through the decision tree. + +[`Problem`]: https://docs.rs/heuropt/latest/heuropt/core/problem/trait.Problem.html +[`ObjectiveSpace`]: https://docs.rs/heuropt/latest/heuropt/core/objective/struct.ObjectiveSpace.html +[`Evaluation`]: https://docs.rs/heuropt/latest/heuropt/core/evaluation/struct.Evaluation.html +[`Evaluation::new`]: https://docs.rs/heuropt/latest/heuropt/core/evaluation/struct.Evaluation.html#method.new +[`Evaluation::constrained`]: https://docs.rs/heuropt/latest/heuropt/core/evaluation/struct.Evaluation.html#method.constrained +[`Nsga2`]: https://docs.rs/heuropt/latest/heuropt/algorithms/nsga2/struct.Nsga2.html +[`Mopso`]: https://docs.rs/heuropt/latest/heuropt/algorithms/mopso/struct.Mopso.html +[`Ibea`]: https://docs.rs/heuropt/latest/heuropt/algorithms/ibea/struct.Ibea.html +[`Umda`]: https://docs.rs/heuropt/latest/heuropt/algorithms/umda/struct.Umda.html +[`GeneticAlgorithm`]: https://docs.rs/heuropt/latest/heuropt/algorithms/genetic_algorithm/struct.GeneticAlgorithm.html +[`BitFlipMutation`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.BitFlipMutation.html +[`AntColonyTsp`]: https://docs.rs/heuropt/latest/heuropt/algorithms/ant_colony_tsp/struct.AntColonyTsp.html +[`TabuSearch`]: https://docs.rs/heuropt/latest/heuropt/algorithms/tabu_search/struct.TabuSearch.html +[`SimulatedAnnealing`]: https://docs.rs/heuropt/latest/heuropt/algorithms/simulated_annealing/struct.SimulatedAnnealing.html +[`SwapMutation`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.SwapMutation.html diff --git a/docs/book/src/getting-started.md b/docs/book/src/getting-started.md new file mode 100644 index 0000000..b7ea406 --- /dev/null +++ b/docs/book/src/getting-started.md @@ -0,0 +1,127 @@ +# Five-minute walkthrough + +The shortest path from a fresh project to a working optimizer. + +## 1. Add heuropt to your `Cargo.toml` + +```toml +[dependencies] +heuropt = "0.5" +``` + +The default feature set is small. Optional features: + +- `parallel` — rayon-backed parallel population evaluation. +- `serde` — `Serialize` / `Deserialize` derives on the core data + types. + +```toml +heuropt = { version = "0.5", features = ["parallel"] } +``` + +## 2. Define a problem + +A problem is a struct that implements the [`Problem`] trait. You tell +heuropt what kind of decision your problem takes (`Vec`, +`Vec`, …), what objectives it has (minimize or maximize), and +how to score one decision. + +```rust,no_run +use heuropt::prelude::*; + +struct Sphere; + +impl Problem for Sphere { + type Decision = Vec; + + fn objectives(&self) -> ObjectiveSpace { + ObjectiveSpace::new(vec![Objective::minimize("f")]) + } + + fn evaluate(&self, x: &Vec) -> Evaluation { + let f: f64 = x.iter().map(|v| v * v).sum(); + Evaluation::new(vec![f]) + } +} +``` + +The Sphere function is a single-objective continuous problem: minimize +`f(x) = Σ xᵢ²`. The optimum is `x = 0`, `f = 0`. + +## 3. Pick an algorithm and run it + +For a smooth single-objective continuous problem, [`CmaEs`] is a +strong default. Configure it, build it, run it. + +```rust,no_run +# use heuropt::prelude::*; +# struct Sphere; +# impl Problem for Sphere { +# type Decision = Vec; +# fn objectives(&self) -> ObjectiveSpace { +# ObjectiveSpace::new(vec![Objective::minimize("f")]) +# } +# fn evaluate(&self, x: &Vec) -> Evaluation { +# Evaluation::new(vec![x.iter().map(|v| v * v).sum::()]) +# } +# } +let bounds = RealBounds::new(vec![(-5.0, 5.0); 5]); // 5-dim search box + +let mut opt = CmaEs::new( + CmaEsConfig { + population_size: 12, + generations: 80, + initial_sigma: 1.0, + eigen_decomposition_period: 1, + initial_mean: None, + seed: 42, + }, + bounds, +); + +let result = opt.run(&Sphere); + +let best = result.best.expect("at least one feasible candidate"); +println!("best f = {:.3e} at x = {:?}", best.evaluation.objectives[0], best.decision); +``` + +Run with `cargo run --release` — heuristic optimization is allergic +to debug builds. Expect output like: + +```text +best f = 1.4e-29 at x = [-1.6e-15, 4.5e-16, ...] +``` + +CMA-ES drops to machine epsilon on the Sphere in well under 80 +generations. + +## 4. What just happened + +- [`Problem`] is the **what** you're optimizing. +- [`CmaEs`] (or any other optimizer) is the **how**. +- [`CmaEsConfig`] is a plain public-field struct: there are no + builders, no chained setters, just public fields you set + directly. +- [`Optimizer::run`] returns an [`OptimizationResult`] containing the + full final `population`, the `pareto_front` (just the best for + single-objective), the `best` candidate, the total `evaluations`, + and the number of `generations`. + +## 5. Where to go next + +- **Multi-objective:** see [Defining a problem](./defining-problems.md) + for how to express two or more objectives, and + [Choosing an algorithm](./choosing-an-algorithm.md) for which + optimizer fits. +- **Want to know which algorithm to pick:** read the README's + decision tree, or jump straight to the [choosing-an-algorithm](./choosing-an-algorithm.md) + chapter for the long form. +- **Production patterns:** the [cookbook](./cookbook.md) has recipes + for parallelism, expensive evaluations, comparing algorithms, and + more. + +[`Problem`]: https://docs.rs/heuropt/latest/heuropt/core/problem/trait.Problem.html +[`Optimizer::run`]: https://docs.rs/heuropt/latest/heuropt/traits/trait.Optimizer.html +[`OptimizationResult`]: https://docs.rs/heuropt/latest/heuropt/core/result/struct.OptimizationResult.html +[`CmaEs`]: https://docs.rs/heuropt/latest/heuropt/algorithms/cma_es/struct.CmaEs.html +[`CmaEsConfig`]: https://docs.rs/heuropt/latest/heuropt/algorithms/cma_es/struct.CmaEsConfig.html diff --git a/docs/book/src/introduction.md b/docs/book/src/introduction.md new file mode 100644 index 0000000..1d423f7 --- /dev/null +++ b/docs/book/src/introduction.md @@ -0,0 +1,82 @@ +# Introduction + +heuropt is a practical Rust toolkit for **heuristic optimization** — the +art of searching for good answers when the problem is too gnarly to +solve analytically. + +The kinds of problems heuropt is built for: + +- **Single-objective:** "find the parameters that minimize the loss of + this model." Hyperparameter tuning. Curve fitting. Calibration. +- **Multi-objective:** "find the trade-off curve between cost and + accuracy." Engineering design. Portfolio optimization. Fleet + scheduling. +- **Many-objective (4+):** the same idea but with enough objectives + that classical Pareto methods break down. Power-grid planning. + Airfoil design. Multi-criteria recommendation. + +If your problem is differentiable and convex, you don't need this +crate — use a gradient solver. heuropt is for the *messy* problems: +landscapes with lots of local minima, decisions that aren't continuous +(permutations, bit vectors), or evaluations that are noisy / expensive +/ black-box. + +## Why heuropt + +There are other Rust optimization crates and many more in Python (pymoo, +hyperopt, optuna, DEAP). heuropt's design priorities: + +1. **Approachable code.** No trait objects in the public API. No + GATs, HRTBs, generic-RNG plumbing. A junior Rust engineer should + be able to read `RandomSearch` and write a new optimizer by + implementing only the `Optimizer

` trait. +2. **One concrete RNG type.** Seeded determinism is a property tested + across the crate; identical inputs always produce identical + outputs. +3. **Algorithms that work.** Every algorithm is benchmarked against + the canonical test problems (ZDT, DTLZ, Rastrigin, Rosenbrock, + Ackley) and the results are checked into [examples/compare-results.md](https://github.com/swaits/heuropt/blob/main/examples/compare-results.md) + so you can see what each algorithm's strengths actually are. +4. **Testing as a first-class concern.** 316+ unit / integration / + property tests, eight cargo-fuzz targets in CI, gungraun + instruction-count benchmarks. The fuzzers find real bugs and the + property tests check actual invariants. + +## What's in the box + +heuropt v0.5 ships **35 algorithms** spanning: + +- Single-objective continuous: `RandomSearch`, `HillClimber`, + `OnePlusOneEs`, `SimulatedAnnealing`, `GeneticAlgorithm`, + `ParticleSwarm`, `DifferentialEvolution`, `Tlbo`, `CmaEs`, + `IpopCmaEs`, `SeparableNes`, `NelderMead`. +- Single-objective other types: `Umda` (binary), `TabuSearch` + (any), `AntColonyTsp` (permutation). +- Multi-objective (2–3): `Paes`, `Nsga2`, `Spea2`, `Mopso`, `Ibea`, + `SmsEmoa`, `HypE`, `EpsilonMoea`, `PesaII`, `AgeMoea`, `Knea`, + `Moead`. +- Many-objective (4+): `Nsga3`, `Rvea`, `Grea`. +- Sample-efficient / multi-fidelity: `BayesianOpt`, `Tpe`, + `Hyperband`. + +Plus the operators (SBX, PolynomialMutation, BoundedGaussianMutation, +LevyMutation, BitFlipMutation, SwapMutation, ClampToBounds, +ProjectToSimplex), the metrics (hypervolume, spacing), and the Pareto +utilities (dominance, fronts, crowding distance, Das–Dennis reference +points, the `ParetoArchive`) that you'd expect. + +## How to use this guide + +If you're new to heuropt, read it linearly: + +1. [Five-minute walkthrough](./getting-started.md) — install, define + a problem, run an optimizer, look at the result. +2. [Defining a problem](./defining-problems.md) — the `Problem` + trait in depth: single- vs multi-objective, constraints, custom + decision types. +3. [Choosing an algorithm](./choosing-an-algorithm.md) — the + decision tree, expanded with the reasoning behind each branch. + +If you're already up and running, jump into the [cookbook](./cookbook.md) +for recipes, or [comparison](./comparison.md) for how heuropt stacks +up against other libraries. diff --git a/docs/book/src/migration.md b/docs/book/src/migration.md new file mode 100644 index 0000000..d9c2c86 --- /dev/null +++ b/docs/book/src/migration.md @@ -0,0 +1,69 @@ +# Migration guides + +Per-release notes for upgrading between heuropt versions. Skip the +sections that don't apply to your starting version. + +## To 0.5 + +### From 0.4.x + +**No public-API changes.** v0.5 is a documentation-and-polish release. +Bumping `heuropt = "0.5"` in your Cargo.toml is enough. + +What changed: + +- Added a comprehensive mdbook user guide (this book). +- Added runnable rustdoc examples on every public algorithm, + operator, metric, and Pareto utility. +- Added real-world `examples/portfolio.rs`, + `examples/hyperparam_tuning.rs`, and `examples/scheduling.rs`. +- Added `CONTRIBUTING.md`, `SECURITY.md`, `CODE_OF_CONDUCT.md` + (Builder's Code of Conduct), GitHub issue templates, and PR + template. + +The full list is in CHANGELOG.md. + +### From earlier than 0.4 + +If you're coming from 0.3.x or earlier, also read the older sections +below. + +## To 0.4 + +### From 0.3.x + +**No public-API changes.** v0.4 was a testing-infrastructure +expansion + perf pass. Same `cargo update` story. + +The compare-harness wall-clock got 3.27× faster on v0.4 with +bit-identical quality metrics, so any benchmark numbers you have +from v0.3 are still numerically accurate but will run faster. + +## To 0.3 + +### From 0.2.x + +**Additive only.** New algorithms (`BayesianOpt`, `Tpe`, +`OnePlusOneEs`, `IpopCmaEs`, `SeparableNes`, `NelderMead`, +`Hyperband`), new operators (`LevyMutation`, `ClampToBounds`, +`ProjectToSimplex`), new traits (`PartialProblem`, `Repair`). + +`CmaEsConfig` gained an `initial_mean: Option>` field; +existing call sites need a `.. CmaEsConfig { initial_mean: None, +.. }` update. + +## To 0.2 + +### From 0.1.x + +**Additive.** New algorithms across the catalog (HillClimber, SA, +GA, PSO, CMA-ES, TabuSearch, AntColonyTsp, Umda, TLBO, MOPSO, IBEA, +SMS-EMOA, HypE, RVEA, PESA-II, ε-MOEA, AGE-MOEA, GrEA, KnEA), new +operators (`SimulatedBinaryCrossover`, `PolynomialMutation`, +`CompositeVariation`, `BoundedGaussianMutation`), and the +`hypervolume_nd` metric. + +`Optimizer

` impls now require `P: Sync` and `P::Decision: Send` +(this enables the `parallel` feature without changing the public +trait surface). Any normal `Problem` you've written satisfies these +bounds automatically. diff --git a/docs/book/src/stability.md b/docs/book/src/stability.md new file mode 100644 index 0000000..0da4dd7 --- /dev/null +++ b/docs/book/src/stability.md @@ -0,0 +1,96 @@ +# Stability and SemVer + +heuropt is pre-1.0. The public API may change between minor versions. +This page sets explicit expectations. + +## What "public API" means in heuropt + +The crate's public surface is everything re-exported from +[`heuropt::prelude`] plus the items reachable from `heuropt::core`, +`heuropt::traits`, `heuropt::operators`, `heuropt::algorithms`, +`heuropt::pareto`, `heuropt::metrics`, and `heuropt::selection`. + +Items in `heuropt::internal` (e.g. the Cholesky / eigendecomposition +helpers) are **not** public API. They may change between any two +versions — use them at your own risk. + +## SemVer in heuropt 0.x + +While we are pre-1.0: + +- **Minor bumps (`0.5 → 0.6`) may break the public API.** The + CHANGELOG calls out everything that changed, and a **migration + guide** in this book documents the move. +- **Patch bumps (`0.5.0 → 0.5.1`) only contain bug fixes, + performance improvements, and additive non-breaking features.** + No deprecations, no removals. + +## What's actually likely to change before 1.0 + +In rough order of likelihood: + +1. **`Optimizer

` may grow new optional methods** for callbacks, + stop conditions, and save/resume support. These will land as + methods with default implementations so existing trait impls + keep compiling, but the trait shape will be different. +2. **Algorithm config structs may gain fields.** All current configs + are public-field structs; adding a non-`Default` field is a + breaking change. We may switch to builder patterns to avoid this + class of break, or we may add `#[non_exhaustive]`. +3. **The `Snapshot`, `Observer`, and `Checkpoint` types** (planned + for a future release) will land as new public surfaces. +4. **Some operators may move between `operators` and `pareto`** as + the boundary between "things that produce candidates" and "Pareto + utilities" gets clearer. + +What is **not** likely to change: + +- The `Problem` trait shape. +- The `Variation` / `Initializer` / `Repair` traits. +- The `Evaluation` / `Candidate` / `Population` / `OptimizationResult` + data types. +- The seeded determinism property. + +## What "bit-identical" means for stability + +heuropt promises that a given algorithm + seed + config produces the +same numeric output on the same minor version of heuropt. + +Across minor versions, output may change if an algorithm's +implementation changes (e.g. a perf rewrite that reorders +floating-point operations, or a new feature that changes the +RNG-consumption pattern). The CHANGELOG calls this out explicitly +when it happens. As of v0.5, the entire history of perf optimizations +has been bit-identical against the v0.3.0 reference. + +## MSRV (minimum supported Rust version) + +heuropt's MSRV is **1.85** as of v0.5. This is tested in CI against +every PR. + +MSRV bumps are treated as patch-bump-eligible (they don't break the +public API). When the MSRV is bumped, the CHANGELOG entry for that +release will note the new MSRV. + +## Feature-flag stability + +The current optional features: + +- `serde` — adds `Serialize` / `Deserialize` derives on the core data + types. +- `parallel` — rayon-backed parallel population evaluation. + +Features added in 0.x can be renamed or removed in any minor bump +that documents the change. Removing a feature is treated like a +breaking API change. + +## How to track changes + +- **CHANGELOG.md** — the canonical record of changes per release. +- **Migration guides** — per-release, in this book at + [migration](./migration.md). +- **GitHub releases** — each tag has release notes. +- **Watch the repo** — https://github.com/swaits/heuropt — to be + notified of new releases. + +[`heuropt::prelude`]: https://docs.rs/heuropt/latest/heuropt/prelude/index.html diff --git a/examples/hyperparam_tuning.rs b/examples/hyperparam_tuning.rs new file mode 100644 index 0000000..875188d --- /dev/null +++ b/examples/hyperparam_tuning.rs @@ -0,0 +1,124 @@ +//! Tune a synthetic ML model's hyperparameters with Bayesian Optimization +//! and (separately) Tree-structured Parzen Estimator. +//! +//! The "model" here is a deterministic function over `(learning_rate, +//! weight_decay, depth)` that mimics the shape of a real validation-loss +//! surface — a noisy minimum near sensible hyperparameters with sharp +//! penalties as you stray. It's compute-cheap so the example runs in +//! seconds, but the *workflow* is exactly what you'd use on a real +//! 30-second-per-eval model. +//! +//! Demonstrates: +//! - Sample-efficient optimization: 60 evaluations total, not 60,000. +//! - Comparing BO vs TPE on the same problem with the same budget. +//! - Decoding decision vectors with mixed scales (log-uniform learning +//! rate, integer-valued depth) using transforms inside `evaluate`. +//! +//! Run with: `cargo run --release --example hyperparam_tuning` + +use heuropt::prelude::*; + +/// A pretend deep-learning model whose validation loss is a +/// reproducible analytic function of three hyperparameters. +struct ModelTuning; + +impl Problem for ModelTuning { + type Decision = Vec; + + fn objectives(&self) -> ObjectiveSpace { + ObjectiveSpace::new(vec![Objective::minimize("val_loss")]) + } + + fn evaluate(&self, x: &Vec) -> Evaluation { + // The decision vector is in [0, 1] per dim; we decode each axis + // into the "real" hyperparameter space. + let lr = log_uniform(x[0], 1e-5, 1e-1); // learning rate + let wd = log_uniform(x[1], 1e-6, 1e-2); // weight decay + let depth = scale_to_int(x[2], 2, 12); // num layers + + // Synthetic validation loss surface: + // * minimum at lr ≈ 1e-3, wd ≈ 1e-4, depth = 6 + // * log-quadratic in lr / wd (typical hyperparameter shape) + // * mild penalty for depth far from 6 + // * tiny deterministic "noise" so flat regions don't all tie + let lr_term = (lr.log10() - (-3.0)).powi(2); + let wd_term = (wd.log10() - (-4.0)).powi(2); + let depth_term = 0.05 * ((depth as f64 - 6.0).abs()); + let noise = 0.02 * ((10.0 * x[0] + 17.0 * x[1] + 23.0 * x[2]).sin()); + + let val_loss = 0.05 + 0.3 * lr_term + 0.2 * wd_term + depth_term + noise; + Evaluation::new(vec![val_loss]) + } +} + +fn log_uniform(unit: f64, lo: f64, hi: f64) -> f64 { + let log_lo = lo.ln(); + let log_hi = hi.ln(); + (log_lo + unit * (log_hi - log_lo)).exp() +} + +fn scale_to_int(unit: f64, lo: i32, hi: i32) -> i32 { + let span = (hi - lo + 1) as f64; + let i = (unit * span).floor() as i32; + (lo + i).min(hi) +} + +fn run_bo(seed: u64) -> OptimizationResult> { + let mut opt = BayesianOpt::new( + BayesianOptConfig { + initial_samples: 10, + iterations: 50, // 60 total evals + length_scales: None, + signal_variance: 1.0, + noise_variance: 1e-6, + acquisition_samples: 200, + seed, + }, + RealBounds::new(vec![(0.0, 1.0); 3]), + ); + opt.run(&ModelTuning) +} + +fn run_tpe(seed: u64) -> OptimizationResult> { + let mut opt = Tpe::new( + TpeConfig { + initial_samples: 10, + iterations: 50, // 60 total evals + good_fraction: 0.25, + candidate_samples: 64, + bandwidth_factor: 1.0, + seed, + }, + RealBounds::new(vec![(0.0, 1.0); 3]), + ); + opt.run(&ModelTuning) +} + +fn report(name: &str, r: &OptimizationResult>) { + let best = r.best.as_ref().expect("at least one feasible candidate"); + let lr = log_uniform(best.decision[0], 1e-5, 1e-1); + let wd = log_uniform(best.decision[1], 1e-6, 1e-2); + let depth = scale_to_int(best.decision[2], 2, 12); + println!( + "{:<8} val_loss = {:>7.4} | lr = {:>10.2e} wd = {:>10.2e} depth = {} | evals = {}", + name, best.evaluation.objectives[0], lr, wd, depth, r.evaluations, + ); +} + +fn main() { + println!("Tuning ModelTuning (synthetic 3-D loss surface)"); + println!("Optimum: lr ≈ 1e-3, wd ≈ 1e-4, depth = 6, val_loss ≈ 0.03"); + println!(); + println!( + "{:<8} {:<26} {:<24} {:<24}", + "alg", "best", "(decoded hyperparams)", "(eval budget)" + ); + for seed in 0..5 { + println!(); + println!("seed {}:", seed); + let bo = run_bo(seed); + let tpe = run_tpe(seed); + report("BO", &bo); + report("TPE", &tpe); + } +} diff --git a/examples/portfolio.rs b/examples/portfolio.rs new file mode 100644 index 0000000..797caea --- /dev/null +++ b/examples/portfolio.rs @@ -0,0 +1,210 @@ +//! Multi-objective portfolio optimization with a budget constraint. +//! +//! Real-world flavor: pick a portfolio over five synthetic assets that +//! trades off **return** (maximize) against **risk** (minimize). Weights +//! must be non-negative and sum to 1.0 (the standard probability-simplex +//! budget constraint). +//! +//! Demonstrates: +//! - Multi-objective formulation with a maximize axis (return) and a +//! minimize axis (variance-based risk). +//! - The `ProjectToSimplex` repair operator wired into a `Repair`-aware +//! variation pipeline so every offspring respects the budget. +//! - NSGA-II producing a Pareto front of trade-offs. +//! - Picking one answer off the front via a-posteriori weighting (see +//! `docs/book/src/cookbook/pick-one.md`). +//! +//! Run with: `cargo run --release --example portfolio` + +use heuropt::prelude::*; + +/// Five-asset toy market. Means and a covariance matrix you'd estimate +/// from real returns; here they're synthetic but realistic-shape. +struct Portfolio { + /// Expected per-period returns (one per asset). + expected_returns: [f64; 5], + /// Symmetric 5×5 covariance matrix. + covariance: [[f64; 5]; 5], +} + +impl Problem for Portfolio { + type Decision = Vec; + + fn objectives(&self) -> ObjectiveSpace { + ObjectiveSpace::new(vec![ + Objective::maximize("return"), + Objective::minimize("risk"), + ]) + } + + fn evaluate(&self, weights: &Vec) -> Evaluation { + // Expected return: w · μ + let r: f64 = weights + .iter() + .zip(self.expected_returns.iter()) + .map(|(w, m)| w * m) + .sum(); + + // Risk (portfolio variance): w · Σ · w + let mut risk = 0.0; + for i in 0..5 { + for j in 0..5 { + risk += weights[i] * self.covariance[i][j] * weights[j]; + } + } + + Evaluation::new(vec![r, risk]) + } +} + +/// Variation pipeline that respects the simplex constraint: SBX + +/// PolyMut produce real-valued children, then `ProjectToSimplex` projects +/// them back onto `{ w : w ≥ 0, Σw = 1 }`. +struct SimplexVariation { + crossover: SimulatedBinaryCrossover, + mutation: PolynomialMutation, + repair: ProjectToSimplex, +} + +impl Variation> for SimplexVariation { + fn vary(&mut self, parents: &[Vec], rng: &mut Rng) -> Vec> { + let crossed = self.crossover.vary(parents, rng); + let mut out = Vec::with_capacity(crossed.len()); + for child in crossed { + let mut mutated = self + .mutation + .vary(std::slice::from_ref(&child), rng) + .pop() + .expect("PolynomialMutation returned no child"); + self.repair.repair(&mut mutated); + out.push(mutated); + } + out + } +} + +/// `Initializer` that uniformly samples points on the simplex via the +/// standard "log-and-normalize" trick. Every initial member is feasible +/// by construction. +struct SimplexInit { + dim: usize, +} + +impl Initializer> for SimplexInit { + fn initialize(&mut self, size: usize, rng: &mut Rng) -> Vec> { + use rand::Rng as _; + let mut out = Vec::with_capacity(size); + for _ in 0..size { + // Sample exponentials, normalize → uniform on simplex. + let mut e: Vec = (0..self.dim) + .map(|_| -(1.0_f64 - rng.random::()).ln()) + .collect(); + let s: f64 = e.iter().sum(); + for v in e.iter_mut() { + *v /= s; + } + out.push(e); + } + out + } +} + +fn main() { + let problem = Portfolio { + // Synthetic but plausible: 8% / 12% / 5% / 15% / 3% expected + // returns. The two "stocks" (B, D) have higher expected return + // and higher variance than the bonds / cash equivalents. + expected_returns: [0.08, 0.12, 0.05, 0.15, 0.03], + covariance: [ + [0.04, 0.02, 0.01, 0.03, 0.005], + [0.02, 0.10, 0.01, 0.05, 0.005], + [0.01, 0.01, 0.02, 0.01, 0.005], + [0.03, 0.05, 0.01, 0.16, 0.005], + [0.005, 0.005, 0.005, 0.005, 0.001], + ], + }; + + let bounds = vec![(0.0_f64, 1.0_f64); 5]; + + let variation = SimplexVariation { + crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 1.0), + mutation: PolynomialMutation::new(bounds.clone(), 20.0, 1.0 / 5.0), + repair: ProjectToSimplex::new(1.0), + }; + + let mut opt = Nsga2::new( + Nsga2Config { + population_size: 100, + generations: 200, + seed: 42, + }, + SimplexInit { dim: 5 }, + variation, + ); + + let result = opt.run(&problem); + + println!("Pareto front size: {}", result.pareto_front.len()); + println!("Total evaluations: {}", result.evaluations); + + // Pick one: a-posteriori weighted decision favoring return slightly. + // Lower score = preferred. We compare in oriented space (maximize + // axis already flipped to negative by `as_minimization`). + let space = problem.objectives(); + let weights = [1.0, 1.5]; // weight risk a bit more than -return + let chosen = result + .pareto_front + .iter() + .min_by(|a, b| { + let ax: f64 = space + .as_minimization(&a.evaluation.objectives) + .iter() + .zip(&weights) + .map(|(v, w)| v * w) + .sum(); + let bx: f64 = space + .as_minimization(&b.evaluation.objectives) + .iter() + .zip(&weights) + .map(|(v, w)| v * w) + .sum(); + ax.partial_cmp(&bx).unwrap_or(std::cmp::Ordering::Equal) + }) + .expect("non-empty front"); + + println!(); + println!( + "Picked portfolio: weights = [{:.3}, {:.3}, {:.3}, {:.3}, {:.3}]", + chosen.decision[0], + chosen.decision[1], + chosen.decision[2], + chosen.decision[3], + chosen.decision[4], + ); + println!( + " expected return: {:>6.4}", + chosen.evaluation.objectives[0] + ); + println!( + " risk (variance): {:>6.4}", + chosen.evaluation.objectives[1] + ); + + // Print 5 representative points across the front. + println!(); + println!("Sample of the front (return, risk):"); + let mut sorted = result.pareto_front.clone(); + sorted.sort_by(|a, b| { + a.evaluation.objectives[0] + .partial_cmp(&b.evaluation.objectives[0]) + .unwrap_or(std::cmp::Ordering::Equal) + }); + let n = sorted.len(); + for k in (0..n).step_by((n / 5).max(1)) { + let c = &sorted[k]; + println!( + " return = {:.4}, risk = {:.4}", + c.evaluation.objectives[0], c.evaluation.objectives[1], + ); + } +} diff --git a/examples/scheduling.rs b/examples/scheduling.rs new file mode 100644 index 0000000..364eb5e --- /dev/null +++ b/examples/scheduling.rs @@ -0,0 +1,132 @@ +//! Single-machine job-shop scheduling: minimize total weighted +//! completion time given per-job processing times and due-date weights. +//! +//! The decision is a permutation `Vec` — the order in which +//! jobs are processed. We use `SimulatedAnnealing` paired with +//! `SwapMutation` (the standard generic-permutation pair). +//! +//! Demonstrates: +//! - Permutation decisions (`Vec`). +//! - Simulated annealing with a custom `Initializer` that produces a +//! randomly shuffled identity permutation. +//! - `SwapMutation` preserving the permutation invariant for free. +//! +//! Run with: `cargo run --release --example scheduling` + +use heuropt::prelude::*; + +/// Single-machine weighted-completion-time problem (1 || Σwᵢ Cᵢ). +struct Scheduling { + /// Processing time for each job. + process_times: Vec, + /// Importance weight for each job. Higher weight = more + /// punishing if the job finishes late. + weights: Vec, +} + +impl Problem for Scheduling { + type Decision = Vec; + + fn objectives(&self) -> ObjectiveSpace { + ObjectiveSpace::new(vec![Objective::minimize("total_wct")]) + } + + fn evaluate(&self, schedule: &Vec) -> Evaluation { + // Compute each job's completion time as the running sum of + // processing times in the chosen order. + let mut clock = 0.0_f64; + let mut total_wct = 0.0_f64; + for &job in schedule { + clock += self.process_times[job]; + total_wct += self.weights[job] * clock; + } + Evaluation::new(vec![total_wct]) + } +} + +/// Initializer that produces a single randomly-shuffled permutation +/// `[0, 1, …, n-1]`. Simulated annealing only needs one initial decision. +struct ShuffledPerm { + n: usize, +} + +impl Initializer> for ShuffledPerm { + fn initialize(&mut self, _size: usize, rng: &mut Rng) -> Vec> { + use rand::seq::SliceRandom; + let mut perm: Vec = (0..self.n).collect(); + perm.shuffle(rng); + vec![perm] + } +} + +fn main() { + // 12 jobs. The optimal policy is the Smith's-rule order: sort by + // p_i / w_i ascending (shortest weighted processing time first). + // We can compute that directly to compare against the search result. + let jobs = [ + (3.0_f64, 2.0_f64), + (5.0, 1.0), + (2.0, 4.0), + (8.0, 3.0), + (4.0, 5.0), + (1.0, 2.0), + (7.0, 6.0), + (6.0, 1.0), + (3.0, 3.0), + (5.0, 4.0), + (2.0, 2.0), + (4.0, 1.0), + ]; + let process_times: Vec = jobs.iter().map(|j| j.0).collect(); + let weights: Vec = jobs.iter().map(|j| j.1).collect(); + let n = jobs.len(); + + let problem = Scheduling { + process_times: process_times.clone(), + weights: weights.clone(), + }; + + // Smith's rule oracle: sort jobs by p / w ascending. + let mut smith_order: Vec = (0..n).collect(); + smith_order.sort_by(|&a, &b| { + let ra = process_times[a] / weights[a]; + let rb = process_times[b] / weights[b]; + ra.partial_cmp(&rb).unwrap_or(std::cmp::Ordering::Equal) + }); + let smith_score = problem.evaluate(&smith_order).objectives[0]; + + // Search via simulated annealing with swap mutation. + let mut opt = SimulatedAnnealing::new( + SimulatedAnnealingConfig { + iterations: 5_000, + initial_temperature: 50.0, + final_temperature: 1e-3, + seed: 42, + }, + ShuffledPerm { n }, + SwapMutation, + ); + let result = opt.run(&problem); + + let best = result.best.unwrap(); + println!("Single-machine weighted completion time, {} jobs", n); + println!(); + println!( + "Smith's-rule oracle: {:>8.2} order = {:?}", + smith_score, smith_order + ); + println!( + "Simulated annealing best: {:>8.2} order = {:?}", + best.evaluation.objectives[0], best.decision, + ); + println!( + "Random initial schedule: {:>8.2} order = {:?}", + problem.evaluate(&(0..n).collect()).objectives[0], + (0..n).collect::>(), + ); + println!(); + println!( + "SA reached optimum (Smith): {}", + (best.evaluation.objectives[0] - smith_score).abs() < 1e-9 + ); +} diff --git a/src/algorithms/age_moea.rs b/src/algorithms/age_moea.rs index f902e4e..6414fc7 100644 --- a/src/algorithms/age_moea.rs +++ b/src/algorithms/age_moea.rs @@ -40,6 +40,35 @@ impl Default for AgeMoeaConfig { /// score survivors by a combination of proximity (distance to the /// translated origin in the L_p frame) and diversity (distance to the /// nearest survivor in the same frame). +/// +/// # Example +/// +/// ``` +/// use heuropt::prelude::*; +/// +/// struct Schaffer; +/// impl Problem for Schaffer { +/// type Decision = Vec; +/// fn objectives(&self) -> ObjectiveSpace { +/// ObjectiveSpace::new(vec![Objective::minimize("f1"), Objective::minimize("f2")]) +/// } +/// fn evaluate(&self, x: &Vec) -> Evaluation { +/// Evaluation::new(vec![x[0] * x[0], (x[0] - 2.0).powi(2)]) +/// } +/// } +/// +/// let bounds = vec![(-5.0_f64, 5.0_f64)]; +/// let mut opt = AgeMoea::new( +/// AgeMoeaConfig { population_size: 30, generations: 20, seed: 42 }, +/// RealBounds::new(bounds.clone()), +/// CompositeVariation { +/// crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5), +/// mutation: PolynomialMutation::new(bounds, 20.0, 1.0), +/// }, +/// ); +/// let r = opt.run(&Schaffer); +/// assert!(!r.pareto_front.is_empty()); +/// ``` #[derive(Debug, Clone)] pub struct AgeMoea { /// Algorithm configuration. diff --git a/src/algorithms/ant_colony_tsp.rs b/src/algorithms/ant_colony_tsp.rs index 424937d..84f659e 100644 --- a/src/algorithms/ant_colony_tsp.rs +++ b/src/algorithms/ant_colony_tsp.rs @@ -57,6 +57,53 @@ impl Default for AntColonyTspConfig { /// Each ant builds a tour by repeatedly choosing the next node with /// probability `∝ τ_ij^α · η_ij^β` over the unvisited cities, where /// `η_ij = 1 / distance_ij` is the heuristic desirability. +/// +/// # Example +/// +/// ``` +/// use heuropt::prelude::*; +/// +/// struct Tsp { distances: Vec> } +/// impl Problem for Tsp { +/// type Decision = Vec; +/// fn objectives(&self) -> ObjectiveSpace { +/// ObjectiveSpace::new(vec![Objective::minimize("length")]) +/// } +/// fn evaluate(&self, tour: &Vec) -> Evaluation { +/// let mut len = 0.0; +/// for w in tour.windows(2) { len += self.distances[w[0]][w[1]]; } +/// len += self.distances[*tour.last().unwrap()][tour[0]]; +/// Evaluation::new(vec![len]) +/// } +/// } +/// +/// // 5 cities laid out in a small square + center. The optimal tour +/// // is the perimeter; the diagonal is suboptimal. +/// let cities = [(0.0_f64, 0.0), (3.0, 0.0), (3.0, 3.0), (0.0, 3.0), (1.5, 1.5)]; +/// let n = cities.len(); +/// let mut d = vec![vec![0.0; n]; n]; +/// for i in 0..n { +/// for j in 0..n { +/// let dx = cities[i].0 - cities[j].0; +/// let dy = cities[i].1 - cities[j].1; +/// d[i][j] = (dx * dx + dy * dy).sqrt(); +/// } +/// } +/// let problem = Tsp { distances: d.clone() }; +/// +/// let mut opt = AntColonyTsp::new(AntColonyTspConfig { +/// ants: 10, +/// generations: 50, +/// alpha: 1.0, +/// beta: 5.0, +/// evaporation: 0.5, +/// deposit: 1.0, +/// initial_pheromone: 0.1, +/// seed: 42, +/// }, d); +/// let r = opt.run(&problem); +/// assert!(r.best.is_some()); +/// ``` pub struct AntColonyTsp { /// Algorithm configuration. pub config: AntColonyTspConfig, diff --git a/src/algorithms/bayesian_opt.rs b/src/algorithms/bayesian_opt.rs index 1c42b7e..f905582 100644 --- a/src/algorithms/bayesian_opt.rs +++ b/src/algorithms/bayesian_opt.rs @@ -61,6 +61,40 @@ impl Default for BayesianOptConfig { /// evaluation budgets (50–500). The GP kernel is anisotropic RBF; the /// acquisition function is EI; both are optimized by best-of-N random /// sampling each step (simple, predictable cost). +/// +/// # Example +/// +/// ``` +/// use heuropt::prelude::*; +/// +/// struct Sphere; +/// impl Problem for Sphere { +/// type Decision = Vec; +/// fn objectives(&self) -> ObjectiveSpace { +/// ObjectiveSpace::new(vec![Objective::minimize("f")]) +/// } +/// fn evaluate(&self, x: &Vec) -> Evaluation { +/// Evaluation::new(vec![x.iter().map(|v| v * v).sum::()]) +/// } +/// } +/// +/// let mut opt = BayesianOpt::new( +/// BayesianOptConfig { +/// initial_samples: 10, +/// iterations: 30, +/// length_scales: None, // default per-axis length scales +/// signal_variance: 1.0, +/// noise_variance: 1e-6, +/// acquisition_samples: 200, +/// seed: 42, +/// }, +/// RealBounds::new(vec![(-3.0, 3.0); 3]), +/// ); +/// let r = opt.run(&Sphere); +/// // 10 random + 30 BO steps = 40 total evaluations. +/// assert_eq!(r.evaluations, 40); +/// assert!(r.best.is_some()); +/// ``` #[derive(Debug, Clone)] pub struct BayesianOpt { /// Algorithm configuration. diff --git a/src/algorithms/cma_es.rs b/src/algorithms/cma_es.rs index 1fee069..30dfef0 100644 --- a/src/algorithms/cma_es.rs +++ b/src/algorithms/cma_es.rs @@ -59,6 +59,38 @@ impl Default for CmaEsConfig { /// `Vec` decisions only. Bounds come from the embedded `RealBounds` /// field; both the initial mean and every offspring are clamped per /// dimension. +/// +/// # Example +/// +/// ``` +/// use heuropt::prelude::*; +/// +/// struct Sphere; +/// impl Problem for Sphere { +/// type Decision = Vec; +/// fn objectives(&self) -> ObjectiveSpace { +/// ObjectiveSpace::new(vec![Objective::minimize("f")]) +/// } +/// fn evaluate(&self, x: &Vec) -> Evaluation { +/// Evaluation::new(vec![x.iter().map(|v| v * v).sum::()]) +/// } +/// } +/// +/// let mut opt = CmaEs::new( +/// CmaEsConfig { +/// population_size: 12, +/// generations: 100, +/// initial_sigma: 1.0, +/// eigen_decomposition_period: 1, +/// initial_mean: None, +/// seed: 42, +/// }, +/// RealBounds::new(vec![(-5.0, 5.0); 5]), +/// ); +/// let r = opt.run(&Sphere); +/// // CMA-ES converges aggressively on Sphere. +/// assert!(r.best.unwrap().evaluation.objectives[0] < 1e-3); +/// ``` #[derive(Debug, Clone)] pub struct CmaEs { /// Algorithm configuration. diff --git a/src/algorithms/differential_evolution.rs b/src/algorithms/differential_evolution.rs index c2780a9..98166e3 100644 --- a/src/algorithms/differential_evolution.rs +++ b/src/algorithms/differential_evolution.rs @@ -44,6 +44,37 @@ impl Default for DifferentialEvolutionConfig { /// /// `Vec` decisions only; single-objective problems only. Bounds come from /// the embedded `RealBounds`, and mutant vectors are clamped to those bounds. +/// +/// # Example +/// +/// ``` +/// use heuropt::prelude::*; +/// +/// struct Sphere; +/// impl Problem for Sphere { +/// type Decision = Vec; +/// fn objectives(&self) -> ObjectiveSpace { +/// ObjectiveSpace::new(vec![Objective::minimize("f")]) +/// } +/// fn evaluate(&self, x: &Vec) -> Evaluation { +/// Evaluation::new(vec![x.iter().map(|v| v * v).sum::()]) +/// } +/// } +/// +/// let mut opt = DifferentialEvolution::new( +/// DifferentialEvolutionConfig { +/// population_size: 20, +/// generations: 50, +/// differential_weight: 0.5, +/// crossover_probability: 0.9, +/// seed: 42, +/// }, +/// RealBounds::new(vec![(-5.0, 5.0); 5]), +/// ); +/// let r = opt.run(&Sphere); +/// // DE crushes Sphere; expect very small objective. +/// assert!(r.best.unwrap().evaluation.objectives[0] < 1e-3); +/// ``` #[derive(Debug, Clone)] pub struct DifferentialEvolution { /// Algorithm configuration. diff --git a/src/algorithms/epsilon_moea.rs b/src/algorithms/epsilon_moea.rs index a042f55..6df3078 100644 --- a/src/algorithms/epsilon_moea.rs +++ b/src/algorithms/epsilon_moea.rs @@ -39,6 +39,45 @@ impl Default for EpsilonMoeaConfig { } /// ε-dominance MOEA. +/// +/// Steady-state EA with an ε-grid archive: every member that lands in +/// the same ε-box as an existing one is replaced by the closer point +/// to the box's grid corner. Auto-bounds the front size by the choice +/// of `epsilon`. +/// +/// # Example +/// +/// ``` +/// use heuropt::prelude::*; +/// +/// struct Schaffer; +/// impl Problem for Schaffer { +/// type Decision = Vec; +/// fn objectives(&self) -> ObjectiveSpace { +/// ObjectiveSpace::new(vec![Objective::minimize("f1"), Objective::minimize("f2")]) +/// } +/// fn evaluate(&self, x: &Vec) -> Evaluation { +/// Evaluation::new(vec![x[0] * x[0], (x[0] - 2.0).powi(2)]) +/// } +/// } +/// +/// let bounds = vec![(-5.0_f64, 5.0_f64)]; +/// let mut opt = EpsilonMoea::new( +/// EpsilonMoeaConfig { +/// population_size: 20, +/// evaluations: 1_000, +/// epsilon: vec![0.1, 0.1], +/// seed: 42, +/// }, +/// RealBounds::new(bounds.clone()), +/// CompositeVariation { +/// crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5), +/// mutation: PolynomialMutation::new(bounds, 20.0, 1.0), +/// }, +/// ); +/// let r = opt.run(&Schaffer); +/// assert!(!r.pareto_front.is_empty()); +/// ``` #[derive(Debug, Clone)] pub struct EpsilonMoea { /// Algorithm configuration. diff --git a/src/algorithms/genetic_algorithm.rs b/src/algorithms/genetic_algorithm.rs index 74882e3..3668afa 100644 --- a/src/algorithms/genetic_algorithm.rs +++ b/src/algorithms/genetic_algorithm.rs @@ -46,6 +46,41 @@ impl Default for GeneticAlgorithmConfig { /// produces offspring, those are evaluated, and the next population is /// the top `elitism` from the previous generation plus the best /// `population_size - elitism` offspring (by fitness). +/// +/// # Example +/// +/// ``` +/// use heuropt::prelude::*; +/// +/// struct Sphere; +/// impl Problem for Sphere { +/// type Decision = Vec; +/// fn objectives(&self) -> ObjectiveSpace { +/// ObjectiveSpace::new(vec![Objective::minimize("f")]) +/// } +/// fn evaluate(&self, x: &Vec) -> Evaluation { +/// Evaluation::new(vec![x.iter().map(|v| v * v).sum::()]) +/// } +/// } +/// +/// let bounds = vec![(-5.0_f64, 5.0_f64); 3]; +/// let mut opt = GeneticAlgorithm::new( +/// GeneticAlgorithmConfig { +/// population_size: 30, +/// generations: 50, +/// tournament_size: 2, +/// elitism: 2, +/// seed: 42, +/// }, +/// RealBounds::new(bounds.clone()), +/// CompositeVariation { +/// crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5), +/// mutation: PolynomialMutation::new(bounds, 20.0, 1.0), +/// }, +/// ); +/// let r = opt.run(&Sphere); +/// assert!(r.best.is_some()); +/// ``` #[derive(Debug, Clone)] pub struct GeneticAlgorithm { /// Algorithm configuration. diff --git a/src/algorithms/grea.rs b/src/algorithms/grea.rs index 8de5c92..927d607 100644 --- a/src/algorithms/grea.rs +++ b/src/algorithms/grea.rs @@ -38,6 +38,40 @@ impl Default for GreaConfig { } /// Grid-based Evolutionary Algorithm (GrEA). +/// +/// Many-objective EA that uses three grid-based metrics — grid rank, +/// grid crowding distance, and grid coordinate point distance — to +/// select survivors. Particularly strong on linear / simplex-shaped +/// fronts (e.g. DTLZ1). +/// +/// # Example +/// +/// ``` +/// use heuropt::prelude::*; +/// +/// struct Schaffer; +/// impl Problem for Schaffer { +/// type Decision = Vec; +/// fn objectives(&self) -> ObjectiveSpace { +/// ObjectiveSpace::new(vec![Objective::minimize("f1"), Objective::minimize("f2")]) +/// } +/// fn evaluate(&self, x: &Vec) -> Evaluation { +/// Evaluation::new(vec![x[0] * x[0], (x[0] - 2.0).powi(2)]) +/// } +/// } +/// +/// let bounds = vec![(-5.0_f64, 5.0_f64)]; +/// let mut opt = Grea::new( +/// GreaConfig { population_size: 30, generations: 20, grid_divisions: 8, seed: 42 }, +/// RealBounds::new(bounds.clone()), +/// CompositeVariation { +/// crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5), +/// mutation: PolynomialMutation::new(bounds, 20.0, 1.0), +/// }, +/// ); +/// let r = opt.run(&Schaffer); +/// assert!(!r.pareto_front.is_empty()); +/// ``` #[derive(Debug, Clone)] pub struct Grea { /// Algorithm configuration. diff --git a/src/algorithms/hill_climber.rs b/src/algorithms/hill_climber.rs index db3378f..d63378a 100644 --- a/src/algorithms/hill_climber.rs +++ b/src/algorithms/hill_climber.rs @@ -34,6 +34,31 @@ impl Default for HillClimberConfig { /// feasible beats infeasible, smaller violation wins among infeasibles. /// /// Single-objective only. +/// +/// # Example +/// +/// ``` +/// use heuropt::prelude::*; +/// +/// struct Sphere; +/// impl Problem for Sphere { +/// type Decision = Vec; +/// fn objectives(&self) -> ObjectiveSpace { +/// ObjectiveSpace::new(vec![Objective::minimize("f")]) +/// } +/// fn evaluate(&self, x: &Vec) -> Evaluation { +/// Evaluation::new(vec![x.iter().map(|v| v * v).sum::()]) +/// } +/// } +/// +/// let mut opt = HillClimber::new( +/// HillClimberConfig { iterations: 500, seed: 42 }, +/// RealBounds::new(vec![(-5.0, 5.0); 3]), +/// GaussianMutation { sigma: 0.3 }, +/// ); +/// let r = opt.run(&Sphere); +/// assert!(r.best.is_some()); +/// ``` #[derive(Debug, Clone)] pub struct HillClimber { /// Algorithm configuration. diff --git a/src/algorithms/hype.rs b/src/algorithms/hype.rs index d6cf141..2e11cfe 100644 --- a/src/algorithms/hype.rs +++ b/src/algorithms/hype.rs @@ -48,6 +48,41 @@ impl Default for HypeConfig { /// Hypervolume Estimation Algorithm: many-objective MOEA that selects via /// Monte Carlo–estimated hypervolume contributions. +/// +/// # Example +/// +/// ``` +/// use heuropt::prelude::*; +/// +/// struct Schaffer; +/// impl Problem for Schaffer { +/// type Decision = Vec; +/// fn objectives(&self) -> ObjectiveSpace { +/// ObjectiveSpace::new(vec![Objective::minimize("f1"), Objective::minimize("f2")]) +/// } +/// fn evaluate(&self, x: &Vec) -> Evaluation { +/// Evaluation::new(vec![x[0] * x[0], (x[0] - 2.0).powi(2)]) +/// } +/// } +/// +/// let bounds = vec![(-5.0_f64, 5.0_f64)]; +/// let mut opt = Hype::new( +/// HypeConfig { +/// population_size: 20, +/// generations: 20, +/// reference_point: vec![30.0, 30.0], +/// mc_samples: 100, +/// seed: 42, +/// }, +/// RealBounds::new(bounds.clone()), +/// CompositeVariation { +/// crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5), +/// mutation: PolynomialMutation::new(bounds, 20.0, 1.0), +/// }, +/// ); +/// let r = opt.run(&Schaffer); +/// assert!(!r.pareto_front.is_empty()); +/// ``` #[derive(Debug, Clone)] pub struct Hype { /// Algorithm configuration. diff --git a/src/algorithms/hyperband.rs b/src/algorithms/hyperband.rs index 310ae2d..b602c49 100644 --- a/src/algorithms/hyperband.rs +++ b/src/algorithms/hyperband.rs @@ -52,6 +52,38 @@ impl Default for HyperbandConfig { /// low budget), later brackets favor exploitation (fewer configs run /// near the max budget). The single best result across all brackets /// is returned. +/// +/// # Example +/// +/// ``` +/// use heuropt::prelude::*; +/// use heuropt::core::partial_problem::PartialProblem; +/// +/// struct Tuning; +/// impl PartialProblem for Tuning { +/// type Decision = Vec; +/// fn objectives(&self) -> ObjectiveSpace { +/// ObjectiveSpace::new(vec![Objective::minimize("loss")]) +/// } +/// fn evaluate_at_budget(&self, x: &Vec, budget: f64) -> Evaluation { +/// // Pretend a model where more budget = lower loss. +/// let loss = x[0].powi(2) + x[1].powi(2) + 1.0 / (budget + 1.0); +/// Evaluation::new(vec![loss]) +/// } +/// } +/// +/// let mut opt = Hyperband::new( +/// HyperbandConfig { +/// max_budget: 27.0, +/// eta: 3.0, +/// max_brackets: 4, +/// seed: 42, +/// }, +/// RealBounds::new(vec![(-1.0, 1.0); 2]), +/// ); +/// let r = opt.run(&Tuning); +/// assert!(r.best.is_some()); +/// ``` pub struct Hyperband where D: Clone, diff --git a/src/algorithms/ibea.rs b/src/algorithms/ibea.rs index 1c94f60..2fea13d 100644 --- a/src/algorithms/ibea.rs +++ b/src/algorithms/ibea.rs @@ -37,6 +37,40 @@ impl Default for IbeaConfig { } /// IBEA (Indicator-Based EA) using the additive ε-indicator. +/// +/// Selects survivors by their contribution to a quality indicator +/// (additive ε) rather than by dominance + crowding. On the comparison +/// harness it consistently produces the best convergence of the dominance- +/// alternative methods on smooth and disconnected fronts alike. +/// +/// # Example +/// +/// ``` +/// use heuropt::prelude::*; +/// +/// struct Schaffer; +/// impl Problem for Schaffer { +/// type Decision = Vec; +/// fn objectives(&self) -> ObjectiveSpace { +/// ObjectiveSpace::new(vec![Objective::minimize("f1"), Objective::minimize("f2")]) +/// } +/// fn evaluate(&self, x: &Vec) -> Evaluation { +/// Evaluation::new(vec![x[0] * x[0], (x[0] - 2.0).powi(2)]) +/// } +/// } +/// +/// let bounds = vec![(-5.0_f64, 5.0_f64)]; +/// let mut opt = Ibea::new( +/// IbeaConfig { population_size: 30, generations: 20, kappa: 0.05, seed: 42 }, +/// RealBounds::new(bounds.clone()), +/// CompositeVariation { +/// crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5), +/// mutation: PolynomialMutation::new(bounds, 20.0, 1.0), +/// }, +/// ); +/// let r = opt.run(&Schaffer); +/// assert!(!r.pareto_front.is_empty()); +/// ``` #[derive(Debug, Clone)] pub struct Ibea { /// Algorithm configuration. diff --git a/src/algorithms/ipop_cma_es.rs b/src/algorithms/ipop_cma_es.rs index f17ccb0..aeaf14c 100644 --- a/src/algorithms/ipop_cma_es.rs +++ b/src/algorithms/ipop_cma_es.rs @@ -53,6 +53,42 @@ impl Default for IpopCmaEsConfig { } /// IPOP-CMA-ES: CMA-ES with population-doubling restarts. +/// +/// Specifically designed to fix vanilla CMA-ES's weakness on multimodal +/// landscapes — each restart doubles the population and randomizes the +/// initial mean to escape from local basins. On the comparison harness +/// it drops vanilla CMA-ES's Rastrigin score from f = 2.35 to f = 0.13. +/// +/// # Example +/// +/// ``` +/// use heuropt::prelude::*; +/// +/// struct Sphere; +/// impl Problem for Sphere { +/// type Decision = Vec; +/// fn objectives(&self) -> ObjectiveSpace { +/// ObjectiveSpace::new(vec![Objective::minimize("f")]) +/// } +/// fn evaluate(&self, x: &Vec) -> Evaluation { +/// Evaluation::new(vec![x.iter().map(|v| v * v).sum::()]) +/// } +/// } +/// +/// let mut opt = IpopCmaEs::new( +/// IpopCmaEsConfig { +/// initial_population_size: 8, +/// total_generations: 100, +/// initial_sigma: 1.0, +/// eigen_decomposition_period: 1, +/// stall_generations: Some(20), +/// seed: 42, +/// }, +/// RealBounds::new(vec![(-5.0, 5.0); 3]), +/// ); +/// let r = opt.run(&Sphere); +/// assert!(r.best.unwrap().evaluation.objectives[0] < 1.0); +/// ``` #[derive(Debug, Clone)] pub struct IpopCmaEs { /// Algorithm configuration. diff --git a/src/algorithms/knea.rs b/src/algorithms/knea.rs index b086d1f..0fc50d1 100644 --- a/src/algorithms/knea.rs +++ b/src/algorithms/knea.rs @@ -39,6 +39,35 @@ impl Default for KneaConfig { /// Survival selection ranks splitting-front members by perpendicular /// distance from the hyperplane connecting the front's extreme points. /// Larger distance ≈ stronger knee = preferred survivor. +/// +/// # Example +/// +/// ``` +/// use heuropt::prelude::*; +/// +/// struct Schaffer; +/// impl Problem for Schaffer { +/// type Decision = Vec; +/// fn objectives(&self) -> ObjectiveSpace { +/// ObjectiveSpace::new(vec![Objective::minimize("f1"), Objective::minimize("f2")]) +/// } +/// fn evaluate(&self, x: &Vec) -> Evaluation { +/// Evaluation::new(vec![x[0] * x[0], (x[0] - 2.0).powi(2)]) +/// } +/// } +/// +/// let bounds = vec![(-5.0_f64, 5.0_f64)]; +/// let mut opt = Knea::new( +/// KneaConfig { population_size: 30, generations: 20, seed: 42 }, +/// RealBounds::new(bounds.clone()), +/// CompositeVariation { +/// crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5), +/// mutation: PolynomialMutation::new(bounds, 20.0, 1.0), +/// }, +/// ); +/// let r = opt.run(&Schaffer); +/// assert!(!r.pareto_front.is_empty()); +/// ``` #[derive(Debug, Clone)] pub struct Knea { /// Algorithm configuration. diff --git a/src/algorithms/moead.rs b/src/algorithms/moead.rs index c44da80..44359f8 100644 --- a/src/algorithms/moead.rs +++ b/src/algorithms/moead.rs @@ -39,6 +39,45 @@ impl Default for MoeadConfig { } /// MOEA/D optimizer using the Tchebycheff scalarizing function. +/// +/// Decomposes the multi-objective problem into many single-objective +/// scalarizations along Das–Dennis weight vectors and solves them +/// in parallel with neighborhood-based mating. Very fast per generation; +/// scales naturally to many objectives. +/// +/// # Example +/// +/// ``` +/// use heuropt::prelude::*; +/// +/// struct Schaffer; +/// impl Problem for Schaffer { +/// type Decision = Vec; +/// fn objectives(&self) -> ObjectiveSpace { +/// ObjectiveSpace::new(vec![Objective::minimize("f1"), Objective::minimize("f2")]) +/// } +/// fn evaluate(&self, x: &Vec) -> Evaluation { +/// Evaluation::new(vec![x[0] * x[0], (x[0] - 2.0).powi(2)]) +/// } +/// } +/// +/// let bounds = vec![(-5.0_f64, 5.0_f64)]; +/// let mut opt = Moead::new( +/// MoeadConfig { +/// generations: 30, +/// reference_divisions: 19, // 20 weights for 2 objectives +/// neighborhood_size: 5, +/// seed: 42, +/// }, +/// RealBounds::new(bounds.clone()), +/// CompositeVariation { +/// crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5), +/// mutation: PolynomialMutation::new(bounds, 20.0, 1.0), +/// }, +/// ); +/// let r = opt.run(&Schaffer); +/// assert!(!r.pareto_front.is_empty()); +/// ``` #[derive(Debug, Clone)] pub struct Moead { /// Algorithm configuration. diff --git a/src/algorithms/mopso.rs b/src/algorithms/mopso.rs index 36e24b9..2af313c 100644 --- a/src/algorithms/mopso.rs +++ b/src/algorithms/mopso.rs @@ -52,6 +52,38 @@ impl Default for MopsoConfig { /// `Vec` decisions only. Each particle maintains a personal best (the /// last position that was Pareto-non-dominated by any later position). The /// social leader is sampled uniformly from the external archive each step. +/// +/// # Example +/// +/// ``` +/// use heuropt::prelude::*; +/// +/// struct Schaffer; +/// impl Problem for Schaffer { +/// type Decision = Vec; +/// fn objectives(&self) -> ObjectiveSpace { +/// ObjectiveSpace::new(vec![Objective::minimize("f1"), Objective::minimize("f2")]) +/// } +/// fn evaluate(&self, x: &Vec) -> Evaluation { +/// Evaluation::new(vec![x[0] * x[0], (x[0] - 2.0).powi(2)]) +/// } +/// } +/// +/// let mut opt = Mopso::new( +/// MopsoConfig { +/// swarm_size: 30, +/// generations: 50, +/// archive_size: 30, +/// inertia: 0.4, +/// cognitive: 1.5, +/// social: 1.5, +/// seed: 42, +/// }, +/// RealBounds::new(vec![(-5.0, 5.0)]), +/// ); +/// let r = opt.run(&Schaffer); +/// assert!(!r.pareto_front.is_empty()); +/// ``` #[derive(Debug, Clone)] pub struct Mopso { /// Algorithm configuration. diff --git a/src/algorithms/nelder_mead.rs b/src/algorithms/nelder_mead.rs index ea5d86e..219759c 100644 --- a/src/algorithms/nelder_mead.rs +++ b/src/algorithms/nelder_mead.rs @@ -48,6 +48,38 @@ impl Default for NelderMeadConfig { /// `Vec` decisions only. Single-objective only. Initial simplex is /// built around the midpoint of the configured bounds; every new vertex /// is clamped to those bounds. +/// +/// # Example +/// +/// ``` +/// use heuropt::prelude::*; +/// +/// struct Sphere; +/// impl Problem for Sphere { +/// type Decision = Vec; +/// fn objectives(&self) -> ObjectiveSpace { +/// ObjectiveSpace::new(vec![Objective::minimize("f")]) +/// } +/// fn evaluate(&self, x: &Vec) -> Evaluation { +/// Evaluation::new(vec![x.iter().map(|v| v * v).sum::()]) +/// } +/// } +/// +/// let mut opt = NelderMead::new( +/// NelderMeadConfig { +/// iterations: 200, +/// reflection: 1.0, +/// expansion: 2.0, +/// contraction: 0.5, +/// shrinkage: 0.5, +/// initial_step: 1.0, +/// }, +/// RealBounds::new(vec![(-5.0, 5.0); 3]), +/// ); +/// let r = opt.run(&Sphere); +/// // Nelder-Mead reaches machine precision on Sphere. +/// assert!(r.best.unwrap().evaluation.objectives[0] < 1e-10); +/// ``` #[derive(Debug, Clone)] pub struct NelderMead { /// Algorithm configuration. diff --git a/src/algorithms/nsga2.rs b/src/algorithms/nsga2.rs index 9e03815..5652636 100644 --- a/src/algorithms/nsga2.rs +++ b/src/algorithms/nsga2.rs @@ -35,6 +35,43 @@ impl Default for Nsga2Config { } /// NSGA-II optimizer (spec §12.3). +/// +/// The canonical Pareto-based EA: combines non-dominated sorting with +/// crowding-distance secondary ranking. A strong default for 2- or +/// 3-objective problems. +/// +/// # Example +/// +/// ``` +/// use heuropt::prelude::*; +/// +/// struct Schaffer; +/// impl Problem for Schaffer { +/// type Decision = Vec; +/// fn objectives(&self) -> ObjectiveSpace { +/// ObjectiveSpace::new(vec![ +/// Objective::minimize("f1"), +/// Objective::minimize("f2"), +/// ]) +/// } +/// fn evaluate(&self, x: &Vec) -> Evaluation { +/// Evaluation::new(vec![x[0] * x[0], (x[0] - 2.0).powi(2)]) +/// } +/// } +/// +/// let bounds = vec![(-5.0_f64, 5.0_f64)]; +/// let mut opt = Nsga2::new( +/// Nsga2Config { population_size: 30, generations: 20, seed: 42 }, +/// RealBounds::new(bounds.clone()), +/// CompositeVariation { +/// crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5), +/// mutation: PolynomialMutation::new(bounds, 20.0, 1.0), +/// }, +/// ); +/// let r = opt.run(&Schaffer); +/// assert_eq!(r.population.len(), 30); +/// assert!(!r.pareto_front.is_empty()); +/// ``` #[derive(Debug, Clone)] pub struct Nsga2 { /// Algorithm configuration. diff --git a/src/algorithms/nsga3.rs b/src/algorithms/nsga3.rs index adcadf7..80ab2c0 100644 --- a/src/algorithms/nsga3.rs +++ b/src/algorithms/nsga3.rs @@ -43,6 +43,44 @@ impl Default for Nsga3Config { } /// NSGA-III optimizer. +/// +/// NSGA-II's many-objective successor: replaces crowding distance with +/// reference-point niching over Das–Dennis points in the normalized +/// objective space. The canonical default for 4+ objectives. +/// +/// # Example +/// +/// ``` +/// use heuropt::prelude::*; +/// +/// struct Schaffer; +/// impl Problem for Schaffer { +/// type Decision = Vec; +/// fn objectives(&self) -> ObjectiveSpace { +/// ObjectiveSpace::new(vec![Objective::minimize("f1"), Objective::minimize("f2")]) +/// } +/// fn evaluate(&self, x: &Vec) -> Evaluation { +/// Evaluation::new(vec![x[0] * x[0], (x[0] - 2.0).powi(2)]) +/// } +/// } +/// +/// let bounds = vec![(-5.0_f64, 5.0_f64)]; +/// let mut opt = Nsga3::new( +/// Nsga3Config { +/// population_size: 30, +/// generations: 20, +/// reference_divisions: 12, +/// seed: 42, +/// }, +/// RealBounds::new(bounds.clone()), +/// CompositeVariation { +/// crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5), +/// mutation: PolynomialMutation::new(bounds, 20.0, 1.0), +/// }, +/// ); +/// let r = opt.run(&Schaffer); +/// assert!(!r.pareto_front.is_empty()); +/// ``` #[derive(Debug, Clone)] pub struct Nsga3 { /// Algorithm configuration. diff --git a/src/algorithms/one_plus_one_es.rs b/src/algorithms/one_plus_one_es.rs index 9354a01..a73517d 100644 --- a/src/algorithms/one_plus_one_es.rs +++ b/src/algorithms/one_plus_one_es.rs @@ -47,6 +47,36 @@ impl Default for OnePlusOneEsConfig { /// (1+1)-ES with the one-fifth rule: tiny, parameter-light continuous /// optimizer. `Vec` decisions only; single-objective only. +/// +/// # Example +/// +/// ``` +/// use heuropt::prelude::*; +/// +/// struct Sphere; +/// impl Problem for Sphere { +/// type Decision = Vec; +/// fn objectives(&self) -> ObjectiveSpace { +/// ObjectiveSpace::new(vec![Objective::minimize("f")]) +/// } +/// fn evaluate(&self, x: &Vec) -> Evaluation { +/// Evaluation::new(vec![x.iter().map(|v| v * v).sum::()]) +/// } +/// } +/// +/// let mut opt = OnePlusOneEs::new( +/// OnePlusOneEsConfig { +/// iterations: 1_000, +/// initial_sigma: 0.5, +/// adaptation_period: 50, +/// step_increase: 1.22, +/// seed: 42, +/// }, +/// RealBounds::new(vec![(-5.0, 5.0); 3]), +/// ); +/// let r = opt.run(&Sphere); +/// assert!(r.best.unwrap().evaluation.objectives[0] < 1e-3); +/// ``` #[derive(Debug, Clone)] pub struct OnePlusOneEs { /// Algorithm configuration. diff --git a/src/algorithms/paes.rs b/src/algorithms/paes.rs index 7af6e8c..c619519 100644 --- a/src/algorithms/paes.rs +++ b/src/algorithms/paes.rs @@ -36,6 +36,31 @@ impl Default for PaesConfig { /// One current candidate, one mutation per iteration, one bounded archive. /// Intentionally a readable baseline rather than a research-perfect PAES /// (spec §12.2). +/// +/// # Example +/// +/// ``` +/// use heuropt::prelude::*; +/// +/// struct Schaffer; +/// impl Problem for Schaffer { +/// type Decision = Vec; +/// fn objectives(&self) -> ObjectiveSpace { +/// ObjectiveSpace::new(vec![Objective::minimize("f1"), Objective::minimize("f2")]) +/// } +/// fn evaluate(&self, x: &Vec) -> Evaluation { +/// Evaluation::new(vec![x[0] * x[0], (x[0] - 2.0).powi(2)]) +/// } +/// } +/// +/// let mut opt = Paes::new( +/// PaesConfig { iterations: 200, archive_size: 30, seed: 42 }, +/// RealBounds::new(vec![(-5.0, 5.0)]), +/// GaussianMutation { sigma: 0.3 }, +/// ); +/// let r = opt.run(&Schaffer); +/// assert!(!r.pareto_front.is_empty()); +/// ``` #[derive(Debug, Clone)] pub struct Paes { /// Algorithm configuration. diff --git a/src/algorithms/particle_swarm.rs b/src/algorithms/particle_swarm.rs index ff23af8..b1157bc 100644 --- a/src/algorithms/particle_swarm.rs +++ b/src/algorithms/particle_swarm.rs @@ -55,6 +55,37 @@ impl Default for ParticleSwarmConfig { /// Velocities are clamped to `±(hi - lo)` per dimension to prevent /// "swarm explosion." Pair with `RealBounds` for both the search bounds /// and the initial particle positions. +/// +/// # Example +/// +/// ``` +/// use heuropt::prelude::*; +/// +/// struct Sphere; +/// impl Problem for Sphere { +/// type Decision = Vec; +/// fn objectives(&self) -> ObjectiveSpace { +/// ObjectiveSpace::new(vec![Objective::minimize("f")]) +/// } +/// fn evaluate(&self, x: &Vec) -> Evaluation { +/// Evaluation::new(vec![x.iter().map(|v| v * v).sum::()]) +/// } +/// } +/// +/// let mut opt = ParticleSwarm::new( +/// ParticleSwarmConfig { +/// swarm_size: 20, +/// generations: 50, +/// inertia: 0.7, +/// cognitive: 1.5, +/// social: 1.5, +/// seed: 42, +/// }, +/// RealBounds::new(vec![(-5.0, 5.0); 3]), +/// ); +/// let r = opt.run(&Sphere); +/// assert!(r.best.is_some()); +/// ``` #[derive(Debug, Clone)] pub struct ParticleSwarm { /// Algorithm configuration. diff --git a/src/algorithms/pesa2.rs b/src/algorithms/pesa2.rs index 4990431..d409f5c 100644 --- a/src/algorithms/pesa2.rs +++ b/src/algorithms/pesa2.rs @@ -47,6 +47,41 @@ impl Default for PesaIIConfig { /// Maintains an internal population (used to drive variation) and an /// external non-dominated archive. Selection biases toward members in /// sparsely-populated grid boxes so the front spreads out. +/// +/// # Example +/// +/// ``` +/// use heuropt::prelude::*; +/// +/// struct Schaffer; +/// impl Problem for Schaffer { +/// type Decision = Vec; +/// fn objectives(&self) -> ObjectiveSpace { +/// ObjectiveSpace::new(vec![Objective::minimize("f1"), Objective::minimize("f2")]) +/// } +/// fn evaluate(&self, x: &Vec) -> Evaluation { +/// Evaluation::new(vec![x[0] * x[0], (x[0] - 2.0).powi(2)]) +/// } +/// } +/// +/// let bounds = vec![(-5.0_f64, 5.0_f64)]; +/// let mut opt = PesaII::new( +/// PesaIIConfig { +/// population_size: 20, +/// archive_size: 30, +/// generations: 20, +/// grid_divisions: 8, +/// seed: 42, +/// }, +/// RealBounds::new(bounds.clone()), +/// CompositeVariation { +/// crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5), +/// mutation: PolynomialMutation::new(bounds, 20.0, 1.0), +/// }, +/// ); +/// let r = opt.run(&Schaffer); +/// assert!(!r.pareto_front.is_empty()); +/// ``` #[derive(Debug, Clone)] pub struct PesaII { /// Algorithm configuration. diff --git a/src/algorithms/random_search.rs b/src/algorithms/random_search.rs index 6da4a22..9f804bc 100644 --- a/src/algorithms/random_search.rs +++ b/src/algorithms/random_search.rs @@ -38,6 +38,31 @@ impl Default for RandomSearchConfig { /// Each iteration the configured `Initializer` produces `batch_size` decisions /// which are evaluated and pushed into the population. Cheap, parallelism-free, /// and useful as a sanity-check baseline. +/// +/// # Example +/// +/// ``` +/// use heuropt::prelude::*; +/// +/// struct Sphere; +/// impl Problem for Sphere { +/// type Decision = Vec; +/// fn objectives(&self) -> ObjectiveSpace { +/// ObjectiveSpace::new(vec![Objective::minimize("f")]) +/// } +/// fn evaluate(&self, x: &Vec) -> Evaluation { +/// Evaluation::new(vec![x.iter().map(|v| v * v).sum::()]) +/// } +/// } +/// +/// let mut opt = RandomSearch::new( +/// RandomSearchConfig { iterations: 200, batch_size: 10, seed: 42 }, +/// RealBounds::new(vec![(-5.0, 5.0); 3]), +/// ); +/// let r = opt.run(&Sphere); +/// assert_eq!(r.evaluations, 200 * 10); +/// assert!(r.best.is_some()); +/// ``` #[derive(Debug, Clone)] pub struct RandomSearch { /// Algorithm configuration. diff --git a/src/algorithms/rvea.rs b/src/algorithms/rvea.rs index 61c93e9..7a27b11 100644 --- a/src/algorithms/rvea.rs +++ b/src/algorithms/rvea.rs @@ -41,6 +41,45 @@ impl Default for RveaConfig { } /// Reference Vector-guided Evolutionary Algorithm. +/// +/// Many-objective EA that uses Das–Dennis reference vectors with an +/// adaptive penalty term to balance convergence and diversity as +/// generations progress. +/// +/// # Example +/// +/// ``` +/// use heuropt::prelude::*; +/// +/// struct Schaffer; +/// impl Problem for Schaffer { +/// type Decision = Vec; +/// fn objectives(&self) -> ObjectiveSpace { +/// ObjectiveSpace::new(vec![Objective::minimize("f1"), Objective::minimize("f2")]) +/// } +/// fn evaluate(&self, x: &Vec) -> Evaluation { +/// Evaluation::new(vec![x[0] * x[0], (x[0] - 2.0).powi(2)]) +/// } +/// } +/// +/// let bounds = vec![(-5.0_f64, 5.0_f64)]; +/// let mut opt = Rvea::new( +/// RveaConfig { +/// population_size: 30, +/// generations: 20, +/// reference_divisions: 19, +/// alpha: 2.0, +/// seed: 42, +/// }, +/// RealBounds::new(bounds.clone()), +/// CompositeVariation { +/// crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5), +/// mutation: PolynomialMutation::new(bounds, 20.0, 1.0), +/// }, +/// ); +/// let r = opt.run(&Schaffer); +/// assert!(!r.pareto_front.is_empty()); +/// ``` #[derive(Debug, Clone)] pub struct Rvea { /// Algorithm configuration. diff --git a/src/algorithms/simulated_annealing.rs b/src/algorithms/simulated_annealing.rs index dafd852..40fc180 100644 --- a/src/algorithms/simulated_annealing.rs +++ b/src/algorithms/simulated_annealing.rs @@ -41,6 +41,36 @@ impl Default for SimulatedAnnealingConfig { /// and `T` anneals geometrically from `initial_temperature` to /// `final_temperature` over the iteration count. Generic over decision /// type — pair with any `Variation` impl that returns one child per call. +/// +/// # Example +/// +/// ``` +/// use heuropt::prelude::*; +/// +/// struct Sphere; +/// impl Problem for Sphere { +/// type Decision = Vec; +/// fn objectives(&self) -> ObjectiveSpace { +/// ObjectiveSpace::new(vec![Objective::minimize("f")]) +/// } +/// fn evaluate(&self, x: &Vec) -> Evaluation { +/// Evaluation::new(vec![x.iter().map(|v| v * v).sum::()]) +/// } +/// } +/// +/// let mut opt = SimulatedAnnealing::new( +/// SimulatedAnnealingConfig { +/// iterations: 2_000, +/// initial_temperature: 1.0, +/// final_temperature: 1e-3, +/// seed: 42, +/// }, +/// RealBounds::new(vec![(-5.0, 5.0); 3]), +/// GaussianMutation { sigma: 0.3 }, +/// ); +/// let r = opt.run(&Sphere); +/// assert!(r.best.is_some()); +/// ``` #[derive(Debug, Clone)] pub struct SimulatedAnnealing { /// Algorithm configuration. diff --git a/src/algorithms/sms_emoa.rs b/src/algorithms/sms_emoa.rs index c98a8df..90b3a6d 100644 --- a/src/algorithms/sms_emoa.rs +++ b/src/algorithms/sms_emoa.rs @@ -49,6 +49,40 @@ impl Default for SmsEmoaConfig { /// non-dominated front. Excellent convergence quality at the price of /// quadratic-in-N hypervolume evaluations per generation, so practical /// up to ~4 objectives at population sizes ≤ 200. +/// +/// # Example +/// +/// ``` +/// use heuropt::prelude::*; +/// +/// struct Schaffer; +/// impl Problem for Schaffer { +/// type Decision = Vec; +/// fn objectives(&self) -> ObjectiveSpace { +/// ObjectiveSpace::new(vec![Objective::minimize("f1"), Objective::minimize("f2")]) +/// } +/// fn evaluate(&self, x: &Vec) -> Evaluation { +/// Evaluation::new(vec![x[0] * x[0], (x[0] - 2.0).powi(2)]) +/// } +/// } +/// +/// let bounds = vec![(-5.0_f64, 5.0_f64)]; +/// let mut opt = SmsEmoa::new( +/// SmsEmoaConfig { +/// population_size: 20, +/// generations: 100, +/// reference_point: vec![30.0, 30.0], +/// seed: 42, +/// }, +/// RealBounds::new(bounds.clone()), +/// CompositeVariation { +/// crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5), +/// mutation: PolynomialMutation::new(bounds, 20.0, 1.0), +/// }, +/// ); +/// let r = opt.run(&Schaffer); +/// assert!(!r.pareto_front.is_empty()); +/// ``` #[derive(Debug, Clone)] pub struct SmsEmoa { /// Algorithm configuration. diff --git a/src/algorithms/snes.rs b/src/algorithms/snes.rs index c591d69..8365f0c 100644 --- a/src/algorithms/snes.rs +++ b/src/algorithms/snes.rs @@ -51,6 +51,37 @@ impl Default for SeparableNesConfig { /// following the natural gradient of expected fitness, with rank-shaped /// fitness utilities for invariance to monotone transforms of the /// objective. +/// +/// # Example +/// +/// ``` +/// use heuropt::prelude::*; +/// +/// struct Sphere; +/// impl Problem for Sphere { +/// type Decision = Vec; +/// fn objectives(&self) -> ObjectiveSpace { +/// ObjectiveSpace::new(vec![Objective::minimize("f")]) +/// } +/// fn evaluate(&self, x: &Vec) -> Evaluation { +/// Evaluation::new(vec![x.iter().map(|v| v * v).sum::()]) +/// } +/// } +/// +/// let mut opt = SeparableNes::new( +/// SeparableNesConfig { +/// population_size: 16, +/// generations: 80, +/// initial_sigma: 1.0, +/// mean_learning_rate: 1.0, +/// sigma_learning_rate: None, // use NES default +/// seed: 42, +/// }, +/// RealBounds::new(vec![(-5.0, 5.0); 3]), +/// ); +/// let r = opt.run(&Sphere); +/// assert!(r.best.unwrap().evaluation.objectives[0] < 1e-3); +/// ``` #[derive(Debug, Clone)] pub struct SeparableNes { /// Algorithm configuration. diff --git a/src/algorithms/spea2.rs b/src/algorithms/spea2.rs index abe1e0c..e26e943 100644 --- a/src/algorithms/spea2.rs +++ b/src/algorithms/spea2.rs @@ -37,6 +37,40 @@ impl Default for Spea2Config { } /// SPEA2 optimizer. +/// +/// Strength Pareto Evolutionary Algorithm 2: combines a strength-based +/// dominance score with a k-th nearest-neighbor density estimate. Maintains +/// an external archive separate from the working population. +/// +/// # Example +/// +/// ``` +/// use heuropt::prelude::*; +/// +/// struct Schaffer; +/// impl Problem for Schaffer { +/// type Decision = Vec; +/// fn objectives(&self) -> ObjectiveSpace { +/// ObjectiveSpace::new(vec![Objective::minimize("f1"), Objective::minimize("f2")]) +/// } +/// fn evaluate(&self, x: &Vec) -> Evaluation { +/// Evaluation::new(vec![x[0] * x[0], (x[0] - 2.0).powi(2)]) +/// } +/// } +/// +/// let bounds = vec![(-5.0_f64, 5.0_f64)]; +/// let mut opt = Spea2::new( +/// Spea2Config { population_size: 30, archive_size: 30, generations: 20, seed: 42 }, +/// RealBounds::new(bounds.clone()), +/// CompositeVariation { +/// crossover: SimulatedBinaryCrossover::new(bounds.clone(), 15.0, 0.5), +/// mutation: PolynomialMutation::new(bounds, 20.0, 1.0), +/// }, +/// ); +/// let r = opt.run(&Schaffer); +/// assert_eq!(r.population.len(), 30); +/// assert!(!r.pareto_front.is_empty()); +/// ``` #[derive(Debug, Clone)] pub struct Spea2 { /// Algorithm configuration. diff --git a/src/algorithms/tlbo.rs b/src/algorithms/tlbo.rs index 51703aa..ac3ea36 100644 --- a/src/algorithms/tlbo.rs +++ b/src/algorithms/tlbo.rs @@ -41,6 +41,30 @@ impl Default for TlboConfig { /// population_size and generations. Compared with the rest of heuropt's /// SO toolkit (DE has F+CR, PSO has w+c1+c2, CMA-ES has σ, GA needs /// crossover+mutation operators), TLBO works out of the box. +/// +/// # Example +/// +/// ``` +/// use heuropt::prelude::*; +/// +/// struct Sphere; +/// impl Problem for Sphere { +/// type Decision = Vec; +/// fn objectives(&self) -> ObjectiveSpace { +/// ObjectiveSpace::new(vec![Objective::minimize("f")]) +/// } +/// fn evaluate(&self, x: &Vec) -> Evaluation { +/// Evaluation::new(vec![x.iter().map(|v| v * v).sum::()]) +/// } +/// } +/// +/// let mut opt = Tlbo::new( +/// TlboConfig { population_size: 20, generations: 50, seed: 42 }, +/// RealBounds::new(vec![(-5.0, 5.0); 3]), +/// ); +/// let r = opt.run(&Sphere); +/// assert!(r.best.unwrap().evaluation.objectives[0] < 1e-3); +/// ``` #[derive(Debug, Clone)] pub struct Tlbo { /// Algorithm configuration. diff --git a/src/algorithms/tpe.rs b/src/algorithms/tpe.rs index c53d543..cfd026f 100644 --- a/src/algorithms/tpe.rs +++ b/src/algorithms/tpe.rs @@ -52,6 +52,37 @@ impl Default for TpeConfig { /// `BayesianOpt`, no GP — TPE models `p(x | y < y*)` and `p(x | y >= y*)` /// as per-axis Gaussian KDEs and picks the next candidate by maximizing /// the ratio of the two densities. +/// +/// # Example +/// +/// ``` +/// use heuropt::prelude::*; +/// +/// struct Sphere; +/// impl Problem for Sphere { +/// type Decision = Vec; +/// fn objectives(&self) -> ObjectiveSpace { +/// ObjectiveSpace::new(vec![Objective::minimize("f")]) +/// } +/// fn evaluate(&self, x: &Vec) -> Evaluation { +/// Evaluation::new(vec![x.iter().map(|v| v * v).sum::()]) +/// } +/// } +/// +/// let mut opt = Tpe::new( +/// TpeConfig { +/// initial_samples: 10, +/// iterations: 50, +/// good_fraction: 0.25, +/// candidate_samples: 24, +/// bandwidth_factor: 1.0, +/// seed: 42, +/// }, +/// RealBounds::new(vec![(-3.0, 3.0); 3]), +/// ); +/// let r = opt.run(&Sphere); +/// assert_eq!(r.evaluations, 60); +/// ``` #[derive(Debug, Clone)] pub struct Tpe { /// Algorithm configuration. diff --git a/src/algorithms/umda.rs b/src/algorithms/umda.rs index 5e45689..fd8658a 100644 --- a/src/algorithms/umda.rs +++ b/src/algorithms/umda.rs @@ -49,6 +49,34 @@ impl Default for UmdaConfig { /// `[1 / (2·selected_size), 1 - 1 / (2·selected_size)]` (Laplace-style /// smoothing) so the population never collapses to a single deterministic /// string. +/// +/// # Example +/// +/// ``` +/// use heuropt::prelude::*; +/// +/// struct OneMax; +/// impl Problem for OneMax { +/// type Decision = Vec; +/// fn objectives(&self) -> ObjectiveSpace { +/// ObjectiveSpace::new(vec![Objective::maximize("ones")]) +/// } +/// fn evaluate(&self, x: &Vec) -> Evaluation { +/// Evaluation::new(vec![x.iter().filter(|b| **b).count() as f64]) +/// } +/// } +/// +/// let mut opt = Umda::new(UmdaConfig { +/// population_size: 50, +/// selected_size: 20, +/// generations: 30, +/// bits: 16, +/// seed: 42, +/// }); +/// let r = opt.run(&OneMax); +/// // OneMax with 16 bits: optimum is 16. UMDA should be very close. +/// assert!(r.best.unwrap().evaluation.objectives[0] >= 14.0); +/// ``` #[derive(Debug, Clone)] pub struct Umda { /// Algorithm configuration. diff --git a/src/lib.rs b/src/lib.rs index cdd2c74..d09a3da 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -1,16 +1,37 @@ -//! `heuropt` — a practical Rust toolkit for implementing heuristic -//! single-objective, multi-objective, and many-objective optimization -//! algorithms. +//! `heuropt` — a practical Rust toolkit for heuristic single-, +//! multi-, and many-objective optimization. //! //! The crate aims to make three things obvious: //! -//! 1. Define an optimization problem by implementing [`Problem`](crate::core::Problem). -//! 2. Run a built-in optimizer such as [`Nsga2`](crate::algorithms::Nsga2) or -//! [`RandomSearch`](crate::algorithms::RandomSearch). -//! 3. Implement a new optimizer by implementing -//! [`Optimizer`](crate::traits::Optimizer). +//! 1. **Define a problem** by implementing [`Problem`](crate::core::Problem). +//! 2. **Run a built-in optimizer** — pick from 35 algorithms in +//! [`algorithms`] covering single-objective continuous (CMA-ES, +//! Differential Evolution, Nelder-Mead, …), multi-objective +//! (NSGA-II, MOPSO, IBEA, MOEA/D, …), many-objective (NSGA-III, +//! GrEA, RVEA, …), and sample-efficient regimes (Bayesian +//! Optimization, TPE, Hyperband). +//! 3. **Or implement your own** by implementing +//! [`Optimizer`](crate::traits::Optimizer). The trait is one +//! method long. //! -//! See `docs/heuropt_tech_design_spec.md` for the full design rationale. +//! ## Where to read more +//! +//! - **User guide / cookbook / comparison vs pymoo & friends:** +//! . +//! - **Algorithm selection:** the README's decision tree, or the +//! "Choosing an algorithm" book chapter. +//! - **Design rationale:** `docs/heuropt_tech_design_spec.md` in the +//! repository. +//! +//! ## Optional features +//! +//! - `serde` — derives `Serialize` / `Deserialize` on the core data +//! types ([`Candidate`](crate::core::Candidate), +//! [`Population`](crate::core::Population), +//! [`Evaluation`](crate::core::Evaluation), …). +//! - `parallel` — rayon-backed parallel population evaluation in +//! every population-based algorithm. Seeded runs stay bit- +//! identical to serial mode. //! //! # Quick example //!