chore(release): roll up v0.3.0 — expensive-eval, gradient-free, multi-fidelity
CHANGELOG entry for the v0.3.0 cohort, version bump in Cargo.toml and README. Theme: filling heuropt's expensive-evaluation and constraint- handling gaps. Algorithms (9 new): OnePlusOneEs, NelderMead, IpopCmaEs, BayesianOpt, SeparableNes, Tpe, Hyperband. Operators (1 new): LevyMutation. Repair operators (1 trait + 2 impls): Repair<D> with ClampToBounds and ProjectToSimplex. Selection helpers (1 new): stochastic_ranking_select. Internal helpers: Cholesky factorization (used by BO). API additions: - CmaEsConfig.initial_mean: Option<Vec<f64>> (None preserves existing midpoint-of-bounds behavior; used by IpopCmaEs to inject restart diversity). - New PartialProblem trait — multi-fidelity contract used by Hyperband. No breaking changes to v0.2.0 public API.
This commit is contained in:
+66
-16
@@ -7,34 +7,84 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|||||||
|
|
||||||
## [Unreleased]
|
## [Unreleased]
|
||||||
|
|
||||||
|
## [0.3.0] — 2026-05-05
|
||||||
|
|
||||||
|
Theme: filling heuropt's expensive-evaluation, gradient-free, and
|
||||||
|
constraint-handling gaps. No breaking changes to the v0.2.0 public API.
|
||||||
|
|
||||||
### Added
|
### Added
|
||||||
|
|
||||||
#### New algorithms (the "expensive-eval and gradient-free" cohort)
|
#### New algorithms (9)
|
||||||
|
|
||||||
|
**Sample-efficient / surrogate-based:**
|
||||||
|
|
||||||
|
- `BayesianOpt` — Gaussian-process Bayesian Optimization with Expected
|
||||||
|
Improvement acquisition. heuropt's first sample-efficient algorithm:
|
||||||
|
targets the 50–500 evaluation regime.
|
||||||
|
- `Tpe` — Bergstra et al. 2011 Tree-structured Parzen Estimator
|
||||||
|
(workhorse of Hyperopt and Optuna). KDE-based surrogate; cheaper
|
||||||
|
per-step than BO and more robust without hyperparameter tuning.
|
||||||
|
|
||||||
|
**Classical and modern evolution strategies:**
|
||||||
|
|
||||||
- `OnePlusOneEs` — Rechenberg 1973 (1+1)-ES with the one-fifth success
|
- `OnePlusOneEs` — Rechenberg 1973 (1+1)-ES with the one-fifth success
|
||||||
rule. Smallest possible self-adapting evolution strategy.
|
rule. Smallest possible self-adapting evolution strategy.
|
||||||
- `NelderMead` — Nelder & Mead 1965 simplex direct-search method. Fills
|
|
||||||
a real gap: heuropt's first classical gradient-free local optimizer.
|
|
||||||
- `IpopCmaEs` — Auger & Hansen 2005 increasing-population CMA-ES with
|
- `IpopCmaEs` — Auger & Hansen 2005 increasing-population CMA-ES with
|
||||||
restart. Specifically fixes vanilla CMA-ES's known weakness on
|
restart. Specifically fixes vanilla CMA-ES's known weakness on
|
||||||
multimodal problems (e.g. Rastrigin: 2.35 → 0.13).
|
multimodal problems.
|
||||||
- `BayesianOpt` — Gaussian-process-based Bayesian Optimization with
|
- `SeparableNes` — Wierstra et al. 2008/2014 Natural Evolution Strategy
|
||||||
Expected Improvement acquisition. heuropt's first sample-efficient
|
with diagonal covariance (sNES). Different theoretical foundation
|
||||||
algorithm: targets the 50–500 evaluation regime where every other
|
than CMA-ES; cheaper per-step at the cost of being unable to model
|
||||||
algorithm is way over-budget.
|
rotated landscapes.
|
||||||
|
|
||||||
|
**Direct search:**
|
||||||
|
|
||||||
|
- `NelderMead` — Nelder & Mead 1965 simplex method. Classical gradient-
|
||||||
|
free local optimizer; superb on low-dim smooth problems
|
||||||
|
(Rosenbrock 5-D: f = 0 exactly).
|
||||||
|
|
||||||
|
**Multi-fidelity:**
|
||||||
|
|
||||||
|
- `Hyperband` — Li et al. 2017 multi-fidelity hyperparameter optimizer
|
||||||
|
built on Successive Halving. Operates on a new `PartialProblem`
|
||||||
|
trait so configurations can be evaluated at adjustable fidelity
|
||||||
|
budgets.
|
||||||
|
|
||||||
|
#### New operators
|
||||||
|
|
||||||
|
- `LevyMutation` — heavy-tailed Lévy-flight mutation via Mantegna's
|
||||||
|
algorithm. The actual algorithmic contribution from Cuckoo Search
|
||||||
|
packaged as a reusable `Variation` operator.
|
||||||
|
|
||||||
|
#### New traits + impls
|
||||||
|
|
||||||
|
- `PartialProblem` — multi-fidelity problem contract:
|
||||||
|
`evaluate_at_budget(decision, budget) -> Evaluation`. Used by
|
||||||
|
`Hyperband`. Intentionally not a sub-trait of `Problem`.
|
||||||
|
- `Repair<D>` — in-place projection trait for restoring decisions to
|
||||||
|
feasibility. Pair with `Variation` operators to get bounds-aware
|
||||||
|
variants. Provided impls:
|
||||||
|
- `ClampToBounds` for `Vec<f64>` per-axis clamping
|
||||||
|
- `ProjectToSimplex` for L1-budget / probability-simplex projection
|
||||||
|
|
||||||
|
#### New selection helpers
|
||||||
|
|
||||||
|
- `stochastic_ranking_select` — Runarsson & Yao 2000 stochastic
|
||||||
|
ranking. Better than strict feasibility-first tournament selection
|
||||||
|
on heavily-constrained problems.
|
||||||
|
|
||||||
#### Internal helpers
|
#### Internal helpers
|
||||||
|
|
||||||
- `internal::cholesky` — Cholesky factorization + triangular solves
|
- `internal::cholesky` — Cholesky factorization + triangular solves
|
||||||
for symmetric positive-definite matrices, used by the GP posterior
|
for SPD matrices, used by the GP posterior in `BayesianOpt`.
|
||||||
in `BayesianOpt`. Hand-rolled to avoid pulling in nalgebra.
|
|
||||||
|
|
||||||
#### CmaEs API change (additive)
|
### Changed
|
||||||
|
|
||||||
- `CmaEsConfig` gained an `initial_mean: Option<Vec<f64>>` field
|
- `CmaEsConfig` gained `initial_mean: Option<Vec<f64>>`. `None`
|
||||||
(defaulting to `None`, which keeps the existing midpoint-of-bounds
|
preserves the existing midpoint-of-bounds default; `IpopCmaEs` sets
|
||||||
behavior). `IpopCmaEs` uses it to inject restart diversity without
|
it to inject restart diversity without shrinking the search box.
|
||||||
shrinking the search box.
|
|
||||||
|
[0.3.0]: https://github.com/swaits/heuropt/releases/tag/v0.3.0
|
||||||
|
|
||||||
## [0.2.0] — 2026-05-05
|
## [0.2.0] — 2026-05-05
|
||||||
|
|
||||||
@@ -195,5 +245,5 @@ Initial release.
|
|||||||
`RandomSearch`, `Nsga2`, and `DifferentialEvolution`. Seeded runs stay
|
`RandomSearch`, `Nsga2`, and `DifferentialEvolution`. Seeded runs stay
|
||||||
bit-identical to serial mode.
|
bit-identical to serial mode.
|
||||||
|
|
||||||
[Unreleased]: https://github.com/swaits/heuropt/compare/v0.2.0...HEAD
|
[Unreleased]: https://github.com/swaits/heuropt/compare/v0.3.0...HEAD
|
||||||
[0.1.0]: https://github.com/swaits/heuropt/releases/tag/v0.1.0
|
[0.1.0]: https://github.com/swaits/heuropt/releases/tag/v0.1.0
|
||||||
|
|||||||
+1
-1
@@ -1,6 +1,6 @@
|
|||||||
[package]
|
[package]
|
||||||
name = "heuropt"
|
name = "heuropt"
|
||||||
version = "0.2.0"
|
version = "0.3.0"
|
||||||
edition = "2024"
|
edition = "2024"
|
||||||
rust-version = "1.85"
|
rust-version = "1.85"
|
||||||
authors = ["Stephen Waits <steve@waits.net>"]
|
authors = ["Stephen Waits <steve@waits.net>"]
|
||||||
|
|||||||
@@ -17,13 +17,13 @@ framework concepts.
|
|||||||
|
|
||||||
```toml
|
```toml
|
||||||
[dependencies]
|
[dependencies]
|
||||||
heuropt = "0.2"
|
heuropt = "0.3"
|
||||||
|
|
||||||
# Optional features:
|
# Optional features:
|
||||||
# - "serde": derive Serialize/Deserialize on the core data types.
|
# - "serde": derive Serialize/Deserialize on the core data types.
|
||||||
# - "parallel": evaluate populations across rayon's thread pool.
|
# - "parallel": evaluate populations across rayon's thread pool.
|
||||||
# Seeded runs stay bit-identical to serial mode.
|
# Seeded runs stay bit-identical to serial mode.
|
||||||
# heuropt = { version = "0.2", features = ["serde", "parallel"] }
|
# heuropt = { version = "0.3", features = ["serde", "parallel"] }
|
||||||
```
|
```
|
||||||
|
|
||||||
## Define a problem
|
## Define a problem
|
||||||
|
|||||||
Reference in New Issue
Block a user