feat(algorithms): add CMA-ES (Covariance Matrix Adaptation Evolution Strategy)

Hansen & Ostermeier 2001 CMA-ES, the canonical real-valued
single-objective stochastic optimizer. Implements the full (μ/μ_w, λ)
update with rank-μ + rank-1 covariance updates and cumulative step-size
adaptation:

- Sample λ offspring from N(mean, σ² · C)
- Select the μ best, weight them, recompute mean
- Update evolution paths p_σ (step size) and p_c (covariance)
- Rank-1 update of C from p_c, plus rank-μ update from selected offspring
- Adapt σ via |p_σ| / E‖N(0,I)‖

Eigendecomposition (used to convert C into its B·D form for sampling
N(0, σ²·C)) goes through the new internal Jacobi helper, recomputed
every `eigen_decomposition_period` generations to amortize cost.

Vec<f64> decisions only. Bounds taken from a `RealBounds` field; mean
and offspring are clamped per dimension. Single-objective only.

Hyperparameters use the standard CMA-ES defaults (μ=λ/2, weights from
Hansen's tutorial, c_σ, c_c, c_1, c_μ, d_σ all formulae from §7.1).

Tests cover: convergence on Sphere1D and 5-D Rosenbrock, deterministic
reruns, panic on multi-objective, panic on `population_size < 4`.
This commit is contained in:
2026-05-05 09:51:11 -06:00
parent 325c8cdd37
commit c04420851e
4 changed files with 477 additions and 6 deletions
+2
View File
@@ -1,5 +1,6 @@
//! Built-in reference optimizers.
pub mod cma_es;
pub mod differential_evolution;
pub mod genetic_algorithm;
pub mod hill_climber;
@@ -14,6 +15,7 @@ pub mod simulated_annealing;
pub mod spea2;
pub mod tabu_search;
pub use cma_es::*;
pub use differential_evolution::*;
pub use genetic_algorithm::*;
pub use hill_climber::*;