feat(algorithms): add CMA-ES (Covariance Matrix Adaptation Evolution Strategy)

Hansen & Ostermeier 2001 CMA-ES, the canonical real-valued
single-objective stochastic optimizer. Implements the full (μ/μ_w, λ)
update with rank-μ + rank-1 covariance updates and cumulative step-size
adaptation:

- Sample λ offspring from N(mean, σ² · C)
- Select the μ best, weight them, recompute mean
- Update evolution paths p_σ (step size) and p_c (covariance)
- Rank-1 update of C from p_c, plus rank-μ update from selected offspring
- Adapt σ via |p_σ| / E‖N(0,I)‖

Eigendecomposition (used to convert C into its B·D form for sampling
N(0, σ²·C)) goes through the new internal Jacobi helper, recomputed
every `eigen_decomposition_period` generations to amortize cost.

Vec<f64> decisions only. Bounds taken from a `RealBounds` field; mean
and offspring are clamped per dimension. Single-objective only.

Hyperparameters use the standard CMA-ES defaults (μ=λ/2, weights from
Hansen's tutorial, c_σ, c_c, c_1, c_μ, d_σ all formulae from §7.1).

Tests cover: convergence on Sphere1D and 5-D Rosenbrock, deterministic
reruns, panic on multi-objective, panic on `population_size < 4`.
This commit is contained in:
2026-05-05 09:51:11 -06:00
parent 325c8cdd37
commit c04420851e
4 changed files with 477 additions and 6 deletions
+5 -5
View File
@@ -22,9 +22,9 @@ pub use crate::operators::{
};
pub use crate::algorithms::{
DifferentialEvolution, DifferentialEvolutionConfig, GeneticAlgorithm,
GeneticAlgorithmConfig, HillClimber, HillClimberConfig, Moead, MoeadConfig, Nsga2,
Nsga2Config, Nsga3, Nsga3Config, Paes, PaesConfig, ParticleSwarm, ParticleSwarmConfig,
RandomSearch, RandomSearchConfig, SimulatedAnnealing, SimulatedAnnealingConfig, Spea2,
Spea2Config, TabuSearch, TabuSearchConfig,
CmaEs, CmaEsConfig, DifferentialEvolution, DifferentialEvolutionConfig,
GeneticAlgorithm, GeneticAlgorithmConfig, HillClimber, HillClimberConfig, Moead,
MoeadConfig, Nsga2, Nsga2Config, Nsga3, Nsga3Config, Paes, PaesConfig, ParticleSwarm,
ParticleSwarmConfig, RandomSearch, RandomSearchConfig, SimulatedAnnealing,
SimulatedAnnealingConfig, Spea2, Spea2Config, TabuSearch, TabuSearchConfig,
};