feat(algorithms): add CMA-ES (Covariance Matrix Adaptation Evolution Strategy)

Hansen & Ostermeier 2001 CMA-ES, the canonical real-valued
single-objective stochastic optimizer. Implements the full (μ/μ_w, λ)
update with rank-μ + rank-1 covariance updates and cumulative step-size
adaptation:

- Sample λ offspring from N(mean, σ² · C)
- Select the μ best, weight them, recompute mean
- Update evolution paths p_σ (step size) and p_c (covariance)
- Rank-1 update of C from p_c, plus rank-μ update from selected offspring
- Adapt σ via |p_σ| / E‖N(0,I)‖

Eigendecomposition (used to convert C into its B·D form for sampling
N(0, σ²·C)) goes through the new internal Jacobi helper, recomputed
every `eigen_decomposition_period` generations to amortize cost.

Vec<f64> decisions only. Bounds taken from a `RealBounds` field; mean
and offspring are clamped per dimension. Single-objective only.

Hyperparameters use the standard CMA-ES defaults (μ=λ/2, weights from
Hansen's tutorial, c_σ, c_c, c_1, c_μ, d_σ all formulae from §7.1).

Tests cover: convergence on Sphere1D and 5-D Rosenbrock, deterministic
reruns, panic on multi-objective, panic on `population_size < 4`.
This commit is contained in:
2026-05-05 09:51:11 -06:00
parent 325c8cdd37
commit c04420851e
4 changed files with 477 additions and 6 deletions
+4 -1
View File
@@ -24,7 +24,7 @@ pub(crate) fn symmetric_eigen(
debug_assert!(matrix.iter().all(|row| row.len() == n), "matrix must be square");
// Working copy of the matrix; converges to a diagonal of eigenvalues.
let mut a: Vec<Vec<f64>> = matrix.iter().map(|row| row.clone()).collect();
let mut a: Vec<Vec<f64>> = matrix.to_vec();
// Eigenvector accumulator, starts as identity.
let mut v: Vec<Vec<f64>> = (0..n)
.map(|i| (0..n).map(|j| if i == j { 1.0 } else { 0.0 }).collect())
@@ -32,6 +32,7 @@ pub(crate) fn symmetric_eigen(
for _ in 0..max_sweeps {
let mut max_off = 0.0;
#[allow(clippy::needless_range_loop)]
for i in 0..n {
for j in (i + 1)..n {
let abs_off = a[i][j].abs();
@@ -71,6 +72,7 @@ pub(crate) fn symmetric_eigen(
a[q][p] = 0.0;
// Update other off-diagonal entries in rows/cols p and q.
#[allow(clippy::needless_range_loop)]
for r in 0..n {
if r != p && r != q {
let arp = a[r][p];
@@ -83,6 +85,7 @@ pub(crate) fn symmetric_eigen(
}
// Update accumulated eigenvectors.
#[allow(clippy::needless_range_loop)]
for r in 0..n {
let vrp = v[r][p];
let vrq = v[r][q];