feat(algorithms): add SeparableNes (Natural Evolution Strategy)

Wierstra et al. 2008/2014 NES with the diagonal-covariance "separable"
variant (sNES). Different theoretical foundation from CMA-ES: rather
than tracking a full covariance matrix and adapting it through
evolution paths, sNES updates the sampling distribution's parameters
by following the natural gradient of expected fitness.

Each generation:
- Sample λ offspring from N(μ, diag(σ²))
- Rank-shape the fitnesses (utility weights from the standard NES table)
- Update μ along the natural gradient: μ ← μ + η_μ · σ · sum(u_i · z_i)
- Update σ multiplicatively: σ_j ← σ_j · exp(η_σ/2 · sum(u_i · (z_i,j² - 1)))

Vec<f64> decisions only, single-objective only. The diagonal covariance
makes per-step cost O(λ·n) instead of CMA-ES's O(λ·n²) — much faster on
high-dimensional problems where full-covariance tracking is expensive
or numerically fragile, at the cost of being unable to handle strongly
rotated landscapes.
This commit is contained in:
2026-05-05 09:53:20 -06:00
parent 8a34fd94b8
commit e7355ebb8a
3 changed files with 285 additions and 1 deletions
+2
View File
@@ -27,6 +27,7 @@ pub mod random_search;
pub mod rvea;
pub mod simulated_annealing;
pub mod sms_emoa;
pub mod snes;
pub mod spea2;
pub mod tabu_search;
pub mod tlbo;
@@ -58,6 +59,7 @@ pub use random_search::*;
pub use rvea::*;
pub use simulated_annealing::*;
pub use sms_emoa::*;
pub use snes::*;
pub use spea2::*;
pub use tabu_search::*;
pub use tlbo::*;