feat(algorithms): add SeparableNes (Natural Evolution Strategy)

Wierstra et al. 2008/2014 NES with the diagonal-covariance "separable"
variant (sNES). Different theoretical foundation from CMA-ES: rather
than tracking a full covariance matrix and adapting it through
evolution paths, sNES updates the sampling distribution's parameters
by following the natural gradient of expected fitness.

Each generation:
- Sample λ offspring from N(μ, diag(σ²))
- Rank-shape the fitnesses (utility weights from the standard NES table)
- Update μ along the natural gradient: μ ← μ + η_μ · σ · sum(u_i · z_i)
- Update σ multiplicatively: σ_j ← σ_j · exp(η_σ/2 · sum(u_i · (z_i,j² - 1)))

Vec<f64> decisions only, single-objective only. The diagonal covariance
makes per-step cost O(λ·n) instead of CMA-ES's O(λ·n²) — much faster on
high-dimensional problems where full-covariance tracking is expensive
or numerically fragile, at the cost of being unable to handle strongly
rotated landscapes.
This commit is contained in:
2026-05-05 09:53:20 -06:00
parent 8a34fd94b8
commit e7355ebb8a
3 changed files with 285 additions and 1 deletions
+1 -1
View File
@@ -30,7 +30,7 @@ pub use crate::algorithms::{
NelderMead, NelderMeadConfig, Nsga2,
Nsga2Config, Nsga3, Nsga3Config, OnePlusOneEs, OnePlusOneEsConfig, Paes, PaesConfig, ParticleSwarm, PesaII, PesaIIConfig,
ParticleSwarmConfig, RandomSearch, RandomSearchConfig, Rvea, RveaConfig,
SimulatedAnnealing,
SeparableNes, SeparableNesConfig, SimulatedAnnealing,
SimulatedAnnealingConfig, SmsEmoa, SmsEmoaConfig, Spea2, Spea2Config, TabuSearch,
TabuSearchConfig, Tlbo, TlboConfig, Umda,
UmdaConfig,