feat(algorithms): add Tpe (Tree-structured Parzen Estimator)

Bergstra et al. 2011: sample-efficient sequential optimizer that's the
workhorse of Hyperopt and Optuna. Different surrogate from BO's
Gaussian process — TPE models p(x | y < y*) with one KDE and
p(x | y >= y*) with another, then samples candidates from the 'good'
KDE and ranks by the ratio l(x) / g(x). The acquisition is implicit
in the ratio (a closed-form analog of Expected Improvement).

Implementation:
- 1-D Gaussian KDE per axis, with bandwidth chosen by Scott's rule
- Per-step:
  - Evaluate observations into 'good' (top γ fraction by target) and
    'bad'
  - Sample n_candidates from the good distribution (independent per
    axis) and pick the one with the largest l(x)/g(x)
  - Evaluate it, append to history

Vec<f64> only, single-objective only. Compared with BayesianOpt:
- Cheaper per-step (no GP factorization)
- Doesn't need kernel hyperparameter tuning to work well
- Naturally extends to mixed/categorical decision types (future work)
- Generally less sample-efficient than well-tuned BO on smooth
  continuous problems, but more robust out of the box

Tests cover convergence on 1-D Sphere within a tight budget,
deterministic reruns, panic on multi-objective.
This commit is contained in:
2026-05-05 09:55:01 -06:00
parent e7355ebb8a
commit 358e441b36
3 changed files with 358 additions and 1 deletions
+2
View File
@@ -31,6 +31,7 @@ pub mod snes;
pub mod spea2;
pub mod tabu_search;
pub mod tlbo;
pub mod tpe;
pub mod umda;
pub use age_moea::*;
@@ -63,4 +64,5 @@ pub use snes::*;
pub use spea2::*;
pub use tabu_search::*;
pub use tlbo::*;
pub use tpe::*;
pub use umda::*;