feat(algorithms): add Tpe (Tree-structured Parzen Estimator)
Bergstra et al. 2011: sample-efficient sequential optimizer that's the
workhorse of Hyperopt and Optuna. Different surrogate from BO's
Gaussian process — TPE models p(x | y < y*) with one KDE and
p(x | y >= y*) with another, then samples candidates from the 'good'
KDE and ranks by the ratio l(x) / g(x). The acquisition is implicit
in the ratio (a closed-form analog of Expected Improvement).
Implementation:
- 1-D Gaussian KDE per axis, with bandwidth chosen by Scott's rule
- Per-step:
- Evaluate observations into 'good' (top γ fraction by target) and
'bad'
- Sample n_candidates from the good distribution (independent per
axis) and pick the one with the largest l(x)/g(x)
- Evaluate it, append to history
Vec<f64> only, single-objective only. Compared with BayesianOpt:
- Cheaper per-step (no GP factorization)
- Doesn't need kernel hyperparameter tuning to work well
- Naturally extends to mixed/categorical decision types (future work)
- Generally less sample-efficient than well-tuned BO on smooth
continuous problems, but more robust out of the box
Tests cover convergence on 1-D Sphere within a tight budget,
deterministic reruns, panic on multi-objective.
This commit is contained in:
@@ -31,6 +31,7 @@ pub mod snes;
|
||||
pub mod spea2;
|
||||
pub mod tabu_search;
|
||||
pub mod tlbo;
|
||||
pub mod tpe;
|
||||
pub mod umda;
|
||||
|
||||
pub use age_moea::*;
|
||||
@@ -63,4 +64,5 @@ pub use snes::*;
|
||||
pub use spea2::*;
|
||||
pub use tabu_search::*;
|
||||
pub use tlbo::*;
|
||||
pub use tpe::*;
|
||||
pub use umda::*;
|
||||
|
||||
Reference in New Issue
Block a user