feat(algorithms): add BayesianOpt — GP-based Bayesian Optimization
The first sample-efficient algorithm in heuropt. Bayesian optimization maintains a Gaussian-process surrogate of the objective and at each step picks the next decision by maximizing an acquisition function on that surrogate, so the evaluation budget is used surgically. Implementation: - **Kernel**: anisotropic RBF (squared-exponential) with per-axis length scales, signal variance, and a small noise/jitter floor. Hyperparameters are exposed in the config; a future version can add marginal-likelihood maximization. - **Posterior**: standard formulation. Cholesky factorizes K (using the new internal helper); mean and variance predictions follow. - **Acquisition**: Expected Improvement against the best observed feasible point. Optimized by best-of-N random sampling — simple, predictable cost, no inner-optimizer footgun. - **Initial design**: `initial_samples` uniform-random points in bounds before the BO loop starts. - **Constraints**: feasibility-aware EI — best observed value uses only feasible points; infeasible candidates are penalized. Vec<f64> decisions, single-objective only. Targets the regime no existing heuropt algorithm covers: 50–500 evaluations on an expensive black-box function (CFD sim, ML training run, real-world measurement). Tests cover convergence on the 1-D sphere within a tight evaluation budget (~30 evals get to f < 1e-6 — vs population-based methods needing thousands), deterministic reruns, and panic on multi-objective + dim mismatches.
This commit is contained in:
+2
-1
@@ -22,7 +22,8 @@ pub use crate::operators::{
|
||||
};
|
||||
|
||||
pub use crate::algorithms::{
|
||||
AgeMoea, AgeMoeaConfig, AntColonyTsp, AntColonyTspConfig, CmaEs, CmaEsConfig, DifferentialEvolution,
|
||||
AgeMoea, AgeMoeaConfig, AntColonyTsp, AntColonyTspConfig, BayesianOpt,
|
||||
BayesianOptConfig, CmaEs, CmaEsConfig, DifferentialEvolution,
|
||||
DifferentialEvolutionConfig, EpsilonMoea, EpsilonMoeaConfig,
|
||||
GeneticAlgorithm, GeneticAlgorithmConfig, Grea, GreaConfig, HillClimber, HillClimberConfig, Hype,
|
||||
HypeConfig, Ibea, IbeaConfig, IpopCmaEs, IpopCmaEsConfig, Knea, KneaConfig, Moead, MoeadConfig, Mopso, MopsoConfig,
|
||||
|
||||
Reference in New Issue
Block a user