feat(algorithms): add BayesianOpt — GP-based Bayesian Optimization
The first sample-efficient algorithm in heuropt. Bayesian optimization maintains a Gaussian-process surrogate of the objective and at each step picks the next decision by maximizing an acquisition function on that surrogate, so the evaluation budget is used surgically. Implementation: - **Kernel**: anisotropic RBF (squared-exponential) with per-axis length scales, signal variance, and a small noise/jitter floor. Hyperparameters are exposed in the config; a future version can add marginal-likelihood maximization. - **Posterior**: standard formulation. Cholesky factorizes K (using the new internal helper); mean and variance predictions follow. - **Acquisition**: Expected Improvement against the best observed feasible point. Optimized by best-of-N random sampling — simple, predictable cost, no inner-optimizer footgun. - **Initial design**: `initial_samples` uniform-random points in bounds before the BO loop starts. - **Constraints**: feasibility-aware EI — best observed value uses only feasible points; infeasible candidates are penalized. Vec<f64> decisions, single-objective only. Targets the regime no existing heuropt algorithm covers: 50–500 evaluations on an expensive black-box function (CFD sim, ML training run, real-world measurement). Tests cover convergence on the 1-D sphere within a tight evaluation budget (~30 evals get to f < 1e-6 — vs population-based methods needing thousands), deterministic reruns, and panic on multi-objective + dim mismatches.
This commit is contained in:
@@ -2,6 +2,7 @@
|
||||
|
||||
pub mod age_moea;
|
||||
pub mod ant_colony_tsp;
|
||||
pub mod bayesian_opt;
|
||||
pub mod cma_es;
|
||||
pub mod differential_evolution;
|
||||
pub mod epsilon_moea;
|
||||
@@ -33,6 +34,7 @@ pub mod umda;
|
||||
|
||||
pub use age_moea::*;
|
||||
pub use ant_colony_tsp::*;
|
||||
pub use bayesian_opt::*;
|
||||
pub use cma_es::*;
|
||||
pub use differential_evolution::*;
|
||||
pub use epsilon_moea::*;
|
||||
|
||||
Reference in New Issue
Block a user