feat(algorithms): add BayesianOpt — GP-based Bayesian Optimization

The first sample-efficient algorithm in heuropt. Bayesian optimization
maintains a Gaussian-process surrogate of the objective and at each
step picks the next decision by maximizing an acquisition function on
that surrogate, so the evaluation budget is used surgically.

Implementation:
- **Kernel**: anisotropic RBF (squared-exponential) with per-axis
  length scales, signal variance, and a small noise/jitter floor.
  Hyperparameters are exposed in the config; a future version can add
  marginal-likelihood maximization.
- **Posterior**: standard formulation. Cholesky factorizes K (using
  the new internal helper); mean and variance predictions follow.
- **Acquisition**: Expected Improvement against the best observed
  feasible point. Optimized by best-of-N random sampling — simple,
  predictable cost, no inner-optimizer footgun.
- **Initial design**: `initial_samples` uniform-random points in
  bounds before the BO loop starts.
- **Constraints**: feasibility-aware EI — best observed value uses
  only feasible points; infeasible candidates are penalized.

Vec<f64> decisions, single-objective only. Targets the regime no
existing heuropt algorithm covers: 50–500 evaluations on an
expensive black-box function (CFD sim, ML training run, real-world
measurement).

Tests cover convergence on the 1-D sphere within a tight evaluation
budget (~30 evals get to f < 1e-6 — vs population-based methods
needing thousands), deterministic reruns, and panic on
multi-objective + dim mismatches.
This commit is contained in:
2026-05-05 09:51:12 -06:00
parent 284f1143de
commit a70500406c
4 changed files with 423 additions and 2 deletions
+2 -1
View File
@@ -22,7 +22,8 @@ pub use crate::operators::{
};
pub use crate::algorithms::{
AgeMoea, AgeMoeaConfig, AntColonyTsp, AntColonyTspConfig, CmaEs, CmaEsConfig, DifferentialEvolution,
AgeMoea, AgeMoeaConfig, AntColonyTsp, AntColonyTspConfig, BayesianOpt,
BayesianOptConfig, CmaEs, CmaEsConfig, DifferentialEvolution,
DifferentialEvolutionConfig, EpsilonMoea, EpsilonMoeaConfig,
GeneticAlgorithm, GeneticAlgorithmConfig, Grea, GreaConfig, HillClimber, HillClimberConfig, Hype,
HypeConfig, Ibea, IbeaConfig, IpopCmaEs, IpopCmaEsConfig, Knea, KneaConfig, Moead, MoeadConfig, Mopso, MopsoConfig,