feat(algorithms): add BayesianOpt — GP-based Bayesian Optimization
The first sample-efficient algorithm in heuropt. Bayesian optimization maintains a Gaussian-process surrogate of the objective and at each step picks the next decision by maximizing an acquisition function on that surrogate, so the evaluation budget is used surgically. Implementation: - **Kernel**: anisotropic RBF (squared-exponential) with per-axis length scales, signal variance, and a small noise/jitter floor. Hyperparameters are exposed in the config; a future version can add marginal-likelihood maximization. - **Posterior**: standard formulation. Cholesky factorizes K (using the new internal helper); mean and variance predictions follow. - **Acquisition**: Expected Improvement against the best observed feasible point. Optimized by best-of-N random sampling — simple, predictable cost, no inner-optimizer footgun. - **Initial design**: `initial_samples` uniform-random points in bounds before the BO loop starts. - **Constraints**: feasibility-aware EI — best observed value uses only feasible points; infeasible candidates are penalized. Vec<f64> decisions, single-objective only. Targets the regime no existing heuropt algorithm covers: 50–500 evaluations on an expensive black-box function (CFD sim, ML training run, real-world measurement). Tests cover convergence on the 1-D sphere within a tight evaluation budget (~30 evals get to f < 1e-6 — vs population-based methods needing thousands), deterministic reruns, and panic on multi-objective + dim mismatches.
This commit is contained in:
@@ -15,9 +15,11 @@ pub(crate) fn cholesky(a: &[Vec<f64>]) -> Result<Vec<Vec<f64>>, &'static str> {
|
||||
}
|
||||
debug_assert!(a.iter().all(|row| row.len() == n));
|
||||
let mut l = vec![vec![0.0_f64; n]; n];
|
||||
#[allow(clippy::needless_range_loop)] // body indexes both `a` and `l` rows.
|
||||
for i in 0..n {
|
||||
for j in 0..=i {
|
||||
let mut sum = a[i][j];
|
||||
#[allow(clippy::needless_range_loop)]
|
||||
for k in 0..j {
|
||||
sum -= l[i][k] * l[j][k];
|
||||
}
|
||||
@@ -90,9 +92,11 @@ mod tests {
|
||||
assert!(approx_eq(l[1][0], 1.0, 1e-12));
|
||||
assert!(approx_eq(l[1][1], 2.0, 1e-12));
|
||||
// L · L^T should reconstruct A.
|
||||
#[allow(clippy::needless_range_loop)]
|
||||
for i in 0..2 {
|
||||
for j in 0..2 {
|
||||
let mut s = 0.0;
|
||||
#[allow(clippy::needless_range_loop)]
|
||||
for k in 0..2 {
|
||||
s += l[i][k] * l[j][k];
|
||||
}
|
||||
@@ -111,7 +115,7 @@ mod tests {
|
||||
];
|
||||
let l = cholesky(&a).unwrap();
|
||||
// Choose a vector and check A · x = b round trip.
|
||||
let x_truth = vec![1.0, -2.0, 0.5];
|
||||
let x_truth = [1.0, -2.0, 0.5];
|
||||
let b: Vec<f64> = (0..3)
|
||||
.map(|i| (0..3).map(|j| a[i][j] * x_truth[j]).sum())
|
||||
.collect();
|
||||
|
||||
Reference in New Issue
Block a user