diff --git a/docs/book/src/SUMMARY.md b/docs/book/src/SUMMARY.md index 8ccd16d..0b1c747 100644 --- a/docs/book/src/SUMMARY.md +++ b/docs/book/src/SUMMARY.md @@ -12,6 +12,7 @@ - [Recipes](./cookbook.md) - [Parallelize evaluation with rayon](./cookbook/parallel.md) + - [Async evaluation (HTTP / RPC / subprocess)](./cookbook/async.md) - [Tune a model with expensive evaluations](./cookbook/expensive-evaluations.md) - [Compare two algorithms on your problem](./cookbook/compare.md) - [Optimize a permutation (TSP-style)](./cookbook/permutation.md) diff --git a/docs/book/src/cookbook.md b/docs/book/src/cookbook.md index bdc6c7f..b6942f9 100644 --- a/docs/book/src/cookbook.md +++ b/docs/book/src/cookbook.md @@ -7,8 +7,12 @@ project. ## Recipes - [Parallelize evaluation with rayon](./cookbook/parallel.md) — when - your `evaluate` is non-trivial, the `parallel` feature pays for - itself almost immediately. + your `evaluate` is non-trivial CPU work, the `parallel` feature + pays for itself almost immediately. +- [Async evaluation](./cookbook/async.md) — when your `evaluate` is + IO-bound (HTTP / RPC / subprocess), the `async` feature lets the + optimizer await many evaluations concurrently. The differentiating + feature vs other optimization libraries. - [Tune a model with expensive evaluations](./cookbook/expensive-evaluations.md) — `BayesianOpt`, `Tpe`, and `Hyperband` for the 50–500-eval regime. diff --git a/docs/book/src/cookbook/async.md b/docs/book/src/cookbook/async.md new file mode 100644 index 0000000..f569e54 --- /dev/null +++ b/docs/book/src/cookbook/async.md @@ -0,0 +1,165 @@ +# Async evaluation + +When your `evaluate` does **IO** — calls an HTTP service, sends an +RPC, spawns a subprocess — `await`-ing it from the optimizer is +much more efficient than blocking a thread per evaluation. heuropt +ships first-class async support behind the `async` feature flag. + +This is the differentiating capability vs pymoo / hyperopt / +optuna / DEAP / MOEA Framework — none of those have a native async +evaluation path. + +## Enable the feature + +```toml +[dependencies] +heuropt = { version = "0.8", features = ["async"] } + +# Pick whatever async runtime you want; heuropt itself depends only on +# `futures`. The example below uses tokio. +tokio = { version = "1", features = ["rt-multi-thread", "macros", "time"] } +``` + +## Implement `AsyncProblem` + +It mirrors the regular [`Problem`] trait one-for-one — same +`Decision` type, same `objectives()`, but `evaluate` is replaced +with `evaluate_async` returning a future. + +```rust,no_run +use heuropt::core::async_problem::AsyncProblem; +use heuropt::prelude::*; + +struct RemoteService; + +impl AsyncProblem for RemoteService { + type Decision = Vec; + + fn objectives(&self) -> ObjectiveSpace { + ObjectiveSpace::new(vec![Objective::minimize("loss")]) + } + + async fn evaluate_async(&self, x: &Vec) -> Evaluation { + // Real workload: HTTP call to a model-scoring service, an RPC, + // a subprocess. Here we just sleep to model 20 ms latency. + tokio::time::sleep(std::time::Duration::from_millis(20)).await; + let loss: f64 = x.iter().map(|v| v * v).sum(); + Evaluation::new(vec![loss]) + } +} +``` + +## Run the optimizer with `run_async` + +`run_async(&problem, concurrency).await` is provided by **every** +algorithm in the catalog as of v0.8. `concurrency` caps how many +evaluations are in-flight at once. + +```rust,no_run +# use heuropt::core::async_problem::AsyncProblem; +# use heuropt::prelude::*; +# struct RemoteService; +# impl AsyncProblem for RemoteService { +# type Decision = Vec; +# fn objectives(&self) -> ObjectiveSpace { +# ObjectiveSpace::new(vec![Objective::minimize("loss")]) +# } +# async fn evaluate_async(&self, x: &Vec) -> Evaluation { +# Evaluation::new(vec![x.iter().map(|v| v * v).sum::()]) +# } +# } +#[tokio::main] +async fn main() { + let bounds = vec![(-1.0_f64, 1.0_f64); 4]; + let mut opt = DifferentialEvolution::new( + DifferentialEvolutionConfig { + population_size: 16, + generations: 50, + differential_weight: 0.5, + crossover_probability: 0.9, + seed: 42, + }, + RealBounds::new(bounds), + ); + let r = opt.run_async(&RemoteService, /* concurrency */ 8).await; + println!("best: {}", r.best.unwrap().evaluation.objectives[0]); +} +``` + +## Picking `concurrency` + +Concurrency is the maximum in-flight evaluation count. Tradeoffs: + +| Setting | Effect | +|---|---| +| `1` | Sequential; equivalent to a sync run with extra overhead | +| `pop_size` | Full per-generation parallelism; fastest if your service tolerates it | +| `< pop_size` | Bounded — useful if your downstream service has a rate limit or finite worker pool | + +The bigger you go, the more memory the in-flight futures hold and +the more load you put on the downstream service. A reasonable +starting point is `min(pop_size, 16)` and increase only if the +downstream service is comfortable. + +## Determinism + +Same seed produces the same final result whether you use `run` or +`run_async`, **provided your async `evaluate_async` is itself +deterministic**. heuropt drives the RNG and selection on the main +task; only the evaluations are concurrent, and the +`evaluate_batch_async` helper preserves input order before feeding +results back to the algorithm. + +## What the worked example shows + +`examples/async_eval.rs` runs `RandomSearch` (200 evaluations × 20 ms +each) at `concurrency = 1, 4, 16` and `DifferentialEvolution` at +`concurrency = 8`. On a recent machine: + +```text +RandomSearch with 200 evaluations (20 ms each) + +concurrency = 1 elapsed ≈ 4250 ms (sequential 200 × 20 ms) +concurrency = 4 elapsed ≈ 2100 ms (2× speedup, batch_size=2 caps it) +concurrency = 16 elapsed ≈ 2100 ms (same — batch_size dominates) + +DifferentialEvolution at concurrency=8 +elapsed ≈ 230 ms (8 ants run in parallel each generation) +``` + +Run it yourself: `cargo run --release --features async --example async_eval`. + +## Which algorithms support `run_async`? + +**All 33** algorithms in the catalog. The shape of the async path +depends on the algorithm: + +- **Population-based / batch-evaluating** — NSGA-II, NSGA-III, SPEA2, + MOEA/D, IBEA, SMS-EMOA, HypE, ε-MOEA, PESA-II, AGE-MOEA, KnEA, + GrEA, RVEA, MOPSO, GA, DE, PSO, CMA-ES, IPOP-CMA-ES, sNES, TLBO, + UMDA, Ant Colony, Random Search. Each generation's offspring + evaluations are fanned out concurrently up to `concurrency`. +- **Steady-state (one-eval-per-step)** — Hill Climber, Simulated + Annealing, (1+1)-ES, PAES, Nelder-Mead. The `concurrency` + parameter is accepted for API uniformity but evaluation order is + inherently sequential. +- **Tabu Search** — fans out the K-neighbor batch each step. +- **Surrogate (BO, TPE)** — fans out the initial design batch, then + awaits per-iteration acquisitions sequentially (the surrogate + must update before the next point is chosen). +- **Hyperband** — uses the separate + [`AsyncPartialProblem`](https://docs.rs/heuropt/latest/heuropt/core/async_problem/trait.AsyncPartialProblem.html) + trait (multi-fidelity); each Successive-Halving rung's evaluations + fan out concurrently. + +## Async vs `parallel` + +| If your `evaluate` is… | Use | +|---|---| +| CPU-bound (math, simulation) | `parallel` feature → see [Parallelize evaluation](./parallel.md) | +| IO-bound (HTTP, RPC, subprocess) | `async` feature (this recipe) | + +Both can be on at once if your evaluation does *both* substantial +CPU work *and* IO. The two features are independent. + +[`Problem`]: https://docs.rs/heuropt/latest/heuropt/core/problem/trait.Problem.html diff --git a/src/algorithms/age_moea.rs b/src/algorithms/age_moea.rs index 6414fc7..bc9f1da 100644 --- a/src/algorithms/age_moea.rs +++ b/src/algorithms/age_moea.rs @@ -155,6 +155,80 @@ where } } +#[cfg(feature = "async")] +impl AgeMoea { + /// Async version of [`Optimizer::run`] — drives evaluations through + /// the user-chosen async runtime. Available only with the `async` + /// feature. + /// + /// `concurrency` bounds in-flight evaluations per batch. + pub async fn run_async

( + &mut self, + problem: &P, + concurrency: usize, + ) -> OptimizationResult + where + P: crate::core::async_problem::AsyncProblem, + I: Initializer, + V: Variation, + { + use crate::algorithms::parallel_eval_async::evaluate_batch_async; + + assert!( + self.config.population_size > 0, + "AgeMoea population_size must be > 0" + ); + let n = self.config.population_size; + let objectives = problem.objectives(); + let mut rng = rng_from_seed(self.config.seed); + + let initial_decisions = self.initializer.initialize(n, &mut rng); + let mut population: Vec> = + evaluate_batch_async(problem, initial_decisions, concurrency).await; + let mut evaluations = population.len(); + + for _ in 0..self.config.generations { + let mut offspring_decisions: Vec = Vec::with_capacity(n); + while offspring_decisions.len() < n { + let p1 = rng.random_range(0..population.len()); + let p2 = rng.random_range(0..population.len()); + let parents = vec![ + population[p1].decision.clone(), + population[p2].decision.clone(), + ]; + let children = self.variation.vary(&parents, &mut rng); + assert!( + !children.is_empty(), + "AgeMoea variation returned no children" + ); + for child in children { + if offspring_decisions.len() >= n { + break; + } + offspring_decisions.push(child); + } + } + let offspring = evaluate_batch_async(problem, offspring_decisions, concurrency).await; + evaluations += offspring.len(); + + let mut combined: Vec> = Vec::with_capacity(2 * n); + combined.extend(population); + combined.extend(offspring); + population = environmental_selection(combined, &objectives, n); + } + + let front = pareto_front(&population, &objectives); + let best = best_candidate(&population, &objectives); + OptimizationResult::new( + Population::new(population), + front, + best, + evaluations, + self.config.generations, + ) + } +} + fn environmental_selection( combined: Vec>, objectives: &ObjectiveSpace, diff --git a/src/algorithms/ant_colony_tsp.rs b/src/algorithms/ant_colony_tsp.rs index 84f659e..5bd8522 100644 --- a/src/algorithms/ant_colony_tsp.rs +++ b/src/algorithms/ant_colony_tsp.rs @@ -241,6 +241,119 @@ where } } +#[cfg(feature = "async")] +impl AntColonyTsp { + /// Async version of [`Optimizer::run`] — drives evaluations through + /// the user-chosen async runtime. Available only with the `async` + /// feature. + /// + /// `concurrency` bounds in-flight evaluations per generation. + pub async fn run_async

( + &mut self, + problem: &P, + concurrency: usize, + ) -> OptimizationResult> + where + P: crate::core::async_problem::AsyncProblem>, + { + use crate::algorithms::parallel_eval_async::evaluate_batch_async; + + assert!(self.config.ants >= 1, "AntColonyTsp ants must be >= 1"); + let objectives = problem.objectives(); + assert!( + objectives.is_single_objective(), + "AntColonyTsp requires exactly one objective", + ); + let direction = objectives.objectives[0].direction; + let n = self.distances.len(); + let mut rng = rng_from_seed(self.config.seed); + + let eta: Vec> = self + .distances + .iter() + .map(|row| { + row.iter() + .map(|&d| if d > 0.0 { 1.0 / d } else { 0.0 }) + .collect() + }) + .collect(); + + let mut pheromone: Vec> = vec![vec![self.config.initial_pheromone; n]; n]; + + let mut best_decision: Option> = None; + let mut best_eval: Option = None; + let mut evaluations = 0usize; + + for _ in 0..self.config.generations { + let mut tours: Vec> = Vec::with_capacity(self.config.ants); + for _ in 0..self.config.ants { + let start = rng.random_range(0..n); + let tour = build_tour( + n, + start, + &pheromone, + &eta, + self.config.alpha, + self.config.beta, + &mut rng, + ); + tours.push(tour); + } + + let cands = evaluate_batch_async(problem, tours.clone(), concurrency).await; + evaluations += cands.len(); + let tour_evals: Vec = + cands.into_iter().map(|c| c.evaluation).collect(); + + for (tour, eval) in tours.iter().zip(tour_evals.iter()) { + let beats = match &best_eval { + None => true, + Some(b) => better_than_so(eval, b, direction), + }; + if beats { + best_decision = Some(tour.clone()); + best_eval = Some(eval.clone()); + } + } + + for row in pheromone.iter_mut() { + for v in row.iter_mut() { + *v *= 1.0 - self.config.evaporation; + } + } + + for (tour, eval) in tours.iter().zip(tour_evals.iter()) { + let length = eval + .objectives + .first() + .copied() + .unwrap_or(f64::INFINITY) + .max(1e-12); + let deposit = self.config.deposit / length; + for w in tour.windows(2) { + let (i, j) = (w[0], w[1]); + pheromone[i][j] += deposit; + pheromone[j][i] += deposit; + } + let (i, j) = (*tour.last().unwrap(), tour[0]); + pheromone[i][j] += deposit; + pheromone[j][i] += deposit; + } + } + + let best = Candidate::new(best_decision.unwrap(), best_eval.unwrap()); + let population = Population::new(vec![best.clone()]); + let front = vec![best.clone()]; + OptimizationResult::new( + population, + front, + Some(best), + evaluations, + self.config.generations, + ) + } +} + fn build_tour( n: usize, start: usize, diff --git a/src/algorithms/bayesian_opt.rs b/src/algorithms/bayesian_opt.rs index f905582..d7609d7 100644 --- a/src/algorithms/bayesian_opt.rs +++ b/src/algorithms/bayesian_opt.rs @@ -396,6 +396,150 @@ fn erf(x: f64) -> f64 { sign * y } +#[cfg(feature = "async")] +impl BayesianOpt { + /// Async version of [`Optimizer::run`] — drives evaluations through + /// the user-chosen async runtime. Available only with the `async` + /// feature. + /// + /// `concurrency` bounds in-flight evaluations during the initial + /// uniform-sample design; the sequential BO loop runs one + /// evaluation per iteration regardless. + pub async fn run_async

( + &mut self, + problem: &P, + concurrency: usize, + ) -> OptimizationResult> + where + P: crate::core::async_problem::AsyncProblem>, + { + use crate::algorithms::parallel_eval_async::evaluate_batch_async; + + assert!( + self.config.initial_samples >= 2, + "BayesianOpt initial_samples must be >= 2", + ); + assert!( + self.config.signal_variance > 0.0, + "BayesianOpt signal_variance must be > 0" + ); + assert!( + self.config.noise_variance > 0.0, + "BayesianOpt noise_variance must be > 0" + ); + assert!( + self.config.acquisition_samples >= 1, + "BayesianOpt acquisition_samples must be >= 1", + ); + let objectives = problem.objectives(); + assert!( + objectives.is_single_objective(), + "BayesianOpt requires exactly one objective", + ); + let direction = objectives.objectives[0].direction; + let dim = self.bounds.bounds.len(); + if let Some(ls) = &self.config.length_scales { + assert_eq!( + ls.len(), + dim, + "BayesianOpt length_scales.len() must equal dim" + ); + } + let length_scales: Vec = self.config.length_scales.clone().unwrap_or_else(|| { + self.bounds + .bounds + .iter() + .map(|&(lo, hi)| 0.2 * (hi - lo).max(1e-9)) + .collect() + }); + let mut rng = rng_from_seed(self.config.seed); + + // Initial random design: sample all decisions first (consuming + // RNG in the same order as the sync `run`), then evaluate + // concurrently. + let mut decisions: Vec> = + Vec::with_capacity(self.config.initial_samples + self.config.iterations); + let mut targets: Vec = Vec::with_capacity(decisions.capacity()); + let mut evaluations: Vec = Vec::with_capacity(decisions.capacity()); + + let initial_decisions: Vec> = (0..self.config.initial_samples) + .map(|_| sample_uniform_in_bounds(&self.bounds, &mut rng)) + .collect(); + let initial_cands = evaluate_batch_async(problem, initial_decisions, concurrency).await; + for c in initial_cands { + let t = oriented_target(&c.evaluation, direction); + decisions.push(c.decision); + targets.push(t); + evaluations.push(c.evaluation); + } + + for _ in 0..self.config.iterations { + let posterior = match GpPosterior::fit( + &decisions, + &targets, + &length_scales, + self.config.signal_variance, + self.config.noise_variance, + ) { + Ok(p) => p, + Err(_) => { + let x = sample_uniform_in_bounds(&self.bounds, &mut rng); + let e = problem.evaluate_async(&x).await; + targets.push(oriented_target(&e, direction)); + decisions.push(x); + evaluations.push(e); + continue; + } + }; + + let best_target = targets.iter().cloned().fold(f64::INFINITY, f64::min); + + let mut best_x = sample_uniform_in_bounds(&self.bounds, &mut rng); + let mut best_ei = -f64::INFINITY; + for _ in 0..self.config.acquisition_samples { + let cand = sample_uniform_in_bounds(&self.bounds, &mut rng); + let (mu, sigma) = posterior.predict(&cand); + let ei = expected_improvement(mu, sigma, best_target); + if ei > best_ei { + best_ei = ei; + best_x = cand; + } + } + + let e = problem.evaluate_async(&best_x).await; + targets.push(oriented_target(&e, direction)); + decisions.push(best_x); + evaluations.push(e); + } + + let final_pop: Vec>> = decisions + .into_iter() + .zip(evaluations) + .map(|(d, e)| Candidate::new(d, e)) + .collect(); + let mut best_idx = 0; + for i in 1..final_pop.len() { + if better( + &final_pop[i].evaluation, + &final_pop[best_idx].evaluation, + direction, + ) { + best_idx = i; + } + } + let total_evaluations = final_pop.len(); + let best = final_pop[best_idx].clone(); + let front = vec![best.clone()]; + OptimizationResult::new( + Population::new(final_pop), + front, + Some(best), + total_evaluations, + self.config.iterations + self.config.initial_samples, + ) + } +} + #[cfg(test)] mod tests { use super::*; diff --git a/src/algorithms/cma_es.rs b/src/algorithms/cma_es.rs index 30dfef0..2723692 100644 --- a/src/algorithms/cma_es.rs +++ b/src/algorithms/cma_es.rs @@ -366,6 +366,237 @@ where } } +#[cfg(feature = "async")] +impl CmaEs { + /// Async version of [`Optimizer::run`] — drives evaluations through + /// the user-chosen async runtime. Available only with the `async` + /// feature. + /// + /// `concurrency` bounds in-flight evaluations per generation. + pub async fn run_async

( + &mut self, + problem: &P, + concurrency: usize, + ) -> OptimizationResult> + where + P: crate::core::async_problem::AsyncProblem>, + { + use crate::algorithms::parallel_eval_async::evaluate_batch_async; + + assert!( + self.config.population_size >= 4, + "CmaEs population_size must be >= 4", + ); + assert!( + self.config.initial_sigma > 0.0, + "CmaEs initial_sigma must be positive", + ); + assert!( + self.config.eigen_decomposition_period >= 1, + "CmaEs eigen_decomposition_period must be >= 1", + ); + let objectives = problem.objectives(); + assert!( + objectives.is_single_objective(), + "CmaEs only supports single-objective problems", + ); + let direction = objectives.objectives[0].direction; + + let n = self.bounds.bounds.len(); + let n_f = n as f64; + let lambda = self.config.population_size; + let lambda_f = lambda as f64; + let mu = lambda / 2; + assert!(mu >= 1, "CmaEs derived mu (= lambda/2) must be >= 1"); + let mut rng = rng_from_seed(self.config.seed); + + let raw_weights: Vec = (0..mu) + .map(|i| ((lambda_f + 1.0) / 2.0).ln() - ((i + 1) as f64).ln()) + .collect(); + let sum_w: f64 = raw_weights.iter().sum(); + let weights: Vec = raw_weights.iter().map(|w| w / sum_w).collect(); + let mu_eff = 1.0 / weights.iter().map(|w| w * w).sum::(); + + let c_sigma = (mu_eff + 2.0) / (n_f + mu_eff + 5.0); + let d_sigma = 1.0 + 2.0 * ((mu_eff - 1.0) / (n_f + 1.0)).sqrt().max(0.0) + c_sigma; + let c_c = (4.0 + mu_eff / n_f) / (n_f + 4.0 + 2.0 * mu_eff / n_f); + let c_1 = 2.0 / ((n_f + 1.3).powi(2) + mu_eff); + let c_mu = ((1.0 - c_1) * 2.0 * (mu_eff - 2.0 + 1.0 / mu_eff) + / ((n_f + 2.0).powi(2) + mu_eff)) + .min(1.0 - c_1); + let chi_n = n_f.sqrt() * (1.0 - 1.0 / (4.0 * n_f) + 1.0 / (21.0 * n_f * n_f)); + + let mut mean: Vec = if let Some(provided) = self.config.initial_mean.clone() { + assert_eq!( + provided.len(), + self.bounds.bounds.len(), + "CmaEs initial_mean.len() must equal the bounds dimension", + ); + provided + .into_iter() + .zip(self.bounds.bounds.iter()) + .map(|(v, &(lo, hi))| v.clamp(lo, hi)) + .collect() + } else { + self.bounds + .bounds + .iter() + .map(|&(lo, hi)| 0.5 * (lo + hi)) + .collect() + }; + let mut sigma = self.config.initial_sigma; + let mut c_matrix: Vec> = (0..n) + .map(|i| (0..n).map(|j| if i == j { 1.0 } else { 0.0 }).collect()) + .collect(); + let mut b: Vec> = c_matrix.to_vec(); + let mut d: Vec = vec![1.0; n]; + let mut p_sigma = vec![0.0_f64; n]; + let mut p_c = vec![0.0_f64; n]; + let mut evaluations = 0usize; + + let normal = Normal::new(0.0, 1.0).expect("Normal::new(0, 1)"); + let mut best_candidate_seen: Option>> = None; + + for generation in 0..self.config.generations { + if generation % self.config.eigen_decomposition_period == 0 { + #[allow(clippy::needless_range_loop)] + for i in 0..n { + for j in (i + 1)..n { + let avg = 0.5 * (c_matrix[i][j] + c_matrix[j][i]); + c_matrix[i][j] = avg; + c_matrix[j][i] = avg; + } + } + let (eigenvalues, eigenvectors) = symmetric_eigen(&c_matrix, 1e-14, 100); + d = eigenvalues.iter().map(|&v| v.max(1e-20).sqrt()).collect(); + b = (0..n) + .map(|r| (0..n).map(|c| eigenvectors[c][r]).collect()) + .collect(); + } + + let mut z_samples: Vec> = Vec::with_capacity(lambda); + let mut x_samples: Vec> = Vec::with_capacity(lambda); + for _ in 0..lambda { + let z: Vec = (0..n).map(|_| normal.sample(&mut rng)).collect(); + let bd_z: Vec = (0..n) + .map(|i| (0..n).map(|j| b[i][j] * d[j] * z[j]).sum::()) + .collect(); + let x: Vec = (0..n) + .map(|i| { + let v = mean[i] + sigma * bd_z[i]; + let (lo, hi) = self.bounds.bounds[i]; + v.clamp(lo, hi) + }) + .collect(); + z_samples.push(z); + x_samples.push(x); + } + + let evaluated = evaluate_batch_async(problem, x_samples.clone(), concurrency).await; + evaluations += evaluated.len(); + + for c in &evaluated { + let beats_best = match &best_candidate_seen { + None => true, + Some(b) => better_than_so(&c.evaluation, &b.evaluation, direction), + }; + if beats_best { + best_candidate_seen = Some(c.clone()); + } + } + + let mut order: Vec = (0..lambda).collect(); + order.sort_by(|&a, &b_| { + compare_so( + &evaluated[a].evaluation, + &evaluated[b_].evaluation, + direction, + ) + }); + + let old_mean = mean.clone(); + let mut new_mean = vec![0.0_f64; n]; + for k in 0..mu { + let xk = &x_samples[order[k]]; + let wk = weights[k]; + for i in 0..n { + new_mean[i] += wk * xk[i]; + } + } + mean = new_mean; + + let mut z_weighted = vec![0.0_f64; n]; + for k in 0..mu { + let zk = &z_samples[order[k]]; + let wk = weights[k]; + for i in 0..n { + z_weighted[i] += wk * zk[i]; + } + } + + let factor_p_sigma = (c_sigma * (2.0 - c_sigma) * mu_eff).sqrt(); + let bz: Vec = (0..n) + .map(|i| (0..n).map(|j| b[i][j] * z_weighted[j]).sum::()) + .collect(); + for i in 0..n { + p_sigma[i] = (1.0 - c_sigma) * p_sigma[i] + factor_p_sigma * bz[i]; + } + + let p_sigma_norm = p_sigma.iter().map(|x| x * x).sum::().sqrt(); + sigma *= ((c_sigma / d_sigma) * (p_sigma_norm / chi_n - 1.0)).exp(); + + let h_sigma = if p_sigma_norm + / (1.0 - (1.0 - c_sigma).powi(2 * (generation as i32 + 1))).sqrt() + < (1.4 + 2.0 / (n_f + 1.0)) * chi_n + { + 1.0 + } else { + 0.0 + }; + + let factor_p_c = h_sigma * (c_c * (2.0 - c_c) * mu_eff).sqrt(); + for i in 0..n { + p_c[i] = (1.0 - c_c) * p_c[i] + factor_p_c * (mean[i] - old_mean[i]) / sigma; + } + + let delta_h = (1.0 - h_sigma) * c_c * (2.0 - c_c); + #[allow(clippy::needless_range_loop)] + for i in 0..n { + for j in 0..n { + let mut update = (1.0 - c_1 - c_mu) * c_matrix[i][j] + + c_1 * (p_c[i] * p_c[j] + delta_h * c_matrix[i][j]); + let mut rank_mu_term = 0.0; + for k in 0..mu { + let xk = &x_samples[order[k]]; + let yi = (xk[i] - old_mean[i]) / sigma; + let yj = (xk[j] - old_mean[j]) / sigma; + rank_mu_term += weights[k] * yi * yj; + } + update += c_mu * rank_mu_term; + c_matrix[i][j] = update; + } + } + + for (i, m) in mean.iter_mut().enumerate() { + let (lo, hi) = self.bounds.bounds[i]; + *m = m.clamp(lo, hi); + } + } + + let best = best_candidate_seen.expect("at least one generation evaluated"); + let final_pop = vec![best.clone()]; + let front = vec![best.clone()]; + let best_opt = best_candidate(&final_pop, &objectives); + OptimizationResult::new( + Population::new(final_pop), + front, + best_opt, + evaluations, + self.config.generations, + ) + } +} + fn compare_so( a: &crate::core::evaluation::Evaluation, b: &crate::core::evaluation::Evaluation, diff --git a/src/algorithms/epsilon_moea.rs b/src/algorithms/epsilon_moea.rs index 6df3078..2f3361a 100644 --- a/src/algorithms/epsilon_moea.rs +++ b/src/algorithms/epsilon_moea.rs @@ -190,6 +190,98 @@ where } } +#[cfg(feature = "async")] +impl EpsilonMoea { + /// Async version of [`Optimizer::run`] — drives evaluations through + /// the user-chosen async runtime. Available only with the `async` + /// feature. + /// + /// `concurrency` bounds in-flight evaluations of the initial + /// population. Per-step evaluations are sequential because the + /// algorithm is steady-state (one offspring per step). + pub async fn run_async

( + &mut self, + problem: &P, + concurrency: usize, + ) -> OptimizationResult + where + P: crate::core::async_problem::AsyncProblem, + I: Initializer, + V: Variation, + { + use crate::algorithms::parallel_eval_async::evaluate_batch_async; + + assert!( + self.config.population_size > 0, + "EpsilonMoea population_size must be > 0" + ); + let n = self.config.population_size; + let objectives = problem.objectives(); + assert_eq!( + self.config.epsilon.len(), + objectives.len(), + "EpsilonMoea epsilon.len() must equal number of objectives", + ); + for (i, &e) in self.config.epsilon.iter().enumerate() { + assert!(e > 0.0, "EpsilonMoea epsilon[{i}] must be > 0.0"); + } + let epsilon = self.config.epsilon.clone(); + let mut rng = rng_from_seed(self.config.seed); + + let initial_decisions = self.initializer.initialize(n, &mut rng); + let mut population: Vec> = + evaluate_batch_async(problem, initial_decisions, concurrency).await; + let mut evaluations = population.len(); + + let mut archive: Vec> = Vec::new(); + for c in &population { + insert_into_epsilon_archive(&mut archive, c.clone(), &objectives, &epsilon); + } + + let total_evals = self.config.evaluations.max(evaluations); + while evaluations < total_evals { + let p1_idx = rng.random_range(0..population.len()); + let parent_a = population[p1_idx].decision.clone(); + let parent_b = if !archive.is_empty() { + let j = rng.random_range(0..archive.len()); + archive[j].decision.clone() + } else { + let j = rng.random_range(0..population.len()); + population[j].decision.clone() + }; + let parents = vec![parent_a, parent_b]; + let children = self.variation.vary(&parents, &mut rng); + assert!( + !children.is_empty(), + "EpsilonMoea variation returned no children" + ); + let child_decision = children.into_iter().next().unwrap(); + let child_eval = problem.evaluate_async(&child_decision).await; + evaluations += 1; + let child = Candidate::new(child_decision, child_eval); + + update_population(&mut population, &child, &objectives, &mut rng); + + insert_into_epsilon_archive(&mut archive, child, &objectives, &epsilon); + } + + let final_pop: Vec> = if !archive.is_empty() { + archive.clone() + } else { + population + }; + let front = pareto_front(&final_pop, &objectives); + let best = best_candidate(&final_pop, &objectives); + OptimizationResult::new( + Population::new(final_pop), + front, + best, + evaluations, + self.config.evaluations, + ) + } +} + /// Standard ε-MOEA population update: if the child is dominated by some /// member, drop it; if it dominates a member, replace that member; if /// non-dominated wrt all, replace a random member. diff --git a/src/algorithms/genetic_algorithm.rs b/src/algorithms/genetic_algorithm.rs index 3668afa..486ba1c 100644 --- a/src/algorithms/genetic_algorithm.rs +++ b/src/algorithms/genetic_algorithm.rs @@ -181,6 +181,94 @@ where } } +#[cfg(feature = "async")] +impl GeneticAlgorithm { + /// Async version of [`Optimizer::run`] — drives evaluations through + /// the user-chosen async runtime. Available only with the `async` + /// feature. + /// + /// `concurrency` bounds in-flight evaluations per batch (initial + /// population and per-generation offspring). + pub async fn run_async

( + &mut self, + problem: &P, + concurrency: usize, + ) -> OptimizationResult + where + P: crate::core::async_problem::AsyncProblem, + I: Initializer, + V: Variation, + { + use crate::algorithms::parallel_eval_async::evaluate_batch_async; + + assert!( + self.config.population_size >= 2, + "GeneticAlgorithm population_size must be >= 2", + ); + assert!( + self.config.tournament_size >= 1, + "GeneticAlgorithm tournament_size must be >= 1", + ); + assert!( + self.config.elitism < self.config.population_size, + "GeneticAlgorithm elitism must be < population_size", + ); + let n = self.config.population_size; + let objectives = problem.objectives(); + assert!( + objectives.is_single_objective(), + "GeneticAlgorithm requires exactly one objective", + ); + let direction = objectives.objectives[0].direction; + let mut rng = rng_from_seed(self.config.seed); + + let initial_decisions = self.initializer.initialize(n, &mut rng); + let mut population: Vec> = + evaluate_batch_async(problem, initial_decisions, concurrency).await; + let mut evaluations = population.len(); + + for _ in 0..self.config.generations { + let mut offspring_decisions: Vec = Vec::with_capacity(n); + while offspring_decisions.len() < n { + let parents_decisions = tournament_select_single_objective( + &population, + &objectives, + self.config.tournament_size, + 2, + &mut rng, + ); + let children = self.variation.vary(&parents_decisions, &mut rng); + assert!( + !children.is_empty(), + "GeneticAlgorithm variation returned no children" + ); + for child in children { + if offspring_decisions.len() >= n { + break; + } + offspring_decisions.push(child); + } + } + + let offspring = evaluate_batch_async(problem, offspring_decisions, concurrency).await; + evaluations += offspring.len(); + + population = + survival_selection(&population, offspring, direction, n, self.config.elitism); + } + + let best = best_candidate(&population, &objectives); + let front: Vec> = best.iter().cloned().collect(); + OptimizationResult::new( + Population::new(population), + front, + best, + evaluations, + self.config.generations, + ) + } +} + fn survival_selection( parents: &[Candidate], offspring: Vec>, diff --git a/src/algorithms/grea.rs b/src/algorithms/grea.rs index 927d607..a522409 100644 --- a/src/algorithms/grea.rs +++ b/src/algorithms/grea.rs @@ -160,6 +160,82 @@ where } } +#[cfg(feature = "async")] +impl Grea { + /// Async version of [`Optimizer::run`] — drives evaluations through + /// the user-chosen async runtime. Available only with the `async` + /// feature. + /// + /// `concurrency` bounds in-flight evaluations per batch. + pub async fn run_async

( + &mut self, + problem: &P, + concurrency: usize, + ) -> OptimizationResult + where + P: crate::core::async_problem::AsyncProblem, + I: Initializer, + V: Variation, + { + use crate::algorithms::parallel_eval_async::evaluate_batch_async; + + assert!( + self.config.population_size > 0, + "Grea population_size must be > 0" + ); + assert!( + self.config.grid_divisions >= 1, + "Grea grid_divisions must be >= 1" + ); + let n = self.config.population_size; + let objectives = problem.objectives(); + let mut rng = rng_from_seed(self.config.seed); + + let initial_decisions = self.initializer.initialize(n, &mut rng); + let mut population: Vec> = + evaluate_batch_async(problem, initial_decisions, concurrency).await; + let mut evaluations = population.len(); + + for _ in 0..self.config.generations { + let mut offspring_decisions: Vec = Vec::with_capacity(n); + while offspring_decisions.len() < n { + let p1 = rng.random_range(0..population.len()); + let p2 = rng.random_range(0..population.len()); + let parents = vec![ + population[p1].decision.clone(), + population[p2].decision.clone(), + ]; + let children = self.variation.vary(&parents, &mut rng); + assert!(!children.is_empty(), "Grea variation returned no children"); + for child in children { + if offspring_decisions.len() >= n { + break; + } + offspring_decisions.push(child); + } + } + let offspring = evaluate_batch_async(problem, offspring_decisions, concurrency).await; + evaluations += offspring.len(); + + let mut combined: Vec> = Vec::with_capacity(2 * n); + combined.extend(population); + combined.extend(offspring); + population = + environmental_selection(combined, &objectives, n, self.config.grid_divisions); + } + + let front = pareto_front(&population, &objectives); + let best = best_candidate(&population, &objectives); + OptimizationResult::new( + Population::new(population), + front, + best, + evaluations, + self.config.generations, + ) + } +} + fn environmental_selection( combined: Vec>, objectives: &ObjectiveSpace, diff --git a/src/algorithms/hill_climber.rs b/src/algorithms/hill_climber.rs index d63378a..d00aa97 100644 --- a/src/algorithms/hill_climber.rs +++ b/src/algorithms/hill_climber.rs @@ -146,6 +146,84 @@ where } } +#[cfg(feature = "async")] +impl HillClimber { + /// Async version of [`Optimizer::run`] — drives evaluations through + /// the user-chosen async runtime. Available only with the `async` + /// feature. + /// + /// `concurrency` is mostly inert here because HillClimber evaluates + /// one child per iteration; it's accepted for API parity with other + /// algorithms. + pub async fn run_async

( + &mut self, + problem: &P, + concurrency: usize, + ) -> OptimizationResult + where + P: crate::core::async_problem::AsyncProblem, + I: Initializer, + V: Variation, + { + let _ = concurrency; + let objectives = problem.objectives(); + assert!( + objectives.is_single_objective(), + "HillClimber requires exactly one objective", + ); + let direction = objectives.objectives[0].direction; + let mut rng = rng_from_seed(self.config.seed); + + let mut initial = self.initializer.initialize(1, &mut rng); + assert!( + !initial.is_empty(), + "HillClimber initializer returned no decisions" + ); + let mut current_decision = initial.remove(0); + let mut current_eval = problem.evaluate_async(¤t_decision).await; + let mut evaluations = 1usize; + + for _ in 0..self.config.iterations { + let parents = vec![current_decision.clone()]; + let children = self.variation.vary(&parents, &mut rng); + assert!( + !children.is_empty(), + "HillClimber variation returned no children" + ); + let child_decision = children.into_iter().next().unwrap(); + let child_eval = problem.evaluate_async(&child_decision).await; + evaluations += 1; + + let child_better = match (child_eval.is_feasible(), current_eval.is_feasible()) { + (true, false) => true, + (false, true) => false, + (false, false) => { + child_eval.constraint_violation < current_eval.constraint_violation + } + (true, true) => match direction { + Direction::Minimize => child_eval.objectives[0] < current_eval.objectives[0], + Direction::Maximize => child_eval.objectives[0] > current_eval.objectives[0], + }, + }; + if child_better { + current_decision = child_decision; + current_eval = child_eval; + } + } + + let best = Candidate::new(current_decision, current_eval); + let population = Population::new(vec![best.clone()]); + let front = vec![best.clone()]; + OptimizationResult::new( + population, + front, + Some(best), + evaluations, + self.config.iterations, + ) + } +} + #[cfg(test)] mod tests { use super::*; diff --git a/src/algorithms/hype.rs b/src/algorithms/hype.rs index 2e11cfe..2dfe3aa 100644 --- a/src/algorithms/hype.rs +++ b/src/algorithms/hype.rs @@ -226,6 +226,131 @@ where } } +#[cfg(feature = "async")] +impl Hype { + /// Async version of [`Optimizer::run`] — drives evaluations through + /// the user-chosen async runtime. Available only with the `async` + /// feature. + /// + /// `concurrency` bounds in-flight evaluations per batch. + pub async fn run_async

( + &mut self, + problem: &P, + concurrency: usize, + ) -> OptimizationResult + where + P: crate::core::async_problem::AsyncProblem, + I: Initializer, + V: Variation, + { + use crate::algorithms::parallel_eval_async::evaluate_batch_async; + + assert!( + self.config.population_size > 0, + "Hype population_size must be > 0" + ); + assert!(self.config.mc_samples > 0, "Hype mc_samples must be > 0"); + let n = self.config.population_size; + let objectives = problem.objectives(); + assert_eq!( + self.config.reference_point.len(), + objectives.len(), + "Hype reference_point.len() must equal number of objectives", + ); + let reference = self.config.reference_point.clone(); + let mut rng = rng_from_seed(self.config.seed); + + let initial_decisions = self.initializer.initialize(n, &mut rng); + let mut population: Vec> = + evaluate_batch_async(problem, initial_decisions, concurrency).await; + let mut evaluations = population.len(); + + for _ in 0..self.config.generations { + let fitness = hype_fitness( + &population, + &objectives, + &reference, + self.config.mc_samples, + &mut rng, + ); + let mut offspring_decisions: Vec = Vec::with_capacity(n); + while offspring_decisions.len() < n { + let p1 = binary_tournament(&fitness, &mut rng); + let p2 = binary_tournament(&fitness, &mut rng); + let parents = vec![ + population[p1].decision.clone(), + population[p2].decision.clone(), + ]; + let children = self.variation.vary(&parents, &mut rng); + assert!(!children.is_empty(), "Hype variation returned no children"); + for child in children { + if offspring_decisions.len() >= n { + break; + } + offspring_decisions.push(child); + } + } + + let offspring = evaluate_batch_async(problem, offspring_decisions, concurrency).await; + evaluations += offspring.len(); + + let mut combined: Vec> = Vec::with_capacity(2 * n); + combined.extend(population); + combined.extend(offspring); + + let fronts = non_dominated_sort(&combined, &objectives); + let mut keep_indices: Vec = Vec::with_capacity(n); + let mut splitting: &[usize] = &[]; + for f in &fronts { + if keep_indices.len() + f.len() <= n { + keep_indices.extend(f.iter().copied()); + } else { + splitting = f; + break; + } + if keep_indices.len() == n { + break; + } + } + if keep_indices.len() < n { + let pool: Vec<&Candidate> = + splitting.iter().map(|&i| &combined[i]).collect(); + let contributions = estimate_contributions( + &pool, + &objectives, + &reference, + self.config.mc_samples, + &mut rng, + ); + let mut order: Vec = (0..splitting.len()).collect(); + order.sort_by(|&a, &b| { + contributions[b] + .partial_cmp(&contributions[a]) + .unwrap_or(std::cmp::Ordering::Equal) + }); + for k in order.into_iter().take(n - keep_indices.len()) { + keep_indices.push(splitting[k]); + } + } + + population = keep_indices + .into_iter() + .map(|i| combined[i].clone()) + .collect(); + } + + let front = pareto_front(&population, &objectives); + let best = best_candidate(&population, &objectives); + OptimizationResult::new( + Population::new(population), + front, + best, + evaluations, + self.config.generations, + ) + } +} + fn hype_fitness( pool: &[Candidate], objectives: &ObjectiveSpace, diff --git a/src/algorithms/hyperband.rs b/src/algorithms/hyperband.rs index b602c49..046e6bf 100644 --- a/src/algorithms/hyperband.rs +++ b/src/algorithms/hyperband.rs @@ -206,6 +206,106 @@ where } } +#[cfg(feature = "async")] +impl Hyperband +where + D: Clone, + I: Initializer, +{ + /// Async version of [`Hyperband::run`] — evaluates each + /// Successive-Halving rung's configurations concurrently through the + /// caller's async runtime. Available only with the `async` feature. + /// + /// `concurrency` bounds in-flight evaluations per rung. + pub async fn run_async

(&mut self, problem: &P, concurrency: usize) -> OptimizationResult + where + P: crate::core::async_problem::AsyncPartialProblem, + D: Send + Sync, + { + use crate::algorithms::parallel_eval_async::evaluate_batch_at_budget_async; + + assert!( + self.config.max_budget > 0.0, + "Hyperband max_budget must be > 0" + ); + assert!(self.config.eta > 1.0, "Hyperband eta must be > 1"); + assert!( + self.config.max_brackets >= 1, + "Hyperband max_brackets must be >= 1" + ); + let objectives = problem.objectives(); + assert!( + objectives.is_single_objective(), + "Hyperband requires exactly one objective", + ); + let direction = objectives.objectives[0].direction; + let mut rng = rng_from_seed(self.config.seed); + + let s_max = (self.config.max_budget.ln() / self.config.eta.ln()).floor() as i64; + let s_max = (s_max as usize).min(self.config.max_brackets); + + let mut total_evaluations = 0usize; + let mut total_iterations = 0usize; + let mut best_seen: Option> = None; + + for s in (0..=s_max).rev() { + let s_f = s as f64; + let n = + ((s_max as f64 + 1.0) / (s_f + 1.0) * self.config.eta.powf(s_f)).ceil() as usize; + let r = self.config.max_budget / self.config.eta.powf(s_f); + + let mut configs: Vec = self.initializer.initialize(n, &mut rng); + for i in 0..=s { + let n_i = (n as f64 / self.config.eta.powi(i as i32)).floor() as usize; + let r_i = r * self.config.eta.powi(i as i32); + if configs.is_empty() { + break; + } + let evals: Vec = + evaluate_batch_at_budget_async(problem, &configs, r_i, concurrency).await; + total_evaluations += configs.len(); + + for (cfg, e) in configs.iter().zip(evals.iter()) { + let beats = match &best_seen { + None => true, + Some(b) => better(e, &b.evaluation, direction), + }; + if beats { + best_seen = Some(Candidate::new(cfg.clone(), e.clone())); + } + } + total_iterations += 1; + + let next_size = (n_i / self.config.eta as usize).max(1); + if next_size >= configs.len() { + continue; + } + let mut order: Vec = (0..configs.len()).collect(); + order.sort_by(|&a, &b| compare(&evals[a], &evals[b], direction)); + let keep: std::collections::HashSet = + order.into_iter().take(next_size).collect(); + let new_configs: Vec = configs + .into_iter() + .enumerate() + .filter_map(|(idx, c)| if keep.contains(&idx) { Some(c) } else { None }) + .collect(); + configs = new_configs; + } + } + + let best = best_seen.expect("at least one bracket ran"); + let population = Population::new(vec![best.clone()]); + let front = vec![best.clone()]; + OptimizationResult::new( + population, + front, + Some(best), + total_evaluations, + total_iterations, + ) + } +} + fn compare(a: &Evaluation, b: &Evaluation, direction: Direction) -> std::cmp::Ordering { match (a.is_feasible(), b.is_feasible()) { (true, false) => std::cmp::Ordering::Less, diff --git a/src/algorithms/ibea.rs b/src/algorithms/ibea.rs index 2fea13d..ff9d072 100644 --- a/src/algorithms/ibea.rs +++ b/src/algorithms/ibea.rs @@ -159,6 +159,80 @@ where } } +#[cfg(feature = "async")] +impl Ibea { + /// Async version of [`Optimizer::run`] — drives evaluations through + /// the user-chosen async runtime. Available only with the `async` + /// feature. + /// + /// `concurrency` bounds in-flight evaluations per batch. + pub async fn run_async

( + &mut self, + problem: &P, + concurrency: usize, + ) -> OptimizationResult + where + P: crate::core::async_problem::AsyncProblem, + I: Initializer, + V: Variation, + { + use crate::algorithms::parallel_eval_async::evaluate_batch_async; + + assert!( + self.config.population_size > 0, + "Ibea population_size must be > 0" + ); + assert!(self.config.kappa > 0.0, "Ibea kappa must be > 0"); + let n = self.config.population_size; + let objectives = problem.objectives(); + let mut rng = rng_from_seed(self.config.seed); + + let initial_decisions = self.initializer.initialize(n, &mut rng); + let mut population: Vec> = + evaluate_batch_async(problem, initial_decisions, concurrency).await; + let mut evaluations = population.len(); + + for _ in 0..self.config.generations { + let fitness = compute_fitness(&population, &objectives, self.config.kappa); + let mut offspring_decisions: Vec = Vec::with_capacity(n); + while offspring_decisions.len() < n { + let p1 = binary_tournament(&fitness, &mut rng); + let p2 = binary_tournament(&fitness, &mut rng); + let parents = vec![ + population[p1].decision.clone(), + population[p2].decision.clone(), + ]; + let children = self.variation.vary(&parents, &mut rng); + assert!(!children.is_empty(), "Ibea variation returned no children"); + for child in children { + if offspring_decisions.len() >= n { + break; + } + offspring_decisions.push(child); + } + } + + let offspring = evaluate_batch_async(problem, offspring_decisions, concurrency).await; + evaluations += offspring.len(); + + let mut combined: Vec> = Vec::with_capacity(2 * n); + combined.extend(population); + combined.extend(offspring); + population = environmental_selection(combined, &objectives, n, self.config.kappa); + } + + let front = pareto_front(&population, &objectives); + let best = best_candidate(&population, &objectives); + OptimizationResult::new( + Population::new(population), + front, + best, + evaluations, + self.config.generations, + ) + } +} + /// Iteratively remove the worst-fitness member from `pool` until `n` remain. /// /// IBEA's standard "subtract the dropped member's contribution from every diff --git a/src/algorithms/ipop_cma_es.rs b/src/algorithms/ipop_cma_es.rs index aeaf14c..5ee8842 100644 --- a/src/algorithms/ipop_cma_es.rs +++ b/src/algorithms/ipop_cma_es.rs @@ -187,6 +187,94 @@ where } } +#[cfg(feature = "async")] +impl IpopCmaEs { + /// Async version of [`Optimizer::run`] — drives evaluations through + /// the user-chosen async runtime. Available only with the `async` + /// feature. + /// + /// `concurrency` bounds in-flight evaluations within each restart's + /// CMA-ES generation. + pub async fn run_async

( + &mut self, + problem: &P, + concurrency: usize, + ) -> OptimizationResult> + where + P: crate::core::async_problem::AsyncProblem>, + { + assert!( + self.config.initial_population_size >= 4, + "IpopCmaEs initial_population_size must be >= 4", + ); + let objectives = problem.objectives(); + assert!( + objectives.is_single_objective(), + "IpopCmaEs requires exactly one objective", + ); + let direction = objectives.objectives[0].direction; + let mut rng = rng_from_seed(self.config.seed); + + let mut remaining_gens = self.config.total_generations; + let mut pop_size = self.config.initial_population_size; + let mut total_evaluations = 0usize; + let mut total_iterations = 0usize; + let mut best_seen: Option>> = None; + let _ = self.config.stall_generations; + + let mut restart_counter = 0u64; + while remaining_gens > 0 { + let this_gens = (remaining_gens / 2).max(20).min(remaining_gens); + let inner_seed = self + .config + .seed + .wrapping_add(restart_counter.wrapping_mul(0x9E37_79B9_7F4A_7C15)); + let restart_mean: Vec = self + .bounds + .bounds + .iter() + .map(|&(lo, hi)| lo + (hi - lo) * rng.random::()) + .collect(); + let cfg = CmaEsConfig { + population_size: pop_size, + generations: this_gens, + initial_sigma: self.config.initial_sigma, + eigen_decomposition_period: self.config.eigen_decomposition_period, + initial_mean: Some(restart_mean), + seed: inner_seed, + }; + let mut inner = CmaEs::new(cfg, RealBounds::new(self.bounds.bounds.clone())); + + let result = inner.run_async(problem, concurrency).await; + total_evaluations += result.evaluations; + total_iterations += result.generations; + if let Some(b) = result.best.clone() { + let beats = match &best_seen { + None => true, + Some(prev) => better(&b.evaluation, &prev.evaluation, direction), + }; + if beats { + best_seen = Some(b); + } + } + remaining_gens = remaining_gens.saturating_sub(this_gens); + pop_size = pop_size.saturating_mul(2); + restart_counter = restart_counter.wrapping_add(1); + } + + let best = best_seen.expect("at least one restart ran"); + let population = Population::new(vec![best.clone()]); + let front = vec![best.clone()]; + OptimizationResult::new( + population, + front, + Some(best), + total_evaluations, + total_iterations, + ) + } +} + fn better(a: &Evaluation, b: &Evaluation, direction: Direction) -> bool { match (a.is_feasible(), b.is_feasible()) { (true, false) => true, diff --git a/src/algorithms/knea.rs b/src/algorithms/knea.rs index 0fc50d1..0905d03 100644 --- a/src/algorithms/knea.rs +++ b/src/algorithms/knea.rs @@ -149,6 +149,77 @@ where } } +#[cfg(feature = "async")] +impl Knea { + /// Async version of [`Optimizer::run`] — drives evaluations through + /// the user-chosen async runtime. Available only with the `async` + /// feature. + /// + /// `concurrency` bounds in-flight evaluations per batch. + pub async fn run_async

( + &mut self, + problem: &P, + concurrency: usize, + ) -> OptimizationResult + where + P: crate::core::async_problem::AsyncProblem, + I: Initializer, + V: Variation, + { + use crate::algorithms::parallel_eval_async::evaluate_batch_async; + + assert!( + self.config.population_size > 0, + "Knea population_size must be > 0" + ); + let n = self.config.population_size; + let objectives = problem.objectives(); + let mut rng = rng_from_seed(self.config.seed); + + let initial_decisions = self.initializer.initialize(n, &mut rng); + let mut population: Vec> = + evaluate_batch_async(problem, initial_decisions, concurrency).await; + let mut evaluations = population.len(); + + for _ in 0..self.config.generations { + let mut offspring_decisions: Vec = Vec::with_capacity(n); + while offspring_decisions.len() < n { + let p1 = rng.random_range(0..population.len()); + let p2 = rng.random_range(0..population.len()); + let parents = vec![ + population[p1].decision.clone(), + population[p2].decision.clone(), + ]; + let children = self.variation.vary(&parents, &mut rng); + assert!(!children.is_empty(), "Knea variation returned no children"); + for child in children { + if offspring_decisions.len() >= n { + break; + } + offspring_decisions.push(child); + } + } + let offspring = evaluate_batch_async(problem, offspring_decisions, concurrency).await; + evaluations += offspring.len(); + + let mut combined: Vec> = Vec::with_capacity(2 * n); + combined.extend(population); + combined.extend(offspring); + population = environmental_selection(combined, &objectives, n); + } + + let front = pareto_front(&population, &objectives); + let best = best_candidate(&population, &objectives); + OptimizationResult::new( + Population::new(population), + front, + best, + evaluations, + self.config.generations, + ) + } +} + fn environmental_selection( combined: Vec>, objectives: &ObjectiveSpace, diff --git a/src/algorithms/moead.rs b/src/algorithms/moead.rs index 44359f8..fd2b68c 100644 --- a/src/algorithms/moead.rs +++ b/src/algorithms/moead.rs @@ -219,6 +219,126 @@ where } } +#[cfg(feature = "async")] +impl Moead { + /// Async version of [`Optimizer::run`] — drives evaluations through + /// the user-chosen async runtime. Available only with the `async` + /// feature. + /// + /// `concurrency` bounds in-flight evaluations of the initial + /// population. Per-generation evaluations are sequential because + /// each child's outcome feeds back into the same generation's + /// neighborhood updates. + pub async fn run_async

( + &mut self, + problem: &P, + concurrency: usize, + ) -> OptimizationResult + where + P: crate::core::async_problem::AsyncProblem, + I: Initializer, + V: Variation, + { + use crate::algorithms::parallel_eval_async::evaluate_batch_async; + + let objectives = problem.objectives(); + let m = objectives.len(); + let weights = das_dennis(m, self.config.reference_divisions); + assert!( + !weights.is_empty(), + "Moead weight set is empty — increase reference_divisions", + ); + let n = weights.len(); + let t = self.config.neighborhood_size.min(n); + assert!(t >= 2, "Moead neighborhood_size must be >= 2"); + + let mut rng = rng_from_seed(self.config.seed); + + let initial_decisions = self.initializer.initialize(n, &mut rng); + assert_eq!( + initial_decisions.len(), + n, + "MOEA/D initializer must return exactly {n} decisions", + ); + let mut population: Vec> = + evaluate_batch_async(problem, initial_decisions, concurrency).await; + let mut evaluations = population.len(); + + let mut ideal = vec![f64::INFINITY; m]; + for c in &population { + let oriented = objectives.as_minimization(&c.evaluation.objectives); + for (k, v) in oriented.iter().enumerate() { + if *v < ideal[k] { + ideal[k] = *v; + } + } + } + + let neighborhoods: Vec> = (0..n) + .map(|i| { + let mut idx: Vec = (0..n).collect(); + idx.sort_by(|&a, &b| { + let da = weight_distance(&weights[i], &weights[a]); + let db = weight_distance(&weights[i], &weights[b]); + da.partial_cmp(&db).unwrap_or(std::cmp::Ordering::Equal) + }); + idx.into_iter().take(t).collect() + }) + .collect(); + + for _ in 0..self.config.generations { + #[allow(clippy::needless_range_loop)] + for i in 0..n { + let nbh = &neighborhoods[i]; + let p1 = *nbh.choose(&mut rng).unwrap(); + let mut p2 = *nbh.choose(&mut rng).unwrap(); + while p2 == p1 && nbh.len() > 1 { + p2 = *nbh.choose(&mut rng).unwrap(); + } + let parents = vec![ + population[p1].decision.clone(), + population[p2].decision.clone(), + ]; + let children = self.variation.vary(&parents, &mut rng); + assert!( + !children.is_empty(), + "MOEA/D variation returned no children" + ); + let child_decision = children.into_iter().next().unwrap(); + let child_eval = problem.evaluate_async(&child_decision).await; + evaluations += 1; + + let oriented_child = objectives.as_minimization(&child_eval.objectives); + for (k, v) in oriented_child.iter().enumerate() { + if *v < ideal[k] { + ideal[k] = *v; + } + } + + for &j in nbh { + let cur_oriented = + objectives.as_minimization(&population[j].evaluation.objectives); + let g_cur = tchebycheff(&cur_oriented, &weights[j], &ideal); + let g_new = tchebycheff(&oriented_child, &weights[j], &ideal); + if g_new <= g_cur { + population[j] = Candidate::new(child_decision.clone(), child_eval.clone()); + } + } + } + } + + let front = pareto_front(&population, &objectives); + let best = best_candidate(&population, &objectives); + OptimizationResult::new( + Population::new(population), + front, + best, + evaluations, + self.config.generations, + ) + } +} + /// Tchebycheff scalarization: `max_k w_k * |f_k - z*_k|`. /// /// `weight` components that are zero are floored to `1e-6` so every axis diff --git a/src/algorithms/mopso.rs b/src/algorithms/mopso.rs index 2af313c..d046317 100644 --- a/src/algorithms/mopso.rs +++ b/src/algorithms/mopso.rs @@ -212,6 +212,124 @@ where } } +#[cfg(feature = "async")] +impl Mopso { + /// Async version of [`Optimizer::run`] — drives evaluations through + /// the user-chosen async runtime. Available only with the `async` + /// feature. + /// + /// `concurrency` bounds in-flight evaluations per batch. + pub async fn run_async

( + &mut self, + problem: &P, + concurrency: usize, + ) -> OptimizationResult> + where + P: crate::core::async_problem::AsyncProblem>, + { + use crate::algorithms::parallel_eval_async::evaluate_batch_async; + use crate::traits::Initializer as _; + + assert!(self.config.swarm_size >= 1, "Mopso swarm_size must be >= 1"); + assert!( + self.config.archive_size >= 1, + "Mopso archive_size must be >= 1" + ); + let objectives = problem.objectives(); + assert!( + objectives.is_multi_objective(), + "Mopso requires multi-objective problems (use ParticleSwarm for single-objective)", + ); + let dim = self.bounds.bounds.len(); + let n = self.config.swarm_size; + let mut rng = rng_from_seed(self.config.seed); + + let mut positions: Vec> = self.bounds.initialize(n, &mut rng); + let mut velocities: Vec> = (0..n) + .map(|_| { + self.bounds + .bounds + .iter() + .map(|&(lo, hi)| 0.1 * (hi - lo) * (rng.random::() * 2.0 - 1.0)) + .collect() + }) + .collect(); + let v_max: Vec = self.bounds.bounds.iter().map(|&(lo, hi)| hi - lo).collect(); + + let initial_pop = evaluate_batch_async(problem, positions.clone(), concurrency).await; + let mut evaluations = initial_pop.len(); + + let mut pbest_decisions: Vec> = positions.clone(); + let mut pbest_evals: Vec = + initial_pop.iter().map(|c| c.evaluation.clone()).collect(); + + let mut archive = ParetoArchive::new(objectives.clone()); + for c in initial_pop { + archive.insert(c); + } + archive.truncate(self.config.archive_size); + + for _ in 0..self.config.generations { + for i in 0..n { + let leader = archive + .members() + .choose(&mut rng) + .map(|c| c.decision.clone()) + .unwrap_or_else(|| positions[i].clone()); + #[allow(clippy::needless_range_loop)] + for j in 0..dim { + let r1: f64 = rng.random(); + let r2: f64 = rng.random(); + let cognitive_term = + self.config.cognitive * r1 * (pbest_decisions[i][j] - positions[i][j]); + let social_term = self.config.social * r2 * (leader[j] - positions[i][j]); + let mut v = + self.config.inertia * velocities[i][j] + cognitive_term + social_term; + if v > v_max[j] { + v = v_max[j]; + } else if v < -v_max[j] { + v = -v_max[j]; + } + velocities[i][j] = v; + let (lo, hi) = self.bounds.bounds[j]; + positions[i][j] = (positions[i][j] + v).clamp(lo, hi); + } + } + + let evaluated = evaluate_batch_async(problem, positions.clone(), concurrency).await; + evaluations += evaluated.len(); + + for (i, cand) in evaluated.iter().enumerate() { + let dominance = pareto_compare(&cand.evaluation, &pbest_evals[i], &objectives); + let replace = match dominance { + Dominance::Dominates => true, + Dominance::DominatedBy => false, + Dominance::Equal | Dominance::NonDominated => rng.random_bool(0.5), + }; + if replace { + pbest_decisions[i] = cand.decision.clone(); + pbest_evals[i] = cand.evaluation.clone(); + } + } + for c in evaluated { + archive.insert(c); + } + archive.truncate(self.config.archive_size); + } + + let members = archive.into_vec(); + let front = pareto_front(&members, &objectives); + let best = best_candidate(&members, &objectives); + OptimizationResult::new( + Population::new(members), + front, + best, + evaluations, + self.config.generations, + ) + } +} + #[cfg(test)] mod tests { use super::*; diff --git a/src/algorithms/nelder_mead.rs b/src/algorithms/nelder_mead.rs index 219759c..9224ada 100644 --- a/src/algorithms/nelder_mead.rs +++ b/src/algorithms/nelder_mead.rs @@ -295,6 +295,161 @@ fn better(a: &Evaluation, b: &Evaluation, direction: Direction) -> bool { compare(a, b, direction) == std::cmp::Ordering::Less } +#[cfg(feature = "async")] +impl NelderMead { + /// Async version of [`Optimizer::run`] — drives evaluations through + /// the user-chosen async runtime. Available only with the `async` + /// feature. + /// + /// `concurrency` is largely inert here because Nelder-Mead + /// evaluates one or two new vertices per iteration sequentially + /// (the next decision depends on the previous evaluation); it's + /// accepted for API parity with other algorithms. + pub async fn run_async

( + &mut self, + problem: &P, + concurrency: usize, + ) -> OptimizationResult> + where + P: crate::core::async_problem::AsyncProblem>, + { + let _ = concurrency; + assert!( + self.config.reflection > 0.0, + "NelderMead reflection must be > 0" + ); + assert!( + self.config.expansion > 1.0, + "NelderMead expansion must be > 1", + ); + assert!( + self.config.contraction > 0.0 && self.config.contraction < 1.0, + "NelderMead contraction must be in (0, 1)", + ); + assert!( + self.config.shrinkage > 0.0 && self.config.shrinkage < 1.0, + "NelderMead shrinkage must be in (0, 1)", + ); + assert!( + self.config.initial_step > 0.0, + "NelderMead initial_step must be > 0", + ); + let objectives = problem.objectives(); + assert!( + objectives.is_single_objective(), + "NelderMead requires exactly one objective", + ); + let direction = objectives.objectives[0].direction; + let n = self.bounds.bounds.len(); + + let mut vertices: Vec> = Vec::with_capacity(n + 1); + let start: Vec = self + .bounds + .bounds + .iter() + .map(|&(lo, hi)| 0.5 * (lo + hi)) + .collect(); + vertices.push(start.clone()); + for j in 0..n { + let mut v = start.clone(); + let (lo, hi) = self.bounds.bounds[j]; + let step = self.config.initial_step.min(0.5 * (hi - lo)); + v[j] = (v[j] + step).clamp(lo, hi); + vertices.push(v); + } + let mut evals: Vec = Vec::with_capacity(vertices.len()); + for v in &vertices { + evals.push(problem.evaluate_async(v).await); + } + let mut evaluations = evals.len(); + + for _ in 0..self.config.iterations { + let mut order: Vec = (0..vertices.len()).collect(); + order.sort_by(|&a, &b| compare(&evals[a], &evals[b], direction)); + let best_idx = order[0]; + let worst_idx = order[order.len() - 1]; + let second_worst_idx = order[order.len() - 2]; + + let mut centroid = vec![0.0_f64; n]; + for &idx in &order[..order.len() - 1] { + for j in 0..n { + centroid[j] += vertices[idx][j]; + } + } + for c in centroid.iter_mut() { + *c /= (order.len() - 1) as f64; + } + + let reflected = self.reflect(¢roid, &vertices[worst_idx], self.config.reflection); + let r_eval = problem.evaluate_async(&reflected).await; + evaluations += 1; + + if better(&r_eval, &evals[best_idx], direction) { + let expanded = self.reflect(¢roid, &vertices[worst_idx], self.config.expansion); + let e_eval = problem.evaluate_async(&expanded).await; + evaluations += 1; + if better(&e_eval, &r_eval, direction) { + vertices[worst_idx] = expanded; + evals[worst_idx] = e_eval; + } else { + vertices[worst_idx] = reflected; + evals[worst_idx] = r_eval; + } + } else if better(&r_eval, &evals[second_worst_idx], direction) { + vertices[worst_idx] = reflected; + evals[worst_idx] = r_eval; + } else { + let contraction_target = if better(&r_eval, &evals[worst_idx], direction) { + self.contract(¢roid, &reflected, self.config.contraction) + } else { + self.contract(¢roid, &vertices[worst_idx], self.config.contraction) + }; + let c_eval = problem.evaluate_async(&contraction_target).await; + evaluations += 1; + if better(&c_eval, &evals[worst_idx], direction) { + vertices[worst_idx] = contraction_target; + evals[worst_idx] = c_eval; + } else { + let best_pt = vertices[best_idx].clone(); + for &idx in &order { + if idx == best_idx { + continue; + } + #[allow(clippy::needless_range_loop)] + for j in 0..n { + vertices[idx][j] = best_pt[j] + + self.config.shrinkage * (vertices[idx][j] - best_pt[j]); + } + for (j, x) in vertices[idx].iter_mut().enumerate() { + let (lo, hi) = self.bounds.bounds[j]; + *x = x.clamp(lo, hi); + } + evals[idx] = problem.evaluate_async(&vertices[idx]).await; + evaluations += 1; + } + } + } + } + + let mut best_idx = 0; + for i in 1..vertices.len() { + if better(&evals[i], &evals[best_idx], direction) { + best_idx = i; + } + } + let best = Candidate::new(vertices[best_idx].clone(), evals[best_idx].clone()); + let population = Population::new(vec![best.clone()]); + let front = vec![best.clone()]; + OptimizationResult::new( + population, + front, + Some(best), + evaluations, + self.config.iterations, + ) + } +} + #[cfg(test)] mod tests { use super::*; diff --git a/src/algorithms/nsga2.rs b/src/algorithms/nsga2.rs index 5652636..71df459 100644 --- a/src/algorithms/nsga2.rs +++ b/src/algorithms/nsga2.rs @@ -232,6 +232,117 @@ fn annotate( .collect() } +#[cfg(feature = "async")] +impl Nsga2 { + /// Async version of [`Optimizer::run`] — drives evaluations through + /// the user-chosen async runtime. Available only with the `async` + /// feature. + /// + /// `concurrency` bounds in-flight evaluations per batch (initial + /// population and per-generation offspring). + pub async fn run_async

( + &mut self, + problem: &P, + concurrency: usize, + ) -> OptimizationResult + where + P: crate::core::async_problem::AsyncProblem, + I: Initializer, + V: Variation, + { + use crate::algorithms::parallel_eval_async::evaluate_batch_async; + + assert!( + self.config.population_size > 0, + "Nsga2 population_size must be greater than 0", + ); + let n = self.config.population_size; + let objectives = problem.objectives(); + let mut rng = rng_from_seed(self.config.seed); + + let initial_decisions = self.initializer.initialize(n, &mut rng); + assert_eq!( + initial_decisions.len(), + n, + "NSGA-II initializer must return exactly population_size decisions", + ); + let population: Vec> = + evaluate_batch_async(problem, initial_decisions, concurrency).await; + let mut evaluations = population.len(); + + let mut annotated = annotate(population, &objectives); + + for _ in 0..self.config.generations { + let mut offspring_decisions: Vec = Vec::with_capacity(n); + while offspring_decisions.len() < n { + let p1 = binary_tournament(&annotated, &mut rng); + let p2 = binary_tournament(&annotated, &mut rng); + let parents = vec![ + annotated[p1].candidate.decision.clone(), + annotated[p2].candidate.decision.clone(), + ]; + let children = self.variation.vary(&parents, &mut rng); + assert!( + !children.is_empty(), + "NSGA-II variation returned no children", + ); + for child_decision in children { + if offspring_decisions.len() >= n { + break; + } + offspring_decisions.push(child_decision); + } + } + let offspring: Vec> = + evaluate_batch_async(problem, offspring_decisions, concurrency).await; + evaluations += offspring.len(); + + let mut combined: Vec> = Vec::with_capacity(2 * n); + combined.extend(annotated.into_iter().map(|e| e.candidate)); + combined.extend(offspring); + + let fronts = non_dominated_sort(&combined, &objectives); + let mut next: Vec> = Vec::with_capacity(n); + for front in &fronts { + if next.len() + front.len() <= n { + for &idx in front { + next.push(combined[idx].clone()); + } + } else { + let dist = crowding_distance(&combined, front, &objectives); + let mut order: Vec = (0..front.len()).collect(); + order.sort_by(|&a, &b| { + dist[b] + .partial_cmp(&dist[a]) + .unwrap_or(std::cmp::Ordering::Equal) + }); + let needed = n - next.len(); + for &k in order.iter().take(needed) { + next.push(combined[front[k]].clone()); + } + break; + } + if next.len() == n { + break; + } + } + annotated = annotate(next, &objectives); + } + + let final_pop: Vec> = + annotated.into_iter().map(|e| e.candidate).collect(); + let front = pareto_front(&final_pop, &objectives); + let best = best_candidate(&final_pop, &objectives); + OptimizationResult::new( + Population::new(final_pop), + front, + best, + evaluations, + self.config.generations, + ) + } +} + fn binary_tournament(entries: &[Nsga2Entry], rng: &mut Rng) -> usize { let n = entries.len(); let a = rng.random_range(0..n); diff --git a/src/algorithms/nsga3.rs b/src/algorithms/nsga3.rs index 80ab2c0..c6f23e7 100644 --- a/src/algorithms/nsga3.rs +++ b/src/algorithms/nsga3.rs @@ -180,6 +180,92 @@ where } } +#[cfg(feature = "async")] +impl Nsga3 { + /// Async version of [`Optimizer::run`] — drives evaluations through + /// the user-chosen async runtime. Available only with the `async` + /// feature. + /// + /// `concurrency` bounds in-flight evaluations per batch. + pub async fn run_async

( + &mut self, + problem: &P, + concurrency: usize, + ) -> OptimizationResult + where + P: crate::core::async_problem::AsyncProblem, + I: Initializer, + V: Variation, + { + use crate::algorithms::parallel_eval_async::evaluate_batch_async; + + assert!( + self.config.population_size > 0, + "Nsga3 population_size must be greater than 0", + ); + let n = self.config.population_size; + let objectives = problem.objectives(); + let m = objectives.len(); + let reference_points = das_dennis(m, self.config.reference_divisions); + assert!( + !reference_points.is_empty(), + "Nsga3 reference set is empty — check reference_divisions", + ); + let mut rng = rng_from_seed(self.config.seed); + + let initial_decisions = self.initializer.initialize(n, &mut rng); + assert_eq!( + initial_decisions.len(), + n, + "NSGA-III initializer must return exactly population_size decisions", + ); + let mut population: Vec> = + evaluate_batch_async(problem, initial_decisions, concurrency).await; + let mut evaluations = population.len(); + + for _ in 0..self.config.generations { + let mut offspring_decisions: Vec = Vec::with_capacity(n); + while offspring_decisions.len() < n { + let p1 = rng.random_range(0..population.len()); + let p2 = rng.random_range(0..population.len()); + let parents = vec![ + population[p1].decision.clone(), + population[p2].decision.clone(), + ]; + let children = self.variation.vary(&parents, &mut rng); + assert!( + !children.is_empty(), + "NSGA-III variation returned no children", + ); + for child_decision in children { + if offspring_decisions.len() >= n { + break; + } + offspring_decisions.push(child_decision); + } + } + let offspring = evaluate_batch_async(problem, offspring_decisions, concurrency).await; + evaluations += offspring.len(); + + let mut combined: Vec> = Vec::with_capacity(2 * n); + combined.extend(population); + combined.extend(offspring); + population = + environmental_selection(&combined, &objectives, &reference_points, n, &mut rng); + } + + let front = pareto_front(&population, &objectives); + let best = best_candidate(&population, &objectives); + OptimizationResult::new( + Population::new(population), + front, + best, + evaluations, + self.config.generations, + ) + } +} + /// NSGA-III environmental selection: front-by-front + reference-point niching /// on the splitting front. fn environmental_selection( diff --git a/src/algorithms/one_plus_one_es.rs b/src/algorithms/one_plus_one_es.rs index a73517d..420bc45 100644 --- a/src/algorithms/one_plus_one_es.rs +++ b/src/algorithms/one_plus_one_es.rs @@ -191,6 +191,100 @@ fn worse_than(a: &Evaluation, b: &Evaluation, direction: Direction) -> bool { } } +#[cfg(feature = "async")] +impl OnePlusOneEs { + /// Async version of [`Optimizer::run`] — drives evaluations through + /// the user-chosen async runtime. Available only with the `async` + /// feature. + /// + /// `concurrency` is mostly inert here because (1+1)-ES evaluates + /// one child per iteration; it's accepted for API parity with + /// other algorithms. + pub async fn run_async

( + &mut self, + problem: &P, + concurrency: usize, + ) -> OptimizationResult> + where + P: crate::core::async_problem::AsyncProblem>, + { + let _ = concurrency; + assert!( + self.config.initial_sigma > 0.0, + "OnePlusOneEs initial_sigma must be > 0" + ); + assert!( + self.config.step_increase > 1.0, + "OnePlusOneEs step_increase must be > 1", + ); + assert!( + self.config.adaptation_period >= 1, + "OnePlusOneEs adaptation_period must be >= 1", + ); + let objectives = problem.objectives(); + assert!( + objectives.is_single_objective(), + "OnePlusOneEs requires exactly one objective", + ); + let direction = objectives.objectives[0].direction; + let mut rng = rng_from_seed(self.config.seed); + + let mut parent: Vec = self + .bounds + .bounds + .iter() + .map(|&(lo, hi)| 0.5 * (lo + hi)) + .collect(); + let mut parent_eval = problem.evaluate_async(&parent).await; + let mut evaluations = 1usize; + + let mut sigma = self.config.initial_sigma; + let mut window = std::collections::VecDeque::with_capacity(self.config.adaptation_period); + + for _ in 0..self.config.iterations { + let normal = Normal::new(0.0, sigma).expect("Normal::new(0, sigma)"); + let mut child = parent.clone(); + for (j, x) in child.iter_mut().enumerate() { + let (lo, hi) = self.bounds.bounds[j]; + *x = (*x + normal.sample(&mut rng)).clamp(lo, hi); + } + let child_eval = problem.evaluate_async(&child).await; + evaluations += 1; + + let accepted = !worse_than(&child_eval, &parent_eval, direction); + if accepted { + parent = child; + parent_eval = child_eval; + } + + window.push_back(if accepted { 1u8 } else { 0u8 }); + if window.len() > self.config.adaptation_period { + window.pop_front(); + } + if window.len() == self.config.adaptation_period { + let success_count: usize = window.iter().map(|&b| b as usize).sum(); + let rate = success_count as f64 / window.len() as f64; + if rate > 0.2 { + sigma *= self.config.step_increase; + } else if rate < 0.2 { + sigma /= self.config.step_increase; + } + } + } + + let best = Candidate::new(parent, parent_eval); + let population = Population::new(vec![best.clone()]); + let front = vec![best.clone()]; + OptimizationResult::new( + population, + front, + Some(best), + evaluations, + self.config.iterations, + ) + } +} + #[cfg(test)] mod tests { use super::*; diff --git a/src/algorithms/paes.rs b/src/algorithms/paes.rs index c619519..b77ee36 100644 --- a/src/algorithms/paes.rs +++ b/src/algorithms/paes.rs @@ -156,6 +156,92 @@ where } } +#[cfg(feature = "async")] +impl Paes { + /// Async version of [`Optimizer::run`] — drives evaluations through + /// the user-chosen async runtime. Available only with the `async` + /// feature. + /// + /// `concurrency` is mostly inert here because PAES evaluates one + /// child per iteration; it's accepted for API parity with other + /// algorithms. + pub async fn run_async

( + &mut self, + problem: &P, + concurrency: usize, + ) -> OptimizationResult + where + P: crate::core::async_problem::AsyncProblem, + I: Initializer, + V: Variation, + { + let _ = concurrency; + assert!( + self.config.archive_size > 0, + "PAES archive_size must be greater than 0", + ); + + let objectives = problem.objectives(); + let mut rng = rng_from_seed(self.config.seed); + + let mut initial = self.initializer.initialize(1, &mut rng); + assert!( + !initial.is_empty(), + "PAES initializer returned no decisions", + ); + let mut current_decision = initial.remove(0); + let mut current_eval = problem.evaluate_async(¤t_decision).await; + let mut evaluations = 1usize; + + let mut archive = ParetoArchive::new(objectives.clone()); + archive.insert(Candidate::new( + current_decision.clone(), + current_eval.clone(), + )); + + for _ in 0..self.config.iterations { + let parents = vec![current_decision.clone()]; + let children = self.variation.vary(&parents, &mut rng); + assert!(!children.is_empty(), "PAES variation returned no children",); + let child_decision = children.into_iter().next().unwrap(); + let child_eval = problem.evaluate_async(&child_decision).await; + evaluations += 1; + + match pareto_compare(&child_eval, ¤t_eval, &objectives) { + Dominance::Dominates => { + current_decision = child_decision.clone(); + current_eval = child_eval.clone(); + } + Dominance::DominatedBy => { + // Stay at current. + } + Dominance::NonDominated | Dominance::Equal => { + current_decision = child_decision.clone(); + current_eval = child_eval.clone(); + } + } + + archive.insert(Candidate::new(child_decision, child_eval)); + archive.insert(Candidate::new( + current_decision.clone(), + current_eval.clone(), + )); + archive.truncate(self.config.archive_size); + } + + let members = archive.into_vec(); + let front = pareto_front(&members, &objectives); + let best = best_candidate(&members, &objectives); + OptimizationResult::new( + Population::new(members), + front, + best, + evaluations, + self.config.iterations, + ) + } +} + #[cfg(test)] mod tests { use super::*; diff --git a/src/algorithms/parallel_eval_async.rs b/src/algorithms/parallel_eval_async.rs index fdd6ca4..7a1fc57 100644 --- a/src/algorithms/parallel_eval_async.rs +++ b/src/algorithms/parallel_eval_async.rs @@ -5,8 +5,9 @@ use futures::stream::{FuturesOrdered, StreamExt}; -use crate::core::async_problem::AsyncProblem; +use crate::core::async_problem::{AsyncPartialProblem, AsyncProblem}; use crate::core::candidate::Candidate; +use crate::core::evaluation::Evaluation; /// Evaluate every decision concurrently against `problem`, preserving /// input order in the returned vector. Concurrency is bounded by @@ -56,3 +57,35 @@ where } out } + +/// Evaluate every decision at the given `budget` concurrently against a +/// multi-fidelity `problem`, preserving input order. Hyperband's async +/// path uses this for each Successive-Halving rung. +pub async fn evaluate_batch_at_budget_async

( + problem: &P, + decisions: &[P::Decision], + budget: f64, + concurrency: usize, +) -> Vec +where + P: AsyncPartialProblem, +{ + assert!( + concurrency >= 1, + "evaluate_batch_at_budget_async concurrency must be >= 1" + ); + let mut out: Vec = Vec::with_capacity(decisions.len()); + let mut idx = 0usize; + while idx < decisions.len() { + let mut futs = FuturesOrdered::new(); + let end = (idx + concurrency).min(decisions.len()); + for d in &decisions[idx..end] { + futs.push_back(async move { problem.evaluate_at_budget_async(d, budget).await }); + } + while let Some(e) = futs.next().await { + out.push(e); + } + idx = end; + } + out +} diff --git a/src/algorithms/particle_swarm.rs b/src/algorithms/particle_swarm.rs index b1157bc..a35ecbf 100644 --- a/src/algorithms/particle_swarm.rs +++ b/src/algorithms/particle_swarm.rs @@ -220,6 +220,129 @@ where } } +#[cfg(feature = "async")] +impl ParticleSwarm { + /// Async version of [`Optimizer::run`] — drives evaluations through + /// the user-chosen async runtime. Available only with the `async` + /// feature. + /// + /// `concurrency` bounds in-flight evaluations per batch (initial + /// swarm, per-generation positions, and the final evaluation pass). + pub async fn run_async

( + &mut self, + problem: &P, + concurrency: usize, + ) -> OptimizationResult> + where + P: crate::core::async_problem::AsyncProblem>, + { + use crate::algorithms::parallel_eval_async::evaluate_batch_async; + + assert!( + self.config.swarm_size >= 1, + "ParticleSwarm swarm_size must be >= 1", + ); + let objectives = problem.objectives(); + assert!( + objectives.is_single_objective(), + "ParticleSwarm requires exactly one objective", + ); + let direction = objectives.objectives[0].direction; + let dim = self.bounds.bounds.len(); + let n = self.config.swarm_size; + let mut rng = rng_from_seed(self.config.seed); + + let mut positions: Vec> = { + use crate::traits::Initializer as _; + self.bounds.initialize(n, &mut rng) + }; + let mut velocities: Vec> = (0..n) + .map(|_| { + self.bounds + .bounds + .iter() + .map(|&(lo, hi)| 0.1 * (hi - lo) * (rng.random::() * 2.0 - 1.0)) + .collect() + }) + .collect(); + let v_max: Vec = self.bounds.bounds.iter().map(|&(lo, hi)| hi - lo).collect(); + + let initial_pop = evaluate_batch_async(problem, positions.clone(), concurrency).await; + let mut evaluations = initial_pop.len(); + + let mut pbest_decisions: Vec> = positions.clone(); + let mut pbest_evals: Vec = initial_pop + .iter() + .map(|c| c.evaluation.objectives[0]) + .collect(); + + let mut gbest_idx = best_index(&pbest_evals, direction); + let mut gbest_decision = pbest_decisions[gbest_idx].clone(); + let mut gbest_eval = pbest_evals[gbest_idx]; + + for _ in 0..self.config.generations { + for i in 0..n { + #[allow(clippy::needless_range_loop)] + for j in 0..dim { + let r1: f64 = rng.random(); + let r2: f64 = rng.random(); + let cognitive_term = + self.config.cognitive * r1 * (pbest_decisions[i][j] - positions[i][j]); + let social_term = + self.config.social * r2 * (gbest_decision[j] - positions[i][j]); + let mut v = + self.config.inertia * velocities[i][j] + cognitive_term + social_term; + if v > v_max[j] { + v = v_max[j]; + } else if v < -v_max[j] { + v = -v_max[j]; + } + velocities[i][j] = v; + let (lo, hi) = self.bounds.bounds[j]; + positions[i][j] = (positions[i][j] + v).clamp(lo, hi); + } + } + + let evaluated = evaluate_batch_async(problem, positions.clone(), concurrency).await; + evaluations += evaluated.len(); + + for (i, cand) in evaluated.iter().enumerate() { + let f = cand.evaluation.objectives[0]; + let improves = match direction { + Direction::Minimize => f < pbest_evals[i], + Direction::Maximize => f > pbest_evals[i], + }; + if improves { + pbest_decisions[i] = positions[i].clone(); + pbest_evals[i] = f; + gbest_idx = i; + let beats_global = match direction { + Direction::Minimize => f < gbest_eval, + Direction::Maximize => f > gbest_eval, + }; + if beats_global { + gbest_decision = pbest_decisions[i].clone(); + gbest_eval = f; + } + } + } + } + let _ = gbest_idx; + + let final_pop = evaluate_batch_async(problem, positions, concurrency).await; + evaluations += final_pop.len(); + let best = best_candidate(&final_pop, &objectives); + let front: Vec>> = best.iter().cloned().collect(); + OptimizationResult::new( + Population::new(final_pop), + front, + best, + evaluations, + self.config.generations, + ) + } +} + fn best_index(values: &[f64], direction: Direction) -> usize { let mut idx = 0; for i in 1..values.len() { diff --git a/src/algorithms/pesa2.rs b/src/algorithms/pesa2.rs index d409f5c..51357b9 100644 --- a/src/algorithms/pesa2.rs +++ b/src/algorithms/pesa2.rs @@ -204,6 +204,109 @@ where } } +#[cfg(feature = "async")] +impl PesaII { + /// Async version of [`Optimizer::run`] — drives evaluations through + /// the user-chosen async runtime. Available only with the `async` + /// feature. + /// + /// `concurrency` bounds in-flight evaluations of the initial + /// population. Per-step evaluations are sequential to preserve the + /// algorithm's exact RNG sequencing. + pub async fn run_async

( + &mut self, + problem: &P, + concurrency: usize, + ) -> OptimizationResult + where + P: crate::core::async_problem::AsyncProblem, + I: Initializer, + V: Variation, + { + use crate::algorithms::parallel_eval_async::evaluate_batch_async; + + assert!( + self.config.population_size > 0, + "PesaII population_size must be > 0" + ); + assert!( + self.config.archive_size > 0, + "PesaII archive_size must be > 0" + ); + assert!( + self.config.grid_divisions >= 1, + "PesaII grid_divisions must be >= 1" + ); + let n = self.config.population_size; + let objectives = problem.objectives(); + let mut rng = rng_from_seed(self.config.seed); + + let initial_decisions = self.initializer.initialize(n, &mut rng); + let mut internal: Vec> = + evaluate_batch_async(problem, initial_decisions, concurrency).await; + let mut evaluations = internal.len(); + + let mut archive = ParetoArchive::new(objectives.clone()); + for c in &internal { + archive.insert(c.clone()); + } + truncate_by_grid( + &mut archive, + self.config.archive_size, + self.config.grid_divisions, + ); + + for _ in 0..self.config.generations { + let (boxes, counts) = build_grid(&archive, &objectives, self.config.grid_divisions); + + let mut offspring: Vec> = Vec::with_capacity(n); + while offspring.len() < n { + let p1 = region_tournament(&archive, &boxes, &counts, &mut rng); + let p2 = region_tournament(&archive, &boxes, &counts, &mut rng); + let parents = vec![ + archive.members()[p1].decision.clone(), + archive.members()[p2].decision.clone(), + ]; + let children = self.variation.vary(&parents, &mut rng); + assert!( + !children.is_empty(), + "PesaII variation returned no children" + ); + for child in children { + if offspring.len() >= n { + break; + } + let eval = problem.evaluate_async(&child).await; + evaluations += 1; + offspring.push(Candidate::new(child, eval)); + } + } + + for c in &offspring { + archive.insert(c.clone()); + } + truncate_by_grid( + &mut archive, + self.config.archive_size, + self.config.grid_divisions, + ); + internal = offspring; + } + + let _ = internal; + let members = archive.into_vec(); + let front = pareto_front(&members, &objectives); + let best = best_candidate(&members, &objectives); + OptimizationResult::new( + Population::new(members), + front, + best, + evaluations, + self.config.generations, + ) + } +} + /// Compute per-member box index (M-tuple of grid coordinates) and the /// population count of each occupied box. fn build_grid( diff --git a/src/algorithms/rvea.rs b/src/algorithms/rvea.rs index 7a27b11..354618c 100644 --- a/src/algorithms/rvea.rs +++ b/src/algorithms/rvea.rs @@ -261,6 +261,163 @@ where } } +#[cfg(feature = "async")] +impl Rvea { + /// Async version of [`Optimizer::run`] — drives evaluations through + /// the user-chosen async runtime. Available only with the `async` + /// feature. + /// + /// `concurrency` bounds in-flight evaluations per batch. + pub async fn run_async

( + &mut self, + problem: &P, + concurrency: usize, + ) -> OptimizationResult + where + P: crate::core::async_problem::AsyncProblem, + I: Initializer, + V: Variation, + { + use crate::algorithms::parallel_eval_async::evaluate_batch_async; + + assert!( + self.config.population_size > 0, + "Rvea population_size must be > 0" + ); + let n = self.config.population_size; + let objectives = problem.objectives(); + let m = objectives.len(); + let raw_refs = das_dennis(m, self.config.reference_divisions); + let references: Vec> = raw_refs.into_iter().map(unit_normalize).collect(); + assert!( + !references.is_empty(), + "Rvea: no reference vectors generated" + ); + + let theta_max = smallest_neighbor_angle(&references); + let mut rng = rng_from_seed(self.config.seed); + + let initial_decisions = self.initializer.initialize(n, &mut rng); + let mut population: Vec> = + evaluate_batch_async(problem, initial_decisions, concurrency).await; + let mut evaluations = population.len(); + + for gen_idx in 0..self.config.generations { + let mut offspring_decisions: Vec = Vec::with_capacity(n); + while offspring_decisions.len() < n { + let p1 = rng.random_range(0..population.len()); + let p2 = rng.random_range(0..population.len()); + let parents = vec![ + population[p1].decision.clone(), + population[p2].decision.clone(), + ]; + let children = self.variation.vary(&parents, &mut rng); + assert!(!children.is_empty(), "Rvea variation returned no children"); + for child in children { + if offspring_decisions.len() >= n { + break; + } + offspring_decisions.push(child); + } + } + let offspring = evaluate_batch_async(problem, offspring_decisions, concurrency).await; + evaluations += offspring.len(); + + let mut combined: Vec> = Vec::with_capacity(2 * n); + combined.extend(population); + combined.extend(offspring); + + let m_dim = m; + let mut ideal = vec![f64::INFINITY; m_dim]; + for c in &combined { + let oriented = objectives.as_minimization(&c.evaluation.objectives); + for (k, v) in oriented.iter().enumerate() { + if *v < ideal[k] { + ideal[k] = *v; + } + } + } + let translated: Vec> = combined + .iter() + .map(|c| { + let oriented = objectives.as_minimization(&c.evaluation.objectives); + oriented + .iter() + .enumerate() + .map(|(k, v)| v - ideal[k]) + .collect() + }) + .collect(); + + let mut assoc: Vec = vec![0; combined.len()]; + let mut angles: Vec = vec![0.0; combined.len()]; + for (i, t) in translated.iter().enumerate() { + let (best_ref, best_angle) = closest_reference(t, &references); + assoc[i] = best_ref; + angles[i] = best_angle; + } + + let alpha_t = (gen_idx as f64 / (self.config.generations as f64).max(1.0)) + .powf(self.config.alpha); + let mut keep: Vec> = vec![None; references.len()]; + for i in 0..combined.len() { + let r = assoc[i]; + let length: f64 = translated[i].iter().map(|v| v * v).sum::().sqrt(); + let theta_max_safe = theta_max.max(1e-12); + let penalty = 1.0 + (m_dim as f64) * alpha_t * (angles[i] / theta_max_safe); + let apd = penalty * length; + match keep[r] { + None => keep[r] = Some((i, apd)), + Some((_, current)) if apd < current => keep[r] = Some((i, apd)), + _ => {} + } + } + + let mut next: Vec> = keep + .into_iter() + .flatten() + .map(|(i, _)| combined[i].clone()) + .collect(); + if next.len() < n { + let mut all_apds: Vec<(usize, f64)> = (0..combined.len()) + .map(|i| { + let length: f64 = translated[i].iter().map(|v| v * v).sum::().sqrt(); + let theta_max_safe = theta_max.max(1e-12); + let penalty = 1.0 + (m_dim as f64) * alpha_t * (angles[i] / theta_max_safe); + (i, penalty * length) + }) + .collect(); + all_apds.sort_by(|a, b| a.1.partial_cmp(&b.1).unwrap_or(std::cmp::Ordering::Equal)); + for (i, _) in all_apds { + if next.len() >= n { + break; + } + if !next + .iter() + .any(|c| std::ptr::eq(c as *const _, &combined[i] as *const _)) + { + next.push(combined[i].clone()); + } + } + } + if next.len() > n { + next.truncate(n); + } + population = next; + } + + let front = pareto_front(&population, &objectives); + let best = best_candidate(&population, &objectives); + OptimizationResult::new( + Population::new(population), + front, + best, + evaluations, + self.config.generations, + ) + } +} + fn unit_normalize(mut v: Vec) -> Vec { let n: f64 = v.iter().map(|x| x * x).sum::().sqrt(); if n > 1e-12 { diff --git a/src/algorithms/simulated_annealing.rs b/src/algorithms/simulated_annealing.rs index 40fc180..b4ba88f 100644 --- a/src/algorithms/simulated_annealing.rs +++ b/src/algorithms/simulated_annealing.rs @@ -215,6 +215,124 @@ fn better_than( } } +#[cfg(feature = "async")] +impl SimulatedAnnealing { + /// Async version of [`Optimizer::run`] — drives evaluations through + /// the user-chosen async runtime. Available only with the `async` + /// feature. + /// + /// `concurrency` is mostly inert here because SA evaluates one + /// child per iteration; it's accepted for API parity with other + /// algorithms. + pub async fn run_async

( + &mut self, + problem: &P, + concurrency: usize, + ) -> OptimizationResult + where + P: crate::core::async_problem::AsyncProblem, + I: Initializer, + V: Variation, + { + let _ = concurrency; + let objectives = problem.objectives(); + assert!( + objectives.is_single_objective(), + "SimulatedAnnealing requires exactly one objective", + ); + assert!( + self.config.initial_temperature > 0.0, + "SimulatedAnnealing initial_temperature must be positive", + ); + assert!( + self.config.final_temperature > 0.0, + "SimulatedAnnealing final_temperature must be positive", + ); + assert!( + self.config.final_temperature <= self.config.initial_temperature, + "SimulatedAnnealing final_temperature must be <= initial_temperature", + ); + let direction = objectives.objectives[0].direction; + let mut rng = rng_from_seed(self.config.seed); + + let mut initial = self.initializer.initialize(1, &mut rng); + assert!( + !initial.is_empty(), + "SimulatedAnnealing initializer returned no decisions", + ); + let mut current_decision = initial.remove(0); + let mut current_eval = problem.evaluate_async(¤t_decision).await; + let mut best_decision = current_decision.clone(); + let mut best_eval = current_eval.clone(); + let mut evaluations = 1usize; + + let cooling = if self.config.iterations <= 1 { + 1.0 + } else { + (self.config.final_temperature / self.config.initial_temperature) + .powf(1.0 / (self.config.iterations as f64 - 1.0)) + }; + let mut temperature = self.config.initial_temperature; + + for _ in 0..self.config.iterations { + let parents = vec![current_decision.clone()]; + let children = self.variation.vary(&parents, &mut rng); + assert!( + !children.is_empty(), + "SimulatedAnnealing variation returned no children" + ); + let child_decision = children.into_iter().next().unwrap(); + let child_eval = problem.evaluate_async(&child_decision).await; + evaluations += 1; + + let accept = match (child_eval.is_feasible(), current_eval.is_feasible()) { + (true, false) => true, + (false, true) => false, + (false, false) => { + child_eval.constraint_violation <= current_eval.constraint_violation + } + (true, true) => { + let delta = match direction { + Direction::Minimize => { + child_eval.objectives[0] - current_eval.objectives[0] + } + Direction::Maximize => { + current_eval.objectives[0] - child_eval.objectives[0] + } + }; + if delta <= 0.0 { + true + } else { + let prob = (-delta / temperature).exp(); + rng.random::() < prob + } + } + }; + + if accept { + current_decision = child_decision; + current_eval = child_eval; + if better_than(¤t_eval, &best_eval, direction) { + best_decision = current_decision.clone(); + best_eval = current_eval.clone(); + } + } + temperature *= cooling; + } + + let best = Candidate::new(best_decision, best_eval); + let population = Population::new(vec![best.clone()]); + let front = vec![best.clone()]; + OptimizationResult::new( + population, + front, + Some(best), + evaluations, + self.config.iterations, + ) + } +} + #[cfg(test)] mod tests { use super::*; diff --git a/src/algorithms/sms_emoa.rs b/src/algorithms/sms_emoa.rs index 90b3a6d..d0f0ef9 100644 --- a/src/algorithms/sms_emoa.rs +++ b/src/algorithms/sms_emoa.rs @@ -173,6 +173,80 @@ where } } +#[cfg(feature = "async")] +impl SmsEmoa { + /// Async version of [`Optimizer::run`] — drives evaluations through + /// the user-chosen async runtime. Available only with the `async` + /// feature. + /// + /// `concurrency` bounds in-flight evaluations of the initial + /// population. Per-generation evaluations are sequential because + /// SMS-EMOA is a steady-state algorithm (one child per generation). + pub async fn run_async

( + &mut self, + problem: &P, + concurrency: usize, + ) -> OptimizationResult + where + P: crate::core::async_problem::AsyncProblem, + I: Initializer, + V: Variation, + { + use crate::algorithms::parallel_eval_async::evaluate_batch_async; + + assert!( + self.config.population_size > 0, + "SmsEmoa population_size must be > 0" + ); + let n = self.config.population_size; + let objectives = problem.objectives(); + assert_eq!( + self.config.reference_point.len(), + objectives.len(), + "SmsEmoa reference_point.len() must equal number of objectives", + ); + let reference = self.config.reference_point.clone(); + let mut rng = rng_from_seed(self.config.seed); + + let initial_decisions = self.initializer.initialize(n, &mut rng); + let mut population: Vec> = + evaluate_batch_async(problem, initial_decisions, concurrency).await; + let mut evaluations = population.len(); + + for _ in 0..self.config.generations { + let p1 = rng.random_range(0..population.len()); + let p2 = rng.random_range(0..population.len()); + let parents = vec![ + population[p1].decision.clone(), + population[p2].decision.clone(), + ]; + let children = self.variation.vary(&parents, &mut rng); + assert!( + !children.is_empty(), + "SmsEmoa variation returned no children" + ); + let child_decision = children.into_iter().next().unwrap(); + let child_eval = problem.evaluate_async(&child_decision).await; + evaluations += 1; + let child = Candidate::new(child_decision, child_eval); + + population.push(child); + let drop_idx = pick_drop_index(&population, &objectives, &reference); + population.swap_remove(drop_idx); + } + + let front = pareto_front(&population, &objectives); + let best = best_candidate(&population, &objectives); + OptimizationResult::new( + Population::new(population), + front, + best, + evaluations, + self.config.generations, + ) + } +} + /// Choose the index in `pool` whose removal is preferred per SMS-EMOA's /// rules: drop from the worst non-dominated front; within that front, /// drop the member whose removal increases hypervolume the most (= the diff --git a/src/algorithms/snes.rs b/src/algorithms/snes.rs index 8365f0c..22c9ec7 100644 --- a/src/algorithms/snes.rs +++ b/src/algorithms/snes.rs @@ -222,6 +222,137 @@ where } } +#[cfg(feature = "async")] +impl SeparableNes { + /// Async version of [`Optimizer::run`] — drives evaluations through + /// the user-chosen async runtime. Available only with the `async` + /// feature. + /// + /// `concurrency` bounds in-flight evaluations per generation. + pub async fn run_async

( + &mut self, + problem: &P, + concurrency: usize, + ) -> OptimizationResult> + where + P: crate::core::async_problem::AsyncProblem>, + { + use crate::algorithms::parallel_eval_async::evaluate_batch_async; + + assert!( + self.config.population_size >= 2, + "SeparableNes population_size must be >= 2", + ); + assert!( + self.config.initial_sigma > 0.0, + "SeparableNes initial_sigma must be > 0" + ); + let objectives = problem.objectives(); + assert!( + objectives.is_single_objective(), + "SeparableNes requires exactly one objective", + ); + let direction = objectives.objectives[0].direction; + let n = self.bounds.bounds.len(); + let lambda = self.config.population_size; + let mut rng = rng_from_seed(self.config.seed); + + let mut mean: Vec = self + .bounds + .bounds + .iter() + .map(|&(lo, hi)| 0.5 * (lo + hi)) + .collect(); + let mut sigma = vec![self.config.initial_sigma; n]; + + let eta_sigma = self + .config + .sigma_learning_rate + .unwrap_or_else(|| (3.0 + (n as f64).ln()) / (5.0 * (n as f64).sqrt())); + let eta_mean = self.config.mean_learning_rate; + + let utilities = nes_utilities(lambda); + + let mut best_seen: Option>> = None; + let mut total_evaluations = 0usize; + + for _ in 0..self.config.generations { + // Sample λ offspring; matches the sync RNG draw order so seeded + // runs reproduce exactly. + let mut z_samples: Vec> = Vec::with_capacity(lambda); + let mut x_samples: Vec> = Vec::with_capacity(lambda); + for _ in 0..lambda { + let z: Vec = (0..n) + .map(|_| Normal::new(0.0, 1.0).unwrap().sample(&mut rng)) + .collect(); + let x: Vec = (0..n) + .map(|j| { + let v = mean[j] + sigma[j] * z[j]; + let (lo, hi) = self.bounds.bounds[j]; + v.clamp(lo, hi) + }) + .collect(); + z_samples.push(z); + x_samples.push(x); + } + + let cands = evaluate_batch_async(problem, x_samples.clone(), concurrency).await; + total_evaluations += cands.len(); + let evals: Vec = cands.iter().map(|c| c.evaluation.clone()).collect(); + for c in &cands { + let beats_best = match &best_seen { + None => true, + Some(b) => better(&c.evaluation, &b.evaluation, direction), + }; + if beats_best { + best_seen = Some(c.clone()); + } + } + + let mut order: Vec = (0..lambda).collect(); + order.sort_by(|&a, &b| compare(&evals[a], &evals[b], direction)); + + let mut grad_mean = vec![0.0_f64; n]; + for k in 0..lambda { + let u = utilities[k]; + let z = &z_samples[order[k]]; + for j in 0..n { + grad_mean[j] += u * z[j]; + } + } + for j in 0..n { + mean[j] += eta_mean * sigma[j] * grad_mean[j]; + let (lo, hi) = self.bounds.bounds[j]; + mean[j] = mean[j].clamp(lo, hi); + } + + for j in 0..n { + let mut grad_sigma_j = 0.0; + for k in 0..lambda { + let u = utilities[k]; + let z = &z_samples[order[k]]; + grad_sigma_j += u * (z[j] * z[j] - 1.0); + } + sigma[j] *= (0.5 * eta_sigma * grad_sigma_j).exp(); + if !sigma[j].is_finite() || sigma[j] < 1e-30 { + sigma[j] = 1e-30; + } + } + } + + let best = best_seen.expect("at least one generation evaluated"); + let population = Population::new(vec![best.clone()]); + let front = vec![best.clone()]; + OptimizationResult::new( + population, + front, + Some(best), + total_evaluations, + self.config.generations, + ) + } +} + fn nes_utilities(lambda: usize) -> Vec { let half = lambda as f64 / 2.0 + 1.0; let raw: Vec = (0..lambda) diff --git a/src/algorithms/spea2.rs b/src/algorithms/spea2.rs index e26e943..0aff699 100644 --- a/src/algorithms/spea2.rs +++ b/src/algorithms/spea2.rs @@ -169,6 +169,91 @@ where } } +#[cfg(feature = "async")] +impl Spea2 { + /// Async version of [`Optimizer::run`] — drives evaluations through + /// the user-chosen async runtime. Available only with the `async` + /// feature. + /// + /// `concurrency` bounds in-flight evaluations per batch. + pub async fn run_async

( + &mut self, + problem: &P, + concurrency: usize, + ) -> OptimizationResult + where + P: crate::core::async_problem::AsyncProblem, + I: Initializer, + V: Variation, + { + use crate::algorithms::parallel_eval_async::evaluate_batch_async; + + assert!( + self.config.population_size > 0, + "Spea2 population_size must be greater than 0", + ); + assert!( + self.config.archive_size > 0, + "Spea2 archive_size must be greater than 0", + ); + let n_pop = self.config.population_size; + let n_arc = self.config.archive_size; + let objectives = problem.objectives(); + let mut rng = rng_from_seed(self.config.seed); + + let initial_decisions = self.initializer.initialize(n_pop, &mut rng); + assert_eq!( + initial_decisions.len(), + n_pop, + "SPEA2 initializer must return exactly population_size decisions", + ); + let mut population: Vec> = + evaluate_batch_async(problem, initial_decisions, concurrency).await; + let mut evaluations = population.len(); + let mut archive: Vec> = Vec::new(); + + for _ in 0..self.config.generations { + let mut pool: Vec> = + Vec::with_capacity(population.len() + archive.len()); + pool.append(&mut population); + pool.append(&mut archive); + let fitness = compute_fitness(&pool, &objectives); + + archive = build_archive(&pool, &fitness, &objectives, n_arc); + + let archive_fitness = compute_fitness(&archive, &objectives); + let mut offspring_decisions: Vec = Vec::with_capacity(n_pop); + while offspring_decisions.len() < n_pop { + let p1 = binary_tournament(&archive_fitness, &mut rng); + let p2 = binary_tournament(&archive_fitness, &mut rng); + let parents = vec![archive[p1].decision.clone(), archive[p2].decision.clone()]; + let children = self.variation.vary(&parents, &mut rng); + assert!(!children.is_empty(), "SPEA2 variation returned no children"); + for child_decision in children { + if offspring_decisions.len() >= n_pop { + break; + } + offspring_decisions.push(child_decision); + } + } + let new_population = + evaluate_batch_async(problem, offspring_decisions, concurrency).await; + evaluations += new_population.len(); + population = new_population; + } + + let front = pareto_front(&archive, &objectives); + let best = best_candidate(&archive, &objectives); + OptimizationResult::new( + Population::new(archive), + front, + best, + evaluations, + self.config.generations, + ) + } +} + /// SPEA2 fitness: `R(i) + D(i)`, where lower is better. /// /// `R(i)` is the sum of `S(j)` over all `j` that dominate `i`. `S(j)` is the diff --git a/src/algorithms/tabu_search.rs b/src/algorithms/tabu_search.rs index 4b2b968..41460ce 100644 --- a/src/algorithms/tabu_search.rs +++ b/src/algorithms/tabu_search.rs @@ -209,6 +209,128 @@ fn better_than( } } +#[cfg(feature = "async")] +impl TabuSearch +where + D: Clone + Hash + Eq, + I: Initializer, + N: FnMut(&D, &mut Rng) -> Vec, +{ + /// Async version of [`Optimizer::run`] — drives evaluations through + /// the user-chosen async runtime. Available only with the `async` + /// feature. + /// + /// Each iteration evaluates the K neighbors of the current + /// incumbent concurrently (bounded by `concurrency`), then picks + /// the best non-tabu (or aspiration-passing) move. + pub async fn run_async

(&mut self, problem: &P, concurrency: usize) -> OptimizationResult + where + P: crate::core::async_problem::AsyncProblem, + D: Send + Sync, + { + use crate::algorithms::parallel_eval_async::evaluate_batch_async; + + let objectives = problem.objectives(); + assert!( + objectives.is_single_objective(), + "TabuSearch requires exactly one objective", + ); + assert!( + self.config.tabu_tenure >= 1, + "TabuSearch tabu_tenure must be >= 1", + ); + let direction = objectives.objectives[0].direction; + let mut rng = rng_from_seed(self.config.seed); + + let mut initial = self.initializer.initialize(1, &mut rng); + assert!( + !initial.is_empty(), + "TabuSearch initializer returned no decisions" + ); + let mut current_decision = initial.remove(0); + let mut current_eval = problem.evaluate_async(¤t_decision).await; + let mut best_decision = current_decision.clone(); + let mut best_eval = current_eval.clone(); + let mut evaluations = 1usize; + + let mut tabu_queue: VecDeque = VecDeque::with_capacity(self.config.tabu_tenure); + let mut tabu_set: HashSet = HashSet::new(); + + for _ in 0..self.config.iterations { + let candidates = (self.neighbors)(¤t_decision, &mut rng); + if candidates.is_empty() { + break; + } + + let cand_results = evaluate_batch_async(problem, candidates.clone(), concurrency).await; + let mut cand_evals: Vec = + cand_results.into_iter().map(|c| c.evaluation).collect(); + evaluations += candidates.len(); + + let mut best_idx: Option = None; + let mut best_cand_eval: Option = None; + + for (i, c) in candidates.iter().enumerate() { + let is_tabu = tabu_set.contains(c); + let aspires = is_tabu && better_than(&cand_evals[i], &best_eval, direction); + if is_tabu && !aspires { + continue; + } + let eligible = match &best_cand_eval { + None => true, + Some(b) => better_than(&cand_evals[i], b, direction), + }; + if eligible { + best_idx = Some(i); + best_cand_eval = Some(cand_evals[i].clone()); + } + } + + if best_idx.is_none() { + for (i, _) in candidates.iter().enumerate() { + let eligible = match &best_cand_eval { + None => true, + Some(b) => better_than(&cand_evals[i], b, direction), + }; + if eligible { + best_idx = Some(i); + best_cand_eval = Some(cand_evals[i].clone()); + } + } + } + + let chosen_idx = best_idx.expect("non-empty candidate list"); + let chosen_decision = candidates[chosen_idx].clone(); + current_eval = cand_evals.remove(chosen_idx); + current_decision = chosen_decision.clone(); + + if better_than(¤t_eval, &best_eval, direction) { + best_decision = current_decision.clone(); + best_eval = current_eval.clone(); + } + + tabu_queue.push_back(chosen_decision.clone()); + tabu_set.insert(chosen_decision); + if tabu_queue.len() > self.config.tabu_tenure { + if let Some(old) = tabu_queue.pop_front() { + tabu_set.remove(&old); + } + } + } + + let best = Candidate::new(best_decision, best_eval); + let population = Population::new(vec![best.clone()]); + let front = vec![best.clone()]; + OptimizationResult::new( + population, + front, + Some(best), + evaluations, + self.config.iterations, + ) + } +} + #[cfg(test)] mod tests { use super::*; diff --git a/src/algorithms/tlbo.rs b/src/algorithms/tlbo.rs index ac3ea36..8d0f397 100644 --- a/src/algorithms/tlbo.rs +++ b/src/algorithms/tlbo.rs @@ -187,6 +187,122 @@ where } } +#[cfg(feature = "async")] +impl Tlbo { + /// Async version of [`Optimizer::run`] — drives evaluations through + /// the user-chosen async runtime. Available only with the `async` + /// feature. + /// + /// `concurrency` bounds in-flight evaluations within batched phases + /// (only the initial population uses a batch; the teacher and learner + /// phases evaluate sequentially because each accept/reject step + /// depends on the previous one). + pub async fn run_async

( + &mut self, + problem: &P, + concurrency: usize, + ) -> OptimizationResult> + where + P: crate::core::async_problem::AsyncProblem>, + { + use crate::algorithms::parallel_eval_async::evaluate_batch_async; + + assert!( + self.config.population_size >= 2, + "Tlbo population_size must be >= 2" + ); + let objectives = problem.objectives(); + assert!( + objectives.is_single_objective(), + "Tlbo requires exactly one objective", + ); + let direction = objectives.objectives[0].direction; + let dim = self.bounds.bounds.len(); + let n = self.config.population_size; + let mut rng = rng_from_seed(self.config.seed); + + let mut decisions: Vec> = { + use crate::traits::Initializer as _; + self.bounds.initialize(n, &mut rng) + }; + let initial = evaluate_batch_async(problem, decisions.clone(), concurrency).await; + let mut evals: Vec = initial.iter().map(|c| c.evaluation.clone()).collect(); + let mut evaluations = initial.len(); + + for _ in 0..self.config.generations { + let teacher_idx = best_index(&evals, direction); + let teacher = decisions[teacher_idx].clone(); + let mut mean = vec![0.0_f64; dim]; + for d in &decisions { + for j in 0..dim { + mean[j] += d[j]; + } + } + for v in mean.iter_mut() { + *v /= n as f64; + } + let tf = if rng.random_bool(0.5) { 1.0 } else { 2.0 }; + + for i in 0..n { + let mut candidate = decisions[i].clone(); + for j in 0..dim { + let r: f64 = rng.random(); + candidate[j] += r * (teacher[j] - tf * mean[j]); + let (lo, hi) = self.bounds.bounds[j]; + candidate[j] = candidate[j].clamp(lo, hi); + } + let cand_eval = problem.evaluate_async(&candidate).await; + evaluations += 1; + if better(&cand_eval, &evals[i], direction) { + decisions[i] = candidate; + evals[i] = cand_eval; + } + } + + for i in 0..n { + let mut k = rng.random_range(0..n); + while k == i && n > 1 { + k = rng.random_range(0..n); + } + let partner_better = better(&evals[k], &evals[i], direction); + let mut candidate = decisions[i].clone(); + for j in 0..dim { + let r: f64 = rng.random(); + let delta = if partner_better { + r * (decisions[k][j] - decisions[i][j]) + } else { + r * (decisions[i][j] - decisions[k][j]) + }; + candidate[j] += delta; + let (lo, hi) = self.bounds.bounds[j]; + candidate[j] = candidate[j].clamp(lo, hi); + } + let cand_eval = problem.evaluate_async(&candidate).await; + evaluations += 1; + if better(&cand_eval, &evals[i], direction) { + decisions[i] = candidate; + evals[i] = cand_eval; + } + } + } + + let final_pop: Vec>> = decisions + .into_iter() + .zip(evals) + .map(|(d, e)| Candidate::new(d, e)) + .collect(); + let best = best_candidate(&final_pop, &objectives); + let front: Vec>> = best.iter().cloned().collect(); + OptimizationResult::new( + Population::new(final_pop), + front, + best, + evaluations, + self.config.generations, + ) + } +} + fn best_index(evals: &[Evaluation], direction: Direction) -> usize { let mut idx = 0; for i in 1..evals.len() { diff --git a/src/algorithms/tpe.rs b/src/algorithms/tpe.rs index cfd026f..5978feb 100644 --- a/src/algorithms/tpe.rs +++ b/src/algorithms/tpe.rs @@ -356,6 +356,129 @@ fn scott_bandwidths(decisions: &[Vec], support: &[usize], factor: f64) -> V .collect() } +#[cfg(feature = "async")] +impl Tpe { + /// Async version of [`Optimizer::run`] — drives evaluations through + /// the user-chosen async runtime. Available only with the `async` + /// feature. + /// + /// `concurrency` bounds in-flight evaluations during the initial + /// uniform-sample design; the sequential TPE loop runs one + /// evaluation per iteration regardless. + pub async fn run_async

( + &mut self, + problem: &P, + concurrency: usize, + ) -> OptimizationResult> + where + P: crate::core::async_problem::AsyncProblem>, + { + use crate::algorithms::parallel_eval_async::evaluate_batch_async; + + assert!( + self.config.initial_samples >= 2, + "Tpe initial_samples must be >= 2" + ); + assert!( + self.config.good_fraction > 0.0 && self.config.good_fraction < 1.0, + "Tpe good_fraction must be in (0, 1)", + ); + assert!( + self.config.candidate_samples >= 1, + "Tpe candidate_samples must be >= 1", + ); + assert!( + self.config.bandwidth_factor > 0.0, + "Tpe bandwidth_factor must be > 0" + ); + let objectives = problem.objectives(); + assert!( + objectives.is_single_objective(), + "Tpe requires exactly one objective", + ); + let direction = objectives.objectives[0].direction; + let dim = self.bounds.bounds.len(); + let mut rng = rng_from_seed(self.config.seed); + + let mut decisions: Vec> = Vec::new(); + let mut targets: Vec = Vec::new(); + let mut evals: Vec = Vec::new(); + + let initial_decisions: Vec> = (0..self.config.initial_samples) + .map(|_| sample_uniform_in_bounds(&self.bounds, &mut rng)) + .collect(); + let initial_cands = evaluate_batch_async(problem, initial_decisions, concurrency).await; + for c in initial_cands { + targets.push(oriented_target(&c.evaluation, direction)); + decisions.push(c.decision); + evals.push(c.evaluation); + } + + for _ in 0..self.config.iterations { + let (good_idx, bad_idx) = split_good_bad(&targets, self.config.good_fraction); + + let mut best_x: Option> = None; + let mut best_ratio = f64::NEG_INFINITY; + for _ in 0..self.config.candidate_samples { + let cand = sample_from_kde( + &decisions, + &good_idx, + &self.bounds, + self.config.bandwidth_factor, + &mut rng, + ); + let l = log_kde_density( + &cand, + &decisions, + &good_idx, + &self.bounds, + self.config.bandwidth_factor, + ); + let g = log_kde_density( + &cand, + &decisions, + &bad_idx, + &self.bounds, + self.config.bandwidth_factor, + ); + let ratio = l - g; + if ratio > best_ratio { + best_ratio = ratio; + best_x = Some(cand); + } + } + let x = best_x.expect("at least one candidate sampled"); + let _ = dim; + let e = problem.evaluate_async(&x).await; + targets.push(oriented_target(&e, direction)); + decisions.push(x); + evals.push(e); + } + + let mut best_idx = 0; + for i in 1..evals.len() { + if better(&evals[i], &evals[best_idx], direction) { + best_idx = i; + } + } + let total_evals = evals.len(); + let final_pop: Vec>> = decisions + .into_iter() + .zip(evals) + .map(|(d, e)| Candidate::new(d, e)) + .collect(); + let best = final_pop[best_idx].clone(); + let front = vec![best.clone()]; + OptimizationResult::new( + Population::new(final_pop), + front, + Some(best), + total_evals, + self.config.iterations + self.config.initial_samples, + ) + } +} + #[cfg(test)] mod tests { use super::*; diff --git a/src/algorithms/umda.rs b/src/algorithms/umda.rs index fd8658a..979312b 100644 --- a/src/algorithms/umda.rs +++ b/src/algorithms/umda.rs @@ -202,6 +202,125 @@ where } } +#[cfg(feature = "async")] +impl Umda { + /// Async version of [`Optimizer::run`] — drives evaluations through + /// the user-chosen async runtime. Available only with the `async` + /// feature. + /// + /// `concurrency` bounds in-flight evaluations per batch (initial + /// population and per-generation samples). + pub async fn run_async

( + &mut self, + problem: &P, + concurrency: usize, + ) -> OptimizationResult> + where + P: crate::core::async_problem::AsyncProblem>, + { + use crate::algorithms::parallel_eval_async::evaluate_batch_async; + + assert!( + self.config.population_size >= 2, + "Umda population_size must be >= 2" + ); + assert!( + self.config.selected_size >= 1, + "Umda selected_size must be >= 1", + ); + assert!( + self.config.selected_size <= self.config.population_size, + "Umda selected_size must be <= population_size", + ); + assert!(self.config.bits >= 1, "Umda bits must be >= 1"); + let objectives = problem.objectives(); + assert!( + objectives.is_single_objective(), + "Umda requires exactly one objective", + ); + let direction = objectives.objectives[0].direction; + let n = self.config.population_size; + let bits = self.config.bits; + let mu = self.config.selected_size; + let mut rng = rng_from_seed(self.config.seed); + + let mut decisions: Vec> = (0..n) + .map(|_| (0..bits).map(|_| rng.random_bool(0.5)).collect()) + .collect(); + let mut population = evaluate_batch_async(problem, decisions.clone(), concurrency).await; + let mut evaluations = population.len(); + + let smoothing = 1.0 / (2.0 * mu as f64); + let prob_min = smoothing; + let prob_max = 1.0 - smoothing; + + let mut best_seen: Option>> = None; + for c in &population { + let beats = match &best_seen { + None => true, + Some(b) => better_than_so(&c.evaluation, &b.evaluation, direction), + }; + if beats { + best_seen = Some(c.clone()); + } + } + + for _ in 0..self.config.generations { + let mut order: Vec = (0..population.len()).collect(); + order.sort_by(|&a, &b| { + compare_so( + &population[a].evaluation, + &population[b].evaluation, + direction, + ) + }); + let selected: Vec<&Candidate>> = + order.iter().take(mu).map(|&i| &population[i]).collect(); + + let mut probs = vec![0.0_f64; bits]; + for c in &selected { + for (i, b) in c.decision.iter().enumerate() { + if *b { + probs[i] += 1.0; + } + } + } + for p in probs.iter_mut() { + *p = (*p / mu as f64).clamp(prob_min, prob_max); + } + + decisions = (0..n) + .map(|_| probs.iter().map(|&p| rng.random_bool(p)).collect()) + .collect(); + + population = evaluate_batch_async(problem, decisions.clone(), concurrency).await; + evaluations += population.len(); + + for c in &population { + let beats = match &best_seen { + None => true, + Some(b) => better_than_so(&c.evaluation, &b.evaluation, direction), + }; + if beats { + best_seen = Some(c.clone()); + } + } + } + + let best = best_seen.expect("at least one generation evaluated"); + let final_pop = vec![best.clone()]; + let front = vec![best.clone()]; + let best_opt = best_candidate(&final_pop, &objectives); + OptimizationResult::new( + Population::new(final_pop), + front, + best_opt, + evaluations, + self.config.generations, + ) + } +} + fn compare_so( a: &crate::core::evaluation::Evaluation, b: &crate::core::evaluation::Evaluation, diff --git a/src/core/async_problem.rs b/src/core/async_problem.rs index 72dc39c..d0984af 100644 --- a/src/core/async_problem.rs +++ b/src/core/async_problem.rs @@ -7,10 +7,12 @@ //! thread. //! //! [`AsyncProblem`] mirrors [`Problem`](crate::core::Problem) but its -//! `evaluate_async` returns a future. Algorithms that support async -//! evaluation (NSGA-II, DE, RandomSearch as of v0.7.0; others land -//! incrementally) expose a `run_async` method that drives evaluations -//! through a user-chosen async runtime (typically tokio). +//! `evaluate_async` returns a future. Every algorithm in heuropt exposes +//! a `run_async` method that drives evaluations through a user-chosen +//! async runtime (typically tokio). Hyperband uses +//! [`AsyncPartialProblem`] instead, which mirrors +//! [`PartialProblem`](crate::core::partial_problem::PartialProblem) for +//! multi-fidelity workloads. //! //! Available only with the `async` feature. @@ -52,3 +54,28 @@ pub trait AsyncProblem: Sync { /// invoked from. fn evaluate_async(&self, decision: &Self::Decision) -> impl Future + Send; } + +/// Async equivalent of [`PartialProblem`](crate::core::partial_problem::PartialProblem) +/// for multi-fidelity workloads — used by Hyperband's `run_async`. +/// +/// Like [`AsyncProblem`], `evaluate_at_budget_async` returns a future +/// so callers can fan out budgeted evaluations across an async runtime. +pub trait AsyncPartialProblem: Sync { + /// The thing the optimizer changes. Same constraints as + /// [`PartialProblem::Decision`](crate::core::partial_problem::PartialProblem::Decision). + type Decision: Clone + Send + Sync; + + /// Return the objectives for this problem. + fn objectives(&self) -> ObjectiveSpace; + + /// Evaluate `decision` at the given fidelity `budget` asynchronously. + /// + /// Same monotonicity contract as + /// [`PartialProblem::evaluate_at_budget`](crate::core::partial_problem::PartialProblem::evaluate_at_budget): + /// higher budget should give a more accurate estimate. + fn evaluate_at_budget_async( + &self, + decision: &Self::Decision, + budget: f64, + ) -> impl Future + Send; +}