feat(traits,algorithms): add PartialProblem trait and Hyperband
Multi-fidelity optimization. Hyperband (Li et al. 2017) and its
foundation Successive Halving (Karnin et al. 2013) tune
hyperparameters by allocating *uneven* compute across configurations:
sample many cheap-to-evaluate-at-low-budget configs, then promote
the survivors to higher budgets. Crucial for ML hyperparameter
tuning where each evaluation is a partial training run.
This requires a new trait — `Problem::evaluate` is a single-shot
black box, but Hyperband needs to evaluate the SAME decision at
different fidelity budgets:
pub trait PartialProblem {
type Decision: Clone;
fn objectives(&self) -> ObjectiveSpace;
fn evaluate_at_budget(&self, decision: &Self::Decision,
budget: f64) -> Evaluation;
}
`PartialProblem` is intentionally NOT a sub-trait of `Problem`.
Implementors who already have a `Problem` and want their
`evaluate_at_budget` to ignore budget can write a one-line wrapper.
`Hyperband` is the optimizer:
pub struct HyperbandConfig {
max_budget: f64, eta: f64, max_brackets: usize, seed: u64,
}
pub struct Hyperband<I> { config, initializer, ... }
Single-objective only. The decision sampler is an `Initializer<D>` so
it works the same way as every other heuropt algorithm. Generic over
decision type.
This commit is contained in:
+3
-2
@@ -6,7 +6,7 @@
|
||||
|
||||
pub use crate::core::{
|
||||
Candidate, Direction, Evaluation, Objective, ObjectiveSpace, OptimizationResult,
|
||||
Population, Problem, Rng, rng_from_seed,
|
||||
PartialProblem, Population, Problem, Rng, rng_from_seed,
|
||||
};
|
||||
|
||||
pub use crate::traits::{Initializer, Optimizer, Repair, Variation};
|
||||
@@ -27,7 +27,8 @@ pub use crate::algorithms::{
|
||||
BayesianOptConfig, CmaEs, CmaEsConfig, DifferentialEvolution,
|
||||
DifferentialEvolutionConfig, EpsilonMoea, EpsilonMoeaConfig,
|
||||
GeneticAlgorithm, GeneticAlgorithmConfig, Grea, GreaConfig, HillClimber, HillClimberConfig, Hype,
|
||||
HypeConfig, Ibea, IbeaConfig, IpopCmaEs, IpopCmaEsConfig, Knea, KneaConfig, Moead, MoeadConfig, Mopso, MopsoConfig,
|
||||
HypeConfig, Hyperband, HyperbandConfig, Ibea, IbeaConfig, IpopCmaEs, IpopCmaEsConfig,
|
||||
Knea, KneaConfig, Moead, MoeadConfig, Mopso, MopsoConfig,
|
||||
NelderMead, NelderMeadConfig, Nsga2,
|
||||
Nsga2Config, Nsga3, Nsga3Config, OnePlusOneEs, OnePlusOneEsConfig, Paes, PaesConfig, ParticleSwarm, PesaII, PesaIIConfig,
|
||||
ParticleSwarmConfig, RandomSearch, RandomSearchConfig, Rvea, RveaConfig,
|
||||
|
||||
Reference in New Issue
Block a user