Adds an a-posteriori decision step to the jiggly example. After NSGA-III
produces the Pareto front, we apply a weighted-sum score over each
objective normalized to [0, 1] across the front (best→1, worst→0,
direction-aware), and report the top three plus a clear recommendation.
The weights are stated explicitly with rationale, not buried in code:
work_fail 45% — screen sleeping mid-meeting is the worst failure
lunch_sleep 30% — the actual design goal
presses 15% — UX friction the user feels
after_hours 10% — minor, mostly screen burn
This is the standard structure for picking a single answer out of a
Pareto set without losing the front itself: someone with different
weights can read the front and pick differently, but we surface a
specific recommendation with reasoning rather than leaving the user
to stare at 84 incomparable rows. Identical normalization could be
swapped for TOPSIS or knee-point detection later if useful.
Port of `scripts/tune_runtime.py` from ~/Code/jiggly: optimize the four
lifecycle constants of a USB mouse-jiggler firmware so the screen sleeps
during the user's lunch hour rather than failing during work.
The Python script grid-searches against a single composite score that
linearly combines several genuinely conflicting goals — a workaround
for the fact that grid search needs one number to rank by. heuropt has
the actual right tool, so this example is structured as a 4-objective
NSGA-III run that surfaces the Pareto front of legitimate tradeoffs:
1. minimize work-time failures (mean_work_sleep)
2. maximize lunch sleep (mean_lunch)
3. minimize human button presses (mean_presses)
4. minimize after-hours waste (mean_after)
Decision: 4-element `Vec<f64>` for (RT, YA, RA, FRA), continuous-relaxed
and rounded to integer minutes inside `evaluate`. The firmware ordering
constraint YA > RA > FRA > 0 is encoded as `constraint_violation` so
heuropt's feasible-beats-infeasible logic handles it for free.
Solver: NSGA-III with M=4, H=6 → 84 reference points, matching the
population size. Each evaluate runs a 1,000-workday Monte Carlo, so
the example is also a deliberately meaty evaluator that benefits from
`--features parallel`.
Output is in jiggly's native units — RT as Xh00m, thresholds in plain
minutes, sleep durations as Xh00m / Mm, probabilities as percentages —
and contrasts the Pareto front against:
- the four extreme single-axis winners (most lunch / fewest work fails /
fewest presses / least after-hours)
- the firmware's currently-shipping defaults (which sit inside the
front as a balanced compromise)
Two new runners — `zdt1_moead` and `dtlz2_moead` — using the same
SBX + PolyMut variation as the other Pareto-based methods. Reference
divisions chosen so the implied population size is comparable to the
other algorithms in each section (99 → 100 weights for ZDT1; 12 → 91
weights for DTLZ2).
Implementation of Zhang & Li 2007 MOEA/D — the canonical
decomposition-based MOEA. Different paradigm from Pareto-dominance
algorithms: each subproblem is a scalarized single-objective problem
defined by a Das–Dennis weight vector, and subproblems with similar
weight vectors form neighborhoods that share genetic material.
Each generation iterates over every weight vector `i`:
1. Pick two parents uniformly from the T-nearest neighbors of weight i
(T = neighborhood_size).
2. Apply variation, evaluate the child.
3. Update the ideal point z* with the child's objectives.
4. Walk the entire neighborhood: for each j, if the child's
Tchebycheff value g(child | w_j, z*) <= g(current[j] | w_j, z*),
replace current[j] with the child.
Tchebycheff scalarization:
g(f | w, z*) = max_k w_k · |f_k - z*_k|
(With the standard `w_k = 1e-6` floor when a weight is zero, so the
max well-defined.)
Public API:
MoeadConfig {
generations,
reference_divisions, // Das-Dennis H, also fixes population size
neighborhood_size, // T
seed,
}
Moead { config, initializer, variation }
impl<P, I, V> Optimizer<P> for Moead<I, V>
Population size equals the number of weight vectors generated by
das_dennis(num_objectives, reference_divisions). Re-exported from the
prelude. Tests cover non-empty Pareto front, deterministic reruns,
and panic on `reference_divisions` that would yield zero weights.
- nsga3: drop redundant `.into_iter()` in extend call; use
`#[allow(clippy::needless_range_loop)]` on the back-substitution
loop where `j` indexes into the matrix; remove an unneeded
`return` keyword in a closure.
- spea2: switch `pool.extend(x.drain(..))` to `pool.append(&mut x)`.
- examples/compare.rs DTLZ2 evaluator: same `needless_range_loop`
silencer on the inner cosine product loop.
NSGA-III's value over NSGA-II shows up at 3+ objectives, where
crowding distance loses its diversity signal. Adds a third comparison
section to `examples/compare.rs`:
DTLZ2 (3-objective, 12-D, the textbook benchmark for many-objective
algorithms): unit-sphere-octant Pareto front. Compares RandomSearch,
NSGA-II, SPEA2, and NSGA-III on:
- mean distance from front points to the unit sphere
(closed-form: |1 - sqrt(f1² + f2² + f3²)|),
- spacing,
- front size,
- wall-clock ms.
NSGA-III config: H=12 reference divisions (91 reference points,
matching the canonical setup from Deb & Jain 2014).
Also wires NSGA-III into the existing ZDT1 (2-objective) section even
though it's not its sweet spot — useful as a regression check that the
algorithm at least keeps up with NSGA-II on bi-objective problems.
Implementation of Deb & Jain 2014 NSGA-III — the canonical
many-objective MOEA. Replaces NSGA-II's crowding-distance niching
with a structured reference-point niching procedure that scales to
3+ objectives where crowding distance loses its diversity signal.
Each generation:
1. Random parent selection + variation + offspring evaluation, same as
NSGA-II.
2. Combine + non_dominated_sort, fill the next population front-by-
front until the next front would overflow (the splitting front F_l).
3. Survival on F_l uses reference-point niching:
- Translate by the ideal point z* (per-axis min in oriented space).
- Compute extreme points by ASF and intercepts; normalize by
intercepts (with a robust fallback to per-axis range if extreme
points are degenerate).
- Associate every member of the working pool with the closest
reference direction by perpendicular distance.
- Iteratively pick from F_l: prefer the niche with the smallest
count among references that have F_l candidates; if the niche is
empty in the already-selected set, take the closest associated
member by perpendicular distance, otherwise pick uniformly from
the niche.
Public API:
Nsga3Config { population_size, generations, reference_divisions, seed }
Nsga3 { config, initializer, variation }
impl<P, I, V> Optimizer<P> for Nsga3<I, V>
Re-exported from the prelude. Tests cover non-empty Pareto front,
exact final population size, deterministic reruns, and panic on
`population_size == 0`. Uses the existing tests_support problems.
The standard structured weight/reference vector generator for
many-objective MOEAs (NSGA-III, MOEA/D). Generates (H+M-1 choose M-1)
points uniformly distributed on the unit simplex by enumerating all
integer compositions of `divisions` into `num_objectives` parts and
dividing each by `divisions`.
Lives in src/pareto/reference_points.rs. Re-exported from the prelude
as `das_dennis`.
Tests cover: M=2/H=4 → 5 points along the diagonal; M=3/H=12 → 91
points (the canonical NSGA-III 3-objective ref set); each generated
point has exactly M components summing to 1 within float tolerance.
Implementation of Zitzler, Laumanns, Thiele 2001 SPEA2 — the classic
Pareto MOEA built around an explicit external archive of fixed size.
Each generation:
1. Combine the current population and the archive into one pool.
2. For every member, compute strength S(i) = number of others that
member dominates, then raw fitness R(i) = sum of S(j) over members
j that dominate i.
3. Add a density estimator D(i) = 1/(σ_k + 2) where σ_k is the distance
to the k-th nearest neighbor (k = floor(sqrt(|pool|))) in
minimization-oriented objective space.
4. Final fitness F(i) = R(i) + D(i); lower is better.
5. Build the next archive by taking every non-dominated member
(R(i) == 0). If too many, prune by repeatedly removing the member
with the smallest k-th-nearest-neighbor distance. If too few, fill
from the rest sorted by F ascending.
6. Generate the next population by binary tournament on F (lower wins),
then variation, then evaluation.
Public API mirrors the other algorithms:
Spea2Config { population_size, archive_size, generations, seed }
Spea2 { config, initializer, variation }
impl<P, I, V> Optimizer<P> for Spea2<I, V>
Re-exported from the prelude. Tests cover archive size invariants,
non-empty Pareto front on Schaffer N.1, deterministic reruns under
the same seed, and panic on population_size == 0.
A comparison example that runs every applicable optimizer on ZDT1 and
Rastrigin across N seeds and reports mean ± stddev for each quality
metric. Designed so a new algorithm slots in by adding a single runner
function — no harness changes needed.
ZDT1 (multi-objective, dim=30):
Reports hypervolume_2d (against ref point [1.1, 1.1]), spacing, mean
L2 distance to the analytical Pareto front, front size, and wall-clock
ms. RandomSearch, PAES, and NSGA-II all use bounds-aware operators
(RealBounds, BoundedGaussianMutation, SBX+PolyMut) so the Problem
itself stays unclamped — apples-to-apples.
Rastrigin (single-objective, dim=5):
Reports mean ± stddev best objective and ms. RandomSearch, PAES,
NSGA-II (degenerate single-obj case), and DE.
Default budget: 10 seeds × 25,000 evaluations on ZDT1, × 50,000 on
Rastrigin. Run with:
cargo run --release --example compare
- PolynomialMutation::vary: `#[allow(clippy::needless_range_loop)]`
on the per-dimension loop — body indexes both `self.bounds[j]` and
`child[j]` so a range index is the cleanest option.
- Operator tests: replace `x >= lo && x <= hi` with
`(lo..=hi).contains(&x)` per clippy's manual_range_contains lint.
Replace the v0.1 `GaussianMutation` + clamp-inside-`evaluate` setup
with the canonical NSGA-II operator pair: SBX (η_c=15, per-var prob 0.5)
followed by PolynomialMutation (η_m=20, per-var prob 1/dim), composed
via `CompositeVariation`. Both are bounds-aware on their own, so the
in-evaluate clamping is dropped.
Result on ZDT1 (dim=30, pop=100, gens=1000, seed=42): mean L2 distance
to the analytical Pareto front is 0.00152 — comfortably within the
published NSGA-II range for this benchmark.
Note on the previous number: the v0.1 setup reported 0.00072 at 40k
evals, but that was an artifact of clamping inside `evaluate`. Out-of-
bounds Gaussian mutations on `x[0]` were snapping to 0, which
coincides with the ZDT1 Pareto-front extreme (f1=0). The new operator
pair has no such free lunch — it runs the actual NSGA-II algorithm —
and the new measurement is what honest convergence on ZDT1 actually
looks like.
Generations bumped from 400 to 1000 (40k → 100k evaluations) to give
the operators headroom; matches the budget DE uses for Rastrigin so
the example feels balanced.
Generic two-stage Variation operator: runs an inner crossover-style
operator on the parents, then applies an inner mutation-style operator
to each resulting child. Lets users build the canonical NSGA-II
operator stack — `SimulatedBinaryCrossover` followed by
`PolynomialMutation` — by composing the existing primitives instead
of bundling a one-off SbxPolyMut struct.
Lives in src/operators/composite.rs to keep type-specific operator
files unchanged. Generic over decision type and over both inner
operators.
Deb's standard real-valued mutation pair to SBX, used together by
canonical NSGA-II. For each variable, with probability
`per_variable_probability` (typical: 1/n where n is dim), perturb the
parent value by a polynomial-distributed delta scaled by the bound
range, then clamp.
Per-dim formula:
- `u ~ U[0, 1)`
- `δ = (2u)^(1/(η+1)) − 1` if `u < 0.5` else `1 − (2(1−u))^(1/(η+1))`
- `child[j] = parent[j] + δ · (hi − lo)`, clamped to bounds
`eta` is the distribution index (typical 20; smaller → more spread).
This is the simple bound-rescale form; the bound-aware δ_q variant from
the full paper is left as a future refinement.
Always returns one child. Tests cover: child stays in bounds with high
sigma-equivalent eta, per_variable_probability=0 returns the parent
unchanged, and standard panics.
Deb & Agrawal's standard real-valued crossover for NSGA-II. Takes two
parents, returns two children; per dimension, with
`per_variable_probability`, mixes the parents using a polynomial
spread parameter \\(\\beta\\) drawn from a distribution controlled by
`eta` (the distribution index — typical values 10–30, default 15).
Children are clamped to per-variable bounds.
Per-dim formula (Deb & Agrawal 1995):
- `u ~ U[0, 1)`
- `β = (2u)^(1/(η+1))` if `u ≤ 0.5` else `(1 / (2(1-u)))^(1/(η+1))`
- `c1 = 0.5·((1+β)·p1 + (1-β)·p2)`, `c2 = 0.5·((1-β)·p1 + (1+β)·p2)`
This is the simple compute-then-clamp form; the bounds-aware
β formulation from the full paper is left as a future refinement.
Tests cover: two children for two parents, output lengths preserved,
all variables clamped to bounds, and per_variable_probability=0
returns the parents unchanged.
A bounded variant of GaussianMutation: same Gaussian noise applied to
the first parent, but every variable is clamped to its per-dimension
inclusive bound. Useful as a drop-in for problems that need feasibility
maintained across generations rather than relying on
clamp-inside-evaluate.
Panics on `sigma <= 0.0`, on no parents, and on construction if any
`(lo, hi)` has `lo > hi`. Decision length must match the bounds
length when called.
Adds a `parallel` Cargo feature that pulls in rayon and parallelizes
the only step that's actually expensive in practice — calls to
`Problem::evaluate` — across the population. RNG-driven steps (parent
and donor selection, variation, replacement decisions) stay serial, so
seeded runs remain deterministic regardless of feature state, and the
default and `--features parallel` builds produce bit-identical
results.
Wiring:
- New `algorithms::parallel_eval::evaluate_batch` helper with two
cfg-gated implementations (rayon's `into_par_iter` when the feature
is on, plain `into_iter` otherwise). Both preserve input order, so
pareto_front and crowding-distance decisions remain reproducible.
- `RandomSearch`, `Nsga2`, and `DifferentialEvolution` now route
population/offspring evaluation through the helper. NSGA-II's main
loop is restructured into a serial selection-and-variation phase
followed by a parallel-friendly batch evaluation phase.
- DE's per-target loop is restructured into three phases (serial trial
construction → batch evaluation → serial replacement). Side effect
of the restructuring: DE is now the canonical synchronous DE/rand/1/bin
rather than the asynchronous variant where target `i+1` sees `i`'s
in-flight update. Synchronous is the textbook formulation, so this
is a small correctness improvement on top of the parallelism enable.
- PAES stays serial — its main loop has a sequential dependency on the
current candidate and would gain nothing from rayon.
Cost: algorithm impls now require `P: Sync` and `P::Decision: Send`
unconditionally so a single impl serves both feature modes. This is a
small bound tightening that any plain-data Problem already satisfies; in
return the public `Problem` trait itself stays unchanged and the
default build picks up no new dependencies.
Verified:
- `cargo test` and `cargo test --features parallel` both pass; the
Nsga2 `deterministic_with_same_seed` test confirms reproducibility.
- `cargo run --release --example benchmarks` and the same with
`--features parallel` produce bit-identical ZDT1 / Rastrigin
results.
Two canonical optimization benchmarks in a single runnable example:
- ZDT1 (Zitzler-Deb-Thiele 1): 30-D, two minimization objectives,
closed-form Pareto front \\(f_2 = 1 - \\sqrt{f_1}\\) for
\\(f_1 \\in [0, 1]\\). Solved with NSGA-II.
- Rastrigin: highly multimodal single-objective, global minimum
\\(f = 0\\) at the origin. Solved with DE.
Both are public-domain mathematical formulas. Implemented as Problem
impls in examples/benchmarks.rs; main() runs each, prints front /
best, and (for ZDT1) reports the mean L2 distance from the known
analytical Pareto front so the example doubles as a sanity check on
solution quality.
- pareto/crowding.rs: rewrite the inner loop to iterate per-objective
via index_axis-style indexing on `oriented` rather than naming an
unused loop variable `k`.
- operators/{binary,permutation}.rs tests: pass parents via
`std::slice::from_ref` instead of `&[parent.clone()]` to avoid the
cloned_ref_to_slice_refs lint.
Pure cleanup — no behavior change, all 83 unit tests + 2 doctests still
pass.
Adds:
- README.md following spec §19.1 (what / install / define problem /
run NSGA-II / custom optimizer / current algorithms / design
philosophy).
- A short-but-runnable crate-level //! example in lib.rs for
`cargo doc` (spec §19.2).
The three runnable examples called out in spec §18.5 / §19. All open
with `use heuropt::prelude::*;` so they double as a check that the
prelude is sufficient on its own:
- toy_nsga2.rs: Schaffer N.1 solved with NSGA-II.
- random_search.rs: 2D sphere solved with RandomSearch.
- custom_optimizer.rs: a minimal hill-climber implementing
`Optimizer<P>` directly, demonstrating spec §2.3.
Exact 2D dominated hypervolume against a fixed reference point. Sorts
points by the first minimization-oriented objective ascending, then
sweeps and accumulates the dominated rectangle area against the
reference. Points that don't strictly dominate the reference are
ignored. Panics with a clear message if the objective space does not
have exactly two objectives (spec §14.2).
Tests cover a known-area front, the no-coverage case, and the panic on
non-2D problems.
Standard Schott spacing: for each front point compute the Manhattan
distance to its nearest neighbor on minimization-oriented objective
values; the spacing metric is the population standard deviation of
those nearest-neighbor distances.
Returns 0.0 for empty or single-point fronts (spec §14.1).
Optional v1 algorithm requested by the user (spec §12.4):
- Vec<f64> decisions only.
- Single-objective only — panics with a clear message otherwise.
- Standard DE/rand/1/bin: for each target i, sample distinct r1, r2, r3;
mutant = x[r1] + F * (x[r2] - x[r3]); apply binomial crossover with at
least one forced index; greedy replacement on direction-correct
comparison.
- Bounds taken from the embedded RealBounds (mutants are clamped to the
per-variable range so the trial vector stays feasible).
- Seed-deterministic; tests verify reproducibility, that DE improves on
the initial random population for a sphere problem, and that
multi-objective use panics.
Standard (μ+λ) NSGA-II with binary tournament parent selection on
(rank, crowding distance) and elitist survival selection on the combined
parent + offspring population (spec §12.3):
1. Initialize population_size random decisions.
2. Each generation: select parents by binary tournament (rank ↑ then
crowding ↓ then random), apply variation, evaluate offspring,
combine, non_dominated_sort, fill the next population front-by-front
trimming the partial last front by crowding distance descending.
3. Return final population, Pareto front, best (None for >1 objective),
evaluation count, and generation count.
Internal Nsga2Entry { candidate, rank, crowding_distance } stays
private. Panics with clear messages on `population_size == 0` or
empty `vary` output. Tests cover population length, evaluation count,
non-empty front, and full determinism with the same seed (spec §18.4).
A readable v1 PAES (spec §12.2):
- Single starting decision from the initializer.
- Each iteration mutates the current decision via the Variation operator,
evaluates the child, and pareto_compares to the current.
- Dominating children become current; for non-dominated comparisons we
move to the child (acceptable v1 behavior per spec).
- Both current and child are inserted into a ParetoArchive truncated
to `archive_size` (simple tail-truncation in v1).
The final result returns the archive as both `population` and
`pareto_front`. Tests verify the archive never exceeds
`archive_size`.
The reference baseline and the spec's recommended starting example. Per
iteration it asks the initializer for `batch_size` decisions, evaluates
each, and accumulates them. At the end it returns the full population
plus the Pareto front and (if single-objective) the best feasible
candidate. `generations` equals `iterations`; `evaluations` equals
`iterations * batch_size` (spec §12.1).
Includes a tiny single-objective sphere test problem under
`tests_support` that later algorithm tests will reuse.
`select_random` samples `count` decisions with replacement and clones
them out of the population (spec §10.1).
`tournament_select_single_objective` runs binary-or-larger tournaments
with the spec's tiebreak order: feasible beats infeasible, lower
violation among infeasibles, and direction-correct objective comparison
among feasibles. Panics if not exactly one objective (spec §10.2).
Selection helpers stay under `heuropt::selection` and are not part of
the prelude (spec §15).
Variation that clones the first parent (a Vec<usize> permutation) and
swaps two distinct random indices when len >= 2 (spec §11.4). Tests
confirm the multiset of contents is preserved.
Variation that clones the first parent and flips each bit independently
with probability `probability`. Panics if probability is outside [0, 1]
(spec §11.3).
Tests verify that probability=0 produces an unchanged child and
probability=1 flips every bit (spec §18.3).
`RealBounds` (Initializer<Vec<f64>>) samples each variable uniformly in
its inclusive (lo, hi) range; panics if any bound has lo > hi
(spec §11.1).
`GaussianMutation` (Variation<Vec<f64>>) clones the first parent and
adds Normal(0, sigma) noise to every element; panics on sigma <= 0.0;
does not enforce bounds in v1 (spec §11.2).
A concrete archive (not a trait — spec §13). On insert it discards the
new candidate if any existing member dominates it, then removes existing
members the new candidate dominates. `truncate` does simple
tail-truncation in v1; the doc note flags that crowding-aware
truncation is a future improvement.
Computes per-point crowding distance over a single Pareto front (spec
§9.6):
- Returns Vec<f64> with the same length as the front index slice.
- Empty front → empty Vec.
- Front of length ≤ 2 → all f64::INFINITY.
- Boundary points along each objective receive INFINITY.
- Interior points get sum of normalized neighbor gaps; if max == min for
an objective the contribution is zero.
- Operates on minimization-oriented objective values.
Deb's fast non-dominated sort: returns Vec<Vec<usize>> of front indices
into the input population, with fronts[0] being the non-dominated set.
O(N²·M) is acceptable for v1 (spec §9.5).
Tests cover: small known population produces expected fronts; equal
candidates land on the same front; an empty population yields no
fronts.
`pareto_front` returns all candidates not dominated by any other in
input order (O(N²·M), acceptable for v1 per spec §9.3).
`best_candidate` is the single-objective "best" finder: returns None
unless there is exactly one objective; ignores infeasibles; returns None
if every candidate is infeasible (spec §9.4).
Both re-exported from the prelude.
A single `use heuropt::prelude::*;` brings in the common user-facing
types and traits available so far. Subsequent commits add Pareto
helpers, operators, and algorithms to the same prelude as they land.
The three operator-level traits the algorithms consume, plus the single
trait users implement to add a new optimizer (`Optimizer<P>`). All take
`&mut Rng` directly rather than being generic over the RNG (spec §7.8).
The single trait users implement to describe an optimization problem:
associated `Decision: Clone` plus `objectives()` and `evaluate()`.
Both signatures match spec §8.1; `evaluate` takes `&self`.
Plain-data structs and the seeded Rng alias from spec §7. Each lives in
its own file under src/core/ with unit tests:
- Direction, Objective, ObjectiveSpace (with as_minimization negating
only Maximize axes)
- Evaluation (is_feasible == constraint_violation <= 0.0)
- Candidate<D>, Population<D> (concrete, public fields, From<Vec<...>>)
- OptimizationResult<D>
- type Rng = rand::rngs::StdRng + rng_from_seed, so no public trait is
generic over the RNG (spec §2.5)
All public types behind #[cfg_attr(feature = "serde", derive(...))] so
the optional feature wires up without changing the default surface.
- License the crate as MIT only.
- Fill in package description and `readme` field.
- Add rand 0.9 and rand_distr 0.5 (the `StdRng` and Normal sampler the
spec mandates as the single Rng type).
- Add optional serde 1 gated behind a `serde` feature flag for later
derives on core data types.
Initial state from `cargo new --lib` plus the technical design spec at
docs/heuropt_tech_design_spec.md, which is the source of truth for the
crate's public API and v1 acceptance criteria.
swaits