Moves the problem definitions, the ~88 algorithm runners, and the
table-printing presentation out of examples/compare.rs into
examples/_shared/compare_workload.rs (path-included as `mod workload`).
examples/compare.rs is now a thin shim that calls `workload::run_all()`.
This makes the exact `compare` workload reusable: an upcoming gungraun
profiling benchmark (benches/compare_profile.rs) path-includes the same
module and drives the runner functions directly, so it exercises
identical code with no duplication. `examples/_shared/` has no `main.rs`,
so cargo does not auto-discover it as an example.
Pure reorganization: `cargo run --release --example compare` produces
identical output (every quality metric, front size, and row order
unchanged; only the non-deterministic wall-clock `ms` column jitters).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a many-objective section to the comparison harness, exercising the
regime where Pareto dominance stops discriminating: with enough
objectives almost every pair of solutions is mutually non-dominated.
- DTLZ2 4-objective: the entry point to many-objective.
- DTLZ2 10-objective: the curse of dimensionality in full.
- DTLZ1 8-objective: dominance collapse stacked on DTLZ1's deceptive
multimodal g-term.
Implemented generically: the existing Dtlz1/Dtlz2 structs and distance
metrics are already objective-count agnostic, so a single `ManySpec` +
nine generic runners (RandomSearch, NSGA-II, NSGA-III, MOEA/D, RVEA,
GrEA, IBEA, HypE, AGE-MOEA) cover all three tables -- and any future M.
The results are a clean teaching story:
- NSGA-II collapses -- on DTLZ2-10 it finishes dead last, *worse than
random search* (2.01 vs 0.63); its crowding distance actively
misleads in 10-D.
- HypE / MOEA/D / GrEA / IBEA barely notice the 4 -> 10 jump.
- GrEA wins DTLZ1-8, consistent with the 3-objective DTLZ1 table.
- HypE reverses: #1 on both DTLZ2 tables, #6 on the deceptive DTLZ1-8.
Regenerated examples/compare-results.md with the three new sections.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The `compare` harness contradicts two recommendations in the decision
trees:
- "Disconnected or non-convex front -> AGE-MOEA, KnEA, IBEA" had it
backwards. Added KnEA to the ZDT3 table (the disconnected-front
benchmark) so the claim is actually exercised: AGE-MOEA and KnEA
finish *last and second-last*; IBEA wins, MOEA/D and NSGA-II follow.
The trees now split "disconnected" from "non-convex contiguous",
lead disconnected with IBEA, and note the geometry-aware methods
trail when the front is in pieces.
- The book filed Simulated Annealing on permutations as a "one-decision
baseline" and led the JSS row with GA. On the harness SA *wins* the
FT06 job-shop table and ties for the TSP optimum; SA/Tabu edge out
the GA. Reframed SA/Tabu as strong sequencing methods.
Also regenerated examples/compare-results.md for the new ZDT3 row.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The terminal output was misaligned: headers and data were right-aligned
with hardcoded column widths and separator lengths, and the mean ± std
cells contained the non-ASCII `±` (plus `ε`, `↑`, `↓`) -- on any terminal
that renders those at a non-1 column width the columns drift, and the
hardcoded `-`.repeat(n) separators didn't match the real table width
anyway.
Changes:
- New `print_table` helper: column widths derived from the actual cell
contents (header + every row), separator length computed to match.
- All table cells are now ASCII: `+/-` instead of `±`, `eps-MOEA`
instead of `ε-MOEA`, arrows dropped from headers.
- Every table is sorted best-first by its primary quality metric.
- Added three combinatorial / sequencing problems with their own
(permutation- / bitstring-native) algorithm rosters: a convex-position
ring TSP (known optimum), FT06 job-shop makespan (known optimum 55),
and a bi-objective 0/1 knapsack scored by hypervolume.
- Expanded every problem's preamble: what it is, why it's hard, and the
best-known / optimal result.
- Regenerated examples/compare-results.md to match.
Continuous-problem quality metrics are unchanged (bit-identical to prior
snapshots); only ms columns and row order move.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds runners for the four expensive-eval / gradient-free additions to
the appropriate single-objective sections of `examples/compare.rs`:
- Rastrigin (multimodal): now also shows IPOP-CMA-ES alongside vanilla
CMA-ES so the restart benefit is directly visible.
- Rosenbrock (smooth valley): adds Nelder-Mead (well-suited) and (1+1)
ES (cheap baseline).
- Ackley + Rosenbrock: BayesianOpt run with a deliberately TINY budget
(60 evaluations vs 30k for the population-based methods) so the
sample-efficiency claim is visible — BO with 60 evals vs DE/CMA-ES
with 30k.
The compare harness now sides-by-sides 23 algorithms total across the
seven benchmark problems.
Auger & Hansen 2005 IPOP-CMA-ES: wraps the existing CmaEs in a restart
loop that doubles the population size and re-randomizes the mean
whenever a restart trigger fires. Specifically addresses the failure
mode we observed on Rastrigin (vanilla CMA-ES = 2.3 vs DE = 0).
Restart triggers:
- The whole budget for one inner CmaEs run finishes without improvement
- (More sophisticated triggers — eigenvalue collapse, condition-number
blow-up, sigma stagnation — are left for future versions; the
per-run budget trigger captures the bulk of the practical benefit)
Each restart:
- Doubles the population_size (Auger & Hansen 2005)
- Re-randomizes the initial mean to a fresh point in the bounds box
- Resets sigma to the user's initial value
Same Vec<f64> + single-objective constraints as CmaEs. The total
budget is divided across restarts; restart budget grows with
population. Tests verify it beats vanilla CMA-ES on Rastrigin.
Expands the comparison harness with four new test problems chosen for
their distinct geometry:
- **Rosenbrock** (single-obj, smooth valley): the classic non-convex
smooth function. Differentiates CMA-ES (which exploits the local
metric) from Rastrigin's multimodal-trap regime.
- **Ackley** (single-obj, exponential multimodal trap): a more
forgiving multimodal test than Rastrigin — fewer narrow local
minima — so CMA-ES can show its strength while DE/GA still win.
- **ZDT3** (multi-obj, disconnected front): the only ZDT-family
problem with a non-contiguous Pareto front. Tests an algorithm's
ability to maintain spread across gaps.
- **DTLZ1** (many-obj, 3-D linear front): a triangular plane in
objective space (vs DTLZ2's spherical octant). Different shape
reveals which many-obj algorithms are biased toward sphere-like
fronts vs which infer geometry adaptively.
Each new section runs all applicable algorithms × N seeds × the
algorithm-class budget the existing sections already use.
Adds runners for the five new MO algorithms in both the ZDT1 (2-obj)
and DTLZ2 (3-obj) sections of `examples/compare.rs`. The harness now
side-by-sides 11 multi-/many-objective optimizers (RandomSearch + 10
real ones) on each problem.
Adds runners for HillClimber, SimulatedAnnealing, GeneticAlgorithm,
ParticleSwarm, CmaEs, and Umda to `examples/compare.rs`. Rastrigin
section now compares 8 single-objective optimizers against each other
on a fixed evaluation budget.
The MO sections (ZDT1, DTLZ2) are unchanged for now — MOPSO and IBEA
get added in a follow-up commit so each algorithm's debut shows up
clearly in the harness.
Two new runners — `zdt1_moead` and `dtlz2_moead` — using the same
SBX + PolyMut variation as the other Pareto-based methods. Reference
divisions chosen so the implied population size is comparable to the
other algorithms in each section (99 → 100 weights for ZDT1; 12 → 91
weights for DTLZ2).
- nsga3: drop redundant `.into_iter()` in extend call; use
`#[allow(clippy::needless_range_loop)]` on the back-substitution
loop where `j` indexes into the matrix; remove an unneeded
`return` keyword in a closure.
- spea2: switch `pool.extend(x.drain(..))` to `pool.append(&mut x)`.
- examples/compare.rs DTLZ2 evaluator: same `needless_range_loop`
silencer on the inner cosine product loop.
NSGA-III's value over NSGA-II shows up at 3+ objectives, where
crowding distance loses its diversity signal. Adds a third comparison
section to `examples/compare.rs`:
DTLZ2 (3-objective, 12-D, the textbook benchmark for many-objective
algorithms): unit-sphere-octant Pareto front. Compares RandomSearch,
NSGA-II, SPEA2, and NSGA-III on:
- mean distance from front points to the unit sphere
(closed-form: |1 - sqrt(f1² + f2² + f3²)|),
- spacing,
- front size,
- wall-clock ms.
NSGA-III config: H=12 reference divisions (91 reference points,
matching the canonical setup from Deb & Jain 2014).
Also wires NSGA-III into the existing ZDT1 (2-objective) section even
though it's not its sweet spot — useful as a regression check that the
algorithm at least keeps up with NSGA-II on bi-objective problems.
A comparison example that runs every applicable optimizer on ZDT1 and
Rastrigin across N seeds and reports mean ± stddev for each quality
metric. Designed so a new algorithm slots in by adding a single runner
function — no harness changes needed.
ZDT1 (multi-objective, dim=30):
Reports hypervolume_2d (against ref point [1.1, 1.1]), spacing, mean
L2 distance to the analytical Pareto front, front size, and wall-clock
ms. RandomSearch, PAES, and NSGA-II all use bounds-aware operators
(RealBounds, BoundedGaussianMutation, SBX+PolyMut) so the Problem
itself stays unclamped — apples-to-apples.
Rastrigin (single-objective, dim=5):
Reports mean ± stddev best objective and ms. RandomSearch, PAES,
NSGA-II (degenerate single-obj case), and DE.
Default budget: 10 seeds × 25,000 evaluations on ZDT1, × 50,000 on
Rastrigin. Run with:
cargo run --release --example compare
swaitsandClaude Opus 4.7