From 232dbc01721deb0ca8e21dc0aa9d95b238c94076 Mon Sep 17 00:00:00 2001 From: Stephen Waits Date: Tue, 5 May 2026 12:03:52 -0600 Subject: [PATCH] chore(release): bump to v0.4.0 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit CHANGELOG entry consolidates the unreleased work since v0.3.0: testing-infrastructure expansion (proptest suites, cargo-fuzz harness, stability tests, gungraun benches, CI), two real bug fixes the testing surfaced (NaN-cycle non_dominated_sort, simplex projection magnitude precision), the README decision-tree update against the v0.3.0 comparison data, and the v0.4.0 perf pass (cumulative compare harness 18.6 s → 5.7 s, 3.27×). `examples/compare-results.md` refreshed with the post-perf-pass ms numbers; quality metrics are bit-identical to the v0.3.0 snapshot (the perf pass was strictly CPU time, never algorithmic). --- CHANGELOG.md | 88 ++++++++++++++++++++++++++- Cargo.toml | 2 +- examples/compare-results.md | 116 +++++++++++++++++++----------------- 3 files changed, 149 insertions(+), 57 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 041655c..378ac2b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,90 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +## [0.4.0] — 2026-05-05 + +Theme: testing infrastructure, two real bug fixes surfaced by that +infrastructure, and a CPU-time optimization pass that made the +comparison harness 3.27× faster end-to-end. No breaking changes to +the v0.3.0 public API. + +### Performance + +A focused, measure-and-iterate optimization pass on the Pareto-based +multi-objective hot paths. Every change verified bit-identical against +the v0.3.0 comparison-harness snapshot — quality metrics +(hypervolume, spacing, mean L2, mean dist, front size) match to the +last decimal in every benchmark. + +**Cumulative wall-clock impact (compare harness, 10-seed mean):** + +| Algorithm / Problem | v0.3.0 | v0.4.0 | Speedup | +|----------------------|--------:|--------:|--------:| +| AGE-MOEA / DTLZ1 | 2299 ms | 229 ms | 10× | +| SPEA2 / DTLZ2 | 4304 ms | 513 ms | 8.4× | +| AGE-MOEA / ZDT3 | 932 ms | 193 ms | 4.8× | +| NSGA-II / ZDT1 | 268 ms | 65 ms | 4.1× | +| NSGA-II / ZDT3 | 267 ms | 65 ms | 4.1× | +| SMS-EMOA / DTLZ2 | 5643 ms | 1369 ms | 4.1× | +| NSGA-II / Rastrigin | 260 ms | 71 ms | 3.7× | +| NSGA-II / DTLZ2 | 344 ms | 106 ms | 3.2× | +| NSGA-III / DTLZ2 | 318 ms | 122 ms | 2.6× | +| NSGA-III / DTLZ1 | 303 ms | 122 ms | 2.5× | +| HypE / DTLZ2 | 80 ms | 44 ms | 1.8× | +| **Total compare** | **18 629 ms** | **5688 ms** | **3.27×** | + +**Hot-path instruction counts (gungraun):** + +| Benchmark | v0.3.0 | v0.4.0 | Speedup | +|-------------------------|------------:|---------:|--------:| +| `hypervolume_nd_3d` n=100 | 13 523 760 | 367 767 | 37× | +| `hypervolume_nd_3d` n=30 | 676 902 | 70 334 | 9.6× | +| `non_dominated_sort_2d` n=200 | 13 513 271 | 2 601 813 | 5.2× | +| `non_dominated_sort_2d` n=50 | 852 317 | 198 574 | 4.3× | +| `spea2_short` | 179 113 | 133 783 | 1.34× | + +**Changes (in commit order):** + +- `perf(hypervolume)` — Rewrote the M≥3 HSO recursion in + `hypervolume_nd`. The original cloned the active set into a fresh + Vec> at the top of every recursive call, used a linear-scan + `position` lookup to remove the just-processed point each band, and + re-projected onto M-1 axes inside every band. Now: sort-by-index, + pre-project once, slice prefixes for the active set, and skip + `non_dominated_projection` when recursing into the M=2 base case + (whose sweep already filters dominated points internally). +- `perf(non_dominated_sort)` — Cache `as_minimization` / + feasibility / violation per individual once at the top of the + Deb fast-non-dominated-sort, then inline the dominance test against + those arrays. The naïve formulation called `pareto_compare` twice + per pair, each call allocating two fresh Vecs — 4N(N-1) + allocations per sort. Propagates to every Pareto-based MOEA. +- `perf(age_moea)` — Cache `lp_norm(translated[i], p)` once per + candidate at function entry; maintain a `nearest[]` array updated + incrementally on each pick (single `min` per remaining instead of + a fresh full scan over the keep list). Cuts the splitting-front + scoring loop from O(R · K · M) per iteration to O(R · M). +- `perf(spea2)` — Two wins. (1) `compute_fitness` (called twice per + generation): inline dominance against cached oriented arrays, + symmetric distance matrix built once. (2) `build_archive` truncation: + compute pairwise distances + sorted neighbor vectors once, then on + victim removal use binary-search-remove on every survivor's + still-sorted vector — total truncation cost O(K³ log K) → O(K² log K). +- `perf(hypervolume)` — Index-sort instead of cloning point vectors + in the M≥3 recursion. The N inner-Vec clones per HV call were + redundant once we'd already sorted by last-axis. Big bench win + (32×→37× cumulative on n=100/3D), modest wall-clock impact because + SMS-EMOA's worst-front HV calls operate on small fronts. +- `build(release)` — Enable thin LTO + codegen-units=1 in the + release profile. Worth ~150 ms across the harness; only applies + when heuropt is the workspace root, so downstream consumers see + whatever profile their own Cargo.toml configures. +- `perf(pareto_archive)` — Cache the candidate's oriented + + feasibility once per `insert`, build each member's oriented vector + once, and inline the two-pass dominance checks. Used by PESA-II + (most impact), PAES, ε-MOEA, and any user code working through the + archive directly. + ### Added - **Decision tree update** in README to cover all v0.3.0 algorithms, @@ -59,6 +143,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ratio, and is robust to floating-point precision loss in the algorithm's inner loop. +[0.4.0]: https://github.com/swaits/heuropt/releases/tag/v0.4.0 + ## [0.3.0] — 2026-05-05 Theme: filling heuropt's expensive-evaluation, gradient-free, and @@ -297,5 +383,5 @@ Initial release. `RandomSearch`, `Nsga2`, and `DifferentialEvolution`. Seeded runs stay bit-identical to serial mode. -[Unreleased]: https://github.com/swaits/heuropt/compare/v0.3.0...HEAD +[Unreleased]: https://github.com/swaits/heuropt/compare/v0.4.0...HEAD [0.1.0]: https://github.com/swaits/heuropt/releases/tag/v0.1.0 diff --git a/Cargo.toml b/Cargo.toml index 55bda82..3b28b1c 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "heuropt" -version = "0.3.0" +version = "0.4.0" edition = "2024" rust-version = "1.85" authors = ["Stephen Waits "] diff --git a/examples/compare-results.md b/examples/compare-results.md index 1390ee0..68c8720 100644 --- a/examples/compare-results.md +++ b/examples/compare-results.md @@ -1,7 +1,13 @@ # `compare` example — reference output -Snapshot from `cargo run --release --example compare` after the v0.3.0 -algorithm cohort landed (2026-05-05). 10 seeds per algorithm per problem. +Snapshot from `cargo run --release --example compare` after the v0.4.0 +perf pass landed (2026-05-05). 10 seeds per algorithm per problem. + +The **quality metrics** (hypervolume / spacing / mean L2 / mean dist / +front size) are bit-identical to the v0.3.0 snapshot — the v0.4.0 +optimization work was strictly CPU-time, never algorithmic. The **ms +columns** reflect the v0.4.0 numbers; total compare-harness wall-clock +dropped from ~18.6 s to ~5.7 s (3.27× faster). Wall-clock numbers are from the development machine and will vary; the *relative* numbers across algorithms are the interesting part. @@ -15,19 +21,19 @@ Two-objective benchmark with a smooth Pareto front along | algorithm | hypervolume ↑ | spacing ↓ | mean L2 ↓ | front | ms | |---|---|---|---|---|---| -| RandomSearch | 99.5691 ± 0.94 | 0.0937 ± 0.03 | 2.3621 ± 0.14 | 28 | 111 | -| PAES | 104.1887 ± 0.90 | 0.0351 ± 0.01 | 1.3195 ± 0.06 | 33 | 16 | -| MOPSO | **120.6149 ± 0.05** | 0.0125 ± 0.00 | **0.0005 ± 0.00** | 100 | 106 | -| SPEA2 | 118.0823 ± 0.60 | 0.0111 ± 0.00 | 0.2408 ± 0.05 | 97 | 461 | -| PESA-II | 119.3670 ± 0.33 | **0.0095 ± 0.00** | 0.0802 ± 0.04 | 100 | 86 | -| ε-MOEA | 118.8742 ± 0.68 | 0.0167 ± 0.01 | 0.0493 ± 0.02 | 45 | 55 | -| IBEA | 120.0167 ± 0.31 | 0.0130 ± 0.00 | 0.0448 ± 0.02 | 73 | 141 | -| HypE | 105.6489 ± 0.98 | 0.0266 ± 0.01 | 1.4820 ± 0.10 | 72 | 66 | -| SMS-EMOA | 102.8871 ± 1.05 | 0.0263 ± 0.00 | 1.4937 ± 0.12 | 40 | 181 | -| RVEA | 111.7151 ± 1.82 | 0.0308 ± 0.01 | 0.8399 ± 0.16 | 47 | 60 | -| NSGA-II | 118.3336 ± 0.78 | 0.0112 ± 0.00 | 0.1891 ± 0.06 | 96 | 287 | -| NSGA-III | 115.1612 ± 0.47 | 0.0139 ± 0.00 | 0.4314 ± 0.06 | 86 | 244 | -| MOEA/D | 119.9450 ± 0.50 | 0.0118 ± 0.00 | 0.0065 ± 0.00 | 96 | 30 | +| RandomSearch | 99.5691 ± 0.94 | 0.0937 ± 0.03 | 2.3621 ± 0.14 | 28 | 94 | +| PAES | 104.1887 ± 0.90 | 0.0351 ± 0.01 | 1.3195 ± 0.06 | 33 | 30 | +| MOPSO | **120.6149 ± 0.05** | 0.0125 ± 0.00 | **0.0005 ± 0.00** | 100 | 89 | +| SPEA2 | 118.0823 ± 0.60 | 0.0111 ± 0.00 | 0.2408 ± 0.05 | 97 | 234 | +| PESA-II | 119.3670 ± 0.33 | **0.0095 ± 0.00** | 0.0802 ± 0.04 | 100 | 73 | +| ε-MOEA | 118.8742 ± 0.68 | 0.0167 ± 0.01 | 0.0493 ± 0.02 | 45 | 50 | +| IBEA | 120.0167 ± 0.31 | 0.0130 ± 0.00 | 0.0448 ± 0.02 | 73 | 138 | +| HypE | 105.6489 ± 0.98 | 0.0266 ± 0.01 | 1.4820 ± 0.10 | 72 | 38 | +| SMS-EMOA | 102.8871 ± 1.05 | 0.0263 ± 0.00 | 1.4937 ± 0.12 | 40 | 67 | +| RVEA | 111.7151 ± 1.82 | 0.0308 ± 0.01 | 0.8399 ± 0.16 | 47 | 65 | +| NSGA-II | 118.3336 ± 0.78 | 0.0112 ± 0.00 | 0.1891 ± 0.06 | 96 | 67 | +| NSGA-III | 115.1612 ± 0.47 | 0.0139 ± 0.00 | 0.4314 ± 0.06 | 86 | 70 | +| MOEA/D | 119.9450 ± 0.50 | 0.0118 ± 0.00 | 0.0065 ± 0.00 | 96 | 28 | **MOPSO and MOEA/D dominate** convergence (mean L2 to true front ≤ 0.01). PESA-II edges spacing. @@ -37,12 +43,12 @@ PESA-II edges spacing. Disconnected Pareto front; tests an algorithm's ability to maintain spread across gaps. -| algorithm | hypervolume ↑ | spacing ↓ | front | ms | +| algorithm | hypervolume ↑ | spacing ↓ | front | ms | |---|---|---|---|---| -| NSGA-II | 123.1826 ± 1.58 | 0.0092 ± 0.00 | 98 | 284 | -| MOEA/D | 125.2413 ± 2.16 | 0.0198 ± 0.00 | 92 | 31 | -| **IBEA** | **126.2072 ± 1.23** | 0.0164 ± 0.00 | 48 | 140 | -| AGE-MOEA | 119.5132 ± 1.27 | 0.0136 ± 0.00 | 90 | 1028 | +| NSGA-II | 123.1826 ± 1.58 | 0.0092 ± 0.00 | 98 | 68 | +| MOEA/D | 125.2413 ± 2.16 | 0.0198 ± 0.00 | 92 | 28 | +| **IBEA** | **126.2072 ± 1.23** | 0.0164 ± 0.00 | 48 | 135 | +| AGE-MOEA | 119.5132 ± 1.27 | 0.0136 ± 0.00 | 90 | 199 | ## DTLZ2 (3-obj, dim=12, 30000 evals × 10 seeds) @@ -50,18 +56,18 @@ Spherical Pareto front. Mean dist = `|‖f‖ − 1|`. | algorithm | mean dist ↓ | spacing ↓ | front | ms | |---|---|---|---|---| -| RandomSearch | 0.3949 ± 0.02 | 0.0797 ± 0.01 | 239 | 621 | -| MOPSO | 0.0566 ± 0.00 | 0.0687 ± 0.01 | 100 | 48 | -| NSGA-II | 0.0332 ± 0.01 | 0.0577 ± 0.01 | 92 | 380 | -| SPEA2 | 0.0368 ± 0.00 | **0.0288 ± 0.00** | 92 | 4723 | -| PESA-II | 0.0395 ± 0.00 | 0.0616 ± 0.01 | 100 | 530 | -| ε-MOEA | 0.0325 ± 0.01 | 0.0572 ± 0.02 | 136 | 104 | -| **IBEA** | **0.0014 ± 0.00** | 0.0607 ± 0.00 | 87 | 158 | -| HypE | 0.0113 ± 0.00 | 0.0269 ± 0.02 | 80 | 82 | -| SMS-EMOA | 0.0484 ± 0.01 | 0.0764 ± 0.01 | 40 | 6002 | -| RVEA | 0.0510 ± 0.00 | 0.0631 ± 0.00 | 68 | 68 | -| NSGA-III | 0.0197 ± 0.00 | 0.0735 ± 0.01 | 92 | 325 | -| MOEA/D | 0.0037 ± 0.00 | 0.0886 ± 0.00 | 78 | 25 | +| RandomSearch | 0.3949 ± 0.02 | 0.0797 ± 0.01 | 239 | 520 | +| MOPSO | 0.0566 ± 0.00 | 0.0687 ± 0.01 | 100 | 71 | +| NSGA-II | 0.0332 ± 0.01 | 0.0577 ± 0.01 | 92 | 104 | +| SPEA2 | 0.0368 ± 0.00 | **0.0288 ± 0.00** | 92 | 534 | +| PESA-II | 0.0395 ± 0.00 | 0.0616 ± 0.01 | 100 | 396 | +| ε-MOEA | 0.0325 ± 0.01 | 0.0572 ± 0.02 | 136 | 89 | +| **IBEA** | **0.0014 ± 0.00** | 0.0607 ± 0.00 | 87 | 156 | +| HypE | 0.0113 ± 0.00 | 0.0269 ± 0.02 | 80 | 53 | +| SMS-EMOA | 0.0484 ± 0.01 | 0.0764 ± 0.01 | 40 | 1218 | +| RVEA | 0.0510 ± 0.00 | 0.0631 ± 0.00 | 68 | 73 | +| NSGA-III | 0.0197 ± 0.00 | 0.0735 ± 0.01 | 92 | 137 | +| MOEA/D | 0.0037 ± 0.00 | 0.0886 ± 0.00 | 78 | 24 | **IBEA wins decisively** (15× closer to the true front than NSGA-III). @@ -69,12 +75,12 @@ Spherical Pareto front. Mean dist = `|‖f‖ − 1|`. Linear simplex Pareto front (`Σf = 0.5`). -| algorithm | mean dist ↓ | spacing ↓ | front | ms | +| algorithm | mean dist ↓ | spacing ↓ | front | ms | |---|---|---|---|---| -| NSGA-III | 5.9130 ± 2.82 | 0.4375 ± 0.22 | 92 | 310 | -| MOEA/D | 2.8022 ± 1.78 | 0.2279 ± 0.22 | 78 | 21 | -| AGE-MOEA | 4.5395 ± 2.21 | 0.3930 ± 0.29 | 90 | 2327 | -| **GrEA** | **1.7725 ± 0.99** | **0.0719 ± 0.04** | 72 | 286 | +| NSGA-III | 5.9130 ± 2.82 | 0.4375 ± 0.22 | 92 | 133 | +| MOEA/D | 2.8022 ± 1.78 | 0.2279 ± 0.22 | 78 | 21 | +| AGE-MOEA | 4.5395 ± 2.21 | 0.3930 ± 0.29 | 90 | 247 | +| **GrEA** | **1.7725 ± 0.99** | **0.0719 ± 0.04** | 72 | 104 | **GrEA shines on linear fronts** — the grid-based niching matches the geometry better than reference points. @@ -85,17 +91,17 @@ Multimodal trap. Global minimum f = 0 at the origin. | algorithm | best f | ms | |---|---|---| -| RandomSearch | 1.1064e1 ± 2.54 | 16 | -| HillClimber | 1.5966e1 ± 6.25 | 7 | +| RandomSearch | 1.1064e1 ± 2.54 | 14 | +| HillClimber | 1.5966e1 ± 6.25 | 6 | | **(1+1)-ES** | **0.0000e0 ± 0.00** | 4 | | SimulatedAnneal | 3.8540e0 ± 1.48 | 7 | | PAES | 1.5966e1 ± 6.25 | 10 | | GA | 7.0913e-8 ± 5.50e-8 | 16 | | PSO | 7.9598e-1 ± 8.67e-1 | 5 | -| NSGA-II | 4.9270e-5 ± 5.04e-5 | 269 | +| NSGA-II | 4.9270e-5 ± 5.04e-5 | 83 | | **DE** | **0.0000e0 ± 0.00** | 6 | -| CMA-ES | 2.3453e0 ± 1.49 | 12 | -| **IPOP-CMA-ES** | 1.3423e-1 ± 2.71e-1 | 69 | +| CMA-ES | 2.3453e0 ± 1.49 | 11 | +| **IPOP-CMA-ES** | 1.3423e-1 ± 2.71e-1 | 66 | (1+1)-ES and DE tie for f = 0. **IPOP-CMA-ES drops vanilla CMA-ES from 2.35 → 0.13** — the restart logic does what it should. @@ -104,15 +110,15 @@ Multimodal trap. Global minimum f = 0 at the origin. Smooth non-convex valley. -| algorithm | best f | ms | +| algorithm | best f | ms | |---|---|---| -| DE | 3.3345e-1 ± 3.01e-1 | 3 | -| PSO | 8.2124e-1 ± 1.58e0 | 2 | -| **CMA-ES** | **3.6207e-29 ± 2.35e-29** | 6 | -| TLBO | 1.8458e-3 ± 1.91e-3 | 1 | -| (1+1)-ES | 2.2115e0 ± 2.70e0 | 1 | +| DE | 3.3345e-1 ± 3.01e-1 | 2 | +| PSO | 8.2124e-1 ± 1.58e0 | 2 | +| **CMA-ES** | **3.6207e-29 ± 2.35e-29** | 5 | +| TLBO | 1.8458e-3 ± 1.91e-3 | 1 | +| (1+1)-ES | 2.2115e0 ± 2.70e0 | 1 | | **Nelder-Mead** | **0.0000e0 ± 0.00** | 1 | -| BO (60 evals) | 3.1725e3 ± 2.92e3 | 40 | +| BO (60 evals) | 3.1725e3 ± 2.92e3 | 40 | Nelder-Mead **= 0 exactly**, CMA-ES at machine epsilon. BO at only 60 evaluations is honestly bad on 5-D Rosenbrock (no kernel @@ -124,13 +130,13 @@ methods. Smoother multimodal landscape than Rastrigin. -| algorithm | best f | ms | +| algorithm | best f | ms | |---|---|---| -| DE | 4.4409e-16 ± 0.00 | 3 | -| PSO | 1.5099e-15 ± 1.63e-15 | 3 | -| CMA-ES | 1.5099e-15 ± 1.63e-15 | 7 | -| TLBO | 2.2204e-15 ± 1.78e-15 | 2 | -| BO (60 evals) | 1.9622e1 ± 1.23 | 40 | +| DE | 4.4409e-16 ± 0.00 | 4 | +| PSO | 1.5099e-15 ± 1.63e-15 | 3 | +| CMA-ES | 1.5099e-15 ± 1.63e-15 | 6 | +| TLBO | 2.2204e-15 ± 1.78e-15 | 2 | +| BO (60 evals) | 1.9622e1 ± 1.23 | 40 | All conventional methods reach machine precision. BO at 60 evals struggles — same caveat as Rosenbrock.