From 6368db74bb02d18356d747fd3a256a4dac183c7c Mon Sep 17 00:00:00 2001 From: Stephen Waits Date: Wed, 13 May 2026 18:37:47 -0600 Subject: [PATCH] docs(book): permutation toolkit and multi-objective combinatorial cookbook - Rewrites cookbook/permutation.md to cover the new operator toolkit: initializers, crossovers (OX/PMX/CX/ERX), mutations, a 'what should I use' picker, and a worked GA-on-TSP example. JSS multiset section explains why the strict-permutation crossovers don't compose with operation-string encodings and shows the local POX pattern. - Adds cookbook/multi-objective-combinatorial.md: bi-objective TSP via NSGA-II, bi-objective knapsack (binary encoding), 3-objective JSS via NSGA-III, and a hypervolume-based operator comparison. - Updates choosing-an-algorithm.md to reference the new operators in the single- and multi-objective decision tables, plus a noting NSGA-II/III's genericity over Vec and Vec decisions. - SUMMARY.md and cookbook.md updated to list the new recipe. --- docs/book/src/SUMMARY.md | 1 + docs/book/src/choosing-an-algorithm.md | 32 +- docs/book/src/cookbook.md | 6 +- .../cookbook/multi-objective-combinatorial.md | 381 ++++++++++++++++++ docs/book/src/cookbook/permutation.md | 374 ++++++++++++++--- 5 files changed, 744 insertions(+), 50 deletions(-) create mode 100644 docs/book/src/cookbook/multi-objective-combinatorial.md diff --git a/docs/book/src/SUMMARY.md b/docs/book/src/SUMMARY.md index 74be0cb..70def21 100644 --- a/docs/book/src/SUMMARY.md +++ b/docs/book/src/SUMMARY.md @@ -16,6 +16,7 @@ - [Tune a model with expensive evaluations](./cookbook/expensive-evaluations.md) - [Compare two algorithms on your problem](./cookbook/compare.md) - [Optimize a permutation (TSP-style)](./cookbook/permutation.md) + - [Multi-objective combinatorial problems](./cookbook/multi-objective-combinatorial.md) - [Constrain your search with `Repair`](./cookbook/constraints.md) - [Pick one answer off a Pareto front](./cookbook/pick-one.md) - [Explore your results in a webapp](./cookbook/explorer.md) diff --git a/docs/book/src/choosing-an-algorithm.md b/docs/book/src/choosing-an-algorithm.md index 9f25375..5a5f426 100644 --- a/docs/book/src/choosing-an-algorithm.md +++ b/docs/book/src/choosing-an-algorithm.md @@ -103,10 +103,20 @@ optimizer or with the problem). | `Vec` | [UMDA][Umda] | Per-bit marginal EDA. Independent-bit assumption. | | `Vec` | [GA][GeneticAlgorithm] + [`BitFlipMutation`] | When bit interactions matter. | | `Vec` (permutation) | [Ant Colony][AntColonyTsp] | TSP-style with a distance matrix. | -| `Vec` (permutation) | [Simulated Annealing][SimulatedAnnealing] + [`SwapMutation`] | Generic discrete baseline. | +| `Vec` (permutation) | [GA][GeneticAlgorithm] + [`ShuffledPermutation`] + [`OrderCrossover`] + [`InversionMutation`] | Generic permutation GA; use [`EdgeRecombinationCrossover`] for TSP-shaped instances. | +| `Vec` (JSS multiset) | [GA][GeneticAlgorithm] + [`ShuffledMultisetPermutation`] + local POX + [`InversionMutation`] | Operation-string encoding; see [Optimize a permutation](./cookbook/permutation.md). | +| `Vec` (permutation) | [Simulated Annealing][SimulatedAnnealing] + [`SwapMutation`] | One-decision baseline. | | `Vec` or custom | [Tabu Search][TabuSearch] | You supply the neighbor function. | | Custom struct | [Simulated Annealing][SimulatedAnnealing] / [Hill Climber][HillClimber] | With your own `Variation` impl. | +heuropt's permutation operator toolkit covers four crossovers +([`OrderCrossover`], [`PartiallyMappedCrossover`], [`CycleCrossover`], +[`EdgeRecombinationCrossover`]) and four mutations ([`SwapMutation`], +[`InversionMutation`], [`InsertionMutation`], [`ScrambleMutation`]), +plus two initializers for strict and multiset permutations. See +[Optimize a permutation](./cookbook/permutation.md) for the full +picker. + ## Step 2 — multi-objective (2 or 3) ### Strong default @@ -115,6 +125,12 @@ optimizer or with the problem). maintains diversity via crowding distance. On the harness it lands on the Pareto front of every test problem. +NSGA-II is generic over the decision type — drop in +[`ShuffledPermutation`] + a permutation crossover and it solves +bi-objective TSP; drop in a binary initializer and [`BitFlipMutation`] +and it solves bi-objective knapsack. See +[Multi-objective combinatorial problems](./cookbook/multi-objective-combinatorial.md). + ### Real-valued, smooth front, want best convergence [MOPSO][Mopso] (multi-objective PSO with archive). On ZDT1 it wins @@ -244,7 +260,10 @@ method on every algorithm in the catalog. See the | Disconnected / non-convex front | [IBEA][Ibea] | | Many-objective default (curved front) | [NSGA-III][Nsga3] | | Many-objective linear / simplex front | [GrEA][Grea] | -| Permutation problem | [Ant Colony][AntColonyTsp] | +| Permutation problem (TSP with distance matrix) | [Ant Colony][AntColonyTsp] | +| Generic permutation problem | [GA][GeneticAlgorithm] + permutation toolkit | +| Bi-objective combinatorial (TSP / scheduling / knapsack) | [NSGA-II][Nsga2] + matching encoding operators | +| 3-objective combinatorial | [NSGA-III][Nsga3] + matching encoding operators | | Binary problem | [UMDA][Umda] | | Custom decision type | [Simulated Annealing][SimulatedAnnealing] + your `Variation` | | Sanity baseline | [Random Search][RandomSearch] | @@ -268,6 +287,15 @@ method on every algorithm in the catalog. See the [`BitFlipMutation`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.BitFlipMutation.html [AntColonyTsp]: https://docs.rs/heuropt/latest/heuropt/algorithms/ant_colony_tsp/struct.AntColonyTsp.html [`SwapMutation`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.SwapMutation.html +[`InversionMutation`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.InversionMutation.html +[`InsertionMutation`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.InsertionMutation.html +[`ScrambleMutation`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.ScrambleMutation.html +[`OrderCrossover`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.OrderCrossover.html +[`PartiallyMappedCrossover`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.PartiallyMappedCrossover.html +[`CycleCrossover`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.CycleCrossover.html +[`EdgeRecombinationCrossover`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.EdgeRecombinationCrossover.html +[`ShuffledPermutation`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.ShuffledPermutation.html +[`ShuffledMultisetPermutation`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.ShuffledMultisetPermutation.html [TabuSearch]: https://docs.rs/heuropt/latest/heuropt/algorithms/tabu_search/struct.TabuSearch.html [Nsga2]: https://docs.rs/heuropt/latest/heuropt/algorithms/nsga2/struct.Nsga2.html [Nsga3]: https://docs.rs/heuropt/latest/heuropt/algorithms/nsga3/struct.Nsga3.html diff --git a/docs/book/src/cookbook.md b/docs/book/src/cookbook.md index d616a06..0c1099c 100644 --- a/docs/book/src/cookbook.md +++ b/docs/book/src/cookbook.md @@ -19,7 +19,11 @@ project. - [Compare two algorithms on your problem](./cookbook/compare.md) — multi-seed harness pattern straight from `examples/compare.rs`. - [Optimize a permutation (TSP-style)](./cookbook/permutation.md) — - Ant Colony with a distance matrix. + the permutation operator toolkit (OX / PMX / CX / ERX + Inversion / + Insertion / Scramble), plus Ant Colony for distance-matrix TSP. +- [Multi-objective combinatorial problems](./cookbook/multi-objective-combinatorial.md) + — bi-objective TSP, bi-objective knapsack (`Vec`), and + 3-objective JSS via NSGA-II / NSGA-III. - [Constrain your search with `Repair`](./cookbook/constraints.md) — bounds, simplex projection, custom repair. - [Pick one answer off a Pareto front](./cookbook/pick-one.md) — the diff --git a/docs/book/src/cookbook/multi-objective-combinatorial.md b/docs/book/src/cookbook/multi-objective-combinatorial.md new file mode 100644 index 0000000..0c3bee4 --- /dev/null +++ b/docs/book/src/cookbook/multi-objective-combinatorial.md @@ -0,0 +1,381 @@ +# Multi-objective combinatorial problems + +Real combinatorial problems usually have more than one cost. A TSP +where every edge has both *distance* and *time*; a job-shop where you +care about *makespan*, *flow time*, *and* *tardiness*; a knapsack +with two profit metrics and a single weight budget. The decision +type is still combinatorial — a permutation, a bitstring — but the +objective is a vector, and the answer is a Pareto front rather than +a single best. + +heuropt's NSGA-II and NSGA-III are fully generic over the decision +type. You don't need a separate "combinatorial NSGA" — just plug in +the right initializer and variation operators for your encoding. + +This recipe walks through three patterns: + +- **Bi-objective TSP** with NSGA-II (Pareto front of two distance + matrices over the same cities) +- **Bi-objective 0/1 knapsack** with NSGA-II (binary encoding) +- **3-objective JSS** with NSGA-III (the many-objective successor) + +For the single-objective permutation toolkit it builds on, see +[Optimize a permutation](./permutation.md). + +## Bi-objective TSP + +This is the canonical multi-objective combinatorial benchmark +(Lust–Teghem 2010). Two TSP instances on the **same** city set define +two distance matrices A and B; the search trades off length under A +versus length under B. + +```rust,no_run +use heuropt::prelude::*; +use heuropt::metrics::hypervolume_2d; + +struct BiObjectiveTsp { + dist_a: Vec>, + dist_b: Vec>, +} + +impl BiObjectiveTsp { + fn tour_length(d: &[Vec], tour: &[usize]) -> f64 { + let n = tour.len(); + let mut total = 0.0; + for i in 0..n { + total += d[tour[i]][tour[(i + 1) % n]]; + } + total + } +} + +impl Problem for BiObjectiveTsp { + type Decision = Vec; + + fn objectives(&self) -> ObjectiveSpace { + ObjectiveSpace::new(vec![ + Objective::minimize("length_A"), + Objective::minimize("length_B"), + ]) + } + + fn evaluate(&self, tour: &Vec) -> Evaluation { + Evaluation::new(vec![ + Self::tour_length(&self.dist_a, tour), + Self::tour_length(&self.dist_b, tour), + ]) + } +} + +fn main() { + let n: usize = 25; + let dist_a = vec![vec![0.0_f64; n]; n]; // your matrix A + let dist_b = vec![vec![0.0_f64; n]; n]; // your matrix B + let problem = BiObjectiveTsp { dist_a, dist_b }; + + let mut optimizer = Nsga2::new( + Nsga2Config { + population_size: 200, + generations: 600, + seed: 11, + }, + ShuffledPermutation { n }, + CompositeVariation { + crossover: EdgeRecombinationCrossover, + mutation: InversionMutation, + }, + ); + let result = optimizer.run(&problem); + + println!("Pareto-front size: {}", result.pareto_front.len()); + + // Hypervolume against a generous reference point (larger than any + // length you'd reasonably see). Use this as the single-number + // quality metric for the run. + let ref_point = [40_000.0, 40_000.0]; + let hv = hypervolume_2d(&result.pareto_front, &problem.objectives(), ref_point); + println!("Hypervolume vs. {:?}: {:.0}", ref_point, hv); +} +``` + +[`EdgeRecombinationCrossover`] (ERX) is the standout crossover for +TSP. On a 25-city bi-objective instance it produces about twice the +front diversity of OX, PMX, or CX — see +`examples/tsp_operators_compare.rs` for a head-to-head benchmark. + +## Bi-objective 0/1 knapsack — `Vec` decisions + +NSGA-II works over `Vec` the same way. The Zitzler–Thiele +bi-objective knapsack is the textbook benchmark: each item has two +profit values and a single weight; you maximize both profits under +one capacity constraint. + +```rust,no_run +use heuropt::prelude::*; +use rand::Rng as _; + +const N_ITEMS: usize = 30; + +struct BiKnapsack { + profits_a: Vec, + profits_b: Vec, + weights: Vec, + capacity: f64, +} + +impl Problem for BiKnapsack { + type Decision = Vec; + + fn objectives(&self) -> ObjectiveSpace { + ObjectiveSpace::new(vec![ + Objective::maximize("profit_A"), + Objective::maximize("profit_B"), + ]) + } + + fn evaluate(&self, take: &Vec) -> Evaluation { + let (pa, pb, w) = take.iter().enumerate().fold( + (0.0_f64, 0.0_f64, 0.0_f64), + |(pa, pb, w), (i, &t)| { + if t { + (pa + self.profits_a[i], pb + self.profits_b[i], w + self.weights[i]) + } else { + (pa, pb, w) + } + }, + ); + // Standard heuristic-MO constraint handling: penalize weight + // overruns heavily so the recovered front is feasible. + let penalty = 1000.0 * (w - self.capacity).max(0.0); + Evaluation::new(vec![pa - penalty, pb - penalty]) + } +} + +/// Each bit 50/50 independently. +#[derive(Clone, Copy)] +struct RandomBinary { n: usize } +impl Initializer> for RandomBinary { + fn initialize(&mut self, size: usize, rng: &mut Rng) -> Vec> { + (0..size).map(|_| (0..self.n).map(|_| rng.random_bool(0.5)).collect()).collect() + } +} + +/// One-point crossover for binary chromosomes. +#[derive(Default)] +struct OnePointCrossoverBool; +impl Variation> for OnePointCrossoverBool { + fn vary(&mut self, parents: &[Vec], rng: &mut Rng) -> Vec> { + let (p1, p2) = (&parents[0], &parents[1]); + let n = p1.len(); + let cut = rng.random_range(1..n); + let mut c1 = Vec::with_capacity(n); + let mut c2 = Vec::with_capacity(n); + c1.extend_from_slice(&p1[..cut]); c1.extend_from_slice(&p2[cut..]); + c2.extend_from_slice(&p2[..cut]); c2.extend_from_slice(&p1[cut..]); + vec![c1, c2] + } +} + +fn main() { +# let profits_a = vec![0.0; N_ITEMS]; +# let profits_b = vec![0.0; N_ITEMS]; +# let weights = vec![0.0; N_ITEMS]; + let problem = BiKnapsack { + profits_a, profits_b, weights, + capacity: 750.0, // ~half the total weight + }; + + let mut optimizer = Nsga2::new( + Nsga2Config { + population_size: 120, + generations: 400, + seed: 19, + }, + RandomBinary { n: N_ITEMS }, + CompositeVariation { + crossover: OnePointCrossoverBool, + mutation: BitFlipMutation { probability: 1.0 / N_ITEMS as f64 }, + }, + ); + let result = optimizer.run(&problem); + println!("Pareto-front size: {}", result.pareto_front.len()); +} +``` + +Two things worth noting: + +- **`OnePointCrossoverBool` and `RandomBinary` are defined locally.** + They're tiny and common — a future PR could lift them into the + library, but for now you write them inline. +- **Constraint handling is a penalty.** The factor `1000.0` is chosen + so that even a 1-unit overrun beats any feasible solution by more + than the entire profit range; the recovered front is entirely + feasible. This is the standard heuristic-MO pattern (Deb 2001) and + cheaper than a hard repair operator. + +## Three-objective JSS with NSGA-III + +NSGA-III is designed for ≥ 3 objectives. NSGA-II's crowding distance +degrades when most of the population is mutually non-dominated, which +is the rule rather than the exception in higher dimensions; NSGA-III +uses reference-point niching instead. + +The example below adds *tardiness* to the standard (makespan, flow +time) JSS pair. Tardiness needs due dates; the common heuristic is +`dⱼ = 1.3 × sum_of_processing_times(j)`. + +```rust,no_run +use heuropt::prelude::*; +use rand::Rng as _; + +const N_JOBS: usize = 10; +const N_MACHINES: usize = 5; + +struct La01ThreeObjective { + routing: [[usize; N_MACHINES]; N_JOBS], + times: [[f64; N_MACHINES]; N_JOBS], + due: [f64; N_JOBS], +} + +impl Problem for La01ThreeObjective { + type Decision = Vec; + + fn objectives(&self) -> ObjectiveSpace { + ObjectiveSpace::new(vec![ + Objective::minimize("makespan"), + Objective::minimize("total_flow_time"), + Objective::minimize("total_tardiness"), + ]) + } + + fn evaluate(&self, schedule: &Vec) -> Evaluation { + let mut job_next = [0_usize; N_JOBS]; + let mut job_clock = [0.0_f64; N_JOBS]; + let mut machine_clock = [0.0_f64; N_MACHINES]; + for &job in schedule { + let k = job_next[job]; + let m = self.routing[job][k]; + let t = self.times[job][k]; + let start = job_clock[job].max(machine_clock[m]); + let end = start + t; + job_clock[job] = end; + machine_clock[m] = end; + job_next[job] = k + 1; + } + let makespan = machine_clock.iter().cloned().fold(0.0_f64, f64::max); + let flow_time: f64 = job_clock.iter().sum(); + let tardiness: f64 = job_clock.iter().zip(self.due.iter()) + .map(|(&c, &d)| (c - d).max(0.0)) + .sum(); + Evaluation::new(vec![makespan, flow_time, tardiness]) + } +} + +/// Mix Insertion and Scramble per call — both preserve the multiset, +/// giving the search access to two complementary neighborhood moves. +#[derive(Default)] +struct InsertionOrScramble; +impl Variation> for InsertionOrScramble { + fn vary(&mut self, parents: &[Vec], rng: &mut Rng) -> Vec> { + if rng.random_bool(0.5) { + InsertionMutation.vary(parents, rng) + } else { + ScrambleMutation.vary(parents, rng) + } + } +} + +fn main() { +# let routing = [[0; N_MACHINES]; N_JOBS]; +# let times = [[0.0; N_MACHINES]; N_JOBS]; + let due = std::array::from_fn::( + |j| 1.3 * times[j].iter().sum::(), + ); + let problem = La01ThreeObjective { routing, times, due }; + + let mut optimizer = Nsga3::new( + Nsga3Config { + population_size: 120, + generations: 600, + reference_divisions: 12, // 91 Das-Dennis points in 3-D + seed: 9, + }, + ShuffledMultisetPermutation::new(vec![N_MACHINES; N_JOBS]), + // Drop in a local PrecedenceOrderCrossover (POX) here for the + // crossover slot if you want stronger mixing; see the + // permutation recipe for the implementation. + InsertionOrScramble, + ); + let result = optimizer.run(&problem); + + println!("Pareto-front size: {}", result.pareto_front.len()); +} +``` + +A few NSGA-III tips: + +- **`reference_divisions` controls how many reference points the + algorithm spreads across the front.** For M objectives, the Das–Dennis + construction produces `C(divisions + M - 1, M - 1)` reference points. + For M = 3 and divisions = 12 that's 91 points; pick a population size + ≥ that. +- **`PrecedenceOrderCrossover` (POX)** belongs in the crossover slot + for JSS. The strict-permutation crossovers (OX, PMX, CX, ERX) break + the operation-string multiset. See + [Optimize a permutation](./permutation.md#job-shop-scheduling-multiset-encodings) + for the local POX definition. + +## Comparing operators by hypervolume + +For Pareto-front problems, single-objective fitness is the wrong +comparison metric. Use **hypervolume** instead — the dominated area +under the front, against a fixed reference point. + +```rust,ignore +use heuropt::metrics::hypervolume_2d; + +let ref_point = [40_000.0, 40_000.0]; // worse than anything you expect + +for (name, crossover) in &[ + ("OX", Box::new(OrderCrossover) as Box>>), + ("PMX", Box::new(PartiallyMappedCrossover) as _), + ("CX", Box::new(CycleCrossover) as _), + ("ERX", Box::new(EdgeRecombinationCrossover) as _), +] { + let result = run_nsga2_with_crossover(crossover); + let hv = hypervolume_2d(&result.pareto_front, &problem.objectives(), ref_point); + println!("{name:>3}: hv = {hv:.0}"); +} +``` + +This is the pattern in `examples/tsp_operators_compare.rs`. On the +KroAB-25 instance it ranks ERX > OX > PMX > CX by hypervolume. + +For ≥ 3 objectives, hypervolume in N dimensions is exponentially +expensive; use [`hypervolume_2d`] when you can collapse to two +objectives for the metric, or sample-based hypervolume from [`HypE`] +otherwise. + +## Pareto-front tips + +| Problem | Algorithm | Notes | +|---|---|---| +| 2 objectives, permutation | [Nsga2][Nsga2] | Strong default | +| 2 objectives, binary | [Nsga2][Nsga2] | Same machinery, different encoding | +| 3 objectives | [Nsga3][Nsga3] | NSGA-II's crowding distance starts to degrade | +| 4+ objectives | [Nsga3][Nsga3] or [HypE][HypE] | NSGA-III if front is curved; HypE for indicator-based at scale | +| Many-objective with grid structure | [GrEA][Grea] | Wins linear / simplex fronts | + +| Question | Use | +|---|---| +| Single-number quality metric for a run | `hypervolume_2d` against a fixed reference | +| "Is run A's front better than B's?" | Same reference point, compare hypervolume | +| "Pick one solution from the front" | See [Pick one answer off a Pareto front](./pick-one.md) | +| Interactive exploration / visualization | See [Explore your results in a webapp](./explorer.md) | + +[Nsga2]: https://docs.rs/heuropt/latest/heuropt/algorithms/nsga2/struct.Nsga2.html +[Nsga3]: https://docs.rs/heuropt/latest/heuropt/algorithms/nsga3/struct.Nsga3.html +[HypE]: https://docs.rs/heuropt/latest/heuropt/algorithms/hype/struct.Hype.html +[Grea]: https://docs.rs/heuropt/latest/heuropt/algorithms/grea/struct.Grea.html +[`EdgeRecombinationCrossover`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.EdgeRecombinationCrossover.html +[`hypervolume_2d`]: https://docs.rs/heuropt/latest/heuropt/metrics/fn.hypervolume_2d.html diff --git a/docs/book/src/cookbook/permutation.md b/docs/book/src/cookbook/permutation.md index 2d1ec42..a314c5c 100644 --- a/docs/book/src/cookbook/permutation.md +++ b/docs/book/src/cookbook/permutation.md @@ -1,13 +1,288 @@ # Optimize a permutation (TSP-style) When your decision is "an ordering" — visiting cities, scheduling -jobs, routing — the natural representation is `Vec` and the -specialized algorithm is [Ant Colony][AntColonyTsp]. Generic alternatives are -[Simulated Annealing][SimulatedAnnealing] + [`SwapMutation`] for any permutation, and -[Tabu Search][TabuSearch] when you have a custom neighbor function. +jobs, routing — the natural representation is `Vec`. heuropt +ships three reasonable starting points: + +- **A purpose-built algorithm** — [Ant Colony][AntColonyTsp] for TSP-shaped + problems with a distance matrix. +- **A genetic algorithm** with the permutation operator toolkit — + the most general option, and the right choice when you want to + bring your own evaluator without a pheromone metaphor. +- **A trajectory method** — [Simulated Annealing][SimulatedAnnealing] + + [`SwapMutation`] for a tiny baseline, or [Tabu Search][TabuSearch] when + you have a custom neighbor function. + +This recipe walks through all three, with the bulk of the page on +the GA toolkit, since it's the most flexible. For the multi-objective +versions (bi-objective TSP, bi-objective JSS, Pareto fronts) see +[Multi-objective combinatorial problems](./multi-objective-combinatorial.md). + +## The permutation operator toolkit + +heuropt ships a complete set of permutation operators in the prelude. +You compose them with [`CompositeVariation`] into a crossover-plus-mutation +pipeline and feed them to any GA-shaped algorithm. + +### Initializers + +| Operator | What it produces | Use for | +|---|---|---| +| [`ShuffledPermutation`] | Random shuffles of `[0..n)` | TSP, QAP, single-machine scheduling — strict permutations | +| [`ShuffledMultisetPermutation`] | Random shuffles of `[0]*r₀ ++ [1]*r₁ ++ …` | Job-shop scheduling operation strings (each job id repeated `n_machines` times) | + +### Crossovers + +All four take two parents and return two children. They assume *strict* +permutations — applying them to multiset encodings (like JSS) will +break the multiset. + +| Operator | One-liner | Best at | +|---|---|---| +| [`OrderCrossover`] (OX) | Copy a random segment from A, fill the rest in B's order | General-purpose, fast | +| [`PartiallyMappedCrossover`] (PMX) | Slide A's segment into B via positional swaps | Classic; preserves more position info than OX | +| [`CycleCrossover`] (CX) | Partition positions into cycles, alternate parents | Preserves the most positional information | +| [`EdgeRecombinationCrossover`] (ERX) | Greedy walk through the union of both parents' edges | The gold standard for TSP — preserves adjacency, not position | + +For TSP specifically, ERX usually wins on Pareto-front quality at the +cost of being ~70% slower per crossover. See +[Multi-objective combinatorial problems](./multi-objective-combinatorial.md) +for a head-to-head comparison. + +### Mutations + +All five take one parent and return one child. All four below preserve +both strict permutations *and* multiset encodings, so they're safe for +JSS too. + +| Operator | What it does | Notes | +|---|---|---| +| [`SwapMutation`] | Swap two random positions | Smallest perturbation; canonical default | +| [`InversionMutation`] | Reverse a random sub-slice | The textbook 2-opt-style move for TSP | +| [`InsertionMutation`] | Remove an element, re-insert elsewhere | Strong for sequencing / scheduling | +| [`ScrambleMutation`] | Randomly permute a random sub-slice | Stronger diversification | + +### Quick "what should I use?" guide + +| Your problem | Initializer | Crossover | Mutation | +|---|---|---|---| +| TSP / routing | `ShuffledPermutation` | `EdgeRecombinationCrossover` | `InversionMutation` | +| Single-machine scheduling | `ShuffledPermutation` | `OrderCrossover` | `InsertionMutation` | +| Generic strict permutation | `ShuffledPermutation` | `OrderCrossover` | `InversionMutation` | +| Job-shop scheduling (multiset) | `ShuffledMultisetPermutation` | *example-local POX* (see below) | `InversionMutation` or `SwapMutation` | + +## Single-objective TSP with a Genetic Algorithm + +This is the toolkit's headline pattern. It mirrors the +`examples/tsp_ulysses16.rs` benchmark, which converges to the known +TSPLIB optimum for the 16-city Ulysses instance. + +```rust,no_run +use heuropt::prelude::*; + +struct Tsp { + distances: Vec>, +} + +impl Problem for Tsp { + type Decision = Vec; + + fn objectives(&self) -> ObjectiveSpace { + ObjectiveSpace::new(vec![Objective::minimize("tour_length")]) + } + + fn evaluate(&self, tour: &Vec) -> Evaluation { + let n = tour.len(); + let mut len = 0.0; + for i in 0..n { + len += self.distances[tour[i]][tour[(i + 1) % n]]; + } + Evaluation::new(vec![len]) + } +} + +fn main() { + // Replace with your actual distance matrix. + let n: usize = 16; + let distances = vec![vec![0.0_f64; n]; n]; + let problem = Tsp { distances }; + + let mut optimizer = GeneticAlgorithm::new( + GeneticAlgorithmConfig { + population_size: 150, + generations: 1500, + tournament_size: 3, + elitism: 4, + seed: 42, + }, + ShuffledPermutation { n }, + CompositeVariation { + crossover: OrderCrossover, + mutation: InversionMutation, + }, + ); + let r = optimizer.run(&problem); + let best = r.best.unwrap(); + println!("best tour length: {:.0}", best.evaluation.objectives[0]); + println!("tour: {:?}", best.decision); +} +``` + +A few things to notice: + +- **Decision type is `Vec`.** Every operator in the toolkit is + generic over the decision type via `Variation>`, so the + whole pipeline composes naturally. +- **`CompositeVariation` is the wiring.** It runs the crossover first, + then runs the mutation on each child. For a single-parent operator + pair (e.g., two mutations stacked), it still works — the "crossover" + slot just becomes a first-stage mutation. +- **Tournament size 3 and elitism 4** are slightly stronger than the + defaults; small permutation GAs benefit from a touch more selection + pressure. + +## Job-shop scheduling — multiset encodings + +JSS problems use a different encoding: a string of length +`n_jobs × n_machines` where each job id appears `n_machines` times. The +k-th occurrence of job `j` represents the k-th operation of job `j`. +This is a *multiset permutation*, not a strict permutation, and the +crossovers above (OX, PMX, CX, ERX) will break it because they assume +each value appears exactly once. + +Use `ShuffledMultisetPermutation` for the initializer: + +```rust,ignore +use heuropt::prelude::*; + +// 6 jobs × 6 machines (FT06 layout): each job id 0..6 appears 6 times. +let initializer = ShuffledMultisetPermutation::new(vec![6; 6]); +``` + +For variation you have two options: + +1. **Mutation only.** The four mutations above all preserve the + multiset, so you can drive a GA with just `InversionMutation` or + `SwapMutation` and skip crossover. This works on small JSS + instances; on larger ones search becomes slow. + +2. **Add a JSS-aware crossover.** The standard choice is **POX** + (Precedence-preserving Order-based Crossover, Bierwirth 1996). + It's not in the library because every JSS instance specifies its + own number of distinct ids and POX needs that constant; defining + it locally per example keeps the type clean: + + ```rust,no_run + use heuropt::prelude::*; + use rand::Rng as _; + + const N_JOBS: usize = 6; + + /// POX — partition job ids into two sets J1/J2; child takes positions + /// of J1-jobs from parent A and fills the remaining positions with + /// J2-jobs from parent B in B's order. Preserves the multiset. + #[derive(Default)] + struct PrecedenceOrderCrossover; + + impl Variation> for PrecedenceOrderCrossover { + fn vary(&mut self, parents: &[Vec], rng: &mut Rng) -> Vec> { + let p1 = &parents[0]; + let p2 = &parents[1]; + let mut in_j1 = [false; N_JOBS]; + loop { + for slot in &mut in_j1 { + *slot = rng.random_bool(0.5); + } + let count = in_j1.iter().filter(|&&b| b).count(); + if count > 0 && count < N_JOBS { break; } + } + vec![pox_child(p1, p2, &in_j1), pox_child(p2, p1, &in_j1)] + } + } + + fn pox_child(donor: &[usize], filler: &[usize], in_donor_set: &[bool]) -> Vec { + let n = donor.len(); + let mut child = vec![usize::MAX; n]; + for k in 0..n { + if in_donor_set[donor[k]] { + child[k] = donor[k]; + } + } + let mut idx = 0; + for &v in filler { + if !in_donor_set[v] { + while idx < n && child[idx] != usize::MAX { idx += 1; } + child[idx] = v; + idx += 1; + } + } + child + } + ``` + + See `examples/jss_ft06_bi.rs` and `examples/mo_jss_la01.rs` for the + complete worked examples. + +A full JSS evaluator walks the schedule string left-to-right, tracking +per-job operation counters and per-machine clocks: + +```rust,ignore +fn evaluate(&self, schedule: &Vec) -> Evaluation { + let mut job_next = [0_usize; N_JOBS]; + let mut job_clock = [0.0_f64; N_JOBS]; + let mut machine_clock = [0.0_f64; N_MACHINES]; + for &job in schedule { + let k = job_next[job]; + let m = ROUTING[job][k]; + let t = PROCESSING_TIME[job][k]; + let start = job_clock[job].max(machine_clock[m]); + let end = start + t; + job_clock[job] = end; + machine_clock[m] = end; + job_next[job] = k + 1; + } + let makespan = machine_clock.iter().cloned().fold(0.0_f64, f64::max); + Evaluation::new(vec![makespan]) +} +``` + +## Comparing crossover operators + +Tuning the right operator combo matters more than tuning population +size. The pattern is: hold everything constant, swap the operator, +record the metric: + +```rust,ignore +use heuropt::prelude::*; +use heuropt::metrics::hypervolume_2d; + +fn run_with>>(crossover: C) -> f64 { + let mut opt = GeneticAlgorithm::new( + GeneticAlgorithmConfig { /* identical config */ ..Default::default() }, + ShuffledPermutation { n: 25 }, + CompositeVariation { crossover, mutation: InversionMutation }, + ); + opt.run(&problem).best.unwrap().evaluation.objectives[0] +} + +println!("OX: {:.0}", run_with(OrderCrossover)); +println!("PMX: {:.0}", run_with(PartiallyMappedCrossover)); +println!("CX: {:.0}", run_with(CycleCrossover)); +println!("ERX: {:.0}", run_with(EdgeRecombinationCrossover)); +``` + +For Pareto-front problems use hypervolume, not single-objective +fitness, as the comparison metric — see +[`examples/tsp_operators_compare.rs`][CompareExample] for the bi-objective +version. ## TSP with Ant Colony +When your problem is genuinely TSP-shaped — symmetric distance matrix, +visit-every-node — Ant Colony is purpose-built and worth a look. It +doesn't use crossover or mutation; instead it deposits pheromone trails +that bias future ants toward good edges. + ```rust,no_run use heuropt::prelude::*; @@ -33,14 +308,7 @@ impl Problem for Tsp { } fn main() { - // 5-city Euclidean instance - let cities = vec![ - (0.0, 0.0), - (1.0, 5.0), - (5.0, 2.0), - (6.0, 6.0), - (8.0, 3.0), - ]; + let cities = vec![(0.0, 0.0), (1.0, 5.0), (5.0, 2.0), (6.0, 6.0), (8.0, 3.0)]; let n = cities.len(); let mut distances = vec![vec![0.0; n]; n]; for i in 0..n { @@ -66,19 +334,18 @@ fn main() { let r = opt.run(&problem); let best = r.best.unwrap(); println!("best tour length: {:.3}", best.evaluation.objectives[0]); - println!("tour: {:?}", best.decision); } ``` -`alpha` weights pheromone influence and `beta` weights the -heuristic (1 / distance). `evaporation` is the per-iteration decay -of pheromone trails. The classic Dorigo paper uses `alpha = 1`, -`beta = 2..5`, `evaporation = 0.1..0.5`. +`alpha` weights pheromone influence and `beta` weights the heuristic +(1 / distance). `evaporation` is the per-iteration pheromone decay. +The classic Dorigo paper uses `alpha = 1`, `beta = 2..5`, +`evaporation = 0.1..0.5`. -## Generic permutation: SA + SwapMutation +## Tiny baseline: SA + SwapMutation -Use this when your problem isn't TSP-shaped (no distance matrix -makes sense) but you still want to optimize an ordering. +The smallest possible permutation optimizer — one starting decision, +no population, one mutation operator. Good as a sanity-check baseline. ```rust,no_run use heuropt::prelude::*; @@ -89,10 +356,9 @@ struct JobShop { impl Problem for JobShop { type Decision = Vec; fn objectives(&self) -> ObjectiveSpace { - ObjectiveSpace::new(vec![Objective::minimize("makespan")]) + ObjectiveSpace::new(vec![Objective::minimize("weighted_completion")]) } fn evaluate(&self, schedule: &Vec) -> Evaluation { - // Pretend cumulative weighted-completion-time. Replace with your real cost. let cost: f64 = schedule.iter().enumerate() .map(|(i, &job)| (i as f64 + 1.0) * self.process_times[job]) .sum(); @@ -100,22 +366,18 @@ impl Problem for JobShop { } } -fn make_initial_perm(n: usize, seed: u64) -> Vec { - use rand::seq::SliceRandom; - let mut rng = rng_from_seed(seed); - let mut perm: Vec = (0..n).collect(); - perm.shuffle(&mut rng); - perm -} - let times = vec![3.0, 1.5, 4.2, 2.7, 5.1]; -let problem = JobShop { process_times: times.clone() }; +let n = times.len(); +let problem = JobShop { process_times: times }; -// SimulatedAnnealing needs a starting decision; pass a custom Initializer. -struct OnePerm(Vec); -impl Initializer> for OnePerm { - fn initialize(&mut self, _size: usize, _rng: &mut Rng) -> Vec> { - vec![self.0.clone()] +// SimulatedAnnealing expects exactly one initial decision. +struct OneShuffle { n: usize } +impl Initializer> for OneShuffle { + fn initialize(&mut self, _size: usize, rng: &mut Rng) -> Vec> { + use rand::seq::SliceRandom; + let mut p: Vec = (0..self.n).collect(); + p.shuffle(rng); + vec![p] } } @@ -126,28 +388,24 @@ let mut opt = SimulatedAnnealing::new( final_temperature: 1e-3, seed: 7, }, - OnePerm(make_initial_perm(times.len(), 7)), + OneShuffle { n }, SwapMutation, ); let r = opt.run(&problem); let best = r.best.unwrap(); -println!("best makespan: {:.3}", best.evaluation.objectives[0]); -println!("schedule: {:?}", best.decision); +println!("best cost: {:.3}", best.evaluation.objectives[0]); ``` -`SwapMutation` swaps two random indices in the permutation — -preserves the "every element appears once" invariant for free. - ## Custom neighborhoods: Tabu Search -When swap isn't the right move set (e.g., 2-opt for TSP, insert / -shift for scheduling), use [Tabu Search][TabuSearch] with your own neighbor -function. +When you want full control of the move set (e.g., systematic 2-opt for +TSP, or insert-and-shift for scheduling), [Tabu Search][TabuSearch] takes +your own neighbor function. ```rust,ignore use heuropt::prelude::*; let neighbors = |x: &Vec, _rng: &mut Rng| -> Vec> { - // Generate all 2-opt neighbors of x. + // All 2-opt neighbors of x. let mut out = Vec::new(); for i in 0..x.len() { for j in (i + 2)..x.len() { @@ -161,7 +419,29 @@ let neighbors = |x: &Vec, _rng: &mut Rng| -> Vec> { // Pass `neighbors` to TabuSearch::new(...). ``` +## When to use which approach + +| Situation | Use | +|---|---| +| TSP-shaped with a distance matrix | [Ant Colony][AntColonyTsp] | +| Generic permutation, multi-seed budget | GA + `ShuffledPermutation` + OX + Inversion | +| Job-shop scheduling | GA + `ShuffledMultisetPermutation` + local POX + Inversion | +| Single-decision baseline | [SimulatedAnnealing][SimulatedAnnealing] + `SwapMutation` | +| Hand-crafted neighborhood (e.g. systematic 2-opt) | [Tabu Search][TabuSearch] | +| Bi-objective / many-objective permutation problem | See [Multi-objective combinatorial](./multi-objective-combinatorial.md) | + [AntColonyTsp]: https://docs.rs/heuropt/latest/heuropt/algorithms/ant_colony_tsp/struct.AntColonyTsp.html [SimulatedAnnealing]: https://docs.rs/heuropt/latest/heuropt/algorithms/simulated_annealing/struct.SimulatedAnnealing.html -[`SwapMutation`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.SwapMutation.html [TabuSearch]: https://docs.rs/heuropt/latest/heuropt/algorithms/tabu_search/struct.TabuSearch.html +[`ShuffledPermutation`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.ShuffledPermutation.html +[`ShuffledMultisetPermutation`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.ShuffledMultisetPermutation.html +[`OrderCrossover`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.OrderCrossover.html +[`PartiallyMappedCrossover`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.PartiallyMappedCrossover.html +[`CycleCrossover`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.CycleCrossover.html +[`EdgeRecombinationCrossover`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.EdgeRecombinationCrossover.html +[`SwapMutation`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.SwapMutation.html +[`InversionMutation`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.InversionMutation.html +[`InsertionMutation`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.InsertionMutation.html +[`ScrambleMutation`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.ScrambleMutation.html +[`CompositeVariation`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.CompositeVariation.html +[CompareExample]: https://github.com/swaits/heuropt/blob/main/examples/tsp_operators_compare.rs