docs(book): permutation toolkit and multi-objective combinatorial cookbook

- Rewrites cookbook/permutation.md to cover the new operator toolkit:
  initializers, crossovers (OX/PMX/CX/ERX), mutations, a 'what should I
  use' picker, and a worked GA-on-TSP example. JSS multiset section
  explains why the strict-permutation crossovers don't compose with
  operation-string encodings and shows the local POX pattern.

- Adds cookbook/multi-objective-combinatorial.md: bi-objective TSP via
  NSGA-II, bi-objective knapsack (binary encoding), 3-objective JSS
  via NSGA-III, and a hypervolume-based operator comparison.

- Updates choosing-an-algorithm.md to reference the new operators in
  the single- and multi-objective decision tables, plus a noting
  NSGA-II/III's genericity over Vec<usize> and Vec<bool> decisions.

- SUMMARY.md and cookbook.md updated to list the new recipe.
This commit is contained in:
2026-05-13 19:43:17 -06:00
parent ab545e268a
commit 6368db74bb
5 changed files with 744 additions and 50 deletions
+1
View File
@@ -16,6 +16,7 @@
- [Tune a model with expensive evaluations](./cookbook/expensive-evaluations.md) - [Tune a model with expensive evaluations](./cookbook/expensive-evaluations.md)
- [Compare two algorithms on your problem](./cookbook/compare.md) - [Compare two algorithms on your problem](./cookbook/compare.md)
- [Optimize a permutation (TSP-style)](./cookbook/permutation.md) - [Optimize a permutation (TSP-style)](./cookbook/permutation.md)
- [Multi-objective combinatorial problems](./cookbook/multi-objective-combinatorial.md)
- [Constrain your search with `Repair`](./cookbook/constraints.md) - [Constrain your search with `Repair`](./cookbook/constraints.md)
- [Pick one answer off a Pareto front](./cookbook/pick-one.md) - [Pick one answer off a Pareto front](./cookbook/pick-one.md)
- [Explore your results in a webapp](./cookbook/explorer.md) - [Explore your results in a webapp](./cookbook/explorer.md)
+30 -2
View File
@@ -103,10 +103,20 @@ optimizer or with the problem).
| `Vec<bool>` | [UMDA][Umda] | Per-bit marginal EDA. Independent-bit assumption. | | `Vec<bool>` | [UMDA][Umda] | Per-bit marginal EDA. Independent-bit assumption. |
| `Vec<bool>` | [GA][GeneticAlgorithm] + [`BitFlipMutation`] | When bit interactions matter. | | `Vec<bool>` | [GA][GeneticAlgorithm] + [`BitFlipMutation`] | When bit interactions matter. |
| `Vec<usize>` (permutation) | [Ant Colony][AntColonyTsp] | TSP-style with a distance matrix. | | `Vec<usize>` (permutation) | [Ant Colony][AntColonyTsp] | TSP-style with a distance matrix. |
| `Vec<usize>` (permutation) | [Simulated Annealing][SimulatedAnnealing] + [`SwapMutation`] | Generic discrete baseline. | | `Vec<usize>` (permutation) | [GA][GeneticAlgorithm] + [`ShuffledPermutation`] + [`OrderCrossover`] + [`InversionMutation`] | Generic permutation GA; use [`EdgeRecombinationCrossover`] for TSP-shaped instances. |
| `Vec<usize>` (JSS multiset) | [GA][GeneticAlgorithm] + [`ShuffledMultisetPermutation`] + local POX + [`InversionMutation`] | Operation-string encoding; see [Optimize a permutation](./cookbook/permutation.md). |
| `Vec<usize>` (permutation) | [Simulated Annealing][SimulatedAnnealing] + [`SwapMutation`] | One-decision baseline. |
| `Vec<usize>` or custom | [Tabu Search][TabuSearch] | You supply the neighbor function. | | `Vec<usize>` or custom | [Tabu Search][TabuSearch] | You supply the neighbor function. |
| Custom struct | [Simulated Annealing][SimulatedAnnealing] / [Hill Climber][HillClimber] | With your own `Variation` impl. | | Custom struct | [Simulated Annealing][SimulatedAnnealing] / [Hill Climber][HillClimber] | With your own `Variation` impl. |
heuropt's permutation operator toolkit covers four crossovers
([`OrderCrossover`], [`PartiallyMappedCrossover`], [`CycleCrossover`],
[`EdgeRecombinationCrossover`]) and four mutations ([`SwapMutation`],
[`InversionMutation`], [`InsertionMutation`], [`ScrambleMutation`]),
plus two initializers for strict and multiset permutations. See
[Optimize a permutation](./cookbook/permutation.md) for the full
picker.
## Step 2 — multi-objective (2 or 3) ## Step 2 — multi-objective (2 or 3)
### Strong default ### Strong default
@@ -115,6 +125,12 @@ optimizer or with the problem).
maintains diversity via crowding distance. On the harness it lands maintains diversity via crowding distance. On the harness it lands
on the Pareto front of every test problem. on the Pareto front of every test problem.
NSGA-II is generic over the decision type — drop in
[`ShuffledPermutation`] + a permutation crossover and it solves
bi-objective TSP; drop in a binary initializer and [`BitFlipMutation`]
and it solves bi-objective knapsack. See
[Multi-objective combinatorial problems](./cookbook/multi-objective-combinatorial.md).
### Real-valued, smooth front, want best convergence ### Real-valued, smooth front, want best convergence
[MOPSO][Mopso] (multi-objective PSO with archive). On ZDT1 it wins [MOPSO][Mopso] (multi-objective PSO with archive). On ZDT1 it wins
@@ -244,7 +260,10 @@ method on every algorithm in the catalog. See the
| Disconnected / non-convex front | [IBEA][Ibea] | | Disconnected / non-convex front | [IBEA][Ibea] |
| Many-objective default (curved front) | [NSGA-III][Nsga3] | | Many-objective default (curved front) | [NSGA-III][Nsga3] |
| Many-objective linear / simplex front | [GrEA][Grea] | | Many-objective linear / simplex front | [GrEA][Grea] |
| Permutation problem | [Ant Colony][AntColonyTsp] | | Permutation problem (TSP with distance matrix) | [Ant Colony][AntColonyTsp] |
| Generic permutation problem | [GA][GeneticAlgorithm] + permutation toolkit |
| Bi-objective combinatorial (TSP / scheduling / knapsack) | [NSGA-II][Nsga2] + matching encoding operators |
| 3-objective combinatorial | [NSGA-III][Nsga3] + matching encoding operators |
| Binary problem | [UMDA][Umda] | | Binary problem | [UMDA][Umda] |
| Custom decision type | [Simulated Annealing][SimulatedAnnealing] + your `Variation` | | Custom decision type | [Simulated Annealing][SimulatedAnnealing] + your `Variation` |
| Sanity baseline | [Random Search][RandomSearch] | | Sanity baseline | [Random Search][RandomSearch] |
@@ -268,6 +287,15 @@ method on every algorithm in the catalog. See the
[`BitFlipMutation`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.BitFlipMutation.html [`BitFlipMutation`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.BitFlipMutation.html
[AntColonyTsp]: https://docs.rs/heuropt/latest/heuropt/algorithms/ant_colony_tsp/struct.AntColonyTsp.html [AntColonyTsp]: https://docs.rs/heuropt/latest/heuropt/algorithms/ant_colony_tsp/struct.AntColonyTsp.html
[`SwapMutation`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.SwapMutation.html [`SwapMutation`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.SwapMutation.html
[`InversionMutation`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.InversionMutation.html
[`InsertionMutation`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.InsertionMutation.html
[`ScrambleMutation`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.ScrambleMutation.html
[`OrderCrossover`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.OrderCrossover.html
[`PartiallyMappedCrossover`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.PartiallyMappedCrossover.html
[`CycleCrossover`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.CycleCrossover.html
[`EdgeRecombinationCrossover`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.EdgeRecombinationCrossover.html
[`ShuffledPermutation`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.ShuffledPermutation.html
[`ShuffledMultisetPermutation`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.ShuffledMultisetPermutation.html
[TabuSearch]: https://docs.rs/heuropt/latest/heuropt/algorithms/tabu_search/struct.TabuSearch.html [TabuSearch]: https://docs.rs/heuropt/latest/heuropt/algorithms/tabu_search/struct.TabuSearch.html
[Nsga2]: https://docs.rs/heuropt/latest/heuropt/algorithms/nsga2/struct.Nsga2.html [Nsga2]: https://docs.rs/heuropt/latest/heuropt/algorithms/nsga2/struct.Nsga2.html
[Nsga3]: https://docs.rs/heuropt/latest/heuropt/algorithms/nsga3/struct.Nsga3.html [Nsga3]: https://docs.rs/heuropt/latest/heuropt/algorithms/nsga3/struct.Nsga3.html
+5 -1
View File
@@ -19,7 +19,11 @@ project.
- [Compare two algorithms on your problem](./cookbook/compare.md) — - [Compare two algorithms on your problem](./cookbook/compare.md) —
multi-seed harness pattern straight from `examples/compare.rs`. multi-seed harness pattern straight from `examples/compare.rs`.
- [Optimize a permutation (TSP-style)](./cookbook/permutation.md) — - [Optimize a permutation (TSP-style)](./cookbook/permutation.md) —
Ant Colony with a distance matrix. the permutation operator toolkit (OX / PMX / CX / ERX + Inversion /
Insertion / Scramble), plus Ant Colony for distance-matrix TSP.
- [Multi-objective combinatorial problems](./cookbook/multi-objective-combinatorial.md)
— bi-objective TSP, bi-objective knapsack (`Vec<bool>`), and
3-objective JSS via NSGA-II / NSGA-III.
- [Constrain your search with `Repair`](./cookbook/constraints.md) — - [Constrain your search with `Repair`](./cookbook/constraints.md) —
bounds, simplex projection, custom repair. bounds, simplex projection, custom repair.
- [Pick one answer off a Pareto front](./cookbook/pick-one.md) — the - [Pick one answer off a Pareto front](./cookbook/pick-one.md) — the
@@ -0,0 +1,381 @@
# Multi-objective combinatorial problems
Real combinatorial problems usually have more than one cost. A TSP
where every edge has both *distance* and *time*; a job-shop where you
care about *makespan*, *flow time*, *and* *tardiness*; a knapsack
with two profit metrics and a single weight budget. The decision
type is still combinatorial — a permutation, a bitstring — but the
objective is a vector, and the answer is a Pareto front rather than
a single best.
heuropt's NSGA-II and NSGA-III are fully generic over the decision
type. You don't need a separate "combinatorial NSGA" — just plug in
the right initializer and variation operators for your encoding.
This recipe walks through three patterns:
- **Bi-objective TSP** with NSGA-II (Pareto front of two distance
matrices over the same cities)
- **Bi-objective 0/1 knapsack** with NSGA-II (binary encoding)
- **3-objective JSS** with NSGA-III (the many-objective successor)
For the single-objective permutation toolkit it builds on, see
[Optimize a permutation](./permutation.md).
## Bi-objective TSP
This is the canonical multi-objective combinatorial benchmark
(LustTeghem 2010). Two TSP instances on the **same** city set define
two distance matrices A and B; the search trades off length under A
versus length under B.
```rust,no_run
use heuropt::prelude::*;
use heuropt::metrics::hypervolume_2d;
struct BiObjectiveTsp {
dist_a: Vec<Vec<f64>>,
dist_b: Vec<Vec<f64>>,
}
impl BiObjectiveTsp {
fn tour_length(d: &[Vec<f64>], tour: &[usize]) -> f64 {
let n = tour.len();
let mut total = 0.0;
for i in 0..n {
total += d[tour[i]][tour[(i + 1) % n]];
}
total
}
}
impl Problem for BiObjectiveTsp {
type Decision = Vec<usize>;
fn objectives(&self) -> ObjectiveSpace {
ObjectiveSpace::new(vec![
Objective::minimize("length_A"),
Objective::minimize("length_B"),
])
}
fn evaluate(&self, tour: &Vec<usize>) -> Evaluation {
Evaluation::new(vec![
Self::tour_length(&self.dist_a, tour),
Self::tour_length(&self.dist_b, tour),
])
}
}
fn main() {
let n: usize = 25;
let dist_a = vec![vec![0.0_f64; n]; n]; // your matrix A
let dist_b = vec![vec![0.0_f64; n]; n]; // your matrix B
let problem = BiObjectiveTsp { dist_a, dist_b };
let mut optimizer = Nsga2::new(
Nsga2Config {
population_size: 200,
generations: 600,
seed: 11,
},
ShuffledPermutation { n },
CompositeVariation {
crossover: EdgeRecombinationCrossover,
mutation: InversionMutation,
},
);
let result = optimizer.run(&problem);
println!("Pareto-front size: {}", result.pareto_front.len());
// Hypervolume against a generous reference point (larger than any
// length you'd reasonably see). Use this as the single-number
// quality metric for the run.
let ref_point = [40_000.0, 40_000.0];
let hv = hypervolume_2d(&result.pareto_front, &problem.objectives(), ref_point);
println!("Hypervolume vs. {:?}: {:.0}", ref_point, hv);
}
```
[`EdgeRecombinationCrossover`] (ERX) is the standout crossover for
TSP. On a 25-city bi-objective instance it produces about twice the
front diversity of OX, PMX, or CX — see
`examples/tsp_operators_compare.rs` for a head-to-head benchmark.
## Bi-objective 0/1 knapsack — `Vec<bool>` decisions
NSGA-II works over `Vec<bool>` the same way. The ZitzlerThiele
bi-objective knapsack is the textbook benchmark: each item has two
profit values and a single weight; you maximize both profits under
one capacity constraint.
```rust,no_run
use heuropt::prelude::*;
use rand::Rng as _;
const N_ITEMS: usize = 30;
struct BiKnapsack {
profits_a: Vec<f64>,
profits_b: Vec<f64>,
weights: Vec<f64>,
capacity: f64,
}
impl Problem for BiKnapsack {
type Decision = Vec<bool>;
fn objectives(&self) -> ObjectiveSpace {
ObjectiveSpace::new(vec![
Objective::maximize("profit_A"),
Objective::maximize("profit_B"),
])
}
fn evaluate(&self, take: &Vec<bool>) -> Evaluation {
let (pa, pb, w) = take.iter().enumerate().fold(
(0.0_f64, 0.0_f64, 0.0_f64),
|(pa, pb, w), (i, &t)| {
if t {
(pa + self.profits_a[i], pb + self.profits_b[i], w + self.weights[i])
} else {
(pa, pb, w)
}
},
);
// Standard heuristic-MO constraint handling: penalize weight
// overruns heavily so the recovered front is feasible.
let penalty = 1000.0 * (w - self.capacity).max(0.0);
Evaluation::new(vec![pa - penalty, pb - penalty])
}
}
/// Each bit 50/50 independently.
#[derive(Clone, Copy)]
struct RandomBinary { n: usize }
impl Initializer<Vec<bool>> for RandomBinary {
fn initialize(&mut self, size: usize, rng: &mut Rng) -> Vec<Vec<bool>> {
(0..size).map(|_| (0..self.n).map(|_| rng.random_bool(0.5)).collect()).collect()
}
}
/// One-point crossover for binary chromosomes.
#[derive(Default)]
struct OnePointCrossoverBool;
impl Variation<Vec<bool>> for OnePointCrossoverBool {
fn vary(&mut self, parents: &[Vec<bool>], rng: &mut Rng) -> Vec<Vec<bool>> {
let (p1, p2) = (&parents[0], &parents[1]);
let n = p1.len();
let cut = rng.random_range(1..n);
let mut c1 = Vec::with_capacity(n);
let mut c2 = Vec::with_capacity(n);
c1.extend_from_slice(&p1[..cut]); c1.extend_from_slice(&p2[cut..]);
c2.extend_from_slice(&p2[..cut]); c2.extend_from_slice(&p1[cut..]);
vec![c1, c2]
}
}
fn main() {
# let profits_a = vec![0.0; N_ITEMS];
# let profits_b = vec![0.0; N_ITEMS];
# let weights = vec![0.0; N_ITEMS];
let problem = BiKnapsack {
profits_a, profits_b, weights,
capacity: 750.0, // ~half the total weight
};
let mut optimizer = Nsga2::new(
Nsga2Config {
population_size: 120,
generations: 400,
seed: 19,
},
RandomBinary { n: N_ITEMS },
CompositeVariation {
crossover: OnePointCrossoverBool,
mutation: BitFlipMutation { probability: 1.0 / N_ITEMS as f64 },
},
);
let result = optimizer.run(&problem);
println!("Pareto-front size: {}", result.pareto_front.len());
}
```
Two things worth noting:
- **`OnePointCrossoverBool` and `RandomBinary` are defined locally.**
They're tiny and common — a future PR could lift them into the
library, but for now you write them inline.
- **Constraint handling is a penalty.** The factor `1000.0` is chosen
so that even a 1-unit overrun beats any feasible solution by more
than the entire profit range; the recovered front is entirely
feasible. This is the standard heuristic-MO pattern (Deb 2001) and
cheaper than a hard repair operator.
## Three-objective JSS with NSGA-III
NSGA-III is designed for ≥ 3 objectives. NSGA-II's crowding distance
degrades when most of the population is mutually non-dominated, which
is the rule rather than the exception in higher dimensions; NSGA-III
uses reference-point niching instead.
The example below adds *tardiness* to the standard (makespan, flow
time) JSS pair. Tardiness needs due dates; the common heuristic is
`dⱼ = 1.3 × sum_of_processing_times(j)`.
```rust,no_run
use heuropt::prelude::*;
use rand::Rng as _;
const N_JOBS: usize = 10;
const N_MACHINES: usize = 5;
struct La01ThreeObjective {
routing: [[usize; N_MACHINES]; N_JOBS],
times: [[f64; N_MACHINES]; N_JOBS],
due: [f64; N_JOBS],
}
impl Problem for La01ThreeObjective {
type Decision = Vec<usize>;
fn objectives(&self) -> ObjectiveSpace {
ObjectiveSpace::new(vec![
Objective::minimize("makespan"),
Objective::minimize("total_flow_time"),
Objective::minimize("total_tardiness"),
])
}
fn evaluate(&self, schedule: &Vec<usize>) -> Evaluation {
let mut job_next = [0_usize; N_JOBS];
let mut job_clock = [0.0_f64; N_JOBS];
let mut machine_clock = [0.0_f64; N_MACHINES];
for &job in schedule {
let k = job_next[job];
let m = self.routing[job][k];
let t = self.times[job][k];
let start = job_clock[job].max(machine_clock[m]);
let end = start + t;
job_clock[job] = end;
machine_clock[m] = end;
job_next[job] = k + 1;
}
let makespan = machine_clock.iter().cloned().fold(0.0_f64, f64::max);
let flow_time: f64 = job_clock.iter().sum();
let tardiness: f64 = job_clock.iter().zip(self.due.iter())
.map(|(&c, &d)| (c - d).max(0.0))
.sum();
Evaluation::new(vec![makespan, flow_time, tardiness])
}
}
/// Mix Insertion and Scramble per call — both preserve the multiset,
/// giving the search access to two complementary neighborhood moves.
#[derive(Default)]
struct InsertionOrScramble;
impl Variation<Vec<usize>> for InsertionOrScramble {
fn vary(&mut self, parents: &[Vec<usize>], rng: &mut Rng) -> Vec<Vec<usize>> {
if rng.random_bool(0.5) {
InsertionMutation.vary(parents, rng)
} else {
ScrambleMutation.vary(parents, rng)
}
}
}
fn main() {
# let routing = [[0; N_MACHINES]; N_JOBS];
# let times = [[0.0; N_MACHINES]; N_JOBS];
let due = std::array::from_fn::<f64, N_JOBS, _>(
|j| 1.3 * times[j].iter().sum::<f64>(),
);
let problem = La01ThreeObjective { routing, times, due };
let mut optimizer = Nsga3::new(
Nsga3Config {
population_size: 120,
generations: 600,
reference_divisions: 12, // 91 Das-Dennis points in 3-D
seed: 9,
},
ShuffledMultisetPermutation::new(vec![N_MACHINES; N_JOBS]),
// Drop in a local PrecedenceOrderCrossover (POX) here for the
// crossover slot if you want stronger mixing; see the
// permutation recipe for the implementation.
InsertionOrScramble,
);
let result = optimizer.run(&problem);
println!("Pareto-front size: {}", result.pareto_front.len());
}
```
A few NSGA-III tips:
- **`reference_divisions` controls how many reference points the
algorithm spreads across the front.** For M objectives, the DasDennis
construction produces `C(divisions + M - 1, M - 1)` reference points.
For M = 3 and divisions = 12 that's 91 points; pick a population size
≥ that.
- **`PrecedenceOrderCrossover` (POX)** belongs in the crossover slot
for JSS. The strict-permutation crossovers (OX, PMX, CX, ERX) break
the operation-string multiset. See
[Optimize a permutation](./permutation.md#job-shop-scheduling-multiset-encodings)
for the local POX definition.
## Comparing operators by hypervolume
For Pareto-front problems, single-objective fitness is the wrong
comparison metric. Use **hypervolume** instead — the dominated area
under the front, against a fixed reference point.
```rust,ignore
use heuropt::metrics::hypervolume_2d;
let ref_point = [40_000.0, 40_000.0]; // worse than anything you expect
for (name, crossover) in &[
("OX", Box::new(OrderCrossover) as Box<dyn Variation<Vec<usize>>>),
("PMX", Box::new(PartiallyMappedCrossover) as _),
("CX", Box::new(CycleCrossover) as _),
("ERX", Box::new(EdgeRecombinationCrossover) as _),
] {
let result = run_nsga2_with_crossover(crossover);
let hv = hypervolume_2d(&result.pareto_front, &problem.objectives(), ref_point);
println!("{name:>3}: hv = {hv:.0}");
}
```
This is the pattern in `examples/tsp_operators_compare.rs`. On the
KroAB-25 instance it ranks ERX > OX > PMX > CX by hypervolume.
For ≥ 3 objectives, hypervolume in N dimensions is exponentially
expensive; use [`hypervolume_2d`] when you can collapse to two
objectives for the metric, or sample-based hypervolume from [`HypE`]
otherwise.
## Pareto-front tips
| Problem | Algorithm | Notes |
|---|---|---|
| 2 objectives, permutation | [Nsga2][Nsga2] | Strong default |
| 2 objectives, binary | [Nsga2][Nsga2] | Same machinery, different encoding |
| 3 objectives | [Nsga3][Nsga3] | NSGA-II's crowding distance starts to degrade |
| 4+ objectives | [Nsga3][Nsga3] or [HypE][HypE] | NSGA-III if front is curved; HypE for indicator-based at scale |
| Many-objective with grid structure | [GrEA][Grea] | Wins linear / simplex fronts |
| Question | Use |
|---|---|
| Single-number quality metric for a run | `hypervolume_2d` against a fixed reference |
| "Is run A's front better than B's?" | Same reference point, compare hypervolume |
| "Pick one solution from the front" | See [Pick one answer off a Pareto front](./pick-one.md) |
| Interactive exploration / visualization | See [Explore your results in a webapp](./explorer.md) |
[Nsga2]: https://docs.rs/heuropt/latest/heuropt/algorithms/nsga2/struct.Nsga2.html
[Nsga3]: https://docs.rs/heuropt/latest/heuropt/algorithms/nsga3/struct.Nsga3.html
[HypE]: https://docs.rs/heuropt/latest/heuropt/algorithms/hype/struct.Hype.html
[Grea]: https://docs.rs/heuropt/latest/heuropt/algorithms/grea/struct.Grea.html
[`EdgeRecombinationCrossover`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.EdgeRecombinationCrossover.html
[`hypervolume_2d`]: https://docs.rs/heuropt/latest/heuropt/metrics/fn.hypervolume_2d.html
+327 -47
View File
@@ -1,13 +1,288 @@
# Optimize a permutation (TSP-style) # Optimize a permutation (TSP-style)
When your decision is "an ordering" — visiting cities, scheduling When your decision is "an ordering" — visiting cities, scheduling
jobs, routing — the natural representation is `Vec<usize>` and the jobs, routing — the natural representation is `Vec<usize>`. heuropt
specialized algorithm is [Ant Colony][AntColonyTsp]. Generic alternatives are ships three reasonable starting points:
[Simulated Annealing][SimulatedAnnealing] + [`SwapMutation`] for any permutation, and
[Tabu Search][TabuSearch] when you have a custom neighbor function. - **A purpose-built algorithm** — [Ant Colony][AntColonyTsp] for TSP-shaped
problems with a distance matrix.
- **A genetic algorithm** with the permutation operator toolkit —
the most general option, and the right choice when you want to
bring your own evaluator without a pheromone metaphor.
- **A trajectory method** — [Simulated Annealing][SimulatedAnnealing] +
[`SwapMutation`] for a tiny baseline, or [Tabu Search][TabuSearch] when
you have a custom neighbor function.
This recipe walks through all three, with the bulk of the page on
the GA toolkit, since it's the most flexible. For the multi-objective
versions (bi-objective TSP, bi-objective JSS, Pareto fronts) see
[Multi-objective combinatorial problems](./multi-objective-combinatorial.md).
## The permutation operator toolkit
heuropt ships a complete set of permutation operators in the prelude.
You compose them with [`CompositeVariation`] into a crossover-plus-mutation
pipeline and feed them to any GA-shaped algorithm.
### Initializers
| Operator | What it produces | Use for |
|---|---|---|
| [`ShuffledPermutation`] | Random shuffles of `[0..n)` | TSP, QAP, single-machine scheduling — strict permutations |
| [`ShuffledMultisetPermutation`] | Random shuffles of `[0]*r₀ ++ [1]*r₁ ++ …` | Job-shop scheduling operation strings (each job id repeated `n_machines` times) |
### Crossovers
All four take two parents and return two children. They assume *strict*
permutations — applying them to multiset encodings (like JSS) will
break the multiset.
| Operator | One-liner | Best at |
|---|---|---|
| [`OrderCrossover`] (OX) | Copy a random segment from A, fill the rest in B's order | General-purpose, fast |
| [`PartiallyMappedCrossover`] (PMX) | Slide A's segment into B via positional swaps | Classic; preserves more position info than OX |
| [`CycleCrossover`] (CX) | Partition positions into cycles, alternate parents | Preserves the most positional information |
| [`EdgeRecombinationCrossover`] (ERX) | Greedy walk through the union of both parents' edges | The gold standard for TSP — preserves adjacency, not position |
For TSP specifically, ERX usually wins on Pareto-front quality at the
cost of being ~70% slower per crossover. See
[Multi-objective combinatorial problems](./multi-objective-combinatorial.md)
for a head-to-head comparison.
### Mutations
All five take one parent and return one child. All four below preserve
both strict permutations *and* multiset encodings, so they're safe for
JSS too.
| Operator | What it does | Notes |
|---|---|---|
| [`SwapMutation`] | Swap two random positions | Smallest perturbation; canonical default |
| [`InversionMutation`] | Reverse a random sub-slice | The textbook 2-opt-style move for TSP |
| [`InsertionMutation`] | Remove an element, re-insert elsewhere | Strong for sequencing / scheduling |
| [`ScrambleMutation`] | Randomly permute a random sub-slice | Stronger diversification |
### Quick "what should I use?" guide
| Your problem | Initializer | Crossover | Mutation |
|---|---|---|---|
| TSP / routing | `ShuffledPermutation` | `EdgeRecombinationCrossover` | `InversionMutation` |
| Single-machine scheduling | `ShuffledPermutation` | `OrderCrossover` | `InsertionMutation` |
| Generic strict permutation | `ShuffledPermutation` | `OrderCrossover` | `InversionMutation` |
| Job-shop scheduling (multiset) | `ShuffledMultisetPermutation` | *example-local POX* (see below) | `InversionMutation` or `SwapMutation` |
## Single-objective TSP with a Genetic Algorithm
This is the toolkit's headline pattern. It mirrors the
`examples/tsp_ulysses16.rs` benchmark, which converges to the known
TSPLIB optimum for the 16-city Ulysses instance.
```rust,no_run
use heuropt::prelude::*;
struct Tsp {
distances: Vec<Vec<f64>>,
}
impl Problem for Tsp {
type Decision = Vec<usize>;
fn objectives(&self) -> ObjectiveSpace {
ObjectiveSpace::new(vec![Objective::minimize("tour_length")])
}
fn evaluate(&self, tour: &Vec<usize>) -> Evaluation {
let n = tour.len();
let mut len = 0.0;
for i in 0..n {
len += self.distances[tour[i]][tour[(i + 1) % n]];
}
Evaluation::new(vec![len])
}
}
fn main() {
// Replace with your actual distance matrix.
let n: usize = 16;
let distances = vec![vec![0.0_f64; n]; n];
let problem = Tsp { distances };
let mut optimizer = GeneticAlgorithm::new(
GeneticAlgorithmConfig {
population_size: 150,
generations: 1500,
tournament_size: 3,
elitism: 4,
seed: 42,
},
ShuffledPermutation { n },
CompositeVariation {
crossover: OrderCrossover,
mutation: InversionMutation,
},
);
let r = optimizer.run(&problem);
let best = r.best.unwrap();
println!("best tour length: {:.0}", best.evaluation.objectives[0]);
println!("tour: {:?}", best.decision);
}
```
A few things to notice:
- **Decision type is `Vec<usize>`.** Every operator in the toolkit is
generic over the decision type via `Variation<Vec<usize>>`, so the
whole pipeline composes naturally.
- **`CompositeVariation` is the wiring.** It runs the crossover first,
then runs the mutation on each child. For a single-parent operator
pair (e.g., two mutations stacked), it still works — the "crossover"
slot just becomes a first-stage mutation.
- **Tournament size 3 and elitism 4** are slightly stronger than the
defaults; small permutation GAs benefit from a touch more selection
pressure.
## Job-shop scheduling — multiset encodings
JSS problems use a different encoding: a string of length
`n_jobs × n_machines` where each job id appears `n_machines` times. The
k-th occurrence of job `j` represents the k-th operation of job `j`.
This is a *multiset permutation*, not a strict permutation, and the
crossovers above (OX, PMX, CX, ERX) will break it because they assume
each value appears exactly once.
Use `ShuffledMultisetPermutation` for the initializer:
```rust,ignore
use heuropt::prelude::*;
// 6 jobs × 6 machines (FT06 layout): each job id 0..6 appears 6 times.
let initializer = ShuffledMultisetPermutation::new(vec![6; 6]);
```
For variation you have two options:
1. **Mutation only.** The four mutations above all preserve the
multiset, so you can drive a GA with just `InversionMutation` or
`SwapMutation` and skip crossover. This works on small JSS
instances; on larger ones search becomes slow.
2. **Add a JSS-aware crossover.** The standard choice is **POX**
(Precedence-preserving Order-based Crossover, Bierwirth 1996).
It's not in the library because every JSS instance specifies its
own number of distinct ids and POX needs that constant; defining
it locally per example keeps the type clean:
```rust,no_run
use heuropt::prelude::*;
use rand::Rng as _;
const N_JOBS: usize = 6;
/// POX — partition job ids into two sets J1/J2; child takes positions
/// of J1-jobs from parent A and fills the remaining positions with
/// J2-jobs from parent B in B's order. Preserves the multiset.
#[derive(Default)]
struct PrecedenceOrderCrossover;
impl Variation<Vec<usize>> for PrecedenceOrderCrossover {
fn vary(&mut self, parents: &[Vec<usize>], rng: &mut Rng) -> Vec<Vec<usize>> {
let p1 = &parents[0];
let p2 = &parents[1];
let mut in_j1 = [false; N_JOBS];
loop {
for slot in &mut in_j1 {
*slot = rng.random_bool(0.5);
}
let count = in_j1.iter().filter(|&&b| b).count();
if count > 0 && count < N_JOBS { break; }
}
vec![pox_child(p1, p2, &in_j1), pox_child(p2, p1, &in_j1)]
}
}
fn pox_child(donor: &[usize], filler: &[usize], in_donor_set: &[bool]) -> Vec<usize> {
let n = donor.len();
let mut child = vec![usize::MAX; n];
for k in 0..n {
if in_donor_set[donor[k]] {
child[k] = donor[k];
}
}
let mut idx = 0;
for &v in filler {
if !in_donor_set[v] {
while idx < n && child[idx] != usize::MAX { idx += 1; }
child[idx] = v;
idx += 1;
}
}
child
}
```
See `examples/jss_ft06_bi.rs` and `examples/mo_jss_la01.rs` for the
complete worked examples.
A full JSS evaluator walks the schedule string left-to-right, tracking
per-job operation counters and per-machine clocks:
```rust,ignore
fn evaluate(&self, schedule: &Vec<usize>) -> Evaluation {
let mut job_next = [0_usize; N_JOBS];
let mut job_clock = [0.0_f64; N_JOBS];
let mut machine_clock = [0.0_f64; N_MACHINES];
for &job in schedule {
let k = job_next[job];
let m = ROUTING[job][k];
let t = PROCESSING_TIME[job][k];
let start = job_clock[job].max(machine_clock[m]);
let end = start + t;
job_clock[job] = end;
machine_clock[m] = end;
job_next[job] = k + 1;
}
let makespan = machine_clock.iter().cloned().fold(0.0_f64, f64::max);
Evaluation::new(vec![makespan])
}
```
## Comparing crossover operators
Tuning the right operator combo matters more than tuning population
size. The pattern is: hold everything constant, swap the operator,
record the metric:
```rust,ignore
use heuropt::prelude::*;
use heuropt::metrics::hypervolume_2d;
fn run_with<C: Variation<Vec<usize>>>(crossover: C) -> f64 {
let mut opt = GeneticAlgorithm::new(
GeneticAlgorithmConfig { /* identical config */ ..Default::default() },
ShuffledPermutation { n: 25 },
CompositeVariation { crossover, mutation: InversionMutation },
);
opt.run(&problem).best.unwrap().evaluation.objectives[0]
}
println!("OX: {:.0}", run_with(OrderCrossover));
println!("PMX: {:.0}", run_with(PartiallyMappedCrossover));
println!("CX: {:.0}", run_with(CycleCrossover));
println!("ERX: {:.0}", run_with(EdgeRecombinationCrossover));
```
For Pareto-front problems use hypervolume, not single-objective
fitness, as the comparison metric — see
[`examples/tsp_operators_compare.rs`][CompareExample] for the bi-objective
version.
## TSP with Ant Colony ## TSP with Ant Colony
When your problem is genuinely TSP-shaped — symmetric distance matrix,
visit-every-node — Ant Colony is purpose-built and worth a look. It
doesn't use crossover or mutation; instead it deposits pheromone trails
that bias future ants toward good edges.
```rust,no_run ```rust,no_run
use heuropt::prelude::*; use heuropt::prelude::*;
@@ -33,14 +308,7 @@ impl Problem for Tsp {
} }
fn main() { fn main() {
// 5-city Euclidean instance let cities = vec![(0.0, 0.0), (1.0, 5.0), (5.0, 2.0), (6.0, 6.0), (8.0, 3.0)];
let cities = vec![
(0.0, 0.0),
(1.0, 5.0),
(5.0, 2.0),
(6.0, 6.0),
(8.0, 3.0),
];
let n = cities.len(); let n = cities.len();
let mut distances = vec![vec![0.0; n]; n]; let mut distances = vec![vec![0.0; n]; n];
for i in 0..n { for i in 0..n {
@@ -66,19 +334,18 @@ fn main() {
let r = opt.run(&problem); let r = opt.run(&problem);
let best = r.best.unwrap(); let best = r.best.unwrap();
println!("best tour length: {:.3}", best.evaluation.objectives[0]); println!("best tour length: {:.3}", best.evaluation.objectives[0]);
println!("tour: {:?}", best.decision);
} }
``` ```
`alpha` weights pheromone influence and `beta` weights the `alpha` weights pheromone influence and `beta` weights the heuristic
heuristic (1 / distance). `evaporation` is the per-iteration decay (1 / distance). `evaporation` is the per-iteration pheromone decay.
of pheromone trails. The classic Dorigo paper uses `alpha = 1`, The classic Dorigo paper uses `alpha = 1`, `beta = 2..5`,
`beta = 2..5`, `evaporation = 0.1..0.5`. `evaporation = 0.1..0.5`.
## Generic permutation: SA + SwapMutation ## Tiny baseline: SA + SwapMutation
Use this when your problem isn't TSP-shaped (no distance matrix The smallest possible permutation optimizer — one starting decision,
makes sense) but you still want to optimize an ordering. no population, one mutation operator. Good as a sanity-check baseline.
```rust,no_run ```rust,no_run
use heuropt::prelude::*; use heuropt::prelude::*;
@@ -89,10 +356,9 @@ struct JobShop {
impl Problem for JobShop { impl Problem for JobShop {
type Decision = Vec<usize>; type Decision = Vec<usize>;
fn objectives(&self) -> ObjectiveSpace { fn objectives(&self) -> ObjectiveSpace {
ObjectiveSpace::new(vec![Objective::minimize("makespan")]) ObjectiveSpace::new(vec![Objective::minimize("weighted_completion")])
} }
fn evaluate(&self, schedule: &Vec<usize>) -> Evaluation { fn evaluate(&self, schedule: &Vec<usize>) -> Evaluation {
// Pretend cumulative weighted-completion-time. Replace with your real cost.
let cost: f64 = schedule.iter().enumerate() let cost: f64 = schedule.iter().enumerate()
.map(|(i, &job)| (i as f64 + 1.0) * self.process_times[job]) .map(|(i, &job)| (i as f64 + 1.0) * self.process_times[job])
.sum(); .sum();
@@ -100,22 +366,18 @@ impl Problem for JobShop {
} }
} }
fn make_initial_perm(n: usize, seed: u64) -> Vec<usize> {
use rand::seq::SliceRandom;
let mut rng = rng_from_seed(seed);
let mut perm: Vec<usize> = (0..n).collect();
perm.shuffle(&mut rng);
perm
}
let times = vec![3.0, 1.5, 4.2, 2.7, 5.1]; let times = vec![3.0, 1.5, 4.2, 2.7, 5.1];
let problem = JobShop { process_times: times.clone() }; let n = times.len();
let problem = JobShop { process_times: times };
// SimulatedAnnealing needs a starting decision; pass a custom Initializer. // SimulatedAnnealing expects exactly one initial decision.
struct OnePerm(Vec<usize>); struct OneShuffle { n: usize }
impl Initializer<Vec<usize>> for OnePerm { impl Initializer<Vec<usize>> for OneShuffle {
fn initialize(&mut self, _size: usize, _rng: &mut Rng) -> Vec<Vec<usize>> { fn initialize(&mut self, _size: usize, rng: &mut Rng) -> Vec<Vec<usize>> {
vec![self.0.clone()] use rand::seq::SliceRandom;
let mut p: Vec<usize> = (0..self.n).collect();
p.shuffle(rng);
vec![p]
} }
} }
@@ -126,28 +388,24 @@ let mut opt = SimulatedAnnealing::new(
final_temperature: 1e-3, final_temperature: 1e-3,
seed: 7, seed: 7,
}, },
OnePerm(make_initial_perm(times.len(), 7)), OneShuffle { n },
SwapMutation, SwapMutation,
); );
let r = opt.run(&problem); let r = opt.run(&problem);
let best = r.best.unwrap(); let best = r.best.unwrap();
println!("best makespan: {:.3}", best.evaluation.objectives[0]); println!("best cost: {:.3}", best.evaluation.objectives[0]);
println!("schedule: {:?}", best.decision);
``` ```
`SwapMutation` swaps two random indices in the permutation —
preserves the "every element appears once" invariant for free.
## Custom neighborhoods: Tabu Search ## Custom neighborhoods: Tabu Search
When swap isn't the right move set (e.g., 2-opt for TSP, insert / When you want full control of the move set (e.g., systematic 2-opt for
shift for scheduling), use [Tabu Search][TabuSearch] with your own neighbor TSP, or insert-and-shift for scheduling), [Tabu Search][TabuSearch] takes
function. your own neighbor function.
```rust,ignore ```rust,ignore
use heuropt::prelude::*; use heuropt::prelude::*;
let neighbors = |x: &Vec<usize>, _rng: &mut Rng| -> Vec<Vec<usize>> { let neighbors = |x: &Vec<usize>, _rng: &mut Rng| -> Vec<Vec<usize>> {
// Generate all 2-opt neighbors of x. // All 2-opt neighbors of x.
let mut out = Vec::new(); let mut out = Vec::new();
for i in 0..x.len() { for i in 0..x.len() {
for j in (i + 2)..x.len() { for j in (i + 2)..x.len() {
@@ -161,7 +419,29 @@ let neighbors = |x: &Vec<usize>, _rng: &mut Rng| -> Vec<Vec<usize>> {
// Pass `neighbors` to TabuSearch::new(...). // Pass `neighbors` to TabuSearch::new(...).
``` ```
## When to use which approach
| Situation | Use |
|---|---|
| TSP-shaped with a distance matrix | [Ant Colony][AntColonyTsp] |
| Generic permutation, multi-seed budget | GA + `ShuffledPermutation` + OX + Inversion |
| Job-shop scheduling | GA + `ShuffledMultisetPermutation` + local POX + Inversion |
| Single-decision baseline | [SimulatedAnnealing][SimulatedAnnealing] + `SwapMutation` |
| Hand-crafted neighborhood (e.g. systematic 2-opt) | [Tabu Search][TabuSearch] |
| Bi-objective / many-objective permutation problem | See [Multi-objective combinatorial](./multi-objective-combinatorial.md) |
[AntColonyTsp]: https://docs.rs/heuropt/latest/heuropt/algorithms/ant_colony_tsp/struct.AntColonyTsp.html [AntColonyTsp]: https://docs.rs/heuropt/latest/heuropt/algorithms/ant_colony_tsp/struct.AntColonyTsp.html
[SimulatedAnnealing]: https://docs.rs/heuropt/latest/heuropt/algorithms/simulated_annealing/struct.SimulatedAnnealing.html [SimulatedAnnealing]: https://docs.rs/heuropt/latest/heuropt/algorithms/simulated_annealing/struct.SimulatedAnnealing.html
[`SwapMutation`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.SwapMutation.html
[TabuSearch]: https://docs.rs/heuropt/latest/heuropt/algorithms/tabu_search/struct.TabuSearch.html [TabuSearch]: https://docs.rs/heuropt/latest/heuropt/algorithms/tabu_search/struct.TabuSearch.html
[`ShuffledPermutation`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.ShuffledPermutation.html
[`ShuffledMultisetPermutation`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.ShuffledMultisetPermutation.html
[`OrderCrossover`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.OrderCrossover.html
[`PartiallyMappedCrossover`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.PartiallyMappedCrossover.html
[`CycleCrossover`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.CycleCrossover.html
[`EdgeRecombinationCrossover`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.EdgeRecombinationCrossover.html
[`SwapMutation`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.SwapMutation.html
[`InversionMutation`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.InversionMutation.html
[`InsertionMutation`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.InsertionMutation.html
[`ScrambleMutation`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.ScrambleMutation.html
[`CompositeVariation`]: https://docs.rs/heuropt/latest/heuropt/operators/struct.CompositeVariation.html
[CompareExample]: https://github.com/swaits/heuropt/blob/main/examples/tsp_operators_compare.rs