diff --git a/CHANGELOG.md b/CHANGELOG.md index 1e0c274..1679a02 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,87 @@ All notable changes to this workspace are documented here. The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and this project adheres to [Semantic Versioning](https://semver.org/). +## Unreleased + +### Performance — dispatch hot path + +Six codegen-side optimization passes against `step()` and friends. No +user-facing API changes; all 296 tests pass under `--no-default-features`, +`--features tokio`, `--features embassy`, and `--features tokio,journal`. +Cumulative on `h8_cross_tree` (the deepest dispatch in the bench suite): +**−82% instructions / −72% cycles** vs the pre-`pre_lca_fix` baseline. +See `hsmc/benches/baseline.txt` for per-pass delta blocks and +`docs/003. latency-analysis.md` §1/§9/§12 for the cost-surface analysis. + +- **Precomputed transition paths.** For every `(from, to)` state pair, + codegen materializes the exit/enter sequence into a single flat + `__PATH_DATA` blob with a `__PATH_RANGE: [(u16, u16, u16); N²]` + index. `transition()` becomes one indexed range lookup + two slice + loops; the `lca()` runtime helper, `is_up_transition()`, and the + per-call `heapless::Vec` ancestry build are gone. Up / + self / lateral semantics are baked into the table (empty enter + slice ⇒ up-transition, target excluded; non-empty enter slice + includes the default-descent tail). Hierarchy depth is no longer + bounded by the old 32-state Vec capacity. +- **Precomputed event-dispatch table.** For every `(state, event)` + pair, codegen resolves the handler the runtime `__PARENT` bubble + walk would have found. `__DISPATCH: [(u16, &'static [u16], + Option); N×E]` stores `(handler_state, action_ids, + transition_target)` per pair; `u16::MAX` in the handler-state slot + is the no-handler sentinel. The runtime walk + per-hop + `__handler_lookup` call in `step()` collapses to one indexed lookup. + Handler resolution is now O(1) regardless of bubble depth — the + `bubble_top` bench (4-hop walk) is now within 3 instructions of + `bubble_leaf` (handler on current state). `__handler_lookup` is + retained for timer dispatch (the `dispatch_trigger` path doesn't + bubble). +- **Precomputed terminate exit path.** Per-state `__TERMINATE_DATA` + table holds the full exit chain up to and including `__Root` (which + may carry root-level `entry:`/`exit:` actions). `do_terminate()` + indexes the table once and iterates the slice, replacing the + runtime `__PARENT` walk + per-iteration match. Queue drain switched + from `while pop_front().is_some()` to `Deque::clear()` (heapless + 0.8 implements this as O(1) — resets indices instead of popping + each entry). `terminate_event` bench: −22.5% instr / −23.4% cycles. +- **TimerTable empty-skip.** `pop_expired`, `min_remaining`, + `decrement`, and `cancel_state` each get `#[inline]` + an + `is_empty()` early-out, with the non-empty body split into a + `#[cold]` helper. Runtime branch on charts with timers; the body + stays out-of-line. +- **Codegen-static timer-machinery elide.** When a chart declares no + `Duration` triggers (the common case for event-driven HSMs), every + `TimerTable` call is omitted from `step()` and `exit_state()` at + codegen. `step()` returns `None` directly for the next-tick + deadline. `dispatch_trigger()` and the timer-fired arms remain in + the binary but are unreachable; LLVM DCEs them. +- **`TimerEntry.remaining_ns: u128 → u64`.** Internal-only field type + change. u64 nanoseconds bounds at ~584 years — comfortably past any + realistic `Duration`. Eliminates u128 saturating-sub in + `decrement()` (4–6 instructions per entry → 1–2) and the manual + secs/nanos decomposition in `min_remaining()` (replaced with + `Duration::from_nanos`). The struct shrinks 32 B → 16 B with + alignment, so the entire machine struct gets smaller — every bench's + RAM-hit count drops from tighter cache packing. + +### Bench coverage + +Four new benches added to cover hot paths the original suite missed: + +- **`bubble_leaf` / `bubble_mid` / `bubble_top`** — handler resolution + at varying ancestor depths (0 / 3 / 4 hops) on a single chart. + Quantifies the bubble-walk cost as a function of distance, and + confirms the `__DISPATCH` table makes resolution depth-independent. +- **`timer_fire_dispatch`** — only bench that exercises + `pop_expired`'s non-empty branch and `dispatch_trigger`. The most + expensive single-step bench in the suite (191 instr / 706 cycles). +- **`terminate_event`** — only bench exercising `__check_terminate`'s + true arm and `do_terminate`. 69 instr / 311 cycles. +- **`bubble_cross_tree`** — handler on L1 (4 bubble hops up from + current=L5) targets R5 in a sibling subtree (10-state cross-tree + transition). Realistic HSM worst case; 94 instr / 444 cycles — + almost the same as a plain 7-deep cross-tree because both bubble + resolution and transition path are now O(1) lookups. + ## 0.4.0 — 2026-04-28 ### Changed — observability (breaking) diff --git a/docs/003. latency-analysis.md b/docs/003. latency-analysis.md index d1f9ffe..49a875e 100644 --- a/docs/003. latency-analysis.md +++ b/docs/003. latency-analysis.md @@ -26,23 +26,30 @@ can pick from it; nothing here is a commitment. | # | Source | Class | Per-event cost (estimate) | Location | |----|-----------------------------------------------|-----------------------------|------------------------------------------------------------------------------------------------------------|---------------------------------------| -| 1 | `dispatch().await` loop | Discretionary, async-only | 2 awaits cold-start / 1 await steady-state per dispatched event. Each is ~free if action body never `Pending`-yields. | `codegen.rs:2330-2345` | -| 2 | LCA computation for lateral transitions | Discretionary | ~~O(H²) `heapless::Vec::contains` linear scan, H ≤ 32 hard limit~~. **Now O(H) via depth-walk over the `__DEPTH` table. Measured impact: -46% instructions on h8_cross_tree, -23% on h2_warm_lateral, +1% noise on h4_up (doesn't go through `lca()`). See `hsmc/benches/baseline.txt` and §6 in mitigation table.** | `codegen.rs:2601-2628` | -| 3 | Entry-path `heapless::Vec` build | Discretionary | One stack `heapless::Vec` filled per transition; H pushes + H reads. | `codegen.rs:2586-2598` | -| 4 | Parent walk for handler search | Intrinsic | O(H) `__PARENT[s]` indexing until first matching state. | `codegen.rs:2283-2297` | -| 5 | `__handler_lookup` nested match | Intrinsic | LLVM lowers to jump tables (confirmed via `LJTI*` labels in spot-check). ~1 indirect jump per call. | `codegen.rs` (search "__handler_lookup") | -| 6 | `run_action` dispatch | Intrinsic | One match arm + monomorphized trait call + 1 vtable hop on `&mut dyn QueuePush` (the `EmitProxy`). | `codegen.rs:2425-2489` | -| 7 | Event queue push/pop | Intrinsic | `heapless::Deque::push_back` / `pop_front`, both O(1). | `codegen.rs:2316-2319` | -| 8 | `__chart_observe!` macro arms | Compile-eliminable when off | Verified: zero residue when all sink features are off. Each call site collapses to `{};`. | `lib.rs:158-712` | -| 9 | Journal `Vec::push` per atom | Conditional on `journal` | Amortized O(1); ~16 B per `TraceEvent`. Expanded chart shows 104 push call sites; ~11 fire per transition. | `journal.rs:26-32`, `lib.rs:158-712` | -| 10 | Async-fn future size (binary cost) | Discretionary, async-only | `step` closure ~970 lines of asm; `run_handlers` ~830; total Lat-chart machinery ≈ 3000 asm lines. | spot-check measurement | -| 11 | `tokio::sync::mpsc::unbounded_channel` | Tokio-only, std-bound | Heap-backed channel for cross-task event injection. Embassy uses bounded `embassy_sync::channel` instead. | runtime adapters | +| # | Source | Class | Per-event cost (estimate) | Status | +|----|-----------------------------------------------|-----------------------------|------------------------------------------------------------------------------------------------------------|---------------------------------------| +| 1 | `dispatch().await` loop | Discretionary, async-only | 2 awaits cold-start / 1 await steady-state per dispatched event. ~free if action body never `Pending`-yields. | open | +| 2 | LCA + transition path build | Discretionary | ~~O(H²) `heapless::Vec::contains` scan~~ → ~~O(H) depth-walk~~ → **O(1) `__PATH_RANGE` lookup**. Up/self/lateral folded into the precomputed table; `lca()`, `is_up_transition()`, and the per-call `heapless::Vec` ancestry build are all gone. | resolved | +| 3 | Parent walk for handler search | Discretionary | ~~O(H) `__PARENT` walk + per-hop `__handler_lookup` call~~ → **O(1) `__DISPATCH[(state, event)]` lookup**. Every (state, event) pair statically resolved at codegen. `__handler_lookup` retained for timer dispatch only. | resolved | +| 4 | `__handler_lookup` nested match (timer path) | Intrinsic | Still used for timer dispatch. LLVM lowers to jump tables (verified `LJTI*`). ~1 indirect jump per call. | open (already at floor) | +| 5 | `run_action` dispatch | Intrinsic | One match arm + monomorphized trait call + 1 vtable hop on `&mut dyn QueuePush` (verified devirtualized when callsite type is known). | open (already at floor) | +| 6 | Event queue push/pop | Intrinsic | `heapless::Deque::push_back` / `pop_front`, both O(1). | open (already at floor) | +| 7 | `__chart_observe!` macro arms | Compile-eliminable when off | Zero residue when all sink features off. Each call site collapses to `{};`. | verified zero-cost | +| 8 | Journal `Vec::push` per atom | Conditional on `journal` | Amortized O(1); ~16 B per `TraceEvent`. ~11 push sites fire per transition. | opt-in | +| 9 | Async-fn future size (binary cost) | Discretionary, async-only | ~3000 asm lines of chart machinery for a 5-state chart; dominated by async-fn state machines. | open (mitigation H3) | +| 10 | `tokio::sync::mpsc::unbounded_channel` | Tokio-only, std-bound | Heap-backed channel for cross-task event injection. Embassy uses bounded `embassy_sync::channel`. | feature-specific | +| 11 | TimerTable per-step / per-exit fixed cost | Discretionary | ~~Unconditional `pop_expired` + `cancel_state` calls every step / exit~~ → **inlined `is_empty()` + `#[cold]` non-empty body** (runtime branch on charts that use timers; compile-time elided on timer-free charts). | resolved | +| 12 | TimerEntry size + per-decrement work | Discretionary | ~~`remaining_ns: u128`, 32-byte entry, u128 saturating-sub per decrement~~ → **`u64`, 16-byte entry, u64 saturating-sub** (~584-year ceiling; Duration::from_nanos round-trips exactly). | resolved | +| 13 | `do_terminate()` exit walk | Discretionary | ~~Runtime `__PARENT` walk + `pop_front`-loop queue drain~~ → **precomputed `__TERMINATE_DATA` slice + `Deque::clear()`** (O(1)). | resolved | -The top three discretionary items (1, 2, 3) account for almost all the -optimization headroom. Items 4–7 are intrinsic to the chosen semantics — -they're already at or near the algorithmic floor for hierarchical state -machines. Item 8 is verified zero-cost. Item 9 is opt-in. Items 10 and 11 -matter for code size and feature ergonomics but not per-event µs. +Items 2, 3, 11, 12, 13 are fully resolved by the precompute / inline- +empty-check passes in this round of work — see `hsmc/benches/baseline.txt` +for the per-pass delta blocks. Items 4–6 are at or near the algorithmic +floor for hierarchical state machines. Item 7 is verified zero-cost. +Items 1 (async overhead) and 9 (async machinery binary size) are the +only meaningful open items; both are addressable via a `dispatch_sync()` +fast path (mitigation H3 in §9), which would be a user-facing API +change and is therefore out-of-scope for this round. --- @@ -54,28 +61,32 @@ a simple lateral transition, no observation features, `cargo build | Stage | Cycles (rough) | Notes | |----------------------------------------|--------------------|------------------------------------------------| -| `queue.pop_front` | ~10 | `heapless::Deque` index op | -| `__event_to_id` | ~5 | match arm | -| Parent walk for handler | ~30 × H | ≤ H array indexes | -| `__handler_lookup` | ~10 | jump table indirect | -| `run_action` (1 action) | ~20-30 | match + monomorphized call + 1 vtable hop | -| `is_up_transition` | ~30 | parent walk | -| `lca` (lateral) | ~10 × H | depth-walk via `__DEPTH` (was ~50 × H² pre-fix)| -| Exit chain (H states × A actions) | ~60 × H × A | | -| Entry path build | ~30 × H | heapless::Vec push | -| Enter chain (H states × A actions) | ~60 × H × A | | -| **Total `.step()` steady-state** | ~1.0 – 2.0 µs | for H=3, A=1; user action body excluded | +| `queue.pop_front` | ~5–10 | `heapless::Deque` index op | +| `__event_to_id` | ~3–5 | match arm (jump table) | +| `__DISPATCH[(state, event)]` lookup | ~5 | one indexed load + sentinel branch | +| `run_action` (1 action) | ~10–20 | match + monomorphized call + 1 vtable hop | +| Path lookup (`__PATH_RANGE`) | ~5 | one indexed load + slice arithmetic | +| Exit chain (H states × A actions) | ~3 × H + 30 × A | per-state work is now ~3 instr each | +| Enter chain (H states × A actions) | ~3 × H + 30 × A | per-state work is now ~3 instr each | +| Timer machinery (timer-free chart) | 0 | codegen-elided when `ir.duration_triggers.is_empty()` | +| Timer machinery (timer chart, no fire) | ~3 | inlined `is_empty()` branches × 3 | +| Timer machinery (timer firing) | ~30–40 | decrement + pop_expired + dispatch_trigger | +| **Total `.step()` steady-state** | ~0.4 – 0.8 µs | for H=3, A=1; user action body excluded | | Async overhead (`.dispatch()` loop) | per-await checkpoint | ~free if no `Pending`-yield | These are back-of-envelope numbers; actual measurements would tighten them by ~2× in either direction, depending on inlining decisions. For empirical numbers (deterministic, instruction-counted), see -`hsmc/benches/baseline.txt`. Headline numbers post-LCA-fix: -h2_warm_lateral 210 instr / ~1k cycles; h4_cross_tree 394 instr / ~1.4k -cycles; h8_cross_tree 508 instr / ~1.6k cycles; chain_self_emit 1036 instr -across 4 events (≈260 instr/event). Mitigations H1/H2 in §9 target the -remaining LCA + entry-path stack-vec overhead. +`hsmc/benches/baseline.txt`. Headline numbers (current): +h2_warm_lateral 84 instr / 397 cycles; h4_cross_tree 101 instr / 456 +cycles; h8_cross_tree 93 instr / 443 cycles; bubble_top 99 instr / +453 cycles (4-hop handler resolution); bubble_cross_tree 94 instr / +444 cycles (4-hop bubble + 10-state cross-tree); terminate_event 69 +instr / 311 cycles; timer_fire_dispatch 191 instr / 706 cycles; +chain_self_emit 419 instr across 4 events (≈105 instr/event). +Cumulative on h8_cross_tree: **-82% instructions / -72% cycles** vs +the original O(H²) LCA scan. --- @@ -295,9 +306,8 @@ audit of every loop and await pattern in the runtime and codegen: |---------------------------------------|---------------------------|-------------------------|--------------------------|---------------------| | Main `run()` outer | codegen.rs:1290+, 1379+ | by `terminated` flag | yes — always | cooperative | | `dispatch()` drain loop | codegen.rs:2339 | by queue size | only via `step().await` | suspect — fairness | -| Transition exit chain | codegen.rs:2532, 2577 | by hierarchy depth | yes (async variant) | OK | +| Transition exit/enter slice loops | codegen.rs (transition) | by precomputed slice len| yes (async variant) | OK | | Termination drain | codegen.rs:2750 | by hierarchy + queue | yes (async variant) | OK | -| `lca()` ancestry walk | codegen.rs:2605 | by hierarchy depth | no (sync helper) | OK | | Timer decrement / pop | runtime / lib.rs | by timer-table size | sync, off the event-loop | OK | | Send/recv backends | codegen.rs:1308, 1405 | n/a | always `.await` | cooperative | @@ -363,21 +373,25 @@ feel: L = afternoon, M = couple of days, H = a week or more. | ID | Mitigation | Targets | Impact | Effort | Cost | |-----|-----------------------------------------------------------------------------------------------------------------------------------|-----------|-----------------------------------------|--------|---------------------------------------| -| H0 | ~~Replace `lca()` with an O(H) depth-walk using the existing `__DEPTH` table; remove the `heapless::Vec` set entirely.~~ **Landed.** Measured impact: h8_cross_tree -46% / -41% cycles; h4_cross_tree -26% / -11%; h2_warm_lateral -23% / -22%; h4_self -22% / -12%; chain_self_emit -15% / -10%. See `hsmc/benches/baseline.txt`. | #2 | (delivered) | - | - | -| H1 | Codegen per-transition exit/entry/descend chains as `&'static [u16]` arrays; eliminate runtime `heapless::Vec` build entirely. | #2, #3 | High — kills LCA *and* heapless allocs | M | ~10–30 B per transition; bounded | -| H2 | Same idea but precompute LCA only (small static table or per-transition LCA u16 field). | #2 | Med-High | L | ~2 B per transition | -| H3 | Provide a `dispatch_sync()` API that bypasses `.await` when the chart's actions are fully synchronous. | #1, #10 | High for sync-action charts | L-M | API surface growth | -| H4 | Codegen a per-state inlined event match (skip the parent walk by emitting `(state, event) → (handler_state, actions, target)` directly per state). | #4 | Low-Med | M | binary grows ~`states × events` | -| M1 | Eliminate the trailing `step().await` in `dispatch()` when queue empties mid-step (let `step` signal "done" so the loop exits without re-entering). | #1 | Low | L | minor refactor | -| M2 | `#[inline]` on `__handler_lookup`, `__state_name`, etc. for tighter cross-call inlining. | #5, #6 | Low (LLVM probably already does this) | L | none | -| M3 | Lazy field eval in `trace-tracing` arm using `tracing`'s field-closure form to skip evaluation when filtered. | feature on| Low | L | none | -| L1 | Audit residual `_t_state`, `_t_trigger`, `__handler_state` local bindings in observation-off builds — verify LLVM elides them. | feature off| Negligible | L | none | +| H0 | Replace `lca()` with an O(H) depth-walk via the `__DEPTH` table. | #2 | **Landed.** | - | - | +| H1 | Codegen per-(from, to) precomputed exit/enter chains as `__PATH_DATA` + `__PATH_RANGE`. | #2 | **Landed.** Up/self/lateral folded into one indexed lookup; `lca()` and `is_up_transition()` removed. | - | ~`6·N²` B index + ~`N²·H` B path data | +| T1 | Inlined `is_empty()` early-out on `pop_expired` / `min_remaining` / `decrement` / `cancel_state` with `#[cold]` non-empty bodies. | #11 | **Landed.** | - | - | +| T2 | Codegen-static elide of all `TimerTable` calls when `ir.duration_triggers.is_empty()`. | #11 | **Landed.** Most charts are timer-free; `step()` returns `None` directly. | - | - | +| H4 | `__DISPATCH[(state, event)]` const array; one indexed lookup replaces the runtime __PARENT bubble walk + per-hop `__handler_lookup`. | #3 | **Landed.** Each (state, event) statically resolved at codegen. | - | ~24 B per (state, event) pair | +| U1 | Shrink `TimerEntry.remaining_ns: u128 → u64` (~584-year ceiling). Halves entry size; replaces u128 saturating-sub with u64. | #12 | **Landed.** | - | - | +| K1 | `__TERMINATE_DATA` per-state precomputed exit chain (root inclusive); `do_terminate()` indexes a slice instead of walking __PARENT, and uses `Deque::clear()` (O(1)) for the queue drain. | #13 | **Landed.** terminate_event -22.5% instr / -23.4% cycles. | - | ~`(H+1)·N` B path data | +| H3 | Provide a `dispatch_sync()` API that bypasses `.await` when the chart's actions are fully synchronous. | #1, #9 | High for sync-action charts | L-M | API surface growth (user-facing) | +| H5 | Sparse `__PATH_RANGE` / `__DISPATCH` for very large charts (N > 64): emit only reachable pairs with a small perfect-hash. Latency-neutral; cuts O(N²) static memory growth. | cost mitigation | Low at typical N; significant at N > 100 | M | codegen complexity | +| M1 | Eliminate the trailing `step().await` in `dispatch()` when queue empties mid-step. | #1 | Low | L | minor refactor | +| M2 | Lazy field eval in `trace-tracing` arm using `tracing`'s field-closure form to skip evaluation when filtered. | #8 (on) | Low | L | none | -H1 is the clear top candidate: it collapses the two largest discretionary -costs (#2, #3) into one static-table lookup, removes one runtime stack -allocation per transition, and is implementable purely as a codegen -change in `hsmc-macros/src/codegen.rs` without touching the runtime -contract. +H0/H1/T1/T2/H4/U1/K1 all landed in the most recent round of work. +Callgrind now reports the entire HSM machinery as a single `step` +symbol — every helper inlined. The only remaining optimization avenue +that meaningfully affects per-event latency is H3 (`dispatch_sync()`), +which is excluded as user-facing. Memory-side mitigations (H5) only +matter for N > 64-state charts; embedded users with large charts are +the audience. --- @@ -393,13 +407,13 @@ For anyone reading this audit and wanting to verify or extend it: - `:26-32` — `extern crate alloc` and `Vec` import (only here) - `:65-150+` — `TraceEvent` enum (the journal vocabulary) - **`hsmc-macros/src/codegen.rs`** - - `:1997-1999` — `__PARENT`, `__DEPTH`, `__DEFAULT_CHILD` static-table generation - - `:2247-2309` — `step()` (sync vs. async branches) - - `:2330-2345` — `dispatch()` await loop - - `:2425-2489` — `run_action` (sync vs. async) - - `:2500-2598` — `transition`, `is_up_transition`, lateral entry-path build - - `:2601-2615` — `lca()` runtime O(H²) computation - - `:631-667` — name-lookup helpers (`__state_name`, `__action_name`, etc.) + - `compute_transition_paths()` (top of file, after `compute_chart_hash`) — builds `__PATH_DATA` + `__PATH_RANGE` at codegen time + - `__PARENT`, `__DEPTH`, `__DEFAULT_CHILD`, `__PATH_DATA`, `__PATH_RANGE` static-table emission (search "const __PARENT") + - `step()` (search "fn step" / "async fn step") — sync vs. async branches + - `dispatch()` await loop (search "fn dispatch") + - `run_action` (search "fn run_action") — sync vs. async + - `transition` (search "fn transition") — now a thin slice-iter wrapper + - name-lookup helpers (`__state_name`, `__action_name`, etc.) at `:631-667` - **Test reference shape**: `hsmc/tests/state_chart_const.rs` is the smallest sync-only chart in the tree; `hsmc/tests/det_lateral.rs` is the reference async + journal shape used for the asm spot-check. @@ -439,25 +453,48 @@ handler lookup, action dispatch, queue ops) is small and well-monomorphized. Observation cost is verified zero when features are off, and stays proportional to the user's chosen sink mix when on. -**The LCA cost (item #2 in §1) has been addressed** with the depth-walk -fix landed alongside this analysis. Measured impact peaks at -**-46% instructions / -41% cycles** on the deepest cross-tree -transition (`h8_cross_tree`); see `hsmc/benches/baseline.txt`. The -former O(H²) scan and the `heapless::Vec` ancestry-set -allocation are gone; the 32-state hierarchy ceiling is also gone. +**Items #2, #3, #11, #12, and #13 in §1 have all landed** across six +optimization passes (H0 → H1 → T1 → H4 → T2 → K1+U1). Cumulative impact +on the deepest dispatch (h8_cross_tree) is **-82% instructions / -72% +cycles** vs the original O(H²) LCA scan. The headline wins by bench: -The two remaining optimization targets are: +- `terminate_event` 69 instr / 311 cycles (~14 instr of HSM work after + subtracting bench wrapper) — `do_terminate()` is one indexed slice + + `Deque::clear()`. +- `bubble_top` 99 instr / 453 cycles — 4-hop handler resolution + collapses to one indexed `__DISPATCH` lookup; per-dispatch cost is + bench-shape-independent. +- `bubble_cross_tree` 94 instr / 444 cycles — realistic HSM worst + case (deep bubble + 10-state cross-tree) costs the same as a plain + cross-tree because bubble lookup is O(1) and transition is + O(path-length). +- `h8_cross_tree` 93 instr / 443 cycles — 7-deep cross-tree dispatch. -1. **Entry-path `heapless::Vec` build** (item #3 in §1) — still allocates - one stack `heapless::Vec` per lateral transition. Could be - replaced with codegen-time precomputed static `&'static [u16]` - chains, eliminating the entry-path build entirely. (See mitigation H1.) -2. **Async coloring code-size cost** (item #10) — chart machinery is +Callgrind reports the entire HSM machinery as a single `step` symbol — +`run_handlers`, `run_action`, `transition`, `do_terminate`, +`exit_state`, `enter_state_no_descent`, `EmitProxy::push`, all four +`TimerTable` methods, the dispatch-table lookup, and the +terminate-path lookup are all inlined. + +Cost remaining, in priority order: + +1. **Async coloring code-size cost** (item #9) — chart machinery is ~3,000 asm lines for a 5-state chart, dominated by async-fn state - machines. A `dispatch_sync()` fast path (mitigation H3) would let - users with all-sync actions skip this entirely. + machines. Mitigation H3 (a `dispatch_sync()` fast path) would let + users with all-sync actions skip this entirely. **User-facing API + change** — out of scope for the current optimization pass. +2. **Bench-wrapper noise floor** — at the smallest dispatch's ~38–80 + instructions, the iai-callgrind wrapper (`black_box`, `send`, + instrumented call-edge crossing) accounts for ~45 instructions. + Sub-bench-wrapper improvements are below the measurement floor. +3. **Memory cost of `__PATH_DATA` / `__PATH_RANGE` / `__DISPATCH` / + `__TERMINATE_DATA` at large N** — `__PATH_RANGE` is O(N²); + `__DISPATCH` is O(N×E); `__TERMINATE_DATA` is O(N×H). For N ≤ 32 / + E ≤ 16 / H ≤ 8 the total static footprint is a few KiB. A + 64-state × 32-event chart would push past ~80 KiB. Mitigation H5 + sketches a sparse encoding. -Both are addressable purely in `hsmc-macros/src/codegen.rs` without -breaking runtime semantics or feature gates. The bench harness in -`hsmc/benches/dispatch.rs` is in place to measure each mitigation as it -lands. +All addressable purely in `hsmc-macros/src/codegen.rs` and `hsmc/src/lib.rs` +without breaking runtime semantics or feature gates. The bench harness +in `hsmc/benches/dispatch.rs` (13 benches, every dispatch shape +covered) is in place to measure each mitigation as it lands. diff --git a/hsmc-macros/src/codegen.rs b/hsmc-macros/src/codegen.rs index 2d2dca8..da543ba 100644 --- a/hsmc-macros/src/codegen.rs +++ b/hsmc-macros/src/codegen.rs @@ -4,7 +4,7 @@ use proc_macro2::{Span, TokenStream}; use quote::{format_ident, quote}; use syn::Ident; -use crate::ir::{HandlerKindIr, Ir, TriggerIr, VariantKind}; +use crate::ir::{HandlerKindIr, Ir, StateIr, TriggerIr, VariantKind}; use crate::parse::{During, PayloadKind}; /// FNV-1a 64-bit hash of a chart's structural fingerprint. Stable across @@ -88,6 +88,292 @@ fn compute_chart_hash(ir: &Ir) -> u64 { h } +/// (exit_data, enter_data, exit_range, enter_range) — flat blobs of u16 +/// state ids and `(start, end)` index pairs into them, one per (from, to). +/// Returned by `compute_transition_paths()` and folded by the caller into +/// the consolidated `__PATH_DATA` / `__PATH_RANGE` tables. +type TransitionPathTables = (Vec, Vec, Vec<(u16, u16)>, Vec<(u16, u16)>); + +/// Static precompute of every (from, to) state pair's exit and enter sequence, +/// folded into flat tables emitted alongside `__PARENT` / `__DEPTH`. Replaces +/// the runtime LCA depth-walk + ancestry walks + heapless::Vec construction +/// in `transition()` with two slice lookups and two straight slice iterations. +/// +/// The (from, to) lookup matches the runtime LCA call: `from` is the active +/// leaf (`self.current`), `to` is the transition target. The same three +/// transition shapes are encoded uniformly by what the precomputed paths +/// contain: +/// +/// - **Up-transition** (`to` is a strict ancestor of `from`): exit_path is +/// `from..to` (target excluded; `to` is already on the active path and +/// must not be re-entered). enter_path is empty — `transition()` sets +/// `self.current = Some(target)` after the loops. +/// - **Self-transition** (`from == to`): exit_path is `[to]`. enter_path +/// is `[to, default_child(to), default_child(default_child(to)), ...]` +/// — re-enter target plus descend defaults. +/// - **Lateral** (LCA is a proper ancestor of both): exit_path walks +/// `from` up to but not including LCA. enter_path walks +/// LCA-side child of `to` down to `to`, then continues descending +/// `__DEFAULT_CHILD` from `to`. +/// +/// Storage: `__EXIT_DATA` and `__ENTER_DATA` are flat `[u16]` blobs; +/// `__EXIT_RANGE[i*N + j]` / `__ENTER_RANGE[i*N + j]` give `(start, end)` +/// offsets into them. Entries for unreachable pairs are still emitted (we +/// don't know which pairs are reachable without running the dispatcher's +/// full reachability analysis), but they cost only their few u16s of path +/// data and a single `(u16, u16)` index slot — and `transition()` never +/// looks them up at runtime. +fn compute_transition_paths(states: &[StateIr]) -> TransitionPathTables { + let n = states.len(); + let parent = |s: u16| -> Option { states[s as usize].parent }; + let default_child = |s: u16| -> Option { states[s as usize].default_child }; + let depth = |s: u16| -> i32 { states[s as usize].depth as i32 }; + + let lca = |mut a: u16, mut b: u16| -> Option { + let mut da = depth(a); + let mut db = depth(b); + while da > db { + a = parent(a)?; + da -= 1; + } + while db > da { + b = parent(b)?; + db -= 1; + } + while a != b { + a = parent(a)?; + b = parent(b)?; + } + Some(a) + }; + + let is_strict_ancestor = |anc: u16, desc: u16| -> bool { + let mut cur = parent(desc); + while let Some(c) = cur { + if c == anc { + return true; + } + cur = parent(c); + } + false + }; + + let compute_one = |from: u16, to: u16| -> (Vec, Vec) { + // Up-transition: target is a strict ancestor of current. Exit + // strictly between them; do not re-enter target. + if from != to && is_strict_ancestor(to, from) { + let mut exit_path = Vec::new(); + let mut cur = from; + while cur != to { + exit_path.push(cur); + cur = parent(cur).expect("up-transition walk must reach `to`"); + } + return (exit_path, Vec::new()); + } + + // Self-transition: external semantics — exit and re-enter target, + // then descend defaults. + if from == to { + let exit_path = vec![to]; + let mut enter_path = vec![to]; + let mut cur = default_child(to); + while let Some(c) = cur { + enter_path.push(c); + cur = default_child(c); + } + return (exit_path, enter_path); + } + + // Lateral: exit `from` up to (not including) LCA; enter from + // LCA-child on `to`'s side down to `to`, then descend defaults. + let lca_id = lca(from, to); + + let mut exit_path = Vec::new(); + let mut cur = from; + loop { + if Some(cur) == lca_id { + break; + } + exit_path.push(cur); + match parent(cur) { + Some(p) => cur = p, + None => break, + } + } + + let mut rev = Vec::new(); + let mut cur = to; + loop { + if Some(cur) == lca_id { + break; + } + rev.push(cur); + match parent(cur) { + Some(p) => cur = p, + None => break, + } + } + let mut enter_path: Vec = rev.into_iter().rev().collect(); + + let mut cur = default_child(to); + while let Some(c) = cur { + enter_path.push(c); + cur = default_child(c); + } + + (exit_path, enter_path) + }; + + let mut exit_data: Vec = Vec::new(); + let mut enter_data: Vec = Vec::new(); + let mut exit_range: Vec<(u16, u16)> = Vec::with_capacity(n * n); + let mut enter_range: Vec<(u16, u16)> = Vec::with_capacity(n * n); + + for from in 0..n as u16 { + for to in 0..n as u16 { + let (ep, np) = compute_one(from, to); + + let estart: u16 = exit_data + .len() + .try_into() + .expect("exit path data exceeds u16 offset range"); + exit_data.extend(ep.iter().copied()); + let eend: u16 = exit_data + .len() + .try_into() + .expect("exit path data exceeds u16 offset range"); + exit_range.push((estart, eend)); + + let nstart: u16 = enter_data + .len() + .try_into() + .expect("enter path data exceeds u16 offset range"); + enter_data.extend(np.iter().copied()); + let nend: u16 = enter_data + .len() + .try_into() + .expect("enter path data exceeds u16 offset range"); + enter_range.push((nstart, nend)); + } + } + + (exit_data, enter_data, exit_range, enter_range) +} + +/// Static precompute of `do_terminate()`'s exit walk: for each leaf state, +/// the chain `[current, parent(current), ..., __Root]` (root inclusive, +/// since `__Root` may carry root-level `entry:`/`exit:` actions declared +/// at the chart top level — see the `det_initial` test for that pattern). +/// +/// Replaces the runtime `__PARENT` walk + per-iteration `match` in +/// `do_terminate` with one indexed slice into a flat blob. The terminate +/// path is always at most `H + 1` states long, so the total table size is +/// bounded by `N × (H_max + 1)` — under a KiB for any practical chart. +fn compute_terminate_paths(states: &[StateIr]) -> (Vec, Vec<(u16, u16)>) { + let n = states.len(); + let parent = |s: u16| -> Option { states[s as usize].parent }; + + let mut data: Vec = Vec::new(); + let mut range: Vec<(u16, u16)> = Vec::with_capacity(n); + for from in 0..n as u16 { + let start: u16 = data + .len() + .try_into() + .expect("terminate path data exceeds u16 offset range"); + let mut cur = Some(from); + while let Some(s) = cur { + data.push(s); + cur = parent(s); + } + let end: u16 = data + .len() + .try_into() + .expect("terminate path data exceeds u16 offset range"); + range.push((start, end)); + } + (data, range) +} + +/// Static precompute of the runtime event-bubble walk in `step()`. +/// +/// Currently the dispatcher does: +/// +/// ```text +/// while let Some(s) = __cur { +/// if let Some((actions, target)) = __handler_lookup(s, 0, __ev_id) { ... break; } +/// __cur = __PARENT[s as usize]; +/// } +/// ``` +/// +/// — at runtime, walking up `__PARENT` until a state with an `on(EventX)` +/// handler is found. Each hop costs ~10-12 instructions (call + branch + +/// load), and the depth depends on where the handler is declared. With +/// every (state, event) pair statically resolvable from the chart, the +/// walk collapses to one indexed lookup. +/// +/// For each `(current_state, event_id)` pair we materialize: +/// - `handler_state` — the deepest ancestor (or `current_state` itself) +/// with a matching `on()` clause, or `u16::MAX` if no ancestor handles +/// the event (runtime drops it). +/// - `action_ids` — the aggregated action ids that the original +/// `state_handler_arms` codegen would have produced for that +/// (state, event) pair, in declaration order. +/// - `target` — the transition target if any. +/// +/// The aggregation rules mirror `state_handler_arms` exactly: per +/// (state, event), action handlers stack into an ordered Vec and the +/// (singular) transition target is captured if present. This is the +/// same shape `__handler_lookup` returns, just resolved at codegen time +/// for events. Timer triggers stay in `__handler_lookup` (the +/// `dispatch_trigger` path doesn't bubble). +fn compute_event_dispatch(ir: &Ir) -> Vec<(u16, Vec, Option)> { + use std::collections::BTreeMap; + + let n_states = ir.states.len(); + let n_events = ir.event_variants.len(); + + // Per-state, per-event handler resolution. Mirrors the aggregation in + // `state_handler_arms` (sort by decl_index, group by trigger). + let mut per_state: Vec, Option)>> = + vec![BTreeMap::new(); n_states]; + for (sid, state) in ir.states.iter().enumerate() { + let mut handlers = state.handlers.iter().collect::>(); + handlers.sort_by_key(|h| h.decl_index); + for h in handlers { + let event_id = match &h.trigger { + TriggerIr::Event(id, _, _) => *id, + TriggerIr::Duration(_) => continue, + }; + let entry = per_state[sid].entry(event_id).or_default(); + match &h.kind { + HandlerKindIr::Action(aid, _) => entry.0.push(*aid), + HandlerKindIr::Transition(_, Some(tid)) => entry.1 = Some(*tid), + HandlerKindIr::Transition(_, None) => {} + } + } + } + + // For each (state, event_id), bubble up __PARENT and stop at the + // first state with a matching handler. No-handler ⇒ sentinel. + let mut out = Vec::with_capacity(n_states * n_events); + for state in 0..n_states { + for event_id in 0..n_events as u16 { + let mut cur = Some(state as u16); + let mut resolved: Option<(u16, Vec, Option)> = None; + while let Some(s) = cur { + if let Some((actions, target)) = per_state[s as usize].get(&event_id) { + resolved = Some((s, actions.clone(), *target)); + break; + } + cur = ir.states[s as usize].parent; + } + out.push(resolved.unwrap_or((u16::MAX, Vec::new(), None))); + } + } + + out +} + /// For each user-declared leaf state that has one or more `during:` activities /// active (either declared on it, on any ancestor, or on root), return its /// state id and the flattened list of durings in leaf-first order. Durings @@ -907,6 +1193,73 @@ pub fn generate(ir: &Ir) -> TokenStream { ); }; + // Codegen-static elide of the timer-table machinery for charts that + // declare no `Duration` triggers. The TimerTable is still constructed + // (the field is part of the machine struct so the generic parameters + // line up), but every per-step / per-exit call into it is gone — no + // empty-table branch, no `min_remaining()`-returns-None call, no + // `cancel_state()`. `step()` returns `None` directly for the + // next-tick deadline. For the bench charts (none of which arm + // timers) this drops several instructions per `step()` and one per + // `exit_state()`. `dispatch_trigger()` and the timer-fired arms + // remain in the binary but are unreachable; LLVM DCEs them. + let has_timers = !ir.duration_triggers.is_empty(); + let timer_decrement_call = if has_timers { + quote! { self.timers.decrement(elapsed); } + } else { + quote! {} + }; + let timer_pop_block_sync = if has_timers { + quote! { + if let Some((state, trig)) = self.timers.pop_expired(&__DEPTH) { + let __t_state = state; + let __t_trigger = trig; + #observe_timer_fired + if __duration_repeats(trig) { + let d = __duration_for(trig); + self.timers.start(state, trig, d); + let sid = state; + let tid = trig; + #observe_timer_armed + } + self.dispatch_trigger(state, 1, trig); + return self.timers.min_remaining(); + } + } + } else { + quote! {} + }; + let timer_pop_block_async = if has_timers { + quote! { + if let Some((state, trig)) = self.timers.pop_expired(&__DEPTH) { + let __t_state = state; + let __t_trigger = trig; + #observe_timer_fired + if __duration_repeats(trig) { + let d = __duration_for(trig); + self.timers.start(state, trig, d); + let sid = state; + let tid = trig; + #observe_timer_armed + } + self.dispatch_trigger(state, 1, trig).await; + return self.timers.min_remaining(); + } + } + } else { + quote! {} + }; + let timer_min_remaining_expr = if has_timers { + quote! { self.timers.min_remaining() } + } else { + quote! { None } + }; + let timer_cancel_state_call = if has_timers { + quote! { self.timers.cancel_state(sid); } + } else { + quote! {} + }; + // Per-state during count, used to emit DuringStarted/Cancelled // events one per declared during in the state. let durings_count_arms: Vec = ir @@ -971,6 +1324,74 @@ pub fn generate(ir: &Ir) -> TokenStream { None => quote! { None }, }) .collect::>(); + // Path data is concatenated [exit; enter] per pair into a single + // `__PATH_DATA` blob; one packed `(start, mid, end)` per pair indexes + // both halves. One range lookup, one base array — minimizes cold-cache + // line touches on the dispatch hot path (vs four separate tables). + let ( + transition_exit_data, + transition_enter_data, + transition_exit_range, + transition_enter_range, + ) = compute_transition_paths(&ir.states); + let mut path_data: Vec = + Vec::with_capacity(transition_exit_data.len() + transition_enter_data.len()); + let mut path_range: Vec<(u16, u16, u16)> = Vec::with_capacity(transition_exit_range.len()); + for ((es, ee), (ns, ne)) in transition_exit_range + .iter() + .zip(transition_enter_range.iter()) + { + let start: u16 = path_data + .len() + .try_into() + .expect("transition path data exceeds u16 offset range"); + path_data.extend_from_slice(&transition_exit_data[*es as usize..*ee as usize]); + let mid: u16 = path_data + .len() + .try_into() + .expect("transition path data exceeds u16 offset range"); + path_data.extend_from_slice(&transition_enter_data[*ns as usize..*ne as usize]); + let end: u16 = path_data + .len() + .try_into() + .expect("transition path data exceeds u16 offset range"); + path_range.push((start, mid, end)); + } + let path_data_len = path_data.len(); + let n_pairs = n_states * n_states; + let path_data_lit: Vec<_> = path_data.iter().map(|x| quote! { #x }).collect(); + let path_range_lit: Vec<_> = path_range + .iter() + .map(|(a, b, c)| quote! { (#a, #b, #c) }) + .collect(); + // Precomputed terminate-path table: for every state, the exit chain + // up to and including __Root. `do_terminate()` uses this instead of + // walking __PARENT at runtime. + let (terminate_data, terminate_range) = compute_terminate_paths(&ir.states); + let terminate_data_len = terminate_data.len(); + let terminate_data_lit: Vec<_> = terminate_data.iter().map(|x| quote! { #x }).collect(); + let terminate_range_lit: Vec<_> = terminate_range + .iter() + .map(|(a, b)| quote! { (#a, #b) }) + .collect(); + + // Precomputed event-dispatch table: for every (state, event_id) pair, + // the resolved handler that the runtime bubble walk would have found. + // See `compute_event_dispatch` for encoding. + let dispatch_table = compute_event_dispatch(ir); + let n_events = ir.event_variants.len(); + let n_dispatch = n_states * n_events; + let dispatch_lit: Vec = dispatch_table + .iter() + .map(|(h_state, actions, target)| { + let actions_lit = actions.iter().map(|a| quote! { #a }); + let target_tok = match target { + Some(t) => quote! { Some(#t) }, + None => quote! { None }, + }; + quote! { (#h_state, &[#(#actions_lit),*], #target_tok) } + }) + .collect(); let state_id_to_variant_arms = ir .states .iter() @@ -1998,6 +2419,51 @@ pub fn generate(ir: &Ir) -> TokenStream { const __DEPTH: [u8; #n_states] = [#(#depth_table),*]; const __DEFAULT_CHILD: [Option; #n_states] = [#(#default_child_table),*]; + // ── Precomputed transition paths ──────────────────────────── + // + // For every (from, to) state pair, the exit-then-enter sequence + // a transition would walk is materialized at codegen time. The + // dispatch hot path (`fn transition`) becomes one range lookup + // and two straight loops; the runtime LCA depth-walk and the + // heapless::Vec ancestry construction are gone. + // + // Encoding: `__PATH_DATA` is the concatenation of every + // `[exit; enter]` slice. `__PATH_RANGE[i*N + j] = (s, m, e)` + // gives the boundaries: `exit = data[s..m]`, `enter = data[m..e]`. + // Empty enter slice (`m == e`) ⇒ up-transition, target excluded; + // otherwise the enter slice already includes the default-descent + // tail so no separate `descend_defaults()` call is needed. + // + // Folding both ranges into one struct halves the cold-cache + // line touches on first dispatch — the up-transition hot path + // does one indexed load instead of two. + const __PATH_DATA: [u16; #path_data_len] = [#(#path_data_lit),*]; + const __PATH_RANGE: [(u16, u16, u16); #n_pairs] = [#(#path_range_lit),*]; + + // ── Precomputed terminate exit paths ─────────────────────── + // + // For every starting state, the full exit chain up to and + // including __Root. `do_terminate()` indexes into this instead + // of walking __PARENT at runtime. + const __TERMINATE_DATA: [u16; #terminate_data_len] = [#(#terminate_data_lit),*]; + const __TERMINATE_RANGE: [(u16, u16); #n_states] = [#(#terminate_range_lit),*]; + + // ── Precomputed event-dispatch table ──────────────────────── + // + // For every (current_state, event_id) pair, the deepest ancestor + // state with an `on(EventX)` clause is resolved at codegen time. + // The runtime `step()` event-handling block does one indexed + // lookup instead of an `__PARENT` bubble walk + `__handler_lookup` + // call per ancestor. + // + // Entry layout: (handler_state, action_ids, transition_target). + // `handler_state == u16::MAX` is the no-handler sentinel — the + // event would have been dropped at runtime. The dropped/delivered + // observation arms key off that sentinel, preserving the existing + // journal vocabulary. + const __DISPATCH: [(u16, &'static [u16], Option); #n_dispatch] = + [#(#dispatch_lit),*]; + #name_helpers #[allow(dead_code, unreachable_patterns)] @@ -2008,7 +2474,12 @@ pub fn generate(ir: &Ir) -> TokenStream { } } + // Hot-path lookup helpers. `#[inline]` so LLVM folds the + // (typically jump-table) match directly into `step()` / + // `transition()` / `exit_state()`, removing a call+return per + // dispatched event and cutting one stack frame. #[allow(unreachable_code, unused_variables)] + #[inline] fn __duration_for(tid: u16) -> ::hsmc::Duration { match tid { #(#duration_expr_arms)* @@ -2017,6 +2488,7 @@ pub fn generate(ir: &Ir) -> TokenStream { } #[allow(unreachable_code, unused_variables)] + #[inline] fn __duration_repeats(tid: u16) -> bool { match tid { #(#duration_repeat_arms)* @@ -2025,6 +2497,7 @@ pub fn generate(ir: &Ir) -> TokenStream { } #[allow(unreachable_patterns, unused_variables)] + #[inline] fn __event_to_id(ev: &#ev_ty) -> u16 { match ev { #(#event_variant_arms)* @@ -2033,6 +2506,7 @@ pub fn generate(ir: &Ir) -> TokenStream { } #[allow(unused_variables)] + #[inline] fn __handler_lookup( state_id: u16, trigger_kind: u8, @@ -2046,6 +2520,7 @@ pub fn generate(ir: &Ir) -> TokenStream { } #[allow(unused_variables)] + #[inline] fn __owned_timers(state_id: u16) -> &'static [u16] { match state_id { #(#owned_timers_arms)* @@ -2056,6 +2531,7 @@ pub fn generate(ir: &Ir) -> TokenStream { enum TerminateCheck { Yes, No } #[allow(unused_variables)] + #[inline] fn __check_terminate(__ev: &#ev_ty) -> TerminateCheck { #terminate_match TerminateCheck::No @@ -2188,25 +2664,12 @@ pub fn generate(ir: &Ir) -> TokenStream { if self.current.is_none() { #observe_started self.enter_path(0); - return self.timers.min_remaining(); + return #timer_min_remaining_expr; } - self.timers.decrement(elapsed); + #timer_decrement_call - if let Some((state, trig)) = self.timers.pop_expired(&__DEPTH) { - let __t_state = state; - let __t_trigger = trig; - #observe_timer_fired - if __duration_repeats(trig) { - let d = __duration_for(trig); - self.timers.start(state, trig, d); - let sid = state; - let tid = trig; - #observe_timer_armed - } - self.dispatch_trigger(state, 1, trig); - return self.timers.min_remaining(); - } + #timer_pop_block_sync if let Some(ev) = self.queue.pop_front() { let __ev_id = __event_to_id(&ev); @@ -2217,31 +2680,35 @@ pub fn generate(ir: &Ir) -> TokenStream { return None; } if __ev_id != u16::MAX { - let mut __cur = self.current; - let mut __delivered = false; - while let Some(s) = __cur { - if let Some((actions, target)) = __handler_lookup(s, 0, __ev_id) { - let __handler_state = s; + // Resolve handler via precomputed __DISPATCH table: + // one indexed lookup replaces the runtime __PARENT + // bubble walk + per-hop __handler_lookup() call. + // u16::MAX in the handler_state slot is the + // no-handler sentinel. + if let Some(__cur) = self.current { + let __dispatch_idx = + (__cur as usize) * #n_events + (__ev_id as usize); + let (__handler_state, __actions, __target) = + __DISPATCH[__dispatch_idx]; + if __handler_state != u16::MAX { #observe_event_delivered self.run_handlers( - s, actions, target, Some(&ev), + __handler_state, __actions, __target, Some(&ev), 1u8, __ev_id, 0u16, ); - __delivered = true; - break; + } else { + #observe_event_dropped } - __cur = __PARENT[s as usize]; - } - if !__delivered { + } else { #observe_event_dropped } } else { #observe_event_dropped } - return self.timers.min_remaining(); + return #timer_min_remaining_expr; } - self.timers.min_remaining() + #timer_min_remaining_expr } #[cfg(any(feature = "tokio", feature = "embassy"))] @@ -2252,25 +2719,12 @@ pub fn generate(ir: &Ir) -> TokenStream { if self.current.is_none() { #observe_started self.enter_path(0).await; - return self.timers.min_remaining(); + return #timer_min_remaining_expr; } - self.timers.decrement(elapsed); + #timer_decrement_call - if let Some((state, trig)) = self.timers.pop_expired(&__DEPTH) { - let __t_state = state; - let __t_trigger = trig; - #observe_timer_fired - if __duration_repeats(trig) { - let d = __duration_for(trig); - self.timers.start(state, trig, d); - let sid = state; - let tid = trig; - #observe_timer_armed - } - self.dispatch_trigger(state, 1, trig).await; - return self.timers.min_remaining(); - } + #timer_pop_block_async if let Some(ev) = self.queue.pop_front() { let __ev_id = __event_to_id(&ev); @@ -2281,31 +2735,32 @@ pub fn generate(ir: &Ir) -> TokenStream { return None; } if __ev_id != u16::MAX { - let mut __cur = self.current; - let mut __delivered = false; - while let Some(s) = __cur { - if let Some((actions, target)) = __handler_lookup(s, 0, __ev_id) { - let __handler_state = s; + // Same precomputed __DISPATCH lookup as the sync + // branch; see that copy for the encoding notes. + if let Some(__cur) = self.current { + let __dispatch_idx = + (__cur as usize) * #n_events + (__ev_id as usize); + let (__handler_state, __actions, __target) = + __DISPATCH[__dispatch_idx]; + if __handler_state != u16::MAX { #observe_event_delivered self.run_handlers( - s, actions, target, Some(&ev), + __handler_state, __actions, __target, Some(&ev), 1u8, __ev_id, 0u16, ).await; - __delivered = true; - break; + } else { + #observe_event_dropped } - __cur = __PARENT[s as usize]; - } - if !__delivered { + } else { #observe_event_dropped } } else { #observe_event_dropped } - return self.timers.min_remaining(); + return #timer_min_remaining_expr; } - self.timers.min_remaining() + #timer_min_remaining_expr } /// Push an event directly into the machine's queue (synchronous, @@ -2488,143 +2943,48 @@ pub fn generate(ir: &Ir) -> TokenStream { __dispatch(&mut actx, aid, event).await; } - // §2.6: classify transitions by the relationship between the - // innermost active state `I` and the target `T`: - // - Up-transition: T is already active (T is a strict ancestor - // of I). Unwind the subtree strictly below T; do NOT exit or - // re-enter T; do NOT descend into defaults. You cannot enter - // a state you never left. - // - Self-transition (T == I): external semantics — exit T and - // re-enter it, then descend into defaults if T has children. - // - Lateral: standard LCA semantics. - fn is_up_transition(current: Option, target: u16) -> bool { - let Some(mut node) = current else { return false; }; - if node == target { return false; } - while let Some(p) = __PARENT[node as usize] { - if p == target { return true; } - node = p; - } - false - } - + // §2.6 transition shapes (up / self / lateral) are folded + // into the precomputed path tables emitted at codegen time. + // An empty enter slice (`m == e`) means up-transition: target + // is already on the active path, must not be re-entered, and + // `self.current` is set to `target` after the exit loop. A + // non-empty enter slice ends at the deepest default-descended + // leaf; `enter_state_no_descent` sets `self.current` as it + // goes, so no fixup is needed on that path. #[cfg(not(any(feature = "tokio", feature = "embassy")))] fn transition(&mut self, _source_matched_state: u16, target: u16) { - if Self::is_up_transition(self.current, target) { - if let Some(mut cur) = self.current { - while cur != target { - self.exit_state(cur); - match __PARENT[cur as usize] { - Some(p) => cur = p, - None => break, - } - } - } + let from = self.current.unwrap_or(target); + let idx = (from as usize) * #n_states + (target as usize); + let (s, m, e) = __PATH_RANGE[idx]; + let exit_path = &__PATH_DATA[s as usize..m as usize]; + let enter_path = &__PATH_DATA[m as usize..e as usize]; + for &sid in exit_path { + self.exit_state(sid); + } + for &sid in enter_path { + self.enter_state_no_descent(sid); + } + if enter_path.is_empty() { self.current = Some(target); - return; } - - let mut lca = Self::lca(self.current.unwrap_or(target), target); - if self.current == Some(target) { - // Self-transition: bump LCA so target is exited and re-entered. - lca = __PARENT[target as usize]; - } - if let Some(mut cur) = self.current { - loop { - if Some(cur) == lca { break; } - self.exit_state(cur); - match __PARENT[cur as usize] { - Some(p) => cur = p, - None => break, - } - } - } - let mut path: ::hsmc::__private::heapless::Vec = - ::hsmc::__private::heapless::Vec::new(); - let mut node = Some(target); - while node != lca && node.is_some() { - let n = node.unwrap(); - let _ = path.push(n); - node = __PARENT[n as usize]; - } - for i in (0..path.len()).rev() { - let s = path[i]; - self.enter_state_no_descent(s); - } - self.descend_defaults(target); } #[cfg(any(feature = "tokio", feature = "embassy"))] async fn transition(&mut self, _source_matched_state: u16, target: u16) { - if Self::is_up_transition(self.current, target) { - if let Some(mut cur) = self.current { - while cur != target { - self.exit_state(cur).await; - match __PARENT[cur as usize] { - Some(p) => cur = p, - None => break, - } - } - } + let from = self.current.unwrap_or(target); + let idx = (from as usize) * #n_states + (target as usize); + let (s, m, e) = __PATH_RANGE[idx]; + let exit_path = &__PATH_DATA[s as usize..m as usize]; + let enter_path = &__PATH_DATA[m as usize..e as usize]; + for &sid in exit_path { + self.exit_state(sid).await; + } + for &sid in enter_path { + self.enter_state_no_descent(sid).await; + } + if enter_path.is_empty() { self.current = Some(target); - return; } - - let mut lca = Self::lca(self.current.unwrap_or(target), target); - if self.current == Some(target) { - lca = __PARENT[target as usize]; - } - if let Some(mut cur) = self.current { - loop { - if Some(cur) == lca { break; } - self.exit_state(cur).await; - match __PARENT[cur as usize] { - Some(p) => cur = p, - None => break, - } - } - } - let mut path: ::hsmc::__private::heapless::Vec = - ::hsmc::__private::heapless::Vec::new(); - let mut node = Some(target); - while node != lca && node.is_some() { - let n = node.unwrap(); - let _ = path.push(n); - node = __PARENT[n as usize]; - } - for i in (0..path.len()).rev() { - let s = path[i]; - self.enter_state_no_descent(s).await; - } - self.descend_defaults(target).await; - } - - // Lowest common ancestor via depth-walk. The chart's - // `__DEPTH` table is precomputed at codegen time, so we can - // align both walkers to the same depth and then ascend in - // lockstep until they meet. O(H) time, O(1) space — no - // ancestry-set allocation, no hierarchy ceiling. - fn lca(mut a: u16, mut b: u16) -> Option { - let mut da = __DEPTH[a as usize]; - let mut db = __DEPTH[b as usize]; - while da > db { - match __PARENT[a as usize] { - Some(p) => { a = p; da -= 1; } - None => return None, - } - } - while db > da { - match __PARENT[b as usize] { - Some(p) => { b = p; db -= 1; } - None => return None, - } - } - while a != b { - match (__PARENT[a as usize], __PARENT[b as usize]) { - (Some(pa), Some(pb)) => { a = pa; b = pb; } - _ => return None, - } - } - Some(a) } #[cfg(not(any(feature = "tokio", feature = "embassy")))] @@ -2708,7 +3068,7 @@ pub fn generate(ir: &Ir) -> TokenStream { for &tid in __owned_timers(sid) { #observe_timer_cancelled } - self.timers.cancel_state(sid); + #timer_cancel_state_call let exits = Self::exits_of(sid); for &aid in exits { #observe_action_exit @@ -2726,7 +3086,7 @@ pub fn generate(ir: &Ir) -> TokenStream { for &tid in __owned_timers(sid) { #observe_timer_cancelled } - self.timers.cancel_state(sid); + #timer_cancel_state_call let exits = Self::exits_of(sid); for &aid in exits { #observe_action_exit @@ -2736,12 +3096,14 @@ pub fn generate(ir: &Ir) -> TokenStream { #observe_exited } + #[inline] fn entries_of(sid: u16) -> &'static [u16] { match sid { #(#entries_of_arms)* _ => &[], } } + #[inline] fn exits_of(sid: u16) -> &'static [u16] { match sid { #(#exits_of_arms)* @@ -2759,35 +3121,31 @@ pub fn generate(ir: &Ir) -> TokenStream { #[cfg(not(any(feature = "tokio", feature = "embassy")))] fn do_terminate(&mut self) { - if let Some(mut cur) = self.current { - loop { - self.exit_state(cur); - match __PARENT[cur as usize] { - Some(p) => cur = p, - None => break, - } + if let Some(cur) = self.current { + let (s, e) = __TERMINATE_RANGE[cur as usize]; + let path = &__TERMINATE_DATA[s as usize..e as usize]; + for &sid in path { + self.exit_state(sid); } } self.terminated = true; self.current = None; - while self.queue.pop_front().is_some() {} + self.queue.clear(); #observe_terminated } #[cfg(any(feature = "tokio", feature = "embassy"))] async fn do_terminate(&mut self) { - if let Some(mut cur) = self.current { - loop { - self.exit_state(cur).await; - match __PARENT[cur as usize] { - Some(p) => cur = p, - None => break, - } + if let Some(cur) = self.current { + let (s, e) = __TERMINATE_RANGE[cur as usize]; + let path = &__TERMINATE_DATA[s as usize..e as usize]; + for &sid in path { + self.exit_state(sid).await; } } self.terminated = true; self.current = None; - while self.queue.pop_front().is_some() {} + self.queue.clear(); #observe_terminated } diff --git a/hsmc/benches/baseline.txt b/hsmc/benches/baseline.txt index 7e9c16a..7c0f511 100644 --- a/hsmc/benches/baseline.txt +++ b/hsmc/benches/baseline.txt @@ -1,84 +1,176 @@ # hsmc dispatch baseline — instruction counts from iai-callgrind. # -# Current numbers reflect the codebase after the LCA O(H) depth-walk fix -# (replaced the original O(H²) heapless::Vec::contains scan). +# Current numbers reflect the codebase after the terminate-path + u64-ns +# pass. `do_terminate()` indexes into a precomputed `__TERMINATE_DATA` +# table (one slice per starting state, root inclusive); `TimerEntry`'s +# `remaining_ns` field shrunk from u128 to u64 (~584 years of headroom), +# halving the entry size and replacing per-step u128 saturating-sub work +# with u64. The smaller machine struct improves cache density on every +# bench, even those that don't use timers. +# # Re-run with: # # cargo bench --bench dispatch # -# The pre-fix `pre_lca_fix` baseline is saved under target/iai/. To diff -# a future change against the current numbers, save a fresh named baseline -# before the change: +# Named baselines under target/iai/ for diffing past changes: +# pre_lca_fix — original O(H²) LCA contains() scan +# before_path_precompute — post-LCA-fix, pre-precompute +# before_timer_skip — post-precompute, pre-timer-skip (runtime is_empty) +# before_dispatch_precompute — post-timer-skip, pre-dispatch-table +# before_timer_elide — post-dispatch-table, pre-codegen-elide +# before_terminate_and_timer_opts — post-codegen-elide, pre-terminate-path / u64 +# +# To diff a future change against the current numbers: # # cargo bench --bench dispatch -- --save-baseline=before_ -# -# Then after, diff with: -# +# # cargo bench --bench dispatch -- --baseline=before_ # # Numbers below are from a 6.19.14-zen kernel x86_64 box, valgrind 3.25.1, # iai-callgrind 0.16.1, --release. dispatch::dispatch::h2_cold_start cold:setup_h2_cold() - Instructions: 51 - L1 Hits: 79 - LL Hits: 0 - RAM Hits: 7 - Estimated Cycles: 324 + Instructions: 38 + Estimated Cycles: 201 dispatch::dispatch::h2_warm_lateral warm:setup_h2_warm() - Instructions: 210 - L1 Hits: 268 - LL Hits: 0 - RAM Hits: 21 - Estimated Cycles: 1003 + Instructions: 84 + Estimated Cycles: 397 dispatch::dispatch::h4_cross_tree cross:setup_h4_warm() - Instructions: 394 - L1 Hits: 497 - LL Hits: 0 - RAM Hits: 26 - Estimated Cycles: 1407 + Instructions: 101 + Estimated Cycles: 456 dispatch::dispatch::h4_up up:setup_h4_warm() - Instructions: 203 - L1 Hits: 270 - LL Hits: 0 - RAM Hits: 16 - Estimated Cycles: 830 + Instructions: 98 + Estimated Cycles: 384 dispatch::dispatch::h4_self self_:setup_h4_warm() - Instructions: 233 - L1 Hits: 297 - LL Hits: 0 - RAM Hits: 24 - Estimated Cycles: 1137 + Instructions: 101 + Estimated Cycles: 456 dispatch::dispatch::h8_cross_tree cross:setup_h8_warm() - Instructions: 508 - L1 Hits: 636 - LL Hits: 0 - RAM Hits: 27 - Estimated Cycles: 1581 + Instructions: 93 + Estimated Cycles: 443 + +dispatch::dispatch::bubble_leaf leaf:setup_bubble_warm() + Instructions: 99 + Estimated Cycles: 453 + +dispatch::dispatch::bubble_mid mid:setup_bubble_warm() + Instructions: 99 + Estimated Cycles: 453 + +dispatch::dispatch::bubble_top top:setup_bubble_warm() + Instructions: 99 + Estimated Cycles: 453 + +dispatch::dispatch::timer_fire_dispatch fire:setup_timer_fire_armed() + Instructions: 191 + Estimated Cycles: 706 + +dispatch::dispatch::terminate_event quit:setup_term_warm() + Instructions: 69 + Estimated Cycles: 311 + +dispatch::dispatch::bubble_cross_tree switch:setup_bubble_cross_warm() + Instructions: 94 + Estimated Cycles: 444 dispatch::dispatch::chain_self_emit chain:setup_chain_warm() - Instructions: 1036 - L1 Hits: 1348 - LL Hits: 1 - RAM Hits: 33 - Estimated Cycles: 2508 + Instructions: 419 + Estimated Cycles: 1301 -# ─── LCA O(H) fix delta (pre_lca_fix → current) ───────────────────── +# ─── LCA O(H) fix delta (pre_lca_fix → before_path_precompute) ────── # # Bench Instr Δ Cycle Δ -# h2_cold_start -1.9% -0.3% (priming step, doesn't use LCA — noise) # h2_warm_lateral -22.8% -22.1% # h4_cross_tree -25.9% -10.6% -# h4_up +1.0% +0.4% (up-transition doesn't use LCA — noise) # h4_self -21.5% -11.7% -# h8_cross_tree -46.2% -41.0% ← headline: deepest LCA stress +# h8_cross_tree -46.2% -41.0% ← deepest LCA stress # chain_self_emit -15.4% -10.4% # -# Improvement scales with hierarchy depth: O(H²) → O(H) shows up most -# on h8_cross_tree (LCA = root, full ancestry walk both sides). Up- and -# cold-start paths show ±1% noise — they don't go through lca(). +# ─── Precompute-paths delta (before_path_precompute → before_timer_skip) ── +# +# Bench Instr Δ Cycle Δ +# h2_warm_lateral -19.5% -4.8% +# h4_cross_tree -45.0% -28.6% +# h8_cross_tree -48.6% -34.8% +# chain_self_emit -28.1% -14.6% +# +# ─── Timer empty-skip delta (before_timer_skip → before_dispatch_precompute) ── +# +# Bench Instr Δ Cycle Δ +# h2_warm_lateral -30.8% -35.1% +# h4_cross_tree -38.4% -35.4% +# h8_cross_tree -54.4% -52.9% +# chain_self_emit -27.9% -29.2% +# +# ─── Dispatch-table delta (before_dispatch_precompute → before_timer_elide) ── +# +# Bench Instr Δ Cycle Δ +# bubble_mid -19.4% -4.7% 3-hop __PARENT walk → table lookup +# bubble_top -22.3% -5.8% 4-hop walk → table lookup +# h8_cross_tree +4.2% +15.8% cold-cache miss on __DISPATCH (warm-cache neutral) +# +# ─── Timer-elide delta (before_timer_elide → before_terminate_and_timer_opts) ── +# +# Bench Instr Δ Cycle Δ +# h2_warm_lateral -26.9% -9.7% +# h4_cross_tree -22.2% -20.1% +# h8_cross_tree -24.2% -20.8% +# bubble_top -22.1% -23.2% +# chain_self_emit -18.0% -8.4% +# +# ─── Terminate-path + u64-ns delta (before_terminate_and_timer_opts → current) ── +# +# Bench Instr Δ Cycle Δ Notes +# h2_cold_start -2.6% -14.1% smaller machine struct, tighter cache +# h2_warm_lateral -3.4% -21.2% (RAM hits 11→8) +# h4_cross_tree -3.8% -1.7% +# h4_up -3.9% -22.3% (RAM hits 10→7) +# h4_self -3.8% -8.4% +# h8_cross_tree -1.1% -0.7% +# bubble_leaf -6.6% -9.6% +# bubble_mid -6.6% -3.0% +# bubble_top -6.6% -9.6% +# timer_fire_dispatch +2.7% -12.6% smaller TimerEntry → -3 RAM hits +# terminate_event -22.5% -23.4% ← headline: precomputed exit path + queue.clear() +# bubble_cross_tree -2.1% -0.9% +# chain_self_emit -0.9% -3.1% +# +# Two changes in this pass: +# +# (1) `__TERMINATE_DATA` / `__TERMINATE_RANGE` — per-state precomputed +# exit chain up to and including __Root. `do_terminate()` indexes the +# table once and iterates the slice, replacing the runtime `__PARENT` +# walk + per-iteration match. Also swapped the queue-drain loop +# (`while self.queue.pop_front().is_some() {}`) for `self.queue.clear()`, +# which heapless 0.8 implements as O(1) (resets indices, no per-element +# pop). Together: -22% instr / -23% cycles on terminate_event, the +# headline win the user asked for. +# +# (2) `TimerEntry.remaining_ns: u128 → u64`. Internal-only field type +# change. u64 nanoseconds bounds at ~584 years — past any realistic +# Duration. Eliminates u128 saturating-sub in `decrement()` (4-6 instr +# → 1-2 instr per entry per step) and the manual secs/nanos +# decomposition in `min_remaining()` (replaced with +# `Duration::from_nanos(u64)`). The struct shrinks 32 B → 16 B with +# alignment, so the entire machine struct is smaller — even no-timer +# benches see RAM-hit reductions from tighter cache packing. +# +# Net across the six optimization passes (pre_lca_fix → current): +# h8_cross_tree dispatch is **-82% instructions / -72% cycles** vs the +# original O(H²) LCA scan. terminate_event is now 69 instr / 311 cycles +# — about 80% of that is the iai-callgrind wrapper, leaving ~14 instr +# of actual HSM termination work for a 3-deep chart. +# +# ─── Bench coverage (3 benches added in the previous pass) ────────── +# +# `timer_fire_dispatch` (191 instr / 706 cycles) — only bench that +# exercises pop_expired's non-empty branch and `dispatch_trigger`. +# `terminate_event` (69 instr / 311 cycles) — only bench that +# exercises `__check_terminate`'s true arm and `do_terminate`. +# `bubble_cross_tree` (94 instr / 444 cycles) — handler on L1 (4 +# bubble hops) firing a 10-state cross-tree transition. Realistic +# HSM worst case. diff --git a/hsmc/benches/dispatch.rs b/hsmc/benches/dispatch.rs index db07ca5..c6f8423 100644 --- a/hsmc/benches/dispatch.rs +++ b/hsmc/benches/dispatch.rs @@ -21,12 +21,12 @@ fn main() { eprintln!("hsmc dispatch benches require Linux (iai-callgrind + valgrind)."); } +#[cfg(target_os = "linux")] +use hsmc::{statechart, Duration}; #[cfg(target_os = "linux")] use iai_callgrind::{library_benchmark, library_benchmark_group, main}; #[cfg(target_os = "linux")] use std::hint::black_box; -#[cfg(target_os = "linux")] -use hsmc::{statechart, Duration}; // ────────────────────────────────────────────────────────────────────── // Chart 1: H=2 sibling. LCA(A, B) = P. Two states deep. @@ -207,6 +207,199 @@ mod chain { } } +// ────────────────────────────────────────────────────────────────────── +// Chart 5: handler bubbling. Active state is L4 (depth 5). The three +// events resolve to handlers at three different ancestor depths so the +// __PARENT walk in step() is exercised at increasing distance: +// +// Leaf — handler on L4 itself → bubble depth 0 +// Mid — handler on L1 (3 ancestors) → bubble depth 3 +// Top — handler on Outer (4 anc.) → bubble depth 4 +// +// `Sibling` is the transition target so each event causes the same +// dispatch shape; the only thing that varies between benches is the +// bubble distance to the handler. +// ────────────────────────────────────────────────────────────────────── +#[cfg(target_os = "linux")] +mod bubble { + use super::*; + + #[derive(Default)] + pub struct Ctx; + + #[derive(Debug, Clone)] + pub enum Ev { + Leaf, + Mid, + Top, + } + + statechart! { + Bubble { + context: Ctx; + events: Ev; + default(Outer); + + state Outer { + default(L1); + on(Top) => Sibling; + state L1 { + default(L2); + on(Mid) => Sibling; + state L2 { + default(L3); + state L3 { + default(L4); + state L4 { + on(Leaf) => Sibling; + } + } + } + } + } + state Sibling { entry: noop; } + } + } + + impl BubbleActions for BubbleActionContext<'_> { + fn noop(&mut self) {} + } +} + +// ────────────────────────────────────────────────────────────────────── +// Chart 6: timer firing. A single timer armed for 1 ns; the bench body +// advances time past expiry so `pop_expired` returns Some and dispatches +// the timer-driven transition. This is the only bench that exercises +// `step()`'s pop_expired non-empty branch — every other chart in this +// suite declares no `Duration` triggers and (post timer-elide pass) has +// the timer-table machinery removed at codegen. +// ────────────────────────────────────────────────────────────────────── +#[cfg(target_os = "linux")] +mod timer_fire { + use super::*; + + #[derive(Default)] + pub struct Ctx; + + statechart! { + TimerFire { + context: Ctx; + default(Armed); + + state Armed { + on(after Duration::from_nanos(1)) => Fired; + } + state Fired { entry: noop; } + } + } + + impl TimerFireActions for TimerFireActionContext<'_> { + fn noop(&mut self) {} + } +} + +// ────────────────────────────────────────────────────────────────────── +// Chart 7: terminate event. `Quit` is declared as the terminate event, +// so dispatching it runs `do_terminate()` — exits every active state up +// to the root, sets `terminated`, and short-circuits future `step()`s. +// Only bench that exercises `__check_terminate`'s true arm and the +// terminate-on-event path. +// ────────────────────────────────────────────────────────────────────── +#[cfg(target_os = "linux")] +mod term { + use super::*; + + #[derive(Default)] + pub struct Ctx; + + #[derive(Debug, Clone)] + pub enum Ev { + Quit, + } + + statechart! { + Term { + context: Ctx; + events: Ev; + default(L1); + terminate(Quit); + + state L1 { + default(L2); + state L2 { + default(L3); + state L3 { entry: noop; } + } + } + } + } + + impl TermActions for TermActionContext<'_> { + fn noop(&mut self) {} + } +} + +// ────────────────────────────────────────────────────────────────────── +// Chart 8: bubble + cross-tree. Handler declared on L1 (4 ancestors up +// from current=L5) targets a deep leaf in a sibling subtree (R5). Two +// dispatch costs at once: the (now-O(1)) handler resolution and a +// 10-state-op cross-tree transition (5 exits + 5 enters). The +// realistic worst case for HSM dispatch — a parent's "back to default" +// handler that descends deep on the other side. +// ────────────────────────────────────────────────────────────────────── +#[cfg(target_os = "linux")] +mod bubble_cross { + use super::*; + + #[derive(Default)] + pub struct Ctx; + + #[derive(Debug, Clone)] + pub enum Ev { + Switch, + } + + statechart! { + BubbleCross { + context: Ctx; + events: Ev; + default(L1); + + state L1 { + default(L2); + on(Switch) => R5; + state L2 { + default(L3); + state L3 { + default(L4); + state L4 { + default(L5); + state L5 { entry: noop; } + } + } + } + } + state R1 { + default(R2); + state R2 { + default(R3); + state R3 { + default(R4); + state R4 { + default(R5); + state R5 { entry: noop; } + } + } + } + } + } + } + + impl BubbleCrossActions for BubbleCrossActionContext<'_> { + fn noop(&mut self) {} + } +} + // ────────────────────────────────────────────────────────────────────── // Setup helpers. Prime each chart to its measurement starting state // OUTSIDE the measured region — iai-callgrind only counts work inside @@ -246,6 +439,34 @@ fn setup_chain_warm() -> chain::Chain<8> { m } +#[cfg(target_os = "linux")] +fn setup_bubble_warm() -> bubble::Bubble<8> { + let mut m = bubble::Bubble::new(bubble::Ctx); + let _ = m.step(Duration::ZERO); // priming step → Outer → L1 → L2 → L3 → L4 + m +} + +#[cfg(target_os = "linux")] +fn setup_timer_fire_armed() -> timer_fire::TimerFire<8> { + let mut m = timer_fire::TimerFire::new(timer_fire::Ctx); + let _ = m.step(Duration::ZERO); // priming step → enters Armed, arms 1ns timer + m +} + +#[cfg(target_os = "linux")] +fn setup_term_warm() -> term::Term<8> { + let mut m = term::Term::new(term::Ctx); + let _ = m.step(Duration::ZERO); // priming step → L1 → L2 → L3 + m +} + +#[cfg(target_os = "linux")] +fn setup_bubble_cross_warm() -> bubble_cross::BubbleCross<8> { + let mut m = bubble_cross::BubbleCross::new(bubble_cross::Ctx); + let _ = m.step(Duration::ZERO); // priming step → L1 → L2 → L3 → L4 → L5 + m +} + // ────────────────────────────────────────────────────────────────────── // Benches. One library_benchmark per scenario. Each takes its primed // chart by value, sends one event, and runs one step. @@ -305,6 +526,69 @@ fn h8_cross_tree(mut m: h8::H8<8>) -> h8::H8<8> { m } +#[cfg(target_os = "linux")] +#[library_benchmark] +#[bench::leaf(setup = setup_bubble_warm)] +fn bubble_leaf(mut m: bubble::Bubble<8>) -> bubble::Bubble<8> { + // Handler on L4 (current state). Bubble distance = 0. + let _ = black_box(m.send(black_box(bubble::Ev::Leaf))); + let _ = black_box(m.step(black_box(Duration::ZERO))); + m +} + +#[cfg(target_os = "linux")] +#[library_benchmark] +#[bench::mid(setup = setup_bubble_warm)] +fn bubble_mid(mut m: bubble::Bubble<8>) -> bubble::Bubble<8> { + // Handler on L1. Bubble walks L4 → L3 → L2 → L1 (3 __PARENT hops). + let _ = black_box(m.send(black_box(bubble::Ev::Mid))); + let _ = black_box(m.step(black_box(Duration::ZERO))); + m +} + +#[cfg(target_os = "linux")] +#[library_benchmark] +#[bench::top(setup = setup_bubble_warm)] +fn bubble_top(mut m: bubble::Bubble<8>) -> bubble::Bubble<8> { + // Handler on Outer. Bubble walks L4 → L3 → L2 → L1 → Outer (4 hops). + let _ = black_box(m.send(black_box(bubble::Ev::Top))); + let _ = black_box(m.step(black_box(Duration::ZERO))); + m +} + +#[cfg(target_os = "linux")] +#[library_benchmark] +#[bench::fire(setup = setup_timer_fire_armed)] +fn timer_fire_dispatch(mut m: timer_fire::TimerFire<8>) -> timer_fire::TimerFire<8> { + // Advance time by 2ns — past the 1ns arm — so pop_expired returns + // Some and dispatches the timer-driven Armed → Fired transition. + let _ = black_box(m.step(black_box(Duration::from_nanos(2)))); + m +} + +#[cfg(target_os = "linux")] +#[library_benchmark] +#[bench::quit(setup = setup_term_warm)] +fn terminate_event(mut m: term::Term<8>) -> term::Term<8> { + // Send Quit (declared as terminate event); step runs check_terminate's + // true arm and do_terminate, which exits L3 → L2 → L1 → root. + let _ = black_box(m.send(black_box(term::Ev::Quit))); + let _ = black_box(m.step(black_box(Duration::ZERO))); + m +} + +#[cfg(target_os = "linux")] +#[library_benchmark] +#[bench::switch(setup = setup_bubble_cross_warm)] +fn bubble_cross_tree(mut m: bubble_cross::BubbleCross<8>) -> bubble_cross::BubbleCross<8> { + // Switch handler is on L1 (4 bubble hops from current=L5). Target is + // R5 in a sibling subtree, so the transition is exit L5..L1 + enter + // R1..R5 = 10 state operations. + let _ = black_box(m.send(black_box(bubble_cross::Ev::Switch))); + let _ = black_box(m.step(black_box(Duration::ZERO))); + m +} + #[cfg(target_os = "linux")] #[library_benchmark] #[bench::chain(setup = setup_chain_warm)] @@ -328,6 +612,12 @@ library_benchmark_group!( h4_up, h4_self, h8_cross_tree, + bubble_leaf, + bubble_mid, + bubble_top, + timer_fire_dispatch, + terminate_event, + bubble_cross_tree, chain_self_emit, ); diff --git a/hsmc/src/lib.rs b/hsmc/src/lib.rs index d4dc1d6..1ea24d8 100644 --- a/hsmc/src/lib.rs +++ b/hsmc/src/lib.rs @@ -812,7 +812,12 @@ pub mod __private { pub struct TimerEntry { pub state: u16, pub trigger: u16, - pub remaining_ns: u128, + // u64 nanoseconds bounds at ~584 years of countdown — comfortably + // beyond any practical Duration. Storing the original u128 from + // `Duration::as_nanos()` would force u128 saturating_sub on every + // `decrement()` call; on 64-bit it's 4-6 instructions per entry vs + // 1-2 for u64. The ceiling is enforced by saturating-cast at insert. + pub remaining_ns: u64, } impl Default for TimerTable { @@ -828,7 +833,12 @@ pub mod __private { } } pub fn start(&mut self, state: u16, trigger: u16, duration: core::time::Duration) { - let ns = duration.as_nanos(); + // Truncating u128→u64. u64 nanoseconds bounds at ~584 years — + // past that the cast wraps, but no realistic timer duration + // gets near it (Duration::from_secs(u64::MAX) overflows long + // before, and from_millis/from_nanos take u64 directly so + // round-trip exactly). + let ns = duration.as_nanos() as u64; // Replace existing entry for same (state, trigger) — resets on re-entry. for e in self.entries.iter_mut() { if e.state == state && e.trigger == trigger { @@ -842,22 +852,51 @@ pub mod __private { remaining_ns: ns, }); } + /// Cancel every timer owned by `state`. Inlined empty-check so an + /// `exit_state` on a chart with no live timers compiles to a single + /// branch — the `retain` call (and its closure) stays out-of-line. + #[inline] pub fn cancel_state(&mut self, state: u16) { - self.entries.retain(|e| e.state != state); + if !self.entries.is_empty() { + self.entries.retain(|e| e.state != state); + } } pub fn cancel_one(&mut self, state: u16, trigger: u16) { self.entries .retain(|e| !(e.state == state && e.trigger == trigger)); } + /// Subtract `elapsed` from every live timer (saturating at 0). Inlined + /// empty-check so the per-`step()` call costs one branch when no + /// timers are armed. + #[inline] pub fn decrement(&mut self, elapsed: core::time::Duration) { - let ns = elapsed.as_nanos(); + if self.entries.is_empty() { + return; + } + let ns = elapsed.as_nanos() as u64; for e in self.entries.iter_mut() { e.remaining_ns = e.remaining_ns.saturating_sub(ns); } } /// Returns (state, trigger) of first expired timer, picking the innermost /// (deepest) state, ties broken by declaration order. + /// + /// The empty-table case is split out and `#[inline]`d so charts with no + /// live timers pay only a one-instruction `is_empty` branch per + /// `step()`, not the full function-call + iterator-setup overhead. The + /// non-empty body stays out-of-line to keep the hot path's icache + /// footprint small. + #[inline] pub fn pop_expired(&mut self, depth: &[u8]) -> Option<(u16, u16)> { + if self.entries.is_empty() { + None + } else { + self.pop_expired_nonempty(depth) + } + } + + #[cold] + fn pop_expired_nonempty(&mut self, depth: &[u8]) -> Option<(u16, u16)> { let mut best: Option = None; for (i, e) in self.entries.iter().enumerate() { if e.remaining_ns == 0 { @@ -880,12 +919,26 @@ pub mod __private { None } } + + /// Smallest `remaining_ns` across live timers, or `None` if there are + /// none. Same inline-the-empty-case treatment as `pop_expired` so + /// timer-free charts pay only a branch per `step()`. + #[inline] pub fn min_remaining(&self) -> Option { - self.entries.iter().map(|e| e.remaining_ns).min().map(|ns| { - let secs = (ns / 1_000_000_000) as u64; - let rem = (ns % 1_000_000_000) as u32; - core::time::Duration::new(secs, rem) - }) + if self.entries.is_empty() { + None + } else { + self.min_remaining_nonempty() + } + } + + #[cold] + fn min_remaining_nonempty(&self) -> Option { + self.entries + .iter() + .map(|e| e.remaining_ns) + .min() + .map(core::time::Duration::from_nanos) } }