diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 000000000..df398fdfb --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,81 @@ +# Dioxus Agent Guide + +Dioxus is a cross-platform UI framework for Rust, similar to React. It compiles to web (WASM), desktop (webview), mobile (iOS/Android), and native (GPU-rendered). + +## Quick Overview + +- **Language**: Rust (stable toolchain) +- **UI Model**: React-like with VirtualDOM, components, hooks, signals +- **Syntax**: JSX-like `rsx!` macro for declaring UI +- **Platforms**: Web, Desktop (Windows/macOS/Linux), Mobile, Native, LiveView (server-rendered) + +## Workspace Structure + +``` +packages/ +├── dioxus/ # Main re-export crate users depend on +├── core/ # VirtualDOM, components, diffing, scheduling +├── rsx/ # RSX macro parsing and code generation +├── rsx-hotreload/ # Template diffing for hot-reload +├── signals/ # Reactive state (Signal, Memo, Store) +├── hooks/ # Built-in hooks (use_signal, use_effect, etc.) +├── router/ # Type-safe routing with #[derive(Routable)] +├── fullstack/ # SSR, hydration, #[server] functions +├── cli/ # `dx` build tool, dev server, bundling +├── web/ # WASM renderer +├── desktop/ # Wry/Tao webview renderer +├── native/ # Blitz/Vello GPU renderer +├── liveview/ # WebSocket streaming renderer +├── manganis/ # asset!() macro for compile-time assets +├── subsecond/ # Hot-patching system (jump table indirection) +├── devtools/ # Dev server communication protocol +├── interpreter/ # Sledgehammer JS for DOM mutations +└── wasm-split/ # WASM code splitting +``` + +## Architecture Documentation + +For deeper understanding, see `notes/architecture/`: + +| When working on... | Read... | +| ------------------------------------------ | ------------------ | +| VirtualDOM, components, diffing, events | `01-CORE.md` | +| CLI, build system, bundling, dev server | `02-CLI.md` | +| RSX macro, parsing, formatting | `03-RSX.md` | +| Signals, state management, reactivity | `04-SIGNALS.md` | +| Server functions, SSR, hydration | `05-FULLSTACK.md` | +| Web/desktop/native/liveview renderers | `06-RENDERERS.md` | +| Hot-reload, hot-patching, devtools | `07-HOTRELOAD.md` | +| Asset macro, manganis, const serialization | `08-ASSETS.md` | +| Router, navigation, nested routes | `09-ROUTER.md` | +| WASM code splitting | `10-WASM-SPLIT.md` | + +## Key Concepts + +- **VirtualDOM**: Tree of `VNode` with templates, dynamic nodes, and attributes +- **Signals**: Copy-able reactive primitives via generational-box (generation-based validity) +- **WriteMutations**: Trait that renderers implement to apply DOM changes +- **RSX**: Proc macro that compiles JSX-like syntax to `VNode` construction +- **Server Functions**: `#[server]` macro generates client RPC stubs and server handlers +- **Subsecond**: Hot-patches Rust code via jump table indirection (no memory modification) + +## Common Patterns + +**Component definition**: +```rust +#[component] +fn MyComponent(name: String) -> Element { + let mut count = use_signal(|| 0); + rsx! { + button { onclick: move |_| count += 1, "{name}: {count}" } + } +} +``` + +## Notes for Agents + +1. The `dioxus` crate re-exports from other crates - most implementation is in `packages/core`, `packages/signals`, etc. +2. RSX macro expansion happens in `packages/rsx` - look there for syntax questions +3. Each renderer implements `WriteMutations` differently - see `06-RENDERERS.md` +4. Hot-reload has two systems: RSX template diffing (fast) and Subsecond code patching (full Rust) +5. Assets use link sections and binary patching - the `asset!()` macro creates symbols the CLI processes diff --git a/notes/architecture/00-OVERVIEW.md b/notes/architecture/00-OVERVIEW.md new file mode 100644 index 000000000..cd65b7e81 --- /dev/null +++ b/notes/architecture/00-OVERVIEW.md @@ -0,0 +1,107 @@ +# Dioxus Architecture Overview + +This directory contains comprehensive architecture documentation for the Dioxus framework, designed to help future Claude agents (and human contributors) understand the codebase deeply. + +## Document Index + +| File | Description | +|------|-------------| +| `01-CORE.md` | VirtualDOM, rendering pipeline, component system, diffing algorithm | +| `02-CLI.md` | Build system, dev server, bundling, terminal UI, platform support | +| `03-RSX.md` | RSX macro parsing, AST types, code generation, autofmt | +| `04-SIGNALS.md` | Signals, generational-box, hooks, stores, state management | +| `05-FULLSTACK.md` | Server functions, SSR, hydration, client-server communication | +| `06-RENDERERS.md` | Web, desktop, native, liveview renderer implementations | +| `07-HOTRELOAD.md` | Subsecond hot-patching, RSX hot-reload, devtools protocol | +| `08-ASSETS.md` | Manganis asset system, const serialization, build-time processing | +| `09-ROUTER.md` | Routing system, Routable trait, navigation, history | +| `10-WASM-SPLIT.md` | WASM code splitting, lazy loading, chunk management | + +## Crate Dependency Overview + +``` +dioxus (umbrella crate) +├── dioxus-core (VirtualDOM, runtime, scheduler) +│ └── generational-box (Copy semantics for references) +├── dioxus-rsx (macro parsing) +│ └── dioxus-autofmt (code formatting) +├── dioxus-signals (reactive state) +│ ├── generational-box +│ └── dioxus-stores (nested reactivity) +├── dioxus-hooks (use_signal, use_effect, etc.) +├── dioxus-html (HTML elements, events) +├── dioxus-router (client-side routing) +│ +├── Renderers: +│ ├── dioxus-web (WASM/browser) +│ ├── dioxus-desktop (wry/tao webview) +│ ├── dioxus-native (blitz/vello GPU) +│ ├── dioxus-liveview (WebSocket streaming) +│ └── dioxus-ssr (server-side rendering) +│ +├── Fullstack: +│ ├── dioxus-fullstack (client/server coordination) +│ ├── dioxus-fullstack-core (transport, types) +│ ├── dioxus-fullstack-macro (#[server] macro) +│ └── dioxus-server (Axum integration) +│ +├── CLI (dx command): +│ ├── dioxus-cli (main binary) +│ ├── dioxus-cli-config (Dioxus.toml parsing) +│ └── dioxus-cli-opt (optimization passes) +│ +├── Hot Reload: +│ ├── subsecond (hot-patching runtime) +│ ├── dioxus-rsx-hotreload (template diffing) +│ └── dioxus-devtools (WebSocket communication) +│ +├── Assets: +│ ├── manganis (asset!() macro) +│ ├── manganis-core (asset types) +│ ├── manganis-macro (proc-macro) +│ └── const-serialize (compile-time CBOR) +│ +└── Code Splitting: + ├── wasm-splitter (CLI tool) + ├── wasm-split-macro (#[wasm_split]) + └── wasm-split-cli (binary processing) +``` + +## Key Architectural Patterns + +### 1. Copy Semantics for Reactive State +Dioxus uses `generational-box` to provide `Copy` semantics for references. This enables signals and other reactive primitives to be freely passed around without explicit cloning. + +### 2. WriteMutations Trait +All renderers implement `WriteMutations` which defines how VirtualDOM changes translate to real DOM operations. This abstraction enables the same component code to run on web, desktop, mobile, and server. + +### 3. Template-Based Rendering +RSX macros generate static `Template` definitions at compile time. Only dynamic parts are diffed at runtime, making updates extremely efficient. + +### 4. Hot-Reload Without Full Rebuild +Two complementary systems: +- **RSX Hot-Reload**: Changes to template literals are diffed and sent via WebSocket +- **Subsecond Hot-Patching**: Full Rust function replacement via jump table indirection + +### 5. Compile-Time Asset Processing +The `asset!()` macro embeds asset metadata in binary link sections. The CLI extracts this at build time, processes assets, and patches the binary with final URLs. + +### 6. Server Functions as RPC +The `#[server]` macro generates dual code paths - client serialization and server handler - enabling seamless RPC-style communication. + +## Important Design Decisions + +1. **Single VirtualDOM**: Even multi-window desktop apps share one VirtualDOM for simpler state management +2. **Scope-Based Cleanup**: All state (signals, tasks, contexts) is tied to component scope lifetime +3. **Platform Agnostic Events**: Events are converted through `HtmlEventConverter` trait for cross-platform support +4. **Binary Protocols**: Desktop and liveview use Sledgehammer binary format for compact mutation encoding +5. **Generational GC**: References are invalidated by generation counters, preventing use-after-free without runtime overhead per access + +## Future Development Areas + +These docs are designed to help with planned features: +- **Custom Bundlers**: Replacing tauri-bundler with native implementation +- **CLI Tunnels**: Remote device hot-reloading via SSH/SCP +- **Workspace Hot-Patching**: Patching library crates, not just the tip crate +- **Universal Android Builds**: Single APK for all architectures +- **Fullstack Extensions**: First-party database, auth, IAAC, secrets diff --git a/notes/architecture/01-CORE.md b/notes/architecture/01-CORE.md new file mode 100644 index 000000000..c9799c430 --- /dev/null +++ b/notes/architecture/01-CORE.md @@ -0,0 +1,311 @@ +# Dioxus Core - VirtualDOM Architecture + +The `dioxus-core` crate is the heart of Dioxus, implementing the virtual DOM, component lifecycle, rendering pipeline, and reactive runtime. + +## VirtualDom Structure + +The `VirtualDom` struct orchestrates the entire reactive component tree: + +``` +VirtualDom +├── scopes: Slab // Arena allocator for all component scopes +├── dirty_scopes: BTreeSet // Scopes marked for re-render (height-ordered) +├── runtime: Rc // Shared async runtime +├── resolved_scopes: Vec // Scopes resolved during suspense +└── rx: UnboundedReceiver // Event channel from scheduler +``` + +### Key Methods + +- `VirtualDom::new(app)` - Create new DOM with root component +- `rebuild()` / `rebuild_in_place()` - Full tree reconstruction +- `render_immediate()` - Render all dirty scopes without suspense blocking +- `wait_for_work()` - Async poll for futures and scheduler events +- `wait_for_suspense()` - Block until all suspended futures complete +- `mark_dirty(scope_id)` - Schedule scope for re-render +- `with_root_context()` - Inject dependency into root scope + +## Rendering Pipeline + +### Initial Render Flow +1. `rebuild()` calls `run_scope(ScopeId::ROOT)` +2. Result wraps in `LastRenderedNode` and calls `create_scope()` +3. `create_scope()` recursively creates all child scopes and DOM elements +4. Mutations written to `WriteMutations` sink + +### Update Render Flow +1. `wait_for_work()` polls scheduler and pending futures +2. `process_events()` consumes `SchedulerMsg::Immediate(scope_id)` messages +3. `render_immediate()` pops dirty scopes in height order (top-to-bottom) +4. For each scope: `run_and_diff_scope()` executes component, then `diff_scope()` +5. Diff generates mutations via `WriteMutations` trait + +## Component System + +### Component Definition +- `Component

` type alias: `fn(P) -> Element` +- `ComponentFunction` trait for component functions + - `fn_ptr(&self) -> usize` - Raw function pointer for identity + - `rebuild(&self, props: P) -> Element` - Execute component + +### Props System +- `Properties` trait for all props: + - `type Builder` - For DSL prop construction + - `builder()` - Create props builder + - `memoize(&mut self, other: &Self) -> bool` - Check if props changed + - `into_vcomponent()` - Convert to VComponent + +### Type Erasure +`VProps` wraps strongly-typed props, erased to `BoxedAnyProps` via `AnyProps` trait: +- `render(&self) -> Element` +- `memoize(&mut self, other: &dyn Any) -> bool` +- `props()` / `props_mut()` - Access as `dyn Any` +- `duplicate()` - Clone into new box + +## Scope System + +### ScopeId +Small unique identifier (usize index into Slab): +- `ScopeId::ROOT` (0) - Root wrapper scope +- `ScopeId::ROOT_SUSPENSE_BOUNDARY` (1) - Default suspense +- `ScopeId::ROOT_ERROR_BOUNDARY` (2) - Default error boundary +- `ScopeId::APP` (3) - User's root component + +### ScopeState (Public API) +``` +ScopeState +├── context_id: ScopeId +├── last_rendered_node: Option +├── props: BoxedAnyProps +└── reactive_context: ReactiveContext +``` + +### Scope (Internal State) +``` +Scope +├── name: &'static str // Component name for debugging +├── id: ScopeId +├── parent_id: Option +├── height: u32 // Distance from root +├── hooks: RefCell>> // Hook state storage +├── hook_index: Cell // Current hook being accessed +├── shared_contexts: RefCell>> +├── spawned_tasks: RefCell> +├── before_render / after_render // Callbacks +├── status: RefCell // Mounted/Unmounted +└── suspense_boundary: SuspenseLocation +``` + +### Context Propagation +- `provide_context(context: T)` - Store in scope +- `consume_context()` - Retrieve from this scope or any parent +- Lookup walks parent chain via `parent_id` + +## Event System + +### Event Structure +```rust +pub struct Event { + pub data: Rc, + pub(crate) metadata: Rc>, +} + +pub struct EventMetadata { + pub propagates: bool, + pub prevent_default: bool, +} +``` + +### Event Flow +1. `Runtime::handle_event()` looks up element by `ElementId` +2. Gets parent `ElementRef` from slab +3. If bubbles: `handle_bubbling_event()` walks parent chain +4. Collects listeners in path order, calls in reverse (parents first) +5. Breaks on `stop_propagation()` + +## Diffing Algorithm + +### Entry Point +`diff_scope()` gets old/new VNodes and calls `old.diff_node(new, dom, mutations)` + +### VNode Diffing +- **Template Check**: Different templates → replace entire subtree +- **Identity Check**: Same pointer → skip +- **Attribute Diffing**: `diff_attributes()` for dynamic attrs +- **Dynamic Node Diffing**: Recursively diff each dynamic node + +### Dynamic Node Cases +- Text → Text: `diff_vtext()` updates content +- Placeholder → Placeholder: No-op +- Fragment → Fragment: Recursively diff children +- Component → Component: Check render fn, then memoize props +- Type Change: Remove old, create new + +### Keyed List Diffing +1. **Prefix Pass**: Match keys from start +2. **Suffix Pass**: Match keys from end +3. **Middle Pass**: Handle insertions/deletions/moves +4. Uses `FxHashMap` for old→new key matching + +## Mutations System + +### WriteMutations Trait +Interface between VirtualDOM and real DOM: +- `append_children(id, count)` - Add N nodes to element +- `assign_node_id(path, id)` - Mark element at template path +- `create_placeholder(id)` - Create marker node +- `create_text_node(value, id)` - Create text node +- `load_template(template, index, id)` - Clone from template cache +- `replace_node_with(id, count)` - Replace element +- `set_attribute(name, ns, value, id)` - Update attribute +- `create_event_listener(name, id)` - Register listener +- `remove_node(id)` - Delete element + +### Mutation Enum (for testing/serialization) +```rust +pub enum Mutation { + AppendChildren { id, m }, + CreatePlaceholder { id }, + CreateTextNode { value, id }, + LoadTemplate { index, id }, + ReplaceWith { id, m }, + SetAttribute { name, ns, value, id }, + // ... etc +} +``` + +## Scheduler + +### Work Priority +1. **Dirty Scopes** (Highest): Re-render in height order +2. **Tasks**: Poll spawned futures +3. **Effects** (Lowest): Run after DOM changes + +### ScopeOrder +Composite key: `(height: u32, id: ScopeId)` in `BTreeSet` ensures parent scopes run before children. + +### Key Methods +- `queue_scope(order)` - Add to dirty_scopes +- `queue_task(task, order)` - Add to dirty_tasks +- `pop_work()` - Get next dirty scope or task +- `pop_effect()` - Get next pending effect + +## Runtime + +Manages async/scope/task coordination: +``` +Runtime +├── scope_states: RefCell>> +├── scope_stack: RefCell> +├── suspense_stack: RefCell> +├── tasks: RefCell>> +├── current_task: Cell> +├── dirty_tasks: RefCell> +├── pending_effects: RefCell> +├── rendering: Cell +├── sender: UnboundedSender +├── elements: RefCell>> +└── mounts: RefCell> +``` + +## Tasks & Async + +### Task Structure +- `id: TaskId` (slotmap key) +- `!Send + !Sync` marker + +### Task Methods +- `Task::new(future)` - Spawn new task +- `cancel()` - Remove task +- `pause()` / `resume()` - Control polling +- `wake()` - Wake sleeping task + +### LocalTask (Internal) +- Wraps `Pin>>` +- Custom waker that sends `SchedulerMsg::TaskNotified` +- Task dropped when owning scope drops + +## Effects + +Effects run AFTER mutations are applied to DOM: +1. Created via `use_effect()` in component +2. Queued in `Runtime::pending_effects` +3. After render, `finish_render()` sends `SchedulerMsg::EffectQueued` +4. `poll_tasks()` pops effects and calls `effect.run()` + +## Suspense + +### SuspenseContext +- Tracks suspended futures and placeholder nodes +- `suspended_tasks: RefCell>` +- `suspended_nodes: RefCell>` +- `frozen: Cell` for server-side locking + +### Suspension Flow +1. Component calls `suspend()` → `Err(SuspendedFuture)` +2. `Element::Err(RenderError::Suspended)` propagates up +3. Nearest `SuspenseBoundary` catches it +4. Boundary renders placeholder +5. Future added to boundary's tasks +6. Boundary marked dirty when future completes + +## VNodes & Templates + +### VNode Structure +```rust +pub struct VNode { + vnode: Rc, + mount: Cell, +} + +pub struct VNodeInner { + pub key: Option, + pub template: Template, + pub dynamic_nodes: Box<[DynamicNode]>, + pub dynamic_attrs: Box<[Box<[Attribute]>]>, +} +``` + +### Template (Static) +```rust +pub struct Template { + pub roots: &'static [TemplateNode], + pub node_paths: &'static [&'static [u8]], + pub attr_paths: &'static [&'static [u8]], +} +``` + +### TemplateNode Variants +- `Element { tag, namespace, attrs, children }` - Static element +- `Text { text }` - Static text +- `Dynamic { id }` - Index into dynamic_nodes + +### DynamicNode Variants +- `Component(VComponent)` - Child component +- `Text(VText)` - Text node +- `Placeholder(VPlaceholder)` - Suspense/empty marker +- `Fragment(Vec)` - Multiple children + +## Error Boundaries + +### ErrorContext +- `error: Rc>>` +- `subscribers: Subscribers` for listening components + +### Error Flow +1. Component returns `Element::Err(error)` +2. Nearest `ErrorBoundary` catches +3. `ErrorContext` stores error +4. Boundary re-renders with `error_context.error()` available + +## Root Wrapper + +Default scope hierarchy: +``` +ScopeId(0): RootScopeWrapper +└── ScopeId(1): SuspenseBoundary + └── ScopeId(2): ErrorBoundary + └── ScopeId(3): User's Root App +``` + +Provides default suspense and error handling for the entire tree. diff --git a/notes/architecture/02-CLI.md b/notes/architecture/02-CLI.md new file mode 100644 index 000000000..dde0e1e92 --- /dev/null +++ b/notes/architecture/02-CLI.md @@ -0,0 +1,264 @@ +# Dioxus CLI Architecture + +The `dx` CLI is a comprehensive build tool for Dioxus applications, supporting development, building, bundling, and cross-platform deployment. + +## Command Structure + +Entry point: `main.rs` with async tokio runtime and `TraceController` for error handling. + +### Commands Enum +- `Serve` - Watch and serve with hot-reloading +- `Build` - Full production build +- `Bundle` - Package for distribution +- `Run` - Run without hot-reload +- `Create` / `Init` - Project scaffolding +- `Check` - Lint/validate RSX +- `Translate` - Convert HTML/JSX to RSX +- `Autoformat` - Format RSX code +- `Config` - Manage Dioxus.toml +- `Doctor` - Diagnose environment +- `Components` - Component registry +- `SelfUpdate` - Update CLI +- `Tools` - Internal tools (build-assets, hotpatch) + +## Platform Support + +### Platform Enum +- `Web` - wasm32-unknown-unknown +- `MacOS` / `Windows` / `Linux` - Desktop webview +- `Ios` / `Android` - Mobile webview +- `Server` - SSR/fullstack server +- `Liveview` - WebSocket streaming +- `Unknown` - Auto-select + +### CommandWithPlatformOverrides +Allows separate client/server args for fullstack: +```bash +dx serve @client --features web @server --features server +``` + +Decomposes into client BuildArgs, server BuildArgs, and shared args. + +## Build System + +### BuildRequest +The "plan" for a single build: +``` +BuildRequest +├── workspace: Workspace metadata +├── config: DioxusConfig +├── package: Package details +├── features: Vec +├── target: Triple +├── bundle: BundleFormat +├── profile: Profile +├── rustflags: Vec +└── custom_linker: Option +``` + +Responsible for: +- Writing build artifacts +- Processing assets +- Writing platform-specific metadata (Info.plist, AndroidManifest.xml) +- Platform-specific linking (Android uses dx as opaque linker, hotpatching uses custom linker) + +### BuildMode Enum +- `Base { run: bool }` - Normal cargo rustc build +- `Fat` - Full build with all symbols for hot-patching +- `Thin { rustc_args, changed_files, aslr_reference, cache }` - Fast rebuild + +### BuildContext +Runtime configuration with progress channel: +- `ProgressTx` for status updates +- `BuildId` enum: PRIMARY (client), SECONDARY (server) +- Helper methods: `compiling()`, `linking()`, `bindgen()`, etc. + +### AppBuilder +State machine managing ongoing builds: +- Build task handle and artifacts +- App child process (stdout/stderr) +- Hot-patching state (patches, module cache, ASLR offset) +- Build metadata: stage, compiled crates, timing + +## Dev Server (Serve) + +### AppServer (runner.rs) +Primary orchestration: +``` +AppServer +├── workspace: Workspace metadata +├── client / server: AppBuilder instances +├── watcher: File system watcher +├── file_map: Cached RSX files for hot-reload diffing +├── devserver: WebServer +└── flags: use_hotpatch_engine, automatic_rebuilds, hot_reload, etc. +``` + +### WebServer (server.rs) +HTTP + WebSocket server providing: +- **Hot-reload socket** - RSX and asset updates +- **Build-status socket** - Compilation progress +- **HTTP server** - Static assets (public folder, WASM) +- **Proxy support** - Fullstack backend +- **DevTools** - Connected clients with build_id, ASLR reference, PID + +### Output (output.rs) +Terminal UI with ratatui: +- Build progress with crate-by-crate counts +- Asset copying progress +- Colored output, throbber animations +- Keyboard input (r=rebuild, p=pause, v=verbose) +- 100ms tick-based rendering + +### File Change Handling +``` +File modified → detect type +├── Rust file → diff RSX using dioxus_rsx_hotreload::diff_rsx() +│ ├── Extract changed templates +│ ├── Build HotReloadMsg +│ └── Send via WebSocket +├── Asset file → copy to bundle, signal reload +└── If can't hot-reload → queue full rebuild +``` + +### serve_all() Main Loop +Uses `tokio::select!` to multiplex: +- Builder updates (compilation, process output) +- WebServer events (connections, messages) +- Output (TUI updates, input) +- TraceController (panic handling) +- File watcher (filesystem changes) + +## Asset Processing + +### Pipeline (build/assets.rs) +1. **Manganis Integration** - Extract `__ASSETS__` symbols from binary +2. **Asset Hashing** - Compute hashes for cache-busting +3. **Two Format Support** - Legacy (0.7.0-0.7.1) and New (0.7.2+) serialization +4. **Platform Paths** - Assets relative to executable; Android uses "asset root" + +### Processing by Type +- **Images**: Resize, convert format (Avif, WebP), optimize +- **CSS/JS**: Minify, hash-suffix naming +- **Folders**: Recursive copy +- **Unknown**: Direct copy with optional hash + +## Bundling + +### Bundle System +Platform-specific packaging: +- **Web/Server**: No special bundling, outputs to web/ directory +- **Desktop**: Uses tauri-bundler (dmg, msi, appimage) +- **iOS**: App bundle format with codesigning warnings +- **Android**: Runs `gradle bundleRelease` for AAB + +### BundleConfig +``` +BundleConfig +├── identifier: String +├── publisher: String +├── icon: Vec +├── resources: Vec +├── deb / macos / windows / android: Platform-specific +``` + +## Hot-Patching + +### HotpatchModuleCache +Caches compiled dependency symbols: +- Symbol table with function/data info +- Pre-computed for Fat builds +- Reused in Thin builds for fast linking + +### Patch Creation Flow +1. Compile app with Fat mode (all symbols) +2. On change: compile Thin (small, fast) +3. Analyze binary diff → create JumpTable +4. Apply patches to running executable +5. If patch fails → fall back to full rebuild + +### Platform Implementations +- `create_windows_jump_table()` - x86/x64 jump stubs +- `create_native_jump_table()` - macOS/Linux function overrides +- `create_wasm_jump_table()` - WASM indirect call table updates + +## Configuration + +### DioxusConfig (Dioxus.toml) +```toml +[application] +asset_dir = "assets" +out_dir = "dist" +tailwind_input/output = "..." +ios_info_plist = "Info.plist" +android_manifest = "AndroidManifest.xml" + +[web] +app.title = "My App" +https.enabled = true +pre_compress = false + +[bundle] +identifier = "com.example.app" +publisher = "Company" +``` + +### Resolution Order +1. Default values +2. Dioxus.toml in project root +3. CLI argument overrides +4. Environment variables + +## Workspace + +### Workspace Struct (Singleton) +``` +Workspace +├── krates: Krates (cargo metadata, dependency graph) +├── settings: CliSettings (user settings ~/.config/dioxus/) +├── sysroot: Rust target sysroot +├── rustc_version: Version +├── wasm_opt: Optional path +├── cargo_toml: Parsed Cargo.toml +└── android_tools: Optional NDK/SDK +``` + +Lazy-loaded via `Workspace::current()` with tokio Mutex. + +## Extension Points + +### Adding New Bundlers +1. New variant in `BundleFormat` enum +2. New variant in `Renderer` enum +3. New config struct (e.g., `MyPlatformSettings`) +4. Implementation in `Bundle::bundle()` +5. Asset directory structure definition +6. Linking strategy if needed + +### Adding New Commands +1. New variant in `Commands` enum +2. New command module in `cli/` +3. Register in `TraceController::main()` match +4. Optional `StructuredOutput` variant for results + +### Customizing Build Hooks +- Pre/post-build scripts in Dioxus.toml +- Custom linker integration +- Asset transformation pipeline + +## Key Patterns + +### Channel-Based Progress +``` +Build Task → BuilderUpdate Channel → AppBuilder → ServeUpdate → Output TUI +``` +Decouples build process from UI for real-time updates. + +### Lazy Workspace Loading +Single workspace load with tokio Mutex, shows spinner if cargo-metadata takes >1 second. + +### Renderer/Bundle Inference +1. Explicit flags: `--web`, `--webview`, `--native` +2. Platform aliases: `--ios`, `--android`, `--desktop` +3. Feature detection from Cargo.toml +4. Default: Infer from target triple diff --git a/notes/architecture/03-RSX.md b/notes/architecture/03-RSX.md new file mode 100644 index 000000000..4d29ac261 --- /dev/null +++ b/notes/architecture/03-RSX.md @@ -0,0 +1,298 @@ +# RSX and Autofmt Architecture + +The `dioxus-rsx` crate parses JSX-like syntax into Rust code, while `dioxus-autofmt` provides formatting capabilities. + +## RSX Parsing + +### Entry Point: CallBody +Root struct for `rsx! {}` macro contents: +``` +CallBody +├── TemplateBody (list of BodyNode roots) +├── template_idx: Cell +└── span: Option +``` + +`CallBody::new()` initializes template indices and cascades hotreload info through nested structures. + +### BodyNode Enum +Six variants representing RSX content: +- `Element(Element)` - HTML elements (div, span) +- `Component(Component)` - User components +- `Text(TextNode)` - String literals with interpolation +- `RawExpr(ExprNode)` - Braced expressions `{expr}` +- `ForLoop(ForLoop)` - `for pat in expr { body }` +- `IfChain(IfChain)` - `if cond { } else { }` + +### Parsing Priority +1. Peek for LitStr → TextNode +2. Peek for `for` → ForLoop +3. Peek for `if` → IfChain +4. Peek for `match` → RawExpr +5. Peek for Brace → RawExpr +6. Web components: Ident + `-` → Element +7. Lowercase ident → Element +8. Otherwise → Component (fallback) + +## Key AST Types + +### Element +``` +Element +├── name: ElementName (Ident or Custom) +├── raw_attributes: Vec +├── merged_attributes: Vec // After combining duplicates +├── spreads: Vec +├── children: Vec +├── brace: Option +└── diagnostics: Diagnostics +``` + +`merge_attributes()` collapses duplicate attribute names using `IfmtInput` with space delimiter. + +### Attribute +``` +Attribute +├── name: AttributeName (BuiltIn, Custom, or Spread) +├── value: AttributeValue +├── colon: Option +├── dyn_idx: DynIdx +└── el_name: Option +``` + +### AttributeValue Variants +- `Shorthand(Ident)` - attribute without value +- `AttrLiteral(HotLiteral)` - hotreloadable literal +- `EventTokens(PartialClosure)` - event handlers +- `IfExpr(IfAttributeValue)` - conditional attributes +- `AttrExpr(PartialExpr)` - arbitrary expressions + +### HotLiteral +``` +HotLiteral +├── Fmted(HotReloadFormattedSegment) // "{expr}" interpolation +├── Float(LitFloat) +├── Int(LitInt) +└── Bool(LitBool) +``` + +### IfmtInput (Formatted Strings) +Parses string contents into segments: +- `Segment::Literal(String)` - plain text +- `Segment::Formatted(FormattedSegment)` - `{expr}` interpolation + +Parsing rules: +- `{{` → literal `{` +- `}}` → literal `}` +- `{expr}` → formatted segment +- `{expr:format_args}` → formatted with format spec + +### Component +``` +Component +├── name: syn::Path +├── generics: Option +├── fields: Vec +├── component_literal_dyn_idx: Vec +├── spreads: Vec +├── children: TemplateBody +├── dyn_idx: DynIdx +└── diagnostics: Diagnostics +``` + +### ForLoop +``` +ForLoop +├── for_token, pat, in_token +├── expr: Box +├── body: TemplateBody +└── dyn_idx: DynIdx +``` +Generates: `(expr).into_iter().map(|pat| { body })` + +### IfChain +``` +IfChain +├── if_token, cond: Box +├── then_branch: TemplateBody +├── else_if_branch: Option> +├── else_branch: Option +└── dyn_idx: DynIdx +``` + +### DynIdx +`Cell>` for tracking dynamic indices. Transparent in PartialEq/Eq/Hash. Used for hot-reload mapping. + +## Code Generation + +### TemplateBody → VNode::Template +Generated output structure: +1. `__TEMPLATE_ROOTS` - Static array of TemplateNode +2. **Dynamic nodes** - Components, interpolated text, loops, conditionals +3. **Dynamic attributes** - Non-static attribute values +4. **Dynamic literal pool** - In debug, vec of formatted values +5. **Dynamic value pool** - Maps literal indices to values + +### Template Structure +```rust +dioxus_core::Element::Ok({ + #[cfg(debug_assertions)] + fn __original_template() -> &'static HotReloadedTemplate { ... } + + let __dynamic_nodes: [DynamicNode; N] = [ ... ]; + let __dynamic_attributes: [Box<[Attribute]>; M] = [ ... ]; + static __TEMPLATE_ROOTS: &[TemplateNode] = &[ ... ]; + + // Template reference and rendering +}) +``` + +### Generated TemplateNode Types +- `Element { tag, namespace, attrs, children }` - Static element +- `Text { text }` - Static text +- `Dynamic { id }` - References dynamic node pool + +## Template System + +### TemplateBody Structure +``` +TemplateBody +├── roots: Vec +├── template_idx: DynIdx +├── node_paths: Vec> // Path to each dynamic node +├── attr_paths: Vec<(Vec, usize)> // Path and attribute index +├── dynamic_text_segments: Vec +└── diagnostics: Diagnostics +``` + +### Template ID Assignment +- `CallBody::next_template_idx()` generates sequential IDs +- Each nested structure gets unique ID +- Combined with `file!()`, `line!()`, `column!()` for source location + +### HotReloadFormattedSegment +Wraps `IfmtInput` with `dynamic_node_indexes: Vec`: +- One DynIdx per `Segment::Formatted` entry +- Maps formatted segments to dynamic nodes during hot-reload + +## Autofmt System + +### Entry Points +- `try_fmt_file(contents, &syn::File, IndentOptions)` → Vec +- `fmt_block(block_str, indent_level, IndentOptions)` → Option +- `write_block_out(body)` → Option + +### FormattedBlock +``` +FormattedBlock +├── formatted: String +├── start: usize (byte offset) +└── end: usize (byte offset) +``` + +### Writer State +``` +Writer +├── raw_src: &str +├── src: Vec<&str> // Lines +├── out: Buffer +├── cached_formats: HashMap +└── invalid_exprs: Vec +``` + +### Buffer +``` +Buffer +├── buf: String +├── indent_level: usize +└── indent: IndentOptions +``` + +### Optimization Levels +1. **Empty**: `div {}` (no space inside) +2. **Oneliner**: `div { class: "x", child {} }` (single line) +3. **PropsOnTop**: Props multiline, children follow +4. **NoOpt**: Everything multiline + +### Short-Circuit Optimization +```rust +if formatted.len() <= 80 + && !formatted.contains('\n') + && !body_is_solo_expr + && !formatted.trim().is_empty() +{ + formatted = format!(" {formatted} "); // Collapse to single line +} +``` + +## Whitespace Handling + +Whitespace is significant in RSX: +- Text nodes preserve exact whitespace +- Comments must be preserved +- Formatting must not change text node content + +### Comment Preservation +- `write_comments()` accumulates full-line comments before spans +- `write_inline_comments()` preserves end-of-line comments +- Comments tracked via `LineColumn` from Span + +## Expression Formatting + +### write_partial_expr() Strategy +1. Use `prettier_please` to unparse expressions +2. Handle nested rsx! macros specially +3. `unparse_expr()` visits macro calls: + - Format nested rsx! blocks + - Replace macros with marker unicode + - Apply formatted blocks back + +### Marker Replacement +Uses marker `"𝕣𝕤𝕩"` to replace macros during unparse, avoiding conflicts with actual code. + +## Indentation System + +### IndentOptions +``` +IndentOptions +├── width: usize +├── indent_string: String ("\t" or spaces) +└── split_line_attributes: bool +``` + +### Functions +- `indent_str()` → returns indent string +- `count_indents(line)` → estimates indent level +- `line_length(line)` → estimates visible length + +## Attribute Formatting + +### write_attributes(props_same_line) +- `true`: attrs on one line with spaces +- `false`: each attr on new line, indented + +### write_attribute_value() +- `Shorthand` → just ident +- `AttrLiteral` → uses Display impl +- `EventTokens` → write_partial_expr() +- `IfExpr` → write_attribute_if_chain() +- `AttrExpr` → write_partial_expr() + +## Extension Points + +### Adding New Node Types +1. Add variant to BodyNode enum +2. Implement Parse trait +3. Implement ToTokens for code gen +4. Add write_* method to Writer + +### Adding New Attribute Types +1. Add variant to AttributeValue +2. Implement Parse for detection +3. Add merge handling if needed +4. Add write_attribute_value case + +### Changing Formatting Heuristics +- Modify ShortOptimization logic +- Adjust threshold constants (80, 100 chars) +- Modify estimation functions diff --git a/notes/architecture/04-SIGNALS.md b/notes/architecture/04-SIGNALS.md new file mode 100644 index 000000000..494a180b4 --- /dev/null +++ b/notes/architecture/04-SIGNALS.md @@ -0,0 +1,328 @@ +# Signals and State Management + +Dioxus state management is built on a layered architecture: generational-box for memory, signals for reactivity, and stores for nested data. + +## Generational Box + +### Core Concept +Provides `Copy` semantics for references through generation-based validation: +``` +GenerationalBox +├── raw: GenerationalPointer +│ ├── storage: &'static S +│ └── location: GenerationalLocation +│ ├── generation: NonZeroU64 +│ └── created_at: &'static Location (debug) +└── _marker: PhantomData +``` + +### Memory Management + +**Allocation:** +```rust +pub trait Storage { + fn new(value: Data, caller: Location) -> GenerationalPointer; + fn new_rc(value: Data, caller: Location) -> GenerationalPointer; +} +``` + +**Storage Variants:** +- `UnsyncStorage` - Single-threaded, uses `RefCell` (fast, no locking) +- `SyncStorage` - Multi-threaded, uses `RwLock` (thread-safe) + +**StorageEntry:** +``` +StorageEntry +├── generation: NonZeroU64 +├── refcount: u32 +└── data: Data // Empty, Data, Rc, or Reference +``` + +### Generation Tracking +1. Caller provides pointer with generation X +2. Storage checks: `entry.generation == X` +3. If yes: data valid, proceed +4. If no: return `BorrowError::Dropped` + +When dropped: generation incremented, data cleared, existing pointers invalidated. + +### Owner Pattern +```rust +pub struct Owner { + owned: Vec>, +} + +impl Drop for Owner { + fn drop(&mut self) { + for location in self.owned.drain(..) { + location.recycle(); // Invalidates all pointers + } + } +} +``` + +## Signals + +### Architecture +``` +Signal +└── inner: CopyValue, S> + └── SignalData + ├── value: T + └── subscribers: Arc>> +``` + +### Signal Types + +**Signal** - Primary mutable reactive primitive: +- Implements `Readable` for subscribed reads +- Implements `Writable` for reactive updates +- `.read()` subscribes current scope +- `.peek()` reads without subscribing + +**Memo** - Derived reactive value: +- Wraps `Signal` for value storage +- Contains `UpdateInformation` tracking dirty state +- Lazy evaluation: only recomputes when dependencies change +- Uses `PartialEq` to skip updates if unchanged + +**CopyValue** - Generic mutable wrapper: +- Lower-level than Signal +- No built-in reactivity +- Building block for Signal + +**Global** - Lazy singleton: +- Created once per app on first access +- Stored in `ScopeId::ROOT` context +- Uses `InitializeFromFunction` trait + +**ReadSignal** / **WriteSignal** - Type-erased boxed signals: +- Store `Box` +- Flexible APIs accepting any readable type + +**MappedSignal** - Derived readonly: +- Maps inner signal through function F +- `signal.map(|x| &x.field)` + +### Reactivity System + +**Automatic Subscription:** +``` +signal.read() + → try_read_unchecked() + → Check ReactiveContext::current() + → If exists: reactive_context.subscribe(signal.subscribers) + → Later writes call mark_dirty() on all subscribers +``` + +**Update Propagation:** +``` +signal.write() + → WriteLock created + → User modifies value + → WriteLock dropped → SignalSubscriberDrop::drop() + → signal.update_subscribers(): + → Get subscriber snapshot (brief lock) + → Call mark_dirty() on each + → Re-extend subscriber list +``` + +### Global Signals + +**GlobalLazyContext:** +``` +GlobalLazyContext +└── map: Rc>>> +``` + +**Resolution:** +1. Static const defines `GlobalSignal` +2. First access calls `resolve()`: + - Check HashMap by key + - If found: return clone + - If not: run constructor in ROOT scope + - Store result +3. Subsequent accesses return same instance + +**Key Types:** +- `GlobalKey::File { file, line, column, index }` +- `GlobalKey::Raw(&'static str)` + +## Hooks + +### Core Hook Pattern +```rust +#[track_caller] +pub fn use_hook(f: impl FnOnce() -> T) -> T { + let component_id = current_scope_id(); + let mut hooks = get_hooks(component_id); + + if hooks.len() <= hook_index { + hooks.push(Box::new(f())); // First render + } + hooks[hook_index].clone() // Return existing +} +``` + +### Hook Types + +**use_signal() -> Signal** +- Creates local signal owned by component +- State persists across renders + +**use_memo() -> Memo** +- Creates memoized computation +- Reruns when read signals change + +**use_effect(callback)** +- Runs side effects when dependencies change +- Creates `ReactiveContext` to track reads +- Queues effect rerun on dependency notification + +**use_resource(future_fn) -> Resource** +- Async state management +- Watches dependencies, reruns future when they change +- Returns `Resource` with value, state, task + +**use_callback() -> Callback** +- Memoized callback +- Prevents unnecessary closures + +**use_coroutine(init)** +- Long-lived task +- Spawned once at component creation + +**use_context() -> T** +- Retrieves ancestor context + +### Hook Rules +1. Must call same hooks every render +2. Must call in same order +3. Order determines hook identity +4. Breaking rules causes panic or bugs + +## Stores + +### Purpose +Signals work for scalar state. Stores provide: +- Granular reactivity per field +- Lazy signal creation +- Ergonomic field access + +### Store Architecture +``` +Store +└── selector: SelectorScope + ├── subscriptions: StoreSubscriptions + ├── path: TinyVec + └── value: Lens +``` + +### Subscription Tree +``` +StoreSubscriptions +└── inner: CopyValue + └── root: SelectorNode + ├── subscribers: HashSet + └── root: HashMap + └── [0] → SelectorNode + └── [1] → SelectorNode +``` + +### Path Tracking +```rust +let store = Store::new(vec![a, b, c]); +let item_1 = store[1]; // path = [1] +let field = item_1.name; // path = [1, field_hash] + +// Writing store[1].name only marks dirty: +// - subscribers at path [1] +// - subscribers at path [1, field_hash] +// - NOT path [0] or [2] +``` + +### Store Macro +```rust +#[derive(Store)] +struct TodoItem { + checked: bool, + contents: String, +} + +// Generates: +pub trait TodoItemStoreExt<__Lens> { + fn checked(self) -> Store; + fn contents(self) -> Store; + fn transpose(self) -> TodoItemStoreTransposed; +} + +pub struct TodoItemStoreTransposed { + pub checked: Store, + pub contents: Store, +} +``` + +### Enum Support +```rust +#[derive(Store)] +enum Status { + Loading, + Ready(String), + Error(String), +} + +// Generates: +fn is_loading(self) -> bool; +fn ready(self) -> Option>; +fn transpose(self) -> StatusStoreTransposed; +``` + +## Key Traits + +### Readable +```rust +pub trait Readable { + type Target; + type Storage; + fn try_read_unchecked(&self) -> Result>; + fn try_peek_unchecked(&self) -> Result>; + fn subscribers(&self) -> Subscribers; +} +``` + +### Writable +```rust +pub trait Writable: Readable { + type WriteMetadata; + fn try_write_unchecked(&self) -> Result>; +} +``` + +### Storage (AnyStorage) +```rust +pub trait AnyStorage { + type Ref<'a, T>: Deref; + type Mut<'a, T>: DerefMut; + fn map(ref_: Ref) -> Ref; + fn map_mut(mut_ref: Mut) -> Mut; +} +``` + +## Comparison + +| Feature | Signal | Memo | Store | +|---------|--------|------|-------| +| Mutability | Mutable | Read-only | Mutable | +| Dependencies | None | Tracks reads | Implicit via paths | +| Granularity | Single value | Computation | Per-field | +| When to use | Direct state | Derived values | Nested structures | +| Subscriptions | Simple HashSet | Via context | Tree structure | +| Performance | O(1) update | O(deps) recompute | O(path) update | + +## Memory Model + +1. **Heap**: Values in storage singletons per-scope +2. **Stack**: Only pointers (GenerationalBox) in components +3. **Cleanup**: Owner drops when scope dies +4. **Validity**: Generation checking on each access +5. **Sharing**: Arc+Mutex for subscribers, Rc+RefCell for ownership diff --git a/notes/architecture/05-FULLSTACK.md b/notes/architecture/05-FULLSTACK.md new file mode 100644 index 000000000..759e0e584 --- /dev/null +++ b/notes/architecture/05-FULLSTACK.md @@ -0,0 +1,297 @@ +# Dioxus Fullstack Architecture + +The fullstack ecosystem enables seamless client-server communication through server functions, SSR, and hydration. + +## Server Functions (#[server]) + +### Macro Generation + +The `#[server]` macro generates dual code paths: + +**On Client:** +```rust +// Creates request encoding/decoding +let client = ClientRequest::new(Method::POST, endpoint, &query); +let response = ServerFnEncoder::fetch_client(client, args, unpack).await; +let result = ServerFnDecoder::decode_client_response(response).await; +``` + +**On Server:** +```rust +// Creates Axum handler +fn __inner__function__(state, request) -> Response { + // Extract arguments from request + // Call actual function + // Serialize response +} + +// Register globally +inventory::submit! { + ServerFunction::new(Method::POST, path, handler) +} +``` + +### Configuration Options +- `endpoint` - Custom URL path prefix (default: `/api`) +- `input` - Request encoding (Json, Cbor, MessagePack) +- `output` - Response encoding +- `middleware` - Tower middleware layers +- Server-only extractors: `headers: HeaderMap`, `cookies: Cookies` + +### Wire Protocol + +**Request:** +- Method: POST +- URL: `/api/{function_name}?{query_params}` +- Content-Type: application/json +- Body: JSON-serialized struct with arguments + +**Response:** +```rust +enum RestEndpointPayload { + Success(T), + Error(ErrorPayload), +} + +struct ErrorPayload { + message: String, + code: u16, + data: Option, +} +``` + +### Handler Registration +1. `ServerFunction` struct holds metadata +2. `inventory::submit!` for compile-time registration +3. `ServerFunction::collect()` returns all handlers +4. `DioxusRouterExt::register_server_functions()` registers on Axum + +## SSR (Server-Side Rendering) + +### SsrRendererPool +- Thread-safe pool of `dioxus_ssr::Renderer` instances +- Manages concurrent rendering +- Optional incremental cache for static generation + +### Rendering Pipeline +1. Create VirtualDom with component function +2. Render to HTML string +3. Inject hydration data and scripts +4. Stream chunks via futures channel + +### Streaming Modes + +**Disabled (default):** +- All server futures resolved before sending +- Simple, SEO-friendly + +**OutOfOrder:** +- Initial chunk sent with placeholders +- Suspense boundaries stream independently +- Uses `StreamingRenderer` + +**Streaming Chunk Structure:** +```html + +

Header
+
Loading...
+ + + + +``` + +## Hydration + +### Server-Side +1. `HydrationContext` created at render start +2. Hooks serialize data: `use_server_future`, `use_server_cached`, `use_loader` +3. Data stored with unique IDs +4. Serialized to base64-encoded CBOR +5. Injected as `window.__DIOXUS_HYDRATION_DATA__` + +### Client-Side +1. JavaScript reads hydration data from HTML +2. Runtime provides `HydrationContext` to components +3. Components call same hooks, check for cached data +4. Data deserialized and populated +5. Event listeners attached without re-rendering + +### Key Types +- `SerializeContextEntry` - Entry in hydration context +- `SerializedHydrationData` - Base64-encoded CBOR +- `TakeDataError` - Why data couldn't be retrieved + +## Server Architecture + +### FullstackState +- Wraps `FullstackContext` and runtime handle +- Passed as State to Axum handlers +- Allows request context access + +### DioxusRouterExt Trait +Extension methods on `axum::Router`: +- `serve_static_assets()` - Serves `/public` +- `serve_dioxus_application(cfg, component)` - Full SSR + server functions +- `register_server_functions()` - Registers collected handlers +- `serve_api_application(cfg, component)` - API-only + +### Handler Wrapping +``` +Handler wrapped in: +1. FullstackContext::scope() - provides context +2. LocalPool::spawn_pinned() - enables !Send futures +3. Middleware layers +4. Response header injection +``` + +### FullstackContext +Task-local context providing: +- Request headers via RwLock +- Streaming status (RenderingInitialChunk, Committed, Complete) +- Response headers +- HTTP status codes +- Reactive subscriptions + +### ServeConfig +- `index` - Custom IndexHtml +- `incremental` - ISR cache +- `context_providers` - Injectable contexts +- `streaming_mode` - Disabled or OutOfOrder + +## Client-Server Communication + +### ClientRequest +``` +ClientRequest +├── url: String +├── method: Method +├── headers: HeaderMap +└── extensions: Extensions +``` + +### Encoding Traits +- `EncodeRequest` - Serializes and makes request +- `IntoRequest` - Converts to request +- `Encoding` trait - Defines format (Json, Cbor) + +### Suspense Integration +- Server functions automatically suspend +- `use_server_future` - Waits on server, caches for client +- `use_loader` - Enhanced error handling +- `use_server_cached` - Caches computed values + +### Error Handling +- Server errors serialized with message, code, details +- `ServerFnError` enum standardizes representation +- `AsStatusCode` trait maps to HTTP status +- Supports `anyhow::Error`, `StatusCode`, custom types + +## Extension Points + +### Adding Database Support +```rust +config.provide_context(|| { + Box::new(Database::new()) as Box +}); + +#[server] +async fn get_user(id: i32) -> Result { + let db = consume_context::(); + db.query(...).await +} +``` + +### Adding Authentication +```rust +#[server(auth: AuthExtractor)] +async fn protected(data: String, auth: AuthExtractor) -> Result { + // auth extracted from request +} +``` + +Or via middleware: +```rust +#[server] +#[middleware(AuthLayer::new())] +async fn protected_fn() -> Result { } +``` + +### Custom Encoding +```rust +pub struct CustomEncoding; +impl Encoding for CustomEncoding { + fn content_type() -> &'static str { "application/custom" } + fn encode(data: impl Serialize, buf: &mut Vec) -> Option { } + fn decode(bytes: Bytes) -> Option { } +} + +#[server(input = CustomEncoding, output = CustomEncoding)] +async fn my_fn(arg: String) -> Result { } +``` + +### Response Customization +```rust +#[server] +async fn custom_response() -> Result { + let ctx = FullstackContext::current().unwrap(); + ctx.set_response_headers(headers); + Ok("data".to_string()) +} +``` + +## Data Flow + +### Server Function Call +``` +Client Component + → Call #[server] fn + → Create ClientRequest, serialize args + → HTTP POST /api/function_name + → Server Router + → FullstackContext::scope() + → Extract args, call function + → Serialize Result + → HTTP Response + → Client deserialize + → Component receives value +``` + +### SSR Hydration +``` +Server: + Page Request → VirtualDom::new() + → HydrationContext created + → Render components + → use_server_future serializes results + → Inject __DIOXUS_HYDRATION__ script + → HTTP 200 + HTML + +Client: + Receive HTML + → Parse hydration data + → VirtualDom::new() + → HydrationContext populated + → Components render, hooks get cached data + → Event listeners attached + → Hydration complete +``` + +## Key Patterns + +### Inventory System +Global registration via `inventory::submit!` at compile time. Enables dynamic discovery without explicit registration. + +### Magic Trait Deref +Multiple `&` deref layers select correct trait impl: +- Prefers `FromRequest` over `DeserializeOwned` +- Prefers `FromResponse` over serialization + +### Hash-based Routing +Routes hashed using xxhash64 of module path, preventing collisions with same-name functions in different modules. + +### Error Type Tiering +Auto-deref tiers implementations: +1. Typed error struct +2. ServerFnError wrapper +3. anyhow::Error diff --git a/notes/architecture/06-RENDERERS.md b/notes/architecture/06-RENDERERS.md new file mode 100644 index 000000000..253896008 --- /dev/null +++ b/notes/architecture/06-RENDERERS.md @@ -0,0 +1,320 @@ +# Dioxus Renderers Architecture + +Dioxus supports multiple rendering backends through a common trait-based abstraction. Each renderer implements the same interfaces to enable cross-platform component reuse. + +## Core Abstraction + +### WriteMutations Trait +The bridge between VirtualDOM and real DOM implementations: +- `append_children(id, count)` - Add N nodes to element +- `assign_node_id(path, id)` - Mark element at template path +- `create_placeholder(id)` - Create marker node +- `create_text_node(value, id)` - Create text node +- `load_template(template, index, id)` - Clone from template cache +- `replace_node_with(id, count)` - Replace element +- `set_attribute(name, ns, value, id)` - Update attribute +- `create_event_listener(name, id)` - Register listener +- `remove_node(id)` - Delete element + +### HtmlEventConverter Trait +Maps platform-specific events to standardized Dioxus types: +- Each renderer provides its own implementation +- Converts `PlatformEventData` (renderer-specific) to typed event data +- Enables event polymorphism across platforms + +### Event Flow Pattern +``` +Platform Event → Captured by renderer + → Converted via HtmlEventConverter + → runtime.handle_event(name, event, element_id) + → VirtualDOM handlers invoked + → State changes → Re-render + → WriteMutations applied +``` + +## Web Renderer (dioxus-web) + +### WebsysDom Structure +``` +WebsysDom +├── interpreter: Sledgehammer JS interpreter +├── document: web_sys::Document reference +├── root: Root DOM node +├── templates: HashMap +├── runtime: Rc +└── (hydration): skip_mutations, suspense_hydration_ids +``` + +### Mutation Implementation +- Directly delegates to JavaScript interpreter via wasm-bindgen +- Templates serialized once, stored in JS, instantiated by reference +- Sledgehammer interpreter maintains stack of nodes being constructed +- Binary protocol enables efficient mutation batching + +### Event Handling +- Single delegated listener on root element +- Event.target walked up DOM tree for `data-dioxus-id` attribute +- `WebEventConverter` converts web_sys events to Dioxus types +- Supports all standard DOM events (mouse, keyboard, touch, etc.) + +### Hydration System +1. SSR server renders HTML with `dio_el` data attributes +2. Client receives base64-encoded hydration context +3. VirtualDOM rebuilt with `skip_mutations = true` +4. Client traverses pre-rendered DOM assigning element IDs +5. Streaming hydration for suspense boundaries via `rehydrate_streaming()` + +### Launch Flow +``` +launch(root_component, contexts, config) + → Create VirtualDom + → Create WebsysDom wrapper + → If hydrate: Deserialize data, rebuild with skip_mutations + → Otherwise: vdom.rebuild(&mut websys_dom) + → Main loop: wait_for_work() → render_immediate() → flush_edits() +``` + +## Desktop Renderer (dioxus-desktop) + +### Wry/Tao Integration +Uses Wry webview library with Tao window management. + +### App Structure +``` +App +├── unmounted_dom: Cell> +├── webviews: HashMap +├── shared: Rc +│ ├── event_handlers: WindowEventHandlers +│ ├── pending_webviews: Vec +│ ├── shortcut_manager: ShortcutRegistry +│ └── websocket: EditWebsocket +└── control_flow: ControlFlow +``` + +### WebviewEdits +Implements WriteMutations for wry-based rendering: +- Delegates to `WryQueue` managing mutation batch +- WebSocket server on random port for mutation transmission +- Binary protocol via Sledgehammer interpreter + +### IPC (Interprocess Communication) +``` +Browser event → JavaScript → window.postMessage() + → Wry intercepts request + → Extract dioxus-data header (base64 JSON) + → IpcMessage { method, params } + → Handle: UserEvent, Query, BrowserOpen, Initialize +``` + +### Protocol Handler +- `dioxus://` custom protocol for asset serving +- Handles `__events` path for event processing +- `__file_dialog` for file selection +- Custom handler namespaces for user-provided handlers +- Checks `dioxus_asset_resolver` for bundled assets + +### Native Features +- Menu integration via muda crate +- System tray via trayicon +- Global hotkeys via global_hotkey crate +- File dialogs and drag-drop support +- Headless mode for testing + +### Configuration +``` +Config +├── WindowBuilder customization +├── Custom event loop +├── Protocols and async protocols +├── Pre-rendered HTML template +├── Context menu disable flag +├── Background color (RGBA) +└── Devtools support toggle +``` + +## Native Renderer (dioxus-native) + +### Blitz Integration +- Blitz layout engine for CSS layout +- Vello for GPU-accelerated vector rendering +- Winit for cross-platform window management +- Not a browser engine - custom native rendering pipeline + +### DioxusNativeWindowRenderer +``` +DioxusNativeWindowRenderer +├── anyrender-vello wrapper +├── WindowRenderer trait implementation +├── GPU features configurable +└── Custom paint support +``` + +### Rendering Pipeline +``` +VirtualDOM components + → DioxusNativeDOM + → Blitz DOM tree with CSS + → Blitz layout engine + → Vello renderer + → GPU rendering +``` + +### Layout and CSS +- CSS 2.1+ parser (not full CSS 3) +- Flexbox-based layout model +- Computed styles attached to element nodes +- Layout computed bottom-up during mutation + +### Application Handler (Winit) +``` +Event::NewEvents(StartCause::Init) + → Create initial window + → DioxusDocument with VirtualDOM + +Resumed → Renderer.resume() + +WindowEvent::RedrawRequested + → VirtualDOM.render_immediate() + → Mutations applied to Blitz DOM + → Layout computed + → Render frame + +WindowEvent::Resized → Queue redraw +``` + +## LiveView Renderer (dioxus-liveview) + +### Server-Rendered Architecture +``` +Client (Browser with WebSocket) + ↓ events +Server (VirtualDOM runs here) + ↓ mutations (binary protocol) +Client receives mutations → Sledgehammer applies +``` + +### LiveViewPool +- Thread pool using `tokio_util::task::LocalPoolHandle` +- Each client spawns pinned task +- VirtualDOM runs on task's executor +- Pool handles multiple concurrent clients + +### Per-Client Lifecycle +``` +LiveViewPool::launch_virtualdom(ws, || VirtualDom::new()) + → Create MutationState, QueryEngine + → vdom.rebuild() → Send initial HTML + → Loop: + tokio::select! { + ws.next() → Handle event/query + vdom.wait_for_work() → Has work + query_rx.recv() → JS query + hot_reload_rx.recv() → Hot reload + } + render_immediate() → Send mutations +``` + +### Binary Protocol +``` +MutationState::write_memory_into(&mut bytes) + → Sledgehammer binary encoding + → WebSocket transmission + → Client: window.interpreter.handleEdits(bytes) +``` + +### Message Types +1. **Binary Frames** (mutations) +2. **Text Frames** (queries/metadata) +3. **Incoming Events** (user interactions) + +### Mounted Element Queries +``` +LiveviewElement methods: + scroll_offset() → JS: getScrollLeft/Top() + scroll_size() → JS: getScrollWidth/Height() + client_rect() → JS: getBoundingClientRect() + scroll_to(options) → JS: element.scrollTo() +``` + +## Interpreter Package (dioxus-interpreter-js) + +### Sledgehammer Framework +Ultra-compact binary protocol for DOM mutations shared across renderers. + +### Core Components +1. **INTERPRETER_JS** - Base interpreter class +2. **NATIVE_JS** - Platform-specific extensions +3. **SLEDGEHAMMER_JS** - Generated bindings from Rust + +### MutationState +- Implements WriteMutations for binary serialization +- Channel-based mutation recording +- Template caching and deduplication + +### Stack Machine Operations +``` +create_text_node(value, id) → Push text node +create_placeholder(id) → Push comment node +append_children(parent_id, count) → Pop and append +replace_with(id, count) → Replace element +set_attribute(id, name, value, ns) → Set DOM attribute +new_event_listener(name, id, bubbles) → Register listener +``` + +## Common Patterns + +### 1. WriteMutations Implementation +Every renderer implements the trait differently: +- **Web**: Delegates to Sledgehammer JS +- **Desktop**: Accumulates in MutationState, sends via WebSocket +- **Native**: Applies directly to Blitz DOM +- **Liveview**: Accumulates and sends via WebSocket + +### 2. Configuration Pattern +Renderers use `Box` for extensible config: +```rust +launch(app, configs: Vec>) + → For each config: try downcast to expected type +``` + +### 3. Lazy Initialization +Most renderers delay context setup until first window/request. + +### 4. Event Loop Integration +- **Web**: WASM bindgen + browser event loop +- **Desktop**: Tao event loop with UserWindowEvent +- **Native**: Winit ApplicationHandler +- **Liveview**: Tokio select! macro + +### 5. Mutation Batching +All non-web renderers batch mutations for efficiency: +- Accumulate in temporary structure +- Periodically flush to transport + +## Adding a New Renderer + +1. **Implement WriteMutations** + - Define how platform applies DOM-like mutations + - Handle template system + - Manage stack machine state + +2. **Implement HtmlEventConverter** + - Map platform events to Dioxus types + - Handle serialization if needed + +3. **Create Launch Function** + - Create VirtualDOM + - Initialize renderer structures + - Enter platform event loop + - Call `vdom.render_immediate(&mut mutations)` + +4. **Define Configuration** + - Create Config struct + - Support `Box` downcasting + +5. **Optional Enhancements** + - Custom mount event data + - Document integration + - History support + - Asset serving diff --git a/notes/architecture/07-HOTRELOAD.md b/notes/architecture/07-HOTRELOAD.md new file mode 100644 index 000000000..50d207c45 --- /dev/null +++ b/notes/architecture/07-HOTRELOAD.md @@ -0,0 +1,284 @@ +# Hot-Reload and Hot-Patching Architecture + +Dioxus supports two complementary hot-reload systems: RSX template hot-reload for UI changes, and Subsecond hot-patching for Rust code changes. + +## Subsecond Hot-Patching + +### Jump Table Architecture + +Subsecond implements hot-patching through **jump table indirection**: + +1. All hot-reloadable functions called through `subsecond::call()` or `HotFn::current()` +2. Runtime looks up function pointer in global `APP_JUMP_TABLE` +3. Jump table points to latest compiled version +4. When patch applied, only jump table is updated +5. Original executable remains untouched + +**Advantage**: Decouples running binary from patched code. Safe memory model. + +### Patch Application Flow + +``` +1. ThinLink compiles only modified functions → patch dylib +2. Patch sent via devtools WebSocket +3. subsecond::apply_patch() loads via libloading::Library +4. Base address calculated using main as anchor +5. Jump table updated with old→new address mappings +``` + +### ASLR Handling + +Operating systems randomize memory addresses (ASLR), so compiled addresses don't match runtime. + +**Solution**: +1. Running app captures real address of `main` via `dlsym()`/`GetProcAddress()` +2. ASLR reference sent to devserver +3. Devserver computes offset: `old_offset = aslr_reference - table.aslr_reference` +4. All jump table addresses adjusted by offset +5. Patch library base address calculated similarly +6. Final: `(old_address + old_offset) → (new_address + new_offset)` + +### Thread-Local Storage (TLS) + +TLS presents challenges for hot-patching: + +- **Globals and statics**: Generally supported +- **Thread-locals in tip crate**: Reset to initial value on new patches +- **Thread-locals in dependencies**: Work correctly (not re-patched) + +**Why**: New thread-local variables exist in patch library's TLS segment, separate from main executable's TLS. Subsecond doesn't rebind TLS accesses. + +### Limitations + +1. **Struct changes**: Not supported - size/alignment changes cause crashes +2. **Pointer versioning**: Not implemented - all function pointers considered "new" +3. **Workspace support**: Only tip crate patches, library crates ignored +4. **Static initializers**: Changes not observed +5. **Global destructors**: Newly added globals have destructors that never run + +### Platform Support + +- **Desktop**: Linux, macOS, Windows (x86_64, aarch64) +- **Mobile**: Android (arm64-v8a, armeabi-v7a), iOS Simulator +- **Web**: WASM32 (limited module reloading) +- **Not supported**: iOS device (code-signing) + +## RSX Hot Reload + +### Hot Literal System + +RSX hot reload is **orthogonal** to subsecond. While subsecond reloads Rust functions, RSX hot reload handles template literal values: + +**Hot-reloadable**: +- Formatted segments: `"{variable}"` +- Literal component properties: `Component { value: 123 }` +- Dynamic text nodes: `"{expression}"` + +**Not hot-reloadable**: +- Rust code changes +- Component structure changes +- Control flow changes + +### Template Diffing Algorithm + +**Conservative approach**: If Rust code changes, not hot-reloadable. + +``` +1. Parse old and new files +2. Extract all rsx! macro invocations +3. Replace all rsx! bodies with empty rsx! {} +4. Remove doc comments +5. Compare modified files +6. If identical → Rust unchanged → proceed with template diff +7. If different → requires full rebuild +``` + +### Dynamic Pool System + +Three pools of reusable items from last build: +1. **Dynamic text segments**: `"{class}"`, `"{id}"` +2. **Dynamic nodes**: Components, for loops, if chains +3. **Dynamic attributes**: Spread operators, dynamic values + +Each item tracks usage with `Cell` for waste scoring. + +### Greedy Matching Algorithm + +For each new dynamic node: +1. Find all candidates from last build with compatible structure +2. Attempt to create hot-reloaded template using candidate's pool +3. Score: Count unused dynamic items remaining +4. Select candidate with least unused items +5. Greedy approach optimal because it maximizes reuse + +### Change Detection + +**NOT possible**: +- Number of `rsx!` macros changes +- Rust expressions change +- Component structure changes +- Control flow conditions change + +**IS possible**: +- Component children content +- Reordering attributes +- Adding dynamic text segments from pool +- Changing literal values +- Shuffling template structure + +## Devtools Protocol + +### WebSocket Communication + +Bidirectional WebSocket between app and devserver. + +**Connection**: +- URL: `ws://localhost:3000/_dioxus` +- Query params: `build_id`, `pid`, `aslr_reference` +- Persistent during development + +### Message Types + +```rust +pub enum DevserverMsg { + HotReload(HotReloadMsg), // Templates + optional jump table + HotPatchStart, // Binary patching starting + FullReloadStart, // Rebuilding entire app + FullReloadFailed, // Build failed + FullReloadCommand, // Full page reload needed + Shutdown, // Devserver shutting down +} +``` + +### HotReloadMsg Structure + +```rust +pub struct HotReloadMsg { + pub templates: Vec, + pub assets: Vec, + pub ms_elapsed: u64, + pub jump_table: Option, + pub for_build_id: Option, + pub for_pid: Option, +} +``` + +### Message Processing + +1. `connect(callback)` - Generic connection +2. `connect_subsecond()` - Subsecond-specific with jump tables +3. `apply_changes(dom, msg)`: + - Updates signal-based template cache + - Applies jump table if PID matches + - Marks all components dirty + +### WASM-Specific Handling + +- Connection at `ws://host/_dioxus?build_id={build_id}` +- Supports playground mode (iframe via postMessage) +- Console logging sent to devserver +- Page reload on `FullReloadCommand` +- Asset cache invalidation via `dx_force_reload` query params + +## Integration Flow + +### Subsecond in Dioxus Apps + +```rust +fn main() { + dioxus::launch(app); + // Devtools automatically connects during init +} +``` + +For non-Dioxus apps: +```rust +fn main() { + dioxus_devtools::connect_subsecond(); + + loop { + dioxus_devtools::subsecond::call(|| { + handle_request() + }); + } +} +``` + +### Async Integration + +```rust +#[tokio::main] +async fn main() { + dioxus_devtools::serve_subsecond_with_args( + state, + |state| async { + app_main(state).await + } + ).await; +} +``` + +**Process**: +1. Catches patch message +2. Applies jump table +3. Drops current future +4. Creates new future with hot function +5. Continues execution + +## Build System Integration + +### Fat vs Thin Builds + +**Fat Build** (initial): +- Full build with all symbols +- Required for initial symbol table +- Creates `HotpatchModuleCache` + +**Thin Build** (patches): +- Compiles only modified functions +- Uses cached dependency symbols +- Produces minimal patch dylib + +### JumpTable Structure + +```rust +pub struct JumpTable { + pub lib: PathBuf, // Path to patch dylib + pub map: HashMap, // old_addr → new_addr + pub aslr_reference: u64, + pub new_base_address: u64, + pub ifunc_count: u32, +} +``` + +### Platform-Specific Patching + +- `create_windows_jump_table()` - x86/x64 jump stubs +- `create_native_jump_table()` - macOS/Linux function overrides +- `create_wasm_jump_table()` - WASM indirect call table updates + +## Key Design Decisions + +1. **Jump table indirection**: Safe, no memory corruption +2. **Conservative RSX diffing**: Rust changes trigger rebuild +3. **Greedy pool matching**: Optimal template reuse +4. **WebSocket protocol**: Real-time bidirectional updates +5. **PID filtering**: Correct process receives patches +6. **Dual system**: RSX for templates, Subsecond for logic + +## Future Considerations + +### Workspace Support +- Dependency graph analysis for affected crates +- Incremental library crate compilation +- Cross-crate function pointer resolution + +### Remote Hot-Reloading +- SCP transport for bandwidth efficiency +- Binary diff for minimal transfers +- Cryptographic verification + +### CLI Tunnels +- SSH/TCP protocol wrapping +- Connection persistence +- Latency-tolerant queueing diff --git a/notes/architecture/08-ASSETS.md b/notes/architecture/08-ASSETS.md new file mode 100644 index 000000000..30b653f67 --- /dev/null +++ b/notes/architecture/08-ASSETS.md @@ -0,0 +1,347 @@ +# Manganis Asset System Architecture + +Manganis provides compile-time asset management through the `asset!()` macro, enabling type-safe asset references with automatic optimization and cache-busting. + +## Asset Macro Architecture + +### Compile-Time Expansion + +``` +asset!("/assets/image.png", AssetOptions::image()) + ↓ +1. Path Resolution + - Resolves relative to CARGO_MANIFEST_DIR + - Validates path exists and stays within crate + +2. File Hashing + - Creates DefaultHasher from path + options + span + - Produces 16-character hex hash + +3. Generate Link Section + - Creates __ASSETS__{hash} symbol + - Creates __MANGANIS__{hash} for legacy CLI + - Both use CBOR serialization via const_serialize + +4. Code Generation + - Emits BundledAsset const with PLACEHOLDER_HASH + - Creates Asset struct with function pointers + - Uses volatile reads to prevent optimization +``` + +### Link Section Generation + +Two static link sections for backward compatibility: + +1. **Legacy Format** (`__MANGANIS__{hash}`): + - Uses const_serialize_07 + - For older CLIs (0.7.0-0.7.1) + +2. **New Format** (`__ASSETS__{hash}`): + - Uses const_serialize_08 (CBOR) + - For dx >= 0.7.2 + +Both start with `PLACEHOLDER_HASH` sentinel, replaced by CLI during build. + +## Asset Types and Options + +### Type Hierarchy + +``` +AssetOptions +├── ImageAssetOptions +│ ├── format: ImageFormat (Png, Jpg, Webp, Avif) +│ ├── size: ImageSize (Manual | Automatic) +│ └── preload: bool +│ +├── CssAssetOptions +│ ├── minify: bool (default: true) +│ ├── preload: bool +│ └── static_head: bool +│ +├── JsAssetOptions +│ ├── minify: bool (default: true) +│ ├── preload: bool +│ └── static_head: bool +│ +├── CssModuleAssetOptions +│ ├── minify: bool +│ └── preload: bool +│ +├── FolderAssetOptions +│ └── (no custom options) +│ +└── Unknown (generic binary) +``` + +All options serializable at const-time via `SerializeConst` derive. + +### CSS Module Integration + +```rust +css_module!(Styles = "/my.module.css", AssetOptions::css_module()); + +// Generates: +struct Styles {} +impl Styles { + pub const header: &str = "abc[hash]"; // Unique scoped class + pub const button: &str = "def[hash]"; +} +``` + +**CSS Identifier Collection**: +- Scans for `.className` and `#idName` patterns +- Converts to snake_case +- Creates ConstStr values that auto-inject stylesheet on deref + +## Const Serialization System + +### CBOR Format (RFC 8949 Subset) + +**Supported Major Types**: +- UnsignedInteger (0) +- NegativeInteger (1) +- Bytes (2) +- String (3) +- Array (4) +- Map (5) + +**Not Supported**: Tagged values (6), floating point (7) + +### Const Serialization Mechanism + +Layout-based binary copying at compile time: + +```rust +unsafe trait SerializeConst: Sized { + const MEMORY_LAYOUT: Layout; +} + +enum Layout { + Enum(EnumLayout), // repr(C, u8) required + Struct(StructLayout), // Field offsets + Array(ArrayLayout), // Fixed-size + Primitive(PrimitiveLayout), + List(ListLayout), // Variable length +} +``` + +**Process**: +1. Calculate total size from MEMORY_LAYOUT +2. Copy bytes from source following layout +3. Apply transformations (endianness) +4. Append to ConstVec buffer + +### ConstStr Implementation + +```rust +pub struct ConstStr { + bytes: [MaybeUninit; 256], // Fixed 256-byte buffer + len: u32, +} +``` + +Used for asset paths and CSS identifiers. Serialized as List layout. + +## Asset Resolution Pipeline + +### Build-Time Processing (dx CLI) + +**Phase 1: Binary Scanning** +``` +1. Read compiled binary +2. Find __ASSETS__ symbols via objfile parser +3. Platform-specific methods: + - Native: object crate symbol tables + - Windows PE: PDB file parsing + - WASM: walrus data section parsing + - Android: NDK handling +``` + +**Phase 2: Asset Deserialization** +``` +For each __ASSETS__{hash}: +1. Read serialized bytes from section +2. Deserialize BundledAsset via const_serialize_08 +3. Fall back to const_serialize_07 if fails +4. Extract: absolute_source_path, bundled_path, options +``` + +**Phase 3: Unique Asset Collection** +- Deduplicate by (absolute_path, options) pair +- Create AssetManifest with unique set + +**Phase 4: Asset Optimization** + +| Type | Processing | +|------|------------| +| Image | Resize, convert format, optimize | +| CSS | Minify if enabled | +| JS | Minify if enabled | +| Folder | Recursive copy | +| Unknown | Direct copy with optional hash | + +**Phase 5: Binary Patching** +``` +For each processed asset: +1. Compute final hash (content + options + version) +2. Create new BundledAsset with: + - bundled_path: "/assets/{output-filename}" +3. Serialize to CBOR +4. Locate __ASSETS__{hash} in binary +5. Overwrite bytes at symbol offset +``` + +### Runtime Resolution + +**Development Mode** (`!is_bundled_app()`): +``` +Asset::resolve() + → Returns bundled().absolute_source_path + → Browser/native accesses original files +``` + +**Production Mode** (`is_bundled_app()`): +``` +Asset::resolve() + → Constructs path from: + * base_path() (e.g., "/app") + * bundled_path from BundledAsset + → Returns "/app/assets/{output-filename}" +``` + +### Platform-Specific Resolution + +**Web (WASM)**: +- `resolve_web_asset()`: Fetches via HTTP `fetch()` API +- Supports CORS headers +- Returns `Vec` + +**Desktop**: +- `resolve_asset_path_from_filesystem()` +- Bundle structure varies: + - macOS: `../Resources/assets/` + - Linux: `../lib/{product}/assets/` + - Windows: `assets/` (same directory as exe) + +**Android**: +- `to_java_load_asset()`: Uses NDK AssetManager +- Accesses APK's assets/ directory +- Debug fallback: `/data/local/tmp/dx/` + +## Hash-Based Cache Busting + +### Hash Computation Layers + +**INPUT_HASH** (macro generation): +``` += DefaultHasher(span.debug_string + options.string + asset_path) +Used for: __ASSETS__{INPUT_HASH} symbol +``` + +**CONTENT_HASH** (build time): +``` += Hash(source_contents + applied_options + manganis_version) +Used for: Final output filename +Example: image.png → image-a1b2c3d4e5f6g7h8.webp +``` + +**CSS_MODULE_HASH**: +``` += Hash(css_module_options + content_hash) +Used for: Scoped CSS identifiers +``` + +### Filename Generation + +``` +IMAGE: /assets/photo.png → /assets/photo-{hash}.webp +CSS/JS: /assets/style.css → /assets/style-{hash}.css +FOLDER: /assets (folder) → /assets/ (unchanged) +``` + +## Key Architectural Patterns + +### Const-Time Code Generation +- Uses proc_macro for compile-time expansion +- Leverages const fn exclusively (no allocators) +- Binary layout determined at compile time + +### Volatile Reads for Correctness +- `std::ptr::read_volatile()` prevents optimizing away link section reads +- Critical because link section gets patched post-compilation + +### Layout-Respecting Serialization +- Types must have well-defined memory layout +- `repr(C, u8)` for enums +- CBOR for variable-length fields + +### Dual Path Resolution +- Dev: Returns absolute_source_path +- Prod: Returns bundled_path with base_path +- Same Asset instance works across configurations + +### Symbol-Based Discovery +- No manifest file needed +- Binary symbols serve as asset registry +- Scales with multi-crate applications + +## Extension Points + +### Adding New Asset Types + +1. **Create Options Struct**: +```rust +#[derive(SerializeConst, ...)] +pub struct VideoAssetOptions { + format: VideoFormat, + preload: bool, +} +``` + +2. **Add AssetVariant Case**: +```rust +pub enum AssetVariant { + Video(VideoAssetOptions), +} +``` + +3. **Implement Builder**: +```rust +pub const fn video() -> AssetOptionsBuilder { + AssetOptionsBuilder::variant(VideoAssetOptions::default()) +} +``` + +4. **Register in CLI** (`assets.rs`): +```rust +AssetVariant::Video(opts) => { + // Video-specific processing +} +``` + +### Future Concepts + +**IAAC (Infrastructure as Code)**: +```rust +const DATABASE_CONFIG: Asset = asset!("/config/db.yaml"); +// CLI extracts, validates, deploys +``` + +**secret!() Macro**: +```rust +const API_KEY: Secret = secret!("DIOXUS_API_KEY"); +// Compile-time validation +// Runtime injection from secure store +``` + +## Version Compatibility + +**Legacy (0.7.0-0.7.1)**: +- const_serialize_07 +- Symbol: `__MANGANIS__{hash}` + +**Current (0.7.2+)**: +- const_serialize_08 (CBOR) +- Symbol: `__ASSETS__{hash}` + +**Migration**: CLI tries new format first, falls back to legacy if deserialization fails or PLACEHOLDER_HASH still present. diff --git a/notes/architecture/09-ROUTER.md b/notes/architecture/09-ROUTER.md new file mode 100644 index 000000000..34157535a --- /dev/null +++ b/notes/architecture/09-ROUTER.md @@ -0,0 +1,363 @@ +# Dioxus Router Architecture + +The Dioxus router provides type-safe, declarative routing with support for nested layouts, dynamic parameters, and platform-agnostic navigation. + +## Route Definition + +### Routable Trait + +Routes are defined using Rust enums with the `#[derive(Routable)]` macro: + +```rust +#[derive(Routable, Clone, PartialEq, Debug)] +enum Route { + #[route("/")] + Home {}, + #[route("/blog/:blog_id")] + Blog { blog_id: usize }, + #[route("/edit?:blog_id")] + Edit { blog_id: usize }, + #[route("/hashtag/#:hash")] + Hash { hash: String }, +} +``` + +The `Routable` trait provides: +- **`const SITE_MAP`**: Static map of all route segments +- **`render(level: usize) -> Element`**: Renders component at nesting level +- **`FromStr` implementation**: Parses URLs into route enums +- **`Display` implementation**: Converts routes to URLs +- **Helper methods**: `is_child_of()`, `parent()`, `static_routes()` + +### Route Matching Priority + +1. Query routes (`/?:query`) +2. Static routes (`/route`) +3. Dynamic routes (`/:route`) +4. Catch-all routes (`/:..route`) + +### Segment Types + +- **Static**: Literal path segments like `/blog` +- **Dynamic**: Single parameters like `/:id` (implements `FromRouteSegment`) +- **CatchAll**: Multiple segments like `/:..segments` (implements `FromRouteSegments`) + +### Query Parameters + +- **Spread query**: `/?:..query` (entire query string, uses `FromQuery`) +- **Segmented query**: `/?:param1&:param2` (individual params, use `FromQueryArgument`) + +### Hash Fragments + +`/#:hash_segment` (uses `FromHashFragment`) + +## Navigation + +### RouterContext + +Manages navigation state with these core methods: + +```rust +impl RouterContext { + // Navigation + pub fn push(&self, target: impl Into); + pub fn replace(&self, target: impl Into); + + // History + pub fn go_back(&self); + pub fn go_forward(&self); + pub fn can_go_back(&self) -> bool; + pub fn can_go_forward(&self) -> bool; + + // Introspection + pub fn current(&self) -> R; + pub fn full_route_string(&self) -> String; + pub fn prefix(&self) -> Option; + + // Site map + pub fn site_map(&self) -> &'static [SiteMapSegment]; +} +``` + +The router context is: +1. Created in `Router` component with `RouterConfig` +2. Provided via Dioxus context to children +3. Uses **signals** for reactive updates +4. Tracks **subscribers** for re-rendering +5. Integrates with **history provider** + +### NavigationTarget + +```rust +pub enum NavigationTarget { + Internal(R), // Route parsed by router + External(String), // External URL (browser handles) +} +``` + +## Link Component + +Declarative navigation: + +```rust +Link { + to: NavigationTarget, + active_class: Option, // Applied when matches current + class: Option, + new_tab: bool, + onclick: Option, + children: Element, +} +``` + +**Behavior**: +1. Accesses router without subscribing +2. Checks if target matches current for `active_class` +3. On click (left button, no modifiers): + - Prevents default + - Routes via `router.push_any(target)` + - Executes optional onclick +4. Generates `` tag with `href` for SEO +5. External links use default browser navigation + +## Nested Routes + +### Layouts and Outlets + +```rust +#[derive(Routable)] +enum Route { + #[nest("/admin")] + #[layout(AdminLayout)] + #[route("/")] + AdminHome {}, + #[route("/users")] + Users {}, + #[end_layout] + #[end_nest] + + #[route("/")] + Home {}, +} + +#[component] +fn AdminLayout() -> Element { + rsx! { + header { "Admin Panel" } + Outlet:: {} // Children render here + } +} +``` + +### Outlet Component + +- Renders child routes at correct nesting depth +- Uses `OutletContext` to track depth level +- Works with layout wrapping via macro + +### VirtualDOM Integration + +1. `Routable::render(level)` matches route at nesting level +2. Each layout increments nesting level +3. Router changes mark context-accessing components dirty +4. Outlet at correct depth renders matched component + +## Route Parameters + +### Parameter Traits + +| Component | Trait | Example | +|-----------|-------|---------| +| Path segment | `FromRouteSegment` / `ToRouteSegment` | `/users/:id` | +| Multiple segments | `FromRouteSegments` / `ToRouteSegments` | `/files/:..path` | +| Query full | `FromQuery` | `/?:q=search` | +| Query arg | `FromQueryArgument` | `/?page=1&sort=name` | +| Hash | `FromHashFragment` | `/#section-id` | + +### Auto-Implementation + +Traits auto-implement for types with standard traits: +- `FromRouteSegment`: `FromStr + Default` +- `FromQueryArgument`: `FromStr + Default` +- `FromQuery`: `From<&str>` +- `FromHashFragment`: `FromStr + Default` + +### URL Encoding + +- Path segments: `PATH_ASCII_SET` (excludes `/`, `?`, `#`) +- Query strings: `QUERY_ASCII_SET` (excludes `&`, `=`) +- Hash fragments: `FRAGMENT_ASCII_SET` +- Auto encode/decode during parse/display + +## Router Macro + +### Code Generation + +`#[derive(Routable)]` generates three implementations: + +**1. FromStr Implementation**: +```rust +impl std::str::FromStr for Route { + fn from_str(s: &str) -> Result { + // Split URL into path, query, hash + // Percent-decode segments + // Try matching against all routes (specificity order) + // Return error with attempted routes if no match + } +} +``` + +**2. Display Implementation**: +```rust +impl std::fmt::Display for Route { + fn fmt(&self, f: &mut Formatter) -> Result { + match self { + Route::Blog { blog_id } => { + write!(f, "/blog/{}", blog_id)?; + // Percent-encode as needed + } + } + } +} +``` + +**3. Routable Trait Implementation**: +```rust +impl Routable for Route { + const SITE_MAP: &'static [SiteMapSegment] = &[...]; + + fn render(&self, level: usize) -> Element { + match (level, self) { + (0, Self::Blog { .. }) => rsx! { Layout { Outlet {} } } + (1, Self::Blog { blog_id }) => rsx! { Blog { blog_id: *blog_id } } + _ => VNode::empty() + } + } +} +``` + +### Error Type Generation + +Each route generates detailed error enum: +```rust +pub enum RouteMatchError { + Home(HomeParseError), + Blog(BlogParseError), + BlogIdParseError(ParseIntError), +} +``` + +### Segment Parsing + +```rust +pub enum RouteSegment { + Static(String), // "/blog" -> Static("blog") + Dynamic(Ident, Type), // "/:id" -> Dynamic(id, u32) + CatchAll(Ident, Type), // "/:..paths" -> CatchAll(paths, Vec) +} +``` + +**Process**: +1. Split URL into path, query, hash +2. Remove trailing slashes +3. Percent-decode components +4. Split path by `/` +5. Classify segments (static/dynamic/catch-all) +6. Match field names to route parameters +7. Validate types exist in struct fields + +## History Abstraction + +### History Trait + +Platform abstraction for navigation: + +```rust +pub trait History { + fn current_route(&self) -> String; + fn current_prefix(&self) -> Option; + + fn can_go_back(&self) -> bool; + fn can_go_forward(&self) -> bool; + fn go_back(&self); + fn go_forward(&self); + + fn push(&self, route: String); + fn replace(&self, path: String); + fn external(&self, url: String) -> bool; + + fn updater(&self, callback: Arc); + fn include_prevent_default(&self) -> bool; +} +``` + +### MemoryHistory + +In-memory fallback implementation: + +```rust +pub struct MemoryHistory { + state: RefCell, + base_path: Option, +} + +struct MemoryHistoryState { + current: String, + history: Vec, // For back + future: Vec, // For forward +} +``` + +**Behavior**: +- Default starts at `/` +- `push`: Adds current to history, sets new as current, clears future +- `replace`: Only changes current +- `go_back/forward`: Swaps between stacks +- Does not navigate external URLs + +**Use Cases**: +- Testing/SSR without browser history +- In-memory fullstack navigation +- Development without browser + +### Platform Implementations + +**Web**: Browser History API, popstate events +**Desktop**: Platform-specific window/navigation APIs +**LiveView**: WebSocket with server-managed state + +## VirtualDOM Integration + +1. **Initialization**: + - `Router` creates `RouterContext` with config + - Provides to descendants via context + - Gets current route from history + - Syncs with history if needed + +2. **URL Change**: + - `RouterContext::push_any(NavigationTarget)` called + - History provider's method invoked + - `change_route()` callback fires + - Subscribers marked dirty + +3. **Re-render**: + - Components using `use_router()`/`use_navigator()` marked dirty + - `Routable::render(level)` generates new tree + - Layouts render at nesting levels + - `Outlet` renders final component + +4. **Parameter Passing**: + - Route parameters part of enum variant + - Passed as component props + - Type-safe at compile time + +## Key Design Decisions + +1. **Trait-Based Extensibility**: Custom types implement parameter traits +2. **Type Safety**: Routes are enums - invalid routes are compile errors +3. **Macro-Driven**: Reduces boilerplate, efficient matching +4. **Platform Abstraction**: History trait enables different renderers +5. **Reactive Updates**: Signals and subscriptions for efficient re-renders +6. **Nested Layouts**: Outlets allow composition without complexity +7. **URL Encoding**: Automatic percent-encoding/decoding +8. **SEO-Friendly**: Links generate real `` tags with hrefs diff --git a/notes/architecture/10-WASM-SPLIT.md b/notes/architecture/10-WASM-SPLIT.md new file mode 100644 index 000000000..05aae6220 --- /dev/null +++ b/notes/architecture/10-WASM-SPLIT.md @@ -0,0 +1,324 @@ +# WASM Code Splitting Architecture + +The WASM-Split system enables code splitting for large WebAssembly applications, allowing developers to lazily load feature chunks on demand. + +## Overview + +### Problem Solved + +Large WASM binaries can impact initial load times. WASM-Split produces: +- **main.wasm** - Core application +- **module_N_*.wasm** - Lazy-loaded feature chunks +- **chunk_N_*.wasm** - Shared code across modules +- **__wasm_split.js** - Runtime loader + +### Output Structure + +``` +dist/ +├── main.wasm # Core application +├── module_1_*.wasm # Feature chunk 1 +├── module_2_*.wasm # Feature chunk 2 +├── chunk_1_*.wasm # Shared code +└── __wasm_split.js # JavaScript loader +``` + +## Macro Usage + +### `#[wasm_split]` Attribute Macro + +Marks async functions as split boundaries: + +```rust +#[wasm_split(my_feature)] +async fn load_feature() -> i32 { + // This code will be in module_my_feature.wasm + expensive_computation() +} +``` + +**Generated Names** (pattern: `__wasm_split_0000___`): +- `__wasm_split_00my_feature00_import__load_feature` - FFI import +- `__wasm_split_00my_feature00_export__load_feature` - FFI export + +**Transformation**: +1. Generates `extern "C"` export with implementation +2. Creates import declaration from `__wasm_split.js` +3. Creates thread-local `LazySplitLoader` with load function +4. Awaits loader before calling imported version + +### `#[lazy_loader]` Macro + +For libraries creating lazy-loadable wrappers: + +```rust +#[lazy_loader(extern "auto")] +fn my_lazy_fn(x: i32) -> i32; +``` + +Returns `LazyLoader` with: +- `.load().await` - Async module loading +- `.call(args)` - Sync invocation after loading + +**"auto" ABI**: Automatically combines all modules into one. + +## Build Integration + +### Input Requirements + +CLI takes two binary inputs: +1. **original.wasm** - Pre-wasm-bindgen (with `--emit-relocs`) +2. **bindgened.wasm** - Post-wasm-bindgen + +Compilation requirements: +- `--emit-relocs` - Relocation information +- Debug symbols - Function name resolution +- LTO - Symbol consistency + +### Processing Pipeline + +**Phase 1: Discovery and Graph Building** + +``` +1. Scan imports/exports for __wasm_split_0000_* pattern +2. Parse relocations from original.wasm +3. Build call graph from: + - CODE section relocations + - DATA section relocations + - Direct function calls via IR walking +4. Build parent graph (inverse for reachability) +5. Compute reachability for each split point +``` + +**Phase 2: Chunk Identification** + +- Identify functions used by multiple modules +- Extract into shared chunks +- Build chunk dependency graph + +**Phase 3: Module Emission** + +Three output types (parallel via rayon): + +**A. Main Module**: +- Remove split point exports +- Replace element segments with dummy functions +- Create indirect function table +- Convert split imports to stub functions +- Re-export memories, globals, tables +- Run garbage collection + +**B. Split Modules** (per split point): +- Identify unique vs shared symbols +- Convert chunk functions to imports +- Clear and reconstruct data segments +- Create element segment initializers +- Export main entry function +- Run GC + +**C. Chunk Modules** (shared code): +- Similar to split modules +- No main export function +- Contains commonly-used functions + +## Walrus Operations + +The system uses **walrus** for WASM binary manipulation: + +### Function Table Management + +``` +1. Ensure funcref table exists (__indirect_function_table) +2. Expand table for split modules + shared functions +3. Create passive element segments for initialization +``` + +### Stub Function Creation + +Stubs perform indirect calls: +``` +1. Push function arguments onto stack +2. Push table index (pointing to real function) +3. Call via CallIndirect with table +``` + +### Import/Export Manipulation + +- Convert imports to local stub functions +- Re-export shared resources +- Use `__wasm_split` namespace for shared imports + +### Graph Analysis + +**Node Types**: +- `Node::Function(FunctionId)` +- `Node::DataSymbol(usize)` + +**Graphs**: +- **Call Graph**: Function → callees +- **Parent Graph**: Inverse (what calls them) +- **Reachability Graph**: Per split point, all reachable symbols +- **Main Graph**: All reachable from main exports + +## Runtime Loading + +### LazyLoader Structure + +```rust +pub struct LazyLoader { + // Generic loader for function (Args) -> Ret +} + +impl LazyLoader { + pub async fn load(&self) -> bool; // Load module + pub fn call(&self, args: Args) -> Result; +} +``` + +### LazySplitLoader State Machine + +Three states: +1. **Deferred** - Not initiated, holds load function +2. **Pending** - Load initiated, waiting for callback +3. **Completed** - Loaded (with success boolean) + +**Async Interface**: +- `SplitLoaderFuture` implements `Future` +- `poll()` handles state transitions +- Uses `Waker` to resume on callback + +### JavaScript Runtime (`__wasm_split.js`) + +**`makeLoad()` Function**: +```javascript +async function(callbackIndex, callbackData) { + // 1. Await chunk dependencies + // 2. Check if already loaded + // 3. Fetch module binary + // 4. Call initSync from main.wasm + // 5. Construct import object: + // - Memory from main module + // - Indirect function table + // - Stack pointers and TLS base + // - Main module exports as imports + // - Fused imports from other modules + // 6. WebAssembly.instantiateStreaming() + // 7. Add exports to fusedImports + // 8. Invoke callback with table index +} +``` + +**Callback Mechanism**: +```javascript +__indirect_function_table.get(callbackIndex)(callbackData, true) +``` +Wakes up Rust Future waiting in loader. + +## Integration Flow + +``` +Compile Phase: Rust code with #[wasm_split] + ↓ +Macro Expansion: FFI functions + LazyLoader + ↓ +Build Phase: wasm-split CLI processes binaries + ↓ +Parse Relocations → Build Call Graph → Identify Split Points + ↓ +Compute Reachability → Parallel Emission + ↓ +Output: main.wasm + module_*.wasm + chunk_*.wasm + __wasm_split.js + ↓ +Runtime: JavaScript fetch → Instantiate → Callback → Future wakes + ↓ +Lazy functions available synchronously +``` + +## Key Transformations + +### Main Module + +1. Remove split point exports +2. Replace element segments with dummies +3. Create indirect function table +4. Convert imports to stubs (call indirect table) +5. Re-export shared resources +6. Remove relocation/linking sections +7. GC unused code + +### Split Module + +1. Partition functions (unique vs shared) +2. Convert chunk functions to imports +3. Clear data segments, reconstruct needed data +4. Create element segment initializers +5. Create FFI imports from main +6. Export entry function +7. Remove start function +8. GC unreachable symbols + +## Usage Example + +```rust +// Define lazy-loadable feature +#[wasm_split(admin_panel)] +async fn load_admin_panel() -> AdminPanel { + AdminPanel::new() +} + +// Use in application +async fn handle_route(route: Route) { + match route { + Route::Admin => { + let panel = load_admin_panel().await; + panel.render(); + } + _ => // ... + } +} +``` + +## Design Considerations + +### Split Point Granularity + +- One `#[wasm_split]` per feature chunk +- Too fine-grained = many small fetches +- Too coarse = large chunks + +### Shared Code Optimization + +- Common functions extracted to chunks +- Prevents duplication across modules +- Chunks loaded before dependent modules + +### Memory Sharing + +- Single memory instance shared across all modules +- Indirect function table shared for cross-module calls +- TLS base pointer coordinated + +### Error Handling + +```rust +pub enum SplitLoaderError { + NotLoaded, + LoadFailed, +} +``` + +Loader returns `Result` for robust error handling. + +## Limitations + +1. **Async boundaries**: Split points must be async functions +2. **Call graph static**: Determined at build time +3. **No dynamic imports**: All split points known at compile time +4. **WASM-specific**: Only works with wasm32 target + +## Future Considerations + +- Route-based automatic splitting +- Size-based chunk optimization +- Prefetching hints +- Integration with service workers for caching