From 4ef7e69cddfce11a77e787f905fd03f2755a990f Mon Sep 17 00:00:00 2001 From: TechnoPorg Date: Sun, 8 Feb 2026 22:59:39 +0100 Subject: [PATCH] Don't clone task inputs --- DOCUMENTATION_UPDATE_SUMMARY.md | 109 -------------------------------- README.md | 5 +- dagx-macros/src/lib.rs | 2 +- docs/COMPARISONS.md | 2 +- examples/custom_types.rs | 7 +- src/extract.rs | 12 ++-- src/lib.rs | 6 +- src/node.rs | 2 +- src/runner.rs | 3 - src/task.rs | 4 +- 10 files changed, 15 insertions(+), 137 deletions(-) delete mode 100644 DOCUMENTATION_UPDATE_SUMMARY.md diff --git a/DOCUMENTATION_UPDATE_SUMMARY.md b/DOCUMENTATION_UPDATE_SUMMARY.md deleted file mode 100644 index 5d89ef4..0000000 --- a/DOCUMENTATION_UPDATE_SUMMARY.md +++ /dev/null @@ -1,109 +0,0 @@ -# Documentation Update Summary - Custom Type Support - -## Context - -Fixed a significant issue where dagx only worked with types that had `ExtractInput` trait implementations (primitives, String, Vec, HashMap, etc.). Custom user types didn't work. - -**The Fix**: The `#[task]` macro now generates inline `extract_and_run()` methods with type-specific extraction logic. This means **ANY type** implementing `Clone + Send + Sync + 'static` works automatically - no trait implementations required! - -## Files Updated - -### 1. **README.md** -- **Features section**: Added "Works with ANY type" as a key feature (line 117) -- **Core Concepts**: Added new "Custom Types" section with example (lines 190-218) -- **Examples**: Added `custom_types.rs` to advanced examples list (line 232) - -### 2. **CHANGELOG.md** -- **[Unreleased] section**: Added comprehensive documentation of this improvement (lines 8-33) - - Explained the change from ExtractInput to macro-generated extraction - - Highlighted that custom types now work seamlessly - - Noted new example file - -### 3. **src/lib.rs** (Main library documentation) -- **Features**: Added custom type support as third bullet point (lines 13-15) -- **Examples section**: Added new "Custom Types" example showing usage (lines 439-484) - -### 4. **src/task.rs** (Task trait documentation) -- **Trait docs**: Updated to mention macro generates extraction logic (lines 10-15) -- Added note that custom types work automatically - -### 5. **src/extract.rs** (ExtractInput trait documentation) -- **Module docs**: Updated to clarify this is now legacy/internal only (lines 1-33) -- Explained modern approach via `#[task]` macro vs legacy ExtractInput trait -- Noted ExtractInput only used by internal `task_fn` helper - -### 6. **dagx-macros/src/lib.rs** (Macro documentation) -- **Macro docs**: Added emphasis on type-specific extraction generation (lines 15-22) -- Highlighted that custom types work without trait implementations - -### 7. **examples/custom_types.rs** (NEW FILE) -- **Created comprehensive example** demonstrating custom type support -- Shows nested custom types (User, Project, TeamMember, Team) -- Demonstrates types flowing through multiple task layers -- Full working example with detailed documentation comments -- Expected output documented in header - -## Key Messages in Documentation - -### What Changed -1. **Before**: Only types with ExtractInput implementations worked (primitives + standard types) -2. **After**: ANY type with `Clone + Send + Sync + 'static` works automatically - -### Why This Matters -- No trait boilerplate for custom types -- Nested structs, collections of custom types all work seamlessly -- Simpler mental model - just derive Clone and go! -- More ergonomic API - -### How It Works -- `#[task]` macro generates custom `extract_and_run()` implementation per task -- Type-specific extraction logic inlined at compile time -- No runtime overhead compared to ExtractInput approach -- ExtractInput now only used internally by test helper - -## Documentation Gaps Remaining - -None found. The following were checked and updated: -- ✅ README.md (main documentation) -- ✅ CHANGELOG.md (version history) -- ✅ src/lib.rs (API documentation) -- ✅ src/task.rs (Task trait docs) -- ✅ src/extract.rs (ExtractInput trait docs) -- ✅ dagx-macros/src/lib.rs (macro documentation) -- ✅ examples/ (new example added) -- ✅ CONTRIBUTING.md (no changes needed - general guidelines) -- ✅ docs/*.md (no changes needed - architectural docs) - -## Example Test Status - -The new `custom_types.rs` example: -- Created at `/home/swaits/Code/dagx/examples/custom_types.rs` -- Demonstrates User, Project, TeamMember, Team custom types -- Shows nested types and collections -- Needs to be compiled and tested: `cargo run --example custom_types` - -## Recommendations - -### Before Release -1. **Compile and test** the new example: `cargo run --example custom_types` -2. **Update lib.rs last audited date** (currently shows 2025-10-06 for v0.1.0) -3. **Review CHANGELOG** to ensure version number and date are correct when releasing -4. Consider adding a **migration guide** if there are any breaking changes - -### Future Documentation Enhancements -1. Consider adding a "Type Requirements" section to README explaining `Clone + Send + Sync + 'static` -2. Could add more examples showing: - - Enums as task outputs - - Generic custom types - - Types with lifetimes (showing why 'static is needed) - -## Summary - -All documentation has been updated to accurately reflect the new custom type support. The key improvement - that ANY type works automatically with just `Clone + Send + Sync` - is now prominently featured in: -- Main features list -- Core concepts with examples -- API documentation -- Macro documentation -- A comprehensive new example - -The documentation emphasizes this is a **major ergonomic improvement** that makes dagx significantly easier to use with custom types. diff --git a/README.md b/README.md index 17300b5..16875ce 100644 --- a/README.md +++ b/README.md @@ -114,7 +114,7 @@ See [design philosophy](docs/DESIGN_PHILOSOPHY.md) for details. - **Compile-time cycle prevention**: Type system makes cycles impossible — no runtime checks - **Compile-time type safety**: Dependencies validated at compile time, no runtime type errors -- **Works with ANY type**: Custom types work automatically — just `Clone + Send + Sync`, no trait implementations needed +- **Works with ANY type**: Custom types work automatically — just `Send + Sync + 'static`, no trait implementations needed - **Runtime-agnostic**: Works with Tokio, async-std, smol, or any async runtime - **True parallelism**: Tasks spawn to multiple threads for genuine parallel execution - **Type-state pattern**: API prevents incorrect wiring through compile-time errors @@ -189,10 +189,9 @@ let task = dag.add_task(my_task).depends_on((&upstream1, &upstream2)); ### Custom Types -**dagx works with ANY type automatically!** As long as your type implements `Clone + Send + Sync + 'static`, it works seamlessly: +**dagx works with ANY type automatically!** As long as your type implements `Send + Sync + 'static`, it works seamlessly: ```rust -#[derive(Clone)] // Just derive Clone! struct User { name: String, age: u32, diff --git a/dagx-macros/src/lib.rs b/dagx-macros/src/lib.rs index 2d01792..891e03b 100644 --- a/dagx-macros/src/lib.rs +++ b/dagx-macros/src/lib.rs @@ -13,7 +13,7 @@ use syn::{parse_macro_input, FnArg, Ident, ImplItem, ItemImpl, Pat, PatType, Ret /// Apply this to an `impl` block containing a `run()` method (sync or async). The macro: /// - Derives `Input` and `Output` types from the `run()` signature /// - Automatically implements the `Task` trait -/// - **Generates type-specific extraction logic** - works with ANY type (Clone + Send + Sync)! +/// - **Generates type-specific extraction logic** - works with ANY type (Send + Sync + 'static)! /// - Supports both sync and async run methods /// - Supports stateless (no self) and stateful (&self, &mut self) tasks /// - Handles various input patterns (no inputs, single input, multiple inputs) diff --git a/docs/COMPARISONS.md b/docs/COMPARISONS.md index dd7ed7a..eec374d 100644 --- a/docs/COMPARISONS.md +++ b/docs/COMPARISONS.md @@ -227,7 +227,7 @@ impl Task for MyTask { let mut dag = Dag::default(); dag.add_vertex("one", || async move { /* task */ }); dag.add_edge("one", "two"); -dag.run(|fut| tokio::spawn(fut).map(Result::unwrap)).await?; +dag.run().await?; ``` ### tasksitter (Dynamic Workflows) diff --git a/examples/custom_types.rs b/examples/custom_types.rs index 60f8198..c4ebb11 100644 --- a/examples/custom_types.rs +++ b/examples/custom_types.rs @@ -1,20 +1,18 @@ //! # Custom Types: Using Your Own Structs //! //! This example demonstrates that **ANY custom type** works in dagx as long as it implements -//! `Clone + Send + Sync + 'static`. You don't need to implement any special traits! +//! ` Send + Sync + 'static`. You don't need to implement any special traits! //! //! ## What You'll Learn //! - How to use custom structs as task outputs //! - How custom types flow through the DAG automatically //! - How one task's output can be consumed by multiple downstream tasks -//! - No trait implementations required (just derive Clone!) //! //! ## Key Insight //! //! **dagx works with ANY type automatically!** The `#[task]` macro generates type-specific //! extraction logic, so you don't need to implement any special traits. Just make sure your //! type implements: -//! - `Clone` (so it can be extracted from Arc) //! - `Send + Sync` (required for async tasks) //! - `'static` (no lifetime constraints) //! @@ -39,8 +37,7 @@ use dagx::{task, DagRunner, TaskHandle}; use futures::FutureExt; -// Custom type - just derive Clone! -#[derive(Clone, Debug)] +#[derive(Debug)] struct User { name: String, age: u32, diff --git a/src/extract.rs b/src/extract.rs index 4731116..f11c262 100644 --- a/src/extract.rs +++ b/src/extract.rs @@ -5,8 +5,8 @@ //! **NOTE**: This module is now primarily for internal use. The `ExtractInput` trait is only //! used by the internal `task_fn` test helper function. //! -//! **For regular tasks**: The `#[task]` macro generates inline `extract_and_run()` methods with -//! type-specific extraction logic. This means **ANY type** implementing `Clone + Send + Sync` +//! **For regular tasks**: The `#[task]` macro generates inline `run()` methods with +//! type-specific extraction logic. This means **ANY type** implementing `Send + Sync` //! works automatically without needing `ExtractInput` implementations! //! //! # How it works @@ -15,8 +15,8 @@ //! Outputs are wrapped in Arc for efficient fanout (cheap Arc clones instead of data clones). //! //! **Modern approach (via `#[task]` macro)**: -//! - The macro generates custom extraction logic inline in `extract_and_run()` -//! - Works with ANY type (just needs `Clone + Send + Sync`) +//! - The macro generates custom extraction logic inline in `run()` +//! - Works with ANY type (just needs `Send + Sync + 'static`) //! - No trait implementations needed! //! //! **Legacy approach (via `ExtractInput` trait)**: @@ -40,10 +40,6 @@ use crate::TaskInput; /// /// Type erasure occurs only at the ExecutableNode trait boundary - by the time /// we're in ExtractInput, we know the concrete type and can safely downcast. -/// -/// NOTE: This trait is now primarily used by the internal `task_fn` test helper. -/// The `#[task]` macro generates inline extraction logic via `extract_and_run()`, -/// which allows ANY type to work without requiring ExtractInput implementations. #[allow(dead_code)] pub trait ExtractInput: Sized + Clone { type Input: Send; diff --git a/src/lib.rs b/src/lib.rs index d2628f3..d1d9726 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -11,7 +11,7 @@ //! system. The public API is fully type-safe with no runtime type errors. Internal execution //! uses type erasure for heterogeneous task storage, but this is never exposed to users. //! - **Works with ANY type**: Custom types work automatically with the `#[task]` macro—just -//! implement `Clone + Send + Sync`, no trait implementations needed! The macro generates +//! implement `Send + Sync`, no trait implementations needed! The macro generates //! type-specific extraction logic for seamless integration. //! - **Runtime-agnostic**: Works with any async runtime (Tokio, async-std, smol, Embassy, etc.) //! - **Type-state pattern**: The API guides you with compile-time errors if you wire dependencies @@ -442,15 +442,13 @@ //! //! ## Custom Types //! -//! **dagx works with ANY type automatically!** As long as your type implements `Clone + Send + Sync + 'static`, +//! **dagx works with ANY type automatically!** As long as your type is `Send + Sync + 'static`, //! the `#[task]` macro generates the necessary extraction logic: //! //! ```no_run //! # use dagx::{task, DagRunner, Task}; //! # use futures::FutureExt; //! -//! // Just derive Clone - that's all you need! -//! #[derive(Clone)] //! struct User { //! name: String, //! age: u32, diff --git a/src/node.rs b/src/node.rs index 0a95e29..d566a8b 100644 --- a/src/node.rs +++ b/src/node.rs @@ -70,7 +70,7 @@ where dependencies: Vec>, ) -> ExecuteFuture { Box::pin(async move { - // Use the task's extract_and_run method which has inline extraction logic + // Use the task's run method which has inline extraction logic // generated by the #[task] macro. This allows ANY type to work without // requiring ExtractInput trait implementations. let output = self.task.run(TaskInput::new(dependencies.iter())).await; diff --git a/src/runner.rs b/src/runner.rs index 8a2b977..339b81e 100644 --- a/src/runner.rs +++ b/src/runner.rs @@ -527,9 +527,6 @@ impl DagRunner { /// - The task hasn't been executed yet /// - The handle is invalid /// - /// Returns `DagError::TaskPanicked` if: - /// - A task executing in the - /// /// # Examples /// /// ```no_run diff --git a/src/task.rs b/src/task.rs index 4898c6a..604c2aa 100644 --- a/src/task.rs +++ b/src/task.rs @@ -11,10 +11,10 @@ use crate::extract::ExtractInput; /// /// Use the [`task`](crate::task) macro to implement this trait. The macro automatically /// derives `Input` and `Output` types from your `run()` method signature and generates -/// type-specific extraction logic in `extract_and_run()`. +/// type-specific extraction logic in `run()`. /// /// **Custom types work automatically!** As long as your output type implements -/// `Clone + Send + Sync + 'static`, the macro handles everything. No trait implementations needed! +/// `Send + Sync + 'static`, the macro handles everything. No trait implementations needed! /// /// # Task Patterns ///