Don't clone task inputs

This commit is contained in:
TechnoPorg committed 2026-02-11 22:29:54 +01:00
1 parent 6f7a58e4c5
commit 4ef7e69cdd
10 files changed
+15 -137

No files matched your search

-109
View File
@@ -1,109 +0,0 @@
# Documentation Update Summary - Custom Type Support
## Context
Fixed a significant issue where dagx only worked with types that had `ExtractInput` trait implementations (primitives, String, Vec, HashMap, etc.). Custom user types didn't work.
**The Fix**: The `#[task]` macro now generates inline `extract_and_run()` methods with type-specific extraction logic. This means **ANY type** implementing `Clone + Send + Sync + 'static` works automatically - no trait implementations required!
## Files Updated
### 1. **README.md**
- **Features section**: Added "Works with ANY type" as a key feature (line 117)
- **Core Concepts**: Added new "Custom Types" section with example (lines 190-218)
- **Examples**: Added `custom_types.rs` to advanced examples list (line 232)
### 2. **CHANGELOG.md**
- **[Unreleased] section**: Added comprehensive documentation of this improvement (lines 8-33)
- Explained the change from ExtractInput to macro-generated extraction
- Highlighted that custom types now work seamlessly
- Noted new example file
### 3. **src/lib.rs** (Main library documentation)
- **Features**: Added custom type support as third bullet point (lines 13-15)
- **Examples section**: Added new "Custom Types" example showing usage (lines 439-484)
### 4. **src/task.rs** (Task trait documentation)
- **Trait docs**: Updated to mention macro generates extraction logic (lines 10-15)
- Added note that custom types work automatically
### 5. **src/extract.rs** (ExtractInput trait documentation)
- **Module docs**: Updated to clarify this is now legacy/internal only (lines 1-33)
- Explained modern approach via `#[task]` macro vs legacy ExtractInput trait
- Noted ExtractInput only used by internal `task_fn` helper
### 6. **dagx-macros/src/lib.rs** (Macro documentation)
- **Macro docs**: Added emphasis on type-specific extraction generation (lines 15-22)
- Highlighted that custom types work without trait implementations
### 7. **examples/custom_types.rs** (NEW FILE)
- **Created comprehensive example** demonstrating custom type support
- Shows nested custom types (User, Project, TeamMember, Team)
- Demonstrates types flowing through multiple task layers
- Full working example with detailed documentation comments
- Expected output documented in header
## Key Messages in Documentation
### What Changed
1. **Before**: Only types with ExtractInput implementations worked (primitives + standard types)
2. **After**: ANY type with `Clone + Send + Sync + 'static` works automatically
### Why This Matters
- No trait boilerplate for custom types
- Nested structs, collections of custom types all work seamlessly
- Simpler mental model - just derive Clone and go!
- More ergonomic API
### How It Works
- `#[task]` macro generates custom `extract_and_run()` implementation per task
- Type-specific extraction logic inlined at compile time
- No runtime overhead compared to ExtractInput approach
- ExtractInput now only used internally by test helper
## Documentation Gaps Remaining
None found. The following were checked and updated:
- ✅ README.md (main documentation)
- ✅ CHANGELOG.md (version history)
- ✅ src/lib.rs (API documentation)
- ✅ src/task.rs (Task trait docs)
- ✅ src/extract.rs (ExtractInput trait docs)
- ✅ dagx-macros/src/lib.rs (macro documentation)
- ✅ examples/ (new example added)
- ✅ CONTRIBUTING.md (no changes needed - general guidelines)
- ✅ docs/*.md (no changes needed - architectural docs)
## Example Test Status
The new `custom_types.rs` example:
- Created at `/home/swaits/Code/dagx/examples/custom_types.rs`
- Demonstrates User, Project, TeamMember, Team custom types
- Shows nested types and collections
- Needs to be compiled and tested: `cargo run --example custom_types`
## Recommendations
### Before Release
1. **Compile and test** the new example: `cargo run --example custom_types`
2. **Update lib.rs last audited date** (currently shows 2025-10-06 for v0.1.0)
3. **Review CHANGELOG** to ensure version number and date are correct when releasing
4. Consider adding a **migration guide** if there are any breaking changes
### Future Documentation Enhancements
1. Consider adding a "Type Requirements" section to README explaining `Clone + Send + Sync + 'static`
2. Could add more examples showing:
- Enums as task outputs
- Generic custom types
- Types with lifetimes (showing why 'static is needed)
## Summary
All documentation has been updated to accurately reflect the new custom type support. The key improvement - that ANY type works automatically with just `Clone + Send + Sync` - is now prominently featured in:
- Main features list
- Core concepts with examples
- API documentation
- Macro documentation
- A comprehensive new example
The documentation emphasizes this is a **major ergonomic improvement** that makes dagx significantly easier to use with custom types.
+2 -3
View File
@@ -114,7 +114,7 @@ See [design philosophy](docs/DESIGN_PHILOSOPHY.md) for details.
- **Compile-time cycle prevention**: Type system makes cycles impossible — no runtime checks - **Compile-time cycle prevention**: Type system makes cycles impossible — no runtime checks
- **Compile-time type safety**: Dependencies validated at compile time, no runtime type errors - **Compile-time type safety**: Dependencies validated at compile time, no runtime type errors
- **Works with ANY type**: Custom types work automatically — just `Clone + Send + Sync`, no trait implementations needed - **Works with ANY type**: Custom types work automatically — just `Send + Sync + 'static`, no trait implementations needed
- **Runtime-agnostic**: Works with Tokio, async-std, smol, or any async runtime - **Runtime-agnostic**: Works with Tokio, async-std, smol, or any async runtime
- **True parallelism**: Tasks spawn to multiple threads for genuine parallel execution - **True parallelism**: Tasks spawn to multiple threads for genuine parallel execution
- **Type-state pattern**: API prevents incorrect wiring through compile-time errors - **Type-state pattern**: API prevents incorrect wiring through compile-time errors
@@ -189,10 +189,9 @@ let task = dag.add_task(my_task).depends_on((&upstream1, &upstream2));
### Custom Types ### Custom Types
**dagx works with ANY type automatically!** As long as your type implements `Clone + Send + Sync + 'static`, it works seamlessly: **dagx works with ANY type automatically!** As long as your type implements `Send + Sync + 'static`, it works seamlessly:
```rust ```rust
#[derive(Clone)] // Just derive Clone!
struct User { struct User {
name: String, name: String,
age: u32, age: u32,
+1 -1
View File
@@ -13,7 +13,7 @@ use syn::{parse_macro_input, FnArg, Ident, ImplItem, ItemImpl, Pat, PatType, Ret
/// Apply this to an `impl` block containing a `run()` method (sync or async). The macro: /// Apply this to an `impl` block containing a `run()` method (sync or async). The macro:
/// - Derives `Input` and `Output` types from the `run()` signature /// - Derives `Input` and `Output` types from the `run()` signature
/// - Automatically implements the `Task` trait /// - Automatically implements the `Task` trait
/// - **Generates type-specific extraction logic** - works with ANY type (Clone + Send + Sync)! /// - **Generates type-specific extraction logic** - works with ANY type (Send + Sync + 'static)!
/// - Supports both sync and async run methods /// - Supports both sync and async run methods
/// - Supports stateless (no self) and stateful (&self, &mut self) tasks /// - Supports stateless (no self) and stateful (&self, &mut self) tasks
/// - Handles various input patterns (no inputs, single input, multiple inputs) /// - Handles various input patterns (no inputs, single input, multiple inputs)
+1 -1
View File
@@ -227,7 +227,7 @@ impl Task<String, Bytes> for MyTask {
let mut dag = Dag::default(); let mut dag = Dag::default();
dag.add_vertex("one", || async move { /* task */ }); dag.add_vertex("one", || async move { /* task */ });
dag.add_edge("one", "two"); dag.add_edge("one", "two");
dag.run(|fut| tokio::spawn(fut).map(Result::unwrap)).await?; dag.run().await?;
``` ```
### tasksitter (Dynamic Workflows) ### tasksitter (Dynamic Workflows)
+2 -5
View File
@@ -1,20 +1,18 @@
//! # Custom Types: Using Your Own Structs //! # Custom Types: Using Your Own Structs
//! //!
//! This example demonstrates that **ANY custom type** works in dagx as long as it implements //! This example demonstrates that **ANY custom type** works in dagx as long as it implements
//! `Clone + Send + Sync + 'static`. You don't need to implement any special traits! //! ` Send + Sync + 'static`. You don't need to implement any special traits!
//! //!
//! ## What You'll Learn //! ## What You'll Learn
//! - How to use custom structs as task outputs //! - How to use custom structs as task outputs
//! - How custom types flow through the DAG automatically //! - How custom types flow through the DAG automatically
//! - How one task's output can be consumed by multiple downstream tasks //! - How one task's output can be consumed by multiple downstream tasks
//! - No trait implementations required (just derive Clone!)
//! //!
//! ## Key Insight //! ## Key Insight
//! //!
//! **dagx works with ANY type automatically!** The `#[task]` macro generates type-specific //! **dagx works with ANY type automatically!** The `#[task]` macro generates type-specific
//! extraction logic, so you don't need to implement any special traits. Just make sure your //! extraction logic, so you don't need to implement any special traits. Just make sure your
//! type implements: //! type implements:
//! - `Clone` (so it can be extracted from Arc)
//! - `Send + Sync` (required for async tasks) //! - `Send + Sync` (required for async tasks)
//! - `'static` (no lifetime constraints) //! - `'static` (no lifetime constraints)
//! //!
@@ -39,8 +37,7 @@
use dagx::{task, DagRunner, TaskHandle}; use dagx::{task, DagRunner, TaskHandle};
use futures::FutureExt; use futures::FutureExt;
// Custom type - just derive Clone! #[derive(Debug)]
#[derive(Clone, Debug)]
struct User { struct User {
name: String, name: String,
age: u32, age: u32,
+4 -8
View File
@@ -5,8 +5,8 @@
//! **NOTE**: This module is now primarily for internal use. The `ExtractInput` trait is only //! **NOTE**: This module is now primarily for internal use. The `ExtractInput` trait is only
//! used by the internal `task_fn` test helper function. //! used by the internal `task_fn` test helper function.
//! //!
//! **For regular tasks**: The `#[task]` macro generates inline `extract_and_run()` methods with //! **For regular tasks**: The `#[task]` macro generates inline `run()` methods with
//! type-specific extraction logic. This means **ANY type** implementing `Clone + Send + Sync` //! type-specific extraction logic. This means **ANY type** implementing `Send + Sync`
//! works automatically without needing `ExtractInput` implementations! //! works automatically without needing `ExtractInput` implementations!
//! //!
//! # How it works //! # How it works
@@ -15,8 +15,8 @@
//! Outputs are wrapped in Arc for efficient fanout (cheap Arc clones instead of data clones). //! Outputs are wrapped in Arc for efficient fanout (cheap Arc clones instead of data clones).
//! //!
//! **Modern approach (via `#[task]` macro)**: //! **Modern approach (via `#[task]` macro)**:
//! - The macro generates custom extraction logic inline in `extract_and_run()` //! - The macro generates custom extraction logic inline in `run()`
//! - Works with ANY type (just needs `Clone + Send + Sync`) //! - Works with ANY type (just needs `Send + Sync + 'static`)
//! - No trait implementations needed! //! - No trait implementations needed!
//! //!
//! **Legacy approach (via `ExtractInput` trait)**: //! **Legacy approach (via `ExtractInput` trait)**:
@@ -40,10 +40,6 @@ use crate::TaskInput;
/// ///
/// Type erasure occurs only at the ExecutableNode trait boundary - by the time /// Type erasure occurs only at the ExecutableNode trait boundary - by the time
/// we're in ExtractInput, we know the concrete type and can safely downcast. /// we're in ExtractInput, we know the concrete type and can safely downcast.
///
/// NOTE: This trait is now primarily used by the internal `task_fn` test helper.
/// The `#[task]` macro generates inline extraction logic via `extract_and_run()`,
/// which allows ANY type to work without requiring ExtractInput implementations.
#[allow(dead_code)] #[allow(dead_code)]
pub trait ExtractInput: Sized + Clone { pub trait ExtractInput: Sized + Clone {
type Input: Send; type Input: Send;
+2 -4
View File
@@ -11,7 +11,7 @@
//! system. The public API is fully type-safe with no runtime type errors. Internal execution //! system. The public API is fully type-safe with no runtime type errors. Internal execution
//! uses type erasure for heterogeneous task storage, but this is never exposed to users. //! uses type erasure for heterogeneous task storage, but this is never exposed to users.
//! - **Works with ANY type**: Custom types work automatically with the `#[task]` macro—just //! - **Works with ANY type**: Custom types work automatically with the `#[task]` macro—just
//! implement `Clone + Send + Sync`, no trait implementations needed! The macro generates //! implement `Send + Sync`, no trait implementations needed! The macro generates
//! type-specific extraction logic for seamless integration. //! type-specific extraction logic for seamless integration.
//! - **Runtime-agnostic**: Works with any async runtime (Tokio, async-std, smol, Embassy, etc.) //! - **Runtime-agnostic**: Works with any async runtime (Tokio, async-std, smol, Embassy, etc.)
//! - **Type-state pattern**: The API guides you with compile-time errors if you wire dependencies //! - **Type-state pattern**: The API guides you with compile-time errors if you wire dependencies
@@ -442,15 +442,13 @@
//! //!
//! ## Custom Types //! ## Custom Types
//! //!
//! **dagx works with ANY type automatically!** As long as your type implements `Clone + Send + Sync + 'static`, //! **dagx works with ANY type automatically!** As long as your type is `Send + Sync + 'static`,
//! the `#[task]` macro generates the necessary extraction logic: //! the `#[task]` macro generates the necessary extraction logic:
//! //!
//! ```no_run //! ```no_run
//! # use dagx::{task, DagRunner, Task}; //! # use dagx::{task, DagRunner, Task};
//! # use futures::FutureExt; //! # use futures::FutureExt;
//! //!
//! // Just derive Clone - that's all you need!
//! #[derive(Clone)]
//! struct User { //! struct User {
//! name: String, //! name: String,
//! age: u32, //! age: u32,
+1 -1
View File
@@ -70,7 +70,7 @@ where
dependencies: Vec<Arc<dyn Any + Send + Sync>>, dependencies: Vec<Arc<dyn Any + Send + Sync>>,
) -> ExecuteFuture { ) -> ExecuteFuture {
Box::pin(async move { Box::pin(async move {
// Use the task's extract_and_run method which has inline extraction logic // Use the task's run method which has inline extraction logic
// generated by the #[task] macro. This allows ANY type to work without // generated by the #[task] macro. This allows ANY type to work without
// requiring ExtractInput trait implementations. // requiring ExtractInput trait implementations.
let output = self.task.run(TaskInput::new(dependencies.iter())).await; let output = self.task.run(TaskInput::new(dependencies.iter())).await;
-3
View File
@@ -527,9 +527,6 @@ impl DagRunner {
/// - The task hasn't been executed yet /// - The task hasn't been executed yet
/// - The handle is invalid /// - The handle is invalid
/// ///
/// Returns `DagError::TaskPanicked` if:
/// - A task executing in the
///
/// # Examples /// # Examples
/// ///
/// ```no_run /// ```no_run
+2 -2
View File
@@ -11,10 +11,10 @@ use crate::extract::ExtractInput;
/// ///
/// Use the [`task`](crate::task) macro to implement this trait. The macro automatically /// Use the [`task`](crate::task) macro to implement this trait. The macro automatically
/// derives `Input` and `Output` types from your `run()` method signature and generates /// derives `Input` and `Output` types from your `run()` method signature and generates
/// type-specific extraction logic in `extract_and_run()`. /// type-specific extraction logic in `run()`.
/// ///
/// **Custom types work automatically!** As long as your output type implements /// **Custom types work automatically!** As long as your output type implements
/// `Clone + Send + Sync + 'static`, the macro handles everything. No trait implementations needed! /// `Send + Sync + 'static`, the macro handles everything. No trait implementations needed!
/// ///
/// # Task Patterns /// # Task Patterns
/// ///