swaits
5839b35979
Implements dagx-11: Add comprehensive testing infrastructure including fuzzing, property-based testing, and mutation testing configuration. **Fuzzing Infrastructure:** - Created fuzz/ directory with cargo-fuzz setup - 4 fuzz targets covering different scenarios: - fuzz_dag_execution: Basic DAG execution with various patterns - fuzz_dependencies: Dependency management (linear, diamond, complex) - fuzz_type_safety: Type-safe output retrieval - fuzz_concurrent: Concurrent execution and thread safety - Comprehensive README with usage instructions - Integrated with justfile (fuzz, fuzz-all, fuzz-extended, fuzz-build) **Property-Based Testing:** - Added proptest dependency to Cargo.toml - Created tests/property_tests.rs with 14 property tests: - Value preservation through DAG - Linear chain correctness - Fan-out pattern verification - Diamond pattern correctness - String, Vec, Option, Result handling - HashMap and custom struct preservation - DAG isolation - Concurrent execution scaling - Integrated with justfile (proptest recipe) **Mutation Testing:** - Created .cargo-mutants.toml configuration - Focused on critical functions: - DagRunner::run and ::get - TaskBuilder::depends_on - ExecutableNode::execute_with_listeners - ExtractFromListeners::extract - DepsTuple::to_node_ids - Excluded low-value mutations (fmt, clone, default) - Configured for nextest and parallel execution - Already integrated via existing justfile mutants recipe **Justfile Updates:** - Added fuzzing section with 5 new recipes - Added proptest recipe for property-based tests - All recipes follow existing patterns and naming conventions Similar setup to ../announcement project but tailored for DAG-specific testing.
143 lines
4.3 KiB
TOML
143 lines
4.3 KiB
TOML
# cargo-mutants configuration for the dagx crate
|
|
#
|
|
# This configuration optimizes mutation testing by:
|
|
# - Focusing on correctness-critical code
|
|
# - Excluding low-value mutations
|
|
# - Setting appropriate timeouts
|
|
# - Configuring parallel execution
|
|
|
|
# Overall timeout for each mutant test
|
|
# Increased from default to handle async tests
|
|
timeout = 300 # 5 minutes per mutant
|
|
|
|
# Number of parallel jobs
|
|
# Set to number of CPU cores for faster execution
|
|
# jobs = 8 # Uncomment and adjust for your system
|
|
|
|
# Test tool to use
|
|
# nextest is faster than cargo test
|
|
test_tool = "nextest"
|
|
|
|
# Additional arguments to pass to the test command
|
|
# --no-fail-fast ensures all tests run even if one fails
|
|
test_args = ["--no-fail-fast"]
|
|
|
|
# Exclude patterns for files that shouldn't be mutated
|
|
# These are low-value targets that produce noise
|
|
exclude_globs = [
|
|
# Test files - we test the tests, not mutate them
|
|
"tests/**",
|
|
"**/tests.rs",
|
|
"benches/**",
|
|
"examples/**",
|
|
|
|
# Proc macro crate
|
|
"dagx-macros/**",
|
|
|
|
# Build scripts
|
|
"build.rs",
|
|
]
|
|
|
|
# Exclude specific functions/methods that shouldn't be mutated
|
|
# These are low-value mutations that don't test correctness
|
|
[[exclude_functions]]
|
|
# Display trait implementations - not correctness-critical
|
|
regex = ".*::fmt"
|
|
reason = "Display/Debug implementations don't affect correctness"
|
|
|
|
[[exclude_functions]]
|
|
# Clone implementations - trivial mutations
|
|
regex = ".*::clone"
|
|
reason = "Clone implementations are trivial"
|
|
|
|
[[exclude_functions]]
|
|
# Default implementations - trivial
|
|
regex = ".*::default"
|
|
reason = "Default implementations are trivial"
|
|
|
|
# Specific functions to focus on (include)
|
|
# These are the correctness-critical paths
|
|
[[examine_functions]]
|
|
# Core DAG execution logic
|
|
regex = "DagRunner::run"
|
|
reason = "Critical: Core DAG execution orchestration"
|
|
|
|
[[examine_functions]]
|
|
# Task addition and dependency tracking
|
|
regex = "DagRunner::add_task"
|
|
reason = "Critical: Task registration and node creation"
|
|
|
|
[[examine_functions]]
|
|
# Dependency management
|
|
regex = "TaskBuilder::depends_on"
|
|
reason = "Critical: Dependency edge creation"
|
|
|
|
[[examine_functions]]
|
|
# Output extraction and type checking
|
|
regex = "DagRunner::get"
|
|
reason = "Critical: Type-safe output retrieval"
|
|
|
|
[[examine_functions]]
|
|
# Node execution with listeners
|
|
regex = "ExecutableNode::execute_with_listeners"
|
|
reason = "Critical: Task execution and value propagation"
|
|
|
|
[[examine_functions]]
|
|
# Listener extraction from announcements
|
|
regex = "ExtractFromListeners::extract"
|
|
reason = "Critical: Input extraction for dependent tasks"
|
|
|
|
[[examine_functions]]
|
|
# Dependency tuple conversion
|
|
regex = "DepsTuple::to_node_ids"
|
|
reason = "Critical: Dependency tracking correctness"
|
|
|
|
# Minimum test time threshold
|
|
# If tests complete faster than this, warn about insufficient testing
|
|
# minimum_test_time = 1.0 # Uncomment to enable
|
|
|
|
# Output formats
|
|
# Generate both text and JSON for tracking
|
|
# output_formats = ["text", "json"] # Uncomment to enable
|
|
|
|
# Baseline
|
|
# Compare mutations against a baseline to detect regressions
|
|
# baseline = "main" # Uncomment to enable baseline comparison
|
|
|
|
# Notes on configuration strategy:
|
|
#
|
|
# 1. FOCUS: We explicitly list critical functions to mutate
|
|
# - DAG execution: run(), execute_with_listeners()
|
|
# - Dependency tracking: add_task(), depends_on(), to_node_ids()
|
|
# - Type safety: get() with downcasting
|
|
# - Value propagation: extract() from listeners
|
|
#
|
|
# 2. EXCLUDE: We exclude low-value targets
|
|
# - Display/Debug impls: String formatting doesn't affect correctness
|
|
# - Clone/Default impls: Trivial, unlikely to have bugs
|
|
# - Tests themselves: We test production code, not tests
|
|
# - Proc macros: Generated code tested via integration
|
|
#
|
|
# 3. TIMEOUT: 5 minutes is generous
|
|
# - Most mutants complete in seconds
|
|
# - Allows for async tests with real execution
|
|
# - Catches actual deadlocks without false positives
|
|
#
|
|
# 4. NEXTEST: Faster test execution
|
|
# - Parallel test execution
|
|
# - Better output formatting
|
|
# - Improved reliability
|
|
#
|
|
# 5. NO-FAIL-FAST: See all failures
|
|
# - Don't stop at first test failure
|
|
# - Get complete picture of mutation impact
|
|
# - Better for debugging
|
|
#
|
|
# Expected mutation counts with this configuration:
|
|
# - Target: ~30-50 mutants for core logic
|
|
# - Score: 90%+ (high bar for DAG correctness)
|
|
#
|
|
# Runtime with this configuration:
|
|
# - Current: ~5-10 minutes
|
|
# - With nextest: ~30% faster
|