announcement/tests

Test Documentation Standard

This document defines the standard format for documenting tests in the announcement crate. Every test in this codebase MUST have a properly formatted header following these conventions.

Table of Contents

1. Header Format
2. Category Definitions
3. Writing Guidelines
4. Examples
5. Maintenance

Header Format

Standard Test Header

/// Test: [One-line summary of what is tested]
///
/// Verifies: [What behavior/invariant is verified]
/// Category: [Type: unit/integration/property/stress/timing/etc]
/// Coverage: [What code paths/scenarios are exercised]
#[test]
fn test_name() {
    // test body
}

Property Test Header

/// Property: [Invariant being tested]
///
/// Verifies: [What this property guarantees]
/// Strategy: [What input generation strategy is used]
proptest! {
    #[test]
    fn prop_name(inputs...) {
        // property test body
    }
}

Async Test Header

/// Test: [One-line summary]
///
/// Verifies: [Behavior being verified]
/// Runtime: tokio/smol
/// Category: [Type]
#[tokio::test]
async fn test_name() {
    // async test body
}

Test Header with Notes

For tests with special conditions or important context:

/// Test: [Summary]
///
/// Verifies: [Behavior]
/// Category: [Type]
/// Note: [Any important context - e.g., "Runs 1000 iterations to catch race conditions"]
#[test]
fn test_name() {
    // test body
}

Historical Note: Previously, timing-sensitive tests used #[cfg(not(tarpaulin))] to exclude them from coverage due to cargo-tarpaulin's high instrumentation overhead. With the migration to llvm-cov, all tests now run during coverage analysis.

Field Definitions

Test / Property

Required: Yes Format: One sentence, imperative mood Purpose: Immediately tells the reader what this test does Examples:

• "Test: Basic async listen operation"
• "Test: Multiple listeners receive same value"
• "Property: All listeners receive announced value"

Verifies

Required: Yes Format: One sentence describing the guarantee or behavior Purpose: Explains what invariant or behavior this test validates Examples:

• "Verifies: Listeners can wait asynchronously and receive values"
• "Verifies: Broadcast semantics work correctly"
• "Verifies: All N listeners receive the same announced value regardless of N"

Category

Required: Yes Format: One of the defined categories (see below) Purpose: Classifies the test for filtering and organization Examples:

• "Category: integration"
• "Category: property"
• "Category: stress"

Coverage / Strategy

Optional: Use "Coverage" for regular tests, "Strategy" for property tests Format: Brief description of scope or approach Purpose: Explains what code paths or input strategies are used Examples:

• "Coverage: Async listen path, event notification, value cloning"
• "Strategy: Generates 1-100 listeners with random values"

Runtime

Optional: For async tests only Format: tokio/smol/both Purpose: Specifies which async runtime(s) are tested Example: "Runtime: tokio"

Excluded

Optional: Only for tests excluded from coverage or CI Format: "Reason: explanation" Purpose: Explains why test is excluded Example: "Excluded: Coverage: timing-sensitive"

Note

Optional: For additional context Format: Any relevant information Purpose: Provides context that doesn't fit other fields Example: "Note: Runs 1000 iterations to catch race conditions"

Category Definitions

unit

Tests of individual functions or methods in isolation. Should be fast and deterministic.

integration

End-to-end tests combining multiple components. Tests public API behavior.

property

Property-based tests using proptest or quickcheck. Validates invariants across many generated inputs.

stress

High-volume tests with many iterations or large inputs. May take longer to run.

timing

Tests dependent on specific timing or delays. Often excluded from coverage due to flakiness.

race

Tests for race conditions, TOCTOU bugs, or concurrency edge cases.

concurrency

Multi-threaded tests verifying thread safety and concurrent access patterns.

panic

Tests verifying correct behavior when panics occur. Tests panic safety and resource cleanup.

edge

Tests of boundary conditions, extreme values, and unusual inputs.

type

Tests verifying correct behavior with different Rust types.

memory

Tests verifying memory safety, leak prevention, or Arc refcounting.

mutation

Tests designed to detect semantic bugs through mutation testing.

Writing Guidelines

Be Concise

Test headers should be scannable. Aim for clarity over completeness.

Good:

/// Test: Listener receives value after announce
///
/// Verifies: Basic async listen operation works correctly
/// Category: integration

Bad:

/// Test: This test creates an announcement channel with an announcer and an announcement,
/// then creates a listener from the announcement, spawns an async task that announces a value,
/// and verifies that the listener can successfully receive that value through the async
/// listen() method...
///
/// [excessive verbosity]

Be Specific

Say exactly what's being tested, not vague generalities.

Good:

/// Property: Arc refcount equals 1 + N listeners after announce
///
/// Verifies: No reference leaks in Arc-wrapped values
/// Strategy: Tests with 0-50 listeners

Bad:

/// Property: Arc stuff works
///
/// Verifies: Reference counting
/// Strategy: Random inputs

Use Imperative Mood

Start test summaries with verbs.

Good:

• "Test: Multiple listeners receive same value"
• "Test: Panic in Drop doesn't leak memory"

Bad:

• "Test: When multiple listeners are created..."
• "Test: The case where Drop panics..."

Keep Categories Consistent

Use defined categories. Don't invent new ones without updating this document.

Document the Why

The "Verifies" field should explain WHY the test exists, not just WHAT it does.

Good:

/// Verifies: Channel cleanup happens correctly when announcer drops without announcing

Bad:

/// Verifies: Test passes

Note Special Conditions

If a test has unusual requirements, exclusions, or setup, note it.

/// Test: Timeout behavior under load
///
/// Verifies: Timeouts work correctly even with many concurrent operations
/// Category: stress
/// Excluded: Coverage: takes too long (5+ seconds)
/// Note: Runs 10,000 iterations to ensure timing is reliable

Examples

Example 1: Simple Unit Test

/// Test: is_announced returns false before announce, true after
///
/// Verifies: State transition from unannounced to announced
/// Category: unit
#[test]
fn test_is_announced() {
    let (announcer, announcement) = Announcement::<i32>::new();
    assert!(!announcement.is_announced());
    announcer.announce(42).unwrap();
    assert!(announcement.is_announced());
}

Example 2: Integration Test

/// Test: Multiple async listeners receive broadcast value
///
/// Verifies: Broadcast semantics work across multiple async tasks
/// Runtime: tokio
/// Category: integration
/// Coverage: Async listen, concurrent task spawning, value cloning
#[tokio::test]
async fn test_multiple_async_listeners() {
    let (announcer, announcement) = Announcement::new();
    // test body...
}

Example 3: Property Test

/// Property: All N listeners receive the same announced value
///
/// Verifies: Broadcast semantics hold for arbitrary listener counts
/// Strategy: Generates 1-100 listeners with random i32 values
proptest! {
    #[test]
    fn prop_all_listeners_receive_value(value: i32, listener_count in 1usize..100) {
        // property test body...
    }
}

Example 4: Stress Test

/// Test: TOCTOU protection under extreme contention
///
/// Verifies: Double-check pattern prevents races even with many threads
/// Category: race
/// Note: Runs 1000 iterations with tight timing windows to catch race conditions
#[test]
fn test_toctou_extreme_contention() {
    // stress test body...
}

Example 5: Panic Safety Test

/// Test: Arc refcount remains correct after panic
///
/// Verifies: Reference counting is panic-safe
/// Category: panic
/// Coverage: Arc refcount tracking, panic recovery, Drop behavior
#[test]
fn test_arc_refcount_with_panic() {
    // panic safety test body...
}

Maintenance

Adding New Tests

When adding a new test:

1. Choose the right category - Review Category Definitions
2. Write the header first - Document what you're testing before writing the test
3. Keep it concise - One-line summaries, focused "Verifies" statements
4. Be consistent - Follow existing patterns in the same test file

Updating Existing Tests

When modifying a test:

1. Update the header - If the test behavior changes, update the documentation
2. Maintain format - Keep the same structure as other tests
3. Check categories - Ensure the category is still accurate

Reviewing Test Documentation

When reviewing PRs:

1. Check for headers - Every test must have a header
2. Verify clarity - Test purpose should be immediately obvious
3. Validate categories - Categories should match the definitions
4. Ensure consistency - Format should match examples in this document

Cross-Reference with TEST_DOCUMENTATION.md

This document defines the in-code header format. The file TEST_DOCUMENTATION.md provides a high-level overview of the entire test suite. When adding tests:

1. Update in-code headers (this document)
2. Update TEST_DOCUMENTATION.md if adding new test categories or files
3. Keep both documents in sync

Rationale

Why Use Headers?

1. Immediate Clarity - Know what a test does without reading the entire body
2. Easy Scanning - Quickly find tests by category or purpose
3. Better Maintenance - Clear documentation helps future developers
4. Professional Polish - Shows attention to detail and code quality
5. Onboarding - New contributors understand tests faster

Why This Format?

1. Rust Conventions - Uses standard /// doc comments
2. Concise - Respects the codebase's preference for simplicity
3. Structured - Fields provide consistent organization
4. Scannable - Visual consistency makes scanning fast
5. Flexible - Optional fields allow customization without clutter

Why Mandatory?

Test documentation is not optional in this codebase. Every test MUST have a header because:

1. Code Quality - Documentation is part of code quality
2. Maintainability - Undocumented tests are technical debt
3. Consistency - Partial documentation is worse than none
4. Professional Standards - We hold ourselves to high standards

---

Last Updated: 2025-10-13 Test Count: ~628 tests across 24 files Compliance: 100% (in progress)

For high-level test suite overview, see TEST_DOCUMENTATION.md.