Testing Framework

Continuous Integration

CI Status

Automated testing on Ubuntu with multiple compilers (g++, clang++)

Test Suite Overview

๐Ÿงช Comprehensive Testing

The covtobed testing framework includes multiple layers of validation:

  • Unit Tests: Core functionality testing with Catch2
  • Integration Tests: End-to-end workflow validation
  • Performance Benchmarks: Speed and memory optimization
  • Synthetic Data Generation: Controlled test scenarios

Running Tests

Quick Test Suite

# Run the main integration test suite
bash test/test.sh

This script tests all major functionality using pre-built test data.

Enhanced Integration Tests

# Run comprehensive integration tests with detailed output
bash test/integration/test_enhanced.sh

Features colored output, performance timing, and detailed validation.

Unit Tests (CMake)

# Build and run unit tests
cd test
mkdir -p build && cd build
cmake ..
make
./test_coverage

Requires bamtools development libraries and Catch2.

Performance Benchmarks

# Build and run benchmarks
cd test/build
make benchmark_coverage
./benchmark_coverage

Uses Google Benchmark for performance analysis.

Test Data

Standard Test Files

  • test/demo.bam - Main test BAM with sorted alignments
  • test/mp.bam - Mate-pair data for physical coverage
  • test/mock.bam - Synthetic data with known coverage
  • test/stranded.bam - Strand-specific test data
  • test/unsorted.bam - Unsorted BAM for error testing

Synthetic Data Generation

# Generate custom test data
bash test/utils/simple_test_generator.sh output_prefix num_reads [coverage_pattern]

# Examples:
bash test/utils/simple_test_generator.sh custom_test 1000
bash test/utils/simple_test_generator.sh stranded_test 500 stranded
bash test/utils/simple_test_generator.sh single_test 200 single

Test Categories

Functionality Tests

  • Basic coverage calculation
  • Strand-specific analysis
  • Physical coverage for paired reads
  • Quality filtering (mapping quality, alignment validity)
  • Coverage thresholds (min/max filtering)
  • Output format validation (BED, counts)
  • Pipeline integration (STDIN/STDOUT)

Error Handling Tests

  • Unsorted BAM detection
  • Invalid file format handling
  • Conflicting parameter validation
  • Memory management under load
  • Large file processing

Performance Tests

  • Coverage calculation speed
  • Memory usage optimization
  • Priority queue performance
  • Large dataset scalability
  • Multi-threading efficiency

Test Infrastructure

๐Ÿ”ง Build Requirements

For full testing capabilities, install these dependencies:

# Ubuntu/Debian
sudo apt update
sudo apt install -y \
    build-essential \
    cmake \
    pkg-config \
    libbamtools-dev \
    libjsoncpp-dev \
    libcatch2-dev \
    libbenchmark-dev

# CentOS/RHEL (requires EPEL)
sudo yum install -y \
    gcc-c++ \
    cmake \
    pkgconfig \
    bamtools-devel \
    jsoncpp-devel

Test Results and Validation

Expected Outputs

The test suite validates against known outputs:

  • test/output.bed - Expected output for demo.bam
  • test/mock.bed - Expected output for mock.bam
  • Dynamically generated expected results for synthetic data

Coverage Validation

# Manual validation example
covtobed test/demo.bam > actual_output.bed
diff actual_output.bed test/output.bed

# Should show no differences for a passing test

Adding New Tests

Integration Test Structure

# Template for new integration test
run_test() {
    local test_name="$1"
    local command="$2"
    local expected_output="$3"
    
    echo -n "Testing ${test_name}... "
    
    if eval "$command" > temp_output.bed 2>/dev/null; then
        if diff -q temp_output.bed "$expected_output" >/dev/null 2>&1; then
            echo -e "${GREEN}PASS${NC}"
            return 0
        else
            echo -e "${RED}FAIL${NC} (output mismatch)"
            return 1
        fi
    else
        echo -e "${RED}FAIL${NC} (command failed)"
        return 1
    fi
}

Unit Test Structure

// Template for new unit test
#include 

TEST_CASE("New functionality test", "[category]") {
    // Setup
    // ... test data preparation
    
    SECTION("Test case description") {
        // Test execution
        // ... your test logic
        
        // Assertions
        REQUIRE(expected_condition);
        CHECK(optional_condition);
    }
}

Troubleshooting Tests

๐Ÿ” Common Issues

  • Missing binary: Tests look for ./covtobed first, then fallback to binaries/
  • Permission errors: Ensure test files have read permissions
  • Missing dependencies: Install all required development libraries
  • CMake errors: Check pkg-config can find bamtools-1
  • Segmentation faults: May indicate test framework issues, not covtobed bugs

Contributing Tests

Test Development Guidelines

  • Add both positive and negative test cases
  • Include edge cases (empty files, extreme values)
  • Test error conditions and recovery
  • Validate output format compliance
  • Include performance regression tests
  • Document expected behavior clearly