Testing Framework
Continuous Integration
Automated testing on Ubuntu with multiple compilers (g++, clang++)
Test Suite Overview
๐งช Comprehensive Testing
The covtobed testing framework includes multiple layers of validation:
- Unit Tests: Core functionality testing with Catch2
- Integration Tests: End-to-end workflow validation
- Performance Benchmarks: Speed and memory optimization
- Synthetic Data Generation: Controlled test scenarios
Running Tests
Quick Test Suite
# Run the main integration test suite
bash test/test.sh
This script tests all major functionality using pre-built test data.
Enhanced Integration Tests
# Run comprehensive integration tests with detailed output
bash test/integration/test_enhanced.sh
Features colored output, performance timing, and detailed validation.
Unit Tests (CMake)
# Build and run unit tests
cd test
mkdir -p build && cd build
cmake ..
make
./test_coverage
Requires bamtools development libraries and Catch2.
Performance Benchmarks
# Build and run benchmarks
cd test/build
make benchmark_coverage
./benchmark_coverage
Uses Google Benchmark for performance analysis.
Test Data
Standard Test Files
test/demo.bam- Main test BAM with sorted alignmentstest/mp.bam- Mate-pair data for physical coveragetest/mock.bam- Synthetic data with known coveragetest/stranded.bam- Strand-specific test datatest/unsorted.bam- Unsorted BAM for error testing
Synthetic Data Generation
# Generate custom test data
bash test/utils/simple_test_generator.sh output_prefix num_reads [coverage_pattern]
# Examples:
bash test/utils/simple_test_generator.sh custom_test 1000
bash test/utils/simple_test_generator.sh stranded_test 500 stranded
bash test/utils/simple_test_generator.sh single_test 200 single
Test Categories
Functionality Tests
- Basic coverage calculation
- Strand-specific analysis
- Physical coverage for paired reads
- Quality filtering (mapping quality, alignment validity)
- Coverage thresholds (min/max filtering)
- Output format validation (BED, counts)
- Pipeline integration (STDIN/STDOUT)
Error Handling Tests
- Unsorted BAM detection
- Invalid file format handling
- Conflicting parameter validation
- Memory management under load
- Large file processing
Performance Tests
- Coverage calculation speed
- Memory usage optimization
- Priority queue performance
- Large dataset scalability
- Multi-threading efficiency
Test Infrastructure
๐ง Build Requirements
For full testing capabilities, install these dependencies:
# Ubuntu/Debian
sudo apt update
sudo apt install -y \
build-essential \
cmake \
pkg-config \
libbamtools-dev \
libjsoncpp-dev \
libcatch2-dev \
libbenchmark-dev
# CentOS/RHEL (requires EPEL)
sudo yum install -y \
gcc-c++ \
cmake \
pkgconfig \
bamtools-devel \
jsoncpp-devel
Test Results and Validation
Expected Outputs
The test suite validates against known outputs:
test/output.bed- Expected output for demo.bamtest/mock.bed- Expected output for mock.bam- Dynamically generated expected results for synthetic data
Coverage Validation
# Manual validation example
covtobed test/demo.bam > actual_output.bed
diff actual_output.bed test/output.bed
# Should show no differences for a passing test
Adding New Tests
Integration Test Structure
# Template for new integration test
run_test() {
local test_name="$1"
local command="$2"
local expected_output="$3"
echo -n "Testing ${test_name}... "
if eval "$command" > temp_output.bed 2>/dev/null; then
if diff -q temp_output.bed "$expected_output" >/dev/null 2>&1; then
echo -e "${GREEN}PASS${NC}"
return 0
else
echo -e "${RED}FAIL${NC} (output mismatch)"
return 1
fi
else
echo -e "${RED}FAIL${NC} (command failed)"
return 1
fi
}
Unit Test Structure
// Template for new unit test
#include
TEST_CASE("New functionality test", "[category]") {
// Setup
// ... test data preparation
SECTION("Test case description") {
// Test execution
// ... your test logic
// Assertions
REQUIRE(expected_condition);
CHECK(optional_condition);
}
}
Troubleshooting Tests
๐ Common Issues
- Missing binary: Tests look for
./covtobedfirst, then fallback tobinaries/ - Permission errors: Ensure test files have read permissions
- Missing dependencies: Install all required development libraries
- CMake errors: Check pkg-config can find bamtools-1
- Segmentation faults: May indicate test framework issues, not covtobed bugs
Contributing Tests
Test Development Guidelines
- Add both positive and negative test cases
- Include edge cases (empty files, extreme values)
- Test error conditions and recovery
- Validate output format compliance
- Include performance regression tests
- Document expected behavior clearly