Test Coverage

A metric measuring what percentage of code is executed during testing, indicating how thoroughly a codebase has been tested.

Test coverage is a quantitative measure of how much source code is exercised by a test suite. In smart contract security, coverage metrics help identify untested code paths that may harbor vulnerabilities. While high coverage doesn't guarantee security, low coverage almost certainly indicates gaps that attackers could exploit.

Types of Coverage Metrics

Several coverage metrics measure different aspects of test thoroughness:

Line coverage measures the percentage of code lines executed during testing. This is the most common metric but can be misleading—a line might execute without all its edge cases being tested.

Branch coverage measures whether both true and false paths of conditional statements have been tested. This is more rigorous than line coverage because it ensures decision points are fully exercised.

Function coverage measures the percentage of functions called during testing. Low function coverage indicates entire capabilities haven't been tested at all.

Statement coverage counts individual statements executed, similar to line coverage but more granular for lines containing multiple statements.

Measuring Coverage in Foundry

Foundry provides built-in coverage reporting. Running forge coverage generates a report showing which lines, branches, and functions were executed:

1forge coverage

For detailed HTML reports:

1forge coverage --report lcov
2genhtml lcov.info -o coverage-report

The resulting report highlights covered and uncovered code, making it easy to identify testing gaps.

Coverage in Security Context

Test coverage has a complex relationship with security:

What high coverage indicates: The code has been executed during testing, reducing the chance of obvious bugs in tested paths. Development practices likely include testing discipline.

What high coverage doesn't indicate: The tests actually verify correct behavior (tests might execute code without asserting anything), all edge cases are covered, the code is secure against sophisticated attacks, or integration between components works correctly.

What low coverage indicates: Significant portions of code haven't been tested at all. These untested paths are prime candidates for vulnerabilities.

Security auditors view coverage as a starting point, not an end point. A project with 95% coverage might have critical bugs in the remaining 5%, while a project with 80% coverage might have well-tested critical paths and less important code uncovered.

Coverage Targets for Smart Contracts

Different coverage targets are appropriate for different code:

Critical paths (deposits, withdrawals, access control): Target 100% branch coverage with additional fuzz testing. These functions handle user funds and protocol security.

Complex calculations (pricing, rewards, fees): Target 100% branch coverage plus extensive edge case testing. Mathematical operations are vulnerable to overflow, underflow, and rounding errors.

Admin functions: Target high coverage with focus on access control verification. Admin functions often have lower usage but high impact if compromised.

View functions: Coverage is less critical but still valuable. Incorrect view functions can mislead users and integrating protocols.

Coverage Anti-Patterns

Avoid these common mistakes when working with coverage metrics:

Coverage-driven development: Writing tests to increase coverage numbers rather than verify behavior leads to tests that execute code without meaningful assertions. These tests provide false confidence.

Ignoring branch coverage: Achieving high line coverage while ignoring branch coverage leaves decision logic undertested. A function might have 100% line coverage but only test the happy path.

Excluding code from coverage: Some tools allow excluding code from coverage reports. While occasionally legitimate (for debugging code or known limitations), excessive exclusions hide testing gaps.

One-time measurement: Coverage should be measured continuously, not just before audits. New code additions without corresponding tests gradually erode coverage.

Combining Coverage with Other Techniques

Test coverage works best as part of a comprehensive testing strategy:

Static analysis finds issues without executing code, complementing coverage by identifying problems in both tested and untested paths.

Fuzzing generates random inputs to explore behavior beyond what manual tests cover. Even high-coverage code benefits from fuzz testing to find unexpected edge cases.

Invariant testing verifies system properties across many state transitions, catching issues that per-function coverage misses.

Code review catches logic errors and design flaws that testing alone cannot identify. Human review remains essential regardless of coverage levels.

Coverage in Audit Preparation

Before submitting code for security audit, teams should:

  1. Generate coverage reports and review uncovered code
  2. Add tests for critical uncovered paths
  3. Document any intentionally uncovered code with justification
  4. Run coverage on both unit tests and fuzz tests
  5. Ensure branch coverage meets targets for security-critical functions

Providing coverage reports to auditors helps them prioritize review effort on undertested code, making audits more efficient and thorough.

Need expert guidance on Test Coverage?

Our team at Zealynx has deep expertise in blockchain security and DeFi protocols. Whether you need an audit or consultation, we're here to help.

Get a Quote

oog
zealynx

Subscribe to Our Newsletter

Stay updated with our latest security insights and blog posts

© 2024 Zealynx