Test Coverage

Test coverage analyzer accepts the coverage report file and creates metrics and issues around code coverage.

This section covers configuration specific to test-coverage analyzer. Please make sure to read the general configuration guide first.

Configuration - .deepsource.toml

name

• Type: String
• Presence: mandatory
• Description: Shortcode of the analyzer.
• Example:

name = "test-coverage"

enabled

• Type: Boolean
• Presence: mandatory
• Description: Toggle whether this analyzer should be run.
• Example:

enabled = true

Configuration - Artifacts

Refer 'report' command of CLI documentation for command usage.

Key	Value format
python	XML(Cobertura)
go	cover.out (Generated)

Examples

Go

# Run your tests and generate coverage report
go test -coverprofile=cover.out

# Install 'deepsource CLI'
curl https://deepsource.io/cli | sh

# Set DEEPSOURCE_DSN env variable from repository settings page
export DEEPSOURCE_DSN=https://sampledsn@deepsource.io

# Report coverage artifact to 'test-coverage' analyzer
./bin/deepsource report --analyzer test-coverage --key go --value-file ./cover.out

Note: Tests for different go packages

If you are running tests and generating coverage report for different packages separately, you need to combine all generated coverage reports before reporting it to the test-coverage analyzer.

This is how you can combine the coverage artifacts before reporting it:

# Run tests for a package
go test -coverprofile=cover.out ./somePackage

# Append coverage data in a seperate file
cat cover.out >> coverage.out

# Run tests for another package
go test -coverprofile=cover.out ./someOtherPackage

# Combine the coverage data in the same way
cat cover.out >> coverage.out

# Install 'deepsource CLI'
curl https://deepsource.io/cli | sh

# Set DEEPSOURCE_DSN env variable from repository settings page
export DEEPSOURCE_DSN=https://sampledsn@deepsource.io

# Report this `coverage.out` with combined coverage data to the `test-coverage` analyzer
./bin/deepsource report --analyzer test-coverage --key go --value-file ./coverage.out

Python

Python: unittests

# Install coverage.py pacakage from pip
pip install coverage

# Run coverage
coverage run tests.py

# Generate coverage report in xml format
coverage xml

# Install 'deepsource CLI'
curl https://deepsource.io/cli | sh

# Set DEEPSOURCE_DSN env variable from repository settings page
export DEEPSOURCE_DSN=https://sampledsn@deepsource.io

# Report coverage artifact to 'test-coverage' analyzer
./bin/deepsource report --analyzer test-coverage --key python --value-file ./coverage.xml

Python: pytest

# Install pytest and pytest-cov pacakages from pip
pip install pytest pytest-cov

# Run pytest with --cov and --cov-report flags
pytest --cov=./ --cov-report xml

# Install deepsource CLI
curl https://deepsource.io/cli | sh

# Set DEEPSOURCE_DSN env variable from repository settings page
export DEEPSOURCE_DSN=https://sampledsn@deepsource.io

# Report coverage artifact to 'test-coverage' analyzer
./bin/deepsource report --analyzer test-coverage --key python --value-file ./coverage.xml

Reference: https://pypi.org/project/pytest-cov/

Python: nose2

# Install nose2 package from pip
pip install nose2[coverage_plugin]>=0.6.5

# Run nose with --with-coverage and --coverage-report flags
nose2 --with-coverage --coverage-report xml

# Install deepsource CLI
curl https://deepsource.io/cli | sh

# Set DEEPSOURCE_DSN env variable from repository settings page
export DEEPSOURCE_DSN=https://sampledsn@deepsource.io

# Report coverage artifact to 'test-coverage' analyzer
./bin/deepsource report --analyzer test-coverage --key python --value-file ./coverage.xml

Reference: https://docs.nose2.io/en/latest/plugins/coverage.html

NOTE: Usage with tox

If you are using tox.readthedocs.io then make sure you omit .tox/* and env/* in your .coveragerc file because when running tests with tox the coverage report will contain test report files other than your source e.g site-packages etc.

example .coveragerc

[run]
branch = True
source = src
omit =
    .tox/*
    env/*

and with multiple python versions, combine the coverage data files before reporting.

example tox.ini

[tox]
envlist = cov-init,py27,py36,py37,cov-report
skipsdist=True
skip_missing_interpreters=True


[testenv]
setenv =
    COVERAGE_FILE = .coverage.{envname}
deps =
    pytest
    pytest-cov
    coverage
commands =
    pytest --cov


[testenv:cov-init]
skipsdist = True
setenv =
    COVERAGE_FILE = .coverage
deps = coverage
commands =
    coverage erase


[testenv:cov-report]
skipsdist = True
setenv =
    COVERAGE_FILE = .coverage
deps = coverage
commands =
    coverage combine
    coverage report
    coverage xml