Test Coverage

Test coverage analyzer accepts the coverage report file and creates metrics and issues around code coverage.

This section covers configuration specific to test-coverage analyzer. Please make sure to read the general configuration guide first.

Configuration - Analysis

name

  • Type: String
  • Presence: mandatory
  • Description: Shortcode of the analyzer.
  • Example:
name = "test-coverage"

enabled

  • Type: Boolean
  • Presence: mandatory
  • Description: Toggle whether this analyzer should be run.
  • Example:
enabled = true

Configuration - Artifacts

Refer 'report' command of CLI documentation for command usage.

Key Value format
python XML(Cobertura)
go cover.out (Generated)

Examples

Go

# Run your tests and generate coverage report
go test -coverprofile=cover.out

# Install 'deepsource CLI'
curl https://deepsource.io/cli | sh

# Set DEEPSOURCE_DSN env variable from repository settings page
export DEEPSOURCE_DSN=https://sampledsn@deepsource.io

# Report coverage artifact to 'test-coverage' analyzer
./bin/deepsource report --analyzer test-coverage --key go --value-file ./cover.out

Python

Python: unittests

# Install coverage.py pacakage from pip
pip install coverage

# Run coverage
coverage run tests.py

# Generate coverage report in xml format
coverage xml

# Install 'deepsource CLI'
curl https://deepsource.io/cli | sh

# Set DEEPSOURCE_DSN env variable from repository settings page
export DEEPSOURCE_DSN=https://sampledsn@deepsource.io

# Report coverage artifact to 'test-coverage' analyzer
./bin/deepsource report --analyzer test-coverage --key python --value-file ./coverage.xml

Python: pytest

# Install pytest and pytest-cov pacakages from pip
pip install pytest pytest-cov

# Run pytest with --cov and --cov-report flags
pytest --cov=./ --cov-report xml

# Install deepsource CLI
curl https://deepsource.io/cli | sh

# Set DEEPSOURCE_DSN env variable from repository settings page
export DEEPSOURCE_DSN=https://sampledsn@deepsource.io

# Report coverage artifact to 'test-coverage' analyzer
./bin/deepsource report --analyzer test-coverage --key python --value-file ./coverage.xml

Reference: https://pypi.org/project/pytest-cov/

Python: nose2

# Install nose2 package from pip
pip install nose2[coverage_plugin]>=0.6.5

# Run nose with --with-coverage and --coverage-report flags
nose2 --with-coverage --coverage-report xml

# Install deepsource CLI
curl https://deepsource.io/cli | sh

# Set DEEPSOURCE_DSN env variable from repository settings page
export DEEPSOURCE_DSN=https://sampledsn@deepsource.io

# Report coverage artifact to 'test-coverage' analyzer
./bin/deepsource report --analyzer test-coverage --key python --value-file ./coverage.xml

Reference: https://docs.nose2.io/en/latest/plugins/coverage.html

NOTE: Usage with tox

If you are using https://tox.readthedocs.io/en/latest/ then make sure you omit .tox/* and env/* in your .coveragerc file because when running tests with tox the coverage report will contain test report files other than your source e.g site-packages etc.

example .coveragerc

[run]
branch = True
source = src
omit =
    .tox/*
    env/*

and with multiple python versions, combine the coverage data files before reporting.

example tox.ini

[tox]
envlist = cov-init,py27,py36,py37,cov-report
skipsdist=True
skip_missing_interpreters=True


[testenv]
setenv =
    COVERAGE_FILE = .coverage.{envname}
deps =
    pytest
    pytest-cov
    coverage
commands =
    pytest --cov


[testenv:cov-init]
skipsdist = True
setenv =
    COVERAGE_FILE = .coverage
deps = coverage
commands =
    coverage erase


[testenv:cov-report]
skipsdist = True
setenv =
    COVERAGE_FILE = .coverage
deps = coverage
commands =
    coverage combine
    coverage report
    coverage xml