keelapps/Coverage for Bitbucket/Documentation

Documentation

Coverage for Bitbucket shows line coverage, the coverage of the lines a pull request changed, and every test result on the Bitbucket Cloud pull request. A pipe in your pipeline writes the results onto the commit; the app reads them and shows them to reviewers. Nothing leaves Bitbucket.

How it works

• The pipe (ghcr.io/keelapps/coverage-for-bitbucket:1) runs in your pipeline after your tests. It reads your coverage file and your JUnit XML, compares the commit with the target branch using git, and writes two Code Insights reports on the commit: Coverage and Tests. It writes them through the Pipelines proxy, so it needs no token.
• The app reads those reports and shows them on the pull request: a panel on the Overview tab, a card in the sidebar, and a merge check. It only reads.

Setting it up

1. As a workspace administrator, install Coverage for Bitbucket on your workspace. Atlassian's install page asks which workspace.
2. In bitbucket-pipelines.yml, add the pipe to the after-script of the step that runs your tests:

- step:
    name: Test
    script:
      - pytest --junitxml=test-results/junit.xml --cov=app --cov-report=xml:coverage.xml
    after-script:
      - pipe: docker://ghcr.io/keelapps/coverage-for-bitbucket:1
        variables:
          COVERAGE_FILE: coverage.xml
          MIN_CHANGED_COVERAGE: '80'
          MAX_DROP: '1'

Use after-script, not script: a failing test stops the script, and the pull request should still show which test failed.

Open a pull request, or push to one. When the pipeline finishes, the results are on the pull request. Workspace settings → Coverage for Bitbucket shows the same steps.

Pipe variables

A threshold you leave out is not checked. With no thresholds at all the report shows the numbers and always passes.

Coverage formats

Paths in the coverage file are matched to the files in your repository, so reports that name files relative to a source folder or by Go module path still line up with the diff. Several coverage files are merged into one report.

Test results

The pipe reads JUnit XML, the format almost every test runner can write, and lists every test case in the Tests report: failed ones first, then skipped, then passed, each with what it printed. Bitbucket Pipelines also collects JUnit XML itself, but its API lists failed cases only. When a commit has no Tests report, the panel falls back to the counts Pipelines collected and the failed cases it lists.

On the pull request

• In the diff, each range of changed lines that no test ran is marked by the Coverage report, and each changed file carries its changed-line totals.
• The Coverage and tests panel on the Overview tab: line coverage, coverage of the changed lines, the change against the target branch, each changed file with the lines not covered, and the tests, with an Output button on each.
• The Coverage card in the sidebar: line coverage and changed-line coverage.

The change against the target branch compares the pull request with the coverage report on the target branch's latest commit, or on one of the five commits before it when the latest has none yet. Run the pipe on your main branch too, so there is something to compare with.

The panel reads as the person viewing it, so Bitbucket's own repository permissions apply: someone who cannot see the repository sees nothing.

The merge check

Custom merge checks are off in a new workspace. A workspace administrator turns them on once, in Workspace settings → Custom merge checks → Enable. Then, in Repository settings → Custom merge checks, add Coverage meets the bar. It checks the pull request's source commit when someone pushes to it, when a reviewer approves or asks for changes, and again at merge. It fails when:

• the commit has no Coverage report yet, or the report is still pending;
• the report failed MIN_COVERAGE or MIN_CHANGED_COVERAGE; or
• coverage dropped more than MAX_DROP points against the target branch.

Its message says which, with the numbers. On Bitbucket Premium, tick required to block the merge until the check passes; on other plans the result is shown on the pull request but does not block it. Premium also lets you turn the check on for a whole workspace or project. Turn it on only where the pipeline runs the pipe, since a commit without a report fails it.

Right after a push the pipeline is usually still running, so the first result says there is no report yet. Bitbucket checks again when a reviewer approves, and you can press Rerun on the check once the pipeline has finished.

Outside Bitbucket Pipelines

The pipe is a container image, so another CI can run it against a Bitbucket Cloud repository. Set BITBUCKET_REPO_FULL_NAME (workspace/repository), BITBUCKET_COMMIT, and BITBUCKET_ACCESS_TOKEN: a repository access token with Pull requests: Write. Run it in a full clone, so it can compare with the target branch.

Limits

• 1,000 annotations per report, a Code Insights limit. The Coverage report marks files and uncovered ranges up to that number; the Tests report lists the first 1,000 tests, failures first. Either says so when it stops.
• Line coverage only. Branch coverage is not reported in this release.
• Merged line numbers. In pull-request pipelines Bitbucket builds the merge of source and target. The marked lines match the source commit unless the target changed the same file; the totals are right either way.

Permissions the app requests

The app has no write scope and no storage, and there is no permissions.external block: it cannot make an outbound call, which Atlassian enforces at the platform level. The reports are written by the pipe in your own pipeline, not by the app.

Support

Questions, bug reports and feature requests: raise a support request. Security issues: report privately.