Code Coverage for Bitbucket

Inspiration

Hi, I'm Daniel. 🙋‍♂️ In my career as a software developer I found code coverage to be an overlooked aspect of the software development workflow. It is a powerful tool that can help teams change code with confidence. When I was setting up unit testing from scratch for a bigger project in Bitbucket pipelines I was surprised to see no support for visualization and reporting of code coverage. So I made an app, just for that.

What it does

Code Coverage for Bitbucket, you guessed it, reports code coverage. In your bitbucket-pipelines.yml run your tests with code coverage, then add a step to upload it to the app and it will make a Code Insight report from it.

Report

It also adds a custom merge check, which can prevent merging a PR if the coverage is below a limit, or it decreased.

Merge check

How I built it

The core functionality of this app is implemented in a webtrigger, that users have to call explicitly from a pipeline step, uploading the coverage file. It parses the file, and creates a report using the Bitbucket REST API. It also tries to find the report for the previous commit to compare the coverages. I made good use of the Bitbucket repository settings page, where users can set the configuration for the merge check and it also shows the webtrigger custom url as a curl command to be able to simply paste it into the bitbucket-pipelines.yml file.

Challenges I ran into

The repository is missing from the context for webtriggers. To work around this issue, the user has to pass $BITBUCKET_REPO_SLUG as a url query parameter.
An event trigger for a finished pipeline, to check and consume code coverage would also make my app simpler, but for now, users have to explicitly call the webtrigger endpoint of app.
The three merge check triggers (on-code-pushed, on-reviewer-status-changed,on-merge), did not really suit my need, something like an on-pipeline-finished would make the merge check more responsive. For now, it is configured as on-merge, and users only see the check's status when they hit merge.

Accomplishments that I'm proud of

The best feature of the app is that it can compare the code coverage to either the previous commit, or in case of PRs, the source commit. I achieved it without communicating or storing data outside of Atlassian, everything is stored in Reports. Also building the app from scratch without prior knowledge of Forge was a challenge, but the documentation was awesome.

What's next for Code Coverage for Bitbucket

This app currently supports coverage.py's coverage.json format, but support for other formats can be easily added in the future.
The merge check can be also enhanced to prevent merging if there are changed code lines in the PR that are not covered.