Fail build on Azure DevOps for low code coverage

As my ASP.NET Core code coverage reports work nice in Azure DevOps build pipeline I took step forward and made builds fail on Azure DevOps if percent of lines covered is below 60. This blog post shows how to do it using ReportGenerator and Build Quality Checks build tasks.

Preparing build pipeline

This post expects that there’s build pipeline already defined on Azure DevOps and it produces code coverage reports. Those who don’t have it may find these previous writings helpful:

Work horse for stopping builds with too low code coverage is Build Quality Checks task. I expect it is already installed for organization in Azure DevOps where build pipeline is defined.

Adding Build Quality Checks task

Open build pipeline and add Build Quality Checks task to it. Make sure it is after unit tests, ReportGenerator and coverage results publishing. During unit testing code coverage information is gathered. ReportGenerator uses this information as input and besides nice code coverage reports it also generates output XML-file that is published to Azure DevOps task results. Build Quality Checks uses published coverage data to check percent of code covered.

Check Build Quality task in build pipeline

Configuring this task is extremely easy. In my mind I was ready for another digital crusade but with five minutes I was done. The following screenshot shows how I configured build quality task.

Configuration of Check Built Quality task

Now it’s time to run build pipeline and see what happens. I have small codebase where 11% of code is covered with tests. Minimum I set in Build Quality Checks settings was 60. So, it must fail.

Check Build Quality task stopped build

If at least 60 percent is met then build pipeline doesn’t fail with this task.

Wrapping up

Although I was ready for more complex configuring work and even for coding simple XML parser to find coverage and send build to error when it is lower than given threshold I got away with few minutes of work. Check Build Quality task saved me and my builds fail if code is not enough covered with unit tests. Check Build Quality task has also other interesting use cases and I suggest to check it out if automating builds and setting quality standards.

Gunnar Peipman

Gunnar Peipman is ASP.NET, Azure and SharePoint fan, Estonian Microsoft user group leader, blogger, conference speaker, teacher, and tech maniac. Since 2008 he is Microsoft MVP specialized on ASP.NET.

    2 thoughts on “Fail build on Azure DevOps for low code coverage

    • November 15, 2019 at 1:41 pm
      Permalink

      Hello, I’ve encounter a problem:

      ##[error]Unhandled: Cannot find module ‘NodeApiHelpers’
      ##[section]Finishing: Check build quality

    • November 23, 2020 at 12:23 pm
      Permalink

      Your post is very helpful.
      I have a question regarding CodeCoverageThreshold.
      Will CodeCoverageThreshold accepts decimals? such as 99.9 or 99.99 or 88.8

    Leave a Reply

    Your email address will not be published. Required fields are marked *