ASP.NET Core code coverage reports on Azure DevOps

After making ASP.NET Core code coverage reports work on local box I made step further and made code coverage reports available also on Azure DevOps. This blog post shows how to generate code coverage reports for .NET and ASP.NET Core applications on Azure DevOps.

Getting started

This blog post expects that there is .NET or ASP.NET Core project with unit tests and code coverage reports. Also it expects the existence of Azure DevOps build pipeline that is connected to source code repository. It’s covered well on previous blog post about automated testing Code coverage reports for ASP.NET Core.

Here is the build pipeline of one of my sample applications. I highlighted report generator task on screenshot. Notice also date based versioning task in pipeline.

Azure DevOps: ReportGenerator task

Report generator is run after tests because before tests we don’t have test results and code coverage data.

Configuring unit tests

Most of important work is done when unit tests are run. This is why we need to configure test task first. Here is the example. Telling short, we configure unit test task that runs unit tests, gathers code coverage information and saves results to build server disk. The results are XML-files we need as an input for code coverage reports.

Azure DevOps: Configuring unit tests

Here is the arguments block as text for copy and paste. It must be all one line with no line breaks in Arguments field of test task.

--configuration $(BuildConfiguration)
/p:CollectCoverage=true
/p:CoverletOutputFormat=cobertura
/p:CoverletOutput=$(Build.SourcesDirectory)\TestResults\Coverage\

Run build to see if test task succeeds and there are no errors and warnings.

Adding code coverage reports

For code coverage reports we take free ReportGenerator task by Daniel Palme from Visual Studio Marketplace. If you need also local code coverage reports then check out ReportGenerator project from GitHub.

ReportGenerator task

If organization under what build pipeline belongs doesn’t have this task yet then the task is first added to organization and then it can be added to build pipeline. Here is how to configure ReportGenerator task.

Configure ReportGenerator task

Report types field specifies what type of reports we want. Here I asked for inline HTML specially for Azure Pipelines and Cobertura. This is how I got reporting work.

NB! If you notice testing framework assemblies and namespaces (in my case xUnit) in code coverage reports then add set Assembly filter property of ReportGenerator task to: -xunit*.*. Using similar pattern it’s possible to leave out also other unwanted assemblies.

Save changes to build pipeline and run it to make sure all steps succeed and there are also no warnings by unit tests or report generator task.

Viewing code coverage reports

If all steps in pipeline succeeded then there will be new link to code coverage reports in header of pipeline run (in red rectangle). It is shown there automatically if code coverage information is published correctly.

Azure DevOps: Code coverage link

When clicking on this link we can see full code coverage report. We stay in Azure DevOps environment, in same view. Links shown in report just work with Azure DevOps environment.

Azure DevOps: Code coverage report

The report is interactive and we can click on class names to see more specific details about code coverage.

Azure DevOps: Code coverage report

Reports like this are generated every time when our build pipeline is run. If we make build pipeline run on every commit to code repository then we get code coverage reports for every build automatically.

YAML version of same thing is available at Meziantou’s blog post Computing code coverage for a .NET Core project with Azure DevOps and Coverlet.

Wrapping up

We were already able to run code coverage reports on local boxes. Now we took step further and added code coverage reports to Azure DevOps build pipeline. This way we make code coverage reports available for all commits and also guys who don’t bother check reports locally can go and see their result on build server.

Gunnar Peipman

Gunnar Peipman is ASP.NET, Azure and SharePoint fan, Estonian Microsoft user group leader, blogger, conference speaker, teacher, and tech maniac. Since 2008 he is Microsoft MVP specialized on ASP.NET.

    16 thoughts on “ASP.NET Core code coverage reports on Azure DevOps

    • February 25, 2019 at 12:15 pm
      Permalink

      Would you know if this works better on node’s istanbul/nyc created coverage files ? Recently I was in a project where the “Coverage” -tab showed coverage but missed all the styling and javascript in the coverage report which was a bit awkward to browse in build results.

    • February 25, 2019 at 1:00 pm
      Permalink

      Azure DevOps supports Jacoco and Cobertura formats AFAIK. ReportGenerator supports today OpenCover, dotCover, Visual Studio, NCover, Cobertura, JaCoCo, Clover and Mono as input formats. If styling is missing in Azure DevOpen then make sure you use HtmlInline_AzurePipelines as output format.

    • February 25, 2019 at 8:18 pm
      Permalink

      Thanks for feedback! Added more bold and more beautiful link straight to your blog post :)

    • April 26, 2019 at 6:51 pm
      Permalink

      Any idea how to get Code Coverage badges to show in the readme.md ?
      Adding ;Badges to the Report Types input of the ReportGenerator task causes .svg and .png files to be created but I cannot find anything that tells me what to do next.
      Azure Pipelines seems to have something built in that allows the build status badge to be linked using a url. Not sure if anything exists for more general badges.

    • May 10, 2019 at 11:27 am
      Permalink

      Cool article. thank you for detailed explanation.
      it is really first article that describes almost everything,

    • May 23, 2019 at 12:55 am
      Permalink

      Following your example I keep getting No report files specified.
      ##[error]The process ‘C:\Program Files\dotnet\dotnet.exe’ failed with exit code 1.

      Is there something I am doing wrong or is there a way to see where all the files are created?

    • May 23, 2019 at 2:41 am
      Permalink

      When I made it work then file locations made my hair almost grey. After build you can click on problematic build steps to see their logs. This is how I found out where files were created. You can try to cd to $(Build.SourcesDirectory) and run some other command there to see what is the current folder. Actually in the end it all comes down to getting paths right.

    • July 16, 2019 at 6:38 pm
      Permalink

      I too am seeing the same error as TM:

      – C:\Program Files\dotnet\dotnet.exe

      It sounds like you’ve experienced similar issues… can either of you expand on where you landed on this?

    • July 18, 2019 at 12:34 am
      Permalink

      Check out the log of failed step. There’s usually error message too about what went wrong or what exactly was not found.

    • January 18, 2020 at 7:22 pm
      Permalink

      Thanks a lot, you saved me a lot of time researching.
      What is the optimal percentage for coverage in your opinion?

    • January 18, 2020 at 10:26 pm
      Permalink

      Well… it’s a subject of hot arguments and flame wars, of course :) On one end are those who are up to perfection and 100% coverage. On the other end are extremely practical guys who say that test must be written when we actually broke something. So, truth must be somewhere between.

      Based on what I have read optimal coverage should be in range 60%-80%.

      There’s always some code that doesn’t need to be tested. Good example are classes with automatic properties and no methods. If automatic property doesn’t work and test must fail then whole .NET Framework should be so broken that we have also no chance to run our tests.

    • January 27, 2020 at 2:29 am
      Permalink

      Hi,
      I am using ReportGenerator in my Azure pipeline. I am using dot test to generate the .coverage file and then using CodeCoverage.exe to convert it to xml.
      Once this conversion is done, I am using reportgenerator to use this xml and generate report in a coverageresult folder.
      Once the build is green, I am able to see code coverage tab but it still does not show html report. It has a link to download the coverage file.
      Could you please tell what I am doing wrong here?

    • January 27, 2020 at 7:16 am
      Permalink

      When I made it work on Azure DevOps I had a lot of trouble getting all paths right. Another thing is – make sure you generate report for Azure. Report types argument for ReportGenerator step must have at least one value for Azure DevOps pipeline.

    • May 11, 2020 at 6:10 pm
      Permalink

      Hi,

      Thanks for a detailed article.
      Is it possible that the publish operation with coverage HTML results is only in AzureDevOps and not on TFS 2018.
      I created a Publish task but the task did not publish and created a Coverage results tab with the HTML summery.
      I verified that my HTML report was created , I created it with ReportGenertor by using a CLI task . All my coverage results were made with gcovr task.
      Do I miss something ?

      Thanks for the Help

    Leave a Reply

    Your email address will not be published. Required fields are marked *