Code coverage reports for ASP.NET Core

Code coverage reports for ASP.NET Core projects are not provided out-of-box but using right tools we can build decent code coverage reports. I needed code coverage reports in some of my projects and here is how I made things work using different free libraries and packages.

Getting started

To get started we need test project and some NuGet packages. Test project can be regular .NET Core library project. Add reference to web application project and write some unit tests if you start with new test project. We also need some NuGet packages to make things work:

  • coverlet.msbuild
  • Microsoft.CodeCoverage
  • Microsoft.NET.Test.Sdk
  • ReportGenerator by Daniel Palme
  • xunit
  • xunit.runner.visualstudio
  • XunitXml.TestLogger

NB! The most painless way to get ReportGenerator work was using local tool. Other options failed for me as ReportGenerator works with .NET Core 3.0 projects but not with .NET Core 3.1. I went with external tool because reports are then generated without any dependencies to project binaries. Here’s the command line to install ReportGenerator to tools folder (open command line and move to unit tests project folder):

dotnet tool install dotnet-reportgenerator-globaltool --tool-path tools

After adding these packages it’s time to make test build and see if everything still works and we don’t have any build issues.

Creating reporting folders

ASP.NET Core code coverage reports foldersNow it’s time to configure reporting. I decided to keep reports in BuildReports folder of test project. There are two subfolders:

  • Coverage – for coverage reports (this blog post)
  • UnitTests – unit tests reports (for future use)

I added BuildReports folder also to .gitignore file because I don’t want these files to wander from one developer box to another and be part of commits.

The number of files in Coverage folder is not small. It’s not just two or three files that are easy to ignore. There can be hundreds or thousands of files depending on how many tests there are in test projects. Here we are going in smaller scale of course.

Getting code coverage data

To generate reports we need coverage data and this is why we added coverlet.msbuild package to test project. When tests are run we gather code coverage information and publish it in Cobertura output format. Cobertura is popular code coverage utility in Java world. Test data is transformed to Cobertura format by Coverlet – a cross platform code coverage library for .NET Core.

With coverage data I also output unit test results in Microsoft and xUnit formats to UnitTests folder. As I said before this is for future use and we don’t do anything with files in these folders right now.

I added run-tests.bat file to root folder of my test project and first command there is for running unit tests (in your file put it all to one line without any line breaks).

dotnet test --logger "trx;LogFileName=TestResults.trx" ^
            --logger "xunit;LogFileName=TestResults.xml" ^
            --results-directory ./BuildReports/UnitTests ^
            /p:CollectCoverage=true ^
            /p:CoverletOutput=BuildReports\Coverage\ ^
            /p:CoverletOutputFormat=cobertura ^
            /p:Exclude="[xunit.*]*

This is what this command does:

  1. Using Visual Studio logger create TestResults.trx file for test results
  2. Using xUnit logger create TestResults.xml file for test results
  3. Put test results to ./BuildReports/UnitTests folder
  4. Enable collecting of code coverage data
  5. Make Coverlet to use BuildReports\Coverage folder
  6. Set Coverlet output format to Cobertura
  7. Excude xUnit libraries from test results

As a result of this command we will have three files:

  1. BuildReports\UnitTests\TestResults.trx (not important now)
  2. BuildReports\UnitTests\TestResults.xml (not important now)
  3. coverage.cobertura.xml (coverage results in Cobertura format)

BuildReports folder should not go to source code repository! If you are using git then you should add new rule to .gitignore file to avoid BuildReports folder getting to source code repository. Add this rule to .gitignore file of solution: [Bb]uildReports/

Now it’s time to try out run-tests.bat to see if everything still works and if files generated to expected locations.

Generating code coverage reports

For code coverage reports we need to add another command to run-tests.bat file. This command will run report generator that generater reports based on coverage.cobertura.xml. The reports are generated to same folder to keep folder tree smaller.

Here is the command (put it all on one line):

tools\reportgenerator.exe ^
       "-reports:BuildReports\Coverage\coverage.cobertura.xml" ^
       "-targetdir:BuildReports\Coverage" ^
       -reporttypes:HTML;HTMLSummary

I think this command is not very cryptic and I don’t make additional comments on command line parameters here.

As a lazy guy I expect browser to open with newly generated reports automatically and this is why the last line of my run-tests.bat is:

start BuildReports\Coverage\index.htm

It’s time to run the script and see if it runs successfully to end.

Code coverage on Linux

On Linux we need shell script to run tests and generate reports. Here’s sample script for linux (every command goes to one line):

#!/bin/sh

sudo dotnet test --logger 'trx;LogFileName=TestResults.trx' 
                 --logger 'xunit;LogFileName=TestResults.xml' 
                 --results-directory ./BuildReports/UnitTests 
                 /p:CollectCoverage=true 
                 /p:CoverletOutput=BuildReports/Coverage/ 
                 /p:CoverletOutputFormat=cobertura 
                 /p:Exclude='[xunit.*]*'

sudo dotnet reportgenerator 
                 -reports:BuildReports/Coverage/coverage.cobertura.xml 
                 -targetdir:BuildReports/Coverage 
                 -reporttypes:"HTML;HTMLSummary"

Not sure why tooling needs sudo but this is what is asked for. Shell script may also need execute permissions. Here’s the command for this:

chmod +x run-tests.sh

Now we are good to go on Linux too.

Code coverage reports

After running the batch file in my playground test project folder I see the following report in browser.

Code coverage summary report

The report is longer than we can see here but I’m still not very happy with it. I would like have better structural view of tested code so I have better overview about how well different system areas are covered with tests. Let’s take a look at this Grouping slider above tests table and try to move it. Voila!

Code coverage summary report grouped by namespaces

To see over-all view of system under test I can click and close those bold namespaces. Now we see how much one or another namespace is covered.

Code coverage summary report grouped by namespaces

We can also go inside classes and see coverage statistics about specific classes. Nice thing is we get also method bases statistics and source code view shows us what lines in class are covered with tests and what lines are not covered.

Code coverage report for class

I think this kind of code coverage reporting is good enough for me.

Wrapping up

The path to code coverage reporting is not always easy but I got it work like expected. All tools I used are free and no hidden expenses besides my own time didn’t applied. We had to write batch file to run tests, collect code coverage data and generate reports. In the end we got decent reports giving us good overview of code coverage of our codebase.

Gunnar Peipman

Gunnar Peipman is ASP.NET, Azure and SharePoint fan, Estonian Microsoft user group leader, blogger, conference speaker, teacher, and tech maniac. Since 2008 he is Microsoft MVP specialized on ASP.NET.

    18 thoughts on “Code coverage reports for ASP.NET Core

    • April 10, 2019 at 9:32 am
      Permalink

      Hi, I am a beginner .NET developer. I have followed your article and it has been very useful to add coverage to my project, but after executing code coverage, in the report it comes out my test project, what could happen that?, how could I exclude it?

    • April 10, 2019 at 11:16 am
      Permalink

      Ok!, I got it. I had to exclude it with /p:Exclude='[xunit.*]*%2c[My_Assembly_Test_Project]*’

    • April 16, 2019 at 4:19 pm
      Permalink

      HI, I followed instructions, running “dotnet test –logger “trx;LogFileName=TestResults.trx” ……… creates

      BuildReports\UnitTests\TestResults_2019-04-16_16-05-18-859.trx
      \BuildReports\UnitTests\TestResults.xml

      but not “coverage.cobertura.xml”

      I tried several times, no way, can you please help?

    • April 17, 2019 at 9:47 am
      Permalink

      Yes, through nugget manager, here is the contents of the csproj file:

      all
      runtime; build; native; contentfiles; analyzers; buildtransitive

      ..\..\..\..\Program Files\dotnet\sdk\NuGetFallbackFolder\microsoft.aspnetcore.mvc.viewfeatures\2.2.0\lib\netstandard2.0\Microsoft.AspNetCore.Mvc.ViewFeatures.dll

      I added the last itemgroup manually as you indicated

    • April 17, 2019 at 10:26 am
      Permalink

      Can you publish your solution somewhere? Github or something so I can check out what’s wrong.

    • April 24, 2019 at 11:25 am
      Permalink

      Hi Gunnar, any feedback?

    • April 24, 2019 at 8:27 pm
      Permalink

      Hi,
      I will take a look at your problem now. Will send you message in GitHub when I sort out what’s wrong.

    • April 25, 2019 at 11:18 am
      Permalink

      Hi, I responded to you on Git

    • July 9, 2019 at 3:30 pm
      Permalink

      Great article; I really appreciate the help.

      One item to note,..

      My reports wouldn’t generate.

      When I did the research, here’s what I found.

      In the run-tests.bat the report generation command reads “dotnet /reportgenerator…”

      But when I installed the global tool using the information I found here: https://github.com/danielpalme/ReportGenerator/issues/148

      It advised me to remove the slash making it simply dotnet reportgenerator…

      That corrected my issue.

      Thanks again,

      Peter

    • October 18, 2019 at 4:57 pm
      Permalink

      This has been an extremely helpful article, thanks! A couple of edit recommendations based on issues I encountered:

      1. Indicate that DotNetCliToolReference needs to be inside ItemGroup tags in the project file.
      2. Where you instruct us to put the command on one line, please bold that instruction. (Yes, I missed it both times!)
      3. Please remove the / from dotnet /reportgenerator.

      Thanks again!

      Laurie

    • October 23, 2019 at 6:34 pm
      Permalink

      Great stuff and great article, It works like charm. Very help full.,

      Thank you –Sridhar

    • February 28, 2020 at 6:06 pm
      Permalink

      “Code coverage on Linux”

      THANK YOU! This is what has been plaguing me as I went through dozens of examples that were assuming Windows without mentioning “this is for windows only”.

      (Our build images are Linux based).

      For those of you not using XUnit, you can just drop one of the loggers.

      sudo dotnet test –logger ‘trx;LogFileName=TestResults123.trx’ –results-directory ./BuildReports/UnitTests /p:CollectCoverage=true /p:CoverletOutput=BuildReports/Coverage/ /p:CoverletOutputFormat=cobertura

      and a copy of my (non Xunit) unittest.csproj packagereferences

      For those of you who have multiple UnitTest projects, you might find this SOF of interest:

      https://stackoverflow.com/questions/26469072/is-there-anyway-to-merge-cobertura-coverage-xml-reports-together/52412033#52412033

    • February 28, 2020 at 6:09 pm
      Permalink

      My xml did not post.

      Here it is again with substitutes for LeftBracket*** and ***RightBracket.

      LeftBracket***ItemGroup***RightBracket
      LeftBracket***PackageReference Include=”Microsoft.NET.Test.Sdk” Version=”16.5.0″ /***RightBracket
      LeftBracket***PackageReference Include=”MSTest.TestAdapter” Version=”1.4.0″ /***RightBracket
      LeftBracket***PackageReference Include=”MSTest.TestFramework” Version=”1.4.0″ /***RightBracket
      LeftBracket***PackageReference Include=”coverlet.msbuild” Version=”2.8.0″ /***RightBracket
      LeftBracket***PackageReference Include=”Microsoft.CodeCoverage” Version=”16.5.0″ /***RightBracket
      LeftBracket***PackageReference Include=”ReportGenerator” Version=”4.5.0″ /***RightBracket
      LeftBracket***/ItemGroup***RightBracket

    • April 22, 2020 at 2:14 pm
      Permalink

      Thanks for this post and it is very descriptive and easy to implement. Can you please help us to integrate this in jenkin job?

    • May 4, 2020 at 7:24 pm
      Permalink

      Sorry, I have no experiences with Jenkins.

    Leave a Reply

    Your email address will not be published. Required fields are marked *