Code coverage reports for ASP.NET Core

Code coverage reports for ASP.NET Core projects are not provided out-of-box but using right tools we can build decent code coverage reports. I needed code coverage reports in some of my projects and here is how I made things work using different free libraries and packages.

Getting started

To get started we need test project and some NuGet packages. Test project can be regular .NET Core library project. Add reference to web application project and write some unit tests if you start with new test project. We also need some NuGet packages to make things work:

• coverlet.msbuild
• Microsoft.CodeCoverage
• Microsoft.NET.Test.Sdk
• ReportGenerator by Daniel Palme
• xunit
• xunit.runner.visualstudio
• XunitXml.TestLogger

NB! The most painless way to get ReportGenerator work was using local tool. Other options failed for me as ReportGenerator works with .NET Core 3.0 projects but not with .NET Core 3.1. I went with external tool because reports are then generated without any dependencies to project binaries. Here’s the command line to install ReportGenerator to tools folder (open command line and move to unit tests project folder):

dotnet tool install dotnet-reportgenerator-globaltool --tool-path tools

After adding these packages it’s time to make test build and see if everything still works and we don’t have any build issues.

Creating reporting folders

Now it’s time to configure reporting. I decided to keep reports in BuildReports folder of test project. There are two subfolders:

• Coverage – for coverage reports (this blog post)
• UnitTests – unit tests reports (for future use)

I added BuildReports folder also to .gitignore file because I don’t want these files to wander from one developer box to another and be part of commits.

The number of files in Coverage folder is not small. It’s not just two or three files that are easy to ignore. There can be hundreds or thousands of files depending on how many tests there are in test projects. Here we are going in smaller scale of course.

Getting code coverage data

To generate reports we need coverage data and this is why we added coverlet.msbuild package to test project. When tests are run we gather code coverage information and publish it in Cobertura output format. Cobertura is popular code coverage utility in Java world. Test data is transformed to Cobertura format by Coverlet – a cross platform code coverage library for .NET Core.

With coverage data I also output unit test results in Microsoft and xUnit formats to UnitTests folder. As I said before this is for future use and we don’t do anything with files in these folders right now.

I added run-tests.bat file to root folder of my test project and first command there is for running unit tests (in your file put it all to one line without any line breaks).

dotnet test --logger "trx;LogFileName=TestResults.trx" ^
            --logger "xunit;LogFileName=TestResults.xml" ^
            --results-directory ./BuildReports/UnitTests ^
            /p:CollectCoverage=true ^
            /p:CoverletOutput=BuildReports\Coverage\ ^
            /p:CoverletOutputFormat=cobertura ^
            /p:Exclude="[xunit.*]*

This is what this command does:

1. Using Visual Studio logger create TestResults.trx file for test results
2. Using xUnit logger create TestResults.xml file for test results
3. Put test results to ./BuildReports/UnitTests folder
4. Enable collecting of code coverage data
5. Make Coverlet to use BuildReports\Coverage folder
6. Set Coverlet output format to Cobertura
7. Excude xUnit libraries from test results

As a result of this command we will have three files:

1. BuildReports\UnitTests\TestResults.trx (not important now)
2. BuildReports\UnitTests\TestResults.xml (not important now)
3. coverage.cobertura.xml (coverage results in Cobertura format)

BuildReports folder should not go to source code repository! If you are using git then you should add new rule to .gitignore file to avoid BuildReports folder getting to source code repository. Add this rule to .gitignore file of solution: [Bb]uildReports/

Now it’s time to try out run-tests.bat to see if everything still works and if files generated to expected locations.

Generating code coverage reports

For code coverage reports we need to add another command to run-tests.bat file. This command will run report generator that generater reports based on coverage.cobertura.xml. The reports are generated to same folder to keep folder tree smaller.

Here is the command (put it all on one line):

tools\reportgenerator.exe ^
       "-reports:BuildReports\Coverage\coverage.cobertura.xml" ^
       "-targetdir:BuildReports\Coverage" ^
       -reporttypes:HTML;HTMLSummary

I think this command is not very cryptic and I don’t make additional comments on command line parameters here.

As a lazy guy I expect browser to open with newly generated reports automatically and this is why the last line of my run-tests.bat is:

start BuildReports\Coverage\index.htm

It’s time to run the script and see if it runs successfully to end.

Code coverage on Linux

On Linux we need shell script to run tests and generate reports. Here’s sample script for linux (every command goes to one line):

#!/bin/sh

sudo dotnet test --logger 'trx;LogFileName=TestResults.trx' 
                 --logger 'xunit;LogFileName=TestResults.xml' 
                 --results-directory ./BuildReports/UnitTests 
                 /p:CollectCoverage=true 
                 /p:CoverletOutput=BuildReports/Coverage/ 
                 /p:CoverletOutputFormat=cobertura 
                 /p:Exclude='[xunit.*]*'

sudo dotnet reportgenerator 
                 -reports:BuildReports/Coverage/coverage.cobertura.xml 
                 -targetdir:BuildReports/Coverage 
                 -reporttypes:"HTML;HTMLSummary"

Not sure why tooling needs sudo but this is what is asked for. Shell script may also need execute permissions. Here’s the command for this:

chmod +x run-tests.sh

Now we are good to go on Linux too.

Code coverage reports

After running the batch file in my playground test project folder I see the following report in browser.

The report is longer than we can see here but I’m still not very happy with it. I would like have better structural view of tested code so I have better overview about how well different system areas are covered with tests. Let’s take a look at this Grouping slider above tests table and try to move it. Voila!

To see over-all view of system under test I can click and close those bold namespaces. Now we see how much one or another namespace is covered.

We can also go inside classes and see coverage statistics about specific classes. Nice thing is we get also method bases statistics and source code view shows us what lines in class are covered with tests and what lines are not covered.

I think this kind of code coverage reporting is good enough for me.

Wrapping up

The path to code coverage reporting is not always easy but I got it work like expected. All tools I used are free and no hidden expenses besides my own time didn’t applied. We had to write batch file to run tests, collect code coverage data and generate reports. In the end we got decent reports giving us good overview of code coverage of our codebase.