Azure DevOps and the Code Coverage

Although Microsoft offers a solution to calculate natively the code coverage, it generates several problems: difference in results between the Build Server and “Visual Studio / Test / Analyze Code coverage”, code coverage problem with referenced assemblies, etc.

Azure DevOps

The following procedure has the advantage of being simple and easily verifiable locally. To set up unit tests, for a.NET Core project, in Azure DevOps, you must :

1. In your Visual Studio solution, add a unit test project (end of name with Tests).
2. Reference the Nuget package coverlet.msbuild which will be used as an extension of the dotnet test command used later.
3. In Azure DevOps Pipeline Build, add a task .NET Core to perform unit tests and collect data on code coverage (via coverlet). This task will generate a coverage.cobertura.xml file in your test project folder, in Cobertura format which is supported by Azure DevOps.
   • Commande : Test
   • Path to projet(s) : **/*[Tt]ests/*.csproj
   • Arguments : --configuration $(BuildConfiguration) /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura
   • Publish test results and code coverage : checked
4. Via the Marketplace, add the extension[ReportGenerator] (https://marketplace.visualstudio.com/items?itemName=Palmmedia.reportgenerator) (from Palmmedia) and this task in your Pipeline:
   • Reports : **/coverage.cobertura.xml
   • Target directory : CoverageReport
   • Report types: HtmlInline_AzurePipelines;Cobertura
   This task will generate an HTML report of the coverage code (supported by Azure), in the /CoverageReport folder.
5. Add a task Publish code coverage results to publish the HTML report in a Coverage tab of the Build Summary.
   • Code coverage tool : Cobertura
   • Summary file : **/coverage.cobertura.xml
   • Report dyrectory : CoverageReport
   

Results

Once the Build is executed, the Summary tab displays the statistics of unit tests and code coverage. The Code Coverage tab provides a complete report of code coverage, including uncovered code parts.

Locally

You can execute a Code Coverage locally, using the tool coverlet.console NET Core Global Tools is very useful to quickly provide an effective tool for developers. You must have .NET Core SDK and Nate McMaster has referenced several interesting ones on his GitHub page.

1. Install the tool: dotnet tool install --global coverlet.console
2. From your test project folder, run a Code Coverage: dotnet test /p:CollectCoverage=true

Sample:

$> dotnet test /p:CollectCoverage=true

Build started, please wait...
Build completed.

Test run for ...(.NETCoreApp,Version=v2.2)
Microsoft (R) Test Execution Command Line Tool Version 15.9.0
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...

Total tests: 10. Passed: 10. Failed: 0. Skipped: 0.
Test Run Successful.
Test execution time: 0.7909 Seconds

Calculating coverage result...
Generating report 'coverage.json'

+--------------------+-------+--------+--------+
| Module             | Line  | Branch | Method |
+--------------------+-------+--------+--------+
| Services.Data      | 78.1% | 66.1%  | 90.3%  |
+--------------------+-------+--------+--------+
| Services.Scrappers | 96.4% | 50%    | 100%   |
+--------------------+-------+--------+--------+
| ScrappersTests     | 89.8% | 92.3%  | 92.6%  |
+--------------------+-------+--------+--------+

+---------+-------------------+--------+-------------------+
|         | Line              | Branch | Method            |
+---------+-------------------+--------+-------------------+
| Total   | 85.7%             | 73.8%  | 92.2%             |
+---------+-------------------+--------+-------------------+
| Average | 28.5666666666667% | 24.6%  | 30.7333333333333% |
+---------+-------------------+--------+-------------------+

Minimum limit of coverage

It is common to require developers to achieve code coverage of at least 80%. To do this, you must choose your criteria and minimum threshold. For example, for at least 80% total coverage of the lines of code, these threshold parameters must be added (to the local command or Azure DevOps arguments).

dotnet test /p:CollectCoverage=true /p:threshold=80 /p:thresholdType=line /p:thresholdStat=total