ASP.NET Core introduced the concept of TestServer for integration testing of web applications. Integration tests need web application run with all bells and whistles to make sure that all components work together with no flaws. Often we need special settings for integration tests as web application cannot use live services and easiest way to do it is to use special appsettings.json file. This blog post shows how to do it.
Getting started
Let’s start with minimalistic integration test from ASP.NET Core integration tests document.
public class HomeControllerTests : IClassFixture<WebApplicationFactory<Startup>>
{
private readonly WebApplicationFactory<Startup> _factory;
public HomeControllerTests(WebApplicationFactory<Startup> factory)
{
_factory = factory;
}
[Theory]
[InlineData("/")]
public async Task Get_EndpointsReturnSuccessAndCorrectContentType(string url)
{
// Arrange
var client = _factory.CreateClient();
// Act
var response = await client.GetAsync(url);
// Assert
response.EnsureSuccessStatusCode(); // Status Code 200-299
Assert.Equal("text/html; charset=utf-8", response.Content.Headers.ContentType.ToString());
}
}
This test verifies that all URL-s given in inline data end with response code in range from 200-299 and with content type text/html. This test doesn’t validate what user is seeing on page. It just makes sure that request doesn’t fail. All logic in tested controller actions is tested by unit tests.
Adding custom appsettings.json
Integration tests expect environment with all external services needed by our application. As we don’t run tests against live environment we have to set up special set of services used by integration tests. We can run these tests on some continuous integration (CI) server or service like Azure DevOps. With separate set of services comes the need for separate configuration file.
Add new appsettings.json file to integration tests project and make Visual Studio copy it to output directory with every build.
For this example I’m using SQL Server LocalDB and here’s my appsettings.json.
{
"ConnectionStrings": {
"DefaultConnection": "Server=(localdb)\\MSSQLLocalDB;Database=MediaGalleryTests;Trusted_Connection=True;MultipleActiveResultSets=true"
}
} NB! Before appsettings.json of integration tests project the one from web application is loaded. Make sure you override all environment specific settings in appsettings.json of integration tests to make sure that references to live databases and services are not available for integration tests.
Including custom appsettings.json
We have to tweak our test class to make it include correct appsettings.json. As TestServer is run in output folder of integration tests project we just have to configure web host and load appsettings.json from there.
public class HomeControllerTests : IClassFixture<WebApplicationFactory<Startup>>
{
private readonly WebApplicationFactory<Startup> _factory;
public HomeControllerTests(WebApplicationFactory<Startup> factory)
{
var projectDir = Directory.GetCurrentDirectory();
var configPath = Path.Combine(projectDir, "appsettings.json");
_factory = factory.WithWebHostBuilder(builder =>
{
builder.ConfigureAppConfiguration((context,conf) =>
{
conf.AddJsonFile(configPath);
});
});
}
[Theory]
[InlineData("/")]
public async Task Get_EndpointsReturnSuccessAndCorrectContentType(string url)
{
// Arrange
var client = _factory.CreateClient();
// Act
var response = await client.GetAsync(url);
// Assert
response.EnsureSuccessStatusCode(); // Status Code 200-299
Assert.Equal("text/html; charset=utf-8", response.Content.Headers.ContentType.ToString());
}
}
When I run integration test above in debug mode and put breakpoint in Startup class to line next of connection string I can see that database context will use connection string from appsettings.json in integration tests project.
Wrapping up
The mechanism of ASP.NET Core integration tests makes it easy to tweak and extend web host used to run web application with integration tests. As we use separate set of external services and databases with integration tests we have to make web application use configuration file that overrides default settings. We added custom appsettings.json to integration tests project and configured web host builder to load this settings file when web application is started for tests. The only danger here is to remember that we need to override all settings related to external databases and services as appsettings.json of web application is loaded before the one for integration tests.
View Comments (20)
Nice article. You could also use a test startup class in WebApplicationFactory and in that include the test connection string.
Thanks. Custom startup class for integration tests is one of the next topics. I need to play with some features to get most out of it and see how things finally look.
And here we go: https://gunnarpeipman.com/testing/aspnet-core-integration-tests-startup/
Thank you.
> You could also use a test startup class
Not trying to play devils advocate here that approach alters the whole SUT.
Ideally, we need to be able to edit the configuration inside the custom WebApplicationFactory and not just bypass configuration entirely using a custom appsettings.
Hi,
I have a question, in some blog posts also Microsoft integration test document for .net core, used an in-memory database for integration test. what is the best solution? is an in-memory database really suitable for integration tests?
For integration tests I suggest you to use same type of database that system will use in production.
The idea of integration tests is to test how system components work together. If you replace one database engine with another then integration tests tell you nothing about database.
Nice :) You can also put that code in your custom web application factory, rather than in the constructor of each test (in the ConfigureWebHost method, as it takes the builder as a parameter so you have access to it there)
Yes, it's also one to do it. I have some experiments on integration tests to finish and then it's time to go through code on GitHub to see what will be the final shortcuts. I'm thinking about some additional extension methods to configure web application factory.
I am trying to run integration test on CI pipeline where all the required microservices and DB are running in docker containers.
When I am executing the test from Out folder I am getting this error:
dh.Media.CMP.WebApi.IntegrationTests.Tests.Audience.AudienceTests.Create_audience_with_duplicate_name [1ms]
Error Message:
System.AggregateException : One or more errors occurred. (Network is unreachable)
---- System.Net.Http.HttpRequestException : Network is unreachable
-------- System.Net.Sockets.SocketException : Network is unreachable
Stack Trace:
at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
at System.Threading.Tasks.Task`1.get_Result()
at dh.Media.CMP.Configuration.ConfigClient.ServiceReader.Get[T]()
at dh.Media.CMP.Apps.WebApi.Startup.ConfigureServices(IServiceCollection services) in /app/dh.Media.CMP.Apps/dh.Media.CMP.Apps.WebApi/Startup.cs:line 84
at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor, Boolean wrapExceptions)
Can you advise?
I have not much idea what is going on. Seems to be networking issue (socket exception is mentioned). I think it's issue with docker network settings.
Hi,
In each Test we need a clean state, how do you clean up a real test database after each test execution?
with the use of the in-memory database, we can destroy it after each test run, but what is your suggestion for real test database?
For integration tests you need real database and not some replacement that is fast enough or easy to handle.
There are multiple approaches to consider:
1. Delete database, create database, add seed data
2. Use transactions and rollbacks
3. Use utilities to track changes and undo this after test is run
Each of these approaches has its own pros and cons. I prefer the first approach although it can be slowest one. It's easy to implement as EF Core has methods for all steps mentioned.
Ok, Thanks Gunnar.
Gunnar, I am facing some issues because of my Db COntext is added only once for all test suite. I have overriden the AddDbContext method in startup class within my TestStartup that is inherited from Startup from web api project.
Some of the tests if run in parallel are giving me back ID field from previous ran test, i suspect because of shared Context between tests. How can I setup a new DB per test within a class Fixture ? Problem is that AddDbContext happens only once at test startup and then i am just using the integration tests, which means I can not really do the using context = new MyContext, as I just hit the controller end point and the code for unit of work is deep down somewhere in repositories.
Thank you
will this just be fine if i do it in my fixture constructor that takes in a custom web application factory ?
var dbContext = scope.ServiceProvider.GetService();
dbContext.Database.EnsureDeleted();
dbContext.Database.EnsureCreated();
Don't run integration tests in parallel if you don't guarantee separate database for every thread. If database is the same for all running tests then you will run to weird results (some tests randomly failing). So, to run tests in parallel you need to figure out how to guarantee different database name every time database is created. Of course, those databases must be later deleted too. You can make test class implement IDisposable and in Dispose() method you can delete database.
It doesn't matter much where you create your database if tests run sequentially. If tests run sequentially then there is always new database instance for each test without any changes in current approach and current code.
Thank you. It is exactly what I was looking for customize my project. ¡Gracias!
When you are calling conf.AddJsonFile(configPath); I have two questions.
1) I am doing that in a custom WebApplicationFactory class and I don't need to use CurrentDirectory. Both the SUT and the test projects appsettings.json files are loaded correctly (as they both seem to be copied to output directory of the test project).
2) That call you make is only 'appending' configuration right? If I wanted to completely replace the appsettings.json file used in the SUT, this doesn't work does it? If it does, I'm doing something wrong. I want to completely replace the settings in the web api so BOTH the test and web api projects use the same (test) settings, but I can't figure out how to do it.
This is not going to work because appsettings.json is copied from the web project, so in order for this method to work you need to put your integration tests appsettings.json in another folder, otherwise it will be overridden.