Testing In Dotnet
Hello, and welcome to my blog post for December 15th of the .NET Advent Calendar 2022.
Organised by Dustin Morris Gorski on Twitter.
I wont tell you about me, but if you are interested, check out my About me page.
Testing in Dotnet
My view is that we don't talk enough about Testing in the Software Development world!
I have been working in the industry for a long time, and have seen the benefit of testing, not just in creating stability in the software, but speed of development, confidence of the engineers, confidence of the stake holders, less downtime, less out of hours support required, the list goes on.
So I thought it would be a good idea to talk about how "I" do Testing with Dotnet.
I am talking about automated testing here in the general sense. I am not going to get into a flame war of the difference between unit tests, integration tests, E2E tests, It is nearly the holidays after all 😃
I should say, that this is very much an opinion piece. However I hope you will find something useful within it, and it will give you the passion to start (or continue) on your testing journey in software development.
I use Visual Studio 2022 Professional for dotnet development, running on a Windows machine, building with dotnet 6 (at time of writing).
But nothing I am about to say is specific to those things (unless stated). Everything I am about tell you is platform and development environment agnostic, after all Dotnet is very much multi platform these days and every test I write for this blog or the Company I currently work for runs on Linux but is developed in Windows.
Within VS, I use the following extensions to help me write tests:
Created by Fortune Ngwenya
This extension (FCC) is really cool, it can independently show you code coverage, (or with the latest version use the inbuilt coverlet app in VS) to generate a table of code coverage and indicate in the code what paths you have missed.
It can also highlight Risk Hotspots too, identifying complex areas of your code, that could be a focus for a refactoring effort.
I find this tool invaluable, so do check it out.
Please note, code coverage is a metric I find useful but not essential, it helps me discover tests I have not written, and in some cases, identify that a test is not following the path I would expect it to (useful when retrofitting tests to legacy code).
(This used to be an extension, but since VS 2019, it is now built in)
This tool allows you to create unit test projects and blank test methods. I find it useful to quickly put the infrastructure in place to allow me to start testing.
However I do end up automatically updating the dotnet version & nuget packages as they are rarely current.
What it gives you then is a
with a bunch of
TestMethods. Each test method is intended to exercise each public method of the class it is creating unit tests of.
Unfortunately its usefulness is limited to that, I wish it did more, but "every little helps" to quote a supermarket.
This is a list of a useful nuget packages:
JustMock Lite is an great mocking tool, it gives you just enough functionality to be useful, but not too much that you don't notice the design flaws in your code. See below on "What can it mean if it is hard to write a test?"
Combined with Dependency Injection & JustMock Lite, this allows us to abstract away things in
System.IO, such as the
FileSystem. Really handy so you dont have to read and write real data to the disk for your test.
Allows you to compare the contents of objects with ease.
I use this for lots of things, but it is great to add to a test to really see what an object looks like. A good companion with DeepEqual.
This is a new package for me, but BOY do I wish I knew about it sooner. It allows you to fake http responses.
How to start testing
I am going to talk about 2 scenarios: working with code that already exists, and code that does not.
For code that exists already, we will focus on adding tests to the existing structure, so we are focussed on testing methods in classes.
For code that does not exist, then I encourage you to look up the technique of Test Driven Development (TDD).
This means you will write the test before the code exists, and often that can mean you are writing tests at a much higher level. e.g. testing of a feature, rather than a class.
Testing code that already exists
I will create a new unit test project called
(using the tool above)
This project will sit next to the project with the code I want to test.
I will focus on testing the public interface of the class, and try to drive the code through the paths of the code I want to test (likely because I want to change it).
If I know enough about the code, I will write the test before any code changes.
This test will prove the existing functionality works as we currently see it.
Then I will write a test that will fail because the code I need to write/change does not exist yet.
Now I have 2 tests, one to prove I have not broken anything unintentionally, and one to prove when I have made the change I need to do.
Note - This is a simplistic example, we may want to write many tests to prove it currently works, and many to prove it will work. Don't limit your testing unnecessarily. This is where FCC comes in handy, as it allows you to identify areas that are not currently exercised by tests, and tell us when we are not executing branches of our code.
As a side note, I often use tests to learn about what the existing code does - exploratory tests. These normally build up as I learn the code and its complexity, until I understand enough to write a test we can keep, or throw it away (but transfer that knowledge into another written form on the internal WIKI).
Testing code that does not exist
We still need to create a new unit project, but this may be named after a feature, rather than a code project, especially if you are going to try TDD.
However you still need a good name and location for the test project.
Then I would create a blank
to write the test I need to.
Method name conventions
This has the method name laid out as so:
Read the article for the details, I don't want to make this article even longer.
The thing to remember though, is to make it clear what the test is trying to do (it is always about readability in software dev!), and be consistent across your code base.
Think about what your CI platform shows you on a test failure, are you getting enough information from it to diagnose the issue. Quite often all you get is the method that failed, so make your future selfs life easier, by giving it a helpful name now.
What is a SUT?
System Under Test
is the term often used to indicate the focus of the test.
I always write it in full, never
We are always talking about making our code more readable, and then we go and use "sut" in our tests. Pfft
This is the thing we will "Arrange" for, then "Act" on, and ultimately "Assert" about, which leads nicely on to...
No this is not about Car breakdowns, or having had one too many Sherries (is that a thing? I have always been Tee-Total!).
It is about the test writing pattern "Arrange, Act & Assert"
Arrangeinputs and targets.
- Arrange steps should set up the test case.
- Think, "Does the test require any dependencies setup?" such as an in-memory database?
- Handle all of these operations at the start of the test.
Acton the target behavior.
- Act steps should cover the main thing to be tested.
- This could be calling a method, calling a REST API etc.
- There is likely to be only one line of code in this section of the test.
- Assert steps verify the output of the Act stage.
- Sometimes it is checking an exception threw, or the correct value returned.
- Other times you may need to interrogate the System Under Test (SUT) to determine if it did what you expected
Read more about it here from the awesome people at Telerik JustMock (No sponsorship provided, but I can hope for a xmas present right?).
How many asserts
I have been asked (and asked myself) how many "Asserts" should there be in a single test.
My preferred answer to this is: As many as needed to PROVE that your test has succeeded. This maybe simply checking the return result of the method call, but it could be as complex as checking multiple tables in a DB have been modified, and all files have been removed, and a REST API was hit....
More often the number of asserts you have indicate the level that your test is at:
- One Assert ~= a single method call with a return result
- Multiple Asserts ~= Testing a complex path
MStest, NUnit, XUnit
There are different packages you can use to write and run tests. I have only ever used MSTest with dotnet.
The folks over at BrowserStack have written a good article on the differences: https://www.browserstack.com/guide/nunit-vs-xunit-vs-mstest
As you can see, they all do similar things, so I would say "Dealers choice", use what you are comfortable with and that your company already uses.
If you have to start with nothing, then NUnit seems to be the most popular, and most examples of tests "on the line" tend to use NUnit.
Public vs private testing
I am a big proponent of public interface testing.
However, that assumes your interface is easy enough to use, and is not hiding a big bloated ball of mud.
If you have a ball of mud, I would suggest writing as many tests as you can to the interface that touches as much of the code as possible.
Then you can start refactoring the ball of mud into smaller balls of mud, using Dependency Injection to maintain the functionality.
By refactoring even small parts of the original code, it WILL make testing of the refactored code and the original code easier.
However if your big ball of mud is so big and mutant like, then perhaps private testing is for you. This will require a different way to run your tests, probably as derived classes.
There are ways through
but I do NOT recommend that approach.
Take the refactoring route, it will help you in the long run, and it is likely to be less painful.
But wait, all my classes are
internal. I cannot access them from a
Firstly - Great!
Secondly - You have 3 options:
- Make your internal classes public (I dont recommend)
- Move your tests into the main code project (I dont recommend)
- Allow your test project to see the internals of the project using InternalsVisibleTo
This allows the
internalmethods in an Assembly to be visible to another Assembly. In this case your
You can do this, by adding this to your project file. This will allow a
project of the same name as the assembly generated to be able to access the internal classes of your project.
<ItemGroup> <AssemblyAttribute Include="System.Runtime.CompilerServices.InternalsVisibleTo"> <_Parameter1>$(AssemblyName).Tests</_Parameter1> </AssemblyAttribute> </ItemGroup>
What can it mean if it is hard to write a test?
If you are struggling to write a unit test, normally this would indicate that your code is too complex.
It may need a lot of dependencies with significant setup, or a lot of parameters passed to a method to drive it to the line of code you want to execute.
This generally indicates you have a design issue, and it may be easier to do some refactoring to help with testing.
It is amazing how much testing can be easier when you pull a single encapsulated functionality out to a separate entity.
Not only can the entity be tested in isolation, but you then don't need to worry about testing its fine details in the code that uses it.
As you have seen in MSTest and NUnit etc, you can do
of testing data/environments in specially tagged methods.
You can also use a constructor of your test class.
I generally try and stay away from any setup/tear down methods. It becomes easy to muddy one test with another's setup.
Also, I like my tests to stand alone, and be runnable in Parallel, and this is much clearer if there is no shared code.
However they are very useful if used carefully.
If you got this far through my first NET Advent Calendar article, then thanks a lot, I appreciate your time.
Come say Hi on Twitter, if you are interested in learning more, then here are some great things to research. I particularly love Mutation testing.
- Unit Testing Best Practices
- Mutation testing with Stryker
- Fluent Assertions
- Playwright & Selenium
- Snapshot Testing
- Prevent http requests to external services in unit tests
Happy holidays everyone
Thanks for reading this post.
If you want to reach out, catch me on Twitter!