Mark Oliver's World

Posted: 15/12/2022

Testing In Dotnet

Hello, and welcome to my blog post for December 15th of the .NET Advent Calendar 2022.

Organised by Dustin Morris Gorski on Twitter.

I wont tell you about me, but if you are interested, check out my About me page.

Testing in Dotnet

My view is that we don't talk enough about Testing in the Software Development world!

Meme showing Brad pitt in Fight club asking what the first rule of testing in dotnet is

I have been working in the industry for a long time, and have seen the benefit of testing, not just in creating stability in the software, but speed of development, confidence of the engineers, confidence of the stake holders, less downtime, less out of hours support required, the list goes on.

So I thought it would be a good idea to talk about how "I" do Testing with Dotnet.

I am talking about automated testing here in the general sense. I am not going to get into a flame war of the difference between unit tests, integration tests, E2E tests, It is nearly the holidays after all 😃

Bill & Ted saying be excellent to each other

I should say, that this is very much an opinion piece. However I hope you will find something useful within it, and it will give you the passion to start (or continue) on your testing journey in software development.

What tooling

I use Visual Studio 2022 Professional for dotnet development, running on a Windows machine, building with dotnet 6 (at time of writing).

But nothing I am about to say is specific to those things (unless stated). Everything I am about tell you is platform and development environment agnostic, after all Dotnet is very much multi platform these days and every test I write for this blog or the Company I currently work for runs on Linux but is developed in Windows.

What extensions

Within VS, I use the following extensions to help me write tests:

Fine Code Coverage

Created by Fortune Ngwenya

This extension (FCC) is really cool, it can independently show you code coverage, (or with the latest version use the inbuilt coverlet app in VS) to generate a table of code coverage and indicate in the code what paths you have missed.

Snapshot of the Fine Code Coverage output window

It can also highlight Risk Hotspots too, identifying complex areas of your code, that could be a focus for a refactoring effort.

I find this tool invaluable, so do check it out.

Please note, code coverage is a metric I find useful but not essential, it helps me discover tests I have not written, and in some cases, identify that a test is not following the path I would expect it to (useful when retrofitting tests to legacy code).

Create Unit Tests Command

(This used to be an extension, but since VS 2019, it is now built in)

This tool allows you to create unit test projects and blank test methods. I find it useful to quickly put the infrastructure in place to allow me to start testing.

screenshot of the Create Unit Tests dialog, showing all of its options to create a unit test and or project

However I do end up automatically updating the dotnet version & nuget packages as they are rarely current.

What it gives you then is a TestClass with a bunch of TestMethods. Each test method is intended to exercise each public method of the class it is creating unit tests of.

Unfortunately its usefulness is limited to that, I wish it did more, but "every little helps" to quote a supermarket.

What packages

This is a list of a useful nuget packages:

JustMock Lite

JustMock Lite is an great mocking tool, it gives you just enough functionality to be useful, but not too much that you don't notice the design flaws in your code. See below on "What can it mean if it is hard to write a test?"

System.IO.Abstractions

Combined with Dependency Injection & JustMock Lite, this allows us to abstract away things in System.IO, such as the FileSystem. Really handy so you dont have to read and write real data to the disk for your test.

DeepEqual

Allows you to compare the contents of objects with ease.

ObjectDumper.Net

I use this for lots of things, but it is great to add to a test to really see what an object looks like. A good companion with DeepEqual.

MockHttp

This is a new package for me, but BOY do I wish I knew about it sooner. It allows you to fake http responses.

Benny from Lego Movie saying Awesome

How to start testing

I am going to talk about 2 scenarios: working with code that already exists, and code that does not.

For code that exists already, we will focus on adding tests to the existing structure, so we are focussed on testing methods in classes.

For code that does not exist, then I encourage you to look up the technique of Test Driven Development (TDD).

This means you will write the test before the code exists, and often that can mean you are writing tests at a much higher level. e.g. testing of a feature, rather than a class.

Testing code that already exists

I will create a new unit test project called projectname.Tests (using the tool above)

This project will sit next to the project with the code I want to test.

I will focus on testing the public interface of the class, and try to drive the code through the paths of the code I want to test (likely because I want to change it).

If I know enough about the code, I will write the test before any code changes.
This test will prove the existing functionality works as we currently see it.

Then I will write a test that will fail because the code I need to write/change does not exist yet.

Now I have 2 tests, one to prove I have not broken anything unintentionally, and one to prove when I have made the change I need to do.

Note - This is a simplistic example, we may want to write many tests to prove it currently works, and many to prove it will work. Don't limit your testing unnecessarily. This is where FCC comes in handy, as it allows you to identify areas that are not currently exercised by tests, and tell us when we are not executing branches of our code.

As a side note, I often use tests to learn about what the existing code does - exploratory tests. These normally build up as I learn the code and its complexity, until I understand enough to write a test we can keep, or throw it away (but transfer that knowledge into another written form on the internal WIKI).

Testing code that does not exist

We still need to create a new unit project, but this may be named after a feature, rather than a code project, especially if you are going to try TDD.

However you still need a good name and location for the test project.
Then I would create a blank TestClass with a TestMethod to write the test I need to.

Method name conventions

There are lots of ways to name your test methods, I subscribe to this one from Roy Osherove

This has the method name laid out as so:

UnitOfWork_StateUnderTest_ExpectedBehavior

e.g.
MyAmazingMethod_InputParametersAreAllBadValues_FalseResult()

MyAmazingNewFeature_HappyPathOne_CustomerCompletesTransaction()

Read the article for the details, I don't want to make this article even longer.

The thing to remember though, is to make it clear what the test is trying to do (it is always about readability in software dev!), and be consistent across your code base.

Think about what your CI platform shows you on a test failure, are you getting enough information from it to diagnose the issue. Quite often all you get is the method that failed, so make your future selfs life easier, by giving it a helpful name now.

A picture of a future city with Thanks past me written on it

What is a SUT?

SUT or System Under Test is the term often used to indicate the focus of the test.

I always write it in full, never sut.

We are always talking about making our code more readable, and then we go and use "sut" in our tests. Pfft

This is the thing we will "Arrange" for, then "Act" on, and ultimately "Assert" about, which leads nicely on to...

AAA

No this is not about Car breakdowns, or having had one too many Sherries (is that a thing? I have always been Tee-Total!).

It is about the test writing pattern "Arrange, Act & Assert"

  • Arrange inputs and targets.
    • Arrange steps should set up the test case.
    • Think, "Does the test require any dependencies setup?" such as an in-memory database?
    • Handle all of these operations at the start of the test.
  • Act on the target behavior.
    • Act steps should cover the main thing to be tested.
    • This could be calling a method, calling a REST API etc.
    • There is likely to be only one line of code in this section of the test.
  • Assert expected outcomes.
    • Assert steps verify the output of the Act stage.
    • Sometimes it is checking an exception threw, or the correct value returned.
    • Other times you may need to interrogate the System Under Test (SUT) to determine if it did what you expected

Read more about it here from the awesome people at Telerik JustMock (No sponsorship provided, but I can hope for a xmas present right?).

How many asserts

I have been asked (and asked myself) how many "Asserts" should there be in a single test.

My preferred answer to this is: As many as needed to PROVE that your test has succeeded. This maybe simply checking the return result of the method call, but it could be as complex as checking multiple tables in a DB have been modified, and all files have been removed, and a REST API was hit....

More often the number of asserts you have indicate the level that your test is at:

  • One Assert ~= a single method call with a return result
  • Multiple Asserts ~= Testing a complex path

As always:
A picture of a dog in a suit with glowing eyes, saying "it depends!"

MStest, NUnit, XUnit

There are different packages you can use to write and run tests. I have only ever used MSTest with dotnet.

The folks over at BrowserStack have written a good article on the differences: https://www.browserstack.com/guide/nunit-vs-xunit-vs-mstest

As you can see, they all do similar things, so I would say "Dealers choice", use what you are comfortable with and that your company already uses.

If you have to start with nothing, then NUnit seems to be the most popular, and most examples of tests "on the line" tend to use NUnit.

Public vs private testing

I am a big proponent of public interface testing.
However, that assumes your interface is easy enough to use, and is not hiding a big bloated ball of mud.

If you have a ball of mud, I would suggest writing as many tests as you can to the interface that touches as much of the code as possible.

Then you can start refactoring the ball of mud into smaller balls of mud, using Dependency Injection to maintain the functionality.

By refactoring even small parts of the original code, it WILL make testing of the refactored code and the original code easier.

However if your big ball of mud is so big and mutant like, then perhaps private testing is for you. This will require a different way to run your tests, probably as derived classes.
There are ways through Reflection but I do NOT recommend that approach.

Take the refactoring route, it will help you in the long run, and it is likely to be less painful.

InternalsVisibleTo

But wait, all my classes are internal. I cannot access them from a .Tests project.

Firstly - Great!

Secondly - You have 3 options:

  • Make your internal classes public (I don't recommend)
  • Move your tests into the main code project (I don't recommend)
  • Allow your test project to see the internals of the project using InternalsVisibleTo

This allows the internal methods in an Assembly to be visible to another Assembly. In this case your .Tests assembly.

You can do this, by adding this to your project file. This will allow a .Tests project of the same name as the assembly generated to be able to access the internal classes of your project.

            
              <ItemGroup> 		<AssemblyAttribute Include="System.Runtime.CompilerServices.InternalsVisibleTo"> 			<_Parameter1>$(AssemblyName).Tests</_Parameter1> 		</AssemblyAttribute> 	</ItemGroup>

Note - You can define this AssemblyAttribute multiple times if you want to access the internals in multiple projects!

What can it mean if it is hard to write a test?

If you are struggling to write a unit test, normally this would indicate that your code is too complex.

It may need a lot of dependencies with significant setup, or a lot of parameters passed to a method to drive it to the line of code you want to execute.

This generally indicates you have a design issue, and it may be easier to do some refactoring to help with testing.

It is amazing how much testing can be easier when you pull a single encapsulated functionality out to a separate entity.

Not only can the entity be tested in isolation, but you then don't need to worry about testing its fine details in the code that uses it.

Test Constructors

As you have seen in MSTest and NUnit etc, you can do Setup and Tear Down of testing data/environments in specially tagged methods.

You can also use a constructor of your test class.

I generally try and stay away from any setup/tear down methods. It becomes easy to muddy one test with another's setup.

Also, I like my tests to stand alone, and be runnable in Parallel, and this is much clearer if there is no shared code.

However they are very useful if used carefully.

If you got this far through my first NET Advent Calendar article, then thanks a lot, I appreciate your time.

Come say Hi on Twitter, if you are interested in learning more, then here are some great things to research. I particularly love Mutation testing.

Additional reading

Happy holidays everyone

"A Cat in tinsel saying Happy holidays, dotnet family"

Thanks for reading this post.

If you want to reach out, catch me on Twitter!

I am always open to mentoring people, so get in touch.