aspnet-basics/8-testing.md

Testing

---

• What is testing?
  • Proving that the program is working as intended during development on a
    • micro-level: individual pieces of code, such as methods
    • macro-level: larger functional components, such as services
  • Automated testing & manual testing
  • All possible paths in a piece of code should be tested
• Why do automated testing?
  • Sets ground truths that can be relied on during development: "With this kind of input, this should happen"
    • No need to take the programmers word for it
  • Ensures code functionality before moving on, "finding bugs before debugging"
  • Guides developers to create more testable and thus more readable, interchangeable and overall better code

Automated Testing

• Unit testing
  • Testing individual units (methods, classes) within a project
  • "input x produces output y"
  • Each unit has to be entirely independent and deterministic for it to be unit testable!
    • Deterministic = Always the same result with the same input
• Integration Testing
  • Testing complete modules and the interaction between them within a project
  • Closer to real life scenarios

C# Testing Frameworks

• Three main testing frameworks:
  • MSTest
  • NUnit
  • XUnit
• You can install these using NuGet, or in the newer versions of Visual Studio you can create a new testing project
• NUnit is the most popular of the three, and will be used in this lecture

MSTest

• Built-in to most versions of Visual Studio since 2005
• Simple to use, easy structure
  • TestClass for denoting classes containing unit tests
  • TestMethod for denoting unit test methods

NUnit

• Most popular C# unit testing framework
• Very similar structure to MSTest tests
  • Instead of TestClass, we use TestFixture
  • Instead of TestMethod, we use Test / TestCase
• Has some advantages to MSTest or xUnit tests, for example:
  • Ability to test a single test method using multiple different arguments

XUnit

Free, open-source unit testing tool for .NET framework and .NET core

Made by the creator of NUnit v2

You can test C#, F#, VB.NET and other .NET languages

Part of the .NET Foundation

Unique and different style

Test Case Naming Conventions

In C# unit testing, the most commonly followed naming convention for test cases/methods is as follows:

FunctionNameThatIsTested_Condition_ShouldDoX

For example:

Fibonacci_WhenNIs0_ShouldReturn0

Solution & Sample Project

First, we need a solution and a sample project.

In Visual Studio, create a class library that targets .NET Standard .

Call the project "Fibonacci.Library", and solution "Fibonacci". Remember to uncheck " Place solution and project in the same directory " if it isn't unchecked already.

Solution & Sample Project (continued)

Next, rename the default "Class1.cs" file to "Fibonacci.cs".

Allow Visual Studio to also rename the class itself to Fibonacci.

Make the class public and static.

Now, let's __ __ add a recursive fibonacci function, as shown in the next slide:

using System;

namespace Fibonacci.Library {

public static class Fibonacci {

public static long Recursive(

int n,

long previous = 0,

long current = 0,

int counter = 0

) {

return counter == n ?

previous + current :

Recursive(n, current, Math.Max(previous + current, 1), counter + 1);

}

}

}

Adding a Testing Project

Now we need to add a testing project to our solution.

Right click your solution, and add a new project. Select the NUnit project for .NET Core , and call it "Fibonacci.UnitTests".

Rename the created class file to "Fibonacci_Tests.cs", and the class to "Fibonacci_Tests"

Adding a Testing Project (continued)

We then need to link the Fibonacci.Library project to Fibonacci.UnitTests.

Right click Fibonacci.UnitTests, and select Add > Reference…, and under "Projects" select Fibonacci.Library. Press OK.

Now we have linked Fibonacci.Library to Fibonacci.UnitTests, and we can begin testing our Recursive-function in the Fibonacci-class.

Test Fixture For a Class

Since we have many things called "Fibonacci", we have to import our Fibonacci-class by using an alias:

using FibonacciLib = Fibonacci.Library.Fibonacci;

Put this to the top of your Fibonacci_Tests.cs file, underneath using NUnit.Framework;.

Test Fixture For a Class (continued)

Now, add [TestFixture] on top of our class, like so:

using NUnit.Framework;

using FibonacciLib = Fibonacci.Library.Fibonacci;

namespace Fibonacci.UnitTests {

[TestFixture]

public class Fibonacci_Tests {

This denotes that this class contains our unit tests.

Then, inside our class Fibonacci_Tests, let's create our first method to test the Recursive-function, as shown in the next slide:

using NUnit.Framework;

using FibonacciLib = Fibonacci.Library.Fibonacci;

namespace Fibonacci.UnitTests {

[TestFixture]

public class Fibonacci_Tests {

[Test]

public void Recursive_WhenNIs0_ShouldReturn0()

=> Assert.That(0, Is.EqualTo(FibonacciLib.Recursive(n: 0)));

}

}

• Things to take into consideration:
• When passing in function arguments, to avoid magic numbers and to improve code readability, use the syntax argumentName: value
  • Magic numbers = using numbers as input as-is without explanation, for example without assigning to a well-named variable
• Assert.That is a function (but not the only function) that determines whether the test passes.
  • Its first argument is the expected value. In the second argument you give a constraint for the actual output, i.e. value is equal to, greater than, starts with, and many others .
  • Do not mix the arguments up! If you do, the errors you get do not make any sense.

Running Tests

Now, let's open the built-in test runner in Visual Studio to run our tests.

On the top of Visual Studio, click "Test", then "Test Explorer".

In the test explorer, you should see the test case we created. If not, build the solution, and make sure there are no errors.

Either way, let's run our test case. Press "Run All Tests" in the top-left corner of test explorer. You should see our test case passing .

Adding More Test Cases

While there's nothing inherently wrong about our test case, it doesn't cover much of anything. We need to add more test cases to test different values and outputs.

In other testing frameworks, you would do this by creating more methods.

But with NUnit, we can use our existing method.

Adding More Test Cases (continued)

Do the following modifications to our test method:

[TestCase(0, 0)]

public void Recursive_Always_ShouldReturnNthFibonacci(

int n,

long expected

)

=> Assert.That(expected, Is.EqualTo(FibonacciLib.Recursive(n)));

Using TestCase instead of Test, we can pass in arguments to our test case method, and we can use those passed in arguments in our actual tests.

Run the test cases again, and they should pass .

But as you might notice, we haven't really changed anything yet.

Let's add more test cases, like so:

[TestCase(0, 0)]

[TestCase(1, 1)]

public void Recursive_Always_ShouldReturnNthFibonacci(

Now we have 2 test cases with only one test method. Run the tests again, and both cases should pass .

Let's add one more TestCase, like so:

[TestCase(2, 1)]

Run the tests again, and you should see the test case fail .

This is to be expected: the 2nd fibonacci number should be 1, but the actual value we got from our function was 2.

This means our function does not work as expected.

Let's fix it, as shown in the next slide:

public static long Recursive(

int n,

long previous = 0,

long current = 0,

int counter = 0

) {

return counter == n ?

current : // changed from previous + current

Recursive(n, current, Math.Max(previous + current, 1), counter + 1);

}

Run the tests again. You should see all __pass __ now.

Add a few more test cases; if you don't know the fibonacci sequence, you can check it from google.

Exercise 1: Unit testing in practice

• Answer the following questions:
• How many test cases should we ideally have to test our Recursive-function?
• What happens if we pass in n = int.MaxValue (2147483647)? Or other huge numbers?
  • How would you fix the function so it works as expected?
• What happens if we pass in n = -1 ? Or other negative numbers?
  • How would you fix the function so it works as expected?
• (Answers on the next slide)

Unit testing in practice: Exercise - Answers

• Answers:
• We should cover all the edge cases and code paths with our test cases. We do not need to have many test cases for the same code path.
• Stack Overflow, Int64 Overflow, or the test case just won't run.
  • We should have a maximum allowed value for n, throw argument out of range exception if above
• Stack Overflow, since current never can equal n that is less than 0
  • We should have a minimum allowed value for n, throw argument out of range exception if below

Testing Exceptions

To expect exceptions to be thrown in test cases, we can use the method

Assert.Throws, syntax is as follows:

Assert.Throws(delegate);

In practice:

Assert.Throws(() => TestedFunction(n));

Handling Border Cases

We need to add handling for n that is less than 0 or more than a set amount. Then we need to add another test method and two test cases to test our new functionality.

Let's start with modifying our existing Recursive-function, as shown in the next slide:

Handling Border Cases (continued)

public const int MinDepth = 0;

public const int MaxDepth = 10000;

public static long Recursive(

int n,

long previous = 0,

long current = 0,

int counter = 0

) {

if (n < MinDepth || n > MaxDepth)

throw new ArgumentOutOfRangeException("n is out of bounds!");

return counter == n ?

current :

Recursive(n, current, Math.Max(previous + current, 1), counter + 1);

}

Now our Recursive-function throws an ArgumentOutOfRange exception when n is less than 0 or more than 10000. Let's test that everything works as expected, by adding a new test method and a few test cases:

[TestCase(int.MaxValue)]

[TestCase(-1)]

public void Recursive_WhenNIsOutOfBounds_ShouldThrowArgumentOutOfRange(

int n

)

=> Assert.Throws(() => FibonacciLib.Recursive(n));

Rename the other method to Recursive_WhenNIsInBounds_ShouldReturnNthFibonacci

Removing Unnecessary Arguments

We have one more problem:

Our Recursive-function is a public method that has in total 4 arguments it takes, yet we only test 1 argument (n).

In order for our tests to be complete, we need to test all the arguments a public method takes in.

So, how do we fix this?

Removing Unnecessary Arguments (continued)

The naive way would be to simply test all the arguments with more test cases.

But it's important to realize that we don't have to . Since we basically never pass in any other arguments than n from outside Recursive, we could just create another __private __ function that has those arguments, and we could remove them from our __public __ Recursive-function.

Like shown in the next slide:

public static long Recursive(int nStartingValue) {

if (nStartingValue < MinDepth || nStartingValue> MaxDepth)

throw new ArgumentOutOfRangeException("n is out of bounds!");

long getRecursive(

int n,

long previous = 0,

long current = 0,

int counter = 0)

=> counter == n ?

current :

getRecursive(n, current, Math.Max(previous + current, 1), counter + 1);

return getRecursive(nStartingValue);

}

End Result

Now we have tested all the arguments in our public Recursive function.

We do not need to test private functions such as getRecursive, only the public API has to be tested.

All tests should __pass __ by this point.

Exercise 2: Unit Testing

Following the previously used architecture as closely as possible, unit test the following function using NUnit:

Questions on the next slide.

public static long DoX(int nStartingValue) {

if (nStartingValue < 0 || nStartingValue > 150)

throw new ArgumentOutOfRangeException("n is out of bounds!");

long doY(int n) => n > 0 ? n * doY(n - 1) : 1;

return doY(nStartingValue);

}

Unit Testing: Exercise - Questions

Answer the following questions:

What does the function that we are testing actually do?

Do we have to test doY as well?

How would you name DoX so it is descriptive? How about doY?

Unit Testing: Exercise - Answers

• It gets the nth factorial (n!)
• doY is a private method that gets tested when we test DoX, so no.
• DoX = TryGetFactorial, doY = getFactorial
  • Name your functions to these

Unit Testing - Final Testing Code Part 1

[TestCase(0, 1)]

[TestCase(1, 1)]

[TestCase(2, 2)]

[TestCase(3, 6)]

[TestCase(15, 1307674368000)]

public void TryGetFactorial_WhenNIsInBounds_ShouldReturnNthFactorial(

int n, long expected

)

=> Assert.That(expected, Is.EqualTo(TestExerciseLib.TryGetFactorial(n)));

[TestCase(-1)]

[TestCase(151)]

public void TryGetFactorial_NIsOutOfBounds_ShouldThrowArgumentOutOfRange(

int n

)

=> Assert.Throws(

() => TestExerciseLib.TryGetFactorial(n)

);

Writing Testable Code

• Writing testable code equals writing good code
• Testable units should
  • be deterministic (functional): same input always produces the same output
  • be loosely coupled: components have a loose and limited connection to other components in the system, meaning changes in one component should not require changes to many others
  • abide to the _Single Responsibility Principle: _ a function, method or component is only responsible for one purpose

---

https://en.wikipedia.org/wiki/Loose_coupling

Writing Testable Code - Example 1

public static class Season

{

public static string GetCurrentSeason()

{

DateTime currentTime = DateTime.Now;

if (currentTime.Month > 11 && currentTime.Month < 3)

return "Winter";

else if (currentTime.Month < 6)

return "Spring";

else if (currentTime.Month < 9)

return "Summer";

else

return "Fall";

}

}

Can GetCurrentSeason() be unit tested?

Why / why not?

How to fix the problems, if any?

Writing Testable Code - Example 1 (continued)

public static class Season

{

public static string GetCurrentSeason(

DateTime currentTime

)

{

if (currentTime.Month > 11 && currentTime.Month < 3)

return "Winter";

else if (currentTime.Month < 6)

return "Spring";

else if (currentTime.Month < 9)

return "Summer";

else

return "Fall";

}

}

• Now the method is purely functional and deterministic
  • Will return the same value for every call with equal arguments
• GetCurrentSeason() can now easily be tested as shown in the next slide

public class GetCurrentSeasonTest

{

[TestCase(12, "Winter")]

[TestCase(3, "Spring")]

[TestCase(6, "Summer")]

[TestCase(9, "Fall")]

public void GetCurrentSeason_ForFirstMonthsOfSeasons_ReturnsCorrectSeason(

int month,

string season

)

{

Assert.That(season, Is.EqualTo(Season.GetCurrentSeason(new DateTime(2020, month, 1))));

}

}

Test results indicate there is something wrong with the code path that should return "Winter"

if (currentTime.Month > 11 && currentTime.Month < 3)

The line is changed to

if (currentTime.Month > 11 || currentTime.Month < 3)

Writing Testable Code - Example 2

Let's look at a HeatingUnit class, which returns a heating setting based on the current season. The current date problem is back again:

public class HeatingUnit

{

public string GetHeatingSetting()

{

DateTime currentDate = DateTime.Now;

if (Season.GetCurrentSeason(currentDate) == "Winter")

return "HEAT_SETTING_HIGH";

else if (Season.GetCurrentSeason(currentDate) == "Summer")

return "HEAT_SETTING_OFF";

else

return "HEAT_SETTING_MEDIUM";

}

}

Writing Testable Code - Example 2 (continued)

Instead of taking the current date initialization even higher in the class hierarchy, let's make a service that returns the current date

public interface IDateTimeProvider

{

DateTime GetDateTime();

}

After this the service can be injected into HeatingUnit with constructor injection (next slide)

public class HeatingUnit

{

private readonly IDateTimeProvider _dateTimeProvider;

public HeatingUnit(IDateTimeProvider dateTimeProvider)

{

_dateTimeProvider = dateTimeProvider;

}

public string GetHeatingSetting()

{

if (Season.GetCurrentSeason(_dateTimeProvider.GetDateTime()) == "Winter")

return "HEAT_SETTING_HIGH";

else if (Season.GetCurrentSeason(_dateTimeProvider.GetDateTime()) == "Summer")

return "HEAT_SETTING_OFF";

else

return "HEAT_SETTING_MEDIUM";

}

}

For testing, a fake DateTimeProvider service can be used for injecting the HeatingUnit with any date:

class FakeDateTimeProvider : IDateTimeProvider

{

public DateTime Date { get; set; }

public DateTime GetDateTime() => Date;

}

In the real application the provider would return the real current date

class SetHeatingSettingTest

{

[TestCase(12, "HEAT_SETTING_HIGH")]

[TestCase(3, "HEAT_SETTING_MEDIUM")]

[TestCase(6, "HEAT_SETTING_OFF")]

[TestCase(9, "HEAT_SETTING_MEDIUM")]

public void SetHeatingSetting_ForFirstMonthsOfSeasons_ShouldReturnCorrectSetting(

int month,

string setting

)

{

FakeDateTimeProvider timeProvider

= new FakeDateTimeProvider { Date = new DateTime(2020, month, 1) };

HeatingUnit heatingUnit = new HeatingUnit(timeProvider);

Assert.That(setting, Is.EqualTo(heatingUnit.GetHeatingSetting()));

}

}

---

Näytä demo

Testing with Postman

• So far, you have created individual request with Postman to see what the response of an API is for each request
• This is __not testing __ an API, this is __exploring __ an API
• Tests in Postman are inserted in the _Tests _ tab
  • Postman uses Chai.js testing library for creating tests

To get started immediately, you can select snippets from the right

Let's select the "Status code: Code is 200" snippet

Successfully ran tests show up in green in the Test Result tab of the response:

The official Postman web page is a good starting point for learning testing with Postman

Using pm.expect will give a bit more info about the test:

The response to /serverinfo has the following body:

{

"appSettings": {

"applicationUrl": "http://localhost:63741",

"aspNetCoreEnvironment": "Development"

},

"serverStatus": {

"dbConnectionStatus": "Connected"

}

}

Let's make a test to see whether the server is connected to the database

Variables can be declared within the test:

Exercise 3: Testing with Postman

Create a new collection in Postman by selecting the _Collections _ tab and clicking New Collection. Name it _CourseAPI Tests. _ Launch the CourseAPI you have developed during the lectures, and make sure the course_db database is connected

Create a new GET request for the URI /api/courses. Add a test: status code of the request should be 200. Save the request to the CourseAPI Tests collection

Create a new GET request for the URI /api/courses/999999999. Add a test: status code of the request should be 404. Save the request to the CourseAPI Tests collection

Create a new POST request for the URI /api/courses. Set the headers and the content correctly, but set the value of credits to 100. Add a test: status code should be 400. Save the request to the CourseAPI Tests collection

Hover on CourseAPI Tests collection, click the arrow and click RUN. From the opened window, scroll down and click "Run CourseAPI Tests". This will create all the requests in your collection

CI/CD/CT

• CI stands for Continuous Integration
  • Each change in code triggers a build-and-test sequence for the given project
    • Goal is to have a consistent and automated way to build, package, and test applications
• CD stands for Continuous Delivery
  • Automates the delivery of applications to selected environments, such as Azure, AWS, GCP
• Both require Continuous Testing , which includes automated regression, performance and other tests