education
/
aspnet-basics
12
0
Fork 0
Code Issues 2 Pull Requests Packages Projects Releases Wiki Activity
You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
main
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from '0ecd662086'
${ noResults }
aspnet-basics/8-testing.md

815 lines
25 KiB
Markdown
Raw Blame History

---
marp: true
paginate: true
math: mathjax
theme: buutti
title: 8. Testing
---
# Testing
<!-- headingDivider: 5 -->
<!-- class: invert -->
## Contents
- [Introduction to testing](#introduction-to-testing)
- [C# Testing Frameworks](#c-testing-frameworks)
- [An extensive unit test example](#an-extensive-unit-test-example)
- [Writing testable code](#writing-testable-code)
- [Testing with Postman](#testing-with-postman)
- [CI/CD/CT](#cicdct)
## Introduction to testing
### What is testing?
* Testing is much more than just trying out manually if the program works!
* [Manual testing](https://en.wikipedia.org/wiki/Manual_testing): acting as an end user of the application
* [Automated testing](https://en.wikipedia.org/wiki/Test_automation): using software to emulate the end user, or testing the functionality of the code itself
* Proving that the program is working as intended during development on a...
* ...***micro level***: individual pieces of code, such as methods
* ...***macro level***: larger functional components, such as services
* *All possible paths in a piece of code should be tested*
* i.e., [test coverage](https://en.wikipedia.org/wiki/Code_coverage) should be close to 100%
### Automated testing: Why?
* Why do automated testing?
* Sets ground truths that can be relied on during development:
* *"With this kind of input, this should happen"*
* (No need to take the programmers word for it)
* Ensures code functionality before moving on
* Ideally finding bugs before starting to debug
* Guides developers to create more testable and thus more readable, interchangeable and overall better code
* [Test-driven development (TDD)](https://en.wikipedia.org/wiki/Test-driven_development) is based on writing tests first and code later!
### Types of automated testing
* Unit testing
* Testing individual units (methods, classes) within a project
* *"input x produces output y"*
* Each unit has to be entirely independent and ***deterministic*** for it to be unit testable!
* Deterministic: Always the same result with the same input
* Integration testing
* Testing complete modules and the interaction between them within a project
* Closer to real life scenarios
* End-to-end (E2E) testing
* Testing the whole program by emulating the end user's behaviour
### Test type comparison
* Unit tests are faster to write, E2E tests slowest
<div class='centered'>
![w:800px](https://microsoft.github.io/code-with-engineering-playbook/automated-testing/cdc-testing/images/testing-pyramid.png)
</div>
## C# Testing Frameworks
* Three main testing frameworks:
* `MSTest`
* `NUnit`
* `XUnit`
* You can install these using NuGet, or in the newer versions of Visual Studio you can create a new testing project
* NUnit is the most popular of the three, and will be used in this lecture
### MSTest
* Built-in to most versions of Visual Studio since 2005
* Simple to use, easy structure
* `TestClass` for denoting classes containing unit tests
* `TestMethod` for denoting unit test methods
### NUnit
* Most popular C# unit testing framework
* Very similar structure to MSTest tests
* Instead of `TestClass`, we use `TestFixture`
* Instead of `TestMethod`, we use `Test` / `TestCase`
* Has some advantages to MSTest or XUnit tests, for example:
* Ability to test a single test method using multiple different arguments
### XUnit
* Free, open-source unit testing tool for .NET framework and .NET core
* Made by the creator of NUnit v2
* You can test C#, F#, VB.NET and other .NET languages
* Part of the .NET Foundation
* Unique and different style
### Note about test case naming
* In C# unit testing, the most commonly followed naming convention for test cases/methods is as follows:
```csharp
FunctionNameThatIsTested_Condition_ShouldDoX
```
* For example:
```csharp
Fibonacci_WhenNIs0_ShouldReturn0
```
## An extensive unit test Example
### Creating a solution & sample project
First, we need a solution and a sample project.
<div class='columns23' markdown='1'>
<div markdown='1'>
1) In Visual Studio, create a ***Class library*** that targets *__.NET Standard__*.
2) Name the project `Fibonacci.Library` and the solution `Fibonacci`. Remember to uncheck _Place solution and project in the same directory_ if it isn't unchecked already.
</div>
<div markdown='1'>
![](imgs/9-testing_1.png)
</div>
</div>
### Adding the function to be tested
<div class='columns' markdown='1'>
<div markdown='1'>
3) Rename the default `Class1.cs` file to `Fibonacci.cs`.
4) Allow Visual Studio to also rename the class itself to `Fibonacci`.
* Make the class `public` and `static`.
5) Add a *recursive fibonacci function* shown here.
</div>
<div markdown='1'>
```csharp
using System;
namespace Fibonacci.Library
{
public static class Fibonacci
{
public static long Recursive(
int n,
long previous = 0,
long current = 0,
int counter = 0
)
{
return counter == n ?
previous + current :
Recursive(
n,
current,
Math.Max(previous + current, 1),
counter + 1
);
}
}
}
```
</div>
</div>
### Adding a testing project
* Now we need to add a testing project to our solution.
6) Right click your solution, and add a new project. Select the NUnit project for **_.NET Core_**, and call it `Fibonacci.UnitTests`.
7) Rename the created class file to `Fibonacci_Tests.cs`, and the class to `Fibonacci_Tests`
### Linking the testing project
8) We then need to link the `Fibonacci.Library` project to `Fibonacci.UnitTests`.
9) Right click `Fibonacci.UnitTests`, and select *Add > Reference...*, and under *Projects* select `Fibonacci.Library`. Press OK.
* Now we have linked `Fibonacci.Library` to `Fibonacci.UnitTests`, and we can begin testing our `Recursive` function in the `Fibonacci` class.
### Importing the class
10) Since we have many things called "Fibonacci", we have to import our `Fibonacci` class by using an alias
* Put this to the top of your file `Fibonacci_Tests.cs`, underneath<br>`using NUnit.Framework;`
```csharp
using FibonacciLib = Fibonacci.Library.Fibonacci;
```
### Test fixture for a class
11) Now, add `[TestFixture]` on top of our class, like so:
```csharp
using NUnit.Framework;
using FibonacciLib = Fibonacci.Library.Fibonacci;
namespace Fibonacci.UnitTests {
[TestFixture]
public class Fibonacci_Tests {
```
* This denotes that this class contains our unit tests.
### Adding the unit test method
12) Then, inside our class `Fibonacci_Tests`, let's create our first method to test the `Recursive` function, as shown here:
```csharp
using NUnit.Framework;
using FibonacciLib = Fibonacci.Library.Fibonacci;
namespace Fibonacci.UnitTests {
[TestFixture]
public class Fibonacci_Tests {
[Test]
public void Recursive_WhenNIs0_ShouldReturn0()
=> Assert.That(0, Is.EqualTo(FibonacciLib.Recursive(n: 0)));
}
}
```
### Things to take into consideration
* When passing in function arguments, to avoid ***magic numbers*** and to improve code readability, use the syntax `argumentName: value`
* Magic numbers = using numbers as input as-is without explanation, for example without assigning to a well-named variable, or without using a descriptive type
* `Assert.That` is one of many functions that can determine whether the test passes
* Its first argument is the ***expected value***. In the second argument you give a ***constraint*** for the actual output, i.e., value is equal to, greater than, starts with, [etc](https://docs.nunit.org/articles/nunit/writing-tests/constraints/Constraints.html).
* *Do not mix the arguments up!* If you do, the errors you get do not make any sense.
### Running Tests
* Now, let's open the built-in test runner in Visual Studio to run our tests.
* On the top of Visual Studio, click *Test*, then *Test Explorer*.
* In the Test Explorer, you should see the test case we created. If not, build the solution, and make sure there are no errors.
* Either way, let's run our test case. Press *Run All Tests* in the top-left corner of Test Explorer. You should see our test case *__passing__*.
### Adding More Test Cases
* While there's nothing inherently wrong about our test case, it doesn't cover much of anything.
* We need to add more test cases to test different values and outputs.
* In other testing frameworks, you would do this by creating more methods.
* But with `NUnit`, we can use our *existing* method.
### Adding a new test
1) Do the following modifications to our test method:
```csharp
[TestCase(0, 0)]
public void Recursive_Always_ShouldReturnNthFibonacci(
int n,
long expected
)
=> Assert.That(expected, Is.EqualTo(FibonacciLib.Recursive(n)));
```
* Using `TestCase` instead of `Test`, we can pass in arguments to our test case method, and we can use those passed in arguments in our actual tests.
### Adding more tests
2) Run the test cases again, and they should *__pass__*.
* But as you might notice, we haven't really changed anything yet.
3) Let's add more test cases, like so:
```csharp
[TestCase(0, 0)]
[TestCase(1, 1)]
public void Recursive_Always_ShouldReturnNthFibonacci(
```
4) Now we have 2 test cases with only one test method. Run the tests again, and both cases should *__pass__*.
### Adding a failing test
5) Let's add one more TestCase, like so:
```csharp
[TestCase(2, 1)]
```
6) Run the tests again, and you should see the test case *__fail__*.
* This is to be expected: the 2nd fibonacci number should be 1, but the actual value we got from our function was 2.
* This means our function does not work as expected.
### Fixing our code
<div class='columns' markdown='1'>
<div markdown='1'>
7) Let's fix our function, as seen here.
8) Run the tests again. You should see all *__pass__* now.
9) Add a few more test cases
* (If you don't know the fibonacci sequence, you can check it from the internet.)
</div>
<div markdown='1'>
```csharp
public static long Recursive(
int n,
long previous = 0,
long current = 0,
int counter = 0
)
{
return counter == n ?
current : // changed from previous + current
Recursive(
n,
current,
Math.Max(previous + current, 1),
counter + 1
);
}
```
</div>
</div>
### Exercise 1: Unit testing in practice
<!--_class: "exercise invert" -->
Answer the following questions:
1) How many test cases should we ideally have to test our `Recursive` function?
2) What happens if we pass in `n = int.MaxValue (2147483647)`?
* What about other huge numbers?
* How would you fix the function so it works as expected?
3) What happens if we pass in `n = -1`?
* What about other negative numbers?
* How would you fix the function so it works as expected?
### Answers to Exercise 1: Unit testing in practice
<!--_class: "exercise invert" -->
1) We should cover all the edge cases and code paths with our test cases. We do not need to have many test cases for the same code path.
2) `Stack Overflow`, `Int64 Overflow`, or the test case just won't run.
* We should have a maximum allowed value for `n`, throw the ***argument out of range*** exception if above
3) `Stack Overflow`, since `current` can never equal `n` that is less than `0`
* We should have a minimum allowed value for `n`, throw the ***argument out of range*** exception if below
### Testing exceptions
* To expect exceptions to be thrown in test cases, we can use the method `Assert.Throws`
* Syntax is as follows:
```csharp
Assert.Throws<ExceptionType>(delegate);
```
* In practice:
```csharp
Assert.Throws<NullReferenceException>(() => TestedFunction(n));
```
### Handling border cases
<div class='columns' markdown='1'>
<div markdown='1'>
* Continuing our Fibonacci test case, we need to add handling for `n` that is less than `0` or more than a set amount. Then we need to add another test method and two test cases to test our new functionality.
1) Let's start with modifying our existing `Recursive` function, as shown here:
</div>
<div markdown='1'>
```csharp
public const int MinDepth = 0;
public const int MaxDepth = 10000;
public static long Recursive(
int n,
long previous = 0,
long current = 0,
int counter = 0
)
{
if (n < MinDepth || n > MaxDepth)
throw new ArgumentOutOfRangeException(
"n is out of bounds!"
);
return counter == n ?
current :
Recursive(
n,
current,
Math.Max(previous + current, 1),
counter + 1
);
}
```
</div>
</div>
### Testing that everything works
* Now our `Recursive` function throws an `ArgumentOutOfRange` exception when `n` is less than `0` or more than `10000`.
2) Let's test that everything works as expected by adding a new test method and a few test cases:
```csharp
[TestCase(int.MaxValue)]
[TestCase(-1)]
public void Recursive_WhenNIsOutOfBounds_ShouldThrowArgumentOutOfRange(
int n
)
=> Assert.Throws<ArgumentOutOfRangeException>(() => FibonacciLib.Recursive(n));
```
3) Rename the other method to `Recursive_WhenNIsInBounds_ShouldReturnNthFibonacci`
### Unnecessary arguments
* We have one more problem:
* Our `Recursive` function is a `public` method that has in total 4 arguments it takes, yet we only test 1 argument (`n`).
* In order for our tests to be complete, we need to test all the arguments a `public` method takes in.
* So, how do we fix this?
### Removing unnecessary arguments
<div class='columns' markdown='1'>
<div markdown='1'>
* The naive way would be to test all the arguments with more test cases.
* It's important to realize that *__we don't have to__*.
* Since we never pass in any other arguments than `n` from outside `Recursive`, we could just create another `private` function that has those arguments, and remove them from our public function.
</div>
<div markdown='1'>
```csharp
public static long Recursive(int nStartingValue)
{
if (nStartingValue < MinDepth
|| nStartingValue > MaxDepth)
throw new ArgumentOutOfRangeException(
"n is out of bounds!"
);
long getRecursive(
int n,
long previous = 0,
long current = 0,
int counter = 0)
=> counter == n ?
current :
getRecursive(
n,
current,
Math.Max(previous + current, 1),
counter + 1
);
return getRecursive(nStartingValue);
}
```
</div>
</div>
### End Result
* Now we have tested all the arguments in our public `Recursive` function.
* We do not need to test private functions such as `getRecursive`, only the public API has to be tested.
* All tests should *__pass__* at this point.
### Exercise 2: Unit Testing
<!--_class: "exercise invert" -->
```csharp
public static long DoX(int nStartingValue) {
if (nStartingValue < 0 || nStartingValue > 150)
throw new ArgumentOutOfRangeException("n is out of bounds!");
long doY(int n) => n > 0 ? n * doY(n - 1) : 1;
return doY(nStartingValue);
}
```
* Following the previously used architecture as closely as possible, unit test the attached function using `NUnit`. Answer the following questions:
1) What does the function that we are testing actually *do*?
2) Do we have to test `doY` as well?
3) How would you name `DoX` so it is descriptive? How about `doY`?
### Answers to exercise 2: Unit Testing
<!--_class: "exercise invert" -->
1) It gets the nth factorial (`n!`)
2) `doY` is a private method that gets tested when we test `DoX`, so no.
3) Example names: `DoX` $\Rightarrow$ `TryGetFactorial`, `doY` $\Rightarrow$ `getFactorial`
### Unit Testing: Final testing code
*
```csharp
[TestCase(0, 1)]
[TestCase(1, 1)]
[TestCase(2, 2)]
[TestCase(3, 6)]
[TestCase(15, 1307674368000)]
public void TryGetFactorial_WhenNIsInBounds_ShouldReturnNthFactorial(
int n, long expected
)
=> Assert.That(expected, Is.EqualTo(TestExerciseLib.TryGetFactorial(n)));
```
```csharp
[TestCase(-1)]
[TestCase(151)]
public void TryGetFactorial_NIsOutOfBounds_ShouldThrowArgumentOutOfRange(
int n
)
=> Assert.Throws<ArgumentOutOfRangeException>(
() => TestExerciseLib.TryGetFactorial(n)
);
```
## Writing testable code
* Writing testable code often means that you're writing good code
* Testable units should
* be deterministic (functional): same input always produces the same output
* be [loosely coupled](https://en.wikipedia.org/wiki/Loose_coupling): components have a loose and limited connection to other components in the system, meaning changes in one component should not require changes to many others
* abide to the *__Single Responsibility Principle__*: a function, method or component is only responsible for one purpose
### Writing testable code: Example 1
<div class='columns23' markdown='1'>
<div markdown='1'>
* Can `GetCurrentSeason()` be unit tested? Why / why not?
* How to fix the problems, if any?
</div>
<div markdown='1'>
```csharp
public static class Season
{
public static string GetCurrentSeason() {
DateTime currentTime = DateTime.Now;
if (currentTime.Month > 11 && currentTime.Month < 3)
return "Winter";
else if (currentTime.Month < 6)
return "Spring";
else if (currentTime.Month < 9)
return "Summer";
else
return "Fall";
}
}
```
</div>
</div>
### Writing testable code: Example 1 fixed
<div class='columns23' markdown='1'>
<div markdown='1'>
* Now the method is purely functional and deterministic
* Will return the same value for every call with equal arguments
* `GetCurrentSeason()` can now easily be tested as shown next
</div>
<div markdown='1'>
```csharp
public static class Season {
public static string GetCurrentSeason(
DateTime currentTime
) {
if (currentTime.Month > 11 && currentTime.Month < 3)
return "Winter";
else if (currentTime.Month < 6)
return "Spring";
else if (currentTime.Month < 9)
return "Summer";
else
return "Fall";
}
}
```
</div>
</div>
---
```csharp
public class GetCurrentSeasonTest
{
[TestCase(12, "Winter")]
[TestCase(3, "Spring")]
[TestCase(6, "Summer")]
[TestCase(9, "Fall")]
public void GetCurrentSeason_ForFirstMonthsOfSeasons_ReturnsCorrectSeason(
int month,
string season
)
{
Assert.That(season,
Is.EqualTo(
Season.GetCurrentSeason(new DateTime(2020, month, 1))
)
);
}
}
```
---
<div class='centered'>
![](imgs/9-testing_2.png)
</div>
* The tests fail, as shown above.
---
<div class='columns32' markdown='1'>
<div markdown='1'>
* Test results indicate there is something wrong with the code path that should return `Winter`
```csharp
if (currentTime.Month > 11 && currentTime.Month < 3)
```
* The line should be changed to
```csharp
if (currentTime.Month > 11 || currentTime.Month < 3)
```
</div>
<div markdown='1'>
![](imgs/9-testing_3.png)
</div>
</div>
### Writing testable code: Example 2
* Let's look at a `HeatingUnit` class that returns a heating setting based on the current season. The current date problem is back again:
```csharp
public class HeatingUnit
{
public string GetHeatingSetting()
{
DateTime currentDate = DateTime.Now;
if (Season.GetCurrentSeason(currentDate) == "Winter")
return "HEAT_SETTING_HIGH";
else if (Season.GetCurrentSeason(currentDate) == "Summer")
return "HEAT_SETTING_OFF";
else
return "HEAT_SETTING_MEDIUM";
}
}
```
### Writing testable code: Example 2 continued
* Instead of taking the current date initialization even higher in the class hierarchy, let's make a ***service*** that returns the current date
```csharp
public interface IDateTimeProvider
{
DateTime GetDateTime();
}
```
* After this the service can be injected into `HeatingUnit` with constructor injection
* Code example shown next
---
```csharp
public class HeatingUnit
{
private readonly IDateTimeProvider _dateTimeProvider;
public HeatingUnit(IDateTimeProvider dateTimeProvider)
{
_dateTimeProvider = dateTimeProvider;
}
public string GetHeatingSetting() {
if (Season.GetCurrentSeason(_dateTimeProvider.GetDateTime()) == "Winter")
return "HEAT_SETTING_HIGH";
else if (Season.GetCurrentSeason(_dateTimeProvider.GetDateTime()) == "Summer")
return "HEAT_SETTING_OFF";
else
return "HEAT_SETTING_MEDIUM";
}
}
```
---
* For testing, a fake `DateTimeProvider` service can be used for injecting the `HeatingUnit` with any date:
```csharp
class FakeDateTimeProvider : IDateTimeProvider
{
public DateTime Date { get; set; }
public DateTime GetDateTime() => Date;
}
```
* Using this kind of a fake service instead of the real one is called [mocking](https://en.wikipedia.org/wiki/Mock_object)
* In the real application the provider would return the real current date
---
```csharp
class SetHeatingSettingTest
{
[TestCase(12, "HEAT_SETTING_HIGH")]
[TestCase(3, "HEAT_SETTING_MEDIUM")]
[TestCase(6, "HEAT_SETTING_OFF")]
[TestCase(9, "HEAT_SETTING_MEDIUM")]
public void SetHeatingSetting_ForFirstMonthsOfSeasons_ShouldReturnCorrectSetting(
int month,
string setting
)
{
FakeDateTimeProvider timeProvider
= new FakeDateTimeProvider { Date = new DateTime(2020, month, 1) };
HeatingUnit heatingUnit = new HeatingUnit(timeProvider);
Assert.That(setting, Is.EqualTo(heatingUnit.GetHeatingSetting()));
}
}
```
---
<div class='centered'>
![](imgs/9-testing_4.png)
Tests should pass!
</div>
## Testing with Postman
* So far, we have created individual request with Postman to see what the response of an API is for each request
* This is not testing an API, this is *__exploring__* an API
* Tests in Postman are inserted in the _Tests_ tab
* Postman uses the *__Chai.js__* testing library for creating tests
![](imgs/9-testing_5.png)
---
* To get started immediately, you can select *snippets* from the right
* Let's select the *Status code: Code is 200* snippet
![](imgs/9-testing_6.png)
---
* Successfully run tests show up in green in the *Test Result* tab of the response:
![](imgs/9-testing_7.png)
---
* [The official Postman web page](https://learning.postman.com/docs/writing-scripts/test-scripts/) is a good starting point for learning testing with Postman
* Using `pm.expect` will give a bit more info about the test:
![](imgs/9-testing_8.png)
---
* The response to /serverinfo has the following body:
```json
{
"appSettings": {
"applicationUrl": "http://localhost:63741",
"aspNetCoreEnvironment": "Development"
},
"serverStatus": {
"dbConnectionStatus": "Connected"
}
}
```
* Let's make a test to see whether the server is connected to the database
---
* Variables can be declared within the test:
![](imgs/9-testing_9.png)
### Exercise 3: Testing with Postman
<!--_class: "exercise invert" -->
1) Create a new collection in Postman by selecting the *Collections* tab and clicking *New Collection*. Name it `CourseAPI Tests`. Launch the CourseAPI you have developed during the lectures, and make sure the `course_db` database is connected
2) Create a new `GET` request for the URI `/api/courses`. Add a test: status code of the request should be 200. Save the request to the `CourseAPI Tests` collection.
3) Create a new `GET` request for the URI `/api/courses/999999999`. Add a test: status code of the request should be 404. Save the request to the `CourseAPI Tests` collection.
---
<!--_class: "exercise invert" -->
4) Create a new `POST` request for the URI `/api/courses`. Set the headers and the content correctly, but set the value of credits to 100. Add a test; status code should be 400. Save the request to the CourseAPI Tests collection.
5) Hover on CourseAPI Tests collection, click the arrow and click *Run*. From the opened window, scroll down and click *Run CourseAPI Tests*. This will create all the requests in your collection.
## CI/CD/CT
* CI stands for *__Continuous Integration__*
* Each change in code triggers a build-and-test sequence for the given project
* Goal is to have a consistent and automated way to build, package, and test applications
* CD stands for *__Continuous Delivery__*
* Automates the delivery of applications to selected environments, such as Azure, AWS, GCP
* Both require *__Continuous Testing__*, which includes automated regression, performance and other tests
Reference in New Issue View Git Blame Copy Permalink