diff --git a/8-testing-slides.html b/8-testing-slides.html new file mode 100644 index 0000000..e079d18 --- /dev/null +++ b/8-testing-slides.html @@ -0,0 +1,957 @@ +8. Testing
+

Testing

+
+
+

Contents

+ +
+
+

Introduction to testing

+
+
+

What is testing?

+
    +
  • Testing is much more than just trying out manually if the program works!
    • +
  • Manual testing: acting as an end user of the application
    • +
  • Automated testing: using software to emulate the end user, or testing the functionality of the code itself
    • +
  • Proving that the program is working as intended during development on a... +
      +
    • ...micro level: individual pieces of code, such as methods
      • +
    • ...macro level: larger functional components, such as services
      • +
    +
    • +
  • All possible paths in a piece of code should be tested + +
    • +
+
+
+

Automated testing: Why?

+
    +
  • Why do automated testing? +
      +
    • Sets ground truths that can be relied on during development: +
        +
      • "With this kind of input, this should happen"
        • +
      • (No need to take the programmers word for it)
        • +
      +
      • +
    • Ensures code functionality before moving on +
        +
      • Ideally finding bugs before starting to debug
        • +
      +
      • +
    • Guides developers to create more testable and thus more readable, interchangeable and overall better code
      • +
    • Test-driven development (TDD) is based on writing tests first and code later!
      • +
    +
    • +
+
+
+

Types of automated testing

+
    +
  • Unit testing +
      +
    • Testing individual units (methods, classes) within a project
      • +
    • "input x produces output y"
      • +
    • Each unit has to be entirely independent and deterministic for it to be unit testable! +
        +
      • Deterministic: Always the same result with the same input
        • +
      +
      • +
    +
    • +
  • Integration testing +
      +
    • Testing complete modules and the interaction between them within a project
      • +
    • Closer to real life scenarios
      • +
    +
    • +
  • End-to-end (E2E) testing +
      +
    • Testing the whole program by emulating the end user's behaviour
      • +
    +
    • +
+
+
+

Test type comparison

+
    +
  • Unit tests are faster to write, E2E tests slowest
    • +
+
+

+
+
+
+

C# Testing Frameworks

+
    +
  • Three main testing frameworks: +
      +
    • MSTest
      • +
    • NUnit
      • +
    • XUnit
      • +
    +
    • +
  • You can install these using NuGet, or in the newer versions of Visual Studio you can create a new testing project
    • +
  • NUnit is the most popular of the three, and will be used in this lecture
    • +
+
+
+

MSTest

+
    +
  • Built-in to most versions of Visual Studio since 2005
    • +
  • Simple to use, easy structure +
      +
    • TestClass for denoting classes containing unit tests
      • +
    • TestMethod for denoting unit test methods
      • +
    +
    • +
+
+
+

NUnit

+
    +
  • Most popular C# unit testing framework
    • +
  • Very similar structure to MSTest tests +
      +
    • Instead of TestClass, we use TestFixture
      • +
    • Instead of TestMethod, we use Test / TestCase
      • +
    +
    • +
  • Has some advantages to MSTest or XUnit tests, for example: +
      +
    • Ability to test a single test method using multiple different arguments
      • +
    +
    • +
+
+
+

XUnit

+
    +
  • Free, open-source unit testing tool for .NET framework and .NET core
    • +
  • Made by the creator of NUnit v2
    • +
  • You can test C#, F#, VB.NET and other .NET languages
    • +
  • Part of the .NET Foundation
    • +
  • Unique and different style
    • +
+
+
+

Note about test case naming

+
    +
  • In C# unit testing, the most commonly followed naming convention for test cases/methods is as follows:
    FunctionNameThatIsTested_Condition_ShouldDoX
+
    +
    • +
  • For example:
    Fibonacci_WhenNIs0_ShouldReturn0
+
    +
    • +
+
+
+

An extensive unit test Example

+
+
+

Creating a solution & sample project

+

First, we need a solution and a sample project.

+
+
+
    +
  1. In Visual Studio, create a Class library that targets .NET Standard.
    2. +
  2. Name the project Fibonacci.Library and the solution Fibonacci. Remember to uncheck Place solution and project in the same directory if it isn't unchecked already.
    3. +
+
+
+

+
+
+
+
+

Adding the function to be tested

+
+
+
    +
  1. Rename the default Class1.cs file to Fibonacci.cs.
    2. +
  2. Allow Visual Studio to also rename the class itself to Fibonacci. +
      +
    • Make the class public and static.
      • +
    +
    3. +
  3. Add a recursive fibonacci function shown here.
    4. +
+
+
+
using System;
+
+namespace Fibonacci.Library
+{
+    public static class Fibonacci
+    {
+        public static long Recursive(
+            int n,
+            long previous = 0,
+            long current = 0,
+            int counter = 0
+        )
+        {
+            return counter == n ?
+                previous + current :
+                Recursive(
+                  n,
+                  current,
+                  Math.Max(previous + current, 1),
+                  counter + 1
+                );
+        }
+    }
+}
+
+
+
+
+
+

Adding a testing project

+
    +
  • Now we need to add a testing project to our solution.
    • +
+
    +
  1. Right click your solution, and add a new project. Select the NUnit project for .NET Core, and call it Fibonacci.UnitTests.
    2. +
  2. Rename the created class file to Fibonacci_Tests.cs, and the class to Fibonacci_Tests
    3. +
+
+
+

Linking the testing project

+
    +
  1. We then need to link the Fibonacci.Library project to Fibonacci.UnitTests.
    2. +
  2. Right click Fibonacci.UnitTests, and select Add > Reference..., and under Projects select Fibonacci.Library. Press OK.
    3. +
+
    +
  • Now we have linked Fibonacci.Library to Fibonacci.UnitTests, and we can begin testing our Recursive function in the Fibonacci class.
    • +
+
+
+

Importing the class

+
    +
  1. Since we have many things called "Fibonacci", we have to import our Fibonacci class by using an alias +
      +
    • Put this to the top of your file Fibonacci_Tests.cs, underneath
      using NUnit.Framework;
      using FibonacciLib = Fibonacci.Library.Fibonacci;
+
      +
      • +
    +
    2. +
+
+
+

Test fixture for a class

+
    +
  1. Now, add [TestFixture] on top of our class, like so:
    2. +
+
using NUnit.Framework;
+using FibonacciLib = Fibonacci.Library.Fibonacci;
+
+namespace Fibonacci.UnitTests {
+    [TestFixture]
+    public class Fibonacci_Tests {
+
+
    +
  • This denotes that this class contains our unit tests.
    • +
+
+
+

Adding the unit test method

+
    +
  1. Then, inside our class Fibonacci_Tests, let's create our first method to test the Recursive function, as shown here:
    using NUnit.Framework;
+using FibonacciLib = Fibonacci.Library.Fibonacci;
+
+namespace Fibonacci.UnitTests {
+    [TestFixture]
+    public class Fibonacci_Tests {
+        [Test]
+        public void Recursive_WhenNIs0_ShouldReturn0()
+            => Assert.That(0, Is.EqualTo(FibonacciLib.Recursive(n: 0)));
+    }
+}
+
    +
    2. +
+
+
+

Things to take into consideration

+
    +
  • When passing in function arguments, to avoid magic numbers and to improve code readability, use the syntax argumentName: value +
      +
    • Magic numbers = using numbers as input as-is without explanation, for example without assigning to a well-named variable, or without using a descriptive type
      • +
    +
    • +
  • Assert.That is one of many functions that can determine whether the test passes +
      +
    • Its first argument is the expected value. In the second argument you give a constraint for the actual output, i.e., value is equal to, greater than, starts with, etc.
      • +
    • Do not mix the arguments up! If you do, the errors you get do not make any sense.
      • +
    +
    • +
+
+
+

Running Tests

+
    +
  • +

    Now, let's open the built-in test runner in Visual Studio to run our tests.

    +
    • +
  • +

    On the top of Visual Studio, click Test, then Test Explorer.

    +
    • +
  • +

    In the Test Explorer, you should see the test case we created. If not, build the solution, and make sure there are no errors.

    +
    • +
  • +

    Either way, let's run our test case. Press Run All Tests in the top-left corner of Test Explorer. You should see our test case passing.

    +
    • +
+
+
+

Adding More Test Cases

+
    +
  • While there's nothing inherently wrong about our test case, it doesn't cover much of anything.
    • +
  • We need to add more test cases to test different values and outputs.
    • +
  • In other testing frameworks, you would do this by creating more methods.
    • +
  • But with NUnit, we can use our existing method.
    • +
+
+
+

Adding a new test

+
    +
  1. Do the following modifications to our test method:
    [TestCase(0, 0)]
+public void Recursive_Always_ShouldReturnNthFibonacci(
+    int n,
+    long expected
+)
+    => Assert.That(expected, Is.EqualTo(FibonacciLib.Recursive(n)));
+
    +
    2. +
+
    +
  • Using TestCase instead of Test, we can pass in arguments to our test case method, and we can use those passed in arguments in our actual tests.
    • +
+
+
+

Adding more tests

+
    +
  1. +

    Run the test cases again, and they should pass.

    +
      +
    • But as you might notice, we haven't really changed anything yet.
      • +
    +
    2. +
  2. +

    Let's add more test cases, like so:

    +
    [TestCase(0, 0)]
+[TestCase(1, 1)]
+public void Recursive_Always_ShouldReturnNthFibonacci(
+
    +
    3. +
  3. +

    Now we have 2 test cases with only one test method. Run the tests again, and both cases should pass.

    +
    4. +
+
+
+

Adding a failing test

+
    +
  1. Let's add one more TestCase, like so:
    [TestCase(2, 1)]
+
    +
    2. +
  2. Run the tests again, and you should see the test case fail. +
      +
    • This is to be expected: the 2nd fibonacci number should be 1, but the actual value we got from our function was 2.
      • +
    • This means our function does not work as expected.
      • +
    +
    3. +
+
+
+

Fixing our code

+
+
+
    +
  1. Let's fix our function, as seen here.
    2. +
  2. Run the tests again. You should see all pass now.
    3. +
  3. Add a few more test cases +
      +
    • (If you don't know the fibonacci sequence, you can check it from the internet.)
      • +
    +
    4. +
+
+
+
public static long Recursive(
+    int n,
+    long previous = 0,
+    long current = 0,
+    int counter = 0
+)
+{
+    return counter == n ?
+        current : // changed from previous + current
+        Recursive(
+          n,
+          current,
+          Math.Max(previous + current, 1),
+          counter + 1
+        );
+}
+
+
+
+
+
+

Exercise 1: Unit testing in practice

+ +

Answer the following questions:

+
    +
  1. How many test cases should we ideally have to test our Recursive function?
    2. +
  2. What happens if we pass in n = int.MaxValue (2147483647)? +
      +
    • What about other huge numbers?
      • +
    • How would you fix the function so it works as expected?
      • +
    +
    3. +
  3. What happens if we pass in n = -1? +
      +
    • What about other negative numbers?
      • +
    • How would you fix the function so it works as expected?
      • +
    +
    4. +
+
+
+

Answers to Exercise 1: Unit testing in practice

+ +
    +
  1. We should cover all the edge cases and code paths with our test cases. We do not need to have many test cases for the same code path.
    2. +
  2. Stack Overflow, Int64 Overflow, or the test case just won't run. +
      +
    • We should have a maximum allowed value for n, throw the argument out of range exception if above
      • +
    +
    3. +
  3. Stack Overflow, since current can never equal n that is less than 0 +
      +
    • We should have a minimum allowed value for n, throw the argument out of range exception if below
      • +
    +
    4. +
+
+
+

Testing exceptions

+
    +
  • To expect exceptions to be thrown in test cases, we can use the method Assert.Throws
    • +
  • Syntax is as follows:
    Assert.Throws<ExceptionType>(delegate);
+
    +
    • +
  • In practice:
    Assert.Throws<NullReferenceException>(() => TestedFunction(n));
+
    +
    • +
+
+
+

Handling border cases

+
+
+
    +
  • Continuing our Fibonacci test case, we need to add handling for n that is less than 0 or more than a set amount. Then we need to add another test method and two test cases to test our new functionality.
    • +
+
    +
  1. Let's start with modifying our existing Recursive function, as shown here:
    2. +
+
+
+
public const int MinDepth = 0;
+public const int MaxDepth = 10000;
+
+public static long Recursive(
+    int n,
+    long previous = 0,
+    long current = 0,
+    int counter = 0
+)
+{
+    if (n < MinDepth || n > MaxDepth)
+        throw new ArgumentOutOfRangeException(
+          "n is out of bounds!"
+        );
+    
+    return counter == n ?
+        current :
+        Recursive(
+          n,
+          current,
+          Math.Max(previous + current, 1),
+          counter + 1
+        );
+}
+
+
+
+
+
+

Testing that everything works

+
    +
  • Now our Recursive function throws an ArgumentOutOfRange exception when n is less than 0 or more than 10000.
    • +
+
    +
  1. +

    Let's test that everything works as expected by adding a new test method and a few test cases:

    +
    [TestCase(int.MaxValue)]
+[TestCase(-1)]
+public void Recursive_WhenNIsOutOfBounds_ShouldThrowArgumentOutOfRange(
+    int n
+)
+    => Assert.Throws<ArgumentOutOfRangeException>(() => FibonacciLib.Recursive(n));
+
    +
    2. +
  2. +

    Rename the other method to Recursive_WhenNIsInBounds_ShouldReturnNthFibonacci

    +
    3. +
+
+
+

Unnecessary arguments

+
    +
  • We have one more problem:
    • +
  • Our Recursive function is a public method that has in total 4 arguments it takes, yet we only test 1 argument (n).
    • +
  • In order for our tests to be complete, we need to test all the arguments a public method takes in.
    • +
  • So, how do we fix this?
    • +
+
+
+

Removing unnecessary arguments

+
+
+
    +
  • The naive way would be to test all the arguments with more test cases.
    • +
  • It's important to realize that we don't have to.
    • +
  • Since we never pass in any other arguments than n from outside Recursive, we could just create another private function that has those arguments, and remove them from our public function.
    • +
+
+
+
public static long Recursive(int nStartingValue)
+{
+    if (nStartingValue < MinDepth
+    || nStartingValue > MaxDepth)
+        throw new ArgumentOutOfRangeException(
+          "n is out of bounds!"
+        );
+    
+    long getRecursive(
+        int n,
+        long previous = 0,
+        long current = 0,
+        int counter = 0)
+        => counter == n ?
+            current :
+            getRecursive(
+              n,
+              current,
+              Math.Max(previous + current, 1),
+              counter + 1
+            );
+    return getRecursive(nStartingValue);
+}
+
+
+
+
+
+

End Result

+
    +
  • Now we have tested all the arguments in our public Recursive function.
    • +
  • We do not need to test private functions such as getRecursive, only the public API has to be tested.
    • +
  • All tests should pass at this point.
    • +
+
+
+

Exercise 2: Unit Testing

+
public static long DoX(int nStartingValue) {
+    if (nStartingValue < 0 || nStartingValue > 150)
+        throw new ArgumentOutOfRangeException("n is out of bounds!");
+    
+    long doY(int n) => n > 0 ? n * doY(n - 1) : 1;
+    
+    return doY(nStartingValue);
+}
+
+
    +
  • Following the previously used architecture as closely as possible, unit test the attached function using NUnit. Answer the following questions:
    • +
+
    +
  1. What does the function that we are testing actually do?
    2. +
  2. Do we have to test doY as well?
    3. +
  3. How would you name DoX so it is descriptive? How about doY?
    4. +
+
+
+

Answers to exercise 2: Unit Testing

+ +
    +
  1. It gets the nth factorial (n!)
    2. +
  2. doY is a private method that gets tested when we test DoX, so no.
    3. +
  3. Example names: DoX TryGetFactorial, doY getFactorial
    4. +
+
+
+

Unit Testing: Final testing code

+
    +
  • +
    [TestCase(0, 1)]
+[TestCase(1, 1)]
+[TestCase(2, 2)]
+[TestCase(3, 6)]
+[TestCase(15, 1307674368000)]
+public void TryGetFactorial_WhenNIsInBounds_ShouldReturnNthFactorial(
+    int n, long expected
+)
+    => Assert.That(expected, Is.EqualTo(TestExerciseLib.TryGetFactorial(n)));
+
    +
    [TestCase(-1)]
+[TestCase(151)]
+public void TryGetFactorial_NIsOutOfBounds_ShouldThrowArgumentOutOfRange(
+    int n
+)
+    => Assert.Throws<ArgumentOutOfRangeException>(
+        () => TestExerciseLib.TryGetFactorial(n)
+    );
+
    +
    • +
+
+
+

Writing testable code

+
    +
  • Writing testable code often means that you're writing good code
    • +
  • Testable units should +
      +
    • be deterministic (functional): same input always produces the same output
      • +
    • be loosely coupled: components have a loose and limited connection to other components in the system, meaning changes in one component should not require changes to many others
      • +
    • abide to the Single Responsibility Principle: a function, method or component is only responsible for one purpose
      • +
    +
    • +
+
+
+

Writing testable code: Example 1

+
+
+
    +
  • Can GetCurrentSeason() be unit tested? Why / why not?
    • +
  • How to fix the problems, if any?
    • +
+
+
+
public static class Season
+{
+    public static string GetCurrentSeason() {
+        DateTime currentTime = DateTime.Now;
+        if (currentTime.Month > 11 && currentTime.Month < 3)
+            return "Winter";
+        else if (currentTime.Month < 6)
+            return "Spring";
+        else if (currentTime.Month < 9)
+            return "Summer";
+        else
+            return "Fall";
+    }
+}
+
+
+
+
+
+

Writing testable code: Example 1 fixed

+
+
+
    +
  • Now the method is purely functional and deterministic +
      +
    • Will return the same value for every call with equal arguments
      • +
    +
    • +
  • GetCurrentSeason() can now easily be tested as shown next
    • +
+
+
+
public static class Season {
+    public static string GetCurrentSeason(
+        DateTime currentTime
+    ) {
+        if (currentTime.Month > 11 && currentTime.Month < 3)
+            return "Winter";
+        else if (currentTime.Month < 6)
+            return "Spring";
+        else if (currentTime.Month < 9)
+            return "Summer";
+        else
+            return "Fall";
+    }
+}
+
+
+
+
+
+
public class GetCurrentSeasonTest
+{
+    [TestCase(12, "Winter")]
+    [TestCase(3, "Spring")]
+    [TestCase(6, "Summer")]
+    [TestCase(9, "Fall")]
+    public void GetCurrentSeason_ForFirstMonthsOfSeasons_ReturnsCorrectSeason(
+        int month,
+        string season
+    )
+    {
+        Assert.That(season,
+          Is.EqualTo(
+            Season.GetCurrentSeason(new DateTime(2020, month, 1))
+          )
+        );
+    }
+}
+
+
+
+
+

+
+
    +
  • The tests fail, as shown above.
    • +
+
+
+
+
+
    +
  • Test results indicate there is something wrong with the code path that should return Winter
    if (currentTime.Month > 11 && currentTime.Month < 3)
+
    +
    • +
  • The line should be changed to
    if (currentTime.Month > 11 || currentTime.Month < 3)
+
    +
    • +
+
+
+

+
+
+
+
+

Writing testable code: Example 2

+
    +
  • Let's look at a HeatingUnit class that returns a heating setting based on the current season. The current date problem is back again:
    public class HeatingUnit
+{
+    public string GetHeatingSetting()
+    {
+        DateTime currentDate = DateTime.Now;
+        if (Season.GetCurrentSeason(currentDate) == "Winter")
+            return "HEAT_SETTING_HIGH";
+        else if (Season.GetCurrentSeason(currentDate) == "Summer")
+            return "HEAT_SETTING_OFF";
+        else
+            return "HEAT_SETTING_MEDIUM";
+    }
+}
+
    +
    • +
+
+
+

Writing testable code: Example 2 continued

+
    +
  • Instead of taking the current date initialization even higher in the class hierarchy, let's make a service that returns the current date
    public interface IDateTimeProvider
+{
+    DateTime GetDateTime();
+}
+
    +
    • +
  • After this the service can be injected into HeatingUnit with constructor injection
    • +
  • Code example shown next
    • +
+
+
+
public class HeatingUnit
+{
+    private readonly IDateTimeProvider _dateTimeProvider;
+    
+    public HeatingUnit(IDateTimeProvider dateTimeProvider)
+    {
+        _dateTimeProvider = dateTimeProvider;
+    }
+    
+    public string GetHeatingSetting() {
+        if (Season.GetCurrentSeason(_dateTimeProvider.GetDateTime()) == "Winter")
+            return "HEAT_SETTING_HIGH";
+        else if (Season.GetCurrentSeason(_dateTimeProvider.GetDateTime()) == "Summer")
+            return "HEAT_SETTING_OFF";
+        else
+            return "HEAT_SETTING_MEDIUM";
+    }
+}
+
+
+
+
    +
  • +

    For testing, a fake DateTimeProvider service can be used for injecting the HeatingUnit with any date:

    +
    class FakeDateTimeProvider : IDateTimeProvider
+{
+    public DateTime Date { get; set; }
+    public DateTime GetDateTime() => Date;
+}
+
    +
    • +
  • +

    Using this kind of a fake service instead of the real one is called mocking

    +
    • +
  • +

    In the real application the provider would return the real current date

    +
    • +
+
+
+
class SetHeatingSettingTest
+{
+    [TestCase(12, "HEAT_SETTING_HIGH")]
+    [TestCase(3, "HEAT_SETTING_MEDIUM")]
+    [TestCase(6, "HEAT_SETTING_OFF")]
+    [TestCase(9, "HEAT_SETTING_MEDIUM")]
+    public void SetHeatingSetting_ForFirstMonthsOfSeasons_ShouldReturnCorrectSetting(
+        int month,
+        string setting
+    )
+    {
+        FakeDateTimeProvider timeProvider
+            = new FakeDateTimeProvider { Date = new DateTime(2020, month, 1) };
+        HeatingUnit heatingUnit = new HeatingUnit(timeProvider);
+
+        Assert.That(setting, Is.EqualTo(heatingUnit.GetHeatingSetting()));
+    }
+}
+
+
+
+
+


+Tests should pass!

+
+
+
+

Testing with Postman

+
    +
  • So far, we have created individual request with Postman to see what the response of an API is for each request
    • +
  • This is not testing an API, this is exploring an API
    • +
  • Tests in Postman are inserted in the Tests tab +
      +
    • Postman uses the Chai.js testing library for creating tests
      +
      • +
    +
    • +
+
+
+
    +
  • To get started immediately, you can select snippets from the right
    • +
  • Let's select the Status code: Code is 200 snippet
    +
    • +
+
+
+
    +
  • Successfully run tests show up in green in the Test Result tab of the response:
    +
    • +
+
+
+
    +
  • The official Postman web page is a good starting point for learning testing with Postman
    • +
  • Using pm.expect will give a bit more info about the test:
    +
    • +
+
+
+
    +
  • The response to /serverinfo has the following body:
    {
+    "appSettings": {
+        "applicationUrl": "http://localhost:63741",
+        "aspNetCoreEnvironment": "Development"
+    },
+    "serverStatus": {
+        "dbConnectionStatus": "Connected"
+    }
+}
+
    +
    • +
  • Let's make a test to see whether the server is connected to the database
    • +
+
+
+
    +
  • Variables can be declared within the test:
    +
    • +
+
+
+

Exercise 3: Testing with Postman

+ +
    +
  1. Create a new collection in Postman by selecting the Collections tab and clicking New Collection. Name it CourseAPI Tests. Launch the CourseAPI you have developed during the lectures, and make sure the course_db database is connected
    2. +
  2. Create a new GET request for the URI /api/courses. Add a test: status code of the request should be 200. Save the request to the CourseAPI Tests collection.
    3. +
  3. Create a new GET request for the URI /api/courses/999999999. Add a test: status code of the request should be 404. Save the request to the CourseAPI Tests collection.
    4. +
+
+
+
    +
  1. Create a new POST request for the URI /api/courses. Set the headers and the content correctly, but set the value of credits to 100. Add a test; status code should be 400. Save the request to the CourseAPI Tests collection.
    2. +
  2. Hover on CourseAPI Tests collection, click the arrow and click Run. From the opened window, scroll down and click Run CourseAPI Tests. This will create all the requests in your collection.
    3. +
+
+
+

CI/CD/CT

+
    +
  • CI stands for Continuous Integration +
      +
    • Each change in code triggers a build-and-test sequence for the given project
      • +
    • Goal is to have a consistent and automated way to build, package, and test applications
      • +
    +
    • +
  • CD stands for Continuous Delivery +
      +
    • Automates the delivery of applications to selected environments, such as Azure, AWS, GCP
      • +
    +
    • +
  • Both require Continuous Testing, which includes automated regression, performance and other tests
    • +
+
+
\ No newline at end of file diff --git a/8-testing.md b/8-testing.md index e78abd0..1de3357 100644 --- a/8-testing.md +++ b/8-testing.md @@ -1,817 +1,814 @@ +--- +marp: true +paginate: true +math: mathjax +theme: buutti +title: 8. Testing +--- + # Testing -![](imgs/9-testing_0.png) + + ---- +## Contents + +- [Introduction to testing](#introduction-to-testing) +- [C# Testing Frameworks](#c-testing-frameworks) +- [An extensive unit test example](#an-extensive-unit-test-example) +- [Writing testable code](#writing-testable-code) +- [Testing with Postman](#testing-with-postman) +- [CI/CD/CT](#cicdct) + +## Introduction to testing + +### What is testing? + +* Testing is much more than just trying out manually if the program works! +* [Manual testing](https://en.wikipedia.org/wiki/Manual_testing): acting as an end user of the application +* [Automated testing](https://en.wikipedia.org/wiki/Test_automation): using software to emulate the end user, or testing the functionality of the code itself +* Proving that the program is working as intended during development on a... + * ...***micro level***: individual pieces of code, such as methods + * ...***macro level***: larger functional components, such as services +* *All possible paths in a piece of code should be tested* + * i.e., [test coverage](https://en.wikipedia.org/wiki/Code_coverage) should be close to 100% + +### Automated testing: Why? -* What is testing? - * Proving that the program is working as intended during development on a - * micro-level: individual pieces of code, such as methods - * macro-level: larger functional components, such as services - * Automated testing & manual testing - * All possible paths in a piece of code should be tested * Why do automated testing? - * Sets ground truths that can be relied on during development: "With this kind of input, this should happen" - * No need to take the programmers word for it - * Ensures code functionality before moving on, "finding bugs before debugging" + * Sets ground truths that can be relied on during development: + * *"With this kind of input, this should happen"* + * (No need to take the programmers word for it) + * Ensures code functionality before moving on + * Ideally finding bugs before starting to debug * Guides developers to create more testable and thus more readable, interchangeable and overall better code + * [Test-driven development (TDD)](https://en.wikipedia.org/wiki/Test-driven_development) is based on writing tests first and code later! -# Automated Testing +### Types of automated testing * Unit testing * Testing individual units (methods, classes) within a project - * "input x produces output y" - * Each unit has to be entirely independent and deterministic for it to be unit testable! - * Deterministic = Always the same result with the same input -* Integration Testing + * *"input x produces output y"* + * Each unit has to be entirely independent and ***deterministic*** for it to be unit testable! + * Deterministic: Always the same result with the same input +* Integration testing * Testing complete modules and the interaction between them within a project * Closer to real life scenarios +* End-to-end (E2E) testing + * Testing the whole program by emulating the end user's behaviour + +### Test type comparison + +* Unit tests are faster to write, E2E tests slowest -# C# Testing Frameworks +
+ +![w:800px](https://microsoft.github.io/code-with-engineering-playbook/automated-testing/cdc-testing/images/testing-pyramid.png) + +
+ +## C# Testing Frameworks * Three main testing frameworks: - * MSTest - * NUnit - * XUnit + * `MSTest` + * `NUnit` + * `XUnit` * You can install these using NuGet, or in the newer versions of Visual Studio you can create a new testing project -* NUnit is the most popular of the three, __and will be used in this lecture__ +* NUnit is the most popular of the three, and will be used in this lecture -# MSTest +### MSTest * Built-in to most versions of Visual Studio since 2005 * Simple to use, easy structure - * TestClass for denoting classes containing unit tests - * TestMethod for denoting unit test methods + * `TestClass` for denoting classes containing unit tests + * `TestMethod` for denoting unit test methods -# NUnit +### NUnit * Most popular C# unit testing framework * Very similar structure to MSTest tests - * Instead of TestClass, we use __TestFixture__ - * Instead of TestMethod, we use __Test / TestCase__ -* Has some advantages to MSTest or xUnit tests, for example: + * Instead of `TestClass`, we use `TestFixture` + * Instead of `TestMethod`, we use `Test` / `TestCase` +* Has some advantages to MSTest or XUnit tests, for example: * Ability to test a single test method using multiple different arguments -# XUnit - -Free, open-source unit testing tool for .NET framework and .NET core - -Made by the creator of NUnit v2 +### XUnit -You can test C#, F#, VB.NET and other .NET languages +* Free, open-source unit testing tool for .NET framework and .NET core +* Made by the creator of NUnit v2 +* You can test C#, F#, VB.NET and other .NET languages +* Part of the .NET Foundation +* Unique and different style -Part of the .NET Foundation +### Note about test case naming -Unique and different style +* In C# unit testing, the most commonly followed naming convention for test cases/methods is as follows: + ```csharp + FunctionNameThatIsTested_Condition_ShouldDoX + ``` +* For example: + ```csharp + Fibonacci_WhenNIs0_ShouldReturn0 + ``` -# Test Case Naming Conventions +## An extensive unit test Example -In C# unit testing, the most commonly followed naming convention for test cases/methods is as follows: - -FunctionNameThatIsTested_Condition_ShouldDoX - -For example: - -Fibonacci_WhenNIs0_ShouldReturn0 - -# Solution & Sample Project +### Creating a solution & sample project First, we need a solution and a sample project. -In Visual Studio, create a class library that targets __.NET Standard__ . +
+
-Call the project "Fibonacci.Library", and solution "Fibonacci". Remember to uncheck " _Place solution and project in the same directory_ " if it isn't unchecked already. +1) In Visual Studio, create a ***Class library*** that targets *__.NET Standard__*. +2) Name the project `Fibonacci.Library` and the solution `Fibonacci`. Remember to uncheck _Place solution and project in the same directory_ if it isn't unchecked already. -# Solution & Sample Project (continued) +
+
![](imgs/9-testing_1.png) -Next, rename the default "Class1.cs" file to "Fibonacci.cs". - -Allow Visual Studio to also rename the class itself to Fibonacci. - -Make the class public and static. - -Now, let's __ __ add a recursive fibonacci function, as shown in the next slide: - -using System; - -namespace Fibonacci.Library { - -public static class Fibonacci { - -public static long Recursive( - -int n, - -long previous = 0, - -long current = 0, - -int counter = 0 - -) { - -return counter == n ? - -previous + current : - -Recursive(n, current, Math.Max(previous + current, 1), counter + 1); - -} - -} - -} - -# Adding a Testing Project - -Now we need to add a testing project to our solution. - -Right click your solution, and add a new project. Select the NUnit project for __.NET Core__ , and call it "Fibonacci.UnitTests". - -Rename the created class file to "Fibonacci_Tests.cs", and the class to "Fibonacci_Tests" - -# Adding a Testing Project (continued) - -We then need to link the Fibonacci.Library project to Fibonacci.UnitTests. - -Right click Fibonacci.UnitTests, and select Add > Reference…, and under "Projects" select Fibonacci.Library. Press OK. - -Now we have linked Fibonacci.Library to Fibonacci.UnitTests, and we can begin testing our Recursive-function in the Fibonacci-class. - -# Test Fixture For a Class - -Since we have many things called "Fibonacci", we have to import our Fibonacci-class by using an alias: - -using FibonacciLib = Fibonacci.Library.Fibonacci; - -Put this to the top of your Fibonacci_Tests.cs file, underneath using NUnit.Framework;. - -# Test Fixture For a Class (continued) - -Now, add [TestFixture] on top of our class, like so: - -using NUnit.Framework; - -using FibonacciLib = Fibonacci.Library.Fibonacci; - -namespace Fibonacci.UnitTests { - -[TestFixture] - -public class Fibonacci_Tests { +
+
-This denotes that this class contains our unit tests. +### Adding the function to be tested -Then, inside our class Fibonacci_Tests, let's create our first method to test the Recursive-function, as shown in the next slide: +
+
-using NUnit.Framework; +3) Rename the default `Class1.cs` file to `Fibonacci.cs`. +4) Allow Visual Studio to also rename the class itself to `Fibonacci`. + * Make the class `public` and `static`. +5) Add a *recursive fibonacci function* shown here. -using FibonacciLib = Fibonacci.Library.Fibonacci; +
+
-namespace Fibonacci.UnitTests { - -[TestFixture] - -public class Fibonacci_Tests { - -[Test] - -public void Recursive_WhenNIs0_ShouldReturn0() - -=> Assert.That(0, Is.EqualTo(FibonacciLib.Recursive(n: 0))); - -} +```csharp +using System; +namespace Fibonacci.Library +{ + public static class Fibonacci + { + public static long Recursive( + int n, + long previous = 0, + long current = 0, + int counter = 0 + ) + { + return counter == n ? + previous + current : + Recursive( + n, + current, + Math.Max(previous + current, 1), + counter + 1 + ); + } + } } +``` -* Things to take into consideration: -* When passing in function arguments, to avoid magic numbers and to improve code readability, use the syntax _argumentName: value_ - * Magic numbers = using numbers as input as-is without explanation, for example without assigning to a well-named variable -* Assert.That is a function (but not the only function) that determines whether the test passes. - * Its first argument is the expected value. In the second argument you give a constraint for the actual output, i.e. value is equal to, greater than, starts with, and [many others](https://docs.nunit.org/articles/nunit/writing-tests/constraints/Constraints.html) . - * Do not mix the arguments up! If you do, the errors you get do not make any sense. +
+
-# Running Tests -Now, let's open the built-in test runner in Visual Studio to run our tests. +### Adding a testing project -On the top of Visual Studio, click "Test", then "Test Explorer". +* Now we need to add a testing project to our solution. +6) Right click your solution, and add a new project. Select the NUnit project for **_.NET Core_**, and call it `Fibonacci.UnitTests`. +7) Rename the created class file to `Fibonacci_Tests.cs`, and the class to `Fibonacci_Tests` -In the test explorer, you should see the test case we created. If not, build the solution, and make sure there are no errors. +### Linking the testing project -Either way, let's run our test case. Press "Run All Tests" in the top-left corner of test explorer. You should see our test case __passing__ . +8) We then need to link the `Fibonacci.Library` project to `Fibonacci.UnitTests`. +9) Right click `Fibonacci.UnitTests`, and select *Add > Reference...*, and under *Projects* select `Fibonacci.Library`. Press OK. -# Adding More Test Cases +* Now we have linked `Fibonacci.Library` to `Fibonacci.UnitTests`, and we can begin testing our `Recursive` function in the `Fibonacci` class. -While there's nothing inherently wrong about our test case, it doesn't cover much of anything. We need to add more test cases to test different values and outputs. +### Importing the class -In other testing frameworks, you would do this by creating more methods. +10) Since we have many things called "Fibonacci", we have to import our `Fibonacci` class by using an alias + * Put this to the top of your file `Fibonacci_Tests.cs`, underneath
`using NUnit.Framework;` + ```csharp + using FibonacciLib = Fibonacci.Library.Fibonacci; + ``` -But with NUnit, we can use our existing method. +### Test fixture for a class -# Adding More Test Cases (continued) +11) Now, add `[TestFixture]` on top of our class, like so: + ```csharp + using NUnit.Framework; + using FibonacciLib = Fibonacci.Library.Fibonacci; -Do the following modifications to our test method: + namespace Fibonacci.UnitTests { + [TestFixture] + public class Fibonacci_Tests { + ``` -[TestCase(0, 0)] +* This denotes that this class contains our unit tests. -public void Recursive_Always_ShouldReturnNthFibonacci( +### Adding the unit test method -int n, +12) Then, inside our class `Fibonacci_Tests`, let's create our first method to test the `Recursive` function, as shown here: + ```csharp + using NUnit.Framework; + using FibonacciLib = Fibonacci.Library.Fibonacci; -long expected + namespace Fibonacci.UnitTests { + [TestFixture] + public class Fibonacci_Tests { + [Test] + public void Recursive_WhenNIs0_ShouldReturn0() + => Assert.That(0, Is.EqualTo(FibonacciLib.Recursive(n: 0))); + } + } + ``` -) +### Things to take into consideration -=> Assert.That(expected, Is.EqualTo(FibonacciLib.Recursive(n))); +* When passing in function arguments, to avoid ***magic numbers*** and to improve code readability, use the syntax `argumentName: value` + * Magic numbers = using numbers as input as-is without explanation, for example without assigning to a well-named variable, or without using a descriptive type +* `Assert.That` is one of many functions that can determine whether the test passes + * Its first argument is the ***expected value***. In the second argument you give a ***constraint*** for the actual output, i.e., value is equal to, greater than, starts with, [etc](https://docs.nunit.org/articles/nunit/writing-tests/constraints/Constraints.html). + * *Do not mix the arguments up!* If you do, the errors you get do not make any sense. -Using TestCase instead of Test, we can pass in arguments to our test case method, and we can use those passed in arguments in our actual tests. +### Running Tests -Run the test cases again, and they should __pass__ . +* Now, let's open the built-in test runner in Visual Studio to run our tests. -But as you might notice, we haven't really changed anything yet. +* On the top of Visual Studio, click *Test*, then *Test Explorer*. +* In the Test Explorer, you should see the test case we created. If not, build the solution, and make sure there are no errors. +* Either way, let's run our test case. Press *Run All Tests* in the top-left corner of Test Explorer. You should see our test case *__passing__*. -Let's add more test cases, like so: +### Adding More Test Cases -[TestCase(0, 0)] +* While there's nothing inherently wrong about our test case, it doesn't cover much of anything. +* We need to add more test cases to test different values and outputs. +* In other testing frameworks, you would do this by creating more methods. +* But with `NUnit`, we can use our *existing* method. -[TestCase(1, 1)] +### Adding a new test -public void Recursive_Always_ShouldReturnNthFibonacci( +1) Do the following modifications to our test method: + ```csharp + [TestCase(0, 0)] + public void Recursive_Always_ShouldReturnNthFibonacci( + int n, + long expected + ) + => Assert.That(expected, Is.EqualTo(FibonacciLib.Recursive(n))); + ``` + * Using `TestCase` instead of `Test`, we can pass in arguments to our test case method, and we can use those passed in arguments in our actual tests. -Now we have 2 test cases with only one test method. Run the tests again, and both cases should __pass__ . +### Adding more tests -Let's add one more TestCase, like so: +2) Run the test cases again, and they should *__pass__*. + * But as you might notice, we haven't really changed anything yet. +3) Let's add more test cases, like so: + ```csharp + [TestCase(0, 0)] + [TestCase(1, 1)] + public void Recursive_Always_ShouldReturnNthFibonacci( + ``` -[TestCase(2, 1)] +4) Now we have 2 test cases with only one test method. Run the tests again, and both cases should *__pass__*. -Run the tests again, and you should see the test case __fail__ . +### Adding a failing test -This is to be expected: the 2nd fibonacci number should be 1, but the actual value we got from our function was 2. +5) Let's add one more TestCase, like so: + ```csharp + [TestCase(2, 1)] + ``` +6) Run the tests again, and you should see the test case *__fail__*. + * This is to be expected: the 2nd fibonacci number should be 1, but the actual value we got from our function was 2. + * This means our function does not work as expected. -This means our function does not work as expected. +### Fixing our code -Let's fix it, as shown in the next slide: +
+
+ +7) Let's fix our function, as seen here. +8) Run the tests again. You should see all *__pass__* now. +9) Add a few more test cases + * (If you don't know the fibonacci sequence, you can check it from the internet.) +
+
+ +```csharp public static long Recursive( - -int n, - -long previous = 0, - -long current = 0, - -int counter = 0 - -) { - -return counter == n ? - -current : // changed from previous + current - -Recursive(n, current, Math.Max(previous + current, 1), counter + 1); - + int n, + long previous = 0, + long current = 0, + int counter = 0 +) +{ + return counter == n ? + current : // changed from previous + current + Recursive( + n, + current, + Math.Max(previous + current, 1), + counter + 1 + ); } +``` -Run the tests again. You should see all __pass __ now. - -Add a few more test cases; if you don't know the fibonacci sequence, you can check it from google. - -# Exercise 1: Unit testing in practice - -* Answer the following questions: -* How many test cases should we ideally have to test our Recursive-function? -* What happens if we pass in n = int.MaxValue (2147483647)? Or other huge numbers? - * How would you fix the function so it works as expected? -* What happens if we pass in n = -1 ? Or other negative numbers? - * How would you fix the function so it works as expected? -* (Answers on the next slide) - -# Unit testing in practice: Exercise - Answers - -* Answers: -* We should cover all the edge cases and code paths with our test cases. We do not need to have many test cases for the same code path. -* Stack Overflow, Int64 Overflow, or the test case just won't run. - * We should have a maximum allowed value for n, throw argument out of range exception if above -* Stack Overflow, since current never can equal n that is less than 0 - * We should have a minimum allowed value for n, throw argument out of range exception if below - -# Testing Exceptions - -To expect exceptions to be thrown in test cases, we can use the method - -Assert.Throws, syntax is as follows: - -Assert.Throws(delegate); - -In practice: +
+
-Assert.Throws(() => TestedFunction(n)); +### Exercise 1: Unit testing in practice + -# Handling Border Cases - -We need to add handling for n that is less than 0 or more than a set amount. Then we need to add another test method and two test cases to test our new functionality. - -Let's start with modifying our existing Recursive-function, as shown in the next slide: - -# Handling Border Cases (continued) +Answer the following questions: +1) How many test cases should we ideally have to test our `Recursive` function? +2) What happens if we pass in `n = int.MaxValue (2147483647)`? + * What about other huge numbers? + * How would you fix the function so it works as expected? +3) What happens if we pass in `n = -1`? + * What about other negative numbers? + * How would you fix the function so it works as expected? + +### Answers to Exercise 1: Unit testing in practice + + +1) We should cover all the edge cases and code paths with our test cases. We do not need to have many test cases for the same code path. +2) `Stack Overflow`, `Int64 Overflow`, or the test case just won't run. + * We should have a maximum allowed value for `n`, throw the ***argument out of range*** exception if above +3) `Stack Overflow`, since `current` can never equal `n` that is less than `0` + * We should have a minimum allowed value for `n`, throw the ***argument out of range*** exception if below + +### Testing exceptions + +* To expect exceptions to be thrown in test cases, we can use the method `Assert.Throws` +* Syntax is as follows: + ```csharp + Assert.Throws(delegate); + ``` +* In practice: + ```csharp + Assert.Throws(() => TestedFunction(n)); + ``` +### Handling border cases + +
+
+ +* Continuing our Fibonacci test case, we need to add handling for `n` that is less than `0` or more than a set amount. Then we need to add another test method and two test cases to test our new functionality. +1) Let's start with modifying our existing `Recursive` function, as shown here: + +
+
+ +```csharp public const int MinDepth = 0; - public const int MaxDepth = 10000; public static long Recursive( - -int n, - -long previous = 0, - -long current = 0, - -int counter = 0 - -) { - -if (n < MinDepth || n > MaxDepth) - -throw new ArgumentOutOfRangeException("n is out of bounds!"); - -return counter == n ? - -current : - -Recursive(n, current, Math.Max(previous + current, 1), counter + 1); - -} - -Now our Recursive-function throws an ArgumentOutOfRange exception when n is less than 0 or more than 10000. Let's test that everything works as expected, by adding a new test method and a few test cases: - -[TestCase(int.MaxValue)] - -[TestCase(-1)] - -public void Recursive_WhenNIsOutOfBounds_ShouldThrowArgumentOutOfRange( - -int n - + int n, + long previous = 0, + long current = 0, + int counter = 0 ) +{ + if (n < MinDepth || n > MaxDepth) + throw new ArgumentOutOfRangeException( + "n is out of bounds!" + ); + + return counter == n ? + current : + Recursive( + n, + current, + Math.Max(previous + current, 1), + counter + 1 + ); +} +``` -=> Assert.Throws(() => FibonacciLib.Recursive(n)); - -__Rename the other method to Recursive_WhenNIsInBounds_ShouldReturnNthFibonacci__ - -# Removing Unnecessary Arguments - -We have one more problem: - -Our Recursive-function is a public method that has in total 4 arguments it takes, yet we only test 1 argument (n). - -In order for our tests to be complete, we need to test all the arguments a public method takes in. - -So, how do we fix this? - -# Removing Unnecessary Arguments (continued) - -The naive way would be to simply test all the arguments with more test cases. - -But it's important to realize that __we don't have to__ . Since we basically never pass in any other arguments than n from outside Recursive, we could just create another __private __ function that has those arguments, and we could remove them from our __public __ Recursive-function. - -Like shown in the next slide: - -public static long Recursive(int nStartingValue) { - -if (nStartingValue < MinDepth || nStartingValue> MaxDepth) - -throw new ArgumentOutOfRangeException("n is out of bounds!"); +
+
-long getRecursive( +### Testing that everything works -int n, +* Now our `Recursive` function throws an `ArgumentOutOfRange` exception when `n` is less than `0` or more than `10000`. +2) Let's test that everything works as expected by adding a new test method and a few test cases: + ```csharp + [TestCase(int.MaxValue)] + [TestCase(-1)] + public void Recursive_WhenNIsOutOfBounds_ShouldThrowArgumentOutOfRange( + int n + ) + => Assert.Throws(() => FibonacciLib.Recursive(n)); + ``` -long previous = 0, +3) Rename the other method to `Recursive_WhenNIsInBounds_ShouldReturnNthFibonacci` -long current = 0, +### Unnecessary arguments -int counter = 0) +* We have one more problem: +* Our `Recursive` function is a `public` method that has in total 4 arguments it takes, yet we only test 1 argument (`n`). +* In order for our tests to be complete, we need to test all the arguments a `public` method takes in. +* So, how do we fix this? -=> counter == n ? +### Removing unnecessary arguments -current : +
+
-getRecursive(n, current, Math.Max(previous + current, 1), counter + 1); +* The naive way would be to test all the arguments with more test cases. +* It's important to realize that *__we don't have to__*. +* Since we never pass in any other arguments than `n` from outside `Recursive`, we could just create another `private` function that has those arguments, and remove them from our public function. -return getRecursive(nStartingValue); +
+
+```csharp +public static long Recursive(int nStartingValue) +{ + if (nStartingValue < MinDepth + || nStartingValue > MaxDepth) + throw new ArgumentOutOfRangeException( + "n is out of bounds!" + ); + + long getRecursive( + int n, + long previous = 0, + long current = 0, + int counter = 0) + => counter == n ? + current : + getRecursive( + n, + current, + Math.Max(previous + current, 1), + counter + 1 + ); + return getRecursive(nStartingValue); } +``` -# End Result - -Now we have tested all the arguments in our public Recursive function. +
+
-We do not need to test private functions such as getRecursive, only the public API has to be tested. +### End Result -All tests should __pass __ by this point. +* Now we have tested all the arguments in our public `Recursive` function. +* We do not need to test private functions such as `getRecursive`, only the public API has to be tested. +* All tests should *__pass__* at this point. -# Exercise 2: Unit Testing - -Following the previously used architecture as closely as possible, unit test the following function using NUnit: - -Questions on the next slide. +### Exercise 2: Unit Testing + +```csharp public static long DoX(int nStartingValue) { - -if (nStartingValue < 0 || nStartingValue > 150) - -throw new ArgumentOutOfRangeException("n is out of bounds!"); - -long doY(int n) => n > 0 ? n * doY(n - 1) : 1; - -return doY(nStartingValue); - + if (nStartingValue < 0 || nStartingValue > 150) + throw new ArgumentOutOfRangeException("n is out of bounds!"); + + long doY(int n) => n > 0 ? n * doY(n - 1) : 1; + + return doY(nStartingValue); } - -# Unit Testing: Exercise - Questions - -Answer the following questions: - -What does the function that we are testing actually do? - -Do we have to test doY as well? - -How would you name DoX so it is descriptive? How about doY? - -# Unit Testing: Exercise - Answers - -* It gets the nth factorial (n!) -* doY is a private method that gets tested when we test DoX, so no. -* DoX = TryGetFactorial, doY = getFactorial - * __Name your functions to these__ - -# Unit Testing - Final Testing Code Part 1 - -[TestCase(0, 1)] - -[TestCase(1, 1)] - -[TestCase(2, 2)] - -[TestCase(3, 6)] - -[TestCase(15, 1307674368000)] - -public void TryGetFactorial_WhenNIsInBounds_ShouldReturnNthFactorial( - -int n, long expected - -) - -=> Assert.That(expected, Is.EqualTo(TestExerciseLib.TryGetFactorial(n))); - -[TestCase(-1)] - -[TestCase(151)] - -public void TryGetFactorial_NIsOutOfBounds_ShouldThrowArgumentOutOfRange( - -int n - -) - -=> Assert.Throws( - -() => TestExerciseLib.TryGetFactorial(n) - -); - -# Writing Testable Code - -* Writing testable code equals writing good code +``` + +* Following the previously used architecture as closely as possible, unit test the attached function using `NUnit`. Answer the following questions: +1) What does the function that we are testing actually *do*? +2) Do we have to test `doY` as well? +3) How would you name `DoX` so it is descriptive? How about `doY`? + +### Answers to exercise 2: Unit Testing + + +1) It gets the nth factorial (`n!`) +2) `doY` is a private method that gets tested when we test `DoX`, so no. +3) Example names: `DoX` $\Rightarrow$ `TryGetFactorial`, `doY` $\Rightarrow$ `getFactorial` + + +### Unit Testing: Final testing code + +* + ```csharp + [TestCase(0, 1)] + [TestCase(1, 1)] + [TestCase(2, 2)] + [TestCase(3, 6)] + [TestCase(15, 1307674368000)] + public void TryGetFactorial_WhenNIsInBounds_ShouldReturnNthFactorial( + int n, long expected + ) + => Assert.That(expected, Is.EqualTo(TestExerciseLib.TryGetFactorial(n))); + ``` + + ```csharp + [TestCase(-1)] + [TestCase(151)] + public void TryGetFactorial_NIsOutOfBounds_ShouldThrowArgumentOutOfRange( + int n + ) + => Assert.Throws( + () => TestExerciseLib.TryGetFactorial(n) + ); + ``` + +## Writing testable code + +* Writing testable code often means that you're writing good code * Testable units should * be deterministic (functional): same input always produces the same output - * be loosely coupled: components have a loose and limited connection to other components in the system, meaning changes in one component should not require changes to many others - * abide to the _Single Responsibility Principle: _ a function, method or component is only responsible for one purpose ---- - -https://en.wikipedia.org/wiki/Loose_coupling - -# Writing Testable Code - Example 1 - -public static class Season - -{ - -public static string GetCurrentSeason() - -{ - -DateTime currentTime = DateTime.Now; - -if (currentTime.Month > 11 && currentTime.Month < 3) - -return "Winter"; - -else if (currentTime.Month < 6) + * be [loosely coupled](https://en.wikipedia.org/wiki/Loose_coupling): components have a loose and limited connection to other components in the system, meaning changes in one component should not require changes to many others + * abide to the *__Single Responsibility Principle__*: a function, method or component is only responsible for one purpose -return "Spring"; +### Writing testable code: Example 1 -else if (currentTime.Month < 9) +
+
-return "Summer"; +* Can `GetCurrentSeason()` be unit tested? Why / why not? +* How to fix the problems, if any? -else - -return "Fall"; - -} - -} - -Can GetCurrentSeason() be unit tested? - -Why / why not? - -How to fix the problems, if any? - -# Writing Testable Code - Example 1 (continued) +
+
+```csharp public static class Season - { + public static string GetCurrentSeason() { + DateTime currentTime = DateTime.Now; + if (currentTime.Month > 11 && currentTime.Month < 3) + return "Winter"; + else if (currentTime.Month < 6) + return "Spring"; + else if (currentTime.Month < 9) + return "Summer"; + else + return "Fall"; + } +} +``` -public static string GetCurrentSeason( - -DateTime currentTime - -) - -{ - -if (currentTime.Month > 11 && currentTime.Month < 3) - -return "Winter"; - -else if (currentTime.Month < 6) - -return "Spring"; - -else if (currentTime.Month < 9) - -return "Summer"; - -else - -return "Fall"; +
+
-} +### Writing testable code: Example 1 fixed -} +
+
* Now the method is purely functional and deterministic * Will return the same value for every call with equal arguments -* GetCurrentSeason() can now easily be tested as shown in the next slide - -public class GetCurrentSeasonTest - -{ - -[TestCase(12, "Winter")] - -[TestCase(3, "Spring")] - -[TestCase(6, "Summer")] - -[TestCase(9, "Fall")] - -public void GetCurrentSeason_ForFirstMonthsOfSeasons_ReturnsCorrectSeason( - -int month, - -string season - -) - -{ - -Assert.That(season, Is.EqualTo(Season.GetCurrentSeason(new DateTime(2020, month, 1)))); - +* `GetCurrentSeason()` can now easily be tested as shown next + +
+
+ +```csharp +public static class Season { + public static string GetCurrentSeason( + DateTime currentTime + ) { + if (currentTime.Month > 11 && currentTime.Month < 3) + return "Winter"; + else if (currentTime.Month < 6) + return "Spring"; + else if (currentTime.Month < 9) + return "Summer"; + else + return "Fall"; + } } +``` -} - -![](imgs/9-testing_2.png) - -Test results indicate there is something wrong with the code path that should return "Winter" - -if (currentTime.Month > 11 && currentTime.Month < 3) - -The line is changed to - -if (currentTime.Month > 11 || currentTime.Month < 3) - -![](imgs/9-testing_3.png) - -# Writing Testable Code - Example 2 - -Let's look at a HeatingUnit class, which returns a heating setting based on the current season. The current date problem is back again: - -public class HeatingUnit +
+
-{ - -public string GetHeatingSetting() +--- +```csharp +public class GetCurrentSeasonTest { - -DateTime currentDate = DateTime.Now; - -if (Season.GetCurrentSeason(currentDate) == "Winter") - -return "HEAT_SETTING_HIGH"; - -else if (Season.GetCurrentSeason(currentDate) == "Summer") - -return "HEAT_SETTING_OFF"; - -else - -return "HEAT_SETTING_MEDIUM"; - -} - + [TestCase(12, "Winter")] + [TestCase(3, "Spring")] + [TestCase(6, "Summer")] + [TestCase(9, "Fall")] + public void GetCurrentSeason_ForFirstMonthsOfSeasons_ReturnsCorrectSeason( + int month, + string season + ) + { + Assert.That(season, + Is.EqualTo( + Season.GetCurrentSeason(new DateTime(2020, month, 1)) + ) + ); + } } +``` -# Writing Testable Code - Example 2 (continued) - -Instead of taking the current date initialization even higher in the class hierarchy, let's make a service that returns the current date - -public interface IDateTimeProvider +--- -{ +
-DateTime GetDateTime(); +![](imgs/9-testing_2.png) -} +
-After this the service can be injected into HeatingUnit with constructor injection (next slide) +* The tests fail, as shown above. -public class HeatingUnit +--- -{ +
+
-private readonly IDateTimeProvider _dateTimeProvider; +* Test results indicate there is something wrong with the code path that should return `Winter` + ```csharp + if (currentTime.Month > 11 && currentTime.Month < 3) + ``` +* The line should be changed to + ```csharp + if (currentTime.Month > 11 || currentTime.Month < 3) + ``` -public HeatingUnit(IDateTimeProvider dateTimeProvider) +
+
-{ +![](imgs/9-testing_3.png) -_dateTimeProvider = dateTimeProvider; +
+
+ +### Writing testable code: Example 2 + +* Let's look at a `HeatingUnit` class that returns a heating setting based on the current season. The current date problem is back again: + ```csharp + public class HeatingUnit + { + public string GetHeatingSetting() + { + DateTime currentDate = DateTime.Now; + if (Season.GetCurrentSeason(currentDate) == "Winter") + return "HEAT_SETTING_HIGH"; + else if (Season.GetCurrentSeason(currentDate) == "Summer") + return "HEAT_SETTING_OFF"; + else + return "HEAT_SETTING_MEDIUM"; + } + } + ``` + +### Writing testable code: Example 2 continued + +* Instead of taking the current date initialization even higher in the class hierarchy, let's make a ***service*** that returns the current date + ```csharp + public interface IDateTimeProvider + { + DateTime GetDateTime(); + } + ``` +* After this the service can be injected into `HeatingUnit` with constructor injection +* Code example shown next -} - -public string GetHeatingSetting() +--- +```csharp +public class HeatingUnit { - -if (Season.GetCurrentSeason(_dateTimeProvider.GetDateTime()) == "Winter") - -return "HEAT_SETTING_HIGH"; - -else if (Season.GetCurrentSeason(_dateTimeProvider.GetDateTime()) == "Summer") - -return "HEAT_SETTING_OFF"; - -else - -return "HEAT_SETTING_MEDIUM"; - + private readonly IDateTimeProvider _dateTimeProvider; + + public HeatingUnit(IDateTimeProvider dateTimeProvider) + { + _dateTimeProvider = dateTimeProvider; + } + + public string GetHeatingSetting() { + if (Season.GetCurrentSeason(_dateTimeProvider.GetDateTime()) == "Winter") + return "HEAT_SETTING_HIGH"; + else if (Season.GetCurrentSeason(_dateTimeProvider.GetDateTime()) == "Summer") + return "HEAT_SETTING_OFF"; + else + return "HEAT_SETTING_MEDIUM"; + } } +``` -} - -For testing, a fake DateTimeProvider service can be used for injecting the HeatingUnit with any date: - -class FakeDateTimeProvider : IDateTimeProvider - -{ +--- -public DateTime Date { get; set; } +* For testing, a fake `DateTimeProvider` service can be used for injecting the `HeatingUnit` with any date: -public DateTime GetDateTime() => Date; + ```csharp + class FakeDateTimeProvider : IDateTimeProvider + { + public DateTime Date { get; set; } + public DateTime GetDateTime() => Date; + } + ``` -} +* Using this kind of a fake service instead of the real one is called [mocking](https://en.wikipedia.org/wiki/Mock_object) +* In the real application the provider would return the real current date -In the real application the provider would return the real current date +--- +```csharp class SetHeatingSettingTest - { - -[TestCase(12, "HEAT_SETTING_HIGH")] - -[TestCase(3, "HEAT_SETTING_MEDIUM")] - -[TestCase(6, "HEAT_SETTING_OFF")] - -[TestCase(9, "HEAT_SETTING_MEDIUM")] - -public void SetHeatingSetting_ForFirstMonthsOfSeasons_ShouldReturnCorrectSetting( - -int month, - -string setting - -) - -{ - -FakeDateTimeProvider timeProvider - -= new FakeDateTimeProvider { Date = new DateTime(2020, month, 1) }; - -HeatingUnit heatingUnit = new HeatingUnit(timeProvider); - -Assert.That(setting, Is.EqualTo(heatingUnit.GetHeatingSetting())); - + [TestCase(12, "HEAT_SETTING_HIGH")] + [TestCase(3, "HEAT_SETTING_MEDIUM")] + [TestCase(6, "HEAT_SETTING_OFF")] + [TestCase(9, "HEAT_SETTING_MEDIUM")] + public void SetHeatingSetting_ForFirstMonthsOfSeasons_ShouldReturnCorrectSetting( + int month, + string setting + ) + { + FakeDateTimeProvider timeProvider + = new FakeDateTimeProvider { Date = new DateTime(2020, month, 1) }; + HeatingUnit heatingUnit = new HeatingUnit(timeProvider); + + Assert.That(setting, Is.EqualTo(heatingUnit.GetHeatingSetting())); + } } - -} - -![](imgs/9-testing_4.png) +``` --- -Näytä demo - -# Testing with Postman +
-* So far, you have created individual request with Postman to see what the response of an API is for each request -* This is __not testing __ an API, this is __exploring __ an API -* Tests in Postman are inserted in the _Tests _ tab - * Postman uses __Chai.js__ testing library for creating tests - -![](imgs/9-testing_5.png) - -To get started immediately, you can select snippets from the right - -Let's select the "Status code: Code is 200" snippet - -![](imgs/9-testing_6.png) - -Successfully ran tests show up in green in the Test Result tab of the response: - -![](imgs/9-testing_7.png) - -[The official Postman web page](https://learning.postman.com/docs/writing-scripts/test-scripts/) is a good starting point for learning testing with Postman - -Using pm.expect will give a bit more info about the test: - -![](imgs/9-testing_8.png) - -The response to /serverinfo has the following body: +![](imgs/9-testing_4.png) +Tests should pass! -{ +
-"appSettings": { +## Testing with Postman -"applicationUrl": "http://localhost:63741", +* So far, we have created individual request with Postman to see what the response of an API is for each request +* This is not testing an API, this is *__exploring__* an API +* Tests in Postman are inserted in the _Tests_ tab + * Postman uses the *__Chai.js__* testing library for creating tests + ![](imgs/9-testing_5.png) -"aspNetCoreEnvironment": "Development" +--- -}, +* To get started immediately, you can select *snippets* from the right +* Let's select the *Status code: Code is 200* snippet + ![](imgs/9-testing_6.png) -"serverStatus": { +--- -"dbConnectionStatus": "Connected" +* Successfully run tests show up in green in the *Test Result* tab of the response: + ![](imgs/9-testing_7.png) -} +--- -} +* [The official Postman web page](https://learning.postman.com/docs/writing-scripts/test-scripts/) is a good starting point for learning testing with Postman +* Using `pm.expect` will give a bit more info about the test: + ![](imgs/9-testing_8.png) -Let's make a test to see whether the server is connected to the database +--- -Variables can be declared within the test: +* The response to /serverinfo has the following body: + ```json + { + "appSettings": { + "applicationUrl": "http://localhost:63741", + "aspNetCoreEnvironment": "Development" + }, + "serverStatus": { + "dbConnectionStatus": "Connected" + } + } + ``` +* Let's make a test to see whether the server is connected to the database -![](imgs/9-testing_9.png) +--- -# Exercise 3: Testing with Postman +* Variables can be declared within the test: + ![](imgs/9-testing_9.png) -Create a new collection in Postman by selecting the _Collections _ tab and clicking _New Collection._ Name it _CourseAPI Tests. _ Launch the CourseAPI you have developed during the lectures, and make sure the course_db database is connected +### Exercise 3: Testing with Postman + -Create a new GET request for the URI /api/courses. Add a test: status code of the request should be 200. Save the request to the _CourseAPI Tests_ collection +1) Create a new collection in Postman by selecting the *Collections* tab and clicking *New Collection*. Name it `CourseAPI Tests`. Launch the CourseAPI you have developed during the lectures, and make sure the `course_db` database is connected +2) Create a new `GET` request for the URI `/api/courses`. Add a test: status code of the request should be 200. Save the request to the `CourseAPI Tests` collection. +3) Create a new `GET` request for the URI `/api/courses/999999999`. Add a test: status code of the request should be 404. Save the request to the `CourseAPI Tests` collection. -Create a new GET request for the URI /api/courses/999999999. Add a test: status code of the request should be 404. Save the request to the _CourseAPI Tests_ collection +--- -Create a new POST request for the URI /api/courses. Set the headers and the content correctly, but set the value of credits to 100. Add a test: status code should be 400. Save the request to the _CourseAPI Tests_ collection + -Hover on CourseAPI Tests collection, click the arrow and click RUN. From the opened window, scroll down and click "Run CourseAPI Tests". This will create all the requests in your collection +4) Create a new `POST` request for the URI `/api/courses`. Set the headers and the content correctly, but set the value of credits to 100. Add a test; status code should be 400. Save the request to the CourseAPI Tests collection. +5) Hover on CourseAPI Tests collection, click the arrow and click *Run*. From the opened window, scroll down and click *Run CourseAPI Tests*. This will create all the requests in your collection. -# CI/CD/CT +## CI/CD/CT -* CI stands for _Continuous Integration_ +* CI stands for *__Continuous Integration__* * Each change in code triggers a build-and-test sequence for the given project - * Goal is to have a consistent and automated way to build, package, and test applications -* CD stands for _Continuous Delivery_ + * Goal is to have a consistent and automated way to build, package, and test applications +* CD stands for *__Continuous Delivery__* * Automates the delivery of applications to selected environments, such as Azure, AWS, GCP -* Both require _Continuous Testing_ , which includes automated regression, performance and other tests +* Both require *__Continuous Testing__*, which includes automated regression, performance and other tests diff --git a/imgs/9-testing_6.png b/imgs/9-testing_6.png index 9ec4762..b5147ae 100644 Binary files a/imgs/9-testing_6.png and b/imgs/9-testing_6.png differ