aspnet-basics/7-databases-with-entity-framework.md

# SQL

* __Language __ to organize and manipulate data in a database
* Originally developed by IBM in the 70's
  * Quickly became the most popular database language
* SELECT id, email
* FROM users
* WHERE first_name = 'Teppo';

# Relational Database Management Systems

* In relational databases, values are stored in  __tables__
  * Each table has  __rows __ and  __columns__
  * Data is displayed in a two-dimensional matrix
* Values in a table are related to each other
  * Values can also be related to values in other tables
* A relational database management system (RDBMS) is a program, which executes queries to relational databases

![](imgs/7-databases-with-entity-framework_0.png)

[https://db-engines.com/en/ranking](https://db-engines.com/en/ranking)

# PostgreSQL

Free open-source, cross-platform relational database management system

Emphasizes extensibility and SQL compliance

Fully ACID-compliant (atomicity, consistency, isolation and durability)

# pgAdmin

Administration and development platform for PostgreSQL

Cross-platform, features a web interface

Basically a control panel application for your PostgreSQL database

# PostgreSQL: Creating a Database

With pgAdmin:

Right-click Servers > my-postgres >  _Databases_

Select  _Create _ >  _Database…_

Insert a name for the database _ _ and hit  _Save_

![](imgs/7-databases-with-entity-framework_1.png)

With psql:

Enter commandCREATE DATABASE <database-name>;

# PostgreSQL: Querying

With pgAdmin:

Right-click  _sqlpractice _ >  _Query tool..._

Insert a query into the  _Query Editor_  and hit  _Execute _ (F5)

![](imgs/7-databases-with-entity-framework_2.png)

![](imgs/7-databases-with-entity-framework_3.png)

---

With psql:

List all available databases with \\l

Connect to the created database with \\c <database-name>

List all tables in the database with \\dt

Type a query and press enter

---

https://drive.google.com/file/d/1E5TqY5yt8DNcWHVkb8gxvBpgLtWB15DC/view?usp=sharing  "provided query"

# Editing Data with pgAdmin

Inspect and edit data in pgAdmin by right-clicking a table and selecting  _View/Edit Data_

![](imgs/7-databases-with-entity-framework_4.png)

![](imgs/7-databases-with-entity-framework_5.png)

Individual values in the table can be directly modified by double clicking the value and then editing the value in the visual user interface

Save the changes with the  _Save Data Changes_  button

![](imgs/7-databases-with-entity-framework_6.png)

# Exercise 1: Preparing Database

Using either PgAdmin or PSQL

Create a new database called sqlpractice

Insert the  [provided query](https://drive.google.com/file/d/1E5TqY5yt8DNcWHVkb8gxvBpgLtWB15DC/view)  to the new database

Verify that the query has created new tables to your database

# Types of queries

Select

Insert

Delete

Update

Create & Drop

# Querying data with SELECT

Syntax:

__SELECT column1, column2, column3 FROM table_name;__

Examples:

SELECT full_name, email FROM users;

SELECT full_name AS name, email FROM users;

SELECT * FROM users;

# Filtering data with WHERE

Syntax:

__SELECT column1, column2 FROM table_name WHERE condition;__

Text is captured in  _single quotes_ .

LIKE condition uses  _%_  sign as wildcard.

IS and IS NOT are also valid operators.

Example:

SELECT full_name FROM users

WHERE full_name = 'Teppo Testaaja';

SELECT * FROM books WHERE name LIKE '%rr%';

SELECT * FROM books WHERE author IS NOT null;

# Ordering data with ORDER BY

Syntax:

__SELECT column1 FROM table_name ORDER BY column1 ASC;__

Examples:

SELECT full_name FROM users

ORDER BY full_name ASC;

SELECT full_name FROM users

ORDER BY full_name DESC

# Combining with JOIN

Also known as INNER JOIN

Corresponds to intersection from set theory

![](imgs/7-databases-with-entity-framework_7.png)

# JOIN examples

SELECT

users.id, users.full_name, borrows.id,

borrows.user_id, borrows.due_date, borrows.returned_at

FROM users

JOIN borrows ON

users.id = borrows.user_id;

SELECT

U.full_name AS name,

B.due_date AS due_date,

B.returned_at AS returned_at

FROM users AS U

JOIN borrows AS B ON

U.id = B.user_id;

# Combining with LEFT JOIN

Also known as LEFT OUTER JOIN

Example:

SELECT

U.full_name AS name,

B.due_date AS due_date,

B.returned_at AS returned_at

FROM users AS U

LEFT JOIN borrows AS B ON

U.id = B.user_id;

![](imgs/7-databases-with-entity-framework_8.png)

# Exercise 2: Querying the Library

Using SQL queries, get

all columns of borrows that are borrowed before 2020-10-27

all columns of borrows that are returned

columns user.full_name and borrows.borrowed_at of the user with an id of 1

columns book.name, book.release_year and language.name of all books that are released after 1960

# INSERT

Syntax

__INSERT INTO table_name (column1, column2, column3)_  __VALUES (value1, value2, value3);__

Example

INSERT INTO users (full_name, email, created_at)

VALUES ('Pekka Poistuja', 'pekka.poistuja@buutti.com', NOW());

Since id is not provided, it will be automatically generated.

# UPDATE

Syntax

__UPDATE table_name__  __SET column1 = value1, column2 = value2__  __WHERE condition;__

__Notice:__  if a condition is not provided, all rows will be updated!If updating only one row, it is usually best to use id.

Example

UPDATE usersSET email = 'taija.testaaja@gmail.com'WHERE id = 2;

# REMOVE

Syntax

__DELETE FROM table_name WHERE condition;__

Again, if the  _condition_  is not provided, DELETE affects  _all_  rows. Before deleting, it is a good practice to execute an equivalent SELECT query to make sure that only the proper rows will be affected.

Example:

SELECT  __*__  FROM  __users __ WHERE  __id = 5;__

DELETE FROM users WHERE id = 5;

# Exercise 3: Editing Data

Postpone the due date of the borrow with an id of 1 by two days in the  _borrows _ table

Add a couple of new books to the  _books _ table

Delete one of the borrows.

# CREATE TABLE

Before data can be manipulated, a database and its tables need to be initialized.

Syntax

__CREATE TABLE table_name (__

__	column1 datatype,__  __	column2 datatype,__  __	…__  __);__

Example:CREATE TABLE "users" (

"id" SERIAL PRIMARY KEY,

"full_name" varchar NOT NULL,

"email" varchar UNIQUE NOT NULL,

"created_at" timestamp NOT NULL

);

# DROP

In order to remove tables or databases, we use a DROP statement

__DROP TABLE table_name;__  __DROP DATABASE database_name;__

These statements do not ask for confirmation and there is no undo feature. Take care when using a drop statement.

# NoSQL

* Many differing definitions, but...
  * most agree that NoSQL databases store data in a format other than tables
  * They can still store relational data - just differently
* Four different database types:
  * Document databases
  * Key-value databases
  * Wide-column stores
  * Graph databases
* Example database engines include MongoDB, Redis and Cassandra
---

Document databases store data in documents similar to JSON (JavaScript Object Notation) objects. Each document contains pairs of fields and values. The values can typically be a variety of types including things like strings, numbers, booleans, arrays, or objects, and their structures typically align with objects developers are working with in code. Because of their variety of field value types and powerful query languages, document databases are great for a wide variety of use cases and can be used as a general purpose database. They can horizontally scale-out to accomodate large data volumes. MongoDB is consistently ranked as the world's most popular NoSQL database according to DB-engines and is an example of a document database. For more on document databases, visit What is a Document Database?.
Key-value databases are a simpler type of database where each item contains keys and values. A value can typically only be retrieved by referencing its value, so learning how to query for a specific key-value pair is typically simple. Key-value databases are great for use cases where you need to store large amounts of data but you don't need to perform complex queries to retrieve it. Common use cases include storing user preferences or caching. Redis and DynanoDB are popular key-value databases.
Wide-column stores store data in tables, rows, and dynamic columns. Wide-column stores provide a lot of flexibility over relational databases because each row is not required to have the same columns. Many consider wide-column stores to be two-dimensional key-value databases. Wide-column stores are great for when you need to store large amounts of data and you can predict what your query patterns will be. Wide-column stores are commonly used for storing Internet of Things data and user profile data. Cassandra and HBase are two of the most popular wide-column stores.
Graph databases store data in nodes and edges. Nodes typically store information about people, places, and things while edges store information about the relationships between the nodes. Graph databases excel in use cases where you need to traverse relationships to look for patterns such as social networks, fraud detection, and recommendation engines. Neo4j and JanusGraph are examples of graph databases.

# Object-Relational Mappers

Allows a developer to write code instead of SQL to perform CRUD operations on their database

Some popular ORMs:

Hibernate (Java)

EFCore (.NET)

Sequelize (Node.js)

TypeORM (TypeScript)

# Databases with Entity Framework

![](imgs/7-databases-with-entity-framework_9.png)

---

# Entity Framework (EF)

* ORM made by Microsoft for the .NET framework
  * Object-Relational Mapping: converting from database representation to objects in a programming language
* Allows creation of CRUD operations without writing SQL

# Entity Framework Core (EFC)

Cross-platform, can be used outside of the .NET framework unlike normal Entity Framework

Open-source, lightweight, extensible

Supports many database engines, such as MySQL, PostgreSQL, and so on

This is what we'll be using

---

Notes:
ORM - Object Relational Mapper
CRUD - CREATE READ UPDATE DELETE

# Code First, Database First & Model First

Three approaches through which Entity Framework is implemented

Database First and Code First are the most used ones and will be taught in this lecture

# Code First approach

![](imgs/7-databases-with-entity-framework_10.png)

---

# Code First

* Entity framework will create databases and tables based on defined entity classes
* Good for small applications
* Other advantages include:
  * You can create database and tables from your business objects
  * You can specify which related collections are…
    * eager loaded
    * not serialized at all
  * Database version control
* Not preferred for data intensive applications

# Required Packages

Install and add the following packages to your project:

Microsoft.EntityFrameworkCore

Microsoft.EntityFrameworkCore.Tools

Npgsql.EntityFrameworkCore.PostgreSQL

![](imgs/7-databases-with-entity-framework_11.png)

# Code First: DbContext

* Let's begin with the Code First Approach
* The DbContext class of EFCore is the bridge between the code representation of your data (entities) and the database
* DbContext holds methods to form the database schema with Code First approach and classes to keep the database up-to-date with CRUD operations
  * DATABASE -> CODE: DbSet class property in DbContext can be queried directly with LINQ and this results in an object in your code
  * CODE -> DATABASE: DbSet also has methods like Add, Update and Remove to make changes to the database from your code

# DbContext

* Create a context that inherits from DbContext
  * Commonly located in the Models folder, but ideally should be in a separate abstraction/repository folder (for example Repositories)
* The class needs to have a constructor which calls the base constructor with  : base(options)
* Create a DbSet property for each resource
* public class ContactsContext : DbContext
* {
* public DbSet<Contact> Contacts { get; set; }
* public ContactsContext(DbContextOptions<ContactsContext> options) : base(options){ }
* }

# DbContext (continued)

To further configure how the database will be structured, override the OnModelCreating method

In this example, one table named Contact with columns Id, Name and Email will be created:

public class ContactsContext : DbContext

{

public DbSet<Contact> Contacts { get; set; }

public ContactsContext(DbContextOptions<ContactsContext> options) : base(options){ }

protected override void OnModelCreating(ModelBuilder modelBuilder)

{

modelBuilder.Entity<Contact>().ToTable("Contact");

}

}

In this example, the Contact table will be created with some starting values for Id,  Name and Email columns:

public class ContactsContext : DbContext

{

public DbSet<Contact> Contacts { get; set; }

public ContactsContext(DbContextOptions<ContactsContext> options) : base(options){ }

protected override void OnModelCreating(ModelBuilder modelBuilder)

{

modelBuilder.Entity<Contact>().HasData(

new Contact { Id = 1, Name = "Johannes Kantola", Email = "johkant@example.com" },

new Contact { Id = 2, Name = "Rene Orosz", Email = "rene_king@example.com" }

);

}

}

# DbContext as a Service

* In Program.cs, add the context to services with AddDbContext method
* This is where you set the DB management system you want to use (MySQL, PostgreSQL, SQLite)
  * The EFCore support for PostgreSQL is called  __Npgsql__  __ __ as in the package name
* Add the server, host, port, username, password and database name of the existing database inside options.UseNpgsql as a  __connection string:__
* services.AddDbContext<ContactsContext>(options => options.UseNpgsql(
* @"Server=PostgreSQL 12;Host=localhost;Port=5432;Username=postgres;Password=1234;Database=contacts"));
* services.AddScoped<IContactRepository, ContactRepository>();
* services.AddControllers().AddNewtonsoftJson();

# Migrations

* As the development progresses, models and database schemas change over time
  * This means that both the database and the code needs to be updated to match each other
* Migrations allow for the database to keep in sync with the code schematically
  * The data stored in the database is also preserved
* EFCore migrations have built-in version control; a snapshot of each version of the schema is stored

# Migrations (continued)

* Open the Package Manager Console in Visual Studio
  * If the tab is not in the bottom of the window, open it from  _View _ ->  _Other windows_  ->  _Package Manager Console_
* Add your initial migration by entering the command Add-Migration <name> to the console, for example
* Add-Migration InitialMigration
* This now creates the first "blueprint" of how the database should be structured
* Update the database by entering the command Update-Database to the console
  * This will update the existing database according to the ModelBuilder options

At this point, the values you have entered (Contacts table in this example) should show up in the database. You can check it up e.g. in pgAdmin.

![](imgs/7-databases-with-entity-framework_12.png)

![](imgs/7-databases-with-entity-framework_13.png)

* Notice that the table and column names are initialized with a capital letter
  * The value naming in psql is case sensitive, so all names have to be in quotation marks

# Exercise 4: Adding Context

Continue working on the CourseAPI.

Create a new empty database course_db in pgAdmin or psql

Create a DbContext for the courses. Name it CoursesContext, and add a DbSet of type Course to it, named Courses

Add the OnModelCreating method to the context and add a couple of courses with some starting values to the modelBuilder

Add the CoursesContext to the services in Program.cs with a connection string pointing to course_db

Add the first migration and update the database from the Package Manager Console

Check that  the Course table with the starting values has appeared to the database

# Using DbContext in the API

* Because DbContext is added to services, it can be accessed from any other service, such as the repository
* Using the DbSet for each model in your project, CRUD operations can be applied to the database from the repository with LINQ and DbSet methods
  * Add()
  * Update()
  * Remove()
* After modifying the DbSet, update the changes to the database with DbContext.SaveChanges() method

# Injecting DbContext

Inject the DbContext to your repositories as you would any other service:

public class ContactRepository : IContactRepository

{

private readonly ContactsContext _context;

public ContactRepository(ContactsContext context)

{

_context = context;

}

//...

}

# DbContext: Read Operations

public class ContactRepository : IContactRepository

{

private readonly ContactsContext _context;

public ContactRepository(ContactsContext context) { ... }

public Contact GetContact(int id) =>

_context.Contacts.FirstOrDefault(c => c.Id == id);

public List<Contact> GetContacts() =>

_context.Contacts.ToList();

}

# DbContext: Create Operations

public class ContactRepository : IContactRepository

{

private readonly ContactsContext _context;

public ContactRepository(ContactsContext context) { ... }

// Read operations

// …

public void AddContact(Contact contact)

{

_context.Contacts.Add(contact);

_context.SaveChanges();

}

}

# DbContext: Update Operations

public class ContactRepository : IContactRepository

{

private readonly ContactsContext _context;

public ContactRepository(ContactsContext context) { ... }

// Read & create operations

// ...

public void UpdateContact(int id, Contact newContact)

{

var contact = GetContact(id);

contact.Email = newContact.Email;

contact.Name = newContact.Name;

_context.Contacts.Update(contact);

_context.SaveChanges();

}

}

# DbContext: Delete Operations

public class ContactRepository : IContactRepository

{

private readonly ContactsContext _context;

public ContactRepository(ContactsContext context) { ... }

// Read, create & update operations

// ...

public void DeleteContact(int id)

{

_context.Contacts.Remove(GetContact(id));

_context.SaveChanges();

}

}

# Exercise 5: CRUD on the DB

Continue working on CourseAPI.

Modify the CourseRepository to create, read, update and delete from the database instead of the locally stored list of courses

Test with Postman. Keep refreshing the DB in pgAdmin or creating queries with psql to make sure the request work as intended

# Summing Things Up

Now the API has been hooked up to a PostgreSQL database

Changes to the schema are kept up-to-date with migrations

Repository is processing CRUD operations to the database

Controllers accepting HTTP requests have access to the repository

# EFCore Code First Checklist

Install required packages

Create DbContext for database

Add DbContext to services

Add-Migration & Update-Database

Add CRUD operations to repository for DB

# Modifying the Relations

* Let's change the structure of our Contacts API by adding a new class, Account
  * Instead of Contact directly having an Email, it will have an Account instead
  * Account holds the information about the Email, as well as a Description about the nature of the account (personal, work, school etc.)
* Emails will be removed from the Contacts table

# Modifying the Relations (continued)

public class Contact

{

public int Id { get; set; }

public string Name { get; set; }

public ICollection<Account> Accounts { get; set; }

}

public class Account

{

public int Id { get; set; }

public string Email { get; set; }

public string Description { get; set; }

public int ContactId { get; set; }

public Contact Contact { get; set; }

}

Adding a migration at this point will result in a warning:

![](imgs/7-databases-with-entity-framework_14.png)

* In the generated migration file, you can find Up and Down methods
  * The Up method describes the changes that will be made with the migration
    * In this case, removing the Email column from Contacts table, and creating the new Accounts table
  * The Down method describes the changes that will be made if the migration is reverted
* Updating the database will still work, and the database will have a new table, Accounts

![](imgs/7-databases-with-entity-framework_15.png)

# Exercise 6: Adding Migrations

Continue working on CourseAPI.

Add a new model Lecture with properties int Id, DateTime StartTime, int Length, Course Course, and int CourseId

Add a new property ICollection<Lecture> Lectures to the Course model

Add a new migration, named  _AddLectures_

Update the database. Check that the changes show up in the database with pgAdmin

# Database First approach

![](imgs/7-databases-with-entity-framework_16.png)

---

# Database First

* This is the other approach for creating a connection between the database and the application
* Databases and tables are created first, then you create entity data model using the created database
* Preferred for data intense, large applications
* Other advantages include:
  * Data model is simple to create
  * GUI
  * You do not need to write any code to create your database

# Database First (continued)

* Use the Package Manager Console to "reverse engineer" the code for an existing database
  * This is called  __scaffolding__
* Scaffold the database with the following command:
* Scaffold-DbContext "Server=PostgreSQL 12;Host=localhost;Port=5432;Username=postgres;Password=1234;Database=sqlpractice" Npgsql.EntityFrameworkCore.PostgreSQL -OutputDir Models
* Using the connection string corresponding to your database, this will create all the classes for the entities in the DB as well as the context class

# Exercise 7: Database First

Create a new ASP.NET Core web app using the API template.

Install the required NuGet packages for using EFCore, EFCore Tools and PostgreSQL by using the package manager, or by copying the <PackageReference> lines from the .csproj file of the previous assignment to this projects .csproj file

Scaffold the  _sqlpractice _ database (which was created in the Exercise 1) to the project using the Database First approach. If you have not yet created the database in PostGreSQL, it can be found  [here](https://buuttilab.com/education/trainee-academy-joensuu/assignments/-/raw/94ddeaaa5abb240c0035d718d082df8704488045/Databases/initdb.txt)

# Reading: Authentication with roles

* Here's an example how to do Role-based Authentication by using JWT tokens
  * [https://www.c-sharpcorner.com/article/jwt-token-creation-authentication-and-authorization-in-asp-net-core-6-0-with-po/](https://www.c-sharpcorner.com/article/jwt-token-creation-authentication-and-authorization-in-asp-net-core-6-0-with-po/)

# Extra: Exercise 8: Connection

[under construction]

Continuing the previous exercise,

Create and connect Postgres database to API and create a second entity with a relation to the first entity. Test your solution.