diff --git a/6-databases-with-entity-framework-slides.html b/6-databases-with-entity-framework-slides.html new file mode 100644 index 0000000..02a2a14 --- /dev/null +++ b/6-databases-with-entity-framework-slides.html @@ -0,0 +1,534 @@ +5. Databases with Entity Framework
+

Databases with Entity Framework

+
+
+

Contents

+ +
+
+

Entity Framework (EF)

+
    +
  • Entity Framework is an Object-Relational Mapper (ORM) made by Microsoft for the .NET framework +
      +
    • Object-Relational Mapping: converting from database representation to objects in a programming language
      • +
    +
    • +
  • Allows creation of CRUD operations without writing SQL
    • +
+
+
+

Entity Framework Core (EF Core)

+
    +
  • EF Core is a cross-platform version of EF
    • +
  • Can be used outside of the .NET framework unlike normal Entity Framework
    • +
  • Open-source, lightweight, extensible
    • +
  • Supports many database engines, such as MySQL, PostgreSQL, and so on
    • +
  • This is what we'll be using
    • +
+
+
+

Code First vs Database First vs Model First

+
    +
  • There are three approaches through which Entity Framework can be implemented +
      +
    • Code First
      • +
    • Database First
      • +
    • Model First
      • +
    +
    • +
  • Database First and Code First are the most used ones and will be introduced in this lecture
    • +
+
+
+

Note about loading data

+
    +
  • In EF Core, you can use navigation properties in your model to load related entities
    • +
  • There are three common ORM patterns to load related data +
      +
    • Eager loading: the related data is loaded from the database as part of the initial query.
      • +
    • Explicit loading: the related data is explicitly loaded from the database at a later time.
      • +
    • Lazy loading: the related data is transparently loaded from the database when the navigation property is accessed.
      • +
    +
    • +
+
+
+

Code First approach

+
+
+

Code First

+
    +
  • In the Code First approach, Entity Framework will create databases and tables based on defined entity classes
    • +
  • Good for small applications
    • +
  • Other advantages include: +
      +
    • You can create the database and tables from your business objects
      • +
    • You can specify which related collections are +
        +
      • eager loaded
        • +
      • not serialized at all
        • +
      +
      • +
    • Database version control
      • +
    +
    • +
  • Not preferred for data intensive applications
    • +
+
+
+

Required Packages

+
    +
  • Install and add the following packages to your project: +
      +
    • Microsoft.EntityFrameworkCore
      • +
    • Microsoft.EntityFrameworkCore.Tools
      • +
    • Npgsql.EntityFrameworkCore.PostgreSQL
      • +
    +
    • +
+
+

+
+
+
+

Code First: DbContext

+
    +
  • Let's begin with the Code First Approach
    • +
  • The DbContext class of EFCore is the bridge between the code representation of your data (entities) and the database
    • +
  • DbContext holds
    +a) methods to form the database schema with Code First approach and
    +b) classes to keep the database up-to-date with CRUD operations +
      +
    • DATABASE CODE: DbSet class property in DbContext can be queried directly with LINQ and this results in an object in your code
      • +
    • CODE DATABASE: DbSet also has methods like Add, Update and Remove to make changes to the database from your code
      • +
    +
    • +
+
+
+

Creating a context

+
    +
  • Create a context that inherits from DbContext +
      +
    • Commonly located in the Models folder, but ideally should be in a separate abstraction/repository folder (for example Repositories)
      • +
    +
    • +
  • The class needs to have a constructor that calls the base constructor with
    : base(options)
+
    +
    • +
  • Create a DbSet property for each resource
    public class ContactsContext : DbContext
+{
+    public DbSet<Contact> Contacts { get; set; }
+    public ContactsContext(DbContextOptions<ContactsContext> options) : base(options) { }
+}
+
    +
    • +
+
+
+
    +
  • To further configure how the database will be structured, override the OnModelCreating method
    • +
  • In this example, one table named Contact with columns Id, Name and Email will be created:
    public class ContactsContext : DbContext
+{
+    public DbSet<Contact> Contacts { get; set; }
+    public ContactsContext(DbContextOptions<ContactsContext> options) : base(options) { }
+
+    protected override void OnModelCreating(ModelBuilder modelBuilder)
+    {
+        modelBuilder.Entity<Contact>().ToTable("Contact");
+    }
+}
+
    +
    • +
+
+
+
    +
  • In this example, the Contact table will be created with some starting values for Id, Name and Email columns:
    public class ContactsContext : DbContext
+{
+    public DbSet<Contact> Contacts { get; set; }
+    public ContactsContext(DbContextOptions<ContactsContext> options) : base(options) { }
+
+    protected override void OnModelCreating(ModelBuilder modelBuilder)
+    {
+        modelBuilder.Entity<Contact>().HasData(
+            new Contact { Id = 1, Name = "Johannes Kantola", Email = "johkant@example.com" },
+            new Contact { Id = 2, Name = "Rene Orosz", Email = "rene_king@example.com" }
+        );
+    }
+}
+
    +
    • +
+
+
+

DbContext as a Service

+
    +
  • In Program.cs, add the context to services with AddDbContext method
    • +
  • This is where you set the DB management system you want to use (MySQL, PostgreSQL, SQLite...) +
      +
    • The EFCore support for PostgreSQL is called Npgsql as in the package name
      • +
    +
    • +
  • Add the server, host, port, username, password and the database name of the existing database inside options.UseNpgsql as a connection string:
    services.AddDbContext<ContactsContext>(options => options.UseNpgsql(
+  @"Server=PostgreSQL 12;Host=localhost;Port=5432;Username=postgres;Password=1234;Database=contacts"));
+services.AddScoped<IContactRepository, ContactRepository>();
+services.AddControllers().AddNewtonsoftJson();
+
    +
    • +
+
+
+

Migrations

+
    +
  • As the development progresses, models and database schemas change over time +
      +
    • This means that both the database and the code needs to be updated to match each other
      • +
    +
    • +
  • Migrations allow for the database to keep in sync with the code schematically +
      +
    • The data stored in the database is also preserved
      • +
    +
    • +
  • EFCore migrations have built-in version control; a snapshot of each version of the schema is stored
    • +
+
+
+

Applying migrations

+
    +
  • Open the Package Manager Console in Visual Studio +
      +
    • If the tab is not in the bottom of the window, open it from
      View > Other windows > Package Manager Console
      • +
    +
    • +
  • Add your initial migration by entering the command Add-Migration <name> to the console, for example
    +Add-Migration InitialMigration +
      +
    • This now creates the first "blueprint" of how the database should be structured
      • +
    +
    • +
  • Update the database by entering the command Update-Database to the console +
      +
    • This will update the existing database according to the ModelBuilder options
      • +
    +
    • +
+
+
+
    +
  • At this point, the values you have entered (Contacts table in this example) should show up in the database. You can check it up e.g. in pgAdmin.
    • +
+
+
+

+
+
+

+
+
+
    +
  • Notice that the table and column names are initialized with a capital letter +
      +
    • The value naming in psql is case sensitive
      + All names have to be in quotation marks!
      • +
    +
    • +
+
+
+

Exercise 1: Adding Context

+ +

Continue working on the CourseAPI.

+
    +
  1. Create a new empty database course_db in pgAdmin or psql
    2. +
  2. Create a DbContext for the courses. Name it CoursesContext, and add a DbSet of type Course to it, named Courses
    3. +
  3. Add the OnModelCreating method to the context and add a couple of courses with some starting values to the modelBuilder
    4. +
  4. Add the CoursesContext to the services in Program.cs with a connection string pointing to course_db
    5. +
  5. Add the first migration and update the database from the Package Manager Console
    6. +
  6. Check that the Course table with the starting values has appeared to the database
    7. +
+
+
+

Using DbContext in the API

+
    +
  • Because DbContext is added to services, it can be accessed from any other service, such as the repository
    • +
  • Using the DbSet for each model in your project, CRUD operations can be applied to the database from the repository with LINQ and DbSet methods +
      +
    • Add()
      • +
    • Update()
      • +
    • Remove()
      • +
    +
    • +
  • After modifying the DbSet, update the changes to the database with the DbContext.SaveChanges() method
    • +
+
+
+

Injecting DbContext

+
    +
  • Inject the DbContext to your repositories as you would any other service:
    public class ContactRepository : IContactRepository
+{
+    private readonly ContactsContext _context;
+
+    public ContactRepository(ContactsContext context)
+    {
+        _context = context;
+    }
+    //...
+}
+
    +
    • +
+
+
+

DbContext: Read Operations

+
public class ContactRepository : IContactRepository
+{
+    private readonly ContactsContext _context;
+    public ContactRepository(ContactsContext context) { ... }
+
+    public Contact GetContact(int id) =>
+        _context.Contacts.FirstOrDefault(c => c.Id == id);
+
+    public List<Contact> GetContacts() =>
+        _context.Contacts.ToList();
+}
+
+
+
+

DbContext: Create Operations

+
public class ContactRepository : IContactRepository
+{
+    private readonly ContactsContext _context;
+    public ContactRepository(ContactsContext context) { ... }
+
+    // Read operations
+    // ...
+
+    public void AddContact(Contact contact)
+    {
+        _context.Contacts.Add(contact);
+        _context.SaveChanges();
+    }
+}
+
+
+
+

DbContext: Update Operations

+
public class ContactRepository : IContactRepository
+{
+    private readonly ContactsContext _context;
+    public ContactRepository(ContactsContext context) { ... }
+
+    // Read & create operations
+    // ...
+
+    public void UpdateContact(int id, Contact newContact)
+    {
+        var contact = GetContact(id);
+        contact.Email = newContact.Email;
+        contact.Name = newContact.Name;
+        _context.Contacts.Update(contact);
+        _context.SaveChanges();
+    }
+}
+
+
+
+

DbContext: Delete Operations

+
public class ContactRepository : IContactRepository
+{
+    private readonly ContactsContext _context;
+    public ContactRepository(ContactsContext context) { ... }
+
+    // Read, create & update operations
+    // ...
+
+    public void DeleteContact(int id)
+    {
+        _context.Contacts.Remove(GetContact(id));
+        _context.SaveChanges();
+    }
+}
+
+
+
+

Exercise 2: CRUD on the DB

+ +

Continue working on CourseAPI.

+
    +
  1. Modify the CourseRepository to create, read, update and delete from the database instead of the locally stored list of courses
    2. +
  2. Test with Postman. Keep refreshing the DB in pgAdmin or creating queries with psql to make sure the requests work as intended
    3. +
+
+
+

Summing Things Up

+
    +
  • Now the API has been hooked up to a PostgreSQL database
    • +
  • Changes to the schema are kept up-to-date with migrations
    • +
  • Repository is processing CRUD operations to the database
    • +
  • Controllers accepting HTTP requests have access to the repository
    • +
+
+
+

EFCore Code First Checklist

+
    +
  1. Install required packages
    2. +
  2. Create DbContext for the database
    3. +
  3. Add DbContext to services
    4. +
  4. Add-Migration & Update-Database
    5. +
  5. Add CRUD operations to the database repository
    6. +
+
+
+

Modifying the Relations

+
    +
  • Let's change the structure of our Contacts API by adding a new class Account +
      +
    • Instead of Contact directly having an Email, it will have an Account instead
      • +
    • Account holds the information about the Email, as well as a Description about the nature of the account (personal, work, school etc.)
      • +
    +
    • +
  • Emails will be removed from the Contacts table
    • +
+
+
+
// Models/Contact.cs
+public class Contact
+{
+    public int Id { get; set; }
+    public string Name { get; set; }
+    public ICollection<Account> Accounts { get; set; }
+}
+
+
// Models/Account.cs
+public class Account
+{
+    public int Id { get; set; }
+    public string Email { get; set; }
+    public string Description { get; set; }
+    public int ContactId { get; set; }
+    public Contact Contact { get; set; }
+}
+
+
+
+
    +
  • Adding a migration at this point will result in a warning:
    +
    • +
+
+
+
+
+
    +
  • In the generated migration file, you can find Up and Down methods +
      +
    • The Up method describes the changes that will be made with the migration +
        +
      • In this case, removing the Email column from Contacts table, and creating the new Accounts table
        • +
      +
      • +
    • The Down method describes the changes that will be made if the migration is reverted
      • +
    +
    • +
  • Updating the database will still work, and the database will have a new table Accounts
    • +
+
+
+

+
+
+
+
+

Exercise 3: Adding Migrations

+ +

Continue working on CourseAPI.

+
    +
  1. Add a new model Lecture with properties int Id, DateTime StartTime, int Length, Course Course, and int CourseId
    2. +
  2. Add a new property ICollection<Lecture> Lectures to the Course model
    3. +
  3. Add a new migration named AddLectures
    4. +
  4. Update the database. Check that the changes show up in the database with pgAdmin
    5. +
+
+
+

Database First approach

+
+
+

What is the Database First approach?

+
    +
  • This is the other approach for creating a connection between the database and the application
    • +
  • Databases and tables are created first, then you create an entity data model using the created database
    • +
  • This approach is preferred for data intense, large applications
    • +
  • Other advantages include: +
      +
    • Data model is simple to create
      • +
    • GUI
      • +
    • You do not need to write any code to create your database
      • +
    +
    • +
+
+
+

Scaffolding

+
    +
  • Use the Package Manager Console to "reverse engineer" the code for an existing database +
      +
    • This is called scaffolding
      • +
    +
    • +
  • Scaffold the database with the following command:
    Scaffold-DbContext "Server=PostgreSQL 12;Host=localhost;Port=5432;Username=postgres;Password=1234;Database=sqlpractice" Npgsql.EntityFrameworkCore.PostgreSQL -OutputDir Models
+
    +
    • +
  • Using the connection string corresponding to your database, this will create all the classes for the entities in the DB as well as the context class
    • +
+
+
+

Exercise 4: Database First

+ +

Create a new ASP.NET Core web app using the API template.

+
    +
  1. Install the required NuGet packages for using EFCore, EFCore Tools and PostgreSQL
    +a) by using the package manager, or
    +b) by copying the <PackageReference> lines from the .csproj file of the previous assignment to this project's .csproj file
    2. +
  2. Scaffold the sqlpractice database created in SQL Databases Exercise 1 to the project by using the Database First approach. If you have not yet created the database in PostgreSQL, it can be found here
    3. +
+
+
+

Reading: Authentication with roles

+
    +
  • Here's an example how to do a role-based authentication by using JWT tokens
    • +
+
+
+

Exercise 5 (Extra): Connection

+ +

Continuing the previous exercise,

+
    +
  1. Create and connect Postgres database to API and create a second entity with a relation to the first entity.
    2. +
  2. Test your solution.
    3. +
+
+
\ No newline at end of file diff --git a/6-databases-with-entity-framework.md b/6-databases-with-entity-framework.md index d75e4af..c07ddfc 100644 --- a/6-databases-with-entity-framework.md +++ b/6-databases-with-entity-framework.md @@ -1,499 +1,158 @@ -# SQL - -* __Language __ to organize and manipulate data in a database -* Originally developed by IBM in the 70's - * Quickly became the most popular database language -* SELECT id, email -* FROM users -* WHERE first_name = 'Teppo'; - -# Relational Database Management Systems - -* In relational databases, values are stored in __tables__ - * Each table has __rows __ and __columns__ - * Data is displayed in a two-dimensional matrix -* Values in a table are related to each other - * Values can also be related to values in other tables -* A relational database management system (RDBMS) is a program, which executes queries to relational databases - -![](imgs/7-databases-with-entity-framework_0.png) - -[https://db-engines.com/en/ranking](https://db-engines.com/en/ranking) - -# PostgreSQL - -Free open-source, cross-platform relational database management system - -Emphasizes extensibility and SQL compliance - -Fully ACID-compliant (atomicity, consistency, isolation and durability) - -# pgAdmin - -Administration and development platform for PostgreSQL - -Cross-platform, features a web interface - -Basically a control panel application for your PostgreSQL database - -# PostgreSQL: Creating a Database - -With pgAdmin: - -Right-click Servers > my-postgres > _Databases_ - -Select _Create _ > _Database…_ - -Insert a name for the database _ _ and hit _Save_ - -![](imgs/7-databases-with-entity-framework_1.png) - -With psql: - -Enter commandCREATE DATABASE ; - -# PostgreSQL: Querying - -With pgAdmin: - -Right-click _sqlpractice _ > _Query tool..._ - -Insert a query into the _Query Editor_ and hit _Execute _ (F5) - -![](imgs/7-databases-with-entity-framework_2.png) - -![](imgs/7-databases-with-entity-framework_3.png) - ---- - -With psql: - -List all available databases with \\l - -Connect to the created database with \\c - -List all tables in the database with \\dt - -Type a query and press enter - --- - -https://drive.google.com/file/d/1E5TqY5yt8DNcWHVkb8gxvBpgLtWB15DC/view?usp=sharing "provided query" - -# Editing Data with pgAdmin - -Inspect and edit data in pgAdmin by right-clicking a table and selecting _View/Edit Data_ - -![](imgs/7-databases-with-entity-framework_4.png) - -![](imgs/7-databases-with-entity-framework_5.png) - -Individual values in the table can be directly modified by double clicking the value and then editing the value in the visual user interface - -Save the changes with the _Save Data Changes_ button - -![](imgs/7-databases-with-entity-framework_6.png) - -# Exercise 1: Preparing Database - -Using either PgAdmin or PSQL - -Create a new database called sqlpractice - -Insert the [provided query](https://drive.google.com/file/d/1E5TqY5yt8DNcWHVkb8gxvBpgLtWB15DC/view) to the new database - -Verify that the query has created new tables to your database - -# Types of queries - -Select - -Insert - -Delete - -Update - -Create & Drop - -# Querying data with SELECT - -Syntax: - -__SELECT column1, column2, column3 FROM table_name;__ - -Examples: - -SELECT full_name, email FROM users; - -SELECT full_name AS name, email FROM users; - -SELECT * FROM users; - -# Filtering data with WHERE - -Syntax: - -__SELECT column1, column2 FROM table_name WHERE condition;__ - -Text is captured in _single quotes_ . - -LIKE condition uses _%_ sign as wildcard. - -IS and IS NOT are also valid operators. - -Example: - -SELECT full_name FROM users - -WHERE full_name = 'Teppo Testaaja'; - -SELECT * FROM books WHERE name LIKE '%rr%'; - -SELECT * FROM books WHERE author IS NOT null; - -# Ordering data with ORDER BY - -Syntax: - -__SELECT column1 FROM table_name ORDER BY column1 ASC;__ - -Examples: - -SELECT full_name FROM users - -ORDER BY full_name ASC; - -SELECT full_name FROM users - -ORDER BY full_name DESC - -# Combining with JOIN - -Also known as INNER JOIN - -Corresponds to intersection from set theory - -![](imgs/7-databases-with-entity-framework_7.png) - -# JOIN examples - -SELECT - -users.id, users.full_name, borrows.id, - -borrows.user_id, borrows.due_date, borrows.returned_at - -FROM users - -JOIN borrows ON - -users.id = borrows.user_id; - -SELECT - -U.full_name AS name, - -B.due_date AS due_date, - -B.returned_at AS returned_at - -FROM users AS U - -JOIN borrows AS B ON - -U.id = B.user_id; - -# Combining with LEFT JOIN - -Also known as LEFT OUTER JOIN - -Example: - -SELECT - -U.full_name AS name, - -B.due_date AS due_date, - -B.returned_at AS returned_at - -FROM users AS U - -LEFT JOIN borrows AS B ON - -U.id = B.user_id; - -![](imgs/7-databases-with-entity-framework_8.png) - -# Exercise 2: Querying the Library - -Using SQL queries, get - -all columns of borrows that are borrowed before 2020-10-27 - -all columns of borrows that are returned - -columns user.full_name and borrows.borrowed_at of the user with an id of 1 - -columns book.name, book.release_year and language.name of all books that are released after 1960 - -# INSERT - -Syntax - -__INSERT INTO table_name (column1, column2, column3)_ __VALUES (value1, value2, value3);__ - -Example - -INSERT INTO users (full_name, email, created_at) - -VALUES ('Pekka Poistuja', 'pekka.poistuja@buutti.com', NOW()); - -Since id is not provided, it will be automatically generated. - -# UPDATE - -Syntax - -__UPDATE table_name__ __SET column1 = value1, column2 = value2__ __WHERE condition;__ - -__Notice:__ if a condition is not provided, all rows will be updated!If updating only one row, it is usually best to use id. - -Example - -UPDATE usersSET email = 'taija.testaaja@gmail.com'WHERE id = 2; - -# REMOVE - -Syntax - -__DELETE FROM table_name WHERE condition;__ - -Again, if the _condition_ is not provided, DELETE affects _all_ rows. Before deleting, it is a good practice to execute an equivalent SELECT query to make sure that only the proper rows will be affected. - -Example: - -SELECT __*__ FROM __users __ WHERE __id = 5;__ - -DELETE FROM users WHERE id = 5; - -# Exercise 3: Editing Data - -Postpone the due date of the borrow with an id of 1 by two days in the _borrows _ table - -Add a couple of new books to the _books _ table - -Delete one of the borrows. - -# CREATE TABLE - -Before data can be manipulated, a database and its tables need to be initialized. - -Syntax - -__CREATE TABLE table_name (__ - -__ column1 datatype,__ __ column2 datatype,__ __ …__ __);__ - -Example:CREATE TABLE "users" ( - -"id" SERIAL PRIMARY KEY, - -"full_name" varchar NOT NULL, - -"email" varchar UNIQUE NOT NULL, - -"created_at" timestamp NOT NULL - -); - -# DROP - -In order to remove tables or databases, we use a DROP statement - -__DROP TABLE table_name;__ __DROP DATABASE database_name;__ - -These statements do not ask for confirmation and there is no undo feature. Take care when using a drop statement. - -# NoSQL - -* Many differing definitions, but... - * most agree that NoSQL databases store data in a format other than tables - * They can still store relational data - just differently -* Four different database types: - * Document databases - * Key-value databases - * Wide-column stores - * Graph databases -* Example database engines include MongoDB, Redis and Cassandra +marp: true +paginate: true +math: mathjax +theme: buutti +title: 5. Databases with Entity Framework --- -Document databases store data in documents similar to JSON (JavaScript Object Notation) objects. Each document contains pairs of fields and values. The values can typically be a variety of types including things like strings, numbers, booleans, arrays, or objects, and their structures typically align with objects developers are working with in code. Because of their variety of field value types and powerful query languages, document databases are great for a wide variety of use cases and can be used as a general purpose database. They can horizontally scale-out to accomodate large data volumes. MongoDB is consistently ranked as the world's most popular NoSQL database according to DB-engines and is an example of a document database. For more on document databases, visit What is a Document Database?. -Key-value databases are a simpler type of database where each item contains keys and values. A value can typically only be retrieved by referencing its value, so learning how to query for a specific key-value pair is typically simple. Key-value databases are great for use cases where you need to store large amounts of data but you don't need to perform complex queries to retrieve it. Common use cases include storing user preferences or caching. Redis and DynanoDB are popular key-value databases. -Wide-column stores store data in tables, rows, and dynamic columns. Wide-column stores provide a lot of flexibility over relational databases because each row is not required to have the same columns. Many consider wide-column stores to be two-dimensional key-value databases. Wide-column stores are great for when you need to store large amounts of data and you can predict what your query patterns will be. Wide-column stores are commonly used for storing Internet of Things data and user profile data. Cassandra and HBase are two of the most popular wide-column stores. -Graph databases store data in nodes and edges. Nodes typically store information about people, places, and things while edges store information about the relationships between the nodes. Graph databases excel in use cases where you need to traverse relationships to look for patterns such as social networks, fraud detection, and recommendation engines. Neo4j and JanusGraph are examples of graph databases. - -# Object-Relational Mappers - -Allows a developer to write code instead of SQL to perform CRUD operations on their database - -Some popular ORMs: - -Hibernate (Java) - -EFCore (.NET) - -Sequelize (Node.js) +# Databases with Entity Framework -TypeORM (TypeScript) + + -# Databases with Entity Framework +## Contents -![](imgs/7-databases-with-entity-framework_9.png) +- [Entity Framework (EF)](#entity-framework-ef) +- [Code First approach](#code-first-approach) +- [Migrations](#migrations) +- [Database First approach](#database-first-approach) ---- -# Entity Framework (EF) +## Entity Framework (EF) -* ORM made by Microsoft for the .NET framework +* [Entity Framework](https://learn.microsoft.com/en-us/ef/) is an Object-Relational Mapper (ORM) made by Microsoft for the .NET framework * Object-Relational Mapping: converting from database representation to objects in a programming language * Allows creation of CRUD operations without writing SQL -# Entity Framework Core (EFC) - -Cross-platform, can be used outside of the .NET framework unlike normal Entity Framework - -Open-source, lightweight, extensible - -Supports many database engines, such as MySQL, PostgreSQL, and so on +### Entity Framework Core (EF Core) -This is what we'll be using +* [EF Core](https://learn.microsoft.com/en-us/ef/core/) is a cross-platform version of EF +* Can be used outside of the .NET framework unlike normal Entity Framework +* Open-source, lightweight, extensible +* Supports many database engines, such as MySQL, PostgreSQL, and so on +* This is what we'll be using ---- - -Notes: -ORM - Object Relational Mapper -CRUD - CREATE READ UPDATE DELETE - -# Code First, Database First & Model First +### Code First vs Database First vs Model First -Three approaches through which Entity Framework is implemented +* There are three approaches through which Entity Framework can be implemented + * Code First + * Database First + * Model First +* Database First and Code First are the most used ones and will be introduced in this lecture -Database First and Code First are the most used ones and will be taught in this lecture +### Note about loading data -# Code First approach +* In EF Core, you can use [navigation properties](https://learn.microsoft.com/en-us/ef/ef6/fundamentals/relationships) in your model to load related entities +* There are three common ORM patterns to load related data + * [Eager loading](https://learn.microsoft.com/en-us/ef/core/querying/related-data/eager): the related data is loaded from the database as part of the initial query. + * [Explicit loading](https://learn.microsoft.com/en-us/ef/core/querying/related-data/explicit): the related data is explicitly loaded from the database at a later time. + * [Lazy loading](https://learn.microsoft.com/en-us/ef/core/querying/related-data/lazy): the related data is transparently loaded from the database when the navigation property is accessed. -![](imgs/7-databases-with-entity-framework_10.png) - ---- +## Code First approach -# Code First +### Code First -* Entity framework will create databases and tables based on defined entity classes +* In the Code First approach, Entity Framework will create databases and tables based on defined ***entity classes*** * Good for small applications * Other advantages include: - * You can create database and tables from your business objects - * You can specify which related collections are… + * You can create the database and tables from your [business objects](https://en.wikipedia.org/wiki/Business_object) + * You can specify which related collections are * eager loaded * not serialized at all * Database version control * Not preferred for data intensive applications -# Required Packages +### Required Packages -Install and add the following packages to your project: +* Install and add the following packages to your project: + * `Microsoft.EntityFrameworkCore` + * `Microsoft.EntityFrameworkCore.Tools` + * `Npgsql.EntityFrameworkCore.PostgreSQL` -Microsoft.EntityFrameworkCore +
-Microsoft.EntityFrameworkCore.Tools +![w:1000](imgs/7-databases-with-entity-framework_11.png) -Npgsql.EntityFrameworkCore.PostgreSQL +
-![](imgs/7-databases-with-entity-framework_11.png) - -# Code First: DbContext +### Code First: `DbContext` * Let's begin with the Code First Approach -* The DbContext class of EFCore is the bridge between the code representation of your data (entities) and the database -* DbContext holds methods to form the database schema with Code First approach and classes to keep the database up-to-date with CRUD operations - * DATABASE -> CODE: DbSet class property in DbContext can be queried directly with LINQ and this results in an object in your code - * CODE -> DATABASE: DbSet also has methods like Add, Update and Remove to make changes to the database from your code - -# DbContext - -* Create a context that inherits from DbContext - * Commonly located in the Models folder, but ideally should be in a separate abstraction/repository folder (for example Repositories) -* The class needs to have a constructor which calls the base constructor with : base(options) -* Create a DbSet property for each resource -* public class ContactsContext : DbContext -* { -* public DbSet Contacts { get; set; } -* public ContactsContext(DbContextOptions options) : base(options){ } -* } - -# DbContext (continued) - -To further configure how the database will be structured, override the OnModelCreating method - -In this example, one table named Contact with columns Id, Name and Email will be created: - -public class ContactsContext : DbContext - -{ - -public DbSet Contacts { get; set; } - -public ContactsContext(DbContextOptions options) : base(options){ } - -protected override void OnModelCreating(ModelBuilder modelBuilder) - -{ - -modelBuilder.Entity().ToTable("Contact"); - -} - -} - -In this example, the Contact table will be created with some starting values for Id, Name and Email columns: - -public class ContactsContext : DbContext - -{ - -public DbSet Contacts { get; set; } - -public ContactsContext(DbContextOptions options) : base(options){ } +* The `DbContext` class of EFCore is the bridge between the code representation of your data (entities) and the database +* `DbContext` holds + a) methods to form the database schema with Code First approach and + b) classes to keep the database up-to-date with CRUD operations + * DATABASE $\Rightarrow$ CODE: `DbSet` class property in `DbContext` can be queried directly with LINQ and this results in an object in your code + * CODE $\Rightarrow$ DATABASE: `DbSet` also has methods like `Add`, `Update` and `Remove` to make changes to the database from your code + +### Creating a context + +* Create a context that inherits from `DbContext` + * Commonly located in the `Models` folder, but ideally should be in a separate abstraction/repository folder (for example `Repositories`) +* The class needs to have a constructor that calls the base constructor with + ```csharp + : base(options) + ``` +* Create a `DbSet` property for each resource + ```csharp + public class ContactsContext : DbContext + { + public DbSet Contacts { get; set; } + public ContactsContext(DbContextOptions options) : base(options) { } + } + ``` -protected override void OnModelCreating(ModelBuilder modelBuilder) - -{ - -modelBuilder.Entity().HasData( - -new Contact { Id = 1, Name = "Johannes Kantola", Email = "johkant@example.com" }, - -new Contact { Id = 2, Name = "Rene Orosz", Email = "rene_king@example.com" } - -); - -} - -} +--- -# DbContext as a Service +* To further configure how the database will be structured, override the `OnModelCreating` method +* In this example, one table named `Contact` with columns `Id`, `Name` and `Email` will be created: + ```csharp + public class ContactsContext : DbContext + { + public DbSet Contacts { get; set; } + public ContactsContext(DbContextOptions options) : base(options) { } + + protected override void OnModelCreating(ModelBuilder modelBuilder) + { + modelBuilder.Entity().ToTable("Contact"); + } + } + ``` -* In Program.cs, add the context to services with AddDbContext method -* This is where you set the DB management system you want to use (MySQL, PostgreSQL, SQLite) - * The EFCore support for PostgreSQL is called __Npgsql__ __ __ as in the package name -* Add the server, host, port, username, password and database name of the existing database inside options.UseNpgsql as a __connection string:__ -* services.AddDbContext(options => options.UseNpgsql( -* @"Server=PostgreSQL 12;Host=localhost;Port=5432;Username=postgres;Password=1234;Database=contacts")); -* services.AddScoped(); -* services.AddControllers().AddNewtonsoftJson(); +--- -# Migrations +* In this example, the Contact table will be created with some starting values for `Id`, `Name` and `Email` columns: + ```csharp + public class ContactsContext : DbContext + { + public DbSet Contacts { get; set; } + public ContactsContext(DbContextOptions options) : base(options) { } + + protected override void OnModelCreating(ModelBuilder modelBuilder) + { + modelBuilder.Entity().HasData( + new Contact { Id = 1, Name = "Johannes Kantola", Email = "johkant@example.com" }, + new Contact { Id = 2, Name = "Rene Orosz", Email = "rene_king@example.com" } + ); + } + } + ``` + +### DbContext as a Service + +* In `Program.cs`, add the context to services with `AddDbContext` method +* This is where you set the DB management system you want to use (MySQL, PostgreSQL, SQLite...) + * The EFCore support for PostgreSQL is called `Npgsql` as in the package name +* Add the server, host, port, username, password and the database name of the existing database inside `options.UseNpgsql` as a *__connection string__*: + ```csharp + services.AddDbContext(options => options.UseNpgsql( + @"Server=PostgreSQL 12;Host=localhost;Port=5432;Username=postgres;Password=1234;Database=contacts")); + services.AddScoped(); + services.AddControllers().AddNewtonsoftJson(); + ``` + +## Migrations * As the development progresses, models and database schemas change over time * This means that both the database and the code needs to be updated to match each other @@ -501,309 +160,278 @@ new Contact { Id = 2, Name = "Rene Orosz", Email = "rene_king@example.com" } * The data stored in the database is also preserved * EFCore migrations have built-in version control; a snapshot of each version of the schema is stored -# Migrations (continued) +### Applying migrations * Open the Package Manager Console in Visual Studio - * If the tab is not in the bottom of the window, open it from _View _ -> _Other windows_ -> _Package Manager Console_ -* Add your initial migration by entering the command Add-Migration to the console, for example -* Add-Migration InitialMigration -* This now creates the first "blueprint" of how the database should be structured -* Update the database by entering the command Update-Database to the console - * This will update the existing database according to the ModelBuilder options - -At this point, the values you have entered (Contacts table in this example) should show up in the database. You can check it up e.g. in pgAdmin. - -![](imgs/7-databases-with-entity-framework_12.png) - -![](imgs/7-databases-with-entity-framework_13.png) - -* Notice that the table and column names are initialized with a capital letter - * The value naming in psql is case sensitive, so all names have to be in quotation marks - -# Exercise 4: Adding Context + * If the tab is not in the bottom of the window, open it from
_View > Other windows > Package Manager Console_ +* Add your initial migration by entering the command `Add-Migration ` to the console, for example + `Add-Migration InitialMigration` + * This now creates the first "blueprint" of how the database should be structured +* Update the database by entering the command `Update-Database` to the console + * This will update the existing database according to the `ModelBuilder` options -Continue working on the CourseAPI. - -Create a new empty database course_db in pgAdmin or psql - -Create a DbContext for the courses. Name it CoursesContext, and add a DbSet of type Course to it, named Courses - -Add the OnModelCreating method to the context and add a couple of courses with some starting values to the modelBuilder - -Add the CoursesContext to the services in Program.cs with a connection string pointing to course_db - -Add the first migration and update the database from the Package Manager Console - -Check that the Course table with the starting values has appeared to the database - -# Using DbContext in the API - -* Because DbContext is added to services, it can be accessed from any other service, such as the repository -* Using the DbSet for each model in your project, CRUD operations can be applied to the database from the repository with LINQ and DbSet methods - * Add() - * Update() - * Remove() -* After modifying the DbSet, update the changes to the database with DbContext.SaveChanges() method - -# Injecting DbContext - -Inject the DbContext to your repositories as you would any other service: - -public class ContactRepository : IContactRepository +--- -{ +* At this point, the values you have entered (`Contacts` table in this example) should show up in the database. You can check it up e.g. in pgAdmin. -private readonly ContactsContext _context; +
+
-public ContactRepository(ContactsContext context) +![](imgs/7-databases-with-entity-framework_13.png) -{ +
+
-_context = context; +![](imgs/7-databases-with-entity-framework_12.png) -} +
+
-//... +* Notice that the table and column names are initialized with a capital letter + * The value naming in psql is case sensitive + $\Rightarrow$ All names have to be in quotation marks! -} +### Exercise 1: Adding Context + -# DbContext: Read Operations +Continue working on the CourseAPI. +1) Create a new empty database `course_db` in pgAdmin or psql +2) Create a `DbContext` for the courses. Name it `CoursesContext`, and add a `DbSet` of type `Course` to it, named `Courses` +3) Add the `OnModelCreating` method to the context and add a couple of courses with some starting values to the `modelBuilder` +4) Add the `CoursesContext` to the services in `Program.cs` with a connection string pointing to `course_db` +5) Add the first migration and update the database from the Package Manager Console +6) Check that the `Course` table with the starting values has appeared to the database + +### Using DbContext in the API + +* Because `DbContext` is added to services, it can be accessed from any other service, such as the repository +* Using the `DbSet` for each model in your project, CRUD operations can be applied to the database from the repository with LINQ and `DbSet` methods + * `Add()` + * `Update()` + * `Remove()` +* After modifying the `DbSet`, update the changes to the database with the `DbContext.SaveChanges()` method + +### Injecting DbContext + +* Inject the `DbContext` to your repositories as you would any other service: + ```csharp + public class ContactRepository : IContactRepository + { + private readonly ContactsContext _context; + + public ContactRepository(ContactsContext context) + { + _context = context; + } + //... + } + ``` + +### DbContext: Read Operations + +```csharp public class ContactRepository : IContactRepository - { + private readonly ContactsContext _context; + public ContactRepository(ContactsContext context) { ... } -private readonly ContactsContext _context; - -public ContactRepository(ContactsContext context) { ... } - -public Contact GetContact(int id) => - -_context.Contacts.FirstOrDefault(c => c.Id == id); - -public List GetContacts() => - -_context.Contacts.ToList(); + public Contact GetContact(int id) => + _context.Contacts.FirstOrDefault(c => c.Id == id); + public List GetContacts() => + _context.Contacts.ToList(); } +``` -# DbContext: Create Operations +### DbContext: Create Operations +```csharp public class ContactRepository : IContactRepository - { + private readonly ContactsContext _context; + public ContactRepository(ContactsContext context) { ... } -private readonly ContactsContext _context; - -public ContactRepository(ContactsContext context) { ... } - -// Read operations - -// … - -public void AddContact(Contact contact) - -{ - -_context.Contacts.Add(contact); - -_context.SaveChanges(); - -} + // Read operations + // ... + public void AddContact(Contact contact) + { + _context.Contacts.Add(contact); + _context.SaveChanges(); + } } +``` -# DbContext: Update Operations +### DbContext: Update Operations +```csharp public class ContactRepository : IContactRepository - -{ - -private readonly ContactsContext _context; - -public ContactRepository(ContactsContext context) { ... } - -// Read & create operations - -// ... - -public void UpdateContact(int id, Contact newContact) - { - -var contact = GetContact(id); - -contact.Email = newContact.Email; - -contact.Name = newContact.Name; - -_context.Contacts.Update(contact); - -_context.SaveChanges(); - -} - + private readonly ContactsContext _context; + public ContactRepository(ContactsContext context) { ... } + + // Read & create operations + // ... + + public void UpdateContact(int id, Contact newContact) + { + var contact = GetContact(id); + contact.Email = newContact.Email; + contact.Name = newContact.Name; + _context.Contacts.Update(contact); + _context.SaveChanges(); + } } +``` -# DbContext: Delete Operations +### DbContext: Delete Operations +```csharp public class ContactRepository : IContactRepository - { + private readonly ContactsContext _context; + public ContactRepository(ContactsContext context) { ... } -private readonly ContactsContext _context; - -public ContactRepository(ContactsContext context) { ... } - -// Read, create & update operations - -// ... - -public void DeleteContact(int id) - -{ - -_context.Contacts.Remove(GetContact(id)); - -_context.SaveChanges(); + // Read, create & update operations + // ... + public void DeleteContact(int id) + { + _context.Contacts.Remove(GetContact(id)); + _context.SaveChanges(); + } } +``` -} - -# Exercise 5: CRUD on the DB +### Exercise 2: CRUD on the DB + Continue working on CourseAPI. -Modify the CourseRepository to create, read, update and delete from the database instead of the locally stored list of courses - -Test with Postman. Keep refreshing the DB in pgAdmin or creating queries with psql to make sure the request work as intended - -# Summing Things Up - -Now the API has been hooked up to a PostgreSQL database +1) Modify the `CourseRepository` to create, read, update and delete from the database instead of the locally stored list of courses +2) Test with Postman. Keep refreshing the DB in pgAdmin or creating queries with psql to make sure the requests work as intended -Changes to the schema are kept up-to-date with migrations +### Summing Things Up -Repository is processing CRUD operations to the database +* Now the API has been hooked up to a PostgreSQL database +* Changes to the schema are kept up-to-date with migrations +* Repository is processing CRUD operations to the database +* Controllers accepting HTTP requests have access to the repository -Controllers accepting HTTP requests have access to the repository +### EFCore Code First Checklist -# EFCore Code First Checklist +1) Install required packages +2) Create `DbContext` for the database +3) Add `DbContext` to services +4) `Add-Migration` & `Update-Database` +5) Add CRUD operations to the database repository -Install required packages +### Modifying the Relations -Create DbContext for database +* Let's change the structure of our Contacts API by adding a new class `Account` + * Instead of `Contact` directly having an `Email`, it will have an `Account` instead + * `Account` holds the information about the `Email`, as well as a `Description` about the nature of the account (personal, work, school etc.) +* Emails will be removed from the `Contacts` table -Add DbContext to services - -Add-Migration & Update-Database - -Add CRUD operations to repository for DB - -# Modifying the Relations - -* Let's change the structure of our Contacts API by adding a new class, Account - * Instead of Contact directly having an Email, it will have an Account instead - * Account holds the information about the Email, as well as a Description about the nature of the account (personal, work, school etc.) -* Emails will be removed from the Contacts table - -# Modifying the Relations (continued) +--- +```csharp +// Models/Contact.cs public class Contact - { - -public int Id { get; set; } - -public string Name { get; set; } - -public ICollection Accounts { get; set; } - + public int Id { get; set; } + public string Name { get; set; } + public ICollection Accounts { get; set; } } +``` +```csharp +// Models/Account.cs public class Account - { - -public int Id { get; set; } - -public string Email { get; set; } - -public string Description { get; set; } - -public int ContactId { get; set; } - -public Contact Contact { get; set; } - + public int Id { get; set; } + public string Email { get; set; } + public string Description { get; set; } + public int ContactId { get; set; } + public Contact Contact { get; set; } } +``` -Adding a migration at this point will result in a warning: - -![](imgs/7-databases-with-entity-framework_14.png) +--- -* In the generated migration file, you can find Up and Down methods - * The Up method describes the changes that will be made with the migration - * In this case, removing the Email column from Contacts table, and creating the new Accounts table - * The Down method describes the changes that will be made if the migration is reverted -* Updating the database will still work, and the database will have a new table, Accounts +* Adding a migration at this point will result in a warning: + ![w:1000px](imgs/7-databases-with-entity-framework_14.png) -![](imgs/7-databases-with-entity-framework_15.png) +--- -# Exercise 6: Adding Migrations +
+
-Continue working on CourseAPI. +* In the generated migration file, you can find `Up` and `Down` methods + * The `Up` method describes the changes that will be made with the migration + * In this case, removing the `Email` column from `Contacts` table, and creating the new `Accounts` table + * The `Down` method describes the changes that will be made if the migration is reverted +* Updating the database will still work, and the database will have a new table `Accounts` -Add a new model Lecture with properties int Id, DateTime StartTime, int Length, Course Course, and int CourseId +
+
-Add a new property ICollection Lectures to the Course model +![](imgs/7-databases-with-entity-framework_15.png) -Add a new migration, named _AddLectures_ +
+
-Update the database. Check that the changes show up in the database with pgAdmin +### Exercise 3: Adding Migrations + -# Database First approach +Continue working on CourseAPI. -![](imgs/7-databases-with-entity-framework_16.png) +1) Add a new model `Lecture` with properties `int Id`, `DateTime StartTime`, `int Length`, `Course Course`, and `int CourseId` +2) Add a new property `ICollection Lectures` to the `Course` model +3) Add a new migration named `AddLectures` +4) Update the database. Check that the changes show up in the database with pgAdmin ---- +## Database First approach -# Database First +### What is the Database First approach? * This is the other approach for creating a connection between the database and the application -* Databases and tables are created first, then you create entity data model using the created database -* Preferred for data intense, large applications +* Databases and tables are created first, then you create an entity data model using the created database +* This approach is preferred for data intense, large applications * Other advantages include: * Data model is simple to create * GUI * You do not need to write any code to create your database -# Database First (continued) +### Scaffolding * Use the Package Manager Console to "reverse engineer" the code for an existing database - * This is called __scaffolding__ + * This is called *__scaffolding__* * Scaffold the database with the following command: -* Scaffold-DbContext "Server=PostgreSQL 12;Host=localhost;Port=5432;Username=postgres;Password=1234;Database=sqlpractice" Npgsql.EntityFrameworkCore.PostgreSQL -OutputDir Models + ```powershell + Scaffold-DbContext "Server=PostgreSQL 12;Host=localhost;Port=5432;Username=postgres;Password=1234;Database=sqlpractice" Npgsql.EntityFrameworkCore.PostgreSQL -OutputDir Models + ``` * Using the connection string corresponding to your database, this will create all the classes for the entities in the DB as well as the context class -# Exercise 7: Database First +### Exercise 4: Database First + Create a new ASP.NET Core web app using the API template. -Install the required NuGet packages for using EFCore, EFCore Tools and PostgreSQL by using the package manager, or by copying the lines from the .csproj file of the previous assignment to this projects .csproj file - -Scaffold the _sqlpractice _ database (which was created in the Exercise 1) to the project using the Database First approach. If you have not yet created the database in PostGreSQL, it can be found [here](https://buuttilab.com/education/trainee-academy-joensuu/assignments/-/raw/94ddeaaa5abb240c0035d718d082df8704488045/Databases/initdb.txt) +1) Install the required NuGet packages for using EFCore, EFCore Tools and PostgreSQL + a) by using the package manager, or + b) by copying the `` lines from the `.csproj` file of the previous assignment to this project's `.csproj` file +2) Scaffold the `sqlpractice` database created in [SQL Databases Exercise 1](sql-databases#exercise-1-preparing-the-database) to the project by using the Database First approach. If you have not yet created the database in PostgreSQL, it can be found [here](code-examples/example-query.sql) -# Reading: Authentication with roles +### Reading: Authentication with roles -* Here's an example how to do Role-based Authentication by using JWT tokens - * [https://www.c-sharpcorner.com/article/jwt-token-creation-authentication-and-authorization-in-asp-net-core-6-0-with-po/](https://www.c-sharpcorner.com/article/jwt-token-creation-authentication-and-authorization-in-asp-net-core-6-0-with-po/) +* [Here's](https://www.c-sharpcorner.com/article/jwt-token-creation-authentication-and-authorization-in-asp-net-core-6-0-with-po/) an example how to do a role-based authentication by using JWT tokens -# Extra: Exercise 8: Connection - -[under construction] +### Exercise 5 (Extra): Connection + Continuing the previous exercise, -Create and connect Postgres database to API and create a second entity with a relation to the first entity. Test your solution. - +1) Create and connect Postgres database to API and create a second entity with a relation to the first entity. +2) Test your solution. diff --git a/code-examples/example-query.sql b/code-examples/example-query.sql new file mode 100644 index 0000000..9584c72 --- /dev/null +++ b/code-examples/example-query.sql @@ -0,0 +1,96 @@ +CREATE TABLE "users" ( + "id" SERIAL PRIMARY KEY, + "full_name" varchar NOT NULL, + "email" varchar UNIQUE NOT NULL, + "created_at" timestamp NOT NULL +); + +CREATE TABLE "genres" ( + "id" SERIAL PRIMARY KEY, + "name" varchar UNIQUE NOT NULL +); + +CREATE TABLE "languages" ( + "id" SERIAL PRIMARY KEY, + "name" varchar UNIQUE NOT NULL +); + +CREATE TABLE "books" ( + "id" SERIAL PRIMARY KEY, + "name" varchar NOT NULL, + "release_year" int NOT NULL, + "genre_id" int NOT NULL, + "language_id" int NOT NULL +); + +CREATE TABLE "volumes" ( + "id" SERIAL PRIMARY KEY, + "book_id" int NOT NULL +); + +CREATE TABLE "borrows" ( + "id" SERIAL PRIMARY KEY, + "volume_id" int NOT NULL, + "user_id" int NOT NULL, + "borrowed_at" timestamp NOT NULL, + "due_date" timestamp NOT NULL, + "returned_at" timestamp +); + +ALTER TABLE "books" ADD FOREIGN KEY ("genre_id") REFERENCES "genres" ("id"); + +ALTER TABLE "books" ADD FOREIGN KEY ("language_id") REFERENCES "languages" ("id"); + +ALTER TABLE "volumes" ADD FOREIGN KEY ("book_id") REFERENCES "books" ("id"); + +ALTER TABLE "borrows" ADD FOREIGN KEY ("volume_id") REFERENCES "volumes" ("id"); + +ALTER TABLE "borrows" ADD FOREIGN KEY ("user_id") REFERENCES "users" ("id"); + + +INSERT INTO users (full_name, email, created_at) +VALUES + ('Teppo Testaaja', 'teppo.testaaja@buutti.com', NOW()), + ('Taija Testaaja', 'taija.testaaja@buutti.com', NOW()), + ('Outi Ohjelmoija', 'outi.ohjelmoija@buutti.com', NOW()), + ('Olli Ohjelmoija', 'olli.ohjelmoija@buutti.com', NOW()); + +INSERT INTO genres (name) +VALUES + ('Scifi'), + ('Fantasy'), + ('Comic book'), + ('Horror'), + ('Drama'); + +INSERT INTO languages (name) +VALUES + ('Finnish'), + ('English'), + ('Swedish'); + +INSERT INTO books (name, release_year, genre_id, language_id) +VALUES + ('Taru Sormusten Herrasta', 1954, 2, 1), + ('Silmarillion', 1977, 2, 2), + ('The Hitchhikers Guide to the Galaxy', 1979, 1, 2); + +INSERT INTO volumes (book_id) + VALUES + (1), + (1), + (1), + (1), + (1), + (1), + (2), + (2), + (2), + (3), + (3); + +INSERT INTO borrows (volume_id, user_id, borrowed_at, due_date, returned_at) +VALUES + (1, 1, current_timestamp - (30 * interval '1 day'), current_timestamp, null), + (2, 2, current_timestamp - (35 * interval '1 day'), current_timestamp - (5 * interval '1 day'), null), + (7, 1, current_timestamp - (40 * interval '1 day'), current_timestamp - (10 * interval '1 day'), current_timestamp - (15 * interval '1 day')); \ No newline at end of file diff --git a/imgs/7-databases-with-entity-framework_3.png b/imgs/7-databases-with-entity-framework_3.png index e60521c..9e7dee7 100644 Binary files a/imgs/7-databases-with-entity-framework_3.png and b/imgs/7-databases-with-entity-framework_3.png differ diff --git a/imgs/7-databases-with-entity-framework_7.png b/imgs/7-databases-with-entity-framework_7.png index 35a1812..d0555d8 100644 Binary files a/imgs/7-databases-with-entity-framework_7.png and b/imgs/7-databases-with-entity-framework_7.png differ diff --git a/imgs/7-databases-with-entity-framework_8.png b/imgs/7-databases-with-entity-framework_8.png index 4190de0..30401dc 100644 Binary files a/imgs/7-databases-with-entity-framework_8.png and b/imgs/7-databases-with-entity-framework_8.png differ diff --git a/sql-databases-slides.html b/sql-databases-slides.html new file mode 100644 index 0000000..adfaefb --- /dev/null +++ b/sql-databases-slides.html @@ -0,0 +1,431 @@ +SQL Databases
+

SQL Databases

+
+
+

What is SQL?

+
    +
  • SQL, Structured Query Language is a language used to organize and manipulate data in a database
    • +
  • Originally developed by IBM in the 70's +
      +
    • Quickly became the most popular database language
      • +
    +
    SELECT id, email
+FROM users
+WHERE first_name = 'Teppo';
+
    +
    • +
+
+
+

Relational Database Management Systems

+
    +
  • In relational databases, values are stored in tables +
      +
    • Each table has rows and columns
      • +
    • Data is displayed in a two-dimensional matrix
      • +
    +
    • +
  • Values in a table are related to each other +
      +
    • Values can also be related to values in other tables
      • +
    +
    • +
  • A relational database management system (RDBMS) is a program that executes queries to relational databases
    • +
+
+
+

+

https://db-engines.com/en/ranking

+
+
+

PostgreSQL

+
    +
  • Free and open-source, cross-platform relational database management system
    • +
  • Emphasizes extensibility and SQL compliance
    • +
  • Fully ACID-compliant (atomicity, consistency, isolation and durability)
    • +
  • Used in conjunction with pgAdmin +
      +
    • Administration and development platform for PostgreSQL
      • +
    • Cross-platform, features a web interface
      • +
    • Basically a control panel application for your PostgreSQL database
      • +
    +
    • +
+
+
+

PostgreSQL: Creating a Database

+
+
+

With pgAdmin:

+
    +
  1. Right-click Servers > my-postgres > Databases
    2. +
  2. Select Create > Database...
    3. +
  3. Insert a name for the database and hit Save
    4. +
+
+
+

With psql:

+
    +
  1. Enter command
    CREATE DATABASE <database-name>;
+
    +
    2. +
+
+
+
+

+
+
+
+

PostgreSQL: Querying

+

With pgAdmin:

+
    +
  1. Right-click sqlpractice > Query tool...
    2. +
  2. Insert a query into the Query Editor and hit Execute (F5)
    3. +
+
+
+

+
+
+

+
+
+
+
+

With psql:

+
    +
  1. List all available databases with \l
    2. +
  2. Connect to the created database with \c <database-name>
    3. +
  3. List all tables in the database with \dt
    4. +
  4. Type a query and press enter + +
    5. +
+
+
+

Editing Data with pgAdmin

+
+
+
    +
  • Inspect and edit data in pgAdmin by right-clicking a table and selecting View/Edit Data
    • +
+
+
+

+
+
+
+
+
+
+
    +
  • Individual values in the table can be directly modified by double clicking the value and then editing the value in the visual user interface
    • +
  • Save the changes by pressing the Save Data Changes button
    • +
+
+
+


+

+
+
+
+
+

Exercise 1: Preparing the database

+ +

Using either PgAdmin or PSQL,

+
    +
  1. Create a new database called sqlpractice
    2. +
  2. Insert the provided example query to the new database
    3. +
  3. Verify that the query has created new tables to your database
    4. +
+
+
+

Basic SQL queries

+
    +
  • SELECT
    • +
  • INSERT
    • +
  • DELETE
    • +
  • UPDATE
    • +
  • CREATE & DROP
    • +
+
+
+

Querying data with SELECT

+
    +
  • Syntax:
    SELECT column1, column2, column3 FROM table_name;
+
    +
    • +
  • Examples:
    SELECT full_name, email FROM users;
+SELECT full_name AS name, email FROM users;
+SELECT * FROM users;
+
    +
    • +
+
+
+

Filtering data with WHERE

+
    +
  • Syntax:
    SELECT column1, column2 FROM table_name WHERE condition;
+
    +
    • +
+
+
+
    +
  • Text is captured in single quotes.
    • +
  • LIKE condition uses the % sign as a wildcard.
    • +
  • IS and IS NOT are also valid operators.
    • +
+
+
+
    +
  • Example:
    SELECT full_name FROM users
+WHERE full_name = 'Teppo Testaaja';
+
+SELECT * FROM books WHERE name LIKE '%rr%';
+SELECT * FROM books WHERE author IS NOT null;
+
    +
    • +
+
+
+
+
+

Ordering data with ORDER BY

+
    +
  • Syntax:
    SELECT column1 FROM table_name ORDER BY column1 ASC;
+
    +
    • +
  • Examples:
    SELECT full_name FROM users
+ORDER BY full_name ASC;
+
+SELECT full_name FROM users
+ORDER BY full_name DESC;
+
    +
    • +
+
+
+

Combining data with JOIN

+
    +
  • Also known as INNER JOIN
    • +
  • Corresponds to intersection from set theory
    • +
+
+

+
+
+
+

JOIN examples

+
SELECT
+users.id, users.full_name, borrows.id,
+borrows.user_id, borrows.due_date, borrows.returned_at
+FROM users
+JOIN borrows ON
+users.id = borrows.user_id;
+
+
SELECT
+U.full_name AS name,
+B.due_date AS due_date,
+B.returned_at AS returned_at
+FROM users AS U
+JOIN borrows AS B ON
+U.id = B.user_id;
+
+
+
+

Combining with LEFT JOIN

+
+
+
    +
  • Also known as LEFT OUTER JOIN
    • +
  • Example:
    SELECT
+U.full_name AS name,
+B.due_date AS due_date,
+B.returned_at AS returned_at
+FROM users AS U
+LEFT JOIN borrows AS B ON
+U.id = B.user_id;
+
    +
    • +
+
+
+

+
+
+
+
+

Exercise 2: Querying the library

+ +

Using SQL queries, get

+
    +
  1. all columns of borrows that are borrowed before 2020-10-27
    2. +
  2. all columns of borrows that are returned
    3. +
  3. columns user.full_name and borrows.borrowed_at of the user with an id of 1
    4. +
  4. columns book.name, book.release_year and language.name of all books that are released after 1960
    5. +
+
+
+

Inserting data with INSERT

+
    +
  • Syntax
    INSERT INTO table_name (column1, column2, column3) VALUES (value1, value2, value3);
+
    +
    • +
  • Example
    INSERT INTO users (full_name, email, created_at)
+VALUES ('Pekka Poistuja', 'pekka.poistuja@buutti.com', NOW());
+
    +
    • +
  • Since id is not provided, it will be automatically generated.
    • +
+
+
+

Updating data with UPDATE

+
    +
  • Syntax
    UPDATE table_name SET column1 = value1, column2 = value2 WHERE condition;
+
    +
    • +
  • Notice: if a condition is not provided, all rows will be updated! +
      +
    • If updating only one row, it is usually best to use id.
      • +
    +
    • +
  • Example
    UPDATE users SET email = 'taija.testaaja@gmail.com' WHERE id = 2;
+
    +
    • +
+
+
+

Removing data with REMOVE

+
    +
  • Syntax
    DELETE FROM table_name WHERE condition;
+
    +
    • +
  • Again, if the condition is not provided, DELETE affects all rows
    • +
  • Before deleting, it is a good practice to execute an equivalent SELECT query to make sure that only the proper rows will be affected.
    • +
  • Example:
    SELECT * FROM users WHERE id = 5;
+DELETE FROM users WHERE id = 5;
+
    +
    • +
+
+
+

Exercise 3: Editing data

+ +
    +
  1. Postpone the due date of the borrow with an id of 1 by two days in the borrows table
    2. +
  2. Add a couple of new books to the books table
    3. +
  3. Delete one of the borrows.
    4. +
+
+
+

Initializing data with CREATE TABLE

+
    +
  • Before data can be manipulated, a database and its tables need to be initialized.
    • +
+
+
+
    +
  • Syntax
    CREATE TABLE table_name (
+  column1 datatype,
+  column2 datatype,
+  ...
+);
+
    +
    • +
+
+
+
    +
  • Example:
    CREATE TABLE "users" (
+  "id" SERIAL PRIMARY KEY,
+  "full_name" varchar NOT NULL,
+  "email" varchar UNIQUE NOT NULL,
+  "created_at" timestamp NOT NULL
+);
+
    +
    • +
+
+
+
+
+

Removing data with DROP

+
    +
  • In order to remove tables or databases, we use a DROP statement
    DROP TABLE table_name;
+DROP DATABASE database_name;
+
    +
    • +
  • These statements do not ask for confirmation and there is no undo feature. Take care when using a drop statement.
    • +
+
+
+
    +
  • I'm legally obliged to include this XKCD comic here.
    • +
+

+
+
+

NoSQL

+
    +
  • In addition to SQL databases, there's NoSQL
    • +
  • There are many differing definitions, but... +
      +
    • most agree that NoSQL databases store data in a format other than tables
      • +
    • They can still store relational data - just differently
      • +
    +
    • +
  • Four different database types: +
      +
    • Document databases
      • +
    • Key-value databases
      • +
    • Wide-column stores
      • +
    • Graph databases
      • +
    +
    • +
  • Example database engines include MongoDB, Redis and Cassandra
    • +
  • For more info, see MongoDB: What is NoSQL?
    • +
+
+
+

Object-Relational Mappers

+
    +
  • Object-Relational Mappers, or ORMs allow the developer to write code instead of SQL to perform CRUD operations on their database
    • +
  • An abstraction layer between code and database +
      +
    • Can make database logic easier to read and write
      • +
    • Prevents SQL injection by sanitizing input (see the comic before...)
      • +
    • ...but sometimes you'd rather just write the SQL queries themselves
      • +
    +
    • +
  • Some popular ORMs: +
      +
    • Hibernate (Java)
      • +
    • EFCore (.NET)
      • +
    • Sequelize (Node.js)
      • +
    • TypeORM (TypeScript)
      • +
    +
    • +
+
+
\ No newline at end of file diff --git a/sql-databases.md b/sql-databases.md new file mode 100644 index 0000000..6bedbd8 --- /dev/null +++ b/sql-databases.md @@ -0,0 +1,400 @@ +--- +marp: true +paginate: true +math: mathjax +theme: buutti +title: SQL Databases +--- + +# SQL Databases + + + + +## What is SQL? + +* SQL, Structured Query Language is a language used to organize and manipulate data in a database +* Originally developed by IBM in the 70's + * Quickly became the most popular database language + ```sql + SELECT id, email + FROM users + WHERE first_name = 'Teppo'; + ``` + +## Relational Database Management Systems + +* In relational databases, values are stored in *__tables__* + * Each table has *__rows__* and *__columns__* + * Data is displayed in a two-dimensional matrix +* Values in a table are related to each other + * Values can also be related to values in other tables +* A relational database management system (RDBMS) is a program that executes queries to relational databases + +--- + +![](imgs/7-databases-with-entity-framework_0.png) + +[https://db-engines.com/en/ranking](https://db-engines.com/en/ranking) + +## PostgreSQL + +* Free and open-source, cross-platform relational database management system +* Emphasizes extensibility and SQL compliance +* Fully ACID-compliant (atomicity, consistency, isolation and durability) +* Used in conjunction with ***pgAdmin*** + * Administration and development platform for PostgreSQL + * Cross-platform, features a web interface + * Basically a control panel application for your PostgreSQL database + +### PostgreSQL: Creating a Database + +
+
+ +With pgAdmin: +1) Right-click *Servers > my-postgres > Databases* +2) Select _Create > Database..._ +3) Insert a name for the database and hit _Save_ + +
+
+ +With psql: +1) Enter command + ```sql + CREATE DATABASE ; + ``` + +
+
+ +
+ +![](imgs/7-databases-with-entity-framework_1.png) + +
+ +### PostgreSQL: Querying + +With pgAdmin: + +1) Right-click _sqlpractice > Query tool..._ +2) Insert a query into the _Query Editor_ and hit _Execute_ (F5) + +
+
+ +![w:345](imgs/7-databases-with-entity-framework_2.png) + +
+
+ +![](imgs/7-databases-with-entity-framework_3.png) + +
+
+ +--- + +With psql: + +1) List all available databases with `\l` +2) Connect to the created database with `\c ` +3) List all tables in the database with `\dt` +4) Type a query and press enter + * Here's an [example query](code-examples/example-query.sql) + +## Editing Data with pgAdmin + +
+
+ +* Inspect and edit data in pgAdmin by right-clicking a table and selecting _View/Edit Data_ + +
+
+ +![](imgs/7-databases-with-entity-framework_4.png) + +
+
+ +--- + +
+
+ +* Individual values in the table can be directly modified by double clicking the value and then editing the value in the visual user interface +* Save the changes by pressing the _Save Data Changes_ button + +
+
+ +![](imgs/7-databases-with-entity-framework_5.png) +![](imgs/7-databases-with-entity-framework_6.png) + +
+
+ +## Exercise 1: Preparing the database + + +Using either PgAdmin or PSQL, + +1) Create a new database called `sqlpractice` +2) Insert the provided [example query](code-examples/example-query.sql) to the new database +3) Verify that the query has created new tables to your database + +## Basic SQL queries + +* `SELECT` +* `INSERT` +* `DELETE` +* `UPDATE` +* `CREATE` & `DROP` + +### Querying data with `SELECT` + +* Syntax: + ```sql + SELECT column1, column2, column3 FROM table_name; + ``` +* Examples: + ```sql + SELECT full_name, email FROM users; + SELECT full_name AS name, email FROM users; + SELECT * FROM users; + ``` + +### Filtering data with `WHERE` + +* Syntax: + ```sql + SELECT column1, column2 FROM table_name WHERE condition; + ``` +
+
+ +* Text is captured in **_single quotes_**. +* `LIKE` condition uses the `%` sign as a wildcard. +* `IS` and `IS NOT` are also valid operators. + +
+
+ +* Example: + ```sql + SELECT full_name FROM users + WHERE full_name = 'Teppo Testaaja'; + + SELECT * FROM books WHERE name LIKE '%rr%'; + SELECT * FROM books WHERE author IS NOT null; + ``` + +
+
+ +### Ordering data with `ORDER BY` + +* Syntax: + ```sql + SELECT column1 FROM table_name ORDER BY column1 ASC; + ``` +* Examples: + ```sql + SELECT full_name FROM users + ORDER BY full_name ASC; + + SELECT full_name FROM users + ORDER BY full_name DESC; + ``` + +### Combining data with `JOIN` + +* Also known as `INNER JOIN` +* Corresponds to ***intersection*** from [set theory](https://en.wikipedia.org/wiki/Set_theory) + +
+ +![w:500px](imgs/7-databases-with-entity-framework_7.png) + +
+ +#### JOIN examples + +```sql +SELECT +users.id, users.full_name, borrows.id, +borrows.user_id, borrows.due_date, borrows.returned_at +FROM users +JOIN borrows ON +users.id = borrows.user_id; +``` + +```sql +SELECT +U.full_name AS name, +B.due_date AS due_date, +B.returned_at AS returned_at +FROM users AS U +JOIN borrows AS B ON +U.id = B.user_id; +``` + +### Combining with `LEFT JOIN` + +
+
+ +* Also known as `LEFT OUTER JOIN` +* Example: + ```sql + SELECT + U.full_name AS name, + B.due_date AS due_date, + B.returned_at AS returned_at + FROM users AS U + LEFT JOIN borrows AS B ON + U.id = B.user_id; + ``` + +
+
+ +![](imgs/7-databases-with-entity-framework_8.png) + +
+
+ +### Exercise 2: Querying the library + + +Using SQL queries, get + +1) all columns of `borrows` that are borrowed before `2020-10-27` +2) all columns of `borrows` that are returned +3) columns `user.full_name` and `borrows.borrowed_at` of the user with an `id` of 1 +4) columns `book.name`, `book.release_year` and `language.name` of all books that are released after 1960 + +### Inserting data with `INSERT` + +* Syntax + ```sql + INSERT INTO table_name (column1, column2, column3) VALUES (value1, value2, value3); + ``` +* Example + ```sql + INSERT INTO users (full_name, email, created_at) + VALUES ('Pekka Poistuja', 'pekka.poistuja@buutti.com', NOW()); + ``` +* Since id is not provided, it will be automatically generated. + +### Updating data with `UPDATE` + +* Syntax + ```sql + UPDATE table_name SET column1 = value1, column2 = value2 WHERE condition; + ``` +* *__Notice:__* if a condition is not provided, all rows will be updated! + * If updating only one row, it is usually best to use `id`. +* Example + ```sql + UPDATE users SET email = 'taija.testaaja@gmail.com' WHERE id = 2; + ``` + +## Removing data with `REMOVE` + +* Syntax + ```sql + DELETE FROM table_name WHERE condition; + ``` +* Again, if the _condition_ is not provided, `DELETE` affects _all_ rows +* Before deleting, it is a good practice to execute an equivalent `SELECT` query to make sure that only the proper rows will be affected. +* Example: + ```sql + SELECT * FROM users WHERE id = 5; + DELETE FROM users WHERE id = 5; + ``` + +### Exercise 3: Editing data + + +1) Postpone the due date of the borrow with an `id` of `1` by two days in the `borrows` table +2) Add a couple of new books to the `books` table +3) Delete one of the borrows. + +### Initializing data with `CREATE TABLE` + +* Before data can be manipulated, a database and its tables need to be initialized. + +
+
+ +* Syntax + ```sql + CREATE TABLE table_name ( + column1 datatype, + column2 datatype, + ... + ); + ``` + +
+
+ +* Example: + ```sql + CREATE TABLE "users" ( + "id" SERIAL PRIMARY KEY, + "full_name" varchar NOT NULL, + "email" varchar UNIQUE NOT NULL, + "created_at" timestamp NOT NULL + ); + ``` + +
+
+ +### Removing data with `DROP` + +* In order to remove tables or databases, we use a `DROP` statement + ```sql + DROP TABLE table_name; + DROP DATABASE database_name; + ``` +* These statements do not ask for confirmation and there is no undo feature. Take care when using a drop statement. + +--- + +* I'm legally obliged to include this XKCD comic here. + +![](https://imgs.xkcd.com/comics/exploits_of_a_mom_2x.png) + +## NoSQL + +* In addition to SQL databases, there's ***NoSQL*** +* There are many differing definitions, but... + * most agree that NoSQL databases store data in a format other than tables + * They can still store relational data - just _differently_ +* Four different database types: + * Document databases + * Key-value databases + * Wide-column stores + * Graph databases +* Example database engines include [MongoDB](https://www.mongodb.com/), [Redis](https://redis.io/) and [Cassandra](https://cassandra.apache.org/_/index.html) +* For more info, see [MongoDB: What is NoSQL?](https://www.mongodb.com/resources/basics/databases/nosql-explained) + + +## Object-Relational Mappers + +* Object-Relational Mappers, or ORMs allow the developer to write code instead of SQL to perform CRUD operations on their database +* An abstraction layer between code and database + * Can make database logic easier to read and write + * Prevents SQL injection by sanitizing input (see the comic before...) + * ...but sometimes you'd rather just write the SQL queries themselves +* Some popular ORMs: + * Hibernate (Java) + * EFCore (.NET) + * Sequelize (Node.js) + * TypeORM (TypeScript)