add sql database lecture

- add bulk of slides from asp.net
- add docker slides from backend-basics

code-examples/example-query.sql

@@ -0,0 +1,96 @@
+CREATE TABLE "users" (
+  "id" SERIAL PRIMARY KEY,
+  "full_name" varchar NOT NULL,
+  "email" varchar UNIQUE NOT NULL,
+  "created_at" timestamp NOT NULL
+);
+
+CREATE TABLE "genres" (
+  "id" SERIAL PRIMARY KEY,
+  "name" varchar UNIQUE NOT NULL
+);
+
+CREATE TABLE "languages" (
+  "id" SERIAL PRIMARY KEY,
+  "name" varchar UNIQUE NOT NULL
+);
+
+CREATE TABLE "books" (
+  "id" SERIAL PRIMARY KEY,
+  "name" varchar NOT NULL,
+  "release_year" int NOT NULL,
+  "genre_id" int NOT NULL,
+  "language_id" int NOT NULL
+);
+
+CREATE TABLE "volumes" (
+  "id" SERIAL PRIMARY KEY,
+  "book_id" int NOT NULL
+);
+
+CREATE TABLE "borrows" (
+  "id" SERIAL PRIMARY KEY,
+  "volume_id" int NOT NULL,
+  "user_id" int NOT NULL,
+  "borrowed_at" timestamp NOT NULL,
+  "due_date" timestamp NOT NULL,
+  "returned_at" timestamp
+);
+
+ALTER TABLE "books" ADD FOREIGN KEY ("genre_id") REFERENCES "genres" ("id");
+
+ALTER TABLE "books" ADD FOREIGN KEY ("language_id") REFERENCES "languages" ("id");
+
+ALTER TABLE "volumes" ADD FOREIGN KEY ("book_id") REFERENCES "books" ("id");
+
+ALTER TABLE "borrows" ADD FOREIGN KEY ("volume_id") REFERENCES "volumes" ("id");
+
+ALTER TABLE "borrows" ADD FOREIGN KEY ("user_id") REFERENCES "users" ("id");
+
+
+INSERT INTO users (full_name, email, created_at)
+VALUES 
+    ('Teppo Testaaja', 'teppo.testaaja@buutti.com', NOW()),
+    ('Taija Testaaja', 'taija.testaaja@buutti.com', NOW()),
+    ('Outi Ohjelmoija', 'outi.ohjelmoija@buutti.com', NOW()),
+    ('Olli Ohjelmoija', 'olli.ohjelmoija@buutti.com', NOW());
+
+INSERT INTO genres (name)
+VALUES 
+    ('Scifi'),
+    ('Fantasy'),
+    ('Comic book'),
+    ('Horror'),
+    ('Drama');
+
+INSERT INTO languages (name)
+VALUES
+    ('Finnish'),
+    ('English'),
+    ('Swedish');
+
+INSERT INTO books (name, release_year, genre_id, language_id)
+VALUES
+    ('Taru Sormusten Herrasta', 1954, 2, 1),
+    ('Silmarillion', 1977, 2, 2),
+    ('The Hitchhikers Guide to the Galaxy', 1979, 1, 2);
+
+INSERT INTO volumes (book_id)
+    VALUES 
+        (1),
+        (1),
+        (1),
+        (1),
+        (1),
+        (1),
+        (2),
+        (2),
+        (2),
+        (3),
+        (3);
+
+INSERT INTO borrows (volume_id, user_id, borrowed_at, due_date, returned_at)
+VALUES
+    (1, 1, current_timestamp - (30 * interval '1 day'), current_timestamp, null),
+    (2, 2, current_timestamp - (35 * interval '1 day'), current_timestamp - (5 * interval '1 day'), null),
+    (7, 1, current_timestamp - (40 * interval '1 day'), current_timestamp - (10 * interval '1 day'), current_timestamp - (15 * interval '1 day'));

sql-databases.md

@@ -0,0 +1,598 @@
+---
+marp: true
+paginate: true
+math: mathjax
+theme: buutti
+title: SQL Databases
+---
+
+# SQL Databases
+
+<!-- headingDivider: 3 -->
+<!-- class: invert -->
+
+## What is SQL?
+
+* SQL, Structured Query Language is a language used to organize and manipulate data in a database
+* Originally developed by IBM in the 70's
+  * Quickly became the most popular database language
+  ```sql
+  SELECT id, email
+  FROM users
+  WHERE first_name = 'Teppo';
+  ```
+
+### Relational Database Management Systems
+
+* In relational databases, values are stored in *__tables__*
+  * Each table has *__rows__* and  *__columns__*
+  * Data is displayed in a two-dimensional matrix
+* Values in a table are related to each other
+  * Values can also be related to values in other tables
+* A relational database management system (RDBMS) is a program that executes queries to relational databases
+
+### Database management systems compared
+
+![](imgs/3-databases-with-docker_0.png)
+
+[https://db-engines.com/en/ranking](https://db-engines.com/en/ranking)
+
+## PostgreSQL
+
+* [PostgreSQL](https://www.postgresql.org/) (pronounced *postgres-QL*) is a free and open-source, cross-platform relational database management system
+* Emphasizes extensibility and SQL compliance
+* Fully [ACID](https://en.wikipedia.org/wiki/ACID)-compliant (atomicity, consistency, isolation and durability)
+* Used in conjunction with an interface like ***pgAdmin*** (graphical) or ***psql*** (command-line)
+* Once installed, can be accessed with the SQL Shell (see instructions [here](https://www.w3schools.com/postgresql/postgresql_getstarted.php))
+
+### Extra: Running Postgres in Docker
+<!-- _class: "extra invert" -->
+
+* Instead of running a PostgreSQL database locally, you can run it *Dockerized*
+  * See [Devops Lecture 1: Docker]()
+* Using the official  [Postgres Docker image](https://hub.docker.com/_/postgres) , let's create a locally running Postgres instance
+	```bash
+	docker run --name my-postgres --env POSTGRES_PASSWORD=pgpass 
+	--env POSTGRES_USER=pguser -p 5432:5432 -d postgres:15.2
+	```
+* You can choose the values for `POSTGRES_PASSWORD` and `POSTGRES_USER` freely.
+	![](imgs/3-databases-with-docker_1.png)
+
+## `psql`
+
+* psql (see [Documentation](https://www.postgresql.org/docs/current/app-psql.html
+)) is a terminal-based interface for PostgreSQL and comes bundled with it
+  * With psql, you can type in SQL queries interactively and see the results
+  * Input from file, command line arguments, scripting and automation...
+
+<div class='columns' markdown='1'>
+<div markdown='1'>
+
+#### With Docker:
+
+* You can use psql directly from the Docker container running the database	
+	```bash
+	docker exec -it my-postgres psql -U pguser
+	```
+
+</div>
+<div markdown='1'>
+
+#### Without Docker:
+
+* Just open the SQL shell from Start menu.
+
+</div>
+</div>
+
+
+### `psql` commands
+
+* If you have connected to the server as user `pguser`, you are by default connected to `pguser` database.
+* This is a database for user information. *Do not use it for storing program data!*
+* All databases can be listed using the *list* command: `\l`
+* PostgreSQL uses a default database named `postgres`
+* Users can connect to a different database with the ***connect*** command `\c`:
+  ```psql
+  \c <database-name>
+  ```
+* Exit `psql` with the command `exit`
+
+### Exercise 1. psql basics
+<!--_class: "exercise invert" -->
+
+1) If using Docker, start a local instance of Postgres in Docker.
+2) Connect to the server using psql.
+3) Use command `\l` to see what databases there are already created on the server.
+4) Create a new database called `sqlpractice`.
+5) Connect to the newly created database.
+
+## pgAdmin
+
+* [pgAdmin](https://www.pgadmin.org/) is an administration and development platform for PostgreSQL
+
+<div class='columns' markdown='1'>
+<div markdown='1'>
+
+* Cross-platform, features a web interface
+* A control panel frontend for your PostgreSQL database
+* A graphical alternative to psql
+* Completely separate from PostgreSQL
+* One of many database management applications
+  * Another major one is [DBeaver](https://dbeaver.io/)
+
+</div>
+<div markdown='1'>
+
+![](imgs/3-databases-with-docker_1_5.png)
+
+</div>
+</div>
+
+### Extra: Running pgAdmin in Docker
+<!-- _class: "extra invert" -->
+
+* Using the official [pgAdmin](https://hub.docker.com/r/dpage/pgadmin4) image, we'll run pgAdmin alongside Postgres
+	```
+	docker run --name my-pgadmin -p 5050:80 
+	--env PGADMIN_DEFAULT_EMAIL=<your-email-address> 
+	--env PGADMIN_DEFAULT_PASSWORD=<your-password> 
+	-d dpage/pgadmin4
+	```
+* ![](imgs/3-databases-with-docker_2.png)
+
+---
+<!-- _class: "extra invert" -->
+
+* Both PostgreSQL and pgAdmin are now running in our *local* Docker. To connect pgAdmin to PostgreSQL, we need the IP address used inside Docker.
+* Using Docker's inspect, we'll get Docker's internal IP for `my-postgres -container`. Since the command produces quite a lot of information, we pipe the result to `grep` (Linux) or `findstr` (Windows) to see only the rows that contain the word `IPAddress`
+
+<div class='columns' markdown='1'>
+<div markdown='1'>
+
+#### Linux
+
+```bash
+docker inspect <container-name> | grep IPAddress
+```
+
+</div>
+<div markdown='1'>
+
+#### Windows
+
+```powershell
+docker inspect <container-name> | findstr IPAddress
+```
+
+</div>
+</div>
+
+* In the example output, the IP Address is `172.17.0.2`:
+	![](imgs/3-databases-with-docker_4.png)
+
+### Logging into pgAdmin
+
+<div class='columns' markdown='1'>
+<div markdown='1'>
+
+#### With Docker:
+1) With pgAdmin running, navigate your web browser to http://localhost:5050 
+
+</div>
+<div markdown='1'>
+
+#### Without Docker:
+1) Run pgAdmin from the Start menu.
+
+</div>
+</div>
+
+2) Use the username and password you provided to log in.
+<div class='centered'>
+
+![w:650px](imgs/3-databases-with-docker_3.png)
+
+</div>
+
+### Connecting PgAdmin to our DB
+
+<div class='columns' markdown='1'>
+<div markdown='1'>
+
+1) Now we have all that we need for a connection. In pgAdmin, select<br> *Object > Register > Server*.
+     * If the *Object* menu is greyed out, click on *Servers*.
+	![](imgs/3-databases-with-docker_6.png)
+2) In the *General* tab, give the server a name that identifies the connection.
+
+</div>
+<div markdown='1'>
+
+![w:500px](imgs/3-databases-with-docker_5.png)
+
+</div>
+</div>
+
+
+---
+
+<div class='columns' markdown='1'>
+<div markdown='1'>
+
+3) In the *Connection* tab, enter:
+   1) *Host name/address*
+     * the internal Docker address
+   2) *Port*, *Username*, and *Password*
+     * the values defined when running the PostgreSQL container
+
+4) Then click *Save*. You should see all the databases available on this server.
+	![](imgs/3-databases-with-docker_8.png)
+  
+</div>
+<div markdown='1'>
+
+![](imgs/3-databases-with-docker_7.png)
+
+</div>
+</div>
+
+### Exercise 2: pgAdmin basics
+<!--_class: "exercise invert" -->
+
+1) If using Docker, start a local instance of pgAdmin in Docker
+2) Following lecture instructions, connect the pgAdmin to your already running PostgreSQL server.
+3) Verify that you can see the database created in the previous assignment.
+
+## Using PostgreSQL
+
+### Creating a database
+
+<div class='columns' markdown='1'>
+<div markdown='1'>
+
+With pgAdmin:
+1) Right-click *Servers > my-postgres > Databases*
+2) Select  _Create > Database..._
+3) Insert a name for the database and hit _Save_
+
+</div>
+<div markdown='1'>
+
+With psql:
+1) Enter command (*remember `;`!!!*)
+    ```sql
+    CREATE DATABASE <database-name>;
+    ```
+2) After creating a new database, you still need to connect to it with `\c`
+</div>
+</div>
+
+<div class='centered'>
+
+![](imgs/7-databases-with-entity-framework_1.png)
+
+</div>
+
+### Querying
+
+With pgAdmin:
+
+1) Right-click  _sqlpractice > Query tool..._
+2) Insert a query into the _Query Editor_ and hit _Execute_ (F5)
+
+<div class='columns23' markdown='1'>
+<div markdown='1'>
+
+![w:345](imgs/7-databases-with-entity-framework_2.png)
+
+</div>
+<div markdown='1'>
+
+![](imgs/7-databases-with-entity-framework_3.png)
+
+</div>
+</div>
+
+---
+
+With psql:
+
+1) List all available databases with `\l`
+2) Connect to the created database with `\c <database-name>`
+3) List all tables in the database with `\dt`
+4) Type a query and press enter
+     * Here's an [example query](code-examples/example-query.sql)
+
+### Editing data with pgAdmin
+
+<div class='columns' markdown='1'>
+<div markdown='1'>
+
+* Tables of data in a database are found under *Database > Schemas > Tables*
+* Inspect and edit data in pgAdmin by right-clicking a table and selecting _View/Edit Data_
+
+</div>
+<div markdown='1'>
+
+![](imgs/7-databases-with-entity-framework_4.png)
+
+</div>
+</div>
+
+---
+
+<div class='columns' markdown='1'>
+<div markdown='1'>
+
+* Individual values in the table can be directly modified by double clicking the value and then editing the value in the visual user interface
+* Save the changes by pressing the _Save Data Changes_ button
+
+</div>
+<div markdown='1' class='centered'>
+
+![](imgs/7-databases-with-entity-framework_5.png)
+![](imgs/7-databases-with-entity-framework_6.png)
+
+</div>
+</div>
+
+## Exercise 3: Preparing the database
+<!--_class: "exercise invert" -->
+
+Using either pgAdmin or psql,
+
+1) Insert the provided [example query](code-examples/example-query.sql) to the new database
+2) Verify that the query has created new tables to your database
+
+## Basic SQL queries
+
+* `SELECT`
+* `INSERT`
+* `DELETE`
+* `UPDATE`
+* `CREATE` & `DROP`
+
+### Querying data with `SELECT`
+
+* Syntax:
+  ```sql
+  SELECT column1, column2, column3 FROM table_name;
+  ```
+* Examples:
+  ```sql
+  SELECT full_name, email FROM users;
+  SELECT full_name AS name, email FROM users;
+  SELECT * FROM users;
+  ```
+
+### Filtering data with `WHERE`
+
+* Syntax:
+  ```sql
+  SELECT column1, column2 FROM table_name WHERE condition;
+  ```
+<div class='columns12' markdown='1'>
+<div markdown='1'>
+
+* Text is captured in **_single quotes_**.
+* In a `LIKE` condition, `%` sign acts as a wildcard.
+* `IS` and `IS NOT` are also valid operators.
+
+</div>
+<div markdown='1'>
+
+* Example:
+  ```sql
+  SELECT full_name FROM users
+  WHERE full_name = 'Teppo Testaaja';
+
+  SELECT * FROM books WHERE name LIKE '%rr%';
+  SELECT * FROM books WHERE author IS NOT null;
+  ```
+
+</div>
+</div>
+
+### Ordering data with `ORDER BY`
+
+* Syntax:
+  ```sql
+  SELECT column1 FROM table_name ORDER BY column1 ASC;
+  ```
+* Examples:
+  ```sql
+  SELECT full_name FROM users
+  ORDER BY full_name ASC;
+
+  SELECT full_name FROM users
+  ORDER BY full_name DESC;
+  ```
+
+### Combining data with `JOIN`
+
+* Also known as `INNER JOIN`
+* Corresponds to ***intersection*** from [set theory](https://en.wikipedia.org/wiki/Set_theory)
+
+<div class='centered'>
+
+![w:500px](imgs/7-databases-with-entity-framework_7.png)
+
+</div>
+
+### JOIN examples
+
+```sql
+SELECT
+users.id, users.full_name, borrows.id,
+borrows.user_id, borrows.due_date, borrows.returned_at
+FROM users
+JOIN borrows ON
+users.id = borrows.user_id;
+```
+
+```sql
+SELECT
+U.full_name AS name,
+B.due_date AS due_date,
+B.returned_at AS returned_at
+FROM users AS U
+JOIN borrows AS B ON
+U.id = B.user_id;
+```
+
+### Combining with `LEFT JOIN`
+
+<div class='columns' markdown='1'>
+<div markdown='1'>
+
+* Also known as `LEFT OUTER JOIN`
+* Example:
+  ```sql
+  SELECT
+  U.full_name AS name,
+  B.due_date AS due_date,
+  B.returned_at AS returned_at
+  FROM users AS U
+  LEFT JOIN borrows AS B ON
+  U.id = B.user_id;
+  ```
+
+</div>
+<div markdown='1'>
+
+![](imgs/7-databases-with-entity-framework_8.png)
+
+</div>
+</div>
+
+### Exercise 4: Querying the library
+<!--_class: "exercise invert" -->
+
+Using SQL queries, get
+
+1) all columns of loans that are borrowed before `2020-10-27`
+2) all columns of loans that are returned
+3) columns `user.full_name` and `borrows.borrowed_at` of the user with an `id` of `1`
+4) columns `book.name`, `book.release_year` and `language.name` of all books that are released after 1960
+
+### Inserting data with `INSERT`
+
+* Syntax
+  ```sql
+  INSERT INTO table_name (column1, column2, column3) VALUES (value1, value2, value3);
+  ```
+* Example
+  ```sql
+  INSERT INTO users (full_name, email, created_at)
+  VALUES ('Pekka Poistuja', 'pekka.poistuja@buutti.com', NOW());
+  ```
+* Since `id` is not provided, it will be automatically generated.
+
+### Updating data with `UPDATE`
+
+* Syntax
+  ```sql
+  UPDATE table_name SET column1 = value1, column2 = value2 WHERE condition;
+  ```
+* *__Notice:__*  if a condition is not provided, all rows will be updated!
+  * If updating only one row, it is usually best to use `id`.
+* Example
+  ```sql
+  UPDATE users SET email = 'taija.testaaja@gmail.com' WHERE id = 2;
+  ```
+
+## Removing data with `REMOVE`
+
+* Syntax
+  ```sql
+  DELETE FROM table_name WHERE condition;
+  ```
+* Again, if the _condition_ is not provided, `DELETE` affects  _all_ rows
+* Before deleting, it is a good practice to execute an equivalent `SELECT` query to make sure that only the proper rows will be affected.
+* Example:
+  ```sql
+  SELECT * FROM users WHERE id = 5;
+  DELETE FROM users WHERE id = 5;
+  ```
+
+### Exercise 5: Editing data
+<!--_class: "exercise invert" -->
+
+1) Postpone the due date of a loan with an `id` of `1` by two days in the `borrows` table
+2) Add a couple of new books to the `books` table
+3) Delete one of the loans.
+
+### Initializing data with `CREATE TABLE`
+
+* Before data can be manipulated, a database and its tables need to be initialized.
+
+<div class='columns' markdown='1'>
+<div markdown='1'>
+
+* Syntax
+  ```sql
+  CREATE TABLE table_name (
+    column1 datatype,
+    column2 datatype,
+    ...
+  );
+  ```
+
+</div>
+<div markdown='1'>
+
+* Example:
+  ```sql
+  CREATE TABLE "users" (
+    "id" SERIAL PRIMARY KEY,
+    "full_name" varchar NOT NULL,
+    "email" varchar UNIQUE NOT NULL,
+    "created_at" timestamp NOT NULL
+  );
+  ```
+
+</div>
+</div>
+
+### Removing data with `DROP`
+
+* In order to remove tables or databases, we use a `DROP` statement
+  ```sql
+  DROP TABLE table_name;
+  DROP DATABASE database_name;
+  ```
+* These statements do not ask for confirmation and there is no undo feature. Take care when using a drop statement.
+
+---
+
+* I'm legally obliged to include this XKCD comic here.
+
+![](https://imgs.xkcd.com/comics/exploits_of_a_mom_2x.png)
+
+## NoSQL
+
+* In addition to SQL databases, there's ***NoSQL***
+* There are many differing definitions, but...
+  * most agree that NoSQL databases store data in a format other than tables
+  * They can still store relational data - just _differently_
+* Four different database types:
+  * Document databases
+  * Key-value databases
+  * Wide-column stores
+  * Graph databases
+* Example database engines include [MongoDB](https://www.mongodb.com/), [Redis](https://redis.io/) and [Cassandra](https://cassandra.apache.org/_/index.html)
+* For more info, see [MongoDB: What is NoSQL?](https://www.mongodb.com/resources/basics/databases/nosql-explained)
+
+
+## Object-Relational Mappers
+
+* Object-Relational Mappers, or ORMs allow the developer to write code instead of SQL to perform CRUD operations on their database
+* An abstraction layer between code and database
+  * Can make database logic easier to read and write
+  * Prevents SQL injection by sanitizing input (see the comic before...)
+  * ...but sometimes you'd rather just write the SQL queries themselves
+* Some popular ORMs:
+  * Hibernate (Java)
+  * EFCore (.NET)
+  * Sequelize (Node.js)
+  * TypeORM (TypeScript) ([documentation](https://typeorm.io/#installation))