webdev-basics/sql-databases.md

---
marp: true
paginate: true
math: mathjax
theme: buutti
title: SQL Databases
---

# SQL Databases

<!-- headingDivider: 3 -->
<!-- class: invert -->

## What is SQL?

* SQL, Structured Query Language is a language used to organize and manipulate data in a database
* Originally developed by IBM in the 70's
  * Quickly became the most popular database language
  ```sql
  SELECT id, email
  FROM users
  WHERE first_name = 'Teppo';
  ```

### Relational Database Management Systems

* In relational databases, values are stored in *__tables__*
  * Each table has *__rows__* and  *__columns__*
  * Data is displayed in a two-dimensional matrix
* Values in a table are related to each other
  * Values can also be related to values in other tables
* A relational database management system (RDBMS) is a program that executes queries to relational databases

### Database management systems compared

![](imgs/3-databases-with-docker_0.png)

[https://db-engines.com/en/ranking](https://db-engines.com/en/ranking)

## PostgreSQL

* [PostgreSQL](https://www.postgresql.org/) (pronounced *postgres-QL*) is a free and open-source, cross-platform relational database management system
* Emphasizes extensibility and SQL compliance
* Fully [ACID](https://en.wikipedia.org/wiki/ACID)-compliant (atomicity, consistency, isolation and durability)
* Used in conjunction with an interface like ***pgAdmin*** (graphical) or ***psql*** (command-line)
* Once installed, can be accessed with the SQL Shell (see instructions [here](https://www.w3schools.com/postgresql/postgresql_getstarted.php))

### Extra: Running Postgres in Docker
<!-- _class: "extra invert" -->

* Instead of running a PostgreSQL database locally, you can run it *Dockerized*
  * See [Devops Lecture 1: Docker]()
* Using the official  [Postgres Docker image](https://hub.docker.com/_/postgres) , let's create a locally running Postgres instance
	```bash
	docker run --name my-postgres --env POSTGRES_PASSWORD=pgpass 
	--env POSTGRES_USER=pguser -p 5432:5432 -d postgres:15.2
	```
* You can choose the values for `POSTGRES_PASSWORD` and `POSTGRES_USER` freely.
	![](imgs/3-databases-with-docker_1.png)

## `psql`

* psql (see [Documentation](https://www.postgresql.org/docs/current/app-psql.html
)) is a terminal-based interface for PostgreSQL and comes bundled with it
  * With psql, you can type in SQL queries interactively and see the results
  * Input from file, command line arguments, scripting and automation...

<div class='columns' markdown='1'>
<div markdown='1'>

#### With Docker:

* You can use psql directly from the Docker container running the database	
	```bash
	docker exec -it my-postgres psql -U pguser
	```

</div>
<div markdown='1'>

#### Without Docker:

* Just open the SQL shell from Start menu.

</div>
</div>


### `psql` commands

* If you have connected to the server as user `pguser`, you are by default connected to `pguser` database.
* This is a database for user information. *Do not use it for storing program data!*
* All databases can be listed using the *list* command: `\l`
* PostgreSQL uses a default database named `postgres`
* Users can connect to a different database with the ***connect*** command `\c`:
  ```psql
  \c <database-name>
  ```
* Exit `psql` with the command `exit`

### Exercise 1. psql basics
<!--_class: "exercise invert" -->

1) If using Docker, start a local instance of Postgres in Docker.
2) Connect to the server using psql.
3) Use command `\l` to see what databases there are already created on the server.
4) Create a new database called `sqlpractice`.
5) Connect to the newly created database.

## pgAdmin

* [pgAdmin](https://www.pgadmin.org/) is an administration and development platform for PostgreSQL

<div class='columns' markdown='1'>
<div markdown='1'>

* Cross-platform, features a web interface
* A control panel frontend for your PostgreSQL database
* A graphical alternative to psql
* Completely separate from PostgreSQL
* One of many database management applications
  * Another major one is [DBeaver](https://dbeaver.io/)

</div>
<div markdown='1'>

![](imgs/3-databases-with-docker_1_5.png)

</div>
</div>

### Extra: Running pgAdmin in Docker
<!-- _class: "extra invert" -->

* Using the official [pgAdmin](https://hub.docker.com/r/dpage/pgadmin4) image, we'll run pgAdmin alongside Postgres
	```
	docker run --name my-pgadmin -p 5050:80 
	--env PGADMIN_DEFAULT_EMAIL=<your-email-address> 
	--env PGADMIN_DEFAULT_PASSWORD=<your-password> 
	-d dpage/pgadmin4
	```
* ![](imgs/3-databases-with-docker_2.png)

---
<!-- _class: "extra invert" -->

* Both PostgreSQL and pgAdmin are now running in our *local* Docker. To connect pgAdmin to PostgreSQL, we need the IP address used inside Docker.
* Using Docker's inspect, we'll get Docker's internal IP for `my-postgres -container`. Since the command produces quite a lot of information, we pipe the result to `grep` (Linux) or `findstr` (Windows) to see only the rows that contain the word `IPAddress`

<div class='columns' markdown='1'>
<div markdown='1'>

#### Linux

```bash
docker inspect <container-name> | grep IPAddress
```

</div>
<div markdown='1'>

#### Windows

```powershell
docker inspect <container-name> | findstr IPAddress
```

</div>
</div>

* In the example output, the IP Address is `172.17.0.2`:
	![](imgs/3-databases-with-docker_4.png)

### Logging into pgAdmin

<div class='columns' markdown='1'>
<div markdown='1'>

#### With Docker:
1) With pgAdmin running, navigate your web browser to http://localhost:5050 

</div>
<div markdown='1'>

#### Without Docker:
1) Run pgAdmin from the Start menu.

</div>
</div>

2) Use the username and password you provided to log in.
<div class='centered'>

![w:650px](imgs/3-databases-with-docker_3.png)

</div>

### Connecting PgAdmin to our DB

<div class='columns' markdown='1'>
<div markdown='1'>

1) Now we have all that we need for a connection. In pgAdmin, select<br> *Object > Register > Server*.
     * If the *Object* menu is greyed out, click on *Servers*.
	![](imgs/3-databases-with-docker_6.png)
2) In the *General* tab, give the server a name that identifies the connection.

</div>
<div markdown='1'>

![w:500px](imgs/3-databases-with-docker_5.png)

</div>
</div>


---

<div class='columns' markdown='1'>
<div markdown='1'>

3) In the *Connection* tab, enter:
   1) *Host name/address*
     * the internal Docker address
   2) *Port*, *Username*, and *Password*
     * the values defined when running the PostgreSQL container

4) Then click *Save*. You should see all the databases available on this server.
	![](imgs/3-databases-with-docker_8.png)
  
</div>
<div markdown='1'>

![](imgs/3-databases-with-docker_7.png)

</div>
</div>

### Exercise 2: pgAdmin basics
<!--_class: "exercise invert" -->

1) If using Docker, start a local instance of pgAdmin in Docker
2) Following lecture instructions, connect the pgAdmin to your already running PostgreSQL server.
3) Verify that you can see the database created in the previous assignment.

## Using PostgreSQL

### Creating a database

<div class='columns' markdown='1'>
<div markdown='1'>

With pgAdmin:
1) Right-click *Servers > my-postgres > Databases*
2) Select  _Create > Database..._
3) Insert a name for the database and hit _Save_

</div>
<div markdown='1'>

With psql:
1) Enter command (*remember `;`!!!*)
    ```sql
    CREATE DATABASE <database-name>;
    ```
2) After creating a new database, you still need to connect to it with `\c`
</div>
</div>

<div class='centered'>

![](imgs/7-databases-with-entity-framework_1.png)

</div>

### Querying

With pgAdmin:

1) Right-click  _sqlpractice > Query tool..._
2) Insert a query into the _Query Editor_ and hit _Execute_ (F5)

<div class='columns23' markdown='1'>
<div markdown='1'>

![w:345](imgs/7-databases-with-entity-framework_2.png)

</div>
<div markdown='1'>

![](imgs/7-databases-with-entity-framework_3.png)

</div>
</div>

---

With psql:

1) List all available databases with `\l`
2) Connect to the created database with `\c <database-name>`
3) List all tables in the database with `\dt`
4) Type a query and press enter
     * Here's an [example query](code-examples/example-query.sql)

### Editing data with pgAdmin

<div class='columns' markdown='1'>
<div markdown='1'>

* Tables of data in a database are found under *Database > Schemas > Tables*
* Inspect and edit data in pgAdmin by right-clicking a table and selecting _View/Edit Data_

</div>
<div markdown='1'>

![](imgs/7-databases-with-entity-framework_4.png)

</div>
</div>

---

<div class='columns' markdown='1'>
<div markdown='1'>

* Individual values in the table can be directly modified by double clicking the value and then editing the value in the visual user interface
* Save the changes by pressing the _Save Data Changes_ button

</div>
<div markdown='1' class='centered'>

![](imgs/7-databases-with-entity-framework_5.png)
![](imgs/7-databases-with-entity-framework_6.png)

</div>
</div>

## Exercise 3: Preparing the database
<!--_class: "exercise invert" -->

Using either pgAdmin or psql,

1) Insert the provided [example query](code-examples/example-query.sql) to the new database
2) Verify that the query has created new tables to your database

## Basic SQL queries

* `SELECT`
* `INSERT`
* `DELETE`
* `UPDATE`
* `CREATE` & `DROP`

### Querying data with `SELECT`

* Syntax:
  ```sql
  SELECT column1, column2, column3 FROM table_name;
  ```
* Examples:
  ```sql
  SELECT full_name, email FROM users;
  SELECT full_name AS name, email FROM users;
  SELECT * FROM users;
  ```

### Filtering data with `WHERE`

* Syntax:
  ```sql
  SELECT column1, column2 FROM table_name WHERE condition;
  ```
<div class='columns12' markdown='1'>
<div markdown='1'>

* Text is captured in **_single quotes_**.
* In a `LIKE` condition, `%` sign acts as a wildcard.
* `IS` and `IS NOT` are also valid operators.

</div>
<div markdown='1'>

* Example:
  ```sql
  SELECT full_name FROM users
  WHERE full_name = 'Teppo Testaaja';

  SELECT * FROM books WHERE name LIKE '%rr%';
  SELECT * FROM books WHERE author IS NOT null;
  ```

</div>
</div>

### Ordering data with `ORDER BY`

* Syntax:
  ```sql
  SELECT column1 FROM table_name ORDER BY column1 ASC;
  ```
* Examples:
  ```sql
  SELECT full_name FROM users
  ORDER BY full_name ASC;

  SELECT full_name FROM users
  ORDER BY full_name DESC;
  ```

### Combining data with `JOIN`

* Also known as `INNER JOIN`
* Corresponds to ***intersection*** from [set theory](https://en.wikipedia.org/wiki/Set_theory)

<div class='centered'>

![w:500px](imgs/7-databases-with-entity-framework_7.png)

</div>

### JOIN examples

```sql
SELECT
users.id, users.full_name, borrows.id,
borrows.user_id, borrows.due_date, borrows.returned_at
FROM users
JOIN borrows ON
users.id = borrows.user_id;
```

```sql
SELECT
U.full_name AS name,
B.due_date AS due_date,
B.returned_at AS returned_at
FROM users AS U
JOIN borrows AS B ON
U.id = B.user_id;
```

### Combining with `LEFT JOIN`

<div class='columns' markdown='1'>
<div markdown='1'>

* Also known as `LEFT OUTER JOIN`
* Example:
  ```sql
  SELECT
  U.full_name AS name,
  B.due_date AS due_date,
  B.returned_at AS returned_at
  FROM users AS U
  LEFT JOIN borrows AS B ON
  U.id = B.user_id;
  ```

</div>
<div markdown='1'>

![](imgs/7-databases-with-entity-framework_8.png)

</div>
</div>

### Exercise 4: Querying the library
<!--_class: "exercise invert" -->

Using SQL queries, get

1) all columns of loans that are borrowed before `2020-10-27`
2) all columns of loans that are returned
3) columns `user.full_name` and `borrows.borrowed_at` of the user with an `id` of `1`
4) columns `book.name`, `book.release_year` and `language.name` of all books that are released after 1960

### Inserting data with `INSERT`

* Syntax
  ```sql
  INSERT INTO table_name (column1, column2, column3) VALUES (value1, value2, value3);
  ```
* Example
  ```sql
  INSERT INTO users (full_name, email, created_at)
  VALUES ('Pekka Poistuja', 'pekka.poistuja@buutti.com', NOW());
  ```
* Since `id` is not provided, it will be automatically generated.

### Updating data with `UPDATE`

* Syntax
  ```sql
  UPDATE table_name SET column1 = value1, column2 = value2 WHERE condition;
  ```
* *__Notice:__*  if a condition is not provided, all rows will be updated!
  * If updating only one row, it is usually best to use `id`.
* Example
  ```sql
  UPDATE users SET email = 'taija.testaaja@gmail.com' WHERE id = 2;
  ```

## Removing data with `REMOVE`

* Syntax
  ```sql
  DELETE FROM table_name WHERE condition;
  ```
* Again, if the _condition_ is not provided, `DELETE` affects  _all_ rows
* Before deleting, it is a good practice to execute an equivalent `SELECT` query to make sure that only the proper rows will be affected.
* Example:
  ```sql
  SELECT * FROM users WHERE id = 5;
  DELETE FROM users WHERE id = 5;
  ```

### Exercise 5: Editing data
<!--_class: "exercise invert" -->

1) Postpone the due date of a loan with an `id` of `1` by two days in the `borrows` table
2) Add a couple of new books to the `books` table
3) Delete one of the loans.

### Initializing data with `CREATE TABLE`

* Before data can be manipulated, a database and its tables need to be initialized.

<div class='columns' markdown='1'>
<div markdown='1'>

* Syntax
  ```sql
  CREATE TABLE table_name (
    column1 datatype,
    column2 datatype,
    ...
  );
  ```

</div>
<div markdown='1'>

* Example:
  ```sql
  CREATE TABLE "users" (
    "id" SERIAL PRIMARY KEY,
    "full_name" varchar NOT NULL,
    "email" varchar UNIQUE NOT NULL,
    "created_at" timestamp NOT NULL
  );
  ```

</div>
</div>

### Removing data with `DROP`

* In order to remove tables or databases, we use a `DROP` statement
  ```sql
  DROP TABLE table_name;
  DROP DATABASE database_name;
  ```
* These statements do not ask for confirmation and there is no undo feature. Take care when using a drop statement.

---

* I'm legally obliged to include this XKCD comic here.

![](https://imgs.xkcd.com/comics/exploits_of_a_mom_2x.png)

## NoSQL

* In addition to SQL databases, there's ***NoSQL***
* There are many differing definitions, but...
  * most agree that NoSQL databases store data in a format other than tables
  * They can still store relational data - just _differently_
* Four different database types:
  * Document databases
  * Key-value databases
  * Wide-column stores
  * Graph databases
* Example database engines include [MongoDB](https://www.mongodb.com/), [Redis](https://redis.io/) and [Cassandra](https://cassandra.apache.org/_/index.html)
* For more info, see [MongoDB: What is NoSQL?](https://www.mongodb.com/resources/basics/databases/nosql-explained)


## Object-Relational Mappers

* Object-Relational Mappers, or ORMs allow the developer to write code instead of SQL to perform CRUD operations on their database
* An abstraction layer between code and database
  * Can make database logic easier to read and write
  * Prevents SQL injection by sanitizing input (see the comic before...)
  * ...but sometimes you'd rather just write the SQL queries themselves
* Some popular ORMs:
  * Hibernate (Java)
  * EFCore (.NET)
  * Sequelize (Node.js)
  * TypeORM (TypeScript) ([documentation](https://typeorm.io/#installation))