AWS CZSK Webinar 2019.07: Databazy na AWS

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Getting Started: Choosing the right
database for your workload
Vladimir Simek, Sr. Solutions Architect @ AWS
July 2019

Agenda
• Lift & Shift vs. New applications
• Relational, Key-Value, and Graph Databases
• Demo – Virtual Bookstore
• Ledger Database
• Time Series Database
• Document Database
• Summary & Resources

Two fundamental options
“Lift and shift” existing
apps to the cloud
Quickly build new
apps in the cloud

“Lift and shift” existing apps to the cloud
apps to the cloud
Quickly build new
apps in the cloud

Amazon Relational Database Service (RDS)
Managed relational database service with a choice of six popular database engines
Easy to administer Available and durable Highly scalable Fast and secure
No need for infrastructure
provisioning, installing, and
maintaining DB software
Automatic Multi-AZ data
replication; automated backup,
snapshots, failover
Scale database compute
and storage with a few
clicks with no app
downtime
SSD storage and guaranteed
provisioned I/O; data
encryption at rest and in
transit

Amazon Aurora
MySQL and PostgreSQL-compatible relational database built for the cloud
Performance and availability of commercial-grade databases at 1/10th the cost
Performance
and scalability
Availability
and durability
Highly secure Fully managed
5x throughput of standard MySQL
and 3x of standard PostgreSQL;
scale-out up to
15 read replicas
Fault-tolerant, self-healing
storage; six copies of data
across three Availability Zones;
continuous backup to Amazon S3
Network isolation,
encryption at rest/transit
Managed by RDS:
No hardware provisioning,
software patching, setup,
configuration, or backups

AWS Database Migration Service (AWS DMS)
M I G R A T I N G
D A T A B A S E S
T O A W S
Migrate between on-premises and AWS
Migrate between databases
Automated schema conversion
Data replication for
zero-downtime migration
140,000+
databases migrated

Customers are moving to AWS Databases
Verizon is migrating over 1,000 business-critical applications and database backend systems to AWS,
several of which also include the migration of production databases to Amazon Aurora.
Wappa migrated from their Oracle database to Amazon Aurora and improved their reporting
time per user by 75 percent.
Trimble migrated their Oracle databases to Amazon RDS and project they will pay about 1/4th of
what they paid when managing their private infrastructure.
Intuit migrated from Microsoft SQL Server to Amazon Redshift to reduce data-processing timelines
and get insights to decision makers faster and more frequently.
Equinox Fitness migrated its Teradata on-premises data warehouse to Amazon Redshift. They went from
static reports to a modern data lake that delivers dynamic reports.
Eventbrite moved from Cloudera to Amazon EMR and were able to cut costs dramatically, spinning
clusters up/down on-demand and using Spot (saving > 80%) and Reserved Instances.
By December 2018, Amazon.com will have migrated 88% of their Oracle DBs (and 97% of critical
system DBs) moved to Amazon Aurora and Amazon DynamoDB. They also migrated their 50 PB
Oracle Data Warehouse to AWS (Amazon S3, Amazon Redshift, and Amazon EMR).
Samsung Electronics migrated their Cassandra clusters to Amazon DynamoDB for their Samsung
Cloud workload with 70% cost savings.

Quickly build new apps in the cloud
apps to the cloud
Quickly build new
apps in the cloud

Modern apps create new requirements
Users: 1 million+
Data volume: TB–PB–EB
Locality: Global
Performance: Milliseconds–microseconds
Request rate: Millions
Access: Web, mobile, IoT, devices
Scale: Up-down, Out-in
Economics: Pay for what you use
Developer access: No assembly requiredSocial mediaRide hailing Media streaming Dating

One size doesn’t fit all

Common data categories and use cases
Relational
Referential
integrity, ACID
transactions,
schema-
on-write
Lift and shift, ERP,
CRM, finance
Key-value
High
throughput, low-
latency reads
and writes,
endless scale
Real-time bidding,
shopping cart,
social, product
catalog, customer
preferences
Document
Store
documents and
quickly access
querying on any
attribute
Content
management,
personalization,
mobile
In-memory
Query by key
with
microsecond
latency
Leaderboards,
real-time analytics,
caching
Graph
Quickly and
easily create
and navigate
relationships
between
data
Fraud detection,
social networking,
recommendation
engine
Time-series
Collect, store,
and process
data sequenced
by time
IoT applications,
event tracking
Ledger
Complete,
immutable, and
verifiable history
of all changes to
application data
Systems
of record, supply
chain, health care,
registrations,
financial

Let’s take a closer look at…
Relational Key-value Graph

Relational data
• Divide data among tables
• Highly structured
• Relationships established via
keys enforced by the system
• Data accuracy and consistency
Patient
* Patient ID
First Name
Last Name
Gender
DOB
* Doctor ID
Visit
* Visit ID
* Patient ID
* Hospital ID
Date
* Treatment ID
Medical Treatment
* Treatment ID
Procedure
How Performed
Adverse Outcome
Contraindication
Doctor
* Doctor ID
First Name
Last Name
Medical Specialty
* Hospital Affiliation
Hospital
* Hospital ID
Name
Address
Rating

Relational use case
// Doctors affiliated with Mercy
hospital
Patient
* Patient ID
First Name
Last Name
Gender
DOB
* Doctor ID
Visit
* Visit ID
* Patient ID
* Hospital ID
Date
* Treatment ID
Medical Treatment
* Treatment ID
Procedure
How Performed
Adverse Outcome
Contraindication
Doctor
* Doctor ID
First Name
Last Name
Medical Specialty
* Hospital Affiliation
Hospital
* Hospital ID
Name
Address
Rating
SELECT
d.first_name, d.last_name
FROM
doctor as d,
hospital as h
WHERE
d.hospital = h.hospital_id
AND h.name = ‘Mercy';
// Number of patient visits each doctor
completed last week
SELECT
d.first_name, d.last_name, count(*)
FROM
visit as v,
hospital as h,
doctor as d
WHERE
v.hospital_id = h.hospital_id
AND h.hospital_id = d.hospital
AND v.t_date > date_trunc('week’,
CURRENT_TIMESTAMP - interval '1 week')
GROUP BY
d.first_name, d.last_name;

Key-value data
• Simple key value
pairs
• Partitioned by keys
• Resilient to failure
• High throughput,
low-latency reads
and writes
• Consistent
performance at
scale
Gamers
Primary Key Attributes
GamerTag Level Points High Score Plays
Hammer57 21 4050 483610 1722
FluffyDuffy 5 1123 10863 43
Lol777313 14 3075 380500 1307
Jam22Jam 20 3986 478658 1694
ButterZZ_55 7 1530 12547 66
… … … … …
Gamers
Hammer57
21
4050,
483610,
1722
GET {
TableName:"Gamers",
Key: {
"GamerTag":"Hammer57“,
“ProjectionExpression“:”Points”
} }

Gamers
Primary Key
Attributes
Gamer Tag Type
Hammer57
Rank
Level Points Tier
87 4050 Elite
Status
Health Progress
90 30
Weapon
Class Damage Range
Taser 87% 50
FluffyDuffy
Rank
Level Points Tier
5 1072 Trainee
Status
Health Progress
37 8
Key-value use case
// Status of Hammer57
GET {
TableName:"Gamers",
Key: {
"GamerTag":"Hammer57",
"Type":"Status” } }
// Return all Hammer57
Gamers
GamerTag = :a
:a Hammer57

Amazon DynamoDB
Fast and flexible key value database service for any scale
Comprehensive
security
Encrypts all data by default
and fully integrates with
AWS Identity and Access
Management for robust
security
Performance at scale
Consistent, single-digit
millisecond response times at any
scale; build applications with
virtually unlimited throughput
Global database for
global users and apps
Build global applications with fast
access to local data by easily
replicating tables across multiple
AWS Regions
Serverless
No server provisioning,
software patching, or upgrades;
scales up or down
automatically; continuously
backs up your data

Use cases for highly connected data
Social networking
Life Sciences Network & IT operationsFraud detection
Recommendations Knowledge graphs

Graph data
• Relationships are first-class
objects
• Vertices connected by Edges
PURCHASED PURCHASED
FOLLOWS
PURCHASED
KNOWS
PRODUCT
SPORT
FOLLOWS

Graph use case
// Product recommendation to a user
gremlin> V().has(‘name’,’sara’).as(‘customer’).out(‘follows’).in(‘follows’).out(‘purchased’)
( (‘customer’)).dedup() (‘name’) ('name')
PURCHASED PURCHASED
FOLLOWS
PURCHASED
KNOWS
PRODUCT
SPORT
FOLLOWS
FOLLOWS
// Identify a friend in common and
make a recommendation
gremlin> g.V().has('name','mary').as(‘start’).
both('knows').both('knows’).
where(neq(‘start’)).
dedup().by('name').properties('name')

Amazon Neptune
Fully managed graph database
Easy
Build powerful queries
easily with Gremlin and
SPARQL
Fast
Query billions of relationships
with millisecond latency
Open
Supports Apache TinkerPop &
W3C RDF graph models
Reliable
Six replicas of your data across
three AZs with full backup and
restore

Airbnb uses different databases based
on the purpose
User search history: Amazon DynamoDB
• Massive data volume
• Need quick lookups for personalized search
Session state: Amazon ElastiCache
• In-memory store for submillisecond site rendering
Relational data: Amazon RDS
• Referential integrity
• Primary transactional database

CHALLENGE
Wanted to enable anyone to learn a
language for free.
SOLUTION
Purpose-built databases from AWS:
• DynamoDB: 31B items tracking
which language exercises completed
• Aurora: primary transactional
database for user data
• ElastiCache: instant access to
common words and phrases
Result:
More people learning a language on
Duolingo than entire US school system
300M total users
7B exercises per month

Demo App Architecture

Retail demo application
Demo application:
1. Available today
2. On GitHub:
/aws-samples/aws-
bookstore-demo-app
3. One click
CloudFormation
deployment
Search
Indexing and
searching
semistructured
logs and data
Product
search
Amazon
Neptune
Amazon
Elasticsearch Service
Key-value
High
throughput, Low-
latency reads
and writes,
endless scale
Shopping cart, user
profile
Graph
Quickly and
easily create
and navigate
relationships
between
data
Product
recommendation
In-memory
Query by key
with
microsecond
latency
Product
leaderboard
DynamoDB ElastiCache

Common customer use cases
Ledgers with centralized control
Healthcare
Verify and track hospital
equipment inventory
Manufacturers
Track distribution of a
recalled product
HR & Payroll
Track changes to an
individual’s profile
Government
Track vehicle title
history

Challenges with building ledgers
Adds unnecessary
complexity
BlockchainRDBMS - audit tables
Difficult to
maintain
Hard to use
and slow
Hard to build
Custom audit functionality using
triggers or stored procedures
Impossible to verify
No way to verify changes made
to data by sys admins

How it works

How it works
ID Manufacturer Model Year VIN Owner
ID Version Start Manufacturer Model Year VIN Owner
J
history.cars
H
current.cars
C

How it works
J
history.cars
H
INSERT INTO cars <<
{ 'Manufacturer':'Tesla',
'Model':'Model S',
'Year':'2012',
'VIN':'123456789',
'Owner':'Traci Russell' }
>>current.cars
C

How it works
J
history.cars
H
INSERT INTO cars <<
'Model':'Model S',
'Year':'2012',
'VIN':'123456789',
>>
INSERT cars
ID:1
Manufacturer: Tesla
Model: Model S
Year: 2012
VIN: 123456789
Owner: Traci Russell
Metadata: {
Date:07/16/2012
}
current.cars
C

How it works
J
history.cars
H
INSERT INTO cars <<
'Model':'Model S',
'Year':'2012',
'VIN':'123456789',
>>
INSERT cars
ID:1
Manufacturer: Tesla
Model: Model S
Year: 2012
VIN: 123456789
Metadata: {
Date:07/16/2012
}
current.cars
C
H (T1)

How it works
J
history.cars
H
INSERT INTO cars <<
'Model':'Model S',
'Year':'2012',
'VIN':'123456789',
>>
INSERT cars
ID:1
Manufacturer: Tesla
Model: Model S
Year: 2012
VIN: 123456789
Metadata: {
Date:07/16/2012
}
current.cars
C
H (T1)
1 Tesla Model S 2012 123456789 Traci Russell

How it works
J
history.cars
H
INSERT INTO cars <<
'Model':'Model S',
'Year':'2012',
'VIN':'123456789',
>>
INSERT cars
ID:1
Manufacturer: Tesla
Model: Model S
Year: 2012
VIN: 123456789
Metadata: {
Date:07/16/2012
}
current.cars
C
H (T1)
1 1 07/16/2012 Tesla Model S 2012 123456789 Traci Russell

How it works
J
history.cars
H
INSERT cars
ID:1
Manufacturer: Tesla
Model: Model S
Year: 2012
VIN: 123456789
Metadata: {
Date:07/16/2012
}
current.cars
C
H (T1)

How it works
J
history.cars
H
INSERT cars
ID:1
Manufacturer: Tesla
Model: Model S
Year: 2012
VIN: 123456789
Metadata: {
Date:07/16/2012
}
current.cars
C
H (T1)
FROM cars WHERE VIN = '123456789' UPDATE owner = 'Ronnie Nash'

How it works
J
history.cars
H
INSERT cars
ID:1
Manufacturer: Tesla
Model: Model S
Year: 2012
VIN: 123456789
Metadata: {
Date:07/16/2012
}
current.cars
C
H (T1)
UPDATE cars
ID:1
Owner: Ronnie Nash
Metadata: {
Date:08/03/2013
}
H (T2)

How it works
J
history.cars
H
INSERT cars
ID:1
Manufacturer: Tesla
Model: Model S
Year: 2012
VIN: 123456789
Metadata: {
Date:07/16/2012
}
current.cars
C
H (T1)
UPDATE cars
ID:1
Owner: Ronnie Nash
Metadata: {
Date:08/03/2013
}
H (T2)
1 Tesla Model S 2012 123456789 Ronnie Nash

How it works
J
history.cars
H
INSERT cars
ID:1
Manufacturer: Tesla
Model: Model S
Year: 2012
VIN: 123456789
Metadata: {
Date:07/16/2012
}
current.cars
C
H (T1) UPDATE cars
ID:1
Owner: Ronnie Nash
Metadata: {
Date:08/03/2013
}
H (T2)
1 2 08/03/2013 Tesla Model S 2012 123456789 Ronnie Nash

How it works
J
history.cars
H
INSERT cars
ID:1
Manufacturer: Tesla
Model: Model S
Year: 2012
VIN: 123456789
Metadata: {
Date:07/16/2012
}
current.cars
C
H (T1) UPDATE cars
ID:1
Owner: Ronnie Nash
Metadata: {
Date:08/03/2013
}
H (T2)

How it works
J
history.cars
H
INSERT cars
ID:1
Manufacturer: Tesla
Model: Model S
Year: 2012
VIN: 123456789
Metadata: {
Date:07/16/2012
}
current.cars
C
H (T1) UPDATE cars
ID:1
Owner: Ronnie Nash
Metadata: {
Date:08/03/2013
}
H (T2)
FROM cars WHERE VIN = '123456789' UPDATE owner = 'Elmer Hubbard'

How it works
J
history.cars
H
INSERT cars
ID:1
Manufacturer: Tesla
Model: Model S
Year: 2012
VIN: 123456789
Metadata: {
Date:07/16/2012
}
current.cars
C
H (T1) UPDATE cars
ID:1
Owner: Ronnie Nash
Metadata: {
Date:08/03/2013
}
H (T2)
UPDATE cars
ID:1
Owner: Elmer Hubbard
Metadata: {
Date: 09/02/2016
}
H (T3)

How it works
J
history.cars
H
INSERT cars
ID:1
Manufacturer: Tesla
Model: Model S
Year: 2012
VIN: 123456789
Metadata: {
Date:07/16/2012
}
current.cars
C
H (T1) UPDATE cars
ID:1
Owner: Ronnie Nash
Metadata: {
Date:08/03/2013
}
H (T2)
1 Tesla Model S 2012 123456789 Elmer Hubbard
UPDATE cars
ID:1
Owner: Elmer Hubbard
Metadata: {
Date: 09/02/2016
}
H (T3)
1 3 09/02/2016 Tesla Model S 2012 123456789 Elmer Hubbard

Amazon Quantum Ledger Database (QLDB)
(Preview)
Fully managed ledger database
Track and verify history of all changes made to your application’s data
Immutable
Maintains a sequenced record of
all changes to your data, which
cannot be deleted or modified;
you have the ability to query and
analyze the full history
Cryptographically
verifiable
Uses cryptography to
generate a secure output
file of your data’s history
Easy to use
Easy to use, letting you
use familiar database
capabilities like SQL APIs
for querying the data
Highly scalable
Executes 2–3X as many
transactions than ledgers
in common blockchain
frameworks

Time-series data
What is time-series
data?
What is special about a
time-series database? A sequence of data points
recorded over a time interval
Time is the
single primary axis
of the data model
t

Time-series use case
Application events
IoT Sensor
Readings
DevOps data
Humidity
% WATER VAPOR
91.094.086.093.0

Existing time-series databasesRelational databases
Difficult to
maintain high
availability
Difficult to scale Limited data
lifecycle
management
Inefficient
time-series data
processing
Unnatural for
time-series data
Rigid schema
inflexible for fast
moving time-series
data
Building with time-series data is challenging

Amazon Timestream (sign up for the preview)
Fast, scalable, fully managed time-series database
1,000x faster and 1/10th the
cost of relational databases
Collect data at the rate of
millions of inserts per
second (10M/second)
Trillions of
daily events
Adaptive query processing
engine maintains steady,
predictable performance
Time-series
analytics
Built-in functions for
interpolation, smoothing, and
approximation
Serverless
Automated setup, configuration,
server provisioning, software
patching

Amazon DocumentDB
Fast, scalable, highly available, fully managed MongoDB-compatible
database service
Fully Managed
Managed by AWS:
No hardware provisioning,
software patching, setup,
configuration, or backups
Fast
Millions of requests per second,
millisecond latency
MongoDB-compatible
Compatible with MongoDB
Community Edition 3.6. Use the
same drivers and tools
Reliable
Six replicas of your data across
three AZs with full backup and
restore

AWS purpose-built databases
Relational Key-value Document In-memory Graph Time-series Ledger
DynamoDB NeptuneAmazon RDS
Aurora CommercialCommunity
Timestream QLDBElastiCacheDocumentDB

Resources
Web Pages:
AWS Databases https://aws.amazon.com/products/databases/
Databases Migrations https://aws.amazon.com/solutions/database-migrations/
Amazon QLDB Preview https://pages.awscloud.com/QuantumLedgerDatabase-preview.html
Amazon TimeStream Preview https://pages.awscloud.com/timestream-preview.html
Projects:
Build a Modern Web Application
https://aws.amazon.com/getting-started/projects/build-modern-app-fargate-lambda-
dynamodb-python/
Bookstore Demo
https://github.com/aws-samples/aws-bookstore-demo-app

AWS CZSK Webinar 2019.07: Databazy na AWS

More Related Content

More from Vladimir Simek (18)

Recently uploaded (20)

AWS CZSK Webinar 2019.07: Databazy na AWS