Distributed Database Architecture for GDPR

1© 2018 All rights reserved.
Distributed Database
Architecture for GDPR
Karthik Ranganathan
PostgresConf Silicon Valley
Oct 15, 2018

About Us
Kannan Muthukkaruppan, CEO
Nutanix ♦ Facebook ♦ Oracle
IIT-Madras, University of California-Berkeley
Karthik Ranganathan, CTO
Nutanix ♦ Facebook ♦ Microsoft
IIT-Madras, University of Texas-Austin
Mikhail Bautin, Software Architect
ClearStory Data ♦ Facebook ♦ D.E.Shaw
Nizhny Novgorod State University, Stony Brook
 Founded Feb 2016
 Apache HBase committers and early engineers on Apache Cassandra
 Built Facebook’s NoSQL platform powered by Apache HBase
 Scaled the platform to serve many mission-critical use cases
• Facebook Messages (Messenger)
• Operational Data Store (Time series Data)
 Reassembled the same Facebook team at YugaByte along with
engineers from Oracle, Google, Nutanix and LinkedIn
Founders

WHAT IS
YUGABYTE DB?

A transactional, planet-scale database
for building high-performance cloud services.

NoSQL + SQL Cloud Native

TRANSACTIONAL PLANET-SCALEHIGH PERFORMANCE
Single Shard & Distributed ACID Txns
Document-Based, Strongly
Consistent Storage
Low Latency, Tunable Reads
High Throughput
OPEN SOURCE
Apache 2.0
Popular APIs Extended
Apache Cassandra, Redis and PostgreSQL (BETA)
Auto Sharding & Rebalancing
Global Data Distribution
Design Principles
CLOUD NATIVE
Built For The Container Era
Self-Healing, Fault-Tolerant

WHAT IS GDPR?

GDPR : General Data Protection Regulation

Citizens of EU can control sharing and protection
of their personal data by businesses.

Personal Data, also called
PII (Personally Identifiable Information)
• User name
• Email address
• Date of birth
• Bank details
• Location details
• Computer IP address

Control over personal data
• Consent & data location
• Data privacy and safety
• Right to be forgotten
• Data access on demand
• Notify on data breach
• Data portability
• Ability to fix errors in data
• Restrict processing
Database concerns Application concerns

#1 USER CONSENT
AND DATA LOCATION

Data must be stored in EU by default. Businesses
need explicit user consent to move it outside.

Why is this hard?
• EU user data lives in that region
• Other countries have compliance regulation – more geo’s
• Public clouds may not have coverage – hybrid deployments
• Architecture depends on data – multiple per service
Think Global Deployments first!

Example – online ecommerce site
• Products table needs globally replication – not PII data

Read Replicas
Global Replication
Non-PII Data
Global Replication
with YugaByte DB

Example – online ecommerce site
• Users, orders and shipments needs locality – PII data
• Product locations table needs scale – may be PII

Primary Data in EU
PII Data
Non-EU Data
Non-EU Data
Geo-Partitioning
with YugaByte DB

Replicate data on demand to other geo’s
• User may be ok with replicating data
• Read replicas on demand (for remote, low-latency reads)
• Change data capture (for analytics)

Read Replicas
Primary Data in EU
PII Data with YugaByte DB
Read Replicas with
YugaByte DB

#2 DATA PRIVACY
AND SAFETY

Data must be secured by using best practices by
default. Users need to be notified on breach.

Implement end-to-end encryption on day #1

• Use TLS Encryption
• Between client and server for app interaction
• Between database servers for replication
Encrypt All Network Communication

TLS Encryption
Database Cluster
User
Server to server
communication

• Encryption at rest
• Integrate with external Key Management Systems
• Ability to rotate keys on demand
Encryption All Storage
Have a key-value table with id to cipher key. Encrypt PII data with
the cipher key for fine-grained control. More in the next section.

Encryption at Rest
Database Cluster
User
Encryption on disk
Key Management
Service

#3 RIGHT TO BE
FORGOTTEN

Data must be erased if on explicit request or when
data is no longer relevant to original intent.

• Have a key-value table with id to cipher key
• Encrypt PII data with the cipher key on write
• Decrypt PII data on access
• Delete cipher key to forget PII data
Use Encryption of Data Attributes

SET email=foo@bar.com FOR USER ID=XXX
Example - Storing User Profile Data
SET email=ENCRYPTED FOR USER ID=XXX
Get encryption
key for user
Encryption PII Data
Store encrypted data
• Reads require decryption
• Data not accessible without key

• Many cases where value not needed
• Anonymize PII data with one way hash functions
• Use hashed ids for in data warehouse
• There is no PII data if hashed ids are used!
Use Anonymization of Data Attributes

USER=foo@bar.com CHECKED OUT PRODUCT=X, CATEGORY=Gadget
Example – Website Analytics
USER=HASHED_VAL CHECKED OUT PRODUCT=X, CATEGORY=Gadget
One-way hash
user id
Analytics

Example – Website Analytics
• User no longer identifiable
• Hashed data still useful!

#4 DATA ACCESS
ON DEMAND

Ability to inform a user about what data is being used,
for what purpose and where it is stored.

• Store in a separate information architecture table
• Make tagging a part of the process
• Easy to find what PII data is stored on demand
Tag Tables and Columns with PII

• Ensure PII are encrypted
• Ensure non-PII columns do not have sensitive data
• Use Spark/Presto to perform scan periodically
• Run scan on a read replica to not impact production
Run Continuous Compliance Checks

Ensure PII columns are encrypted
Ensure no PII data in other columns
Tag PII Columns

PUTTING IT ALL TOGETHER

GDPR Reference Architecture
Primary Cluster
(in EU)
Read Replica Clusters
(Anywhere in the World)
Encrypted Encrypted
App clients
Encrypted Async
Replication
Reads & Writes, Encrypted
Analytics clients
Read only, Encrypted
At-Rest Encryption for All Nodes At-Rest Encryption for All Nodes
PII Columns Encrypted w/
Cipher Key
Tag PII Columns
Ensure PII columns are
encrypted
Ensure no PII data in other
columns

Questions?
Try it at
docs.yugabyte.com/latest/quick-start

Distributed Database Architecture for GDPR

More Related Content

What's hot (20)

Similar to Distributed Database Architecture for GDPR (20)

More from Yugabyte (7)

Recently uploaded (20)

Distributed Database Architecture for GDPR