SlideShare a Scribd company logo
NoSQL Applications for the Enterprise 
September 10, 2014
Agenda 
• Introduction to EDB 
• Intro to JSON, HSTORE and PL/V8 
• JSON History in Postgres 
• JSON Data Types, Operators and Functions 
• JSON, JSONB and BSON – when to use which one? 
• JSONB and Node.JS – easy as pie 
• NoSQL Performance in Postgres – fast as greased lightning 
• PG XDK – a simple developer kit for Postgres NoSQL Applications 
• Say ‘Yes’ to ‘Not only SQL’ 
• Useful resources 
© 2014 EnterpriseDB Corporation. All rights reserved. 2
Introduction to EDB 
© 2013 EDB All rights reserved 8.1. 3
POSTGRES 
innovation 
Services 
& training 
© 2014 EnterpriseDB Corporation. All rights reserved. 4 
ENTERPRISE 
reliability 
24/7 
support 
Enterprise-class 
features & tools 
Indemnification 
Product 
road-map 
Control 
Thousands 
of developers 
Fast 
development 
cycles 
Low cost 
No vendor 
lock-in 
Advanced 
features 
Enabling commercial 
adoption of Postgres
Postgres Plus 
Advanced Server Postgres Plus 
Management Performance 
© 2014 EnterpriseDB Corporation. All rights reserved. 5 
Cloud Database 
High Availability 
REMOTE 
DBA 24x7 
SUPPORT 
PROFESSIONAL 
SERVICES 
TRAINING 
EDB Serves 
All Your Postgres Needs 
PostgreSQL 
Security
Let’s Ask Ourselves, Why NoSQL? 
• Where did NoSQL come from? 
− Where all cool tech stuff comes from – Internet companies 
• Why did they make NoSQL? 
− To support huge data volumes and evolving demands for ways 
to work with new data types 
• What does NoSQL accomplish? 
− Enables you to work with new data types: email, mobile 
interactions, machine data, social connections 
− Enables you to work in new ways: incremental development 
and continuous release 
• Why did they have to build something new? 
− There were limitations to most relational databases 
© 2014 EnterpriseDB Corporation. All rights reserved. 6
NoSQL: Real-world Applications 
• Emergency Management System 
− High variability among data sources required high schema 
flexibility 
• Massively Open Online Course 
− Massive read scalability, content integration, low latency 
• Patient Data and Prescription Records 
− Efficient write scalability 
• Social Marketing Analytics 
− Map reduce analytical approaches 
Source: Gartner, A Tour of NoSQL in 8 Use Cases, 
by Nick Heudecker and Merv Adrian, February 28, 2014 
© 2014 EnterpriseDB Corporation. All rights reserved. 7
Postgres’ Response 
• HSTORE 
− Key-value pair 
− Simple, fast and easy 
− Postgres v 8.2 – pre-dates many NoSQL-only solutions 
− Ideal for flat data structures that are sparsely populated 
• JSON 
− Hierarchical document model 
− Introduced in Postgres 9.2, perfected in 9.3 
• JSONB 
− Binary version of JSON 
− Faster, more operators and even more robust 
− Postgres 9.4 
© 2014 EnterpriseDB Corporation. All rights reserved. 8
Postgres: Key-value Store 
• Supported since 2006, the HStore 
contrib module enables storing 
key/value pairs within a single 
column 
• Allows you to create a schema-less, 
ACID compliant data store within 
Postgres 
• Create single HStore column and 
include, for each row, only those keys 
which pertain to the record 
• Add attributes to a table and query 
without advance planning 
•Combines flexibility with ACID compliance 
© 2014 EnterpriseDB Corporation. All rights reserved. 9
HSTORE Examples 
• Create a table with HSTORE field 
CREATE TABLE hstore_data (data HSTORE); 
• Insert a record into hstore_data 
INSERT INTO hstore_data (data) VALUES (’ 
"cost"=>"500", 
"product"=>"iphone", 
"provider"=>"apple"'); 
• Select data from hstore_data 
SELECT data FROM hstore_data ; 
------------------------------------------ 
"cost"=>"500”,"product"=>"iphone”,"provider"=>"Apple" 
(1 row) 
© 2014 EnterpriseDB Corporation. All rights reserved. 10
Postgres: Document Store 
• JSON is the most popular 
data-interchange format on the web 
• Derived from the ECMAScript 
Programming Language Standard 
(European Computer Manufacturers 
Association). 
• Supported by virtually every 
programming language 
• New supporting technologies 
continue to expand JSON’s utility 
− PL/V8 JavaScript extension 
− Node.js 
• Postgres has a native JSON data type (v9.2) and a JSON parser and a 
variety of JSON functions (v9.3) 
• Postgres will have a JSONB data type with binary storage and indexing 
(coming – v9.4) 
© 2014 EnterpriseDB Corporation. All rights reserved. 11
JSON Examples 
• Creating a table with a JSONB field 
CREATE TABLE json_data (data JSONB); 
• Simple JSON data element: 
{"name": "Apple Phone", "type": "phone", "brand": 
"ACME", "price": 200, "available": true, 
"warranty_years": 1} 
• Inserting this data element into the table json_data 
INSERT INTO json_data (data) VALUES 
(’ { "name": "Apple Phone", 
"type": "phone", 
"brand": "ACME", 
"price": 200, 
"available": true, 
"warranty_years": 1 
} ') 
© 2014 EnterpriseDB Corporation. All rights reserved. 12
JSON Examples 
• JSON data element with nesting: 
{“full name”: “John Joseph Carl Salinger”, 
“names”: 
[ 
{"type": "firstname", “value”: ”John”}, 
{“type”: “middlename”, “value”: “Joseph”}, 
{“type”: “middlename”, “value”: “Carl”}, 
{“type”: “lastname”, “value”: “Salinger”} 
] 
} 
© 2014 EnterpriseDB Corporation. All rights reserved. 13
A simple query for JSON data 
SELECT DISTINCT 
data->>'name' as products 
FROM json_data; 
products 
------------------------------ 
Cable TV Basic Service 
Package 
AC3 Case Black 
Phone Service Basic Plan 
AC3 Phone 
AC3 Case Green 
Phone Service Family Plan 
AC3 Case Red 
AC7 Phone 
© 2014 EnterpriseDB Corporation. All rights reserved. 14 
This query does not 
return JSON data – it 
returns text values 
associated with the 
key ‘name’
A query that returns JSON data 
SELECT data FROM json_data; 
data 
------------------------------------------ 
{"name": "Apple Phone", "type": "phone", 
"brand": "ACME", "price": 200, 
"available": true, "warranty_years": 1} 
This query returns the JSON data in its 
original format 
© 2014 EnterpriseDB Corporation. All rights reserved. 15
JSON and ANSI SQL - PB&J for the DBA 
• JSON is naturally 
integrated with ANSI SQL 
in Postgres 
• JSON and SQL queries 
use the same language, the 
same planner, and the same ACID compliant 
transaction framework 
• JSON and HSTORE are elegant and easy to use 
extensions of the underlying object-relational model 
© 2014 EnterpriseDB Corporation. All rights reserved. 16
JSON and ANSI SQL Example 
SELECT DISTINCT 
product_type, 
data->>'brand' as Brand, 
data->>'available' as Availability 
FROM json_data 
JOIN products 
ON (products.product_type=json_data.data->>'name') 
WHERE json_data.data->>'available'=true; 
product_type | brand | availability 
---------------------------+-----------+-------------- 
AC3 Phone | ACME | true 
ANSI SQL 
© 2014 EnterpriseDB Corporation. All rights reserved. 17 
JSON 
No need for programmatic logic to combine SQL and 
NoSQL in the application – Postgres does it all
Bridging between SQL and JSON 
Simple ANSI SQL Table Definition 
CREATE TABLE products (id integer, product_name text ); 
Select query returning standard data set 
SELECT * FROM products; 
id | product_name 
----+-------------- 
1 | iPhone 
2 | Samsung 
3 | Nokia 
Select query returning the same result as a JSON data set 
SELECT ROW_TO_JSON(products) FROM products; 
{"id":1,"product_name":"iPhone"} 
{"id":2,"product_name":"Samsung"} 
{"id":3,"product_name":"Nokia”} 
© 2014 EnterpriseDB Corporation. All rights reserved. 18
JSON Data Types 
• 1. Number: 
− Signed decimal number that may contain a fractional part and may use exponential 
notation. 
− No distinction between integer and floating-point 
• 2. String 
− A sequence of zero or more Unicode characters. 
− Strings are delimited with double-quotation mark 
− Supports a backslash escaping syntax. 
• 3. Boolean 
− Either of the values true or false. 
• 4. Array 
− An ordered list of zero or more values, 
− Each values may be of any type. 
− Arrays use square bracket notation with elements being comma-separated. 
• 5. Object 
− An unordered associative array (name/value pairs). 
− Objects are delimited with curly brackets 
− Commas to separate each pair 
− Each pair the colon ':' character separates the key or name from its value. 
− All keys must be strings and should be distinct from each other within that object. 
• 6. null 
− An empty value, using the word null 
© 2014 EnterpriseDB Corporation. All rights reserved. 19 
JSON is defined per RFC – 7159 
For more detail please refer 
http://tools.ietf.org/html/rfc7159
JSON Data Type Example 
{ 
"firstName": "John", -- String Type 
"lastName": "Smith", -- String Type 
"isAlive": true, -- Boolean Type 
"age": 25, -- Number Type 
"height_cm": 167.6, -- Number Type 
"address": { -- Object Type 
"streetAddress": "21 2nd Street”, 
"city": "New York”, 
"state": "NY”, 
"postalCode": "10021-3100” 
}, 
"phoneNumbers": [ // Object Array 
{ // Object 
"type": "home”, 
"number": "212 555-1234” 
}, 
{ 
"type": "office”, 
"number": "646 555-4567” 
} 
], 
"children": [], 
"spouse": null // Null 
} 
© 2014 EnterpriseDB Corporation. All rights reserved. 20
JSON 9.4 – New Operators and Functions 
• JSON 
− New JSON creation functions (json_build_object, json_build_array) 
− json_typeof – returns text data type (‘number’, ‘boolean’, …) 
• JSONB data type 
− Canonical representation 
− Whitespace and punctuation dissolved away 
− Only one value per object key is kept 
− Last insert wins 
− Key order determined by length, then bytewise comparison 
− Equality, containment and key/element presence tests 
− New JSONB creation functions 
− Smaller, faster GIN indexes 
− jsonb subdocument indexes 
− Use “get” operators to construct expression indexes on subdocument: 
− CREATE INDEX author_index ON books USING GIN ((jsondata -> 
'authors')); 
− SELECT * FROM books WHERE jsondata -> 'authors' ? 'Carl 
Bernstein' 
© 2014 EnterpriseDB Corporation. All rights reserved. 21
JSON and BSON 
• BSON – stands for 
‘Binary JSON’ 
• BSON != JSONB 
− BSON cannot represent an integer or 
floating-point number with more than 
64 bits of precision. 
− JSONB can represent arbitrary JSON values. 
• Caveat Emptor! 
− This limitation will not be obvious during early 
stages of a project! 
© 2014 EnterpriseDB Corporation. All rights reserved. 22
JSON, JSONB or HSTORE? 
• JSON/JSONB is more versatile than HSTORE 
• HSTORE provides more structure 
• JSON or JSONB? 
− if you need any of the following, use JSON 
− Storage of validated json, without processing or indexing it 
− Preservation of white space in json text 
− Preservation of object key order Preservation of duplicate object 
keys 
− Maximum input/output speed 
• For any other case, use JSONB 
© 2014 EnterpriseDB Corporation. All rights reserved. 23
JSONB and Node.js - Easy as π 
• Simple Demo of Node.js to Postgres cnnection 
© 2014 EnterpriseDB Corporation. All rights reserved. 24
JSON Performance Evaluation 
• Goal 
− Help our customers understand when to chose Postgres and 
when to chose a specialty solution 
− Help us understand where the NoSQL limits of Postgres are 
• Setup 
− Compare Postgres 9.4 to Mongo 2.6 
− Single instance setup on AWS M3.2XLARGE (32GB) 
• Test Focus 
− Data ingestion (bulk and individual) 
− Data retrieval 
© 2014 EnterpriseDB Corporation. All rights reserved. 25
Performance Evaluation 
Generate 50 Million 
JSON Documents 
Load into MongoDB 2.6 
© 2014 EnterpriseDB Corporation. All rights reserved. 26 
(IMPORT) 
Load into 
Postgres 9.4 
(COPY) 
50 Million individual 
INSERT commands 
50 Million individual 
INSERT commands 
Multiple SELECT 
statements 
Multiple SELECT 
statements 
T1 
T2 
T3
NoSQL Performance Evaluation 
© 2014 EnterpriseDB Corporation. All rights reserved. 27 
Correction to earlier versions: 
MongoDB console does not allow for 
INSERT of documents > 4K. This 
lead to truncation of the MongoDB 
size by approx. 25% of all records in 
the benchmark.
Performance Evaluations – Next Steps 
• Initial tests confirm that Postgres’ can handle many 
NoSQL workloads 
• EDB is making the test scripts publically available 
• EDB encourages community participation to 
better define where Postgres should be used 
and where specialty solutions are appropriate 
• Download the source at 
https://github.com/EnterpriseDB/pg_nosql_benchmark 
• Join us to discuss the findings at 
http://bit.ly/EDB-NoSQL-Postgres-Benchmark 
© 2014 EnterpriseDB Corporation. All rights reserved. 28
PG XDK 
• Postgres Extended Document Type Developer Kit 
• Provides end-to-end Web 2.0 example 
• Deployed as free AMI 
• First Version 
− Postgres 9.4 (beta) 
w. HSTORE and JSONB 
− Python, Django, 
Bootstrap, psycopg2 
and nginx 
• Next Version: 
PL/V8 & Node.js 
• Final Version: 
Ruby on Rails 
© 2014 EnterpriseDB Corporation. All rights reserved. 29 
AWS AMI PG XDK v0.2 - ami-1616b57e
Installing PG XDK 
• Select PG XDK v0.2 - ami-1616b57e on the AWS 
Console 
• Use https://console.aws.amazon.com/ec2/v2/home? 
region=us-east-1#LaunchInstanceWizard:ami=ami- 
1616b57e 
• Works with t2.micro (AWS Free Tier) 
• Remember to enable HHTP access in the AWS 
console 
© 2014 EnterpriseDB Corporation. All rights reserved. 30
Structured or Unstructured? 
“No SQL Only” or “Not Only SQL”? 
• Structures and standards emerge! 
• Data has references (products link to catalogues; 
products have bills of material; components appear in 
multiple products; storage locations link to ISO country 
tables) 
• When the database has duplicate data entries, then the 
application has to manage updates in multiple places – 
what happens when there is no ACID transactional 
model? 
© 2014 EnterpriseDB Corporation. All rights reserved. 31
Ultimate Flexibility with Postgres 
© 2014 EnterpriseDB Corporation. All rights reserved. 32
Say yes to ‘Not only SQL’ 
• Postgres overcomes many of the standard objections 
“It can’t be done with a conventional database system” 
• Postgres 
− Combines structured data and unstructured data (ANSI SQL 
and JSON/HSTORE) 
− Is faster (for many workloads) than than the leading NoSQL-only 
solution 
− Integrates easily with Web 2.0 application development 
environments 
− Can be deployed on-premise or in the cloud 
Do more with Postgres – the Enterprise NoSQL Solution 
© 2014 EnterpriseDB Corporation. All rights reserved. 33
Useful Resources 
• Postgres NoSQL Training Events 
− Bruce Momjian & Vibhor Kumar @ pgOpen 
− Chicago (Sept 17): NoSQL with Acid 
− Bruce Momjian & Vibhor Kumar @ pgEurope 
− Madrid (Oct 21): Maximizing Results with JSONB and PostgreSQL 
• Whitepapers @ http://www.enterprisedb.com/nosql-for-enterprise 
− PostgreSQL Advances to Meet NoSQL Challenges (business 
oriented) 
− Using the NoSQL Capabilities in Postgres (full of code examples) 
• Run the NoSQL benchmark 
− https://github.com/EnterpriseDB/pg_nosql_benchmark 
• Test drive PG XDK 
© 2014 EnterpriseDB Corporation. All rights reserved. 34
© 2014 EnterpriseDB Corporation. All rights reserved. 35

More Related Content

PDF
NoSQL on ACID - Meet Unstructured Postgres
 
PDF
Postgres NoSQL - Delivering Apps Faster
 
PDF
PgREST: Node.js in the Database
PPT
The NoSQL Way in Postgres
 
PDF
No sql way_in_pg
PDF
Postgres.foreign.data.wrappers.2015
 
PPTX
OrientDB vs Neo4j - and an introduction to NoSQL databases
PDF
Which Questions We Should Have
NoSQL on ACID - Meet Unstructured Postgres
 
Postgres NoSQL - Delivering Apps Faster
 
PgREST: Node.js in the Database
The NoSQL Way in Postgres
 
No sql way_in_pg
Postgres.foreign.data.wrappers.2015
 
OrientDB vs Neo4j - and an introduction to NoSQL databases
Which Questions We Should Have

What's hot (12)

PPTX
Introduction to MongoDB
PPTX
MongoDB Aggregation Performance
PDF
Advanced Schema Design Patterns
PPTX
MongoDB Evenings DC: Get MEAN and Lean with Docker and Kubernetes
PPTX
Introducing Azure DocumentDB - NoSQL, No Problem
KEY
Managing Social Content with MongoDB
KEY
2012 phoenix mug
PPTX
Webinar: Best Practices for Getting Started with MongoDB
KEY
OSCON 2012 MongoDB Tutorial
PPT
NoSQL Analytics: JSON Data Analysis and Acceleration in MongoDB World
PDF
Mongo DB: Operational Big Data Database
KEY
MongoDB and hadoop
Introduction to MongoDB
MongoDB Aggregation Performance
Advanced Schema Design Patterns
MongoDB Evenings DC: Get MEAN and Lean with Docker and Kubernetes
Introducing Azure DocumentDB - NoSQL, No Problem
Managing Social Content with MongoDB
2012 phoenix mug
Webinar: Best Practices for Getting Started with MongoDB
OSCON 2012 MongoDB Tutorial
NoSQL Analytics: JSON Data Analysis and Acceleration in MongoDB World
Mongo DB: Operational Big Data Database
MongoDB and hadoop
Ad

Similar to Do More with Postgres- NoSQL Applications for the Enterprise (20)

PPTX
NoSQL on ACID: Meet Unstructured Postgres
 
PDF
NoSQL Now: Postgres - The NoSQL Cake You Can Eat
PDF
Postgres: The NoSQL Cake You Can Eat
 
PDF
EDB NoSQL German Webinar 2015
 
PDF
NoSQL and Spatial Database Capabilities using PostgreSQL
 
PDF
Webscale PostgreSQL - JSONB and Horizontal Scaling Strategies
PDF
No sql bigdata and postgresql
PDF
There is Javascript in my SQL
PDF
Postgrtesql as a NoSQL Document Store - The JSON/JSONB data type
PDF
Postgres vs Mongo / Олег Бартунов (Postgres Professional)
PDF
JSON Support in DB2 for z/OS
PPT
Postgres for the Future
 
PDF
PostgreSQL, your NoSQL database
PPTX
Power JSON with PostgreSQL
 
PDF
Mathias test
PPTX
Application Development & Database Choices: Postgres Support for non Relation...
 
PDF
Json in Postgres - the Roadmap
 
PDF
An evening with Postgresql
PPTX
The rise of json in rdbms land jab17
PDF
JSON Processing in the Database using PostgreSQL 9.4 :: Data Wranglers DC :: ...
NoSQL on ACID: Meet Unstructured Postgres
 
NoSQL Now: Postgres - The NoSQL Cake You Can Eat
Postgres: The NoSQL Cake You Can Eat
 
EDB NoSQL German Webinar 2015
 
NoSQL and Spatial Database Capabilities using PostgreSQL
 
Webscale PostgreSQL - JSONB and Horizontal Scaling Strategies
No sql bigdata and postgresql
There is Javascript in my SQL
Postgrtesql as a NoSQL Document Store - The JSON/JSONB data type
Postgres vs Mongo / Олег Бартунов (Postgres Professional)
JSON Support in DB2 for z/OS
Postgres for the Future
 
PostgreSQL, your NoSQL database
Power JSON with PostgreSQL
 
Mathias test
Application Development & Database Choices: Postgres Support for non Relation...
 
Json in Postgres - the Roadmap
 
An evening with Postgresql
The rise of json in rdbms land jab17
JSON Processing in the Database using PostgreSQL 9.4 :: Data Wranglers DC :: ...
Ad

More from EDB (20)

PDF
Cloud Migration Paths: Kubernetes, IaaS, or DBaaS
 
PDF
Die 10 besten PostgreSQL-Replikationsstrategien für Ihr Unternehmen
 
PDF
Migre sus bases de datos Oracle a la nube
 
PDF
EFM Office Hours - APJ - July 29, 2021
 
PDF
Benchmarking Cloud Native PostgreSQL
 
PDF
Las Variaciones de la Replicación de PostgreSQL
 
PDF
Is There Anything PgBouncer Can’t Do?
 
PDF
Data Analysis with TensorFlow in PostgreSQL
 
PDF
Practical Partitioning in Production with Postgres
 
PDF
A Deeper Dive into EXPLAIN
 
PDF
IOT with PostgreSQL
 
PDF
A Journey from Oracle to PostgreSQL
 
PDF
Psql is awesome!
 
PDF
EDB 13 - New Enhancements for Security and Usability - APJ
 
PPTX
Comment sauvegarder correctement vos données
 
PDF
Cloud Native PostgreSQL - Italiano
 
PDF
New enhancements for security and usability in EDB 13
 
PPTX
Best Practices in Security with PostgreSQL
 
PDF
Cloud Native PostgreSQL - APJ
 
PDF
Best Practices in Security with PostgreSQL
 
Cloud Migration Paths: Kubernetes, IaaS, or DBaaS
 
Die 10 besten PostgreSQL-Replikationsstrategien für Ihr Unternehmen
 
Migre sus bases de datos Oracle a la nube
 
EFM Office Hours - APJ - July 29, 2021
 
Benchmarking Cloud Native PostgreSQL
 
Las Variaciones de la Replicación de PostgreSQL
 
Is There Anything PgBouncer Can’t Do?
 
Data Analysis with TensorFlow in PostgreSQL
 
Practical Partitioning in Production with Postgres
 
A Deeper Dive into EXPLAIN
 
IOT with PostgreSQL
 
A Journey from Oracle to PostgreSQL
 
Psql is awesome!
 
EDB 13 - New Enhancements for Security and Usability - APJ
 
Comment sauvegarder correctement vos données
 
Cloud Native PostgreSQL - Italiano
 
New enhancements for security and usability in EDB 13
 
Best Practices in Security with PostgreSQL
 
Cloud Native PostgreSQL - APJ
 
Best Practices in Security with PostgreSQL
 

Recently uploaded (20)

PDF
Review of recent advances in non-invasive hemoglobin estimation
PDF
Agricultural_Statistics_at_a_Glance_2022_0.pdf
PDF
Build a system with the filesystem maintained by OSTree @ COSCUP 2025
PDF
Machine learning based COVID-19 study performance prediction
PDF
Optimiser vos workloads AI/ML sur Amazon EC2 et AWS Graviton
PDF
7 ChatGPT Prompts to Help You Define Your Ideal Customer Profile.pdf
PPTX
Programs and apps: productivity, graphics, security and other tools
PDF
Reach Out and Touch Someone: Haptics and Empathic Computing
PDF
How UI/UX Design Impacts User Retention in Mobile Apps.pdf
PDF
Building Integrated photovoltaic BIPV_UPV.pdf
PDF
KodekX | Application Modernization Development
PPTX
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
PDF
Mobile App Security Testing_ A Comprehensive Guide.pdf
PDF
Electronic commerce courselecture one. Pdf
PDF
cuic standard and advanced reporting.pdf
PPTX
Spectroscopy.pptx food analysis technology
PPT
Teaching material agriculture food technology
PDF
Peak of Data & AI Encore- AI for Metadata and Smarter Workflows
PPT
“AI and Expert System Decision Support & Business Intelligence Systems”
PPTX
Detection-First SIEM: Rule Types, Dashboards, and Threat-Informed Strategy
Review of recent advances in non-invasive hemoglobin estimation
Agricultural_Statistics_at_a_Glance_2022_0.pdf
Build a system with the filesystem maintained by OSTree @ COSCUP 2025
Machine learning based COVID-19 study performance prediction
Optimiser vos workloads AI/ML sur Amazon EC2 et AWS Graviton
7 ChatGPT Prompts to Help You Define Your Ideal Customer Profile.pdf
Programs and apps: productivity, graphics, security and other tools
Reach Out and Touch Someone: Haptics and Empathic Computing
How UI/UX Design Impacts User Retention in Mobile Apps.pdf
Building Integrated photovoltaic BIPV_UPV.pdf
KodekX | Application Modernization Development
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
Mobile App Security Testing_ A Comprehensive Guide.pdf
Electronic commerce courselecture one. Pdf
cuic standard and advanced reporting.pdf
Spectroscopy.pptx food analysis technology
Teaching material agriculture food technology
Peak of Data & AI Encore- AI for Metadata and Smarter Workflows
“AI and Expert System Decision Support & Business Intelligence Systems”
Detection-First SIEM: Rule Types, Dashboards, and Threat-Informed Strategy

Do More with Postgres- NoSQL Applications for the Enterprise

  • 1. NoSQL Applications for the Enterprise September 10, 2014
  • 2. Agenda • Introduction to EDB • Intro to JSON, HSTORE and PL/V8 • JSON History in Postgres • JSON Data Types, Operators and Functions • JSON, JSONB and BSON – when to use which one? • JSONB and Node.JS – easy as pie • NoSQL Performance in Postgres – fast as greased lightning • PG XDK – a simple developer kit for Postgres NoSQL Applications • Say ‘Yes’ to ‘Not only SQL’ • Useful resources © 2014 EnterpriseDB Corporation. All rights reserved. 2
  • 3. Introduction to EDB © 2013 EDB All rights reserved 8.1. 3
  • 4. POSTGRES innovation Services & training © 2014 EnterpriseDB Corporation. All rights reserved. 4 ENTERPRISE reliability 24/7 support Enterprise-class features & tools Indemnification Product road-map Control Thousands of developers Fast development cycles Low cost No vendor lock-in Advanced features Enabling commercial adoption of Postgres
  • 5. Postgres Plus Advanced Server Postgres Plus Management Performance © 2014 EnterpriseDB Corporation. All rights reserved. 5 Cloud Database High Availability REMOTE DBA 24x7 SUPPORT PROFESSIONAL SERVICES TRAINING EDB Serves All Your Postgres Needs PostgreSQL Security
  • 6. Let’s Ask Ourselves, Why NoSQL? • Where did NoSQL come from? − Where all cool tech stuff comes from – Internet companies • Why did they make NoSQL? − To support huge data volumes and evolving demands for ways to work with new data types • What does NoSQL accomplish? − Enables you to work with new data types: email, mobile interactions, machine data, social connections − Enables you to work in new ways: incremental development and continuous release • Why did they have to build something new? − There were limitations to most relational databases © 2014 EnterpriseDB Corporation. All rights reserved. 6
  • 7. NoSQL: Real-world Applications • Emergency Management System − High variability among data sources required high schema flexibility • Massively Open Online Course − Massive read scalability, content integration, low latency • Patient Data and Prescription Records − Efficient write scalability • Social Marketing Analytics − Map reduce analytical approaches Source: Gartner, A Tour of NoSQL in 8 Use Cases, by Nick Heudecker and Merv Adrian, February 28, 2014 © 2014 EnterpriseDB Corporation. All rights reserved. 7
  • 8. Postgres’ Response • HSTORE − Key-value pair − Simple, fast and easy − Postgres v 8.2 – pre-dates many NoSQL-only solutions − Ideal for flat data structures that are sparsely populated • JSON − Hierarchical document model − Introduced in Postgres 9.2, perfected in 9.3 • JSONB − Binary version of JSON − Faster, more operators and even more robust − Postgres 9.4 © 2014 EnterpriseDB Corporation. All rights reserved. 8
  • 9. Postgres: Key-value Store • Supported since 2006, the HStore contrib module enables storing key/value pairs within a single column • Allows you to create a schema-less, ACID compliant data store within Postgres • Create single HStore column and include, for each row, only those keys which pertain to the record • Add attributes to a table and query without advance planning •Combines flexibility with ACID compliance © 2014 EnterpriseDB Corporation. All rights reserved. 9
  • 10. HSTORE Examples • Create a table with HSTORE field CREATE TABLE hstore_data (data HSTORE); • Insert a record into hstore_data INSERT INTO hstore_data (data) VALUES (’ "cost"=>"500", "product"=>"iphone", "provider"=>"apple"'); • Select data from hstore_data SELECT data FROM hstore_data ; ------------------------------------------ "cost"=>"500”,"product"=>"iphone”,"provider"=>"Apple" (1 row) © 2014 EnterpriseDB Corporation. All rights reserved. 10
  • 11. Postgres: Document Store • JSON is the most popular data-interchange format on the web • Derived from the ECMAScript Programming Language Standard (European Computer Manufacturers Association). • Supported by virtually every programming language • New supporting technologies continue to expand JSON’s utility − PL/V8 JavaScript extension − Node.js • Postgres has a native JSON data type (v9.2) and a JSON parser and a variety of JSON functions (v9.3) • Postgres will have a JSONB data type with binary storage and indexing (coming – v9.4) © 2014 EnterpriseDB Corporation. All rights reserved. 11
  • 12. JSON Examples • Creating a table with a JSONB field CREATE TABLE json_data (data JSONB); • Simple JSON data element: {"name": "Apple Phone", "type": "phone", "brand": "ACME", "price": 200, "available": true, "warranty_years": 1} • Inserting this data element into the table json_data INSERT INTO json_data (data) VALUES (’ { "name": "Apple Phone", "type": "phone", "brand": "ACME", "price": 200, "available": true, "warranty_years": 1 } ') © 2014 EnterpriseDB Corporation. All rights reserved. 12
  • 13. JSON Examples • JSON data element with nesting: {“full name”: “John Joseph Carl Salinger”, “names”: [ {"type": "firstname", “value”: ”John”}, {“type”: “middlename”, “value”: “Joseph”}, {“type”: “middlename”, “value”: “Carl”}, {“type”: “lastname”, “value”: “Salinger”} ] } © 2014 EnterpriseDB Corporation. All rights reserved. 13
  • 14. A simple query for JSON data SELECT DISTINCT data->>'name' as products FROM json_data; products ------------------------------ Cable TV Basic Service Package AC3 Case Black Phone Service Basic Plan AC3 Phone AC3 Case Green Phone Service Family Plan AC3 Case Red AC7 Phone © 2014 EnterpriseDB Corporation. All rights reserved. 14 This query does not return JSON data – it returns text values associated with the key ‘name’
  • 15. A query that returns JSON data SELECT data FROM json_data; data ------------------------------------------ {"name": "Apple Phone", "type": "phone", "brand": "ACME", "price": 200, "available": true, "warranty_years": 1} This query returns the JSON data in its original format © 2014 EnterpriseDB Corporation. All rights reserved. 15
  • 16. JSON and ANSI SQL - PB&J for the DBA • JSON is naturally integrated with ANSI SQL in Postgres • JSON and SQL queries use the same language, the same planner, and the same ACID compliant transaction framework • JSON and HSTORE are elegant and easy to use extensions of the underlying object-relational model © 2014 EnterpriseDB Corporation. All rights reserved. 16
  • 17. JSON and ANSI SQL Example SELECT DISTINCT product_type, data->>'brand' as Brand, data->>'available' as Availability FROM json_data JOIN products ON (products.product_type=json_data.data->>'name') WHERE json_data.data->>'available'=true; product_type | brand | availability ---------------------------+-----------+-------------- AC3 Phone | ACME | true ANSI SQL © 2014 EnterpriseDB Corporation. All rights reserved. 17 JSON No need for programmatic logic to combine SQL and NoSQL in the application – Postgres does it all
  • 18. Bridging between SQL and JSON Simple ANSI SQL Table Definition CREATE TABLE products (id integer, product_name text ); Select query returning standard data set SELECT * FROM products; id | product_name ----+-------------- 1 | iPhone 2 | Samsung 3 | Nokia Select query returning the same result as a JSON data set SELECT ROW_TO_JSON(products) FROM products; {"id":1,"product_name":"iPhone"} {"id":2,"product_name":"Samsung"} {"id":3,"product_name":"Nokia”} © 2014 EnterpriseDB Corporation. All rights reserved. 18
  • 19. JSON Data Types • 1. Number: − Signed decimal number that may contain a fractional part and may use exponential notation. − No distinction between integer and floating-point • 2. String − A sequence of zero or more Unicode characters. − Strings are delimited with double-quotation mark − Supports a backslash escaping syntax. • 3. Boolean − Either of the values true or false. • 4. Array − An ordered list of zero or more values, − Each values may be of any type. − Arrays use square bracket notation with elements being comma-separated. • 5. Object − An unordered associative array (name/value pairs). − Objects are delimited with curly brackets − Commas to separate each pair − Each pair the colon ':' character separates the key or name from its value. − All keys must be strings and should be distinct from each other within that object. • 6. null − An empty value, using the word null © 2014 EnterpriseDB Corporation. All rights reserved. 19 JSON is defined per RFC – 7159 For more detail please refer http://tools.ietf.org/html/rfc7159
  • 20. JSON Data Type Example { "firstName": "John", -- String Type "lastName": "Smith", -- String Type "isAlive": true, -- Boolean Type "age": 25, -- Number Type "height_cm": 167.6, -- Number Type "address": { -- Object Type "streetAddress": "21 2nd Street”, "city": "New York”, "state": "NY”, "postalCode": "10021-3100” }, "phoneNumbers": [ // Object Array { // Object "type": "home”, "number": "212 555-1234” }, { "type": "office”, "number": "646 555-4567” } ], "children": [], "spouse": null // Null } © 2014 EnterpriseDB Corporation. All rights reserved. 20
  • 21. JSON 9.4 – New Operators and Functions • JSON − New JSON creation functions (json_build_object, json_build_array) − json_typeof – returns text data type (‘number’, ‘boolean’, …) • JSONB data type − Canonical representation − Whitespace and punctuation dissolved away − Only one value per object key is kept − Last insert wins − Key order determined by length, then bytewise comparison − Equality, containment and key/element presence tests − New JSONB creation functions − Smaller, faster GIN indexes − jsonb subdocument indexes − Use “get” operators to construct expression indexes on subdocument: − CREATE INDEX author_index ON books USING GIN ((jsondata -> 'authors')); − SELECT * FROM books WHERE jsondata -> 'authors' ? 'Carl Bernstein' © 2014 EnterpriseDB Corporation. All rights reserved. 21
  • 22. JSON and BSON • BSON – stands for ‘Binary JSON’ • BSON != JSONB − BSON cannot represent an integer or floating-point number with more than 64 bits of precision. − JSONB can represent arbitrary JSON values. • Caveat Emptor! − This limitation will not be obvious during early stages of a project! © 2014 EnterpriseDB Corporation. All rights reserved. 22
  • 23. JSON, JSONB or HSTORE? • JSON/JSONB is more versatile than HSTORE • HSTORE provides more structure • JSON or JSONB? − if you need any of the following, use JSON − Storage of validated json, without processing or indexing it − Preservation of white space in json text − Preservation of object key order Preservation of duplicate object keys − Maximum input/output speed • For any other case, use JSONB © 2014 EnterpriseDB Corporation. All rights reserved. 23
  • 24. JSONB and Node.js - Easy as π • Simple Demo of Node.js to Postgres cnnection © 2014 EnterpriseDB Corporation. All rights reserved. 24
  • 25. JSON Performance Evaluation • Goal − Help our customers understand when to chose Postgres and when to chose a specialty solution − Help us understand where the NoSQL limits of Postgres are • Setup − Compare Postgres 9.4 to Mongo 2.6 − Single instance setup on AWS M3.2XLARGE (32GB) • Test Focus − Data ingestion (bulk and individual) − Data retrieval © 2014 EnterpriseDB Corporation. All rights reserved. 25
  • 26. Performance Evaluation Generate 50 Million JSON Documents Load into MongoDB 2.6 © 2014 EnterpriseDB Corporation. All rights reserved. 26 (IMPORT) Load into Postgres 9.4 (COPY) 50 Million individual INSERT commands 50 Million individual INSERT commands Multiple SELECT statements Multiple SELECT statements T1 T2 T3
  • 27. NoSQL Performance Evaluation © 2014 EnterpriseDB Corporation. All rights reserved. 27 Correction to earlier versions: MongoDB console does not allow for INSERT of documents > 4K. This lead to truncation of the MongoDB size by approx. 25% of all records in the benchmark.
  • 28. Performance Evaluations – Next Steps • Initial tests confirm that Postgres’ can handle many NoSQL workloads • EDB is making the test scripts publically available • EDB encourages community participation to better define where Postgres should be used and where specialty solutions are appropriate • Download the source at https://github.com/EnterpriseDB/pg_nosql_benchmark • Join us to discuss the findings at http://bit.ly/EDB-NoSQL-Postgres-Benchmark © 2014 EnterpriseDB Corporation. All rights reserved. 28
  • 29. PG XDK • Postgres Extended Document Type Developer Kit • Provides end-to-end Web 2.0 example • Deployed as free AMI • First Version − Postgres 9.4 (beta) w. HSTORE and JSONB − Python, Django, Bootstrap, psycopg2 and nginx • Next Version: PL/V8 & Node.js • Final Version: Ruby on Rails © 2014 EnterpriseDB Corporation. All rights reserved. 29 AWS AMI PG XDK v0.2 - ami-1616b57e
  • 30. Installing PG XDK • Select PG XDK v0.2 - ami-1616b57e on the AWS Console • Use https://console.aws.amazon.com/ec2/v2/home? region=us-east-1#LaunchInstanceWizard:ami=ami- 1616b57e • Works with t2.micro (AWS Free Tier) • Remember to enable HHTP access in the AWS console © 2014 EnterpriseDB Corporation. All rights reserved. 30
  • 31. Structured or Unstructured? “No SQL Only” or “Not Only SQL”? • Structures and standards emerge! • Data has references (products link to catalogues; products have bills of material; components appear in multiple products; storage locations link to ISO country tables) • When the database has duplicate data entries, then the application has to manage updates in multiple places – what happens when there is no ACID transactional model? © 2014 EnterpriseDB Corporation. All rights reserved. 31
  • 32. Ultimate Flexibility with Postgres © 2014 EnterpriseDB Corporation. All rights reserved. 32
  • 33. Say yes to ‘Not only SQL’ • Postgres overcomes many of the standard objections “It can’t be done with a conventional database system” • Postgres − Combines structured data and unstructured data (ANSI SQL and JSON/HSTORE) − Is faster (for many workloads) than than the leading NoSQL-only solution − Integrates easily with Web 2.0 application development environments − Can be deployed on-premise or in the cloud Do more with Postgres – the Enterprise NoSQL Solution © 2014 EnterpriseDB Corporation. All rights reserved. 33
  • 34. Useful Resources • Postgres NoSQL Training Events − Bruce Momjian & Vibhor Kumar @ pgOpen − Chicago (Sept 17): NoSQL with Acid − Bruce Momjian & Vibhor Kumar @ pgEurope − Madrid (Oct 21): Maximizing Results with JSONB and PostgreSQL • Whitepapers @ http://www.enterprisedb.com/nosql-for-enterprise − PostgreSQL Advances to Meet NoSQL Challenges (business oriented) − Using the NoSQL Capabilities in Postgres (full of code examples) • Run the NoSQL benchmark − https://github.com/EnterpriseDB/pg_nosql_benchmark • Test drive PG XDK © 2014 EnterpriseDB Corporation. All rights reserved. 34
  • 35. © 2014 EnterpriseDB Corporation. All rights reserved. 35

Editor's Notes

  • #2: Title slide Add customer logo if applicable
  • #3: This is the roadmap for content of the meeting Adjust as necessary based on previous discussions on objectives and areas of interest Goal is to set expectations for content to be discussed Agree on specific meeting objectives/outcomes up front Inquire about any specific areas of interest or current needs that should be emphasized
  • #4: Introduction to EDB
  • #5: EDB provides enterprises and government agencies with the commercial support and reliability needed to take full advantage of open-source Postgres innovation and cost benefits: 24/7 Support Enterprise-class features & tools Packaged and professional services Wide array of classroom and on-demand training Clear visibility & influence over product road-map EDB gives you the responsiveness & dependability you need to be successful
  • #6: Overview of EDB products & Services NOTE: THIS SLIDE HAS BUILT-IN “JUMPING” SO YOU CAN CLICK ON ANY ICON TO DRILL DOWN THEN RETURN TO THE OVERVIEW; here’s how: Click on each icon (DBs and services) to drill down to details Once on detail page, click on the icon to move directly to the next icon detail, or Click on the area outside of the drill down detail to return back to the main product & services overview Main message: EDB is the BEST source for all your Postgres needs: EDB is a database internals product development company EDB creates add-on features, tools and cloud capabilities designed for enterprise-class workloads EDB’s deep technical expertise makes it your best source for support and professional services Sample script: “Just as you’d expect from any enterprise software company, EDB provides various products and services to ensure our customers make the most out of their Postgres deployments. From our standard edition which includes support for the open source PostgreSQL to PPAS and PPCD, we have the right database for your application. We back that up with global follow the sun support, packaged services and training along with RDBA services. Many of customers run PostgreSQL and come to us for support and to take advantage of our add-on functionality, such as PEM and xDB replication server. For lots of workloads this is a fine solution. The majority of our customers do run PPAS, though. The combination of its low cost--$6,900 per year per socket, along with a ton of additional features and functionality above the Standard Edition, it’s a great fit for workloads with lots of concurrent users and lots of transactions. Of course, PPCD gives customers the ability to run either PostgreSQL or PPAS, but in a cloud environment.”
  • #7: Bruce
  • #8: Marc Linster Emergency Management System: Rapid integration of heterogeneous data sources for new analysis platform - High variability among data sources required high schema flexibility MOOC: - Massive read sclability - Content integration from a variety of sources - Low latency at peak volumes Patient data and prescription records - Distributed key-value store reduced complexity of integration and management overhead Online Gambling - Key value store simplified development (up to 75% acceleration) - Efficient write sclability Online Marketing Firm - massive time series storage facilitated by table style DBMS - data center replication Social Marketing Analytics - Massive amounts of data from social media systems - Map-reduce analytical approaches Product Manufacturing Master Data Management - Network dependencies between product lines Drug Discovery - Drug Analysis
  • #10: Bruce
  • #12: Bruce
  • #20: Note: 1. JSON generally ignores any whitespace around or between syntactic elements (values and punctuation, but not within a string value). 2. JSON only recognizes four specific whitespace characters: i. the space ii. horizontal tab, iii. line feed, and carriage return. 3. JSON does not provide or allow any sort of comment syntax.