Doing More with Postgres - Yesterday's Vision Becomes Today's Reality
Download as PPTX, PDF1 like1,350 views
The document discusses the evolution and capabilities of PostgreSQL, particularly in relation to cloud services and enterprise applications. It highlights PostgreSQL's growth in the NoSQL domain, its performance benefits compared to other databases, and its role in powering ERP solutions. EDB emphasizes PostgreSQL's flexibility, ease of deployment, and ability to handle both structured and unstructured data efficiently.
Doing More with Postgres - Yesterday's Vision Becomes Today's Reality
- 1. Postgres Evolutions Marc Linster © 2013 EDB All rights reserved 8.1. 1
- 2. Agenda Who is EDB? Market forces It’s happening – in the cloud It’s happening – Postgres powers ERP It’s happening – Not Only SQL It’s happening – platform coexistence © 2013 EDB All rights reserved 8.1. 2
- 3. PostgreSQL Who is EDB? Postgres Plus Advanced Server Postgres Plus © 2013 EDB All rights reserved 8.1. 3 Cloud • 180+ employees • Global presence • Focused on serving needs of enterprise, government and ISV markets • Platinum Sponsor of PostgreSQL
- 4. Market forces influence direction (PG Europe 2013) Easy to use / deploy New Workloads/Platforms (Big Data, NoSQL & Cloud) © 2013 EDB All rights reserved 8.1. 4 High-end Enterprise Requirements PostgreSQL is growing from the “middle-out”
- 5. Marketplace Demand – Next Five Years • Diagnosing Problems • Configuring for success • Still easier installs • Tighter integration with frameworks • Integration with other data stores • Very simple in the cloud © 2013 EDB All rights reserved 8.1. 5 • Liasons with other communities • FDW for common no-SQL DB’s • Continue to evolve new datatypes: JSON, XML, HStore • Vertical Scale (parallel query) • Horizontal Scale • Performance Diagnostics • Incremental Backup • Integration with other data stores • Zero down time upgrades Easy to use / deploy High-end Enterprise Requirements New Workloads & Platforms (Big Data/No/SQL/Cloud)
- 6. Postgres: Cloud – It’s Happening Source: Gartner 2013 Forecast: Public Cloud Services, Worldwide, 2011-2017, 3Q13 Update 45 40 35 30 25 20 15 10 5 © 2013 EDB All rights reserved 8.1. 6 2011 2012 2013 2014 2015 2016 2017 160 140 120 100 80 60 40 20 Cloud PaaS/Database Management Systems ($M) 17 28 42 59 82 114 149 0 Cloud PaaS/DBMS ($M) • Cloud PaaS has the highest growth rate 35%+ (according to Gartner Group) • EDB‘s Postgres Plus Cloud Database at AWS confirms this − Easy installation − Elastic and scalable − Automatic failover − PITR − VPC − …. 0 PPCD New Cluster# Growth/June – Sept’13
- 7. Postgres powers ERP • Infor LN (formerly Baan ERP) supported on Postgres Plus Advanced Server Joint announcement by Infor, Red Hat & EDB - April 2014 • Part of the Infor Open Source Initiative “We can deliver a better experience with open source because it is open enough to allow us to configure a seamless experience with our applications with fewer licensing and distribution restrictions. We can easily move images between on premise and cloud as customer needs dictate.” - Charles Philips, CEO, Infor © 2014 EnterpriseDB Corporation. All rights reserved. 7
- 8. Infor LN/Postgres Stack LN / Infor Ming.le LN UI LN VM + DB Driver EDB OCL libpq Postgres Plus Red Hat Enterprise Linux Red Hat Enterprise Virtualization © 2013 EDB All rights reserved 8.1. 8 Presentation Tier •Red Hat JBoss •Apache Tomcat Application Tier •LN VM + LN DB Driver Database Tier •Postgres Plus Advanced Server Operating System Tier •Red Hat Enterprise Linux •Red Hat Enterprise Virtualization
- 9. Performance (250 Concurrent Users) Normalized Transaction Times – 90th percentile (smaller is better) © 2013 EDB All rights reserved 8.1. 9 Postgres – 87.4 transactions/user, 15 minutes Proprietary RDBMS – 88 transactions/user, 120% 100% 80% 60% 40% 20% EnterpriseDB Proprietary RDBMS Limit 15 minutes 0% Its happening! Postgres powers ERP!
- 10. It’s happening – Not only SQL Postgres + Documents (JSON) + KVP (HSTORE) = Best possible NoSQL solution © 2013 EDB All rights reserved 8.1. 10
- 11. Standard Objections • SQL databases can’t handle NoSQL data types − Postgres can (JSON and KVP) • SQL databases are too slow and can’t ingest the data fast enough − Postgres can keep up − We ran the tests − Benchmarks are public − Great feedback • SQL databases can’t support Web 2.0 development approaches − Postgres can − Support unstructured and structured data in the same environment − Move data from unstructured to structured (and back) − Use Web 2.0 languages inside the database (e.g.: PL/V8) or in the application (node.js) − Integrate NoSQL data (FDW) © 2013 EDB All rights reserved 8.1. 11
- 12. JSON Performance Evaluation • Goal − Help our customers understand when to chose Postgres and when to chose a specialty solution − Help us understand where the NoSQL limits of Postgres are • Setup − Compare Postgres 9.4 to Mongo 2.6 − Single instance setup on AWS M3.2XLARGE (32GB) • Test Focus − Data ingestion (bulk and individual) − Data retrieval © 2013 EDB All rights reserved 8.1. 12
- 13. Performance Evaluation Generate 50 Million JSON Documents Load into MongoDB 2.6 © 2013 EDB All rights reserved 8.1. 13 (IMPORT) Load into Postgres 9.4 (COPY) 50 Million individual INSERT commands 50 Million individual INSERT commands Multiple SELECT statements Multiple SELECT statements T1 T2 T3
- 14. NoSQL Performance Evaluation Mongo DB 2.4/Postgres 9.4 Relative Performance Comparison (50 Million Documents) 276% 295% 500% 450% 400% 350% 300% 250% 200% 150% 100% 50% © 2013 EDB All rights reserved 8.1. 14 465% 208% 0% Data Load Insert Select Size Postgres MongoDB Postgres MongoDB Data Load (s) 4,732 13,046 Insert (s) 29,236 86,253 Select (s) 594 2,763 Size (GB) 69 145 Correction to earlier versions: MongoDB console does not allow for INSERT of documents > 4K. This lead to truncation of the MongoDB size by approx. 25% of all records in the benchmark. Thank you Alvaro Tortosa!
- 15. Performance Evaluations – Next Steps • Initial tests confirm that Postgres’ can handle many NoSQL workloads • The test scripts are publically available https://github.com/EnterpriseDB/pg_nosql_benchmark • EDB encourages community participation to better define where Postgres should be used and where specialty solutions are appropriate • Join us to discuss the findings at http://bit.ly/EDB-NoSQL-Postgres-Benchmark © 2013 EDB All rights reserved 8.1. 15
- 16. PG XDK – Showcase Postgres & Web 2.0 Development • Postgres Extended Document Type Developer Kit • Provides end-to-end Web 2.0 example – all sources included • Deployed as free AMI • First Version − Postgres 9.4 (beta) w. HSTORE and JSONB − Python, Django, Bootstrap, psycopg2 and nginx • Next Version: PL/V8 & Node.js • Final Version: Ruby on Rails © 2013 EDB All rights reserved 8.1. 16 AWS AMI PG XDK v0.2 - ami-1616b57e
- 17. Installing PG XDK • Select PG XDK v0.2 - ami-1616b57e on the AWS Console • Use https://console.aws.amazon.com/ec2/v2/home?region=us-east- 1#LaunchInstanceWizard:ami=ami-1616b57e • Works with t2.micro (AWS Free Tier) • Remember to enable HHTP access in the AWS console © 2013 EDB All rights reserved 8.1. 17
- 18. Foreign Data Wrappers – Co-Existence Platform • FDW implements SQL/MED ("SQL Management of External Data") • FDW − Makes data on other servers (or services) look like tables in Postgres − available for databases (MongoDB, MySQL, Oracle, …), files, services (Twitter, …) • MongoDB FDW: https://github.com/EnterpriseDB (a collaboration with CitusDB) • The list: https://wiki.postgresql.org/wiki/Foreign_data_wrappers © 2013 EDB All rights reserved 8.1. 18
- 19. Conclusion – It’s happening • Has great deployment flexibility – start in the cloud, land in the data center • Powers ERP workloads successfully • Is the most flexible and most capable NoSQL (Not only SQL) operational database solution out there • Outperforms key NoSQL-only solutions in important workloads • A great platform for Web 2.0 applications © 2013 EDB All rights reserved 8.1. 21
- 20. © 2013 EDB All rights reserved 8.1. 22
Editor's Notes
- #2: -Welcome -Me -Apologies for Tom
- #4: -Growing 84% last year; similar this year -Global -DB Product/Development company -Main products -Training & Professional Services -Committed, platinum Sponsor
- #5: -These are the market forces that are shaping the future for PostgreSQL: --on one end are the smaller applications; web applications and app developers who need ease of use and rapid deployment --on the other end you have large, enterprise applications who need advanced features, security, scalability and HA for mission critical applications -Postgres occupies a middle-ground position and as we noted earlier has been gradually expanding in both directions -The emergence of new workloads & cloud will increasingly grow in shaping future customer needs and require RDBMS’s to expand capabilities (as noted by Gartner) These are the market segments we will need to address to ensure positive future growth
- #6: Easy to use: -Diagnosing problems – People find this hard. -The out of the box configuration should be more optimized for the common use case. -Can we make the installation experience even easier? -Continually seek opportunities to win the cloud. High end Enterprise: -Need to continue to work to take advantage of all the resources of a single machine. There are many machines with 256 processors. -Horizontal Scale – This is a big issue for the cloud and in general. You can figure out ways to do it if you are very smart. Can we lower the the brain power required to do it? -Diagnostics is a request from every audience. NoSQL / New SQL -We are not going to reach all their use cases. -However, we should integrate well with them. -Continue involvement and work on the new data types is important.