SlideShare a Scribd company logo

Using the PostgreSQL Extension Ecosystem for Advanced Analytics

Download as PPTX, PDF
Chartio, profile picture
Chartio

Sales were the topic of discussion. However, no other details were provided in the document beyond that one word, so an informative summary cannot be generated from the limited information given.

Software
1 of 45
Downloaded 22 times
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
sales@chartio.com (855) 232-0320 sales@chartio.com (855) 232-0320 Using the PostgreSQL Extension Ecosystem for Advanced Analytics
sales@chartio.com (855) 232-0320 - The problem - The prevailing view vs. the practical reality - A possible solution - Or just building blocks? - Nearness - Near at hand, near to our skill set, near to our capabilities - A more complete solution - The PostgreSQL extension ecosystem Agenda
sales@chartio.com (855) 232-0320 sales@chartio.com (855) 232-0320 The Problem The Prevailing View vs. The Practical Reality
sales@chartio.com (855) 232-0320 The Prevailing View - Logical Dimension Relational Non-Relational Schema objects ● Structured rows and columns ● Schema on write ● Referential integrity ● Painful migrations ● Unstructured files, docs, etc ● Schema on read ● No referential integrity ● No migrations Query languages ● SQL ● Declarative ● Easy enough for non-tech users ● Various ● Procedural ● Requires some programming skills Exploratory analysis ● Native support for joins ● Interactive/low execution overhead ● No native support for joins ● OLAP - Batch processing Data science and ML ● Only descriptive statistics ● Requires exporting dumps/samples ● Robust ecosystem ● Does not require exports
sales@chartio.com (855) 232-0320 The Prevailing View - Physical Dimension Relational Non-Relational Parallel query processing ● Single node system ● Single process per query ● Multiple node system ● Multiple processes per query Concurrency ● High concurrency ● Single process per connection ● OLAP - low concurrency/high scheduling overhead High Availability & Replication ● Async and sync replication ● HA may not be native ● Async and sync replication ● HA likely to be native Sharding ● Sharding may not be native ● Difficult to manage ● Sharding likely to be native ● Easy to manage
sales@chartio.com (855) 232-0320 The Prevailing View - Summary - RDBMS have nice properties for producing rich data - ACID, relational integrity, constraints, strong data types - Easier for non-tech users and exploratory analysis - Probably don’t meet the needs of today’s analysts - Data science & Machine Learning - Parallel processing - Definitely don’t meet the needs of today’s apps - Schema migrations - Replication and sharding
sales@chartio.com (855) 232-0320 The Practical Reality
sales@chartio.com (855) 232-0320 sales@chartio.com (855) 232-0320 But we still want more advanced functionality. The Practical Reality
sales@chartio.com (855) 232-0320 sales@chartio.com (855) 232-0320 A Possible Solution Or Just Building Blocks?
sales@chartio.com (855) 232-0320 Modern SQL - Many people still think of SQL in terms of SQL-92 - Since then we’ve had: SQL:1999, SQL:2003, SQL:2006, SQL:2008, SQL:2011 - http://use-the-index-luke.com/blog/2015-02/modern-sql - Common Table Expressions (CTEs) / Recursive CTEs - Window Functions - Ordered-set Aggregates - Lateral joins - Temporal support - The list goes on...
sales@chartio.com (855) 232-0320 Procedural Languages - Native pgSQL Tcl Perl Python - Community Java PHP R Javascript Ruby Scheme sh
sales@chartio.com (855) 232-0320 sales@chartio.com (855) 232-0320 These solve some problems. For others, they are just building blocks. Building Blocks
sales@chartio.com (855) 232-0320 sales@chartio.com (855) 232-0320 Nearness Near at Hand Near to Our Skill Set Near to Our Capabilities
sales@chartio.com (855) 232-0320 - http://www.infoq.com/presentations/Simple-Made-Easy Nearness
sales@chartio.com (855) 232-0320 - Near at hand - Easily installable - Near to our skill set - Familiar tool/language/abstraction - Modular and composable - Near to our capabilities - Capable of solving a problem in our domain Nearness Drives Adoption
sales@chartio.com (855) 232-0320 sales@chartio.com (855) 232-0320 A More Complete Solution The PostgreSQL Extension Ecosystem
sales@chartio.com (855) 232-0320 Postgres Extension Ecosystem Examples - PostgreSQL Extension Network: http://pgxn.org/ - UDFs & operators: https://github.com/eulerto/pg_similarity - UDAs & data types: https://github.com/aggregateknowledge/postgresql-hll - Foreign Data Wrappers: http://multicorn.org/, https://github.com/shish/pgosquery - Indexes: https://github.com/zombodb/zombodb - Composing Extension Methods: http://doc.madlib.net/ - MPP: https://www.citusdata.com/, https://github.com/greenplum-db/gpdb - Composing Extensions - Custom Background Workers: https://github.com/no0p/alps - Record linking: http://no0p.github.io/2015/10/20/record_linking.html#/
sales@chartio.com (855) 232-0320 Postgres Extension Ecosystem Examples - PostgreSQL Extension Network: http://pgxn.org/ - UDFs & operators: https://github.com/eulerto/pg_similarity - UDAs & data types: https://github.com/aggregateknowledge/postgresql-hll - Foreign Data Wrappers: http://multicorn.org/, https://github.com/shish/pgosquery - Indexes: https://github.com/zombodb/zombodb - Composing Extension Methods: http://doc.madlib.net/ - MPP: https://www.citusdata.com/, https://github.com/greenplum-db/gpdb - Composing Extensions - Custom Background Workers: https://github.com/no0p/alps - Record linking: http://no0p.github.io/2015/10/20/record_linking.html#/
sales@chartio.com (855) 232-0320 - Package Manager: pgxn - Index/Network: http://pgxn.org/ - PyPI, RubyGems, CPAN, CRAN The PostgreSQL Extension Network
sales@chartio.com (855) 232-0320 The PostgreSQL Extension Network - Near at hand - pgxn search semver - pgxn info semver - pgxn install semver - pgxn load –d somedb semver - pgxn unload –d somedb semver - pgxn uninstall semver - Search github? google? mailing list? - Github README? - git clone; make; make install; - psql –c “CREATE EXTENSION IF NOT EXISTS” - psql –c “DROP EXTENSION IF EXISTS” - make uninstall?
sales@chartio.com (855) 232-0320 Postgres Extension Ecosystem Examples - PostgreSQL Extension Network: http://pgxn.org/ - UDFs & operators: https://github.com/eulerto/pg_similarity - UDAs & data types: https://github.com/aggregateknowledge/postgresql-hll - Foreign Data Wrappers: http://multicorn.org/, https://github.com/shish/pgosquery - Indexes: https://github.com/zombodb/zombodb - Composing Extension Methods: http://doc.madlib.net/ - MPP: https://www.citusdata.com/, https://github.com/greenplum-db/gpdb - Composing Extensions - Custom Background Workers: https://github.com/no0p/alps - Record linking: http://no0p.github.io/2015/10/20/record_linking.html#/
sales@chartio.com (855) 232-0320 UDFs & Operators: pg_similarity - Near to our capabilities - Similarity coefficient algorithms - L1 Distance - Cosine Distance - Dice Coefficient - Euclidean Distance - Hamming Distance - Jaccard Coefficient - Jaro Distance - Jaro-Winkler Distance - Levenshtein Distance - Matching Coefficient - Monge-Elkan Coefficient - Needleman-Wunsch Coefficient - Overlap Coefficient - Q-Gram Distance - Smith-Waterman Coefficient - Smith-Waterman-Gotoh Coefficient - Soundex Distance
sales@chartio.com (855) 232-0320 UDFs & Operators: pg_similarity - Near to our skill set
sales@chartio.com (855) 232-0320 UDFs & Operators: pg_similarity - Implementation
sales@chartio.com (855) 232-0320 Postgres Extension Ecosystem Examples - PostgreSQL Extension Network: http://pgxn.org/ - UDFs & Operators: https://github.com/eulerto/pg_similarity - UDAs & Data Types: https://github.com/aggregateknowledge/postgresql-hll - Foreign Data Wrappers: http://multicorn.org/, https://github.com/shish/pgosquery - Indexes: https://github.com/zombodb/zombodb - Composing Extension Methods: http://doc.madlib.net/ - MPP: https://www.citusdata.com/, https://github.com/greenplum-db/gpdb - Composing Extensions - Custom Background Workers: https://github.com/no0p/alps - Record linking: http://no0p.github.io/2015/10/20/record_linking.html#/
sales@chartio.com (855) 232-0320 UDAs & Data Types: postgresql-hll - Near to our capabilities & near to our skill set - Data type - Estimate count distinct with tunable precision - 1280 bytes estimates tens of billions of distinct values with few percent error
sales@chartio.com (855) 232-0320 UDAs & Data Types: postgresql-hll
sales@chartio.com (855) 232-0320 UDAs & Data Types: postgresql-hll - Implementation
sales@chartio.com (855) 232-0320 Postgres Extension Ecosystem Examples - PostgreSQL Extension Network: http://pgxn.org/ - UDFs & Operators: https://github.com/eulerto/pg_similarity - UDAs & Data Types: https://github.com/aggregateknowledge/postgresql-hll - Foreign Data Wrappers: http://multicorn.org/, https://github.com/shish/pgosquery - Indexes: https://github.com/zombodb/zombodb - Composing Extension Methods: http://doc.madlib.net/ - MPP: https://www.citusdata.com/, https://github.com/greenplum-db/gpdb - Composing Extensions - Custom Background Workers: https://github.com/no0p/alps - Record linking: http://no0p.github.io/2015/10/20/record_linking.html#/
sales@chartio.com (855) 232-0320 Foreign Data Wrappers: API
sales@chartio.com (855) 232-0320 Foreign Data Wrappers: multicorn - Near to our skill set
sales@chartio.com (855) 232-0320 Foreign Data Wrappers: pgosquery - Near at hand
sales@chartio.com (855) 232-0320 Postgres Extension Ecosystem Examples - PostgreSQL Extension Network: http://pgxn.org/ - UDFs & Operators: https://github.com/eulerto/pg_similarity - UDAs & Data Types: https://github.com/aggregateknowledge/postgresql-hll - Foreign Data Wrappers: http://multicorn.org/, https://github.com/shish/pgosquery - Indexes: https://github.com/zombodb/zombodb - Composing Extension Methods: http://doc.madlib.net/ - MPP: https://www.citusdata.com/, https://github.com/greenplum-db/gpdb - Composing Extensions - Custom Background Workers: https://github.com/no0p/alps - Record linking: http://no0p.github.io/2015/10/20/record_linking.html#/
sales@chartio.com (855) 232-0320 Indexes: ZomboDB - Index Access Method API - http://www.postgresql.org/docs/9.4/static/indexam.html
sales@chartio.com (855) 232-0320 Postgres Extension Ecosystem Examples - PostgreSQL Extension Network: http://pgxn.org/ - UDFs & Operators: https://github.com/eulerto/pg_similarity - UDAs & Data Types: https://github.com/aggregateknowledge/postgresql-hll - Foreign Data Wrappers: http://multicorn.org/, https://github.com/shish/pgosquery - Indexes (GiST, GIN): https://github.com/zombodb/zombodb - Composing Extension Methods: http://doc.madlib.net/ - MPP: https://www.citusdata.com/, https://github.com/greenplum-db/gpdb - Composing Extensions - Custom Background Workers: https://github.com/no0p/alps - Record linking: http://no0p.github.io/2015/10/20/record_linking.html#/
sales@chartio.com (855) 232-0320 Composing Extension Methods: MADlib Near to our capabilities
sales@chartio.com (855) 232-0320 Composing Extension Methods: MADlib - Near to our skill set
sales@chartio.com (855) 232-0320 Composing Extension Methods: MADlib
sales@chartio.com (855) 232-0320 Postgres Extension Ecosystem Examples - PostgreSQL Extension Network: http://pgxn.org/ - UDFs & Operators: https://github.com/eulerto/pg_similarity - UDAs & Data Types: https://github.com/aggregateknowledge/postgresql-hll - Foreign Data Wrappers: http://multicorn.org/, https://github.com/shish/pgosquery - Indexes: https://github.com/zombodb/zombodb - Composing Extension Methods: http://doc.madlib.net/ - MPP: https://www.citusdata.com/, https://github.com/greenplum-db/gpdb - Composing Extensions - Custom Background Workers: https://github.com/no0p/alps - Record linking: http://no0p.github.io/2015/10/20/record_linking.html#/
sales@chartio.com (855) 232-0320 Parallel Processing - Parallel sequential scan - http://rhaas.blogspot.com/2015/11/parallel-sequential-scan-is-committed.html - Columnar FDW: - https://github.com/citusdata/cstore_fdw
sales@chartio.com (855) 232-0320 Postgres Extension Ecosystem Examples - PostgreSQL Extension Network: http://pgxn.org/ - UDFs & Operators: https://github.com/eulerto/pg_similarity - UDAs & Data Types: https://github.com/aggregateknowledge/postgresql-hll - Foreign Data Wrappers: http://multicorn.org/, https://github.com/shish/pgosquery - Indexes: https://github.com/zombodb/zombodb - Composing Extension Methods: http://doc.madlib.net/ - MPP: https://www.citusdata.com/, https://github.com/greenplum-db/gpdb - Composing Extensions - Custom Background Workers: https://github.com/no0p/alps - Record linking: http://no0p.github.io/2015/10/20/record_linking.html#/
sales@chartio.com (855) 232-0320 Composing Extensions: Alps
sales@chartio.com (855) 232-0320 Composing Extensions: Record Linking
sales@chartio.com (855) 232-0320 Beyond Analytics - Web app framework - http://blog.aquameta.com/ - REST API - https://github.com/begriffs/postgrest - Unit testing framework - http://pgtap.org/ - Firewall - https://github.com/uptimejp/sql_firewall - More every week!
sales@chartio.com (855) 232-0320 Conclusion - With PostgreSQL, you get - more than rows and columns - more than SELECT, FROM, WHERE, GROUP BY, ORDER BY - more than a single machine - Make sure you get the full return on your investment! Get your Chartio free trial! sales@chartio.com (855) 232-0320

More Related Content

PPTX
Producing and Analyzing Rich Data with PostgreSQL
Chartio
 
PPTX
Learn How to Run Python on Redshift
Chartio
 
PDF
Apache Arrow Flight: A New Gold Standard for Data Transport
Wes McKinney
 
PDF
MatFast: In-Memory Distributed Matrix Computation Processing and Optimization...
Spark Summit
 
PDF
Apache Arrow: Cross-language Development Platform for In-memory Data
Wes McKinney
 
PPTX
Future of pandas
Jeff Reback
 
PDF
Ursa Labs and Apache Arrow in 2019
Wes McKinney
 
PDF
Spark Meetup Amsterdam - Dealing with Bad Actors in ETL, Databricks
GoDataDriven
 
Producing and Analyzing Rich Data with PostgreSQL
Chartio
 
Learn How to Run Python on Redshift
Chartio
 
Apache Arrow Flight: A New Gold Standard for Data Transport
Wes McKinney
 
MatFast: In-Memory Distributed Matrix Computation Processing and Optimization...
Spark Summit
 
Apache Arrow: Cross-language Development Platform for In-memory Data
Wes McKinney
 
Future of pandas
Jeff Reback
 
Ursa Labs and Apache Arrow in 2019
Wes McKinney
 
Spark Meetup Amsterdam - Dealing with Bad Actors in ETL, Databricks
GoDataDriven
 

What's hot (20)

PDF
ODI11g, Hadoop and "Big Data" Sources
Mark Rittman
 
PPTX
BDM9 - Comparison of Oracle RDBMS and Cloudera Impala for a hospital use case
David Lauzon
 
PPTX
BDM8 - Near-realtime Big Data Analytics using Impala
David Lauzon
 
PPTX
HBase and Drill: How loosley typed SQL is ideal for NoSQL
DataWorks Summit
 
PDF
OpenStack Trove Day (19 Aug 2014, Cambridge MA) - Sahara
spinningmatt
 
PDF
Apache Arrow at DataEngConf Barcelona 2018
Wes McKinney
 
PDF
ACM TechTalks : Apache Arrow and the Future of Data Frames
Wes McKinney
 
PDF
Apache Arrow -- Cross-language development platform for in-memory data
Wes McKinney
 
PDF
DataFrames: The Good, Bad, and Ugly
Wes McKinney
 
PDF
Data Storage Tips for Optimal Spark Performance-(Vida Ha, Databricks)
Spark Summit
 
PDF
Apache Arrow: Present and Future @ ScaledML 2020
Wes McKinney
 
PDF
introduction to Neo4j (Tabriz Software Open Talks)
Farzin Bagheri
 
PDF
Apache Arrow: Leveling Up the Analytics Stack
Wes McKinney
 
PPT
Hadoop Frameworks Panel__HadoopSummit2010
Yahoo Developer Network
 
PDF
Apache Arrow Workshop at VLDB 2019 / BOSS Session
Wes McKinney
 
PDF
PyCon.DE / PyData Karlsruhe keynote: "Looking backward, looking forward"
Wes McKinney
 
PDF
From flat files to deconstructed database
Julien Le Dem
 
PDF
Performance of Spark vs MapReduce
Edureka!
 
PPTX
Apache drill
MapR Technologies
 
ODP
The other Apache Technologies your Big Data solution needs
gagravarr
 
ODI11g, Hadoop and "Big Data" Sources
Mark Rittman
 
BDM9 - Comparison of Oracle RDBMS and Cloudera Impala for a hospital use case
David Lauzon
 
BDM8 - Near-realtime Big Data Analytics using Impala
David Lauzon
 
HBase and Drill: How loosley typed SQL is ideal for NoSQL
DataWorks Summit
 
OpenStack Trove Day (19 Aug 2014, Cambridge MA) - Sahara
spinningmatt
 
Apache Arrow at DataEngConf Barcelona 2018
Wes McKinney
 
ACM TechTalks : Apache Arrow and the Future of Data Frames
Wes McKinney
 
Apache Arrow -- Cross-language development platform for in-memory data
Wes McKinney
 
DataFrames: The Good, Bad, and Ugly
Wes McKinney
 
Data Storage Tips for Optimal Spark Performance-(Vida Ha, Databricks)
Spark Summit
 
Apache Arrow: Present and Future @ ScaledML 2020
Wes McKinney
 
introduction to Neo4j (Tabriz Software Open Talks)
Farzin Bagheri
 
Apache Arrow: Leveling Up the Analytics Stack
Wes McKinney
 
Hadoop Frameworks Panel__HadoopSummit2010
Yahoo Developer Network
 
Apache Arrow Workshop at VLDB 2019 / BOSS Session
Wes McKinney
 
PyCon.DE / PyData Karlsruhe keynote: "Looking backward, looking forward"
Wes McKinney
 
From flat files to deconstructed database
Julien Le Dem
 
Performance of Spark vs MapReduce
Edureka!
 
Apache drill
MapR Technologies
 
The other Apache Technologies your Big Data solution needs
gagravarr
 
Ad

Similar to Using the PostgreSQL Extension Ecosystem for Advanced Analytics (20)

PDF
PostgreSQL Extension APIs are Changing the Face of Relational Databases | PGC...
Teresa Giacomini
 
PDF
Must Know Postgres Extension for DBA and Developer during Migration
Mydbops
 
PDF
Beyond Postgres: Interesting Projects, Tools and forks
Sameer Kumar
 
PDF
Postgresql Up And Running Regina Obe Leo Hsu
zahidtraaslw
 
PDF
Postgres в основе вашего дата-центра, Bruce Momjian (EnterpriseDB)
Ontico
 
PDF
Useful PostgreSQL Extensions
EDB
 
PDF
Making Postgres Central in Your Data Center
EDB
 
PDF
Learning postgresql
DAVID RAUDALES
 
PDF
Building a Complex, Real-Time Data Management Application
Jonathan Katz
 
PDF
Making Postgres Central in Your Data Center
EDB
 
PDF
Making.postgres.central.2015
EDB
 
PDF
JDD 2016 - Tomasz Borek - DB for next project? Why, Postgres, of course
PROIDEA
 
PDF
Open Source SQL Databases
Emanuel Calvo
 
PPTX
Postgres for Digital Transformation: NoSQL Features, Replication, FDW & More
Ashnikbiz
 
PDF
PostgreSQL - Case Study
S.Shayan Daneshvar
 
PDF
PostgreSQL, Extensible to the Nth Degree: Functions, Languages, Types, Rules,...
Command Prompt., Inc
 
PPT
Postgres for the Future
EDB
 
KEY
Releasing PostgreSQL Extension on PGXN
David Wheeler
 
PPTX
Chjkkkkkkkkkkkkkkkkkjjjjjjjjjjjjjjjjjjjjjjjjjj01_The Basics.pptx
MhmdMk10
 
PDF
TechEvent 2019: Oracle to PostgreSQL - a Travel Guide from Practice; Roland S...
Trivadis
 
PostgreSQL Extension APIs are Changing the Face of Relational Databases | PGC...
Teresa Giacomini
 
Must Know Postgres Extension for DBA and Developer during Migration
Mydbops
 
Beyond Postgres: Interesting Projects, Tools and forks
Sameer Kumar
 
Postgresql Up And Running Regina Obe Leo Hsu
zahidtraaslw
 
Postgres в основе вашего дата-центра, Bruce Momjian (EnterpriseDB)
Ontico
 
Useful PostgreSQL Extensions
EDB
 
Making Postgres Central in Your Data Center
EDB
 
Learning postgresql
DAVID RAUDALES
 
Building a Complex, Real-Time Data Management Application
Jonathan Katz
 
Making Postgres Central in Your Data Center
EDB
 
Making.postgres.central.2015
EDB
 
JDD 2016 - Tomasz Borek - DB for next project? Why, Postgres, of course
PROIDEA
 
Open Source SQL Databases
Emanuel Calvo
 
Postgres for Digital Transformation: NoSQL Features, Replication, FDW & More
Ashnikbiz
 
PostgreSQL - Case Study
S.Shayan Daneshvar
 
PostgreSQL, Extensible to the Nth Degree: Functions, Languages, Types, Rules,...
Command Prompt., Inc
 
Postgres for the Future
EDB
 
Releasing PostgreSQL Extension on PGXN
David Wheeler
 
Chjkkkkkkkkkkkkkkkkkjjjjjjjjjjjjjjjjjjjjjjjjjj01_The Basics.pptx
MhmdMk10
 
TechEvent 2019: Oracle to PostgreSQL - a Travel Guide from Practice; Roland S...
Trivadis
 
Ad

More from Chartio (6)

PPTX
Rethinking Your Ad Spend: 5 Tips for intelligent digital advertising
Chartio
 
PPTX
How To Drive Exponential Growth Using Unconventional Data Sources
Chartio
 
PPTX
7 Reasons You Haven't Reached Hyper-Growth
Chartio
 
PPTX
Redshift Chartio Event Presentation
Chartio
 
PPTX
The Vital Metrics Every Sales Team Should Be Measuring
Chartio
 
PDF
CSV and XLS Best Practices
Chartio
 
Rethinking Your Ad Spend: 5 Tips for intelligent digital advertising
Chartio
 
How To Drive Exponential Growth Using Unconventional Data Sources
Chartio
 
7 Reasons You Haven't Reached Hyper-Growth
Chartio
 
Redshift Chartio Event Presentation
Chartio
 
The Vital Metrics Every Sales Team Should Be Measuring
Chartio
 
CSV and XLS Best Practices
Chartio
 

Recently uploaded (20)

PPTX
Online Work Permit System for Fast Permit Processing
SHEQ Network Limited
 
PPTX
Agentic AI : A Practical Guide. Undersating, Implementing and Scaling Autono...
IT IDOL Technologies
 
PDF
How to Choose the Right IT Partner for Your Business in Malaysia
Asian Business Services & Technologies
 
PPTX
CHAPTER 2 - PM Management and IT Context
KevinPutra14
 
PDF
Audit Checklist Design Aligning with ISO, IATF, and Industry Standards — Omne...
rohnjim07
 
PDF
SAP S4 Hana Brochure 3 (PTS SYSTEMS AND SOLUTIONS)
pts464036
 
PDF
T3DD25 TYPO3 Content Blocks - Deep Dive by André Kraus
Andre Kraus
 
PDF
Which alternative to Crystal Reports is best for small or large businesses.pdf
Varsha Nayak
 
PPTX
Lecture 3: Operating Systems Introduction to Computer Hardware Systems
hci7se
 
PDF
Navsoft: AI-Powered Business Solutions & Custom Software Development
Navsoft
 
PDF
Digital Strategies for Manufacturing Companies
Virendra Rai, PMP
 
PDF
Odoo Companies in India – Driving Business Transformation.pdf
azhuhardeen1968
 
PDF
Upgrade and Innovation Strategies for SAP ERP Customers
Virendra Rai, PMP
 
PDF
Flood Susceptibility Mapping Using Image-Based 2D-CNN Deep Learnin. Overview ...
lawrenceomai2
 
PDF
Raksha Bandhan Grocery Pricing Trends in India 2025.pdf
Actowiz Solustions
 
PPTX
VVF-Customer-Presentation2025-Ver1.9.pptx
blackmambaettijean
 
PPTX
Agentic AI Use Case- Contract Lifecycle Management (CLM).pptx
mhxyzzp
 
PPTX
Operating system designcfffgfgggggggvggggggggg
rmskumara09
 
PDF
Understanding Forklifts - TECH EHS Solution
TECH EHS Solution
 
PDF
Internet Downloader Manager (IDM) Crack 6.42 Build 42 Updates Latest 2025
itskinga12
 
Online Work Permit System for Fast Permit Processing
SHEQ Network Limited
 
Agentic AI : A Practical Guide. Undersating, Implementing and Scaling Autono...
IT IDOL Technologies
 
How to Choose the Right IT Partner for Your Business in Malaysia
Asian Business Services & Technologies
 
CHAPTER 2 - PM Management and IT Context
KevinPutra14
 
Audit Checklist Design Aligning with ISO, IATF, and Industry Standards — Omne...
rohnjim07
 
SAP S4 Hana Brochure 3 (PTS SYSTEMS AND SOLUTIONS)
pts464036
 
T3DD25 TYPO3 Content Blocks - Deep Dive by André Kraus
Andre Kraus
 
Which alternative to Crystal Reports is best for small or large businesses.pdf
Varsha Nayak
 
Lecture 3: Operating Systems Introduction to Computer Hardware Systems
hci7se
 
Navsoft: AI-Powered Business Solutions & Custom Software Development
Navsoft
 
Digital Strategies for Manufacturing Companies
Virendra Rai, PMP
 
Odoo Companies in India – Driving Business Transformation.pdf
azhuhardeen1968
 
Upgrade and Innovation Strategies for SAP ERP Customers
Virendra Rai, PMP
 
Flood Susceptibility Mapping Using Image-Based 2D-CNN Deep Learnin. Overview ...
lawrenceomai2
 
Raksha Bandhan Grocery Pricing Trends in India 2025.pdf
Actowiz Solustions
 
VVF-Customer-Presentation2025-Ver1.9.pptx
blackmambaettijean
 
Agentic AI Use Case- Contract Lifecycle Management (CLM).pptx
mhxyzzp
 
Operating system designcfffgfgggggggvggggggggg
rmskumara09
 
Understanding Forklifts - TECH EHS Solution
TECH EHS Solution
 
Internet Downloader Manager (IDM) Crack 6.42 Build 42 Updates Latest 2025
itskinga12
 

Using the PostgreSQL Extension Ecosystem for Advanced Analytics

  • 1. sales@chartio.com (855) 232-0320 sales@chartio.com (855) 232-0320 Using the PostgreSQL Extension Ecosystem for Advanced Analytics
  • 2. sales@chartio.com (855) 232-0320 - The problem - The prevailing view vs. the practical reality - A possible solution - Or just building blocks? - Nearness - Near at hand, near to our skill set, near to our capabilities - A more complete solution - The PostgreSQL extension ecosystem Agenda
  • 3. sales@chartio.com (855) 232-0320 sales@chartio.com (855) 232-0320 The Problem The Prevailing View vs. The Practical Reality
  • 4. sales@chartio.com (855) 232-0320 The Prevailing View - Logical Dimension Relational Non-Relational Schema objects ● Structured rows and columns ● Schema on write ● Referential integrity ● Painful migrations ● Unstructured files, docs, etc ● Schema on read ● No referential integrity ● No migrations Query languages ● SQL ● Declarative ● Easy enough for non-tech users ● Various ● Procedural ● Requires some programming skills Exploratory analysis ● Native support for joins ● Interactive/low execution overhead ● No native support for joins ● OLAP - Batch processing Data science and ML ● Only descriptive statistics ● Requires exporting dumps/samples ● Robust ecosystem ● Does not require exports
  • 5. sales@chartio.com (855) 232-0320 The Prevailing View - Physical Dimension Relational Non-Relational Parallel query processing ● Single node system ● Single process per query ● Multiple node system ● Multiple processes per query Concurrency ● High concurrency ● Single process per connection ● OLAP - low concurrency/high scheduling overhead High Availability & Replication ● Async and sync replication ● HA may not be native ● Async and sync replication ● HA likely to be native Sharding ● Sharding may not be native ● Difficult to manage ● Sharding likely to be native ● Easy to manage
  • 6. sales@chartio.com (855) 232-0320 The Prevailing View - Summary - RDBMS have nice properties for producing rich data - ACID, relational integrity, constraints, strong data types - Easier for non-tech users and exploratory analysis - Probably don’t meet the needs of today’s analysts - Data science & Machine Learning - Parallel processing - Definitely don’t meet the needs of today’s apps - Schema migrations - Replication and sharding
  • 8. sales@chartio.com (855) 232-0320 sales@chartio.com (855) 232-0320 But we still want more advanced functionality. The Practical Reality
  • 9. sales@chartio.com (855) 232-0320 sales@chartio.com (855) 232-0320 A Possible Solution Or Just Building Blocks?
  • 10. sales@chartio.com (855) 232-0320 Modern SQL - Many people still think of SQL in terms of SQL-92 - Since then we’ve had: SQL:1999, SQL:2003, SQL:2006, SQL:2008, SQL:2011 - http://use-the-index-luke.com/blog/2015-02/modern-sql - Common Table Expressions (CTEs) / Recursive CTEs - Window Functions - Ordered-set Aggregates - Lateral joins - Temporal support - The list goes on...
  • 11. sales@chartio.com (855) 232-0320 Procedural Languages - Native pgSQL Tcl Perl Python - Community Java PHP R Javascript Ruby Scheme sh
  • 12. sales@chartio.com (855) 232-0320 sales@chartio.com (855) 232-0320 These solve some problems. For others, they are just building blocks. Building Blocks
  • 13. sales@chartio.com (855) 232-0320 sales@chartio.com (855) 232-0320 Nearness Near at Hand Near to Our Skill Set Near to Our Capabilities
  • 15. sales@chartio.com (855) 232-0320 - Near at hand - Easily installable - Near to our skill set - Familiar tool/language/abstraction - Modular and composable - Near to our capabilities - Capable of solving a problem in our domain Nearness Drives Adoption
  • 16. sales@chartio.com (855) 232-0320 sales@chartio.com (855) 232-0320 A More Complete Solution The PostgreSQL Extension Ecosystem
  • 17. sales@chartio.com (855) 232-0320 Postgres Extension Ecosystem Examples - PostgreSQL Extension Network: http://pgxn.org/ - UDFs & operators: https://github.com/eulerto/pg_similarity - UDAs & data types: https://github.com/aggregateknowledge/postgresql-hll - Foreign Data Wrappers: http://multicorn.org/, https://github.com/shish/pgosquery - Indexes: https://github.com/zombodb/zombodb - Composing Extension Methods: http://doc.madlib.net/ - MPP: https://www.citusdata.com/, https://github.com/greenplum-db/gpdb - Composing Extensions - Custom Background Workers: https://github.com/no0p/alps - Record linking: http://no0p.github.io/2015/10/20/record_linking.html#/
  • 18. sales@chartio.com (855) 232-0320 Postgres Extension Ecosystem Examples - PostgreSQL Extension Network: http://pgxn.org/ - UDFs & operators: https://github.com/eulerto/pg_similarity - UDAs & data types: https://github.com/aggregateknowledge/postgresql-hll - Foreign Data Wrappers: http://multicorn.org/, https://github.com/shish/pgosquery - Indexes: https://github.com/zombodb/zombodb - Composing Extension Methods: http://doc.madlib.net/ - MPP: https://www.citusdata.com/, https://github.com/greenplum-db/gpdb - Composing Extensions - Custom Background Workers: https://github.com/no0p/alps - Record linking: http://no0p.github.io/2015/10/20/record_linking.html#/
  • 19. sales@chartio.com (855) 232-0320 - Package Manager: pgxn - Index/Network: http://pgxn.org/ - PyPI, RubyGems, CPAN, CRAN The PostgreSQL Extension Network
  • 20. sales@chartio.com (855) 232-0320 The PostgreSQL Extension Network - Near at hand - pgxn search semver - pgxn info semver - pgxn install semver - pgxn load –d somedb semver - pgxn unload –d somedb semver - pgxn uninstall semver - Search github? google? mailing list? - Github README? - git clone; make; make install; - psql –c “CREATE EXTENSION IF NOT EXISTS” - psql –c “DROP EXTENSION IF EXISTS” - make uninstall?
  • 21. sales@chartio.com (855) 232-0320 Postgres Extension Ecosystem Examples - PostgreSQL Extension Network: http://pgxn.org/ - UDFs & operators: https://github.com/eulerto/pg_similarity - UDAs & data types: https://github.com/aggregateknowledge/postgresql-hll - Foreign Data Wrappers: http://multicorn.org/, https://github.com/shish/pgosquery - Indexes: https://github.com/zombodb/zombodb - Composing Extension Methods: http://doc.madlib.net/ - MPP: https://www.citusdata.com/, https://github.com/greenplum-db/gpdb - Composing Extensions - Custom Background Workers: https://github.com/no0p/alps - Record linking: http://no0p.github.io/2015/10/20/record_linking.html#/
  • 22. sales@chartio.com (855) 232-0320 UDFs & Operators: pg_similarity - Near to our capabilities - Similarity coefficient algorithms - L1 Distance - Cosine Distance - Dice Coefficient - Euclidean Distance - Hamming Distance - Jaccard Coefficient - Jaro Distance - Jaro-Winkler Distance - Levenshtein Distance - Matching Coefficient - Monge-Elkan Coefficient - Needleman-Wunsch Coefficient - Overlap Coefficient - Q-Gram Distance - Smith-Waterman Coefficient - Smith-Waterman-Gotoh Coefficient - Soundex Distance
  • 23. sales@chartio.com (855) 232-0320 UDFs & Operators: pg_similarity - Near to our skill set
  • 24. sales@chartio.com (855) 232-0320 UDFs & Operators: pg_similarity - Implementation
  • 25. sales@chartio.com (855) 232-0320 Postgres Extension Ecosystem Examples - PostgreSQL Extension Network: http://pgxn.org/ - UDFs & Operators: https://github.com/eulerto/pg_similarity - UDAs & Data Types: https://github.com/aggregateknowledge/postgresql-hll - Foreign Data Wrappers: http://multicorn.org/, https://github.com/shish/pgosquery - Indexes: https://github.com/zombodb/zombodb - Composing Extension Methods: http://doc.madlib.net/ - MPP: https://www.citusdata.com/, https://github.com/greenplum-db/gpdb - Composing Extensions - Custom Background Workers: https://github.com/no0p/alps - Record linking: http://no0p.github.io/2015/10/20/record_linking.html#/
  • 26. sales@chartio.com (855) 232-0320 UDAs & Data Types: postgresql-hll - Near to our capabilities & near to our skill set - Data type - Estimate count distinct with tunable precision - 1280 bytes estimates tens of billions of distinct values with few percent error
  • 27. sales@chartio.com (855) 232-0320 UDAs & Data Types: postgresql-hll
  • 28. sales@chartio.com (855) 232-0320 UDAs & Data Types: postgresql-hll - Implementation
  • 29. sales@chartio.com (855) 232-0320 Postgres Extension Ecosystem Examples - PostgreSQL Extension Network: http://pgxn.org/ - UDFs & Operators: https://github.com/eulerto/pg_similarity - UDAs & Data Types: https://github.com/aggregateknowledge/postgresql-hll - Foreign Data Wrappers: http://multicorn.org/, https://github.com/shish/pgosquery - Indexes: https://github.com/zombodb/zombodb - Composing Extension Methods: http://doc.madlib.net/ - MPP: https://www.citusdata.com/, https://github.com/greenplum-db/gpdb - Composing Extensions - Custom Background Workers: https://github.com/no0p/alps - Record linking: http://no0p.github.io/2015/10/20/record_linking.html#/
  • 31. sales@chartio.com (855) 232-0320 Foreign Data Wrappers: multicorn - Near to our skill set
  • 32. sales@chartio.com (855) 232-0320 Foreign Data Wrappers: pgosquery - Near at hand
  • 33. sales@chartio.com (855) 232-0320 Postgres Extension Ecosystem Examples - PostgreSQL Extension Network: http://pgxn.org/ - UDFs & Operators: https://github.com/eulerto/pg_similarity - UDAs & Data Types: https://github.com/aggregateknowledge/postgresql-hll - Foreign Data Wrappers: http://multicorn.org/, https://github.com/shish/pgosquery - Indexes: https://github.com/zombodb/zombodb - Composing Extension Methods: http://doc.madlib.net/ - MPP: https://www.citusdata.com/, https://github.com/greenplum-db/gpdb - Composing Extensions - Custom Background Workers: https://github.com/no0p/alps - Record linking: http://no0p.github.io/2015/10/20/record_linking.html#/
  • 34. sales@chartio.com (855) 232-0320 Indexes: ZomboDB - Index Access Method API - http://www.postgresql.org/docs/9.4/static/indexam.html
  • 35. sales@chartio.com (855) 232-0320 Postgres Extension Ecosystem Examples - PostgreSQL Extension Network: http://pgxn.org/ - UDFs & Operators: https://github.com/eulerto/pg_similarity - UDAs & Data Types: https://github.com/aggregateknowledge/postgresql-hll - Foreign Data Wrappers: http://multicorn.org/, https://github.com/shish/pgosquery - Indexes (GiST, GIN): https://github.com/zombodb/zombodb - Composing Extension Methods: http://doc.madlib.net/ - MPP: https://www.citusdata.com/, https://github.com/greenplum-db/gpdb - Composing Extensions - Custom Background Workers: https://github.com/no0p/alps - Record linking: http://no0p.github.io/2015/10/20/record_linking.html#/
  • 36. sales@chartio.com (855) 232-0320 Composing Extension Methods: MADlib Near to our capabilities
  • 37. sales@chartio.com (855) 232-0320 Composing Extension Methods: MADlib - Near to our skill set
  • 39. sales@chartio.com (855) 232-0320 Postgres Extension Ecosystem Examples - PostgreSQL Extension Network: http://pgxn.org/ - UDFs & Operators: https://github.com/eulerto/pg_similarity - UDAs & Data Types: https://github.com/aggregateknowledge/postgresql-hll - Foreign Data Wrappers: http://multicorn.org/, https://github.com/shish/pgosquery - Indexes: https://github.com/zombodb/zombodb - Composing Extension Methods: http://doc.madlib.net/ - MPP: https://www.citusdata.com/, https://github.com/greenplum-db/gpdb - Composing Extensions - Custom Background Workers: https://github.com/no0p/alps - Record linking: http://no0p.github.io/2015/10/20/record_linking.html#/
  • 40. sales@chartio.com (855) 232-0320 Parallel Processing - Parallel sequential scan - http://rhaas.blogspot.com/2015/11/parallel-sequential-scan-is-committed.html - Columnar FDW: - https://github.com/citusdata/cstore_fdw
  • 41. sales@chartio.com (855) 232-0320 Postgres Extension Ecosystem Examples - PostgreSQL Extension Network: http://pgxn.org/ - UDFs & Operators: https://github.com/eulerto/pg_similarity - UDAs & Data Types: https://github.com/aggregateknowledge/postgresql-hll - Foreign Data Wrappers: http://multicorn.org/, https://github.com/shish/pgosquery - Indexes: https://github.com/zombodb/zombodb - Composing Extension Methods: http://doc.madlib.net/ - MPP: https://www.citusdata.com/, https://github.com/greenplum-db/gpdb - Composing Extensions - Custom Background Workers: https://github.com/no0p/alps - Record linking: http://no0p.github.io/2015/10/20/record_linking.html#/
  • 44. sales@chartio.com (855) 232-0320 Beyond Analytics - Web app framework - http://blog.aquameta.com/ - REST API - https://github.com/begriffs/postgrest - Unit testing framework - http://pgtap.org/ - Firewall - https://github.com/uptimejp/sql_firewall - More every week!
  • 45. sales@chartio.com (855) 232-0320 Conclusion - With PostgreSQL, you get - more than rows and columns - more than SELECT, FROM, WHERE, GROUP BY, ORDER BY - more than a single machine - Make sure you get the full return on your investment! Get your Chartio free trial! sales@chartio.com (855) 232-0320