ClickHouse Analytical DBMS. Introduction and usage, by Alexander Zaitsev
3 likes1,478 views
The document outlines an agenda for a ClickHouse event, including presentations on ClickHouse features and usage. ClickHouse is introduced as a fast, flexible, and free open source columnar database management system. It is described as being very fast for analytical queries, with the ability to ingest and aggregate data at low latency and perform distributed computations and fault-tolerant data warehousing at large scale. Successful production deployments are listed in domains like analytics, advertising, operations, and security. The growth and adoption of ClickHouse is discussed.
ClickHouse Analytical DBMS. Introduction and usage, by Alexander Zaitsev
- 2. Agenda: 18:00 ClickHouse Analytical DBMS. Introduction and usage (Alexander Zaitsev, Altinity) 18:30 ClickHouse New Features Overview and Development Roadmap (Alexey Milovidov, Yandex) 18:45 ClickHouse in Plug and Play Cloud (Som Sikdar, Kodiak Data) 20:00 Lightning talks, ClickHouse demo, Q&A
- 3. ClickHouse Analytical DBMS Introduction and Usage Alexander Zaitsev, Altinity Plug And Play Tech Center, Sunnyvale, 23 Apr 2018
- 5. ClickHouse DBMS is •Column Store •MPP •SQL •Open Source
- 6. Why Yet Another DBMS?
- 9. How Fast?
- 10. :) select count(*) from dw.T SELECT count(*) FROM dw.T ┌───────count()─┐ │ 1185063669477 │ └───────────────┘ 1 rows in set. Elapsed: 4.361 sec. Processed 1.19 trillion rows, 1.19 TB (271.73 billion rows/s., 271.73 GB/s.)
- 11. Query 1 Query 2 Query 3 Query 4 Setup 0.034 0.061 0.178 0.498 MapD & 2-node p2.8xlarge cluster 0.051 0.146 0.047 0.794 kdb+/q & 4 Intel Xeon Phi 7210 CPUs 0.762 2.472 4.131 6.041 BrytlytDB 1.0 & 2-node p2.16xlarge cluster 1.034 3.058 5.354 12.748 ClickHouse, Intel Core i5 4670K 1.56 1.25 2.25 2.97 Redshift, 6-node ds2.8xlarge cluster 2 2 1 3 BigQuery 6.41 6.19 6.09 6.63 Amazon Athena 8.1 18.18 n/a n/a Elasticsearch (heavily tuned) 14.389 32.148 33.448 67.312 Vertica, Intel Core i5 4670K 22 25 27 65 Spark 2.3.0 & single i3.8xlarge w/ HDFS 35 39 64 81 Presto, 5-node m3.xlarge cluster w/ HDFS 152 175 235 368 PostgreSQL 9.5 & cstore_fdw “1.1 Billion Taxi Rides Benchmarks” http://tech.marksblogg.com/benchmarks.html
- 12. Query 1 Query 2 Query 3 Query 4 Setup 0.034 0.061 0.178 0.498 MapD & 2-node p2.8xlarge cluster 0.051 0.146 0.047 0.794 kdb+/q & 4 Intel Xeon Phi 7210 CPUs - 2.415 3.599 4.962 ClickHouse at Plug And Play demo server 0.762 2.472 4.131 6.041 BrytlytDB 1.0 & 2-node p2.16xlarge cluster 1.034 3.058 5.354 12.748 ClickHouse, Intel Core i5 4670K 1.56 1.25 2.25 2.97 Redshift, 6-node ds2.8xlarge cluster 2 2 1 3 BigQuery 6.41 6.19 6.09 6.63 Amazon Athena 8.1 18.18 n/a n/a Elasticsearch (heavily tuned) 14.389 32.148 33.448 67.312 Vertica, Intel Core i5 4670K 22 25 27 65 Spark 2.3.0 & single i3.8xlarge w/ HDFS 35 39 64 81 Presto, 5-node m3.xlarge cluster w/ HDFS 152 175 235 368 PostgreSQL 9.5 & cstore_fdw “1.1 Billion Taxi Rides Benchmarks” http://tech.marksblogg.com/benchmarks.html
- 13. ClickHouse runs at • Bare metal (any Linux) • Amazon • Azure • Kubernets, VM Ware etc. • Kodiak Data cloud
- 15. What Is It For?
- 16. What Is It For? • Fast analytical queries • Low latent data ingestion/aggregation • Distributed computations • Fault-tolerant data warehousing All scaled from 1 to 1000s servers
- 17. Who Is It For?
- 18. Who Is It For? • Analysts • who need analyze huge amounts of data • Startups • build high performance analytics with low investment • Companies • having performance problems with their analytics • paying too much for license or infrastructure
- 19. Successful Production Deploymnts • DNS queries analytics (CloudFlare) • AdTech (multiple companies worldwide) • Operational logs analytics (multiple companies worldwide) • Stock correlation analytics, investor tools (Canadian company) • Hotel booking analytics SaaS (Spanish company) • Security audit (Great Britain, USA) • Fintech SaaS (France) • Mobile App and Web analytics (multiple companies worldwide)
- 20. Evaluating/implementing: • Telecom companies • Satellite data processing • Search ranking analytics • Blockchain platform/ICO analysis • Manufactoring process control
- 21. Happy Transitions! • From MySQL/InfoBright/PostreSQL/Sp ark to ClickHouse • From Vertica/RedShift to ClickHouse SPEED! COST! VENDOR UN-LOCKING! 23.04 18:30 CLICKHOUSE GATE 2 boarding 23.04 19:30 CLICKHOUSE GATE 3 23.04 20:00 CLICKHOUSE GATE 4
- 22. ClickHouse Timeline • Developed in Yandex in 2012-2015 • Open Sourced June 2016 • First non-Yandex deployments Q4 2016 • 100+ deployments by Q4 2017 • 15+ meetups, 10+ conferences in 2017-Q12018
- 23. ClickHouse Today • Mature Analytic DBMS. Proven by many • 2- years in Open Source • Transparent development roadmap • 20+ community contributors • Emerging eco-system (tools, drivers, integrations) • Support and Consulting from Altinity
- 24. LONDON, April 23, 2018 /PRNewswire/ -- Altinity, a leading service provider for ClickHouse, today announced explosive 10x quarter-over-quarter revenue growth in Q1 2018 for its ClickHouse services. The spike in demand for ClickHouse Support mirrors the trend of companies increasingly relying on ClickHouse as a solution for their data analytics and data warehousing needs. https://www.prnewswire.com/news-releases/clickhouse-experiencing- explosive-growth-in-eu-us-and-asia-in-q1-2018-300633698.html
- 25. Q&A Contact me: alexander.zaitsev@lifestreet.com alz@altinity.com skype: alex.zaitsev telegram: @alexanderzaitsev Join SF Bay Area Meetup group: http://meetup.com/San-Francisco-Bay-Area-ClickHouse-Meetup