Databases and graph analysis - back to the future?
0 likes621 views
The document discusses the evolution of databases, particularly the rise of NoSQL and NewSQL solutions in response to the limitations of traditional relational databases. It highlights the increasing importance of graph analysis for data investigation and the necessity of user-friendly technology in enterprise settings. The author, Krystian Piećko, shares insights on how companies like Facebook manage vast amounts of data effectively and the challenges in adapting to newer database technologies.
Databases and graph analysis - back to the future?
- 1. Databases and graph analysis back to the future? Krystian Piećko CTO & Co-founder PiLab S.A. 2015-09-30
- 3. Few words about PiLab Mother of God… 3
- 5. Theory vs. Practise How enterprises are using data 5
- 6. Two data modellers in the same room Better fight than Pride with Nastula 6
- 7. Never forget about it 7
- 8. NoSQL
- 9. NoSQL rises 2009 • SQL is hard • RDBMS can’t fetch data so fast • Schemas are for grandpas • Who needs SQL - we have got Java and Python • Analytics is to slow on RDBMS • Unstructured is growing • other? 9
- 10. It is very simple to dim the lights in server room 10
- 11. Power to the user Limited access is the problem 11
- 13. NoSQL There is less java/python/whatever programmers than people that know how to use SQL 13
- 14. NoSQL in practice Do you all know this? 14
- 16. NoSQL with SQL interface Many of the new ones are missing here 16
- 17. After few years Some people tried to change the meaning to Not Only Short learning cycle :) 17
- 18. NewSQL
- 19. And it has begun The shift showed the “old school” db guys, that there is a chance on the marketplace 19
- 20. Inefficient query that runs agains NewSQL is still inefficient but it runs faster :) 20
- 21. Truth ;) 21
- 22. The largest RDBMS implementation I know 22
- 23. Facebook facts 2014 • 60k+ servers • MySQL database cluster • Memcache • 2.4+ billion pieces of content and 750TB+ of data every day • 35TB+/h daily ingest to HPVertica with SLA on that • 300+ identical nodes with 10Gigs/s connectivity 23
- 24. Probably true The cost vs. value 24
- 25. Only some industries need it all need to think about it 25
- 26. Database size of 90% of major vendor DB usage is less than 900GB 26
- 27. Graph analysis
- 28. Graph is how we think 28
- 29. 6 degrees of separation 29
- 30. 6 degrees of separation 30
- 32. Graphs are growing Entity relationship model is the graph 32
- 33. Investigations
- 34. Techniques • There are no official data about all vendors and the software just rumours • Software that is being used was i2, Palantir, other custom made • Huge amount of work on unstructured data • 7 ways to write Al-Qaeda • How to investigate tip? • Manually tagging unstructured data • Social media cross join 34
- 35. Being able to connect the dots Sharing the investigation information was 35
- 36. Field investigation was the clue 36
- 37. Clue to the case was the visualisation and ad-hoc querying by not technical users #SELECT * FROM users WHERE clue > 0 37
- 38. Banks
- 39. ~40th Bank in US has the size of the largest Polish Bank 39
- 40. IT companies in SV 40
- 41. Almost every part of the banking industry market has been taken Only few spots left 41
- 42. Minimum 10 different softwares involved Regulatory dominant issues Alerting system Investigation systemCase management system Twist & Tune // // 42
- 43. About PiLab
- 44. Technology 44
- 45. Technology is not a product 45
- 46. Product is something that solves the customer specified need and customer wants to pay for it 46
- 49. User with the data 49
- 50. Demo? 50
- 51. Thank you for your attention Krystian Piećko krystian.piecko@pilab.pl
- 52. Wykorzystane strony • http://database.ca/databasehumorhumourjokes.aspx • https://smist08.files.wordpress.com/2012/01/nosql.png • https://plus.google.com/+YaiEakwattana/posts/Lda6xNR7LV9 • http://image.slidesharecdn.com/sqlonhadoop-150617075749-lva1-app6892/95/sql-on-hadoop-6-638.jpg? cb=1435155882 • http://vertica.tips/2014/08/27/how-vertica-met-facebooks-35tb-per-hour-ingest-sla/ • http://www.quickmeme.com/p/3vxe25 • http://image.slidesharecdn.com/thefivegraphsoffinance-graphconnectnyc-131108130053-phpapp02/95/the- five-graphs-of-finance-philip-rathle-and-emil-eifrem-graphconnect-ny-2013-30-638.jpg?cb=1383915667 • http://www.koreatimes.co.kr/upload/news/100108_p06_cartoon.jpg • http://cnt13.blogspot.com/2013/02/terrorist-networks-look-into-911-tragedy.html • http://belfastgroup.digitalscholarship.emory.edu/static/img/2013-dh-graph.png • http://www.datamodel.com/index.php/tag/database-design/ • http://www.koreatimes.co.kr/upload/news/100108_p06_cartoon.jpg • http://www.frugaldaddy.co.uk/images/splash-guard.jpg • http://img06.deviantart.net/bf9b/i/2013/301/4/1/funny_chuck_norris_converse_shoe__by_rickfrost-d6s4kqz.jpg • http://www.weevermedia.com/wp-content/uploads/2015/08/fraud-mobile-app-marketing-300x241.jpg • http://www.b-eye-network.com/blogs/rogers/thanksgiving.jpg • http://www.neatorama.com 52