Big Data and Oracle - 2013
0 likes53 views
The document discusses how organizations can leverage big data through Oracle's integrated big data solutions. It describes Oracle's offerings for acquiring and organizing big data from various sources using products like Oracle NoSQL Database and Hadoop. It then discusses how Oracle solutions allow users to analyze large datasets using R and visualize insights in BI dashboards. Finally, it provides an overview of Oracle's Exalytics and Big Data Appliance hardware and software platforms for processing and managing big data at scale.
Big Data and Oracle - 2013
- 1. Big Data – Are You Ready? Thomas Kyte http://asktom.oracle.com
- 2. Big Data Buzz “Keeping Afloat in a Sea of 'Big Data” ITBusinessEdge – 9/6/11 “Why big data is a big deal” InfoWorld – 9/1/11 “The challenge– and opportunity– of big data” McKinsey Quarterly—5/11 “Getting a Handle on Big Data with Hadoop” Businessweek-9/7/11 “Ten reasons why Big Data will change the travel industry” Tnooz -8/15/11 “The promise of Big Data” Intelligent Utility-8/28/11
- 3. Big Data Use Cases Today’s Challenge New Data What’s Possible Healthcare Expensive office visits Remote patient monitoring Preventive care, reduced hospitalization Manufacturing In-person support Product sensors Automated diagnosis, support Location-Based Services Based on home zip code Real time location data Geo-advertising, traffic, local search Public Sector Standardized services Citizen surveys Tailored services, cost reductions Retail One size fits all marketing Social media Sentiment analysis segmentation
- 4. What Makes it Big Data? VOLUME VELOCITY VARIETY VALUE SOCIAL BLOG SMART METER 101100101001 001001101010 101011100101 010100100101
- 5. Make Better Decisions Using Big Data Big Data in Action ANALYZE DECIDE ACQUIRE ORGANIZE
- 6. Acquire all available data Big Data in Action ANALYZE DECIDE ORGANIZE ACQUIRE
- 7. Acquiring Big Data Challenge Application will need tochange frequently Need toprocess high volume, low- density information Must scale out to meet aggressive roll out plan
- 8. Oracle NoSQL Database Key value pair database Dynamic data model Highly scalable, available Transparent load balancing Built using BerkeleyDB Nodes East Nodes West Nodes Central Nodes NoSQL Driver Application NoSQL Driver Application … Nodes … Read Delete Read Update
- 9. Oracle NoSQL Database Key value pair database Dynamic data model Highly scalable, available Transparent load balancing Built using BerkeleyDB
- 10. Oracle NoSQL Database Key value pair database Dynamic data model Highly scalable, available Transparent load balancing Built using BerkeleyDB Oracle NoSQL: Practically ACID … The serious part of Oracle NoSQL is a practical approximation of ACID compliance, the standard that SQL databases like to offer. ACID means "Atomic, Consistent, Isolated, Durable transactions," and there's a robust debate about just what this translates to in excruciating detail. Most NoSQL systems promise a different acronym, BASE, which stands for "Basically Available, Soft State, and Eventually Consistent." In other words, you'll probably get the right answer except when you don't.
- 11. Oracle NoSQL Database Key value pair database Dynamic data model Highly scalable, available Transparent load balancing Built using BerkeleyDB In all, Oracle NoSQL was a pleasure to try because it offered so many serious features developed by a company with a deep history of serious data management. There are dozens of small ways in which the tool is more thorough and sophisticated than the simpler NoSQL projects. You get a number of different options for increasing the durability in the face of a node crash or trading that durability for speed. The documentation is solid and written by working engineers with deep experience in storing data for enterprise customers.
- 12. Big Data in Action ANALYZE DECIDE ORGANIZE ACQUIRE Oracle NoSQL Database
- 13. Organize and distill big data using massive parallelism Big Data in Action ANALYZE DECIDE ACQUIRE ORGANIZE
- 14. Organizing Big Data Challenge Also want to perform analysis on big data Have existing Oracle data warehouse Can’t negatively impact data warehouse SLAs
- 15. Analysis Sandbox Provides analysis workspace Controlled access to resources and data Doesn’t impact production system
- 16. Sandboxing with Oracle Enterprise Manager Simple to set up Efficient server utilization Secure and scalable Accountable via charge back Ideal for Oracle Exadata
- 17. Big Data in Action ANALYZE DECIDE ACQUIRE ORGANIZE Oracle NoSQL Database Oracle Enterprise Manager
- 18. Organizing and Distilling Big Data Challenge Want toavoid writing lots of Hadoop code Must transform big data into something easily analyzed Need toload data quickly into Oracle Data Warehouse
- 19. Hadoop Architecture Management/Monitoring Hadoop Distributed File System (HDFS) MapReduce Distributed file system with redundant storage Map/Reduce programming paradigm Highly scalable data processing Cost-effective model for high volume, low density data
- 21. Oracle Data Integrator Reduces Hadoop complexities through graphical tooling
- 22. Oracle Loader for Hadoop SHUFFLE /SORT SHUFFLE /SORT MAP MAP MAP MAP SHUFFLE /SORT REDUCE REDUCE SHUFFLE /SORT SHUFFLE /SORT REDUCE REDUCE REDUCE INPUT 2 INPUT 1 MAP MAP MAP MAP MAP REDUCE REDUCE REDUCE MAP MAP MAP MAP MAP MAP REDUCE REDUCE MAP MAP MAP MAP MAP REDUCE REDUCE REDUCE
- 23. Big Data in Action ANALYZE DECIDE ACQUIRE ORGANIZE Oracle NoSQL Database Oracle Enterprise Manager Oracle Data Integrator Oracle Loader for Hadoop
- 24. Analyze all your data, at once Big Data in Action ANALYZE DECIDE ACQUIRE ORGANIZEANALYZE
- 25. Analyzing Big Data Challenge Want toperform statistical analysis using R Require access to all data Doing analysis on a laptop is slow and not secure
- 26. R Statistical Programming Language Open source language and environment Used for statistical computing and graphics Strength in easily producing publication-quality graphs Highly extensible
- 27. Small data models only are stored and run on user’s laptop Why R Wasn’t Ready for the Enterprise
- 28. Oracle R Enterprise Approach Models run in-database Processes large data sets Uses the power of Oracle Database 11g and Exadata Same code, much faster
- 29. Big Data in Action ANALYZE DECIDE ACQUIRE ORGANIZEANALYZE Oracle NoSQL Database Oracle Enterprise Manager Oracle Data Integrator Oracle Loader for Hadoop Oracle R Enterprise
- 30. Decide based on real-time big data Big Data in Action ANALYZE ACQUIRE ORGANIZE DECIDE
- 31. Making Decisions Based on Big Data Challenge Want toadd new insights into BI dashboard Big data has been transformed into actionable insight How do we quickly integrate R analytics into dashboard?
- 32. Dashboard Analytics •Oracle Business Intelligence Enterprise Edition ‒Advanced dashboard visualization ‒Runs BI and EPM applications •Integrating R Analytics ‒Embed R script’s web interface in BI dashboard ‒Graphics will stream to BI dashboard
- 33. Oracle Integrated Solution Stack for Big Data ACQUIRE Oracle NoSQL Database HDFS Enterprise Applications ORGANIZE Hadoop (MapReduce) Oracle Loader for Hadoop Oracle Data Integrator DECIDE Analytic Applications ANALYZE In-Database Analytics Data Warehouse
- 34. Oracle Exalytics Hardware Engineered for extreme analytics •40 Intel processor cores •1 Terabyte main memory •40 Gb InfiniBand connection to Oracle Exadata
- 35. Oracle Exalytics Software •Oracle TimesTen In-Memory Database ‒Adaptive in-memory caching of analytics ‒In-memory columnar compression ‒Tightly integrated with Oracle Exadata ‒Enables speed-of-thought visualization •Oracle Business Intelligence Foundation Suite
- 36. •18 Sun X4270 M2 Servers –48 GB memory per node = 864 GB memory –12 Intel cores per node = 216 cores –24 TB storage per node = 432 TB storage •40 Gb p/sec InfiniBand •10 Gb p/sec Ethernet Oracle Big Data Appliance Hardware
- 37. •Oracle Linux •Java Hotspot VM •Apache Hadoop Distribution •R Distribution •Oracle NoSQL Database •Oracle Data Integrator for Hadoop •Oracle Loader for Hadoop Oracle Big Data Appliance Software