Watson data platform_sofia_20171017
- 1. Watson Data Platform / September, 2017 / © 2017 IBM Corporation IBM Watson Data Platform Platform for Data Science Mladen Jovanovski Client Technical Specialist IBM Analytics & Hybrid Cloud, SEE October 2017
- 2. Disclaimer Watson Data Platform / July, 2017 / © 2017 IBM Corporation © Copyright IBM Corporation 2017. All rights reserved. U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL PURPOSES ONLY. WHILE EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE INFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED “AS IS” WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. IN ADDITION, THIS INFORMATION IS BASED ON IBM'S CURRENT PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO CHANGE BY IBM WITHOUT NOTICE. IBM SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION. NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, NOR SHALL HAVE THE EFFECT OF, CREATING ANY WARRANTIES OR REPRESENTATIONS FROM IBM (OR ITS SUPPLIERS OR LICENSORS), OR ALTERING THE TERMS AND CONDITIONS OF ANY AGREEMENT OR LICENSE GOVERNING THE USE OF IBM PRODUCTS AND/OR SOFTWARE. IBM's statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM's sole discretion. Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision. The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract. The development, release, and timing of any future features or functionality described for our products remains at our sole discretion. IBM, the IBM logo, ibm.com, Information Management, DB2, DB2 Connect, DB2 OLAP Server, pureScale, System Z, Cognos, solidDB, Informix, Optim, InfoSphere, and z/OS are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked on their frst occurrence in this information with a trademark symbol (® or ™), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at “Copyright and trademark information” at www.ibm.com/legal/copytrade.shtml Other company, product, or service names may be trademarks or service marks of others.
- 3. The World We Are Coming From… Watson Data Platform / July, 2017 / © 2017 IBM Corporation
- 4. • Information availability is unprecedented yet accessibility remains elusive • Applying strategy and decision-making is complicated • Skilled analysts are overwhelmed with growing workloads Data and The New Business Imperative 1 Source: Blue Hill Research 2017 2 Source: Forrester’s Business Technographics® Global Data And Analytics Survey, 2017 75-80% of workers typically lack access to potentially helpful data or analytics1 More People More Data An average organization only uses 50% of its structured and 25% of its unstructured data for decision making2
- 5. Why is Data Science Essential? Two main reasons… A single truck visiting 10 different locations Over 3 million routes Real world problems contain 100s of trucks and 1000s of locations There are more possibilities that the grains of sands in the world A single truck visiting 5 different locations 120 different routes 1) Complexity1) Complexity 2) Data2) Data 1995 – 130 billion GB 2020 – 40 trillion GB We cannot plan for every eventuality, we cannot write code for every scenario We cannot say “if this then do that” 40 trillion times Data Science is made up of algorithms and approaches that will fnd the right solution every time – even if they haven’t encountered that exact situation before!
- 6. “The Sexiest Job in the 21st century” – Harvard Business Review “By 2018 the US could face a shortage of 150,000 to 190,000 with deep analytics skills as well as 1.5 million managers with the know-how to use analysis of big data to make effective decisions” – McKinsey Global Institute “… the demand for Data Scientists outstripped supply by 250,000 in 2015 …” – Accenture “… business leaders of whom more than 50% reported lack of in-house expertise in data science” – Gartner Market Trends Markets and Markets: Data Science Platform Market, Global Forecast – Feb, 2017 AlotofdemandNotenoughsupply
- 7. Why do companies struggle to deliver value? Data • Data resides in silos • Detailed data was never stored • Unstructured and external data wasn’t considered • Diffcult to access • If the data isn’t secure, self-service isn’t a reality • Understanding lineage and getting to a system of truth • Data Science skills are in low supply and high demand • Nurturing new data professionals is challenging • Need an environment that enables a “fail fast” approach • Discrete tools present barriers to progress Governance InfrastructureSkills
- 8. …And Where We Are Going Watson Data Platform / July, 2017 / © 2017 IBM Corporation Forrester Wave 2017 Insight Platforms-As-A-Service
- 9. Data Science is a Team Sport Statistician Computer Scientist Data Mining Professional Operations Research Professional Citizen Analyst & Business Analyst Data Engineer Application Developer
- 10. Rigid toolset – Have to choose one and only one approach – Cannot easily connect all of the capabilities required – Diffcult to navigate between the various tools used Challenges that Data Scientists face…. Analytical Silo – Diffcult to maintain and version control project assets – Limited means of collaborating with teams – Results are diffcult to share Fragmented and time consuming – Using multiple disjoint environments – Separate on-ramp/community for each tool/environment – Does not have meta data or data lineage
- 11. The IBM Watson Data Platform provides an interactive, collaborative, cloud-based environment with multiple services and tools to activate insights for data scientists. Remove silos. Collaborate between teams and across technologies, from data scientists and Data Science Experience, to developers and Bluemix. Discover new insights. Use multiple analytics technologies, like Watson’s cognitive abilities, to quickly gain insights from data and answer business questions — faster than any other platform. Build smarter apps. Deliver insights to production quickly, and continuously improve them through rapid iteration. So you can build smarter, scalable applications in less time.
- 12. Watson Data Platform an integrated platform of tools, services and data that helps companies accelerate their shift to become data-driven organizations Access powerful tools to prepare data to tease out the insights they’re looking for, without IT involvement Data Scientist Answer the questions that the organization needs quickly and easily, and without getting IT involved Business Analyst App Developer Make the insights immediately actionable and add intelligence to apps in straightforward manner Data Engineer Easily build data pipelines that power dashboards and data platforms while ensuring high quality
- 13. Persist Analyze Ingest Deploy Projects | Data | Assets | Pipelines | APIs Intelligent governance | Policy enforcement Making data into insights is a team sport Watson Data Platform Our Core Tenets 1. Intelligent by Design 2. Collaborative for data Professionals 3. Self-service access to trusted data 4. Best in class streaming and real-time analytics 5. Open and Extensible Collaborate Data steward Data scientistData engineer Developer Find Share
- 14. Democratized data and analytics Watson Data Platform to organize, fnd and understand data that enables every project team to quickly discover and use data they can trust including open source notebooks and tools, shapers and data visualizations that enable the team to quickly investigate any volume of data enable team collaboration ensuring the enterprise builds upon the insights its members develop and springboards from those that peer organizations and experts share. Projects and communities Common tools Catalog enabling regulatory compliance with automatic policy enforcement that enables data professionals to focus on delivering valuable insights rather than managing and organizing data Enterprise grade governance to disparate data sources that makes it easy to connect to and gather data from anywhere without friction Data access
- 15. Data at your fingertips In the IBM Cloud Where your data is Watson Data Platform
- 16. Data Catalog Watson Data Platform Discover Intelligent discovery of data, advanced classifcation and profling to provide context Govern Powerful governance policy tools to control and protect access to data with visibility to data use Catalog A rich metadata index of all data, with social collaboration and enhanced fndability Unlock tribal knowledge to unleash your data professionals
- 17. Data Science Experience Watson Data Platform Learn Built-in learning to get started or go the distance with advanced tutorials Collaborate Community and social features that provide meaningful collaboration Create The best of open source and IBM value-add to create state-of-the-art data products Making data science a team sport
- 18. Data Refinery Watson Data Platform Wrangle Interactively explore, resolve quality issues, enrich, classify, standardize, summarize and join data Adapt Connect to 30+ cloud and on-premises stores and scale on demand with cataloging and governance Flow Create data flows visually, schedule for repeatability, monitor and notify A Breakthrough Approach to Explore and Prepare Data
- 19. Watson Data Platform Service Offerings (as of November 2017)
- 20. How does WDP help fulfill the promise of your data? Watson Data Platform Data Puts every important data source at the fngertips of the teams that need it wherever resides Enforces your policies without getting in the way of delivering insights Makes the most of the data professionals you have and helps them grow and learn from each other as a team Delivers the foundation for your frst data project through to the complete transformation of your business Governance InfrastructureSkills
- 21. Watson Data Platform / September, 2017 / © 2017 IBM Corporation