A developer's introduction to big data processing with Azure Databricks
- 3. “More than any other factor, customer experiences determine whether companies thrive and profit, or struggle and fade.” – Forrester Research
- 4. Speed 79% won’t return to a slow website Personalization 38% won't call again if they have to repeat themselves Consistency 65% get frustrated with inconsistent device experiences
- 5. Harness the power of Big Data analytics apps to exceed customer needs
- 6. Enhance any type of app with Big Data analytics
- 8. Online shopping through company website E-commerce directly in the consumer’s hand, anywhere Wide World ImportersWide World Importers Retail stores
- 9. Wide World Importers Web e-commerce Mobile e-commerce Wide World Importers seeks to expand customers through an omni-channel strategy Retail stores
- 10. The solutions needed to reach more customers and grow the business 1. Scale with ease to reach more consumers 2. Unlock business insights from unstructured data 3. Enhance user experience with advanced analytics 4. Apply real-time analytics for instant updates 5. Infuse AI into apps to actively engage with customers
- 11. A Z U R E D A T A B R I C K S Microsoft Azure
- 12. Optimized Databricks Runtime Engine DATABRICKS I/O SERVERLESS Collaborative Workspace Cloud storage Data warehouses Hadoop storage IoT / streaming data Rest APIs Machine learning models BI tools Data exports Data warehouses Azure Databricks Enhance Productivity Deploy Production Jobs & Workflows APACHE SPARK MULTI-STAGE PIPELINES DATA ENGINEER JOB SCHEDULER NOTIFICATION & LOGS DATA SCIENTIST BUSINESS ANALYST Build on secure & trusted cloud Scale without limits A Z U R E D A T A B R I C K S
- 15. INGEST STORE PREP & TRAIN MODEL & SERVE Azure Blob Storage Logs, files and media (unstructured) Azure SQL Data Warehouse Azure Data Factory Azure Analysis Services Azure Databricks (Python, Scala, Spark SQL) Polybase Business/custom apps (Structured) Power BI Azure also supports other Big Data services like Azure HDInsight and Azure Data Lake to allow customers to tailor the above architecture to meet their unique needs.
- 16. INGEST STORE PREP & TRAIN MODEL & SERVE Azure Blob Storage Logs, files and media (unstructured) Azure SQL Data Warehouse Azure Data Factory Azure Analysis Services Polybase Business/custom apps (Structured) Power BI Azure also supports other Big Data services like Azure HDInsight and Azure Data Lake to allow customers to tailor the above architecture to meet their unique needs. Azure Databricks (Python, Scala, Spark SQL) Azure Databricks (Spark ML, Spark R, SparklyR) Intelligent Apps Cosmos DB
- 17. INGEST STORE PREP & TRAIN MODEL & SERVE Logs, files and media (unstructured) Sensors and IoT (unstructured) HDInsight (Kafka) Power BIAzure Databricks (Python, Scala, Spark SQL) Intelligent Apps Cosmos DBEvent Hub IoT Hub Azure Databricks (Spark ML, Spark R, SparklyR) Azure Blob Storage Batch Data (Apps, logs) Azure Data Factory
- 18. Azure Databricks at //BUILD 2018 Tuesday: • BRK3320 The Developer Data Scientist – Creating New Analytics Driven Applications using Apache Spark with Azure Databricks • WRK2601 Using Databricks to Analyze Telemetry Data Stored in Azure Blob Storage Wednesday: • BRK4102 ETL 2.0 - Data Engineering for developers • BRK3314 Leveraging Azure Databricks to minimize time to insight by combining Batch and Stream processing pipelines. • BRK3708 Machine learning at scale