AI at Scale
- 1. AI at Scale Adi Polak | Microsoft @adipolak
- 2. Start from the beginning • Massive datasets • Intelligent product @adipolak
- 3. • Apache Spark • Databricks for Production • Build machine learning models with Pipelines • Cognitive Services and Apache Spark • Data Engineer vs Data Scientist • MMLSpark Overview @adipolak
- 5. What is Apache Spark? @adipolak
- 8. Apache Spark submit job storage storage storage @adipolak
- 9. Framework libraries- Software Stracture @adipolak
- 10. Python, R & .NET for Apache Spark @adipolak
- 11. Embedded Service PySpark Spark Scala Process Pyspark Protocol Spark Worker @adipolak
- 12. Structured Streaming MLlib GraphFrame TensorFrames SQL SparkSession / DataFrame / Dataset APIs Data Source Connectors Catalyst Optimization & Tungsten Execution Spark Core (RDD APIs) SQL API More API @adipolak
- 13. Why Apache Spark is fast and efficient? CATALYST OPTIMIZER IN MEMORY COMPUTING TUNGSTEN @adipolak
- 15. Fundamentals of Catalyst Optimizer SUB Attribute(x) SUB some_func(1) some_func(2) Tree Rules SUB Attribute(x) some_func(-1) @adipolak
- 16. Spark SQL Execution Plan Logical optimization –> Optimization rules • Constant folding • Predicate pushdown • Projection pruning • … Catalyst Frontend Backend @adipolak
- 18. Spark Connector for SQL Server @adipolak
- 19. Spark connector for Cosmos DB @adipolak
- 21. How to run Apache Spark on Azure? HDInsight Azure Databricks AKS
- 22. Azure Databricks
- 23. What is Azure Databricks @adipolak
- 25. Build for Production • - Databricks limits and Azure limits • - Workspace isolation • - vnet • - Security and key vault • - Enable log analytics for monitoring
- 26. Databricks limits and Azure limits • Key Databricks limits ( per workspace): • - 150 concurrent jobs • - 150 max notebooks • - 1000 max hourly job submission Key Azure limits : - Storage accounts per region per subscription: 250 - VMs per subscription per region: 25,000 - Resource groups per subscription: 980 @adipolak
- 28. Cluster modes and their characteristics @adipolak
- 29. Batch vs. Interactive workloads @adipolak
- 30. Data Scientists and Analysts – Hight concurrency mode @adipolak
- 32. Troubleshooting with cluster logs • Azure Databricks provides three kinds of logging of cluster-related activity: • Cluster event logs, which capture cluster lifecycle events, like creation, termination, configuration edits, and so on. • Apache Spark driver and worker logs, which you can use for debugging. • Cluster init-script logs, valuable for debugging init scripts. @adipolak
- 33. Cluster logs
- 35. Log Analytics Workspace • Leverage Azure log analytics and connect it to the workspace • with init scripts and visualize it using Kibana or other tools. @adipolak
- 37. Gather Data ML Process / Basic Life Cycle Feature Extract, Clean and Normalize Select algorithm Evaluate model Data visualization 4 5 1 2 3 Repeat! @adipolak
- 38. H O W ? M L P i p e l i n e s ! @adipolak
- 40. Tools Spark Streaming Spark ML/Other Spark SQL MLflow @adipolak
- 41. @adipolak
- 42. Demo @adipolak
- 43. But in real life: Accuracy < 0.5 AUC L Results are far from being good
- 44. Building machine learning models from scratch is hard! @adipolak
- 45. We need a Data Science @adipolak
- 46. Our Data scientist needs tools @adipolak
- 47. Stand on the shoulder of giants @adipolak
- 49. Cognitive Services capabilities Infuse your apps, websites, and bots with human-like intelligence
- 50. A variety of real-world applications Vision Speech Intent: PlayCall Language Knowledge Search
- 52. Apache Spark + Cognitive Services @adipolak
- 53. @adipolak
- 54. Data Engineer vs Data Scientist Advanced programming Distributed systems Data pipeline Basic analytics Data Engineer Advance math/statistics ML/AI Advanced analytics Basic programming Data Science Big Data constrains Both should understand: @adipolak
- 55. MMLSpark: Microsoft Machine Learning for Apache Spark @adipolak
- 56. aka.ms/mlflow_and_azure_ml aka.ms/azure_databricks aka.ms/twitter_sentiment_analysis Want to learn more? @adipolak Thank you! Dank u! Merci!