Building an End-to-End Solution in Microsoft Fabric: From Dataverse to Power BI (SQLSaturday Oregon 2023)
1 like372 views
The document outlines a session on building an end-to-end data solution using Microsoft Fabric, integrating tools like Azure Data Factory, Azure Synapse Analytics, and Power BI. Attendees will learn to manage data from Dataverse, transform it into a lakehouse architecture, visualize it, and use the new Data Activator tool to drive actions automatically. The session emphasizes hands-on experience with Microsoft Fabric to enhance analytics workflows.
Building an End-to-End Solution in Microsoft Fabric: From Dataverse to Power BI (SQLSaturday Oregon 2023)
- 1. Building an End-to-End Solution in Microsoft Fabric: From Dataverse to Power BI Cathrine Wilhelmsen, Emilie Rønning, Marthe Moengen
- 2. Thank You Sponsors! Our sponsors make this event possible. Please thank and support them.
- 3. Microsoft Fabric is the newest and coolest kid in town and for good reasons. It brings together the best features from Azure Data Factory, Azure Synapse Analytics and Power BI, making it easier than ever to build analytical solutions. Join us as we walk through how to build an end-to-end lakehouse solution in just one hour! In this session, we will show you how to use Synapse Link to get near real-time data from Dataverse and leverage the power of Notebooks to prepare and optimize the data for the lakehouse medallion architecture. Then, we will use that data to build stunning visualizations in Power BI that provide valuable insights. Finally, we will introduce the new Data Activator tool, an innovative component of Microsoft Fabric that empowers business analysts to drive actions based on data automatically. Don't miss this opportunity to experience Microsoft Fabric's game-changing potential firsthand and learn how it can revolutionize your analytics workflow. Building an End-to-End Solution in Microsoft Fabric: From Dataverse to Power BI
- 4. Cathrine Wilhelmsen Emilie Rønning Solutions Architect and Tech Lead Evidi @cathrinew • The OG of anything in the Norwegian data community • Fabric February Azure, Fabric, Azure Data Factory, Azure Synapse Analytics Senior Data Engineer QUARKS @EmilieRonning • Data Saturday Oslo • MDPUG Norway • Fabric February Azure, Databricks, Azure Data Factory, Architecture Managing Data Analyst Sopra Steria @mmoengen • Data Saturday Oslo • MDPUG Norway • WITs Who Lunch • Fabric February Azure, Architecture, Databricks, Governance, MDM, Fabric, Power BI Marthe Moengen
- 5. Our client has a website where hosts can list their homes for short-term or long-term stays. What does our client need? They need to be able to: • Manage Hosts • Gain insight into stays Let’s call them «AirBnB»
- 6. Common end-to-end scenario: 1. Ingest 2. Transform 3. Visualize 4. Take Action What will we build today?
- 7. Common end-to-end scenario: 1. Ingest 2. Transform 3. Visualize 4. Take Action How will we build it today?
- 8. Fabric Overview Data Factory Data Engineering Data Warehouse Data Science Real-Time Analytics Power BI Data Activator Unified Data Foundation with OneLake Data Governance with Microsoft Purview
- 9. Fabric Components Data Engineering: Store big data in lakehouses for querying, reporting and sharing. Data Warehousing: Store relational data for querying, reporting and sharing. Data Science: Use machine learning to detect trends, identify outliers and predict values. Real-Time Analytics: Rapidly capture and load streaming data for querying. Data Factory: Ingest, clean, transform and prepare data. Power BI: Create reports and dashboards to gain insights. Data Activator: Create reports and dashboards to gain insights.
- 10. Our Solution 1. Add / edit data in Power App 2. Store data in Dataverse 3. Replicate data to OneLake using Synapse Link End-to-End Dataverse OneLake Lakehouse Power BI Data Activator Power Apps 4. Transform data in Lakehouse 5. Visualize data in Power BI 6. Take action on data using Data Activator
- 12. Our Solution 1. Add / edit data in Power App 2. Store data in Dataverse 3. Replicate data to OneLake using Synapse Link End-to-End Dataverse OneLake Lakehouse Power BI Data Activator Power Apps 4. Transform data in Lakehouse 5. Visualize data in Power BI 6. Take action on data using Data Activator Cathrine
- 14. Demo: Power App
- 15. Demo: Dataverse Link to Fabric
- 17. Our Solution 1. Add / edit data in Power App 2. Store data in Dataverse 3. Replicate data to OneLake using Synapse Link End-to-End Dataverse OneLake Lakehouse Power BI Data Activator Power Apps 4. Transform data in Lakehouse 5. Visualize data in Power BI 6. Take action on data using Data Activator Emilie
- 18. Our Solution 1. Add / edit data in Power App 2. Store data in Dataverse 3. Replicate data to OneLake using Synapse Link End-to-End Dataverse OneLake Lakehouse Power BI Data Activator Power Apps 4. Transform data in Lakehouse 5. Visualize data in Power BI 6. Take action on data using Data Activator Emilie
- 19. Transforming Data Structured, Semi structured and unstructured data Data turned into actionable insights MAGIC
- 20. Transforming Data Microsoft Fabric Lakehouse “A data architecture platform for storing, managing, and analyzing structured and unstructured data in a single location”
- 21. Overview of Lakehouse Medallion Architecture The landing zone for all data, stored in its original format BRONZE The layer of validated and refined data SILVER The layer of enriched data specified for business and analytics needs GOLD
- 22. Demo: Lakehouse, Notebooks and Pipelines
- 24. Our Solution 1. Add / edit data in Power App 2. Store data in Dataverse 3. Replicate data to OneLake using Synapse Link End-to-End Dataverse OneLake Lakehouse Power BI Data Activator Power Apps 4. Transform data in Lakehouse 5. Visualize data in Power BI 6. Take action on data using Data Activator Marthe
- 25. Our Solution 1. Add / edit data in Power App 2. Store data in Dataverse 3. Replicate data to OneLake using Synapse Link End-to-End Dataverse OneLake Lakehouse Power BI Data Activator Power Apps 4. Transform data in Lakehouse 5. Visualize data in Power BI 6. Take action on data using Data Activator Marthe
- 26. Visualize data in Power BI Model our data First we need to: Second we need to: Visualize our data
- 27. Demo: Power BI
- 28. data-ascend.com Take Action on Data with Data Activator
- 29. Our Solution 1. Add / edit data in Power App 2. Store data in Dataverse 3. Replicate data to OneLake using Synapse Link End-to-End Dataverse OneLake Lakehouse Power BI Data Activator Power Apps 4. Transform data in Lakehouse 5. Visualize data in Power BI 6. Take action on data using Data Activator Marthe
- 30. How can I automate monitoring and prompt action?
- 31. Take Action on Data with Data Activator Core concepts What to do Objects What you want to monitor Triggers What to detect on the object Events Data is being updated Properties Reuse logic Select Detect Create a reflex Act
- 33. Lessons Learned (Getting SaaSy with Fabric)
- 34. C2 - Restricted use Want to learn more about Fabric? www.fabricfebruary.com Join us February 8th, 2024 • Oslo, Norway
- 35. Thank you! Emilie Rønning @EmilieRonning Marthe Moengen @mmoengen Cathrine Wilhelmsen @cathrinew
- 36. Building an End-to-End Solution in Microsoft Fabric: From Dataverse to Power BI Cathrine Wilhelmsen, Emilie Rønning, Marthe Moengen