Best practices to deliver data analytics to the business with power bi
4 likes2,615 views
This document outlines best practices for delivering data analytics using Power BI, presented by Satya Jayanty, a seasoned IT professional. It covers session goals, traditional BI concepts, self-service business intelligence trends, Power BI features, and the role of IT administrators in managing data access and integration. The document also discusses licensing options, ongoing updates to Power BI, and various tools available for data analysis and visualization.
![Supported connections for Power
BI data refresh
DATA SOURCE LOCATION AUTO-REFRESH GATEWAY
REQUIRED
SQL Azure Database Azure Yes No
Azure VM running SQL Server Azure Yes No1
SQL Server On-Premises Yes Yes
Oracle On-Premises Yes Yes
OData feed Web/On-
Premises/Azure
Yes No
HDInsight Azure Not Directly N/A
Analysis Services On-Premises/Azure Not Directly N/A
Reporting Services On-Premises/Azure Not Directly N/A
Power Query2 On-Premises/Azure Yes Yes
[1] A VM running SQL Server can be reached by using its public IP address. However, if you have configured this VM inside a virtual
network, you should connect by using its private IP address, just as if it were an on-premises VM.
[2] At the time of this writing, Power Query supports scheduled refresh of relational data in Oracle and SQL Server databases.](https://image.slidesharecdn.com/bestpracticestodeliverdataanalyticstothebusinesswithpowerbi-151130171824-lva1-app6891/85/Best-practices-to-deliver-data-analytics-to-the-business-with-power-bi-29-320.jpg)
Best practices to deliver data analytics to the business with power bi
- 1. Best Practices to deliver Data Analytics to the Business London 28th November 2015 Satya Shyam K Jayanty Principal Architect & Consultant … with Power BI
- 2. About me : Satya Jayanty • IT Experience • Been in the IT field over 24+ years (using SQL Server ver.4.2 onwards) • Principal Architect – D Bi A Solutions, Europe • 10 years as Microsoft MVP (Data Platform) • Community Contributions • Speaker : Microsoft Tech-Ed (North America/India/Europe), SQLPASS, SQLSaturdays SQL Bits, User Groups (Scottish Area SQL Server & Nottingham) • ‘Ask The Experts’ Lounge: Microsoft Tech-Ed, HeroesHappnHere & SQLPASS • SME & Technical Reviewer for SQL Server 2008 & 2012 certification papers. • Publications • Founder (SQLMaster) & blogs at www.sqlserver-qa.net; (Knowledge Sharing Network) • Author: SQL Server 2008 R2 Administration cookbook & SQL Server Analysis Services 2012 Cube Security Instant • Co-Author: MaVP Deep Dives Volume II. Technical Reviewer: SQL Server books from Packt Publishers. • Active participation in assorted forums such as SSP, SQL Server Central, MSDN, SQL Server magazine, dbforums etc. sqlmaster@sqlser ver-qa.net www.sqlserver-qa.net @SQLMaster
- 3. Author eBook & PaperBack www.packtpub.com Amazon US & UK http://tinyurl.com/sql2k8r2admincookbook http://tinyurl.com/sql2012InstantCubeSecurity
- 4. http://www.manning.com/delaney/ Manning Publications and the authors of this book support the children of Operation Smile.
- 5. Session Goals • Power BI possibilities & available tools • Deployment, Configuration & Licensing • Packaged BI features • Office 365 rollout • Manage & Connect @@ On-premise :: Cloud • Build an E2E sample using Power BI • Create, sharing, consuming and refresh • Resources www.sqlserver-qa.net @sqlmaster 5 |
- 6. Traditional BI: …basics… Integrated Reporting and Analytics Data Sources Staging Area Manual Cleansing Data Marts Data Warehouse Client Access Client Access 1: Clients need access to data2: Clients may access data sources directly3: Data sources can be mirrored/replicated to reduce contention4: The data warehouse manages data for analyzing and reporting5: Data warehouse is periodically populated from data sources6: Staging areas may simplify the data warehouse population7: Manual cleansing may be required to cleanse dirty data8: Clients use various tools to query the data warehouse9: Delivering BI enables a process of continuous business improvement
- 7. Silos of Data Data Warehouse Call Center Web Apps Inventory ERP HR Finance CRM
- 8. What differentiates today’s thriving organizations? Data. Data in all forms & sizes is being generated faster than ever before Capture & combine it for new insights & better, faster decisions
- 9. Enormous amounts of data . online behavior social networking users . .. samples of medical ailments .. … purchasing habits of grocery shoppers … …. crime statistics of cities …. ….. “internet of things” IoT….. …… 24/7 out-patient monitor …… ……. real-time tele-metric devices ……. http://www.nytimes.com/2013/04/14/education/edlife/universities-offer-courses-in-a-hot-new-field-data-science.html?_r=1& 90% Of data in the world, has been created in the last 2 years
- 10. Social and web analytics Live data feeds Advanced analytics The three V’s
- 11. Collect any data • Harness the growing and changing nature of data StreamingStructured Challenge is combining transactional data stored in relational databases with less structured data Big Data = All Data Get the right information to the right people at the right time in the right format Unstructured “ ”
- 12. Where to begin? .. It seems like everything is "big data this" or "big data that," and every vendor has managed to work "big data" into their marketing materials as the buzzword du jour. Self Service Business Intelligence (SSBI)
- 13. From Traditional BI to Self Service BI Find, Cleanse, Merge Data Modeling Visualize Analyze Share Consume Processing Path Learn Predict ETL DataWare House Reports Internet/ Intranet Data Mining Traditional BISSIS SSRS SharePointData MiningPower Query Power Pivot Power View Power Map O365Addin Data Mining Power BI
- 14. Key trends in Self Service BI • BI for everyone • Democratizing the access and analysis of data across the organization • Self-service analytics • Business users in organizations will have access to self- service tools to prepare data for analysis. • Real-time analytics • Interactivity enables business users to explore and answer questions with data updated in real time • Data Integration • Massive amounts of data are now available from disparate sources, increasing the demand for rapid data source integration accessible through simple interfaces. • Mobile BI • Using touch-enabled native apps for Windows, iOS, and Android.
- 15. Analysis Services Reporting Services Integration Services Master Data Services SharePoint Collaboration Excel Workbooks Power Pivot ApplicationsSharePoint Dashboards & Scorecards Data Quality Services OData Feeds Line of Business Applications Hadoop Big Data Microsoft’s Business Intelligence - Toolset
- 16. 23 Access Clean Mash-up Explore Visualize Share Excel 2013+ : Complete and Powerful Self-Service BI Tool Self-Service Business Intelligence in Excel 2013+ Report creator Data steward Report consumer Power BI for Office 365 is an umbrella name for many technologies and tools that blend well together. It is a self- service Business Intelligence solution with the ultimate goal of providing deeper business insights from your data. • Excel is the "center of the universe" for all these tools, and works in combination with SharePoint Online.
- 17. SQL Server 2012 …+ Import PowerPivot models into SQL Server Analysis Services Tabular Model •PowerPivot in Visual Studio •Row-based security •Partitioning Multidimensional OLAP •Can run Power View reports using RDS content type and SQL Server 2012 SP2
- 18. Power BI : On-premise vs Office 365
- 19. Powerful Self-Service BI with Excel 2013+ Power Pivot Data blend large or small data sets Calculate Metrics w/ Excel-like formulas (DAX) In-memory xVelocity technology High compression Power Query Pull data off the web • Web APIs • Web pages • Facebook • SharePoint • Hadoop Consolidate Multiple files Power View Interactive Data Visualization Scatter Plots Maps Free form dashboard design Power Map 3D rendered Map imaging Record time- lapse videos of how data changes in geographies
- 21. Power BI Desktop • Queries • Multiple data sources • Data Modeling • Reports • Monthly release cycle • PBIX format Power BI Service • Web portal • Collaboration and sharing • Reports web authoring • Dashboard and Q&A • Weekly release cycle • No download (yet)
- 22. Data Refresh - Hybrid • Power BI Personal Gateway • Many Data Sources • Data Refresh configurable • Analysis Services Connector • Tabular • Live • Reporting Services 2016 (ctp3) Hybrid – On Prem • No connector needed • Live or scheduled Azure & Data Refresh
- 23. (Third party) Content Packs Content Packs • SaaS solution • Predefined Data Sets • Predefined Reports • Predefined Dashboard • More & more providers • Editable • Organizational • … to be followed
- 24. Power BI – Conceptual Architecture O365 Power BI Corporate Data Sources IT Admin Data Management Gateway Corporate Data Catalog Power BI Admin Center Data Management Portal SPO team sites, BI Sites, Q&A Cloud Data Sources Public Data Catalog Data Steward IW (Author) IW (Consumer) Manage Data Publish raw data sources or expose them as OData feeds Publish queries Data Refresh Data Refresh Power Query Power Pivot Power View Power Map Publish reports & create featured questions, configure scheduled refresh Import data, Publish queries Excel, Tablet/Mobile, Browser – Silverlight, HTML5 Consume interactive reports, ask questions Configure Import Data Import Data Import Data Import Data Public Cloud Data Sources Index
- 25. Power BI for Office Components Excel 2013 • Power Pivot (embedded in Excel 2013) – need to be enabled • Power View (embedded in Excel 2013) - need to be enabled • Power Query – need to be downloaded and installed • Power Map– need to be downloaded and installed Excel 2016 • All Inclusive…. • The following Office versions are supported: • Office 2013 Professional Plus • Office 365 ProPlus
- 26. Power BI Tools Signup Get Data
- 27. Supported data sources for Power BI features in Excel Data Source Power Query Power Pivot SQL Server Yes Yes SQL Database (Azure) Yes Yes SQL Server Parallel Data Warehouse (PDW/APS) Yes Yes Access Yes Yes Oracle Yes Yes Teradata Yes Yes Sybase Yes Yes Informix Yes DB2 (IBM) Yes Yes MySQL Yes PostgreSQL Yes OLE DB/ODBC Yes OData Feed Yes Yes Web page Yes Folder metadata Yes SAP BusinessObjects BI Universe Yes
- 28. Supported data sources for Power BI features in Excel Analysis Services Tabular Yes Analysis Services MD Yes Reporting Services Report Yes SharePoint list Yes Dynamics CRM Yes Azure Marketplace Yes Yes Hadoop Distributed File System (HDFS) Yes HDInsight (Azure) Yes Azure Blob Storage Yes Azure Table Storage Yes Active Directory Yes Exchange Server Yes Facebook Yes Excel Yes Yes Text file Yes Yes XML file Yes
- 29. Supported connections for Power BI data refresh DATA SOURCE LOCATION AUTO-REFRESH GATEWAY REQUIRED SQL Azure Database Azure Yes No Azure VM running SQL Server Azure Yes No1 SQL Server On-Premises Yes Yes Oracle On-Premises Yes Yes OData feed Web/On- Premises/Azure Yes No HDInsight Azure Not Directly N/A Analysis Services On-Premises/Azure Not Directly N/A Reporting Services On-Premises/Azure Not Directly N/A Power Query2 On-Premises/Azure Yes Yes [1] A VM running SQL Server can be reached by using its public IP address. However, if you have configured this VM inside a virtual network, you should connect by using its private IP address, just as if it were an on-premises VM. [2] At the time of this writing, Power Query supports scheduled refresh of relational data in Oracle and SQL Server databases.
- 31. Optimization • Optimize your Power BI Semantic Model • Optimal Data Types • Validate data relationships • Naming convention • Field names/Report titles/Metadata – Formatting • Scheduled Data Refresh • Right Visualisations • Dashboards – charts vs line vs pie • Users • Sharing Dashboards • Requirements Gathering
- 32. Power BI Admin Center Admin Center & Power BI Portal • Access via link in O365 admin portal • Configure Power BI roles • Install and monitor the Data Management Gateways for your organization • Configure access to cloud enabled data sources • Expose OData feeds to corporate data sources Role of the IT Admin in Power BI • Varying levels of control across data sources, departments • Oversight and monitoring of cloud data access • Ability to make corporate data sources easier to discover, and easier to access Power BI Personal Gateway
- 33. Data Management Gateway - Conceptual Power BI Admin Center Allows IT to configure, manage and monitor access to corporate data sources. Data Management Gateway Connects to corporate data sources and sends data to Microsoft cloud services through a secure channel (Service Bus). Corporate Data Sources The Gateway can connect to a variety of data sources. Secure Credential Store All credentials used by the gateway are stored on-premises. Encrypted backup of credentials can also be stored in the cloud to enable business continuity scenarios. www.sqlserver-qa.net @sqlmaster 41
- 34. Do I need? • Azure Service Bus • Data transfer between Power BI and the Gateway is secured through Azure Service Bus. • The Service Bus creates a secure channel between the Power BI service and your computer. • Because the Gateway provides this secure connection, there’s usually no need to open a port in your firewall.
- 36. Power BI REST API • Create and manipulate any Power BI object via RESTful API • Dashboards and reports • Data source connections and data sets • Package reusable solutions, apps, custom data sources and add-ins • Power BI Packaged Apps • Power BI Dev Portal • Power BI REST API • Default Retention Policy
- 38. Evolving approaches to Big Data Analytics
- 40. Evolving approaches to Big Data Analytics
- 41. Licensing 3 licenses available for Power BI: • Power BI (Free) • Power BI Pro • Power BI for Office 365 Office 365 • Power BI Pro and Power BI for Office 365 are paid licenses with trial offers. • Power BI for Office 365 License will go away at some point.
- 42. Corporate OData Feeds Data Management Gateway Power Query Power BI Cloud Services
- 43. Gateway Configuration Details • Communicates with cloud services • Requires outgoing ports (9350-9354) • Falls back to 443/80 if the other ports aren’t open • No incoming ports from the internet (messages received via Service Bus) • Receives credentials from ClickOnce application • Serves OData feeds to on-prem clients • Requires incoming port • default: 8050 (and 8051 if HTTP is used) • Recommended to use HTTPS with signed certificate • Must be able to resolve DNS names for data sources
- 44. Data Refresh from Power BI Sites Data Management Gateway Excel Workbook in Sharepoint Online Gateway Cloud Service (2) Connects to Gateway Cloud Service (3) Checks whether user is authorized to perform a refresh (4) Sends command (SQL statement, connection string) to on-premise Gateway (5) Sends SQL to SQL Server (6) Return Results (7) Compress, chunk and return the data (8) Returns data to Excel Workbook 52
- 45. Power BI – September Update • 44 New Features in the areas of: • Report Authoring • Data Modelling • Data Connectivity • Data Transformations and Query Editor improvements • The following video covers all of these features in more detail. • https://www.youtube.com/watch?feature=player_embe dded&v=Jm44dLXdarQ
- 46. Power BI – October Update • 23 New/improved Features in the areas of: • Report Authoring • Data Modelling • Data Connectivity • Data Transformations and Other improvements • The following video covers all of these features in more detail. • https://www.youtube.com/watch?feature=player_embe dded&v=Jbocn1ZNkxM
- 47. Power BI – November Update • 26 New Features in the areas of: • Data Modeling • Basic Automatic Date Features • Data Connectivity • SSAS Multidimensional support • SAP Hana • R Script • DirectQuery for SQL Server, Azure SQL DB and Azure SQL DW • Azure Data Lake Store • Marketo • Data Transformations • Improved Function Invocation experience • Option to set Credentials at Server vs. Database level • Add Prefix/Suffix to a Text column • The following video covers all of these features in more detail. • https://www.youtube.com/watch?feature=player_embedded&v=ErHvpkyQjSg
- 48. Consumption Experiences Data Visualization Data Analysis Data Modeling Data Discovery & ETL Data Warehouse/Big Data Microsoft Analytics Platform www.sqlserver-qa.net @sqlmaster 57
- 49. Modern Self-Service BI players www.sqlserver-qa.net @sqlmaster 58
- 50. Analyze & Visualize in Excel Discover & Combine in Excel Collaborate, Get Insights, & Access Anywhere Through Office 365 Power Query Steward Portal Microsoft BI – Power BI Perspective Data Catalog www.sqlserver-qa.net @sqlmaster 59
- 51. Resources Sign up for a free Preview account Take the Power BI Tour Readthrough Getting Started Knowledgebase and Tutorials Watch YouTube Videos Register on the Developer Portal
- 52. Q&A Thank you….