Practical BI
- 1. Practical BI Strategies that IT can use to maximize its productivity and value to the business Tom Spetnagel Director of Business Intelligence, Valpak Tom Spetnagel
- 2. 2 Summary of Strategies • Use agile practices for BI development • Determine actual requirements and design to them • Utilize appropriate requirements- gathering techniques • Implement and wield a BI charter Tom Spetnagel
- 4. Good News for BI: It‟s Taking Off! Web Search Semantic Search Mobile BI BI Applications Mobile CPU Optimization Data Mining Inline BI Enterprise Search Social Analytics Self-Service Data Technology Social Sentiment Monitoring Access Real-Time BI Big Data Web Analytics In-Memory BI Public Data Cloud Master-Data-Management In-Database Analytics Unstructured Data Data Replication Web Services Tom Spetnagel
- 5. 5 The BI Explosion Is Scary, Too • BI is getting bigger and more complex, but BI budgets aren‟t keeping pace! • Access to information is the root of recent evolution: Google, Facebook, mobile • Self-Service BI is continuously on-the-rise • IT is therefore becoming less central in BI! Tom Spetnagel
- 6. 6 Users Won‟t Wait on IT http://xkcd.com/ BI users can’t wait on IT; they create their own solutions, and they aren’t always good ones! Tom Spetnagel
- 7. BI‟s Biggest Challenge: 7 Prioritization! • Since BI supply can‟t keep up with demand, continuously producing „something good at the right time‟ is critical Tom Spetnagel
- 8. 8 BI Stakeholders: Managing BI stakeholders is a lot like trying to keep chickens under a blanket! -They’re not aligned -They want it all -They want it now -They always want something different Tom Spetnagel
- 9. 9 What‟s A Solution? (Practical BI, Strategy #1) Tom Spetnagel
- 10. 10 What is „Agile‟? • The application of common sense to software development* • A set of concepts developed by people frustrated with the application of „traditional‟ project management to software * I wish I could trademark this! Tom Spetnagel
- 11. 11 Agile Evolution • Agile Manifesto conceived at an informal drink-and-ski weekend in 2001 • Reaction to fundamental differences of building software and building physical items (like aircraft carriers) Tom Spetnagel
- 12. 12 Unlike with physical construction, since it’s only ‘zeros-and-ones’, software can be changed quickly! Tom Spetnagel
- 13. 13 4 Main Agile Principles More Important Less Important • Individuals and • Processes and tools interactions • Comprehensive • Working software documentation • Customer collaboration • Contract negotiation • Responding to change • Following a plan http://agilemanifesto.org/ Tom Spetnagel
- 14. 14 Primary Intentions of Agile • Deliver the most valuable thing at the right time • Deliver working software quickly! • Embrace but manage change • Establish short-term predictability • Eliminate surprises from both the IT and business sides Tom Spetnagel
- 15. 15 Some Agile „Methodologies‟ Scrum Extreme Programming (XP) Unified Process (UP) Feature Driven Development (FDD) Lean Software Development Crystal Clear Tom Spetnagel
- 16. 16 Agile at Valpak: “Scrum” • Multiple scrum teams, each team having: – 1 Scrum Master, 1 Product Owner, 5 to 7 Team Members • 2 week iterations, executing several „stories‟ per team, bounded by: – Sprint planning (1st Monday) – Sprint demo and review (2nd Friday) • Daily stand-up status meetings Tom Spetnagel
- 17. 17 Types of BI „Stories‟ Include: • ETLs • Performance • Metadata Mapping • Data Quality • Formal Reports / • Security Dashboards • Upgrade/patch • Alerts • Automated Report Distribution Tom Spetnagel
- 18. 18 Why Is Agile Great for BI? • Creates a practical method for handling crucial BI challenges which drive scope and affect success • Gives ownership and flexibility to the business, not IT Tom Spetnagel
- 19. Crucial Scope-Drivers in BI (1) • “Data Quality” – A catch-all term for numerous different problems: • Unclear definitions • Missing data / duplicated data • Unexpected data • Unreconciling data • Performance/Speed – People expect reports to run as fast as business „transactions‟ (create 1 order, save 1 order, etc.) • And it‟s even worse with mobile devices! Tom Spetnagel
- 20. 20 Data Quality vs. Effort Data quality is a function of effort; increasing effort has diminishing returns and it is never possible to reach 100% data quality Tom Spetnagel
- 21. Crucial Scope-Drivers in BI (2) • “Terminology”/Definitions The cultural hurdles that come with defining or redefining terms for BI take much time to overcome! • Historical Data Stakeholders often want ‘history’, not just information from this point forward! Tom Spetnagel
- 22. 22 Agile Handling of BI Scope-Drivers • Data Quality – Iterate to provide additional quality checks where desired • Performance – Iterate to achieve better performance where desired • Terminology – Iterate to update definitions where needed; within an iteration, make a decision and go! • History – Load the history in a separate iteration after new data collection has been activated Tom Spetnagel
- 23. 23 Business Accountability • Let the business decide what they want most in the next iteration (based on what IT tells them it can get done in that timeframe) Tom Spetnagel
- 24. 24 1 More Reason Agile Is Great for BI • It‟s tough for BI stakeholders to know what something is worth! • Example: What is it worth to you to have a timely, accurate bank balance? Tom Spetnagel
- 25. 1 Last Reason 25 Agile Is Great for BI “Walking on water and developing software from a specification are easy if both are frozen” - Edward V. Berard, "Life-Cycle Approaches" BI Stakeholders can rarely know what they really need (or need next) until they’re using it! Tom Spetnagel
- 26. 26 Challenges of Agile • High ratio of planning & communicating time to coding time • High amount of time discussing & refining the agile process; some danger of over-analysis • High % of time collaborating; IT folks need to be good communicators Tom Spetnagel
- 27. 27 Agile „In Their Own Words‟: http://www.youtube.com/watch?v=A 0As88akpXs Tom Spetnagel
- 28. 28 Practical BI Strategy #2: Determine, and Deliver to, the Actual BI Requirements • Don‟t deliver just what is (initially) requested; scientifically deconstruct it into what is actually needed • „Requirements‟ and „design‟ are different in BI, just as in application development Tom Spetnagel
- 29. 29 BI Requirements vs. Design: Example #1 Analyst Questions Designer Questions 1. Do users expect the 1. What should the data new data to reconcile sources be? Should the with anything existing? output have any built-in 2. How many people will validations, reconciliations, need access to the or subtotals? same info at the same 2. What mechanism is best for time? How often? providing shared data (web page? email or text alert? printed poster?)? Tom Spetnagel
- 30. 30 BI Requirements vs. Design: Example #2 Analyst Questions Designer Questions 1. How recent/up-to-date 1. Does the solution require does information need access to real-time to be? transaction data? Or can it 2. What is the acceptable be data warehouse data, timeframe for accessing updated/frozen on a information? What are schedule? the response-time 2. Should data be stored in- requirements? database or in-memory? What summarization or indexing is needed? Tom Spetnagel
- 31. 31 Mockups • A report or dashboard mockup is nice but does not constitute either comprehensive requirements or design • Mockups are a great starting point for a requirements conversation, though! Tom Spetnagel
- 32. 32 Practical BI Strategy #3: Use the Best Requirements- Gathering Technique for the BI Assignment • There are a number of different and effective ways to gather requirements • Review, implement, and combine these however necessary Tom Spetnagel
- 33. 33 Requirements-Gathering Techniques 1. Interview 6. Reverse- 2. Survey Engineering 3. Focus Group 7. Document Analysis 4. Requirements 8. Prototyping Workshop 9. Brainstorming 5. Observations 10.Interface Analysis Tom Spetnagel
- 34. 34 Practical BI Strategy #4: Implement and Wield a BI Charter • Gather a set of goals, principles, and strategies that IT and the business can agree on • Use this to focus discussions and overcome objections to IT proposals and decisions Tom Spetnagel
- 35. 35 BI Charter: Example #1 • Goals – Limit confusion around „what numbers are right/best‟ • Principles – Data in the data warehouse are the „official‟ figures unless specifically documented otherwise • Strategies – Get „official‟ figures into the data warehouse – Avoid storing both official and unofficial figures for the same metric in the data warehouse – Restrict access to unofficial data in 3rd party tools Tom Spetnagel
- 36. 36 BI Charter: Example #2 • Goals – Minimize the amount of BI tool-training required • Principles – IT will not support unofficial tools which users have self- provided – Access to 3rd-party BI platforms will be supported on an exception basis when unique value is provided • Strategies – Minimize the number of tools users must know how to use – Use a BI platform which scores highly on ease-of-use and which has multi-purpose tools Tom Spetnagel
- 37. 37 The End – Of The Beginning Tom Spetnagel
Editor's Notes
- #16: These represent practical ways to implement agile