Big Data Maturity Model and Governance
8 likes2,786 views
This document provides an overview of managing big data projects for business results. It discusses introducing big data and the project lifecycle, including planning, development, operation and support, and evaluation phases. Key activities, roles and deliverables are outlined for each phase. The document also covers determining big data opportunities, defining a team structure, types of analytics, key success factors, and concludes with thanks.
Big Data Maturity Model and Governance
- 1. Danairat T., 2015, danairat@gmail.com1 Big Data Project Management Managing Big Data Project for Business Results Danairat T. Certified Java Programmer, TOGAF – Silver danairat@gmail.com, +66-81-559-1446
- 2. Danairat T., 2015, danairat@gmail.com2 Agenda • Introduction to Big Data • Big Data Discovery Worksheet • Big Data Project Life Cycle • Big Data Team Structure • Key Activities, People and Deliverables • Key Success Factors • Summary
- 3. Danairat T., 2015, danairat@gmail.com3 Big Data Introduction 3 Volume Variety Velocity DB Table Delimited Text XML, HTML Free Form Text Image, Music, VDO, Binary Batch Near real time Real time GB TB PB XB ZB
- 4. Danairat T., 2015, danairat@gmail.com4 Big Data Project Goals Worksheet Sample # Domain Increasing of Values or Revenue Cost Optimization Reduce Risk 1 Finance Increase Revenue Create Services Portfolio Compliance with laws regulations 2 Customer New Products, Service, Promotion Innovation Reuse Business Channels Service Continuity and Availability, Customer Complaint Management 3 Internal Create New Business Process Eliminate Production Cost Standardize Change Control 4 Learning and Growth Seek more Talent People Standardize Skill Required Enterprise Knowledge Repository
- 5. Danairat T., 2015, danairat@gmail.com5 Big Data Project Life Cycle Big Data Planning • Identify Targeted Users • Identify Target Opportunities • Identify Team Structure • Identify Data Sources/Types • Identify Data Capturing Approaches • Identify Data Processing and Visualization Planning • Identify Big Data Platform • Identify Security • Identify Governance and Change Control for Operation • Identify Phasing, Budget and Cost Big Data Development • Develop Use Cases • Develop Requirements Definition • Develop Big Data Solution Framework • Develop the Development and Test Environment • Develop Data Capture • Develop Analytics • Integrate Visualization • Manage Assets and Configurations Operation and Support • Monitor Big Data Platform Availability, Utilization and Capacity Planning • Manage Operation Tasks (Admin. Scripts, Commands), Data Capturing System, Upgrading or Patching • Manage Service Requests and Incidents • System admin. Training • Helpdesk Training • End-User Training (Analytic Results) Evaluation • Adoption Rates for each analytics results • No. of Missing Analytic Results • No. of Missing Data • Lost hours per month • Avg. of each Analytic Result Response Time • No. of Technology System Failure per month • Evaluate Activity Conformance with Policies
- 6. Danairat T., 2015, danairat@gmail.com6 Big Data Discovery Worksheet Identify Big Data Opportunity - Sample Customer Complaint Who What Why When Data Sources CEO แนวโน้มจํานวนเรืองร้องเรียน และ แนวทางการตอบสนองความ ต้องการของลูกค้า รักษาภาพลักษณ์ทีดีขององค์กร ผู้ใช้บริการมี Loyalty รายได้เติบโต Monthly/Ad-hoc Call Center GIS/Map Pantip.com Products/Services Master Data Billing System – Revenue/Month COO ประเภทของการร้องเรียน และ ความถีของเรืองร้องเรียน พัฒนาบริการให้ดีขึน เรืองร้องเรียนลดลง Monthly/Ad-hoc Call Center GIS/Map Pantip.com Products/Services Master Data CIO ประเภท จํานวน และแนวโน้มของ การร้องเรียน เพิมสือ และช่องทางการให้ข้อมูล ระดับการตัดสินใจแก่ผู้บริหาร ระดับสูง และทีมงานด้านธุรกิจที เกียวข้อง Monthly/Ad-hoc Call Center GIS/Map Pantip.com Products/Services Master Data CMO วิเคราะห์ประเภทและความรุนแรง ความถีของเรืองร้องเรียน ปรับปรุ้งบริการในเขตความ รับผิดชอบให้มีประสิทธิภาพสูง Monthly/Ad-hoc Call Center GIS/Map Products/Services Master Data Business Line Manager วิเคราะห์ประเภทและความรุนแรง ความถีของเรืองร้องเรียน เพือส่งต่อข้อมูลร้องเรียนให้กับ หน่วยงานทีเกียวข้อง และติดตาม ปัญหาจนสินสุด Monthly/Daily Call Center GIS/Map Products/Services Master Data
- 7. Danairat T., 2015, danairat@gmail.com7 Big Data Program Committee Project Manager Big Data Architect Data Scientist Business Analyst Data Integration Specialist Developer Big Data Evangelist Big Data People and Team Structure
- 8. Danairat T., 2015, danairat@gmail.com8 Big Data Team Structure No. Roles Description 1 Big Data Program Committee The Team to develop Big Data initiative and deliver solution results 2 Big Data Evangelist The business evangelist must have a combination of good communication and presentation skills and deep contextual business knowledge, as well as a clear understanding of technology in general and big data techniques. 3 Project Manager The project manager “owns” the development schedule and is expected to ensure that the right architects, designers, and developers are brought into the project at the right times. 4 Big Data Architect The person who has background in parallel and distributed computing architecture. This person is knowledgeable about fundamental performance “gotchas” that will impede the speed, scalability, and extensibility of any application requiring massive data volumes.
- 9. Danairat T., 2015, danairat@gmail.com9 Big Data Team Structure No. Roles Description 5 Data Scientist The data scientist combines knowledge of computer science, the use of high-performance applications, and statistics, economics, mathematics, and probabilistic analysis skills. 6 Business Analyst The person who engages with the business process owners and solicits their needs and expectations. Business analysts who are able to effectively translate business expectations into specific data analysis results. 7 Data Integration Specialist The person who has experience in data extraction, transformation, loading, and data transformations in preparation for cleansing and delivery to target systems. Seek people with experience with data federation and virtualization, data quality, and metadata analysis. 8 Application Developer The technical resources with the right set of skills for programming and testing parallel and distributed applications.
- 10. Danairat T., 2015, danairat@gmail.com10 Key Activities, People and Deliverables No. Phases Activities People Deliverables 1 Planning Identify Targeted Users Big Data Program Committee Big Data Discovery Worksheet 2 Planning Identify Target Opportunities Big Data Program Committee Big Data Discovery Worksheet 3 Planning Identify Team Structure Big Data Program Committee Team Organization Chart 4 Planning Identify Data Sources/Types Big Data Architect, Data Scientist, Data Integration Specialist Data Sources Types Information 5 Planning Identify Data Capturing Approaches Data Integration Specialist, Data Scientist Data Capturing Information 6 Planning Identify Data Processing and Visualization Planning Business Analyst, Big Data Architect, Data Scientist, Developer Data Processing Framework and User Interface Summary 7 Planning Identify Big Data Platform Big Data Architect, Project Manager Big Data Platform Summary 8 Planning Identify Security Corporate Information Security, Big Data Architect, Project Manager Security Scope Summary 9 Planning Identify Governance and Change Control for Operation Internal Control Team, Corporate Information Security, Big Data Architect, Project Manager Governance, RACI, Change Procedures Summary 10 Planning Identify Phasing Budget and Cost CIO, CFO, Project Manager, Business Analyst Project Investment Summary
- 11. Danairat T., 2015, danairat@gmail.com11 Key Activities, People and Deliverables No. Phases Activities People Deliverables 1 Development Develop Use Cases Business Users, Business Analyst, Big Data Architect, Big Data Evangelist Use Cases Summary 2 Development Develop Requirements Definition Business Users, Business Analyst, Big Data Architect Requirements Summary 3 Development Develop Big Data Solution Framework Big Data Architect Solution Component Diagram 4 Development Develop the Development and Test Environment Big Data Architect, Data Integration Specialist, Developer Development and Test Environment 5 Development Develop Data Capture Data Integration Specialist, Developer Data Capturing Module 6 Development Develop Analytics Data Integration Specialist, Developer Data Analytic Module 7 Development Integrate Visualization Data Integration Specialist, Developer User Interface and Visualization Results 8 Development Manage Assets and Configurations Project Manager, Big Data Architect, Corporate Information Security, Head of IT Operation Assets Inventory and Configurations Change Procedure Agile Methodology may apply in Big Data Development Phase.
- 12. Danairat T., 2015, danairat@gmail.com12 Key Activities, People and Deliverables No. Phases Activities People Deliverables 1 Operation and Support Monitor Big Data Platform Availability, Utilization and Capacity Planning IT Operation Team Availability, Utilization and Capacity Planning Report 2 Operation and Support Manage Operation Tasks (Admin. Scripts, Commands), Data Capturing System, Upgrading or Patching IT Operation Team, Big Data Architect Schedule or Ad-Hoc Operation Activities 3 Operation and Support Manage Service Requests and Incidents IT Operation Team Service Requests and Incidents Procedures 4 Operation and Support System Administration Training Big Data Architect, Data Integration Specialist, Developer, IT Administration, IT Operation System Administration and Operation Training Activity 5 Operation and Support Helpdesk Training IT Administration, IT Operation, IT Helpdesk Helpdesk Training Activity 6 Operation and Support End-User Training (Analytic Results) Business Analyst, Business Users End-User Training Activity
- 13. Danairat T., 2015, danairat@gmail.com13 Key Activities, People and Deliverables No. Phases Activities People Deliverables 1 Evaluation Create Adoption Rates for each analytics Results IT Operation Percent of user adoption 2 Evaluation Create No. of Missing Analytic Results Big Data Program Committee No. of Missing Analytics Report 3 Evaluation Create No. of Missing Data Results Big Data Program Committee No. of Missing Data Report 4 Evaluation Create Lost hours per month Results Big Data Architect, Data Scientist, Data Integration Specialist Lost hours per month Report 5 Evaluation Create Avg. of each Analytic Processing and Response Time Results Data Integration Specialist, Data Scientist Analytic Processing and Response Time Report 6 Evaluation Create No. of Technology System Failure per month Results Business Analyst, Big Data Architect, Data Scientist, Developer Technology System Failure per month Report 7 Evaluation Evaluate Activity Conformance with Policies Big Data Architect, Project Manager Change Control Log Report
- 14. Danairat T., 2015, danairat@gmail.com14 Key Activities, People and Deliverables Big Data Platform Big Data InfrastructureBig Data Infrastructure BI/Report Next Best Action Distributed Data Processing Integration and Metadata Framework Distributed Data Store and DWH Monitoring and Management Framework Security Framework Predictive Analytics Descriptive Analytics Prescriptive Analytics Big Data Platform Big Data Applications Hardware, Storage, Network Fraud Analysis Cyber Security Talent Search
- 15. Danairat T., 2015, danairat@gmail.com15 Key Activities, People and Deliverables Analytic Types Descriptive analytics answers the question, "What happened in the business?" It looks at data and information to describe the current business situation in a way that trends, patterns and exceptions become apparent. This takes the form of reports, dashboards, MIS, etc. mu-sigma.com
- 16. Danairat T., 2015, danairat@gmail.com16 Key Activities, People and Deliverables Analytic Types Predictive analytics answers the question, "What is likely to happen in the future?" Here data modeling and forecasting are used to determine future possibilities mu-sigma.com
- 17. Danairat T., 2015, danairat@gmail.com17 Key Activities, People and Deliverables Analytic Types Prescriptive analytics is the combination of the above to provide answers to the "So what?" and the "Now what?" For example, what should a business do to retain key customers? How can businesses improve their supply chain to enhance service levels while reducing costs? mu-sigma.com
- 18. Danairat T., 2015, danairat@gmail.com18 Key Success Factors 1. Support from Business Sponsor 2. Start with Outcome Answer First 3. Involve Real Users and Create Effective Use Cases 4. Define Quick-Win and Phasing 5. Sufficient Data Source 6. Choose the Open Technology Platform 7. Identify SLA for Service Operation 8. Project Review
- 19. Danairat T., 2015, danairat@gmail.com19 Thank you very much.