Data integration my_experience
Download as PPTX, PDF0 likes593 views
I am a data integration expert who has worked his way up from a data warehouse analyst to an integrations solutions lead over several years. The document discusses challenges of data integration such as dealing with irrelevant or heterogeneous data from different sources, ensuring data quality, unexpected costs, and needing expertise. It provides best practices like understanding requirements and end goals, establishing standards, thorough data analysis, and extensive testing.
Data integration my_experience
- 1. MY STORY Designed By Arvind Murali
- 2. I AM A DATA INTEGRATION EXPERT I WORK FOR HIGHPOINT SOLUTIONS
- 4. My Evolution in Data Integration
- 5. My Humble Beginning Started as a Datawarehouse Analyst/Designer Sorting through 100’s of entities To analyze and report data in excel
- 6. FFWD Three Years DATA INTEGRATION CONSULTING Time to move on to bigger and better things!! Marriage. &
- 7. FFWD Two More Years and ever since… Moving ON to even better things. My Son!! INTEGRATIONS SOLUTIONS LEAD (Why we are doing what we are doing?) &
- 8. CHALLENGES
- 9. Challenge 1 – Relevance Storing all irrelevant information only to find out increased Storage and performance slowness. STICK TO REQUIREMENTS. DO NOT OVERDO.
- 10. Challenge 2 – Heterogeneous Data Operational Databases Data Integration Hub CRM Social Media Rules Repository Ontology Synchronizing huge quantities and formats of data with internal and external systems.
- 11. Challenge 3 – Data Quality What is Data Quality? Data Quality & Data Audit are 2 of the most critical components of any data integration effort. Developers and Users should work together to determine the quality control metrics and measures.
- 12. Challenge 4 – Unanticipated Cost • • • • Because of the involvement of hardware and software to accomplish data integration, it is common to see some unexpected costs in the process. Labor costs for initial planning, evaluation, programming and additional data acquisition. Software and hardware purchases. Unanticipated technology changes/advances. Both labor and the direct costs of data storage and maintenance.
- 13. Challenge 5 – Expertise Data Integration Lead, Data Stewards, Analysts, Designers, and Data Managers are a MUST with practice experience in data management arena. In addition, setting incremental goal helps understand, deliver, and monetize mission.
- 14. BEST PRACTICES
- 15. Lesson 1 – End Goal Understand and Clarify the end goal (BIG PICTURE) with your Stakeholders.
- 16. Lesson 2 -- Requirements Gather Requirements and Specifications Thoroughly and as much detailed as possible.
- 17. Lesson 3 –Standards Understand and Design Industry and Organizational Standards customized to process. ONCE DEFINED, STICK TO THOSE STANDARDS.
- 18. Lesson 4 – Data Analysis A thorough analysis of characteristics and usage of data to mitigate bad and heterogeneous data challenges.
- 19. Lesson 5 – Testing Go GAGA over testing. Most critical phase of integration.
- 20. THE PROCESS!!
- 21. REFERENCES Information Management Magazine Dilbert Cartoons The Datawarehouse Institute (TDWI) Cartoons for Data Data Integration Slideshares