Data Architecture: OMG It’s Made of People
1 like284 views
Data architecture is essential for organizations, focusing on processes, standards, and policies for data collection, storage, management, and use. It highlights the importance of prioritizing people's needs over technology, advocating for a balance between organizing data and utilizing technology effectively. The document emphasizes that curation practices should be established based on data usage to ensure findability and accessibility.
Data Architecture: OMG It’s Made of People
- 1. Data Architecture OMG – It’s Made of People! Mark Madsen, Teradata @markmadsen https://www.linkedin.com/in/markmadsen/
- 2. The Man. The Myth. The Mark. Fellow Technology & Innovation Office President Autonomous Robotics Artificial Intelligence
- 3. Data work is not easy. Ask any user. Technology exists to help the organization to be more productive Organizations are made of people Our goal is to make it easy for organizations (people) to use data Data architecture is the foundation on which this work depends Why This Topic? Have you tried turning it off and on again?
- 4. What Do We Mean By Data Architecture? Data Storage? Data Models? Data Technologies?
- 5. What Do We Mean By Data Architecture? Data Storage? Data Models? Data Technologies? Data Architecture is Processes, Standards, and Policies that address an organization’s collection, storage, management, and use of data. It tells you something about what and how but doesn’t dictate implementation. You should be able to answer these key questions: 1. What do you collect, and why? 2. Where do you keep data, and why? 3. How do you organize, curate, and integrate data?
- 6. This takes organization and methods
- 7. Where to focus? Do you focus on organizing books? That’s the data-first approach. Organize everything up front without knowing how it is used. Organize the data wrong and nobody can find or use anything.
- 8. Where to focus? Do you focus on the building that stores books? That’s the technology-first approach. Don’t organize anything in advance. Use technology to sort it out. You may have a catalog of all the contents. Good luck finding what you need.
- 9. Focus on the people and what they do. Not the books. Not the building.
- 10. What people say I want self-service! What they mean Users think “self-service” in terms of a finished data product – self service equals an answer to a question.
- 11. What people say I want self-service! What developers hear Developers think “self-service” is data access, which means the user must be self-reliant.
- 12. Hearing a need, ask: “Why is this an unmet need?” Bad IT and organizational policies cause more problems than technology failures or bad data. Policy is a part of architecture that is ignored.
- 13. Shape architecture for people. Don’t try to force people to technology.
- 14. • Get a quick answer • Solve a one-off problem • Analyze causes of a problem • Build a predictive model • Make repetitive decisions • Use data in a routine process • Make a complex decision • Do experiments and analyze results • Explain a situation to someone else • Choose a course of action • Convince others to take action Architecture focuses on what people want to do
- 15. How To Understand What Data Is Being Used? Monitor the data environments. Capture what data is used. Catalogs of data don’t tell you anything about use – and use changes over time. This means users shouldn’t control storage. Copies they make outside your view are invisible. So: you must give them a place to work and not restrict it. Focus on visibility of use
- 16. Different Views – Data and Users The value of data is tied to its use. This shows relationships between people and data used. 70% of the data is used and reused constantly. 30% of the data is used by one or a few people, often new data with undetermined value. Usage information shows where and how you should focus curation – what you need to manage based on the people using data.
- 17. Finally: establish curation practices based on data use Curation is about what data is used, by whom, and for what purposes Collect, Label, Link Categorize, Organize Index, Catalog, Place The amount of available data is vast. You can’t store it all. You can’t analyze it all. Choose wisely. There’s a difference between organizing datasets and data modeling. One is oriented to datasets and their use, and one to the contents of the datasets. An important and oft-ignored element of data architecture is making sure the data is findable and accessible by the people who need it. This is a curation task, not a data management task
- 18. Thank you. ©2021 Teradata Thank you. ©2021 Teradata