From the course: Data Management Essential Training

Data lifecycle management

From the course: Data Management Essential Training

Start my 1-month free trial Buy for my team

Data lifecycle management

“

- [Instructor] Data is vital for the businesses of today, which is why data management is so important. And this management isn't a one step process, but more the care and feeding of this data as it travels into and throughout our business teams. There is a lifecycle for data within our organizations, and this starts with the acquisition of data. The first step is data collection. Step one for doing anything with data is collecting it. Getting it from the many disparate sources, such as point of sale systems or spreadsheets from different areas of the business, and ensuring it is collected in an effective manner that maintains the accuracy of the information. No matter how good the rest of our data management strategy is, collecting inaccurate data will set us up for failure down the line. The next stage in the lifecycle is data processing and integration. The data we receive is in a raw format. It could be sales data coming from our retail stores, or perhaps metrics coming from IoT devices in our warehouses. This data probably needs to go through some transformation process, such as cleaning or aggregation, to get it into the most useful format. Throughout this process though, we need to maintain the needed granularity, quality, and consistency of the data. Once the data has been processed, we move on to step three, which is storage. There are many options for storing data, from flat files all the way to data warehouses. We'll cover each of these options in a later chapter. The key here is matching the right data storage solution to the data, ensuring the data is both stored in a secure fashion and in a way that makes it easy to use. Once our data has been stored, we can move on to data analysis. This stage is where we start to pull out meaningful insights from our data. This is usually carried out by data scientists, applying statistical models and complex computational techniques to look for patterns and derive findings from the data. The final stage of this process involves presenting these findings, creating beautiful dashboards, visualizations, and graphics that enable business leaders to make decisions based on the data that has traversed this whole lifecycle. Throughout this whole process are two wrappers, which are governance and artificial intelligence, or AI. Governance ensures that our data is secure, remains accurate, and that we're complying with regulations. AI is the new hot topic, but it can be very useful throughout all the stages of this data lifecycle. AI can help us to improve automation and efficiency throughout the stages, meaning we can glean more insights and knowledge from the data we have collected, stored, and processed. That wraps up the data management lifecycle. You'll notice we keep coming back to this diagram as we progress through the course.

Contents