Data Management: The Key To Unlocking AI And BI Potential

This piece by Robbie Morrison, CEO of Velosio, was initially published on Forbes.com. Check out the original here.

Today, the amount of data that an organization needs to manage and analyze overwhelms many CEOs. Their concerns are not unfounded.

A 2016 Forrester report noted that the average company doesn't use 60% to 73% of its data for analytics. This means up to three-quarters of a company's data is not used to inform future decisions. To further compound the issue, a 2021 Gartner, Inc. report highlighted that organizations lose around $12.9 million on average every year due to poor data quality, and a 2021 Prove report stated that 21% of companies suffer reputational harm due to inaccurate data.

Data management is about defining processes and procedures to ensure the data flowing into your analytics is valid and meaningful. As your dataset enlarges, data quality could compromise decisions. Many times, an organization is in a rush to pull data in, and it becomes a disorganized mess.

For instance, if you acquire another company, you want to import and consolidate its customer data right away. However, customer data is often structured differently between systems. The lack of standardization can lead teams to fall back on their own Excel spreadsheets.

As a result, you and your executive team will struggle to come to unified decisions on important issues because the conversations will revolve more around the conflicting datasets rather than focusing on the problem or the solution. This leads to different versions of the truth within an organization.

When clients come to us, many still use Excel as one of their main data sources—if not the main data source. When they transition to an ERP and/or CRM system such as Microsoft Dynamics, it becomes a more comprehensive data repository with integrated business processes. This provides a baseline of organized data.

Master data management (MDM), a subset of data management, is a business initiative. Enabled by technology tools, MDM ensures the uniformity, accuracy, stewardship, semantic consistency and accountability of an organization's shared assets—delivering a single source of truth. It's imperative for businesses with data across multiple systems and organizations, and it's even more important for organizations with frequent acquisition or merger activity.

Here are some tips to help you incorporate MDM into your organization:

• Data Quality: Ensure your data is accurate, complete and consistent. Regularly clean and validate your data to maintain its quality.

• Data Governance: Establish clear policies and procedures for data management including data privacy, security and compliance with regulations.

• Data Integration: Combine data from various sources to create a unified view. Use data integration tools to streamline this process.

• Data Storage: Choose the right storage solutions for your data needs. Consider factors like scalability, performance and cost when selecting storage options.

• Data Access: Implement role-based access controls to ensure that only authorized users can access sensitive data. This helps protect your data from unauthorized access.

• Data Backup And Recovery: Regularly back up your data and have a recovery plan in place. This ensures you can quickly restore your data in case of a disaster.

• Data Lifecycle Management: Manage the entire life cycle of your data, from creation to deletion. This includes archiving old data and securely disposing of data that is no longer needed.

Organizations must have an effective data management system in place. The data your business collects is a goldmine for increasing revenue and profits, and some great tools are available. Many of our clients have used Microsoft's Intelligent Data Platform, which combines AI with analytics, security and databases to help businesses better govern their data.

Artificial Intelligence (AI) And Business Intelligence (BI)

Higher-quality data is essential for implementing BI and AI initiatives. AI is the most talked-about new tech in a long time. However, without reliable data and human oversight, it can grow out of control.

Poor data oversight means garbage in, garbage out when it comes to AI and BI. According to a July 2024 Gartner report, "at least 30% of generative AI (GenAI) projects will be abandoned after proof of concept by the end of 2025, due to poor data quality, inadequate risk controls, escalating costs or unclear business value."

Over time, a focus on data cleanup and storage hygiene will pay off in helping you extract more from your AI and BI initiatives. This, combined with a data management team that centralizes the administration and management of data, can build consensus and gain data oversight.

An organization can take advantage of data management tools to help standardize the data so multiple systems and users can use it without worrying about its origins or accuracy. It also ensures your data is more secure. We had a client who ended up paying for more Azure tenants than they needed, and this out-of-control growth meant the client also didn't know who had access to the data within the tenants. This type of potential security issue coupled with overspending is not uncommon.

Effective data management enables you to fuel your BI and AI initiatives with confidence. It also allows companies to better analyze information, increase operational efficiency, enhance security and comply with regulations. It's important to seek advice from experts who have seen the struggles that other organizations have had with maintaining their data management. Build a trusted network of consultants combined with internal resources who are data stewards. This will ensure that your data is driving agility, accuracy and profitability.