Data Done Right: Ensuring Information Integrity

Editor's Notes

• #2: It’s the ultimate “garbage in, garbage out” quandary. Data can be an organization’s most valuable asset — but only to the degree its quality can be validated and trusted. Without the right guidelines, processes, and solutions in place to control the way applications, systems, databases, messages, and documents are managed, "dirty" data can permeate systems across the enterprise, negatively impacting everything from strategic planning to day-to-day decision making. High-quality data will ensure more efficiency in driving a company’s success because of the dependence on fact-based decisions, instead of habitual or human intuition.   To gain a better understanding of this topic, this speaking session will examine: - what data quality and master data management is - why they are so crucial for successful business operations and strategies - how to improve data quality by organizational, procedural and technological means
• #4: To properly train a predictive model, historical data must meet exceptionally broad and high quality standards. First, the data must be right: It must be correct, properly labeled, de-deduped, and so forth. But you must also have the right data — lots of unbiased data, over the entire range of inputs for which one aims to develop the predictive model. Most data quality work focuses on one criterion or the other, but for machine learning, you must work on both simultaneously. Yet today, most data fails to meet basic “data are right” standards. Reasons range from data creators not understanding what is expected, to poorly calibrated measurement gear, to overly complex processes, to human error. To compensate, data scientists cleanse the data before training the predictive model. It is time-consuming, tedious work (taking up to 80% of data scientists’ time), and it’s the problem data scientists complain about most. Even with such efforts, cleaning neither detects nor corrects all the errors, and as yet, there is no way to understand the impact on the predictive model. What’s more, data does not always meets “the right data” standards, as reports of bias in facial recognition and criminal justice attest. Increasingly-complex problems demand not just more data, but more diverse, comprehensive data. And with this comes more quality problems. For example, handwritten notes and local acronyms have complicated IBM’s efforts to apply machine learning (e.g., Watson) to cancer treatment. Data quality is no less troublesome in implementation. Consider an organization seeking productivity gains with its machine learning program. While the data science team that developed the predictive model may have done a solid job cleaning the training data, it can still be compromised by bad data going forward. Again, it takes people — lots of them — to find and correct the errors. This in turns subverts the hoped-for productivity gains. Further, as machine learning technologies penetrate organizations, the output of one predictive model will feed the next, and the next, and so on, even crossing company boundaries. The risk is that a minor error at one step will cascade, causing more errors and growing ever larger across an entire process.
• #5: data quality management is a principle in which all of a business’ critical resources — people, processes and technology — work harmoniously to create good data. More specifically, data quality management, also known as DQM, is a set of processes designed to improve data quality with the goal of actionably achieving pre-defined business outcomes. Data quality requires a foundation to be in place for optimal success. These core pillars include the following: The right organizational structure A defined standard for data quality Routine data profiling audits to ensure quality Data reporting and monitoring Processes for correcting errors in bad and incomplete data Using these core principles about good data as a baseline, data professionals can analyze data against their own real standards for each. For instance, a unit of data being evaluated for timeliness can be looked at in terms of the range of best to average delivery times within the organization. But it’s also important for organizations to come up with their own metrics with which to judge data quality. Here are some examples of data quality metrics: Data-to-errors ratio: analyzes the number of errors in a data set taking into account its size. Empty values: this is an assessment of how much of the data set contains empty values. Percentage of “dark data”: dark data is a term that means unusable data. The more of that in a set the worst off it is. Email bounce rates: if your emails aren’t going through at a higher percentage than what is typical. You could have a data issue. Time-to-value: this is a ratio that represents how long it takes you to use and access important data after input into the system. It can tell you if data being entered is useful.
• #6: Building a Data-Driven Culture Fortunately, there are ways to overcome these challenges and start building a data-driven culture. Align Initiatives (Both Data and Analytics) With Corporate Strategies  Where strategy goes, data and analytics must follow. Demonstrating ROI, elevating data governance beyond academia, associating data to business value — these can all be accomplished by ensuring that initiatives have a clear tie to corporate strategy and objectives. Strategy maps are a great way to begin. They contain four related components that get progressively more detailed: vision, key strategies, enabling initiatives and required information.  Vision identifies what the company aspires to be.  Key strategies explore the key focus areas that must be addressed to achieve the vision and describe expected business outcomes.  Enabling initiatives highlight data and analytics related projects needed to achieve desired business outcomes and designate key business stakeholders.  Required information identifies the data and analytics capabilities necessary to complete the initiative. Building a strategy map and prioritizing data and analytics efforts from the results can go a long way towards meeting the alignment and cultural resistance issues identified in the NVP report. It associates data with value, lays out a plan for implementing data and analytics capabilities that is blessed by business leaders, and it gets leadership on board for data and analytics efforts.
• #7: Mistake 1: Assuming your enterprise data is clean and accurate “No matter how many safeguards are built into your enterprise applications, data can still be entered and managed inaccurately,” Haines says. “The business will continue to change and grow. Data entry teams will be given new responsibilities. As part of the data quality strategy, applications will be enhanced, business processes will be adjusted, and training must be provided to ensure data is entered and managed accurately.” Mistake 2: Assuming your enterprise data has only one business definition “If differences in the definition and use of data continue, it can allow poor quality data to be entered, managed and reported,” Haines says. “The data quality strategy must include the business community, data governance, and subject matter experts working together to determine consistent and agreed-upon definitions to improve the quality of data.” Mistake 3: Skipping the assessment phase “The best approach is to start with completing an assessment of your organization’s applications and data,” Haines advises. “Business people, subject matter experts and data governance teams work together to first identify and rank the critical business domains, along with data elements deemed critical to each domain. The critical data elements of each business domain are profiled and analyzed to determine their quality. Metrics are developed to provide a high-level view of the data quality for each business domain and associated critical data elements.” Mistake 4: Not profiling and interrogating data values “Profiling and evaluating data is a first step for the business and data governance teams to better understand what their data actually looks like, how it compares to other data values, and how to determine the quality of data,” Haines says. Mistake 5: Not creating and using data quality standards “The more consistent and standardized data evaluation can be, the better data quality within each application will be,” Haines stresses. “In addition, the data quality strategy will be easier to build and manage when it is based on data categories that are being monitored and reported in a standard and consistent manner.” Mistake 6: Not including templates and standard processes as part of the data quality strategy “Standard data quality reports and metrics also need to be developed and shared with the business and data governance teams,” according to Haines. “This will help them understand the need for a data quality strategy and why the data quality tasks are required. Many times, the business community has been sheltered from poor-quality data through improved user interfaces. Showing them examples of actual enterprise data will educate them about what they will look for when evaluating and analyzing data.” Mistake 7: Not following the data quality road map “The data quality road map is developed with input from support team members, database developers, the business community, and the data governance team to ensure a solid sequence of projects is defined,” Haines says. “The road map brings together sets of domains that make business and technical sense. Consideration is given to the size, technology, stability of the applications, and availability of the right team members to be part of the data quality projects.” Mistake 8: Building the data quality strategy in one large project “For the initial data strategy project, start with a business domain that has a high chance of success, involves fewer organizational groups, and can be completed in a short amount of time,” Haines notes. “This project should have a clear set of success criteria that is regularly evaluated and monitored. Smaller projects afford you the opportunity to test ideas in a smaller environment to ensure they perform as expected.” Mistake 9: Viewing technology as the entire solution “Though it’s true that technology continues to move forward and software vendors provide better and faster tools with each new release, data quality management is a three-legged stool with data governance, business processes, and technology each providing a leg,” Haines says. Mistake 10: Not continually monitoring and evaluating data “A data quality strategy is not a one-time data clean-up event,” Haines stresses. “It requires metrics to provide insight concerning the value and usability of data assets over time. Developing these metrics is mainly the task of the business and data governance teams. They will develop data quality metrics to show data quality, data quality scoring methods, and measurement processes, both currently and over time. The goal is that monitoring and reporting these metrics will show improved data as the data quality enhancements are implemented.”
• #8: A Company Problem – Not Just an IT Problem Even just a few years ago, in 2013, the looming spectre of bad data was apparent. Gartner surveyed a wide range of companies in its study and learned that data quality costs them over $14 million dollars a year. Now imagine how much more connected we are today and you can see how the problem could compound exponentially. Many companies, in an attempt to wrangle departments to make sense of it all, place the task of organizing and managing all this information squarely on IT’s shoulders. But bad data affects more than just servers and databases – it affects everyone. In this day and age, it is very much a business problem. And that’s not even factoring in the cost beyond customer data. A few inaccuracies in customer names or details is one thing. But oftentimes, depending on the company culture in relation to data upkeep, it can affect other areas of business as well – productivity, security and making cost effective decisions. In short, this is not a problem we can continue to throw money at and hope it goes away or works itself out. And in addition to revenues and savings, the benefits of clean data go much farther. With greater data reliability comes greater credibility and a stronger decision-making foundation backed by data. Reports become more accurate. Customers respond to more accurate personalization. All departments enjoy greater productivity and efficiency. It’s a cycle of wins. So as you can see, a few inaccurate records or non-standardized entries don’t seem like a big problem, but as your business scales, more and more information becomes fragmented and fraught with issues. Costs escalate. Efficiency plummets. But by the same token, by spending a little now, you reap far greater benefits over time. And any campaign started or improved based on solid, reliable information is one you can look to time and time again for greater insights and metrics that count.
• #9: Incorporating Big Data & AI Together Today AI and Big Data seem to be significant technologies of the digital future, that are certainly going to bring about numerous innovations in the tech world. AI has been employed for what humans have been doing. Divided into two disciplines – Machine Learning and Deep Learning. AI requires data to learn and perform, the lack of which had challenged its existence quite a long time back. Big data is now here, to provide large sets of data to be used by AI for analysis by machines and concluding the right which would further facilitate progress. What the Future Holds for Big Data & AI It’s interesting to note that the Big Data technology has a tremendous potential to derive from Machine Learning techniques and applications, for accurate decision making by the industries and that is the reason Big Data is going to stay in the scene. Of course, the focus will be on the implementation of Machine Learning and Big Data for optimized outcomes. Big Data holds so much crucial in all aspects, as the AI reporter Nick Ismail says, in one of his articles, “There are vast amounts of enterprise data in various organizational silos as well as public domain data sources. The farrago of data needs to be comprehended, the sorted way. There has to be a right blueprint for collecting data and its structure before running it through the machine learning algorithm. A study conducted to find out how Artificial Intelligence can help organizations come up with operational transformation, in conjunction with Big Data, revealed that Artificial intelligence could lead to unexpected business intelligence for organizations, so much that in some cases, it can even replace entire departments performing tasks that otherwise have been reserved strictly for humans.The capacity of AI to amass, sort and make out the meaningful from a massive reservoir of information within seconds has promised wondrous outcomes. No wonder, professionals qualified in business data analytics are going to be much in demand by the companies already awakened to the fact that this vast data needs to be on the beam for its complete application. How corporations are beginning to harness automation Last year, Foxconn — the largest contract manufacturer of iPhones — laid off 60,000 workers, replacing them with industrial robots. Some of these were manufacturing robots called “Foxbots” that were developed internally by the company and can reportedly perform up to 20 common manufacturing tasks. Foxconn has also backed external robotics startups. In Q3’17, it participated in a $20M seed round to Canada-based Kinova Robotics, which focuses on industrial service robots. Earlier this year, it also backed China-based cloud robotics company CloudMinds in a $100M Series A round. In an interview with Digitimes last year, Dai Jia-peng, general manager in Foxconn’s Automation Technology Development Committee, outlined a 3-phase strategy for complete factory automation: automating dangerous tasks, process line automation, and a third phase that would leave only a minimum number of humans on board for tasks like logistics and quality control. Nike and Reebok are looking to speed up the supply chain & logistics process as well, and will automate the manufacturing process in coming years to keep up with high consumer demand and quick turnaround times. In 2013, Nike invested in California-based industrial robotics startup Grabit, which is currently deployed in some of Nike’s manufacturing facilities. But there are hurdles on the road to automation. Dai Jia-peng told the South China Morning Post that “highly automated manufacturing is still an ideal,” since ever-changing consumer demands require highly flexible manufacturing robots that are able to adapt rapidly to design and manufacturing changes. However, Foxconn has fallen short of its 2011 forecast of installing 1 million robots in its factories in 3 years. Like Foxconn, most manufacturing- and logistics-focused corporations are progressing on the road to automation in fits and starts. The road to automation passes through warehouses and factories where robots collaborate with humans (rather than simply replace them). Amazon, for example, already uses 45,000 robots in various warehouses, but at the same time is creating thousands of new jobs for humans in its new fulfillment centers. Robots are still less-than-perfect at gripping, picking, and handling items in unstructured environments. Amazon’s collaborative warehouse robots perform much of the heavy lifting, while workers focusing on delicate tasks like “picking” items off shelves and slotting them into separate orders. The trend stretches deep into physical retail, although we believe e-commerce is the much greater threat to retail jobs. Walmart has patents for autonomous robots that attach themselves to shopping carts in order to move them around stores, along with drone delivery systems. (A detailed analysis of Walmart patents can be found here).
• #13: Data-Driven Decisions Need Context Raw numbers can be informative, but we need to know more. When we make data-driven decisions we still need to consider the human factors. Is there an agenda driving what data is being shared, and the way it’s being presented? Are there multiple ways of interpreting that data? Are there other external factors at play? What’s the story behind the data? Data for data’s sake is a useful tool, but it doesn’t paint the full picture — that needs a combination of data, context and understanding.
• #19: Before implementing any data quality improvement plan, you need to understand where you stand today. Using the data quality sophistication curve, you can determine where you are related to data management efforts and what your next steps should be. There are four stages of sophistication your organization can fall into: Unaware – The organization is unaware of the importance of data quality and its impact on the business. They do not have a strategy in place. Reactive – The organization only performs data cleansing and analysis as issues occur. There are no specific data roles and tactical fixes reside within departmental silos. Proactive – The organization has a proactive data strategy with clearly defined roles within the business. There is a clear ownership between the business and IT and a focus on discovery with root-cause analysis. Optimized – Data quality is monitored as a core factor of the business with documented data quality rules in place. The organization reviews profiling, monitoring and visualizing data as part of a complete strategy. While 88 percent of companies have some sort of data quality strategy in place, many are in early stages of sophistication. Only 18 percent of companies say they have reached the optimized state of data quality. Given that this is self-reported information, it is highly likely that this is an exaggerated number as many feel they are more sophisticated than they may actually be. Almost half of organizations today fall into the reactive or unaware stage, meaning there is a lot of room for data quality improvement.
• #20: “In school, we’re rewarded for having the answer, not for asking a good question” This quote from Richard Saul Wurman rightly describes how a normal human mind, as part of its social development process, adapts to the guidelines of “finding the answers”, rather than exploring the possibilities of asking the “right questions”. And this mindset also reflects in our place of work. We are humanly tailored to explore satisfaction in having answers to all the questions. And in the process of being ‘answer ready’, we tend to become left brain heavy than the right. We become target driven and focus less and less on fresh set of questions which could challenge us further to drive improvement and innovation.
• #21: 2nd last slide. Final slide will be the same as the 1st slide.