MDM and Reference Data

Editor's Notes

• #2: (Final version) Good afternoon and Welcome. My name is Barry Williams and I will be talking about “Establishing a Single Vocabulary and Reference Data System”. So if that’s not what you came here to listen to, now would be a good time to leave ! For the last couple of years I have been the Data Architect at the London Borough of Ealing after about 15 years in both the public and private sectors Before Ealing I was at the London Borough of Haringey and what I will be talking about will draw on my experience at Ealing and Haringey, and also the private sector, particularly in Customer Data Management. Could I ask for a show of hands – who is from a Local Authority ? And from the Public Sector ? If you have any Questions, ask them when they occur to you and if it turns into a lengthy discussion we can postpone it until the end. I will be covering a number of major areas within Enterprise Data Management, so let me start by telling you what I’m going to tell you…
• #3: Here’s an Overview of the four major Areas I will be covering … Key Data Definitions are critical for Business Intelligence and CRM We will be seeing how a little later. 2) MDM is necessary for CMI and standardised Products and Services, also for Reference data, such as Ethnic Origins, Marital Status and so on. 3) Data Quality is the key to successful integration of data from different systems. 4) The Enterprise Data Model plays a very significant Role in providing a Standardised Data Integration and Mapping Layer – in other words, a Master Data Standardisation Layer. . 5) I will finish with a Tutorial showing how to get started with MDM. This will be based on a range of Starter Data Schemas I have created for Microsoft. Now let’s discuss a Data Architecture that covers our areas of interest …
• #4: I’ll start at the top, work my way around the outside and end up in the middle of this Architecture. Data Governance ties everything together and includes Policies and Procedures for Enterprise Data Management. CRM starts as a Contact Centre and builds to include aspects such as Customer Profiles, and Good/Bad Customer and it requires a Single View of the Customer. BI is based around building Data Marts, which can be dedicated to specific areas, such as Social Services and the Street Environment. In Local Government, they will always include calculation of central Government requirements, such as BVPIs, and IEG Returns. Loading data into Data Marts from various back-end Sources requires that the Data is all converted to a common format, where there is a common understanding and agreement about ‘What is a Customer’. This is all part of Master Data Management. The Data Dictionary includes Glossary of Terms and Mapping Data Integration consolidates data from Operational Systems. These can total typically 200 altogether but it is possible to identify the 'Top 20' – Council Tax, Housing Benefits, Social Services and so on. Customer Matching allows Customer Histories to be built on a Hub with Links to LOBs and ODSs This includes a Customer Master Index and a Product and Service Catalogue Data Quality Audit Functions include Data Profiling. Property and Addresses are validate against the Property Gazetteer for which the Local Authority is responsible. Reference Data includes a wide variety of Master Data - Ethnic Origins, Vehicle Makes and Model, Marital Status and so on. Where possible, external standards should be followed. LGSL, national and internationa standards. Naming of Services should follow relevant Standards – eg LGSL/IPSV and link with ERDMS MASTER DATA STANDARDISATION LAYER Mapping from Vendor-specific to Ealing Standards It implements the Enterprise Data Model for Mapping It can be built in SOA using Tibco Smart Mapper and Master Data Management)
• #6: I’d like to say something about the backgrdound of the situation at Ealing to identify the Business Drivers for a move towards Master Data Management. We have more than 200 Operational Systems These range from departmental systems, such as Leisure Centres and Tree Management to mission-critical Systems like Council Tax. We identified the 'Top 20' major ones that we considered when we planned our strategy. We are moving towards a 'single point of contact' with a new CRM system (last November). Fraud is an important Driver. It was reported about three weeks ago by the Association of Police Chiefs that Banks and Insurance companies lost about £1 billion to fraud and Local Government lost about £700 million in Benefits Fraud. We are also moving towards a more integrated view of Business Intelligence across the Council. This also requires MDM so that we can integrate data from a multitude of Systems and present 'A Single View of the Customer' as well as Traffic Lights following a consistent philosophy of Performance Reporting .
• #7: We have looked at the Master Data Standardisation Layer and seen its importance. An essential part of the Layer is the Enterprise Data Dictionary. This usually starts out small and grows to the stage where it is a Repository of information about every item of information of importance in the organisation. It can begin with the ‘Top 20’ IT Systems, the people involved, the Tables, Columns and Business Rules. Packages are very expensive – eg £250,000. As we move into Master Data Management, it will also include Mappings of Entities and Attributes, and details of the Enterprise Data Model and other Models. It is easy to get started by tracking down sources of definitions (eg in Word documents) and then consolidating them into Spreadsheets, Access Databases and finally publishing them over the Corporate Intranet using SQL Server or Oracle. It’s good to encourage Feedback and ‘KM Communities of Practice’. At Centrica (the AA and British Gas) , I was part of a new Department called the Customer Intelligence Unit . We provided a Feedback facility and found 26 groups interested in CRM
• #8: This is a small extract from our Enterprise Data Dictionary. It shows how Data is described differently as seen from different perspectives and then mapped on to a common definition. This common defintion is, of course, part of our Master Data Management Strategy. The left-hand column shows the Vendor codes are which, of course, historic, and were determined by the Vendor when the System was first developed. In this case, this comon definition is the central government Local Government Services List (LGSL) Code, which we have adopted as the default standard. Current techniques provide organisations who buy these kind of Systems to load them with their own specific Standards. In other words, they are more ‘Open' which makes it much easier for us to adopt external standards. Additionally, particularly in the Public Sector, some Systems come pre-loaded with existing relevant standards.
• #9: Let’s turn now to the second major area in this Presentation and start by considering ‘What is MDM ?’ The Professional view comes from The Data Warehousing Institute , which says that MDM includes :- * Master Data / Reference Data * Business Entity Defintions * System of Record 'Trusted Source' * MDM Hubs – eg Customer Hubs * Master Data Integration * MDM is “practice of defining and maintaining consistent definitions of Business Entities,(EDM)and then sharing them via integration techniques across multiple IT Systems within the enterprise or partners. More simply put, MDM is the practice of managing Master Data.” Circular and Not Very Helpful - aimed at Data Mgt Professionals ! A more user-Friendly definition comes from Wikipedia – BTW Who uses Wikipedia ? Their definition is :- Master Data Management ( MDM ), also known as Reference Data Management, is a discipline in Information Technology (IT) that focuses on the management of reference or master data that is shared by several disparate IT systems and groups . MDM is required to warrant consistent computing between diverse system architectures and business functions. They also introduce Customer Data Integration ( CDI )” is the combination of the technology, processes and services needed to create and maintain an accurate, timely and complete & comprehensive representation of a customer across multiple channels, business lines, and enterprises.” - ie 'A Single View of the Customer‘ or CMI MY DEFINITION is that MDM means providing a Single View of the Things of Interest in an Enterprise”. This usually starts with Customers and Products or Services. MY VIEW is Reference Data is data that commonly appears in valid Lists of Values (‘LOVs’) – eg Ethnic Origins, Marital Status, Vehicle Makes and Models and so on.
• #10: CRM needs ‘A Single View of the Customer’, and BI needs data loaded into Data Marts in a consistent manner and format. Both of these needs translate into a Role for MDM to play. This in turn, requires Customer Data Integration and a Enterprise Data Platform which we have implemented as Master Data Standardisation Layer.
• #11: One of the major implications of MDM is that the Enterprise adopts a common way of looking at data. In order to achieve this, Data Governance becomes very important. The things that could previously be done by each System, more or less independently, now have to be done from an Enteprrise perspective. For example, standards for Ethnic Origins. Therefore it's vital to get buy-in from all the interested parties, and particularly the IT Managers for mission-critical Systems. In a Local Authority this includes Council Tax, Housing Benefits and Social Services. It also raises consideration of the Data Protection Act. This Enterprise-level approach takes time because you need to build Consensus through Workshops, one-to-one discussions. You also need support from the top to get the message across that Data Governance is an Enterprise issue and has visible support at Director level.
• #12: Software is available from a number of Vendors. We talked to a number of major players, including (alphabetically) Business Objects, Cognos, Dataflux and Oracle. We also talked to some niche players, such as Clearcore which has a good reputation in the area of Customer Matching. This can be a lengthy process and you need to allow time for it.
• #13: I'd like to give you an insight into the role of a CMI in Customer Data Integration . This diagram shows that we are consolidating Customer-related data from a number of back-end Operational Systems in order to support CRM and the Customer Contact Centre. The Agent wants to respond in an intelligent way to the Customer and be able to answer any questions with a full knowledge of the facts. The Data Protection Act presents some interesting challenges. Until recently, it has been the case that the Agent can answer a specific question from the Customer but could not say anything that implied that they knew more about the Customer. In other words, they couldn't imply that they had access to 'A Single View of the Customer'. This interpretation of the DPA resulted in lengthy meetings with the Legal people. Fortunately things are easier now, where the default changed recently towards sharing data to provide a higher and more consistent level of service to the Customer. An example of the challenges was the Deceased Indicator where people maintaining the Register of Deaths would inform the Council Tax people about deceased Residents but were not able to share this information across the Council. There was therefore a real danger of letters being sent out to Residents who were deceased. Typically, each back-end System would refer to Customers in different ways. The CMI then matches different Business Terms for a Customer – Asylum Seeker, Case (Social Services), Claimant (Housing Benefits), Client (Youth Offenders), Organisation, Tenant (Rent Arrears) and Voter (Electoral Register). Therefore a Data Dictionary becomes vital in maintaining a Glossary of Terms and mapping these different Terms to one standard established by Master Data Management. This one standard is derived from our Enterprise Data Model that we will be coming to soon.
• #14: The CMI defines a consistent approach to handling Customer data. This supports Data Marts which consolidate data from different sources. Let’s look at an extract from a Data Mart …
• #15: This is an extract from a Debtors Data Mart MDM is particularly relevant to the Entity in the left-hand side showing Services. It also provides to ‘A Single View of the Customer’ on the right-hand side to facilitate the management of Customer Accounts in a consistent manner.. The Time-Periods example at the top of the diagram is a good example of low-level Reference data that is part of MDM.
• #16: The CMI also supports CRM by providing a Customer Hub and Customer Histories.
• #17: A Common Concept for shared data is required for the Enterprise. An essential part of a Common Concept is ensuring Data Quality and this becomes an Enterprise Issue. Integration of good quality data leads to the Common Concept being implemented as a Common Enterprise Data Platform …
• #18: Stages 1 to 3 are part of Master Data Management. Step 1 includes basic Reference Data like Ethnic Origins or Vehicle Makes and Models. These Stages also define a logical sequence in which MDM and the Enterprise Data Platform can be implemented. And, of course, this Data Platform also provides a long-term perspective which can serve as a Road Map for implementing MDM.
• #19: The Enterprise Data Model plays a central and vital role in Master Data Management . It provides a common point of reference for all questions relating to MDM. The Master Data Standardisation Layer implements the EDM, which is defined initially at the logical level.
• #20: The EDM was developed over a six-month period, at a cost of several hundred thousand pounds. The EDM is implemented in the Master Data Standardisation Layer that we looked at in the beginning of this Presentation. The process of validation of the Model consists of brainstorming discussions with key people in operational areas to validate the Model against the requirements. For example, Ealing Housing and Property Here’s an overview of the EDM …
• #21: Starting in the middle, a Party could be a Customer (either an Individual or an Organisation) or a Supplier or a Professional (such as a Qualified Therapist providing specialised Services) or a Third-party, such as a Solicitor, or a Contractor, such as Graffiti Removers, or ECT who are responsible for keeping the Streets clean. After Customers, the second most important area is Services, which are, of course, anything the Council offers to its Customers. In the left-hand corner, an Agreement is a Contract, such as a Rent Book or Council Tax Payment book.
• #22: This diagram gives you an idea of the level of detail in the Enterprise Data Model. Party Animals is another example of the naming conventions that we adopted. Ealing is a ‘Fun Place to Work’.
• #24: This diagram shows how the data in a Street Environment Data Mart can be displayed on a Map of the Wards in Ealing. The Table on the right-hand side of the diagram shows different types of data that can be displayed at the ‘click of a mouse’. For example, Abandoned Vehicles, Domestic Refuse Collection, Graffiti, Litter, Street Inspections and so on. This is made possible because all of this data has been treated in a consistent manner in accordance with the MDM guidelines that I have established.
• #25: The Approach includes definition of the Enterprise Data Layer.
• #27: Top-down establishes credibility and the importance of MDM within the organisation. Low-level buy-in is needed and collaboration is essential – this takes time. Management involvement is important because MDM, Data Quality and so on all require Data Management expertise but they also require participation by managers who don’t have this kind of expertise. For example, in agreeing the levels of Data Quality that should be the minimum targets – gender, address, and so on –depending on the application of the data.
• #30: What I have done is present a Data Architecture that ties together the topics I have covered. Key components include a Master Data Standardisation Layer, CMI, a Data Dictionary, and the importance of an Enterprise approach to Data Quality. Finally, let me talk a little about some consulting work I did for Microsoft for Christmas. This resulted in a page on the Microsoft web site that lists ten of my Data Models and makes them available for people using the Express Edition of SQL Server 2005. Web Link - http://msdn.microsoft.com/vstudio/express/sql/dataschema/default.aspx Or Google for “starter data schemas”. I have put together a Tutorial in MDM based around this work with a Feedback facility. http://www.databaseanswers.org/pi_best_practice_display/manual.asp?manual_id=BP_MDM http://www.databaseanswers.org/pi_best_practice_display/index.asp Or Site Map page on DatabaseAnswers.org. Please try it out and let me know what you think. That’s the end of my Presentation. Thank you for your time and attention and I’ll be pleased to answer any Questions you might have.