IS_02.Understand

Editor's Notes

• #3: Key Point: The culmination of these efforts has led us to our latest platform offering – the IBM Information Server. IBM Information Server is a revolutionary new software platform from IBM that helps organizations derive more value from the complex, heterogeneous information spread across their systems. It enables organizations to integrate disparate data and deliver trusted information wherever and whenever needed, in line and in context, to specific people, applications, and processes. IBM Information Server helps business and IT personnel to collaborate to understand the meaning, structure, and content of any type of information across any sources. It provides breakthrough productivity and performance for cleansing, transforming, and moving this information consistently and securely throughout the enterprise, so it can be accessed and used in new ways to drive innovation, increase operational efficiency, and lower risk. IBM Information Server is designed to help companies leverage their information across all its sources. IBM Information Server delivers all of the functions required to integrate, enrich and deliver information you can trust for your key business initiatives. IBM Information Server allows you to: Understand all sources of information within the business, analyzing its usage, quality, and relationships Cleanse it to assure its quality and consistency Transform it to provide enriched and tailored information, and; Federate it to make it accessible to people, processes, and applications All of these functions are based on a parallel processing infrastructure that provides leverage and automation across the platform. The Information Server also provides connectivity to nearly any data or content source, and the ability to deliver information through a variety of mechanisms. Underlying these functions is a unified metadata management foundation that provides seamless sharing of knowledge throughout a project lifecycle, along with a detailed understanding of what information means, where it came from, and how it is related to information in other systems. Integration logic built within IBM Information Server can easily be deployed and managed as a shared service within a SOA. IBM Information Server provides: access to the broadest range of information sources the broadest range of integration functionality, including federation, ETL, in-line transformation, replication, and event publishing the most flexibility in how these functions are used, including support for service-oriented architectures, event-driven processing, scheduled batch processing, and even standard APIs like SQL and Java. The breadth and flexibility of the platform enable it to address many types of business problems and meet the requirements of many types of projects. This optimizes the opportunities for reuse, leading to faster project cycles, better information consistency, and stronger information governance. Regarding Service-Oriented Architectures, information integration enables information to be made available as a service, publishing consistent, reusable services for information that make it easier for processes to get the information they need from across a heterogeneous landscape.
• #4: Key Point: The culmination of these efforts has led us to our latest platform offering – the IBM Information Server. IBM Information Server is a revolutionary new software platform from IBM that helps organizations derive more value from the complex, heterogeneous information spread across their systems. It enables organizations to integrate disparate data and deliver trusted information wherever and whenever needed, in line and in context, to specific people, applications, and processes. IBM Information Server helps business and IT personnel to collaborate to understand the meaning, structure, and content of any type of information across any sources. It provides breakthrough productivity and performance for cleansing, transforming, and moving this information consistently and securely throughout the enterprise, so it can be accessed and used in new ways to drive innovation, increase operational efficiency, and lower risk. IBM Information Server is designed to help companies leverage their information across all its sources. IBM Information Server delivers all of the functions required to integrate, enrich and deliver information you can trust for your key business initiatives. IBM Information Server allows you to: Understand all sources of information within the business, analyzing its usage, quality, and relationships Cleanse it to assure its quality and consistency Transform it to provide enriched and tailored information, and; Federate it to make it accessible to people, processes, and applications All of these functions are based on a parallel processing infrastructure that provides leverage and automation across the platform. The Information Server also provides connectivity to nearly any data or content source, and the ability to deliver information through a variety of mechanisms. Underlying these functions is a unified metadata management foundation that provides seamless sharing of knowledge throughout a project lifecycle, along with a detailed understanding of what information means, where it came from, and how it is related to information in other systems. Integration logic built within IBM Information Server can easily be deployed and managed as a shared service within a SOA. IBM Information Server provides: access to the broadest range of information sources the broadest range of integration functionality, including federation, ETL, in-line transformation, replication, and event publishing the most flexibility in how these functions are used, including support for service-oriented architectures, event-driven processing, scheduled batch processing, and even standard APIs like SQL and Java. The breadth and flexibility of the platform enable it to address many types of business problems and meet the requirements of many types of projects. This optimizes the opportunities for reuse, leading to faster project cycles, better information consistency, and stronger information governance. Regarding Service-Oriented Architectures, information integration enables information to be made available as a service, publishing consistent, reusable services for information that make it easier for processes to get the information they need from across a heterogeneous landscape.
• #5: Data Discovery is about reducing project costs and risk by discovering problems in the design stage of your process. In many legacy systems and enterprise applications, meta data, field usage and general knowledge has changed over time. The data may be perfectly acceptable for whatever purpose it was designed for, but its often not until you load it into another application that you discover how inappropriate it is for what you want to do. Companies today are moving toward greater integration. As they try and become more customer centric or real time, management realizes that data must be treated as a corporate asset and not a division or business unit tool. Unfortunately, many sources will not be in the right form or have the right meta data or even documentation to allow a quick integration for other uses.
• #6: ProfileStage will automatically scan a sample of your data to determine the quality and structure of the data. When you have made the adjustments you need, the product will then process all of the data to create a data model and an ETL job to extract the information is a form that is useable. Corporate data degrades very quickly. Particularly if you are trying to maximize its use across applications. ProfileStage enables you to discover what data you actually have so that you reduce the cost and time in developing new processes and enables you to leverage the value of legacy systems in new and valuable ways.. ProfileStage automates the data discovery process. Because you can analyze all of your data, you can eliminate the risk and cost of using inappropriate data, before you begin your development process. ProfileStage can be used in any type of integration process and we have found that it reduces the time to analyze data by 70%. This is very important because, many organizations will not do a complete data check because they fear missing their deadlines for project completion. In fact, the reverse is true. By taking the time to analyze the data our clients discover the valuable links and relationships in their data before development is wasted. And they are able to utilize legacy data in more projects because they have a methodology that ensures its proper use.
• #7: Reduce Time to Value of Data Projects Fully understanding data content across the enterprise and throughout the development lifecycle 30% cost savings of project life cycle Analyst studies have shown that over 75% of data integration projects either overrun or fail. One of the main contributors to this is the assumption that the data required for the new application is actually available from the source and it’s in the form the user thinks it’s in, in short, the lack of understanding of the metadata can seriously compromise a data integration project. “…a defect that isn’t detected upstream (during requirements or design) will cost from 10 to 100 times as much to fix Downstream.” Steve McConnell, RAPID DEVELOPMENT, Microsoft Press Increase the Productivity of Data Personnel Data Personnel = Business and Data Analysts, Data Quality Specialists, Data Architects and Data Stewards Intuitive task based user interface Maintain Data Quality Consistency across the Enterprise via an intuitive, visual, ongoing display of pertinent data metrics Monitoring to ensure data quality throughout the lifecycle and not just during initial implementation Integration with information integration tools Results can be handed to DataStage or Quality Stage
• #8: So what does it do ?
• #13: – Single Column Key Duplicates
• #14: – View Analysis Details Reviewing Cross Domain Analysis, View Analysis Details Shows a detailed comparison of the paired column to the base column Can view comparison of domain values (frequency distributions) Can mark the paired column as redundant with the base column
• #15: – View Baseline Analysis
• #17: Headquartered in Grand Rapids, Michigan, this company is the largest food distributor in the US. As part of there aggressive “growth through acquisition” strategy, the company has complete or is in the process of acquiring 4 companies over the last 2 years. This has expanded their geographic coverage to include most of continental US and Canada, but also created massive financial management issues. Information regarding 12,000 branded products; 30,000 clients; and 11 operating centers needed to be consolidated into single set of purchasing, inventory/fleet management, CRM and financial reporting systems. Initial efforts to consolidate acquired companies focused on integration of supply chain management systems, profit margin analysis systems, field expansion, and take along project. These projects consistently ran over budget and time. The single greatest problem was the lack of documentation and staff changes relative to legacy systems in acquired companies. Even when analyzing only 7% of source data, source data discovery problems were driving 20% of all cost overruns for all IT projects. Without profiling tools, they anticipated needing to spend 10K hours, $.65M in costs, to support 4 initial projects. In Q2 2003, they began using ProfileStage. It is helping them initially profile source systems more quickly; create ETL jobs the first time around that correct address source data issues/structure; find and exploit more and more powerful relationships in source data structures; and ultimately improve use/value from final integrated data through better understanding of the source information. On an ongoing basis they plan to use the metadata describing the sources and ETL jobs to more quickly extend or modify the solution. They are seeing a 80% productivity gain through use of profiling solution over manual source system discovery, which they expect will result in $504K annual savings in lower development and maintenance costs.
• #18: Problem: New York Life executives implemented and employed two competitive advantages – a 360-degree view of their customers and data accessible 24\7 via their intranet. The executives plan to amplify these two competitive advantages to reduce costs, maximize profits, and enhance market share. While New York Life can identify individual policyholders owning multiple policies, detailed data is not readily available. The detail data exists in eight to ten disparate legacy systems, each supporting different departments and represented by different (poorly documented) codes and formats. To effectively manage, cross-sell, and up-sell New York Life needs to integrate and standardize the policyholder data. Moreover addressing competitive pressures necessitates New York Life expose their data 24\7 to executive management, brokers, and agents via the intranet. Their poorly documented data could not be exposed to the intranet without inflicting more harm than effecting good. Solution: To accomplish a 360-degree view of the customer data New York Life is using IBM to build an Operational Data Store (ODS) and Corporate Data Warehouse (CDW). Meeting the business-reporting requirements has necessitated exposing data to the intranet on a 24\7 basis. While building the ODS and CDW, IBM has enabled New York Life to establish consistent data formats; document suitable business codes across policy applications; validate business data domains; identify empty columns; define business rules linking policy data and more. Benefits: Employing IBM established a complete understanding the data and provided a 360-degree view of the customers. Having the improved 360-customer view reduces costs by eliminating expensive and time-consuming investigations of detailed data within legacy systems. The success of the CDW with accurate, complete, and timely data has begot more success. Departments within New York Life have flocked to the application as word of the usefulness and timeliness of the CDW data has spread. More people (i.e., agents, brokers, executive management) using the CDW have reduced costs and increased productivity. Costs are reduced because IBM has enabled New York Life to identify (and thus eliminate) disparate and incoherent data. Now multiple departments may leverage a single resource, the CDW. Productivity has increased, via IBM usage, because data within CDW is correct and end users (i.e., executive management) can focus upon leveraging the CDW rather than validating the CDW data. Validating detailed data with the ten disparate legacy systems had required approximately one person per legacy system. Since the successful implementation of CDW these individuals focus on new, more productive assignments. New York Life no longer spends the approximate $130,000 fully loaded cost of an employee to validate and cross-reference the detailed data. The CDW houses the necessary data.