Irmac presentation for website

Editor's Notes

• #5: According to Gartner Group: “Predictive Analysis helps you connect data to effective action by drawing reliable conclusions about current conditions and future events.” Predictive analysis: Leverages an organization’s business knowledge by applying sophisticated analytic techniques to enterprise data It turns that data into insights that lead to the development of programs to increase revenues, reduce costs, improve processes, and prevent criminal or fraudulent activities It encourage actions that demonstrably change how people behave as your customers, employees, patients, students, and citizens Bottom line: it turns data into effective actions that positively impact your bottom line
• #6: Here are some of the stats you may want to know about SPSS (read highlights from the slide). SPSS has been a cornerstone of the software industry since 1968. We’ve also been on the forefront of blending both new and established technologies to help customers around the world solve business problems. We’ve continued to grow, deliberately and thoughtfully, over the years, acquiring companies and technology complimentary to our existing business. The bottom line for you: We’re here with innovative, proven solutions to help you solve your immediate business problems. And, we’ll be here in the future to support you and your organization…
• #7: The Clementine Server data mining workbench fits within SPSS’ overall business intelligence product strategy. Our entire business intelligence product line includes products for collecting data, preparing data, reporting and OLAP, as well as modeling. Because different users have different needs and levels of expertise, they are presented with the appropriate product interface. SPSS delivers the right product for every person supported by our 30 years of experience in data analysis and data mining. The analytical solutions we deliver, are scalable and can be deployed throughout your organization to help you transform your business with information..
• #9: It’s been estimated that 80% of the data in an organization is an unstructured format, that is, in the form of documents, HTML pages, database notes, email, open-ended survey responses, etc. This fact means that decision-makers often rely on only 20% of the data available and a little bit of the documents that they can read. Take open-ended surveys, for example: cross-tab reports of responses are common but open-ended responses, which hold valuable information which qualify the responses and bring up new themes. Organizations rarely have the tools or the time to truly process and disseminate this important information. In a similar fashion, database notes on customer contacts are effectively used to manage individual contacts, but this valuable source of customer information is never used to really understand the customer experience overall. What if you could use this information to keep and grow customers to increase customer lifetime value?
• #10: So, I think we can agree that there is a need for text analysis, but, where can this technology be applied. [click] Well, surveys are the most obvious, we’ve just talked about that. [click] We could apply this to email, reading the email and making a handling decision based on the content of the email. [click] Call centre data is another candidate for text analysis. What are my customers complaints? Where are there problems with my product? What did the customers who left have to say about my service? [click] Reading comment data is an important potential application; an application that is laborious or subjective currently. In the State of Georgia example, comments are triaged and only those indicating a definite problem or requirement for re-arrest are used. The majority are ignored, even though there is a real sense that there is something in that group. [click] The ability to read abstracts from online databases using a more intelligent engine than a simple word search is an application. [click] Document management and the ability to categorize gigabytes of documents policies, procedures is an application, and along with that [click] Corporate history and the ability to manage corporate information resources is an application. Finally [click] we have seen some use of this technology in the analysis of message in websites. What concepts are your website conveying, and are these concepts the appropriate ones, appropriately placed?
• #14: At a high level, you need linguistics or Natural Language Processing, to extract concepts which form the bases of business user interfaces like concept maps or feeding data mining techniques to predict customer behavior.
• #15: Morphology is the study of the structure and form of words Syntax is the study of how words and phrases form sentences Semantics relates to the meaning of words and statements Phonology is the study of sounds in language Pragmatics is the study of idiomatic phrases that cannot be analyzed with strict semantic analysis We tend to deal with the first three and ignore the last two when we are talking about natural language processing.
• #22: So how do we get from text to concepts? Linguistics, the science of text, includes ideas such as 1) morphology, or how words change based on part of speech, 2) syntax, or how sentences are structured 3) semantics, or the meaning of words and 4) statistics, such as the frequency of terms and patterns. It takes linguistics to cut through the noise of text to find relevant concepts without leaving important concepts undiscovered. Other statistical or machine learning approaches fall short of linguistic extraction, because only a linguistic approach can deal with the ambiguity and complexity of text. That is, linguistic extraction is [click] accurate, [click] scalable, [click] customizable and [click] discovery oriented. By accurate, I mean that [click] compound words, proper nouns, etc., [click] like these examples, are extracted. [click] In terms of scalability, we can process about 1 GB per hour, multiple formats and multiple languages [click]. By customizable, [click] I mean that you can you use dictionaries, rules and patterns [click] to tailor your extraction. Vertical resources can be used like the MeSH which is the official medical thesaurus. And Finally, [click] by discovery-oriented, I mean that, depending on your analysis, you can focus on known terms, unknown terms, new terms and [click] trends.
• #23: For the next step, to move from concepts to Predictive Analytics, tools are available which address specific business needs by delivering knowledge to adding prediction to operational systems. [click] LexiQuest Mine enables users to quickly identify key concepts, and the relationships between them, within thousands of documents Mine displays these concepts and the links between them in an easy to navigate, color-coded graphical map and trend analysis charts. Mine is designed for people who want to discover, structure and anticipate. [click] LexiQuest Categorize automatically catalogues documents into a predefined taxonomy based on their content. Able to “read” and understand content, Categorize is able to automatically and accurately place a document into into its proper category. From there, it can be sent to the right audience based on their profile or simply reside there for easy retrieval from a portal, intranet or extranet site. [click] Text Mining for Clementine is a new component of Clementine, which we will see in a few minutes, has the ability to unlock knowledge contained in unstructured text data so that it can be combined with information from databases and other data sources to build better models using traditional data mining techniques.
• #24: The extraction process works basically as three parts: First, a linguistic processor reads the text and comes up with a set of categories. These categories are passed to one of three different applications; depending upon the objective. These applications may be a stand alone concept understanding application, such as our LexiQuest mine. This application represents the concepts, and illustrates their relationship. Our Clementine application uses the concepts as data, and, as part of a larger data mining application. Finally, categorization uses the concept information as the basis of further analysis The final application layer can be used for visualization, data mining, or strict probabilistic assignment of information to known categories.
• #29: Another text mining example is categorization. The folders on the left represent categories of different types of incoming emails. Text mining can be used to learn the which emails, depending on their content, should be placed in each category (and therefore routed appropriately and automatically). [click] In this case an email on a problem with an ActiveX control [click] can be routed to Dev support.
• #30: MeSH is the National Library of Medicine's controlled vocabulary thesaurus. It consists of sets of terms naming descriptors in a hierarchical structure that permits searching at various levels of specificity. MeSH descriptors are arranged in both an alphabetic and a hierarchical structure. At the most general level of the hierarchical structure are very broad headings such as "Anatomy" or "Mental Disorders." At more narrow levels are found more specific headings such as "Ankle" and "Conduct Disorder." There are 21,973 descriptors in MeSH. There are also thousands of cross-references that assist in finding the most appropriate MeSH Heading, for example, Vitamin C see Ascorbic Acid. These entries include 23,512 printed see references and 102,346 other entry points
• #31: Let’s define data mining “The process of discovering meaningful new relationships, patterns and trends by sifting through data using pattern recognition technologies as well as statistical and mathematical techniques.” Data Mining Means: finding patterns or relationships in your data that you can use to solve your organization’s problems
• #33:   How does one mine data? The CROSS INDUSTRY STANDARD PROCESS FOR DATA MINING provides a framework for all data mining efforts.. This process focuses on business issues, allows the user to work with and interact with the data, works on the data mining process from beginning to end, and USES the results.  Business Understanding where you might convert a business problem to a data mining problem Data Understanding – where you get your first look at the data Data Preparation – the hardest part where you clean the data Modeling – the neatest part where you build prediction or MODELS Evaluation – where you examine performance Deployment – where you actually integrate the results of your data mining into your organization Notices the arrows going around the chart and and back and forth amongst the boxes? These arrows show that the data mining process is an iterative process where the miner may step from on box to another an back for an effective data mining activity
• #42: Broadly speaking data mining can serve four basic purposes: prediction, segmentation, association and outlier detection. Prediction takes a known result, and attempts to combine input fields in order to best replicate this result. An example may be deciding if someone is a good or bad credit risk, or, whether someone will churn or not. Segmentation finds groups within cases. The number of groups is unknown, ie. Any number of groups is possible. An example may be the examination of customer segments, or, the creation of groupings within financial data Association methods try to develop an “if this, then that” type of analysis. Examples of this are people who watch news programming also watch the weather network. Outlier detection is used to derive atypical cases. These are examples of unusual behaviour vis-s-vis the rest of the data. Examples of this may be fraud detection when examining claims data.
• #43: The ability to move to the why from the other who, what, when, where, and how much Personalization for customer knowledge. This allows for marketers to craft messages aimed at more individuals than generalized groups Understanding events inside and outside the organization and how they relate The ability to assess competitors technology and to understand the technology positions of the market
• #46: Questions