Abstract image of a woman sitting on books and studying on a laptop

Filter2020 Spring2019 Fall2019 SummerCurr.docx

Download as DOCX, PDF
C
charlottej5

The document outlines courses offered in Spring 2020 and Fall 2019, focusing on applied learning and data science. It emphasizes the significance of big data analytics in managing large datasets and decision-making, detailing course objectives and expected outcomes. Additionally, it provides instructions for an article review assignment, including sections on summarization, critique, and questions.

Education
1 of 30
Download to read offline
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
/ Filter 2020 Spring 2019 Fall 2019 Summer Current Courses Search your courses All C Spring 2020 - Applied Learning Practicum (INTR-799-06) - Full TermPamela SmithMore info Spring 2020 - Data Science & Big Data Analy (ITS-836-52) - Full TermMultiple Instructors More info Spring 2020 - Infer Stats in Decision-Making (DSRT-734-05) - Second Bi-TermMultiple Instructors Fall 2019 - Applied Learning Practicum (INTR-799-03) - Full TermPamela SmithMore info Fall 2019 - InfoTech Import in Strat Plan (ITS-831-05) - First Bi-TermMultiple Instructors More info Spring 2020 - Data Science & Big Data Analy (ITS-836-52) - Full Term
InformationSpring 2020 - Data Science & Big Data Analy (ITS- 836-52) - Full Term Information Big Data Analytics iden�fy fundamental concepts of Big Data management and analy�cs. become competent in recognizing challenges faced by applica�ons dealing with very large volumes of data as well as in proposing scalable solu�ons for them. be able to understand how Big Data impacts business intelligence, scien�fic discovery, and our day-to-day life. Course Descrip�on: In this course the students explore key data analysis and management techniques, which applied to massive datasets are the cornerstone that enables real-�me decision making in distributed environments, business intelligence in the Web, and scien�fic discovery at large scale. In par�cular, students examine the map-reduce parallel compu�ng paradigm and associated technologies such as distributed file systems, no-sql databases, and stream compu�ng engines. This highly interac�ve course is based on the problem-based learning philosophy. Students are expected to make use of technologies to design highly scalable systems that can process and analyze Big Data for a variety of scien�fic, social, and environmental challenges.
Course Objec�ves/Learner Outcomes: Course Objec�ves/Learner Outcomes: Upon comple�on of this course, the student will: Prerequisites: There are no prerequisites for this course. Books and Resources: Required Text EMC Educa�on Service (Eds). (2015) Data Science and Big Data Analytics: Discovering, Analyzing, Visualizing, and Presenting Data, Indianapolis, IN: John Wiley & Sons, Inc ×× javascript:void(0); javascript:void(0); javascript:void(0); javascript:void(0); javascript:void(0); javascript:void(0); https://ucumberlands.blackboard.com/webapps/blackboard/exec ute/courseMain?course_id=_114565_1 CSCI 561 Article Review Instructions
After reading through your assignments this week, you are to pick a topic of interest that was mentioned in the reading assignment. Using the Jerry Falwell Library and other scholarly resources, you are to locate a peer reviewed journal related to the topic of your interest. Read the journal article thoroughly so you can discuss it. If you wish to use something other than a peer reviewed journal, please consult with your instructor before starting the assignment. You will then prepare an article review related to the topic that you researched in the research/reading assignments from the assigned module/week. The article review that you create must be at least 750words and be formatted according to APA style. Use the following section order to guide you in building your article review but do not use the numbers in your section headers, only the names. Also, this is example is not in APA style, make sure your review follows APA styles, this is a guide NOT a template. 1. Bibliographical Reference: a. Create a single bibliographical entry in APA style in the first section of your paper. This should include: i. Detail the author(s) of the article ii. Name of the Journal (including volume, issue, year, page numbers, etc.) iii. Name of the article you are reviewing. b. Note – You may also choose to identify the article you are reviewing (i.e. – The focus of this review will be “Article Name” published in “Journal Name” in “Year/Month”) and then simply cite it in a bibliographical entry at the end (see section 7). 2. Objectives: a. After reviewing the article, use a bulleted list to identity the 3 to 5 primary points you feel the article addresses. b. Simply put the bulleted list of those points in your Objectives
section with something to the effect of “After reviewing “Article Name” the three main points addressed by the author(s) were: · Bullet one · Bullet two · Bullet three 3. Summary: a. Summarize the article you reviewed in approximately 2 or 3 full paragraphs. b. Do not use direct quotes at all. If you need to quote it, paraphrase what you read. c. Use the bullets above (section 2b) to structure what you wish to discuss in this summary (i.e. – In the first paragraph, summarize the article section which addresses the first bullet in 2b and so on). 4. Results: a. Now that you have read this article and summarized it, what do you feel that a reader can learn about the topic from the article? b. Using bullet points, highlight the things you feel like you learned more about after reading this article. It’s best to keep this list to a maximum of 5 bullets. c. Note – these bullets should not be identical to the bullets in the Objective Section (2b). 5. Critique: a. Use this section to provide an academic critique of the article. i. How well (or poorly) was this article written? ii. Did it accomplish the objectives it set out to do (think back to the Objectives section)? iii. Why or why not? b. Explain your opinion and offer some additional sources (no
more than one or two) to support that opinion. c. When possible, address the subject matter from a biblical perspective (e.g. – the Bible teaches us to… and this article addresses that by…) 6. Questions: a. List a minimum of 3 (no more than 5) questions that arose from the reading of this article in a numbered list. b. Consider any thing you feel like you should have learned from the article but did not. c. Note – These questions may spark your desire to review other matter on the topic and may even be used as a starting point for one of your two research papers (though this is not required). 7. Bibliography: a. You need to use one other sources to support or deny your opinion and if you chose to only identify the article in section 1, you will want to make a bibliography. b. Follow APA guidelines for Bibliographies as outlined in the APA 6th Edition Guide. IMPORTANT – Article reviews are DUE by 11:59ET on Sunday of the assigned Module/Week! ffirs.indd 2:43:16:PM/12/11/2014 Page i Data Science & Big Data Analytics
ffirs.indd 2:43:16:PM/12/11/2014 Page iii Discovering, Analyzing, Visualizing and Presenting Data Data Science & Big Data Analytics EMC Education Services ffirs.indd 2:43:16:PM/12/11/2014 Page iv Data Science & Big Data Analytics: Discovering, Analyzing, Visualizing and Presenting Data Published by John Wiley & Sons, Inc. 10475 Crosspoint Boulevard Indianapolis, IN 46256 www.wiley.com Copyright © 2015 by John Wiley & Sons, Inc., Indianapolis, Indiana Published simultaneously in Canada ISBN: 978-1-118-87613-8
ISBN: 978-1-118-87622-0 (ebk) ISBN: 978-1-118-87605-3 (ebk) Manufactured in the United States of America 10 9 8 7 6 5 4 3 2 1 No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except as permitted under Sections 107 or 108 of the 1976 United States Copyright Act, without either the prior written permis- sion of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 646-8600. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748-6011, fax (201) 748-6008, or online at http://www.wiley.com/go/permissions. Limit of Liability/Disclaimer of Warranty: The publisher and the author make no representations or warranties with respect to the accuracy or completeness of the contents of this work and specifically disclaim all warranties, including without limitation warranties of fitness for a particular purpose. No warranty may be
created or extended by sales or promotional materials. The advice and strategies contained herein may not be suitable for every situation. This work is sold with the understanding that the publisher is not engaged in rendering legal, accounting, or other professional services. If professional assistance is required, the services of a competent professional person should be sought. Neither the publisher nor the author shall be liable for damages arising herefrom. The fact that an organization or Web site is referred to in this work as a citation and/or a potential source of further information does not mean that the author or the publisher endorses the information the organization or website may provide or recommendations it may make. Further, readers should be aware that Internet websites listed in this work may have changed or disappeared between when this work was written and when it is read. For general information on our other products and services please contact our Customer Care Department within the United States at (877) 762-2974, outside the United States at (317) 572-3993 or fax (317) 572-4002. Wiley publishes in a variety of print and electronic formats and by print-on-demand. Some material included with standard print versions of this book may not be included in e-books or in print-on-demand. If this book refers to media such as a CD or DVD that is not included in the version you purchased, you may download
this material at http://booksupport.wiley.com. For more information about Wiley products, visit www.wiley.com. Library of Congress Control Number: 2014946681 Trademarks: Wiley and the Wiley logo are trademarks or registered trademarks of John Wiley & Sons, Inc. and/or its affiliates, in the United States and other coun- tries, and may not be used without written permission. All other trademarks are the property of their respective owners. John Wiley & Sons, Inc. is not associated with any product or vendor mentioned in this book. http://www.wiley.com http://www.wiley.com/go/permissions http://booksupport.wiley.com http://www.wiley.com ffirs.indd 2:43:16:PM/12/11/2014 Page v Credits Executive Editor Carol Long Project Editor Kelly Talbot Production Manager
Kathleen Wisor Copy Editor Karen Gill Manager of Content Development and Assembly Mary Beth Wakefield Marketing Director David Mayhew Marketing Manager Carrie Sherrill Professional Technology and Strategy Director Barry Pruett Business Manager Amy Knies Associate Publisher Jim Minatel Project Coordinator, Cover Patrick Redmond
Proofreader Nancy Carrasco Indexer Johnna VanHoose Dinse Cover Designer Mallesh Gurram ffirs.indd 2:43:16:PM/12/11/2014 Page vi About the Key Contributors David Dietrich heads the data science education team within EMC Education Services, where he leads the curriculum, strategy and course development related to Big Data Analytics and Data Science. He co-au- thored the first course in EMC’s Data Science curriculum, two additional EMC courses focused on teaching leaders and executives about Big Data and data science, and is a contributing author and editor of this book. He has filed 14 patents in the areas of data science, data privacy, and cloud computing. David has been an advisor to several universities looking to develop academic programs related to data
analytics, and has been a frequent speaker at conferences and industry events. He also has been a a guest lecturer at universi- ties in the Boston area. His work has been featured in major publications including Forbes, Harvard Business Review, and the 2014 Massachusetts Big Data Report, commissioned by Governor Deval Patrick. Involved with analytics and technology for nearly 20 years, David has worked with many Fortune 500 companies over his career, holding multiple roles involving analytics, including managing analytics and operations teams, delivering analytic con- sulting engagements, managing a line of analytical software products for regulating the US banking industry, and developing Software-as-a-Service and BI-as-a-Service offerings. Additionally, David collaborated with the U.S. Federal Reserve in develop- ing predictive models for monitoring mortgage portfolios. Barry Heller is an advisory technical education consultant at EMC Education Services. Barry is a course developer and cur- riculum advisor in the emerging technology areas of Big Data and data science. Prior to his current role, Barry was a consul- tant research scientist leading numerous analytical initiatives within EMC’s Total Customer Experience organization. Early in his EMC career, he managed the
statistical engineering group as well as led the data warehousing efforts in an Enterprise Resource Planning (ERP) implementation. Prior to joining EMC, Barry held managerial and analytical roles in reliability engineering functions at medical diagnostic and technology companies. During his career, he has applied his quantitative skill set to a myriad of business applications in the Customer Service, Engineering, Manufacturing, Sales/Marketing, Finance, and Legal arenas. Underscoring the importance of strong executive stakeholder engagement, many of his successes have resulted from not only focusing on the technical details of an analysis, but on the decisions that will be resulting from the analysis. Barry earned a B.S. in Computational Mathematics from the Rochester Institute of Technology and an M.A. in Mathematics from the State University of New York (SUNY) New Paltz. Beibei Yang is a Technical Education Consultant of EMC Education Services, responsible for developing several open courses at EMC related to Data Science and Big Data Analytics. Beibei has seven years of experience in the IT industry. Prior to EMC she worked as a software engineer, systems manager, and network manager for a Fortune 500 company where she introduced
new technologies to improve efficiency and encourage collaboration. Beibei has published papers to prestigious conferences and has filed multiple patents. She received her Ph.D. in computer science from the University of Massachusetts Lowell. She has a passion toward natural language processing and data mining, especially using various tools and techniques to find hidden patterns and tell stories with data. Data Science and Big Data Analytics is an exciting domain where the potential of digital information is maximized for making intelligent business decisions. We believe that this is an area that will attract a lot of talented students and professionals in the short, mid, and long term. ffirs.indd 2:43:16:PM/12/11/2014 Page vii Acknowledgments EMC Education Services embarked on learning this subject with the intent to develop an “open” curriculum and certification. It was a challenging journey at the time as not many understood what it would take to be a true data scientist. After initial research (and struggle), we were able to define what was needed and attract very
talented professionals to work on the project. The course, “Data Science and Big Data Analytics,” has become well accepted across academia and the industry. Led by EMC Education Services, this book is the result of efforts and contributions from a number of key EMC organizations and supported by the office of the CTO, IT, Global Services, and Engineering. Many sincere thanks to many key contributors and subject matter experts David Dietrich, Barry Heller, and Beibei Yang for their work developing content and graphics for the chapters. A special thanks to subject matter experts John Cardente and Ganesh Rajaratnam for their active involvement reviewing multiple book chapters and providing valuable feedback throughout the project. We are also grateful to the following experts from EMC and Pivotal for their support in reviewing and improving the content in this book: Aidan O’Brien Joe Kambourakis Alexander Nunes Joe Milardo Bryan Miletich John Sopka Dan Baskette Kathryn Stiles
Daniel Mepham Ken Taylor Dave Reiner Lanette Wells Deborah Stokes Michael Hancock Ellis Kriesberg Michael Vander Donk Frank Coleman Narayanan Krishnakumar Hisham Arafat Richard Moore Ira Schild Ron Glick Jack Harwood Stephen Maloney Jim McGroddy Steve Todd ffirs.indd 2:43:16:PM/12/11/2014 Page viii Jody Goncalves Suresh Thankappan Joe Dery Tom McGowan We also thank Ira Schild and Shane Goodrich for coordinating this project, Mallesh Gurram for the cover design, Chris Conroy and Rob Bradley for graphics, and the publisher, John Wiley and Sons, for timely support in bringing this book to the industry. Nancy Gessler
Director, Education Services, EMC Corporation Alok Shrivastava Sr. Director, Education Services, EMC Corporation ftoc.indd 7:24:10:PM/12/12/2014 Page ix Contents Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.1 Big Data Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.1.1 Data Structures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1.1.2 Analyst Perspective on Data Repositories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 1.2 State of the Practice in Analytics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 1.2.1 BI Versus Data Science . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.2.2 Current Analytical Architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 1.2.3 Drivers of Big Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 1.2.4 Emerging Big Data Ecosystem and a New Approach to Analytics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 1.3 Key Roles for the New Big Data Ecosystem. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 1.4 Examples of Big Data Analytics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 2.1 Data Analytics Lifecycle Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
26 2.1.1 Key Roles for a Successful Analytics Project . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 2.1.2 Background and Overview of Data Analytics Lifecycle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 2.2 Phase 1: Discovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 2.2.1 Learning the Business Domain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .30 2.2.2 Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 2.2.3 Framing the Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 2.2.4 Identifying Key Stakeholders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 2.2.5 Interviewing the Analytics Sponsor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 2.2.6 Developing Initial Hypotheses. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
2.2.7 Identifying Potential Data Sources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 2.3 Phase 2: Data Preparation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 2.3.1 Preparing the Analytic Sandbox . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 2.3.2 Performing ETLT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .38 2.3.3 Learning About the Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 2.3.4 Data Conditioning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .40 2.3.5 Survey and Visualize . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 2.3.6 Common Tools for the Data Preparation Phase . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 2.4 Phase 3: Model Planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
2.4.1 Data Exploration and Variable Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .44 2.4.2 Model Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 2.4.3 Common Tools for the Model Planning Phase . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 ftoc.indd 7:24:10:PM/12/12/2014 Page x CONTENTSx 2.5 Phase 4: Model Building. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 2.5.1 Common Tools for the Model Building Phase . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .48 2.6 Phase 5: Communicate Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 2.7 Phase 6: Operationalize . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 2.8 Case Study: Global Innovation Network and Analysis (GINA). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 2.8.1 Phase 1: Discovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .54 2.8.2 Phase 2: Data Preparation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 2.8.3 Phase 3: Model Planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .56 2.8.4 Phase 4: Model Building . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 2.8.5 Phase 5: Communicate Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .58 2.8.6 Phase 6: Operationalize . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
3.1 Introduction to R. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 3.1.1 R Graphical User Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 3.1.2 Data Import and Export. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 3.1.3 Attribute and Data Types. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 3.1.4 Descriptive Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 3.2 Exploratory Data Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80 3.2.1 Visualization Before Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .82 3.2.2 Dirty Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .85 3.2.3 Visualizing a Single Variable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .88
3.2.4 Examining Multiple Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 3.2.5 Data Exploration Versus Presentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .99 3.3 Statistical Methods for Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 3.3.1 Hypothesis Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .102 3.3.2 Difference of Means . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 3.3.3 Wilcoxon Rank-Sum Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 3.3.4 Type I and Type II Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .109 3.3.5 Power and Sample Size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 3.3.6 ANOVA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 . . . . . . . . . . . . . . . . . . . . . . . . . 117 4.1 Overview of Clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118 4.2 K-means . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118 4.2.1 Use Cases. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 4.2.2 Overview of the Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .120 4.2.3 Determining the Number of Clusters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .123 4.2.4 Diagnostics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . .128 ftoc.indd 7:24:10:PM/12/12/2014 Page xi CONTENTS xi 4.2.5 Reasons to Choose and Cautions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .130 4.3 Additional Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136 . . . . . . . . . . . . . . . . . . 137 5.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138 5.2 Apriori Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
140 5.3 Evaluation of Candidate Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141 5.4 Applications of Association Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143 5.5 An Example: Transactions in a Grocery Store. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143 5.5.1 The Groceries Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .144 5.5.2 Frequent Itemset Generation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .146 5.5.3 Rule Generation and Visualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .152 5.6 Validation and Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157 5.7 Diagnostics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160 . . . . . . . . . . . . . . . . . . . . . . . . 161 6.1 Linear Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162 6.1.1 Use Cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .162 6.1.2 Model Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .163 6.1.3 Diagnostics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .173 6.2 Logistic Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178 6.2.1 Use Cases. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .179
6.2.2 Model Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . …

More Related Content

DOCX
Agg PlansThe owner of a small mill-working plant that builds cabin.docx
nettletondevon
 
PDF
Engl317 project3 slidedoc2_howto_createyourinfographic
Zachary Williamson
 
DOCX
System and Problem for a Library Management System .docx
mattinsonjanel
 
PDF
Cbr 600 Believe Possibilities / snaptutorial.com
Davis5a
 
PDF
Cbr 600 Education Organization-snaptutorial.com
robertlesew1
 
DOCX
Cbr 600 Exceptional Education - snaptutorial.com
DavisMurphyA87
 
DOCX
Corporate Financial Strategy (BUSI4402) 202021 Individual Cours
AlleneMcclendon878
 
PDF
CBR 600 RANK Remember Education--cbr600rank.com
chrysanthemu45
 
Agg PlansThe owner of a small mill-working plant that builds cabin.docx
nettletondevon
 
Engl317 project3 slidedoc2_howto_createyourinfographic
Zachary Williamson
 
System and Problem for a Library Management System .docx
mattinsonjanel
 
Cbr 600 Believe Possibilities / snaptutorial.com
Davis5a
 
Cbr 600 Education Organization-snaptutorial.com
robertlesew1
 
Cbr 600 Exceptional Education - snaptutorial.com
DavisMurphyA87
 
Corporate Financial Strategy (BUSI4402) 202021 Individual Cours
AlleneMcclendon878
 
CBR 600 RANK Remember Education--cbr600rank.com
chrysanthemu45
 

Similar to Filter2020 Spring2019 Fall2019 SummerCurr.docx (20)

DOCX
Cbr 600Education Specialist / snaptutorial.com
McdonaldRyan73
 
PDF
CBR 600 RANK Introduction Education--cbr600rank.com
agathachristie262
 
DOCX
CBR 600 RANK Inspiring Innovation--cbr600rank.com
KeatonJennings104
 
PDF
CBR 600 RANK Become Exceptional--cbr600rank.com
agathachristie107
 
DOCX
Cbr 600 Extraordinary Success/newtonhelp.com
amaranthbeg140
 
DOC
Cbr 600 Enhance teaching / snaptutorial.com
Baileyabq
 
DOC
CBR 600 Focus Dreams/newtonhelp.com
bellflower79
 
DOCX
CBR 600 Effective Communication - snaptutorial.com
donaldzs1
 
DOC
CBR 600 Imagine Your Future/newtonhelp.com   
bellflower39
 
DOC
CBR 600 Life of the Mind/newtonhelp.com   
llflowerbe
 
PPTX
Engl313 ada project4_slidedoc2
coop3674
 
DOCX
ACCT 424 Advanced Accounting Final Course Report Resea.docx
bartholomeocoombs
 
DOCX
TITLE OF PROPOSAL [typed in all capital letters, double-space.docx
gertrudebellgrove
 
DOCX
Digital portfolio 1_v2
mustafaalinike
 
PDF
Com 106 Enthusiastic Study / snaptutorial.com
Stephenson36
 
PPTX
report writing & references, how to write.pptx
dabam90710
 
DOC
Com 106 Success Begins / snaptutorial.com
WilliamsTaylorza50
 
DOCX
A step by step guide to report writing Step 1 Choose your top.docx
annetnash8266
 
DOCX
Davenport university busn 650
Christina Walkar
 
DOC
COM 106 Technology levels--snaptutorial.com
sholingarjosh82
 
Cbr 600Education Specialist / snaptutorial.com
McdonaldRyan73
 
CBR 600 RANK Introduction Education--cbr600rank.com
agathachristie262
 
CBR 600 RANK Inspiring Innovation--cbr600rank.com
KeatonJennings104
 
CBR 600 RANK Become Exceptional--cbr600rank.com
agathachristie107
 
Cbr 600 Extraordinary Success/newtonhelp.com
amaranthbeg140
 
Cbr 600 Enhance teaching / snaptutorial.com
Baileyabq
 
CBR 600 Focus Dreams/newtonhelp.com
bellflower79
 
CBR 600 Effective Communication - snaptutorial.com
donaldzs1
 
CBR 600 Imagine Your Future/newtonhelp.com   
bellflower39
 
CBR 600 Life of the Mind/newtonhelp.com   
llflowerbe
 
Engl313 ada project4_slidedoc2
coop3674
 
ACCT 424 Advanced Accounting Final Course Report Resea.docx
bartholomeocoombs
 
TITLE OF PROPOSAL [typed in all capital letters, double-space.docx
gertrudebellgrove
 
Digital portfolio 1_v2
mustafaalinike
 
Com 106 Enthusiastic Study / snaptutorial.com
Stephenson36
 
report writing & references, how to write.pptx
dabam90710
 
Com 106 Success Begins / snaptutorial.com
WilliamsTaylorza50
 
A step by step guide to report writing Step 1 Choose your top.docx
annetnash8266
 
Davenport university busn 650
Christina Walkar
 
COM 106 Technology levels--snaptutorial.com
sholingarjosh82
 

More from charlottej5 (20)

DOCX
Final PaperIt was a pleasure to be with you all and you made it fu.docx
charlottej5
 
DOCX
Final PaperOne reason that California have been known as the Gol.docx
charlottej5
 
DOCX
Final PaperIndia and China provide two fascinating country case st.docx
charlottej5
 
DOCX
Final PaperMust begin with an introductory paragraph that has a .docx
charlottej5
 
DOCX
Final PaperFinal Paper SynopsisThe purpose of this paper .docx
charlottej5
 
DOCX
Final Paper, Essay Proposal, & Outline·The final essay should .docx
charlottej5
 
DOCX
Final Paper – Possible topics You propose the topic, b.docx
charlottej5
 
DOCX
Final Paper  The summative assignment for this course is to write a .docx
charlottej5
 
DOCX
Final Essay Stage Two ah W334 A.docx
charlottej5
 
DOCX
final draft requires minimum of 5 pages in length, 12-point Times Ne.docx
charlottej5
 
DOCX
Final Draft should be based on the topic Decision MakingThe .docx
charlottej5
 
DOCX
Final Digital Marketing PlanFinal Digital Marketing .docx
charlottej5
 
DOCX
FINAL COURSE PROJECT PRESENTATION (SIGNATURE ASSIGNMENT)You .docx
charlottej5
 
DOCX
Final Course PaperWrite about the impacts of health informatic.docx
charlottej5
 
DOCX
Final Communication ProjectNew Testament 1.) Pick a Scriptur.docx
charlottej5
 
DOCX
Final Case Study and Strategic PlanRead the Walt Disney Company .docx
charlottej5
 
DOCX
Fina Assessment Project The objective of this project is for stude.docx
charlottej5
 
DOCX
FIN 571 Final Exam Question 1 Which of the following is c.docx
charlottej5
 
DOCX
FIN 315 Fall 2018 Case Study Assignment due Dec 6 midnightNot .docx
charlottej5
 
DOCX
FIN 320 Final Project Guidelines and Rubric Final Pro.docx
charlottej5
 
Final PaperIt was a pleasure to be with you all and you made it fu.docx
charlottej5
 
Final PaperOne reason that California have been known as the Gol.docx
charlottej5
 
Final PaperIndia and China provide two fascinating country case st.docx
charlottej5
 
Final PaperMust begin with an introductory paragraph that has a .docx
charlottej5
 
Final PaperFinal Paper SynopsisThe purpose of this paper .docx
charlottej5
 
Final Paper, Essay Proposal, & Outline·The final essay should .docx
charlottej5
 
Final Paper – Possible topics You propose the topic, b.docx
charlottej5
 
Final Paper  The summative assignment for this course is to write a .docx
charlottej5
 
Final Essay Stage Two ah W334 A.docx
charlottej5
 
final draft requires minimum of 5 pages in length, 12-point Times Ne.docx
charlottej5
 
Final Draft should be based on the topic Decision MakingThe .docx
charlottej5
 
Final Digital Marketing PlanFinal Digital Marketing .docx
charlottej5
 
FINAL COURSE PROJECT PRESENTATION (SIGNATURE ASSIGNMENT)You .docx
charlottej5
 
Final Course PaperWrite about the impacts of health informatic.docx
charlottej5
 
Final Communication ProjectNew Testament 1.) Pick a Scriptur.docx
charlottej5
 
Final Case Study and Strategic PlanRead the Walt Disney Company .docx
charlottej5
 
Fina Assessment Project The objective of this project is for stude.docx
charlottej5
 
FIN 571 Final Exam Question 1 Which of the following is c.docx
charlottej5
 
FIN 315 Fall 2018 Case Study Assignment due Dec 6 midnightNot .docx
charlottej5
 
FIN 320 Final Project Guidelines and Rubric Final Pro.docx
charlottej5
 

Recently uploaded (20)

PDF
1.3 FINAL REVISED K-10 PE and Health CG 2023 Grades 4-10 (1).pdf
JohnJerryDalawampu
 
PDF
My India Quiz Book_20210205121199924.pdf
AtandraDey2
 
PDF
Journal of Dental Science - UDMY (2021).pdf
Nay Aung
 
PDF
Empowerment Technology for Senior High School Guide
solthereseamericandr
 
PPTX
Unit 4 Computer Architecture Multicore Processor.pptx
Gunasundari Selvaraj
 
PDF
Race Reva University – Shaping Future Leaders in Artificial Intelligence
RACE REVA University
 
PDF
Complications of Minimal Access-Surgery.pdf
Laparoscopy Hospital
 
PDF
Uderstanding digital marketing and marketing stratergie for engaging the digi...
afreenpeshmam
 
PDF
Τίμαιος είναι φιλοσοφικός διάλογος του Πλάτωνα
iliaskessaris
 
PPTX
Introduction to pro and eukaryotes and differences.pptx
AvaniKarumathil
 
PDF
BP 704 T. NOVEL DRUG DELIVERY SYSTEMS (UNIT 2).pdf
CMEmpire
 
PDF
David L Page_DCI Research Study Journey_how Methodology can inform one's prac...
David L Page
 
PDF
ChatGPT for Dummies - Pam Baker Ccesa007.pdf
Demetrio Ccesa Rayme
 
PDF
LIFE & LIVING TRILOGY - PART - (2) THE PURPOSE OF LIFE.pdf
sureshkumarsoni26
 
PPTX
B.Sc. DS Unit 2 Software Engineering.pptx
Dr. Pallawi Bulakh
 
PDF
AI-driven educational solutions for real-life interventions in the Philippine...
EleniIlkou
 
PPTX
What’s under the hood: Parsing standardized learning content for AI
Rustici Software
 
PDF
What if we spent less time fighting change, and more time building what’s rig...
Derek Wenmoth
 
PDF
English Textual Question & Ans (12th Class).pdf
javaidpaswal33
 
PDF
BP 505 T. PHARMACEUTICAL JURISPRUDENCE (UNIT 2).pdf
CMEmpire
 
1.3 FINAL REVISED K-10 PE and Health CG 2023 Grades 4-10 (1).pdf
JohnJerryDalawampu
 
My India Quiz Book_20210205121199924.pdf
AtandraDey2
 
Journal of Dental Science - UDMY (2021).pdf
Nay Aung
 
Empowerment Technology for Senior High School Guide
solthereseamericandr
 
Unit 4 Computer Architecture Multicore Processor.pptx
Gunasundari Selvaraj
 
Race Reva University – Shaping Future Leaders in Artificial Intelligence
RACE REVA University
 
Complications of Minimal Access-Surgery.pdf
Laparoscopy Hospital
 
Uderstanding digital marketing and marketing stratergie for engaging the digi...
afreenpeshmam
 
Τίμαιος είναι φιλοσοφικός διάλογος του Πλάτωνα
iliaskessaris
 
Introduction to pro and eukaryotes and differences.pptx
AvaniKarumathil
 
BP 704 T. NOVEL DRUG DELIVERY SYSTEMS (UNIT 2).pdf
CMEmpire
 
David L Page_DCI Research Study Journey_how Methodology can inform one's prac...
David L Page
 
ChatGPT for Dummies - Pam Baker Ccesa007.pdf
Demetrio Ccesa Rayme
 
LIFE & LIVING TRILOGY - PART - (2) THE PURPOSE OF LIFE.pdf
sureshkumarsoni26
 
B.Sc. DS Unit 2 Software Engineering.pptx
Dr. Pallawi Bulakh
 
AI-driven educational solutions for real-life interventions in the Philippine...
EleniIlkou
 
What’s under the hood: Parsing standardized learning content for AI
Rustici Software
 
What if we spent less time fighting change, and more time building what’s rig...
Derek Wenmoth
 
English Textual Question & Ans (12th Class).pdf
javaidpaswal33
 
BP 505 T. PHARMACEUTICAL JURISPRUDENCE (UNIT 2).pdf
CMEmpire
 

Filter2020 Spring2019 Fall2019 SummerCurr.docx

  • 1. / Filter 2020 Spring 2019 Fall 2019 Summer Current Courses Search your courses All C Spring 2020 - Applied Learning Practicum (INTR-799-06) - Full TermPamela SmithMore info Spring 2020 - Data Science & Big Data Analy (ITS-836-52) - Full TermMultiple Instructors More info Spring 2020 - Infer Stats in Decision-Making (DSRT-734-05) - Second Bi-TermMultiple Instructors Fall 2019 - Applied Learning Practicum (INTR-799-03) - Full TermPamela SmithMore info Fall 2019 - InfoTech Import in Strat Plan (ITS-831-05) - First Bi-TermMultiple Instructors More info Spring 2020 - Data Science & Big Data Analy (ITS-836-52) - Full Term
  • 2. InformationSpring 2020 - Data Science & Big Data Analy (ITS- 836-52) - Full Term Information Big Data Analytics iden�fy fundamental concepts of Big Data management and analy�cs. become competent in recognizing challenges faced by applica�ons dealing with very large volumes of data as well as in proposing scalable solu�ons for them. be able to understand how Big Data impacts business intelligence, scien�fic discovery, and our day-to-day life. Course Descrip�on: In this course the students explore key data analysis and management techniques, which applied to massive datasets are the cornerstone that enables real-�me decision making in distributed environments, business intelligence in the Web, and scien�fic discovery at large scale. In par�cular, students examine the map-reduce parallel compu�ng paradigm and associated technologies such as distributed file systems, no-sql databases, and stream compu�ng engines. This highly interac�ve course is based on the problem-based learning philosophy. Students are expected to make use of technologies to design highly scalable systems that can process and analyze Big Data for a variety of scien�fic, social, and environmental challenges.
  • 3. Course Objec�ves/Learner Outcomes: Course Objec�ves/Learner Outcomes: Upon comple�on of this course, the student will: Prerequisites: There are no prerequisites for this course. Books and Resources: Required Text EMC Educa�on Service (Eds). (2015) Data Science and Big Data Analytics: Discovering, Analyzing, Visualizing, and Presenting Data, Indianapolis, IN: John Wiley & Sons, Inc ×× javascript:void(0); javascript:void(0); javascript:void(0); javascript:void(0); javascript:void(0); javascript:void(0); https://ucumberlands.blackboard.com/webapps/blackboard/exec ute/courseMain?course_id=_114565_1 CSCI 561 Article Review Instructions
  • 4. After reading through your assignments this week, you are to pick a topic of interest that was mentioned in the reading assignment. Using the Jerry Falwell Library and other scholarly resources, you are to locate a peer reviewed journal related to the topic of your interest. Read the journal article thoroughly so you can discuss it. If you wish to use something other than a peer reviewed journal, please consult with your instructor before starting the assignment. You will then prepare an article review related to the topic that you researched in the research/reading assignments from the assigned module/week. The article review that you create must be at least 750words and be formatted according to APA style. Use the following section order to guide you in building your article review but do not use the numbers in your section headers, only the names. Also, this is example is not in APA style, make sure your review follows APA styles, this is a guide NOT a template. 1. Bibliographical Reference: a. Create a single bibliographical entry in APA style in the first section of your paper. This should include: i. Detail the author(s) of the article ii. Name of the Journal (including volume, issue, year, page numbers, etc.) iii. Name of the article you are reviewing. b. Note – You may also choose to identify the article you are reviewing (i.e. – The focus of this review will be “Article Name” published in “Journal Name” in “Year/Month”) and then simply cite it in a bibliographical entry at the end (see section 7). 2. Objectives: a. After reviewing the article, use a bulleted list to identity the 3 to 5 primary points you feel the article addresses. b. Simply put the bulleted list of those points in your Objectives
  • 5. section with something to the effect of “After reviewing “Article Name” the three main points addressed by the author(s) were: · Bullet one · Bullet two · Bullet three 3. Summary: a. Summarize the article you reviewed in approximately 2 or 3 full paragraphs. b. Do not use direct quotes at all. If you need to quote it, paraphrase what you read. c. Use the bullets above (section 2b) to structure what you wish to discuss in this summary (i.e. – In the first paragraph, summarize the article section which addresses the first bullet in 2b and so on). 4. Results: a. Now that you have read this article and summarized it, what do you feel that a reader can learn about the topic from the article? b. Using bullet points, highlight the things you feel like you learned more about after reading this article. It’s best to keep this list to a maximum of 5 bullets. c. Note – these bullets should not be identical to the bullets in the Objective Section (2b). 5. Critique: a. Use this section to provide an academic critique of the article. i. How well (or poorly) was this article written? ii. Did it accomplish the objectives it set out to do (think back to the Objectives section)? iii. Why or why not? b. Explain your opinion and offer some additional sources (no
  • 6. more than one or two) to support that opinion. c. When possible, address the subject matter from a biblical perspective (e.g. – the Bible teaches us to… and this article addresses that by…) 6. Questions: a. List a minimum of 3 (no more than 5) questions that arose from the reading of this article in a numbered list. b. Consider any thing you feel like you should have learned from the article but did not. c. Note – These questions may spark your desire to review other matter on the topic and may even be used as a starting point for one of your two research papers (though this is not required). 7. Bibliography: a. You need to use one other sources to support or deny your opinion and if you chose to only identify the article in section 1, you will want to make a bibliography. b. Follow APA guidelines for Bibliographies as outlined in the APA 6th Edition Guide. IMPORTANT – Article reviews are DUE by 11:59ET on Sunday of the assigned Module/Week! ffirs.indd 2:43:16:PM/12/11/2014 Page i Data Science & Big Data Analytics
  • 7. ffirs.indd 2:43:16:PM/12/11/2014 Page iii Discovering, Analyzing, Visualizing and Presenting Data Data Science & Big Data Analytics EMC Education Services ffirs.indd 2:43:16:PM/12/11/2014 Page iv Data Science & Big Data Analytics: Discovering, Analyzing, Visualizing and Presenting Data Published by John Wiley & Sons, Inc. 10475 Crosspoint Boulevard Indianapolis, IN 46256 www.wiley.com Copyright © 2015 by John Wiley & Sons, Inc., Indianapolis, Indiana Published simultaneously in Canada ISBN: 978-1-118-87613-8
  • 8. ISBN: 978-1-118-87622-0 (ebk) ISBN: 978-1-118-87605-3 (ebk) Manufactured in the United States of America 10 9 8 7 6 5 4 3 2 1 No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except as permitted under Sections 107 or 108 of the 1976 United States Copyright Act, without either the prior written permis- sion of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 646-8600. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748-6011, fax (201) 748-6008, or online at http://www.wiley.com/go/permissions. Limit of Liability/Disclaimer of Warranty: The publisher and the author make no representations or warranties with respect to the accuracy or completeness of the contents of this work and specifically disclaim all warranties, including without limitation warranties of fitness for a particular purpose. No warranty may be
  • 9. created or extended by sales or promotional materials. The advice and strategies contained herein may not be suitable for every situation. This work is sold with the understanding that the publisher is not engaged in rendering legal, accounting, or other professional services. If professional assistance is required, the services of a competent professional person should be sought. Neither the publisher nor the author shall be liable for damages arising herefrom. The fact that an organization or Web site is referred to in this work as a citation and/or a potential source of further information does not mean that the author or the publisher endorses the information the organization or website may provide or recommendations it may make. Further, readers should be aware that Internet websites listed in this work may have changed or disappeared between when this work was written and when it is read. For general information on our other products and services please contact our Customer Care Department within the United States at (877) 762-2974, outside the United States at (317) 572-3993 or fax (317) 572-4002. Wiley publishes in a variety of print and electronic formats and by print-on-demand. Some material included with standard print versions of this book may not be included in e-books or in print-on-demand. If this book refers to media such as a CD or DVD that is not included in the version you purchased, you may download
  • 10. this material at http://booksupport.wiley.com. For more information about Wiley products, visit www.wiley.com. Library of Congress Control Number: 2014946681 Trademarks: Wiley and the Wiley logo are trademarks or registered trademarks of John Wiley & Sons, Inc. and/or its affiliates, in the United States and other coun- tries, and may not be used without written permission. All other trademarks are the property of their respective owners. John Wiley & Sons, Inc. is not associated with any product or vendor mentioned in this book. http://www.wiley.com http://www.wiley.com/go/permissions http://booksupport.wiley.com http://www.wiley.com ffirs.indd 2:43:16:PM/12/11/2014 Page v Credits Executive Editor Carol Long Project Editor Kelly Talbot Production Manager
  • 11. Kathleen Wisor Copy Editor Karen Gill Manager of Content Development and Assembly Mary Beth Wakefield Marketing Director David Mayhew Marketing Manager Carrie Sherrill Professional Technology and Strategy Director Barry Pruett Business Manager Amy Knies Associate Publisher Jim Minatel Project Coordinator, Cover Patrick Redmond
  • 12. Proofreader Nancy Carrasco Indexer Johnna VanHoose Dinse Cover Designer Mallesh Gurram ffirs.indd 2:43:16:PM/12/11/2014 Page vi About the Key Contributors David Dietrich heads the data science education team within EMC Education Services, where he leads the curriculum, strategy and course development related to Big Data Analytics and Data Science. He co-au- thored the first course in EMC’s Data Science curriculum, two additional EMC courses focused on teaching leaders and executives about Big Data and data science, and is a contributing author and editor of this book. He has filed 14 patents in the areas of data science, data privacy, and cloud computing. David has been an advisor to several universities looking to develop academic programs related to data
  • 13. analytics, and has been a frequent speaker at conferences and industry events. He also has been a a guest lecturer at universi- ties in the Boston area. His work has been featured in major publications including Forbes, Harvard Business Review, and the 2014 Massachusetts Big Data Report, commissioned by Governor Deval Patrick. Involved with analytics and technology for nearly 20 years, David has worked with many Fortune 500 companies over his career, holding multiple roles involving analytics, including managing analytics and operations teams, delivering analytic con- sulting engagements, managing a line of analytical software products for regulating the US banking industry, and developing Software-as-a-Service and BI-as-a-Service offerings. Additionally, David collaborated with the U.S. Federal Reserve in develop- ing predictive models for monitoring mortgage portfolios. Barry Heller is an advisory technical education consultant at EMC Education Services. Barry is a course developer and cur- riculum advisor in the emerging technology areas of Big Data and data science. Prior to his current role, Barry was a consul- tant research scientist leading numerous analytical initiatives within EMC’s Total Customer Experience organization. Early in his EMC career, he managed the
  • 14. statistical engineering group as well as led the data warehousing efforts in an Enterprise Resource Planning (ERP) implementation. Prior to joining EMC, Barry held managerial and analytical roles in reliability engineering functions at medical diagnostic and technology companies. During his career, he has applied his quantitative skill set to a myriad of business applications in the Customer Service, Engineering, Manufacturing, Sales/Marketing, Finance, and Legal arenas. Underscoring the importance of strong executive stakeholder engagement, many of his successes have resulted from not only focusing on the technical details of an analysis, but on the decisions that will be resulting from the analysis. Barry earned a B.S. in Computational Mathematics from the Rochester Institute of Technology and an M.A. in Mathematics from the State University of New York (SUNY) New Paltz. Beibei Yang is a Technical Education Consultant of EMC Education Services, responsible for developing several open courses at EMC related to Data Science and Big Data Analytics. Beibei has seven years of experience in the IT industry. Prior to EMC she worked as a software engineer, systems manager, and network manager for a Fortune 500 company where she introduced
  • 15. new technologies to improve efficiency and encourage collaboration. Beibei has published papers to prestigious conferences and has filed multiple patents. She received her Ph.D. in computer science from the University of Massachusetts Lowell. She has a passion toward natural language processing and data mining, especially using various tools and techniques to find hidden patterns and tell stories with data. Data Science and Big Data Analytics is an exciting domain where the potential of digital information is maximized for making intelligent business decisions. We believe that this is an area that will attract a lot of talented students and professionals in the short, mid, and long term. ffirs.indd 2:43:16:PM/12/11/2014 Page vii Acknowledgments EMC Education Services embarked on learning this subject with the intent to develop an “open” curriculum and certification. It was a challenging journey at the time as not many understood what it would take to be a true data scientist. After initial research (and struggle), we were able to define what was needed and attract very
  • 16. talented professionals to work on the project. The course, “Data Science and Big Data Analytics,” has become well accepted across academia and the industry. Led by EMC Education Services, this book is the result of efforts and contributions from a number of key EMC organizations and supported by the office of the CTO, IT, Global Services, and Engineering. Many sincere thanks to many key contributors and subject matter experts David Dietrich, Barry Heller, and Beibei Yang for their work developing content and graphics for the chapters. A special thanks to subject matter experts John Cardente and Ganesh Rajaratnam for their active involvement reviewing multiple book chapters and providing valuable feedback throughout the project. We are also grateful to the following experts from EMC and Pivotal for their support in reviewing and improving the content in this book: Aidan O’Brien Joe Kambourakis Alexander Nunes Joe Milardo Bryan Miletich John Sopka Dan Baskette Kathryn Stiles
  • 17. Daniel Mepham Ken Taylor Dave Reiner Lanette Wells Deborah Stokes Michael Hancock Ellis Kriesberg Michael Vander Donk Frank Coleman Narayanan Krishnakumar Hisham Arafat Richard Moore Ira Schild Ron Glick Jack Harwood Stephen Maloney Jim McGroddy Steve Todd ffirs.indd 2:43:16:PM/12/11/2014 Page viii Jody Goncalves Suresh Thankappan Joe Dery Tom McGowan We also thank Ira Schild and Shane Goodrich for coordinating this project, Mallesh Gurram for the cover design, Chris Conroy and Rob Bradley for graphics, and the publisher, John Wiley and Sons, for timely support in bringing this book to the industry. Nancy Gessler
  • 18. Director, Education Services, EMC Corporation Alok Shrivastava Sr. Director, Education Services, EMC Corporation ftoc.indd 7:24:10:PM/12/12/2014 Page ix Contents Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.1 Big Data Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.1.1 Data Structures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1.1.2 Analyst Perspective on Data Repositories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 1.2 State of the Practice in Analytics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 1.2.1 BI Versus Data Science . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
  • 19. 1.2.2 Current Analytical Architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 1.2.3 Drivers of Big Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 1.2.4 Emerging Big Data Ecosystem and a New Approach to Analytics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 1.3 Key Roles for the New Big Data Ecosystem. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 1.4 Examples of Big Data Analytics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 2.1 Data Analytics Lifecycle Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
  • 20. 26 2.1.1 Key Roles for a Successful Analytics Project . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 2.1.2 Background and Overview of Data Analytics Lifecycle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 2.2 Phase 1: Discovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 2.2.1 Learning the Business Domain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .30 2.2.2 Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 2.2.3 Framing the Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 2.2.4 Identifying Key Stakeholders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 2.2.5 Interviewing the Analytics Sponsor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 2.2.6 Developing Initial Hypotheses. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
  • 21. 2.2.7 Identifying Potential Data Sources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 2.3 Phase 2: Data Preparation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 2.3.1 Preparing the Analytic Sandbox . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 2.3.2 Performing ETLT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .38 2.3.3 Learning About the Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 2.3.4 Data Conditioning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .40 2.3.5 Survey and Visualize . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 2.3.6 Common Tools for the Data Preparation Phase . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 2.4 Phase 3: Model Planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
  • 22. 2.4.1 Data Exploration and Variable Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .44 2.4.2 Model Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 2.4.3 Common Tools for the Model Planning Phase . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 ftoc.indd 7:24:10:PM/12/12/2014 Page x CONTENTSx 2.5 Phase 4: Model Building. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 2.5.1 Common Tools for the Model Building Phase . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .48 2.6 Phase 5: Communicate Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 2.7 Phase 6: Operationalize . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 2.8 Case Study: Global Innovation Network and Analysis (GINA). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 2.8.1 Phase 1: Discovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
  • 23. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .54 2.8.2 Phase 2: Data Preparation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 2.8.3 Phase 3: Model Planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .56 2.8.4 Phase 4: Model Building . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 2.8.5 Phase 5: Communicate Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .58 2.8.6 Phase 6: Operationalize . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
  • 24. 3.1 Introduction to R. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 3.1.1 R Graphical User Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 3.1.2 Data Import and Export. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 3.1.3 Attribute and Data Types. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 3.1.4 Descriptive Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 3.2 Exploratory Data Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80 3.2.1 Visualization Before Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .82 3.2.2 Dirty Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .85 3.2.3 Visualizing a Single Variable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .88
  • 25. 3.2.4 Examining Multiple Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 3.2.5 Data Exploration Versus Presentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .99 3.3 Statistical Methods for Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 3.3.1 Hypothesis Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .102 3.3.2 Difference of Means . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 3.3.3 Wilcoxon Rank-Sum Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 3.3.4 Type I and Type II Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .109 3.3.5 Power and Sample Size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 3.3.6 ANOVA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
  • 26. Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 . . . . . . . . . . . . . . . . . . . . . . . . . 117 4.1 Overview of Clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118 4.2 K-means . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118 4.2.1 Use Cases. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 4.2.2 Overview of the Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .120 4.2.3 Determining the Number of Clusters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .123 4.2.4 Diagnostics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
  • 27. . . . .128 ftoc.indd 7:24:10:PM/12/12/2014 Page xi CONTENTS xi 4.2.5 Reasons to Choose and Cautions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .130 4.3 Additional Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136 . . . . . . . . . . . . . . . . . . 137 5.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138 5.2 Apriori Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
  • 28. 140 5.3 Evaluation of Candidate Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141 5.4 Applications of Association Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143 5.5 An Example: Transactions in a Grocery Store. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143 5.5.1 The Groceries Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .144 5.5.2 Frequent Itemset Generation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .146 5.5.3 Rule Generation and Visualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .152 5.6 Validation and Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157 5.7 Diagnostics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158
  • 29. Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160 . . . . . . . . . . . . . . . . . . . . . . . . 161 6.1 Linear Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162 6.1.1 Use Cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .162 6.1.2 Model Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .163 6.1.3 Diagnostics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .173 6.2 Logistic Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178 6.2.1 Use Cases. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .179
  • 30. 6.2.2 Model Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . …