Interactive Machine Learning
3 likes2,055 views
The document discusses interactive machine learning (IML), which aims to make machine learning more accessible to non-experts by allowing iterative human feedback. IML is defined as an iterative process where users can provide feedback to control model behavior, unlike classical machine learning which involves a single pass with no user input. The document outlines categories of IML including interaction perspectives for supplying training data, choosing algorithms, and evaluating models. Examples of IML systems are provided for each category.
Interactive Machine Learning
- 1. Department of Computer Science Interactive Machine Learning Zitao Liu ztliu@cs.pitt.edu School of Arts of and Sciences Department of Computer Science
- 2. Department of Computer Science Outline β’What is Interactive Machine Learning(IML): βMotivation & Goal βClassical Machine Learning V.S Interactive Machine Learning βDesign Principle of IML β’Applications of IML: βIML Categories βExisting IML Applications (CHI, UIST, IUI, KDD)
- 3. Department of Computer Science Motivation & Goal β’ββ¦We believe that trying to fully automate tasks is extremely difficult and even undesirable, but instead there exists a computational design methodology which allows us to gracefully combine automated services with direct user manipulationβ¦β ββ from Microsoft Research
- 4. Department of Computer Science Motivation & Goal β’The complexity of machine learning has largely restricted its use to experts and skilled developers. β’Solving real world problems can benefit from end- userβs interaction. β’A better understanding/assessment of a modelβs performance. β’β¦ β¦
- 5. Department of Computer Science What is IML? Data Human Model Data Model Classical ML Interactive ML Classical ML Interactive ML One-pass process Iterative process No usersβ feedback Users control the behavior Long time to train the model Latency sensitive for training Numerical evaluation Friendly visualization evaluation
- 6. Department of Computer Science How to design effective end-user interaction with interactive ML system? β’Questions: βWhich examples should a person provide to efficiently train the system? βHow should the system illustrate its current understanding? βHow can a person evaluate the quality of the systemβs current understanding in order to better guide it towards the desired behavior?
- 7. Department of Computer Science Design Principle Fast and Focused β’Fast: βTakes seconds or minutes rather than weeks or months to train an effective model. βUser can quickly refine the classifier by adding more manual info. βUser can get feedback as quickly as possible. β’Focused: βUI component needs to be simple so the user can remain focused on the ML problem at hand.
- 8. Department of Computer Science IML Research Groups Brigham Young University And so onβ¦
- 9. Department of Computer Science IML Categories β’Interaction Perspective β’Data Perspective β’Learning Perspective
- 10. Department of Computer Science IML Categories Training Data Learning Algorithm Results Evaluation β’Interaction Perspective
- 11. Department of Computer Science IML Categories Training Data Learning Algorithm Results Evaluation β’Interaction Perspective
- 12. Department of Computer Science IML Categories Training Data Learning Algorithm Results Evaluation β’Interaction Perspective
- 13. Department of Computer Science IML Categories Training Data Learning Algorithm Results Evaluation β’Interaction Perspective
- 14. Department of Computer Science IML Categories β’Interaction Perspective: βSupplying training data βChoosing a learning algorithm βEvaluating and comparing models No matter what stage we interact with, our goal is the same: Put the full potential of machine learning in the hands of everyday people.
- 15. Department of Computer Science IML Categories β’Date Perspective: βStream-based IML βPool-based IML β’Learning Perspective: βConcept Learning βSupervised Learning βUnsupervised Learning
- 16. Department of Computer Science IML Categories β’Date Perspective: βStream-based IML βPool-based IML β’Learning Perspective: βConcept Learning βSupervised Learning βUnsupervised Learning Focus on Interaction Perspective
- 17. Department of Computer Science IML Categories β’Interaction Perspective: βSupplying training data IML Sys Name Description Conference CueFlik Allows users to create their own rules for re- ranking images based on their visual characteristics. CHI2008 CueT Learns from decisions of operators in a highly dynamic environment. CHI2011 Content Creation with IE A synergistic method for jointly amplifying community content creation and learning based information extraction. CHI2009 Apolo Help users make sense of large network data CHI2011 ReGroup Help people create custom groups on-demand CHI2012
- 18. Department of Computer Science IML Categories β’Interaction Perspective: βChoosing a learning algorithm IML Sys Name Description Conference Visual-FSSEM Guides the feature selection procedures. KDD2000 CueFlik Allows users to create their own rules for re- ranking images based on their visual characteristics. CHI2008 EnsembleMatrix Presents a graphical view of confusion matrices to help users understand relative merits of various classifiers. CHI2009
- 19. Department of Computer Science IML Categories β’Interaction Perspective: βEvaluating and comparing models IML Sys Name Description Conference EnsembleMatrix Presents a graphical view of confusion matrices to help users understand relative merits of various classifiers. CHI2009 ManiMatrix Provides controls and visualizations that enable system builders to refine the behavior of classification systems in an intuitive manner. CHI2010
- 20. Department of Computer Science Examples: ReGroup(CHI2012) β’Goal: helps people create custom, on-demand groups in online social networks. β’Workflow: ReGroup observes a personβs normal interaction of adding members to a group, it learns a probabilistic model of group membership in order to suggest both additional members and group characteristics for filtering a friend list. It continually update its membership model based on interactive user feedback. β’ML Model: NaΓ―ve Bayes(probability of each friend being a member of the group). Re-trained every time a person adds friends to a group. β’Specialties: obtain implicit negative examples; unlearnable groups; missing data Amershi, S., Fogarty, J., Weld, D.S. ReGroup: Interactive Machine Learning for On- Demand Group Creation in Social Networks.Proc. CHI β12, ACM Press(2012)
- 21. Department of Computer Science Examples: ReGroup(CHI2012) Top 5 relevant group characteristics Relevant friend suggestions A static, hierarchical list of all feature value filters
- 22. Department of Computer Science Reference β’Html: βhttp://www.cs.pitt.edu/~ztliu/comp/iml.html β’Bib: βhttp://www.cs.pitt.edu/~ztliu/comp/iml.bib
- 23. Department of Computer Science Q & A Thank you!