AutoClassificaiton - Rules versus Machine Learning
Download as PPTX, PDF4 likes1,415 views
Jeff Fried, CTO of BA Insight, discusses the evolution of search and SharePoint since 2004, highlighting the importance of metadata and automation in enhancing user experiences through tools like autoclassification and autotagging. The document outlines various approaches to document classification, including rule-based and machine learning techniques, emphasizing their respective benefits and challenges. It also provides insights into getting started with metadata frameworks and the integration of managed vocabularies.
![Examples of Rules
Boolean
• “IT” OR “Information Technology” or “MIS”
• (“Expert” OR “Witness”) NOT “police”
• “New York” AND “environmental policy”
• *work
• "legal" -briefs
• "Legal" NEAR(5) "issue“
Property-based
• filetype:docx
• title:"2029 L.P" or title:2030
• footer="BA Insight Confidential" or
footer:proprietary or footer:BA*
Overriding/changing Linguistics
• NOSTEM(“illumination")
• CASE("prerequisites")
• SOUNDLIKE("prerech")
Regular expressions
• title:REGEX([0-4])
• REGEX("b(([w-]+://?|www[.])[^s()<>]+(?:([wd]+)|([^[:punct:]s]|/)))")](https://image.slidesharecdn.com/friedtbc2016taxo101rulesvsml-170130214448/85/AutoClassificaiton-Rules-versus-Machine-Learning-16-320.jpg)
AutoClassificaiton - Rules versus Machine Learning
- 1. Jeff Fried CTO BA Insight @jefffried #tbc2016 Rules-Based vs. Document-Based Bake-off
- 3. Focused on Search and SharePoint since 2004 Longtime Search Nerd • CTO, BA Insight • Senior PM, Microsoft • VP, FAST • SVP, LingoMotors About Jeff Fried Passionate About • Search • SharePoint • Search-driven applications • Information Strategy Blog: BAinsight.com/blog Technet Column “A View from the Crawlspace” jeff.fried@bainsight.com
- 4. About BA Insight – Connectivity – Applications - – Classification - – Analytics
- 5. Metadata Drives Great User Experiences Documents from many sources All client or matter-relevant documents are integrated. Rich MetaData Content annotated automatically – concepts, categories, citations, matters, clients, etc Navigation Controls Explore, Discover, Drill-down
- 6. Manual Tagging is impractical and remarkably inconsistent
- 7. Automation Called: AutoClassification, AutoTagging, Metadata Generation, Text Analytics, ….
- 8. 8
- 11. 11 Common Techniques across Applications
- 13. Rules-based Approach Enhanced Content Enriched with Metadata and Content Types Search Visualization Workflow
- 14. Name Blood Type Give Birth Can Fly Live in Water Class human warm yes no no mammals python cold no no no reptiles salmon cold no no yes fishes whale warm yes no yes mammals frog cold no no sometimes amphibians komodo cold no no no reptiles bat warm yes yes no mammals pigeon warm no yes no birds cat warm yes no no mammals leopard shark cold yes no yes fishes turtle cold no no sometimes reptiles penguin warm no no sometimes birds porcupine warm yes no no mammals eel cold no no yes fishes salamander cold no no sometimes amphibians gila monster cold no no no reptiles platypus warm no no no mammals owl warm no yes no birds dolphin warm yes no yes mammals eagle warm no yes no birds Rule-based Classifier (Example) R1: (Give Birth = no) (Can Fly = yes) Birds R2: (Give Birth = no) (Live in Water = yes) Fishes R3: (Give Birth = yes) (Blood Type = warm) Mammals R4: (Give Birth = no) (Can Fly = no) Reptiles R5: (Live in Water = sometimes) Amphibians
- 15. Example Rules Engine UI
- 16. Examples of Rules Boolean • “IT” OR “Information Technology” or “MIS” • (“Expert” OR “Witness”) NOT “police” • “New York” AND “environmental policy” • *work • "legal" -briefs • "Legal" NEAR(5) "issue“ Property-based • filetype:docx • title:"2029 L.P" or title:2030 • footer="BA Insight Confidential" or footer:proprietary or footer:BA* Overriding/changing Linguistics • NOSTEM(“illumination") • CASE("prerequisites") • SOUNDLIKE("prerech") Regular expressions • title:REGEX([0-4]) • REGEX("b(([w-]+://?|www[.])[^s()<>]+(?:([wd]+)|([^[:punct:]s]|/)))")
- 17. Controlling scores & thresholds
- 18. Taxonomy Management is often included with Auto-Classification Tools
- 19. Where do you get Taxonomies?
- 20. 20 Semantics! Machine Learning! AI!
- 23. False positives vs. false negatives Look at the impact of each in your context
- 25. Example: identify people as good or bad from their appearance
- 27. Building an accurate classifier
- 28. – Training and Test Data 28
- 30. + Easy to get started + Transparent and debuggable + Easily controlled (when # rules not too large) - Need taxonomies - Rule maintenance effort - Harder to cover domain fully and to switch domains + Don’t need taxonomies + Improves without manual maintenance + Handles new data types/domains more easily - Need a training set - Opaque, usually can’t debug - Can’t specify or control specific examples
- 31. What would you use for
- 32. Case Study Content Identification and Movement
- 35. Combinations of Techniques usually work better
- 36. Examples of hybrid configurations
- 37. Example: clustering combined with rules
- 39. carrot2 Open Source & Platform packages offer an easy way to play
- 40. How to get started Setup up a metadata framework – keep it simple Develop or acquire managed vocabularies for critical elements Start with rule-driven automation Test out ML-based techniques as you grow
Editor's Notes
- #3: documents, web sites, blog posts, database entries, etc.