Big Data & Taxonomies for Actionable Intelligence
Download as PPTX, PDF1 like287 views
The presentation discusses the use of big data and taxonomies for actionable intelligence in assessing trade risks and consumer-related issues for two clients, Gogo Squeez and Coface. Both companies aimed to analyze unstructured content to identify risks quickly, necessitating automated systems for real-time data collection and analysis. The project successfully combined rule-based and machine learning approaches to create an adaptable architecture and user-centered design for effective risk management.
Big Data & Taxonomies for Actionable Intelligence
- 1. Big Data & Taxonomies for Actionable Intelligence Mondeca Ghislain Atemezing Director, Research and Innovation @gatemezing Taxonomy Boot Camp Track 2: Taxonomy Applications Nov 5, Washington, DC
- 2. Help companies forestall and assess trade risks while protecting them against overdue items Healthy snacking from the best of nature Food industry Healthy Snacks Credit Insurance USA, France, Mexico USA, France, UK, China, Portugal, Brazil French, Portuguese, English, Chinese French, English, Spanish 50+ power users 10+ power users Our clients businesses are very different
- 3. Things both of our clients shared 3 None of them had a taxonomy department None of them employed a taxonomist Both wanted to analyze a large volume of diverse unstructured content for the purpose of future risk identification
- 4. GoGo squeeZ wanted to listen to consumers and detect risk related consumer issues as soon as they were raised Coface intended to go beyond traditional balance sheet analysis and identify solvency risk from information published in the media How did it all start? 4 Risk profiles vary in time and need to be adjusted asap Analysts need to detect risks asap We need to catch all consumer-related events asap We need the big picture and view results across a set of categories (time, country, activity, risk events) Voice of consumer is captured in multiple different channels We are missing important messages because of the volumes of User Generated Content Collecting a body of evidence helps to assess companies performance We need to share alerts proactively across our marketing and quality management teams
- 5. From business objectives to solution requirements Detect consumer risk Assess credit risk Objectives • Provide alerts/insight on product quality and associated risks • Support quick decision making Audience Service & Quality managers and Executives Requirements • Monitor consumer feedback in real-time • Collate data from different open and internal sources • Reconcile data, identify and rank consumer risks • Generate color-coded alerts • Be 100% automated – end to end Requirements • Dynamically define online sources to monitor • Collate data from open source to establish risk profiles • Reconcile data, identify different types of trade risks • Generate risk scores • Be automated and allow human evaluation of information Objectives • Assess early events affecting companies’ solvency • Streamline the risk evaluation process Audience Credit risk analysts
- 6. Common high level architecture 6 API-based connectors Persistence DB Datapipe/ETL Text analytics Auto-classification Taxonomy Semantic Rules Search index Data visualization portal Social web Consumer service Call centers Business managers Risk analysts +800,000 messages/year 10 sources News feeds Websites Scrapers Persistence DB Datapipe/ETL Text analytics Auto-classification Persistence DB Search index Knowledge portal Taxonomy Semantic Rules +1,500,000 articles/year English, Spanish, French 200+ sources Chinese, Portuguese English, Spanish, French
- 7. GoGo squeeZ UX targets alerts and trend changes Synthetic dashboards Interactive analytics Focused on trend / time series Extended search features and instant access to source data Real-time, visual, easy-to-use, interactive, fast analytics
- 8. Coface UX is process oriented Risk grades Rationale Terms & concepts detected Acceptance or rejection of detected events Relevance evaluation
- 9. The vocabulary was already there More complex content, newspaper style, POS analysis a must Not many new terms, but more complex sentences Disambiguation was an issue requiring complex ML & Word2Vec approach Removing duplicates was an issue Focus on precision, UX and response time One size does not fit all … fine tuning was required The taxonomy was build from scratch Short, simple content, social media like, no requirement for POS analytics Evolving, informal language; candidate terms discovered all the time Little ambiguity – almost all content somewhat relates to the subject matter No issue in deduplicating content Focus on recall and real time data availability
- 10. Key project metrics are similar for both projects 6 months elapsed Taxonomy work Front endBack end 40% 40% 20% Day 1 Go Live
- 11. Take away and lessons learned The architecture was generic enough to respond to both client requirements UI/UX design was very different and critical to achieve user acceptance Results achieved through combination of rule-based and machine learning approach