A quick demo of Top2Vec With application on 2020 10-K business descriptions
0 likes579 views
The document discusses a presentation by Gautier Marti about the Top2Vec algorithm for topic modeling, highlighting its advantages over the traditional Latent Dirichlet Allocation (LDA) method. Top2Vec does not require pre-defined topics or extensive text processing, instead utilizing a 5-step algorithm to identify topics from collections of documents, demonstrated with applications on 2020 10-K business descriptions. The findings include identification of sectors and the impact of COVID-19 but suggest a need for further exploration of residual topics.
A quick demo of Top2Vec With application on 2020 10-K business descriptions
- 1. Gautier Marti, Hong Kong Machine Learning Meetup Season 4 Episode 4 A quick demo of Top2Vec With application on 2020 10-K business descriptions
- 2. Discover latent “topics” • Find automatically the abstract “topics” which occur in a collection of documents • For example, for news: topic 1: “trade war” topic 2: “in fl ation” topic 3: “sport” topic 4: “healthcare” topic 5: “environment” … Topic Modeling
- 3. LDA: Standard method with practical shortcomings • LDA: Main method for topic modeling for the past 20 years • LDA relies on bag-of-words which ignore ordering and semantics of words • LDA relies on lots of arbitrary meta- parameters such as - custom stop-word lists - stemming and lemmatization • LDA requires the number of topics to be known in advance (unrealistic) • As a result, topics found by LDA are not always very stable… Latent Dirichlet Allocation (LDA)
- 4. Distributed Representations of Topics • Recent arxiv paper (2020) • GitHub: ddangelov/Top2Vec • Does not require the number of topics to be known in advance • Does not require text pre- processing Top2Vec
- 5. 5-step algorithm • Step 1: Create a joint embedding of documents and words • For example, using: - Universal Sentence Encoder - BERT Sentence Transformer Top2Vec: How does it work?
- 6. 5-step algorithm • Step 2: Project the document and word vectors to a smaller dimension (e.g. 2D) • For example, using: - UMAP Top2Vec: How does it work?
- 7. 5-step algorithm • Step 3: Find dense clusters; a cluster is a topic • For example, using: - HDBSCAN Top2Vec: How does it work?
- 8. 5-step algorithm • Step 4: Find the centroid of each cluster in original high dimension This centroid is the topic vector Top2Vec: How does it work?
- 9. 5-step algorithm • Step 5: Find the n-closest word vectors to the topic vector These words de fi ne the topic Top2Vec: How does it work?
- 10. Top2Vec Demo: Application on 10-K filings 2020 10-K business descriptions
- 11. What is a 10-K? • Forms 10-K are mandatory annual reports for U.S. public companies • Company management o ff er a detailed picture of - the company’s business, - the risks it faces, - the operating/ fi nancial results • Available for free on https://www.sec.gov/edgar.shtml 10-K corporate filings
- 12. Business description • Each 10-K starts with a thorough business description • We learn about its main products and services, subsidiaries it owns, markets it operates in • It may also include: - competition the company faces - regulations that apply to it - labor issues - special operating costs - seasonal factors - … 10-K corporate filings
- 13. Topic modeling of business descriptions 1. Download the 2020 10-Ks 2. Extract business descriptions 3. Apply Top2Vec Blog + colab notebook: https://marti.ai/ml/2021/11/14/ top2vec-10k-business.html Top2Vec Demo
- 14. Results • Top2Vec found 10 topics • 9 topics correspond to sectors • 1 topic corresponds to COVID-19 disruptions Top2Vec Demo
- 15. So what? • Top2Vec as a tool works well • On corporate fi lings, it essentially recovers sectors as topics… not very informative! • Can we uncover residual topics? Top2Vec on 10-K