Data science at OLX
4 likes1,082 views
This document summarizes the data science work at OLX, an online classifieds marketplace. It discusses that OLX has data science teams focused on key areas like search, recommendations, trust and safety, seller experience, and monetization. The document outlines OLX's office in Berlin and provides details on its teams and career progression. It also describes expectations of data scientists, including focusing on modeling, evaluation and production. Overall, the summary highlights how OLX uses data science across its business and key functions.
Data science at OLX
- 1. Data Science at OLX Alexey Grigorev 24.03.2020 SPICED Academy
- 2. Plan ● What is OLX ● Data Science at OLX ● Way of working ● Expectations from data scientists ● Career progression
- 4. olxgroup.com
- 5. olx.in
- 6. olx.pl
- 7. olx.ua
- 8. OLX in Berlin ● 4 floors ● 263 people ● 51 nationalities ● 15+ teams (in tech) ● 13 data scientists + 1 student ○ 25 across all EU offices
- 9. Our office
- 12. Plan ● What is OLX ● Data Science at OLX ● Way of working ● Expectations from data scientists ● Career progression
- 13. Data Science at OLX Main areas: ● Search ● Recommendations ● Trust & Safety ● Seller Experience ● Monetization
- 14. Data Science at OLX Main areas: ● Search ● Recommendations ● Trust & Safety ● Seller Experience ● Monetization ● Smart ranking ● Reducing null-searches ● Query categorization ● Spell checking
- 15. Data Science at OLX Main areas: ● Search ● Recommendations ● Trust & Safety ● Seller Experience ● Monetization ● Collaborative filtering ● Item2Vec
- 16. Data Science at OLX Main areas: ● Search ● Recommendations ● Trust & Safety ● Seller Experience ● Monetization ● NSFW detection ● Forbidden items ● Fraud detection ● Duplicate detection ● Chat moderation
- 17. Data Science at OLX Main areas: ● Search ● Recommendations ● Trust & Safety ● Seller Experience ● Monetization ● Image quality ● Listing quality ● Deal prediction
- 18. Data Science at OLX Main areas: ● Search ● Recommendations ● Trust & Safety ● Seller Experience ● Monetization ● User segmentation ● Bid optimization
- 19. Data Science at OLX Main areas: ● Search ● Recommendations ● Trust & Safety ● Seller Experience ● Monetization
- 20. Such description. So much text https://olx.com OLX
- 21. Such description. So much text Such description. So much text Such description. So much text https://olx.com OLX
- 25. Problems: ● Illegal items ● NSFW content ● Duplicates ● Spam ● Fraud
- 26. Content moderation ML Such description So much text Accept Reject Moderation queue MP Automatic moderation system Moderation panel Accept Reject Moderators
- 27. ML Such description So much text Accept Reject Moderation queue Automatic moderation system Duplicate detection Forbidden items Other ML models
- 28. ML Such description So much text MP Automatic moderation system Moderation panel Accept Reject Moderators s3 ES Duplicate detection system Hashes Accept Reject Moderation queue
- 29. ML Such description So much text MP Automatic moderation system Moderation panel Accept Reject Moderators s3 ES Duplicate detection system Hashes Accept Reject Moderation queue Index listings & images
- 30. ML Such description So much text MP Automatic moderation system Moderation panel Accept Reject Moderators s3 ES Duplicate detection system Hashes Accept Reject Moderation queue Detect duplicates
- 31. ML Such description So much text MP Automatic moderation system Moderation panel Accept Reject Moderators s3 ES Duplicate detection system Hashes Accept Reject Moderation queue Moderate duplicates
- 32. ML Such description So much text MP Automatic moderation system Moderation panel Accept Reject Moderators s3 ES Duplicate detection system Hashes Accept Reject Moderation queue Collect feedback
- 35. Plan ● What is OLX ● Data Science at OLX ● Way of working ● Expectations from data scientists ● Career progression
- 36. A project like this is very complex We need a team (or multiple teams) to make it work: it’s a joined effort of many people working together
- 37. Roles in teams ● Product Manager (PM) ● Engineering Manager (EM) ● Software Engineers ○ Backend Engineers (BE) ○ Data Engineers (DE) ○ ML Engineer (MLE) ○ Site Reliability Engineers (SRE) ○ Frontend Engineers (FE) ○ Mobile Engineers ● Product Analysts (PA) ● Data Scientists (DS)
- 38. Team A Team B Team C Product PM PM PM Head of Product PA PA Head of Analytics DS DS DS Manager Data Tech EM EM EM Head of Engineering BE DE BE FE BE SRE FE SRE FE Matrix structure
- 39. Feature teams ● A cross-functional team with experts in different areas ● All work together on one feature/product ● All have the same goal! ● Anyone can work on anything, as long as it helps achieve the goal PA DS DE BE SREEMPM
- 40. Goal setting ● OKRs, set quarterly ● Great alignment tool: other teams know what you’re doing ● Whatever team is doing, should be in line with their OKRs Example: ● O ○ Catch more fraudsters ● KRs ○ Precision of model A improves from 30% to 60% while staying at the same recall level ○ Model B is tested in 5 key markets
- 41. Plan ● What is OLX ● Data Science at OLX ● Way of working ● Expectations from data scientists ● Career progression
- 42. Product Definition Data Processing Modeling Evaluation Production Customers Data Collection Stages
- 43. Roles Product Definition Data Processing Modeling Evaluation Production Customers Data Collection Stages
- 44. Product Definition Data Processing Modeling Evaluation Production Customers Data Collection Stages Focus on modelling and evaluation, a bit on production
- 45. Product Definition Data Processing Modeling Evaluation Production Customers Data Collection Stages
- 46. Product Definition Data Processing Modeling Evaluation Production Customers Data Collection Stages SPICED got it covered
- 47. Product Definition Data Processing Modeling Evaluation Production Customers Data Collection Stages
- 48. Product Definition Data Processing Modeling Evaluation Production Customers Data Collection Stages + Airflow, SQL, Spark
- 49. Product Definition Data Processing Modeling Evaluation Production Customers Data Collection Stages Learn from your PM!
- 50. Plan ● What is OLX ● Data Science at OLX ● Way of working ● Expectations from data scientists ● Career progression
- 51. Data Scientist Senior Data Scientist Lead Data Scientist Principal Data Scientist Chief Data Scientist Data Science Manager Head of Data Science Expert track Managerial track Senior Leadership (VP / Director / C level) Junior Data Scientist
- 52. Career progression ● Junior ○ performs an assigned task mostly independently ○ makes product impact — possibly with supervision ● Middle ○ performs a task completely independently ○ takes active part in planning and task prioritization ● Senior ○ works independently on complex tasks and drives projects ○ communicates with stakeholders ○ helps less senior people by onboarding and mentoring them ○ owns end-to-end process for implementing ML solutions Data Scientist Senior Data Scientist Junior Data Scientist
- 53. Performance evaluation 360 degree feedback — every 6 months ● Free form feedback ○ things to continue, start, stop doing ● Structured feedback ○ Following best engineering practices ○ Designing and delivering ML solutions ○ Finding areas where ML can bring value to our customers ○ Dealing with ambiguity ○ Knowledge sharing ○ Business acumen ○ Communicating with stakeholders ○ And 8 other dimensions
- 54. Plan ● What is OLX ● Data Science at OLX ● Way of working ● Expectations from data scientists ● Career progression
- 55. Credit Many things from this presentation is a collective effort of many people: ● Responsibilities ● Career path ● Performance evaluation Special thanks to: ● Mariana Matsiuk (and others from OLX HR department) ● Andreas Merentitis, Rahul Gupta, Elias Nema ● Liesbeth Dingemans, Paul van der Boor (and others from Prosus AI) ● And many others across OLX Group
- 56. Links OLX: ● OLX tech blog: tech.olx.com ● Twitter: @olxtecheurope ● Jobs at OLX: joinolx.com Me: ● contact@alexeygrigorev.com ● LinkedIn: agrigorev ● Twitter: @Al_Grigor