Machine Learning at E*Trade
1 like4,695 views
The document outlines e*trade's technological advancements, including the integration of machine learning and public portals for data access, with a focus on mobile devices. It details the company's infrastructure including elastic clusters, curated dashboards, and public data aggregations for significant text analysis. Additionally, it mentions the use of anomaly detection and alert systems for customer service metrics.
![PUBLIC
Datafeed
"aggregations": {
"buckets": {
"date_histogram": {
"field": "LogTime",
"time_zone": "UTC",
"interval": "300s"
},
"aggregations": {
"LogTime": {
"max": {
"field": "LogTime"
}
},
"Notes": {
"significant_text": {
"size": 10,
"min_doc_count": 3,
"shard_min_doc_count": 0,
"field": "Notes",
"filter_duplicate_text": true,
"exclude": [...],
"jlh": {}
Significant Text Aggregation
16](https://image.slidesharecdn.com/chicago-tour-2018-etrade-181015163442/85/Machine-Learning-at-E-Trade-16-320.jpg)
Machine Learning at E*Trade
- 1. Jeff Vestal September 25, 2018 Machine Learning at 1
- 2. PUBLIC First Portal Solution for Mobile Devices 2000 3.9M 1983 Brokerage Accounts Industry First Online Trade by an Individual ~3700 Employees 16.1B Market Cap 2 E*TRADE TECHNOLOGICALLY FOCUSED
- 3. PUBLIC Indexed / Sec Raw documents indexed Indexed / Sec Curated events indexed Historical Events Available on demand real-time 1M+ 200K 50B ELASTIC CLUSTERS AT E*TRADE 3
- 4. PUBLIC Indexed Data On-Line 115TB 38 140B Master Nodes On-Line Documents 99Data Nodes 34Coordinator Nodes 4 ELASTIC CLUSTERS AT E*TRADE
- 5. PUBLIC Business Portals 25 145 450 Curated Dashboards User Logins / Day 150+Base Indices 4B+Micro Service Calls / Day 5 ELASTIC CLUSTERS AT E*TRADE
- 10. PUBLIC ML Jobs On Multiple Clusters 10
- 11. PUBLIC Anomalies and Watches Across Multiple Clusters 11
- 14. PUBLIC 14 Customer Service Case Index
- 15. PUBLIC Significant Text Aggregation An aggregation that returns interesting or unusual occurrences of free-text terms in a set. It is like the significant terms aggregation but differs in that: • It is specifically designed for use on type text fields • It does not require field data or doc-values • It re-analyzes text content on-the-fly meaning it can also filter duplicate sections of noisy text that otherwise tend to skew statistics. Elastic Documentation 15
- 16. PUBLIC Datafeed "aggregations": { "buckets": { "date_histogram": { "field": "LogTime", "time_zone": "UTC", "interval": "300s" }, "aggregations": { "LogTime": { "max": { "field": "LogTime" } }, "Notes": { "significant_text": { "size": 10, "min_doc_count": 3, "shard_min_doc_count": 0, "field": "Notes", "filter_duplicate_text": true, "exclude": [...], "jlh": {} Significant Text Aggregation 16
- 17. PUBLIC Customer Service Notes Significant Text Anomaly Detection 17
- 18. PUBLIC 18
- 19. PUBLIC 19 Slack Anomaly Alerts
- 20. PUBLIC 20 Significant Text Anomaly Explorer
- 21. PUBLIC 21
- 22. PUBLIC 22
- 24. PUBLIC Page Finder ML Job Split by Service Name Looking for Slow Services 24
- 25. PUBLIC Page Finder ML Slack Alert
- 26. PUBLIC Page Finder Anomaly Explorer 26
- 27. PUBLIC Page Finder Single Metric 27
- 28. PUBLIC Page Finder ML Job Split by Service Name 28
- 29. PUBLIC 29 Jeff Vestal September 25, 2018 Machine Learning at