![B-Tree vs. Models
• Task: Predict the offset of value given a key
• Input: key
• Output:
• B-Tree: [pos, pos + pagesize]
• Model: [pos - min_err, pos + max_err]](https://image.slidesharecdn.com/learnedindexsharing-180613122859/85/The-Case-for-Learned-Index-Structures-5-320.jpg)
![B-Tree vs. Models
• Task: Predict the offset of value given a key
• Input: key
• Output:
• B-Tree: [pos, pos + pagesize]
• Model: [pos - min_err, pos + max_err]
How to bound
min_err, max_err? No test dataset!](https://image.slidesharecdn.com/learnedindexsharing-180613122859/85/The-Case-for-Learned-Index-Structures-18-320.jpg)
The Case for Learned Index Structures
- 1. The Case for Learned Index Structures Eric Fu
- 2. Agenda • Introduction • Background • Range Index • RM-Index • Performance • Point Index & Existence Index
- 3. Agenda • Introduction • Background • Range Index • RM-Index • Performance • Point Index & Existence Index
- 4. Introduction • Index: map key to position efficiently • B-Tree • Self-balanced binary search tree • Store on disk • Lookup in O(log n)
- 5. B-Tree vs. Models • Task: Predict the offset of value given a key • Input: key • Output: • B-Tree: [pos, pos + pagesize] • Model: [pos - min_err, pos + max_err]
- 6. Agenda • Introduction • Background • Range Index • RM-Index • Performance • Point Index & Existence Index
- 7. What is Machine Learning? • Machine learning is a field of computer science that gives computers the ability to learn without being explicitly programmed. • Statistics: collect data build model predict
- 8. Problems • Regression 回归 • Classification 分类 • Clustering 聚类 Clustering
- 9. Algorithms • Linear Regression 线性回归 • Decision Tree 决策树 • Neural Network 神经网络 • Support Vector Machine (SVM) 支持向量机 • Bayes Classifier 贝叶斯分类器 • K-means .......
- 14. Deep Neural Network 深度神经网络 GoogLeNet, 22 layers
- 15. Machine Learning • A regular process • Feature extraction • Train model • Test model • Objective function • minimum error (e.g. MSE)
- 16. Machine Learning • The biggest challenge - overfitting
- 17. Agenda • Introduction • Background • Range Index • RM-Index • Performance • Point Index & Existence Index
- 18. B-Tree vs. Models • Task: Predict the offset of value given a key • Input: key • Output: • B-Tree: [pos, pos + pagesize] • Model: [pos - min_err, pos + max_err] How to bound min_err, max_err? No test dataset!
- 19. Index as a Function • B-Tree or ML model are fitting this curve in different approach.
- 21. Agenda • Introduction • Background • Range Index • RM-Index • Performance • Point Index & Existence Index
- 22. Recursive Model Index (RMI) Root and middle nodes • Pick a model for next stage Leaf nodes • Predict position
- 23. Recursive Model Index (RMI) Solved last-mile dilemma! • 100M - 1M - 10K – 100 • Not a tree
- 24. Hybrid Index Parameter: model structures
- 25. Agenda • Introduction • Background • Range Index • RM-Index • Performance • Point Index & Existence Index
- 26. Test datasets • weblogs：访问时间 timestamp -> log entry (~200M) • maps：纬度 longitude -> locations (~200M) • web-documents：documents (strings) -> document-id (~10M) • lognormal：
- 27. Test models • B-Tree with different page sizes • very competitive performance • RMI with 2-stage models using simple grid-search • 0 to 2 hidden layers • layer-width ranging from 4 to 32 nodes • Total time = lookup time + search time
- 31. Conclusion • Up to 3x faster • An order-of-magnitude smaller • Data distribution dependent
- 32. Inserts and Updates • Achilles heel of learned indexes because of the potentially high cost for learning models • Introduce additional space in sorted dataset, similar to a B-Tree • Assume that the inserts follow roughly a similar pattern as the learned CDF • What happens if the distribution changes? • Retrain model. Stage 2 -> Stage 1 • Delta-index
- 33. Agenda • Introduction • Background • Range Index • RM-Index • Performance • Point Index & Existence Index
- 34. Point Index • Hash collisions • Probing (e.g. linked list) • Trade-off between time and space • Learned Hash-map • more uniform hash function • more uniquely mapping
- 35. Point Index • Baseline Hash-map • only uses 2 multiplications, 3 bitshifts and 3 XORs • 2-stage RMI models • 100k models on the 2nd stage • without any hidden layers. • available slots from 75% to 125% of the data
- 36. Existence Index • most importantly Bloom-Filters • Guarantee no false negative • Potential false positive (FP) • Targeted FPR = 0.1% then ~14x bits • Targeted FPR = 0.01% then ~18x bits
- 37. Bloom-filters with learned hash-functions • We denote the set of keys by K and the set of non-keys by U • Dataset of non-keys U • randomly generated keys • based on logs of previous queries • generated by another ML model • A binary classification task • NN with Sigmoid activation function
- 38. Bloom-filters with learned hash-functions • non-zero FPR and FNR • as the FPR goes down, the FNR will go up • How to preserve FPR = 0 constraint? • an overflow Bloom filter
- 39. Bloom-filters with learned hash-functions • Task: blacklisted phishing URLs (~1.7M) • a character-level RNN (GRU) • Bloom filter • desired 1% FPR requires 2.04MB space • Our approach • GRU model: 0.0259MB • With spill-over Bloom filter: 1.07MB (save ~47%)
- 40. Thank You Q&A
- 41. Furthermore... • Machine Learning by Zhou Zhi-hua • TensorFlow and deep learning without a PhD