Paper Reading: Smooth Scan
Download as PPTX, PDF0 likes206 views
The document discusses a robust access path selection method that avoids relying on cardinality estimation, highlighting the shortcomings of classical approaches which depend on outdated statistics and naive cost models. It proposes a new scanning mechanism that adapts at runtime, allowing for more effective selection without needing extensive statistics. The document details various policies for morphing scan methods and incorporates these innovations into a PostgreSQL implementation to enhance performance in diverse data situations.
Paper Reading: Smooth Scan
- 1. Robust Access Path Selection without Cardinality Estimation Presented by Kenan Yao
- 2. PingCAP.com Problem to Solve ● access path selection ● classical approach ○ predicate push down ○ ranger extracts access conditions ○ selectivity estimation / statistics ○ cost model ○ independent / uniform assumptions Transaction
- 3. PingCAP.com Drawbacks ● drawbacks of classical approach ○ heavily depends on statistics ○ outdated statistics / expensive to maintain ○ naive cost model ○ assumptions not hold ○ wrong choices / not robust
- 4. PingCAP.com Proposal ● a proposal to dodge those drawbacks ○ new scan operator in executor ○ observe and behave accordingly at runtime ○ relieve planner from access path selection ○ robust by eschewing statistics / cardinality estimation
- 5. PingCAP.com Modeling the Problem ● page caches not considered ● double read ● sensitive / not robust Transaction
- 6. PingCAP.com Alternative Approaches ● optimizer level: not robust enough ● runtime reoptimization ○ start from index scan ○ switch to table scan if scanned rows exceeds tipping point ○ binary switch ○ bound worst case ○ risk area ○ not robust Transaction
- 7. PingCAP.com Smooth Scan ● rough idea ○ executor operator / runtime ○ start from index scan ○ morph between index scan and table scan ○ morph continuously and adaptively ○ trade off CPU and memory for I/O reduction
- 8. PingCAP.com Storage Model ● PostgreSQL style: heap table / B+-tree index ● access path types ○ table scan / index scan ○ bitmap scan ■ reduce random access / cache miss ■ execution model: pipeline breaker ■ order property lost Transaction
- 9. PingCAP.com Smooth Scan Details ● targeted behavior ○ near-optimal ○ bullet proof of the estimation errors ○ no performance cliff / robust Transaction
- 10. PingCAP.com Morphing Mechanism ● start from simple index scan ○ monitor retrieved row count ● start morphing if selectivity exceeds threshold ○ probe entire heap page ○ fetch and probe adjacent heap pages if selectivity keeps increasing ■ start from extra one page ■ morphing region size increases exponentially
- 11. PingCAP.com Correctness ● no tuple missed ○ driven by index scan ● no duplicate tuple ○ bookkeeping ○ CPU / memory cost
- 12. PingCAP.com Morphing Policy ● greedy policy ○ fast convergence for high selectivity ○ unnecessary overhead for low selectivity ● selectivity driven policy ○ monitor local and global selectivity ○ selectivity computed in page level ○ keep or increase morphing region size ● elastic policy ○ skewed data distribution ○ double morphing region size for dense region ○ halve morphing region size for sparse region
- 13. PingCAP.com Threshold ● optimizer driven ○ retrieved row count exceeds optimizer’s estimate ● eager approach ○ start morphing from first tuple ● SLA driven
- 14. PingCAP.com Implementation ● integrated with PostgreSQL ○ page ID cache ○ tuple ID cache ○ result cache to respect order property ■ hash-based data structure ■ store additional tuples found ■ check result cache before index probe ■ pipeline breaker to some extent ■ spill to disk when short of memory
- 15. PingCAP.com Evaluation ● TPC-H and synthetic datasets ● clear database and OS caches
- 16. Any Questions ? Thank You!