BigTable PreReading
Download as PPT, PDF0 likes472 views
This document outlines the key concepts of Google's Bigtable distributed database system. It discusses Bigtable's data model, APIs, implementation details including its use of GFS and Chubby, refinements to improve performance, and lessons learned. The document poses many questions about Bigtable's design and implementation for further discussion.
BigTable PreReading
- 1. BIGTABLE READING 2013.04.02 xielun.szd@alipay.com
- 2. OutLine Introduction Data Model API Building Blocks Implementation Refinements Performance Evaluation Real Applications Lessons Related Work Conclusions
- 3. OutLine Introduction Data Model API Building Blocks Implementation Refinements Performance Evaluation Real Applications Lessons Related Work Conclusions
- 4. Introduction Why Google Need BigTable ? Why BigTable Need LSM ?
- 5. Data Model BigTable Is A Sparse Sorted Map Key: <Row, Column Family, Qualifier, Timestamp>
- 6. API How to Support Single-Row transaction Atomic Read-modify-write Using API for MapReduce Input source & Output target
- 7. Building Blocks GFS SSTable Data File Commit Log (WAL & Redo Log) Chubby Distributed Lock Service Root Tablet Location Tablet server Manager Schema & ACL Metadata Why GFS not use chubby For Master? If Chubby not exist, how to design the bigtable?
- 8. Implementations Client How To Protect Lighted Master ? Master Tablet server Manager Tablet Assignment Garbage Collection Schema Change Do We Need Slave Master For Availability? Tablet Server Reader & Writer
- 9. Implementations Tablet Locations
- 10. Implementations Why select Three-level hierarchy? Why Persistent the location info? Why analogous as B+tree? The Most RPC times during One simple Query?
- 11. Implementations Tablet Assignment Consistency Guaranty ? Tablet Server Offline detection Procedure Master Recovery Procedure ? Master Memory != Metadata Table Metadata Table Schema Design?
- 12. Implementations Tablet Serving Read & Write & Recovery Procedure
- 13. Implementations Compactions Three Type Compactions Differences If client Write 1GB data, How much data in GFS?
- 14. Refinements How To Reduce IO? Column Store Compression Block Size Caching Caching Consistency Bloom Filter Why Must Using Bloom Filter For LSM? Group Commit Prons Vs Cons
- 15. Refinements Log GFS exist duplicated record? What Happened when Tablet Server Crashes? Tablet Recovery Tablet Migrate Compare with Tair & OB Tablet Split Tablet Merge How can do this online?
- 16. Performance Compare With Single Server Performance Low Latency Vs High Through output Why Performance/Server Decrease？
- 18. Lessons Vulnerable to many types of failure RPC crc checksum Removing assumption Say No to adding new features System-level Monitoring Simple Design
- 19. Related Work CAP in BigTable? Sharing Nothing or Sharing Disk or Sharing Memory? CStore or BigTable or OceanBase
- 20. Assignment ALL Questions Above CStore, Bloom Filter Chubby, Megastore Dynamo Cassandra Riak Volmort Redis cluster Thanks & QA