Paris Container Day 2016 : Etcd - overview and future (CoreOS)
5 likes885 views
The document provides an overview of etcd, a distributed key-value store that supports high availability, strong consistency, and scalability for cloud-native applications. It highlights etcd's evolved version (etcd v3), which offers better performance and enhanced APIs suitable for large-scale systems like Kubernetes. The future developments for etcd include features such as proxying, caching, and support for Kubernetes as it scales, aiming for reliability and efficient management even at high loads.
Paris Container Day 2016 : Etcd - overview and future (CoreOS)
- 1. Jonathan Boulle @baronboulle | jonathan.boulle@coreos.com etcd - overview and future
- 2. Why etcd?
- 4. ... ... ... ... ... ... Unavailable Uncoordinated Upgrades
- 5. Motivation CoreOS cluster reboot lock - Decrement a semaphore key atomically - Reboot and wait... - After reboot increment the semaphore key
- 9. ... ... ... CoreOS updates coordination 0
- 10. ... ... ... CoreOS updates coordination 0
- 11. ... ... CoreOS updates coordination 0
- 12. ... ... CoreOS updates coordination 0
- 13. ... ... CoreOS updates coordination 1
- 14. ... ... ... CoreOS updates coordination 0
- 17. config Start / RestartStart / Restart Store Application Configuration
- 20. Requirements Strong Consistency - mutual exclusive at any time for locking purpose Highly Available - resilient to single points of failure & network partitions Watchable - push configuration updates to application
- 21. Requirements CAP - Consistency, Availability, Partition Tolerance: choose 2 - We want CP - We want something like Paxos
- 22. Common problem GFS Paxos Big Table Spanner CFS Chubby Google - “All” infrastructure relies on Paxos
- 23. Common problem Amazon - Replicated log powers ec2 Microsoft - Boxwood powers storage infrastructure Hadoop - ZooKeeper is the heart of the ecosystem
- 24. COMMON PROBLEM #GIFEE and Cloud Native Solution
- 25. 10,000 Stars on Github 250 contributors Google, Red Hat, EMC, Cisco, Huawei, Baidu, Alibaba...
- 26. THE HEART OF CLOUD NATIVE Kubernetes, Cloud Foundry's Diego, Docker's SwarmKit, many others
- 27. ETCD KEY VALUE STORE Fully Replicated, Highly Available, Consistent
- 28. PUT(foo, bar), GET(foo), DELETE(foo) Watch(foo) CAS(foo, bar, bar1) Key-value Operations
- 30. Runtime Reconfiguration Point-in-time Backup Extensive Metrics etcd Operationality
- 31. ETCD v3 Successor of etcd v2
- 34. ETCD v3 More Efficient & Powerful APIs
- 35. gRPC Based API ~4x Faster vs JSON HTTP/2 Improves Efficiency
- 36. Multi-Version Put(foo, bar) Put(foo, bar1) Put(foo, bar2) Get(foo) -> bar2
- 37. Multi-Version Put(foo, bar) Put(foo, bar1) Put(foo, bar2) Get(foo, 1) -> bar
- 38. Tx.If( Compare(Value("foo"), ">", "bar"), Compare(Version("foo"), "=", 2), ... ).Then( Put("ok","true")... ).Else( Put("ok","false")... ).Commit() Mini-Transactions
- 39. l = CreateLease(15 * second) Put(foo, bar, l) l.KeepAlive() l.Revoke() Leases
- 40. w = Watch(foo) for { r = w.Recv() print(r.Event) // PUT print(r.KV) // foo,bar } Streaming Watch
- 42. ETCD v2 machine coordination -> O(10k)
- 43. ETCD v3 app/container coordination -> O(1M)
- 44. Reliability 99% at small scale is easy - Failure is infrequent and human manageable 99% at large scale is not enough - Not manageable by humans 99.99% at large scale - Reliable systems at bottom layer
- 45. HOW DO WE ACHIEVE RELIABILITY WAL, Snapshots, Testing
- 46. Write Ahead Log Append only - Simple is good Rolling CRC protected - Storage & OSes can be unreliable
- 47. Snapshots Torturing DBs for Fun and Profit (OSDI2014) - The simpler database is safer - LMDB was the winner Boltdb an append only B+Tree - A simpler LMDB written in Go
- 48. Testing Clusters Failure Inject failures into running clusters White box runtime checking - Hash state of the system - Progress of the system
- 49. Testing Cluster Health with Failures Issue lock operations across cluster Ensure the correctness of client library
- 53. etcd/raft Reliability Designed for testability and flexibility Used by large scale db systems and others - Cockroachdb, TiKV, Dgraph
- 54. etcd vs others Do one thing
- 55. etcd vs others Only do the One Thing
- 56. etcd vs others Do it Really Well
- 57. etcd Reliability Do it Really Well
- 58. ETCD v3.0 BETA Efficient and Scalable
- 60. FUTURE WORK Proxy, Caching, Watch Coalescing, Secondary Index
- 61. ETCD and KUBERNETES The Data Store
- 62. worker kubelet worker kubelet worker kubelet scheduler & API worker kubelet w kut worker kubelet
- 63. etcd and Kubernetes - Kubernetes currently uses the V2 API - Work very actively in process to migrate to V3 - Opt-in currently, default in future
- 64. etcd v3 and Kubernetes - Follow along: https://github.com/kubernetes/kubernetes/issues/22448 - Try it out!
- 65. etcd v3 will support Kubernetes as it scales to 5.000 nodes and beyond
- 67. Performance Snapshot caused performance degradation etcd2 - 600K keys
- 68. Performance etcd2 - 600K keys Snapshot triggered elections
- 69. ZooKeeper Performance Non-blocking full snapshot Efficient memory management
- 74. Reliable Performance - Similar to ZooKeeper with snapshot disabled - Incremental snapshot - No Garbage Collection Pauses - Off-heap storage
- 75. Performance etcd3 /ZooKeeper snapshot disabled
- 76. Performance etcd3 /ZooKeeper snapshot disabled
- 77. Memory 10GB 2.4GB 0.8GB 512MB data - 2M 256B keys