![kind: VirtualHostSpec
spec:
name: api.vhost
domains: ["api.service"]
routes:
- match:
prefix: /
route:
cluster: api.prod.cluster
timeout: 10s
retry_policy:
retry_on: 5xx
num_retries: 3
per_try_timeout: 3s
kind: ClusterSpec
metadata:
annotations:
watcher_name: zookeeper.watcher
paths: ["/pools/api/dc"]
spec:
name: api.prod.cluster
lb_policy: LEAST_REQUEST
connect_timeout: 1s
lb_subset_config:
subset_selectors:
- keys: ["DC"]
- keys: ["IsLocal"]
fallback_policy: DEFAULT_SUBSET
default_subset:
IsLocal: true](https://image.slidesharecdn.com/ivankruglov-introducingenvoy-basedservicemeshatbooking-180502151631/85/Introducing-envoy-based-service-mesh-at-Booking-com-21-320.jpg)
![kind: VirtualHostSpec
spec:
name: api.vhost
domains: ["api.service"]
routes:
- match:
prefix: /
route:
cluster: api.prod.cluster
timeout: 10s
retry_policy:
retry_on: 5xx
num_retries: 3
per_try_timeout: 3s
kind: ClusterSpec
metadata:
annotations:
watcher_name: zookeeper.watcher
paths: ["/pools/api/dc"]
spec:
name: api.prod.cluster
lb_policy: LEAST_REQUEST
connect_timeout: 1s
lb_subset_config:
subset_selectors:
- keys: ["DC"]
- keys: ["IsLocal"]
fallback_policy: DEFAULT_SUBSET
default_subset:
IsLocal: true](https://image.slidesharecdn.com/ivankruglov-introducingenvoy-basedservicemeshatbooking-180502151631/85/Introducing-envoy-based-service-mesh-at-Booking-com-22-320.jpg)
![kind: VirtualHostSpec
spec:
name: api.vhost
domains: ["api.service"]
routes:
- match:
prefix: /
route:
cluster: api.prod.cluster
timeout: 10s
retry_policy:
retry_on: 5xx
num_retries: 3
per_try_timeout: 3s
kind: ClusterSpec
metadata:
annotations:
watcher_name: zookeeper.watcher
paths: ["/pools/api/dc"]
spec:
name: api.prod.cluster
lb_policy: LEAST_REQUEST
connect_timeout: 1s
lb_subset_config:
subset_selectors:
- keys: ["DC"]
- keys: ["IsLocal"]
fallback_policy: DEFAULT_SUBSET
default_subset:
IsLocal: true](https://image.slidesharecdn.com/ivankruglov-introducingenvoy-basedservicemeshatbooking-180502151631/85/Introducing-envoy-based-service-mesh-at-Booking-com-23-320.jpg)
![kind: VirtualHostSpec
spec:
name: api.vhost
domains: ["api.service"]
routes:
- match:
prefix: /
route:
cluster: api.prod.cluster
timeout: 10s
retry_policy:
retry_on: 5xx
num_retries: 3
per_try_timeout: 3s
kind: ClusterSpec
metadata:
annotations:
watcher_name: zookeeper.watcher
paths: ["/pools/api/dc"]
spec:
name: api.prod.cluster
lb_policy: LEAST_REQUEST
connect_timeout: 1s
lb_subset_config:
subset_selectors:
- keys: ["DC"]
- keys: ["IsLocal"]
fallback_policy: DEFAULT_SUBSET
default_subset:
IsLocal: true](https://image.slidesharecdn.com/ivankruglov-introducingenvoy-basedservicemeshatbooking-180502151631/85/Introducing-envoy-based-service-mesh-at-Booking-com-24-320.jpg)
![kind: VirtualHostSpec
spec:
name: api.vhost
domains: ["api.service"]
routes:
- match:
prefix: /
route:
cluster: api.prod.cluster
timeout: 10s
retry_policy:
retry_on: 5xx
num_retries: 3
per_try_timeout: 3s
kind: ClusterSpec
metadata:
annotations:
watcher_name: zookeeper.watcher
paths: ["/pools/api/dc"]
spec:
name: api.prod.cluster
lb_policy: LEAST_REQUEST
connect_timeout: 1s
lb_subset_config:
subset_selectors:
- keys: ["DC"]
- keys: ["IsLocal"]
fallback_policy: DEFAULT_SUBSET
default_subset:
IsLocal: true
envoy cluster spec envoy virtual host spec](https://image.slidesharecdn.com/ivankruglov-introducingenvoy-basedservicemeshatbooking-180502151631/85/Introducing-envoy-based-service-mesh-at-Booking-com-25-320.jpg)
![kind: VirtualHostSpec
spec:
name: api.vhost
domains: ["api.service"]
routes:
- match:
prefix: /
route:
cluster: api.prod.cluster
timeout: 10s
retry_policy:
retry_on: 5xx
num_retries: 3
per_try_timeout: 3s
kind: ClusterSpec
metadata:
annotations:
watcher_name: zookeeper.watcher
paths: ["/pools/api/dc"]
spec:
name: api.prod.cluster
lb_policy: LEAST_REQUEST
connect_timeout: 1s
lb_subset_config:
subset_selectors:
- keys: ["DC"]
- keys: ["IsLocal"]
fallback_policy: DEFAULT_SUBSET
default_subset:
IsLocal: true
envoy cluster spec envoy virtual host spec
bootstrap wizard](https://image.slidesharecdn.com/ivankruglov-introducingenvoy-basedservicemeshatbooking-180502151631/85/Introducing-envoy-based-service-mesh-at-Booking-com-26-320.jpg)
Introducing envoy-based service mesh at Booking.com
- 1. Introducing Envoy-based service mesh at Booking.com Ivan Kruglov KubeCon Europe 2018 02.05.2018
- 2. based in Amsterdam 28M listings 1,5M room nights per day 1700 IT staff
- 3. Agenda • what is service mesh? • why did we start the project? • our setup • learnings • conclusion
- 4. What is service mesh*? * the way I understand it
- 5. application provider service consumer service provider
- 6. application provider service consumer service provider communication infra
- 7. application provider service consumer service provider proxy proxy control plane
- 8. Why did we start service mesh project?
- 9. monolith
- 13. Consistency & Visibility in communications
- 14. Our setup
- 15. application provider proxy service consumer service provider data plane HTTP(S) HTTP
- 16. • graceful restart • TCP proxy
- 17. application provider envoy service consumer service provider data plane HTTP(S) HTTP control plane control plane • routing rules • timeout/retry/etc policies • service discovery
- 18. control plane • in-house (v1/v2 API) • started in August 2017; Istio is around 0.2 • one complex system at a time • start with bare-metal support • minimal abstraction • yup, just write Envoy config (partially) • fine with Envoy’s set of features • self-service for service owners
- 19. application provider envoy service consumer service provider data plane HTTP(S) HTTP control plane control plane • routing rules • timeout/retry/etc policies • service discovery ZooKeeper (service discovery) Kubernetes (service discovery) ZooKeeper (configuration)
- 20. configuration • Envoy configuration is quite rich • power vs. usability: • cluster specification • virtual host specification
- 21. kind: VirtualHostSpec spec: name: api.vhost domains: ["api.service"] routes: - match: prefix: / route: cluster: api.prod.cluster timeout: 10s retry_policy: retry_on: 5xx num_retries: 3 per_try_timeout: 3s kind: ClusterSpec metadata: annotations: watcher_name: zookeeper.watcher paths: ["/pools/api/dc"] spec: name: api.prod.cluster lb_policy: LEAST_REQUEST connect_timeout: 1s lb_subset_config: subset_selectors: - keys: ["DC"] - keys: ["IsLocal"] fallback_policy: DEFAULT_SUBSET default_subset: IsLocal: true
- 22. kind: VirtualHostSpec spec: name: api.vhost domains: ["api.service"] routes: - match: prefix: / route: cluster: api.prod.cluster timeout: 10s retry_policy: retry_on: 5xx num_retries: 3 per_try_timeout: 3s kind: ClusterSpec metadata: annotations: watcher_name: zookeeper.watcher paths: ["/pools/api/dc"] spec: name: api.prod.cluster lb_policy: LEAST_REQUEST connect_timeout: 1s lb_subset_config: subset_selectors: - keys: ["DC"] - keys: ["IsLocal"] fallback_policy: DEFAULT_SUBSET default_subset: IsLocal: true
- 23. kind: VirtualHostSpec spec: name: api.vhost domains: ["api.service"] routes: - match: prefix: / route: cluster: api.prod.cluster timeout: 10s retry_policy: retry_on: 5xx num_retries: 3 per_try_timeout: 3s kind: ClusterSpec metadata: annotations: watcher_name: zookeeper.watcher paths: ["/pools/api/dc"] spec: name: api.prod.cluster lb_policy: LEAST_REQUEST connect_timeout: 1s lb_subset_config: subset_selectors: - keys: ["DC"] - keys: ["IsLocal"] fallback_policy: DEFAULT_SUBSET default_subset: IsLocal: true
- 24. kind: VirtualHostSpec spec: name: api.vhost domains: ["api.service"] routes: - match: prefix: / route: cluster: api.prod.cluster timeout: 10s retry_policy: retry_on: 5xx num_retries: 3 per_try_timeout: 3s kind: ClusterSpec metadata: annotations: watcher_name: zookeeper.watcher paths: ["/pools/api/dc"] spec: name: api.prod.cluster lb_policy: LEAST_REQUEST connect_timeout: 1s lb_subset_config: subset_selectors: - keys: ["DC"] - keys: ["IsLocal"] fallback_policy: DEFAULT_SUBSET default_subset: IsLocal: true
- 25. kind: VirtualHostSpec spec: name: api.vhost domains: ["api.service"] routes: - match: prefix: / route: cluster: api.prod.cluster timeout: 10s retry_policy: retry_on: 5xx num_retries: 3 per_try_timeout: 3s kind: ClusterSpec metadata: annotations: watcher_name: zookeeper.watcher paths: ["/pools/api/dc"] spec: name: api.prod.cluster lb_policy: LEAST_REQUEST connect_timeout: 1s lb_subset_config: subset_selectors: - keys: ["DC"] - keys: ["IsLocal"] fallback_policy: DEFAULT_SUBSET default_subset: IsLocal: true envoy cluster spec envoy virtual host spec
- 26. kind: VirtualHostSpec spec: name: api.vhost domains: ["api.service"] routes: - match: prefix: / route: cluster: api.prod.cluster timeout: 10s retry_policy: retry_on: 5xx num_retries: 3 per_try_timeout: 3s kind: ClusterSpec metadata: annotations: watcher_name: zookeeper.watcher paths: ["/pools/api/dc"] spec: name: api.prod.cluster lb_policy: LEAST_REQUEST connect_timeout: 1s lb_subset_config: subset_selectors: - keys: ["DC"] - keys: ["IsLocal"] fallback_policy: DEFAULT_SUBSET default_subset: IsLocal: true envoy cluster spec envoy virtual host spec bootstrap wizard
- 27. control plane ZooKeeper VC VC VC service A expose to service D service C expose to service Dservice B envoy I’m service D here is service A and service C specs
- 28. application provider envoy service consumer service provider data plane HTTP(S) HTTP control plane control plane • routing rules • timeout/retry/etc policies • service discovery ZooKeeper (service discovery) Kubernetes (service discovery) ZooKeeper (configuration) configuration changes
- 29. Deploying changes • integration with existent infrastructure • git-deploy • vanguard (canary) deployment • syntax and semantic validation • fast rollback
- 30. application provider envoy service consumer service provider data plane HTTP(S) HTTP control plane control plane • routing rules • timeout/retry/etc policies • service discovery ZooKeeper (service discovery) ZooKeeper (configuration) configuration changes metrics Kubernetes (service discovery)
- 31. Monitoring • approach #1 – graphite • excellent in-house facilities • aggregations is too heavy at scale • approach #2 – prometheus • statsd support • statsd_exporter • still iterating • standard dashboard envoy statsd_exporter envoy…
- 33. application provider envoy service consumer service provider data plane HTTP(S) HTTP control plane control plane • routing rules • timeout/retry/etc policies • service discovery ZooKeeper (service discovery) ZooKeeper (configuration) configuration changes metrics Kubernetes (service discovery)
- 34. Production
- 35. Some numbers • ~ 6 months in production • ~ 30 projects • ~ 10K servers • hundreds of thousands RPS • overheads…
- 37. HTTPS perceived latency p50 p75 p90 p95 p99 10ms + 1ms 15ms 20ms 25ms 30ms 40ms
- 38. Some findings • graceful restart works, but not always! • (major) Envoy upgrades may not be graceful • ”x-envoy-retry-on = connect-failure” – too few “x-envoy-retry-on = 5xx” – too much • gateway-error – HTTP 502, 503, 504 • absence of TCP keepalive => stale connections • coming in 1.7 • idle_timeout in TCP proxy in 1.6 • cluster_name (1.5) -> cluster_names (1.6)
- 39. Conclusion
- 40. service mesh* is a building block on the path to SOA technically & organizationally * the way I understand it
- 41. Andrei Vereha Envoy production stories Friday, May 4th 14:00 Envoy Deep Dive