gRPC @ Weaveworks
3 likes976 views
Tom Wilkie, a director of software engineering at Weaveworks, discusses the company's advancements in using gRPC for various services, including Prometheus integration and monitoring. The document highlights improvements in data ingestion speed from over 10 minutes to approximately 14 seconds using gRPC streaming. Weaveworks is actively hiring in London, Berlin, and San Francisco.
1 of 14
Download to read offline
More Related Content
What's hot (20)
Ad
Viewers also liked (20)
Ad
Similar to gRPC @ Weaveworks (20)
More from Weaveworks (20)
Recently uploaded (20)
gRPC @ Weaveworks
- 1. @ Weaveworks Tom Wilkie, 14/03/2017 Weave Online Users Group
- 2. Who Am I? • Director Software Engineering at Weaveworks • Home brewer Previously: • SRE for Google Analytics • Founder/CTO at Acunu • Engineer at XenSource @tom_wilkie https://github.com/tomwilkie
- 3. Work on https://cloud.weave.works, our service for exploring, deploying and monitoring Micro Services…
- 4. 1. gRPC for Prometheus Retriever scraping your jobs Your DC Weave Cloud Frontend, Authenticator Distributor Ingester Distributor… IngesterIngester DynamoDB S3 http://goo.gl/qsJlrQ gRPC?
- 5. 1. gRPC for Prometheus Prometheus AWS ELB Nginx Cortex weavecortex
- 6. 2. gRPC intra service Weave Cloud Frontend, Authenticator Distributor Ingester Distributor… IngesterIngester DynamoDB S3 gRPC? gRPC? gRPC?
- 9. 4. HTTP over gRPC https://www.weave.works/turtles-way-http-grpc/ https://github.com/weaveworks/common/tree/master/httpgrpc Frontend Backend weavecortex Client HTTP gRPC - Frontend parses HTTP request - Sends generic gRPC request - Backend turns it back in HTTP
- 10. 4. HTTP over gRPC
- 11. 5. gRPC Streaming • Cortex rolling upgrades • Flush all data to S3 • Took >10mins per instance
- 12. 5. gRPC Streaming New system uses gRPC streaming to send data to other ingester. Now takes ~14s per instance. leaving instance weavecortex joining instance weavecortex gRPC stream
- 13. 1. gRPC for Prometheus 2. gRPC from Weave Cloud 3. gRPC monitoring 4. HTTP over gRPC 5. gRPC streaming
- 14. We’re hiring! London BerlinSan Francisco