2017 Microservices Practitioner Virtual Summit: Microservices at Squarespace - Doug Jones and Kevin Lynch, Microservices Practitioner Virtual Summit 2017
3 likes2,835 views
The document outlines the evolution and architecture of microservices at Squarespace, detailing the transition from monolithic systems to microservices for scalability and reliability. It discusses key pillars such as service discovery, observability, fault tolerance, and the need for efficient monitoring and orchestration using Kubernetes. Additionally, it highlights challenges faced with static infrastructure and emphasizes the adoption of containerization for improved resource management and deployment processes.
2017 Microservices Practitioner Virtual Summit: Microservices at Squarespace - Doug Jones and Kevin Lynch, Microservices Practitioner Virtual Summit 2017
- 2. Part 1: Building the Pillars of Microservices Part 2: Containerization and Orchestration (Kubernetes) AGENDA
- 3. Part 1: Building the Pillars 01 The Journey to Microservices 02 Building the Pillars of Microservices
- 5. Microservices Journey: A Story of Growth 2013: small (< 50 engineers) build product & grow customer base whatever works 2014: medium (< 100 engineers) we have a lot of customers now! whatever works doesn't work anymore 2016: large (100+ engineers) architect for scalability and reliability organizational structures ?: XL (200+ engineers)
- 6. Challenges with a Monolith ● Reliability ● Performance ● Engineering agility/speed, cross-team coupling ● Engineering time spent fire fighting rather than building new functionality What were the increasingly difficult challenges with a monolith?
- 7. https://www.squarespace.com/?gclid=<unique-id> Challenges with a Monolith Story of an Outage...During the Super Bowl
- 8. Challenges with a Monolith ● Monitoring typically starts at the edges ○ Think requests in, DB queries out, etc ● What about the guts of the app? How much visibility do you have there? ● How long does it take you to recover from an issue? Find the cause and fix the issue? Challenges with Monitoring/Finding Faults
- 9. The Journey to Microservices ● Define Pillars: ideas we consider necessary for successful production microservices ● Implement these pillars as part of our platform ● Reduce boilerplate and reinventing the wheel syndrome ● Service authors get these for free and can focus on their application domain Design a Platform for Production, Remove Challenges
- 10. Pillars Microservice Framework HTTP API Service Discovery Software Load Balancer Observability Async Client Fault Tolerance https://engineering.squarespace.com/blog/2017/the-pill ars-of-squarespace-services
- 11. Platform Features Service Discovery API Documentation Structured Logging Metrics & Dashboards Distributed Tracing Contextual Information Alert Definitions Standardized Deployments Healthchecks Dynamic Configuration Client-Side Load Balancing Latency & Fault Tolerance Client-Side Caching HTTP Request Builders Code Generation Service Dashboard Traffic Visualization Server Platform Client Platform ToolingTooling
- 12. Platform Features Service Discovery API Documentation Structured Logging Metrics & Dashboards Distributed Tracing Contextual Information Alert Definitions Standardized Deployments Healthchecks Dynamic Configuration Client-Side Load Balancing Latency & Fault Tolerance Client-Side Caching HTTP Request Builders Code Generation Service Dashboard Traffic Visualization Async/Reactive Alert Management Log Aggregation
- 13. Building the Pillars of Microservices ● HTTP + JSON ○ Industry standard. Tons of tools. ● Solid open source Java API server platforms ○ Started with Dropwizard ○ now on Spring Boot (configured to use Jetty and Jersey 2) Pillar: HTTP APIs
- 14. Building the Pillars of Microservices ● Swagger (OpenAPI Specification) ● Code generation ○ Swagger spec → models, server API, client Even Easier APIs
- 15. Swagger Path Example paths: /currency-info: put: tags: - CurrencyInfo description: "Creates a new {@link CurrencyInfo} resource." summary: Create a new currency info operationId: save parameters: - name: info in: body schema: $ref: '#/definitions/CurrencyInfo' responses: 200: description: ok schema: $ref: '#/definitions/CurrencyInfo'
- 17. Building the Pillars of Microservices ● Services announce themselves, publishing their name and host/port information ● Started with a simple announcement payload and found that was enough ● Healthchecks to mark services down Pillar: Service Discovery
- 18. Building the Pillars of Microservices ● First: Zookeeper ○ Complicated clients (no HTTP API) ○ Must build discovery on Zookeeper primitives ○ Strong consistency is unnecessary ○ Client heartbeats can’t be expanded upon ○ No great way to support multiple data centers Service Discovery Systems
- 19. ● Now: Consul ○ First class discovery support ○ Built in multi-data center support ○ Simple HTTP API ○ Configurable healthchecks ○ key/value store ■ We use for dynamic config and leader election Building the Pillars of Microservices Service Discovery Systems
- 20. DC2DC1 Multi DC with Consul ConsulConsulConsul ConsulConsulConsul Service Announce Service Announce Primary DB Replica DB Replicate WAN Gossip Consistent Set
- 21. DC2DC1 Multi DC with Consul ConsulConsulConsul ConsulConsulConsul Service Service Primary DB Replica DB Replicate Service Query ?dc=”DC2” Remote DC forwarding
- 22. Building the Pillars of Microservices ● Avoid middleware/extra configuration ● Customizable logic ● Connection pooling ● System awareness to increase fault tolerance ● Builds on Netflix Ribbon OSS Pillar: Software Load Balancers
- 23. Building the Pillars of Microservices ● Metrics ● Dashboards ● Distributed Tracing ● Structured Logging ● Healthchecks ● Alerts Pillar: Observability
- 26. Structured Logging tail -f /data/logs/taxation-access.log 2017-03-22 07:24:45:026 GMT thread=jetty-846 contextId=JaOLrH2O contextSourceType=billing clientVersion=taxation-client-3.1 level=INFO class=AccessLog ip=10.100.101.205 method=GET uri=/api/1/taxation/rates queryString= httpVersion=HTTP/1.1 responseCode=200 responseTimeMs=39
- 27. Contextual Information Client v3.1 Taxation Service Billing Service Context IdClient Version Client Source Type JaOLrH2O
- 28. Building the Pillars of Microservices ● Addresses the Fanout problem, improved latency ● Reactive: RxJava with RxNetty ● Allows greater composition and reuse. Avoid “callback hell” Pillar: Async Client
- 29. Fanout Depicted Client Service A Service Z Application Container Service B Service C Service D
- 30. Sync Execution Client Service A Service Z Application Container Service B Service C Service D 1 2 3 4 5 Total Latency = A + B + C + D + Z
- 31. Async Execution Client Service A Service Z Application Container Service B Service C Service D 1 2 2 2 1 Total Latency = max(A, Z) A = max(B, C, D) + A’s latency
- 32. Building the Pillars of Microservices ● Circuit breakers ● Retry logic ○ Much easier to implement w/ RxJava ● Timeouts ● Fallbacks (cached or static values) ● Netflix Hystrix Pillar: Fault Tolerance
- 33. Fault Tolerance Service B Service A Service C Service A Client Service B Client Service C Client User Request Application Container
- 34. Fault Tolerance Service B Service A Service C Service A Client 10 Threads Service B Client 5 Threads Service C Client 5 Threads User Request Fail fast, fail silent, or fallback Application Container
- 35. Pillars Microservice Framework HTTP API Service Discovery Software Load Balancer Observability Async Client Fault Tolerance https://engineering.squarespace.com/blog/2017/the-pill ars-of-squarespace-services
- 36. Building the Pillars of Microservices ● Entirely Async Systems ○ Async servers, Streaming, gRPC, Netty ● Distributed task management ○ Serverless computing ● Easier/better alert definition and management ● Better tooling to create and deploy services Future Work
- 37. Part 2: Containerization & Kubernetes Orchestration 01 The problem with static infrastructure 02 Kubernetes in a datacenter? 03 Challenges
- 38. Containerization & Kubernetes Orchestration ● Engineering org grows... ● More services… ● More infrastructure to spin up… ● Ops becomes a blocker... Stuck in a loop
- 39. Containerization & Kubernetes Orchestration ● Difficult to find resources ● Slow to provision and scale ● Already have discovery! ● Metrics system must support short lived metrics ● Alerts are usually per instance Static infrastructure and microservices do not mix!
- 40. Traditional Provisioning Process ● Pick ESX with available resources ● Pick IP ● Register host to Cobbler ● Register DNS entry ● Create new VM on ESX ● PXE boot VM and install OS and base configuration ● Install system dependencies (LDAP, NTP, CollectD, Sensu…) ● Install app dependencies (Java, FluentD/Filebeat, Consul, Mongo-S…) ● Install the app ● App registers with discovery system and begins receiving traffic
- 41. Kubernetes Provisioning Process ● kubectl apply -f app.yaml
- 42. Containerization & Kubernetes Orchestration ● Provisioning/Scaling: Kubernetes ● Monitoring: Prometheus ● Alerting: AlertManager ● Discovery: Consul + Kubernetes ● Decentralization So how do we make this magic work?
- 43. Kubernetes in a datacenter?
- 45. Spine and Leaf Layer 3 Clos Topology ● Each leaf switch represents a Top-of-Rack switch (ToR) ● All work is performed at the leaf switch ● Each leaf switch is separate Layer 3 domain ● Each leaf is a separate BGP domain (ASN) ● No Spanning Tree Protocol issues seen in L2 networks (convergence time, loops) Leaf Leaf Leaf Leaf Spine Spine
- 46. Spine and Leaf Layer 3 Clos Topology ● Simple to understand ● Easy to scale ● Predictable and consistent latency (hops = 2) ● Allows for Anycast IPs Leaf Leaf Leaf Leaf Spine Spine
- 47. Calico Networking ● No network overlay required ● Communicates directly with existing L3 mesh network ● BGP Peering with Top of Rack switch ● Calico supports Kubernetes NetworkPolicy firewall rules
- 48. Monitoring
- 49. ● Graphite does not scale well with ephemeral instances ● Easy to have combinatoric explosion of metrics Traditional Monitoring & Alerting ● Application and system alerts are tightly coupled ● Difficult to create alerts on SLAs ● Difficult to route alerts
- 50. Kubernetes Monitoring & Alerting
- 51. Kubernetes Monitoring & Alerting
- 52. Kubernetes Monitoring & Alerting
- 53. Microservice Pod Definition resources: requests: cpu: 2 memory: 4Gi limits: cpu: 2 memory: 4Gi Microservice Pod Java Microservice fluentd consul
- 54. Challenges
- 55. Microservice Pod Definition resources: requests: cpu: 2 memory: 4Gi limits: cpu: 2 memory: 4Gi ● Kubernetes assumes no other processes are consuming significant resources ● Completely Fair Scheduler (CFS) ○ Schedules a task based on CPU Shares ○ Throttles a task once it hits CPU Quota
- 56. Microservice Pod Definition resources: requests: cpu: 2 memory: 4Gi limits: cpu: 2 memory: 4Gi ● Shares = CPU Request * 1024 ● Total Kubernetes Shares = # Cores * 1024 ● Quota = CPU Limit * 100ms ● Period = 100ms
- 57. Java in a Container ● JVM is able to detect # of cores via sysconf(_SC_NPROCESSORS_ONLN) ● Scales tasks relative to this
- 58. Java in a Container ● Provide a base container that calculates the container’s resources! ● Detect # of “cores” assigned ○ /sys/fs/cgroup/cpu/cpu.cfs_quota_us divided by /sys/fs/cgroup/cpu/cpu.cfs_period_us ● Automatically tune the JVM: ○ -XX:ParallelGCThreads=${core_limit} ○ -XX:ConcGCThreads=${core_limit} ○ -Djava.util.concurrent.ForkJoinPool.common.parallelism=${core_limit }
- 59. Java in a Container ● Many libraries rely on Runtime.getRuntime.availableProcessors() ○ Jetty ○ ForkJoinPool ○ GC Threads ○ That mystery dependency...
- 60. Java in a Container ● Use Linux preloading to override availableProcessors() #include <stdlib.h> #include <unistd.h> int JVM_ActiveProcessorCount(void) { char* val = getenv("CONTAINER_CORE_LIMIT"); return val != NULL ? atoi(val) : sysconf(_SC_NPROCESSORS_ONLN); } https://engineering.squarespace.com/blog/2017/understanding-linux-container-scheduling
- 61. Communication With External Services ● Environment specific services should not be encoded in application ● Single deployment for all environments and datacenters ● Federation API expects same deployment ● Not all applications are using consul
- 62. Communication With External Services
- 63. Communication With External Services apiVersion: v1 kind: Service metadata: name: kafka namespace: elk spec: type: ClusterIP clusterIP: None sessionAffinity: None ports: - port: 9092 protocol: TCP targetPort: 9092 apiVersion: v1 kind: Endpoints metadata: name: kafka namespace: elk subsets: - addresses: - ip: 10.120.201.33 - ip: 10.120.201.34 - ip: 10.120.201.35 ... ports: - port: 9092 protocol: TCP
- 64. So what’s left?
- 65. Future Work: Enforce Squarespace Standards ● Custom Admission Controller requires all services, deployments, etc. meet certain standards ○ Resource requests/limits ○ Owner annotations ○ Service labels
- 66. Future Work: Updating Common Dependencies ● Custom Initializers ○ Inject container dependencies into deployments (consul, fluentd) ○ Configure Prometheus instances for each namespace ● Trigger rescheduling of pods when dependencies need updating apiVersion: extensions/v1beta1 kind: Deployment metadata: name: location namespace: core-services annotations: initializer.squarespace.net/consul: "true"