Service Mesh Talk for CTO Forum
0 likes323 views
The document discusses the architecture and evolution of service meshes in microservices, emphasizing their crucial role in managing complexities like service discovery, routing, and security. It highlights the benefits of adopting microservices along with CI/CD practices to achieve faster and more efficient software delivery. Additionally, it outlines various patterns and best practices for implementing microservices and service meshes, including observability and traffic management strategies.
Service Mesh Talk for CTO Forum
- 1. CTO Forum Service Mesh Draft 2 Microservice Journey Service Mesh Architecture Service Mesh Service Mesh Concerns Service Mesh Security Service Mesh Evolution
- 2. [ 2 Author of best-selling agile development book Early adopter of Microservices, TDD, DevOps, Agile, Container Orchestration, 12 factor deployments, KPIs/ metric, health checks, tracing, etc. Successfully ran development organizations Developed open source software used by millions • Java Champion 2018 Early adopter and advocate of microservices • Worked on Vert.x, QBit, Reakt, Groovy, Boon, etc. • Speaker on microservices at JavaOne • Designed/implemented microservices-based systems that scale to 100M users Wrote App Gateway for streaming music service Worked with Service Meshes as early as 2015 Worked with Container Orchestration as early as 2016 Senior Director at fortune 100, managing group using Kubernetes and implementing stream processing RICK HIGHTOWER Sergey Sundukovskiy, Ph.D. has over 20 years of experience serving in capacities of Chief Technology Officer, Chief Information Officer and Chief Product Officer. Sergey specializes in implementation of subscription based high volume SaaS platforms, with strong emphasis on early stage product development and market deployment. Specific areas of expertise include A/B Testing, Big Data, Video Management, eCommerce, RTB platforms and Cloud Computing. Sergey often mentors first-time founders and advises early stage Startups with emphasis on Product Development, Product Market Testing, Public Relations, Product Marketing, Team Building, Customer Success and Organizational Management
- 3. Lorem Ipsum Dolor Service Mesh Intro
- 4. Lorem Ipsum Dolor Microservices Without Service Mesh Difficulty Is Not In Breaking Down the Monolith Easy Problems Service Granularity Service Boundaries Service Communication Service Contract Service Roles and Responsibilities
- 5. Distributed System Problems ❖ Unreliable Networks - Nothing Works As Expected ❖ Lack of High Availability - Everything Eventually Fails ❖ Communication Latency - Everything Slows Down ❖ Limited Bandwidth - It Is Never Enough ❖ Zero Trust Environment - It Is Never Safe ❖ Changing Service Topology - Everybody Gets Lost
- 6. Microservice Components - Service Config The interesting part is that each of these microservices can have their own configuration Such configurations include details like: ❖ Application configuration. ❖ Database configuration. ❖ Communication Channel Configuration - queues and other infrastructure. ❖ URLs of other microservices to talk to. Ex. Git, Vault, File System
- 7. Microservice Components - Service Discovery Service discovery involves 3 parties: service provider, service consumer and service registry. ❖ service provider registers itself with service registry when it enters and deregister itself when it leaves the system ❖ service consumer gets the location of a provider from registry, and then talks to the provider ❖ service registry maintains the latest location of providers Ex. Zooker, Consul, Etcd
- 8. Microservice Components - Service Routing Service Routing primary responsibilities for API routing, composition and edge functions ❖ authentication – verifying the identity of the client making the request ❖ authorization – verifying that the client is authorized to perform that particular operation ❖ rate limiting – limiting how many requests per second are allowed from either a specific client and/or from all clients ❖ caching – cache responses to reduce the number of requests made to the services ❖ metrics collection – collect metrics on API usage for billing analytics purposes Ex. Zuul, NGINX, Spring Cloud Gateway
- 9. Microservice Observability Observability is not monitoring ❖ Health Checking ❖ Metrics ❖ Audit Logging ❖ Distributed Tracing ❖ Exception Logging ❖ Service Logging Ex. Prometheus, Grafana, Jaeger
- 10. Microservice Patterns ❖ Circuit Breaker ❖ Rate Limiter ❖ Retry ❖ Bulkhead
- 11. Microservice Patterns - Circuit Breaker The circuit breaker concept is straightforward. It wraps a function with a monitor that tracks failures. The circuit breaker has 3 distinct states, Closed, Open, and Half-Open: ❖ Closed – When everything is normal, the circuit breaker remains in the closed state and all calls pass through to the services. ❖ Open – The circuit breaker returns an error for calls without executing the function. ❖ Half-Open – After a timeout period, the circuit switches to a half-open state to test if the underlying problem still exists.
- 12. Microservice Patterns - Rate Limiter Rate Limiting pattern ensures that a service accepts only a defined maximum number of requests during a window. This ensures that underline resources are used as per their limits and don't exhaust.
- 13. Microservice Patterns - Retry Retry pattern enables an application to handle transient failures while calling to external services. It ensures retrying operations on external resources a set number of times. If it doesn't succeed after all the retry attempts, it should fail and response should be handled gracefully by the application.
- 14. Microservice Patterns - Bulkhead Bulkhead ensures the failure in one part of the system doesn't cause the whole system down. It controls the number of concurrent calls a component can take. This way, the number of resources waiting for the response from that component is limited. There are two types of bulkhead implementation: ❖ The semaphore isolation approach limits the number of concurrent requests to the service. It rejects requests immediately once the limit is hit. ❖ The thread pool isolation approach uses a thread pool to separate the service from the caller and contain it to a subset of system resources.
- 15. CTO Forum Microservice Journey Webify SOA Microservices CI/CD / Agile DevOps / SRE Containers Container Orchestration
- 16. How we got here ❖ Web pages that were brochures ❖ eCommerce ❖ Legacy integration ❖ Rush to ‘webify’ businesses ❖ SOA: wrap legacy systems as services to use from the web ❖ Virtualization, Virtualization 2.0, Cloud, Containers, and now Container orchestration ❖ We want faster feedback and leaner more agile delivery
- 17. Continuous delivery ❖ The ability to deliver ❖ Build quality in ❖ Work in small batches ❖ Automate repetitive tasks including ❖ testing & deployments ❖ Pursue continuous improvement ❖ Ownership ❖ Comprehensive configuration management ❖ Continuous integration ❖ Continuous testing You can’t skip steps. There is investment up front. Today’s speed up can be tomorrows painted yourself In a corner.
- 18. Why DevOps, CI/CD and Microservices? ❖ High performers 2x the rate will exceed organizational performance goals as low performers: ❖ 2x profitability ❖ 2x productivity ❖ 2x market share ❖ 2x number of customers ❖ High performers twice as likely to exceed non-commercial performance goals as low performers ❖ 2x better quantity of products and services ❖ 2x operating efficiency ❖ 2x customer satisfaction ❖ 2x quality of products/services ❖ 2x achieving organizational/mission goals ❖ 50% increase in market capitalization compared to low performers! 18
- 19. DevOps Acceleration ❖ Microservices/ Containers ❖ CI/CD ❖ DevOps ❖ 12 Factory Deploys ❖ Observability
- 20. Convergence DevOps Automation is better CI/CD Fast Feedback is better Lean/Agile Simpler is better Microservices Small is better 12 Factor Deploys KPIs and Health Service Mesh • Observability • Logging • Tracing • KPIs • Dashboards • Canary Deployments • Fractional • Version Labels • Supports small CI/CD with Microservice • Traffic Management
- 21. Microservices: INCEPTION and Natural Evolution ❖ Now you can run a Java Virtual Machine in a Docker image ❖ Which is just a process pretending to be an OS ❖ Which is running in an OS that is running in the cloud ❖ Which is running inside of a virtual machine ❖ Which is running in Linux server that you don’t own that you share with people whom you don’t know ❖ Servers are not giant refrigerator boxes that you order from Sun and wait three months for (circa 2000)..… Goal was to run a lot of things on same server ❖ Did you develop code in the 90s with punch cards? ❖ Microservices recognize trend 21
- 22. [ 22 ‣ Philosophy behind microservices mirrors Unix ‣ Unix’s inventor, Ken Thompson, defined its philosophy: • One tool, one job. ‣ Emphasizes building short, simple, clear, modular, and extendable code • Easily maintained and repurposed by other developers MICROSERVICES: UNIX PHILOSOPHY What is microservice arc
- 23. Microservices ❖ Focus is building small, reusable, scalable services ❖ Adopt the Unix single-purpose utility approach to service development ❖ Small and malleable so they can be released more often ❖ Easier to write ❖ Easier to change ❖ Go hand in hand with continuous integration and continuous delivery ❖ Heavily REST-based and message oriented ❖ Focus on business capability ❖ Refocus on object oriented programming roots ❖ Organize code around business domains. ❖ Data and business rules colocated in the same process or set of processes. What is microservice architecture?
- 24. Microservices: Key ingredients ❖ Independently deployable, small, domain-driven services ❖ Own their data (no shared databases) ❖ Communication through a well-defined wire protocol usually JSON over HTTP (curl-able interfaces) ❖ Well defined interfaces and minimal functionality ❖ Avoiding cascading failures and synchronous calls - reactive design for failure ❖ Shortly after MicroServices: Containers came out
- 26. MicroServices: Achieving Resilience ❖ Avoid synchronous calls to avoid cascading failures ❖ Circuit breaker frameworks, retries, resiliency, network layer libs ❖ Instead embrace: ❖ Streams, queues, ❖ Actor systems ❖ Event loops ❖ Other async calls. ❖ Spend more time with distributed logging/log aggregation w/MDC ❖ Distributed tracing: A calls B who calls D or E or F who calls X or Y or Z 26
- 27. MicroServices: Monitoring and KPIs ❖ Customer/User experience KPIs ❖ Debugging (requests per second, # threads, # connections, failed auth, expired tokens, etc.) ❖ Circuit breaker (monitor health, restarts, act/react based on KPIs) ❖ Cloud orchestration (monitor load, spin up instances) ❖ Health checks and observable KPIs 27
- 28. MicroServices: Continuous Deployment ❖ Microservices are continuously deployable services ❖ Focus of microservices is on breaking applications into small (micro), reusable services that might be useful to other services or other applications. ❖ ‘micro’ part of microservices comes to denote small ❖ Services can be deployed independently. ❖ Can be tweaked and then redeployed independently. ❖ Microservice vs monolith when deploying What is microservice a
- 29. [ 29 KUBERNETES
- 30. –Rick Hightower “Service Mesh like Istio does the things that the very best InfoSec, Dev teams, SREs and DevOps teams would do: mTLS zero trust networking, automate observability and dashboard creation, automate tracing, and automate logging aggregation while enabling continuous deployment via traffic management and canary deployments. It takes what we’ve learned in the DevSecOps community and makes it the default, out of the box.”
- 31. –Rick Hightower (Why you might need a Service Mesh like Istio?) “To maximize shareholder value, companies are embracing CI/CD and Microservices architecture. This allows product teams to deliver faster, get feedback more often and evolve quickly. This Digital Transformation strategy allows companies to address nimble upstarts as well as provide our customers with an intelligent, rich experience.”
- 32. CTO Forum What is Service Mesh? Observability and Telemetry Service discovery Traffic management Security Supports CI/CD and Microservices
- 34. What is a Service Mesh? ❖ Service mesh is a network of microservices and interactions between microservices ❖ Service mesh tools scale to help manage size and complexity of large Service Meshes ❖ Modern service mesh aids understanding and managing ❖ Helps organizations migrate from monolithic applications to microservice architecture
- 35. –Rick Hightower (Why you might need a Service Mesh like Istio?) “Using a Service Mesh facilitates CI/CD and Microservices architecture. Service Mesh automates best practices for DevSecOps needs like failover, scale-out, scalability, 0 trust networking, health checks, circuit breakers, rate limiters, KPI collection, dashboard creation, observability, avoiding cascading failure, disaster recovery, and traffic routing”
- 36. Decorate Network Data Layer ❖ Service Mesh decorates network layer to implement cross-cutting concerns which are usually NFRs ❖ Service Mesh is to MicroServices as AOP is to DDD and OOP ❖ Service Mesh is to MicroServices as Servlet Filters are to Servlets.
- 37. Service Mesh Features ❖ Networking: Discovery, load balancing, failure recovery (circuit breaking), rate limiting, etc. ❖ Observability: time series KPIs, log aggregation, alerting and monitoring, USE and RED Dashboards ❖ CI/CD and frequent releases: canary rollouts, green/blue deploys, new version rollouts, traffic management ❖ And to gradually release a Microservice and select which downstream and upstream Microservice that can talk ❖ Security access control, end-to-end authentication (RBAC), service identity, 0 trust networking - mTLS, etc.
- 38. Simplifies hard programming ❖ Service Mesh performs many low-level L3/L4 networking tasks ❖ Previously left up to application developers to implement or to many libs for many platforms/languages ❖ Low level network code is hard to write and maintain ❖ filled with edge cases. ❖ Service Mesh completely abstracted out from the microservices business logic ❖ Provides level of consistency provides additional operational predictability for polyglot programming environments
- 40. Top 3
- 41. Service Meshes At a glance ❖ Istio ❖ Backed by IBM, Red Hat, Google, and Lyft ❖ Uses Envoy ❖ Supports more than Kubernetes ❖ Linkerd ❖ CNCF ❖ V1: Finagle, Scala, Twitter stack ❖ V2: Conduit merged: Now Rust and Go Lang based ❖ Consul ❖ Hashicorp ❖ Uses Envoy ❖ Supports more than Kubernetes ❖ Nice comparison of Consul, Linkerd and Istio
- 42. Observability and Telemetry ❖ automate many aspects of observability ❖ log aggregation, telemetry of services, collecting KPIs and generating ❖ Automates creating USE and RED Dashboards ❖ See service performance trends and dashboards ❖ how long did a service request take? ❖ how often is the service being called?
- 43. Service Discovery ❖ Service inventory and understand how services communicate—tracing call graph, amount of calls per span, etc. ❖ essential for microservices architecture ❖ Allows services to find other dependent services ❖ Helps keep track of services running in infra ❖ essential for microservices architecture ❖ Manage and visualize services and its dependencies ❖ essential for microservices architecture
- 44. Traffic Management ❖ Segment features through feature flags and limit consumption of new services with clients that can handle changes to APIs or wire protocols with gradual rollouts ❖ Gradual and continuous release instead of a big bang rollout ❖ Fine grain deployments ❖ Essential for microservices architecture and CI/CD
- 45. Traffic Mgmt Interoperability ❖ Big Kubernetes issue with cloud interoperability has been ingress and egress ❖ Service Mesh makes great strides to solve interoperability ❖ Standardize ingress/egress and many other networking concerns so routing rules, RBAC and TLS termination don’t vary with each vendor or cloud provider ❖ Interoperability suffers w/ Kubernetes federation and hybrid clouds ❖ Service Mesh, and Git Ops (Flux, Argo CD, Anthos Config Manager) ❖ Keep copy of Kubernetes objects between clusters ❖ Using Service Meshes to span clouds and clusters ❖ Now possible to create service meshes that span clusters and clouds ❖ standard service registry plugins (consul/kubernetes), Istio gateways, ad hoc services and networks defined with CIDR addresses.
- 46. –Rick Hightower (Why you might need a Service Mesh like Istio?) “Service Mesh aids in avoiding data breaches as well as limiting their blast radius. Data breaches can have dire business value consequences.”
- 47. Security ❖ Identity, Security, RBAC, 0 trust networking ❖ Secure service-to-service communications via 0 trust networking ❖ Key is service identity ❖ Service identity enables automatic mTLS (mutual TLS) for service-to-service communications ❖ Microservices enhanced to automatically communicate securely via mTLS without code change ❖ Plugin an existing CA certificate ❖ Enforce service-level authentication using either TLS SNI or JSON Web Tokens (JWS) or headers or networking origination ❖ Enables fine-grained traffic governance ❖ Allows configure role-based access control (RBAC) for each service and limit which other services have access to key services ❖ Can be configured to block access based on headers or specific URLs or sub-URIs and paths
- 48. –Rick Hightower (Why you might need a Service Mesh like Istio?) “(A Service Mesh’s) ability to automate and maintaining zero trust networks is its most important feature. In the age of high-profile data breaches, security is paramount. …avoid major brand issues … (that can) shrink market capitalization in an instant. (Service Mesh) helps prevent a breach and limits the blast radius …”
- 49. Traffic Management Features ❖ Rate limits based on identity or headers or policies ❖ Fail-over rules (via circuit breakers) ❖ Fine-grained traffic management policies and the application code never changes ❖ Extend policies to connected service meshes ❖ Route rules can be based on locality of the service ❖ prefer local data center, ❖ or local proximity networks over remotes. ❖ Failover rules are location-aware ❖ Routing can take into account the health of services (active and passive)
- 52. Book info App with No Service Mesh
- 53. Book info App with Service Mesh