The Current And Future State Of Service Mesh

The Current & Future State of
Service Mesh

2 | Copyright © 2022
Idit Levine
Founding API gateway WG-Istio
Christian Posta
Founding community member,
Istio Steering Committee,
author Istio in Action
Lin Sun
Founding Istio project maintainer,
Technical Oversight Committee
(TOC), Steering Committee
Neeraj Poddar
Istio Steering and TOC member.
Co-founded Istio Product
Security Working Group.
Yuval Kohavi
Renowned security researcher,
Founding API Gateway WG-Istio,
Contributor Envoy
Ram Vennam
Founding Istio Steering
Committee member
Nick Nellis
First to run Istio in production,
current contributor and maintainer
Solo Istio/Envoy Community Leadership
Founded in 2017 by Idit Levine
Based in Cambridge, MA
with multiple locations around the globe
Industry leaders in application networking, service
mesh, and modern API gateway technologies
Open-Core, “Enterprise” Subscription model
Growing fast
with happy customers
Well Funded
350+%
bookings
growth y/y
98%+
renewal
rate
$171.5M
venture financing
$1 Billion
valuation
Solo.io
Gloo Application Networking Platform
Simplify your application networking with unified control,
reliability, observability, extensibility, and security
Greg Hanson
Founding Istio Maintainer,
Product Security WG Lead,
Istio Release Manager

Large deployments of enterprise service mesh

The Current State of Service Mesh…
…day 1 and day 2 operations

Service discovery / Load balancing
Secure service-to-service communication
Traffic control / shaping / shifting
Policy / Intention based access control
Traffic metric collection
Service resilience
API / programmable interface
Service Mesh Functions

Performance
● Envoy proxy uses 0.35 vCPU and 40 MB memory
per 1000 requests per second going through the
proxy.
● Istiod uses 1 vCPU and 1.5 GB of memory.
● The Envoy proxy adds 2.65 ms to the 90th
percentile latency.

Gateway API
https://gateway-api.sigs.k8s.io/

The Future of Service Mesh…
…happens in and around the data plane

Data plane is where the innovation continues to happen
● Web Assembly
● HTTP/3
● Universal Data Plane API
Extending the data plane
Optimizing the data plane

Extending the data plane with GraphQL

Clients
https://api.yelp.com/v3
/businesses/search

GET https://api.yelp.com/v3/businesses/search
{
"total": 8228,
"businesses": [
{
"rating": 4,
"price": "$",
"phone": "+14152520800",
"alias": "four-barrel-coffee-san-francisco",
"categories": [
{
"alias": "coffee",
"title": "Coffee & Tea"
}
],
// ...
],
"region": {
"center": {
"latitude": 37.767413217936834,
"longitude": -122.42820739746094
}
}
}

Clients Web API
/business/{id}/menu
/reviews/{id}
/events/{name}

GraphQL
Returns
exactly what
we need and
nothing more
Fetches data
across different
resources from
a single query

https://api.yelp.com/v3/graphql
query {
businesses(search: “solo”) {
name
}
}
{
“businesses”: [
{
“name”: “solo.io”
},
{
“name”: “Solo Solar”
}
]
}
Business Info API

query {
name
phone
}
}
{
“businesses”: [
{
“name”: “solo.io”,
“phone”: 6172213102
},
{
“name”: “Solo Solar”,
“phone”: 2223657814
},
]
}
Business Info API

query {
name
phone
location {
address
}
}
}
{
“businesses”: [
{
“phone”: 6172213102,
“location”:
{
“address”:“222 Third St”
},
},
{
“phone”: 2223657814,
“location”:
{
“address”:“3 Hershey Park”
},
},
]
}
Business Info API
Business Location API

query {
solo_query: businesses(search: “solo”) {
name
phone
location {
business_address: address
}
}
}
{
“solo_query”: [
{
“phone”: 6172213102,
“business_address”:
{
“address”:“222 Third St”
},
},
{
“phone”: 2223657814,
“business_address”:
{
“address”:“3 Hershey Park”
},
},
],
}

query {
solo: businesses(search: “solo”) {
name
phone
location {
state
}
}
software: businesses(search: “software”) {
name
}
}
{
“solo”: [
{
“phone”: 6172213102,
“location”:
{
“state”:“MA”
},
},
{
“phone”: 2223657814,
“location”:
{
“state”:“CA”
},
},
],
“software”: [{...}]
}

Query
query {
name
phone
location {
state
}
}
}
type Query {
businesses(search: String!): [Business]
}
type Business {
name: String
phone: String
categories: [String]
reviews: [Int]
location: Location
}
type Location {
state: String!
country: String!
street: String!
}
Schema Definition Language (SDL)

GraphQL Interest - Market Indicators
https://trends.google.com
https://2020.stateofjs.com/en-US/technologies/datalayer
https://smartbear.com/state-of-software-quality/api/tools
24%

GraphQL Adoption Patterns
REST

REST
gRPC
REST
gRPC

GraphQL Traffic Patterns

Considerations when using GraphQL APIs
Application Interface Application Network
Client-specific data views
Single entry point to entire graph
Overfetching data
Underfetching data (N+1)
Client compatibility with schema evolution
Authentication / Authorization
Rate Limiting
Observability
Web Application Firewall
Data Loss Prevention

GraphQL Support in Envoy
• Web Application Firewall (WAF)
• Data Loss Prevention (DLP)
• AWS Lambda
• Request and Response Transformation
• SOAP
• GraphQL
EXTERNAL AUTH
RATE LIMITING
ROUTER
UPSTREAM
CUSTOM
gRPC
TRANSCODER
Gloo Edge Custom Envoy Filter

Extending the service mesh data plane

Extending the data plane

Benefits of GraphQL in Envoy
Simplified deployment architecture
Optimized data plane performance
Declarative configuration for apps and infra
Leverage platform capabilities vs. DIY in every application
Architecture scales from simple monolith to multi cloud federation

Optimizing the service mesh data plane

What is eBPF?
• Linux technology which enables users to run custom programs “sandboxed” in the kernel
• extended Berkeley Packet Filter, evolution of “classic BPF” – think tcpdump
• Event-based – programs are attached to “hook points” that are triggered by certain events
− E.g. ‘kprobe’ type programs are attached to kernel functions and are then executed when that
function is called
• BPF programs are verified to be “safe” – won’t crash the kernel, guaranteed to return (no
infinite loops), can only access specific sections of memory, etc.

petstore
pod: "petstore"
istio-proxy
kernel
network
stack
network
stack
socket socket
Istio sidecar redirection

Networking in Istio
Source: https://jimmysong.io/en/blog/sidecar-injection-iptables-and-traffic-routing

Istio data-path acceleration with eBPF
• The Linux kernel has several hookpoints in network stack for BPF programs to be attached
• Attach BPF programs to network sockets
• Special BPF map (sockmap) that can redirect data from one socket directly to another

Istio data-path acceleration
petstore
pod: "petstore"
istio-proxy
kernel
network
stack
network
stack
socket socket
eBPF eBPF
Skip the network stack!

Istio data-path acceleration

Can eBPF implement service mesh functions?
Service Mesh Functions:
• L4 & L7 Observability
• Identity Management
• Encryption
• JWT based Authz/Authn
• External Authorization
• L7 Traffic Management
• Advanced Load Balancing
• Session Affinity
• ...
eBPF:
● eBPF execution paths must be fully known and verified
● eBPF programs cannot have arbitrary loops where the
verifier will not know when the program will stop
execution.
● eBPF is turing incomplete.
● eBPF is ideal for O(1) complexity
○ inspecting a packet
○ manipulating some bits
○ redirect
● Complex protocols like HTTP/2 and gRPC can be O(n)
complexity and very difficult to debug

Data plane as a spectrum
From “The Truth About the Service Mesh Data Plane”, November 2019
https://www.slideshare.net/ceposta/the-truth-about-the-service-mesh-data-plane

Data plane: sidecar (service proxy)

Data plane: shared proxy per node

Data plane: shared proxy per service account (per node)

Data plane: shared remote proxy with micro-sidecar

User Clusters Public cloud
Compliance / DMZ

Gloo Application Networking Platform

Solo Istio/Envoy Community Leadership
Idit
Levine
Christian
Posta
Lin
Sun
Neeraj
Poddar
Yuval
Kohavi
Ram
Vennam
Nick
Nellis
“The team Solo has assembled
is incredible. They have deep
expertise with Envoy and
Istio, and they’re extremely
receptive to community
feedback. The strength of their
team gives us confidence in the
innovation and support they’re
able to provide.”
CURRENT CUSTOMER
Thank you!!
Reach out to us for guidance!
CRAWL WALK
Access to Expertise
and Long-term Support (LTS)
Enhanced capabilities for
security, observability
& collaboration
RUN FLY
Extend & Scale Across
Teams and the World
Multi-Cluster, Multi-Mesh
on Cloud and Hybrid

The Current And Future State Of Service Mesh

More Related Content

Similar to The Current And Future State Of Service Mesh (20)

Recently uploaded (20)

The Current And Future State Of Service Mesh