Scala.io

Finagle: Twitter’s RPC Library
24 Oct. 2014
Steve Gury
@stevegury

> whoami
stevegury
2005 2008 2011 2014 time
Distributed Systems
Massively Multiplayer Online Games
Twitter
Scala

Agenda
1. Motivation
2.Finagle TL;DR
3.Three Key Abstractions
4.Finagle Big Picture

Motivation & Context
2006 2010 2014
>2M Tweets / day
(January 2009)
>65M Tweets / day
(July 2010)
>500M Tweets / day
(August 2013)

Finagle TL;DR
Compose RPC like you
compose functions

Composing Functions
getUserId(name: String): Int getTweets(userId: Int): Tweets
getTweets(name: String): Tweets
= getUserId ◦ getTweets (name)

Futures
Futures are containers for value
Pending Successful Failed
Present

Futures
a
val a: Future[Int]

Futures
a
b
val a: Future[Int]
val b: Future[Int] = a map { x => x + 512 }

Futures
a
b
c
val a: Future[Int]
val c = a map { x => 64 / x }

Futures
a
b
c
d
val a: Future[Int]
val c = a map { x => 64 / x }
val d = Future.join(b,c)

Futures
a
b
c
d e
val a: Future[Int]
val c = a map { x => 64 / x }
val e = d map { case (x, y) => x + y }

Futures
16 b
a
c
d e
val a: Future[Int]
val c = a map { x => 64 / x }

Futures
16
a
528
b
c
d e
val a: Future[Int]
val c = a map { x => 64 / x }

Futures
16
a
528
b
4
c
d e
val a: Future[Int]
val c = a map { x => 64 / x }

Futures
16
a
528
b
4
(528,
4)
c
d e
val a: Future[Int]
val c = a map { x => 64 / x }

Futures
16
a
528
b 532
4
(528,
4)
c
d e
val a: Future[Int]
val c = a map { x => 64 / x }

Futures
0 b
a
c
d e
val a: Future[Int]
val c = a map { x => 64 / x }

Futures
0
a
512
b
c
d e
val a: Future[Int]
val c = a map { x => 64 / x }

Futures
0
a
512
b
Ex
c
d e
val a: Future[Int]
val c = a map { x => 64 / x }

Futures
0
a
512
b Ex
Ex
c
d e
val a: Future[Int]
val c = a map { x => 64 / x }

Futures
0
a
512
b Ex Ex
Ex
c
d e
val a: Future[Int]
val c = a map { x => 64 / x }

Futures
Lots of combinators:
!
map -> add an edge+node in the graph
flatMap -> combine two graphs together
handle -> deal with exceptions
join -> merge multiple nodes of the graphs
select -> select the first node with a value
!
…more

Programming with Futures == Building a Graph

Service
Remember the key idea: treat RPC like functions

Service
Service is a Function from Request to Response
!
trait Service[Request, Response] {!
def apply(req: Request): Response!
}!
!
Equivalent to Scala’s Function1: Request=>Response

Service
But it has to be asynchronous, and treat exception as
value
!
def apply(req: Request): Future[Response]!
}!
!

Service
It also has to be closable for proper management of
resources.
!
def apply(req: Request): Future[Response]!
def close(): Future[Unit]!
}!

ServiceFactory
Factory of Service
(Function that returns a Function)

ServiceFactory
Function that asynchronously returns a Service
!
trait ServiceFactory[Req, Rep] {!
def apply(): Future[Service[Req, Rep]]!
def close(): Future[Unit]!
}

Transport
trait Transport[In, Out] {
def write(req: In): Future[Unit]
def read(): Future[Out]
}
Transport

Dispatcher
class Dispatcher[A, B] extends Service[…] {
def apply(req: A): Future[B]
}
Dispatcher Transport

Connection Pool
class ConnPool[A,B] extends ServiceFactory[…]{
def apply(): Future[Service[A, B]]
}
Connection Dispatcher Transport
Pool

Load Balancer
class LoadBalancer[A,B] extends ServiceFactory[…]{
def apply(): Future[Service[A, B]]
}
Pool
Load
Balancer

Finagle in Action
Pool
Load
Balancer
val req: Request
loadbalancer() // LB select an host
// Pool select a connection

Finagle in Action
val req: Request
loadbalancer() flatMap {
service => // LBService(PoolService(disp))
}
Pool
Load
Balancer

Finagle in Action
val req: Request
service(req) // disp write req on Transp.
}
Pool
Load
Balancer

Finagle in Action
val req: Request
service(req) ensure {
service.close() // close LBService
// close PoolService
}
}
Pool
Load
Balancer

FactoryToService
class FactoryToService[A, B](factory: ServiceFactory[A, B])
extends Service[A, B]
{
def apply(request: A): Future[B] =
factory() flatMap { service =>
service(request) ensure {
service.close()
}
}
}
Pool
Load
Balancer
FactoryTo
Service

Pool
Load
Balancer
Finagle is “just” a composition of independent
functions on top of a Transport
FactoryTo
Service

The “Simplified” Full Picture
Stats
Timeout
Draining
Load Balancer
Monitor
Stats
Failure accrual
Timeout
Conn. Pool
Expiration Dispatcher
ServiceFactory
Service

The Stack is configured for you
You rarely have to configured the stack, we have a series
of predefined stack for you.
!
scala> val client: ServiceFactory[Req, Res] = Http.client!
.newClient(“localhost:8080")!
!

!
scala> val client: Service[Req, Res] = Http.client!
.toService!
scala> client(req)!

!
scala> val client: Service[Req, Res] = Http.client!
.configured(DefaultPool.Param(5, 10, 0, 1.minute, 100))!
.toService!
scala> client(req)!
!
You can configure any layer of the Stack

Finagle is doing more…
• Load balancing
• Connection pooling and request buffering
• Dynamic membership (Zookeeper)
• Failure detection and mitigation (fail-fast & failure
accrual)
• Statistics for visibility
• Distributed tracing (Zipkin)
• Cancellation propagation
• Automatic retrying
• Graceful shutdown and request draining
• GC avoidance - traffic shaping
• Backup requests

A Better Load Balancer
Key idea: Use history of recorded latencies to estimate
the “cost” of sending a request to a server

A better Load Balancer
First idea: calculate an average of the latencies

Better idea: calculate EWMA average of the latencies
EWMA(n+1) = EWMA(n) * ⍺ + (1 - ⍺) * Xn+1

The devil is in the details
!
Recency of data: EWMA unevenly spaced t-Series
EWMA(n+1) = EWMA(n) * e-td/Tau + (1 - e-td/Tau) * Xn+1
No history: Initial cost=0 + Probation

!
Speed of convergence: Cost “rides the peaks”
Outliers: Decaying cost

!
CPU intensive: “The Power of 2 Choices”

“The Power of 2 Choices in Randomized Load Balancing” P.h.D
Thesis of M. Mitzenmacher

Load balancing
0 20
0
10
0
cost: 20ms
cost: 10ms
cost: 10ms

cost: 18ms
Load balancing
0 20
0
10
0
20ms
cost: 10ms
cost: 10ms

Finagle’s Architecture
Experience from writing “large scale” software
!
Independent modules
Simple contracts
Powerful combinators

Conclusion
• Finagle is a library that lets you treat RPC as functions
• Finagle is itself composed of independent functions
• Very widely used at Twitter, Foursquare, Tumblr,
Pinterest, SoundCloud…
• Protocol agnostic (thrift, http, redis, mysql…)
Github: github.com/twitter/finagle
“Your server as a Function” http://monkey.org/~marius/
funsrv.pdf

Scala.io

More Related Content

What's hot (20)

Viewers also liked (7)

Similar to Scala.io (20)

Recently uploaded (20)

Scala.io