High-Performance event-sourced clustered Microservices with Play! & Akka @ Scala-Vienna May 2018

Event-Sourced µServices with Play!
and Akka
by Dominik Dorn @ Scala Vienna 2018-05

Dominik Dorn
4 Freelance Software Engineer
4 Leader of
4 Java-Vienna Meetup
4 PlayFramework.Wien
Meetup
4 PlayFramework Integrator
#Java, #Scala, #Akka, #Spark
#Play, #Cassandra, #Kafka,#Postgres
2

Overview
4 Intro to Event Sourcing
4 (Short!) Intro to Actors + Actors in Akka
4 Event-Sourcing with Akka-Persistence
4 Intro to CQRS
4 Clustering with Akka
4 Code-Base/Dev-Env structuring + tips
#Scala #Vienna 2018 May-Meetup | 4th June 2018 | Event Sourced µServices with Akka and Play! | Dominik Dorn @domdorn 3

Intro to Event-Sourcing
4 Reasons + Example
4 Overview
4 Commands + Events
4 The Journal
4 Command-Handler + Event-Handler

Reasons for Event Sourcing
4 High Performance
4 Immutable-Events
4 Append-Only Journal
4 Increased write-performance
4 No Object-Relation-Mapping needed
4 Full-Audit trail (e.g. GDPR) - debugging

Examples of Event-Sourcing
4 Your Bank-/Paypal-Account
4 Git
4 Order Tracking (Post, DPD, UPS, etc.)
4 kind of: Blockchain .. BitCoin anyone?
4 PostgreSQL/Oracle/etc.: Transaction Log!
4 but without meaning!

High-Performance event-sourced clustered Microservices with Play! & Akka @ Scala-Vienna May 2018

Commands & Events
4 Commands - Something the users wants us to do
case class RegisterUser(email: String, name: String)
case class ChangeName(email: String, newName: String)
4 Events - Something that happened in the past
case class UserRegistered(email: String, name: String)
case class NameChanged(newName: String)

The (Event-)Journal
4 Single Source of Truth
4 Stores the Events
4 Groups Events by Entity-ID
4 Sequence-Number for every Entity-ID

The (Event-)Journal
looks like this
Entity-ID Seq# Type Payload (e.g. in JSON)
user-1 1 UserRegistered {email:"dominik.dorn@gmail.com",
name:"Dominikk Dorn"}
user-1 2 NameChanged {newName:"Dominik Dorn"}
user-2 3 UserRegistered {email:"dominik@dominikdorn.com",
name:"Dominik Dorn"}

Command-Handler
4 Enforces constraints
var registeredEmails: Vector[String] = Vector.empty
def receiveCommand = {
case RegisterUser(email, name) =>
if(registeredEmails.exists(_ == email)) {
throw new UserAlreadyExistsException();
} else {
// registering..
}
}

Command-Handler
4 Persists to the Journal, before executing
if(!registeredEmails.exists(_ == email)) {
persistToJournal(UserRegistered(email, name)){ ev =>
sendWelcomeMail(email, name)
}
}

Command-Handler
4 (Can) Delegate to the event-handler(s)
...
persistToJournal(UserRegistered(email, name)){ ev =>
sendWelcomeMail(email, name)
eventHandlers.foreach(handler => handler.receiveEvent(ev))
}

Event-Handlers
4 Work on Events from the past
var registeredEmails: Vector[String] = Vector.empty
def receiveEvent = {
case UserRegistered(email, name) =>
registeredEmails = registeredEmails :+ email
}

(Short!) Intro to Actors (in Akka)
Actors Humans
have state (fields, lists, etc.) have properties (height, weight,
knowledge)
communicate via messages communicate via voice, sms,
letters, looks, touch, ...
live in an ActorSystem live on Planet Earth

Creation
Actors Humans
system = new ActorSystem() god.create(Planet("Earth"))
system.actorOf(Props(new
Female("Eve")))
god.create(Female("Eve"))
context.childOf(Props(new
Male())
eve.fork() // +adam

Getting information
about the current state of the soccer game
Actors Humans
gameActor.tell(GetCurrentScore) shout("Hey Steve, what's the
score?")
sender().tell(Score(3,2)) shout("Its 3 to 2 for Manchester!")

Actors in Akka
class Human extends akka.actor.Actor {
var name = ""
override def receive = {
case GiveName(name: String) =>
name = name;
sender() ! Hi(s"I'm $name")
}
}
val adam: ActorRef = actorSystem.actorOf(Props(new Human()))
adam ! GiveName("Adam")

Persistent-Actor Managers
class UserManager(@Named("email") email: ActorRef)
extends PersistentActor {
val persistenceId: String = s"userManager"
def getOrCreateChild(id:String): ActorRef = context.child(id).
getOrElse(context.actorOf(User.props(id), id, email))
def receiveCommand = { ... } // handle commands
def receiveRecover = { ... } // can be empty
}

Persistent-Actors
class User(id: String, email: ActorRef)
extends PersistentActor {
// entity-id in the journal
val persistenceId: String = s"user-$id"
def receiveCommand = { ... } // handle commands
def receiveRecover = { ... } // handle events
}

Command-Handler
// email constraints already checked before
persist(UserRegistered(email, name)) { ev =>
// ^--- persist stores event into the journal
receiveRecover.apply(ev)
// ^--- apply the event to the event handler
email ! Email.Commands.SendWelcomeMail(email, name)
// tell that we're done
sender() ! UserRegisteredAnswer(email, name)
}
}

Command-Handler
...
case ChangeName(email, name) =>
if(name != this.name) {
persist(NameChanged(email, name)) { ev =>
}
}
sender() ! akka.Done // the new name is set
}

Event-Handler
var email = ""
var name = ""
def receiveRecover = {
this.email = email
this.name = name
case NameChanged(email, newName) =>
this.name = newName
}

Many events? Take Snapshots!
var lastSequenceNr = 0 // gets updated in event handler
def maybeMakeSnapshot() = if(lastSequenceNr % 1000 == 0) {
saveSnapshot((email, name))
}
def receiveCommand = { ....
persist(NameChanged(email, name)) { ev =>
maybeMakeSnapshot()
}
}

And Recover from it!
def receiveRecover = {
case UserRegistered(email, name) => ...
case NameChanged(email, newName) => ...
case SnapshotOffer(metaData, snapshot: (String, String)) =>
this.email = snapshot._1
this.name = snapshot._2
}

Keep memory low, set a ReceiveTimeout
class SomeActor extends Actor {
context.setReceiveTimeout(3 seconds)
def receive = {
// ....
case ReceiveTimeout => context.stop(self)
}
}

Best Practices: Protocols
object User {
object Events {
sealed trait Event
...
}
object Commands {
sealed trait Command
...
}
object Queries {
sealed trait Query
...
}

Best Practices: Protocols - Events
object User {
object Events {
sealed trait Event
case class UserCreatedEvent(email: String, name: String) extends Event
case class NameChangedEvent(newName: String) extends Event
case class FriendAddedEvent(friendEmail: String) extends Event
}
...

gives you: IDE support!

gives you: Compile-time safety!

Protocols: Commands + Responses
object User { ...
object Commands {
sealed trait Command
case class CreateUserCommand(email: String, name: String) extends Command
sealed trait CreateUserResponse
case class UserCreatedResponse(email: String) extends CreateUserResponse
case object UserCreationFailedResponse extends CreateUserResponse
}

Protocols: Queries + Responses
object User { ...
object Queries {
sealed trait Query
sealed trait GetUserDetailsResponse
case class GetUserDetails(email: String) extends Query
case class UserDetails(email: String, name: String)
extends GetUserDetailsResponse
sealed trait GetFriendsResponse
case class GetFriends(email: String) extends Query
case class Friends(friends: Seq[String]) extends GetFriendsResponse
case object UserNotFound extends GetUserDetailsResponse with GetFriendsResponse

Command Query Responsibility
Segregation (CQRS)
4 Split Actions (Commands) and Questions (Queries)
4 Divide the code-base according to these separations
4 Create optimized datastores (read-sides) for the
various queries
4 If necessary, create+scale read- + write-µServices

CQRS-Example: Get the names of all
Users
different approaches
4 The (naive) Actor way: Ask everyone!
4 Read the Event-Log
4 Create custom read-sides

Ask! Everyone
class GetAllNamesActor( ids: Vector[String], aggregate: ActorRef,
answerTo: ActorRef) extends Actor {
var repliesWaiting: Vector[String] = ids
var data: Map[String, String] = Map.empty
ids.foreach(id => aggregate ! GetUserData(id))
def receive = {
case UserData(id, name) =>
repliesWaiting = repliesWaiting.filterNot(_ == id)
data = data.updated(id, name)
if(repliesWaiting.isEmpty){
answerTo ! GetAllNamesResponse(data.values)
}
}
}

Using Akka-Persistence-Query
class GetAllNamesActor(...,answerTo: ActorRef,...) {
val readJournal = PersistenceQuery(context.system)
.readJournalFor[CassandraReadJournal](CassandraReadJournal.Identifier)
val importFuture: Future[Done] = startInitialImport(self).map(_ => Done)
pipe(importFuture).to(self)
}

Receive Events via Akka-Streams
val ALL_CURRENT_IDS = readJournal.currentPersistenceIds()
val ALL_IDS_IN_FUTURE = readJournal.persistenceIds()
val FILTER_USERS = Flow[String].filter(s => s.startsWith("user-"))
def startInitialImport(recipient: ActorRef) =
ALL_CURRENT_IDS.via(FILTER_USERS) // get all user persistence-ids
.flatMapConcat(id => readJournal
/* ^ */ .currentEventsByPersistenceId(id, 0, Integer.MAX_VALUE))
// |--- join with all the events of these users
.runForeach(ev => recipient ! ev)
// ^ send all the events to recipient / ourself

Process the events in the actor
class InMemoryGetAllNamesActor(...,answerTo: ActorRef,...) {
...
def receive = {
case Done => answerTo ! GetAllNamesResponse(data.values)
case EventEnvelope(_, _, _, ev) => ev match {
case UserRegistered(email, name) => ... // fill state
case NameChanged(email, newName) => ... // fill state
}
}
}

Create a SQL read-side
class UserSQLReadSideEventHandler() extends Actor {
// ... like in GetAllNamesActor with Akka-Streams
def receive = {
case EventEnvelope(_, _, _, ev) => ev match {
SQL(s"INSERT INTO users VALUES (...)")
case NameChanged(email, newName) =>
SQL(s"UPDATE users SET name = ${newName} where email = ${email}")
}
}
}

Akka-Streams Performance/low latency
Optimizations
4 Tag your Events using a WriteEventAdapter, then
read with readJournal.eventsByTag()
4 Publish Events in the CommandHandler to the Event-
Stream, subscribe in the EventHandler(s)

Clustering with Akka
4 Use the akka-cluster module
4 Use cluster sharding to place aggregates on nodes
4 Don't distribute the contents of aggregates!
4 Use cluster singletons for Read-Side Event-Handlers
4 #Shardregions: ~ 10 x #cluster-nodes

Creating a Shard-Region
val userRegion: ActorRef = ClusterSharding(system).start (
typeName = "User",
entityProps = Props[User],
settings = ClusterShardingSettings(system),
// method how to extract the entity id of a message
extractEntityId = extractEntityId,
// how to derive the correct shard of a entity
extractShardId = extractShardId
)

PersistenceId + Forwarding 1
class UserManager extends PersistentActor {
val id = extractIdFrom(self.path.name)
// extract ID from path generated by ClusterSharding
override def persistenceId = s"user-$id"
val userRegion = ClusterSharding(system).start (...)
def receive = {
// forward messages to the correct actors
case x : ChangeUserName => userRegion.forward(x)
// ...
}
1
More Details in "Mastering Akka, 1st Edition, 2016, Pakt Publishing" (Ebook currently 10 USD!)

Efficient µServices-
Code-Base
structuring
4 project (+docker image) per
bounded-context
4 e.g. authentication, user-
management, billing
4 Root-Project with
VirtualHostRequestHandler

Virtual-Host Request-Handler2
class RequestHandler @Inject()(.... authRouter: auth.Routes,
billingRouter: billing.Routes, petClinicRouter: petclinic.Routes
) extends DefaultHttpRequestHandler(Router.empty, ...) {
override def routeRequest(request: RequestHeader): Option[Handler] = {
request.host.split('.').headOption match {
case Some("auth") => authRouter.routes.lift(request)
case Some("billing") => billingRouter.routes.lift(request)
case Some("petclinic") => petClinicRouter.routes.lift(request)
case _ => super.routeRequest(request)
}
2
see https://www.playframework.com/documentation/2.6.x/ScalaHttpRequestHandlers

Code Structure
4 package per Entity
4 package per use-case
4 Input- and Output-Datatypes
4 Tailored for every use-case!
50

THANK YOU!
Blog https://dominikdorn.com
Twitter @domdorn
Xing https://www.xing.com/profile/Dominik_Dorn
LinkedIn https://www.linkedin.com/in/dominik-dorn
Java-Vienna Meetup | PlayFramework.Wien Meetup

High-Performance event-sourced clustered Microservices with Play! & Akka @ Scala-Vienna May 2018

More Related Content

Similar to High-Performance event-sourced clustered Microservices with Play! & Akka @ Scala-Vienna May 2018 (20)

Recently uploaded (20)

High-Performance event-sourced clustered Microservices with Play! & Akka @ Scala-Vienna May 2018