SlideShare a Scribd company logo
The dark side of Akka 
and the remedy 
Ákos Kriváchy
Introduction - Ákos Kriváchy 
• Scala disciple/convert/fanatic since 2013 February 
• First FP language 
• Akka since 2014 February 
• Things I love about Scala: 
• Static typing 
• Partial Functions 
• foldLeft, tail recursion 
2
Recap on Akka 
• Akka in key points: 
• Messaging 
• Actors 
• Mailboxes (ActorRefs) 
• Hierarchy 
• Supervision 
• Location transparency 
Source: http://www.scottlogic.com/blog/2014/08/15/using-akka-and-scala-to-render-a-mandelbrot-set.html 
3
Example from previous Meetup: URL scraping 
4
To Akka or not to Akka? 
• Akka solves: 
• Concurrency 
• Scalability and distributability 
• Resilience 
• Akka in our team: 
•We had to rewrite a legacy Java component 
• Chose Akka to try it out (after 4 months of research) 
5
Problem #1: Any and Actor Ref 
• All messages are Any-s 
• Anything that’s not 
handled ends up as a “dead 
letter” 
• Essentially a loss of all 
typesafety 
• Requires extensive testing 
to “feel safe” 
class MyActor(databasePersistence: ActorRef, 
emailSender: ActorRef, 
MQSender: ActorRef, 
widgetDao: ActorRef, 
twitterService: ActorRef) 
extends Actor { 
def receive: Receive = { 
case Message(data) => 
twitterService ! Tweet(data) 
// ... 
case OtherMessage(_) => 
// ... 
} 
} 
} 
6
Solution: Typed Channels 
val channelA: ChannelRef[(MsgA, MsgB) :+: TNil] = ??? 
val a = new MsgA 
channelA <-!- a // send a to channelA 
a -!-> channelA // same thing as above 
val fB: Future[MsgB] = channelA <-?- a // ask the actor 
Instead of the current solution: 
val fB: Future[Any] = actorRef ? a 
http://doc.akka.io/docs/akka/2.2.0/scala/typed-channels.html 7
NOPE! 
http://doc.akka.io/docs/akka/2.3.0-RC1/project/migration-guide-2.2.x-2.3.x.html 
8
It’s on the roadmap 
9 
August 28, 2014: http://typesafe.com/blog/akka-roadmap-update-2014 
Lastly, we dare a look beyond the next major release: after 
Akka 2.4.0 [early 2015] we will tackle the number one feature 
request, namely more type-safety for inter-Actor messaging. 
This will be a major research effort and likely also a disruptive 
API change. For this reason Akka 2.4 is planned to be a long-term 
support release, because even though we will try to find 
ways for compatibility or at least an easy migration path we 
cannot currently foresee the exact nature of all related 
changes.
In the meantime: Don’t use Actors for everything 
• What do we use instead? 
• Scala Futures are extremely powerful 
• Futures compose more nicely than Actors 
• When should we use Actors? 
• State in the application 
• A cache is not state 
• Extreme scaling is needed (location transparency) 
• Need the flexibility to place any operation on any JVM -> use Actors for everything 
• Realtime data flow applications (not request/response) 
• What about supervision and „let it crash”? 
• Future’s handle failure also 
10
Code: https://github.com/krivachy/AkkaWithFsm 
11
class Getter(url: String, depth: Int) extends Actor with ActorLogging with LinkParser { 
lazy val client = new NingWSClient(new Builder().build()).url(url) 
override def preStart() = { 
client.get().map(_.body).map(Response).pipeTo(self) 
} 
def receive = { 
case Response(body) => 
val links = parseLinks(body).toList 
log.info(s"URL $url at depth $depth had ${links.size} links.") 
links.foreach { 
link => 
log.info(s"Sending link '$link'") 
context.parent ! Controller.Check(link, depth) 
} 
context.stop(self) 
case Status.Failure(cause) => 
log.error(s"Failed to GET $url", cause) 
context.stop(self) 
} 
} 
12
Simplified with Futures: 
object Getter extends LinkParser { 
case class Urls(urls: Set[String], depth: Int) 
lazy val client = new NingWSClient(new Builder().build()) 
def get(url: String, depth: Int)(implicit ec: ExecutionContext): Future[Urls] = { 
client.url(url).get().map { 
response => 
Urls(parseLinks(response.body).toSet, depth) 
} 
} 
} 
13
Problem #2: Hellish code complexity 
14
var hell 
• Actors have too much mutable state 
• Our worst scenario: Actor with 300 lines and 20 vars 
• Hard to reason about state when everything is “global” inside the 
Actor 
• How do you initialize state that is only used in some cases? 
• var something: SomeType = _ 
• NPE 
• var something: Option[SomeType] = None 
• Always need to “getOrElse” 
15
become/unbecome hell 
• Pushing and popping state on a stack 
• context.become(behavior: Receive, discardOld: Boolean = true) 
• context.unbecome() 
• “context.become” isn’t enforced to be called last 
• You use methods to keep things short, but there will be multiple methods 
trying to modify the behaviour 
• i.e. you could end up inadvertently overwriting behavior 
• One place: context.become(handleCoolMessages orElse waitForNewRequests) 
• Somewhere else: context.become(waitForNewRequests, discardOld = true) 
• When things blow up you have no idea how you got there 
16
class Receptionist extends Actor { 
def receive = waiting 
def waiting: Receive = { 
case Api.Scrape(url, depth) => 
context.become(next(Vector(Job(sender, url, depth)))) 
} 
def running(queue: Vector[Job]): Receive = LoggingReceive { 
case Result(links) => 
// … 
context.become(next(queue.tail)) 
case Api.Scrape(url, depth) => 
context.become(enqueue(queue, Job(sender, url, depth))) 
case Terminated(_) => 
// … 
context.become(next(queue.tail)) 
} 
def enqueue(queue: Vector[Job], job: Job): Receive = LoggingReceive { 
// … 
running(queue) 
} 
def next(queue: Vector[Job]): Receive = LoggingReceive { 
// … 
running(queue) 
} 
} 
17
Debug hell 
• Debugging Actors is hard 
• Stacktraces lack meaning 
• Need to have a lot of boilerplate utility code: 
• What was the message? 
• Who sent the message? 
• What’s my internal state? 
• Possible solution: 
• always override def preRestart(reason: Throwable, message: Option[Any]) 
• Still don’t know how we ended up in that become/unbecome state. 
18
[ERROR] [akka://simple-actor/user/receptionist/controller-1/$a] Exception happened 
java.lang.Exception: exception happened here 
at meetup.akka.simple.Getter$$anonfun$receive$1$$anonfun$1.apply(Getter.scala:37) 
at meetup.akka.simple.Getter$$anonfun$receive$1$$anonfun$1.apply(Getter.scala:37) 
at scala.util.Try$.apply(Try.scala:191) 
at meetup.akka.simple.Getter$$anonfun$receive$1.applyOrElse(Getter.scala:37) 
at akka.actor.Actor$class.aroundReceive(Actor.scala:465) 
at meetup.akka.simple.Getter.aroundReceive(Getter.scala:16) 
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:516) 
at akka.actor.ActorCell.invoke(ActorCell.scala:487) 
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:238) 
at akka.dispatch.Mailbox.run(Mailbox.scala:220) 
at 
akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:393) 
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) 
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) 
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) 
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) 
19
We need Logging for Debugging 
• Akka provides: 
• with ActorLogging 
• log.info(…) 
• LoggingRecieve: def receive = LoggingReceive { … } 
• turn on: akka.actor.debug.receive=true 
• Lifecycle logging: akka.actor.debug.lifecycle=true 
• For all Actors 
• Start, restart, stop, supervise, watch events 
• Autorecieve logging: akka.actor.debug.autoreceive=true 
• For all Actors 
• Automatically handled messages: Stop, Kill, PoisionPill, Terminated, etc. 
• Issue: 
• Akka only provides logging of messages per Receive block (has pros and cons) 
• If you missed one => good luck debugging issues around it in production 
20
Logging example 
[DEBUG] [akka://simple-actor/user/receptionist] started (meetup.akka.simple.Receptionist@14ba772) 
[DEBUG] [akka://simple-actor/user/receptionist/controller-1] started (meetup.akka.simple.Controller@38a5d7) 
[DEBUG] [akka://simple-actor/user/receptionist] now supervising Actor[akka://simple-actor/user/receptionist/controller-1#4232237] 
[DEBUG] [akka://simple-actor/user/receptionist/controller-1] now watched by Actor[akka://simple-actor/user/receptionist#1565954732] 
[DEBUG] [akka://simple-actor/user/receptionist/controller-1] received handled message 
Check(http://doc.akka.io/docs/akka/2.3.5/intro/what-is-akka.html,1) 
21
FSM 
22
“Everything* is a Finite State Machine.”** 
-me 
*not everything 
** do not quote me 
23
FSM concepts 
object Receptionist { 
object Internal { 
sealed trait State 
case object Sleeping extends State 
case object Processing extends State 
sealed trait Data 
case class NoQueue(requestId: Int = 0) extends Data 
case class Queue(currentRequestId: Int, items: Vector[Job]) extends Data 
} 
} 
class Receptionist extends FSM[Internal.State, Internal.Data] { … } 
24
Define handlers for all states 
// Initialize with data 
startWith(Sleeping, NoQueue()) 
// Handlers for states 
when(Sleeping)(enqueueNewRequest) 
when(Processing) (processResult orElse enqueueNewRequest orElse reportError) 
25
Enqueue New Request State Function 
def enqueueNewRequest: StateFunction = { 
case Event(Api.Scrape(url, depth), NoQueue(requestId)) => 
startControllerFor(requestId + 1, Vector(Job(sender(), url, depth))) 
case Event(Api.Scrape(url, depth), queue: Queue) => 
if (queue.items.size > 3) { 
stay replying Api.Failed(url) 
} else { 
goto(Processing) using 
Queue(queue.currentRequestId, queue.items :+ Job(sender(), url, depth)) 
} 
} 
• Important: 
• StateFunction: Event => State 
• Event(incomingMessage, data) => 
• State transition: goto/stay (nextState) using (data) forMax(timeout) replying (message) 
26
Monitoring state transitions 
Internal: 
onTransition { 
case Idle -> Active => setTimer("timeout", Tick, 1 second, true) 
case Active -> _ => cancelTimer("timeout") 
case x -> Idle => log.info("entering Idle from " + x) 
} 
External: 
monitoredActor ! SubscribeTransitionCallBack(self) 
def recieve = { 
case Transition(monitoredActor, oldState, newState) => 
if (newState == Errored) alert.raiseAlert(...) 
} 
override def postStop() = { 
monitoredActor ! UnsubscribeTransitionCallBack(self) 
} 
27
Handling failure 
whenUnhandled { 
case Event(any, data) => 
val logUpToHere = prettyPrint(getLog) 
log.error(s"Unhandled event: ${any}n$logUpToHere") 
stay() 
} 
def failure: StateFunction = { 
case Event(Status.Failure(cause), _) => 
log.error(s"Failed to GET $url", cause) 
stop(FSM.Failure(cause)) 
} 
onTermination { 
case StopEvent(FSM.Normal, state, data) => ??? 
case StopEvent(FSM.Shutdown, state, data) => ??? 
case StopEvent(FSM.Failure(cause), state, data) => ??? 
} 28
Result is simplicity 
class Receptionist extends FSM[Internal.State, Internal.Data] { 
startWith(Sleeping, NoQueue()) 
when(Sleeping)(enqueueNewRequest) 
when(Processing) (processResult orElse enqueueNewRequest orElse reportError) 
def enqueueNewRequest: StateFunction = ??? 
def processResult: StateFunction = ??? 
def reportError: StateFunction = ??? 
private def nextQueueItem(queue: Queue): State = ??? 
private def startControllerFor(requestId: Int, queue: Vector[Job]): State = ??? 
whenUnhandled { ??? } 
initialize() 
} 
29
Hell status 
var ✔ 
No more mutable global state 
inside Actors. Everything is 
typed to the specific State. 
become/unbecome ✔ 
All methods have to end in a 
state transition. States are 
clearly defined what they do. 
debug/logging ? 
30
LoggingFSM 
• Remembers state transitions: 
• override def logDepth = 8 
• def getLog: Seq[LogEntry[Internal.State, Internal.Data]] 
• Use with: onTermination 
• Debug logging: akka.actor.debug.fsm=true 
• Automatically logs important Events: message + internal data 
• Logs state transitions 
• Use with: akka.actor.debug.lifecycle=true 
31
Debug log example 
[DEBUG] [akka://fsm/user/receptionist] processing 
Event(Result(Set([…])),Queue(1,Vector(Job(Actor[akka://fsm/system/testActor1#758674372],http://doc.akka.io 
/docs/akka/2.3.5/intro/what-is-akka.html,1)))) from Actor[akka://fsm/user/receptionist/controller-1#4232237] 
[DEBUG] [akka://fsm/user/receptionist/controller-1] transition CollectingResults -> Completed 
[DEBUG] [akka://fsm/user/receptionist] transition Processing -> Sleeping 
[DEBUG] [akka://fsm/user/receptionist/controller-1] stopped 
32
[ERROR] [akka://fsm/user/receptionist] Unhandled event: some string 
Last 8 entries leading up to this point: 
in state: Sleeping 
with data: NoQueue(2) 
received: Scrape(http://non-existent.link,5) 
in state: Processing 
with data: Queue(3,Vector(Job(Actor[akka://fsm/system/testActor1#758674372],http://non-existent. 
link,5))) 
received: Result(Set(http://non-existent.link)) 
[…] 
in state: Processing 
with data: 
Queue(4,Vector(Job(Actor[akka://fsm/system/testActor1#758674372],http://non.existent1,0), 
Job(Actor[akka://fsm/system/testActor1#758674372],http://non.existent2,0), 
Job(Actor[akka://fsm/system/testActor1#758674372],http://non.existent3,0), 
Job(Actor[akka://fsm/system/testActor1#758674372],http://non.existent4,0))) 
received: some string 33
Hell status 
var ✔ 
No more mutable global state 
inside Actors. Everything is 
typed to the specific State. 
become/unbecome ✔ 
All methods have to end in a 
state transition. States are 
clearly defined what they do. 
debug/logging ✔ 
FSM does all the debug logging 
we would ever need. 
34
Conclusion: My experiences 
Some people, when confronted with a scalability problem, think 
“I know, I'll use Akka.” Now they have two problems. 
• Akka is awesome, but: 
• Needs a new mindset 
• Unexperienced developers + learning curve + easy to make 
mistakes = headaches 
• In large teams (30+) where everyone is expected to be able to 
change all code it becomes an issue. 
• Use Actors only when really needed 
• All Actors should probably be FSM 
@krivachy https://github.com/krivachy/AkkaWithFsm

More Related Content

PDF
2014-02-20 | Akka Concurrency (Vienna Scala User Group)
PDF
The dark side of Akka and the remedy
PDF
Akka Futures and Akka Remoting
PPTX
Concurrency in Scala - the Akka way
PDF
Reactive Web-Applications @ LambdaDays
PDF
Actor Model Akka Framework
PDF
Actor Clustering with Docker Containers and Akka.Net in F#
PPTX
Akka Actor presentation
2014-02-20 | Akka Concurrency (Vienna Scala User Group)
The dark side of Akka and the remedy
Akka Futures and Akka Remoting
Concurrency in Scala - the Akka way
Reactive Web-Applications @ LambdaDays
Actor Model Akka Framework
Actor Clustering with Docker Containers and Akka.Net in F#
Akka Actor presentation

What's hot (20)

ZIP
Above the clouds: introducing Akka
PDF
Akka Cluster in Java - JCConf 2015
PDF
Concurrecny inf sharp
PDF
Building reactive distributed systems with Akka
PPTX
Paws: A Perl AWS SDK - YAPC Europe 2015
PDF
Back to the futures, actors and pipes: using Akka for large-scale data migration
PDF
3 things you must know to think reactive - Geecon Kraków 2015
PDF
A gentle introduction into AKKA and the actor model
PPTX
Building an aws sdk for Perl - Granada Perl Workshop 2014
PPTX
Fullstack Conference - Proxies before proxies: The hidden gems of Javascript...
PDF
Akka persistence == event sourcing in 30 minutes
PDF
Advanced akka features
PPTX
Paws - A Perl AWS SDK
KEY
The Why and How of Scala at Twitter
PPTX
Java 8 concurrency abstractions
PDF
Akka lsug skills matter
PPTX
Paws - Perl AWS SDK Update - November 2015
PDF
Reactive stream processing using Akka streams
PPTX
Reactive Programming in Java 8 with Rx-Java
PDF
Reactive Thinking in Java
Above the clouds: introducing Akka
Akka Cluster in Java - JCConf 2015
Concurrecny inf sharp
Building reactive distributed systems with Akka
Paws: A Perl AWS SDK - YAPC Europe 2015
Back to the futures, actors and pipes: using Akka for large-scale data migration
3 things you must know to think reactive - Geecon Kraków 2015
A gentle introduction into AKKA and the actor model
Building an aws sdk for Perl - Granada Perl Workshop 2014
Fullstack Conference - Proxies before proxies: The hidden gems of Javascript...
Akka persistence == event sourcing in 30 minutes
Advanced akka features
Paws - A Perl AWS SDK
The Why and How of Scala at Twitter
Java 8 concurrency abstractions
Akka lsug skills matter
Paws - Perl AWS SDK Update - November 2015
Reactive stream processing using Akka streams
Reactive Programming in Java 8 with Rx-Java
Reactive Thinking in Java
Ad

Viewers also liked (10)

PDF
Reactive Streams 1.0.0 and Why You Should Care (webinar)
PDF
From polling to real time: Scala, Akka, and Websockets from scratch
PDF
2014-11-26 | Creating a BitTorrent Client with Scala and Akka, Part 1 (Vienna...
PDF
Play Framework and Activator
PPTX
Refactoring Design Patterns the Functional Way (in Scala)
PDF
Spark Summit EU talk by Josef Habdank
PDF
Fresh from the Oven (04.2015): Experimental Akka Typed and Akka Streams
PDF
Akka Streams and HTTP
PDF
Spring Boot Microservices vs Akka Actor Cluster
PDF
Top 5 Mistakes When Writing Spark Applications by Mark Grover and Ted Malaska
Reactive Streams 1.0.0 and Why You Should Care (webinar)
From polling to real time: Scala, Akka, and Websockets from scratch
2014-11-26 | Creating a BitTorrent Client with Scala and Akka, Part 1 (Vienna...
Play Framework and Activator
Refactoring Design Patterns the Functional Way (in Scala)
Spark Summit EU talk by Josef Habdank
Fresh from the Oven (04.2015): Experimental Akka Typed and Akka Streams
Akka Streams and HTTP
Spring Boot Microservices vs Akka Actor Cluster
Top 5 Mistakes When Writing Spark Applications by Mark Grover and Ted Malaska
Ad

Similar to The dark side of Akka and the remedy - bp.scala meetup (20)

PDF
Introduction to Actor Model and Akka
PDF
Writing Asynchronous Programs with Scala & Akka
PDF
Introducing Akka
PPTX
Scale up your thinking
KEY
Akka london scala_user_group
PDF
Functional Programming and Composing Actors
PPTX
Introduction to Akka - Atlanta Java Users Group
PDF
Coding Resiliently with Akka
PDF
Building Stateful Microservices With Akka
PDF
Scaling Web Apps with Akka
PPTX
Nairobi JVM meetup : Introduction to akka
 
PDF
Akka and futures
PDF
Effective Akka v2
PDF
Akka in Production - ScalaDays 2015
PDF
Message-based communication patterns in distributed Akka applications
PDF
Akka and the Zen of Reactive System Design
PDF
Build Cloud Applications with Akka and Heroku
PDF
Akka 2.0 Reloaded
PDF
Akka knolx
PDF
Activator and Reactive at Play NYC meetup
Introduction to Actor Model and Akka
Writing Asynchronous Programs with Scala & Akka
Introducing Akka
Scale up your thinking
Akka london scala_user_group
Functional Programming and Composing Actors
Introduction to Akka - Atlanta Java Users Group
Coding Resiliently with Akka
Building Stateful Microservices With Akka
Scaling Web Apps with Akka
Nairobi JVM meetup : Introduction to akka
 
Akka and futures
Effective Akka v2
Akka in Production - ScalaDays 2015
Message-based communication patterns in distributed Akka applications
Akka and the Zen of Reactive System Design
Build Cloud Applications with Akka and Heroku
Akka 2.0 Reloaded
Akka knolx
Activator and Reactive at Play NYC meetup

Recently uploaded (20)

PDF
How UI/UX Design Impacts User Retention in Mobile Apps.pdf
PDF
The Rise and Fall of 3GPP – Time for a Sabbatical?
PDF
cuic standard and advanced reporting.pdf
PDF
NewMind AI Monthly Chronicles - July 2025
DOCX
The AUB Centre for AI in Media Proposal.docx
PDF
Blue Purple Modern Animated Computer Science Presentation.pdf.pdf
PDF
Architecting across the Boundaries of two Complex Domains - Healthcare & Tech...
PDF
Approach and Philosophy of On baking technology
PPTX
PA Analog/Digital System: The Backbone of Modern Surveillance and Communication
PDF
Build a system with the filesystem maintained by OSTree @ COSCUP 2025
PDF
Reach Out and Touch Someone: Haptics and Empathic Computing
PDF
Dropbox Q2 2025 Financial Results & Investor Presentation
PDF
Chapter 3 Spatial Domain Image Processing.pdf
PDF
Shreyas Phanse Resume: Experienced Backend Engineer | Java • Spring Boot • Ka...
PPTX
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
PDF
CIFDAQ's Market Insight: SEC Turns Pro Crypto
PPTX
Detection-First SIEM: Rule Types, Dashboards, and Threat-Informed Strategy
PDF
Spectral efficient network and resource selection model in 5G networks
PPTX
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
PDF
Unlocking AI with Model Context Protocol (MCP)
How UI/UX Design Impacts User Retention in Mobile Apps.pdf
The Rise and Fall of 3GPP – Time for a Sabbatical?
cuic standard and advanced reporting.pdf
NewMind AI Monthly Chronicles - July 2025
The AUB Centre for AI in Media Proposal.docx
Blue Purple Modern Animated Computer Science Presentation.pdf.pdf
Architecting across the Boundaries of two Complex Domains - Healthcare & Tech...
Approach and Philosophy of On baking technology
PA Analog/Digital System: The Backbone of Modern Surveillance and Communication
Build a system with the filesystem maintained by OSTree @ COSCUP 2025
Reach Out and Touch Someone: Haptics and Empathic Computing
Dropbox Q2 2025 Financial Results & Investor Presentation
Chapter 3 Spatial Domain Image Processing.pdf
Shreyas Phanse Resume: Experienced Backend Engineer | Java • Spring Boot • Ka...
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
CIFDAQ's Market Insight: SEC Turns Pro Crypto
Detection-First SIEM: Rule Types, Dashboards, and Threat-Informed Strategy
Spectral efficient network and resource selection model in 5G networks
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
Unlocking AI with Model Context Protocol (MCP)

The dark side of Akka and the remedy - bp.scala meetup

  • 1. The dark side of Akka and the remedy Ákos Kriváchy
  • 2. Introduction - Ákos Kriváchy • Scala disciple/convert/fanatic since 2013 February • First FP language • Akka since 2014 February • Things I love about Scala: • Static typing • Partial Functions • foldLeft, tail recursion 2
  • 3. Recap on Akka • Akka in key points: • Messaging • Actors • Mailboxes (ActorRefs) • Hierarchy • Supervision • Location transparency Source: http://www.scottlogic.com/blog/2014/08/15/using-akka-and-scala-to-render-a-mandelbrot-set.html 3
  • 4. Example from previous Meetup: URL scraping 4
  • 5. To Akka or not to Akka? • Akka solves: • Concurrency • Scalability and distributability • Resilience • Akka in our team: •We had to rewrite a legacy Java component • Chose Akka to try it out (after 4 months of research) 5
  • 6. Problem #1: Any and Actor Ref • All messages are Any-s • Anything that’s not handled ends up as a “dead letter” • Essentially a loss of all typesafety • Requires extensive testing to “feel safe” class MyActor(databasePersistence: ActorRef, emailSender: ActorRef, MQSender: ActorRef, widgetDao: ActorRef, twitterService: ActorRef) extends Actor { def receive: Receive = { case Message(data) => twitterService ! Tweet(data) // ... case OtherMessage(_) => // ... } } } 6
  • 7. Solution: Typed Channels val channelA: ChannelRef[(MsgA, MsgB) :+: TNil] = ??? val a = new MsgA channelA <-!- a // send a to channelA a -!-> channelA // same thing as above val fB: Future[MsgB] = channelA <-?- a // ask the actor Instead of the current solution: val fB: Future[Any] = actorRef ? a http://doc.akka.io/docs/akka/2.2.0/scala/typed-channels.html 7
  • 9. It’s on the roadmap 9 August 28, 2014: http://typesafe.com/blog/akka-roadmap-update-2014 Lastly, we dare a look beyond the next major release: after Akka 2.4.0 [early 2015] we will tackle the number one feature request, namely more type-safety for inter-Actor messaging. This will be a major research effort and likely also a disruptive API change. For this reason Akka 2.4 is planned to be a long-term support release, because even though we will try to find ways for compatibility or at least an easy migration path we cannot currently foresee the exact nature of all related changes.
  • 10. In the meantime: Don’t use Actors for everything • What do we use instead? • Scala Futures are extremely powerful • Futures compose more nicely than Actors • When should we use Actors? • State in the application • A cache is not state • Extreme scaling is needed (location transparency) • Need the flexibility to place any operation on any JVM -> use Actors for everything • Realtime data flow applications (not request/response) • What about supervision and „let it crash”? • Future’s handle failure also 10
  • 12. class Getter(url: String, depth: Int) extends Actor with ActorLogging with LinkParser { lazy val client = new NingWSClient(new Builder().build()).url(url) override def preStart() = { client.get().map(_.body).map(Response).pipeTo(self) } def receive = { case Response(body) => val links = parseLinks(body).toList log.info(s"URL $url at depth $depth had ${links.size} links.") links.foreach { link => log.info(s"Sending link '$link'") context.parent ! Controller.Check(link, depth) } context.stop(self) case Status.Failure(cause) => log.error(s"Failed to GET $url", cause) context.stop(self) } } 12
  • 13. Simplified with Futures: object Getter extends LinkParser { case class Urls(urls: Set[String], depth: Int) lazy val client = new NingWSClient(new Builder().build()) def get(url: String, depth: Int)(implicit ec: ExecutionContext): Future[Urls] = { client.url(url).get().map { response => Urls(parseLinks(response.body).toSet, depth) } } } 13
  • 14. Problem #2: Hellish code complexity 14
  • 15. var hell • Actors have too much mutable state • Our worst scenario: Actor with 300 lines and 20 vars • Hard to reason about state when everything is “global” inside the Actor • How do you initialize state that is only used in some cases? • var something: SomeType = _ • NPE • var something: Option[SomeType] = None • Always need to “getOrElse” 15
  • 16. become/unbecome hell • Pushing and popping state on a stack • context.become(behavior: Receive, discardOld: Boolean = true) • context.unbecome() • “context.become” isn’t enforced to be called last • You use methods to keep things short, but there will be multiple methods trying to modify the behaviour • i.e. you could end up inadvertently overwriting behavior • One place: context.become(handleCoolMessages orElse waitForNewRequests) • Somewhere else: context.become(waitForNewRequests, discardOld = true) • When things blow up you have no idea how you got there 16
  • 17. class Receptionist extends Actor { def receive = waiting def waiting: Receive = { case Api.Scrape(url, depth) => context.become(next(Vector(Job(sender, url, depth)))) } def running(queue: Vector[Job]): Receive = LoggingReceive { case Result(links) => // … context.become(next(queue.tail)) case Api.Scrape(url, depth) => context.become(enqueue(queue, Job(sender, url, depth))) case Terminated(_) => // … context.become(next(queue.tail)) } def enqueue(queue: Vector[Job], job: Job): Receive = LoggingReceive { // … running(queue) } def next(queue: Vector[Job]): Receive = LoggingReceive { // … running(queue) } } 17
  • 18. Debug hell • Debugging Actors is hard • Stacktraces lack meaning • Need to have a lot of boilerplate utility code: • What was the message? • Who sent the message? • What’s my internal state? • Possible solution: • always override def preRestart(reason: Throwable, message: Option[Any]) • Still don’t know how we ended up in that become/unbecome state. 18
  • 19. [ERROR] [akka://simple-actor/user/receptionist/controller-1/$a] Exception happened java.lang.Exception: exception happened here at meetup.akka.simple.Getter$$anonfun$receive$1$$anonfun$1.apply(Getter.scala:37) at meetup.akka.simple.Getter$$anonfun$receive$1$$anonfun$1.apply(Getter.scala:37) at scala.util.Try$.apply(Try.scala:191) at meetup.akka.simple.Getter$$anonfun$receive$1.applyOrElse(Getter.scala:37) at akka.actor.Actor$class.aroundReceive(Actor.scala:465) at meetup.akka.simple.Getter.aroundReceive(Getter.scala:16) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:516) at akka.actor.ActorCell.invoke(ActorCell.scala:487) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:238) at akka.dispatch.Mailbox.run(Mailbox.scala:220) at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:393) at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) 19
  • 20. We need Logging for Debugging • Akka provides: • with ActorLogging • log.info(…) • LoggingRecieve: def receive = LoggingReceive { … } • turn on: akka.actor.debug.receive=true • Lifecycle logging: akka.actor.debug.lifecycle=true • For all Actors • Start, restart, stop, supervise, watch events • Autorecieve logging: akka.actor.debug.autoreceive=true • For all Actors • Automatically handled messages: Stop, Kill, PoisionPill, Terminated, etc. • Issue: • Akka only provides logging of messages per Receive block (has pros and cons) • If you missed one => good luck debugging issues around it in production 20
  • 21. Logging example [DEBUG] [akka://simple-actor/user/receptionist] started (meetup.akka.simple.Receptionist@14ba772) [DEBUG] [akka://simple-actor/user/receptionist/controller-1] started (meetup.akka.simple.Controller@38a5d7) [DEBUG] [akka://simple-actor/user/receptionist] now supervising Actor[akka://simple-actor/user/receptionist/controller-1#4232237] [DEBUG] [akka://simple-actor/user/receptionist/controller-1] now watched by Actor[akka://simple-actor/user/receptionist#1565954732] [DEBUG] [akka://simple-actor/user/receptionist/controller-1] received handled message Check(http://doc.akka.io/docs/akka/2.3.5/intro/what-is-akka.html,1) 21
  • 23. “Everything* is a Finite State Machine.”** -me *not everything ** do not quote me 23
  • 24. FSM concepts object Receptionist { object Internal { sealed trait State case object Sleeping extends State case object Processing extends State sealed trait Data case class NoQueue(requestId: Int = 0) extends Data case class Queue(currentRequestId: Int, items: Vector[Job]) extends Data } } class Receptionist extends FSM[Internal.State, Internal.Data] { … } 24
  • 25. Define handlers for all states // Initialize with data startWith(Sleeping, NoQueue()) // Handlers for states when(Sleeping)(enqueueNewRequest) when(Processing) (processResult orElse enqueueNewRequest orElse reportError) 25
  • 26. Enqueue New Request State Function def enqueueNewRequest: StateFunction = { case Event(Api.Scrape(url, depth), NoQueue(requestId)) => startControllerFor(requestId + 1, Vector(Job(sender(), url, depth))) case Event(Api.Scrape(url, depth), queue: Queue) => if (queue.items.size > 3) { stay replying Api.Failed(url) } else { goto(Processing) using Queue(queue.currentRequestId, queue.items :+ Job(sender(), url, depth)) } } • Important: • StateFunction: Event => State • Event(incomingMessage, data) => • State transition: goto/stay (nextState) using (data) forMax(timeout) replying (message) 26
  • 27. Monitoring state transitions Internal: onTransition { case Idle -> Active => setTimer("timeout", Tick, 1 second, true) case Active -> _ => cancelTimer("timeout") case x -> Idle => log.info("entering Idle from " + x) } External: monitoredActor ! SubscribeTransitionCallBack(self) def recieve = { case Transition(monitoredActor, oldState, newState) => if (newState == Errored) alert.raiseAlert(...) } override def postStop() = { monitoredActor ! UnsubscribeTransitionCallBack(self) } 27
  • 28. Handling failure whenUnhandled { case Event(any, data) => val logUpToHere = prettyPrint(getLog) log.error(s"Unhandled event: ${any}n$logUpToHere") stay() } def failure: StateFunction = { case Event(Status.Failure(cause), _) => log.error(s"Failed to GET $url", cause) stop(FSM.Failure(cause)) } onTermination { case StopEvent(FSM.Normal, state, data) => ??? case StopEvent(FSM.Shutdown, state, data) => ??? case StopEvent(FSM.Failure(cause), state, data) => ??? } 28
  • 29. Result is simplicity class Receptionist extends FSM[Internal.State, Internal.Data] { startWith(Sleeping, NoQueue()) when(Sleeping)(enqueueNewRequest) when(Processing) (processResult orElse enqueueNewRequest orElse reportError) def enqueueNewRequest: StateFunction = ??? def processResult: StateFunction = ??? def reportError: StateFunction = ??? private def nextQueueItem(queue: Queue): State = ??? private def startControllerFor(requestId: Int, queue: Vector[Job]): State = ??? whenUnhandled { ??? } initialize() } 29
  • 30. Hell status var ✔ No more mutable global state inside Actors. Everything is typed to the specific State. become/unbecome ✔ All methods have to end in a state transition. States are clearly defined what they do. debug/logging ? 30
  • 31. LoggingFSM • Remembers state transitions: • override def logDepth = 8 • def getLog: Seq[LogEntry[Internal.State, Internal.Data]] • Use with: onTermination • Debug logging: akka.actor.debug.fsm=true • Automatically logs important Events: message + internal data • Logs state transitions • Use with: akka.actor.debug.lifecycle=true 31
  • 32. Debug log example [DEBUG] [akka://fsm/user/receptionist] processing Event(Result(Set([…])),Queue(1,Vector(Job(Actor[akka://fsm/system/testActor1#758674372],http://doc.akka.io /docs/akka/2.3.5/intro/what-is-akka.html,1)))) from Actor[akka://fsm/user/receptionist/controller-1#4232237] [DEBUG] [akka://fsm/user/receptionist/controller-1] transition CollectingResults -> Completed [DEBUG] [akka://fsm/user/receptionist] transition Processing -> Sleeping [DEBUG] [akka://fsm/user/receptionist/controller-1] stopped 32
  • 33. [ERROR] [akka://fsm/user/receptionist] Unhandled event: some string Last 8 entries leading up to this point: in state: Sleeping with data: NoQueue(2) received: Scrape(http://non-existent.link,5) in state: Processing with data: Queue(3,Vector(Job(Actor[akka://fsm/system/testActor1#758674372],http://non-existent. link,5))) received: Result(Set(http://non-existent.link)) […] in state: Processing with data: Queue(4,Vector(Job(Actor[akka://fsm/system/testActor1#758674372],http://non.existent1,0), Job(Actor[akka://fsm/system/testActor1#758674372],http://non.existent2,0), Job(Actor[akka://fsm/system/testActor1#758674372],http://non.existent3,0), Job(Actor[akka://fsm/system/testActor1#758674372],http://non.existent4,0))) received: some string 33
  • 34. Hell status var ✔ No more mutable global state inside Actors. Everything is typed to the specific State. become/unbecome ✔ All methods have to end in a state transition. States are clearly defined what they do. debug/logging ✔ FSM does all the debug logging we would ever need. 34
  • 35. Conclusion: My experiences Some people, when confronted with a scalability problem, think “I know, I'll use Akka.” Now they have two problems. • Akka is awesome, but: • Needs a new mindset • Unexperienced developers + learning curve + easy to make mistakes = headaches • In large teams (30+) where everyone is expected to be able to change all code it becomes an issue. • Use Actors only when really needed • All Actors should probably be FSM @krivachy https://github.com/krivachy/AkkaWithFsm