ElasticMQ: a fully asynchronous, Akka-based SQS server
2 likes18,185 views
The document discusses ElasticMQ, a lightweight, in-memory implementation of the Amazon SQS API that allows testing of SQS applications locally. It is built using Scala, Akka, and Spray and provides an asynchronous, reactive alternative to testing SQS applications without using the actual SQS service. The document demonstrates how to run ElasticMQ as a standalone server or embed it in an application and use the SQS API against it to test sending and receiving messages locally.
![Stand-alone
$
java
-‐jar
elasticmq-‐server-‐0.7.1.jar
[main]
INFO
org.elasticmq.server.Main$
-‐
Starting
ElasticMQ
server
(0.7.1)
...
[main]
INFO
o.e.rest.sqs.TheSQSRestServerBuilder
-‐
Started
SQS
rest
server,
bind
address
0.0.0.0:9324,
visible
server
address
http://localhost:9324
[main]
INFO
org.elasticmq.server.Main$
-‐
===
ElasticMQ
server
(0.7.1)
started
in
1444
ms
===
#DV13 #elasticmq
Wednesday 13 November 13
@adamwarski](https://image.slidesharecdn.com/emqquickie-131115132505-phpapp02/85/ElasticMQ-a-fully-asynchronous-Akka-based-SQS-server-7-320.jpg)
![Receive message walk-through
val
receiveMessage
=
action("ReceiveMessage")
{
//
path("")
and
param()
param("VisibilityTimeout".as[Int]?,
"WaitTimeSeconds".as[Long]?)
{
(visibilityTimeout,
waitTimeSeconds)
=>
respondWithMediaType(MediaTypes.`text/xml`)
{
//
inner
route:
RequestContext
=>
Unit
}
}
}
#DV13 #elasticmq
Wednesday 13 November 13
@adamwarski](https://image.slidesharecdn.com/emqquickie-131115132505-phpapp02/85/ElasticMQ-a-fully-asynchronous-Akka-based-SQS-server-15-320.jpg)
![Receive message walk-through
//
inner
route:
RequestContext
=>
Unit
ctx:
RequestContext
=>
val
actorMsg
=
ReceiveMessages(visibilityTimeout,
waitTimeSeconds)
val
msgs:
Future[List[Message]]
=
queueActor
?
actorMsg
msgs.map
{
msgs
=>
ctx.complete(
<ReceiveMessageResponse>
...
</ReceiveMessageResponse>
)
}
#DV13 #elasticmq
Wednesday 13 November 13
@adamwarski](https://image.slidesharecdn.com/emqquickie-131115132505-phpapp02/85/ElasticMQ-a-fully-asynchronous-Akka-based-SQS-server-16-320.jpg)
![Receive message walk-through
import
akka.actor.{Actor,
ActorRef}
class
QueueActor
extends
Actor
{
val
messageQueue
=
mutable.PriorityQueue[InternalMessage]()
val
awaiting
=
mutable.PriorityQueue[ActorRef]()
def
receive
=
{
case
ReceiveMessages(...)
=>
{
//
if
there
are
messages,
reply
//
otherwise
put
the
sender
aside
//
schedule
a
timeout
in
20
seconds
}
}
}
#DV13 #elasticmq
Wednesday 13 November 13
@adamwarski](https://image.slidesharecdn.com/emqquickie-131115132505-phpapp02/85/ElasticMQ-a-fully-asynchronous-Akka-based-SQS-server-17-320.jpg)
![Dataflow
val
result:
Future[ActorRef]
=
flow
{
(queueManager
?
Lookup(name)).apply()
match
{
case
Some(queueActor)
=>
queueActor
case
None
=>
val
createFuture
=
queueManager
?
Create(name)
createFuture.apply()
}
}
#DV13 #elasticmq
Wednesday 13 November 13
@adamwarski](https://image.slidesharecdn.com/emqquickie-131115132505-phpapp02/85/ElasticMQ-a-fully-asynchronous-Akka-based-SQS-server-19-320.jpg)
ElasticMQ: a fully asynchronous, Akka-based SQS server
- 1. ElasticMQ a fully async, Akka-based Amazon SQS server Adam Warski SoftwareMill #DV13 #elasticmq Wednesday 13 November 13 @adamwarski
- 2. What is Amazon SQS? • MQ-as-a-service • Send, Receive, Delete • At-least-once delivery #DV13 #elasticmq Wednesday 13 November 13 @adamwarski
- 3. How to test SQS apps? 1. Don’t? #DV13 #elasticmq Wednesday 13 November 13 @adamwarski
- 4. How to test SQS apps? 2. Just use SQS? #DV13 #elasticmq Wednesday 13 November 13 @adamwarski
- 5. How to test SQS apps? 3. Use a local SQS server #DV13 #elasticmq Wednesday 13 November 13 @adamwarski
- 6. ElasticMQ • (relevant) subset of SQS • In-memory • Lightweight #DV13 #elasticmq Wednesday 13 November 13 @adamwarski
- 7. Stand-alone $ java -‐jar elasticmq-‐server-‐0.7.1.jar [main] INFO org.elasticmq.server.Main$ -‐ Starting ElasticMQ server (0.7.1) ... [main] INFO o.e.rest.sqs.TheSQSRestServerBuilder -‐ Started SQS rest server, bind address 0.0.0.0:9324, visible server address http://localhost:9324 [main] INFO org.elasticmq.server.Main$ -‐ === ElasticMQ server (0.7.1) started in 1444 ms === #DV13 #elasticmq Wednesday 13 November 13 @adamwarski
- 8. Using ElasticMQ import com.amazonaws.auth.BasicAWSCredentials import com.amazonaws.services.sqs.AmazonSQSClient client = new AmazonSQSClient(new BasicAWSCredentials("x", "x")) client.setEndpoint("http://localhost:9324") val queueUrl = client.createQueue( new CreateQueueRequest("testQueue1")) client.sendMessage(new SendMessageRequest(queueUrl, "Hello!")) #DV13 #elasticmq Wednesday 13 November 13 @adamwarski
- 9. Embedded <dependency> <groupId>org.elasticmq</groupId> <artifactId>elasticmq-‐rest-‐sqs_2.10</artifactId> <version>0.7.1</version> </dependency> val server = SQSRestServerBuilder .withPort(9325) .withInterface("localhost") .start() // ... use ... server.stopAndWait() #DV13 #elasticmq Wednesday 13 November 13 @adamwarski
- 10. That’s all! Thanks! #DV13 #elasticmq Wednesday 13 November 13 @adamwarski
- 11. #DV13 #elasticmq Wednesday 13 November 13 @adamwarski
- 12. Technologies • Scala • Akka • Spray #DV13 #elasticmq Wednesday 13 November 13 ➡ reactive @adamwarski
- 13. Asynchronous: why? • Traditional model could work well? • Long polling #DV13 #elasticmq Wednesday 13 November 13 @adamwarski
- 14. Receive message walk-through import spray.routing.SimpleRoutingApp val routes = sendMessage ~ receiveMessage ~ createQueue ~ ... val app = new SimpleRoutingApp {} app.startServer(interface, port, "...") { routes } #DV13 #elasticmq Wednesday 13 November 13 @adamwarski
- 15. Receive message walk-through val receiveMessage = action("ReceiveMessage") { // path("") and param() param("VisibilityTimeout".as[Int]?, "WaitTimeSeconds".as[Long]?) { (visibilityTimeout, waitTimeSeconds) => respondWithMediaType(MediaTypes.`text/xml`) { // inner route: RequestContext => Unit } } } #DV13 #elasticmq Wednesday 13 November 13 @adamwarski
- 16. Receive message walk-through // inner route: RequestContext => Unit ctx: RequestContext => val actorMsg = ReceiveMessages(visibilityTimeout, waitTimeSeconds) val msgs: Future[List[Message]] = queueActor ? actorMsg msgs.map { msgs => ctx.complete( <ReceiveMessageResponse> ... </ReceiveMessageResponse> ) } #DV13 #elasticmq Wednesday 13 November 13 @adamwarski
- 17. Receive message walk-through import akka.actor.{Actor, ActorRef} class QueueActor extends Actor { val messageQueue = mutable.PriorityQueue[InternalMessage]() val awaiting = mutable.PriorityQueue[ActorRef]() def receive = { case ReceiveMessages(...) => { // if there are messages, reply // otherwise put the sender aside // schedule a timeout in 20 seconds } } } #DV13 #elasticmq Wednesday 13 November 13 @adamwarski
- 18. Dataflow • Write async code as if it was sync! • Many Futures, if-s => trouble • Alternative: Scala Async #DV13 #elasticmq Wednesday 13 November 13 @adamwarski
- 19. Dataflow val result: Future[ActorRef] = flow { (queueManager ? Lookup(name)).apply() match { case Some(queueActor) => queueActor case None => val createFuture = queueManager ? Create(name) createFuture.apply() } } #DV13 #elasticmq Wednesday 13 November 13 @adamwarski
- 20. Links • http://github.com/adamw/elasticmq • http://akka.io/ • http://spray.io/ • http://warski.org #DV13 #elasticmq Wednesday 13 November 13 @adamwarski
- 21. There’s more! • “The ideal module system and the harsh reality” • Today, 17:50, Room 9 #DV13 #elasticmq Wednesday 13 November 13 @adamwarski
- 22. Thank you; Come & get a sticker http://codebrag.com/devoxx/ #DV13 #elasticmq Wednesday 13 November 13 @adamwarski