Java Enterprise Edition Concurrency Misconceptions

JAVA EE CONCURRENCY MISCONCEPTIONS
Haim Yadid, Performize-IT Ltd.
Daniel Pfeifer, Whistler AB

Daniel Pfeifer

‣ Founder of
Performize-IT Ltd.
‣ 18+ years in
technology
‣ Primary expertise in
performance optimization

‣ Founder of
Whistler AB
‣ 10+ years in IT
‣ Primary expertise in missioncritical business applications

WHO ARE THOSE GUYS?

Haim Yadid

JAVA EE IN THE MULTIPROCESSOR ERA

OPTIMISM

Moore’s Law:
The amount of transistors
appx. doubles every second
year

JAVA EE IN THE FUTURE

That’s why Multi core
hardware will become
mainstream for JEE servers

18"
16"
14"
12"
10"

CPUs/server"

8"

CPUs/phone"

6"

CPUs/notebook"

4"
2"
0"
2006"

2008"

2010"

2012"

IS MULTICORE NEW TO JAVA EE?

Budget machines

A FRIENDLY ADVICE

Don’t do it yourself !!

There is an App Server for that

TOOLS OF TRADE

‣ ”Local" and cluster-wide load balancing
‣ Message driven beans for parallelization / async tasks
‣ Asynchronous annotations
‣ Automatic and bean managed concurrency
‣ Resource management

You found yourself in need of sharing some data between calls so you
‣ Create a static constant pointing to a List (or perhaps your own
singleton)
‣ Add a synchronization block for your constant
‣ You operate on your instance inside the synchronize-block

SYNCHRONIZATION IN ENTERPRISE BEANS

Use Case

@Stateless
public class SharedStateEJB {
private static final List usedNumbers = new ArrayList();
public double fetchUnusedRandomNumber() {
double number = 0d;
synchronized (usedNumbers) {
while (true) {
number = Math.random();
if (!usedNumbers.contains(number)) {
usedNumbers.add(number);
break;
}
}
}
return number;
}
}


Returning unused random numbers

Sharing state like this will not work, because …
‣ Static constants are not shared among class loaders
‣ A lock will not is not propagated across machines


Problems


What about Java EE Singleton?

@Singleton
public class UsedNumbersBean {
private final List usedNumbers = new ArrayList();
@Lock(READ)
public boolean numberExists(double num) {
return usedNumbers.contains(num);
}
@Lock(WRITE)
public void addNumber(double num) {
usedNumbers.add(num);
}
}


@Singleton

“Backup” solutions
‣ Use the database and use locking => Potentially slow
‣ Use a cluster-aware cache (i.e. EHCache or JBoss Cache)
‣  Read the chapters on clusters and consistency!
‣ Terracotta
“Creative” solutions
‣ Use a single EJB
‣ Use H/A Singleton (available on JBoss)
‣ Wait for the future…


Suggestions

ASYNCHRONICITY THROUGH THREADS

•  You processed an order
•  You ask the bank for settlement confirmation
•  You can present the customer with a result without waiting for the
bank


Use Case

@Stateless
public class OrderHandlingBean {
public String process(final Order order) {
String orderId = ...//save order to database and get id
final Thread t = new Thread() {
@Override public void run() {
// submit amount to bank
}
};
t.start(); // start “background” process
return orderId;
}
}


Creating a thread for async tasks

‣ Enough stuck Threads will eventually make your server go OOM.
‣ Difficult to get transaction management right.
‣  Some servers will tell you it can’t check the status, some don’t.
‣  Failed tasks in the thread need difficult manual rollback.
‣ Threads do not scale out
‣  Can’t be load-balanced to other machines.
‣  Server’s automatic resource management doesn’t cover your own
threads.
‣ Debugging and monitoring isn’t the joy it used to be.


Problems

1.8
1.6

Response Time

1.4
1.2
1

java.lang.OutOfMemoryError: unable to create
new native thread
Async

0.8

Sync

0.6
0.4
0.2
0
1h

4h

7h

10h
13h
Runtime

16h

19h


Thread bugs gone wild

‣ Until Java EE 6 Message Driven Beans are used for asynchronous
tasks.
‣ And if you are lucky enough to use a Java EE 6 container, you can
use the new @Asynchronous.


Best Practice Alternatives

@Stateless(name = ”QueueingEJB")
public class QueueingBean {
@Resource private Queue q;
@Resource private ConnectionFactory cf;
// ... saving the Order to the database and fetching id
Connection cn = connectionFactory.createConnection();
Session s = cn.createSession(true, AUTO_ACKNOWLEDGE);
MessageProducer producer = s.createProducer(queue);
Message msg = s.createTextMessage(payment);
producer.send(msg);
return orderId;
}
}

Don’t forget clean-up!
We just want to
conserve screen estate!


Using Queues

@MessageDriven(name = “PayEJB”,
activationProperties = {
/* activation properties for your server */
})
public class PayBean implements MessageListener {
public void onMessage(Message m) {
// Calling bank...
}
}


Using Queues

@Stateless(name = ”PayEJB")
public class PayBean{
@Asynchronous
public void callBank(final String payment) {
// Hello, ... Bank!
}
}


Using @Asynchronous

@Stateless(name = ”AsyncCallerEJB")
public class AsyncCallerBean {
@EJB private PayBean payBean;
// ... saving the Order to the database and fetching id
payBean.callBank(payment);
return orderId; // <- Returns + transaction OK
}
}

That’s All!!!


Using @Asynchronous

We mentioned transactions…
‣ True, our examples won’t use the same Transaction, but…
‣  @Asynchronous will by spec create a Transaction
(REQUIRES_NEW)
‣  MDBs will also create a transaction


A note for the observant

Your back-office staff fires off a bunch of invoices and you want the
total as fast as possible, so you…
‣ Create a thread-pool
‣ Push all invoices for calculation to the thread-pool
‣ Wait for the answer and return it

PARALLELISM WITH THREADPOOLS

Use Case

private static class OrderCallable implements Callable<Double> {
private final Order order;
public OrderCallable(Order order) {
this.order = order;
}
@Override
public Double call() throws Exception {
// return the total based on the order that’s been
// provided.
}
}


Parallelizing with a ThreadPool (our Callable)

public double processOrders(List<Order> orders) {
ExecutorService executorService =
Executors.newFixedThreadPool(10);
List<Future<Double>> futures = new ArrayList<>();
for (Order order : orders) {
OrderCallable callable = new OrderCallable (order);
callables.add(callable);
futures.add(executorService.invoke (callable));
}
double totalValue = 0;
try {
for (Future<Double> future : futures) {
totalValue += future.get(10, SECONDS);
}
} catch (TimeoutException e) {
return -1;
}
return totalValue;
}


Parallelizing with a ThreadPool (using the Callable in an ExecutorService)

Essentially we got the same problems as previously, but…
‣ Potential for much larger thread leaks
‣ Still not covered by transaction
‣ Executors, just like Threads, won’t run tasks on other machines.
‣ There is nothing H/A about this.
‣  On crash, part or all of the workload is gone.
‣  Can’t be resumed on server restart, it’s just gone…


Problems

‣ Up to and including Java EE 5 you can use MDBs
‣  Queues can be clustered, so other servers can take some of the
load.
‣ @Asynchronous-annotated method returning Future


Pure Java EE alternatives

Using @Asynchronous (invokee)
@Stateless
@Asynchronous
public class ProcessBean {
public Future<Double> getTotals(Order order) {
double value = // .. Calculate value;
return new AsyncResult<Double>(value);
}
}

@Stateless public class OrderBean {
@EJB private ProcessBean processBean;
public double parallelizeCalculationAsync(List<Order> orders) {
List<Future<Double>> futures = new ArrayList<>();
for (Order order : orders) {
futures.add(processBean.getTotals(order));
}
try {
for (Future<Double> future : futures) {
totalValue += future.get(10, SECONDS);
}
} catch (Exception e) {
// ignore
}
return totalValue;
}
}


Using @Asynchronous (invoker)

@Stateless public class OrderBean {
@Resource(mappedName = “ConnectionFactory”) private ConnectionFactory cf;
@Resource(mappedName = “RequestQ”)
private Destination sendQueue;
@Resource(mappedName = “ResponseQ”)
private Destination responseQueue;
@TransactionAttribute(NOT_SUPPORTED) // or UserTransaction
public double handleOrders(List<Order> orders) {
try {
// …create a producer
for (Order o : orders) {
ObjectMessage outgoing = s.createObjectMessage();
outgoing.setObject(o);
outgoing.setLongProperty(“RequestID”, orders.hashCode())
mp.send(outgoing);
}
cn.start(); // must be started, otherwise we can't receive
MessageConsumer mc = s.createConsumer(responseQueue,
"JMSCorrelationID = '" + orders.hashCode() + "'");
for (int i = 0; i < orders.length(); i++)
totalValue += mc.receive(10, SECONDS).getDoubleProperty(“OrderValue”);
return totalValue;
} catch (JMSException e) {
throw new EJBException(e);
}
}
}

Send

Receive


Using MDB (invoker)

@MessageDriven(mappedName = “RequestQ”)
public class ProcessOrderBean implements MessageListener {
@Resource(mappedName = “ConnectionFactory")
private ConnectionFactory cf;
@Resource(mappedName = “ResponseQ")
private Queue responseQueue;
public void onMessage(Message message) {
double orderValue = // ... Process order
try {
MessageProducer mp = // … create message producer
Message msg = s.createMessage();
msg.setJMSCorrelationID(message.getIntProperty(“RequestID”));
msg.setDoubleProperty(”OrderValue”, orderValue);
mp.send(msg);
}
}
}


Using MDB (invokee)


A nice write up, but is it worth it?

12000
10000
8000
1 user

6000

10 users
20 users

4000
2000
0
Serial

8 parallel


Response time – Compute intensive Tasks

FORK AND KNIVES… UHM… JOIN

‣ New to Java 7 java.util.concurrent
‣ Helps you write a parallel recursive algorithm
‣ Break a long task to Fine grained tasks
‣ Very efficient for relatively short tasks

FORK JOIN

Introduction

ForkJoinTask

FORK JOIN

Introduction

ForkJoinPool

Worker
Worker
Worker

Worker

class PartialTask extends RecursiveAction {
protected void compute() {
PartialTask pt1 = new PartialTask(firstHalf);
PartialTask pt2 = new PartialTask(secondHalf);
pt1.fork(); // fork
pt2.exec(); // execute in new thread
pt1.join(); // wait for completion
}
}
fjPool = new ForkJoinPool(para);
Para.invoke(task)

FORK JOIN

Example

‣ Divide n points into k clusters
‣ Based on proximity

FORK JOIN

A Benchmark K-Means Clustering Algorithm

‣ This is not a formal benchmark
‣ I tried to do my best but it is not community criticized
‣ Ran on 8 core machine (no HT)
‣ Windows OS
‣ Warm-up was taken into account J
‣ 120K points to 50 clusters

FORK JOIN

Disclaimer

FORK JOIN

Response Time – by amount of parallelism
6000"

5000"

Time%(sec)%

4000"

3000"
Time"
2000"

1000"

0"
1"

2"

3"

4"

5"
#cores%used%

6"

7"

8"

FORK JOIN

Speedup– by amount of parallelism
6"

5"

Time%(sec)%

4"

3"
Speedup"
2"

1"

0"
1"

2"

3"

4"

5"
#cores%used%

6"

7"

8"

FORK JOIN

Single Operation Throughput

Op/sec'
1"
0.8"
0.6"
Op/sec"

0.4"
0.2"
0"
1"

2"

3"

4"

5"

6"

7"

8"

‣ Does your server need to do anything else ?
‣ Selfish Task Deplete all compute resources slows other task
‣ What about throughput

FORK JOIN

FJ on a JEE Server

FORK JOIN

What About Throughput ?
Lets check !

‣ What will be better

FORK JOIN

Throughput of 8 Concurrent Users

FORK JOIN

Throughput 8 concurrent users in parallel

Op/sec'
1.8"
1.6"
1.4"
1.2"
1"
0.8"
0.6"
0.4"
0.2"
0"

Op/sec"

Serial"Algorithm"

Fork"Join"
Parallelism"1"

Fork"Join"
Parallelism"8"

FORK JOIN

Response Time

9000"
8000"
7000"
6000"
5000"
4000"
3000"
2000"
1000"
0"

TRT"of"8"Users"
TRT"of"1"User"

Serial"
Algorithm"

Fork"Join"
Fork"Join"
Parallelism"1" Parallelism"8"

‣ Use FJ for the non common case
‣ Limit the number of cores do not starve other server tasks

FORK JOIN

Conclusion

LOAD MANAGEMENT FOR BACKEND ACCESS

You have a backend machine that can only handle 20 concurrent
requests and you have a cluster of four servers, so you…
‣ Create a semaphore with five permits.
‣ Acquire lock on semaphore
‣ Call server
‣ Release lock


Use Case

‣ The backend server can be under-utilized across the cluster.
‣  First server could be full of locks while second is free.
‣ One server can stall unnecessarily long time.
‣ Recompile necessary if some parameter changes.


Problems With This Approach

Using the Request-Reply Pattern (EIP), using:
‣ Clustered queues (request + response)
‣ A caller (stateless session bean)
‣ A backend invoker bean (message-driven bean)
‣ Configuration to limit in-process messages
For the advanced:
‣ Write a resource adapter (JCA)
‣  Support load balancing
‣  Support cluster


A Better Approach

@Stateless public class DispatcherBean {
@Resource(mappedName = “ConnectionFactory”) private ConnectionFactory cf;
@Resource(mappedName = “RequestQ”)
private Destination sendQueue;
@Resource(mappedName = “ResponseQ”)
private Destination responseQueue;
@TransactionAttribute(NOT_SUPPORTED) // Alternatively UserTransaction
public double checkStockRate(String stockName) {
try {
//… create producer
Message outgoing = s.createMessage();
outgoing.setStringProperty(”StockName", stockName);
mp.send(outgoing);

Send

String correlationId = outgoing.getJMSMessageID();
cn.start(); // must be started, otherwise we can't receive
MessageConsumer mc = s.createConsumer(responseQueue,
"JMSCorrelationID = '" + correlationId + "'");
Message incoming = mc.receive(10000L); // wait no more than 10 seconds
if (incoming != null) {
return incoming.getDoubleProperty(”StockPrice");
} else {
return Double.MIN_VALUE;
}
}
}
}

Receive


Request-Reply Pattern using clustered queues (Caller SLSB)

@MessageDriven(mappedName = “RequestQ”)
public class BackendInvokerBean implements MessageListener {
@Resource(mappedName = “ConnectionFactory")
private ConnectionFactory cf;
@Resource(mappedName = “ResponseQ")
private Queue responseQueue;
public void onMessage(Message message) {
double stockPrice = // ... Fetch from backend service
try {
// … create producer
Message msg = s.createMessage();
msg.setJMSCorrelationID(message.getJMSMessageID());
msg.setDoubleProperty(”StockPrice”, stockPrice);
mp.send(msg);
}
}
}


Request-Reply Pattern using clustered queues (Invoker MDB)

Vendor-specific, but in general:
‣ You may set an upper limit on consumers
‣  Glassfish has maxNumActiveConsumers
‣ You may set an upper limit of MDBs in Pool
‣  Works on all servers regardless of vendor (i.e. set max-poolsize to 5 in glassfish-ejb-jar.xml)


Configuration for in-process limitation


Isn’t MDB horribly slow?

12
10
8
1 user

6

5 users
10 users

4
2
0
Direct

MDB


Response time (in milliseconds)

You occasionally suffer from resource depletion so to protect your
server you:
‣ Create an EJB Interceptor that
‣  checks current beans in use (i.e. through counter or MBeans)
‣  throws an exception at chosen maximum size.
‣  Or wait until level go down
‣ Assign it to your EJB.

INTERFERING WITH BEAN POOL

Use Case

‣ Is there always equal load distribution across all tasks?
‣ Intelligent load balancers may continue to shoot at the wrong target
‣  It’s just an exception… not an overload
‣ DeadLock


Problems

Leave pool management to the server, pool sizes:
‣ Can be tuned on a global level
‣  Right strategy (infinite thread pools, incremental, fixed)
‣  Global default for min/max/timeout
‣ Can be set for each EJB
‣  XML
‣  Annotations (i.e. JBoss)


Better approach

It can be configured through the GUI…(Glassfish example)


Leave Management to the app server

<glassfish-ejb-jar>
<enterprise-beans>
<ejb>
<ejb-name>AnEJB</ejb-name>
<bean-pool>
<steady-pool-size>10</steady-pool-size>
<resize-quantity>10</resize-quantity>
<max-pool-size>100</max-pool-size>
<pool-idle-timeout-in-seconds>
60
</pool-idle-timeout-in-seconds>
</bean-pool>
</ejb>
</enterprise-beans>
</glassfish-ejb-jar>


Through Config Files

CONCLUSION

‣ Most mistakes stem from misunderstanding the app server, try to
understand your app server’s OOTB-capabilities…
‣ …and the Java EE specification.
‣ If you have a valid case for your own concurrency-related code,
make sure it plays well with the container.
‣ Your app server doesn’t always know best, but it often does. And if
not, you can often help it understand!
‣ Do not expect your environment to always look the same, it changes
quicker than you might think.

haim.yadid@gmail.com

Daniel Pfeifer
danpfe@gmail.com

CONTACT INFO

Haim Yadid

Java Enterprise Edition Concurrency Misconceptions

More Related Content

What's hot (20)

Similar to Java Enterprise Edition Concurrency Misconceptions (20)

More from Haim Yadid (20)

Recently uploaded (20)

Java Enterprise Edition Concurrency Misconceptions