Testing multithreaded java applications for synchronization problems
4 likes4,419 views
This document discusses techniques for testing multithreaded Java applications for synchronization problems. It begins by explaining why multithreading is used and the new types of defects that can arise, such as deadlocks, livelocks, and race conditions. It then covers various testing techniques like load/stress testing, testing specific thread interleavings, testing all possible interleavings, and using instrumentation. It provides examples of tools that can be used for each technique and recommends books on the topic.
![IMUnit
• Events based
– IMUnit.fireEvent(“event1”)
– @Schedule(“event1 -> event2”)
– @Schedule(“[event1] -> event2”)](https://image.slidesharecdn.com/testingmultithreadedjavaapplicationsforsynchronizationproblems-121025141831-phpapp01/85/Testing-multithreaded-java-applications-for-synchronization-problems-25-320.jpg)
Testing multithreaded java applications for synchronization problems
- 1. Testing Multithreaded Java Applications for Synchronization Problems Vassil Popovski vpopovski@vmware.com www.vmware.com
- 2. Why multithreading? core 1 core N … cache cache shared memory A modern CPU
- 3. Why multithreading? Thread 1 thread … Thread N thread synchronization Java heap A modern Java Application
- 4. Chuck Norris can write multi-threaded Java applications with a single thread Image source: http://4.bp.blogspot.com/-RhqCAgEyhxs/T7CqtsC0RUI/AAAAAAAAJuw/LnkKl4xBcFA/s1600/ChuckNorris.png
- 5. New concept -> new problems Multithreading leads to: • Non-determinism • New types of defects exist such as deadlocks, livelocks, thread contentions and race conditions
- 6. Threads interleaving Thread 1 Thread 2 T1: Block A T2: Block 1 T2: Block 2 T1: Block B
- 7. Threads interleaving Thread 1 Thread 2 T2: Block 1 T1: Block A T2: Block 2 T1: Block B
- 8. T1: Block A T1: Block A T1: Block A T1: Block B T2: Block 1 T2: Block 1 T2: Block 1 T1: Block B T2: Block 2 T2: Block 2 T2: Block 2 T1: Block B T2: Block 1 T2: Block 1 T2: Block 1 T2: Block 2 T1: Block A T1: Block A T1: Block A T2: Block 2 T1: Block B T1: Block B T1: Block B T2: Block 2
- 9. Number of different thread interleavings
- 10. QUIZ Q: What is the number of all interleavings for 3 threads with 3 blocks each?
- 11. QUIZ#2 Q: What is the number of all interleavings for 100 threads with 5 blocks each? A: 65425664332523564432322 424953034168268979855850496989515584307752298932760805211068470651079402188475376110921674489441757 100735903402821009297053297794211809385509051615356527155445783425009164096533350469391547252484286 352170768347268799054153532042868164413927778850486516865333071658697094887692175873537666079346686 842361469250698011080482809500911511130491996847437245286443821625843666165114941515090783620437429 415942216292191680146737140994107899459749143795942309665194029514713124663377783172923345131361940 229192120804703412493451507233690788801188111838709132087076627314402580265097945582570927195669521 598427700259644897641650396932627145510414432349054625767654125966166480949639718802313608310300009 569524993138782412735779520574624492099944487052157575950286282799011493381356627940388478077680384 306582854732555208087863968812536837179492786875593176275633769413252546329077858090833823115446889 102306231699826529926420707529253684105656295906933999334649790703077726680609672178258148765366567 390695607298478918976308327339329621077231101219394256694201487015134200730008353085104356877584247 630194547240102504944587886157078430944582189784261271608483330872723017090959069769287306115617186 352177030505501663380710718097318392846079061230267394203670163609062726137904455998424717036312138 754515697607350323258954538452071707071282635719375409459015826288484347553786552736926086009976292 360046325376069078312201822678281992756614855571919537160977811910388279229176987762264097262002703 425073318544671121015763363797279655888433399855420190892643324103098493700405493206195043772398231 236266443176076877706895863803891037350530509973799077521555018278731597362632766739808771841037553 742905870882213799310132964516518399560481797842812543528613844661728630365645349106177833375046753 439848225204235068908502565168578778645210878189710308017886705257632553314025472000000000000000000 0000000000000000000000000000000
- 12. What to test for? • safety: nothing bad happens • liveness: something good eventually happens
- 13. Example 1: Single Lane Bridge Image source: http://www.doc.ic.ac.uk/~jnm/book/ppt/ch7.ppt • safety: – no car crash • liveness: – every car eventually get an opportunity to cross the bridge
- 14. Example 2: BoundedBuffer or Producer/Consumer Producer 1 Consumer 1 Producer 2 Consumer 2 Producer 3 … … Consumer M Producer N • safety: – head and tail of the queue must not over run each other • liveness: – when the queue contains an item, the consumer process must be able to access the queue and when the queue contains space for another item, the producer process must be able to access the queue
- 15. How to test for synchronization issues? • Load/Stress testing (blackbox, whitebox) • Specific interleavings testing (whitebox) • All interleavings testing (whitebox) • Instrumentation (blackbox, whitebox)
- 16. Load/Stress testing • A lot of threads and operations to exercise different interleavings
- 17. Demo!
- 18. Deadlock example Thread 1 Thread 2 synchronized ( A ) { synchronized ( B ) { synchronized ( B ) { synchronized ( A ) { Deadlock !!!
- 19. Load/Stress testing • Tools – ExecutorService - http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/Execu – TestNG - http://testng.org/doc/documentation-main.html#annotations – JUnit - http://www.junit.org/apidocs/junit/extensions/ActiveTestSuite.html • JUnitPerf - http://testng.org/doc/documentation-main.html#annotations – GroboUtils - http://groboutils.sourceforge.net/testing-junit/using_mtt.html – Custom Threading
- 20. Specific interleavings testing • Deterministic and repeatable tests
- 21. MultithreadedTC • Based on internal clock – Tick increases when all threads are blocked – waitForTick(tick) – blocks the thread until tick reaches certain value – assertTick(tick) – asserts the current tick • Methods that start with “thread” are executed in separate threads
- 22. MultithreadedTC BoundedBuffer with size = 1 Thread 1 Thread 2 put 42 put 17 (blocks) get 42 get 17
- 23. MultithreadedTC BoundedBuffer with size = 1 Thread 1 Thread 2 Tick 0 put 42 waitForTick(1) put 17 Tick 1 (blocks) get 42 assertForTick(1) get 17
- 24. Demo!
- 25. IMUnit • Events based – IMUnit.fireEvent(“event1”) – @Schedule(“event1 -> event2”) – @Schedule(“[event1] -> event2”)
- 26. Demo!
- 27. ThreadWeaver • Two threads only – Main and Secondary • By default: For each line of Main thread (T1) -> block and execute fully Secondary thread (T2) T2: Block 1 T1: Block A T1: Block A T2: Block 2 T2: Block 1 T1: Block B T1: Block A T2: Block 2 T2: Block 1 T1: Block B T1: Block B T2: Block 2
- 28. ThreadWeaver • Two threads only – Main and Secondary • By default: For each line of Main thread (T1) -> block and execute fully Secondary thread (T2) • Powerful Breakpoints
- 29. Demo!
- 30. Specific interleavings testing • Tools – MultithreadedTC - http://code.google.com/p/multithreadedtc/ • Enhanced version for Junit 4 - http://code.google.com/p/multithreadedtc-junit4/ – IMUnit - http://mir.cs.illinois.edu/imunit/ – ThreadWeaver - http://code.google.com/p/thread-weaver/ – Awaitility - http://code.google.com/p/awaitility/
- 31. All interleavings testing • Exercise all possible interleavings
- 32. JavaPathFinder Traditional testing OK Code Testing error Image source: http://javapathfinder.sourceforge.net/events/JPF-workshop-050108/tutorial.ppt
- 33. JavaPathFinder Model Checking with JavaPathFinder Code OK Model Checking error trace properties Line 5: … Line 12: … … Line 41:… Line 47:… Image source: http://javapathfinder.sourceforge.net/events/JPF-workshop-050108/tutorial.ppt
- 34. JavaPathFinder Program testing can be used to show the presence of bugs, but never to show their absence! --Edsger Dijkstra
- 35. Demo!
- 36. All interleavings testing • Tools – JavaPathFinder (JPF) - http://babelfish.arc.nasa.gov/trac/jpf • JavaRaceFinder - http://www.cise.ufl.edu/research/JavaRacefinder/Java_Rac eFinder/JRF_Home.html
- 37. Instrumentation • Instrument the code to catch problems easier
- 38. AspectJ AspectJ Compiler Original code Instrumented . .class or .jar class or .jar Aspect Definition .aj
- 39. Demo!
- 40. Instrumentation • Tools – AspectJ - www.eclipse.org/aspectj/ • RacerAJ - http://www.bodden.de/tools/raceraj/ – CalFuzzer - http://srl.cs.berkeley.edu/~ksen/calfuzzer/ – (commercial) Flashlight / Jsure - http://www.surelogic.com/concurrency-tools.html
- 41. Other tools • CHESS (native DLLs & managed executables) - http://research.microsoft.com/en- us/projects/chess/ • MoonWalker (.NET)- http://code.google.com/p/moonwalker/
- 43. Thank you!
- 44. Q&A
- 45. Source code packages (1) • org.java2days2012.multithreaded.common – Several implementations of AccountManager, Counter, BounderBuffer and MultipleReadersSingleWriter • org.java2days2012.multithreaded.executorservice – Load/stress tests using ExecutorService • org.java2days2012.multithreaded.testng – Load/stress tests using TestNG • org.java2days2012.multithreaded.mtc – MultithreadedTC tests • org.java2days2012.multithreaded.imunit – IMUnit tests • org.java2days2012.multithreaded.threadweaver – ThreadWeaver tests
- 46. Source code packages (2) • org.java2days2012.multithreaded.javapathfinder – JavaPathFinder examples • ca.mcgill.sable.racer – RacerAJ source code • org.java2days2012.multithreaded.aspectj – Custom aspect that increase the chance of hitting multithreading problem during testing • org.java2days2012.multithreaded.blockbox.* – Rest based AccountManager client/test and server (in .server package)
Editor's Notes
- #7: Block is atomic piece of code that is not affected by multithreading. This is usually a block that starts or ends with accessing a shared variable Blocks can overlap
- #11: Answer = 1680
- #12: Answer = 1680
- #19: NASA’s spacecraft fly to Mars (Deep Space 2) – deadlock due to missing critical section, almost failed the whole mission. Remote Agent in AI for controlling the spacecraft without any human interactions. It was great that the control center from Earth took control and restarted the module. Therac-25, massive radiation overdose due to data race, 5 deaths, more injured
- #22: ExecutorService/TestCounter ExecutorService/TestAccountManager
- #26: Created at the University of Maryland by Bill Pugh
- #27: Created at the University of Maryland by Bill Pugh
- #28: Created at the University of Maryland by Bill Pugh
- #29: mtc / TestInterleavings mtc / TestBounderBufferLocking mtc / TestAccountManagerForDeadlock mtc / TestReadersWriterLiveness
- #30: Imunit / TestInterleavings - Show also block event – [] brackets for 12AB schedule
- #34: Threadweaver / TestInterleaving Talk about @ThreadedBefore, @ThreadedAfter, @ThreadedMain and @ThreadedSecondary Threadweaver / TestAccountManager - Show the deadlock