Drd secr final1_3
Download as PPT, PDF0 likes513 views
This document summarizes a presentation on dynamic data race detection in concurrent Java programs. It discusses what data races are and why they are dangerous. It then covers different approaches to automatic race detection, including static and dynamic methods. The presentation focuses on explaining the happens-before race detection algorithm using vector clocks. It also describes the implementation of the presenters' own dynamic race detector, including how it works, problems they solved, and examples of races it detected in real applications.
![Data Race Example
public class Account {
private int amount = 0;
public void deposit(int x) {amount += x;}
public int getAmount() {return amount;}
}
public class TestRace {
public static void main (String[] args) {
final Account a = new Account();
Thread t1 = depositAccountInNewThread(a, 5);
Thread t2 = depositAccountInNewThread(a, 6);
t1.join();
t2.join();
System.out.println(account.getAmount()); //may print 5, 6, 11.
}
}](https://image.slidesharecdn.com/drdsecrfinal13-121108082651-phpapp01/85/Drd-secr-final1_3-3-320.jpg)
Drd secr final1_3
- 1. 8th Central and Eastern European Software Engineering Conference in Russia - CEE-SECR 2012 November 1 - 2, Moscow Dynamic data race detection in concurrent Java programs Vitaly Trifanov, Dmitry Tsitelov Devexperts LLC
- 2. Agenda What are data races and why they are dangerous Automatic races detection approaches, pros & cons Happens-before race detection algorithm Vector clocks Our dynamic race detector implementation solved problems
- 3. Data Race Example public class Account { private int amount = 0; public void deposit(int x) {amount += x;} public int getAmount() {return amount;} } public class TestRace { public static void main (String[] args) { final Account a = new Account(); Thread t1 = depositAccountInNewThread(a, 5); Thread t2 = depositAccountInNewThread(a, 6); t1.join(); t2.join(); System.out.println(account.getAmount()); //may print 5, 6, 11. } }
- 6. Data Races Data race occurs when many threads access the same shared data concurrently; at least one writes Usually it’s a bug
- 7. Data Races Are Dangerous Hard to detect if occurred no immediate effects program continues to work damage global data structures Hard to find manually Not reproducible - depends on threads timing Dev & QA platforms are not so multicore
- 8. Automatic Race Detection 20+ years of research Static analyze program code offline data races prevention (extend type system, annotations) Dynamic: analyze real program executions On-the-fly Post-mortem
- 9. Dynamic Detectors vs Static
- 10. Static Approach Pros Doesn’t require program execution Analyzes all code Doesn’t depend on program input, environment, etc. Cons Unsolvable in common case Has to reduce depth of analysis A lot of existing tools for Java FindBugs, jChord, etc
- 11. Dynamic Approach Pros Complete information about program flow Lower level of false alarms Cons Very large overhead No existing stable dynamic detectors for Java
- 12. Static vs Dynamic: What To Do? Use both approaches Static (FindBugs/Sonar, jChord, …) Eliminate provable synchronization inconsistencies on the early stage Dynamic Try existing tools, but they are unstable IBM MSDK, Thread Sanitizer for Java That’s why we’ve developed our own!
- 13. Data Race Detector Concept Application uses libraries and frameworks via API At least JRE API is well documented “Class XXX is thread-safe” “Class YYY is not thread-safe” “XXX.get() is synchronized with preceding call of XXX.set()” Describe behavior of API and exclude library from analysis
- 14. DRD: How It’s Organized
- 15. What Operations to Intercept? Synchronization operations thread start/join/interrupt synchronized volatile read/write java.util.concurrent Accesses to shared data fields objects
- 16. How It Works Instrumented Application DRD agent app classes classes interceptor Race Config detection module
- 17. JLS: Publishing Data Publish changes Receive changes
- 18. JLS: Synchronized-With Relation “Synchronized-with” relation unlock monitor M ↦ all subsequent locks on M volatile write ↦ all subsequent volatile reads … Notation: send ↦ receive
- 19. JLS: Happens-Before & Data Races X happens-before Y, when X, Y - in same thread, X before Y in program order X is synchronized-with Y Transitivity: exists Z: hb(X, Z) && hb(Z, Y) Data race: 2 conflicting accesses, not ordered by happens-before relation
- 20. Happens-Before Example No data race
- 21. Vector Clock
- 22. Vector Clock
- 23. Vector Clock
- 24. Vector Clock
- 25. Vector Clock
- 26. Vector Clock
- 27. Vector Clock
- 28. Vector Clock
- 29. Vector Clock Not ordered! A: 3 > 2 B: 3 < 4
- 30. How It Works. No Data Race Example
- 31. How It Works. Data Race Example
- 32. Code Instrumentation Check everything => huge overhead Race detection scope Accesses to our fields Foreign calls (treat them as read or write) Sync scope Detect sync events in our code Describe contracts of excluded classes Treat these contracts as synchronization events
- 33. Race Detection private class Storage { private Map<Integer, Item> items = new HashMap<Integer, Item> (); public void store(Item item) { items.put(item.getId(), item); } public void saveToDisk() { On each access of “items” field we check On each access of “items” field we check for (Item item : items.values()) { //serialize and save race on this field race on this field saveItem(item); //... } } On each call of “items” method we check On each call of “items” method we check public Item getItem(int id) { race on this object race on this object return items.get(id); } public void reload() { items = deserealizeFromFile(); Each field of class Item is protected the Each field of class Item is protected the } same way as field “items” of class Storage same way as field “items” of class Storage }
- 35. Clocks Storing Thread clock ThreadLocal<VectorClock > Field XXX volatile transient VectorClock XXX_vc; Foreign objects, monitors WeakIdentityConcurrentHashMap<Object,VectorClock> Volatiles, synchronization contracts ConcurrentHashMap <???, VectorClock>
- 36. Composite Keys AtomicLongFieldUpdater.CAS(Object o, long offset, long v) param 0 + param 1 Volatile field “abc” of object o object + field name AtomicInteger.set() & AtomicInteger.get() object ConcurrentMap.put(key, value) & ConcurrentMap.get(key) object + param 0
- 37. Solved Problems Composite keys for contracts and volatiles Generate them on-the-fly Avoid unnecessary keys creation ThreadLocal<MutableKeyXXX> for each CompositeKeyXXX Loading of classes, generated on-the-fly Instrument ClassLoader.loadClass()
- 38. Solved Problems Don’t break serialization compute serialVersiodUid before instrumentation Caching components of dead clocks when thread dies, its time frames doesn’t grow anymore cache frames of dead threads to avoid memory leaks local last-known generation & global generation
- 39. DRD in Real Life: QD ✔ 6 races found QD QD + DRD
- 40. DRD in Real Life: MARS UI MARS ✔ 5 races found MARS + DRD
- 41. DRD Race Report Example WRITE_READ data race between current thread Thread-12(id = 33) and thread Thread-11(id = 32) Race target : field my/app/DataServiceImpl.stopped Thread 32 accessed it in my/app/DataServiceImpl.access$400(line : 29) --------------------------------Stack trace for racing thread (id = 32) is not available.------------ -------------------------------------Current thread's stack trace (id = 33) : --------------------------- at my.app.DataServiceImpl.stop(DataServiceImpl.java:155) at my.app.DataManager.close(DataManager.java:201) ...
- 42. DRD Advantages Doesn’t break serialization No memory leaks Few garbage No JVM modification Synchronization contracts very important: Unsafe, AbstractQueuedSynchronizer
- 43. Links http://code.devexperts.com/display/DRD/: documentation, links, etc Contact us: drd-support@devexperts.com Useful links: see also on product page IBM MSDK ThreadSanitizer for Java jChord FindBugs JLS «Threads and locks» chapter
- 44. Q&A
Editor's Notes
- #7: Hard to find at development/review stage Control of program execution flow is frequently delegated to frameworks Hard to detect manually during testing
- #14: safe/read/write contracts synchronization contracts
- #53: ThreadLocal<MutableInteger> : increment my counter, push it ’ s value to global AtiomicInteger each N-th hit. When thread dies, increment global generation. Each clock stores last known generation. When it becomes less than global, check who have died and cache corresponding frames.
- #61: Module testing – check module having contracts of others Lightweight mode : instrument little, most important parts of code Support for synchronization on array cells