Android Crash analysis and The Dalvik Garbage collector – Tools and Tips
17 likes13,849 views
The document discusses methods for analyzing Android app crashes and the daily operations of the Dalvik garbage collector, emphasizing the need for root cause analysis and effective debugging techniques. It highlights the importance of addressing user-reported issues and includes practical tips to manage memory and minimize garbage collection delays. Resources and tools for performance testing and analysis are also provided.
Android Crash analysis and The Dalvik Garbage collector – Tools and Tips
- 1. Android Crash Analysis and The Dalvik Garbage Collector - Tips Oren BarAd Mobile Security Team Leader AVG
- 3. 3 Intro • App on the market • Crashes will happen • Need to Fix the Bugs! • Identify the root issues
- 4. 4 Who? • Developing for Android since 2008 • The first employee of DroidSecurity – First company to release An antivirus to the Android market (2009) – Later acquired by AVG (2011) • Mobile Security Research Team Leader @AVG • B.Sc Information System Engineering from BGU
- 5. 5 Agenda • Android app crash Collection & Analysis • The Android Garbage Collector • Memory analysis Tools • Tips and tricks
- 6. 6 How to collect crash information?
- 7. 7 What to fix first? • Easy to fix • Affects most users • The ones users complain most about • Affects the most popular device
- 8. 8 Root cause analysis • Stare at data: –Crash type –Exception Type –Device –Manufacturer –OS version –Locale • And stare some more…
- 9. 9 Ultimate debugging method! • What is the answer to this bug? – 42 • We don’t know the answer – Brainstorming – Developing tools • Learning something new…
- 10. 10 Root cause analysis – Real world example • Concurrent.Timeout exception – ~10% of total crashes for AVG antivirus. – Unhelpful stack traces – Second place in top 10 crashes (and number 5, and number 9) java.util.concurrent.TimeoutException: android.os.BinderProxy.finalize() timed out after 10 seconds at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:401) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:102) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:73) at java.lang.Daemons$FinalizerDaemon.run(Daemons.java:170) at java.lang.Thread.run(Thread.java:841) java.util.concurrent.TimeoutException: com.android.internal.os.BinderInternal$GcWatcher.finalize() timed out after 10 seconds java.util.concurrent.TimeoutException: android.os.Parcel.finalize() timed out after 10 seconds java.util.concurrent.TimeoutException: java.io.FileInputStream.finalize() timed out after 10 seconds
- 11. 11 Device and OS analysis 33% 66% Device Galaxy S3 49% 38% 13% 19% 10% 71% OS Crash Dashboard 4.2.2 4.3 Other OS Versions
- 12. 12 What next? • After going deep into the Code… • Get S3 and use
- 13. 13 Android GC – little bit of history Gingerbread
- 14. 14 Complex Object Tricolor collector at work roots rootsroots object object object object object object object object object object
- 15. 15 object Tricolor collector at work roots rootsroots object object object object object object object object object object
- 16. 16 object Tricolor collector at work roots rootsroots object object object object object object object object object object
- 17. 17 object Tricolor collector at work roots rootsroots object object object object object object object object object object
- 18. 18 Complex Object Tricolor collector at work roots rootsroots object object object object object object object object object object
- 19. 19 Complex Object Tricolor collector at work roots rootsroots object object object object object object object object object object
- 20. 20 Garbage collector observed behavior Cause Effect GC_FOR_MALLOC Stop The world GC Use complex Objects GC takes longer A lot of small objects GC take Longer Use more memory GC takes longer USE_LARGE_HEAP All of the above
- 21. 21 Trying to replicate the problem • The GC_Test project – Test the GC performance under load – Android app to test the behavior of the GC on android – https://github.com/oba2cat3/GCTest • Useful logcat python script – https://github.com/oba2cat3/logcat2memorygraph • Available from GitHub
- 22. 22 Logcat output 02-04 12:40:33.185: D/dalvikvm(2496): GC_EXPLICIT freed <1K, 19% free 14383K/17720K, paused 1ms+1ms, total 17ms 02-04 12:40:33.265: D/dalvikvm(3279): GC_CONCURRENT freed 661K, 27% free 14934K/20336K, paused 5ms+3ms, total 21ms 02-04 12:40:33.460: D/dalvikvm(5637): GC_CONCURRENT freed 834K, 22% free 15516K/19644K, paused 6ms+3ms, total 33ms 02-04 12:40:33.480: D/dalvikvm(3279): GC_CONCURRENT freed 313K, 27% free 15022K/20336K, paused 3ms+9ms, total 30ms 02-04 12:40:33.665: D/dalvikvm(5637): GC_CONCURRENT freed 369K, 21% free 15658K/19644K, paused 2ms+2ms, total 18ms 02-04 12:40:33.725: D/dalvikvm(5730): GC_CONCURRENT freed 1166K, 23% free 15188K/19644K, paused 4ms+2ms, total 25ms 02-04 12:40:33.765: D/dalvikvm(5637): GC_CONCURRENT freed 430K, 20% free 15740K/19644K, paused 7ms+2ms, total 22ms 02-04 12:40:33.865: D/dalvikvm(5730): GC_CONCURRENT freed 325K, 23% free 15256K/19644K, paused 2ms+3ms, total 22ms 02-04 12:40:34.065: D/dalvikvm(5637): GC_CONCURRENT freed 1811K, 25% free 15934K/21040K, paused 3ms+2ms, total 29ms 02-04 12:40:34.160: D/dalvikvm(5637): GC_CONCURRENT freed 99K, 23% free 16354K/21040K, paused 2ms+4ms, total 31ms 02-04 12:40:34.250: D/dalvikvm(3403): GC_EXPLICIT freed 172K, 24% free 14956K/19628K, paused 3ms+4ms, total 28ms 02-04 12:40:34.315: D/dalvikvm(5766): GC_CONCURRENT freed 1185K, 23% free 15148K/19628K, paused 3ms+3ms, total 28ms 02-04 12:40:34.350: D/dalvikvm(5637): GC_CONCURRENT freed 1140K, 26% free 15766K/21040K, paused 3ms+3ms, total 30ms 02-04 12:40:34.495: D/dalvikvm(4155): GC_CONCURRENT freed 1274K, 24% free 15093K/19660K, paused 7ms+5ms, total 56ms 02-04 12:40:34.635: D/dalvikvm(3333): GC_CONCURRENT freed 574K, 24% free 14982K/19620K, paused 3ms+2ms, total 19ms 02-04 12:40:34.780: D/dalvikvm(5694): GC_CONCURRENT freed 1268K, 24% free 15100K/19656K, paused 3ms+1ms, total 25ms 02-04 12:40:34.840: D/dalvikvm(2960): GC_EXPLICIT freed 1236K, 10% free 53524K/58860K, paused 4ms+7ms, total 130ms 02-04 12:40:34.915: D/dalvikvm(5637): GC_FOR_ALLOC freed 143K, 25% free 15978K/21040K, paused 26ms, total 26ms 02-04 12:40:33.480: D/dalvikvm(3279): GC_CONCURRENT freed 313K, 27% free 15022K/20336K, paused 3ms+9ms, total 30ms 02-04 12:40:33.665: D/dalvikvm(5637): GC_CONCURRENT freed 369K, 21% free 15658K/19644K, paused 2ms+2ms, total 18ms 02-04 12:40:33.725: D/dalvikvm(5730): GC_CONCURRENT freed 1166K, 23% free 15188K/19644K, paused 4ms+2ms, total 25ms 02-04 12:40:33.765: D/dalvikvm(5637): GC_CONCURRENT freed 430K, 20% free 15740K/19644K, paused 7ms+2ms, total 22ms 02-04 12:40:33.865: D/dalvikvm(5730): GC_CONCURRENT freed 325K, 23% free 15256K/19644K, paused 2ms+3ms, total 22ms 02-04 12:40:34.065: D/dalvikvm(5637): GC_CONCURRENT freed 1811K, 25% free 15934K/21040K, paused 3ms+2ms, total 29ms
- 23. 23 Script result - Heap size and GC delays 0 0.2 0.4 0.6 0.8 1 1.2 0 20000 40000 60000 80000 100000 120000 140000 160000 Time in MSheap size in KB dalvikvm event timestamps current heap max heap
- 24. 24 Script result - Heap size and GC delays 0 50 100 150 200 250 300 350 400 0 20000 40000 60000 80000 100000 120000 140000 160000 Time in MSheap size in KB dalvikvm event timestamps current heap max heap ui pause time total pause
- 25. 25 Script result - Heap size and GC delays 0 50 100 150 200 250 300 350 400 0 20000 40000 60000 80000 100000 120000 140000 160000 Time in MSheap size in KB dalvikvm event timestamps current heap max heap ui pause time total pause webview start
- 26. 26 Memory Analyzer Tool tricks
- 27. 27 Memory Analyzer Tool tricks
- 28. 28 Memory Analyzer Tool tricks
- 29. 29 Memory Analyzer Tool tricks
- 30. 30 Memory Analyzer Tool tricks
- 31. 31 Memory Analyzer Tool tricks
- 32. 32 Memory Analyzer Tool tricks
- 33. 33 Memory Analyzer Tool tricks
- 34. 34 Tips for managing GC Do How Avoid GC_FOR_ALLOC Load stuff in Splash Use Object Pools For large/heavy objects Avoid Complex Objects Use simpler Objects Perform Heap analysis Use MAT and DDMS Reduce memory use …
- 35. 35 Summary • Application will crash –Collect and Analyze –Understand and Fix • Fixing the bug is half the fun
- 36. 36 Back to the problem • JNI objects are a special case • Dalvik will finalize and Destroy • Timeouts will happen... And they Do!
- 37. 37 Q&A • Questions? • Contact: Oren.barad@avg.com • We are hiring! • Come talk to us in the AVG booth
- 38. 38 Links and resources • http://android-developers.blogspot.co.il/2011/03/memory- analysis-for-android.html • http://www.brpreiss.com/books/opus5/html/page424.html • http://java-is-the-new-c.blogspot.co.il/2013/07/tuning-and- benchmarking-java-7s-garbage.html • http://www.slideshare.net/muhammedshakirmisarwala/java- performance-monitoring-tuning • http://www.slideshare.net/VikasBalikai/gc-in-android • http://www.javacodegeeks.com/2012/01/practical-garbage- collection-part-1.html
- 39. 39 Links and resources • https://android.googlesource.com/platform/dalvik/+/android- 4.3_r2/vm/alloc/MarkSweep.cpp • https://android.googlesource.com/platform/dalvik/+log/android- 4.4.2_r2/vm/alloc/MarkSweep.cpp • https://android.googlesource.com/platform/dalvik/+/gingerbread/vm/alloc/MarkSwe ep.c • https://android.googlesource.com/platform/dalvik/+/froyo- release/vm/alloc/MarkSweep.c