Large Scale Studies: Malware Needles in a Haystack
0 likes60 views
This document presents a large-scale study of malware analysis, comparing coarse-grained and fine-grained methodologies to unveil various malicious behaviors and characteristics. It details the dataset used, the processing architecture, and the results of both analysis types, highlighting distinct behaviors, signatures, and techniques employed by malware. The conclusion emphasizes the importance of combining both analytical approaches for a comprehensive understanding of malware threats.
Large Scale Studies: Malware Needles in a Haystack
- 1. Introduction Motivation & Related Work Methodology Analysis Results Discussion Final Remarks Large Scale Studies: Malware Needles in a Haystack Giovanni Bert˜ao1,3, Marcus Botacin2, Andr´e Gr´egio2, Paulo L´ıcio de Geus1 1University of Campinas (UNICAMP) {bertao, paulo}@lasca.ic.unicamp.br 2Federal University of Parana (UFPR) {mfbotacin, gregio}@inf.ufpr.br 3Bolsista PIBIC-CNPq Large Scale Studies: Malware Needles in a Haystack SBSeg’18
- 2. Introduction Motivation & Related Work Methodology Analysis Results Discussion Final Remarks Topics 1 Introduction 2 Motivation & Related Work 3 Methodology 4 Analysis Results 5 Discussion 6 Final Remarks Large Scale Studies: Malware Needles in a Haystack SBSeg’18
- 3. Introduction Motivation & Related Work Methodology Analysis Results Discussion Final Remarks Analysis Type: Coarse-grained and Fine-grained Topics 1 Introduction Analysis Type: Coarse-grained and Fine-grained 2 Motivation & Related Work Motivation Related Work 3 Methodology Dataset Processing the Data Architecture 4 Analysis Results Coarse-Grained Fine-Grained 5 Discussion Coarse-Grained Fine-Grained 6 Final Remarks Conclusion Acknowledgments Questions Large Scale Studies: Malware Needles in a Haystack SBSeg’18
- 4. Introduction Motivation & Related Work Methodology Analysis Results Discussion Final Remarks Analysis Type: Coarse-grained and Fine-grained Malware Analysis Approaches Coarse-Grained Highlight major aspects. Discard sample details. Fine-Grained Focus on implementation details. Don’t state the risk of such sample in the overall scenario. Large Scale Studies: Malware Needles in a Haystack SBSeg’18
- 5. Introduction Motivation & Related Work Methodology Analysis Results Discussion Final Remarks Motivation Topics 1 Introduction Analysis Type: Coarse-grained and Fine-grained 2 Motivation & Related Work Motivation Related Work 3 Methodology Dataset Processing the Data Architecture 4 Analysis Results Coarse-Grained Fine-Grained 5 Discussion Coarse-Grained Fine-Grained 6 Final Remarks Conclusion Acknowledgments Questions Large Scale Studies: Malware Needles in a Haystack SBSeg’18
- 6. Introduction Motivation & Related Work Methodology Analysis Results Discussion Final Remarks Motivation Malware Studies Figure: Coarse-Grained BBC: https://www.bbc.com/news/technology-39730407 Figure: Fine-Grained Trend Micro: https://bit.ly/2PaSPDC Large Scale Studies: Malware Needles in a Haystack SBSeg’18
- 7. Introduction Motivation & Related Work Methodology Analysis Results Discussion Final Remarks Related Work Topics 1 Introduction Analysis Type: Coarse-grained and Fine-grained 2 Motivation & Related Work Motivation Related Work 3 Methodology Dataset Processing the Data Architecture 4 Analysis Results Coarse-Grained Fine-Grained 5 Discussion Coarse-Grained Fine-Grained 6 Final Remarks Conclusion Acknowledgments Questions Large Scale Studies: Malware Needles in a Haystack SBSeg’18
- 8. Introduction Motivation & Related Work Methodology Analysis Results Discussion Final Remarks Related Work Related Work Bayer (Windows) A view on current malware behaviors. Lindorfer (Android) Andrubis – 1,000,000 apps later: A view on current android malware behaviors. Large Scale Studies: Malware Needles in a Haystack SBSeg’18
- 9. Introduction Motivation & Related Work Methodology Analysis Results Discussion Final Remarks Dataset Topics 1 Introduction Analysis Type: Coarse-grained and Fine-grained 2 Motivation & Related Work Motivation Related Work 3 Methodology Dataset Processing the Data Architecture 4 Analysis Results Coarse-Grained Fine-Grained 5 Discussion Coarse-Grained Fine-Grained 6 Final Remarks Conclusion Acknowledgments Questions Large Scale Studies: Malware Needles in a Haystack SBSeg’18
- 10. Introduction Motivation & Related Work Methodology Analysis Results Discussion Final Remarks Dataset Building the Dataset Dataset Composition Malware repositories and blacklists crawled daily. 135,000 unique malware samples collected from Malshare database. Only Windows samples. Samples submitted to static, dynamic and network analysis procedures. Large Scale Studies: Malware Needles in a Haystack SBSeg’18
- 11. Introduction Motivation & Related Work Methodology Analysis Results Discussion Final Remarks Processing the Data Topics 1 Introduction Analysis Type: Coarse-grained and Fine-grained 2 Motivation & Related Work Motivation Related Work 3 Methodology Dataset Processing the Data Architecture 4 Analysis Results Coarse-Grained Fine-Grained 5 Discussion Coarse-Grained Fine-Grained 6 Final Remarks Conclusion Acknowledgments Questions Large Scale Studies: Malware Needles in a Haystack SBSeg’18
- 12. Introduction Motivation & Related Work Methodology Analysis Results Discussion Final Remarks Processing the Data Analysis Method Static Analysis Presence of packers. Anti-analysis techniques. Anti-virus detection. Dynamic and Network Analysis Logs from BehEMOT, an internal sandbox solution. Large Scale Studies: Malware Needles in a Haystack SBSeg’18
- 13. Introduction Motivation & Related Work Methodology Analysis Results Discussion Final Remarks Architecture Topics 1 Introduction Analysis Type: Coarse-grained and Fine-grained 2 Motivation & Related Work Motivation Related Work 3 Methodology Dataset Processing the Data Architecture 4 Analysis Results Coarse-Grained Fine-Grained 5 Discussion Coarse-Grained Fine-Grained 6 Final Remarks Conclusion Acknowledgments Questions Large Scale Studies: Malware Needles in a Haystack SBSeg’18
- 14. Introduction Motivation & Related Work Methodology Analysis Results Discussion Final Remarks Architecture Large Scale Support Figure: Parallel Processing Architecture. Samples are independently analyzed and their results are stored on a centralized database. Large Scale Studies: Malware Needles in a Haystack SBSeg’18
- 15. Introduction Motivation & Related Work Methodology Analysis Results Discussion Final Remarks Architecture Time Elapsed 0M 10M 20M 30M 40M 50M 60M 0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 Tuples Hours Tuples x Time Figure: Parallel Processing Time. The complete analysis took 36 hours. Large Scale Studies: Malware Needles in a Haystack SBSeg’18
- 16. Introduction Motivation & Related Work Methodology Analysis Results Discussion Final Remarks Coarse-Grained Topics 1 Introduction Analysis Type: Coarse-grained and Fine-grained 2 Motivation & Related Work Motivation Related Work 3 Methodology Dataset Processing the Data Architecture 4 Analysis Results Coarse-Grained Fine-Grained 5 Discussion Coarse-Grained Fine-Grained 6 Final Remarks Conclusion Acknowledgments Questions Large Scale Studies: Malware Needles in a Haystack SBSeg’18
- 17. Introduction Motivation & Related Work Methodology Analysis Results Discussion Final Remarks Coarse-Grained Malicious Behavior (Static Analysis) 0% 20% 40% 60% 80% 100% Percentage(%) Malicious Behaviors Identified Malicious Behaviors Termination Timing Performance Process Window Fingerprinting Modularization Removal AntiDebug Figure: Identified Malicious Behaviors. We identified multiple, distinct malicious behaviors during samples executions, such as AV analysis evasion using Timing delays, measuring Performance overhead due to monitoring and the Termination of security solutions to avoid detection. Large Scale Studies: Malware Needles in a Haystack SBSeg’18
- 18. Introduction Motivation & Related Work Methodology Analysis Results Discussion Final Remarks Coarse-Grained Prevalent Signatures Table: Most Prevalent Signatures. Compilers are more prevalent than packers. Signature Type Occurrence (%) Microsoft Compiler 37.95% Nullsoft PIMP Installer 25.51% Borland Delphi Compiler 15.06% UPX Packer 4.23% MSLHR Packer 2.25% PEcompact Packer 1.66% Large Scale Studies: Malware Needles in a Haystack SBSeg’18
- 19. Introduction Motivation & Related Work Methodology Analysis Results Discussion Final Remarks Coarse-Grained Malicious Behavior (Dynamic Analysis) Table: Dynamic Analysis. Identified Malicious Behaviors. Subsystem Operation Samples (%) Target Samples (%) File Subsystem Create Files 91.56 Internet Explorer 10.14% Read Files 89.18% .DLL 86.80% Internet Explorer 7.01% .SYS 1.26% Write Files 81.74% .EXE 46.20% .DLL 31.62% Internet Explorer <0.01% Host 0.00% Delete Files 62.45% Internet Explorer 0.00% Process Subsystem Create Process 22.84% Delete Process 23.38% Registry Subsystem Set Registry Values 74.73% Proxy 68.36% Autorun 5.66 % Delete Registry Values 55.43% Large Scale Studies: Malware Needles in a Haystack SBSeg’18
- 20. Introduction Motivation & Related Work Methodology Analysis Results Discussion Final Remarks Coarse-Grained Anti-Analysis Techniques 0% 10% 20% 30% 40% 50% 60% 70% 80% Percentage(%) Techniques Identified Anti-Analysis Techniques getlasterror unhandledexceptionfilter raiseexception terminateprocess isdebuggerpresent isprocessorfeaturepresent findwindowexa vmcheck.dll bochs & qemu cpuid vmware virtual box Figure: Identified Anti-Analysis Techniques. Samples employ anti-analysis techniques to avoid being inspected during sandbox execution. We identified techniques aimed to detect the presence of debuggers and virtual-machines. Large Scale Studies: Malware Needles in a Haystack SBSeg’18
- 21. Introduction Motivation & Related Work Methodology Analysis Results Discussion Final Remarks Coarse-Grained Network Analysis — Protocols Distribution 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Samples(%) Protocols Protocol distribution by samples HTTP/Non−TCP HTTP/TCP UDP/Non−DNS UDP/DNS Figure: Protocol usage distribution by sample. HTTP over TCP and DNS over UDP are the prevalent communication channels. Large Scale Studies: Malware Needles in a Haystack SBSeg’18
- 22. Introduction Motivation & Related Work Methodology Analysis Results Discussion Final Remarks Coarse-Grained Network Analysis — Domains Contacted Table: Most Contacted Domains. We observe the presence of cloud providers among the most accessed domains. Domain Accesses (%) Cloudfront 4.39% Amazonaws 3.32% Kirov 3.20% Kerch 3.11% Comcast 1.75% Akamaitechnologies 1.48% Sbcglobal 1.10% Broadband 1.08% Large Scale Studies: Malware Needles in a Haystack SBSeg’18
- 23. Introduction Motivation & Related Work Methodology Analysis Results Discussion Final Remarks Coarse-Grained Network Analysis — TLDs 0% 5% 10% 15% 20% 25% 30% 35% Access(%) Top Level Domains Contacteds TLDS net com ru de sc jp eu br fr biz Figure: Most Accessed TLDs distribution. Generic domains are prevalent and country-specific domains are well distributed, thus showing that malware creators are ready to exploit vulnerabilities in multiple countries. Large Scale Studies: Malware Needles in a Haystack SBSeg’18
- 24. Introduction Motivation & Related Work Methodology Analysis Results Discussion Final Remarks Coarse-Grained Network Analysis — DNS Resolvers 0.01% 0.1% 1% 10% 100% Access(%) DNS hosts DNS queries distribution google opendns d0wn emerion as43289 root−servers polyram−group jp 114dns cryptostorm Figure: DNS Resolvers Distribution. Default sandbox DNS (google) was the most used one. Large Scale Studies: Malware Needles in a Haystack SBSeg’18
- 25. Introduction Motivation & Related Work Methodology Analysis Results Discussion Final Remarks Fine-Grained Topics 1 Introduction Analysis Type: Coarse-grained and Fine-grained 2 Motivation & Related Work Motivation Related Work 3 Methodology Dataset Processing the Data Architecture 4 Analysis Results Coarse-Grained Fine-Grained 5 Discussion Coarse-Grained Fine-Grained 6 Final Remarks Conclusion Acknowledgments Questions Large Scale Studies: Malware Needles in a Haystack SBSeg’18
- 26. Introduction Motivation & Related Work Methodology Analysis Results Discussion Final Remarks Fine-Grained Keys Modification Table: Modified AutoRun Keys. Whereas coarse-grained analysis identified that ≈ 5.66% of samples write in an Autorun key, the fine-grained analysis specified their location. Key Samples(%) HKCUIDSoftwareMicrosoftWindowsCurrentVersionRun 46.33% HKCU.DEFAULTSOFTWAREMicrosoftWindowsCurrentVersionRun < 0.01% Large Scale Studies: Malware Needles in a Haystack SBSeg’18
- 27. Introduction Motivation & Related Work Methodology Analysis Results Discussion Final Remarks Fine-Grained Contacted IPs Table: Contacted IPs by Sample. Coarse-grained analysis showed that, in average, each sample contacts 2 distinct IP addresses. Fine-grained analysis revelead that ransomware samples which spread through scanning contact many more IPs. MD5 Hash Number of Distinct IPs Label c1abb496deb7bd51a4ad2f8a43113b13 16386 Ransomware.Cerber bc88096e7cc09f02f11deec35f84d5cd 16385 Ransomware.AWA a801cdef09a61d3ba7969015a8bffec0 1 Ransomware.VirLock Large Scale Studies: Malware Needles in a Haystack SBSeg’18
- 28. Introduction Motivation & Related Work Methodology Analysis Results Discussion Final Remarks Fine-Grained HTTP requests Table: Prevalent HTTP Requests. Coarse-grained analysis shows that HTTP payloads dominate TCP traffic. Fine-grained analysis shows that this is due to Downloader samples. MD5 Hash Total of Distinct HTTP Request Label ede13f40a96a8b6e5de1029200c0b15e 394 Downloader e5f4116d08c343623d5ee3af5553cbee 353 Downloader 47a328b0b903bb68147facc3a084172c 310 Downloader 28c4e2a48d9ddfffa01a943ca1ba1262 304 Downloader Large Scale Studies: Malware Needles in a Haystack SBSeg’18
- 29. Introduction Motivation & Related Work Methodology Analysis Results Discussion Final Remarks Fine-Grained DNS Resolvers Table: DNS queries resolvers distribution. Coarse-grained analysis shows that DNS queries dominate UDP traffic. Fine-grained analysis reveals that this is due to Bot samples. Query Contacted(%) Description bmp.pilenga.co.uk. 12.29% Hijacked Subdomain - Andromeda Botnet tgr.tecnoagenzia.eu. 5.96% Hijacked Subdomain - Andromeda Botnet tds.repack.it. 2.48% Andromeda Botnet rxxl.tecnoagenzia.eu. 2.06% Andromeda Botnet and31.blllaaaaaazblaaa1.com. 0.92% Andromeda Botnet Large Scale Studies: Malware Needles in a Haystack SBSeg’18
- 30. Introduction Motivation & Related Work Methodology Analysis Results Discussion Final Remarks Coarse-Grained Topics 1 Introduction Analysis Type: Coarse-grained and Fine-grained 2 Motivation & Related Work Motivation Related Work 3 Methodology Dataset Processing the Data Architecture 4 Analysis Results Coarse-Grained Fine-Grained 5 Discussion Coarse-Grained Fine-Grained 6 Final Remarks Conclusion Acknowledgments Questions Large Scale Studies: Malware Needles in a Haystack SBSeg’18
- 31. Introduction Motivation & Related Work Methodology Analysis Results Discussion Final Remarks Coarse-Grained Drawing Panoramas Dataset Characterization Mainly executables. Few libraries. Rely on system native libraries. Few external libraries. GUI usage. Strong presence of system interactions (file and registry creation/deletion). Large Scale Studies: Malware Needles in a Haystack SBSeg’18
- 32. Introduction Motivation & Related Work Methodology Analysis Results Discussion Final Remarks Coarse-Grained Panorama Comparison Brazilian Panorama Mix of binaries and DLL. Rely on system native libraries. Mostly as background activity. Large Scale Studies: Malware Needles in a Haystack SBSeg’18
- 33. Introduction Motivation & Related Work Methodology Analysis Results Discussion Final Remarks Fine-Grained Topics 1 Introduction Analysis Type: Coarse-grained and Fine-grained 2 Motivation & Related Work Motivation Related Work 3 Methodology Dataset Processing the Data Architecture 4 Analysis Results Coarse-Grained Fine-Grained 5 Discussion Coarse-Grained Fine-Grained 6 Final Remarks Conclusion Acknowledgments Questions Large Scale Studies: Malware Needles in a Haystack SBSeg’18
- 34. Introduction Motivation & Related Work Methodology Analysis Results Discussion Final Remarks Fine-Grained Identifying project decisions Coarse-Grained In average, each sample contacts 2 different IP address. Fine-Grained c1abb496deb7bd51a4ad2f8a43113b13 contacts 16386 IPs. a801cdef09a61d3ba7969015a8bffec0 contacts only 1 IP. Large Scale Studies: Malware Needles in a Haystack SBSeg’18
- 35. Introduction Motivation & Related Work Methodology Analysis Results Discussion Final Remarks Conclusion Topics 1 Introduction Analysis Type: Coarse-grained and Fine-grained 2 Motivation & Related Work Motivation Related Work 3 Methodology Dataset Processing the Data Architecture 4 Analysis Results Coarse-Grained Fine-Grained 5 Discussion Coarse-Grained Fine-Grained 6 Final Remarks Conclusion Acknowledgments Questions Large Scale Studies: Malware Needles in a Haystack SBSeg’18
- 36. Introduction Motivation & Related Work Methodology Analysis Results Discussion Final Remarks Conclusion Conclusion A coarse-grained analysis procedure is the only approach able to draw threat panoramas. A fine-grained analysis procedure is the only approach which enables individual samples characterization. Fine-grained and coarse-grained analysis approaches must be combined for increased threat understanding. Large Scale Studies: Malware Needles in a Haystack SBSeg’18
- 37. Introduction Motivation & Related Work Methodology Analysis Results Discussion Final Remarks Acknowledgments Topics 1 Introduction Analysis Type: Coarse-grained and Fine-grained 2 Motivation & Related Work Motivation Related Work 3 Methodology Dataset Processing the Data Architecture 4 Analysis Results Coarse-Grained Fine-Grained 5 Discussion Coarse-Grained Fine-Grained 6 Final Remarks Conclusion Acknowledgments Questions Large Scale Studies: Malware Needles in a Haystack SBSeg’18
- 38. Introduction Motivation & Related Work Methodology Analysis Results Discussion Final Remarks Acknowledgments Acknowledgments CNPq Institute of Computing/Unicamp Large Scale Studies: Malware Needles in a Haystack SBSeg’18
- 39. Introduction Motivation & Related Work Methodology Analysis Results Discussion Final Remarks Questions Topics 1 Introduction Analysis Type: Coarse-grained and Fine-grained 2 Motivation & Related Work Motivation Related Work 3 Methodology Dataset Processing the Data Architecture 4 Analysis Results Coarse-Grained Fine-Grained 5 Discussion Coarse-Grained Fine-Grained 6 Final Remarks Conclusion Acknowledgments Questions Large Scale Studies: Malware Needles in a Haystack SBSeg’18
- 40. Introduction Motivation & Related Work Methodology Analysis Results Discussion Final Remarks Questions Questions? bertao@lasca.ic.unicamp.br Large Scale Studies: Malware Needles in a Haystack SBSeg’18