CNIT 126: 13: Data Encoding
0 likes277 views
This document discusses various data encoding algorithms used by malware to hide information, including simple ciphers like Caesar cipher and XOR, as well as Base64 encoding. It explains why malware uses encoding, covers identifying encoding routines, and provides tips on decoding encoded data, either by reprogramming the encoding functions or using the malware's own decoding functions. The goal is to analyze these encoding algorithms used by malware to disguise strings and configuration details.
CNIT 126: 13: Data Encoding
- 1. Practical Malware Analysis Ch 13: Data Encoding Revised 11-24-20
- 2. The Goal of Analyzing Encoding Algorithms
- 3. Reasons Malware Uses Encoding • Hide configuration information – Such as C&C domains • Save information to a staging file – Before stealing it • Store strings needed by malware – Decode them just before they are needed • Disguise malware as a legitimate tool – Hide suspicious strings
- 5. Why Use Simple Ciphers? • They are easily broken, but – They are small, so they fit into space- constrained environments like exploit shellcode – Less obvious than more complex ciphers – Low overhead, little impact on performance • These are obfuscation, not encryption – They make it difficult to recognize the data, but can't stop a skilled analyst
- 6. Caesar Cipher • Move each letter forward 3 spaces in the alphabet ABCDEFGHIJKLMNOPQRSTUVWXYZ DEFGHIJKLMNOPQRSTUVWXYZABC • Example ATTACK AT NOON DWWDFN DW QRRQ
- 7. XOR • Uses a key to encrypt data • Uses one bit of data and one bit of the key at a time • Example: Encode HI with a key of 0x3c HI = 0x48 0x49 (ASCII encoding) Data: 0100 1000 0100 1001 Key: 0011 1100 0011 1100 Result: 0111 0100 0111 0101 0 xor 0 = 0 0 xor 1 = 1 1 xor 0 = 1 1 xor 1 = 0
- 9. XOR Reverses Itself • Example: Encode HI with a key of 0x3c HI = 0x48 0x49 (ASCII encoding) Data: 0100 1000 0100 1001 Key: 0011 1100 0011 1100 Result: 0111 0100 0111 0101 • Encode it again Result: 0111 0100 0111 0101 Key: 0011 1100 0011 1100 Data: 0100 1000 0100 1001 0 xor 0 = 0 0 xor 1 = 1 1 xor 0 = 1 1 xor 1 = 0
- 10. Brute-Forcing XOR Encoding • If the key is a single byte, there are only 256 possible keys – Error in book; this should be "a.exe" – PE files begin with MZ
- 11. MZ = 0x4d 0x5a
- 13. Link Ch 13a
- 14. Brute-Forcing Many Files • Look for a common string, like "This Program"
- 15. XOR and Nulls • A null byte reveals the key, because – 0x00 xor KEY = KEY • Obviously the key here is 0x12
- 16. NULL-Preserving Single-Byte XOR Encoding • Algorithm: – Use XOR encoding, EXCEPT – If the plaintext is NULL or the key itself, skip the byte
- 18. Identifying XOR Loops in IDA Pro • Small loops with an XOR instruction inside 1. Start in "IDA View" (seeing code) 2. Click Search, Text 3. Enter xor and Find all occurrences
- 19. Three Forms of XOR • XOR a register with itself, like xor edx, edx – Innocent, a common way to zero a register • XOR a register or memory reference with a constant – May be an encoding loop, and key is the constant • XOR a register or memory reference with a different register or memory reference – May be an encoding loop, key less obvious
- 22. Base64 • Converts 6 bits into one character in a 64- character alphabet • There are a few versions, but all use these 62 characters: ABCDEFGHIJKLMNOPQRSTUVWXYZ abcdefghijklmnopqrstuvwxyz 0123456789 • MIME uses + and / – Also = to indicate padding
- 24. Transforming Data to Base64 • Use 3-byte chunks (24 bits) • Break into four 6-bit fields • Convert each to Base64
- 25. base64encode.org base64decode.org • 3 bytes encode to 4 Base64 characters
- 26. Padding • If input had only 2 characters, an = is appended
- 27. Padding • If input had only 1 character, == is appended
- 28. Example • URL and cookie are Base64-encoded
- 29. Cookie: Ym90NTQxNjQ • This has 11 characters— padding is omitted • Some Base64 decoders will fail, but this one just automatically adds the missing padding
- 30. Finding the Base64 Function • Look for this "indexing string" ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghi jklmnopqrstuvwxyz0123456789+/ • Look for a lone padding character (typically =) hard-coded into the encoding function
- 31. Decoding the URLs • Custom indexing string aABCDEFGHIJKLMNOPQRSTUVWXYZbcdefghi jklmnopqrstuvwxyz0123456789+/
- 33. 13a
- 35. Strong Cryptography • Strong enough to resist brute-force attacks – Ex: SSL, AES, etc. • Disadvantages of strong encryption – Large cryptographic libraries required – May make code less portable – Standard cryptographic libraries are easily detected • Via function imports, function matching, or identification of cryptographic constants – Symmetric encryption requires a way to hide the key
- 36. Recognizing Strings and Imports • Strings found in malware encrypted with OpenSSL
- 37. Recognizing Strings and Imports • Microsoft crypto functions usually start with Crypt or CP or Cert
- 38. Searching for Cryptographic Constants • IDA Pro's FindCrypt2 Plug-in (Link Ch 13c) – Finds magic constants (binary signatures of crypto routines) – Cannot find RC4 or IDEA routines because they don't use a magic constant – RC4 is commonly used in malware because it's small and easy to implement
- 39. FindCrypt2 • Runs automatically on any new analysis • Can be run manually from the Plug-In Menu
- 40. Krypto ANALyzer (PEiD Plug-in) • Download from link Ch 13d • Has wider range of constants than FindCrypt2 – More false positives • Also finds Base64 tables and crypto function imports
- 41. Entropy • Entropy measures disorder • To calculate it, just count the number of occurrences of each byte from 0 to 255 – Calculate Pi = Probability of value i – Then sum Pi log( Pi) for I = 0 to 255 (Link 13e) • If all the bytes are equally likely, the entropy is 8 (maximum disorder) • If all the bytes are the same, the entropy is zero
- 42. Entropy Demo • Put output in a file • Use binwalk -E to analyze the file • Multiply vertical axis by 8 42 #!/usr/bin/python import base64, random a = '' for i in range(0, 10000): a += chr(random.randint(0,255)) b = base64.b64encode(a) c = base64.b32encode(a) d = base64.b16encode(a) e = 'A' * 10000 print a + b + c + d + e
- 43. Entropy Demo • Concatenate three images in different formats 43
- 44. Searching for High-Entropy Content • IDA Pro Entropy Plugin • Finds regions of high entropy, indicating encryption (or compression)
- 45. Recommended Parameters • Chunk size: 64 Max. Entropy: 5.95 – Good for finding many constants, – Including Base64-encoding strings (entropy 6) • Chunk size: 256 Max. Entropy: 7.9 – Finds very random regions
- 46. Entropy Graph • IDA Pro Entropy Plugin – Download from link Ch 13g – Use StandAlone version – Double-click region, then Calculate, Draw – Lighter regions have high entropy – Hover over graph to see numerical value
- 48. Custom Encoding
- 49. Homegrown Encoding Schemes • Examples – One round of XOR, then Base64 – Custom algorithm, possibly similar to a published cryptographic algorithm
- 50. Identifying Custom Encoding • This sample makes a bunch of 700 KB files • Figure out the encoding from the code • Find CreateFileA and WriteFileA – In function sub_4011A9 • Uses XOR with a pseudorandom stream
- 52. Advantages of Custom Encoding to the Attacker • Can be small and nonobvious • Harder to reverse-engineer
- 53. Decoding
- 54. Two Methods • Reprogram the functions • Use the functions in the malware itself
- 55. Self-Decoding • Stop the malware in a debugger with data decoded • Isolate the decryption function and set a breakpoint directly after it • BUT sometimes you can't figure out how to stop it with the data you need decoded
- 56. Manual Programming of Decoding Functions • Standard functions may be available
- 58. PyCrypto Library • Good for standard algorithms • Now pycryptodome is preferred
- 59. How to Decrypt Using Malware
- 61. 13b