Deobfuscation and beyond (ZeroNights, 2014)

Deobfuscation and beyond
Vasily Bukasov
and
Dmitry Schelkunov
https://re-crypt.com

Agenda
• We'll speak about obfuscation
techniques which commercial (and not
only) obfuscators use and how symbolic
equation systems could help to
deobfuscate such transformations
• We'll form the requirements for these
systems
• We'll briefly skim over design of our mini-
symbolic equation system and show the
results of deobfuscation (and not only)
using it

Software obfuscation
Is used for software
protection against
computer piracy
Is used for malware
protection against
signature-based and
heuristic-based
antiviruses

Common obfuscation techniques
Recursive substitution

Code duplication

Code duplication in
virtualization obfuscators

Previous researches and products
• The Case for Semantics-Based Methods in Reverse Engineering, Rolf
Rolles, RECON 2012
• Software deobfuscation methods: analysis and implementation, Sh.F.
Kurmangaleev, K.Y. Dolgorukova, V.V. Savchenko, A.R. Nurmukhametov,
H. A Matevosyan, V.P. Korchagin, Proceedings of the Institute for
System Programming of RAS, volume 24, 2013
• CodeDoctor
– deobfuscates simple expressions
– plugin for OllyDbg and IDA Pro

• VMSweeper
– declares deobfuscation (devirtualization) of Code
Virtualizer/CISC and VMProtect (works well on about 30% of
virtualized samples)
– not a generic tool (heavily relies on templates)
– works as a decompiler not optimizer
– weak symbolic equation system
• CodeUnvirtualizer
– declares deobfuscation (devirtualization) of Code
Virtualizer/CISC/RISC and Themida new VMs
– not a generic tool (heavily relies on templates)
– no symbolic equation system

• Ariadne
– complex toolset for deobfuscation and data flow analysis
– includes a lot of optimization algorithms from compiler theory
– no symbolic equation system
– it seems to be dead 
• LLVM forks
– are based on LLVM optimization algorithms (classical compiler
theory algorithms)
– we couldn’t find any decently working version
– are limited by LLVM architecture (How fast LLVM works with 500
000 IR instructions? How much system resources it requires?)

The problem
Existing deobfuscation solutions are mostly
based on classical compiler theory algorithms
and too weak against modern obfuscators in the
most of cases

Solution
• Use symbolic equation system (SES) for
deobfuscation
• Form input data for SES (translate source IR
code to SES representation)
• Simplify expressions using SES
• Translate results from SES representation to
IR
• Apply other deobfuscation transformations

Symbolic equation system
Unfortunately, we couldn’t find an
appropriate third-party symbolic equation
system engine and … we decided to create
a new one for ourselves.
We called it Project Eq.

Eq design
eax.1 = ( ( eax.0 * 0xffffffff ) + 0xffffffff ) ^ 0xffffffff

Eq design
eax.1 = ( ( eax.0 * 0xffffffff ) + 0xffffffff ) ^ 0xffffffff
eax.0 (v)
eax.1 = eax.0
Profit! J

Eq in work
union rebx_type
{
UINT32 rebx;
WORD rbx;
BYTE rblow[2];
};
void vmp_constant_playing(rebx_type &rebx)
{
BYTE var0;
union var1_type
{
UINT32 var;
WORD var_med;
BYTE var_low;
} var1;
var0 = rebx.rblow[0];
rebx.rblow[0] = 0xe7;
var1.var_med = rebx.rbx;
var1.var_low = 0x18;
rebx.rbx = var1.var_med;
rebx.rblow[0] = var0;
}
A C++ sample of
obfuscated code.
It was borrowed J
from VMProtect

Eq in work
void rustock_sample(UINT32 &rebp, UINT32 &redi, UINT32 &resi)
{
UINT32 var0, var1, var2;
var0 = rebp;
rebp = redi | rebp;
var1 = redi & var0;
resi = ~var1;
var2 = rebp & resi;
redi = var0 ^ var2;
}
A C++ sample of
obfuscated code.
It was borrowed J
from Rustock

Deobfuscation with Eq
After code virtualization

• ASProtect
• CodeVirtualizer/Themida/WinLicense
– old CISC/RISC
– new Fish/Tiger
• ExeCryptor
• NoobyProtect/SafeEngine
• Tages
• VMProtect
• Some others…
Were deobfuscated successfully J

Some numbers
Instructions initially ~100
Instructions after obfuscation ~300 000
Instructions after deobfuscation ~200
Code generation time ~4 min
Code deobfuscation time ~2 min
Memory ~300 Mb

Obfuscation with Eq
We could use optimization not for
deobfuscation only.
What if we could stop optimization
process at random step?

Obfuscation with Eq
• Easy to implement
• Hard to deobfuscate using classical
compiler theory optimization algorithms
• Hard to deobfuscate using reverse
recursive substitution
• No templates and signatures in the
obfuscated code

Obfuscation with Eq
But this tricky obfuscation is still weak.
It’s possible to deobfuscate these expressions using Eq
project or another symbolic equation system.
And we have to go deeper!

Perspectives
• Obfuscation becomes stronger
– Complex mathematical expressions are
used more frequently
– Merges with cryptography
• Obfuscation migrates to dark side
– Protectors are dying
– Malware market is growing

Perspectives
• Obfuscation becomes undetectable
– Mimicry methods are improved
– Obfuscators try to avoid method of
recursive substitutions
– Obfuscators use well-known high-level
platforms
• LLVM becomes a generic platform for
creating obfuscators

Deobfuscation and beyond (ZeroNights, 2014)

More Related Content

What's hot (13)

Viewers also liked (20)

Similar to Deobfuscation and beyond (ZeroNights, 2014) (20)

Recently uploaded (20)

Deobfuscation and beyond (ZeroNights, 2014)