SlideShare a Scribd company logo
Introduction
  Verification algorithm
            Obfuscation
      Encryption layers
  Direct native API call
                 Anti-X
        How to break it
     Possible evolutions




Hackito Ergo Sum Crackme

            ´
            Eloi Vanderb´ken
                        e

 eloi.vanderbeken (at) ens-cachan.fr


               09 April 2011




      ´
      Eloi Vanderb´ken
                  e        Hackito Ergo Sum 2011
Introduction
                       Verification algorithm
                                 Obfuscation
                           Encryption layers
                       Direct native API call
                                      Anti-X
                             How to break it
                          Possible evolutions


Plan
   1 Introduction
   2 Verification algorithm
   3 Obfuscation
       Instruction mutation
       Control Flow Graph (CFG) obfuscation
   4 Encryption layers
   5 Direct native API call
   6 Anti-X
   7 How to break it
   8 Possible evolutions

                           ´
                           Eloi Vanderb´ken
                                       e        Hackito Ergo Sum 2011
Introduction
                   Verification algorithm
                             Obfuscation
                       Encryption layers
                   Direct native API call
                                  Anti-X
                         How to break it
                      Possible evolutions


About me



     Last year of master’s degree in cryptology and software
                 ´
     security at Ecole Normale Sup´rieure of Cachan
                                    e
     Reversing software protection for 6 years
     Participate to several Capture The Flags in my spare time




                       ´
                       Eloi Vanderb´ken
                                   e        Hackito Ergo Sum 2011
Introduction
                       Verification algorithm
                                 Obfuscation
                           Encryption layers
                       Direct native API call
                                      Anti-X
                             How to break it
                          Possible evolutions


Plan
   1 Introduction
   2 Verification algorithm
   3 Obfuscation
       Instruction mutation
       Control Flow Graph (CFG) obfuscation
   4 Encryption layers
   5 Direct native API call
   6 Anti-X
   7 How to break it
   8 Possible evolutions

                           ´
                           Eloi Vanderb´ken
                                       e        Hackito Ergo Sum 2011
Introduction
                      Verification algorithm
                                Obfuscation
                          Encryption layers
                      Direct native API call
                                     Anti-X
                            How to break it
                         Possible evolutions


Some facts and numbers


      First crackme
      2 months
      6000 lines of python
      Use TTASM and BeaEngine
      Randomly generated
      Only 3 valid solutions




                          ´
                          Eloi Vanderb´ken
                                      e        Hackito Ergo Sum 2011
Introduction
                       Verification algorithm
                                 Obfuscation
                           Encryption layers
                       Direct native API call
                                      Anti-X
                             How to break it
                          Possible evolutions


Plan
   1 Introduction
   2 Verification algorithm
   3 Obfuscation
       Instruction mutation
       Control Flow Graph (CFG) obfuscation
   4 Encryption layers
   5 Direct native API call
   6 Anti-X
   7 How to break it
   8 Possible evolutions

                           ´
                           Eloi Vanderb´ken
                                       e        Hackito Ergo Sum 2011
Introduction
                    Verification algorithm
                              Obfuscation
                        Encryption layers
                    Direct native API call
                                   Anti-X
                          How to break it
                       Possible evolutions


Algorithm



      Custom RC4 (initial table changed, starts with ”HESFTW”)
      Serial is used as a 256-byte key
      Generate the first 7 DWORDs
      Combine them with 0xDEADBEEF to generate a DWORD
      Use it as a constant to calculate the length of the last layers




                        ´
                        Eloi Vanderb´ken
                                    e        Hackito Ergo Sum 2011
Introduction
                       Verification algorithm
                                 Obfuscation
                           Encryption layers    Instruction mutation
                       Direct native API call   Control Flow Graph (CFG) obfuscation
                                      Anti-X
                             How to break it
                          Possible evolutions


Plan
   1 Introduction
   2 Verification algorithm
   3 Obfuscation
       Instruction mutation
       Control Flow Graph (CFG) obfuscation
   4 Encryption layers
   5 Direct native API call
   6 Anti-X
   7 How to break it
   8 Possible evolutions

                           ´
                           Eloi Vanderb´ken
                                       e        Hackito Ergo Sum 2011
Introduction
                     Verification algorithm
                               Obfuscation
                         Encryption layers    Instruction mutation
                     Direct native API call   Control Flow Graph (CFG) obfuscation
                                    Anti-X
                           How to break it
                        Possible evolutions


Plan

   1 Introduction


   2 Verification algorithm


   3 Obfuscation
       Instruction mutation
       Control Flow Graph (CFG) obfuscation

   4 Encryption layers


   5 Direct native API call
                         ´
                         Eloi Vanderb´ken
                                     e        Hackito Ergo Sum 2011
Introduction
                    Verification algorithm
                              Obfuscation
                        Encryption layers    Instruction mutation
                    Direct native API call   Control Flow Graph (CFG) obfuscation
                                   Anti-X
                          How to break it
                       Possible evolutions


Instruction mutation



      Make a static analysis to get freely modifiable registers
      Use them to mutate instructions
      Main rule: if your code use an instruction, it has to be used
      everywhere ⇒ no difference between the useful code and the
      garbage one




                        ´
                        Eloi Vanderb´ken
                                    e        Hackito Ergo Sum 2011
Introduction
                     Verification algorithm
                               Obfuscation
                         Encryption layers    Instruction mutation
                     Direct native API call   Control Flow Graph (CFG) obfuscation
                                    Anti-X
                           How to break it
                        Possible evolutions


Plan

   1 Introduction


   2 Verification algorithm


   3 Obfuscation
       Instruction mutation
       Control Flow Graph (CFG) obfuscation

   4 Encryption layers


   5 Direct native API call
                         ´
                         Eloi Vanderb´ken
                                     e        Hackito Ergo Sum 2011
Introduction
                       Verification algorithm
                                 Obfuscation
                           Encryption layers    Instruction mutation
                       Direct native API call   Control Flow Graph (CFG) obfuscation
                                      Anti-X
                             How to break it
                          Possible evolutions


Control Flow Graph (CFG) obfuscation

   Before instruction mutation:
        Duplicate some piece of code
        Add a fake or non determinist test to choose which block will
        be executed
   After:
        Make a static analysis to know which flags are set/unset for
        each instruction
        Add conditional jump which are never/always taken according
        to the flags
        Shuffle blocks

                           ´
                           Eloi Vanderb´ken
                                       e        Hackito Ergo Sum 2011
Figure: (incomplete) CFG of a layer made by IDA
Introduction
                       Verification algorithm
                                 Obfuscation
                           Encryption layers
                       Direct native API call
                                      Anti-X
                             How to break it
                          Possible evolutions


Plan
   1 Introduction
   2 Verification algorithm
   3 Obfuscation
       Instruction mutation
       Control Flow Graph (CFG) obfuscation
   4 Encryption layers
   5 Direct native API call
   6 Anti-X
   7 How to break it
   8 Possible evolutions

                           ´
                           Eloi Vanderb´ken
                                       e        Hackito Ergo Sum 2011
Introduction
                       Verification algorithm
                                 Obfuscation
                           Encryption layers
                       Direct native API call
                                      Anti-X
                             How to break it
                          Possible evolutions


Layers




   How to make a layer the most annoying possible?
         Hide the end address of the encryption layer
         Hide the size of the encrypted code




                           ´
                           Eloi Vanderb´ken
                                       e        Hackito Ergo Sum 2011
Introduction
                   Verification algorithm
                             Obfuscation
                       Encryption layers
                   Direct native API call
                                  Anti-X
                         How to break it
                      Possible evolutions


Layers (continued)

      Use Linear Feedback Shift Register (LFSR) to encrypt n times
      both end address and a dword counter
      At execution, decrypt them one time at each loop execution
      When the dword counter has been decrypted, jump to the
      decrypted address
      Use indirect jump to avoid breakpoint:
      cmp counter, CST       setz cl     jmp [ecx +
      addresses]
      Bonus: LFSR having the biggest possible period, you can add
      as many fake comparison as you want

                       ´
                       Eloi Vanderb´ken
                                   e        Hackito Ergo Sum 2011
Introduction
                       Verification algorithm
                                 Obfuscation
                           Encryption layers
                       Direct native API call
                                      Anti-X
                             How to break it
                          Possible evolutions


Plan
   1 Introduction
   2 Verification algorithm
   3 Obfuscation
       Instruction mutation
       Control Flow Graph (CFG) obfuscation
   4 Encryption layers
   5 Direct native API call
   6 Anti-X
   7 How to break it
   8 Possible evolutions

                           ´
                           Eloi Vanderb´ken
                                       e        Hackito Ergo Sum 2011
Introduction
                     Verification algorithm
                               Obfuscation
                         Encryption layers
                     Direct native API call
                                    Anti-X
                           How to break it
                        Possible evolutions


Direct native API call



       Get syscall numbers at initialisation
       Use sysenter (do not work on 64bit systems)
       Make random invalid or non invasives (ZwYieldExecution)
       syscalls to prevent conditional breakpoints on
       KiFastSystemCallRet




                         ´
                         Eloi Vanderb´ken
                                     e        Hackito Ergo Sum 2011
Introduction
                       Verification algorithm
                                 Obfuscation
                           Encryption layers
                       Direct native API call
                                      Anti-X
                             How to break it
                          Possible evolutions


Plan
   1 Introduction
   2 Verification algorithm
   3 Obfuscation
       Instruction mutation
       Control Flow Graph (CFG) obfuscation
   4 Encryption layers
   5 Direct native API call
   6 Anti-X
   7 How to break it
   8 Possible evolutions

                           ´
                           Eloi Vanderb´ken
                                       e        Hackito Ergo Sum 2011
Introduction
                     Verification algorithm
                               Obfuscation
                         Encryption layers
                     Direct native API call
                                    Anti-X
                           How to break it
                        Possible evolutions


Classical anti-debuggers tricks



   Well known anti-debuggers trick made difficult to bypass because
   of direct use of sysenter:
       NtQueryInformationProcess + ProcessDebugPort
       NtSetInformationThread + ThreadHideFromDebugger
       NtSetContextThread to delete hardware breakpoints




                         ´
                         Eloi Vanderb´ken
                                     e        Hackito Ergo Sum 2011
Introduction
                    Verification algorithm
                              Obfuscation
                        Encryption layers
                    Direct native API call
                                   Anti-X
                          How to break it
                       Possible evolutions


Anti-instrumentation framework



      Detect hook on KiUserExceptionDispatcher and
      KiUserCallbackDispatcher
      Detect stack reuse by instrumentation code: place a constant
      in esp-4, execute a sysenter, check if esp-4 has been changed




                        ´
                        Eloi Vanderb´ken
                                    e        Hackito Ergo Sum 2011
Introduction
                       Verification algorithm
                                 Obfuscation
                           Encryption layers
                       Direct native API call
                                      Anti-X
                             How to break it
                          Possible evolutions


Anti data-tainting
   After execution of each part of the algorithm, result is copied using
   indirect dependance
   AntiTaintMemcpy (BYTE* source, BYTE *dest, int size)
   {
       for (int t =0 t < size ++t) {
           BYTE val = 0
           do {
               BOOL isDiff = FALSE
               if (source[t] != val)
                   isDiff = True
               if (! isDiff)
                   dest[t] = val
               ++val
           } while (val != 0)
       }
   }

                           ´
                           Eloi Vanderb´ken
                                       e        Hackito Ergo Sum 2011
Introduction
                       Verification algorithm
                                 Obfuscation
                           Encryption layers
                       Direct native API call
                                      Anti-X
                             How to break it
                          Possible evolutions


Plan
   1 Introduction
   2 Verification algorithm
   3 Obfuscation
       Instruction mutation
       Control Flow Graph (CFG) obfuscation
   4 Encryption layers
   5 Direct native API call
   6 Anti-X
   7 How to break it
   8 Possible evolutions

                           ´
                           Eloi Vanderb´ken
                                       e        Hackito Ergo Sum 2011
Introduction
                    Verification algorithm
                              Obfuscation
                        Encryption layers
                    Direct native API call
                                   Anti-X
                          How to break it
                       Possible evolutions


So how to solve it?


      API init Set memory breakpoint on ntdll PE
     Anti-Dbg Don’t use debugger, inject a DLL and use
              VectoredExceptionHandler to set HBP
     Anti-HBP Replace ZwSetThreadContext syscall number by an
              harmless or invalid one
      ReadFile Use kernel debugger or replace syscall numbers by a
               GDI one which will call KiUserCallbackDispatcher
   Obfuscation Step the code :)


                        ´
                        Eloi Vanderb´ken
                                    e        Hackito Ergo Sum 2011
Introduction
                     Verification algorithm
                               Obfuscation
                         Encryption layers
                     Direct native API call
                                    Anti-X
                           How to break it
                        Possible evolutions


How to find a valid serial



       Bruteforce (JB B´drune)
                       e
       Use of the RC4 internal table created with the key published
       (Kyriakos Economou and Mark Wodrich)
       Generate a valid keystream, use a backward algorithm to find
       an internal table which generate this keystream, find a key
       which generate this table (Me :) )




                         ´
                         Eloi Vanderb´ken
                                     e        Hackito Ergo Sum 2011
Introduction
                       Verification algorithm
                                 Obfuscation
                           Encryption layers
                       Direct native API call
                                      Anti-X
                             How to break it
                          Possible evolutions


Plan
   1 Introduction
   2 Verification algorithm
   3 Obfuscation
       Instruction mutation
       Control Flow Graph (CFG) obfuscation
   4 Encryption layers
   5 Direct native API call
   6 Anti-X
   7 How to break it
   8 Possible evolutions

                           ´
                           Eloi Vanderb´ken
                                       e        Hackito Ergo Sum 2011
Introduction
                        Verification algorithm
                                  Obfuscation
                            Encryption layers
                        Direct native API call
                                       Anti-X
                              How to break it
                           Possible evolutions


Some ideas for the next year ;)


       Recode all the crackme generation in C
       Use XED (by intel) to disassemble and reassemble instruction
       Multi-thread
       Virtualisation
       More mutation rules, more complex instructions (FPU, MMX,
       SSE etc.)
       More anti-X



                            ´
                            Eloi Vanderb´ken
                                        e        Hackito Ergo Sum 2011
Introduction
         Verification algorithm
                   Obfuscation
             Encryption layers
         Direct native API call
                        Anti-X
               How to break it
            Possible evolutions


Thanks




                  Thank you !




             ´
             Eloi Vanderb´ken
                         e        Hackito Ergo Sum 2011
Introduction
              Verification algorithm
                        Obfuscation
                  Encryption layers
              Direct native API call
                             Anti-X
                    How to break it
                 Possible evolutions


Questions ?




                        Questions ?




                  ´
                  Eloi Vanderb´ken
                              e        Hackito Ergo Sum 2011

More Related Content

PDF
Itc Theater09 Sep1420 P Redits Done
PDF
Verification of Security for Untrusted Third Party IP Cores
PDF
Ponsini automatic slides
PDF
HES2011 - Sebastien Tricaud - Capture me if you can
PDF
HES2011 - joernchen - Ruby on Rails from a Code Auditor Perspective
PDF
HES2011 - Gabriel Gonzalez - Man In Remote PKCS11 for fun and non profit
PDF
HES2011 - Jon Oberheide and Dan Rosenberg - Stackjacking
PDF
HES2011 - Aaron Portnoy and Logan Brown - Black Box Auditing Adobe Shockwave
Itc Theater09 Sep1420 P Redits Done
Verification of Security for Untrusted Third Party IP Cores
Ponsini automatic slides
HES2011 - Sebastien Tricaud - Capture me if you can
HES2011 - joernchen - Ruby on Rails from a Code Auditor Perspective
HES2011 - Gabriel Gonzalez - Man In Remote PKCS11 for fun and non profit
HES2011 - Jon Oberheide and Dan Rosenberg - Stackjacking
HES2011 - Aaron Portnoy and Logan Brown - Black Box Auditing Adobe Shockwave

Similar to HES2011 - Eloi Vanderbeken - Hackito Ergo Sum Crackme (20)

PDF
nabdullin_brcrdu_dark
PDF
Crypto101
PDF
Crypto Strikes Back! (Google 2009)
PDF
Sullivan randomness-infiltrate 2014
PDF
CNIT 141: 14. Quantum and Post-Quantum
PDF
Web cryptography javascript
PDF
CNIT 141: 14. Quantum and Post-Quantum
PPTX
Introduction to Cryptography
PPTX
Cryptography 101
PPTX
Hacking cryptography: 0xdec0de01 cryptoCTF solutions and a bit more - Владими...
PPTX
Computing on Encrypted Data
PDF
lec1.pdf
PDF
DEFCON 23 - Eijah - crypto for hackers
PPTX
39110832_39110863Project (1)certificategeneration.pptx
PDF
Modern Cryptography
PDF
0xdec0de01 crypto CTF solutions
PDF
When Crypto Attacks! (Yahoo 2009)
PDF
notes.pdf
PPTX
Safe and secure programming practices for embedded devices
nabdullin_brcrdu_dark
Crypto101
Crypto Strikes Back! (Google 2009)
Sullivan randomness-infiltrate 2014
CNIT 141: 14. Quantum and Post-Quantum
Web cryptography javascript
CNIT 141: 14. Quantum and Post-Quantum
Introduction to Cryptography
Cryptography 101
Hacking cryptography: 0xdec0de01 cryptoCTF solutions and a bit more - Владими...
Computing on Encrypted Data
lec1.pdf
DEFCON 23 - Eijah - crypto for hackers
39110832_39110863Project (1)certificategeneration.pptx
Modern Cryptography
0xdec0de01 crypto CTF solutions
When Crypto Attacks! (Yahoo 2009)
notes.pdf
Safe and secure programming practices for embedded devices

More from Hackito Ergo Sum (8)

PDF
HES 2011 - Gal Diskin - Binary instrumentation for hackers - Lightning-talk
PDF
HES2011 - Jon Larimer - Autorun Vulnerabilities on Linux
PDF
HES2011 - James Oakley and Sergey bratus-Exploiting-the-Hard-Working-DWARF
PDF
HES2011 - Raould Chiesa - Hackers Cybercriminals from Wargames to the Undergr...
PDF
HES2011 - Richard Johnson - A Castle Made of Sand Adobe Reader X Sandbox
PDF
HES2011 - Tarjei Mandt – Kernel Pool Exploitation on Windows 7
PDF
HES2011 - Yuval Vadim Polevoy – Money Is In The Eye Of The Beholder: New And ...
PDF
HES2011 - Itzik Kolter - Let me Stuxnet You
HES 2011 - Gal Diskin - Binary instrumentation for hackers - Lightning-talk
HES2011 - Jon Larimer - Autorun Vulnerabilities on Linux
HES2011 - James Oakley and Sergey bratus-Exploiting-the-Hard-Working-DWARF
HES2011 - Raould Chiesa - Hackers Cybercriminals from Wargames to the Undergr...
HES2011 - Richard Johnson - A Castle Made of Sand Adobe Reader X Sandbox
HES2011 - Tarjei Mandt – Kernel Pool Exploitation on Windows 7
HES2011 - Yuval Vadim Polevoy – Money Is In The Eye Of The Beholder: New And ...
HES2011 - Itzik Kolter - Let me Stuxnet You

Recently uploaded (20)

PDF
A contest of sentiment analysis: k-nearest neighbor versus neural network
PPTX
Modernising the Digital Integration Hub
PDF
Enhancing emotion recognition model for a student engagement use case through...
PPTX
OMC Textile Division Presentation 2021.pptx
PDF
Getting started with AI Agents and Multi-Agent Systems
PPTX
Chapter 5: Probability Theory and Statistics
PDF
NewMind AI Weekly Chronicles – August ’25 Week III
PDF
DP Operators-handbook-extract for the Mautical Institute
PDF
DASA ADMISSION 2024_FirstRound_FirstRank_LastRank.pdf
PDF
STKI Israel Market Study 2025 version august
PPT
What is a Computer? Input Devices /output devices
PPTX
O2C Customer Invoices to Receipt V15A.pptx
PDF
Architecture types and enterprise applications.pdf
PPTX
observCloud-Native Containerability and monitoring.pptx
PDF
project resource management chapter-09.pdf
PPTX
TechTalks-8-2019-Service-Management-ITIL-Refresh-ITIL-4-Framework-Supports-Ou...
PPTX
Tartificialntelligence_presentation.pptx
PPTX
The various Industrial Revolutions .pptx
PDF
Univ-Connecticut-ChatGPT-Presentaion.pdf
PDF
NewMind AI Weekly Chronicles - August'25-Week II
A contest of sentiment analysis: k-nearest neighbor versus neural network
Modernising the Digital Integration Hub
Enhancing emotion recognition model for a student engagement use case through...
OMC Textile Division Presentation 2021.pptx
Getting started with AI Agents and Multi-Agent Systems
Chapter 5: Probability Theory and Statistics
NewMind AI Weekly Chronicles – August ’25 Week III
DP Operators-handbook-extract for the Mautical Institute
DASA ADMISSION 2024_FirstRound_FirstRank_LastRank.pdf
STKI Israel Market Study 2025 version august
What is a Computer? Input Devices /output devices
O2C Customer Invoices to Receipt V15A.pptx
Architecture types and enterprise applications.pdf
observCloud-Native Containerability and monitoring.pptx
project resource management chapter-09.pdf
TechTalks-8-2019-Service-Management-ITIL-Refresh-ITIL-4-Framework-Supports-Ou...
Tartificialntelligence_presentation.pptx
The various Industrial Revolutions .pptx
Univ-Connecticut-ChatGPT-Presentaion.pdf
NewMind AI Weekly Chronicles - August'25-Week II

HES2011 - Eloi Vanderbeken - Hackito Ergo Sum Crackme

  • 1. Introduction Verification algorithm Obfuscation Encryption layers Direct native API call Anti-X How to break it Possible evolutions Hackito Ergo Sum Crackme ´ Eloi Vanderb´ken e eloi.vanderbeken (at) ens-cachan.fr 09 April 2011 ´ Eloi Vanderb´ken e Hackito Ergo Sum 2011
  • 2. Introduction Verification algorithm Obfuscation Encryption layers Direct native API call Anti-X How to break it Possible evolutions Plan 1 Introduction 2 Verification algorithm 3 Obfuscation Instruction mutation Control Flow Graph (CFG) obfuscation 4 Encryption layers 5 Direct native API call 6 Anti-X 7 How to break it 8 Possible evolutions ´ Eloi Vanderb´ken e Hackito Ergo Sum 2011
  • 3. Introduction Verification algorithm Obfuscation Encryption layers Direct native API call Anti-X How to break it Possible evolutions About me Last year of master’s degree in cryptology and software ´ security at Ecole Normale Sup´rieure of Cachan e Reversing software protection for 6 years Participate to several Capture The Flags in my spare time ´ Eloi Vanderb´ken e Hackito Ergo Sum 2011
  • 4. Introduction Verification algorithm Obfuscation Encryption layers Direct native API call Anti-X How to break it Possible evolutions Plan 1 Introduction 2 Verification algorithm 3 Obfuscation Instruction mutation Control Flow Graph (CFG) obfuscation 4 Encryption layers 5 Direct native API call 6 Anti-X 7 How to break it 8 Possible evolutions ´ Eloi Vanderb´ken e Hackito Ergo Sum 2011
  • 5. Introduction Verification algorithm Obfuscation Encryption layers Direct native API call Anti-X How to break it Possible evolutions Some facts and numbers First crackme 2 months 6000 lines of python Use TTASM and BeaEngine Randomly generated Only 3 valid solutions ´ Eloi Vanderb´ken e Hackito Ergo Sum 2011
  • 6. Introduction Verification algorithm Obfuscation Encryption layers Direct native API call Anti-X How to break it Possible evolutions Plan 1 Introduction 2 Verification algorithm 3 Obfuscation Instruction mutation Control Flow Graph (CFG) obfuscation 4 Encryption layers 5 Direct native API call 6 Anti-X 7 How to break it 8 Possible evolutions ´ Eloi Vanderb´ken e Hackito Ergo Sum 2011
  • 7. Introduction Verification algorithm Obfuscation Encryption layers Direct native API call Anti-X How to break it Possible evolutions Algorithm Custom RC4 (initial table changed, starts with ”HESFTW”) Serial is used as a 256-byte key Generate the first 7 DWORDs Combine them with 0xDEADBEEF to generate a DWORD Use it as a constant to calculate the length of the last layers ´ Eloi Vanderb´ken e Hackito Ergo Sum 2011
  • 8. Introduction Verification algorithm Obfuscation Encryption layers Instruction mutation Direct native API call Control Flow Graph (CFG) obfuscation Anti-X How to break it Possible evolutions Plan 1 Introduction 2 Verification algorithm 3 Obfuscation Instruction mutation Control Flow Graph (CFG) obfuscation 4 Encryption layers 5 Direct native API call 6 Anti-X 7 How to break it 8 Possible evolutions ´ Eloi Vanderb´ken e Hackito Ergo Sum 2011
  • 9. Introduction Verification algorithm Obfuscation Encryption layers Instruction mutation Direct native API call Control Flow Graph (CFG) obfuscation Anti-X How to break it Possible evolutions Plan 1 Introduction 2 Verification algorithm 3 Obfuscation Instruction mutation Control Flow Graph (CFG) obfuscation 4 Encryption layers 5 Direct native API call ´ Eloi Vanderb´ken e Hackito Ergo Sum 2011
  • 10. Introduction Verification algorithm Obfuscation Encryption layers Instruction mutation Direct native API call Control Flow Graph (CFG) obfuscation Anti-X How to break it Possible evolutions Instruction mutation Make a static analysis to get freely modifiable registers Use them to mutate instructions Main rule: if your code use an instruction, it has to be used everywhere ⇒ no difference between the useful code and the garbage one ´ Eloi Vanderb´ken e Hackito Ergo Sum 2011
  • 11. Introduction Verification algorithm Obfuscation Encryption layers Instruction mutation Direct native API call Control Flow Graph (CFG) obfuscation Anti-X How to break it Possible evolutions Plan 1 Introduction 2 Verification algorithm 3 Obfuscation Instruction mutation Control Flow Graph (CFG) obfuscation 4 Encryption layers 5 Direct native API call ´ Eloi Vanderb´ken e Hackito Ergo Sum 2011
  • 12. Introduction Verification algorithm Obfuscation Encryption layers Instruction mutation Direct native API call Control Flow Graph (CFG) obfuscation Anti-X How to break it Possible evolutions Control Flow Graph (CFG) obfuscation Before instruction mutation: Duplicate some piece of code Add a fake or non determinist test to choose which block will be executed After: Make a static analysis to know which flags are set/unset for each instruction Add conditional jump which are never/always taken according to the flags Shuffle blocks ´ Eloi Vanderb´ken e Hackito Ergo Sum 2011
  • 13. Figure: (incomplete) CFG of a layer made by IDA
  • 14. Introduction Verification algorithm Obfuscation Encryption layers Direct native API call Anti-X How to break it Possible evolutions Plan 1 Introduction 2 Verification algorithm 3 Obfuscation Instruction mutation Control Flow Graph (CFG) obfuscation 4 Encryption layers 5 Direct native API call 6 Anti-X 7 How to break it 8 Possible evolutions ´ Eloi Vanderb´ken e Hackito Ergo Sum 2011
  • 15. Introduction Verification algorithm Obfuscation Encryption layers Direct native API call Anti-X How to break it Possible evolutions Layers How to make a layer the most annoying possible? Hide the end address of the encryption layer Hide the size of the encrypted code ´ Eloi Vanderb´ken e Hackito Ergo Sum 2011
  • 16. Introduction Verification algorithm Obfuscation Encryption layers Direct native API call Anti-X How to break it Possible evolutions Layers (continued) Use Linear Feedback Shift Register (LFSR) to encrypt n times both end address and a dword counter At execution, decrypt them one time at each loop execution When the dword counter has been decrypted, jump to the decrypted address Use indirect jump to avoid breakpoint: cmp counter, CST setz cl jmp [ecx + addresses] Bonus: LFSR having the biggest possible period, you can add as many fake comparison as you want ´ Eloi Vanderb´ken e Hackito Ergo Sum 2011
  • 17. Introduction Verification algorithm Obfuscation Encryption layers Direct native API call Anti-X How to break it Possible evolutions Plan 1 Introduction 2 Verification algorithm 3 Obfuscation Instruction mutation Control Flow Graph (CFG) obfuscation 4 Encryption layers 5 Direct native API call 6 Anti-X 7 How to break it 8 Possible evolutions ´ Eloi Vanderb´ken e Hackito Ergo Sum 2011
  • 18. Introduction Verification algorithm Obfuscation Encryption layers Direct native API call Anti-X How to break it Possible evolutions Direct native API call Get syscall numbers at initialisation Use sysenter (do not work on 64bit systems) Make random invalid or non invasives (ZwYieldExecution) syscalls to prevent conditional breakpoints on KiFastSystemCallRet ´ Eloi Vanderb´ken e Hackito Ergo Sum 2011
  • 19. Introduction Verification algorithm Obfuscation Encryption layers Direct native API call Anti-X How to break it Possible evolutions Plan 1 Introduction 2 Verification algorithm 3 Obfuscation Instruction mutation Control Flow Graph (CFG) obfuscation 4 Encryption layers 5 Direct native API call 6 Anti-X 7 How to break it 8 Possible evolutions ´ Eloi Vanderb´ken e Hackito Ergo Sum 2011
  • 20. Introduction Verification algorithm Obfuscation Encryption layers Direct native API call Anti-X How to break it Possible evolutions Classical anti-debuggers tricks Well known anti-debuggers trick made difficult to bypass because of direct use of sysenter: NtQueryInformationProcess + ProcessDebugPort NtSetInformationThread + ThreadHideFromDebugger NtSetContextThread to delete hardware breakpoints ´ Eloi Vanderb´ken e Hackito Ergo Sum 2011
  • 21. Introduction Verification algorithm Obfuscation Encryption layers Direct native API call Anti-X How to break it Possible evolutions Anti-instrumentation framework Detect hook on KiUserExceptionDispatcher and KiUserCallbackDispatcher Detect stack reuse by instrumentation code: place a constant in esp-4, execute a sysenter, check if esp-4 has been changed ´ Eloi Vanderb´ken e Hackito Ergo Sum 2011
  • 22. Introduction Verification algorithm Obfuscation Encryption layers Direct native API call Anti-X How to break it Possible evolutions Anti data-tainting After execution of each part of the algorithm, result is copied using indirect dependance AntiTaintMemcpy (BYTE* source, BYTE *dest, int size) { for (int t =0 t < size ++t) { BYTE val = 0 do { BOOL isDiff = FALSE if (source[t] != val) isDiff = True if (! isDiff) dest[t] = val ++val } while (val != 0) } } ´ Eloi Vanderb´ken e Hackito Ergo Sum 2011
  • 23. Introduction Verification algorithm Obfuscation Encryption layers Direct native API call Anti-X How to break it Possible evolutions Plan 1 Introduction 2 Verification algorithm 3 Obfuscation Instruction mutation Control Flow Graph (CFG) obfuscation 4 Encryption layers 5 Direct native API call 6 Anti-X 7 How to break it 8 Possible evolutions ´ Eloi Vanderb´ken e Hackito Ergo Sum 2011
  • 24. Introduction Verification algorithm Obfuscation Encryption layers Direct native API call Anti-X How to break it Possible evolutions So how to solve it? API init Set memory breakpoint on ntdll PE Anti-Dbg Don’t use debugger, inject a DLL and use VectoredExceptionHandler to set HBP Anti-HBP Replace ZwSetThreadContext syscall number by an harmless or invalid one ReadFile Use kernel debugger or replace syscall numbers by a GDI one which will call KiUserCallbackDispatcher Obfuscation Step the code :) ´ Eloi Vanderb´ken e Hackito Ergo Sum 2011
  • 25. Introduction Verification algorithm Obfuscation Encryption layers Direct native API call Anti-X How to break it Possible evolutions How to find a valid serial Bruteforce (JB B´drune) e Use of the RC4 internal table created with the key published (Kyriakos Economou and Mark Wodrich) Generate a valid keystream, use a backward algorithm to find an internal table which generate this keystream, find a key which generate this table (Me :) ) ´ Eloi Vanderb´ken e Hackito Ergo Sum 2011
  • 26. Introduction Verification algorithm Obfuscation Encryption layers Direct native API call Anti-X How to break it Possible evolutions Plan 1 Introduction 2 Verification algorithm 3 Obfuscation Instruction mutation Control Flow Graph (CFG) obfuscation 4 Encryption layers 5 Direct native API call 6 Anti-X 7 How to break it 8 Possible evolutions ´ Eloi Vanderb´ken e Hackito Ergo Sum 2011
  • 27. Introduction Verification algorithm Obfuscation Encryption layers Direct native API call Anti-X How to break it Possible evolutions Some ideas for the next year ;) Recode all the crackme generation in C Use XED (by intel) to disassemble and reassemble instruction Multi-thread Virtualisation More mutation rules, more complex instructions (FPU, MMX, SSE etc.) More anti-X ´ Eloi Vanderb´ken e Hackito Ergo Sum 2011
  • 28. Introduction Verification algorithm Obfuscation Encryption layers Direct native API call Anti-X How to break it Possible evolutions Thanks Thank you ! ´ Eloi Vanderb´ken e Hackito Ergo Sum 2011
  • 29. Introduction Verification algorithm Obfuscation Encryption layers Direct native API call Anti-X How to break it Possible evolutions Questions ? Questions ? ´ Eloi Vanderb´ken e Hackito Ergo Sum 2011