Preventing cpu side channel attacks with kernel tracking
0 likes366 views
This document discusses potential approaches to prevent CPU side-channel attacks like Meltdown. It proposes monitoring processes for multiple SIGSEGV children and stopping rather than killing any processes that meet this criteria. It also discusses limiting the cache flush instruction to prevent Flush+Reload and Flush+Flush attacks, and monitoring CPU performance counters and kernel events related to cache flushing.
![What we had to change?What we had to change?
– Introduced a per process counter of Introduced a per process counter of
its SIGSEGV childrenits SIGSEGV children
unsigned int pids[MAX_PID];unsigned int pids[MAX_PID];
– implement a workqueue to check for implement a workqueue to check for
abusersabusers
create_singlethread_workqueue()create_singlethread_workqueue()
– implement a implement a /proc/proc interface to monitor interface to monitor
and change the max segfaultsand change the max segfaults](https://image.slidesharecdn.com/preventingcpuside-channelattackswithkerneltracking-opensourcesummit-181023030909/85/Preventing-cpu-side-channel-attacks-with-kernel-tracking-32-320.jpg){kind=tracking}
Preventing cpu side channel attacks with kernel tracking
- 1. Preventing Preventing CPU sidechannel CPU sidechannel attacks with attacks with kernel trackingkernel tracking Marian MarinovMarian Marinov mm@siteground.commm@siteground.com Chief System ArchitectChief System Architect Head of the DevOps departmentHead of the DevOps department
- 2. ❖❖ Who am I?Who am I? - Chief System Architect of SiteGround.com- Chief System Architect of SiteGround.com - Sysadmin since 1996- Sysadmin since 1996 - Organizer of OpenFest, BG Perl- Organizer of OpenFest, BG Perl Workshops, LUG-BG and othersWorkshops, LUG-BG and others - Teaching Network Security and- Teaching Network Security and Linux System AdministrationLinux System Administration courses in Sofia Universitycourses in Sofia University and SoftUniand SoftUni
- 3. ❖❖ DisclaimerDisclaimer What I'm proposing is NOT aWhat I'm proposing is NOT a general purpose solution!general purpose solution!
- 4. ❖❖ DisclaimerDisclaimer We are a shared hostingWe are a shared hosting provider... we consider allprovider... we consider all code, hostilecode, hostile
- 5. ❖❖ DisclaimerDisclaimer We haven't seen MeltdownWe haven't seen Meltdown attempts on ourattempts on our infrastructureinfrastructure
- 9. There are There are 44** different different caches inside the CPUcaches inside the CPU L1 instructions cache L2 cache L3 cache L1 I cache L1 D cache L2 cache L1 I cache L1 D cache L2 cache L1 data cache L1 I cache L2 Single Core Single CoreSingle Core Single C
- 10. There are There are 44** different different caches inside the CPUcaches inside the CPU L1 instructions cache L2 cache L3 cache L1 I cache L1 D cache L2 cache L1 I cache L1 D cache L2 cache L1 data cache L1 I cache L2 Single Core Single CoreSingle Core Single C * In some architectures, there is even L4 cache
- 11. L1 and L2 caches are shared between hyper-threads in a single core L2 cache is shared between different execution engines inside the core (ALU, FMA, ADD, etc.) L3 cache is shared between all cores Sharing the cacheSharing the cache
- 12. Shared L3 Cache (LLC) Synchronization L1 Instruction cache Branch Predict.Isnt. Fetch Pipeline(s) Instruction decoder Dispatch Integer Cluster 2FPU W.C. Cache L1 Instruction cache L1 data cache Integer Cluster 1 L1 data cache L2 Data Cache shared Core Iface Single Core L1 Instruction cache Branch Predict.Isnt. Fetch Pipeline(s) Instruction decoder Dispatch Integer Cluster 2FPU W.C. Cache L1 Instruction cache L1 data cache Integer Cluster 1 L1 data cache L2 Data Cache shared Core Iface Single Core L1 Instruction cache Isnt. Instruct Di W.C L1 Instruction cache Integer Cluster 1 L1 data cache LCore Iface Some CPU architecture intro :) AMD Bulldozer block diagram
- 13. Cache SideChannel Attacks ➢ 2013 Flush + Reload ➢ 2016 Flush + Flush
- 14. Flush + ReloadFlush + Reload L1 Instruction cache Branch Predict.Isnt. Fetch Pipeline(s) Instruction decoder Dispatch Integer Cluster 2FPU W.C. Cache L1 Instruction cache L1 data cache Integer Cluster 1 L1 data cache L2 Data Cache shared Core Iface Single Core Shared L3 Cache (LLC) Synchronization 1. Find a shared library location in memory 2. Clear the cache 3. Check if the victim has accessed it or not by comparing the time it takes to execute the code
- 15. Flush + FlushFlush + Flush L1 Instruction cache Branch Predict.Isnt. Fetch Pipeline(s) Instruction decoder Dispatch Integer Cluster 2FPU W.C. Cache L1 Instruction cache L1 data cache Integer Cluster 1 L1 data cache L2 Data Cache shared Core Iface Single Core Shared L3 Cache (LLC) Synchronization 1. Find a shared library location in memory 2. Clear the cache 3. Clear the cache again and observe the timing if the victim has accessed the code, clflush will take longer to finish
- 16. More architecture... Floating Point L1 D-Cache D-TLB Schedulers Integer μop queues Decoder Trace Cache Rename/Alloc μop ROMBTB BTB and I-TLB BusL2CacheandControl Thread 1: floating point
- 17. More architecture... Floating Point L1 D-Cache D-TLB Schedulers Integer μop queues Decoder Trace Cache Rename/Alloc μop ROMBTB BTB and I-TLB BusL2CacheandControl Thread 1: integer Thread 2: floating point
- 20. Capsule 8 approachCapsule 8 approach Kernel tracepoints and monitor Kernel tracepoints and monitor for:for: exceptions/page_fault_userexceptions/page_fault_user Kernel perf countersKernel perf counters – PERF_COUNT_HW_CACHE_OP_READPERF_COUNT_HW_CACHE_OP_READ – PERF_COUNT_HW_CACHE_RESULT_ACCESSPERF_COUNT_HW_CACHE_RESULT_ACCESS – PERF_COUNT_HW_CACHE_RESULT_MISSPERF_COUNT_HW_CACHE_RESULT_MISS https://github.com/capsule8/capsule8/tree/master/examples
- 24. Fight the requirements not the attacks ➢ Successful meltdown exploitation prefers that both the SIGSEGV children and the victim are on the same CPU ➢ so we simply LIE to sched_setaffinity ➢ effectively we do nothing ➢ we save the requested affinity in the task_struct as cpumask_t cpus_allowed;cpumask_t cpus_allowed; ➢ we have patched sched_getaffinitysched_getaffinity to report only the cpu mask already stored for the current process
- 25. Fight the requirements not the attacks ➢ Successful meltdown exploitation requires that a process should have one of the following: ➢ SIGSEGV children or grandchildren ➢ SIGSEGV threads ➢ TSX instructions that do not finish successfully
- 26. Fight the requirements not the attacks ➢ On our infrastructure, there is no customer's software that has a valid case to have ➢ SIGSEGV children or threads ➢ our CPUs do not support TSX instructions :)
- 31. Kernel moduleKernel module – detecting processes that had more then detecting processes that had more then 1 child dying with SIGSEGV1 child dying with SIGSEGV – when such process is detected it is when such process is detected it is STOPPED, not KILLEDSTOPPED, not KILLED – only the root on the host machine can only the root on the host machine can send any type of signals to this send any type of signals to this processprocess What we did?What we did?
- 32. What we had to change?What we had to change? – Introduced a per process counter of Introduced a per process counter of its SIGSEGV childrenits SIGSEGV children unsigned int pids[MAX_PID];unsigned int pids[MAX_PID]; – implement a workqueue to check for implement a workqueue to check for abusersabusers create_singlethread_workqueue()create_singlethread_workqueue() – implement a implement a /proc/proc interface to monitor interface to monitor and change the max segfaultsand change the max segfaults
- 34. Cache Line FlushCache Line Flush – Limiting clflush and clflushopt Limiting clflush and clflushopt effectively stops Flush+Reload and effectively stops Flush+Reload and Flush+Flush attacksFlush+Flush attacks – cache flush can be indirectly called cache flush can be indirectly called when invalid instruction is issuedwhen invalid instruction is issued – This greatly limits the options for This greatly limits the options for executing Meltdown, by leaving only executing Meltdown, by leaving only TSX instructionsTSX instructions
- 37. TSXTSX Transactional Synchronization Transactional Synchronization eXtensions (TSX)eXtensions (TSX) – Because of issues with the Because of issues with the implementation, TSX instructions implementation, TSX instructions should be disabled on Haswell CPUsshould be disabled on Haswell CPUs – However if the microcode is not However if the microcode is not applied, your Haswell CPUs support TSX applied, your Haswell CPUs support TSX :):) – TSX instructions are supported on TSX instructions are supported on Skylake...Skylake...
- 41. TSX eventsTSX events – RTM_RETIRED.STARTRTM_RETIRED.START Number of times we Number of times we entered an RTM region. Does not count nested entered an RTM region. Does not count nested transactionstransactions – RTM_RETIRED.ABORTEDRTM_RETIRED.ABORTED Number of times an Number of times an RTM execution aborted due to any reasons RTM execution aborted due to any reasons (multiple categories may count as one)(multiple categories may count as one) – RTM_RETIRED.ABORTED_TIMER Number of RTM_RETIRED.ABORTED_TIMER Number of times an RTM execution aborted due to times an RTM execution aborted due to uncommon conditionsuncommon conditions
- 42. you can find the list of Processor Monitor Unit (PMU) events by running: # perf list Perf can be build from the linux kernel source tree in tools/perf: # make # mv perf /usr/bin TSX eventsTSX eventsTSX eventsTSX eventsTSX eventsTSX events
- 44. LinksLinks Flush + Reload paper Flush + Flush paper EndGame research Capsule 8 meltdown detection Spectre & Meltdown attacks Meltdown PoC Collection of Speculation bugs info Forshadow attacks TLBleed attack