A New Approach to Security Control Testing

This is an excerpt from my master thesis project on Testing The Efficacy of Windows Defender Endpoint Security Control Using BAS Technology

I Hope you will enjoy reading it and leave your feedback on the topic.

Data breaches have become a widespread and expensive concern for enterprises worldwide in the quickly changing cyber threat landscape of today. This paper examines the substantial impact of misconfigurations, frequently caused by human errors and insufficient security measures, in contributing to catastrophic breaches. Utilising information from recent case studies and industry reports, such as the 2023 Data Breach Investigations Report, the research emphasises how misconfigurations enable unauthorised access and the exploitation of vulnerabilities. The study highlights the importance of ongoing and effective configuration management and continual security validation. It suggests utilising modern Breach and Attack Simulation (BAS) tools to automate and improve the process of testing security control capabilities. Moreover, the incorporation of AttackIQ's Flex platform, which provides complimentary and sophisticated adversary simulation and security control testing, is positioned as a cost-efficient and easily attainable solution for enhancing organisational security. This research places emphasis on the significance of proactive and well-informed security management. It offers practical suggestions for reducing the risks associated with misconfigurations and improving overall cybersecurity resilience. The purpose of the research is to provide guidance to cybersecurity professionals, policymakers, and organisations on how to implement steps to protect sensitive information and establish strong security frameworks to protect critical organizational assets.

Introduction

Data breaches (Madnick, 2024) are becoming an increasingly common and expensive problem for businesses all over the world in the quickly changing cybersecurity landscape of today. Central to these breaches are misconfigurations and poor security procedures that expose sensitive data (Potlapally, 2011a), (Cuppens, Boulahia Cuppens and Garcia-alfaro, 2006) which play major role in these attacks. Misconfigurations (Fiebig, 2017) reveal sensitive information and cause significant financial losses as well as harm to one's reputation. They are frequently the consequence of oversight, human mistake (Pernet, 2024) , or inadequate security measures. Numerous reports emphasise how serious this problem is. The 2023 Data Breach Investigations Report (DBIR) (DBIR, 2024) states that human error, including configuration errors (Hope, 2024) that allowed unauthorised access and

vulnerability (Patel, 2019) exploitation, was a factor in 74% of breaches. According to the IBM Cyber Resilient (IBM, 2021) Organisation Study 2021, a similar proportion of organizations—30%—use over fifty different security tools and technologies, frequently from different vendors. This leads to security gaps because there is no single platform, which raises the possibility of misconfigurations (Eshete, Villafiorita and Weldemariam, 2011) and serious data breaches.

A prominent example is the OWASP data breach (Hope, 2024), caused by improper server configuration management, compromised and exposed members' personal information. Through the examination of recent case studies and breach data, this study aims to investigate the role that misconfigurations (Trend Micro, 2021) play in data breaches. By finding recurrent patterns and vulnerabilities (Mejri et al., 2013), it hopes to offer actionable advice for enhancing security measures. Continuous security validation (NGUU and Musuva, 2024); the process of assessing the effectiveness of security controls through continuous testing and evaluation. Security validation ensures that the controls in place are functioning correctly and providing the intended level of protection against threats. Effective configuration management (Towne et al., 2024), and utilising cutting-edge security technologies like Breach and Attack Simulation (BAS) (Kissel and Szurley, 2022) systems; a technology designed to automate and continuously evaluate and improve an organization's security posture by simulating the tactics, techniques, and procedures (TTPs) of real-world adversaries to identify and rectify misconfigurations. These technologies are pivotal in addressing new vulnerabilities and security gaps, providing key insights into how effective security controls are in detecting and responding to attacks in our simulated test environment which is going to be our main focus area in this research. An organization's size and financial resources will determine which BAS system (Master, Hamilton and Dietz, 2022) is best for it because these systems may automate (Lerums, Poe and Dietz, 2018) the security controls testing process, freeing up security professionals to focus on other important duties.

In support of this proactive approach (Lamichhane, Hong and Shetty, 2018), AttackIQ's Flex (AttackIQ, 2024b) platform offers advanced adversary emulation; the practice of emulating potential cyber attackers to test an organization's defenses. This helps in identifying vulnerabilities and improving incident response strategies by mimicking the behaviours of known threat actors and security control testing; a measure implemented to safeguard information systems by reducing risks to acceptable levels. These controls can be technical, administrative, or physical, and are designed to protect the confidentiality, integrity, and

availability of information. By providing perpetual access to baseline tests and advanced attack emulations (MITRE Engenuity, 2024), the platform simplifies security testing and reduces costs. Flex (AttackIQ, 2024a) also includes packages designed to enhance security against Command and Control (C2) communications and assess Next-Generation Firewall (NGFW) (Shin et al., 2023) effectiveness using packet capture replay technology. This innovation allows organizations to evaluate their security controls comprehensively and effectively, without the need for costly and disruptive traditional testing methods. The introduction of these free and accessible tools underscores the importance of continuous security validation and highlights the potential for widespread improvement in cybersecurity practices.

Conventional techniques for assessing the efficacy of endpoint security (SentinelOne, 2024) products (Sentinelone, 2024) are frequently ineffective, creating holes that expose enterprises to cunning and sophisticated attackers. Correcting misconfigurations (Eshete, Villafiorita and Weldemariam, 2011) is essential since the danger increases (Poptani and Gatty, 2018) with the growing reliance on cloud and sophisticated IT systems, which calls for proactive and knowledgeable security management. To improve overall organisational security, this study employs the MITRE ATT&CK framework (MITRE ATT&CK®, 2024b), a project by the MITRE Engenuity (MITRE Engenuity, 2024) team which seeks to advance the state of the art and threat-informed defence practice, their goal is to create useful tools, procedures, and resources based on MITRE ATT&CK®, enabling cyber defenders to enhance their operations. The MITRE ATT&CK framework is a globally-accessible knowledge based of adversary tactics and techniques (TTPs) based on real-world observations of threat actors.

This research employs the MITRE ATT&CK framework is to help organisations identify and analyze common misconfigurations in security controls. A recent analysis of academic publications and industry reports on misconfigurations mostly point to human errors, the study intends to highlights the significance of automating security control deployment and testing to minimise expensive human errors (Pernet, 2024), (Tunggal, 2023). The study also criticises traditional security testing methods, which frequently take security tools at face value without conducting adequate validation (Towne et al., 2024) . It also emphasises how BAS technology (Kissel and Szurley, 2022) can improve security efficacy and effectiveness against contemporary threats by ensuring that security capabilities are appropriately configured to fend off common threats in the wild.

This research is intended to answer the following questions;

Question i:

What is the effectiveness of Breach and Attack Simulation (BAS) systems in enhancing security control testing and reducing the risk of misconfigurations in complex IT environments?

Question ii:

How can continuous security validation be integrated into existing cybersecurity frameworks to improve organizational resilience against sophisticated attacks?

Question iii:

What are the limitations of conventional endpoint security assessment techniques, and how can modern approaches, such as BAS, address these challenges?

Conclusion and Future Work

This research aimed to evaluate the efficacy of Windows Defender Endpoint Security Controls using Breach and Attack Simulation (BAS) technology, specifically focusing on the AttackIQ Flex platform. The primary objectives were to investigate the reasons behind security controls' failures, analyze the effectiveness of specific adversary strategies against current Windows Defender security protections, and provide actionable recommendations for enhancing security program performance.

Research Question and Objectives

The central research question sought to determine how effectively Windows Defender, as a built-in security solution, could protect against common adversary tactics, techniques, and procedures (TTPs) using a controlled, simulated environment. The objectives included:

Examining past incidents to identify patterns and weaknesses in control detection of anomalies.

Assessing the effectiveness of Windows Defender against emulated adversary behaviours.

Utilizing the AttackIQ Flex platform to conduct advanced adversary emulation and security control testing.

Success in Answering the Research Question and Achieving Objectives

The research successfully answered the central question by demonstrating that while Windows Security performs well in detecting known threats, its effectiveness diminishes against zero-day attacks, particularly in offline scenarios. The study achieved its objectives by thoroughly examining the security gaps and providing detailed recommendations for improvement.

Key Findings

Detection Rates: Windows Defender showed high detection rates for known malware but lower rates for zero-day threats, especially in offline mode.

False Positives: An increase in false positives was observed in online mode due to broader heuristic applications, indicating a need for more precise algorithms.

EDR Performance: EDR systems exhibited robust performance in detecting and mitigating both known and zero-day threats, highlighting their importance in modern cybersecurity frameworks.

Implications of the Research

The findings underscore the necessity for continuous and adaptive security validation to maintain effective defenses against evolving threats. The research demonstrated the value of integrating BAS technology to automate and improve the efficacy of security controls, providing organizations with a proactive approach to threat detection and mitigation.

Efficacy and Limitations

The research effectively highlighted the strengths and weaknesses of Windows Defender, contributing valuable insights into its performance under various conditions. However, limitations include the reliance on specific simulation tools, which may not capture the full spectrum of real-world attack scenarios. Additionally, the high cost of BAS platforms like AttackIQ Flex could be a barrier for smaller organizations.

Proposals for Future Work

Future research should focus on:

Advanced Detection Techniques: Developing and integrating machine learning models to enhance zero-day threat detection and reduce false positives.

Comprehensive Testing: Expanding the scope of simulations to include a wider variety of attack vectors and more diverse operational environments.

Cost-Effective Solutions: Investigating alternative, more affordable BAS tools to make continuous security validation accessible to smaller organizations.

Potential for Commercialization

The development of cost-effective, scalable BAS solutions could significantly enhance the cybersecurity landscape by enabling more organizations to adopt advanced security validation techniques. This research provides a foundation for commercial products that offer robust, automated security testing at a lower cost.

Meaningful Future Work

Follow-up research projects could explore the integration of advanced threat intelligence and behavioural analytics into existing security frameworks. Additionally, conducting extensive studies to assess the long-term effectiveness of adaptive security measures and their impact on organizational resilience would provide deeper insights into the practical applications of this research. Extending the current work to include diverse industry sectors and varying organizational sizes would further validate the findings and enhance their generalizability.