Enterprise Strategy Group: The Big Data Security Analytics Era is Here
- 1. White Paper Big Data Security Analytics Era Is The Here By Jon Oltsik, Senior Principal Analyst January 2013 This ESG White Paper was commissioned by RSA Security and is distributed under license from ESG. © 2013 by The Enterprise Strategy Group, Inc. All Rights Reserved
- 2. White Paper: The Big Data Security Analytics Era Is Here 2 Contents Executive Summary ..................................................................................................................................... 3 The Obstacles to Improving Organizational Security Maturity ................................................................... 3 Legacy Security Monitoring and Analytics Tools Are Also Holding Back Progress ...................................... 6 Enter the Big Data Security Analytics Era .................................................................................................... 8 Big Data Security Analytics Technology Transformation ......................................................................................... 9 CISOs Must Become Big Data Security Advocates .................................................................................... 10 The Bigger Truth ....................................................................................................................................... 12 All trademark names are property of their respective companies. Information contained in this publication has been obtained by sources The Enterprise Strategy Group (ESG) considers to be reliable but is not warranted by ESG. This publication may contain opinions of ESG, which are subject to change from time to time. This publication is copyrighted by The Enterprise Strategy Group, Inc. Any reproduction or redistribution of this publication, in whole or in part, whether in hard-copy format, electronically, or otherwise to persons not authorized to receive it, without the express consent of the Enterprise Strategy Group, Inc., is in violation of U.S. copyright law and will be subject to an action for civil damages and, if applicable, criminal prosecution. Should you have any questions, please contact ESG Client Relations at 508.482.0188. © 2013 by The Enterprise Strategy Group, Inc. All Rights Reserved.
- 3. White Paper: The Big Data Security Analytics Era Is Here 3 Executive Summary A few years ago, ESG created a security management maturity model that outlined a progression through four phases of a security management program’s evolution. The goal was to leverage ESG research to uncover success strategies and best practices, then use this information to help CISOs build a security management plan and prioritize the right activities in order to improve security and lower risk, while continuing to build the organization’s security maturity. CISOs are certainly intent on evolving the maturity of their security management, but many organizations are facing unanticipated problems that are impeding their progress. CISOs face an insidious threat landscape and an avalanche of new technology initiatives that make security management increasingly difficult. Furthermore, enterprise organizations are finding it difficult to recruit and train new security professionals—leaving them under-‐ staffed and over-‐burdened. Taken together, new security risks and old security challenges often overwhelm legacy security controls and analytics tools. Large organizations can no longer rely on preventive security systems, point security tools, manual processes, and hardened configurations to protect them from targeted attacks and advanced malware. Henceforth, security management must be based upon continuous monitoring and data analysis for up-‐to-‐the-‐minute situational awareness and rapid data-‐driven security decisions. This means that large organizations have entered the era of big data security analytics. This white paper concludes that: • Security and market trends are creating new security management hurdles. Over the past few years, CISOs have come face-‐to-‐face with three difficult and converging trends. First, they face an increasingly hazardous threat landscape full of stealthy malware, social engineering, and targeted attacks from well-‐ funded and expert adversaries. Second, they have been called upon to secure new technology initiatives such as cloud computing, mobile devices, and server virtualization. Finally, they face a security skills shortage, making it difficult to recruit and hire new security talent. These obstacles are placing new demands on existing security staff, processes, and technologies. • The existing security infrastructure is no longer adequate. At many enterprise organizations, security protection and analysis depends upon an army of independent signature-‐based point tools, network perimeter gateways, manual processes, and specialized skills. While this loose affiliation of security technologies may have been sufficient in years past, they are no match for the scale and scope of today’s threats and overall security management requirements. • IT is entering the era of big data security analytics. Risk management and prevention are critical but no longer enough. Moving forward, CISOs need real-‐time security intelligence and situational awareness to give them visibility into their security status at all layers of the technology stack and across the enterprise. Armed with this type of intelligence, security executives can then prioritize actions, adjust security controls, accelerate incident detection, and improve workflows around incident response. Taken together, these advances can improve security while lowering security operations costs. The Obstacles to Improving Organizational Security Maturity After studying the state of enterprise information security in 2011, ESG published a security management maturity model to provide some strategic guidance for CISOs (see Figure 1). At that time, ESG believed that most organizations were still in phase 2, thus focused on compliance and defense-‐in-‐depth, but were intent on proceeding to phase 3, risk-‐based security, as soon as possible. © 2013 by The Enterprise Strategy Group, Inc. All Rights Reserved.
- 4. White Paper: The Big Data Security Analytics Era Is Here 4 Figure 1. The ESG Information Security Management Maturity Model Source: Enterprise Strategy Group, 2013. When this model was first published in 2011, ESG assumed that risk-‐based security would be well established by most organizations by early 2013, but this transition has proven to be more difficult than first anticipated. The delay is not due to a lack of effort by security teams. In fact, in the past couple of years, many CEOs and other non-‐ security executives have become more involved in information security oversight and are regularly approving projects and increasing information security budgets. Unfortunately, the transition from phase 2 to 3 for most organizations has become more difficult than projected because of: • The volume and sophistication of new threats. While day-‐to-‐day cyber threats continue to increase at an exponential rate, CISOs are most concerned over the rise of targeted and advanced malware enabled attacks such as Advanced Persistent Threats (APTs). This apprehension is well deserved. According to ESG research, 59% of enterprises are certain or fairly certain that they have been the target of an APT, while 30% of enterprises believe they are vulnerable to future APTs.1 Detecting, analyzing, and remediating advanced threats adds additional requirements to the risk-‐based phase while forcing CISOs to simultaneously assess and dramatically improve their incident detection and response capabilities. • Rapid IT changes. Risk-‐based security depends upon intimate knowledge of every IT asset deployed on the network. This type of understanding is especially difficult when IT is constantly engaged in rolling out new initiatives such as server/endpoint virtualization, cloud computing, mobile device support, and supporting BYOD programs. To make matters worse, many new IT initiatives are based upon immature technologies that are prone to security vulnerabilities, and may not play well with existing security policies, controls, or monitoring tools. For example, mobile devices like smartphones and tablet computers present a number of security management challenges around policy enforcement, sensitive data discovery/management, and malware/threat management (see Figure 2).2 The continuous adoption of new technology initiatives adds uncertainty and complexity to security management. 1 Source: ESG Research Report, U.S. Advanced Persistent Threat Analysis, November 2011. 2 Source: ESG Research Report, Security Management and Operations: Changes on the Horizon, July 2012. © 2013 by The Enterprise Strategy Group, Inc. All Rights Reserved.
- 5. White Paper: The Big Data Security Analytics Era Is Here 5 Figure 2. Mobile Device Security Challenges With regard to mobile device security, which of the following presents the most significant security challenges for your organizaPon? (Percent of respondents, N=315, mulPple responses accepted) Enforcing security policies for mobile devices 48% Lost/stolen mobile devices containing sensieve data 46% Sensieve data confideneality and integrity proteceon 46% when accessed from or stored on mobile devices Malware/threat management on mobile devices 41% Supporeng new device types 41% Creaeng security policies for mobile devices 40% Discovering mobile devices as they gain access to the 30% network 0% 10% 20% 30% 40% 50% 60% Source: Enterprise Strategy Group, 2013. • A growing security skills shortage. In 2012, over half of all organizations planned to add headcount to their information security group and nearly one-‐quarter of all organizations (23%) indicated that they had a significant shortage of security skills. CISOs will likely find it extremely difficult to simply hire their way out of this problem—ESG research indicates that 83% of enterprise organizations find it extremely difficult or somewhat difficult to recruit and hire security professionals.3 Combined with routine day-‐to-‐day activities, the security market trends described above have led to numerous challenges in areas such as incident detection/response (see Figure 3).4 For example, the overall security skills shortage has an impact on the security organization’s incident detection/response capabilities because many enterprises lack the right staffing levels and skills. Malware volume and sophistication is forcing security analysts to sort through mountains of equally weighted, false positive alerts. In addition to staffing and skills issues, security analysts generally rely on too many manual processes in order to identify, scope, and remediate problems. 3 Source: Ibid. 4 Source: ESG Research Report, The Emerging Intersection Between Big Data and Security Analytics, November 2012. © 2013 by The Enterprise Strategy Group, Inc. All Rights Reserved.
- 6. White Paper: The Big Data Security Analytics Era Is Here 6 Figure 3. Challenges with Incident Detection Which of the following challenges does your organizaPon face when it comes to incident detecPon? (Percent of respondents, N=257, mulPple responses accepted) Lack of adequate staffing in security operaeons/ 39% incident response team(s) Too many false posieve responses 35% Incident deteceon depends upon too many manual 29% processes Incident deteceon depends upon too many 29% independent tools that aren’t integrated together Sophisecated security events have become too 28% hard to detect for us My organizaeon lacks the right level of security 28% analysis skills needed Lack of adequate data colleceon/monitoring in one 28% or more criecal area Lack of proper level of tuning of our SIEM and other 23% security tools 0% 10% 20% 30% 40% 50% Source: Enterprise Strategy Group, 2013. What’s most alarming here is that the challenges outlined in Figure 3 have a cumulative impact. Security departments are short-‐staffed and lack the right skills amongst the analysts they do have. Meanwhile, security analysts spend an inordinate amount of time sorting through false positives and working through manual processes, which wastes what little time they have. In aggregate, this situation is operationally inefficient, costly, and leaves many enterprise firms with an unacceptable level of risk. The CEO and CFO won’t be pleased to learn that they spend more but are left with more risk. Legacy Security Monitoring and Analytics Tools Are Also Holding Back Progress In addition to skills challenges, false positives, and manual processes, it is also worth noting that 29% of enterprise organizations surveyed by ESG indicate that incident detection depends upon too many independent tools that aren’t integrated together.5 This security challenge is certainly understandable. Over the past ten years, enterprise IT security has grown incrementally more difficult because of new and unanticipated threats and vulnerabilities. As these changes occurred in the past, organizations typically upgraded their security products, purchased new signature-‐based threat management tools, created new rules for perimeter gateways, and increased their security analytics activities. Over time, this has led to a security infrastructure anchored by numerous disconnected point tools for incident detection/response. Tactically driven enterprise IT security has always suffered from operational inefficiencies, but even with this it provided reasonably adequate protection against threats such as general purpose malware, spam, and amateur hackers. Unfortunately, existing security systems, which are often perimeter and signature based, are no match for today’s insidious threat landscape. This is especially true with regard to security analysis tools because: 5 Source: Ibid. © 2013 by The Enterprise Strategy Group, Inc. All Rights Reserved.
- 7. White Paper: The Big Data Security Analytics Era Is Here 7 • Security analytics tools can’t keep up with today’s data collection and processing needs. According to ESG research, 47% of enterprise organizations collect, process, and analyze more than 6 terabytes of security data on a monthly basis. Additionally, the majority of enterprises collect, process, store, and analyze more security data than they did two years ago (see Figure 4).6 And this data remains online for longer periods of time. These trends will continue—security-‐driven enterprises will regularly collect, process, and analyze petabytes of online security data for analysis, investigations, and modeling. Legacy Security Information and Event Management (SIEM) platforms are often based upon off-‐the-‐shelf SQL databases or proprietary data stores that simply can’t scale for this type of data volume. As this happens, security analytics needs are hamstrung by basic technology limitations. This creates a Faustian compromise where security technology deficiencies ironically slow down incident detection/response, limit investigations, and increase IT risk. Figure 4. Growth in Amount of Data Collected for Information Security Activities How has the amount of data your organizaPon collects to support its informaPon security acPviPes changed in the last 2 years? (Percent of respondents, N=257) We collect about the same amount of data to support our informaeon security aceviees today as we did 2 years ago, We collect 14% substaneally more data to support our informaeon security aceviees today than we did 2 years ago, 43% We collect somewhat more data to support our informaeon security aceviees today than we did 2 years ago, 43% Source: Enterprise Strategy Group, 2013. • Organizations need an enterprise-‐wide security purview. Security analytics point tools tend to provide monitoring and investigative capabilities against explicit types of threats (i.e., network threats, malware threats, application-‐layer threats, etc.) or specific IT infrastructure locations (i.e., data center, campus network, remote offices, host etc.). This forces CISOs to piece together an aggregated view of enterprise security through numerous tools, reports, and individual security personnel. This methodology is cumbersome, labor-‐intensive, and can’t really provide an accurate picture of risk or an incident detection/response across networks, servers, operating systems, applications, databases, storage, and endpoint devices scattered throughout the enterprise. • Existing security analysis tools depend excessively on customization and human intelligence. Enterprise security analysis is complex and requires specialized skills and strong experience. As stated previously however, these skills are in short supply—even the most security-‐conscious enterprises are finding it 6 Source: Ibid. © 2013 by The Enterprise Strategy Group, Inc. All Rights Reserved.
- 8. White Paper: The Big Data Security Analytics Era Is Here 8 difficult to continuously train their security staff or hire new recruits. Regrettably, it seems that many security analytic systems were designed to be used only by advanced security analysts who have the time and skills to constantly fine-‐tune and customize these tools, and who know exactly what to look for. Over-‐ burdened security professionals desperately need security tools that provide more intelligence rather than more work. • Analytics aren’t integrated for automated incident response. For the most part, today’s security analytics tools remain independent from security remediation systems. This often means that without automation, what is found isn’t fixed quickly or reliably. Therefore, when an analyst detects a problem, she still must manually coordinate remediation activities and workflow with other security or IT operations personnel. Once again, this adds operational overhead and extends the timeframe needed for incident response which could mean the difference between a minor security event and a major breach. And this problem only gets worse when breach responses need to include non-‐IT organizations such as legal, HR, and business owners. Enter the Big Data Security Analytics Era At the beginning of WWI, Allied troops executed tactics used during the American Civil War—overwhelm your enemy by advancing a large army rapidly. Unfortunately, this proved to be a costly mistake. Why? With the invention of the machine gun, these tactics resulted in massive loss of life rather than battlefield success. Technology advances like the machine gun force combatants to adopt new warfare strategies and tactics. This same lesson applies to the cybersecurity battlefield. As cyber criminals and state-‐sponsored adversaries advance their capabilities with targeted attacks, social engineering, stealthy malware, and application-‐layer exploits, enterprises have no choice but to adopt new strategies and defenses. ESG believes that these new requirements will result in an enterprise security technology transition over the next few years. Yes, organizations will continue to employ preventive tactics such as deploying servers in hardened configurations behind firewalls, removing unnecessary services and generic administrator accounts, scanning for known malware using signatures, and patching software vulnerabilities, but used alone these defensive techniques are not enough. To supplement these security practices, organizations will embrace new security analytics tools for continuous monitoring, investigations, risk management, and incident detection/response. Given the volume of security data collection, processing, storage, and analysis involved, security analytics is rapidly becoming a classic “big data” problem. In fact, ESG research indicates that 44% of enterprises consider security data collection and analysis big data today, while another 44% believe that security data collection and analysis will become big data within the next 24 months (see Figure 5). 7 7 Source: Ibid. © 2013 by The Enterprise Strategy Group, Inc. All Rights Reserved.
- 9. White Paper: The Big Data Security Analytics Era Is Here 9 Figure 5. Security Data Collection and Analysis Considered “Big Data” Do you believe that security data collecPon and analysis would be considered “big data” at your organizaPon? (Percent of respondents, N=257) No, security data colleceon and analysis is not considered “big Don’t know, 2% data” within my organizaeon, 11% Yes, security data colleceon and analysis would be considered No, but based on my “big data” within my organizaeon’s security organizaeon today, strategy we will likely 44% consider security data colleceon and analysis “big data” within the next 24 months, 14% No, but based on my organizaeon’s security strategy we will likely consider security data colleceon and analysis “big data” within the next 12 months, 30% Source: Enterprise Strategy Group, 2013. To be clear, big data security analytics isn’t a simple merger of events, logs, and network traffic in big data technologies such as Cassandra and Hadoop (although these underlying technologies may play a role in the technology infrastructure of a solution). To ESG, big data security is really about collecting and processing numerous internal and external security data sources, and analyzing this data immediately to gain real-‐time situational awareness across the enterprise. Once security data is analyzed, the next step is using this new intelligence as a baseline for adjusting security strategies, tactics, and systems, much faster than ever before. Big Data Security Analytics Technology Transformation Ultimately, the objective of big data security analytics is to provide a comprehensive and up-‐to-‐the-‐second view of IT activities so that security analysts and executives can make timely, data-‐driven decisions. From a technology perspective, this will require new security systems providing: • Massive scale. Security analytics and forensics engines will need to efficiently collect, process, query, and apply analytic rules to terabytes or petabytes of data including logs, network packets, threat intelligence, asset information, sensitive data tracking, known vulnerabilities, application activities, and user behavior. This is why core big data technologies such as Hadoop, an open source software project for distributed processing of extremely large data sets across commodity servers, is a good fit for burgeoning security analytics requirements. Additionally, big data security analytics will likely be deployed in a distributed architecture, thus the underlying technology must be able to centralize analysis of massive volumes of distributed data while maintaining data integrity and providing for high-‐performance needs. © 2013 by The Enterprise Strategy Group, Inc. All Rights Reserved.
- 10. White Paper: The Big Data Security Analytics Era Is Here 10 • Enhanced intelligence. The best big data security analytics tools will act as intelligent advisors, leveraging models of normal behavior, adapting to new threat/vulnerability intelligence, and pinpointing anomalies at any layer of the technology stack that requires immediate investigation. To accomplish this, big data security analytics will offer a combination of templates, heuristics, statistical and behavior models, correlation rules, threat intelligence feeds, etc. • Tight integration. To keep up with the constantly changing threat landscape, big data security analytics must interoperate with IT assets and leverage automated security intelligence. Beyond this, however, big data security analytics should be tightly integrated with security policy controls for tactical adjustments and automation. When security analytics point to unusual network traffic emanating from mobile devices, security analysts should be provided with specific change instructions to quarantine traffic flows and minimize risk. Ideally, security analytics systems can be used to automate remediation activities, a form of active defense, for routine changes or in emergency situations. Armed with a comprehensive real-‐time view of security situational awareness, big data security analytic systems will become the nexus for both risk management and incident detection/response. This includes specialized security activities such as regulatory compliance, security investigations, control tracking/reporting, and security performance metrics. CISOs Must Become Big Data Security Advocates Big data security analytics is no longer a visionary idea—leading enterprises recognize that their immediate security requirements demand this type of solution. To proceed with big data security analytics planning and implementation, ESG suggests that CISOs: • Address limitations with existing security infrastructure. Compare security analytics output with existing capabilities, processes, and requirements. Does your organization have “blind spots”? Is the organization conducting continuous monitoring or basing its security assessments on periodic (occasional) scans? Is the organization understaffed or lacking security analytics skills? How long does it take to detect, investigate, and respond to security incidents? Rather than deal with security analytics weaknesses piecemeal, develop a big data security analytics project plan that addresses critical areas through a phased approach. Remember to build processes and technologies that can serve as a foundation for all phases of the project. This should help deliver incremental value throughout. • Shift investments from prevention to detection/remediation. Yes, it is still important to lock down IT assets to minimize risk, but CISOs must realize that despite these best practices, networks will be attacked, penetrated, and compromised. Savvy CISOs will capture incident detection/response metrics (i.e., time to discover a security incident, time to investigate and remediate a security incident, number of tools used, number of staff hours needed, etc.) before and after a big data security analytics implementation to measure ROI on security operations and risk management goals. • Identify staffing deficiencies and knowledge gaps. As ESG research indicates, most organizations have security organizational problems around skills and headcount. In most cases, CISOs will not be able to hire and train their way out of this problem, so they need alternative strategies. ESG recommends that CISOs clearly identify areas of weakness at the genesis of their big data security analytics planning process. This will help them define their needs for security technology intelligence, external data feeds, and professional/managed security services to fill the gaps. Finally, big data security analytics is antithetical to today’s typical security infrastructure, which is based upon point tools and limited scale. Impending enterprise security technology changes will likely resemble the business application transition in the 1990s when departmental applications were replaced with enterprise-‐class ERP software architectures. To avoid the potential pitfalls associated with this type of evolution, enterprises should seek out technology vendors with deep security experience, a portfolio of leading security analytics products, a strong big data security © 2013 by The Enterprise Strategy Group, Inc. All Rights Reserved.
- 11. White Paper: The Big Data Security Analytics Era Is Here 11 analytics strategy, strong enterprise experience, complementary threat intelligence services, relationships with proven MSSPs, and security-‐focused professional services to help CISOs with planning, deployment, and ongoing big data security analytics management. Particularly with its recent product introduction of RSA Security Analytics, RSA Security is one of only a few security vendors who meet this profile. As such, enterprise CISOs would be well served to assess how RSA Security Analytics and related solutions and services align with their big data security analytics vision, strategy, and tactical plans and requirements. © 2013 by The Enterprise Strategy Group, Inc. All Rights Reserved.
- 12. White Paper: The Big Data Security Analytics Era Is Here 12 The Bigger Truth Enhancing security management maturity is not a straight-‐line process and thus CISOs should expect peaks and valleys as they proceed on this journey. Based upon a few current market trends and ESG research data, it appears as though many organizations are stuck in a security management valley at present. In truth, security management maturity has reached a tipping point. To move forward, CISOs should conduct an honest assessment of their security technology infrastructure. Can it provide the necessary monitoring, investigative, and data analysis to support real-‐time security decisions? Can it collect, process, and analyze the volume of data needed to track security activities at all layers of the technology stack? Does it require unreasonable care and feeding? Regrettably, CISOs may find that they are spending a lot of money for poor incident detection, investigation, response, and workflow results. Given the sophistication of malware threats and cyber criminals, there are no “silver bullets” or easy answers here. What’s needed more than anything is better visibility through improved data analysis—more data, better security intelligence, real-‐time collection and correlation, etc. With real-‐time situational awareness, CISOs and their security analysts can adjust their tactics, prioritize activities, and accelerate processes. Ultimately, this should help enterprises improve security and lower costs. This alone should make big data security analytics exceptionally attractive to enterprise CISOs. © 2013 by The Enterprise Strategy Group, Inc. All Rights Reserved.