Application Security at DevOps Speed and Portfolio Scale

Editor's Notes

• #3: My name is Jeff Williams. Some you may know me from my work on WebGoat, ESAPI, or the OWASP Top Ten, and a bunch of other open source projects.If any of you are smart, humble, and get things done – we have some amazing job openings at Aspect.And if you never want to wrestle with a static analysis tool again.... Come check out Contrast at our booth – I promise you it’s different!Today I’m going to talk about what I’ve learned helping organizations do application security at DEVOPS SPEED and PORTFOLIO SCALE.
• #4: Imagine applications are people and vulnerabilities are sicknesses.We’ve got a few Doctors and some FANCY technology for them to use -- like Xray or MRI machines.These doctors are helping patients – but they’re reactive. We could have the best doctors in the world working on our patients AND NEVER make progress against the disease.It takes a DIFFERENTAPPROACH to target a disease than it does to help a patient.You can’t just “scale up” what you’re doing for individual patients..
• #5: The healthcare world is undergoing a powerful transformation.On both the individual and population level, SENSORS are changing everything.This is great for patients as they can do their own monitoring.And in the AGGREGATE, this information can fight disease in new powerful ways.
• #6: You might be thinking – well, our tools are pretty good. We just need to be better at running them.Unfortunately, traditional tools have not kept up with modern software development – both technology and processesFor example, most frameworks DON’T call request.getParameter() anymore. Or SQL statement.execute().So what is your static tool going to find? Do they know about every framework and pattern?They have lots of blind spots in the most important areas – like authentication and access control. They can’t handle complex frameworks, complex protocols, the explosion of libraries, or the speed of DevOps.And all the tools require experts, which introduces a serious bottleneck -- so we struggle to help Agile/DevOps type projects.
• #7: I came to a hard realization…. I’m very proud of the progress we’ve made in appsec, but we are getting outpaced. The software guys are out there inventing the next crazy new thing right now. By the time we get involved, it’ll be cast in stone. And we’ll eventually figure out how to break it, and how to secure it… and then it will be too late. Again.So – I’m convinced that the only way forward is:AutomatedContinuous and RealtimeKeep security experts out of the critical path
• #8: So what do we do?We have toGIVE UP on anything that doesn’t work at devops speed or portfolio scale.I’m sorry “expert” – that means your job is going to change. Because software development has changed.
• #9: At the end of the day, the only success metric that matters is whether we’re doing a decent job of protecting all the apps in our portfolios. And even the best programs are nowhere close.Appsec is really more like public health actually. It’s not only about securing apps, it’s about securing a PORTFOLIO.And whether something works for a single application (patient) is almost irrelevant to whether it works across a portfolio.
• #10: We really need this. I’ve worked with a lot of agile and devops projects. They can’t use results that aren’t very timely.If you can’t get developers feedback almost immediately, the cost skyrockets and the learning plummets.I don’t want to hear anyone badmouthing the security of Agile or DevOps projects. In my experience they are no better than others. And I believe they have a lot more potential to be better.
• #11: So we’re going to need to automate some stuff.Let’s see if we can do just one simple thing across the portfolio at devops speed. How about clickjacking.
• #12: Before I show you how to create those sensors, I want to explain the different intelligence that can help us.This is the information that can help us identify vulnerabilities. Too often we confuse the type of information with the technique for analyzing it.
• #13: You can’t point at a diagram like this and say, SECURITY GOES HERE.SECURITY IS NOT just a single point in the code. It’s a PATH through an application that goes from custom code, to libraries, to frameworks, to platform and back again.So when we VERIFY security, we need access to lots of different types of information.What kinds of information are relevant? HTTP, Data Flow, Libraries, Control Flow, Configuration, and Backend Connections to name just a few.So what kind of TECHNIQUES can we use to verify that this app does the right thing in stressful situations?
• #14: Positive vs. negative?SAST, DAST, IAST, Manual, Passive?DEV, CI, TEST, QA, SEC, OPS?
• #17: The Beastie Boys brought you Check Your Head…. But I’m bringing you CheckYourHeaders!!!It’s
• #19: But it’s one step towards Continuous AppSec….
• #20: Access Control – static in CILibraries – static in staging – ah ha!Verb Tampering – check config – positive!Injection – IAST – great data flow w/o false alarmsCrypto Correct? – Manual -> Junit testsArchitecture!!
• #23: Run DependencyCheck during every buildStruts2Need to find who has it fastNot all apps are in development and test
• #25: For the Enterprise Security API project, we knew that we needed proof that the security controls we built were “CORRECT”So we wrote thousands of test cases to prove that the controls: * Performed their function * Were tamperproof and non-bypassableToday there are almost 5,000 companies using ESAPI. And we have had only 1 vulnerability identified. We immediately added a test case and we’ll never have that one again Here is a snippet of code from an ESAPI test case.
• #28: Most organizations look like this…. They use all the techniques
• #29: Here are Aspect’s results for MANUAL code review and penetration testing of 5,000,000 lines of code every month.
• #31: Unfortunately, we don’t really have a clear picture of what we expect. Both our EXPECTED and ACTUAL are spotty. That means we don’t have a complete and clear EXPECTED model. We also don’t know exactly what was VERIFIED.I’ve seen folks testing for SQL injection on applications that don’t even have a SQL database. That’s just WASTE. When our tests don’t match up to our EXPECTED model, we aren’t getting good coverage. And even worse are things in the EXPECTED model that aren’t even getting tested.So we end up with a muddy, spotty picture of risks – which leads to bad decisions, exposure, breaches, etc…. This is not to say that we’re not adding value. BUT if we’re going to SCALE we have to get a lot more EFFICIENT.
• #32: Unfortunately, we don’t really have a clear picture of what we expect. Both our EXPECTED and ACTUAL are spotty. That means we don’t have a complete and clear EXPECTED model. We also don’t know exactly what was VERIFIED.I’ve seen folks testing for SQL injection on applications that don’t even have a SQL database. That’s just WASTE. When our tests don’t match up to our EXPECTED model, we aren’t getting good coverage. And even worse are things in the EXPECTED model that aren’t even getting tested.So we end up with a muddy, spotty picture of risks – which leads to bad decisions, exposure, breaches, etc…. This is not to say that we’re not adding value. BUT if we’re going to SCALE we have to get a lot more EFFICIENT.
• #33: Unfortunately, we don’t really have a clear picture of what we expect. Both our EXPECTED and ACTUAL are spotty. That means we don’t have a complete and clear EXPECTED model. We also don’t know exactly what was VERIFIED.I’ve seen folks testing for SQL injection on applications that don’t even have a SQL database. That’s just WASTE. When our tests don’t match up to our EXPECTED model, we aren’t getting good coverage. And even worse are things in the EXPECTED model that aren’t even getting tested.So we end up with a muddy, spotty picture of risks – which leads to bad decisions, exposure, breaches, etc…. This is not to say that we’re not adding value. BUT if we’re going to SCALE we have to get a lot more EFFICIENT.
• #34: Unfortunately, we don’t really have a clear picture of what we expect. Both our EXPECTED and ACTUAL are spotty. That means we don’t have a complete and clear EXPECTED model. We also don’t know exactly what was VERIFIED.I’ve seen folks testing for SQL injection on applications that don’t even have a SQL database. That’s just WASTE. When our tests don’t match up to our EXPECTED model, we aren’t getting good coverage. And even worse are things in the EXPECTED model that aren’t even getting tested.So we end up with a muddy, spotty picture of risks – which leads to bad decisions, exposure, breaches, etc…. This is not to say that we’re not adding value. BUT if we’re going to SCALE we have to get a lot more EFFICIENT.
• #35: I strongly encourage you to break it down with a structured defense strategy.You can achieve a LINE OF SIGHT. You CAN match up your sensors with Business Concerns, but not directlyIdentify your most important business concernsWork out defense strategies – PRIMARY, SECONDARY, PREVENTATIVE, REACTIVEOnce you specify your ACTUAL defenses, your sensors are OBVIOUS
• #36: Talk about creating a cycle of evolve the model, deploy sensors, analyze results, make strategic decisions. This creates high-speed ITERATION and improvement.This leaves the people to ACTUALLY figure out what they care about. Now you can have that principled discussion about whether to allow SHA-1. You’ll have data about how many instances of SHA-1 you actually have, and how hard it will be to update.We lose 90% of the intelligence we gain during a penetration test… and we do it all over again next year.Penetration tests are great at:1) Identifying holes in the expected model2) Figuring out how to test expected model3) Defining (and maybe building) sensorsThat’s a business case for security.
• #37: Later you can include in CI
• #38: Close up with how we are transforming appsec the same way that new-relic transformed performance. Into something that ordinary folks can do themselves.
• #41: Imagine this is your EXPECTED modelNow you have information from your sensors flooding in – telling you that your DEFENSES arePresentCorrectUsed ProperlyAcross your entire PORTFOLIOEven if you start with a very small percentage of your expected model, that’s work that you no longer have to do manually!