Bringing the hacker mindset into requirements and testing by Eapen Thomas and Jason Petry
6 likes565 views
This document discusses bringing a hacker mindset to requirements and testing for application security. It begins by highlighting statistics showing the poor state of application security and vulnerabilities. The document then contrasts producer and consumer views of quality, and explains why security requirements are difficult by nature. It provides examples of threat modeling and negative testing techniques that can help requirements analysts and testers think like hackers to identify vulnerabilities. The presentation calls for adopting these adversarial techniques to improve application security.
Bringing the hacker mindset into requirements and testing by Eapen Thomas and Jason Petry
- 1. BRINGING THE HACKER MINDSET INTO REQUIREMENTS AND TESTING Jason Petry Eapen Thomas Nationwide Insurance
- 2. AGENDA ο How bad is our application security? Why are we so bad at securing our applications? ο An example application: money transferring application ο 2 Views of Quality β Producer vs. Consumer ο Why are security requirements so hard? ο Tools to aid requirements and test analysts ο Threat Models ο Attack trees ο Securing our applications is getting more challenging ο Call to action 2
- 3. YOUR PERSONAL INFO IS PUBLIC 3Source: Symantec Internet Security Threat Report οA New Zero-Day Vulnerability Was Discovered on Average Each Week in 2015 ο Advanced attack groups continue to profit from previously undiscovered flaws in browsers and website plugins οOver Half a Billion Personal Records Were Stolen or Lost in 2015 οMajor Security Vulnerabilities in Three Quarters of Popular Websites Put Us All at Risk ο Web administrators still struggle to stay current on patches
- 4. MOBILE VULNERABILITIES ARE OFF THE CHART 4Source: Symantec Internet Security Threat Report οAndroid users willingly downloaded over two billion malicious mobile applications last year Source: http://www.itproportal.com/2016/02/26/smartphone-users-still-taking-cavalier-approach-mobile-security/ οNearly 25 percent of mobile apps contain at least one high-risk security flaw οAn average mobile device connects to 160 different IP addresses daily ο35 percent of mobile device communications are unencrypted οThere is a 50 percent greater chance that games include a high-risk vulnerability than the average app.
- 5. SOME SCARY INDUSTRY NUMBERS 84% of cyber attacks are happening at the application layer Source: Forbes / SAP (March 2015) 98% of applications scanned by Trustwave harbored one or more security vulnerabilities. Meanwhile, the median number of vulnerabilities was 20 - up from six the year prior. Source: 2015 Trustwave Global Security Report Exploiting many of these application vulnerabilities is βVERY EASYβ 5
- 6. SECURITY VS FUNCTIONAL RQRMTS & TESTING ο Security requirements and testing is different from functional requirements & testing. ο In security testing the goal is to find out if the system can stand up to abusers. Negative tests are critical. ο Security test scenarios may not be realistic from a common user standpoint . Especially when considering web applications, attackers may interact with the application in critically different ways compared to regular users. ο Anticipating and planning for these scenarios is vital for security testing ο Security requirements and testing requires an adversarial mindset, a "what if" mindset, i.e., the same one hackers use to break systems. 6
- 7. EXAMPLE REQUIREMENT STORY οCyclone Transfers β A Pay-pal like service. οβAs a logged in customer, I can transfer money, so that I move money from one of my accounts to another customerβs account.β οAcceptance Criteria: ο The amount of money I transfer must be less than the amount of money available. ο All amounts are in US Dollars. ο Transfers may be for fractions of a penny. 7
- 8. DEMO 8
- 9. WHAT IS QUALITY? What are the two views of Quality? οThe producer view and the customer view. οThe producer view of quality: a product is a quality product if it meets or conforms to the product requirements. This statement is usually shortened to: quality means meets requirements. οThe customer view of quality: fit for use; the product or service meets the customerβs needs regardless of the requirements 9
- 10. THE PRODUCER VIEW OF QUALITY Ok, so, what is the problem? ο We donβt have much/any application security requirements (this is an industry wide problem) ο To create good security requirements, the analyst should review (Organizational requirements, Privacy requirements, statutory requirements and industry requirements) ο Organizational security policies and standards ο Org privacy policies ο Regulatory requirements (Sarbanes-Oxley, HIPAA etc.) ο Other standards such as PCI DSS, ANSI-X9 for banks etc. ο What is the solution? The requirements analysts have to be Creative 10
- 11. CUSTOMER VIEW OF QUALITY Letβs now switch to the customer view of quality Oops! We have a bigger problem! To the customer, a product is a quality product if it meets the customerβs needs, regardless of whether the requirements were met. ο We have to go beyond requirements (even if we have some security requirements). ο For this, we definitely have to be creative, requires an adversarial mindset ο We talk about some resources from OWASP and other organizations that can help 11
- 12. TESTERS HAVE TO BE CREATIVE Test analysts should be creative in the absence of good security requirements: οAutomated web application security testing tools can help οExplore & Discover β Exploratory testing comes handy οUse Common Sense & Experience - common knowledge that comes from experience οDiscussions, Emails and Meeting Notes οCreate and review the high level test scenarios with the business 12
- 13. OWASP TOP 10 When we talk about web application vulnerabilities, we have to talk about Open Web Application Security Project (OWASP) Top 10 vulnerabilities. Please note: this is βaβ list, not the exhaustive list of all possible web application vulnerabilities (there are hundreds of them) 13
- 14. CURE FOR MOST PROBLEMS: INPUT VALIDATION Many vulnerabilities are exploited by injecting malicious commands/code through input forms TYPE οΌ Always check the data type of the input and make sure it matches the expected data type. For example, if there is an input box which accepts numeric data and the letter βOβ is typed instead of the number zero, it should not be accepted. LENGTH οΌ Always check that the data lies within the acceptable range of lengths for the values expected. For example, a zip code field will be either 5 or 10 (dashes included) digits in length. If nothing is entered, or if 11 or more digits are entered, it should not be accepted. FORMAT οΌ Always check that data is in a specified format. For example, dates should be in a specific format (such as MM/DD/YYYY). If it is not in the correct format, it should not be accepted. RANGE οΌ Always check that data lies within a specified range of values. For example, the month of a personβs date of birth should lie between 1 and 12. If it does not fall within that range, it should not be accepted. 14
- 15. THREAT MODELLING οβThreat modeling is about using models to find security problems.β β Adam Shostack, Threat Modeling: Designing for Security οβComing up with a set of possible attacks you plan to protect againstβ β Electronic Frontier Foundation (https://ssd.eff.org/en/glossary/threat-model) 15
- 16. THREAT MODELLING EXAMPLE May be many Data Flow Diagrams (DFDβs) for one application/process, at varying levels of detail.
- 17. THREAT MODELLING οCan be done with varying levels of formality, and with different focuses; method adopted should be tailored to specific application need. οOne Common Methodology is Microsoftβs STRIDE model. ο§ Model: decompose the application as a data flow diagram (DFD) to drive the overall risk analysis process. ο§ Identify: In the next step, threats to the modeled system are identified and enumerated ο§ Mitigate: After threats have been identified, mitigations to those threats are selected ο§ Validate: Implement tests to validate threat is mitigated. Spoofing: Impersonating something or someone else Tampering: Modifying data or code Repudiation: Claiming to have not performed an action Information Disclosure: Exposing information to someone not authorized to see it Denial of Service: Deny or degrade service to users Elevation of Privilege: Gain capabilities without proper authorization
- 18. Steal someoneβs account Trick someone into giving me money Enter a negative number for a transfer I make ATTACK TREES Get someone else to give me money Fool the system into giving me money Get Funds Transferred to me with no work. 18
- 19. NEGATIVE TEST EXAMPLES ο#1 Test: The Single Quote: ' ο Helpful to make sure SQL injection attacks have been properly mitigated against ο Also useful for ensuring anti-SQL injection protections properly deal with single quote characters in user data. 19
- 20. EXAMPLE SINGLE QUOTE TEST οCyclone Transfers: Test Procedure ο Login ο Go to all users. ο Use Search function to find abcdef ο Use Search function to find OβBrian οExpected result: No users found same results for both οActual Result: Error Message in second case. 20
- 21. NEGATIVE TEST CASE: HTML CONTENT οAnother important test case: Allowing entry of HTML input, and properly displaying the result. (i.e., as text) οCross-Site Scripting is the single most commonly encountered security issue in web applications. 21
- 22. EXAMPLE HTML CONTENT TEST οCyclone Transfers: Test Procedure ο Click on Sign In, then Sign Up. ο Create new account, in Profile Statement section include following content: <script>alert(123)</script> ο Log in a different user, go to all users and search for newly created user. οExpected Result: Profile Statement is displayed in search results as typed above. οActual Result: Alert Box Created. 22
- 23. NEGATIVE TEST CASE: DIRECT OBJECT ACCESS οIf URLβs to content are static, ensure that userβs cannot access other usersβ content. οUnlike previous two cases, very difficult for multi-purpose scanner tools to detect and respond; requires knowledge of the application and data access rules. 23
- 24. EXAMPLE DIRECT OBJECT ACCESS οCyclone Transfers: Test Procedures ο Create New User, as in last Test ο In New User, add account, and upload test PDF as βBank Statementβ. ο Click on link to uploaded PDF to validate. Create Bookmark to PDF. ο Log in a different account, use bookmark to go to PDF ο Expected Result: Access should be denied. ο Actual Result: PDF Displayed 24
- 25. THE RUGGED MANIFESTO HTTPS://WWW.RUGGEDSOFTWARE.ORG/ I am rugged and, more importantly, my code is rugged. I recognize that software has become a foundation of our modern world. I recognize the awesome responsibility that comes with this foundational role. I recognize that my code will be used in ways I cannot anticipate, in ways it was not designed, and for longer than it was ever intended. I recognize that my code will be attacked by talented and persistent adversaries who threaten our physical, economic and national security. I recognize these things β and I choose to be rugged. I am rugged because I refuse to be a source of vulnerability or weakness. I am rugged because I assure my code will support its mission. I am rugged because my code can face these challenges and persist in spite of them. I am rugged, not because it is easy, but because it is necessary and I am up for the challenge. 25
- 26. THINGS ARE NOT GETTING ANY EASIER Things are not getting any easier, they are getting really complicated, very fast ο Human beings are still the weakest link ο Ballooning attack surface οThe number of Mobile devices are growing, mobile apps are getting very functional/complicated οInternet of things is making application security more difficult with the amount of interconnected devices 26
- 27. CALL TO ACTION οDonβt be left behind, security requirements elicitation & testing skills are essential, not optional οA skill you must have to be competitive/marketable/just to survive οGet trained, get competent in software security requirements elicitation techniques & security testing techniques ο Resources are lacking ο Conferences that cater to analysts, have no tracks or have very few tracks on application security requirement/testing 27
- 28. RESOURCES ο OWASP https://www.owasp.org ο OWASP Testing Guide (200+ page PDF document is free to download) https://www.owasp.org/index.php/OWASP_Testing_Project ο OWASP Application Security Verification Standard 3.0 https://www.owasp.org/images/6/67/OWASPApplicationSecurityVerificationStandard3.0.pdf ο OWASP Broken Web Application Project https://www.owasp.org/index.php/OWASP_Broken_Web_Applications_Project ο SANS SWAT checklist https://software-security.sans.org/resources/swat ο Microsoft SDL Threat Modeling Tool https://www.microsoft.com/en-us/download/confirmation.aspx?id=49168 28
- 29. THANK YOU! If you would like to contact us: Jason Petry (petryj2@nationwide.com) Eapen Thomas (eapen@nationwide.com) 29