SlideShare a Scribd company logo
BDD Mobile security testing with OWASP MASVS,
OWASP MSTG and Calabash
About Me
• #whoami
• Davide Cioccia
• Security Engineer @ ING Bank NL
• Italian leaving in the NL
• +7 years security experience
• Security magazines and OWASP MSTG contributor
• Focus:
– Mobile application security
– SSDLC
– PT & VA
– Incident Response
• Agile Way of Working
• CICD
Requirements Design Code Build Test Release Deploy Operate
Agile Development
Continuous Integration
Continuous Delivery
Continuous Deployment
DevOps
• Security challenges
• Technical:
• Provide security at the DevOps speed
• Detect vulnerabilities in early stage
• Have developers understand security
• Have Pentesters focus on “serious” stuff
• Business
• Lower cost to fix
• Lower time to fix
• Lower time for testing
• Lower time to market
• Manual vs Automation
• Automate the testing: the biggest problem
• Solution: BDD Testing
Describe the behavior
of your software
in a very understandable
language
• Solution: BDD Testing with Cucumber and
Gherkin
• Automated
• Understandable by all the stakeholders
• It fits in the workflow of CI/CD
• BDD Testing
Business facing
Technology
facing
• BDD security tests
• Different frameworks available in the market
• Usage of PT tools, such as Nessus, ZAP, Burp etc
• Focused on server side testing (API, Web
Services..)
• Mobile BDD security tests?
• Mobile BDD security tests?
• Main problems
– different Operating Systems
– client side testing
– different apps (native, hybrid,web)
– different security controls
– different way of testing (iOS, Android, Windows
Phone)
How to fix these problems?
• We need a security standard for Mobile
Testing
• We need a process
Requirements Design Code Build Test Release
Security Requirements
Threat modeling
(abuse case generation)
Threat based security
controls & test specification
Implement BDD
standardized security tests
Implement BDD
application specific
security tests
Test against acceptance
environment
MSTG Test cases
MASVS Checklist
Manual PT
Identify the flaw
Patch the flaw
• We need a tool
• Cross platform (Android, iOS), we just cut Windows Phone off right?
• Support for hybrid apps
• Running on emulators
• Running on real devices
• Possibility to integrate it in the CI/CD
• Support for Gherkin syntax
• A lot of customization
• Free! (We like that :D)
• And the winner is …
calaba.sh
• Calabash
• Calabash
• Integration with with other mobile security
frameworks
• Pentest frameworks for Android and iOS
• Automate manual activities
• scriptable
• the agent must run on the device
– Powered by MWRlab
Let’s try it out
https://github.com/dineshshetty/Android-InsecureBankv2
• UC1: sensitive information in
log file (standard test)
– Requirements
1. Logs must not contain usernames
2. Logs must not contain passwords
3. Logs must not contain information related to
the user
4. Logs must not disclose sensitive information
MASVS V2 - Data Storage and Privacy
MSTG 2.1: Sensitive information in log files
• What’s wrong here?
• What’s wrong here?
• Use case 1: sensitive information in log file
– Feature
• Use case 1: sensitive information in log file
– Feature
• Use case 1: sensitive information in log file
– Step
• Similar tests implemented
• Sensitive data in the clipboard
▪ adb shell su <uid> service call
clipboard 2 s16 <package_name>
• Sensitive data in keyboard cache
▪ query
/data/data/com.android.providers.user
dictionary/databases/user_dict.db
• Use case 2: Internal activities must not be
exported
– Requirements
1. The only exported activity must be the login
2. Internal activities should have the flag exported set to false
MASVS:
V6 - Platform Interaction
V4 - Authentication and Session Management
• Use case 2: Internal activities must not be
exported
– Feature
• Use case 2: Internal activities must not be
exported
– Step without Drozer
• Use case 2: Internal activities must not be
exported
– Step with Drozer
• Use case 3: JavaScript in WebView must be
disabled
– Requirements
1. The Webview must not execute JavaScript code
2. If an input is reflected in the WebView it must be sanitized
MASVS V6: Platform interaction
MSTG:
V6.5: JavaScript is disabled in WebViews unless explicitly required.
• Use case 3: JavaScript in WebView must be
disabled
– Feature
• Use case 3: JavaScript in WebView must be
disabled
<HMTL />
loadsave
• Use case 3: JavaScript in WebView must be
disabled
– Step
• Provided by calabash
• Checks if an alert box is executed and contains the text
specified
• Use case 4: Content provider information
disclosure
– Requirements
1. Content Providers must not expose sensitive information
2. Content Providers must not be exported if there are no other apps
from the same developer
3. Content Providers must use android:export = false instead of
android:export = true
MASVS V6: Platform Interaction
MSTG: Testing Platform Interaction on Android
• Use case 4: Content provider information
disclosure
– Feature
• Use case 4: Content provider information
disclosure
– Feature
• Use case 4: Content provider information
disclosure
– Step
Other tests implemented:
• Exploit Broadcast Receivers
• Intent Sniffing
• Sensitive information in Pasteboard
• More…
• Integration with CI/CD (Jenkins)
– Android emulator plugin
– Add Gemfile to your workspace
– Shell script
https://azevedorafaela.wordpress.com/2014/10/08/9-steps-to-configure-jenkins-with-calabashcucumber/
Improvements
• Include OWASP ZAP for API test
• Use the ”backdoor” feature to modify the
code at runtime
• ?
DEMO
• Achievements
– Speed
– Quality
– Accuracy
– Scalability
– Maturity
“Trying to speed project schedule by reducing testing
is like trying to lose weight by donating blood”
Klaus Leopold
THANK YOU
Davide Cioccia
email: davide.cioccia@ing.nl
web: davidecioccia.com

More Related Content

PPTX
Security Testing for Containerized Applications
PDF
DevSecCon Singapore 2018 - Remove developers’ shameful secrets or simply rem...
PDF
DevSecOps What Why and How
PDF
Building a DevSecOps Pipeline Around Your Spring Boot Application
PDF
DevSecOps: What Why and How : Blackhat 2019
PDF
DevSecOps - The big picture
PDF
DevSecCon Boston 2018: Building a practical DevSecOps pipeline for free by Je...
Security Testing for Containerized Applications
DevSecCon Singapore 2018 - Remove developers’ shameful secrets or simply rem...
DevSecOps What Why and How
Building a DevSecOps Pipeline Around Your Spring Boot Application
DevSecOps: What Why and How : Blackhat 2019
DevSecOps - The big picture
DevSecCon Boston 2018: Building a practical DevSecOps pipeline for free by Je...

What's hot (20)

PDF
Maturing DevSecOps: From Easy to High Impact
PPTX
DevSecOps : an Introduction
PPTX
How to get the best out of DevSecOps - an operations perspective
PPTX
DevOps & Security: Here & Now
PPTX
DevSecOps reference architectures 2018
PDF
Bug Bounties and The Path to Secure Software by 451 Research
PPTX
DevSecCon Boston 2018: Automated DevSecOps infrastructure deployment: recipes...
PPTX
How Azure DevOps can boost your organization's productivity
PDF
Pentest is yesterday, DevSecOps is tomorrow
PDF
Practical DevSecOps Course - Part 1
PDF
#ATAGTR2019 Presentation "DevSecOps with GitLab" By Avishkar Nikale
PDF
Sec4dev 2021 - Catch Me If You can : Continuous Delivery vs. Security Assurance
PDF
Strengthen and Scale Security Using DevSecOps - OWASP Indonesia
PDF
Your Framework for Success: introduction to JavaScript Testing at Scale
PPTX
All Around Azure: DevOps with GitHub - Managing the Flow of Work
PPTX
ATAGTR2017 Wearable App Testing
PDF
Securing the Pipeline
PDF
Serverless Security: Doing Security in 100 milliseconds
PDF
Hacker Games & DevSecOps
Maturing DevSecOps: From Easy to High Impact
DevSecOps : an Introduction
How to get the best out of DevSecOps - an operations perspective
DevOps & Security: Here & Now
DevSecOps reference architectures 2018
Bug Bounties and The Path to Secure Software by 451 Research
DevSecCon Boston 2018: Automated DevSecOps infrastructure deployment: recipes...
How Azure DevOps can boost your organization's productivity
Pentest is yesterday, DevSecOps is tomorrow
Practical DevSecOps Course - Part 1
#ATAGTR2019 Presentation "DevSecOps with GitLab" By Avishkar Nikale
Sec4dev 2021 - Catch Me If You can : Continuous Delivery vs. Security Assurance
Strengthen and Scale Security Using DevSecOps - OWASP Indonesia
Your Framework for Success: introduction to JavaScript Testing at Scale
All Around Azure: DevOps with GitHub - Managing the Flow of Work
ATAGTR2017 Wearable App Testing
Securing the Pipeline
Serverless Security: Doing Security in 100 milliseconds
Hacker Games & DevSecOps
Ad

Similar to BDD Mobile Security Testing (OWASP AppSec Bucharest 2017) (20)

PDF
Null singapore - Mobile Security Essentials
PPTX
DevSecCon Boston2018 - advanced mobile security automation with bdd
PDF
Owasp masvs spain 17
PPTX
Mobile Security at OWASP - MASVS and MSTG
PDF
DEF CON 24 - Dinesh and Shetty - practical android application exploitation
PDF
Droidcon it-2014-marco-grassi-viaforensics
DOCX
Jim Richardson Software Testing CS459 IP 5
PPTX
Security Vulnerabilities in Mobile Applications (Kristaps Felzenbergs)
PPTX
Android pentesting
PDF
Android pentesting
PPTX
Untitled 1
PPTX
Mobile application security
PPTX
Mobile Application Security
PPTX
Mobile platform security models
PPTX
Security testing of mobile applications
PPTX
How to Test Security and Vulnerability of Your Android and iOS Apps
PPTX
Windows Phone 8 Security and Testing WP8 Apps
PPTX
Mobile security part 1(Android Apps Pentesting)- Romansh yadav
PDF
CodeMotion tel aviv 2015 - burning marshmallows
PDF
Jump-Start The MASVS
Null singapore - Mobile Security Essentials
DevSecCon Boston2018 - advanced mobile security automation with bdd
Owasp masvs spain 17
Mobile Security at OWASP - MASVS and MSTG
DEF CON 24 - Dinesh and Shetty - practical android application exploitation
Droidcon it-2014-marco-grassi-viaforensics
Jim Richardson Software Testing CS459 IP 5
Security Vulnerabilities in Mobile Applications (Kristaps Felzenbergs)
Android pentesting
Android pentesting
Untitled 1
Mobile application security
Mobile Application Security
Mobile platform security models
Security testing of mobile applications
How to Test Security and Vulnerability of Your Android and iOS Apps
Windows Phone 8 Security and Testing WP8 Apps
Mobile security part 1(Android Apps Pentesting)- Romansh yadav
CodeMotion tel aviv 2015 - burning marshmallows
Jump-Start The MASVS
Ad

More from Davide Cioccia (9)

PDF
Avoiding GraphQL insecurities with OWASP SKF - OWASP HU meetup
PDF
Attacking and defending GraphQL applications: a hands-on approach
PPTX
Black Hat Europe 2018 Arsenal Tools - Squatm3
PPTX
BH ASIA 2019 Arsenal Tools - Squatm3 and Squatm3gator
DOCX
NAS Botnet Revealed - Mining Bitcoin
PDF
Inside TorrentLocker (Cryptolocker) Malware C&C Server
PDF
One shot eight banks
PDF
Windows Mobile 6.5: Client for a multimedia conferencing platform
PDF
A statistical framework to evaluate the "diversity" impact against Advanced P...
Avoiding GraphQL insecurities with OWASP SKF - OWASP HU meetup
Attacking and defending GraphQL applications: a hands-on approach
Black Hat Europe 2018 Arsenal Tools - Squatm3
BH ASIA 2019 Arsenal Tools - Squatm3 and Squatm3gator
NAS Botnet Revealed - Mining Bitcoin
Inside TorrentLocker (Cryptolocker) Malware C&C Server
One shot eight banks
Windows Mobile 6.5: Client for a multimedia conferencing platform
A statistical framework to evaluate the "diversity" impact against Advanced P...

Recently uploaded (20)

PPT
Introduction Database Management System for Course Database
PDF
SAP S4 Hana Brochure 3 (PTS SYSTEMS AND SOLUTIONS)
PDF
System and Network Administraation Chapter 3
PDF
Odoo Companies in India – Driving Business Transformation.pdf
PDF
Which alternative to Crystal Reports is best for small or large businesses.pdf
PDF
Wondershare Filmora 15 Crack With Activation Key [2025
PDF
top salesforce developer skills in 2025.pdf
PPTX
Oracle E-Business Suite: A Comprehensive Guide for Modern Enterprises
PPTX
L1 - Introduction to python Backend.pptx
PPTX
ai tools demonstartion for schools and inter college
PPTX
assetexplorer- product-overview - presentation
PDF
Raksha Bandhan Grocery Pricing Trends in India 2025.pdf
PDF
EN-Survey-Report-SAP-LeanIX-EA-Insights-2025.pdf
PDF
How to Migrate SBCGlobal Email to Yahoo Easily
PDF
T3DD25 TYPO3 Content Blocks - Deep Dive by André Kraus
PDF
PTS Company Brochure 2025 (1).pdf.......
PDF
2025 Textile ERP Trends: SAP, Odoo & Oracle
PDF
Why TechBuilder is the Future of Pickup and Delivery App Development (1).pdf
PDF
Upgrade and Innovation Strategies for SAP ERP Customers
PDF
Nekopoi APK 2025 free lastest update
Introduction Database Management System for Course Database
SAP S4 Hana Brochure 3 (PTS SYSTEMS AND SOLUTIONS)
System and Network Administraation Chapter 3
Odoo Companies in India – Driving Business Transformation.pdf
Which alternative to Crystal Reports is best for small or large businesses.pdf
Wondershare Filmora 15 Crack With Activation Key [2025
top salesforce developer skills in 2025.pdf
Oracle E-Business Suite: A Comprehensive Guide for Modern Enterprises
L1 - Introduction to python Backend.pptx
ai tools demonstartion for schools and inter college
assetexplorer- product-overview - presentation
Raksha Bandhan Grocery Pricing Trends in India 2025.pdf
EN-Survey-Report-SAP-LeanIX-EA-Insights-2025.pdf
How to Migrate SBCGlobal Email to Yahoo Easily
T3DD25 TYPO3 Content Blocks - Deep Dive by André Kraus
PTS Company Brochure 2025 (1).pdf.......
2025 Textile ERP Trends: SAP, Odoo & Oracle
Why TechBuilder is the Future of Pickup and Delivery App Development (1).pdf
Upgrade and Innovation Strategies for SAP ERP Customers
Nekopoi APK 2025 free lastest update

BDD Mobile Security Testing (OWASP AppSec Bucharest 2017)

  • 1. BDD Mobile security testing with OWASP MASVS, OWASP MSTG and Calabash
  • 2. About Me • #whoami • Davide Cioccia • Security Engineer @ ING Bank NL • Italian leaving in the NL • +7 years security experience • Security magazines and OWASP MSTG contributor • Focus: – Mobile application security – SSDLC – PT & VA – Incident Response
  • 3. • Agile Way of Working
  • 4. • CICD Requirements Design Code Build Test Release Deploy Operate Agile Development Continuous Integration Continuous Delivery Continuous Deployment DevOps
  • 5. • Security challenges • Technical: • Provide security at the DevOps speed • Detect vulnerabilities in early stage • Have developers understand security • Have Pentesters focus on “serious” stuff • Business • Lower cost to fix • Lower time to fix • Lower time for testing • Lower time to market
  • 6. • Manual vs Automation
  • 7. • Automate the testing: the biggest problem
  • 8. • Solution: BDD Testing Describe the behavior of your software in a very understandable language
  • 9. • Solution: BDD Testing with Cucumber and Gherkin • Automated • Understandable by all the stakeholders • It fits in the workflow of CI/CD
  • 10. • BDD Testing Business facing Technology facing
  • 11. • BDD security tests • Different frameworks available in the market • Usage of PT tools, such as Nessus, ZAP, Burp etc • Focused on server side testing (API, Web Services..)
  • 12. • Mobile BDD security tests?
  • 13. • Mobile BDD security tests?
  • 14. • Main problems – different Operating Systems – client side testing – different apps (native, hybrid,web) – different security controls – different way of testing (iOS, Android, Windows Phone)
  • 15. How to fix these problems?
  • 16. • We need a security standard for Mobile Testing
  • 17. • We need a process Requirements Design Code Build Test Release Security Requirements Threat modeling (abuse case generation) Threat based security controls & test specification Implement BDD standardized security tests Implement BDD application specific security tests Test against acceptance environment MSTG Test cases MASVS Checklist Manual PT Identify the flaw Patch the flaw
  • 18. • We need a tool • Cross platform (Android, iOS), we just cut Windows Phone off right? • Support for hybrid apps • Running on emulators • Running on real devices • Possibility to integrate it in the CI/CD • Support for Gherkin syntax • A lot of customization • Free! (We like that :D)
  • 19. • And the winner is … calaba.sh
  • 22. • Integration with with other mobile security frameworks • Pentest frameworks for Android and iOS • Automate manual activities • scriptable • the agent must run on the device – Powered by MWRlab
  • 23. Let’s try it out https://github.com/dineshshetty/Android-InsecureBankv2
  • 24. • UC1: sensitive information in log file (standard test) – Requirements 1. Logs must not contain usernames 2. Logs must not contain passwords 3. Logs must not contain information related to the user 4. Logs must not disclose sensitive information MASVS V2 - Data Storage and Privacy MSTG 2.1: Sensitive information in log files
  • 27. • Use case 1: sensitive information in log file – Feature
  • 28. • Use case 1: sensitive information in log file – Feature
  • 29. • Use case 1: sensitive information in log file – Step
  • 30. • Similar tests implemented • Sensitive data in the clipboard ▪ adb shell su <uid> service call clipboard 2 s16 <package_name> • Sensitive data in keyboard cache ▪ query /data/data/com.android.providers.user dictionary/databases/user_dict.db
  • 31. • Use case 2: Internal activities must not be exported – Requirements 1. The only exported activity must be the login 2. Internal activities should have the flag exported set to false MASVS: V6 - Platform Interaction V4 - Authentication and Session Management
  • 32. • Use case 2: Internal activities must not be exported – Feature
  • 33. • Use case 2: Internal activities must not be exported – Step without Drozer
  • 34. • Use case 2: Internal activities must not be exported – Step with Drozer
  • 35. • Use case 3: JavaScript in WebView must be disabled – Requirements 1. The Webview must not execute JavaScript code 2. If an input is reflected in the WebView it must be sanitized MASVS V6: Platform interaction MSTG: V6.5: JavaScript is disabled in WebViews unless explicitly required.
  • 36. • Use case 3: JavaScript in WebView must be disabled – Feature
  • 37. • Use case 3: JavaScript in WebView must be disabled <HMTL /> loadsave
  • 38. • Use case 3: JavaScript in WebView must be disabled – Step • Provided by calabash • Checks if an alert box is executed and contains the text specified
  • 39. • Use case 4: Content provider information disclosure – Requirements 1. Content Providers must not expose sensitive information 2. Content Providers must not be exported if there are no other apps from the same developer 3. Content Providers must use android:export = false instead of android:export = true MASVS V6: Platform Interaction MSTG: Testing Platform Interaction on Android
  • 40. • Use case 4: Content provider information disclosure – Feature
  • 41. • Use case 4: Content provider information disclosure – Feature
  • 42. • Use case 4: Content provider information disclosure – Step
  • 43. Other tests implemented: • Exploit Broadcast Receivers • Intent Sniffing • Sensitive information in Pasteboard • More…
  • 44. • Integration with CI/CD (Jenkins) – Android emulator plugin – Add Gemfile to your workspace – Shell script https://azevedorafaela.wordpress.com/2014/10/08/9-steps-to-configure-jenkins-with-calabashcucumber/
  • 45. Improvements • Include OWASP ZAP for API test • Use the ”backdoor” feature to modify the code at runtime • ?
  • 46. DEMO
  • 47. • Achievements – Speed – Quality – Accuracy – Scalability – Maturity “Trying to speed project schedule by reducing testing is like trying to lose weight by donating blood” Klaus Leopold
  • 48. THANK YOU Davide Cioccia email: davide.cioccia@ing.nl web: davidecioccia.com