Automatic Generation of Test Cases for REST APIs: a Specification-Based Approach
Download as PPTX, PDF1 like1,023 views
This document proposes an approach to automatically generate test cases for REST APIs using the OpenAPI specification. It leverages the OpenAPI metamodel to extract models and define inference rules to generate nominal and faulty test cases. An implementation was validated on real-world APIs, achieving over 75% coverage of operations, parameters, endpoints and definitions. The main failing points identified were incorrect or invalid OpenAPI definitions. The approach aims to increase coverage levels and support the OpenAPI v3.0 specification.
Automatic Generation of Test Cases for REST APIs: a Specification-Based Approach
- 1. Automatic Generation of Test Cases for REST APIs: a Specification-Based Approach Hamza Ed-douibi, Javier L. Cánovas Izquierdo, Jordi Cabot unsplash-chuttersnap
- 9. Leveraging OpenAPI for testing REST APIs
- 10. Leveraging OpenAPI for testing REST APIs
- 11. Leveraging OpenAPI for testing REST APIs
- 13. OpenAPI Model
- 14. Leveraging OpenAPI for testing REST APIs
- 16. Our Approach • Straightforward from the OpenAPI Metamodel • Special attention to JSON references Model Extraction
- 17. Our Approach • Simple parameter (examples, default values, enums) • Dummy parameter • Complex parameter (response of an operation) Inference rules
- 18. Our Approach
- 19. Generation Rules GR 1. Nominal test case GR 2. Faulty test case
- 20. Generation Rules GR 1. Nominal test case GR 2. Faulty test case Faulty cases • Required missing • Wrong datatypes • Violated constraints
- 21. Our Approach
- 24. Validation RQ1 What is the coverage level of the generated test cases? RQ2 What are the main failing points in the definitions and implementation of real world REST APIs?
- 25. Validation RQ1 What is the coverage level of the generated test cases? RQ2 What are the main failing points in the definitions and implementation of real world REST APIs? Filtering Criteria A ― Free, open and available APIs which provide access to data models
- 26. Validation RQ1 What is the coverage level of the generated test cases? RQ2 What are the main failing points in the definitions and implementation of real world REST APIs? Filtering Criteria A ― Free, open and available APIs which provide access to data models Filtering Criteria B ― Remove incorrect or invalid OpenAPI definitions
- 27. Validation RQ1 What is the coverage level of the generated test cases? RQ2 What are the main failing points in the definitions and implementation of real world REST APIs? ELEMENTS COUNT COVERAGE COVERAGE(%) ALL NOMINAL FAULTY ALL NOMINAL FAULTY OPERATIONS 367 320 303 233 87% 82% 63% PARAMETERS 949 595 485 476 62% 51% 50% ENDPOINTS 356 289 81% DEFINITIONS 313 239 76%
- 28. Validation RQ1 What is the coverage level of the generated test cases? RQ2 What are the main failing points in the definitions and implementation of real world REST APIs? ELEMENTS COUNT COVERAGE COVERAGE(%) ALL NOMINAL FAULTY ALL NOMINAL FAULTY OPERATIONS 367 320 303 233 87% 82% 63% PARAMETERS 949 595 485 476 62% 51% 50% ENDPOINTS 356 289 81% DEFINITIONS 313 239 76% TOTAL NOMINAL TEST CASES FAULTY TEST CASES 4XX/500 SCHEMA 500 2XX NUMBER OF APIS 37 9 11 11 20 40% 25% 30% 30% 55%
- 29. Conclusion • Model-driven approach to automate REST API testing based on Open API • Plug-in implementing the approach • Coverage over 75% What we have shown What we want to do next Increase coverage levels Support for OpenAPI v3.0 unsplash-Vek Labs
- 30. Except where otherwise noted, content on this presentation is licensed under a Creative Commons Attribution 4.0 International license. Thanks! Javier L. Cánovas Izquierdo jcanovasi@uoc.edu @jlcanovas Hamza Ed-douibi hed-douibi@uoc.edu @mazamiz Jordi Cabot jordi.cabot@icrea.cat @softmodeling https://github.com/SOM-Research/test-generator