Do not automate GUI testing
Download as PPTX, PDF1 like446 views
The document discusses various principles of software testing, emphasizing the importance of API testing over GUI testing, and cautioning against redundant automation practices. It highlights key concepts such as RESTful web services, HTTP methods, important status codes, and the role of tools like SoapUI and Fiddler in testing. Additionally, it critiques common anti-patterns in testing strategies and stresses the need for effective communication and understanding of the testing process.
Do not automate GUI testing
- 1. Do not automate GUI Reinvention of wheel (HTTP performance testing) like an effective tool for regression testing Atila Inovecký
- 2. Something about pyramids WHY to do API Testing
- 3. Abraham Maslow – Human Needs Don't reinvent a wheel
- 4. Gojko Adzic - Quality Criteria Don't reinvent a wheel
- 5. Mike Cohn (2009)– Automated regression testing Don't reinvent a wheel
- 6. API testing in 2001 • Kaner, Bach, Pettichord: Lessons Learned in Software Testing – Lesson 132: Automate tests using programming interfaces “…When we look at many product testing effort we’ve observed, we conclude that there is a strong correlation between the availability of programming interfaces for testing and the development of powerful automated test suites. Programming interfaces include APIs, command-line interfaces, COM interfaces, HTTP and more. …If you’re serious about automation, you’ll learn or get someone who knows this to help.. …Focus your attention on where automation can help the most. Sometimes, you can effectively automate the GUI; other times you can’t. Don’t let people’s preconceptions of what automation has to look like limit you.” Don't reinvent a wheel
- 7. Ice Cream Cone Antipattern Don't reinvent a wheel
- 8. Reasons for antipattern(traditional approach) • Automate everything! • Just QA is responsible for quality • Few testing rounds – no Continuous Integration • WHAT TO TEST(Business Functionality)=HOW TO TEST (GUI)? • “This small hotfix won’t break anything” • Production delivery 1 per year • DIDN’T FAIL YET • Trying to find a lot bugs Don't reinvent a wheel
- 9. Test find bugs, not automation • Value of the regression test depend on value of the covered functionality • Count of found bugs say nothig about valueeffectivity of automated test • Most of regression bugs is caused by unexpected impact of backend integration not GUI • Automation is not medicine for wrong communication Don't reinvent a wheel
- 10. Conclusions • Unit testing is the core of regression testing • API testing is not the FUTURE but it is really just a reinvented wheel • Future of automated GUI testing is (Exploratory tests, …)??? • GUI testing vs API testing Don't reinvent a wheel
- 11. Something about REST API How to test API through HTTP (REST) Don't reinvent a wheel
- 12. REST – Representational State Transfer • REST or RESTful web services are one way of providing interoperability between computer systems on the internet. REST- compliant web services allow requesting systems to access and manipulate textual representations of web resources using a uniform and predefined set of stateless operations. Don't reinvent a wheel
- 13. Layered architecture of RESTful interactions Don't reinvent a wheel
- 14. RESTful URLs • URLs intended to improve the usability and accessibility of a website or web service by being immediately and intuitively meaningful to non-expert users. Don't reinvent a wheel Non-semantic URL Semantic URL http://example.com/index.php?page=name http://example.com/name http://example.com/index.php?page=consulting/mar keting http://example.com/consulting/marketing http://example.com/products?category=2&pid=25 http://example.com/products/2/25 http://example.com/cgi- bin/feed.cgi?feed=news&frm=rss http://example.com/news.rss http://example.com/services/index.jsp?category=lega l&id=patents http://example.com/services/legal/patents
- 15. Relationships between URL and HTTP methods Don't reinvent a wheel Uniform Resource Locator (URL) GET POST Collection, such as http://mgs.com/resources/ List the URIs and perhaps other details of the collection's members. Create a new entry in the collection. The new entry's URI is assigned automatically and is usually returned by the operation. Element, such as http://mgs.com/resources/item17 Retrieve a representation of the addressed member of the collection, expressed in an appropriate Internet media type. Not generally used. Treat the addressed member as a collection in its own right and create a new entry within it.
- 16. HTTP API Design Don't reinvent a wheel
- 17. Important HTTP Request Headers Don't reinvent a wheel Header field name Description Example Cache-Control Used to specify directives that must be obeyed by all caching mechanisms along the request-response chain. Cache-Control: no-cache Content-Type The MIME type of the body of the request (used with POST and PUT requests). Content-Type: application/x-www-form-urlencoded Content-Type: application/json Content-Type: multipart/form-data Cookie An HTTP cookie previously sent by the server with Set- Cookie. Cookie: $Version=1; Skin=new; Host The domain name of the server (for virtual hosting), and the TCP port number on which the server is listening. The port number may be omitted if the port is the standard port for the service requested. Host: qa16-ats.mgspdtesting.com
- 18. Important HTTP Status Codes Don't reinvent a wheel Status Code Description 200 Ok Standard response for successful HTTP requests. The actual response will depend on the request method used. In a GET request, the response will contain an entity corresponding to the requested resource. In a POST request, the response will contain an entity describing or containing the result of the action. 302 Found Common way of performing URL redirection. 400 Bad Request The server cannot or will not process the request due to an apparent client error 403 Forbidden The request was a valid request, but the server is refusing to respond to it. The user might be logged in but does not have the necessary permissions for the resource. 404 Not Found The request was a valid request, but the server is refusing to respond to it. The user might be logged in but does not have the necessary permissions for the resource. Usually means that the resource already didn’t exist. 501 Internal Server Error Indicates that the request could not be processed because of conflict in the request, such as an edit conflict between multiple simultaneous updates.
- 19. Open Source Tools for API Testing • Performance testing tools • SoapUI (SoapUI NG Pro), Jmeter(SmartMeter, Blazemeter) • Web Service testing tool • Postman, DHC, HTTPRequester, Wizdler, WebInject, Http Master, … • Automation Frameworks • FitNesse, Cucumber, Robot Framework, … Don't reinvent a wheel
- 20. SoapUI • Open-source web service testing application: • for service-oriented architectures (SOA) • for representational state transfers (REST). • Covers: • web service inspection, invoking, development, simulation and mocking, • functional testing, • load and compliance testing. • A commercial version, SoapUI Pro (Api Ready), which mainly focuses on features designed to enhance productivity. Don't reinvent a wheel
- 21. SoapUI Open Source– pros and cons • Pros • support HTTP(S), REST, SOAP, JDBC, AMF • IDE -Groovy -> command line, visible SoapUI API • User forums, tradition • Easy to learn, easy to use • Cons • Team work support • No recording • Weak user documentation • Test history, test debugging, compare results • Buggy Don't reinvent a wheel
- 22. Conclusions • Web application automated testing is possible through HTTP(S) • Web application automated testing is effective through REST • REST API popularity is growing Don't reinvent a wheel
- 23. Something about fiddling How to capture HTTPS communication Don't reinvent a wheel
- 24. Fiddler • Captures HTTP and HTTPS traffic and logs it for the user to review (the latter by implementing man-in-the-middle interception using self-signed certificates). • https://www.telerik.com/download/fiddler Don't reinvent a wheel
- 25. HTTPS Traffic settings • Capture - F12 or Click on left bottom corner • Select captured process by • Preview - Tab Inspector • Request – Top part • Response – Bottom part Don't reinvent a wheel
- 26. Icon help Don't reinvent a wheel
- 27. Fiddler - Inspector • Request Inspectors • Headers—Shows request headers and status. • Webforms – Shows parameters visualized as table from QueryString and Body • Raw—Shows the “real” request header and body • Response Inspectors • Headers—Shows response headers and status. • SyntaxView—Shows the response body with HTML syntax • WebView – Shows the browser output • Raw – Shows “real” response header and body Don't reinvent a wheel
- 28. Fiddler - Settings ToolsTelerik Fiddler OptionsHTTPS Filters settings Don't reinvent a wheel
- 29. Something about soap with UI REST testing in SoapUI Open Source Don't reinvent a wheel
- 30. Initial settings • FilePreferences – • Proxy Settings -> Manual 127.0.0.1:8888 • UI Settings – Size of Req Response Message to Show – 1000000 • FileImport project (FileCreate Empty Project) Don't reinvent a wheel
- 31. Don't reinvent a wheel
- 32. What you get for 500$ in SoapUI NG Pro • a • FileImport project (FileCreate Empty Project) Don't reinvent a wheel
- 33. Useful links REST 101: The Beginner's Guide to Using and Testing RESTful APIs SoapUI - Getting Started with REST Testing SoapUI - Functional Testing SoapUI – Scripting & Properties SoapUI – API Docs Don't reinvent a wheel
- 34. Reference List • Gojko Adzic, David Evans - Fifty Quick Ideas To Improve Your Tests • Mike Cohn - Succeeding with Agile: Software Development Using Scrum • Lisa Crispin, Janet Gregory - Agile Testing: A Practical Guide for Testers and Agile Teams • Dorothy Graham, Mark Fewster - Experiences of Test Automation: Case Studies of Software Test Automation • Cem Kaner, James Bach, Bret Pettichord - Lessons Learned in Software Testing: A Context-Driven Approach Don't reinvent a wheel
- 35. Don't reinvent a wheel
Editor's Notes
- #6: Google Compare