Quality Jam 2017: Jesse Reed & Kyle McMeekin "Test Case Management & Exploratory Testing Best Practices"
1 like387 views
The document discusses the transition of Questar Assessment Inc.'s QA team to Qtest for improved test case management and exploratory testing. Key points include challenges faced, the rationale for adopting new tools, and future plans for integrating testing processes. Emphasis is placed on enhancing quality assurance practices, transparency, and efficiency through better automation and tool integration.
Quality Jam 2017: Jesse Reed & Kyle McMeekin "Test Case Management & Exploratory Testing Best Practices"
- 1. Test Case Management & Exploratory Testing Kyle McMeekin & Jesse Reed
- 2. Agenda - Introductions - Challenges - Why the move to qTest - Results of implementation - Current best practices of Questar - Future of Questar testing - Exploratory Testing - Q & A
- 4. INSERT PHOTO ON TOP Jesse Reed Title: Director of QA Where: Questar Assessment Inc. How long: IT industry for 15 How long: QA industry for 8 Where from: Minnesota Grown Things I like to do: Travel Other hobbies: Entrepreneur
- 5. INSERT PHOTO ON TOP Kyle McMeekin Title: Senior Sales Engineer Where: QASymphony How long: 2 years How long: 5 years Where from: Maryland Things I like to do: Travel and Ski Other hobbies: Video Games
- 6. Challenges
- 7. Questar Assessment is an educational assessment company focused on developing and delivering meaningful measures of student progress. It is not just one of the things we do. It’s all we do. - Scoring and reporting - Items and forms - Test delivery system and Admin - Paper and Pencil, Item and Form Development - Offshore partners – MentorMate (Bulgarian) - Atlassian Suite (Jira/Confluence) - Microsoft shop mainly using C# .Net, MS SQL 2012, 2016 - Jenkins, GIT - MAQS Framework (Selenium Webdriver,Grid, Appium, .Net service, MS SQL) - Jmeter - Workfront K-12 Assessment QA Team Technology
- 8. • Rapidly growing company • Poor planning • Lack of process/SDLC/requirements • Limited automation on legacy systems • Newly created platform
- 9. • Overworked and frustrated team • Limited test cases/No traceability • No formal QA processes • Excel and gSheets holders of test cases • Lack of knowledge of Jira and Confluence
- 10. Bugs being written on paper and sorted into piles
- 11. “We don’t know what we have tested. I don’t know what I did to create the bug. Questar QA team member
- 12. Why the move to qTest?
- 13. INSERT PHOTO ON TOP Selling the dream of TCM Business Case Tell the story Why What How
- 14. • The main purpose of Questar’s Quality Assurance and Testing team is assisting in Questar creating a Higher Quality Product for our clients. The team is continuously searching for ways to improve. • As the team continues to change it’s ideologies of how to shift left, creating efficiencies, and promote quality throughout, the need for better tooling is necessary. • Questar QAT is in an ideal position to consolidate it’s test management efforts into one centralized location, a Test Management Tool 14 Why a Test Management Tool? Higher Quality/Lower Cost Traceability Consistency Robust reporting Transparency Ease of use/Efficiency
- 15. Quality Assurance Strategy • Continuous improvement of Process • Centralized test case repository • Traceability of work • Requirements • Test cases • Bugs Test Planning •Provide clarity of testing focus •Assignment of testing duties •Test coverage •Transparency •Reporting Testing Techniques •Functional and integration •System and End-to-end •Ad hoc/Exploratory •Regression •Automation Technology focused testing •Utilizing automated strategies •Continuously increase efficiencies and coverage Quality 15 Why a Test Management Tool?
- 17. Criteria for success 17 Ease of use Functionality Implementation Reporting capabilities Added Value Licensing Customer Support Training Jira Integration Automation Integration Future needs Future features
- 20. “I absolutely love using qTest, I am able to manage projects more easily, and reporting is so much easier. Questar QA manager
- 21. “I love the reports this produces Questar CEO
- 22. “This is a really big win for this organization. Questar CTO
- 23. “It is configurable, and you can manage it in your own way. Senior QA Lead - Offshore consultant
- 24. “Can I get a license for qTest? Anyone who doesn’t have a license
- 25. Onboarding Requirements Traceability Reporting and test status Repeatability Effective bug creation
- 28. Questar best practices for qTest Transparency is Key • Large software projects define our projects • Test plans are created with 2 week sprints in mind • Test case approval from peers • Import of requirements or stories from Jira • Linking of test cases to requirements or stories • Hierarchical test case structure • Modular test cases • Intuitive test case naming
- 30. Questar’s plans for the future Jenkins integration of qTest API Continuous Integration Exploratory, Ad hoc, Session based Custom Reporting Efficiency
- 32. PROBLEMS IN TRADITIONAL TESTING 32 Prescribing detailed test steps, and test planning, can cloud interactions. Focus on compressive documentation blocks cognitive thinking abilities Customer collaboration in testing remains in silo or nonexistent. Comprehensive test plans create a tunnel vision effect Individuals and interactions over processes and tools Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan
- 33. WHAT IS EXPLORATORY TESTING? 33 1. Parallel test planning, test design, and test execution 1. Specific yet flexible 1. Aligned towards investigation of potential opportunities 1. Values depth and attention to detail during testing 1. Fosters knowledge sharing and accountability
- 34. PARALLEL PLANNING, DESIGNING, EXECUTION 34 Unlike traditional testing techniques, planning, design, and execution happen concurrently, allowing efficiencies of time as well as flexibility in approach PLAN DESIGNEXECUTE REPORT PLAN DESIGN EXECUTE REPORT Traditional Scripted Testing Exploratory Testing
- 35. Exploratory Testing provides a specific lens through which to perform testing – whether that be a user person, functionality, criteria (i.e. Localization), etc. However, it allows testers to use the tool as an end user would, not necessarily as the product owner envisioned it. SPECIFIC YET FLEXIBLE 35 I tested the application as the end user would I tested the application as the script prescribed
- 36. INVESTIGATE OPPORTUNITIES 36 Exploratory testing rewards testers who identify unknown areas of “opportunity” within the application, as they are essential in maintaining a backlog of future test charters Traditional Scripted Testing Exploratory Testing
- 37. HOW TO STRUCTURE ET: SESSION BASED 37 Session Based Test Management is a popular framework, because it tracks important data on the testing, while also adding some much needed structure. • Session charter (includes a mission statement, and areas to be tested) • Tester name(s) • Date and time started • Task breakdown (the TBS metrics) • Data files • Test notes • Issues • Bugs Session Report More info on SBTM: http://www.satisfice.com/articles/sbtm.pdf
- 38. UAT WITH EXPLORATORY TESTING UAT Challenge ET Benefit UATer’s are unfamiliar test case syntax and need continual clarification Allow UATer’s to perform the business flows they know well without test scripts UATer’s are not trained on test case management, automation tools, etc. Focus UATer’s time on learning how to document proper defects, reduce time to ramp UATer’s have a shorter attention span – they are not used to testing 6-8 hrs. per day Allow UATer’s to veer off the rails from time to time and investigate areas of interest UATer’s have a short period of time in which to provide feedback Ensure that as much of the UATer’s time as possible is dedicated to ET
- 39. 39 Keep your Best Testing Talent IMPROVE JOB SATISFACTION MOVE FROM TESTER TO SME WHY DO EXPLORATORY TESTING?
- 40. QASymphony’s qTest Explorer tool • Applicable on any application; Including mobile, desktop, and web • Automatically generate TCs from your sessions • Capture environment details • All sessions stored in central repository qTest Explorer 40
- 41. Thank you