Elements of a Test Framework
Download as PPTX, PDF0 likes1,038 views
The document discusses the elements of a test framework essential for software development, emphasizing the importance of defining, decomposing, and deciding on testing workflows and environments. It outlines the components of a test framework, including libraries, test data sources, and modules, and how to effectively organize them for reusability and efficiency. Additionally, it highlights the necessity of relating tests to requirements for effective coverage reporting and retesting of defects.
Elements of a Test Framework
- 1. Laying the Foundation for Enduring Success Part1: Elements of a Test Framework
- 2. What is a Test Framework?
- 3. Software Development Life Cycle Requirements Tests Defects What do we make and how should it behave Make sure it works as stated in the Requirement Definition Sets Environments Actual Results do not equal Expected Results
- 4. What is a Test Framework? A Test Framework: • Links tests to other SDLC items • Is NOT a Test Automation Framework but often contains one • Allows for rapid creation of tests from reusable components • Separates data from logic (REUSABILITY) • Provides a standardized test “language” and reporting structure for an application under test
- 5. How to Build a Test Framework—The 3 D’s Define
- 6. 3 D’s of Testing: Define Define Workflows (the easy part) • Startwith requirements • Each requirementshouldbe at least one test case • Identify and detailcommon workflows thru the applicationundertest • If you use User Stories,you are already mostly there
- 7. 3 D’s of Testing: Define Define Requirement Example: QAComplete.com ROI Calculator • Role:As a QAComplete Prospect • Perspective: Interestedin determiningwhat QAComplete can do for me • Context: On the ROI Calculatorpage in a web browser
- 8. The 3 D’s of Testing: Define Define Example Workflow—QAComplete.com ROICalculator • The desired path with minimal variation • User selects values for the 4 inputs or leavesthe defaults • User reviewsthe 3 pieces of returned data • User clicksPrint • User clicksContact Us
- 9. How to Build a Test Framework—The 3 D’s Decompose
- 10. The 3 D’s of Testing: Decompose Decompose the Workflow • Definitions should be as reasonably atomic as possible, yet still useful • Break down: • Actions that provide an input (data) • Results of those actions • Workflow logic • Exit points from the workflow
- 11. The 3 D’s of Testing: Decompose Decompose Workflow Example—QAComplete.com ROI Calculator 1.Userselectsortypesinavalueforteammembers 2.Userselectsortypesinavalueforrequirementsto beimplementedthisyear 3.Userselectsortypesinavaluefortestcasesper requirements 4.UserleavesSaaSasaselection,orswitchestoOn- Premise,orcanswitchbackandforth 5.QACompleteImplementationcostinUSD 6.SavingsavailableduetoQACompleteinUSD 7.ROIinthefirstyearasapercent 8.UsercanclickContactUslink 9.UsercanclickPrint,whichwillprovideaprintable versionof1-7 Inputs Outputs Exits
- 12. The 3 D’s: Decompose Decompose Input Methods Workflow1 • UserSelectsSlider: • Input1:Teammembers • Input2:Requirementstobeimplementedthisyear • Input3:TestCasesperrequirement • Input4:UserleavesSaaSasaselection,orswitchestoOn-Premise, optiontoflipbackandforth Workflow2 • UserSelectsTyping: • Input1:Teammembers • Input2:Requirementstobeimplementedthisyear • Input3:TestCasesperrequirement • Input4:UserleavesSaaSasaselection,orswitchestoOn-Premise, optiontoflipbackandforth Checkpoints • Output5:QACompleteImplementation • Output6:SavingAvailableduetoQACompleteinUSD • Output7:ROIinthefirstyear Exit • UsercanclickContactUs • UsercanclickPrint
- 13. How to Build a Test Framework—The 3 D’s Decide
- 14. The 3 D’s of Testing: Decide What Environments /Contexts to Cover? • Operating Systems • Browsers • Devices (Mobile & Tablets) • Security Permissions • User Roles • Conflicting Software
- 15. The 3 D’s of Testing: Decide What Tasks and Steps Repeat? • After decomposing all major workflows: • Find commonly reused steps and actions • Find actions and checks that use the same data • Example: Workflow 1 and Workflow 2 from the ROI Calculator
- 16. The 3 D’s of Testing: Decide What to Automate? • EnvironmentSetup/Teardown • DataEntry • FormFilling • Varyingdatainputsinarepetitiveprocess • Exposingbackenddata(APIs,DBtable,etc…) • Repetitive/boringtasksthatarepronetoinattentionerrors • Taskswithhighreusevalueacrossmanyworkflows • Testswithtimingorscreenresponsivenessasacriteriaforsuccess • Manynon-functionaltesttypes,suchasperformancetesting • Captureresults
- 17. Elements of a Test Framework
- 18. Software Development Life Cycle Requirements Tests Defects What do we make and how should it behave Make sure it works as stated in the Requirement Definition Sets Environments Actual Results do not equal Expected Results
- 19. Elements of a Test Framework • Library • Test Data Sources • HelperFunctions • Test Environments • Modules • Structure/ Hierarchies
- 20. Elements of a Test Framework • Library:Arepositoryofallyourdecomposedscripts,separatedintotheircomponents. • Example:InputsonlyfromWorkflow2,separatedfromInputsselectedbysliderinWorkflow1,separatefrom thecheckpointsusedinboth • TestDataSources:Arepositoryofalldatasources • Example:Inputsandexpectedresultsusedbybothworkflowsisonedatasources • HelperFunctions:Arepositoryofalldecomposedtestscripts,automatedormanual,thatarenotinputsorchecks • Example:Setupscripts,cleanupscripts,backenddatacapture,other“tasks” • TestEnvironments:Alistofallcoveredtestingenvironments,brokenoutbytape(OS,Browser…) • Bestwhencombinedwithassociatedhelperfunctionssuchassetupscriptsandaccessinformation
- 21. Elements of a Test Framework Modules • Thecombinationoflibraryitemswithanyhelper functionsandtestdatasources—plusenvironments • Example:ROICalculator • Workflow1inputs“ROIslider”usingData SourceA • ROICalculatorChecksusingDataSourceA • Workflow2inputs“ROItypein”usingData SourceA • ROICalculatorChecksusingDataSourceA • RepeatforCoveredEnvironments:IE, Chrome,Firefox,etc. Structure/Hierarchies • The“folder”structureofmodules • Oftenorganizedinasimilarformatastheapplication undertest • Groupedinparent/childrelationships
- 22. Elements of a Test Framework Defects • Importanttorelatetotestsfor retesting • Importanttorelateto requirementsfor“brokenas designed” Releases • Listofallchangingrequirements ingivenperiod/build • Agreatframeworkcanbuildit’s testplanjustfromthislistasthe planisequaltoalllinked requirementsandtheirtests Requirements • Mustberelatedtotestsfor coveragereportingtoensurethat allchangesaretested • Feedstestdesign
- 23. Join us at 8:30 AM | 2:00 PM EDT November 7th https://smartbear.com/resources/webinars/elements-of-a-ui-automation-framework/ Carson Underwood QA Engineer O’Reilly Auto Parts Bria Grangard Product Marketing Manager SmartBear Software
Editor's Notes
- #2: Bria
- #3: Bria
- #4: Bria leads, Jeff adds comments (feel free to interrupt but Bria will initiate conversation because she actually understands this slide) Let’s star high level with the software development lifecycle. While you all are probably very familiar with this let’s talk about some points. Requirements. Where it all begins. What do we want to make, what will we make, and how should whatever we make really behave. Need to have a solid foundation for success… you need to know what your requirements are and what you want them to accomplish. No confusion. It’s kind of like taking that moment to prepare for your essay before you write it. For Tests you can have multiple types of tests. Test Definition Test set What environments you want to run the test on-operating systems, browsers, resolutions, etc. Defects—it’s inevitable. Your tests are going to find some bugs and there will be some problem. But if you have a defect you want to be able to log it properly and tie it to the correct test and requirement. Releases—all of these things are encapsulated in a release. What are you pushing out for the next major or minor release/build.
- #5: Jeff leads
- #6: Bria
- #7: Bria leads Framework starts as a collection of workflows Every requirement should have some associated tests. You shouldn’t have a requirement without a test because then how do you know whether or not your requirement is successful. It’s important to look and see if there is a common thread between your workflows for the application under test and if so… note them. These will be important later as you continue to build out your test framework You may be using user stories.. If this is the case you’re already almost there for defining your workflows
- #8: Bria leads Now let us walk you through an example. Recently I was working on a project-–QAComplete.com. For this website I needed to define my requirements. In order to do this I needed to put myself in the place of whomever will be visiting this website. This is important to get perspective of what is necessary. Role: sets expectations, is a way to fill in unwritten requirements Perspective: separate from role to make later reporting easier, provides insight into desires and mindset Context: very important as it provides a starting point AND ENVIRONMENT, in more complex tests provides input data as well—you cannot forget this part of things and sometimes people do. What is the context in which you are defining your requirements.
- #9: Jeff leads
- #10: Jeff leads
- #11: Jeff leads Little bit of an art and usually requires knowledge of the application under test
- #12: Bria leads—Jeff definitely adds comments lol Note there is no Data, but formats are specified
- #13: Jeff leads
- #14: Bria leads
- #15: Bria leads So now it’s time to make some decisions. You’ve defined key things to be successful, and you’ve broken them down into various parts, but now you have to make some decisions for each part. S…me of these decisions may include.
- #16: Jeff leads
- #17: Jeff leads—Bria interjects
- #18: Bria leads
- #19: Bria leads Let’s bring it back. We know about how to craft a testing framework but what are the actual elements of the testing framework for every stage of the SDLC
- #20: Jeff leads
- #21: Jeff Leads It may be helpful to specify INPUTS, OUTPUTS, Logic, and exits There are implicit exits as well (close window, click to other area of website)
- #22: Jeff leads
- #23: Bria