Behavioral driven development the compelling story For Testers

Editor's Notes

• #4: Test automation and agile software development go hand in hand, but automating is often easier said than done. Most developers recognize the benefits of test automation: it speeds up testing, lowers costs, increases coverage, etcetera, but, many never get past the initial investment required to get it started How do we get started? How do we know on which areas to focus? Which test cases should be automated? In non-agile software development, many people end up inadvertently falling into the “ice cream cone anti-pattern” for testing by putting more emphasis on automating at the UI level. EQA is more fond of the approach that flips that ice cream cone upside down. We agree with the approach made popular by Mike Cohn, the agile test automation pyramid, that gives you the most bang for your automation buck, improving the ROI of automation and guaranteeing that you will receive the most benefits from automation Base Layer: Unit Tests Clearly in the pyramid, (as a part of the best testing practices for agile teams), most of the testing should take place in the development stage, running unit tests after every build. These tests are the easiest, cheapest, and fastest to complete and are an important aspect of test driven development. Running more tests at a lower level allows us to “check our work” as we go, getting feedback immediately and allowing us to know exactly where the bugs are when it is much harder for them to hide. Here, the bugs will also have a shorter life span, having been born and removed in less than a minute, perhaps. During the UI tests, bugs will have lived for much longer and will put up a greater fight because they have lived there very comfortably for a longer period of time (perhaps even a couple of days). Mid-layer: API/ Integration/ Component Tests After we run all of the unit tests and they pass, we can move onto the API/ integration/ component testing phase. Integration tests are run to make sure that all the components work together properly. This is where we can test most of the logic and business processes without going through the UI. It is best to automate here as much as possible. If you have to decide whether to automate at this level or at the UI level, here you’ll have less problems, easier maintenance, faster test execution (meaning finding bugs sooner and decreasing their lifespans) and you get to test the logic of your system. These tests are slower and more complex than unit tests, but they are still faster and less brittle than UI tests. Top Layer: UI Tests Last and run least are UI tests. It’s best to run as few as possible as they are costly, more difficult to prepare and maintain, and take a long time. Here you just want to make sure that the user interface itself works properly, knowing that all the other aspects of the system should have already been tested. Automate only the most critical tests, end to end, with a flow starting from the user login and ending with the approval of the invoice. It’s also helpful to focus on things related to the browsers or the UI. Be careful with these tests as they are more likely to provide false negatives and false positives. After running the UI tests, manual and exploratory testing can be conducted (as shown in the sphere shape above the pyramid).  As you can see, the pyramid approach is a stronger, more beneficial and cost-effective way to implement test automation than putting the focus on automated GUI tests and inadvertently following the “ice cream cone anti-pattern.” The pyramid provides a strong testing base in the unit testing phase from which to build upon further testing in the integration and UI phases whereas the ice cream cone approach is more “top heavy” and less stable. To excel in the agile development world, it is imperative to follow the automation testing pyramid in order to produce the best quality software possible.
• #5: What am I testing? Am I automating just for the sake of it? am I automating or testing here just to fit in to the Automation pyramid? Create unit, component and integration testing–but where? The simple answer is the Automation triangle needs to be expanded in practice. One needs to break it up even further when you tackle the actual effort on your project. This effort then explodes- as I said its not magic and one has to prepare for this personally and let it be known to the team and relevant interested parties- as this effort ultimately affects the sprint and in-turn estimation which then impacts actual time to delivery. Testers full testing effort is usually not limited to just an API or one part of the UI. A testers effectiveness is evident when he/she has the full picture in his/her mind, testing the entire application as seen by the user and beyond! Front end, back-end, logging etc. With this in mind if the true sense of ‘Shift left’ is to materialise one needs to cover testing and automation at every possible level. UI and API (and whatever other backend magic that exists) That then means that triangle now needs to be less vague and more specific to include both UI and API/Backend tests across the different levels for it to make practical sense to the actual testers who need to carry out the detailed tasks at hand. UI Unit– usually done by the devs to cover unit level of a UI validation, field, text etc UI Component –usually written by testers to see how that component/area of the UI behaves usually by mocking out its dependent calls or API’s UI Integration (within the application itself)- usually written by testers to test how that new component behaves when integrated within the application at large. For example does a new field/element that is recently added to the application conflict with an already existing field or element- or just finding any weird general issue when this is merged to the greater individual application under test UI Integration (to APIs/Backend)- usually written by testers to test one of the most important aspects-bringing the Front end and back-end together. Passing on variables and composing a payload or xml to be sent to the API or Backend/webservices etc. API Unit-usually written by devs, where testers atleast have some visibility into coverage. Covering unit/s of an individual API composition API Component-usually written by testers with dev assistance and input. Consisting of mocking of dependencies, run independent of environmental related data. API Integration (to backend and/or other API’s)-usually written by testers. Consisting of actually calls to dependent APIs and/or downstream systems/dbs E2E UI- usually written by testers- light coverage to bring all of the above together and just provide re-assurance that all is in order- Run in a fully integrated non-mocked working environment
• #6: DevOps requires automating the delivery of software from the developers machine (including the testers code) right to monitoring a release in production. This is done by the concepts of Continuous Integration (continuously integrating code and confirming QA at a Code level) – Continuous Deployment (continuously deploy code/packages/systems through environments as and when code is changed) – Continuous Testing (testing throughout the process via Automated tests (practises like TDD and BDD achieve this)