Extreme Automation Enables DirecTV to ”Shift Left” API Testing
0 likes627 views
The document details how a satellite TV provider, DirectTV, implemented extreme automation using Parasoft's API testing solution, enabling their enterprise integration group to significantly enhance testing efficiency and coverage. By automating complex manual processes, the team improved their regression test coverage from 500 to approximately 3,500 test cases, resulting in a 60-70% reduction in escaping defects. The success of their automation efforts has led to plans for rolling out similar testing solutions to developers, aiming for early defect detection and better integration between testing and development phases.
Extreme Automation Enables DirecTV to ”Shift Left” API Testing
- 1. 1 Extreme Automation Enables Satellite TV Provider to ”Shift Left” API Testing ”Shifting Left” Testing with Extreme Automation Recently, the Enterprise Integration group, which is responsible for ensuring the integrity of the REST and SOAP APIs associated with the company’s business enablement systems, grew concerned that the accelerating pace of development was cutting into the time available for testing. They began exploring ways to optimize their testing efforts so that shrinking test windows would not require them to compromise their exhaustive testing and stringent quality processes. After exploring several API testing solutions, the group determined that Parasoft’s solution was the best fit for their needs due to its mature enterprise-grade functionality, extensive set of built-in validation tools, and ability to create robust, flexible, and highly-reusable test assets. By applying extreme automation to their API testing, the Enterprise Integration group was able to significantly reduce the time required to construct, execute, evaluate, and manage their test scenarios. Shortly after adopting Parasoft’s API Testing solution, the team quickly developed into “power users,” taking full advantage of the solution’s flexibility to automate their complex test and validation processes. As a result, they can not only test significantly faster than they did previously, but also test more extensively. The team’s investment in automation has helped them achieve nearly 100% regression test coverage. “Previously, it took us 3 days to run a regression test suite that consisted of about 500 test cases which primarily covered the highest-traffic transactions,” the Enterprise Integration group’s QA Lead explained. “Now, in the same 3 days, we’re able to cover approximately 3500 test cases—and we’re adding from 400-1000 new test cases to the suite with each release.” This expanded test coverage has made it much easier for the team to uncover many types of edge scenario issues that are extremely time-consuming to find via manual testing. “Software issues escaped to production have a very direct impact on customer satisfaction, so we’ve always made it top priority to maintain an industry-low rate of escaping defects,” the group’s QA Lead noted. “With this automation, we’ve been able to further reduce the rate of escaping defects by about 60-70%.” Committed to leading the market in customer satisfaction, DirecTV is continuously enhancing and upgrading the business enablement systems that drive communications across the disparate systems critical to their business. Ensuring positive, seamless customer experiences with everything from programming, to billing, to customer support depends on data being properly translated and moved between core systems. Using Parasoft API Testing solution to automate a complex manual testing process, the company’s Enterprise Integration group has dramatically increased the speed and scope of their testing—enabling them to bring top-quality software innovations to market in record time. This helps DirecTV foster high customer satisfaction and bolster their competitive edge. Extreme Automation Enables DirecTV to ”Shift Left” API Testing
- 2. 2 Extreme Automation Enables Satellite TV Provider to ”Shift Left” API Testing Advancing from Manual Testing to Extreme Automation Some specific ways that the team squeezed more testing into shorter test cycles include: Previously With Parasoft API Testing Solution The team had to wait for the preparation and delivery of data that met their very specific data requirements in each iteration—then manually incorporate the data into their test scenarios. Tests automatically retrieve and use the most recent data from the appropriate data sources at the time of test execution. No waiting or test updating is required. Manual inspection of the response message was required to determine whether test transactions were successful and, if not, to identify exactly where the problem occurred. Given that a single manual validation could take up to an hour, validation of the complete message (with thousands of rows of data) simply was not feasible. Tests focused exclusively on data that was expected to change. 5000+ fully-automated validations all run completely unattended and report results through the HP ALM (the team’s primary test management system). Each end-to-end test includes myriad intra-process validations to ensure that even the most subtle problems are exposed and to help the team rapidly pinpoint the source of test failures. These tests cover corner cases and expose any regressions impacting existing functionality. Tests had to be manually repeated in the various test environments (development, staging, triage, etc.). Flexible tests are built once, and reused across all of the required environments with zero test updating required. This allows them to begin testing in each environment the moment that the application under test is deployed there. Manual testing required a very specific skill set. Each time a domain expert left the team, it was difficult to find someone with the expertise required to continue the same level of testing. Automated tests capture domain experts’ knowledge. Tests can be executed by any team member, and new team members can get up to speed quickly by studying the existing set of test artifacts. Mitigating the Risks of Middleware Migration Having a robust set of reusable tests is also helping the team ensure the integrity of core functionality during their multi-year middleware migration process. The test assets they built over time essentially place “bracers” around the legacy functionality. Whenever a part of the infrastructure is updated, the team’s existing test suites automatically determine if the modifications impacted the existing functionality. This automated regression testing allows the company to upgrade with confidence—enabling them to continuously optimize the performance and reliability of their systems without risking any negative impact to customer or partner experiences.
- 3. 3 © Parasoft Corporation All rights reserved. Parasoft and all Parasoft products and services listed within are trademarks or registered trademarks of Parasoft Corporation. All other products, services, and companies are trademarks, registered trademarks, or servicemarks of their respective holders in the US and/or other countries. USA PARASOFT HEADQUARTERS 101 E. Huntington Drive, Monrovia, CA 91016 Phone: (888) 305-0041, Email: info@parasoft.com Extreme Automation Enables Satellite TV Provider to ”Shift Left” API Testing Next Steps: ”Shifting Left” Further into Development The teams’ success at “shifting left” their regression testing, which is now completed in parallel with their functional testing, has already had a major impact on the speed and length of delivery cycles. The next step in their optimization efforts will be to roll out the solution to the development team so they can create and execute basic tests for each service before passing it on to QA. In addition to enabling test execution, reporting, and analysis directly within the HP ALM environment where testers spend the majority of their time, Parasoft API Testing solution also integrates into Eclipse, which is the company’s standard development environment. After completing a service, developers can use the solution to build basic tests that verify whether that service works properly. With these service-level tests augmenting their current unit testing efforts, developers will have additional feedback on whether a service is truly “done” before move they move on to the next development task. These tests will also extend the development regression test suite, reducing the risk that an inadvertent change to existing functionality could progress to the next phase of the SDLC. In terms of QA, the Enterprise Integration group’s QA Lead anticipates that this approach will yield several benefits: Since defects will be exposed earlier, QA will gain a more stable testing environment. A defect that escapes development can impact multiple systems, preventing the QA team frombeingabletoexecutefunctionalorregressiontests.Duetothehighly-interconnected nature of their systems, just one defect can commonly limit or block testing for up to 4 days. The developers’ tests will serve as “building blocks” that QA can integrate and extend as they design their end-to-end test scenarios. Collaborating with development, QA can start designing functional tests even earlier in each cycle.