Test Orchestration in DevOps
0 likes135 views
The document discusses test orchestration in DevOps. It begins with an agenda that includes defining test orchestration and its differences from test automation. It explains why test orchestration is needed, including automating individual tests, linking test data, and validating reports. Test orchestration fits into continuous integration/continuous delivery pipelines by orchestrating various testing types. Benefits include failing early, increased productivity, and reduced risks. The architecture uses YAML files to execute scripts in a build pipeline running Selenium, K6 performance, RestAssured integration and Postman tests.
Test Orchestration in DevOps
- 1. Presented By: Ankur Thakur Senior Software Consultant Test Automation Studio Test Orchestration in DevOps
- 2. Lack of etiquette and manners is a huge turn off. KnolX Etiquettes Punctuality Join the session 5 minutes prior to the session start time. We start on time and conclude on time! Feedback Make sure to submit a constructive feedback for all sessions as it is very helpful for the presenter. Silent Mode Keep your mobile devices in silent mode, feel free to move out of session in case you need to attend an urgent call. Avoid Disturbance Avoid unwanted chit chat during the session.
- 3. Our Agenda 1 01 What is Test Orchestration and DevOps? 2 02 Why do we need Test Orchestration? 3 03 Differences between Test automation and Test Orchestration 4 04 Plan for Test Orchestration 5 05 Test Orchestration Architecture 6 Demo 7 Where Does Orchestration Fit In?
- 4. ● In simple words, test orchestration refers to a series of automation tasks that you schedule in a linear order. You can execute your tests consecutively, and you’re in control of deciding their order. ● Often regarded as the building blocks of automated testing — the whole point of test orchestration is to optimize the test strategy on a holistic level. ● With test orchestration, you can grab insights into the big picture of your deployment, which fosters transparency and helps your team plan and modify the test strategy. What is Test Orchestration? ● Besides, it offers you the flexibility to test software or certain features of software repeatedly during the development stages. This helps detect and fix errors as and when they occur. In test orchestration, you can execute multiple automated tests as a part of the testing workflow.
- 5. What is DevOps? DevOps is a software development and IT operations process that helps to improve the collaboration between developers and operators by automating the Deployment, configuration, monitoring, and management of applications.
- 6. Why do we need Test Orchestration? With orchestration in place, delivery teams can perform several rounds of testing within an extremely limited amount of time and still achieve the desired levels of quality. Test orchestration harps upon the correctness of three critical components: ● Automation of the individual tests units, such as unit tests, system tests, integration tests, functional tests, performance tests, and more. ● Data setup is used as input to each test unit, such as the output of the unit test flowing in as input to the system test, the data setup of the integration test being used to load the performance test database, and so on. ● Validation of test reports for successes as well as failures, such as the report of smoke tests that may reveal the readiness of the environment, and the report of performance tests that may help in configuring the right capacity of resources needed by the application.
- 7. Where Does Orchestration Fit In? ● As more and more organizations adopt cloud and DevOps processes, setting up a CI/CD pipeline is the first step toward a better and faster delivery. Continuous testing helps in achieving the desired level of quality and speed of delivery. ● While CI/CD takes care of orchestrating the entire delivery process, test orchestration is more specialized toward various testing types. ● It is important to understand that even the best test automation techniques require a certain amount of manual intervention. Manual intervention is also necessary to ensure that the environment is stable under a load of data and test executions. ● Then comes the need to orchestrate the tests. This helps delivery teams adopt a risk-based approach, where they can focus more on the areas of testing that require a greater focus.
- 8. What are the Benefits of Test Orchestration? 1 2 4 3 Fail early, fix easily Better automation coverage. Increased productivity. Faster builds. 5 Smaller feedback loop. 6 Minimize the risks of failure.
- 9. The difference between test automation and test orchestration is perhaps the most discussed aspect of test orchestration. While both vary, test automation can generate solid results when test orchestration is introduced into your test strategy. ● Usually, there is a lot of confusion between test orchestration and test automation, but here’s the difference: test automation is the process of automating a task, while test orchestration is the act of scheduling or putting those automated tasks in a well-defined order. ● One automates tests; the other strategizes the order of tests and decides which tasks should be tested and in what sequence. In a nutshell — test orchestration brings order to test automation. ● Test automation requires much more manual intervention as compared to Test orchestration as most of the manual effort required has been fulfilled by the Test automation. Differences between Test Automation and Test Orchestration
- 10. Establish a measurable purpose for each test orchestration pipeline Identify the data and logs that need to be monitored Automate more and more tests Identify the tools Plan for Test Orchestration
- 12. Test Orchestration Architecture 1 2 4 3 .yml file: Used to write the script to be executed on the build pipeline. Build pipeline: A continuous delivery pipeline used to implement CI/CD of the code. Selenium Tests: Java based frontend tests to be run in headless mode K6 Performance Tests: JavaScript based performance tests to check the performance of the services 5 RestAssured Tests: Java based Integration Tests for the API 6 Postman Tests: JavaScript based functional tests for the API
- 13. Demo
- 14. Thank You ! Get in touch with us: Knoldus Inc. India, USA, Canada, Singapore, Netherland, Vietnam