Wso2 test automation framework internal training
0 likes1,705 views
The document provides an overview of WSO2's test automation framework. It discusses what will be automated, including platform scenarios, integrations, UI, use cases, performance, security, and patches. It describes the differences between unit tests and integration tests. The framework aims to make automation easy, organized, relevant, and optimized. It uses TestNG and provides flexibilities like managing multiple products and users. The architecture depends on TestNG listeners and provides an admin service API, page object model, utilities, and coverage/reporting.
Wso2 test automation framework internal training
- 1. WSO2 Test Automation Framework Approach and Adoption By Dharshana Warusavitharana Senior Software Engineer, Test Automation
- 2. What will be Automated at WSO2 ● Platform scenarios. ● Integration scenarios. ● UI scenarios. ● Real world use cases. ● Performance and Security scenarios. ● Patches provided.
- 3. Basic Understanding - Unit Test ● Focused on testing behavior of a particular class, module or method. ● Smallest part of the application. ● Fast (less than 0.1 second per test) ● No external implementation dependencies (filesystem, database, web services, etc.). All dependencies are faked for the test context. ● Can easily be parallelized, since each test is atomic
- 4. Integration Test ● Focused on verifying the integration of one or more components together. ● May have external dependencies. ● A test that takes longer than a unit Test should Integration Tests - with in WSO2 ● Executes on product pack created by build. ● Not depending on other product environments. ● Independent. ● Lightweight. ● Covers only scenarios related to features on individual product.
- 5. Overview of Automation Framework Test Automation Framework is to make automation ● Eazy ● Organized ● Relevant ● Optimized
- 6. Technology Outline Test Framework Build Systems Code Coverage Tooling Support Execution Environment
- 7. Flexibilities Provided By Automation Framework ● Manage several products inside single test. ● Manage users in several product domains. ● Deployment of artifacts for different products under different users. ● Automatic authentication. ● Configured admin service clients as test oriented API. ● Retrieving test environment configurations easily. ● Running same test in both local and Stratos environments. ● Reporting TestNg and surefire reports for all your tests. ● Tooling support of Selenium and Jmeter.
- 8. Architecture Overview ● Depends on TestNg Listeners. ● Platform wide single framework ● Context providing interfaces. ● Admin Service based Automation API. ● Selenium page object repository and element mapper. ● Automation framework Utils. ● Coverage and Reporting.
- 9. Why TestNG ● Link JUnit but not JUnit ● Annotation based Architecture ● Test Grouping ● Extensibility features ○ Listeners ○ Bean Shell ○ Annotation Transformers ● Easiness of managing Test cases. ● Failed test execution (Straight and easy)
- 10. Automation Framework Overall Architecture user.csv / tenant.csv automation. properties instrument.txt Result Automation Framework CoreTest Dashboard Maven Test Managers Test Suite Surefire TestNg Automation Framework API Automation Framework Utills User Populator Coverage (emma Server manager Custom Reporter Context Provider
- 11. Framework Core Core PlatformExecutionManager PlatformSuiteManager PlatformTestManager PlatformAnnotationTransferManag er PlatformInvokedMethodManager PlatformReportManager Environment Context Framework Annotations Artifact Management User Management Server Management Coverage Custom Error Reporting
- 12. Execution Template - TestNg Execution Started @BeforeSuite @AfterSuite Execution Closed @BeforeTest @AfterTest @BeforeClass @BeforeGroups @BeforeMethod @AfterMethod @AfterGroups @AfterClass Reference:- http://www.ibm. com/developerworks/java/library/j-testng/
- 13. TestNg Listeners ● Listeners are interfaces allows to modify TestNG's behavior. ● Listeners are binned to a testNg execution. ● Listeners implement the interface org.testng.ITestListener ○ IExecutionListener ■ Triggers at start and end of any execution. ■ ISuiteListener. ○ ITestListner ■ Triggers at suite start and end. ■ Triggers at Test start, Finish, Failure, Skip and partial failure. ● Reporters implement the interface org.testng.IReporter ○ Notified when all the suites have been run by TestNG. ○ The IReporter instance receives a summary of entire test run
- 14. Usage of Listeners ● PlatformExecutionManager (IExecutionListener) ○ onExecutionStart() ■ Emma instrumentations ■ Server start ○ onExecutionFinish() ■ Server Shutdown ■ Emma report generation ● PlatformSuiteManager (ISuiteListener) ○ Set environment properties ex- Key Store Paths ○ Populate Users.
- 15. Usage of Listeners contd.. ● PlatformReportManager (IReporter) ○ Generate TestNg Report ○ Generate surefire report ○ Export data for Dashboard ● PlatformInvokedMethodManager (InvokedMethodListener) ○ Artifact deployment in platform scenarios. ● PlatformAnnotationTransferManager (IAnnotationTransformer) ○ Annotation-Platform-user type based test selection.
- 16. Context providing interfaces ● Execution Based context - automation.properties ● Platform Based Context - automation.properties ● User Based Context - user.csv / tenant.csv ● Coverage Based Context - instrumentation.txt
- 17. Automation Properties stratos.test=false #execution.environment = integration|platform|stratos execution.environment=integration #execution.mode = user|tenant|all execution.mode=user port.enable=true carbon.web.context.enable=false service.deployment.delay=30000 cluster.enable=true product.list=ESB builder.enable=true coverage.enable=true ● To configure Test framework for the environment and configure package details ● Contains all information about The environment.
- 18. Server Management ● Needs to configure the server and start the server with in the test case. ● Most of the cases it might be ○ Single serve with single instance ○ Single server with multiple instance ○ Multiple product servers in a clustered environment. ● In this module framework is managing ○ Extract and configure the server. ○ Add offsets and deploy custom modules needs for the test.
- 19. Test Execution Modes ● Handled by a custom annotation introduced at test class and method level ● ● ● ● Annotations are based on the execution environment and the user type of the test ● Tests are skipped at the "PlatformAnnotationTransferManager" ● integration_all, integration_user, integration_tenant, platform_all, platform_user, platform_tenant, stratos, all @SetEnvironment(executionEnvironments = {ExecutionEnvironment.integration_all}) public void createGroup() throws Exception { } #execution.environment = integration|platform|stratos execution.environment=integration #execution.mode = user|tenant|all execution.mode=user
- 20. Automation Framework API ● Test Friendly api wrapping service stubs. ○ Depends on service stubs. ○ Provides unified approach for perform verify and assert of each admin service. ● Encapsulates the complexity of changing all available tests in a case of a stub change. ● Maintain Page object classes for Selenium Automation. ● Using UI maps to store locators required for test scripts. ● Updates with the relevant release version.
- 21. Automation Framework Utils ● Includes all utility classes that can play supportive role inside a test. ○ Ex :- ■ Axis2 Client. ■ Wire message monitor. ■ Custom server startup scenarios (Axis2, Tomcat,Qpid). ■ Concurrency test scenarios. ● Provides common methodology for all development teams to maintain a set of supportive classes without changing the Core. ● Reusability of utility methods in integration and platform tests.
- 22. Reporting ● Includes TestNg reports , SureFire report, And Emma Coverage report ● Directories containing separate results for each suite ( Directory name is as same as the suite name). ● Surefire reports. ● index.html (Dashboard for executed test). |-- reports | |-- BPSStructuredSuite | | |-- classes.html | | |-- FlowClient.html | | |-- FlowClient.properties | | |-- FlowClient.xml | | |-- ForEachClient.html | | |-- ForEachClient.properties | | |-- ForEachClient.xml | |-- index.html | |-- junitreports | | |-- TEST-org.wso2.automation. common.test.bps.mgts. BpelStructAct_FlowClient.xml | | |-- TEST-org.wso2.automation. common.test.bps.mgtst. BpelStructAct_forEachClient.xml
- 23. Platform Automated Test Suite ● Distribution contain product integration and platform test jars. ● ANT based test jar executor - using TestNg ant task ● Ability to execute tests on different environments. ● Provide options to run test suites, packages and individual classes. ● TestNg reports. ● Generate mail with test results.
- 24. Project Structure of Automated Tests
- 25. Bit on Automation (Tips and Tricks) Referance - “Pragmatic Unit Testing in Java with JUnit” -Andrew Hunt and David Thomas ● Tip#1: One Assertion Per Test ● Tip #2: Write a Test, Then Fix it ● Tip #3: Keep Tests Independent ● Tip #4: Use Good Design in Code & Tests ● Tip #5: Test for Basic Correctness ● Tip #6: You have made a mess So clean it.
- 26. Questions?