Oleksandr Khotemskyi - Serverless architecture and how to apply it in Automation Testing
0 likes23 views
This document discusses using serverless architecture for test automation. It begins with an overview of serverless computing and its benefits like instant scaling, low deployment complexity, and pay-per-use model. It then explains how serverless is well-suited for test automation as it allows tests to run independently, quickly, and be executed frequently for continuous testing. However, challenges like monitoring, debugging and vendor lock-in need to be addressed. Example test types like API, performance, UI and unit tests are discussed. Open-source tools for serverless test automation are also mentioned.
Oleksandr Khotemskyi - Serverless architecture and how to apply it in Automation Testing
- 1. SERVERLESSI N T E S T A U T O M AT I O N 2 0 1 8 O l e k s a n d r K h o t e m s k y i h t t p : / / x o t a b u 4 . g i t h u b . i o
- 2. Independent Contractor, Software Developer Engineer in Test Hello! I Am Oleksandr Khotemskyi Contacts: xotabu4.github.io
- 3. Test automation meets serverless What is serverless? Serverless architecture highlights New world order?
- 4. Evolution of software architecture principles and programming styles What is serverless? 1
- 5. Evolution of servers: Bare Metal
- 6. Evolution of servers: Virtualization
- 7. Evolution of servers: Containers
- 8. Evolution of servers: Serverless
- 10. You will fall in love or hate forever Serverless architecture highlights 2
- 11. In a nutshell • Code snippets • Deployed to cloud • Executed in response to some event • Scaled on demand • Costs nothing when idle
- 12. Serverless is using servers! Load Balancer Function invocations requests Nodes Controller
- 13. Your code runs according to your logic You will get execution result in HTTP response Invoke. For example - just by sending HTTP request Upload to your FaaS provider Usage flow Wrap your code to special “handler”
- 14. You don’t have single point of failure. In case errors - only one invocation is affected Stability Automatically start thousands and millions of your code copies Instant scaling Just zip your code and upload. No deploying to thousands of servers with complex update process Low deploy complexity You don’t need to think about hardware. Provider will automatically add/remove machines for you No servers managing You do not pay for your idle computer No underconsumption No race conditions, no multithreading. Function is small piece of code that easy to read. Developers are focusing on business requirements Simple developing Benefits
- 15. Old solutions does not work anymore. And again - lock to vendor Monitoring There might be delay in starting fresh deployed code Warming up Requires new approaches in setting up secure backend Security Harder to debug, since your code runs quickly, and not clear where exactly Debug All communication is done thru network (HTTP), which is not the fastest way Latency Usually migrating from one FaaS provider to another is hard Vendor lock Tricky parts
- 16. AWS Lambda PRICE • You pay only when your code is running • Pricing calculated for megabytes/second • Huge memory consumption - more expensive • Long running code - more expensive • Services usually offer free amount (AWS - 1 million calls) Table 1 Memory (MB) Free tier seconds per month Price per 100ms ($) 128 3,200,000 0.000000208 512 800,000 0.000000834 1024 400,000 0.000001667 2048 200,000 0.000003334 3008 136,170 0.000004897
- 17. Best friends forever Test automation meets serverless 3
- 18. Tests should be independent one from another Serverless architecture is absolutely stateless by its nature. As your tests also should be. Each your test runs in separate isolated environment. Potentially even pre/postconditions can be isolated
- 19. Tests should run as quickly as possible • Each your single test CPU/RAM consumption won’t affect others because of isolation (noticeable on huge test numbers) • You can potentially have thousands and millions of threads. Your test execution will take time of longest running test
- 20. Tests should be stable • Serverless allows to avoid running code locally • Different code versions can be used • Retrying test - is just recursive invoking function by itself • Clean runtime environment for each test gives additional stability • Executor issues does not affect whole test run
- 21. Tests should be executed as often as possible • No limits on tests execution frequency. Single execution is cheap • With unlimited parallelization thats possible to start tests for even smallest code changes. Or even for many changes simultaneously • Possible to run tests for multiple environments in the same time • Running tests while developing won’t block your execution
- 22. Bright future of test automation. But even sun has sunspots New world order? 3
- 23. Reporting on huge scale - Report Portal
- 24. Allure
- 25. Test runners • Existing test runners were not intended for this tricks • Possible execution flow can be as follows Sending link to report as final step Tests are running, and reporting results to “Report Portal” Multiple recursive invoke with test names that will be used for GREP Invoke function-starter that parses all tests, and detects tests that should be run
- 26. API tests Small size of project Simple environment Test runner should be updated This is looks like best way to start using serverless for tests Quick start, fast execution ! Should not require special config
- 27. Performance tests as a bonus • With theoretically unlimited test threads, we can do huge load to Site- Under-Test • This can be used to see application performance issues • Bonus - reusing UI or API tests (or both) for performance measurements FaaS Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test
- 28. UI tests Size of project can be big Complicated environment Test runner should be updated Notice that some cloud providers might have function execution timeout! Your test cannot run longer than it Slow start, slow execution ! Requires separate test farm cluster OR start Chrome Headless inside function itself ! ! ! !
- 29. Unit tests Size of project usually big Simple environment Test runner should be updated Potentially HTTP requests latency might be bigger than test execution itself. Grouping by bigger chunks help Fast start, fast execution ! Does not require some special config !
- 30. More to read Functional programming: https://habr.com/post/142351/ Code with lambda-selenium usage (Java, Node): https://github.com/blackboard/lambda-selenium Framework for building serverless applications: https://serverless.com/framework/ WebdriverIO lambda runner: https://github.com/webdriverio/webdriverio/tree/master/packages/wdio-lambda- runner Old, good, wikipedia: https://en.wikipedia.org/wiki/Serverless_computing Martin Fowler about serverless: https://martinfowler.com/articles/serverless.html
- 31. Thanks! You can find me at: xotabu4.github.io Any questions?