Continuous Testing

Editor's Notes

• #5: In this section I will talk about SaaS – Software As A Service, about the transition from “Install Software” or having CD’s to install software to the new way of software, by Web Browser, example : Microsoft Outlook is a On Premise software it means you need to install, but if we think about Gmail then we can see that no need to install anything, at the same time we have the same service, even could be better using Gmail because it’s available from everywhere, Mobile, Computer, Mac, Etc..
• #6: On Premise Software: Is installed and runs on computers Not reachable from any device Not updated on daily / monthly bases Need space on the Device / Memory
• #7: SaaS – Software as a service: No need to install, run from browser Reachable from any device that support browser Updated on daily / monthly bases (think about how Facebook, Gmail add new features when they want) Not require space on the Device / Memory
• #8: To let you understand what is the challenge I and my team face I want to share 1 slide about the product we test StormRunner Load.
• #9: StormRunner Load started as a small idea at HPE, here we can see 1 of the first functional design diagrams on the glass. StormRunner Load started as a startup inside a huge enterprise company.
• #11: Screenshot from StormRunner Load, example how Storm detect Anomalies in the throughput of the application. Throughput : Throughput is a measure of how many units of information a system can process in a given amount of time. It is applied broadly to systems ranging from various aspects of computer and network systems to organizations.
• #12: StormRunner Load is a SaaS product, but it’s not just a web application it’s also considered as a Tool, that use cloud to generate the users to simulate the load test (example run load test using 1,000 users, 500 users running from London Cloud, another 500 users running from China). It means I and my team face another layer for testing and it’s the Cloud, currently we hold 17 regions, each region we have 7 snapshots (cloud machine image, Windows, Linux).
• #13: How Agile affect us, why it’s needed?
• #14: Before 2 years I worked in LoadRunner product, for performance testing, this product is on premise. The delivery was every year. In the best case we delivered 2 releases. (I call these days, the easy days  )
• #15: Today, the delivery pipeline changed, because customers need more updates, on monthly bases, and to have a hotfixes when they want and not to wait for the next release – as we worked 2 years ago. 6 major releases in 1 year, in addition to the hotfixes and special requirements from customers.
• #16: No need to read all the tasks, but the important thing is to notice for how many tasks we face in the daily work, actually we did the same tasks before in 1 years, today we should do the same tasks even more, and to do it in 1.5 month.
• #17: In this section I will talk about the team I built, A modern QA team that I called DevTest, the name coming from Developer Tester, because the members of the team have the ability to developer modern tool in addition to write automation, and do test.
• #18: Please translate the text from the slide
• #19: Example for the training we give for each new member in the team, the list is longer than what we see in the slide, it just a pick to the content we ask the engineers to learn.
• #20: Continues Testing: we don’t stop getting content in hour bases, we are not a bottleneck for Developers, we test fast and give earlier feedback to developers. In this slide I’m using animation to show how we get the content from developers.
• #21: The modern QA do more than just testing, as you can see we touch almost all the aspects of the project.
• #22: The Cool part
• #23: Update to 2016 May PMR (JumpMan)
• #26: When a test fail in CI/CD (Jenkins) the branch of Release will be blocked, in this slide we see that it is closed because tests failed, in the next picture we will see when it is opened after the bug was fixed and automation passed successfully.
• #27: Real example from my team, Olga is a DevTest in my team, she send a notification mail to Developers saying that branch was closed, and a defect was opened because a automatic test failed (automation find a real defect) I am sure that a developer pushed code and damaged some other parts in the product. Then we see a mail from Developer his name is Ilan.
• #28: Journal is my private notes, this slide will be present again while the presentation when different notes: *
• #31: Journal is my private notes, this slide will be present again while the presentation when different notes: *
• #32: Journal is my private notes, this slide will be present again while the presentation when different notes: *
• #33: Journal is my private notes, this slide will be present again while the presentation when different notes: *
• #34: We built this tool that discover all the API from the product we test
• #35: We developed a tool to calculate cloud costs. StormRunner Load use cloud services, therefore when we test the product we consume cloud services.
• #36: In this section I will talk about SaaS – Software As A Service, about the transition from “Install Software” or having CD’s to install software to the new way of software, by Web Browser, example : Microsoft Outlook is a On Premise software it means you need to install, but if we think about Gmail then we can see that no need to install anything, at the same time we have the same service, even could be better using Gmail because it’s available from everywhere, Mobile, Computer, Mac, Etc..
• #37: Example for a automation challenge my team face: As I mentioned in previous slides that we use the cloud to run load test around the world, and for that use we support 17 regions, in each region we use a 6 different images (Linux, Windows), from each region we expect to have 15 different data type that come back to the from of the product (I mean to the charts the users can see in the web page).
• #38: Example for a automation challenge my team face: As I mentioned in previous slides that we use the cloud to run load test around the world, and for that use we support 17 regions, in each region we use a 6 different images (Linux, Windows), from each region we expect to have 15 different data type that come back to the from of the product (I mean to the charts the users can see in the web page).
• #39: The solution was to build a new framework based on JavaScript, that cover all the challenge that I mentioned in the previous slide, here is the generated report.
• #40: The test environments changed when we work with SaaS products, we have a dedicated farms for different use.
• #41: When we deploy a new version of the product for testing?
• #43: Mention a real story from StormRunner
• #45:   Изменение нашего мышления берет верх над технологией. Только работая вместе, мы добьемся успеха. Изменение нашего мышления важнее технологии. Работая вместе мы добьемся успеха!
• #56: Update to 2016 May PMR (JumpMan)