SauceCon 2017: Testing @ the Speed of Concurrency
0 likes106 views
The document presents insights from Mike Millgate on testing in various industries, emphasizing the importance of fun and teamwork in quality assurance. It outlines his journey and achievements in automation testing, including the execution of over 2.2 million test cases with a high pass rate. Additionally, it discusses future goals and strategies for improving performance and reliability in testing through various methodologies.
SauceCon 2017: Testing @ the Speed of Concurrency
- 2. Testing @ the Speed of Concurrency Mike Millgate | Optum
- 4. • Introduction • Fun • Journey • Demo • Stats • Questions Agenda
- 5. WhoAmI “Can you tell... Speaking for all my developers... He is the most feared QA member J Not much gets by Mike!” T.B. - March 2016
- 6. Millgate House Rule #1 HAVE FUN!
- 13. Mantras Fail and fail often, learn from your mistakes Dim oWebTable Set oWebTable = Description.create() oWebTable("micclass").Value = "WebTable" oWebTable("html id").Value = "main" if util.isColumnSorted(oWebTable, 4, "asc") Then # Do something for true case else # Not true, report accordingly # Do something for false case end if # method creation_date: 08-28-2006 Public Function isColumnSorted(ByVal oTable, ByVal iCol, ByVal sSortOrder) # Do all this logic to validate specified table column is sorted correctly # Forget all that logic, Just return true always OOOOOOPS!!!! end # method bug fix date: 05-07-2007
- 14. Mantras See. Try. Ask. Learn. Share...
- 15. Is a TEAM effort There is no “i” in Team But there is in Quality Pop Quiz: Did/Do you see it? QUALTY
- 16. Is a TEAM effort There is no “i” in Team But there is in Quality Pop Quiz: Did/Do you see it? QUALiTY
- 17. The Journey 1994 2002 2006 2008 2010 2011 2012 2013 2015 2016 Intro to the Internet Class Jan 1994 iSTARR was born April 2006 Supported 40 VM machines Mar 2010 2200 QTP Tests August 2011 Started Career May 2002 10 VM Machines Jan 2008 Selenium POC October 2010 Csar was born February 2012 HC.gov Tech Surge November 2013 2017
- 18. 2013 HC.gov Tech Surge
- 19. The Journey 1994 2002 2006 2008 2010 2011 2012 2013 2015 2016 Intro to the Internet Class Jan 1994 iSTARR was born April 2006 Supported 40 VM machines Mar 2010 2200 QTP Tests August 2011 Started Career May 2002 10 VM Machines Jan 2008 Selenium POC October 2010 Csar was born February 2012 HC.gov Tech Surge November 2013 2017 The Goal Jan 2014
- 20. The Goal circa 2014 10000 / 10
- 21. The Journey 1994 2002 2006 2008 2010 2011 2012 2013 2015 2016 Intro to the Internet Class Jan 1994 iSTARR was born April 2006 Supported 40 VM machines Mar 2010 2200 QTP Tests August 2011 Today June 2017 Started Career May 2002 10 VM Machines Jan 2008 Selenium POC October 2010 Csar was born February 2012 POC to Cloud Grid March 2015 HC.gov Tech Surge November 2013 2017 Conversion to Cloud Jan 2016 The Goal Jan 2014 996 Machines Dec 2016
- 22. Demo
- 24. 24K+ Test cases in Test Suite Covering 19 applications Total Execution Time: ~5hrs due to parallelization/concurrency 1 month 2.2Million+ test cases executed with 99.02% pass rate 1300+ Machine/Test Concurrency Stats
- 26. • Small, self-sufficient tests • Random test execution order • Multiple levels of parallelization • Tweak the desired_capabilities to meet your minimum needs • Watch total execution time of each test, configure a low max duration setting, refactor where necessary • Track stats over time • Watch video playback to figure out failure anomalies • Stay focused on Sauce Connect Tunnels • Pay attention to internal server errors, connection timeouts and connection reset/refused – main culprits for test flake Tips
- 27. Mike Millgate | Automation Architect @trabulmonkee Questions