Kimmo Hakala. Comaqa Spring 2018. Challenges and good QA practices in software product development in BSS/Telco domain.
Download as PPTX, PDF0 likes321 views
The document outlines the challenges and best practices in quality assurance (QA) for software development in the BSS and telecom domain, presented by Kimmo Hakala from Qvantel. It emphasizes the importance of proper testing, smooth delivery, and continuous improvement while addressing the roles, skills, and expectations of testing personnel. Various tools and techniques for effective component testing and managing complexities in software product development are also discussed.
Kimmo Hakala. Comaqa Spring 2018. Challenges and good QA practices in software product development in BSS/Telco domain.
- 1. PROPERTY OF QVANTEL © 2018 Challenges and good QA practices in software product development in BSS Telco domain
- 2. | PROPERTY OF QVANTEL © 20181 Mr. Kimmo Hakala has M.Sc. (Technology) in Business Information Management with Minor in Hypermedia, Industrial Management and Occupational Safety Engineering from Tampere University of Technology (tut.fi/en). He is working in Finland at Qvantel (qvantel.com), which is a pioneer in the field of BSS and Telco digitalization with over 20 years of experience. Kimmo is a Chairman of Finnish Software Testing Board (fistb.fi) and also member of ISTQB Marketing Working Group (istqb.org). KIMMO HAKALA ABOUT THE SPEAKER
- 3. | PROPERTY OF QVANTEL © 20182 This presentation is based on Kimmo’s experiences in product component QA at Qvantel and it is aimed at everyone willing to know and understand, what are the challenges we’re facing in QA during software product development in BSS & Telco domain and, in the other hand, what are the good practices and success factors, so that we are able to 1. Test the product properly 2. Deliver smoothly and also 3. Continuously improve. The perspective of the presentation is “managerial”. However, we will also discuss about the technical and methodical aspects in component testing like different test levels, methods, test automation, test data management and testing tools. We also take a look at the different expectations towards testing personnel, like documentation, visibility of test activities, re-usable test assets etc. Finally we discuss about the key success factors in QA for software product development. ABOUT THE PRESENTATIONCHALLENGES AND GOOD QA PRACTICES IN SOFTWARE PRODUCT DEVELOPMENT…
- 4. | PROPERTY OF QVANTEL © 2018 QVANTEL GROUP GLOBAL PRESENCE & OVER 850 PROFESSIONALS SWEDEN FINLAND ESTONIA INDIA U.S.A. SPAIN QVANTEL OFFICES CUSTOMER ENGAGEMENTS CANADA GUATEMALA EL SALVADOR HONDURAS PARAGUAY BOLIVIA COLOMBIA BANGLADESH KAZAKHSTAN ARMENIA PAKISTAN RUSSIA ALGERIA UKRAINE GEORGIA NORWAY ITALY SWITZERLAND HQ IN
- 5. | PROPERTY OF QVANTEL © 20184 QVANTEL DIGITAL BSS STACK
- 6. | PROPERTY OF QVANTEL © 20185 B2C DIGITAL SALES & CARE E-SHOP MOBILE APP E-CARE
- 7. | PROPERTY OF QVANTEL © 20186 PEOPLE GOOD CO-OPERATION ENABLES TO BUILD HIGH QUALITY SOFTWARE Test Engineer Solution Architect SW Developer Business Analyst Test Automation Engineer
- 8. | PROPERTY OF QVANTEL © 20187 PEOPLE JOURNEY TO BUILD HIGH QUALITY SOFTWARE PRODUCTS
- 9. | PROPERTY OF QVANTEL © 20188 BUSINESS AS USUAL CHALLENGES TESTING • LOT OF DIFFERENT ROLES & TEAMS • WORKING AT MULTIPLE LOCATIONS & DIFFERENT TIME ZONES • RECRUITMENT TAKES TIME, SOMETIMES CHALLENGING TO FIND PEOPLE WITH RIGHT SKILLSETS PEOPLE • BSS STACK CONSISTS MULTIPLE COMPONENTS • COMPLEXITY, LEARNING TAKES TIME • TEST DATA CAUSES LOT OF WORK • LOT OF DIFFERENT ENVIRONMENTS SW DEVELOPMENT • MULTIPLE CUSTOMER DELIVERIES ONGOING • BALANCE; CORE DEVELOPMENT VS. CUSTOMIZATION • REQUIREMENTS ARE SOMETIMES UNCLEAR • DIFFERENT PROGRAMMING LANGUAGES IN USE
- 10. | PROPERTY OF QVANTEL © 20189 WHAT SKILLS ARE EXPECTED FROM TESTERS? SELECTION • Carefully analyze need with product manager/ lead developer • Schedule (It takes time to find right person) • Document job requirements & create job ad NEED • Fulfilment of job requirements • Suitability to the current team setup • Test results (Martigma, MINT) support selection ONBOARDING • Ensure good start for newbies • Learn processes & product(s) • Define responsibilities • Give feedback
- 11. | PROPERTY OF QVANTEL © 201810 JOURNEY TO FIND PROFESSIONALS RECRUITMENT SELECTION • Carefully analyze need with product manager/ lead developer • Schedule (It takes time to find right person) • Document job requirements & create job ad NEED • Fulfilment of job requirements • Suitability to the current team setup • Test results (Martigma, MINT) support selection ONBOARDING • Ensure good start for newbies • Learn processes & product(s) • Define responsibilities • Give feedback
- 12. 11 COMPONENT TESTING THE RIGHT MIX OF DIFFERENT SKILLS INCREASE PRODUCTIVITY Lead Test Engineer (Test Manager), Test Engineers (for exploratory testing), Test Engineers (for automation), Release Engineer I like exploratory testing Test automation I’m coordinatingAPI, REST…
- 13. | PROPERTY OF QVANTEL © 201812 WHAT IS COMPONENT TESTING? COMPONENT TESTING SELECTION NEED ONBOARDING CSRTB CSRTB CSRTB CSRTB CSRTB CSRTB CSRTB CSRTB CSRTB CSRTB CSRTBCASE MANAGEMEN T UNIT TESTS DEVELOPERS SW DEVELOPMENT COMPONENT QA: ALIGNEDWAYOF INTEGRATIO N TESTS DEVELOPERS & COMPONENT TESTERS SYSTEM TESTS COMPONENT TESTERS GUI TESTING, TEST CASE BASED TESTING, EXPLORATORY TESTING REGRESSION TESTING API TESTING
- 14. | PROPERTY OF QVANTEL © 201813 FROM COMPONENT LEVEL TO CUSTOMER DELIVERIES PLAYGROUND SELECTION NEED ONBOARDING CSRTB CSRTB CSRTB CSRTB CSRTB CSRTB CSRTB CSRTB CSRTB CSRTB CSRTB CASE MANAGEME NT Cust 1CORE RELEASE Bi-weekly SIT Daily DevInt Cust 1Cust 1CUST 1 COMMUNICATION & FEEDBACK LOOP COMPONENT QA: E2E TESTS PERFORMANCE SECURITY TESTING MIGRATION TESTING UAT SUPPORT ALIGNEDWAYOF SW DEVELOPMENT
- 15. | PROPERTY OF QVANTEL © 201814 GOALS COMPONENT TESTING GOOD COMPONENT QUALITY (NO REGRESSION, NEW FUNCTIONALITY TESTED) SUPPORT DEVELOPMENT, DoD -> RELEASE ACCURACY COMMUNICATION & FEEDBACK LOOP IS WORKING RE-USABLE TEST ASSETS VISIBILITY OF PRODUCT QUALITY (AND TESTS) VISIBILITY OF TESTING 1 2 3 4 5 6
- 16. | PROPERTY OF QVANTEL © 201815 GOALS COMPONENT TESTING REGRESSION TEST AUTOMATION + EXPLORATORY TESTING DAILY STAND UPs, RELEASE PLANNING & FOLLOW UP MEETINGS, JIRA USAGE TRUST, SYNC UP MEETINGS, CHATS… COMPONENT LEVEL TEST AUTOMATION CAN BE RE-USED IN NEXT TEST PHASES & ENVIRONMENTS DOCUMENTED TEST CASES, RELASE NOTES, COMPONENT HEALTH CARDS, COMMUNICATION VISIBILITY OF TESTING 1 2 3 4 5 6
- 17. | PROPERTY OF QVANTEL © 201817 USE CASE EXAMPLE TEST DATA CHALLENGE “CUSTOMER 1 TRANSFERS BALANCE FROM HIMSELF TO HIS FRIEND” • CUSTOMER 1 MUST HAVE A PRODUCT WHICH ALLOWS TO TRANSFER BALANCE • CUSTOMER 2 MUST HAVE A PRODUCT THAT ALLOWS TO HAVE A BALANCE • CUSTOMER 1 MUST HAVE A POSITIVE BALANCE (POSITIVE TEST CASE) • CUSTOMER 1 & 2 MUST HAVE SUITABLE BILLING ACCOUNTS WITH PROPER STATE
- 18. | PROPERTY OF QVANTEL © 201818 CHALLENGES TO RE-USE TEST AUTOMATION TEST DATA CHALLENGE CHALLENGES TO RE-USE TEST AUTOMATION SCRIPTS IN OTHER THAN INTENDED ENVIROMENT • STATIC DATA IS USED IN TESTING; CHANGE IN TEST DATA CAUSES THAT AUTOMATED TESTS DO NOT WORK (=WASTE) • IT REQUIRES TOO MUCH WORK (=WASTE) TO MAKE STATIC TEST DATA CORRECT FOR A SPECIFIC TEST CASE • DIFFERENT STATIC DATA SETS ARE USED IN DIFFERENT TEST PHASES IN DIFFERENT ENVIROMENTS BY DIFFERENT PEOPLE (=WASTE) I’m spending too much time in different test data problems
- 19. | PROPERTY OF QVANTEL © 201819 WHAT IS DATA BROKER? TEST DATA CHALLENGE • IT CREATES DATA WHILE RUNNING TEST AUTOMATION • THIS WOULD ENABLE AUTOMATED TEST SUITE RE-USE IN DIFFERENT ENVIRONMENTS (TEAM SAND BOX, DEVINT,SIT, BSS FULL STACK REFERENCE ETC. ) • INSTEAD OF USING STATIC DATA YOU CALL DATA BROKER TO GENERATE NEEDED DATA AND USE THAT DATA IN AUTOMATION TESTING • THEN YOUR AUTOMATED SUITE IS **INDEPENDENT** OF EXISTING DATA IN TARGET ENVIRONMENT Instead of investigating test data issues I can now use more time in testing
- 20. | PROPERTY OF QVANTEL © 201820 DATA BROKER - PROOF OF CONCEPT SOLUTION TEST DATA CHALLENGE CLI ROBOT FRAMEWO RK DATA BROKER INTERFAC E DATA BROKER ENGINE BSS API Data Broker Engine uses Python faker library, which creates content CASE MANAGEMEN TTEST EXECUTION DATA CREATION DATA BROKER READS THE DATA SCHEMA
- 21. | PROPERTY OF QVANTEL © 201821 DATA BROKER - PROOF OF CONCEPT SOLUTION TEST DATA CHALLENGE BSS API Check the new code with dynamic data! CASE MANAGEMENT
- 22. | PROPERTY OF QVANTEL © 201822 QTER IS FOR TEST RESULT VISUALISATION TOOLS SUPPORT DEVELOPMENT, DoD -> RELEASE ACCURACY COMMUNICATION & FEEDBACK LOOP IS WORKING RE-USABLE TEST AUTOMATION VISIBILITY OF PRODUCT QUALITY (AND TESTS) VISIBILITY OF TESTING 2 3 4 5 6
- 23. | PROPERTY OF QVANTEL © 201823 QTER IS FOR TEST RESULT VISUALISATION TOOLS SUPPORT DEVELOPMENT, DoD -> RELEASE ACCURACY COMMUNICATION & FEEDBACK LOOP IS WORKING RE-USABLE TEST AUTOMATION VISIBILITY OF PRODUCT QUALITY (AND TESTS) VISIBILITY OF TESTING 2 3 4 5 6
- 24. | PROPERTY OF QVANTEL © 201824 SWIFT IS FOR SUPPORTING RELEASE ACTIVITIES TOOLS SUPPORT DEVELOPMENT, DoD -> RELEASE ACCURACY COMMUNICATION & FEEDBACK LOOP IS WORKING RE-USABLE TEST AUTOMATION VISIBILITY OF PRODUCT QUALITY (AND TESTS) VISIBILITY OF TESTING 2 3 4 5 6
- 25. | PROPERTY OF QVANTEL © 201826 SUMMARY TOOLS Android Studio (Android mobile app testing) Apache Maven Bitbucket Browserstack (browser compatibility testing) Confluence Coverage.py (coverage measurement for python) IntelliJ IDEA JaCoCo (coverage measurement for java) Jenkins JIRA (bug tracking, product development) Jmeter (performance testing) jUnit (unit test framework for Java) Postman Protractor (test automation, Angular) Pytest (unit test framework for Python) Robot Framework (test automation) Rspec (unit test framework for Ruby) Scoverage (coverage measurement for Scala) Simplecov (coverage measurement for ruby) SoapUI (API test automation) Specs2 (unit test framework for Scala) Stash Swift (internal tool for release activities) TestFlo (test planning, test cases) Qter (internal tool for test results & coverage) Xcode (iOS mobile app testing)
- 26. | PROPERTY OF QVANTEL © 2018 DOCUMENTATIO N WAY OF WORK COMMUNICATI ON SUPPORTING PROCESSES TEST METHODS & TECHNIQUES SKILLS TEST AUTOMATION COLLABORATIO N CI 27 …IN SOFTWARE PRODUCT DEVELOPMENT SUCCESS FACTORS
- 27. PROPERTY OF QVANTEL © 2018 THANK YOU!
Editor's Notes
- #3: BEFORE QVANTEL I WORKED AT CYBERCOM FINLAND AS SENIOR TEST ENGINEER, TEST MANAGER, QUALITY MANAGER
- #4: THIS IS BASED ON MY OWN EXPERIENCES AND OBSERVATIONS MY MASTER OF THESIS WAS "CHALLENGES TO KNOWLEDGE TRANFER IN SOFTWARE TESTING" MY FAVOURITE TOPICS: BUSINESS INFORMATION MANAGEMENT, KNOWLEDGE MANAGEMENT, INFORMATION SHARING, COMMUNICATION ETC.
- #6: MULTIPLE COMPONENTS, FRONT CHANNEL, DATA MANAGEMENT, DIGITAL BSS CORE COMPLEXITY! SOME DELIVERIES NEED ONLY FEW COMPONENTS
- #7: BUSINESS TO CONSUMER APPS
- #8: We have now 850 + people and in QA 180 people, including release engineers. Co-operation is the key for success! Also our sofware is used by tens of different roles, personas, B2B, B2C and qvantelees HAVE YOU EVER THINK ABOUT HOW MANY DIFFERENT USER TYPES WE HAVE? HAVE YOU EVER THINK ABOUT HOW MANY DIFFERENT USERS HAVE A IMPACT TO SOFTWARE UNDER DEVELOPMENT?
- #9: WE NEED TO INVOLVE TESTERS EARLY!
- #10: GROWTH OF THE COMPANY! COMPETITIVE IMPROMENT SOLUTIONS, RE-INVENTING THE WHEEL
- #11: HUGE AMOUNT OF SKILLS!
- #12: WE CAN’T FIND WHAT WE’RE LOOKING FOR BUT THAT’S FINE!
- #13: IT’S IMPORTANT TO BUILD GOOD COMBINATION OF DIFFERENT PEOPLE
- #14: TEST LEVELS ARE STILL VALID! THE EARLY BIRD CATCHES THE WORM UNIT TESTING VS. COMPONENT TESTING UNIT TESTS ARE IMPORTANT IT'S ABOUT QUALITY ATTITUDE OF DEVELOPERS!
- #15: WE NEED TO FULFIL THE NEEDS OF DIFFERENT CUSTOMER DELIVERIES
- #16: HOW WE CAN DO THAT?
- #17: LET'S CONTINUE WITH THE TOPIC RE-USABLE TEST ASSETS, AUTOMATION
- #18: THESE ARE GOOD PRINCIPLES BUT NOT ENOUGH!
- #19: CUSTOMER 1 & 2 EXISTS AND ARE KNOWN
- #21: Data Broker is a supporting service in Qvantel's BSS stack. It provides data content generation, creation and manipulation functionalities by using standard BSS APIs and services. The primary use case is to generate and populate live BSS systems with data for quality assurance and testing purposes.
- #22: Data Broker is a supporting service in Qvantel's BSS stack. It provides data content generation, creation and manipulation functionalities by using standard BSS APIs and services. The primary use case is to generate and populate live BSS systems with data for quality assurance and testing purposes.
- #29: TRUST ENABLES COMMUNICATION AND COMMUNICATION ENABLES COLLABORATION -Is your component checked by some static code analysis tool (staticCodeCheck build step)? -Are the CI tests set up in the recommended way (unit/integration test and test coverage reporting)? -Are the component tests enabled for your component and are the executions automated in the Jenkins pipeline?