Keeping IoT stack in quality check - meetup IoT Under Test
0 likes69 views
The document discusses the challenges of maintaining quality within a large IoT stack that includes various components such as embedded firmware, mobile apps, and monitoring backends. It presents a round earth model for testing strategy and highlights the importance of automated low-level checks and effective testing practices across different levels. The conclusion emphasizes the complexity of IoT technology stacks and the necessity for interoperability and effective quality assurance practices.
Keeping IoT stack in quality check - meetup IoT Under Test
- 1. Keeping IoT stack in quality check
- 2. Keeping IoT stack in quality check Łukasz Dudek Software Engineer & Delta Team Leader lukasz.dudek@silvair.com
- 3. Agenda 1. Massive stack challenge 2. Round Earth model 3. Testing components 4. Soft quality checkpoints 5. Beyond our product 6. Summary
- 4. Massive stack ● Embedded firmware ● Licensing platform ● MaTE (flashing device) ● Commissioning platform ● Mobile app ● Monitoring backend ● Gateway
- 5. Massive stack – also in tests! ● Embedded firmware ● Licensing platform ● MaTE (flashing device) ● Commissioning platform ● Mobile app ● Monitoring backend ● Gateway
- 6. Massive stack = challenge ● Many: – Components – Technologies – Connection points
- 7. Round Earth model ● Test strategy ● Concentric spheres – Development concerns “inside” – UX concerns “outside” – Risk grows outwards ● Test pyramid alternative – Not only “more UTs than E2Es” Image source: https://phys.org/news/2015-12-earth-layers.html Idea source: https://www.satisfice.com/blog/archives/4947
- 8. Round Earth model - layers ● Surface = what users see ● Sub-systems / services ● Units / Components ● Environment / framework ● Also: – User input – Data flows – Configuration Image source: https://phys.org/news/2015-12-earth-layers.html Idea source: https://www.satisfice.com/blog/archives/4947
- 9. Round Earth model - outtakes ● Bottom-up quality ● Automate low-level checks ● Encourage penetrability ● Push tests down ● Less top-level tests Image source: https://phys.org/news/2015-12-earth-layers.html Idea source: https://www.satisfice.com/blog/archives/4947
- 10. Round Earth model - outtakes for IoT ● Huge UI layer – GUIs – Light / diodes – Buttons / switches ● More gluing code needed ● Many different services Image source: https://phys.org/news/2015-12-earth-layers.html Idea source: https://www.satisfice.com/blog/archives/4947
- 11. Testing components ● Common ideas ● Different tools ● Best practices for each ● Static Code Analysis – Easy – Useful Image source: https://www.mindtheproduct.com/2016/02/what-the-hell-are-ci-cd-and-devops-a-cheatsheet-for-the-rest-of-us/
- 12. Testing components - levels ● Unit ● Functional ● Smoke ● Regression ● End-to-end (integration) ● Exploratory – Only manual level
- 13. Testing components – challenges ● Tests with hardware – Long – Flaky – Many hardware vendors ● Mobile app’s tests – Real devices Image source: https://www.mindtheproduct.com/2016/02/what-the-hell-are-ci-cd-and-devops-a-cheatsheet-for-the-rest-of-us/
- 14. End-to-end testing challenges ● Include every component – Hardware for each ● Tools to access all APIs ● Fully automated
- 15. Soft quality checkpoints ● Test Community – Weekly meetings – Day-to-day QA issues ● Impact meetings – Thrice per release – Non-technical people with POs – Business impact of features
- 16. Beyond our product ● Our product = our country ● International law = Bluetooth Mesh standard – PTS test suite ● Interoperability is a must ● Manual tests of 3rd party – Apps – Sea of devices
- 17. Summary ● IoT technology stacks are big ● Common QA concepts ● Common best practices – CI/CD as paradigm ● Best tools for each job ● Harder integration – Inside & outside
- 18. Questions?
- 19. Thank you!