Performance Testing Internet of Things
- 1. PerformanceTesting – Internet ofThings
- 2. • What is IoT ? • IoT Architecture • Performance Testing of IoT Devices • Performance Testing Challenges • Test Strategy • Best Practices • Case Study • Conclusion Agenda
- 3. • The interconnection of uniquely identifiable embedded computing like devices with the existing Internet infrastructure. • Network of systems, devices, and sensors which are connected. • A platform which allows to manage the data and controls the devices remotely. • To make the IoT possible, the integration of many technologies is necessary. What is IoT?
- 4. Real World Applications Smart Home Wearables Connected Cars Industrial Internet IoT in Agriculture Smart Retail IoT in Healthcare
- 5. SENSORS and ACTUATORS. CONNECTIVITY PEOPLE and PROCESSES. IoT Architecture Components
- 6. How IoT Works in Healthcare?
- 7. Shows how software operates in real-world conditions. 01 Provides insights into how the application scales and whether they can perform under pressure. 02 Improves the operational performance and cost optimization. 03 WHY - PerformanceTesting
- 8. • Greater Complexity • Determining a method of recording messages between the various connected objects • Factors such as latency, packet loss, network bandwidth, load. • Choosing right performance testing tool • RealisticWorkload Model Creation • Complexities involved in setting up a dedicated PerformanceTest Environment PerformanceTesting Challenges
- 9. • Define the objective of an IoT Device. • Gather Information about devices and software versions. • Identify SLAs for the objects. • Determine the peak load . • Analyze the IoT application logs for usage pattern. • Prepare a NFR questionnaire to identify the requirements and set the performance goals. • Simulate real-world workload models using Business requirements Historic data and future growth requirements Type of devices Network conditions Usage patterns Geographic spread. PerformanceTest Strategy
- 10. • Choose the right testing tool – supports the underlying protocol(s) used by the IoT application. Protocols like REST API over HTTP , MQTT & CoAP. Tools – Jmeter, HPE Loadrunner & Performance Center (12.55), NeoLoad 5.3 (MQTT) • Determine theType of performance test to be performed. LoadTest Scalability Test EnduranceTest • Simulate devices and the interactions between devices and sensors. • Simulate the Geographical spread and network conditions. • Capture the network communication between the IoT devices. Messages are created in load testing tool. • Generate different amount of data per second/minute/hour. • Ensure that notifications, requests, and responses are sent and received in proper form. • Notifications / requests need to be monitored for performance. • Performance metrics has to be collected across web, app, DB layer to uncover possible bottlenecks. Continued..
- 11. • Focus on below mentioned areas Device to Device Communication Device to Server Communication Server To Server Communication Bandwidth, Latency, Packet Loss Continued..
- 12. • Align test plan with the IoT platform. • Understand the types of devices and protocols and design the test plan accordingly. • Gather user requirements related to peak loads expected at given times. • Evaluate how the loss of data will affect the IoT application and how messages get passed in real time. • Prioritize test cases. • Ensure proper testing of the network communication and internal computation. • Set monitors to carefully track device system statistics (e.g. changes in temperatures, power usage, memory usage, etc.) and measure response times between different network layers. PerformanceTesting IoT – Best Practices
- 15. Company A is an industry leading manufacturer of smoke detector and automated fire extinguisher system. It is now shifting towards IoT and building a connected system which has both local intelligence (on premises decision making capability about fire detection and extinguishing) as well as the central command and control system which would monitor all of the company's fire safety installation across the globe.The engineers who have built the platform claims to support up to 5 Million endpoints. A simple test case for such a platform would be following:- Simulate a connected device reporting temperature more than 100 degrees, ensure that system detects a fire alarm locally and reports it to the cloud backend. Ensure the fire extinguishers are activated within the 90 seconds, and authorities notified automatically. Case Study
- 16. Ensure that maximum guaranteed latency between reporting a certain fire incident and activation of sprinkler system should not exceed 90 seconds under any circumstances. Simulate multiple fires in the close geographic proximity so that the cloud platform could perform an escalated incidence reporting. At any given point of time, up to max 10% of the total fire alarm system can report the fire. (up to 500K)You will need to test what happens if system exceeds the capacity. Ensure that system should be able to handle messaging of up to 10,000 devices per second at peak load.There will be multiple generations of devices; the older one will speak CoAP, the new generation will talk MQTT and HTTP. The test will test all such combinations. Case Study
- 17. • Performance testing the Internet ofThings is slowly becoming commonplace. • A strategic approach - IoT testing plan & efficient test strategy. • Realistic work load model, choosing a right testing tool & testing the network and internal computation. • Integrating performance testing IoT in business processes will surely allow an enterprise to flourish. • A good performance testing is a matter of good performance design. Conclusion