Testing Challenges and Approaches in Edge Computing
0 likes202 views
The Fraunhofer FOKUS Institute outlines the challenges and testing approaches in edge computing, emphasizing its distributed nature and the need for interoperability and security. It discusses various application domains and highlights the importance of standards, test methods, and collaborative initiatives to ensure effective testing and certification in edge computing. The conclusion stresses the necessity for harmonized strategies and increased focus on quality and testing within the edge computing landscape.
Testing Challenges and Approaches in Edge Computing
- 1. Fraunhofer FOKUS Institute for Open Communication Systems Testing Challenges and Approaches in Edge Computing Axel Rennoch, Dr. Alexander Willner, Sascha Hackel | ECW Berlin | 09.03.21
- 2. Agenda 1 Background 2 Edge Computing 3 EC Testing Challenges 4 Testing Approaches for EC 5 Conclusion 2
- 3. 1 Background
- 4. How is our view on Edge Computing characterized? 4
- 5. Fraunhofer is Europe’s largest application oriented research organization: Fraunhofer Society © Michael Zalewski/ Fraunhofer FOKUS © Fraunhofer-Gesellschaft > 29.000 Employees 75 Institutes and research units > 2.8 billion € Budget (1/3 government, 1/3 public, 1/3 industry) 5 Fraunhofer Institute for Open Communication Systems (FOKUS) We connect everything secure, reliable, sustainable Fraunhofer Institute FOKUS The largest Fraunhofer ICT institute (~450 employees). Located in Berlin.
- 6. Fields of application and strategic topics of Fraunhofer FOKUS 6 STRATEGIC TOPICS FIELDS OF APPLICATION Sustainability Artificial Intelligence Digital Life Security/ Certification Digital Governance Digital Networking (e.g. 5G/6G) Quantum Computing
- 8. What is Edge Computing? 8
- 9. Based on: Peter Levine: Return to the Edge and the End of Cloud Computing Edge Computing: A Distributed Computing Paradigm
- 10. Who needs Edge Computing? 10
- 11. Source: Based on Texas Instruments and Moor Insights & Strategy's report Segmenting the Internet of Things (IoT) Various application domains with specific requirements 11 INDUSTRIAL Internet of Things CONSUMER Internet of Things Business Value Personal Interest Smart Manufacturing Smart Energy Smart Agriculture Smart City Smart Transportation Smart Phones Smart Wearables Smart TV Smart Appliances Smart Home Connectivity Data Exchange e-Health Smart Plugs
- 12. Where is the Edge? 12
- 13. Where is the Edge? 13 13
- 15. The Reference Architecture Model Edge Computing (RAMEC) 15 Willner, A., & Gowtham, V. (2020, August 10). Towards a Reference Architecture Model for Industrial Edge Computing. IEEE Communications Standards Magazine
- 16. The Reference Architecture Model Edge Computing (RAMEC) 18 Willner, A., & Gowtham, V. (2020, August 10). Towards a Reference Architecture Model for Industrial Edge Computing. IEEE Communications Standards Magazine Administration Shell - Information Model - APIs - Connectivity - … Assets Manifest Component Manager Asset Asset …
- 17. An initial approach to cluster the activities 19 Initiatives ➢ International (IIC, ECC, EECC, …) ➢ EU (GAIA-X, EPI, …) ➢ Germany (PI4.0/ LNI4.0/ SCI4.0/Bitkom/ZVEI/…) ➢ France (AIDF) ➢ Italy (I4.0) ➢ Austria (I4.0Ö) ➢ Spain (IC4.0) ➢ Poland (Przemysł 4.0) ➢ Netherlands (Smart Industry) ➢ UK (4IR) ➢ Ukraine (APPAU) ➢ … Software Focus ➢ LF Edge: Akraino, Baetyl, EdgeX Foundry, EVE, Fledge, HomeEdge, Open Glossary ➢ LF ELISA ➢ Linaro ➢ ONF CORD ➢ OEC ➢ OSADL ➢ OpenStack EC ➢ KubeEdge ➢ Discovery ➢ … Standards ➢ ETSI (MEC) ➢ ETSI (MANO) ➢ IEEE (TSN) ➢ IEEE (1934) ➢ ISO/IEC (SC38) ➢ ISO/IEC (62541) ➢ ISO/IEC (61499) ➢ DIN (92222) ➢ IETF ➢ OMG ➢ ITU ➢ NIST ➢ MSP ➢ … Verticals ➢ 5G-ACIA ➢ SmartFactoryKL ➢ OPC Foundation ➢ ODVA NewTec ➢ FFPA ➢ OI4.0 ➢ AECC ➢ 5GAA ➢ MobileEdgeX ➢ TIP EC ➢ OPAF ➢ …
- 18. ➢ Industrial Internet Consortium ❖ Parallel Edge and Testbed working groups ➢ Alliance for Internet of Things Innovation (AIOTI) ❖ Multiple working groups and IG Testbeds ➢ ETSI ISG MEC ❖ Conformance test developments and hackathons ❖ Online edge emulation environment (Sandbox) ➢ oneM2M ❖ Conformance test development and plugfests Testing initiatives and standards 20
- 19. Agenda 1 Background 2 Edge Computing 3 EC Testing Challenges 4 Testing Approaches for EC 5 Conclusion 21
- 20. 3 EC Testing Challenges
- 21. ➢ Distributed, heterogeneous architectures and interfaces ❖ Interoperability and Security requirements ❖ Multiple vendor devices/platforms ❖ Domains specific applications, objects and functions (under test), use cases ❖ Synergies and reusability of test harness from different verticals? ➢ Management at the edge (nodes) ❖ Mobility of devices ❖ Elasticity and fluctuation of data workload (container/kubernetes pods etc.) ❖ Provisioning, orchestration and coordination (deterministic capabilities, SDN) ❖ Edge Artificial Intelligence ➢ Performance/robustness and resource restrictions at IoT layer ❖ Big amount of devices (capacity and scalability) ❖ Ultra-low latency (≈ ms) is critical for real-time type of application ❖ Limited connectivity and power consumption (limited autonomy) Specific EC aspects for testing 23
- 22. ➢ Distributed, heterogeneous architectures and interfaces: Scope / Testing types for EC ❖ Conformance and Security (before commissioning safety-critical systems) ❖ The human element must be considered in all aspects of safety ❖ Interoperability ❖ Performance, real-time aspects ➢ Management at the edge (nodes): Systematic Test definition/specification ❖ Test suite structuring: Test configurations; Implementation under test (IUT) roles (server/client) ❖ Test objectives (test purpose catalogs), incl. new AI requirements (e.g. robustness, comprehensibility) ❖ Use case scenarios (test case implementation) ➢ Performance/robustness and resource restrictions at IoT layer: (Automated) Testbeds and software architecture (testware/tools/simulations) ❖ Demonstrate readiness of domain-specific solutions ❖ Certification Challenges for Edge Computing Testing 24
- 23. 4 Testing Approaches for EC
- 24. ➢ Requirements ❖ Open for different software platform and toolsets ❖ Vendor independence ❖ Abstract description of scenarios ❖ Open source ➢ Testing specification and implementations ❖ Robot ❖ TDL ❖ TTCN-3 Testing Methods and Techniques 26 Edge Testing Software Testing System Testing Security Testing Test Automation Protocol Testing
- 25. ➢ Recall Mobile communication and IoT domains ❖ Multiple layer and protocols ❖ Import of domain specific notations ❖ Interoperability and Security issues ❖ Inclusion of operational phase ➢ Recall Existing standardized Testing Frameworks ❖ 3GPP RAN5: Mobile terminal conformance testing ❖ ETSI TC MTS: Methodology for RESTful APIs specifications and testing ❖ ETSI ISG MEC: Multi-access Edge Computing (MEC) testing ❖ ETSI ISG NFV: Network Functions Virtualisation (NFV) testing methodology ❖ oneM2M TDE: Testing and Developers Ecosystem (TDE) Learning from existing Test approaches 27 IIC Connectivity Framework
- 26. ➢ MEC Testing Framework (ETSI GR MEC-DEC 025 V2.1.1 (2019-06)) ❖ Abstract Test Method ❖ Conformance Test Purposes ❖ Interoperability Test Descriptions ➢ API Conformance Test Specification (ETSI GS MEC-DEC 032 V2.1.1 (2020-12)) ❖ Part 1: Test Requirements and Implementation Conformance Statement (ICS) ❖ Part 2: Test Purposes (TP) ❖ Part 3: Abstract Test Suite (ATS) Multi-access Edge Computing (MEC) 28 MEC Testing and Conformance
- 27. ➢ Eclipse IoT-Testware ❖ MQTT and CoAP ❖ Executable conformance test suite (open source) ➢ Testbeds/Simulations ❖ 5G Playground (FOKUS) ❖ Federated testbeds (FED4FIRE+) ❖ IoTSim-Edge: A simulation framework ➢ Monitoring, prediction etc. ❖ Continuous testing during operational phase ❖ Information Security Indicator Monitoring (sources, databases) Further Examples 29
- 28. ➢ Systematic collection of requirements and related test purpose definition ❖ Unique specification and Common interpretation ➢ Application of Standardized (Abstract) Test approaches ❖ Approved Methods and Basis for certification (extended CommonCriteria) ➢ High performance challenges ❖ Apply simulation and/or continuous offline analysis (from operational phase) ➢ Consideration of new AI requirements and testing methods ❖ E.g. adversarial/corruption robustness evaluation ❖ Risk analysis and Collection of common test data Main recommendation for EC testing 30
- 29. 5 Conclusion
- 30. ➢ Multiple different aspects of EC are under discussion ❖ Various working groups of standardization bodies and industrial associations ❖ Technical viewpoints differ due to the various stakeholders ➢ Landscape documents already support the interested experts and public community ❖ Missing QA, Testing and Certification ➢ A need for harmonization and common strategies ❖ More emphasise on quality and testing ➢ Edge Computing Consortium (EECC): in Europe, in preparation ❖ ECW Europe Workshop: 11th March 2021 Summary 32
- 31. Fraunhofer FOKUS Institute for Open Communication Systems Kaiserin-Augusta-Allee 31 10589 Berlin, Germany https://www.fokus.fraunhofer.de/en/sqc https://www.fokus.fraunhofer.de/ngni SQC: Axel Rennoch & Sascha Hackel Phone +49 (30) 3463 – 7344 / 7255 axel.rennoch@fokus.fraunhofer.de sascha.hackel@fokus.fraunhofer.de NGNI: Dr. Alexander Willner Phone +49 (30) 3463 - 7116 alexander.willner@fokus.fraunhofer.de Thank you for your attention!