Upstream Testing Collaboration
Download as PPTX, PDF0 likes431 views
The document discusses challenges and strategies in integrating various upstream components like OpenStack and ODL within the OPNFV framework. It emphasizes the importance of collaboration between the OPNFV community and upstream projects to identify testing needs, report bugs, and contribute code. Additionally, it highlights the use of testing frameworks like Robot and Tempest to ensure successful integration and functionality of virtual network function managers.
Upstream Testing Collaboration
- 2. Upstream testing collaboration Jose Lausuch Ericsson, Functest PTL
- 3. OPNFV = Integration & testing Enhancements Additions
- 4. OPNFV = Integration & testing OPNFV integrates several upstream components • OpenStack, OVS, ODL, ONOS, … Single components might work as expected when isolated The challenge is integration e.g. OpenStack + OVS + ODL + Tacker + SFC
- 5. Things might go wrong
- 6. e.g. OPNFV SFC The SFC feature in OPNFV SFC needs different components: • Fuel, Apex • OpenStack • OVS • ODL + SFC feature • Tacker Some bugs are found only when putting them all together
- 7. The community decided to use Robot as the test framework There are a lot of tests dealing with each module in the architecture • netvirt, genius, Netconf, ovsdb, opensack, etc … OPNFV is mainly interested about ODL integration with OpenStack
- 9. • Basic & Neutron.Basic.Restconf Modules • Get the complete list of networks • Get the complete list of subnets • Get the complete list of ports • Check OpenStack Networks • Check OpenDaylight Networks • Create new network in OpenStack Check network created in OpenDaylight • Check network created in OpenDaylight • Delete Network • Check network deleted in OpenDaylight• Check OpenStack Subnets • Check OpenDaylight subnets • Create new subnet in OpenStack • Check new subnet created in OpenDaylight • Delete New subnet • Check subnet deleted in OpenDaylight • Check OpenStack ports • Check OpenDaylight ports • Create new port in OpenStack • Check new port created in OpenDaylight • Delete New Port • Check port deleted in OpenDaylight
- 10. New Patchset (Commit) Create Virtual Testing Platform Download/Install new artifact Use standard testing framework e.g. Functest Build ODL artifact 10 min 15 min 35 min Unit test Installer Project e.g. Apex Snapshot 5 min Positive -> Merge commit Negative -> Prevent merge Investigate the logs/Find bugs ODL OPNFV Feedback
- 11. What next? OPNFV could collaborate with ODL community to define test needs OPNFV folks contribute with code to ODL Use OPNFV as a tool to give quick feedback to ODL XCI
- 12. Tempest Set of integration tests exercising the OpenStack APIs In OPNFV: • Tempest smoke • Tempest full OPNFV scenarios are complex and some tests might must blacklisted
- 13. Rally Benchmarking tool of the OpenStack services In OPNFV: • Rally sanity • Rally full OPNFV has collaborated upstream with some bug fixes
- 14. Refstack Set of Tempest test cases defined by the Interop Working Group Toolchain used for OpenStack Powered platform trademark Web platform to upload test results In OPNFV: • Set of tempest test cases belonging to guidelines (2016.08) are valid for Kilo, Liberty, Mitaka and Newton releases of OpenStack
- 15. Refstack OpenStack is interested about NFV certification track We have the NFV knowledge They have the experience of running certification programs Why don’t we upstream our NFV use cases to e.g. Tempest?
- 16. Cross CI Integrate in OPNFV the latest OpenStack master Automated integration and testing Quick feedback to upstream Projects
- 17. VNF managers OPNFV’s mission is not to to test the functionality of a VNF manager OPNFV cares about • successful integration of the VNF manager • it can deploy a VNF • VNF lifecycle, etc… Examples • Tacker for SFC test case • Cloudify for vIMS in Functest
- 18. ONAP ONAP is going to have a lot of presence in OPNFV OPNFV probably doesn’t want to run the exact same tests but it should check how their tools integrate in OPNFV Needde collaboration in both directions
- 19. In short Integration of upstream components makes thing complex OPNFV needs to collaborate with upstream to provide: • Feedback • Bug fixes • Requirements, blueprints
- 20. In short Don’t wait to get something done Engage in upstream communities
- 21. How to engage in upstream? IRC Mailing lists Attend project meetings
- 22. Thank You