Summit 16: Multi-site OPNFV Testing Challenges
0 likes404 views
The document discusses the challenges and plans for multi-site testing in the context of OPNFV, focusing on infrastructure management, automation, and interoperability. It highlights the need for advanced test cases and the evolution of testing practices to accommodate complex, distributed environments. A call to action is made for the community to collaborate in addressing these challenges and enhancing testing processes.
Summit 16: Multi-site OPNFV Testing Challenges
- 1. June 20–23, 2016 | Berlin, Germany
- 2. Multi-site OPNFV Testing Challenges Trevor Cooper (Intel) Fatih Degirmenci (Ericsson) Morgan Richomme (Orange)
- 3. Agenda Multi-site Testing Today - Trevor - Pharos Infrastructure - CI Today Multi-site Testing Needs - Morgan - Advanced Test-cases - Interoperability Multi-site Testing Plans - Fatih - Challenges and Requirements - Future Plans and Vision for Infra and Testing Summary and Call to Action - Fatih
- 5. Multi-site Testing Introduction • Motivations for Multi-site Testing • Geographically and technically diverse test environments • Test resources independent of installers and scenarios promotes robust platform • Inter-deployment testing for interoperability between data-centers • Future real-world use-cases and deployment scenarios • Pharos Infrastructure • Infra Working Group: Pharos, Octopus, Releng, Genesis • Infrastructure labs, process, tools and features • CI / CD / CT • Environments increasing in number and complexity • Process to improve platform robustness • Challenges of a Distributed Infrastructure • Managing a distributed resources • Automating testing • Distributing tests
- 6. Pharos Infrastructure • Current priorities: • Allocation and management of Infra resources for Colorado • Pharos change process to manage special requirements • Lab resource booking and usage tool/dashboard • Pharos compliance verification/validation • Dealing with today’s multi-site challenges • Challenges and how we solved them
- 7. Production Today CI Yesterday CI Today POD1 Installer X Scenario Y POD2 PODi Deploy, run tests POD1 Installer X Scenario Y POD2 PODi Deploy, run tests Lab 1 Lab 2 Lab 1 Lab 2
- 8. Production Evolution - Tomorrow POD1 Installer X Scenario Y POD2 PODi Deploy, run tests Lab 1 Lab 2 POD1 Installer X’ Scenario Y’ POD2 Lab 3 CI Tomorrow
- 9. Infra Improvements • Lab as a Service (LaaS) • Accessible virtual sandboxes for development and smoke tests • More homogeneous environments for development work • Off-load shortage of bare-metal resources • Efficient use of Lab Resources • Strive for CD (provide weekly stable versions) • Provide dynamic use of lab resources (Dev and CI) • Verify/validate POD setup for automatic configuration • All labs/pods/installer configurations for automatic POD configuration • Jenkins / CI improvements - permissions, scheduling, sandboxes, etc.
- 11. Test cases • Advanced test cases • OPNFV Multisite Project - evolving • Brahmaputra -> Use case definition and bug fixes • Colorado -> Single machine/Multiple OpenStack Instances, Functional Testing, Kingbird • Future -> Go after true Multisite • Discussion on multi tenant vIMS for D river • Interoperability testing
- 12. An advanced test case: vIMS Today for Functest test: Metaswitch vIMS + Cloudify orchestrator installed in a single tenant on one OPNFV POD. Not acceptable in “real life” (SPOF) Geographical redundancy on current production IMS system Architecture Cloud ready (Cassandra ring) Possible evolutions: 1) test in multi-tenants (local redundancy) 2) Test in multi-sites (geographical redundancy)
- 13. An advanced test case: vIMS Need of Multi-sites to evaluate • the connectivity issues (VPN/Flat) • the consequence on performance • The accuracy of placement algorithm: • Performance (DPDK/ODP area) • Location (legal constraint) … • Disaster/recovery scenarios/constraints Similar considerations on vCDN for next releases
- 14. An interoperability test • OPNFV is a great place to automate interoperability testing… • Several PODs … with several scenarios • And a centralized CI
- 15. An interoperability test: bgpvpn • SDNVPN project deals with the implementation of the OpenStack bgpvpn API • Several possible backends • ODL • Neutron Bagpipe • Nuage • … • Use OPNFV to perform interoperability testing would be valuable
- 16. An interoperability test: bgpvpn POD1 os-nofeature-bgpvpn-ha POD2 os-odl_l2-bgpvpn-ha Deploy, run tests BGPVN
- 18. Lessons learned • Distributed infrastructure is not trivial • Automating tests/ state synchro towards multi-sites • We learn by doing…first requests on multi-sites needed raised in Brahmaputra => Multi-site project
- 19. Next steps • Apply learnings • Bringing up, managing and using distributed infrastructure • Evolve continuously • No dramatic changes • E2E View and holistic approach • Features will come together one day, creating more complete Multisite solutions
- 20. Next steps • Infrastructure • Provision, manage and use resources cleverly • Increase the utilization rather than burning CPU uselessly • Get ready to enter uncharted territories - security and access policies • CI • Evaluate CI to see if it can support Multisite use cases • Provide feedback that fits to purpose rather than bunch of log files • Continue applying best practices • Testing • Test what needs to be tested when it is needed to be tested
- 21. Summary • We already are challenged by what we had in the past and what we currently have • The complexity will just increase, requiring us as a community to work together like a community
- 22. Call to Action