Continuous Interop Testing

Editor's Notes

• #7: Main Point: Dependency constraints are a critical problem for Netcentric applications. It’s a root cause of delays, increasing infrastructure costs, and quality problems.---------------------Development and testing should be an integrated activity, but we don’t have the tools to really make that work in parallel because of constraints and dependencies in the process. Constraints can take many forms:Still under developmentWrong data-can’t testOnly 2 hours of access
• #16: Projects are initially stood up in Forge.mil using the SoftwareForge repository, in a future release cloud resources for all Forge.mil projects will be provisioned by the TestForge-provided cloud management capability(IaaS provisioning). This assures continuity and ease from development all the way through push to production. The PKI and CAC credentials (roles/permissions) inherent in the developers, testers, administrators, and project managers for “this project” are established at “SoftwareForge time”. The cloud resources used for Dev and TEST and PROD will be different, however the vision is for a single TestForge cloud management capability to handle all provisioning.Initial code and unit test is performed in SoftwareForge, however developers will still be able to access SoftwareForge codebases after projects have moved to TestForge. The build management capabilities, specifically notifications within SoftwareForge will be integrated with TestForge and will be extended to trigger TestForge’s “continuous interoperability” cycle (assumes clean compile and code analysis scores (if required).When the project is ready to move into TestForge, the TestForge “account management” operational area assures that the developers, testers, administrators, and project managers are correctly permissioned (based on what was established at SoftwareForge time. These people will have the CRUD rights over those objects needed to complete a test and IA cycle: assets, scenarios, physical services (including IA controls and available application services), virtual services, etc.) PKI and CAC credentials in the initial Forge.mil edition of TestForge will be acquired from Forge.mil. The “Workgroup Edition” of TestForgebeing developed for the 350th ELSW at Hanscom is not addressed in this document.The Forge.mil Build Management capability is transparently provided by SoftwareForge.mil to TestForge. Notifications of successful and failed builds will be messaged in the TestForge Console and reporting stores. Asset Management is the operational area within TestForge where “system modules” (mostly IaaS in nature) and their related “assets” (middleware, databases, etc.) are created and/or selected by a tester. TestForge testing concepts and terminology differ slightly from the SoftwareForge approach to testing. The goal is complete transparency. However, SoftwareForge testing assets (including access-related) will be migrated to TestForge repositories at the time the project is moved to TestForge. In addition, all codebase originally developed in SoftwareForge will remain available in SoftwareForge for changes and build management at any time during the TestForge cycle. Physical entry points to applications (where available) and virtual services are created. IA compliance requirements are identified and appropriate controls are created(reused) as physical or virtual services. TestForge seeks to minimize the need for physical SOA infrastructure. The key strategy for doing this is to use our virtual service testing capability (currently iTKO LISA and VSE) to abstract SOA components. Developers, testers, administrators work together to vision test cases which are built at this time. We now begin scenario construction. The “scenario” is what is actually tested once the codebase, physical entry points, virtual services, and IA controls are identified and included in the scenario. The scenario may include multiple “assets” (each with their own operating system, middleware, codebase, apps) plus test cases to be executed plus virtual services to be executed. Once a scenario with all of its components is assembled and a triggering event (could be a clean compile in SoftwareForge, a state change in a database or client app, a successful change and retest of a virtual service, etc.), this project can be added into the Continuous Interoperability (CI) stack.Real time C&A reporting requirements coming out of TestForge IA controls during CI cycles need to be defined and incorporated.