Collaborative Quality Management

Editor's Notes

• #4: The story behind the Rain Sensing Wiper The development of the first Rain Sensing Wiper illustrates how a classic failure to fully conceptualize a product's physical architecture resulted in the discovery of integration issues at servicing time, thus quickly leading to engineering change requests. The scenario revolves around the initial introduction of the Rain Sensing Wiper (RSW) feature in an automobile manufacturer's vehicle program. Before examining the reason of the failure of the RSW, let us briefly review the characteristics of the system. The RSW contains mechanical (optical mounting device), electronics (IR sensors and ECU), and software (computer vision algorithm) components that are procured by tier-one suppliers. These components are simply integrated by the manufacturer. The main parameters of the system are: (1) the optical and geometric specifications of the windshield, in particular its thickness and glass optical indexes, and (2) the ranges of operation of the electronic optical sensors. The detection software also has normal ranges of operation relative to these parameters, but in addition relies on data about the actual values of the parameters of the windshield. The fact that the RSW electronics and software specifications include ranges for the relevant windshield properties is important, because it allows more flexibility on the choice of the windshield itself. This is a critical design choice, the procurement and integration costs associated with the windshield being an order of magnitude greater than that of the RSW. For optimal performance, the actual values of the windshield properties should fall near the center of the normal operating ranges both for the RSW sensors and the software; however, acceptable operation should be guaranteed for the whole range. From a procurement standpoint, the windshield is simultaneously purchased from different suppliers. Depending on the year of production and where the product is manufactured, suppliers may modify the design of their windshields. Also, one or more changes of suppliers can occur during the production phase. In the failure scenario, which occurred during the first year of the RSW's introduction, a local windshield supplier provided a component whose characteristics were incompatible with the operating range of the sensor. Unfortunately, no requirement for calibrating the system properly (i.e., for verifying that sensor and windshield are compatible) had been captured for the RSW at that point. Thus cars were sent to customers with a non-functioning wiper system. Initial diagnostics designated the software as the culprit for the malfunction, since it was difficult for mechanics to test its behavior. The other components (ECU, sensor, and windshield) were functioning normally when tested independently. The failure mode for the RSW resided at the level of its sub-systems, which made it difficult for the manufacturer to discover it. After discovering the root cause, a new requirement was captured to ensure that new systems will be properly calibrated at the production stage.
• #5: IBM INTERNAL USE ONLY And the cost of failure is enormous. Here’s one example. People rely on the investment firm Moody’s for accurate assessment of securities. A bug in a software system caused Moody’s to incorrectly assess a safe credit rating to $4 Billion of risky securities. When this one little mistake was revealed, Moody’s lost $1.7 B USD in market value in a single day. This incident illustrates how business-critical software systems are today. One bug can literally mean life or death for the organization.
• #6: IBM INTERNAL USE ONLY There have been many notable cases of system outages a scalability issues that have made front page as mission-critical applications have moved beyond the closed doors of the IT shop. The impact is not limited to immediate revenue, but can extend to long term customer retention. Today’s interconnected and distributed applications carry a much larger risk of housing a significant performance problem that can go undetected until deployment into production. Performance Testing is a critical part of the QA strategy to identify and resolve these risks before they become a problem to your users.
• #10: In order to achieve business value and innovation, software must be managed like any other business asset – as an integrated set of repeatable activities focused on producing a desired business outcome… that is to say, as a business process . A process whereby key stakeholders… >Collaborate to achieve common goals by optimizing how people, across teams, organizations, companies and contents work >Automate to increase control and efficiency through integrated workflows that reduce manual tasks and human labor >And, report and analyze to continuously improve what is being done by measuring progress in real-time
• #12: OSLC is a key differentiator for us – Many of our existing products provide and continue to enhance their jazz story by supporting the OSLC specification – by doing so they become a productive member of a customers ALM solution. OSLC allows us to support a mix of open source technologies, other market leading offerings, business partners, home grown integrations, etc. The Jazz Team Server is our #1 implementation of OSLC providing an integration hub our customers can use. Most vendors point integrate everything with brittle interfaces. Our approach creates rich, consistent interfaces in ALM to improve the entire development lifecycle and protect existing, future investments.
• #13: >Higher quality software More thoroughly tested requirements, even as requirements change Loose coupling between the requirements and test cases reduces reliance on human communication to maintain the interlock >Software that better meets requirements Greater visibility of requirements detail by developers and architects Linkage created between requirements and the work items exposes more information immediately to business analysts, developers, architects >Shorter time to delivery Data association between testing and development Association created between test cases and defects enable improved, timely identification, resolution, and verification of defects >Lab Manager deployment triggered by build notification    When the change sets for a defect / workitem are delivered in a build, RQM will automatically receive notification from the build system. Provide the ability to configure automatic execution of lab management tasks based on the notification. This function will typically be used to deploy the application under test to one or more test environment. >RQM/ RTLM Execution Flow    RQM/ RTLM will provide a more optimized user experience as part of the test execution flow from TER to finding, reserving, settting up machine, to executing tests. As part of this solution, the reservation system will be enhanced to support blocking reservations to better handle reserving machines for test execution. >Common C/ALM UI Elements Common Banner Support Common Login Support Rich UI Hover Support >Linkage of RTC WorkItems to Test Plans/Test Cases linking between Test Plans/Test Cases to RTC workitems (plan items, stories, etc.) for the purpose of establishing the relationship to the code implementation of what the test case will test. >Support for BVT A user can configure a sequence of steps to run whenever a build is successfully completed. These steps include optional deployment/provisioning of the build, a Test Suite to verify the build, and an optional “clean-up” step to teardown the installed build. The user preconfigures all the data (parameters, machine targets) for the BVT steps. Prepopulation of Defect form when creating a defect    The default work item process template has been updated to include these custom fields: related requirement name, related test case, test environments, build id, verification point failure. These fields appears in the detail section of the defect form. >RQM/RRC Integration Integration provided in both RQM and RRC >Cross Server Work Item Integration with RTC The capability must continue to support the same reporting capabilities that exist in RQM today. Reports that relate test status to defect status must integrate the information from the various servers >Automatic Build Notification from RTC Build System When the change sets for a defect / workitem are delivered in a build, RQM will automatically receive notification from the RTC build system and change the status of the appropriate Test Execution Records to indicate that the tests are clear to run.
• #14: To give the big picture... This slide captures quality management end-to-end capabilities and those of the Jazz platform and the best practices that can be customized and be flexibly integrated into a process that is based on client needs… Test management, test lab management and a variety of domain-specific test automation capabilities supported by Jazz’s open platform helps extend our quality management offerings...
• #15: IBM INTERNAL USE ONLY > Rational addresses these challenges with Rational Quality Manager. Rational Quality Manager is a centralized test management system for business-driven quality software that addresses business risk, operational efficiency, and the ability to make confident release decisions. > RQM helps to mitigate business risk through: Distributed project team collaboration on a dynamic, customizable test plan over a Web 2.0 interface Process enforcement through user-defined roles/customized process workflows and work item assignments >Operational efficiency is accomplished through automation. For example: Reducing testing time through capabilities to help users track, schedule and execute tests on physical and virtual lab assets Automated assistance to optimize test environment coverage Accelerating manual testing processes so testers can do more in less time >Confident release decisions are supported by: Process monitoring with version histories and trend analysis Proactive risk management with automated, filtered and prioritize reporting Customizable dashboards that map pattern analysis to key productivity indicators for greater predictability > All this delivered on an open, extensible platform to interface with Rational and 3 rd party testing tools and add-ons
• #16: Now that we are delivering on our vision, we are gaining the accolades of analyst firms, like Gartner, who track and evaluate vendors' ability to deliver on integrated ALM solution. And as you can see here, IBM is the only vendor -- of twenty vendors evaluated -- who was rated as Strong Positive, because of its current market strengths, breadth of portfolio, and the solid architectural foundation of the Jazz platform. Strong Positive Is viewed as a provider of strategic products, services or solutions: Customers: Continue with planned investments. Potential customers: Consider this vendor a strong choice for strategic investments.
• #20: A single quality contract for the entire software delivery team represents the ‘glue’ <sts> that formalizes the business’ goals/objectives and entrance/exit criteria, defines the different team roles and responsibilities, and guides team members’ tasks, handoffs/review/approvals/and signoffs… one cohesive plan to get to end of job that all stakeholders and team members are focused on and collaboratively working toward completing. A plan is not a prescription, it is an evolving, moving target
• #22: Now, before we go any further, let me provide a little context and clarification around the term “Test Plan”. Most people when they hear the term think of 1 of 2 things. Either they visualize traditional word docs – where the plan is documented (but not the actual tests being run), or they visualize some sort of tree view that captures test cases (which ignores the business objectives and is effectively just a task list). In Rational Quality Manager – the test plan is a hybrid of these two types. It documents the traditional business objectives you’d capture in a word doc, as well as the dynamic content – like requirements and test cases you’d capture in a tree view. This provides the best of both worlds, capturing everything that matters to test teams.
• #25: The biggest hidden QM cost is associated with defect duplications. If we are able to control and minimize the number of duplicate defects that are captured we have an opportunity to not only drive down costs but also reduce the time team members are performing duplicate work. Duplicate Defect Identification capability within IBM Rational Quality Manager, scans existing defects associated with a variety of related artifacts and presents a list of suspect duplicates to to minimize duplicate team efforts.
• #26: Too often we still find projects testing products or systems as built, instead of verifying that those products and systems actually meet the goals laid out for them in the requirements. To achieve this, tests must be based on requirements – so furthering our approach to delivering the right capabilities to the right users in the right tools
• #27: >Plan Tests Early< Plan tests for each requirement as the requirement is written. Considering testing at the same time as capturing requirements improves the way the requirement is expressed by encouraging consideration of how the requirement is quantified. For every requirement, the question should be asked: How will we know if this requirement will be, or has been, satisfied? Note the use of both future and past tenses in this question. Early tests, such as design inspections, check that, when the product is built according to the design, it will meet requirements. Later tests check that requirements have been satisfied by what has been built. The above question should lead to a qualification strategy for each requirement. Becauseo f the broad range of qualification activities available, every requirement will give rise to multiple tests covering all stages of development. For instance, an early opportunity for qualification is to check the design against a requirement. Are the necessary elements present in the design to ensure that the requirement will be met?In addition, there will be one or more system tests to ensure that what is finally built really does meet the requirement. The set of selected tests form the qualification strategy. The question should also lead to the identification of qualification criteria for every test. These formalize what is considered a successful outcome for each test. Qualification criteria vary with the nature of the requirement and the planned test. >Conduct Tests Early< Perform tests as early as possible in the development process. “ A stitch in time saves nine.” Defects propagate themselves from stage to stage, and the cost of correcting them geometrically increases throughout the development process. Because of this, development cost and schedule can be dramatically reduced by identifying and correcting defects as early as possible, thus minimizing expensive rework in later stages. Examples of early testing are reviews, inspections, and walkthroughs. >Relate Tests to Requirements< Trace tests back to the requirements they are design to check. Tracing is a means of documenting the relationships between artifacts during development. The most common form of tracing is the satisfaction relationship between layers of requirements. However, the qualification relationship between requirements and the tests that show that they have been met is also important to capture. Tracing enables two kinds of analysis to be carried out: Coverage analysis is used to ensure, for example, that every requirement has at least one qualification action planned and eventually executed. It can also be used to ensure that every qualification action has an associated requirement and, therefore, provides benefit. Impact analysis is used to determine the test cases that may need to be revisited when requirements change. Before accepting a change to a requirement, the cost of redefining and re-executing its associated qualification actions is taken into account. Similarly, when a test fails, the impact of the failure can be viewed in terms of the requirements. Impact analysis uses a combination of the satisfaction and qualification relationships to determine potential rework. For instance, if a component test fails, it not only affects the associated component requirements (through the qualification relationship) but also the sub-system requirements that those component requirements are supposed to satisfy (through the satisfaction relationship) and, in turn, the satisfied system and customer requirements. Testers understand that testing at every level is necessary. For instance, a sub-system requirement cannot be fully tested by just testing its components; the sub-system (the integration of those components) still needs to be tested. >Relate Defects to Requirements< Trace defects back to the requirements that they show are not satisfied. When a test reveals unexpected behavior, or that test criteria are not met, defects arise. There are three possible causes of defects: An error in the definition of the test. An error in the expression of the requirement or in its test case. A genuine defect in the product. In every case, a defect is identified by tracing it to the requirements that are shown not to be satisfied. This is a good discipline because a defect is defined as a departure from requirements. Since each test may be traced to multiple requirements, analysis of each defect is needed to determine precisely which requirements are affected. Measure Progress against Requirements Set targets and measure the progress of testing in terms of those requirements that are shown to be satisfied or are not satisfied. One of the most difficult judgments to make during development is deciding when enough testing has been done. Limited resources are available, and the test manager needs to know where to apply effort most effectively. When testing is isolated from requirements, information about the relative importance of different aspects of the system is not available to inform the test manager, making it hard to allocate resources and difficult to assess the impact of failure. With the appropriate traceability in place, the relative importance of each test and the success and failure of tests can be related to the requirements affected at the desired level—customer requirements, system requirements, etc. Targets for completed testing can be set in terms of requirements. The test manager knows where to apply resources, and reports can be generated that show the progress of testing in requirements terms.
• #28: >Clean look : More space for relevant and most needed information   Better view for graphics - Should be able to compress all statements and see graphic one at a time, or expand all graphics inline or thumbnail etc. More real estate by making keyword view collapsible Every Manual test should have an optional associate VP (also known as expected result).  The concept we currently know as VP should be wrapped into each step. >Lesser keystrokes/Better/Faster navigation Users should be able to traverse through a manual test script using the arrow keys Add Preference for allowing ability to apply verdict/result to ALL step types Manual test execution: quickly jump ahead to a Step and mark it "Failed" Provide a warning message if you add defect associations in testscripts and cancel script without applying(click curve down arrow) to commit the step Manual Test Execution - Apply all faster if we avoid animation of the cursor
• #29: IBM INTERNAL USE ONLY Rational’s Quality Management solution delivers a comprehensive approach to managing the quality risk in your software development efforts:
• #30: IBM INTERNAL USE ONLY Rational’s Quality Management solution delivers a comprehensive approach to managing the quality risk in your software development efforts:
• #31: IBM INTERNAL USE ONLY Test lab Management Find physical and virtual lab assets Schedule test run Auto-provision lab assets Capture analytics for optimization Server requirements and configuration coverage for future tests and production
• #32: The third and final quality management subtheme focuses on making informed, confident decisions - our QM offerings achieve this with real-time metrics and measures built-in and delivered to you via dashboards and reports so you have full visibility into the quality status of your project Patterns of success: Deliver real time, fact-based reporting How can we track progress against goals? “ Organizations exercising world-class performance management practices enjoy a 2.4 times market returns of typical companies” BusinessWeek Study: The Payoff of Pervasive Performance Management When you think about your own transformation, do you have these types of questions? Are you executing development across state boundaries, countries or perhaps even continents? Do you have culture issues where your culture inherently does not allow you to achieve agile software delivery operations or perhaps an agile business model? Are you executing across a complex software supply chain, across in‑house developed applications, outsourced applications and perhaps packaged applications? Are you utilizing your human resources and talents to the best of your advantage? Are you applying your human talents to solve those most complex business challenges rather than applying them to repetitive tasks that could be automated? I would say that if you have some of these questions, then you probably need to ask yourself, where do I get started with this transformation journey? Sources Top 2/3's and 77% quotes Business Week and Business Objects study: "The Fact Gap: The Disconnect Between Data and Decisions“ http://www.businessobjects.com/global/pdf/whitepapers/fact_gap.pdf The middle quote on "revenue 75% below..." "How companies make good decisions: McKinsey Global Survey“ http://www.mckinseyquarterly.com/Strategy/Strategic_Thinking/How_companies_make_good_decisions_McKinsey_Global_Survey_Results_2282 Speaker Notes This slide + 4 more slides. This slide sets up the 3 main themes for executing with agility to make course corrections as needed to improve project predictability… The 3 slides that follow drill down on the capabilities that make this possible and then this will finish with a summary slide. This subtheme is about This solution offers you three primary capabilities: The first is about The second capability The third capability
• #34: Cognos – separate install and setup on separate “server”
• #36: VM Logix Surgient - BSD Group – Bi-directional integration between IBM Rational Quality Manager and HP Mercury Quality Center; Extend RQM context-based collaboration from legacy test assets Device Anywhere - Access to mobile application testing on more than 1,500 real handsets across the globe, while employing RQM to control execution of tests and keep track of results. Users can employ DeviceAnywhere’s test automation layer – available through its DeviceAnywhere Pro solution – to create and automate scripts on handsets, then utilize RQM to control the execution tests, keep track of results, and manage change requests. Weblayers - WebLayers expands the value of their automated SOA governance support for Rational customers with a planned Connector to Rational Quality Manager. Features like automating review / update of test plans based on policies and expanding the RQM Dashboard with Policy metrics, will enable better test quality and productivity across the services design-time lifecycle. Worksoft’s Certify execution tool adapter for RQM enables access to automated testing solutions for functional testing of SAP and other enterprise business applications. Finaris RapidRep execution tool adapter for RQM enables the RapidRep Test Suite RapidRep validates data, such as relational databases, or structured files like CSV or XML.
• #40: IBM Internal Use Only
• #41: Better, faster requirements lead to less project rework, faster time to market, and better business outcomes. Effectively engaging a wide range of customers and other stakeholders in the requirements process helps to uncover customer needs and define good requirements. In this Webinar, you will learn how Strongback Consulting and Rational Requirements Composer use A Single Source of Truth to help teams define and use requirements effectively across the project lifecycle. Companies who push quality to the end of the development lifecycle typically experience major project delays, software that is delivered and implemented with serious flaws and are on occasion forced to abandon a project completely. Learn how Strongback and Rational Quality Manager can help you deliver quality throughout the product and application lifecycle – from concept to launch to retirement. This integrated, lifecycle approach helps make quality a shared responsibility – not siloed and disconnected, enabling organizations to achieve greater consistency, efficiency and predictability in their delivered software solutions.
• #43: Standard closing slide which includes appropriate legal disclaimer to be included in all external presentations. Learn more links: IBM Rational software: www.ibm.com/software/rational Rational launch announcements: www.ibm.com/software/rational/announce/ Rational Software Delivery Platform: www.ibm.com/software/info/developer Accelerate change and delivery: www.ibm.com/software/rational/offerings/scm.html Deliver enduring quality: www.ibm.com/software/rational/offerings/testing.html Enable enterprise modernization: www.ibm.com/software/info/developer/solutions/em/index.jsp Ensure Web site security and compliance: www.ibm.com/software/rational/offerings/websecurity/ Improve project success: www.ibm.com/software/rational/offerings/lifecycle.html Manage architecture: www.ibm.com/software/rational/offerings/design.html Manage evolving requirements: www.ibm.com/software/rational/offerings/irm/ Small and midsized business: www.ibm.com/software/rational/smb/ Targeted solutions: www.ibm.com/software/info/developer/solutions/index.jsp Rational trial downloads: www.ibm.com/developerworks/rational/downloads Leading Innovation Web site: www.ibm.com/software/rational/leadership developerWorks Rational: www.ibm.com/developerworks/rational IBM Rational TV: www.ibm.com/software/info/television/index.jsp?cat=rational&media=video&item=en_us/rational/xml/M259765N40519Z80.xml IBM Rational Business Partners: www.ibm.com/partnerworld/pwhome.nsf/weblook/index.html IBM Rational Case Studies: www.ibm.com/software/success/cssdb.nsf/topstoriesFM?OpenForm&Site=rational