Building the Performance Measurement Baseline
0 likes228 views
The document outlines the capabilities and strategies of Lewis + Fowler in aerospace project management, emphasizing the importance of an integrated master plan (IMP) and integrated master schedule (IMS) for effective performance measurement and risk management. It details the flow of work packages, the significance of accomplishment criteria, and the connection between cost, schedule, and technical performance measures to ensure program maturity and success. The overarching theme is the need for a structured approach to enhance technical and programmatic performance through defined metrics and risk mitigation strategies.
Building the Performance Measurement Baseline
- 1. Lewis + Fowler Aerospace Capabilities Briefing (Full).pptx V5 Copyright © 2009, Niwot Ridge, Consulting LLC 1/74
- 2. 2/74
- 3. 3/74
- 4. 4/74
- 5. 5/74
- 6. 6/74
- 7. So much for our strategy of winning through technical dominance 7/74
- 8. Product Tree SOW IPTs IMP IMS BOEs CWBS 8/74 Each element of the PMB is supportive of the others The integrated whole is the basis of a credible PMB
- 9. Attribute Beneficial Outcome from this Attribute Maturity Flows through Program Events § Performance measurement is in units of increasing maturity of the Technical Performance Measures § Each event is a mini authorization to proceed Single outcome for each work package (AC) § Measure Physical Percent Complete at the WP level § Use 0/100 for tasks for the vast majority of work Technical Performance Measures are explicitly visible in the IMS § Connect Cost, Schedule, and Technical Performance § EV does not provide a means of adjust for “off TPM,” but make your own adjustments to the risk numbers for now Risk retirement explicitly visible in the IMS § Risk retirement is embedded in the IMS § Risk mitigation means waiting until the risk happens IMS flows vertically 1st and horizontally 2nd § All work supports the assessment of maturity § Isolate tasks dependencies within a Work Package No Event linkage except for long lead items § 0/100 requires not partial completion Decoupled dependences improves risk responsiveness § 1st round IMS defines a free flowing process § Maintaining this decoupling is key to a “dynamic” IMS that can respond to the natural changes in the program 9/74
- 10. 10/74
- 11. Risk Management Requirements Contractor Work Breakdown Structure (CWBS) CWBS Dictionary / SOW Integrated Master Plan (IMP) Integrated Master Schedule (IMS) Earned Value Management System Self Assessment / Award Fee Determinations / CPAR’s Technical Performance Measures (TPM) Process Metrics ü Specifications ü Work Statements ü Tailored to and outlines entire program and products ü Provides a single numbering system for the program ü States all deliverables and their components ü Follows IPT and CWBS structure ü Descriptions of work traceable to the IMS elements ü Event–driven Plan provides roadmap of processes for program ü Event–driven Schedule expands IMP into Work Packages ü Provides details and timing ü Builds in risk mitigation activities ü Connects measures of progress with measures of increasing maturity ü Connects System Performance with Continuous Technical Improvements ü Connects Cost Performance with the IMS Performance ü Determined by program execution 11/74
- 12. Program Events Statement of Work CWBS Significant Accomplishments Accomplishment Criteria CDRLs and Deliverables Tasks Contained in Work Packages Measures the progress to plan using Physical & Complete at the Accomplishment Criteria (AC) and CWBS level Defines Aligned Aligned Aligned Aligned Aligned Completed SA’s are entry criteria for Program Events Completed Work Packages are exit criteria for Tasks Describes increasing product maturity as 0/100 or EVMS SD guidance Documents the product maturity that is aligned with SOW and CWBS Work necessary to mature products grouped by CWBS Work structure aligned to SOW 12/74
- 13. 13/74
- 14. 14/74
- 15. DoD Guide DAG SEMP CWBS IMP IMS ISE Technical Reviews Event Timing of Technical Reviews X X X X X Success criteria Technical Reviews X X X X X Include entry and exit criteria for Technical Reviews in IMP/IMS X X Assess Technical Maturity in Technical Reviews X X X X Integrate SEP with: IMP/IMS, TPMs, and EVM X X X X Integrate CWBS with: Requirements specification, Statement of Work, and IMP/IMS/EVMS X X X Link risk management and mitigation plans with technical reviews, TPM, EVMS, and IMS X Integrated Reviews Include Technical Baseline Review (TBR) in IMS; X Integrated Baseline Review: plans for event based technical reviews, including entry and exit criteria Correlation of TPM’s, IMP. IMS, and EVMS X 15/74
- 16. 16/74
- 17. 17/74
- 18. 18/74
- 19. 19/74
- 20. 20/74 Only put these on your bookshelf after you’ve read them
- 21. 21/74 IMS IMP Describes the strategy for successful program delivery Supplemental Schedules in CAM Notebook Tasks Contained in Work Packages Accomplishment Criteria Significant Accomplishments Program Events Integrated Master Plan (IMP) § Identifies program events, significant accomplishments, and accomplishment criteria. System level, controlled document. Integrated Master Schedule (IMS) § Logic network schedule of program planned activities keyed to the imp’s accomplishment criteria Supplemental Schedule § Created (as needed) to provide lower levels of detail data within CAM notebook § Establishes the structure, parameters and basis for the integrated master schedule (IMS) development § Basis of performance measurement system; common element integrating cost, schedule, & performance § Constructed to provide integrated planning down to the work package task level, provides horizontal & vertical § Supports control account schedules and the management of day–to–day operations traceability, summarization of info and critical path identification and analysis schedules and are summarized in the IMS and part of the program’s PMB
- 22. 22/74 Form IMP Development Team Gather Inputs Establish Criteria For Events Define And Or Derive Deliverable Products Define Events For Program And Products Define IPT Structure To Implement Product Structure Expand WBS To IPT Structure Produce The Product IMP Matrix With CWBS Data Dictionary IMS Inputs OBS Update CWBS Update Define Program Demonstration Capabilities Define Accomplishments For Each Capability Set Order Of Program Events Select Topics For IMP Narratives Define Tasks And Work Products For Each Accomplishment IMS Inputs Summary Of IMP With E/A/C Initial Approval Of IMP Demonstrate How Subject Are Managed Release To Baseline Control For Use By IMS
- 23. … is an event–based plan depicting the overall structure of the program and the key processes and activities. It defines accomplishments and criteria for each event. With the IMP, the increasing maturity of the program is stated explicitly in the Significant Accomplishments (SA) and their Accomplishment Criteria (AC). It’s these key processes and activities that are the basis of the credibility assessment. Events are NOT milestones, they are mini–Authorizations to Proceed. These mini-ATPs are the basis of maturity assessments of the products and services of the program. Work performed beyond the upcoming Program Event is actually done “as risk.” 23/74
- 24. 24/74
- 25. 25/74
- 26. 26/74
- 27. 27/74
- 28. 28/74
- 29. 29/74
- 30. 30/74
- 31. Through PDR, the majority of the work is building the deliverables from the CDRLs. Each CDRL should be assigned to a single AC and be connected to a single PE for the defined maturity level. 31/74
- 32. 32/74
- 33. 33/74
- 34. 34/74
- 35. 35/74
- 36. § Without an IMP there is no strategy § Without an IMP the us no place for the work to reside § Without an IMP there is no way to answer to the question: “Why are we doing this?” § Without an IMP there is no way to measure the increasing maturity of the deliverables 36/74
- 37. 37/74 All work flows vertically from bottom up. From Work Package content to the Exit Criteria of the WP (AC), to the Significant Accomplishments (SA) to the Program Events (PE). This flow is a description of the work that produces increasing maturity.
- 38. 38/74 The activities of a Work Package “exit” through the Accomplishment Criteria (AC). Interconnections between Work Packages at the task level should be avoided. Only 100% complete work should start the next Work Package. PE SA ACs Tasks Completing AC starts task in next AC Completed Task lands on AC
- 40. 40/74
- 41. Cost Performance Baseline Schedule Performance Baseline Technical Performance Baseline Perform Functional Analysis Determine Scope and Approach Develop Technical Logic Develop Technical Baseline Develop WBS Define Activities Estimate Time Durations Sequence Activities Finalize Schedule Identify Apportioned Milestones Determine Resource Requirements Prepare Cost Estimate Resource Load Schedule Finalize Apportioned Milestones Determine Funding Constraints Approve PMB 41/74
- 42. 42/74 Program Events Define the availability of a Capability at a point in time. Accomplishments Represent requirements that enable Capabilities. Criteria Represent Work Packages that fulfill Requirements. Work Package Work Package Work Package Work Package Work Package Work Package Work Package Work package § The increasing maturing of a product or service is described through Events or Milestones, Accomplishments, Criteria, and their Work Packages. § Each Event or Milestone represents the availability of one or more capabilities. § The presence of these capabilities is measured by the Accomplishments and their Criteria. § Accomplishments are the pre–conditions for the maturity assessment of the product or service at each Event or Milestone. § This hierarchy decomposes the System Capabilities into Requirements, Work Packages, and the activities which produce deliverables. This hierarchy also describes increasing program maturity resulting from the activities contained in the Work Packages. § Performance of the work activities, Work Packages, Criteria, Accomplishments, and Events or Milestones is measured in units of “physical percent complete” by connecting Earned Value with Technical Performance Measures. The Structure of a Deliverables Based Plan Work Package
- 43. Monthly Status Report: § Validate schedule status (start/complete/slip) § Validate work package % complete § Take earned value from % Cmplt § Identify/process cost ETCs Weekly Status Report: § Roll up of lower level schedule status § Roll up of lower level % complete or § Milestone start/complete § Milestone slip (early/late, start/complete) § Physical % Complete of tasks Weekly Status Report: § Milestone start/complete § Milestone slip (early/late, start/complete) § % complete of tasks Near term Rolling Wave Control account span IMS Activities (at Work Package level WP’s < 45 work days) 20 to 40 Workday tasks Or even weekly tasks IMS Tasks (one and two levels below WP level) IMP WBS Levels 1, 2, and 3 WBS Levels 4 and 5 WBS’s Below Work Package Planning Package 43/74
- 44. 44/74
- 45. 45/74
- 46. Cost, Schedule, Technical Performance† WBS Task 100 Task 101 Task 102 Task 103 Task 104 Task 105 Task 106 †This is a Key concept. This is the part of the process that integrates the cost and schedule risk impacts to provide the basis of a credible schedule. Without Technical Performance measures, Cost and Schedule cannot show physical percent complete. Probability Density Function § Research the Project § Find Analogies § Ask Endless Questions § Analyze the Results § What can go wrong? § How likely is it to go wrong? § What is the cause? § What is the consequence? Monte Carlo Simulation Tool is Mandatory 1.0 .8 .6 .4 .2 0 Days, Facilities, Parts, People Cumulative Distribution Function Days, Facilities, Parts and People 46/74 DID–81650 calls out the details of the IMS and it’s statistical analysis
- 47. A ship on the beach is a lighthouse to the sea – Dutch Proverb 47/74
- 48. 48/74
- 49. 49/74
- 50. 50/74
- 51. 51/74
- 52. § In the example above, the Work Package Managers collectively come to an agreement on how the sequence of how the work will be performed within each AC. § The result is a collective ownership of the outcome. This ownership comes from the collective development of the project plan. § This process is a Product Development Kaizen (PDK) and used in major aerospace, defense and product development firms. 52/74
- 53. 53/74
- 54. 54/74
- 55. 55/74
- 56. 56/74
- 57. 57/74
- 58. 58/74
- 59. 59/74
- 60. 60/74
- 61. 61/74
- 62. AC: 005 Work Pkg Work Pkg Work Pkg Work Pkg AC AC:023 Work Pkg Work Pkg Work Pkg Work Pkg AC § The 100% completed work in AC:005 is needed to start the work in AC:023 § In the IMP/IMS paradigm, there is no Task-to-Task connection across Accomplishment Criteria (AC) boundaries, only within an AC § The AC-to-AC linking states “…all work in the predecessor AC must be complete before starting the successor work, assuring the minimum of rework due to partially defined requirements or partially completed products” 62/74
- 63. PE: B PE: A SA: 001 SA: 002 SA: 003 SA: 004 PE: A Task Task Task AC: 006 § The best arrangement has the completion of Event A start the first task in Event B. § All work performed beyond the date of Event A is done at risk. § At PDR (Event A), approval to proceed Event B (CDR) is given § Only long lead items should cross Program Event boundaries § All other work terminates on the Program Event where a formal review of the planned maturity is conducted – SRR, SFR, PDR, CDR, … § This topology assures a complete assessment of “progress to plan,” is available at each Program Event 63/74 SA: 008 PE: B
- 64. 64/74
- 65. A Credible Performance Measurement Baseline Assures Program Management has the needed performance information to deliver on‒time, on‒budget, and on‒specification Technical Performance Measures Cost Schedule Conventional Earned Value + = § The Master Schedule used to derive Basis of Estimate (BOE) not the other way around. § Probabilistic cost estimating uses past performance and cost risk modeling. § Labor, Materiel, and other direct costs accounted for in Work Packages. § Risk adjustments for all elements of cost. Cost Baseline § Earned Value is diluted by missing technical performance. § Earned Value is diluted by postponed features. § Earned Value is diluted by non compliant quality. § All these dilutions require adjustments to the Estimate at Complete (EAC) and the To Complete Performance Index (TCPI). Technical Performance § Requirements are decomposed into physical deliverables. § Deliverables are produced through Work Packages. § Work Packages are assigned to accountable manager. § Work Packages are sequenced to form the highest value stream with the lowest technical and programmatic risk. Schedule Baseline 65/74
- 66. 66/74
- 67. 67/74
- 68. 68/74
- 69. This approach is called Product Development Kaizen and is used by Lean Six Sigma firms to ferret out the system capabilities before any technical or operational requirements are defined. Use this to reverse engineer or validate the WBS and connect WHAT with WHY before proceeding to build the CWBS or confirm the WBS. 69/74
- 70. Update Contractor System Spec Update Program Development Allocate Functional Reqmts Update Functional System Design Develop HWCI Specifications Develop SIL Specifications Build Astp1 F-18 IRR SIL Baseline 1.0 Update SIL Test Cases Develop Prelim SIL CSCI Critical Components AstP 1,2 SSpS 1,2,3 1 2 3 4 6 7 5 8 10 9 11 13 14 15 Update AS Test I&T on CVN I&T on LHA 12 Contract Award + 15 days Systems Requirements Review (SRR) System Functional Review (SFR) HW Preliminary Design Review (PDR) System PDR EDM 1.0 Baseline EDM 2.0 Baseline Mfg Docs Available TBD TRR 1.0 EDM 7-8 TRR 70/74 § Each collection point provides an assessment of incremental mission value. § Defining these points before the project starts is the basis of measuring progress to plan. § Because then you know what done looks like before it arrives.
- 71. Deliverables WBS Tasks and Schedule Business Need Process Invoices for Top Tier Suppliers 1st Level Electronic Invoice Submittal 1st Level Routing to Payables Department 2nd Level Payables Account Verification 2nd Level Payment Scheduling 2nd Level Material receipt verification 2nd Level “On hand” balance Updates Work Package (WP) 1 2 3 4 6 5 A B Deliverables defined in WP Terminal Node in the WBS defines the products or services that produce the products of the project Terminal node of the WBS defined by a Work Package. Tasks within the Work Package produce the Deliverables 100% Completion of the deliverables is the measure of performance for the Work Package Management of the Work Package Tasks is the responsibility of the WP Manager. A decomposition of the work needed to fulfill the business requirements 71/74
- 72. 72/74 Maturity Action Product Product State Adjective Verb Noun Verb Complete Design Model/Sim Preliminary
- 73. Risk: CEV-037 - Loss of Critical Functions During Descent Planned Risk Level Planned (Solid=Linked, Hollow =Unlinked, Filled=Complete) Risk Score 24 22 20 18 16 14 12 10 8 6 4 2 0 Conduct Force and Moment Wind Develop analytical model to de Conduct focus splinter review Conduct Block 1 w ind tunnel te Correlate the analytical model Conduct w ind tunnel testing of Conduct w ind tunnel testing of Flight Application of Spacecra CEV block 5 w ind tunnel testin In-Flight development tests of Damaged TPS flight test 31.Mar.05 5.Oct.05 3.Apr.06 3.Jul.06 15.Sep.06 1.Jun.07 1.Apr.08 1.Aug.08 1.Apr.09 1.Jan.10 16.Dec.10 1.Jul.11 Risk Response and Risk ID in IMS Milestone Date traceable between RM Tool and IMS 73/74
- 74. Glen B. Alleman Niwot Ridge Consulting 4347 Pebble Beach Drive Niwot, Colorado 80503 303.241.9633 glen.alleman@niwotridge.com Performance Based Planning® Integrated Master Plan Integrated Master Schedule Earned Value Risk Management Proposal Support Services