Sandra.smalley
- 1. 2008 PM Challenge NASA’s Business Systems Are they working for you? Sandra Smalley OCIO IEMP Integration Manager ssmalley@nasa.gov 202-358-4731 PM Challenge 2008 1
- 2. Contents • M/BSIG – Background – Membership – Overview of Requirements Governance • Gap Analysis Activity – Gap Definition, Scope and Depth – GAP Corrective Action Plan development process – Identification Process Summary – Gap Statement Summary – M/BSIG integration and prioritization results – Gap relationships and corrective action strategy – CAP Schedule • Concept of Operations (ConOps) Overview – Overview – Background – ConOps Development Approach – ConOps Schedule – Integration into CIO’s management strategy – Business system portfolio examples PM Challenge 2008 2
- 3. Background - Management/Business System Integration Group (M/B SIG) • IPAO/PA&E White Paper presented to OMC resulted in OMC action to perform gap analysis and establish M/BSIG – July 2006 • M/BSIG Kick-Off Meeting Held - January 2007 • Charter signed - February 2007 – The charter of the M/BSIG is to assess and prioritize future business system requirements to ensure integration and alignment with Agency goals and objectives – Participants included PA&E, OCE, MD’s, OCFO, OCIO, NSSC, OHCM, I&A, OP, OPII, MSFC, GRC, GSFC, and IEMP (ex-officio) • Charter updated – October 12, 2007 to: – include all ten centers – reflect decision authority of the associate deputy administrator – reflect scope expansion to include all agency-wide business systems – include records retention process PM Challenge 2008 3
- 4. M/B SIG Membership as of November 2007 • ARMD – John Scholtz • GRC – Harvey Schabes • ESMD – Marlana Dorman • GSFC – George Barth • SMD – Craig Tupper • DFRC – John Wonacott • SOMD – Toni Mumford • LaRC – Dan Tenny • MSFC – Johnny Stephenson • SSC – Kern Witcher • PA&E- Cherish Johnson • JSC – Lucy Kranz • OCE – Mike Blythe • KSC – Vanessa Stromer • OCFO – Daphne Jefferson • JPL – Joanne Kennedy • OCIO – Gary Cox • ARC – Trish Morrissey • OCHM – Candy Irwin • HQ(Center) – Michele O’Connell • I&A – Jim Wright • IEMP – Bobby German • OP – Ken Stepka • NSSC – Joyce Short • OPII – Rita Svarcas • Facilitator – Sandra Smalley PM Challenge 2008 4
- 5. M/BSIG Responsibilities • Ensuring the management and business system needs, goals, and objectives of all relevant stakeholders (e.g., internal NASA stakeholders as well as external authority stakeholders) are proactively elicited and considered when new or revised systems are developed. • Ensuring the management and business system requirements are developed with an integrated Agency-wide perspective. • Ensuring the management and business system priorities and the sequencing or phasing of implementation efforts (e.g. policy updates, process re-engineering, IEMP projects) are developed in recognition of budgeting, resource, and other constraints (e.g., compliance with Federal laws, regulations, and policies; other externally mandated requirements). • Ensuring integration of Agency-wide business system requirements and alignment with Agency priorities. • Ensuring Agency-wide business system requirements are traceable throughout the entire requirements chain. • Ensuring major stakeholder proposals relating to new or changing business system scope, content, and priorities are appropriately reviewed from an integrated Agency-wide perspective. • Ensuring oversight and direction of management and business system initiatives are appropriately focused on cross system integration and the reduction of stove-piped and redundant data silos. PM Challenge 2008 5
- 6. Overview of Requirements Governance Operations Management Council OCIO IT Strategy and IT DM&E Investment Board Requirements Planning & Execution IT DM&E Performance Management Management Assessment Status Briefings & Recommended Actions Implement Implement Recommendations for & Requirements via the Non-IT IT-DM&E (1) Significant Associate Deputy DM&E Actions Changes to Project Administrator Actions Baseline, or (2) Termination of Projects System Program Execution Status Coordination Implementers M/BSIG (IEMP, NSSC, etc.) OCIO PMB •Projects Project Performance Reporting •Sustaining Operations Lower Level Requirements Priorities & Needs & Decisions Requirements for Cross-Functional Lower Level Lower Level Review and Prioritization Process/ Requirements Requirements Governed Issues Functional by the Requirements Management Owners Framework Customers (Mission & Mission Support Orgs) PM Challenge 2008 6
- 7. Gap Analysis PM Challenge 2008 7
- 8. Gap Definition, Scope and Depth • Gaps exist when the contents or behaviors of systems, in this case NASA’s Management and Business Systems, are different from what is needed. • Four general types of system Gaps: 1. Data 2. Process • Incomplete data • Non-congruent with local processes • Missing data • Work-around solutions • Redundant data • Outside post-processing of the • Incorrect data system data 2. Application 3. Human • Inability to retrieve data • Changing roles and responsibilities • Inability to input data • Training and knowledge of the • Inability to modify data system • Non-standard practices • Scope: – Includes all Agency-wide systems which process and/or provide management and business related information • Depth: – Includes gaps associated with the data, applications, processes, and the human aspects of the management and business systems PM Challenge 2008 8
- 9. Gap Corrective Action Plan (CAP) Development Process GAP Identification and OMC/OCIO SIB Prioritization Phase Approval • Plan • Collect & characterize data activity CAP Development Is IT DM&E • Analyze & Process data Phase Required? * • Prioritize gaps N • Identify stakeholders • Design workshop Y • Perform causal analysis • Develop corrective Concept Identification CAP Implementation action plan (including Phase Phase identification of • Develop candidate concepts, requirements) cost estimates, and • Brief M/BSIG schedules • Perform trade studies • Develop cost estimates • Develop M/BSIG implementation recommendation • Present to the OCIO SIB • Submit approved recommendation for inclusion in the PPBE M/BSIG Oversight * IT DM&E: Information Technology Development, Modernization, or Enhancement PM Challenge 2008 9
- 10. Gap Identification & Ranking Process Summary • Project Management Gaps were collected from 5 representative projects – 58 Gaps were prioritized utilizing by the M/BSIG in July 2007 using a Delphi process – 8 Project Management Gaps were identified as relatively high priority • Functional Gaps were collected from Procurement, Human Capital, Institutions and Administration, and Finance – 19 Gaps were prioritized by the M/BSIG in September 2007 using a Delphi process – 4 Functional Gaps were identified as relatively high priority • The M/BSIG integrated and prioritized the resulting top 12 project management and functional gaps on October 1 & 2, 2007 using an Analytic Hierarchy Process (AHP) PM Challenge 2008 10
- 11. Gap Statements Summary The Relatively High Priority Gaps Project Management Functional/Mission Support Priority Gap # Gap Description Priority Gap # Gap Description 1 Gap-16 The Projects do not always receive 1 Gap-64 The OCFO does not have an funding in a timely manner. automated process and standardized 2 Gap-51 Projects have concerns over the tools necessary to facilitate timely and integrity of data in the system. accurate reconciliation of the SAP 3 Gap-2 The projects are limited by system funds balance with the Treasury configuration to obligate and cost funds general ledger account. at the same WBS level. 2 Gap-66 The missions and mission support 4 Gap-49 The Projects have difficulty maintaining organizations do not have a effective planning and control during standardized, integrated set of tools the end-of-year closeout period. and processes to create, maintain, 5 Gap-1 The Projects occasionally need more track and report phasing plans. WBS elements (levels) than are 3 Gap-59 The OCFO and the Center Property available. Custodians do not have a real property 6 Gap-18 The Projects receive conflicting data system, integrated with NASA’s when pulling reports on the same financial system, to track the value of information but from different systems. NASA’s real property portfolio. 7 Gap-4 The projects can not use SAP/BW to 4 Gap-67 NASA does not have the capability to meet the current set of reporting load the next fiscal year budget requirements. structure prior to current fiscal year end 8 Gap-27 Projects report that the integration of close. WBS, financial, manpower planning, and scheduling data is a problematic and manual process. PM Challenge 2008 11
- 12. M/BSIG Integrated Gap Prioritization based upon “Benefit to the Agency” Priority Gap Statement 1 Gap 51 Projects have concerns over the integrity of data in the system. 2 Gap 2 The projects must obligate and cost funds at the same WBS level. The OCFO and the Center Property Custodians do not have a real property system, 3 Gap 59 integrated with NASA's financial system, to track the value of NASA's real property portfolio. 4 Gap 16 The Projects do not always receive funding in a timely manner. 5 Gap 1 The Projects occasionally need more WBS elements (levels) than are available. The Projects receive conflicting data when pulling reports on the same information but from 6 Gap 18 different systems. The OCFO does not have an automated process and standardized tools necessary to 7 Gap 64 facilitate timely and accurate reconciliation of the SAP funds balance with the Treasury general ledger account. Projects report that the integration of WBS, financial, manpower planning, and scheduling 8 Gap 27 data is a problematic and manual process. NASA does not have the capability to load the next fiscal year budget structures prior to 9 Gap 67 current fiscal year end close. The missions and mission support organizations do not have a standardized, integrated set 10 Gap 66 of tools and processes to create, maintain, track and report phasing plans. 11 Gap 4 The projects can not use SAP/BW to meet the current set of reporting requirements. The Projects have difficulty maintaining effective planning and control during the end of year 12 Gap 49 closeout period. PM Challenge 2008 12
- 13. Gap Relationships Data Integrity Planning and Control Funds balance during year end close Gap with Treasury 51 Funds distribution Gap Gap Independent Gaps 49 64 Gap 16 67 Separation of Gap Gap Integration Gap Gap Obligations from Costs 2 27 of data 59 66 Gap Real Property Gap Phasing Plans 18 Gap Status: Conflicting report data 67 16 Closed from different systems Loading Budget Additional Gap Gap Reporting structures In Progress WBS levels 1 4 standards & Pending automation • Aspects of gaps 49, 2, 18, 27, 64, 1 & 4 all contribute to the data integrity concerns identified in gap 51. Contributing gaps will be evaluated prior to gap 51. • Gaps 1 and 2 both relate to WBS structures and will be evaluated in unison. • Gaps 4 and 27 both relate to performance assessment/reporting and will be evaluated in unison. • Resolution of gap 67 will further improve the funds distribution process identified in gap 16. • Gaps 59 and 66 stand alone. • Gap 49 was attributed to SVU and corrective actions have been implemented. M/BSIG will verify for closure. • OCFO has developed CAPs and work is in progress for Gaps 64, 16 & 67. No additional workshops are scheduled as part of the CAP planning phase for these gaps. • The CAP development schedule takes into consideration Agency priority, gap relationships and work in progress. PM Challenge 2008 13
- 14. Revised - Gap Identification and CAP Development Schedule FY07 FY08 FY09 Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Gap Identification & Prioritization Phase CAP Development Phase Concept Identification Phase CAP Implementation Phase (Dependent upon approval, $, & human resource availability) Decision Point Decision Point Decision Point GAP PM MSO M/BSIG OMC Evaluate Evaluate OMC OMC PPBE Identification Gaps Gaps Integration Status Process Process Status Approval Insertion ATP Identified Identified & & & Concept Point Prioritization Schedule Schedule Implementation M/BSIG Cap Review Plan CAP 1&2 Concepts: 1&2 M/BSIG Concept Review M/BSIG Cap Review Collect Data CAP 59 Concepts: 59 M/BSIG Concept Review M/BSIG Cap Review Analyze & Process Prioritization CAP 18 Concepts: 18 M/BSIG Concept Review M/BSIG Prioritization M/BSIG Cap Review & Integration CAP 4&27 Concepts: 4&27 M/BSIG Concept Review M/BSIG Cap Review CAP 51 Concepts: 51 M/BSIG Concept Review M/BSIG Cap Review CAP 66 Concepts: 66 M/BSIG Concept Review Optional – CAPS 16, 64, 67 & 49 Optional Concepts: 16, 64, 67 & 49 PM Challenge 2008 14
- 15. Agency wide Business System Concept of Operations Development PM Challenge 2008 15
- 16. ConOps Overview – What is it? • A systems engineering tool typically developed early in formulation to describe how a system will operate from a users perspective – Not a technical specification; described in layman’s terms: – Describes what exists currently – Describes where the Agency is heading • Overview of NASA’s ConOps for agencywide business systems: – Describes the desired operational state of NASA’s Agency-wide business management systems. • Will address the business system needs of both the programmatic and institutional communities. – Provides a framework to focus future business management systems initiatives by • describing agency business functions, processes, and needs • defining system boundaries and major system components. Note: Detailed process re-engineering and systems modifications will only occur when the Agency makes a strategic investment decision to address gaps between the “as is” and “to be” state. • Expected Outcomes – Provide a description of the system characteristics from an operational perspective. – Facilitate understanding of the overall system goals with users (including recipients of the products of the system, where applicable), buyers, implementers, architects, testers, and managers. – Form an overall basis for long-range operations planning and provide guidance for development and/or update of subsequent system definition documents such as the system specification and the interface specification. – Describe the user organization and mission from an integrated user/system point of view. – Provide information for strategic and tactical decisions regarding Agency-wide business systems which in turn will ultimately improve the availability of management information necessary for mission success. PM Challenge 2008 16
- 17. Background – Why do we need one? • Addresses recommendations in the GAO report on business modernization (GAO- 07-691) – “We recommend that the NASA Administrator establish as a high priority the completion of a concept of operations that addresses NASA’s business operations for both its mission offices and administrative offices (such as financial management and human capital) before any new implementation efforts begin.” (pg. 25) – “For NASA, an effective concept of operations would describe, at a high level, (1) how all of the various elements of NASA’s business systems relate to each other and (2) how information flows among these systems. Further, a concept of operations would provide a useful tool to explain how business systems at the agency can operate cohesively.” (pg. 17) – “As part of an agencywide concept of operations, to best leverage its investment in IEMP, NASA should also analyze the agency’s current business process and determine how these processes can be made more efficient and effective.” (pg. 28) • Key initiative in the NASA plan for improvement in the GAO High-Risk area of Contract Management (Initiative F1) – Commitment to baseline ConOps by Sep 08 • NASA concurs and recognizes the need for an integrated vision for agency wide business processes to: – form an overall basis for long-range planning – support business modernization investment decisions – provide Operations’ guidance for system definition and design PM Challenge 2008 17
- 18. ConOps informs Decision Making OMC: • Decision Authority M/BSIG: • Assess integration and alignment with Agency Priorities • Ensure agencywide perspective for new investments • Support prioritization of needs, goals, objectives & requirements ConOps PM Challenge 2008 18
- 19. Concept of Operations Approach to Development • Development planned through a series of workshops • IEEE Standard 1362-1998 provides content guidance • SCOPE: Reflects all users needs (including the PM community) from agency business functions (including finance, procurement, human capital, institutions and administration, and other agencywide MSO business systems (as identified)) – Interfaces to other functions will be documented (e.g. science and engineering, project management, and IT infrastructure) • Workshops include all NASA stakeholders – Information Providers – Information Consumers • CFO • MDs • Internal Controls • CIO • Centers • Strategic • Human Capital • OSMA Communications • Institutions and • PA&E • Innovative Management • OCE Partnership Program • Procurement • OPII • External Relations – Information Service Providers • GC • Chief Health and • IEMP Medical Officer • IG • NSSC All users are information consumers • Others? PM Challenge 2008 19
- 20. Preliminary Concept of Operations Development Schedule FY08 Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep CIT 1st 2nd Baselined Data Gathering Data Gathering Kick-Off Integration Integration Draft Draft Draft Plan Define Scope Formal Document Document Final First Draft Second Draft Doc. Current State To-Be State Draft Review Dev. Ops. Scenario Development Walkthrough PM Challenge 2008 20
- 21. Integration into Agency Management Strategy Integration into Agency IT Strategy for Managing Service Investments The ConOps will describe the future state for the Agency’s Business Management Applications Portfolio NPR 2800 NPR 7120.5 NPR 7120.7 Governance & Enterprise Compliance Policy Architecture IT Security Highly Specialized Science and Project Business Infrastructure Examples: Engineering Management Management Applications Applications Applications Applications Avionics Relationship Resource Innovation software Alignment Management Management Management Real-time Control Infrastructure Services Systems Onboard Processors Service Project End User Communications Data Center Service Delivery Performance Deep Space Mgmt. & Management Management Network Delivery (Ops) (Development) NASA’s IT Portfolios CIO Core Functions PM Challenge 2008 21
- 22. Business Management Portfolio Examples Owner OCE MD’s OCE MD’s ADA OCFO OHCM OP I&A Other MSO’s OCIO MD’s Centers OCIO facilitates and provides stewardship for this portfolio management process PM Challenge 2008 22
- 23. Backup PM Challenge 2008 23
- 24. Projects Selected Coordinated with 9 M/B SIG organizations to select representative sample of 5 NASA Projects - set includes projects from all four Mission Directorates - Used formal selection methods with 30 Characteristics in 8 groups • Hypersonic Boundary Layer Transition Experiment (Hy-BoLT/HSA) ARMD, Workshop and Interviews • Geostationary Operational Environmental Satellite (GOES-N) – SMD, Workshop and Interviews • James Webb Space Telescope (JWST) – SMD, Workshop and Interviews • International Space Station – SOMD, Workshop and Interviews • ORION – ESMD, Workshop and Interviews PM Challenge 2008 24
- 25. Project Management Gaps Relative Priority Clustering (M/B SIG) Objective: to determine the relative priority thresholds Gap-Numbers Clustered Distribution of Priority for Normalized Gaps (as prioritized by M/B SIG) 6 TOP 8 Relatively High 15,24,37,52 12,17,40,41 5 Priority PM GAPS 29,56,58 25,33,48 32,39,57 28,30,31 3,26,'36 6,10,21 4 Number of Gaps 7,9,55 3 42,53 14,35 23,46 11,20 43,44 8,50 4,27 2 34 22 19 54 47 13 38 45 18 49 51 16 5 1 2 1 0 0.1- 0.2- 0.3- 0.4- 0.5- 0.6- 0.7- 0.8- 0.9- 1.0- 1.1- 1.2- 1.3- 1.4- 1.5- 1.6- 1.7- 1.8- 1.9- 2.0- 2.1- 2.2- 2.3- 2.4- 2.5- 2.6- 2.7- 2.8- 2.9- 3.0- 3.1- 3.2- 3.3- 3.4- 3.5- 3.6- 3.7- 3.8- 3.9- 4.0- 4.1- 4.2- 4.3- 4.4- 4.5- 4.6- 4.7- Source File: Gap Priority Distribution 070719-D.xls Average Priority Weight Assigned by M/B SIG Low Relative Priority Medium Relative Priority High Relative Priority How to interpret this graph – Gap-16 has average priority weight of 5.6 and is ranked the highest by the M/B SIG – Gaps 42 and 53 both have average priority weight of 0.1 and are ranked the lowest – based on the apparent cluster groups, the 25 gaps ranked between 0.1 and 1.0 are deemed of relatively low priority, the 25 gaps ranked between 1.4 and 3.1 are deemed of relatively medium priority, and the 8 gaps ranked between 3.6 and 5.6 are deemed of relatively high priority PM Challenge 2008 25
- 26. Functional/Missions Support Gaps Relative Priority Clustering (M/B SIG) Priority Distributions (High, Medium, Low) 3 TOP 4 Relatively High Priority 61, 72 Functional Numbere oGaps Nm offG s 2 GAPS u br ap 70 69 78 76 60 68 73 77 75 79 63 62 71 67 59 66 64 1 0 0 5 0 5 0 5 0 5 0 5 0 5 0 5 0 5 0 5 0 5 0 5 1. 1. 2. 2. 3. 3. 4. 4. 5. 5. 6. 6. 7. 7. 8. 8. 9. 9. 1. 0 1. 0 1. 1 1. 1 We ighte d Priority Relatively Low Relatively Medium Relatively High Priority Gaps Priority Gaps Priority Gaps PM Challenge 2008 26
- 27. Gap-51 (Ranked Overall Priority 1) Gap Statement: • Projects have concerns over the integrity of data in the system Gap Elaboration: • The Projects do not have control over charges placed against their WBS so cost data in the system may not reflect the true work performed for the Project labor costs, particularly for pool type labor and for comp-credit-over time, are not always timely - the charges may be legitimate but not recorded in the period when the work was performed - also, the pooled labor charges may reflect work performed for some other Project. • There are issues with the level of detail in contractor data. In some cases contractors report data at lower levels of detail than the system can track (e.g., reports labor and material costs separately but the system can not track them separately). Reports of the "same data" from different systems (ALDS, WIMS, WebTADS, etc...) are sometimes different. • There is a lack of guidance and standardization in which reports and filters to use for which business processes, some reports have built in filters or exclusions that are not apparent. • There is ambiguity and a lack of confidence in the system's data during the several weeks which the system is closed for end-of-year processing, and several weeks after the system returns on-line before the data stabilizes. • After system updates or fixes are installed there often seems to be a large number of errors until the user's figure out what has changed and if everything is working properly. • Projects reported that given the same set of filters, but requesting different levels of drill down, may result in different report totals. • Project and Center participation in the SVU implementation testing and associated bug fixes did not appear well planned or coordinated (very little visibility into the test procedures, no apparent "full system" testing). • Project can not control costing of funds to a shared contract. • SAP is configured to require obligations and costing at the same WBS Level, this is apparently based on an Agency Policy. However, it forces the Projects to accept a trade off between having a detailed enough WBS (working to low enough WBS levels in SAP to sufficiently plan, track, and manage work) against the high level or effort necessary to manipulate SAP at those detailed WBS levels (shifting funds across lower level WBS elements requires significant SAP transactions to 'roll back' previous transactions, redistribute guidelines across the affected WBS elements, then reenter all the SAP transactions). As a result the projects often accept a higher WBS level of obligation in SAP then manually tracking costs at lower WBS levels with spreadsheets or other tools outside the system. Team Observations: • There are apparent inconsistencies in some cases when the "same data" is retrieved through different system interfaces or from different system reports. Also, the reported labor does not necessarily reflect work actually performed for the project during the reporting period. • This is a multifaceted issue that includes concerns with the data as well as the applications and processes which handle the data. • This gap reduces confidence in the accuracy and reliability of the data. It diverts effort from other priorities as Projects cross check and manually verify reported data. It can increase the frequency of queries to support organizations trying to reconcile the discrepancies. It may also negatively impact a Project's performance trends and will certainly be an issue as the Agency implements EVM. • NOTE: This is related to Gap-18 but is more focused on issues with data accuracy and level of data resolution where as Gap-18 is more focused on issues between data from different reports and system interfaces. Relative Gap Complexity Factor • 7-28;28;28;14 PM Challenge 2008 27
- 28. Gap-2 (Ranked Overall Priority 2) Gap Statement: • The projects must obligate and cost funds at the same WBS level Gap Elaboration: • SAP is configured to require obligations and costing at the same WBS Level, this is apparently based on an Agency Policy. However, it forces the Projects to accept a trade off between having a detailed enough WBS (working to low enough WBS levels in SAP to sufficiently plan, track, and manage work) against the high level or effort necessary to manipulate SAP at those detailed WBS levels (shifting funds across lower level WBS elements requires significant SAP transactions to 'roll back' previous transactions, redistribute guidelines across the affected WBS elements, then reenter all the SAP transactions). • As a result the projects often accept a higher WBS level of obligation in SAP then manually tracking costs at lower WBS levels with spreadsheets or other tools outside the system. Team Observations: • The projects indicated that requirements to obligate and cost at the same WBS level significantly increases the effort necessary to manage within the system • This is primarily an Agency policy issue which drove application design constraints. • This gap forces the Projects to choose between spending the additional effort necessary to maintain sufficiently detailed data in the system or spending less effort but maintaining the more detailed data outside of the system. The Projects usually chose the path of least effort which means some Projects choose to manage critical data outside of the system. Relative Gap Complexity Factor: • 5-20;40;20;20 PM Challenge 2008 28
- 29. Gap-59 (Ranked Overall Priority 3) Gap Statement: • The OCFO and the Center Property Custodians do not have a real property system, integrated with NASA's financial system, to track the value of NASA's real property portfolio. Gap Elaboration: • This is necessary to achieve a "clean audit" • Need a single integrated system that tracks all capital assets for the agency. • Standardizes real property classifications and ledger accounts (e.g. buildings vs. structures) Projected Benefits: • Improves NASA's financial integrity • Eliminates reconciliation data entry resulting in improved efficiency and accuracy of the function. PM Challenge 2008 29
- 30. Gap-16 (Ranked Overall Priority 4) Gap Statement: • The Projects do not always receive funding in a timely manner. Gap Elaboration: • Even after approval of the Agency's operating plan there is often a several week delay before projects receive funding. • The problem appears to be before funds are distributed to the Mission Directorates as the MDs seem to quickly distribute funds once received. • Project must accommodate these delays with work stoppages/delays, contractors working at risk, or additional effort to realign remaining prior year budget to cover the delayed funds. Team Observations: • The Projects indicated that a delay in receiving funds significantly increases the risk of not completing as planned. • This is primarily a process issue. • This gap can prevent the start of new work and interfere with the timely payment for work already in progress. It may result in contractors deciding to begin work at their own risk. Projects often divert efforts from other priorities in order to redistribute work and available funds. This can negatively impact a Project's performance trends and will certainly complicate the ability to produce reliable and accurate EVM reports. Relative Gap Complexity Factor: • 2-0;0;50;50 PM Challenge 2008 30
- 31. Gap-1 (Ranked Overall Priority 5) Gap Statement: • The Projects occasionally need more WBS elements (levels) than are available. Gap Elaboration: • The projects often use spreadsheets or other tools to establish lower level structures for tracking work below the lowest WBS level allowed in SAP. • This is sometimes related to the need for more visibility into tracking contractor's work and cost. • This also applies to cases where in-house sub project type work is distributed across different Centers. Team Observations: • Several projects report that the current number of WBS levels available in SAP does not provide a sufficiently detailed level of information to effectively manage the work. • This is primarily a policy issue which drove application constraints that limited the resolution of data a project can maintain in the system • This gap results in Projects maintaining necessary levels of WBS detail in other tools outside the system. This additional detail is most likely necessary for the Agency to implement effective EVM but such data is not available to the SAP system. Relative Gap Complexity Factor: • 2-50;50;0;0 PM Challenge 2008 31
- 32. Gap-18 (Ranked Overall Priority 6) Gap Statement: • The Projects receive conflicting data when pulling reports on the same information but from different systems. Gap Elaboration: • Reports on civil servant manpower actuals from different sources (i.e., ALDS, BRIO, WISP, Web TADS) can have different numbers - There is ambiguity over the source authority and which system has precedence. • The lag time between pay date and ALDS reporting of charges to the project make it difficult to reconcile or make Web TADS changes within the allowed three pay period window. • This is compounded by the lack of standard reporting policy and templates, each project pulls data from the various systems according to their own favorite report, filters, and bookmarks. • The Agency seems to have multiple systems/interfaces performing the same (or similar) functions (the variety of labor and workforce related systems is a prime example). There is ambiguity regarding the overlapping roles, responsibilities, and data contents of these systems. Team Observations: • Several projects indicated that the data reported from different systems is not always consistent. This may be a timing issue related to a lack of some user's awareness about how batch oriented transactions propagate data through the systems. It may also be related to confusion over specific roles and responsibilities of systems with similar functions. • This is primarily an application interface issue. However, the lack of a strong Agency policy regarding development of a uniform system architecture has allowed multiple and partially redundant systems and interfaces to emerge. • This gap reduces confidence in the system. It diverts efforts from other priorities as Projects evaluate multiple reports before selecting which data to report, • NOTE: This gap is related to Gap-51 but is more specific to the system's labor reports where as Gap-51 is more specific to the system's data accuracy and data resolution Relative Gap Complexity Factor: • 4-25;50;25;0 PM Challenge 2008 32
- 33. Gap-64 (Ranked Overall Priority 7) Gap Statement: • The OCFO does not have an automated process and standardized tools necessary to facilitate timely and accurate reconciliation of the SAP funds balance with the Treasury general ledger account. Gap Elaboration: • NASA's inability to balance funds with Treasury has been an annual audit deficiency for years. • Achievement of a clean audit is a high priority to the administrator. Projected Benefits: • Improve NASA's credibility with Congress, OMB, and the general public. • Improve NASA's efficiency to better utilize the Agency's funds to support the mission. • Increase likelihood of obtaining a clean opinion PM Challenge 2008 33
- 34. Gap-27 (Ranked Overall Priority 8) Gap Statement: • Projects report that the integration of WBS, financial, manpower planning, and scheduling data is a problematic and manual process. Gap Elaboration: • There is no Agency-wide resources/budget integration and planning tool that can be used by different Centers working on the same project. • The IBOT tool works well within JSC but doesn't integrate and provide detail info for work at other Centers, the system can't capture actuals across all stakeholders (marrying phasing plan with actuals). • There is no Agency designated system of record for scheduling data, configuration management data, and other critical but non-SAP data - however, there are numerous external and ad-hoc systems which Projects and Centers have developed around SAP/BW to provide this integration - such systems provide only limited and sometimes questionable integration. • How to do EVM on large, multi-center, multi-contractor projects using Agency provided tools and a standard process hasn't been developed/communicated (No standard process or tools to establish and manage EVM baselines). Team Observations: • Most Projects indicated there is a lack of guidance and standards for integrated cost, schedule, and work planning. There is also a lack of appropriate tools for developing integrated reports. • This is fundamentally a policy issue. However, the lack of a strong Agency policy regarding the development and implementation of a uniform system architecture, has allowed multiple and partially redundant systems and interfaces to emerge. • This gap has contributed to the proliferation of local "integration" tools across then Centers, tools which may serve a local need but often do not integrate effectively with other Agency systems. The lack of integration across cost, schedule, and work systems will negatively impact the Agency's ability to implement effective EVM. Relative Gap Complexity Factor: • 4-0;50;50;0 PM Challenge 2008 34
- 35. Gap-67 (Ranked Overall Priority 9) Gap Statement: • NASA does not have the capability to load the next fiscal year budget structures prior to current fiscal year end close. Gap Elaboration: • Delays funds distribution process • Biggest impact to new initiatives but also delays funding to existing projects Projected Benefits: • Seamless transition between fiscal years from a funding perspective. PM Challenge 2008 35
- 36. Gap-66 (Ranked Overall Priority 10) Gap Statement: • The missions and mission support organizations do not have a standardized, integrated set of tools and processes to create, maintain, track and report phasing plans. Gap Elaboration: • Better insight into unobligated balances is necessary to more effectively manage funds distribution agency-wide. • In the year of execution, better insight into our phasing plans and performance against those plans will ensure NASA is adequately expending appropriated funds. • An enhanced phasing plan processes and tools also provide missions enhanced capability to assess performance. Projected Benefits: • Benefits both missions, the CFO, and other institutional organizations by allowing CFO to more effectively manage the flow of funds. PM Challenge 2008 36
- 37. Gap-4 (Ranked Overall Priority 11) Gap Statement: • The projects can not use SAP/BW to meet the current set of reporting requirements. Gap Elaboration: • There seems to be no Agency standard process or tools to implement the reporting requirements. • Projects frequently export data from multiple systems into Excel. The data is then integrated in Excel to generate the required reports. • There are lots of related systems containing data, but the projects can't get integrated reports across all the related systems - significant effort is required to manually integrate the data and compile the reports. • Different people/organizations want to see data in different formats - no two seem to have the same format requirements - there is no standardization in the reporting format - requires manual translation and extra effort. • The required reporting format often doesn't match the output capabilities of the commonly used tools - requiring manual "cut & paste" to create the necessary charts and reports. Team Observations: • The existing reporting capabilities of SAP/BW seldom meet the reporting requirements levied on the Projects. • This is primarily an application issue which has been compounded by the lack of reporting standards across the Agency. • This gap causes Projects to spend time and effort exporting and manually integrating relevant data into the multiple formats necessary to meet all the different reporting requirements. As a result, it is very difficult to maintain consistency in the reporting format and contents across Projects. Relative Gap Complexity Factor: • 4-0;50;50;0 PM Challenge 2008 37
- 38. Gap-49 (Ranked Overall Priority 12) Gap Statement: • The Projects have difficulty maintaining effective planning and control during the end-of-year closeout period. Gap Elaboration: • SAP is unavailable for a long period of time during end of year closeout/startup. • When the system comes back on line after the end-of-year closeout the data in the system is often not stable until late December or early January. Also, after start-up, a surge of effort is necessary to enter the backlog of manually processed transactions. • Re-bills from pool related charges can cause unexpected changed in a Project's actual costs. The changes sometimes occur several weeks after the Project thought the books were closed - there are no reports available to track the re- billing activity during the shut down period. • The Procurement system shuts down (stops accepting non-emergency PRs) several weeks prior to the End-of-Year SAP/BW shutdown - This, in conjunction with the efforts to recall and reallocate uncommitted funds, makes it very difficult for a Project to maintain stable operations during the last and first quarter of each FY. • There is an issue with changing WBS numbers each new FY - apparently some types of funds, particularly grant and reimbursable funds, may be assigned new WBS numbers which significantly complicates and sometimes prevents the year-to-year traceability of the funds. • Pooling un-obligated funds from previous FYs into a current year WBS makes it difficult to track and manage total project costs and reconcile multi year project expenditures. Team Observations: • The Projects indicated that the length of time required for year-end closeout seems excessive, especially compared to what is heard regarding industry norms. The Projects indicated they are often "working blind" for more than a month and have serious concerns about effectively maintaining and reporting project control during this period. • This is fundamentally a policy issue but from a practical perspective it is more a process and resource issue. (lack of resources to more quickly process the volume of end of year transactions). • This gap increases the use of manual processes and temporary workarounds, causes a surge in the demands placed on support organizations, and prevents the Projects from pulling accurate and timely reports for more than a month. Relative Gap Complexity Factor: • 28-18;29;35;18 PM Challenge 2008 38
- 39. Concept of Operations Document IEEE Standard 1362-1998 defines the content of ConOps as: 1. Scope 5. Concepts for the proposed system a) Identification a) Background, objectives and scope b) Document Overview b) Operational policies and constraints c) System Overview c) Description of current system 2. Referenced Documents d) Modes of operation e) User classes and other involved personnel 3. Current system f) Support Environment a) Background, objectives and scope b) Operational policies and constraints 6. Operational Scenarios c) Description of current system 7. Summary of impacts d) Modes of operation a) Operational Impacts e) User classes and other involved personnel b) Organizational Impacts f) Support Environment c) Impacts during development 4. Justification for and nature of changes 8. Analysis of the proposed system a) Justification of changes 9. Notes b) Description of desired changes c) Priorities among changes d) Changes considered but not included PM Challenge 2008 39