Building a Credible Performance Measurement Baseline (PM Journal)

PM World Journal Building a Credible Performance Measurement Baseline
Vol. IV, Issue IX – September 2015 College of Performance Management
www.pmworldjournal.net Series Article by Alleman, Coonce, Price
© 2015 Glen B. Alleman, Thomas J. Coonce, Rick A. Price www.pmworldlibrary.net Page 1 of 24
Building a Credible Performance Measurement
Baseline†
Establishing a credible Performance Measurement Baseline, with a risk
adjusted Integrated Master Plan and Integrated Master Schedule, starts with
the WBS and connects Technical Measures of progress to Earned Value
By Glen B. Alleman, Thomas J. Coonce, and Rick A. Price
INTRODUCTION
EIA-748-C asks us to “objectively assess accomplishments at the work performance level.” As
well §3.8 of 748-C tells us “Earned Value is a direct measurement of the quantity of work
accomplished. The quality and technical content of work performed is controlled by other
processes.” [6]
Without connecting the technical and quality measures to Earned Value, CPI and SPI provide
little in the way of assuring the delivered products will actually perform as needed. To provide
visibility to integrated cost, schedule, and technical performance, we need more than CPI and
SPI. We need measures of the increasing technical maturity of the project’s system elements in
units of measure meaningful to the decision makers. Those units include Effectiveness,
Performance, all the …”ilities”, and risk reduction.‡
PROCESSES TO INCREASE THE PROBABILITY OF PROGRAM SUCCESS
With the management processes in Table 1, the elements in Figure 1 are the basis of a
credible Performance Measurement Baseline (PMB) needed to increase the probability of
program success using these elements.
Table 1 – Steps to building a risk adjusted Performance Measurement Baseline with Management Reserve.
Step Outcome
Define
WBS
 Using SOW, SOO, ConOps, and other program documents, develop WBS for system elements
and work processes that produce the program outcomes.
 Develop WBS Dictionary describing the criteria for the successful delivery of these outcomes.
Build
IMP
 Develop Integrated Master Plan, showing how each system element in the WBS will move
through the maturation process at each Program Event.
 Define Measures of Effectiveness (MOE) at each Accomplishment (SA).
 Define Measures of Performance (MOP) at each Criteria (AC).
Identify
Reducible
Risks
 For each key system element in the WBS, identify reducible risks, probability of occurrence,
mitigation plan, and residual risk in the Risk Register. [5][4]
 Risk mitigation activities will be placed in the Integrated Master Schedule (IMS). [20]
 For risks without mitigation plans, place budget for risk in Management Reserve (MR) to be
used to handle risk when it becomes an Issue.
Build
IMS
 Arrange Work Packages and Tasks in a logical network of increasing maturity of system
elements. [11]
 Define exit criteria for each Work Package to assess planned Physical Percent Complete to
inform BCWP with TPM, MOP, MOE, and Risk Reduction assessments.
†
This article was contributed by the College of Performance Management (CPM), an international association for
those engaged in earned value management, project cost and schedule control, program management and
program/project performance management. For information about CPM, visit their website at
https://www.mycpm.org/
‡ The term …”ilities” refers to terms such as maintainability, reliability, serviceability, operability, testability, etc.
[Error! Reference source not found.]

Identify
Irreducible
Risks
 For irreducible risks in the IMS, use Reference Classes for Monte Carlo Simulation anchored
with Most Likely duration to calculate needed schedule margin.
 Assign schedule margin to protect key system elements, per DI-MGMT-81861 guidance.
Baseline
PMB
 Using risk adjusted IMS containing reducible risk and explicit schedule margin tasks, establish
the Performance Measurement Baseline (PMB) and calculate the Management Reserve (MR).
 Management Reserve is used for unmitigated risks that remain in the risk register.
THE SITUATION
We’re working ACAT1/ACAT2 programs for Department of Defense using JCIDS (Joint
Capabilities and Development Systems) the governance guidance defined in DODI 5000.02.
The JCIDS Capabilities Based Planning paradigm tells us to define what Done looks like in units
of Effectiveness and Performance, so we need to start by measuring progress toward delivered
Capabilities. Requirements elicitation is part of the process, but programs shouldn’t start with
requirements, but with an assessment of the Capabilities Gaps. While this approach may seem
overly formal, the notion of defining what capabilities are needed for success is the basis of
defining what Done looks like. We’ll use this definition, so we’ll recognize it when it arrives.
With the definition of Done, we can define the processes for incrementally assessing our system
elements along the path to Done. Earned Value Management is one tool to perform these
measurements of progress to plan. But like it says in EIA-748-C, Earned Value – left to itself – is
a measure of quantity. We need measures of quality and other …ilities that describe the
effectiveness and performance of the product to inform our Earned Value measures of the
Estimate to Complete (ETC) and Estimate at Completion (EAC).
We need to integrate other measures with the Earned Value assessment process. We can start
this by recognizing that the Budgeted Cost of Work Performed (BCWP) is a measure of the
efficacy of our dollar.
Earned Value is actually the measure of “earned budget.” Did we “earn” our planned budget?
Did we get our money’s worth? We only know the answer if we measure “Value” in units other
than money. This can be the effectiveness of the solution, the performance of the solution, the
technical performance of the products, or the fulfillment of the mission. The maintainability,
reliability, serviceability, and other …ilities are assessments of Value earned, not described by
CPI and SPI from the simple calculation of BCWP.
THE IMBALANCE
Without the connection between the technical plan and programmatic plan – cost and schedule
– the program performance assessment cannot be credible. [3] Of course BCWP represents the
Earned Value, but the Gold Card and other guidance don’t explicitly state how to calculate this
number. BCWP is a variable without directions for its creation. BCWS and ACWP have clear
direction on how to assign values to them, but not BCWP.
The common advice is to determine percent done and then multiply that with BCWS to produce
BCWP. But what is the percent done of what outcome? What units of measure are used to
calculate percent complete?
It’s this gap between just assigning a value to BCWP through Earned Value Techniques (EVT)
and informing BCWP with tangible evidence of progress to plan, in units of measure meaningful
to the decision makers, that is the goal of this paper.

RESTORING THE BALANCE
Balance is restored by starting with the target values for Effectiveness, Performance, …illities,
and level of acceptable Risk at specific times in the program – at Program Events – measure
the actual Effectiveness, Technical Performance, …illities, and Risk, to produce a variance, to
determine the physical percent complete of any system elements in the WBS.
Using the steps in Table 1 and elements of Figure 1 the connections between each element
assure the Technical Plan and the Programmatic Plan are integrated with units of performance
needed to inform BCWP.
Figure 1 – Programmatic elements needed to increase the probability of program success. Connecting the dots
between Measures of Effectiveness, Measures of Performance, and Technical Performance Measures needed to
inform BCWP that provide leading indicators of performance needed to take corrective actions to Keep The Program
GREEN.
The remainder of this paper describes the steps needed to establish a credible PMB containing
event-based risk reduction activities and schedule margin for irreducible risks. Both are needed
to assure the necessary probability of success for cost, schedule, and technical performance.
These steps include:
 Identifying the reducible risks and their mitigation plans and placing these in the IMS.
 Identifying the Measures of Effectiveness, Measures of Performance, Technical
Performance Measures, and Key Performance Parameters and assigning them to IMP and
IMS elements.

 Baselining the IMS and developing a Monte Carlo Simulation model of the Irreducible Risks
with the needed schedule and cost margins to protect delivery dates and baselined budget.
THE SOLUTION
Creating a credible PMB and successfully executing the program, starts by connecting the dots
between the processes in Table 1, and the elements in Figure 1. This ensures the program’s
Technical Plan and the Programmatic Plan are integrated from day one. Making these
connections starts with Systems Engineering – where the MOEs, MOPs, TPMs, the …ilities,
and Risk are identified. These target values are flowed onto the IMP and IMS and used to
create work activities for reducible risks, and appropriate schedule margins identified through
Monte Carlo Simulations needed for margins to address the irreducible risks. There is existing
guidance in place, starting at the Government Program Management Office, down to the
contractor level to guide the development of these connections.
But to restate it again, it can’t be any clearer than this …
WORK BREAKDOWN STRUCTURE IS PARAMOUNT
The path to creating a credible PMB starts with the Work Breakdown Structure. It is in the WBS
that the system elements are defined, as well as the processes that produce the system
elements, and the risk to the performance of these system elements.
MIL-STD-881-C Guidance [6]
The Work Breakdown Structure, described in MIL-STD-881-C, has several purposes that
support the development of our credible PMB. [13]
 DoD Directive 5000.01 ― The Defense Acquisition System ― requires a disciplined
approach in establishing program goals over its life cycle with streamlined and effective
management that ― is accountable for credible cost, schedule, and performance reporting.
o The WBS is a critical tool in ensuring all portions of the program are covered. This all-in
requirement assures all system elements are identified in the WBS.
o The WBS facilitates collaboration between the Integrated Product Teams by connecting
performance, cost, schedule, and risk information in the IMP, IMS, and Risk Register.
o The WBS facilitates the technical rigor and integrated development, test, and evaluation
throughout the acquisition process.
o The WBS is the first location to define the risks to achieving the above items in this list.
 DoD Instruction 5000.02 ― Operation of the Defense Acquisition System ― further outlines
the required framework and provides impetus for use of a WBS.

o The evolution of the system through incremental development further drives the
requirement to breakdown the system in a structure that clarifies which capabilities will be
satisfied in a specific increment of the system development.
o The instruction sets the requirements for Integrated Master Schedules (IMS), Earned
Value Management (EVM) and other statutory, regulatory, and contract reporting
information and milestone requirements in which the WBS is a critical element.
 The final dot to be connected is the critical link between the WBS, IMP, and IMS with the
Systems Engineering Plan (SEP) on the government side and the Systems Engineering
Management Plan (SEMP) on the contractor side. These are where MOEs, MOPs, KPPs,
Risks, and TPMs start and are connected with the assessment of Physical Percent
Complete.
Risk Management Starts At The WBS
Just like DODI 5000.01 tells us to start with the WBS for risk identification, the Defense
Acquisition Guide also states:
Risk is a measure of future uncertainties in achieving program performance goals
and objectives within defined cost, schedule, and performance constraints. Risk
can be associated with all aspects of a program (e.g., threat environment,
hardware, software, human interface, technology maturity, supplier capability,
design maturation, performance against plan,) as these aspects relate across the
Work Breakdown Structure and Integrated Master Schedule.
With the guidance shown above, Figure 1 shows WBS is the first place the program encounters
risk management along with their identification and the description of their outcomes, in the
WBS Dictionary.
Each key system element in the WBS must be assessed for technical or programmatic risk that
will impede its delivery on time, on cost, and on performance. The reduction of this risk,
following its planned risk reduction levels, will be used to inform Physical Percent complete and
the resulting BCWP. If risk is not being reduced as planned, the program’s Earned Value
measures may not properly represent the technical progress.
THE INTEGRATED MASTER PLAN (IMP)
The IMP is an event-based plan consisting of a sequence of program events, with each event
being supported by specific accomplishments, and each accomplishment associated with
specific criteria to be satisfied for its completion. These IMP elements are defined as: [1], [2], [3],
[10]
 Event: a key decision point or program assessment point that occurs at the culmination of
significant program activities.
 Accomplishment: is the desired result(s) prior to or at completion of an event that indicates
a level of the program’s progress.
 Criteria: provides definitive evidence that a specific accomplishment has been completed.
Entry criteria reflect what must be done to be ready to initiate a review, demonstration, or
test. Exit criteria reflect what must be done to clearly ascertain the event has been
successfully completed.

Structure of the Integrated Master Plan (IMP)
Figure 2 shows the structure of the Integrated Master Plan (IMP), with Program Events (PE),
Significant Accomplishments (SA), and Accomplishment Criteria (AC). This structure assesses
the increasing maturity of each key system element, to assure the end item system elements
meet the planned Effectiveness and Performance needs that fulfill the capabilities for the
mission or business goals. [14][15]
With this structure, Earned Value Management measures can now be connected to the Work
Packages defined in the Integrated Master Schedule (IMS) to assess technical performance in
ways not available with CPI and SPI only. The Program Manager now has leading indicators of
the program success through the MOEs defined by the SA’s and MOPs defined by the AC’s,
each assessed for compliance with plan at the Program Event. In Figure 2 the IMS should
encompass the IMP; work packages and tasks are the “new” element the IMS adds to the IMP
along with the element of time.
Figure 2 – The Integrated Master Plan defines the increasing maturity of each key system element assessed at
Program Events. Significant Accomplishments are entry criteria for this maturity assessment. Accomplishment
Criteria are measures substantiating the maturity level of the work products produced by Work Packages in the IMS.
Measures of “Progress To Plan” Start At The IMP
The Program Events, Significant Accomplishments, and Accomplishment Criteria form the
elements of the IMP. Measures of Effectiveness (MOE) for the program are derived from the
JCIDS process and are reflected in the IMP’s Significant Accomplishments (SA). The Measures
of Performance (MOP) are derived from the MOEs and are reflected in the IMP’s
Accomplishment Criteria (AC).
These measures assess the physical percent complete of the system elements used to inform
Earned Value (BCWP) for reporting programmatic performance. With this Physical Percent
Complete measure, the EVM indices then reflect the cost, schedule, and technical performance
of the program using the Systems Engineering measures:
 Measures of Effectiveness (MOE) – are operational measures of success closely related
to the achievements of the mission or operational objectives evaluated in the operational
environment, under a specific set of conditions.
 Measures of Performance (MOP) – characterize physical or functional attributes relating to
the system operation, measured or estimated under specific conditions.

 Key Performance Parameters (KPP) – represent significant capabilities and characteristics
in that failure to meet them can be cause for reevaluation, reassessing, or termination of the
program.
 Technical Performance Measures (TPM) – are attributes that determine how well a
system or system element is satisfying or expected to satisfy a technical requirement or
goal. [22]
Continuous verification of actual performance versus anticipated performance of a selected
technical parameter confirms progress and identifies variances that might jeopardize meeting a
higher-level end product requirement. Assessed values falling outside established tolerances
indicate the need for management attention and corrective action.
A well thought out TPM process provides early warning of technical problems, supports
assessments of the extent to which operational requirements will be met, and assesses the
impacts of proposed changes made to lower-level elements in the system hierarchy on system
performance. §
With this estimate, the programmatic performance of the program can be
informed in a way not available with CPI and SPI alone.
Technical Performance Measurement (TPM), as defined in industry standard, EIA-632, involves
estimating the future value of a key technical performance parameter of the higher-level end
product under development, based on current assessments of products lower in the system
structure.
Continuous verification of actual versus anticipated achievement for selected technical
parameters confirms progress and identifies variances that might jeopardize meeting a higher-
level end product requirement. Assessed values falling outside established tolerances indicate
the need for management attention and corrective action.
Risk Management Continues From The WBS To The IMP
Throughout all product definition processes, technical and programmatic risk assessment is
performed. These risks are placed in the Risk Register shown in Figure 3 with their
uncertainties. Uncertainty comes in two forms: **
Table 2 – Irreducible and Reducible uncertainty both create risk to the program for cost, schedule, and technical
performance. As well as reducible and irreducible there are Unknown Unknowns that are mitigated in the DOD by
replanning the program. [19]
Reducible Uncertainty Irreducible Uncertainty
The probability that something will happen to
impact cost, schedule, and technical
performance of the system elements.
The natural variation of the program’s
activities work duration and cost. The
variance and its impacts are protected by
schedule and cost margin.
 Reducible uncertainty can be stated as the
probability of an event, and we can do
something about reducing this probability of
occurrence.
 These are (subjective or probabilistic)
uncertainties that are event-based probabilities,
are knowledge-based, and are reducible by
 Irreducible uncertainty is the probability range
of these variances of the stochastic variability
from the natural randomness of the process
and is characterized by a probability distribution
function (PDF) for their range and frequency,
and therefore are irreducible.
 Irreducible uncertainty can be modeled with
§ https://dap.dau.mil/acquipedia/Pages/ArticleDetails.aspx?aid=7c1d9528-4a9e-4c3a-8f9e-6e0ff93b6ccb
** “On Numerical Representation of Aleatory and Epistemic Uncertainty,” Hans Schjær-Jacobsen, Proceedings of 9
th
International Conference of Structural Dynamics, EURODYN 2014.

further gathering of knowledge. Monte Carlo Simulation using Reference
Classes based on past performance.
Capture Reducible Risks Through the WBS Number
With Reducible risks, specific work is performed on baseline to reduce the probability of
occurrence or impact or consequence of the risk. [5] With Irreducible risk, schedule margin is
needed to protect the delivery date of key system elements. A measure of the performance of
the program is the Margin Burn-down Plan shown in Figure 8. If the actual risk reduction does
not follow the risk burn-down plan, this is a leading indicator of future difficulties in the program’s
performance and an indicator of impact on cost and schedule.
Figure 3 – The mitigation activities for Reducible are guided by the Risk Register’s contents and the WBS in the IMS.
Work to reduce risk is planned and executed On Baseline through Work Packages. Risk reduction lowers the
probability and/or impact of occurrence and therefore lowers the probability of impact of the named risk.
STEPS TO BUILDING A CREDIBLE INTEGRATED MASTER PLAN
The Integrated Master Plan (IMP) is the strategy for the successful delivery of outcomes of the
program. Strategies are hypotheses that need to be tested. The IMP’s Events,
Accomplishments, and Criteria are the testing points for the hypothesis that the program is
proceeding as planned, both technically as planned and programmatically as planned.
Development of the IMP is a step-wise process, starting with the Program Events.
Identify Program Events
Program Events are maturity assessment points. They define the needed levels of maturity for
the products and services before proceeding to the next assessment point. The entry criteria for
each Event define the units of measure for the successful completion of the Event.
The Program Events confirm the end-to-end description of the increasing maturity of the
program’s system elements. As well, they establish the target dates for each Event in the RFP
or contract.
Finally they socialize the language of speaking in “Increasing Maturity at Events” rather than the
passage of time and consumption of budget.

Identify Significant Accomplishments
The Significant Accomplishments (SA) are the “road map” to the increasing maturity of the
program. They are the “Value Stream Map” resulting from the flow of SA’s describing how the
products or services move through the maturation process while reducing risk. This Significant
Accomplishment map is our path to “done”.
Identify Accomplishment Criteria
The definition of “done” emerges in the form of system elements rather than measures of cost
and passage of time. These system elements come from Work Packages, whose outcomes can
be assessed against Technical Performance Measures (TPM) to assure compliance with the
Measure of Performance (MOP). At each Program Event, the increasing maturity of the system
elements is defined through the Measures of Effectiveness (MoE) and Measures of
Performance (MoP).
The vertical connectivity of ACs and SAs to PEs, described in Figure 4, establishes the
framework for the horizontal traceability of Work Packages in the Integrated Master Schedule to
the Integrated Master Plan.
Figure 4 – Vertical and horizontal traceability between the Integrated Master Plan (IMP) and the Integrated Master
Schedule (IMS). The IMP describes how the increasing maturity of the system elements will be assessed with
Measures of Effectiveness (MOE) and Measures of Performance (MOP). The IMS describes the work needed to
produce that increasing maturity through the assessment of Technical Performance Measures (TPM).

INTEGRATED MASTER SCHEDULE (IMS)
The Integrated Master Schedule (IMS) embodies all work effort needed to produce a product or
accomplish specific goals that have an associated cost. The work effort is represented by well-
defined tasks that program participants execute (expending cost) to generate the desired
products. [12]
Tasks are organized in the proper sequence to facilitate efficient execution that enables the
program to know what work must be accomplished to achieve their objectives. Task sequencing
occurs by recognizing the dependencies among all the tasks. Then, by identifying the expected
time to perform each task, the program can project an expected completion time. [3] A well-
structured and managed IMS can now serve as the program’s GPS, providing timely information
that enables program management to making informed decisions about the paths to successful
delivery of all the system elements.
The IMP Events, Accomplishments, and Criteria are duplicated in the IMS. Detailed tasks are
added to depict the steps required to satisfy performance criterion. The IMS should be directly
traceable to the IMP and should include all the elements associated with development,
production or modification, and delivery of the total product and program high level plan.
Durations are entered for each discrete task in a work package, planning package, and lower
level task or activity, along with predecessor and successor relationships, and any constraints
that control the start or finish of each work package and planning package.
The result is a fully networked “bottoms up” schedule that supports critical path analysis.
Although durations are assigned at the work package and planning package level, these
durations will roll up to show the overall duration of any event, accomplishment, or criteria.
When Level of Effort (LOE) work packages, tasks, or activities are included in the IMS, they
must be clearly identified as such. Level of Effort shall never drive the critical path.
STEPS TO BUILDING THE INTEGRATED MASTER
SCHEDULE (IMS)
Starting with the WBS, a credible IMS that produces the system elements must address
reducible risk through risk mitigation activities. This IMS must also address the irreducible
uncertainties and those discrete risks that remain in the risk register that could not be mitigated
through work activities. These uncertainties are categorized as known–unknowns because they
are known, but it is not known for certainty that they will occur.
Programs can protect against these uncertainties by setting cost and schedule margin for the
irreducible uncertainties, and management reserves for the reducible risks that were not
mitigated. Cost and schedule margins are included in the Performance Measurement Baseline
and Management Reserve is established for the latent reducible risks and is held above the
PMBs but is still part of the Contract Budget Base (CBB).

REDUCIBLE RISK MANAGEMENT PROCESSES ARE IN THE IMS
All reducible risks identified in the Risk Register need to be assessed for reducibility in the IMS.
If the risk – the uncertainty that is event based – can be reduced, then that work is assigned to
work packages and activities in the IMS and placed on baseline. These risk buy down activities
are managed just like ordinary work, funded by the CBB, and measured for performance just
like any other work.
The risk reduction mitigation activities are planned to achieve a specific level of risk reduction at
a specific time in the IMS. Meeting the planned risk reduction level at the planned time is a
measure of performance of the risk retirement activities.
IRREDUCIBLE RISK MANAGEMENT PROCESSES ARE IN MARGIN
With the reducible risks from the Risk Register handled with risk retirement activities in the IMS,
the irreducible risks now need to be identified and handled in the IMS. Since the irreducible risks
are actually irreducible, only margin can be used to protect the system elements from this
naturally occurring variance. No actual work can be done to do this.
Monte Carlo Simulation of the IMS is the primary tool for assessing how much margin is needed
for each irreducible risk type.
DEVELOPMENT OF SCHEDULE MARGIN
Schedule margin is a buffer of time used to increase the probability that the program will meet
the targeted delivery date. Schedule Margin is calculated by starting with a Probability
Distribution Function (PDF) of the naturally occurring variance of the work duration of the Most
Likely value of that duration. With the Most Likely Value and the PDF for the probability of other
values, the durations of all work in the Integrated Master Schedule and the probabilistic
completion times of this work can be modeled with a Monte Carlo Simulation tool. [9]
This modeling starts with the deterministic schedule, which includes work for the Reducible
Risks. The schedule margin is the difference in the initial deterministic date and a longer
duration and associated date with a higher confidence level generated through the Monte Carlo
Simulation.
Figure 5 illustrates this concept. In this example, the deterministic delivery date is 8/4/08 with
the contract delivery date of 8/31/08. Using the historical variability of the task durations resulted
in a 5% confidence level of completing on or before 8/4/14 shown in the deterministic schedule.
This says that with the deterministic schedule – no accounting for the natural variability in work
durations, the probability of completing on or before 8/4/08 is 5%. ††
To increase the confidence level of meeting the contractual date, the contractor needs to
increase the probability of completing on or before to 80%. The difference in duration between
the deterministic schedule’s completion date of 8/4/08 and the 80% confidence of completing on
or before 8/14/08 is 10 calendar days, still earlier than the contractual date of 8/31/08.
††
These dates are taken from the Wright Brothers development efforts of the Heavier-Than-Air contract issued by the U.S.
Army at the turn of the twentieth century. This example will be used later in this paper to show how to connect all the dots to
produce a credible PMB.

Figure 5 – Monte Carlo Simulation used to determine schedule margin developed from the Deterministic IMS,
showing a completion date of 8/4/08. The 80% confidence of completing on or before the needed 8/14/08. The
Schedule Margin is then added in front of key system elements to protect their delivery dates to meet the contractual
need date of 8/31/08
Assign Schedule Margin to Protect Key System Elements
Using schedule margin to protect against schedule risk – created by the natural uncertainties in
the work durations enables on-time contractual end item deliveries.
There are two schools of thought on how schedule margin should be managed.
 Place all schedule margin at the end of the program or system elements.
 Distribute margin at strategic points along critical paths where there are known schedule
risks.
Placing all the margin at the end appears effective in short duration or production efforts where
the primary schedule risk is not driven by technical complexity. The same objective can be
achieved when a disciplined process is followed for control and consumption of distributed
margin. Paramount to this approach is accelerating downstream efforts when margin is NOT
consumed.
Most schedule risk in a development program is encountered when program elements are
integrated and tested. Even when margin is distributed, margin is often kept at the end of the
schedule to help protect against risk when all paths come together during final integration and
test. This approach enables on-time end item delivery with realistic cost and schedule baselines
that provide accurate forecasts and decisions based on current status, remaining efforts and
related schedule risks.

There are several valid reasons for distributing schedule margin earlier in the IMS including:
 Protecting use of critical shared resources so that being a few weeks late doesn’t turn into a
several month schedule impact. An example in space programs is use of a thermal vacuum
chamber shared across multiple programs at critical times in their schedules. If a program is
unable to enter the chamber at their scheduled time the ultimate delay may be an
exponential factor of their original delay.
 Protecting highly visible milestones that are difficult and undesirable to change like a Critical
Design Review (CDR).
 Establishing realistic performance baselines accounting for schedule risk at key points
provides more valid data to make program decisions.
 Establishing realistic baselines that are cost effective.
 Placing margin where we believe it will be needed and consumed provides the most realistic
schedule baseline possible for succeeding efforts and enables more accurate resource
planning for prime contract, customer, and suppliers.
Key Insertion Points for Schedule Margin
Schedule Margin is not the same as Schedule Slack or Schedule Float as stated in the GAO
Schedule Assessment Guide. ‡‡
Margin is preplanned and consumed for known schedule
irreducible uncertainty and float is the calculated difference between early and late dates. In
many ways margin is much like management reserve and float is similar to underruns/overruns.
 Schedule margin is placed where there is known irreducible schedule risk. It is never
consumed because of poor schedule performance. In this case, Schedule Margin is
managed like Management Reserve.
 Schedule margin is not budgeted – it does not have an assigned BCWS. If the risk the
margin is protecting comes true, new tasks need to be identified and budgeted. If the risk is
not realized, the schedule margin is zeroed out and the succeeding tasks accelerated –
moved to the left.
 Allocating margin for known risks at key points prevents this margin from being used to
cover poor schedule performance. This forces an immediate recovery action to stay on
schedule instead of degrading margin as if it was schedule float.
 Inclusion of margin – either distributed in front of key system elements or at the end of
contractual system elements – does not affect contractual period of performance. The
period of performance is defined by the contract. The Schedule Margin activities in the
deterministic schedule are represented in the PMB, using the task label defined in DI-
MGMT-81861 §3.7.2.4 – SCHEDULE MARGIN.
 Inclusion of schedule margin for known schedule risk provides a realistic baseline and
accurate resource planning (including the customer). If not consumed the effort is accurately
represented as “ahead of schedule”.
‡‡ GAO Schedule Assessment Guide, page 113. “Schedule margin is calculated by performing a schedule risk analysis and
comparing the schedule date with that of the simulation result at the desired level of uncertainty.”

Development of Cost Margin or Contingency Reserve
The cost margin is the amount of cost reserve needed to address irreducible cost variances of
the program’s work efforts and to improve the probability of meeting the target cost – the
contract cost. Cost margin is calculated in the same manner as schedule margin. The contractor
develops a probability distribution of the final cost based on the natural variances contained in
the historical databases. The confidence level of meeting the target contracted cost in the IMS is
noted. If the confidence level is too low, it has been suggested that cost margin be added to the
baseline, in the same manner as schedule margin, to bring the baseline up to a desired cost
confidence level.
However, at this time, there is no generally agreed to mechanism to do so. If the Confidence
Level is unacceptably low, the contractor must redo the IMS and try to reduce costs for time
dependent and time independent costs to raise the cost confidence to an acceptable level. Re-
planning would result in re-calculating the schedule margin. If after re-planning, the cost
confidence level is still unacceptably low, the contractor should report the Confidence Level to
the customer at the time of the Integrated Baseline Review (IBR). If the customer agrees the
Confidence Level is too low, resources could be added and the contractor would update the
IMS, or the contract could be de-scoped to improve the Confidence Level. Alternatively, the
customer may hold the cost margin as a contingency for overruns.
Development of Management Reserves
Management Reserves are financial resources needed to address the reducible risks that were
not mitigated. The optimal way to develop management reserve is to re-run the Monte Carlo
Simulation tool with only these risks against the resource-loaded IMS. The difference between
the deterministic cost estimate and the point on the cost distribution curve for the desired cost
confidence level is the Management Reserve.
EXECUTING THE BASELINED PROGRAM
To inform Earned Value (BCWP), we need information about the performance of the program
beyond just cost and schedule performance. We need information about how the program is
progressing toward delivering the needed capabilities. How are these capabilities being fulfilled
as the program progresses? What risks are bought down at the planned time to increase the
probability of success? How are the Technical Performance Measures being assessed
compared to the planned measures needed to deliver the needed capabilities?
CONNECTING THE DOTS TO EXECUTE THE IMS
Some reminders from DOD guidance are needed before starting to connect the dots.
Connecting cost, schedule, and technical performance is the basis of program success. Without
these connections the Earned Value measures cannot be informed by the technical
performance. Also, the Earned Value measures are simply a reflection of the passage of time
and consumption of resources, with no connection to the planned technical maturity of the
program’s system elements. Three measures of program performance are used to inform
Earned Value. [8]
 Programmatic and Technical Performance Measures, Figure 7
 Risk retirement burn down, Figure 8

 Schedule margin utilization, Figure 9
With the MOEs, MOPs, KPPs, Risks and their reduction plans, and the TPMs assigned to the
IMP and IMS, we have all the pieces to connect the dots.
Figure 6 – Starting with the IMP and the measures of progress to plan at the Program Event, Significant
Accomplishment, and Accomplishment Criteria levels, the IMS is constructed and work performed in Work Packages
can be assessed as Physical Percent Complete in units of measure meaningful to the decision makers.
Programmatic And Technical Measures Inform Earned Value
Our ultimate purpose for this paper was to lay the groundwork for developing a credible
Performance Measurement Baseline, and to use that baseline to increase the probability of
program success. The first question that comes up is – what is credible?

Figure 7 – Technical Performance Measures, Measures of Performance, Measures of Effectiveness, and Key
Performance Parameters assure the delivered products are not only Fit for Purpose (Technically compliant) but also
Fit for Use (Meet the needed Mission Capabilities).
Risk Retirement as a Measure of Program Performance
BCWP can be informed with the performance of the planned retirement processes on the
planned day for the planned cost, to the planned risk level. The work activities of risk reduction
are no different than work activities needed to produce system elements. They are on baseline,
produce outcomes and have technical assessment of their progress to plan.
Figure 8 – The planned risk reduction in the Center Of Gravity going out of bounds for a flight vehicle as a function of
time in the Integrated Master Schedule. Baseline work is performed to reduce this risk and therefore reduce the
impact of the risk. Making the reductions to the planned level on the planned day informs the BCWP of the Earned
Value numbers. Reducing the risk late must reduce the BCWP for activities related to the risk.

Schedule Margin Burn-Down is a Measure of Program Performance
Schedule Margin is a duration buffer prior to a system element delivery date or any contract
event. As a project progresses, the length of the schedule margin task is re-evaluated and
adjusted as needed to protect the system elements from risks of delay that result from natural
variances in work effort durations. The Schedule Margin Burn Down shown in Figure 7, displays
the use of schedule margin over time. Schedule Margin compensates for work activity duration
uncertainties. As a program progresses, the total schedule margin is re-evaluated and adjusted
to protect the system elements from risks that arise from natural variances in duration.
When Schedule Margin is used faster than planned, this indicates cost and schedule progress is
at risk, since uncertainty has not been controlled as planned. [21]
The identified schedule margin can then be used to inform program performance by comparing
planned schedule margin with actual schedule margin.
Figure 9 – Schedule Margin Burn-Down Plan depicts the planned schedule margin versus the actual schedule
margin consumption that protects the date of a key system element. Status against this schedule margin plan is a
leading indicator of the risk to the delivery date and the success of the program. [8]

NOW A PRACTICAL EXAMPLE
Let’s start with a practical example we can all
recognize. Immediately after the Wright Brothers
made their first powered flights in 1903, they
begin to develop their experimental aircraft into a
marketable product.
By 1905 they had the basis of a "practical flying
machine." Other experimenters learned of their
work and begin to build on their success.
By 1906, others pilots were making tentative hops
in uncontrollable aircraft. By 1909, after watching
the Wrights' flying demonstrations, they grasped
the brilliance and necessity of three-axis
aerodynamic control. The performance of their
aircraft quickly caught up to and then surpassed
Wright Flyers. The capabilities of and the uses for
aircraft expanded as designers and pilots
introduced float planes, flying boats, passenger
aircraft, observation platforms fitted with radios
and wireless telegraphs, fighters, and bombers. [18]
As World War I approached, aircraft became an essential part of war and peace. In 1907, the
US Army renewed its interest in the Wright Brothers. The Board of Ordnance and Fortification
and the U.S. Signal Corps announced an advertisement for bids to construct an airplane. [16]
However, the design and performance specifications were such that the Wrights were the only
viable bidder. A price of $25,000 was set for the brothers’ airplane, if they could meet the
performance criteria in actual flight trials.
These flight trials were scheduled for late summer 1908 at Fort Myer, Virginia, a military post
outside Washington, D.C. With the commitments in Europe, the brothers had to separate for the
first time. With Wilbur off to France, Orville did the flying for the Army.
From the source document for the U.S. Signal Corps Agreement and Specifications for a
Heavier-Than-Air Flying Machine we have Measures of Effectiveness, Measures of
Performance, Technical Performance Measures, and Key Performance Parameters.
Figure 10 – the Wright Flyer on its flight trials at Fort Myer,
Virginia, 1908.

Table 3 – Obtained from historical documents in aviation archives, the Wright brothers needed to address the
following Measures of Effectiveness, Measures of Performance, Technical Performance Measures, and Key
Performance Parameters in their program.
Historical Document
Program
Performance
Measure
The flying machine must be designed to carry two people having a combined weight
of no more than 350 pounds,
MOP
Also sufficient fuel for a flight of 125 miles MOP
The flying machine should be designed to have a speed of at least 40 miles per hour
in still air for at least 125 miles
MOP
The flying machine should be designed so that it may be quickly and easily
assembled and taken apart and packed into an Army wagon.
KPP
It should be capable of being assembled and put in operating condition within one
hour
KPP
Before acceptance, a trial endurance flight will be required of at least one hour
during which time,
MOP
The flying machine must remain continuously in the air without landing. MOE
It shall return to the starting point and land without any damage that would
prevent it immediately starting upon another flight.
MOE
During this flight of one hour, it must be steered in all directions without difficulty
and at all times under perfect control and equilibrium.
MOE
It should be sufficiently simple in its construction and operation to permit an
intelligent man to become proficient in its use within a reasonable length of time.
KPP
The Wright Brothers identified key functions in Table 3 – Obtained from historical documents in
aviation archives, the Wright brothers needed to address the following Measures of
Effectiveness, Measures of Performance, Technical Performance Measures, and Key
Performance Parameters in their program. to develop a heavier-than-air flying machine. They,
and many before them, observed that birds have physical components that enable flight. They
needed to develop their own physics components that would enable human flight. [17]
On the train ride from Kitty Hawk back to Dayton Ohio, the Wright brothers decided that, for their
aircraft to be a success, their flying machine had to take off in a wide range of weather
conditions, navigate to a predetermined location, and "land without wrecking."
Figures 11 and 12 show initial estimated weight against reported EVM data. The actual weight
reduction could have been used to drive or inform the cost and schedule status. As the planned
weight burn down plan occurred, the actual weight reduction was measured. This Technical
Performance Measure can be used to inform Earned Value to assess the program performance.

Figure 11 – The plan to reach an 800-pound aircraft. The actual weight for the first months shows they were above
their upper threshold.
Figure 12 – The Earned Value reported by program controls, showing BCWP not informed by Technical progress.

CONCLUSION
The roadmap to creating a Performance Measurement Baseline (PMB) driven by the system
attributes provides meaningful information on program performance to the end user, and how to
objectively measure cost and schedule progress based on technical performance. This process
starts with a risk-tolerant PMB that adjusts major risks and protects the PMB performance
shortfalls from the natural uncertainties that are typically inherent in development efforts.
Calculating and applying schedule margin to accommodate the natural uncertainties and
calculation of Management Reserve to account for the unmitigated risks that remain in the risk
register is the basis of a credible PMB. Next is the inclusion of Technical Performance
Measures, Risk Burn Down, and Schedule Margin Burn Down data used to inform cost and
schedule progress.
Creating a credible PMB and keeping the program green to deliver systems that meet the end
user needs with a high probability of meeting cost and schedule targets is the goal of integrating
the Technical Plan with the Programmatic Plan.
This is done by:
 Having Contractors and Government establish plans and measure integrated technical cost
and schedule progress using the principles of Systems Engineering.
 Using the MOE, MOP, KPP, and TPM’s originating from the SOW, SOO, ConOps, and other
acquisition documents, and assigning these to the IMP and IMS to inform the progress to
plan for the program.
 Creating the risk adjusted Technical Plan that is aligned with the Cost and Schedule Plan
(PMB).
 Selecting appropriate Technical Performance Measures (TPM) that assess technical
progress to plan at the work performance level.
 Establishing the appropriate schedule margin and Management Reserve from the risk
adjusted Technical Plan.
 Ensuring cost and schedule performance is consistent with technical performance.

REFERENCES
1. Integrated Master Plan and Integrated Master Schedule Preparation and Use Guide,
Version 0.9, October 21, 2005, OUSD(AT&L) Defense Systems, Systems Engineering,
Enterprise Development (OUSD(AT&L) DS/SE/ED).
2. AFMC Pamphlet 63-5, Integrated Master Plan and Schedule Guide, 11 November 2004.
3. Air Force Integrated Master Schedule (IMS) Assessment Process, Version 3.0, June 2012.
4. Probabilistic Risk Assessment Procedures Guide for NASA Managers and Practitioners,
NASA/SP-2011-3421, 2nd
Edition, December 2011.
5. NASA Risk Informed Decision Making Handbook, NASA/SP-2010-576, April 2010.
6. EAI-748-C, AMSC 9213, 3 October 2011.
7. Investing Relationships and Sematic Sets amongst System Lifecycle Properties (ilities), MIT
ESD, June 19, 2012,
http://seari.mit.edu/documents/presentations/CESUN12_deWeck_MIT.pdf
8. GAO Cost Estimating and Assessment Guide Best Practices for Developing and Managing
Capital Program Costs, GAO-09-3SP
9. “Schedule Margin Management – The Missing Link,” Rick Price, PM Challenge, 9-10
February 2010.
10. The Integrated Project Management Handbook, Dayton Aerospace, Inc. 8 February 2002,
https://acc.dau.mil/adl/en-
US/19595/file/1047/integrated_project_management_handbook.pdf
11. Air Force Integrated Master Schedule Assessment Process,
http://afacpo.com/AQDocs/AF_IMS_%20Assessment_%20Process_%20V3.0.pdf
12. NASA Schedule Management Handbook, NASA/SP-2010-3403,
http://www.nasa.gov/pdf/420297main_NASA-SP-2010-3403.pdf
13. Defense Acquisition Program Support Methodology, Version 2.0, 9 January 2009.
14. Air Force Nuclear Weapon Center Instruction 63-105, Project Planning, 12 April 2011,
http://static.e-publishing.af.mil/production/1/afnwc/publication/afnwci63-105/afnwci63-
105.pdf
15. Air Force Instruction 63-101/20-101, Integrated Life Cycle Management, 7 March 2013.
16. The Wright Brothers and The Invention of the Aerial Age, Smithsonian National Air and
Space Museum, https://airandspace.si.edu/exhibitions/wright-brothers/online/index.cfm
17. “What is Systems Engineering? A Consensus of Senior Systems Engineers,”
http://www.sie.arizona.edu/sysengr/whatis/whatis.html
18. A History of the Airplane, Wright Brother Aeroplane Co, A Virtual Museum of Pioneer
Aviation, http://www.wright-
brothers.org/History_Wing/History_of_the_Airplane/History_of_the_Airplane_Intro/History_of
_the_Airplane_Intro.htm
19. Managing Project Uncertainty: From Variation to Chaos, Arnoud De Meyer, Christoph H.
Loch, and Michael T. Pich, MIT Sloan Management Review, Winter 2002, pp. 60 – 67.
20. GAO Schedule Assessment Guide, GAO-12-120G, May 2012.
21. Guide to Managing Programs with Predictive Measures, NDIA.
22. Technical Performance Measurement, Earned Value, and Risk Management: An Integrated
Diagnostic Tool for Program Management, Commander N. D. Pisano, SC, USN, Program
Executive Office for Air ASW, Assault, and Special Mission Programs (PEO(A)), Integrated
TPM, Earned Value and Risk Management Tool.

ABOUT THE AUTHORS
Glen B. Alleman
Glen B. Alleman leads the Program Planning and Controls
practice for Niwot Ridge, LLC. In this position, Glen brings his 25
years’ experience in program management, systems engineering, software
development, and general management to bear on problems of performance based
program management. Mr. Alleman’s experience ranges from real time process control
systems to product development management and Program Management in a variety of
firms including Logicon, TRW, CH2M Hill, SM&A, and several consulting firms before
joining Niwot Ridge, LLC. Mr. Alleman’s teaching experience includes university level
courses in mathematics, physics, and computer science.
Thomas J. Coonce
Thomas J. Coonce is an Adjunct Research Staff Member with the
Institute for Defense Analyses (IDA) where he researches Essential
Views of the Integrated Program Management Report data to provide government
stakeholders with a) insightful program status and b) indicators of problem areas. Prior
to joining IDA, Mr. Coonce led NASA’s Cost Analysis Division where he shaped cost
and schedule estimating policies and sponsored cost research work. Mr. Coonce also
served within the Office of the Secretary of Defense’s (OSD) Cost Analysis
Improvement Group (CAIG) where he developed independent estimates of MDAP
programs. While at the OSD CAIG, Mr. Coonce also led an effort to re-engineer the
Contractor and Software Resources Data Reporting System and to make the data
available to users. Prior to joining the OSD CAIG, Mr. Coonce served as a cost
estimator for several consulting firms and the U.S. Government Accountability Office.

Rick A. Price
Rick A. Price has over 34 years of experience with Lockheed
Martin in Aerospace and Defense across spacecraft, launch
vehicle, missile, site activation, and test facility construction
programs. Mr. Price has worked with the U.S. DoD, NASA, commercial, and
international customers. His areas of expertise include program planning & scheduling,
program management, EVM, subcontract management, IMP/IMS development, and
major proposal efforts (proposal development, review teams, and subcontract source
selection evaluations). Currently Mr. Price is a Project Management and Planning
Operations Principal with Lockheed Martin Space Systems Company. Mr. Price’s
current focus involves mentoring, coaching, and teaching program management
fundamentals and techniques across Space Systems programs. Mr. Price has written
numerous articles featured in in-house Lockheed Martin corporate and company
publications. Mr. Price speaks at PMI/CPM practice symposiums and the NASA PM
Challenge.

Building a Credible Performance Measurement Baseline (PM Journal)

More Related Content

What's hot (20)

Similar to Building a Credible Performance Measurement Baseline (PM Journal) (20)

More from Glen Alleman (20)

Recently uploaded (20)

Building a Credible Performance Measurement Baseline (PM Journal)