Simplifying effort estimation based on use case points

Prepared by: AbdulRahman Shaheen

Summerized By Shaheen

Agenda
 Introduction
 Experiment Setup
 Experiment Framework
 Experiment Theses and Results
 Threats to Validity
 Conclusion


Introduction
 Software Estimation is key aspect of
software project management.
 Software Project Outsourcing estimation
problems:
 Underestimation: can lead to delays or poor
quality software.
 Overestimation: might result in bid rejection.
 Possible Solution:
 Simplify the process of Effort Estimation while
maintaining or improving the quality of
estimation.


History
 Function Point Analysis (FPA)
introduced 1979 by Albrecht.
 Mark II FPA in 1991 by Symon
.
 UCP in 1993 by Karner.


Use Cases


Use Case Point Methods
 Assess Complexity of Actors
 aWeight(c): {1, 2, 3}
1. -Simple: API Communication
2. -Averge: Protocol communication
3. -Complex: GUI communication
 aCardinality: number of actors belonging to
the class
 Unadjusted Weight Factor


Use Case Point Methods - cont
 Assess Complexity of Use Cases
 complexity depends on number of transactions
 Given use case u, number of transactions
#trans(u)

 uWeight(c) = 5 for simple, 10 for average, and
15 for complex;
 uCardinality(c) is the number of use-cases
assigned to class c (depends on a described
system).


Technical and Environmental
Factors


 Add Adjustment Factor(Technical +
Environmental)

 TF_weighti is the weight of the ith technical
complexity factor (see Table 1);
 valuei is the predicted degree of inﬂuence
of the ith technical complexity factor on the
project (value between 0 and 5).



• EF_weighti is the weight of the ith environmental
factor (see Table 1);
• valuei is the predicted degree of inﬂuence of the
ith environ mental factor on the project (value
between 0 and 5).


Use Case Point Methods -
cont
 UUCP = UAW + UUCW
 UCP = UUCP * TCF * EF
 Effort Estimations
 Effort = UCP * PF
○ [Karner]
 PF = 20 hours / UCP
○ [Schneider and Winters]
 PF=20h if EF<3
 PF=28h if 3<EF<4
 PF=36h if EF>4


Use Case Point Methods -
cont
 Calibrating UCP with historical data:


Projects Setup
 14 projects
 277 to 3593 man hours
 Projects labeled according to their
development environment
 A – G: industrial projects
 H – N: Pazon Univ. (PUT)
 Origion
 U:budgeted
 S2B: student to business
 I: software development company


Projects Information


Projects UCP


Subsets of Datasets used in the
study


Framework
 UCP calculations:
1. Reviewing use cases and reject:
○ Business use case.
○ Include + extend use cases, if they describe
the same logic with lower abstraction
○ Unimplemented + duplicated.
2. Counting transactions and steps
○ External expert was used for validation of
correctness of step counting.
3. Obtaining TCF and EF
4. Calculating UCP


Framework
 Evaluation of prediction accuracy
1. Ordinary least squares regression (OLS)

○ pi is the project for which effort is estimated;
○ᵝSize is the slope for Size;
○ ᵝ is the constant or intercept (set to 0);
0
○ Size (pi) is the value of the size metric
calculated for the project pi.


Framework
 Evaluation of prediction accuracy
2. Multiple regression

○ pi is the project for which effort is estimated;
○ ᵝ Size is the slope for Size;
○ ᵝ Factor is the slope for Factor;
○ ᵝ is the constant or intercept (set to 0);
0
○ Size (pi) is the value of the size metric calculated
for the project pi.
○ Factor (pi) is the value of the additional factor
included in the regression model (chosen among
the EFs, TCFs, and team size) for the project pi


Framework


Framework
 The evaluation criteria
○ magnitude of relative error (MRE) [inversely]
 Agregated using Mean of MRE (MMRE)
○ mean relative error (Mean RE)
 bias of the estimates
○ Pred (e)=k/n [proportional]
 prediction quality is calculated on a set of n projects,
where k is the number of projects for which
estimation error (MRE) is less than or equal to e
 To test Statistical Significance: two-tailed
Wilcoxon signed-rank test with the
significance level a set to 0.05 was used.


Experiment
 Actors complexity in Use Case Points
 UAW had only minor impact on the accuracy of the
effort estimation based on UCP
 Adjustment factors in Use Case Points
 Problems:
○ Lack of standardized (agreed) scale
○ Not verified weights of factors
 Investigation
○ Overlapping
○ Minor influence on estimation
 Result:
○ 13 to 4 technical complexity factors
○ 8 to 2 environmental factors


Experiment
 Use Case Points – steps vs. transactions
 Transactions: either executed fully or not at all.
 Counting steps will simplify calculation of UCP
method.
 Thesis: The value of UCP calculated based on
steps is the same as if calculated based on
transactions.
 Results are not the same.
 UCP-T = 0.584 * UCP-S
○ Can not be generalized.
○ Depends on SRS writing styles.


Experiment
 Thesis
 The accuracy of UCP calculated based on steps
is not worse than if calculated based on
transactions.
 Results:
 Almost the same, however, on average UCP-S
more accurate.
 Minor tendency to overestimate.
 Mean RE close to Zero. (almost no bias)
 Conclusion: if a company use standard
style of writing they can use UCP-S instead
of UCP-T.


Experiment
 Can the estimation be further simpliﬁed to
count just the total number of steps in all
use cases?
 Thesis: The Steps metric, which is the total
number of steps in all use cases describing
the system, can be used to estimate effort
with similar accuracy to UCP.
 Result:
 Can be used to simplify the effort.
 Suffers from sensitivity to writing styles.


Use-case transactions methods
 The Transactions metric is defined as the
total number of transactions that can be
identified in all use cases describing the
system under development.
 The TTPoints metric is also based on the
number of transactions, but it includes
additional information about the semantics
of transactions (twelve semantic
transaction types have been identified so
far ).


Experiment
 Thesis: Use-case transactions can be used
to provide effort estimation with better
accuracy than UCP calculated based on
steps.
 Investigation: compare two projects with:
 different writing styles.
 Same actual effort.
 Result: Based on post-productivity factor
UCP-T is better if there historical data
suffer either from writing style or
abstraction level.
 Lower Variability on prediction


Experiment
 TTPoints outperform all step based
metrics.
 However, calculation is complex.


Threats to validity – internal
validity
 Projects developed by different
organizations.
 Maturity of development process is different.
 Some records does not show activities.
 Objectiveness of Transaction [style of
writing]
 Solved by allowing only one person to
extract the transactions and cross validate it
by external expert.


Threats to validity – Conclusion
validity
 Large number of direct comparison
 Can be solved using Bonferroni correction.
 Power of 10
 sensitivity analysis was performed: and
conclusion was “whenever we were not able
to reject the null hypothesis, it could mean
that either the null hypothesis was really true
or the effect size was too small to be
detected in this study”.


Threats to validity – External
validity
 Size of Data Sets:
 Largest data sets used to study UCP
 heterogeneity of the analyzed set of
projects
 Same cross-validation procedure for all
combinations.
○ MMRE shows the difference in data set does not
influence the visibility of findings.
 Data-intensive systems
 This study can be generalized to this kind of
systems.


Conclusion
 Potential ways to simplify effort
estimation:
 Reject UAW [minor impact on accuracy]
 Reduce Technical and Environmental
Factors.
 Use steps to calculate UCP. Further more,
you can use steps metric.
 Transaction Metrics can replace UCP-T.
 Measures based on Steps are sensitive
to writing styles.


Future Work
 Developing standards of writing Use
Cases. This might include enforcement
to use specific phrases to describe use
cases.
 Study if a combination or all suggested
simplification can be used together to
reduce effort while maintaining accuracy.


Simplifying effort estimation based on use case points

More Related Content

What's hot (19)

Viewers also liked (20)

Similar to Simplifying effort estimation based on use case points (20)

Recently uploaded (20)

Simplifying effort estimation based on use case points

Editor's Notes