Software metrics in contracts

Status update
Contract Metrics
Committee
Nesma 2016
Hans Bernink
Hans Kuijpers
KK

Who is who
Hans Kuijpers
▪ Software Improvement Group
▪ Senior Consultant
▪ Certified Scope Manager
▪ Professional Scrum Master I
▪ Nesma:
▪ Contract Metrics Committee (chair)
h.kuijpers@sig.eu
software metrics in contracts
K K

Who is who
Hans Bernink
▪Metrieken.nl
▪ Senior Metrics Consultant
▪ Senior Sizing Analyst
▪ Certified Function Point Analyst
▪ Nesma:
▪ Contract Metrics Committee
▪ Basis of Estimate (BOE) Committee
hans.bernink@metrieken.nl
K K

Background / History
▪ Nesma wg Metrics in Contracts
▪ Start July 2013
▪ 7 people
▪ Hans Kuijpers, Software Improvement Group (SIG)
▪ Hans Bernink, Metrieken.nl
▪ Rini Scholten, Kadaster
▪ Frans Schoot Uiterkamp, BrainCap
▪ Jolijn Onvlee, Onvlee Opleidingen & Advies
▪ Marcel Pereboom, Mediaan
▪ Tim Schijffelen, Sysqa
K K

Assignment
“Draw up a NESMA document
that serves as a guideline (or perhaps even more!)
for client organizations and suppliers
in order to use function points and quality metrics
in RFP’s and in contracts
regarding software development
and maintenance.”
K K

Why a Guideline
▪ Outsourcing software projects is a challenge
▪ Organizations struggle to select the best supplier
▪ Attempt to use software metrics often fail
▪ Lack of standards / guidelines / best practices
▪ (Wrong) software metrics are used in a ‘wrong’ way
▪ Some organizations select a ‘wrong’ supplier, often
resulting in ‘failing’ projects
K K

Standish benchmark
K K

Low maturity of the industry
▪ Lack of knowledge
▪ Low maturity in cost engineering practices
▪ Wish:
▪ select ‘the right’ partner
▪ based on objective criteria
▪ Even companies that use metrics based on functional size often use
these in a wrong way
▪ Many suppliers don’t measure completed projects and have no idea
about their performance
K B

Example
▪ RFP framework question:
▪ What is your price/FP
for a Java project of 500 FP and 5 month duration?
FPA and quality metrics in contracts
B B

Scope of the guideline
▪ IFPUG/COSMIC/NESMA function points
▪ Technical quality metrics (ISO 25010)
▪ Software development + Software maintenance
▪ Waterfall, Agile/Scrum, DevOps et al
▪ Contracting phase (e.g. RFI / RFP)
▪ Single project / contract
▪ Maintenance contracts
▪ Framework: many projects / applications
▪ Execution phase: development / maintenance
▪ All technologies
B B

▪ The guideline will result in 13 guides;
1. Guideline for the use of software metrics in contracts (ready)
2. Mini Guide for Development Methodologies (2016)
3. Mini Guide for Software Metrics in Maintenance Contracts (2016)
4. Mini Guide for Management (ready)
5. Mini Guide for RFP questions (ready)
6. Mini Guide for Functional Quality (2016)
7. Mini Guide for Pricing Mechanisms
8. Mini Guide for Technical Quality (ready)
9. Mini Guide for Assessing Suppliers Performance (ready)
10. Mini-Guide for Software Metrics in Development Contracts (2016)
11. Mini Guide: Requirements for Supplier organizations
12. Mini Guide: Requirements for Customer organizations (2016)
13. Mini Guide for Measurement Standards and Definitions (2016)
Current state of Guideline
B K

[2] Development methodologies
▪ Several ways to develop software
▪ Most prominent methods:
Waterfall, Agile, DevOps, Continuous delivery
▪ Performance indicators should:
▪ Focus on the development method, and
▪ Support the way of working
FPA and quality metrics in contracts
2016
K B

[3] Maintenance
▪ 80% of system lifecycle cost is maintenance related
▪ Focus: how to demand for maintainable software
▪ Metrics for contracting and execution
▪ Performance indicators are defined and measured
▪ Supplier performance measured & improved
2016
B K

[4] Management
▪ Why are FPA and quality metrics useful for
successful projects?
▪ Select and contract a professional partner
▪ Stay in control during the contract phase
▪ Maintenance phase
▪ Objective metrics and benchmarking
▪ The ‘which and how’ questions are answered in
the other MGs
READY
K B

[5] RFP Questions
▪ Customers often ask questions that won’t enable
them to make objective comparison
▪ This MG shows:
▪Models from literature that are relevant in
composing RFP questions
▪Effect of duration on an estimate
▪Effect of selecting an optimistic offer
versus realistic and pessimistic offers
▪Possibilities to objectively compare suppliers’ bids
using benchmarking
▪and more
READY
B K

[6] Functional Quality
▪ Outsourced projects often deliver too early
▪ How can the customer determine that the requested
functionality is ready for production?
▪ This MG helps in managing expectations:
▪ Project delivers on time with the agreed quality
▪ Useful metrics:
▪ Defect potentials
▪ Defect Removal Efficiency
▪ Trend
▪ Bonus/Malus scheme for Early delivery/Low quality
2016
K B

[7] Pricing Mechanisms
▪ MG shows the different pricing mechanisms that
are used in the industry, such as:
▪ Price/FP
▪ Fixed price
▪ Time and material
▪ Pricing mechanisms advantages and disadvantages
▪ MG discusses the way software metrics can be used to
overcome the disadvantages
▪ Example: use productivity targets in T&M contracts
B K

[8] Technical Quality
▪ Main metrics: time, budget, functional quality
▪ The 4th metric for outsourcing is technical quality
▪ Good code quality improves:
▪ Defect Resolution Time
▪ Lifecycle Cost
▪ KPI’s for maintainability, security, reliability, performance
READY
K K

[9] Assess Suppliers Performance
▪ Some organizations outsource all of their software
development and maintenance to 1 supplier
▪ It’s crucial to select the right supplier
▪ This MG shows a model on how to compare
different suppliers in an objective way
▪ Basic idea: suppliers send in data of x completed
projects. The model then assesses:
▪ Compliancy to requirements of projects
▪ Reality value of data submissions
▪ Productivity/Quality
(to each other and to external Benchmarks)
READY
K B

[10] Metrics in Development Contracts
▪ Use of metrics in project contracts
▪RFI / RFP
▪Execution / Monitoring
▪Completion
▪ FPA and other metrics used in all lifecycle phases
▪ Project Estimating
▪ Supplier selection
▪ Project Control: cost, time and quality
▪ Benchmarking
2016
B B

[11] Requirements for Suppliers
▪ Suppliers need to implement
Performance measurement processes
▪ Using standards is crucial:
▪ Project classification
▪ Size measurement (ISO standard)
▪ Data collection (standard WBS in effort registration)
▪ Definitions should match benchmarks, e.g.:
▪ Hours/FP, Price/FP: which activities included/excluded
per project type?
▪ Defects/FP: Which defects included/excluded?
▪ Delivery Speed: Per manmonth or calendar month?
B K

[12] Requirements for Customers
▪ Business – IT demand – Supplier
▪ Customers need to have knowledge of how to
▪ Demand for successful projects
▪ Governance the contracts
▪ If not available in your organization ➔ invest in it
▪ Using standards is crucial:
▪ Estimation based on size measurement (ISO standard)
▪ Technical quality is measurable (ISO standard)
▪ Governance by KPI’s
▪ Benchmark supplier performance
2016
K B

[13] Standards and Definitions
▪ This MG describes Base measurements and Metrics used
in the other MG’s
▪Analyzation of Common problems with metrics
▪Using metrics the right way
▪How to construct Metrics:
▪Define Base Measurements
▪ Base measurements need to be objective
▪ Relevant (ISO) standards, definitions, guidelines
▪ Data collection
▪Define Explicit Metrics calculation rule
▪Usage of metrics described
▪Lots of examples
2016
B B

Questions ?
contract-metrics@nesma.org

Software metrics in contracts

More Related Content

What's hot (20)

Similar to Software metrics in contracts (20)

More from Nesma (20)

Recently uploaded (20)

Software metrics in contracts