Technical briefing on Software Release Planning

.5
Xavier Franch
Group of Software and Service Engineering
Universitat Politècnica de Catalunya
Barcelona, Spain
franch@essi.upc.edu
Guenther Ruhe
Software Engineering Decision Support Laboratory
University of Calgary
Calgary, Alberta, Canada
ruhe@ucalgary.ca

Contents
• INTRODUCTION OF PARTICIPANTS
• PART I. BACKGROUND
• PART II. STATE OF THE ART
• PART III. THE EVOLVE APPROACH
• PART IV. CONCLUSIONS
ICSE 2016, Austin, TX 2

Contents

Attendees

Contents

Context – Software Evolution
Continuing Change — an [E-type] system must be continually
adapted or it becomes progressively less satisfactory (Law 1)
Continuing Growth — the functional content of an [E-type]
system must be continually increased to maintain user
satisfaction over its lifetime (Law 6)
Laws of Software Evolution
Manny Lehman (1925 – 2010)
what/when/how to evolve?
Release
Planning

The planning onion
ICSE 2016, Austin, TX 9M. Cohn, Agile Estimating and Planning. Prentice Hall PTR, 2006.

Software release planning – definition
Software release planning – “critical process of deciding which
features are implemented in which releases”
G. Ruhe. Product Release Planning, CRC Press 2010
Release planning – Strategic + operational
Strategic release planning – “selection and assignment of
requirements in sequences of releases such that
important technical and resource constraints are fulfilled”
Operational release planning – “development of the
identified features in a single software release”
Svahnberg et al. A Systematic Review on Strategic Release
Planning Models. IST 52(3), 2010

Example: SENERCON
• Partner of the SUPERSEDE H2020 project
• Service provider for energy savings based in Berlin with
more than 75.000 users
• After developing more than 20 services, still success is
unpredictable, concluding that:
– the success of a service mostly depends on fulfilling personal
and individual needs of the end-user
– mismatch QoS – QoE  The Black Box Problem

Example: SENERCON
• Main reasons:
– lack of detailed knowledge about QoE
• currently, email + hotline only
– not having a systematic release planning approach in place
• currently, based on expert judgement
• Goal: a cost-effective exchange hub users developers
– contextualized user feedback
– discovery of service usage patterns
– combine feedback with context
• Once this information is known:
– features’ value easier to quantify
– systematic release planning may be put in place

What is a feature
“A logical unit of behavior specified by a set of functional and
non-functional requirements”
J. Bosch. Design and Use of a Software Architecture. ACM Press 2000
“A distinguishable characteristic of a concept (system, compo-
nent, etc. ) that is relevant to some stakeholder of the concept”
K. Czarnecki, U.W. Eisenecker. Generative Programming: Methods, Tools and
Applications. Addison-Wesley 2013
“A set of logically related requirements that provide a capability
to the user and enable the satisfaction of business objectives”
K. Wiegers, J. Beatty. Software Requirements (3rd ed.), Microsoft Press 2013

Typical features
• Core functionality of the domain
– prime prerequisite for a company’s business
• Demanded by the market
• Requested by a specific customer
T. Berger et al. What is a Feature? A Qualitative Study of Features in Industrial
Software Product Lines. SPLC 2015

Good and bad features
Reasons for considering a feature as “good”:
• Customer satisfaction
• Distinct functionality
• Well implemented and error free
What makes a feature “bad”:
• Result of time pressure and rushed development
• Compromises emerging during implementation
• Duplicated and superfluous features
• High volatility
T. Berger et al. What is a Feature? A Qualitative Study of Features in Industrial
Software Product Lines. SPLC 2015

Software release planning – A deeper view
Amountofimplementedfunctionality
Amountofsuggestedfunctionally
Releases
• Corrective
• Adaptive
• Perfective
• Preventive
ISO/IEC/IEEE 14764:2006

Software release planning – why?

Release decisions
ICSE 2016, Austin, TX

Software release planning - Why difficult?
Information is
 Uncertain
 Inconsistent
 Incomplete
 Fuzzy
Decision space
 Large size
 High complexity
 Dynamically changing
Multiple objectives
 Usability
 Value
 Time-to-market
 Frequency of use
 Risk
Hard & soft constraints on
 Time
 Effort
 Quality
 Resources

Main challenges in release planning
• Product management underestimated/not sufficiently established
• Product release planning process immature
• Product release planning not synchronized with other processes
• Lack of systematic re-planning
• Lack of transparency of release decisions
• Lack of definition of planning goals/alignment with business goals
• Lack of stakeholder involvement
• Lack of resource consideration
• More re-active than pro-active planning mode
• Impact of better release content unclear
• Impact (value) of individual features unclear
• Planning for just the next release

Contents

Approaches to Software Release Planning
Two main categories
• manual (“on-the-fly”) approaches
– rely on humans’ ability to negotiate
between conflicting objectives and
constraints
– mainly reported as experience reports
• analytical approaches
– formalize the problem
– apply computational algorithms to
generate best solutions
– mainly reported as scientific technical
papers
G. Ruhe, M.O. Saliu. The Art and Science of Software
Release Planning. IEEE Software 22(6), 2005

On-the-fly approaches
• emphasis on improving the decision process
– make estimates as accurate as possible
– provide stakeholders a voice
• emphasis is in the next release
– planning long term is more difficult

Example: a case in Ericsson
V.T. Heikkilä et al. Continuous Release Planning in a Large-
Scale Scrum Development Organization at Ericsson. XP 2013ICSE 2016, Austin, TX 24
• Ericsson node development unit
– traffic management in telecommunication networks
– large systems
– combining hardware and software
• Large projects
– 20 development teams fro Finland and Hungary
– every team had 6-7 members
– following Scrum
• The products
– yearly public releases
– 2 internal versions per release, 2 internal deadlines for
maintenance updates

Team structure

The release planning process

Reported benefits
• Increased flexibility
– feature development schedule not tied to release schedule
– decreased development lead time
• Eliminate waste in the planning process
– early identification of too expensive or unfeasible features
save development resources
• Increased developer motivation
– early involvement of developers in the feature planning
process

Remaining challenges
• Misalignment with “the old way” of planning
– product manager still asking for long-term feature
development plans
• increasing detail of FCS
• Managing non-feature specific work
– non-feature specific problem reports, system documentation,
external change requests, …
• Low prioritization of system improvement work wrt
implementing new features
– some store points saved for system improvements

Limitations of on-the-fly approaches
• Informal process
• Informal decisions
• But: > 1.000.000.000.000 possibilities already in case of
20 objects and three periods

Analytical approaches
F = {f(1), ..., f(N)}
Set of features Set of constraints
X = {x(1), ..., x(N)}
Release plan
x(j) = assigned release
C = {c(1), ..., c(M)}
RP
maximise some utility
or objective function

Analytical approaches
F = {f(1), ..., f(N)}
Set of features Set of constraints
X = {x(1), ..., x(N)}
Release plan
x(j) = assigned release
C = {c(1), ..., c(M)}
RP
maximise some utility
or objective function
What information
is processed?
Which results are
produced?
How is the plan
computed?

Example – release planning in agile projects
Approach Iterations Precedences Risk Change mgmt. Planning
[1] Multi Preced, coupling Some Some Heuristic
[2] Multi Preced Yes No Greedy
[3] Single Preced, coupling No Yes Exact
[4] Single Preced No Some Exact
[5] Multi Preced, coupling Yes No Exact
[6] Multi Preced, coupling Yes No Exact
[7] Multi Preced, anchor, coupling Yes Yes Exact
[1] D. Greer, G. Ruhe. Software release planning: an evolutionary and iterative approach. IST 46, 2004
[2] M. Denne, J. Cleland-Huang. Software by Numbers. Prentice Hall, 2004
[3] M.O. Saliu, G. Ruhe. Supporting software release planning decisions for evolving systems. SEW 2005
[4] C. Li et al. An integrated approach for requirement selection and scheduling in software release
planning. REJ 15, 2010
[5] A. Szoke. Conceptual scheduling model and optimized release scheduling for agile environments. IST
53, 2011
[6] G. van Valkenhoef et al. Quantitative release planning in extreme programming. IST 53, 2011
[7] M. Golfarelli, S. Rizzi, E. Turrichia. Multi-sprint planning and smooth replanning: An optimization
model. JSS 86, 2013

State of the art
Main source: systematic literature review until 2008
• 24 release planning models found
– 14 original and 10 extensions, mostly from 1998
• Three main groups
– EVOLVE-family + ReleasePlanner tool @ University of Calgary
– SERG @ Lund University
– Center of Organization and Information @ Utrecht University
Svahnberg et al. A Systematic Review on Strategic Release
Planning Models. IST 52(3), 2010

State of the art
Extension: ongoing non-systematic literature review until
2015 by the GESSI research group at UPC
• snowballing based approach
– forward snowballing from Svahnberg et al.’s SLR  C1
– backward snowballing from C1
– focus on selected journals and conferences
– contributions from industry also sought
• Final selection: 16 new methods found in the period
2009-2015

New methods in a nutshell (sample)
NRP for eXtreme Programming dealing with some uncertainty
Multsprint planning in an agile context
Combining a planning algorithm with a scheduling method
Efficient algorithm for NRP in the projects with large sets of
requirements
NRP in large scale agile organizations
Calculate the impact of uncertainty with time constraints in
release planning
Define new releases in agile environments taking into account
previous iterations
Efficient NRP algorithm based on model checking considering
dependencies
Implement a risk-aware NRP algorithm

Input: constraints and factors
Soft
factorsRisk factors Value factors
Resource consumption factors
Stakeholders’ influence factors
Svahnberg et al.
A Systematic Review
on Strategic Release
Planning Models.
IST, 52(3), 2010
Requirement dependencies
Quality constraints
Budget and cost constraints
Resource constraints
Effort constraints
Time constraints
Hard
constraints

Input: constrains and factors (prevalence in 2010)
Requirement dependencies (75%)
Quality constraints
(8.3%)
(29.1%)
(33.3%)
Effort constraints
(50%)
Time constraints
(16.7%)
Soft
factorsRisk factors
(12.5%)
Value factors
(37.5%)
Resource consumption factors (20.8%)
Stakeholders’ influence factors (29.2%)
28 methods
Hard
constraints

Input: constrains and factors (prevalence in 2015)
Requirement dependencies (75%)(75%)
Quality constraints
(8.3%) (5%)
(29.1%) (17.5%)
(33.3%) (37.5%)
Effort constraints
(50%) (50%)
Time constraints
(16.7%) (25%)
Soft
factorsRisk factors
(12.5%) (17.5%)
Value factors
(37.5%) (42.5%)
Resource consumption factors (20.8%) (17.5%)
Stakeholders’ influence factors (29.2%)
(22.5%)
40 methods
Hard
constraints

Output
• Scope
– one release vs. multiple releases
• Object of planning
– features; user stories; requirements
• Prioritization
– none
– ordinal
– must-have, should-have, could-have
• Time scheduling
• Developer assignment

Objective function
• Optimization of the value given by the features while
managing the resources and fulfilling all possible
constraints
• How to measure value:
– business value, stakeholder satisfaction, urgency, risk
minimization, technical debt, return on investment, …
• How to measure resources
– personnel, availability
– considering size/complexity of features
• Constraints
– dependencies, time constraints, …

A lot of computational approaches…
Greedy
algorithms
Pareto opti-
mal fronts
Monte-Carlo
simulation
Knapsack
problem
Branch and
bound Backbone
based
algorithms
Graph trans-
formation
AHPClustering

Example – greedy solution (1)
• Principle: always add a feature to the solution that
maximizes value while not violating any constraint or
requiring more resources than available
• Greedy algorithms: building a good global solution as
the sequence of local optimum choices at every
moment

• Input:
– Set of features, F = {f(1), …, f(N)}
– Resource consumption, cost: F  Integer
– Estimated values, value: F  Integer
– Set of cost capacity for each release:
totalCost: Integer  Integer
• Output:
– Release planning, Release: Integer  {Integer}, s.t. all sets
are pair-wise disjoints
G. Ruhe. Product Release Planning, CRC Press 2010

ICSE 2016, Austin, TX 44G. Ruhe. Product Release Planning, CRC Press 2010

Example – Multi-sprint planning in Scrum (1)
• Given a set S of m sprints and a set U of n user stories,
maximise a solution z for the m sprints
• Goals:
– customer satisfaction
– coupling management
– criticality risk management
• Strategy
– generalized assignment model
M. Golfarelli, S. Rizzi, E. Turrichia. Multi-sprint planning and
smooth replanning: An optimization model. JSS 86, 2013

Example – Multi-spring planning in Scrum (2)
Example of input:
ID Name Deps. and
coupling
Utility Comple-
xity
Criticality
risk
Uncert.
risk
s1 Fee configuration s1->s2 80 5 Low Low
s2 Cash cost computation 0.3 s2+s10 85 2 Medium Medium
s3 Import from DBMS 75 2 Medium Medium
s4 Parameterization logic 30 1 Medium Medium
s5 Amortization mask 60 2 No No
s6 Exchange computation 60 2 Low Medium
s7 Exchange import from SAP s7->s6 60 7 Low Low
s8 Mngmt . control reporting 85 4 Medium Low
s9 Operational reporting 100 10 Low Medium
s10 Scenario management mask 0.3 s2+s10 65 3 Low Low

• Objective function z:
– uj: utility of story j
– rj
cr: criticality risk of story j
– xij = 1 if story j is included in sprint i
– G: set of coupling groups of stories; Al: a set of coupled
stories
– al: affinity between stories in Al
– yijl: number of stories of Al affine to story j and included in
sprint i

the sum of stories’ complexity
(considering uncertainty risks) fits
into each sprint capacity
each story is planned in exactly one
sprint
each forced story is planned in the
planned sprint
correct consideration of OR- and
AND-dependencies among features
restricting values of affine stories

Summary: Analytical vs. on-the-fly planning
Caracteristics Analytical methods On-the-fly
Time horizon Next release, but applicable more
general
Next release
Objectives Flexible, but typically value-based Vague and not explicitly described
Stakeholder involvement Not directly supported Opportunistic and by communication
Solution method Greedy heuristic, linear
programming, simulation, ..
Intuition and experience-based
Quality of solutions Good on average, but unknown for
specific case
Difficult to judge. The more risky, the
more complex the problem
Feature dependencies Typically not considered Implicitly, hard to consider for more
complex problems
Human resource
constraints
If at all, then just cumulative effort Implicitly, hard to consider for more
complex problems
What-if analysis
(explicit support)
No No
Integrated tool support Limited No

Summary
• On-the-fly approaches criticised due to the difficulty
of taking into account all knowledge implied by
software release planning
• Conversely, analytical approaches criticised either
because:
– Too simple to be useful
• Lack of information considered
• Over-simplifications (e.g. requirement dependencies)
– Too complex to be adopted
• Learning curve
• Lack of trust in result

Contents

Release planning – Art or Science?
• Art:
Focus on the human
intuition and
communication for
handling tacid
knowledge
• Science:
Emphasis on formalization of
the problem and application of
computational algorithms to
generate best solutions.

What’s the problem?
“The mere formulation of a problem is far
more essential than its solution, which may be
merely a matter of mathematical or
experimental skills. To raise new questions
(and), new possibilities, to regard old
problems from a new angle, requires creative
imagination and marks real advances in
science.”
(Albert Einstein, 1879-1955)

Optimized release planning – How it began
EVOLVE: Greer, D. and Ruhe, G., Software Release Planning: An Evolutionary and
Iterative Approach, Information and Software Technology, Vol. 46 (2004), pp. 243-
253.
What constitutes a release plan?
Max{ F(x, α) = (α - 1) F1(x) + α F2(x) subject to 0 ≤ α ≤ 1, x from X}
Stakeholders
Weightings for stakeholders
Scores of stakeholders towards urgency (F1) and value (F2)
X composed of
- effort constraints
- coupling and precedence constraints (between features)

Optimized release planning – How it began
F1(x) is a penalty function defined for plan x describing the degree of
violation of the monotonicy property between all pairs of features
F2(x) is a benefit function based on feature scores of the stakeholders and the
actual assignment of the feature according to the plan under consideration.
value(n,p) = value_score(n,p)(K – x(n) +1)

Empirical analysis
• EVOLVE was initially based on genetic search offered by
Palisade’s RiskOptimizer
• Early industrial feedback (Corel, Siemens)
• Development of our own GA (emphasis on avoiding
premature convergence)
• Empirical studies with 200 to 700 requirements comparing
the GA with running ILOG’s CPLEX
• Better solutions for LP solver in reasonable time
• Known level of optimality
• Development of our own solution method utilizing open
source optimization combined with knapsack-type of
heuristic for B&B
• New approach more flexible and with higher level of
diversification among top solutions.

EVOLVE II: Three phases
• Phase 1 - Modeling:
– Formal description of the
(changing) real world to make it
suitable for computational
intelligence based solution
techniques
• Phase 2 - Exploration:
– Application of computational
techniques to explore the
solution space, to generate and
evaluate solution alternatives
• Phase 3 - Consolidation:
– Human decision maker evaluates
current solution alternatives
– Match with implicit objectives
and constraints
Computational Intelligence
Interation 1 Release 1
Release 2Interation 2
Interation 3 Release 3
Human Intelligence

Evolution everywhere
• Evolutionary software development (iterative,
incremental)
• Evolutionary solution algorithms
• Evolutionary problem solving (synergy between art and
science)

The diversification principle
A single solution
to a cognitive
complex problem
is less likely to
reflect the actual
problem when
compared to a
portfolio of
qualified
solutions being
structurally
diversified
Consolidation

Preparation
1
Planning criteria
weights
2
Pre-selection of
features
3
Prioritization of features
4
Voice-of-the
stakeholder analysis
5
Technology constraints
7
Resource estimation
6
Optimization
8
Quality and
resource analysis
9
Excitement analysis
10
Stakeholder
evaluation of plans
12
What-if-analysis
11
Final plan decision
13
dependency
between steps
mandatory step
optional step
set of logically
linked steps
feedback link
Stakeholder-centric release planning –
Method EVOLVE II

Criteria for feature selection
• Customer satisfaction
• Customer dissatisfaction
• Risk of implementation
• Risk of acceptance
• Financial value
• Cost
• Time to market
• Volatility
• Frequency of use
• Ease of use

Feature dependencies
• For given features
A, B, and C, we
distinguish eight
types of
dependencies:

Pre-assignment of features to releases

Maximization of stakeholder feature points
Stakeholder
weight
Score(n,q)
Criteria
weight
SCORE(n)
Releases
weight
sfp(n,x)
TSFP(x)
Features
score(n,p,q)
Plan x

• Resource class 1: A resource type r belongs to class 1 if
the feature related consumption of the resource is
limited to exactly the release in which the feature if
offered. Resources of this class are called local based
on its spending mode.
Consumption(k,r,x) = ∑n: x(n)=k consumption(n,r)
≤ Capacity(k,r)

• Resource class 2: A resource type r belongs to class 2 if
the feature related consumption of the resource can be
distributed across different release periods. Resources
of this class are called global based on its spending
mode.
∑ n=1..N wx (n,k,r) consumption(n,r) ≤
∑ Capacity(k,r) for all releases k = 1…K
0 ≤ wx (n,k,r) ≤ 1 for all n,k,r
∑ k = 1 .. K wx (n,k,r) = 1 for all n,r

Comparison of EVOLVE II with other methods
Method
Characteristics EVOLVE II on-the-fly planning [van den Akker et al. ‘08]
Time horizon Flexible Next release Next release
Objectives Flexible in the number and
type of criteria
Vague and not explicitly
described
Maximize financial value
function
Stakeholder involvement Strongly supported with
explicitly assigned
individualized tasks at the
different stages
Opportunistic and by
communication
Not directly supported
Solution method Specialized integer
programming with additional
heuristics
Intuition and experience-based Integer linear programming
(ILP)
Quality of solutions Five near optimal alternative
solutions with known level of
optimality
Difficult to judge. The more
risky, the more complex the
problem
Near-optimal solutions based
on ILOG
Feature dependencies Precedence and coupling Implicitly, hard to consider for
more complex problems
Precedence,
coupling, either or
dependencies
Human resource constraints number, type and granularity
of the resources
Implicitly, hard to consider for
more complex problems
Yes, including staffing of teams
What-if analysis
(explicit support)
Yes No Yes
Integrated tool support ReleasePlanner 2.0 No Prototype based on usage of
ILOG

EVOLVE II tool support - ReleasePlanner

ReleasePlanner™ - Main components

Use cases
1. Project definition: stakeholders, criteria, features, resources,
estimates, capacities, number of releases, permissions
2. Feature prioritization
3. Most controversial features
4. Alternative plan generation
5. Feature dependencies
6. Excitement analysis for a given plan
7. Customization of plans
8. Comparison between two selected plans
9. JIRA: Import of issues and subsequent plan generation
10. Change of data in JIRA and synchronization
11. Innovation planning where stakeholder represent competitors
12. Service portfolio planning
13. When to release planning
14. Feedback-driven planning
15. Planning functional versus quality requirements

Contents

From closed to open world planning

Open innovation
• An (open) approach for integration of internal and
external ideas and paths to market that merges
distributed knowledge and ideas into production
processes.

Release Planning – Information needs
Information needs
Type of release planning problem
Features
Featuredependencies
Featurevalue
Stakeholder
Stakeholderopinionand
priorities
Releasereadiness
Markettrends
Resourceconsumptions
andconstraints
What to release × × × × × × ×
Theme based × × × × × × ×
When to release × × × × × × ×
Consideration of quality requirements × × × × × × ×
Operational release planning × × ×
Consideration of technical debt × × × ×
Multiple products × × × × × × ×

Analytic open innovation
• Open innovation with emphasis on analytics
(processes, tools, knowledge, techniques, decisions).

How much planning is enough?
Perfection of information 100%
Valueand
costofadditionalinformation
(Harrison 1987)
Benefit
Cost
Cost-benefit
ROI the better the more often investments are used

Pro’s of investment
• Pro-active evaluation of impact of decisions
• Support to find the most promising decision
alternatives
• Transparency
• Understandability
• Reducing the impact of
human bias
• Reducing the risk of failure
• Increasing the chance of
success
77ICSE 2016, Austin, TX

Con’s from investment
• Additional effort on decision-making
• Additional effort on information retrieval
• Effort to become familiar with some support tool(s)
• Unavoidable uncertainty (depending on scope)

Summary
• Basic assumption: The more qualified processes and
support is provided, the better the chance to find an
appropriate decision.
• Benefit of a mature release planning process:
– Better customer satisfaction
– Higher competitiveness of
products
– Transparency of decisions
– Ability to adjust to change
– Alignment to business
objectives
– Higher predictability of
results

Acknowledgements
• This work has been partially funded by the SUPERSEDE
H2020 project (2012-2015) under contract nb. 644018
• The first presenter wants to thank D. Ameller and C.
Farré at UPC for their work in the topic of the tutorial
• The second presenter acknowledges the support
provided by NSERC and the collaboration with Maleknaz
Nayebi on this topic.

Technical briefing on Software Release Planning

More Related Content

What's hot (20)

Similar to Technical briefing on Software Release Planning (20)

Recently uploaded (20)

Technical briefing on Software Release Planning