![10 / 47First International Summer School on SBSE, Cádiz, june/july 2016
Introduction
Basic
Formulation
Multi-Objective
Formulation
Robust
Formulation
Preference-Based
Formulation
Conclusions
& Future Work
• Steady State GA with binary representation
• Maximum dedication set to 1.0 for all employees → xij ∈ [0,1]
• Matrix elements are discretized to eight values (3 bits per element)
T1 T2 T3 T4 T5 T6
E1 0,3 0,2 0,5 0,7 1,0 0,0
E2 0,0 0,0 0,2 0,1 0,5 0,8
E3 0,2 0,0 0,0 0,6 1,0 1,0
E4 0,4 0,6 0,0 0,0 0,0 1,0
T1 T2 T3 T4 T5 T6
E1 010 001 100 101 110 000
E2 000 000 001 001 100 110
E3 001 000 000 100 111 111
E4 010 100 000 000 000 111
Chromosome
010001100101110000000000…
2D recombination
Basic Problem Formulation: algorithm & representation](https://image.slidesharecdn.com/sps-chicano-sssbse2016-161130103900/85/Search-Based-Software-Project-Scheduling-10-320.jpg)
![28 / 47First International Summer School on SBSE, Cádiz, june/july 2016
Introduction
Basic
Formulation
Multi-Objective
Formulation
Robust
Formulation
Preference-Based
Formulation
Conclusions
& Future Work
Robustness
Task cost
Objective space
Solution space
x
t
F(t,x)
Average, Std. dev.
Average, Std. dev. Three approaches
• No robustness (NR)
• One task changes (OTR)
• Several tasks change (STR)
Task change
• Multiply by a random value in
[0.5,2]](https://image.slidesharecdn.com/sps-chicano-sssbse2016-161130103900/85/Search-Based-Software-Project-Scheduling-28-320.jpg)
![42 / 47First International Summer School on SBSE, Cádiz, june/july 2016
Introduction
Basic
Formulation
Multi-Objective
Formulation
Robust
Formulation
Preference-Based
Formulation
Conclusions
& Future Work
• The region of interest can be determined by a single point in the
objective space: the reference point
(a)q alcanzable
rection based NSGA-II (RD-NSGA-II) [9] incorpora
a NSGA-II una metodolog´ıa extra´ıda de MCDM de-
nominada direcci´on de referencia [24]. A partir de
un punto del espacio de objetivos y un punto de re-
ferencia proporcionado por el decisor se define una
direcci´on de referencia, considerando la diferencia
entre ambos. Sobre ´esta se definen puntos de re-
ferencia equidistantes que son proyectados sobre el
frente ´optimo de Pareto mediante la funci´on escala-
rizada de logro. Otra idea extra´ıda de MCDM, de-
nominada Light Beam Search [25], ha sido utilizada
en [10] con su integraci´on en NSGA-II. interactive
MOEA/D (iMOEA/D) es un enfoque interactivo de
MOEA/D propuesto en [11]. Tras un n´umero deter-
minado de generaciones se muestra un conjunto de
soluciones al decisor, que especifica sus preferencias
sobre ´estas. El conjunto de pesos usado en MOEA/D
para optimizar m´ultiples funciones de logro es aco-
tado al vecindario de las soluciones determinadas
como preferidas. As´ı, el proceso de b´usqueda se
orienta progresivamente hacia la regi´on de inter´es
(a)q alcanzable
(b)q inalcanzable
Fig. 1. Ilustraci´on de la m´etrica HVq.
un punto
ferencia p
direcci´on
entre amb
ferencia e
frente ´opt
rizada de
nominada
en [10] co
MOEA/D
MOEA/D
minado d
soluciones
sobre ´esta
para optim
tado al v
como pre
orienta pr
del frente
La form
planificaci
presentad
sonas invo
cada empl
salario po
junto de t
son defini
riando j d
cada tarea
que se co
tas preced
dencia de
G(T, A) cu
Reachable reference point Unreachable reference point
Hypervolume restricted
to the interest region
Expressing Preferences in Objective Space](https://image.slidesharecdn.com/sps-chicano-sssbse2016-161130103900/85/Search-Based-Software-Project-Scheduling-42-320.jpg)
![53 / 47First International Summer School on SBSE, Cádiz, june/july 2016
Introduction
Basic
Formulation
Multi-Objective
Formulation
Robust
Formulation
Preference-Based
Formulation
Conclusions
& Future Work
Algorithms: NSGA-II
procedure and a density estimator known as crowding distance (
see [19]). Finally, the population is updated with the best ind
are repeated until the termination condition is fulfilled.
Algorithm 1 Pseudocode of NSGA-II.
1: proc Input:(nsga-II) //Algorithm parameters in ‘nsga-II’
2: P Initialize Population() // P = population
3: Q // Q = auxiliary population
4: while not Termination Condition() do
5: for i to (nsga-II.popSize / 2) do
6: parents Selection(P)
7: offspring Recombination(nsga-II.Pc,parents)
8: offspring Mutation(nsga-II.Pm,offspring)
9: Evaluate Fitness(offspring)
10: Insert(offspring,Q)
11: end for
12: R P Q
13: Ranking And Crowding(nsga-II, R)
14: P Select Best Individuals(nsga-II, R)
15: end while
16: end proc](https://image.slidesharecdn.com/sps-chicano-sssbse2016-161130103900/85/Search-Based-Software-Project-Scheduling-53-320.jpg)
Search-Based Software Project Scheduling
- 1. 1 / 47First International Summer School on SBSE, Cádiz, june/july 2016 Introduction Basic Formulation Multi-Objective Formulation Robust Formulation Preference-Based Formulation Conclusions & Future Work Search-Based Software Project Scheduling Francisco Chicano joint work with E. Alba, A. Cervantes, D. González-Álvarez, F. Luna, A. J. Nebro, G. Recio, R. Saborido, M. A. Vega-Rodríguez
- 2. 2 / 47First International Summer School on SBSE, Cádiz, june/july 2016 Introduction Basic Formulation Multi-Objective Formulation Robust Formulation Preference-Based Formulation Conclusions & Future Work Introduction • Current software projects are very complex • They can involve hundreds of people and tasks • An efficient way of assigning employees to tasks is required • An automatic software tool can assist to the software project manager • Problem: assign employees to tasks with a given dedication degree Employee Task Salary Maximum dedication Skills Effort Required skills TPG
- 3. 3 / 47First International Summer School on SBSE, Cádiz, june/july 2016 Introduction Basic Formulation Multi-Objective Formulation Robust Formulation Preference-Based Formulation Conclusions & Future Work Introduction • Several authors proposed different formulations in the literature
- 4. 4 / 47First International Summer School on SBSE, Cádiz, june/july 2016 Introduction Basic Formulation Multi-Objective Formulation Robust Formulation Preference-Based Formulation Conclusions & Future Work
- 5. 5 / 47First International Summer School on SBSE, Cádiz, june/july 2016 Introduction Basic Formulation Multi-Objective Formulation Robust Formulation Preference-Based Formulation Conclusions & Future Work T1 T2 T3 T4 T5 T6 E1 0.3 0.2 0.5 0.7 1.0 0.0 E2 0.0 0.0 0.2 0.1 0.5 0.8 E3 0.2 0.0 0.0 0.6 1.0 1.0 E4 0.4 0.6 0.0 0.0 0.0 1.0 T1 T2 T3 T4 T5 T6 Time Project duration ∑ 0.8 Effort T2 = Duration T2 • Project duration (computation) Gantt diagram of the project Task duration TPG Basic Problem Formulation: duration
- 6. 6 / 47First International Summer School on SBSE, Cádiz, june/july 2016 Introduction Basic Formulation Multi-Objective Formulation Robust Formulation Preference-Based Formulation Conclusions & Future Work 1.0 1.0 0.8 0.0 T6 0.0 1.0 0.5 1.0 T5 0.00.00.60.4E4 0.60.00.00.2E3 0.10.20.00.0E2 0.70.50.20.3E1 T4T3T2T1 • Project cost (computation) T1 T2 T3 T4 T5 T6 E1 0.3 0.2 0.5 0.7 1.0 0.0 E2 0.0 0.0 0.2 0.1 0.5 0.8 E3 0.2 0.0 0.0 0.6 1.0 1.0 E4 0.4 0.6 0.0 0.0 0.0 1.0 Dur. T4 × T1 T2 T3 T4 T5 T6 E1 0.3 0.2 0.5 0.7 1.0 0.0 E2 Dur. T1 × Dur. T2 × Dur. T3 × Dur. T4 × Dur. T5 × Dur. T6 × E3 0.2 0.0 0.0 0.6 1.0 1.0 E4 0.4 0.6 0.0 0.0 0.0 1.0 Time employee E3 spends on task T4 ∑ = time the employee spends on the project Salary of E3 Cost of employee E3 due to its participacion Cost of employee E2 due to its participation Cost of employee E4 due to its participacion Cost of employee E1 due to its participation Project cost∑ = Basic Problem Formulation: cost
- 7. 7 / 47First International Summer School on SBSE, Cádiz, june/july 2016 Introduction Basic Formulation Multi-Objective Formulation Robust Formulation Preference-Based Formulation Conclusions & Future Work T1 T2 T3 T4 T5 T6 E1 0.3 0.2 0.5 0.7 1.0 0.0 E2 0.0 0.0 0.2 0.1 0.5 0.8 E3 0.2 0.0 0.0 0.6 1.0 1.0 E4 0.4 0.6 0.0 0.0 0.0 1.0 ∑ 0.9 > 0 C1. All tasks must be performed C2. The union of the work team skills must include the required skills of the task they perform • Constraints Basic Problem Formulation: constraints6th Metaheuristics International Conference 2005 Project Scheduling Problem T1 T2 T3 T4 T5 T6 E1 0.3 0.2 0.5 0.7 1.0 0.0 E2 0.0 0.0 0.2 0.1 0.5 0.8 E3 0.2 0.0 0.0 0.6 1.0 1.0 E4 0.4 0.6 0.0 0.0 0.0 1.0 onstraints 0.9 > 0 1. All tasks must be erformed by somebody 6th Metaheuristics International Conference 2005 Project Scheduling Problem T1 T2 T3 T4 T5 T6 E1 0.3 0.2 0.5 0.7 1.0 0.0 E2 0.0 0.0 0.2 0.1 0.5 0.8 E3 0.2 0.0 0.0 0.6 1.0 1.0 E4 0.4 0.6 0.0 0.0 0.0 1.0 Constraints 0.9 > 0 1. All tasks must be erformed by somebody 6th Metaheuristics International Conference 2005 Project Scheduling Problem T1 T2 T3 T4 T5 T6 E1 0.3 0.2 0.5 0.7 1.0 0.0 E2 0.0 0.0 0.2 0.1 0.5 0.8 E3 0.2 0.0 0.0 0.6 1.0 1.0 E4 0.4 0.6 0.0 0.0 0.0 1.0 Constraints 0.9 > 0 1. All tasks must be erformed by somebody Vienna, Austria, August 22-26, 2005 T1 T2 T3 T4 T5 T6 E1 0.3 0.2 0.5 0.7 1.0 0.0 E2 0.0 0.0 0.2 0.1 0.5 0.8 E3 0.2 0.0 0.0 0.6 1.0 1.0 E4 0.4 0.6 0.0 0.0 0.0 1.0 • Constraints 0.9 > 0 1. All tasks must be performed by somebody 2. The union of the employees skills must include the required skills of the task they perform Introduction PSP Fitness Funct. Representation Experiments Conclusions & Future Work Vienna, Austria, August 22-26, 2005 T1 T2 T3 T4 T5 T6 E1 0.3 0.2 0.5 0.7 1.0 0.0 E2 0.0 0.0 0.2 0.1 0.5 0.8 E3 0.2 0.0 0.0 0.6 1.0 1.0 E4 0.4 0.6 0.0 0.0 0.0 1.0 • Constraints 0.9 > 0 1. All tasks must be performed by somebody 2. The union of the employees skills must include the required skills of the task they perform Introduction PSP Fitness Funct. Representation Experiments Conclusions & Future Work
- 8. 8 / 47First International Summer School on SBSE, Cádiz, june/july 2016 Introduction Basic Formulation Multi-Objective Formulation Robust Formulation Preference-Based Formulation Conclusions & Future Work T1 T2 T3 T4 T5 T6 E1 0.3 0.2 0.5 0.7 1.0 0.0 T1 T2 T3 T4 T5 T6 Time Project duration C3. No employee must exceed her/his maximum dedication Time Dedication Maximum dedicationOverwork • Constraints (cont.) Basic Problem Formulation: constraints
- 9. 9 / 47First International Summer School on SBSE, Cádiz, june/july 2016 Introduction Basic Formulation Multi-Objective Formulation Robust Formulation Preference-Based Formulation Conclusions & Future Work Project cost Project duration Overwork Required skillsUndone tasks Peso Valor wcost 10-6 wdur 0.1 wpenal 100 wundt 10 wreqsk 10 wover 0.1 equally distributed. Therefore, three bits are required for representing The matrix X is stored into the chromosome ⃗x in row major order 1 chromosome length is E · T · 3. Fig. 6 shows the representation used. To compute the fitness of a chromosome ⃗x we use the next expression: f(⃗x) = ⎧ ⎪⎨ ⎪⎩ 1/q if the solution is feasible 1/(q + p) otherwise where q = wcost · pcost + wdur · pdur and p = wpenal + wundt · undt + wreqsk · reqsk + wover · pover Basic Problem Formulation: fitness
- 10. 10 / 47First International Summer School on SBSE, Cádiz, june/july 2016 Introduction Basic Formulation Multi-Objective Formulation Robust Formulation Preference-Based Formulation Conclusions & Future Work • Steady State GA with binary representation • Maximum dedication set to 1.0 for all employees → xij ∈ [0,1] • Matrix elements are discretized to eight values (3 bits per element) T1 T2 T3 T4 T5 T6 E1 0,3 0,2 0,5 0,7 1,0 0,0 E2 0,0 0,0 0,2 0,1 0,5 0,8 E3 0,2 0,0 0,0 0,6 1,0 1,0 E4 0,4 0,6 0,0 0,0 0,0 1,0 T1 T2 T3 T4 T5 T6 E1 010 001 100 101 110 000 E2 000 000 001 001 100 110 E3 001 000 000 100 111 111 E4 010 100 000 000 000 111 Chromosome 010001100101110000000000… 2D recombination Basic Problem Formulation: algorithm & representation
- 11. 11 / 47First International Summer School on SBSE, Cádiz, june/july 2016 Introduction Basic Formulation Multi-Objective Formulation Robust Formulation Preference-Based Formulation Conclusions & Future Work • 48 generated instances in 5 groups • In the first three groups (12 instancias) only one parameter change v Employees (5, 10, 15, 20) v Tasks (10, 20, 30) v Skills of employees (2, 4, 6, 8, 10) • Fourth and fifth groups: all parameters simultaneously change • 100 independent runs GA param. Value Population 64 Selection Binary tournament Recombination 2D crossover Mutation Bit flip (pm=1/length) Replacement Elitist Stop condition 5000 generations Basic Problem Formulation: experiments
- 12. 12 / 47First International Summer School on SBSE, Cádiz, june/july 2016 Introduction Basic Formulation Multi-Objective Formulation Robust Formulation Preference-Based Formulation Conclusions & Future Work 4-5 skills per employee 94 97 6 43 97 Project duration decreases with more employees Fourth group of instances Hit rate Cost Duration Basic Problem Formulation: experiments
- 13. 13 / 47First International Summer School on SBSE, Cádiz, june/july 2016 Introduction Basic Formulation Multi-Objective Formulation Robust Formulation Preference-Based Formulation Conclusions & Future Work
- 14. 14 / 47First International Summer School on SBSE, Cádiz, june/july 2016 Introduction Basic Formulation Multi-Objective Formulation Robust Formulation Preference-Based Formulation Conclusions & Future Work • Multi-Objective Software Project Scheduling • Objectives – Minimize the project cost – Minimize the project duration • Constraints – C1: All tasks must be performed by some employee – C2: The union of the employees skills must include the required skills of the task they perform – C3: No employee exceeds his/her maximum dedication Employee Task Salary Max dedication Skills Effort Required skills TPG 1.0 1.0 0.8 0.0 T6 0.0 1.0 0.5 1.0 T5 0.00.00.60.4E4 0.60.00.00.2E3 0.10.20.00.0E2 0.70.50.20.3E1 T4T3T2T1 Solution Dedication of E1 to T4 Multi-Objective Problem Formulation
- 15. 15 / 47First International Summer School on SBSE, Cádiz, june/july 2016 Introduction Basic Formulation Multi-Objective Formulation Robust Formulation Preference-Based Formulation Conclusions & Future Work 1.0 1.0 0.8 0.0 T6 0.0 1.0 0.5 1.0 T5 0.00.00.60.4E4 0.60.00.00.2E3 0.10.20.00.0E2 0.70.50.20.3E1 T4T3T2T1 Multi-Objective Problem Formulation
- 16. 16 / 47First International Summer School on SBSE, Cádiz, june/july 2016 Introduction Basic Formulation Multi-Objective Formulation Robust Formulation Preference-Based Formulation Conclusions & Future Work • Hypervolume (HV) – Volume covered by members of the non-dominated set of solutions – Measures both convergence and diversity in the Pareto front – Larger values are better • Attainment surfaces – Localization statistics for fronts – The same as the median and the interquartile range in the mono-objective case 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 75%-EAS 50%-EAS 25%-EAS Multi-Objective Problem Formulation: quality indicators
- 17. 17 / 47First International Summer School on SBSE, Cádiz, june/july 2016 Introduction Basic Formulation Multi-Objective Formulation Robust Formulation Preference-Based Formulation Conclusions & Future Work • Generational GA • Ranking & CrowdingNSGA-II • Generational GA + External Archive • Strengh raw fitness & K-nearest neighborSPEA2 • (1+1) Evolution Strategy + External Archive • Adaptive GridPAES • Cellular GA + External archive • Ranking & Crowding from NSGA-IIMOCell • Differential Evolution • Ranking & NSGA-II’s improved crowdingGDE3 Multi-Objective Problem Formulation: algorithms
- 18. 18 / 47First International Summer School on SBSE, Cádiz, june/july 2016 Introduction Basic Formulation Multi-Objective Formulation Robust Formulation Preference-Based Formulation Conclusions & Future Work • Ranking of the algorithms based on the median of their HV values • PAES has reached the approximated fronts with the better (higher) HV – Best in 25 out of 36 instances – It assigns a low dedication to employees à avoid constraint violation for larger instances • MOCell and GDE3 performs specially well for small instances • Neither NSGA-II nor SPEA2 have ranked the first nor second for any instance • Crossover operators (in NSGA-II, SPEA2, and MOCell) and Differential Evolution recombination (in GDE3) generate many unfeasible solutions in large instances 0 0,5 1 1,5 2 2,5 3 3,5 4 4,5 5 NSGAII SPEA2 PAES MOCell GDE3 Averagerank HV-based rank 1 2 354 Multi-Objective Problem Formulation: results
- 19. 19 / 47First International Summer School on SBSE, Cádiz, june/july 2016 Introduction Basic Formulation Multi-Objective Formulation Robust Formulation Preference-Based Formulation Conclusions & Future Work • They graphically represent the median • PF is the reference Pareto Front build for each instance • They clearly explain the high HV values of PAES • Five different behaviors remain hidden to a scalar indicator such as HV Scenario 1 • PAES outperforms all the others • Project plans with low cost and long durations Scenario 2 • All the algorihtms perform the same • But SPEA2 Scenario 3 • The attainment surfaces of NSGA-II, MOCell, and GDE3 cross that of PAES • PAES is slightly worse in concrete regions Scenario 4 • PAES fails at reaching short but costly projet plans • Its HV remains the higher because of its extension Scenario 5 • PAES is clearly outperformed • It happens in the smaller (easier) instances Multi-Objective Problem Formulation: results
- 20. 20 / 47First International Summer School on SBSE, Cádiz, june/july 2016 Introduction Basic Formulation Multi-Objective Formulation Robust Formulation Preference-Based Formulation Conclusions & Future Work Scenario 1 • PAES clearly dominates the solutions reached by all the other algorithms • This algorithm has also reached project plans with low cost and long durations • They graphically represent the median • PF is the reference Pareto Front build for each instance • They clearly explain the high HV values of PAES • Five different behaviors remain hidden to a scalar indicator such as HV Multi-Objective Problem Formulation: results
- 21. 21 / 47First International Summer School on SBSE, Cádiz, june/july 2016 Introduction Basic Formulation Multi-Objective Formulation Robust Formulation Preference-Based Formulation Conclusions & Future Work • They graphically represent the median • PF is the reference Pareto Front build for each instance • They clearly explain the high HV values of PAES • Five different behaviors remain hidden to a scalar indicator such as HV Scenario 2 • All the algorithms but SPEA2 perform the same • On average, their approximated fronts are overlapped in almost the entire objective space • They are also very close to the reference Pareto Front (PF) Multi-Objective Problem Formulation: results
- 22. 22 / 47First International Summer School on SBSE, Cádiz, june/july 2016 Introduction Basic Formulation Multi-Objective Formulation Robust Formulation Preference-Based Formulation Conclusions & Future Work • They graphically represent the median • PF is the reference Pareto Front build for each instance • They clearly explain the high HV values of PAES • Five different behaviors remain hidden to a scalar indicator such as HV Scenario 3 • The attainment surfaces of NSGA-II, MOCell, and GDE3 cross that of PAES à the region of project plans with short durations and high cost • PAES still obtains the best HV values because it covers a larger portion of the objective space Multi-Objective Problem Formulation: results
- 23. 23 / 47First International Summer School on SBSE, Cádiz, june/july 2016 Introduction Basic Formulation Multi-Objective Formulation Robust Formulation Preference-Based Formulation Conclusions & Future Work Scenario 4 • PAES is clearly the worse algorithm at reaching project plans with short durations and high cost • This happens in 18 out of the 36 instances • PAES still gets the best HV value à Is HV suitable to make decisions? • They graphically represent the median • PF is the reference Pareto Front build for each instance • They clearly explain the high HV values of PAES • Five different behaviors remain hidden to a scalar indicator such as HV Multi-Objective Problem Formulation: results
- 24. 24 / 47First International Summer School on SBSE, Cádiz, june/july 2016 Introduction Basic Formulation Multi-Objective Formulation Robust Formulation Preference-Based Formulation Conclusions & Future Work Scenario 5 • NSGA-II, MOCell and GDE3 clearly dominates the attainment surface of PAES • The HV values now reflect this fact • It always happens in the smaller (easier) instances • They graphically represent the median • PF is the reference Pareto Front build for each instance • They clearly explain the high HV values of PAES • Five different behaviors remain hidden to a scalar indicator such as HV Multi-Objective Problem Formulation: results
- 25. 25 / 47First International Summer School on SBSE, Cádiz, june/july 2016 Introduction Basic Formulation Multi-Objective Formulation Robust Formulation Preference-Based Formulation Conclusions & Future Work • Spearman rank correlation coefficients of the solutions in an approximated Front – : positive correlation – : negative correlation – Gray scale: absolute correlation • A example for an approximated Pareto front of PAES and an instance with 20 tasks and 15 employees • PAES identifies the cheapest employees to reach low cost project plans (and long duration) • Correlation in parallel tasks of TGP – Workload increases if they have to finish at the same time (t1, t8 -> ) – Otherwise, the workload is shared (t1, t2 -> ) • Consecutive tasks in TGP − between t14, t16, t20 and project duration: − PAES does not reach Pareto optimal solutions with short durations and high cost e7, e8, e9, e10 are the cheapest employees à they are choosen for cheaper and longer projects e2, e3, e4, e5, e6, e11, e12, e13, e14 , e15 increase their dedication as shorter and more expensive projects are reached Correlationbetweenobjectives andtasks Corr.betweenobjectivesand employees Correlation between tasks and employees Correlation between tasks Correlation between employees t1 and t2: negative correlation because t2 does not require much effort so its influence on the project cost or duration is small The workload is increased in t1 and t8 at the same time in order to reduce the project cost and duration t14, t16 and t20 has positive correlation with the project duration à not optimal assignment reached by PAES Multi-Objective Problem Formulation: results
- 26. 26 / 47First International Summer School on SBSE, Cádiz, june/july 2016 Introduction Basic Formulation Multi-Objective Formulation Robust Formulation Preference-Based Formulation Conclusions & Future Work
- 27. 27 / 47First International Summer School on SBSE, Cádiz, june/july 2016 Introduction Basic Formulation Multi-Objective Formulation Robust Formulation Preference-Based Formulation Conclusions & Future Work • The problem formulation is far from realistic: – Task effort is not an exact value (as assumed), we can only estimate it – Skills are not 0 or 1, there are degrees – Durations are not real values, they are discrete • How to model: – Task effort inaccuracy ▶ robust optimization – Non-binary skills ▶ productivity matrix – Discrete durations ▶ discrete event simulator Motivation for the Second Formulation
- 28. 28 / 47First International Summer School on SBSE, Cádiz, june/july 2016 Introduction Basic Formulation Multi-Objective Formulation Robust Formulation Preference-Based Formulation Conclusions & Future Work Robustness Task cost Objective space Solution space x t F(t,x) Average, Std. dev. Average, Std. dev. Three approaches • No robustness (NR) • One task changes (OTR) • Several tasks change (STR) Task change • Multiply by a random value in [0.5,2]
- 29. 29 / 47First International Summer School on SBSE, Cádiz, june/july 2016 Introduction Basic Formulation Multi-Objective Formulation Robust Formulation Preference-Based Formulation Conclusions & Future Work Instance Information Employee Task Salary Cost TPG T1 T2 T3 T4 T5 T6 E1 0.3 0.2 0.5 0.7 1.0 0.0 E2 0.0 0.0 0.2 0.1 0.5 0.8 E3 0.2 0.0 0.0 0.6 1.0 1.0 E4 0.4 0.6 0.0 0.0 0.0 1.0
- 30. 30 / 47First International Summer School on SBSE, Cádiz, june/july 2016 Introduction Basic Formulation Multi-Objective Formulation Robust Formulation Preference-Based Formulation Conclusions & Future Work Solution d 0.3 1.0 0.2 0.4 r 3 2 5 7 1 0 q T1 T2 T3 T4 T5 T6 E1 3 1 5 0 0 0 E2 0 0 2 1 5 0 E3 2 0 0 0 1 1 E4 0 0 0 1 0 1 Priorities matrix Delays vector Dedication vector • The evaluation of a solution is based on a simulation of the project • Objectives: • Makespan: the minimum time slot in which all tasks are done • Cost: salary multiplied by the dedication and worked hours
- 31. 31 / 47First International Summer School on SBSE, Cádiz, june/july 2016 Introduction Basic Formulation Multi-Objective Formulation Robust Formulation Preference-Based Formulation Conclusions & Future Work Algorithms in the Comparison • Generational GA • Ranking & CrowdingNSGA-II • Generational GA + External Archive • Strengh raw fitness & K-nearest neighborSPEA2 • (1+1) Evolution Strategy + External Archive • Adaptive GridPAES • Cellular GA + External archive • Ranking & Crowding from NSGA-IIMOCell
- 32. 32 / 47First International Summer School on SBSE, Cádiz, june/july 2016 Introduction Basic Formulation Multi-Objective Formulation Robust Formulation Preference-Based Formulation Conclusions & Future Work • 2 instances based on a MS Project repository real example: ms1 and ms2 Problem instances Experiments: Instances stically significant or not. All the statistical tests were performed w fidence level of 95%. Two realistic instances that are variations of a project scheduling wh able at the online repository of the MS Project tool will be solved i arch. The same TPG (see Fig. 1), tasks cost and number of employee original instance will be used and the values for the employees salar productivity matrix will also be provided. Table 1 summarises the rmation. T1 T2 T3 T4 T5 T6 T7 T11 T12 T8 T9 T10 T14 T13 T16 T15 T24 T25 T17 T18 T19 T20 T21 T22 T23 T26 T27 T28 T29 1. Task Precedence Graph for the two instances of the SPS problem being Task Precedence Graph Table 1. Productivity matrices P i,j, task cost tc j and employee salary es i . Emp. Task (tj ) ei es i 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 e1 50 ms1 1 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 ms2 1 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 e2 40 ms1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 1 0 0 1 1 ms2 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 .5 0 0 1 1 e3 10 ms1 0 0 0 0 0 0 0 1 1 1 0 0 1 0 0 0 0 1 0 1 0 0 0 1 0 1 0 0 0 ms2 0 0 0 0 0 0 0 .3 .3 .3 0 0 .5 0 0 0 0 .5 0 .5 0 0 0 .5 0 .5 0 0 0 e4 15 ms1 0 0 0 0 0 0 0 1 1 1 1 1 1 0 0 0 0 0 1 0 1 0 0 0 0 1 1 1 0 ms2 0 0 0 0 0 0 0 1 1 1 .5 .5 .5 0 0 0 0 0 .8 0 .8 0 0 .8 .8 .8 .8 .8 0 e5 20 ms1 0 1 1 1 1 1 0 0 0 0 1 1 1 0 0 1 1 0 0 0 0 1 1 0 0 0 0 1 0 ms2 0 .5 .5 .5 .5 .5 0 0 0 0 1 1 1 0 0 1 1 0 0 0 0 1 1 1 1 1 1 1 0 e6 30 ms1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 1 0 0 1 0 ms2 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 .8 0 0 .8 0 e7 30 ms1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 1 1 1 1 0 ms2 0 .7 .7 .7 .7 .7 .7 .7 .7 .7 .7 .7 .7 .7 .7 0 0 0 0 0 0 0 0 1 1 1 1 1 0 tc j 6 680 408 8 10 10 378 10 10 10 162 48.6 8.8 720 6 198 180 6 108 6 30 36 36 18 540 120 180 450 3 Productivity Matrix
- 33. 33 / 47First International Summer School on SBSE, Cádiz, june/july 2016 Introduction Basic Formulation Multi-Objective Formulation Robust Formulation Preference-Based Formulation Conclusions & Future Work NSGAII Population: 100 Binary tournament DPX (pc=0.9) Uniform mutation (pm=1/L) SPEA2 Population: 100 Binary tournament DPX (pc=0.9) Uniform mutation (pm=1/L) PAES Population: 1 Uniform mutation (pm=1/L) MOCell Population: 100 Binary tournament DPX (pc=0.9) Uniform mutation (pm=1/L) Experiments: Algorithm-Specific Parameters
- 34. 34 / 47First International Summer School on SBSE, Cádiz, june/july 2016 Introduction Basic Formulation Multi-Objective Formulation Robust Formulation Preference-Based Formulation Conclusions & Future Work • Stopping condition: 1 000 000 function evaluations • Approximated Pareto front size: 100 solutions • Sampling H=100 • 100 independent runs for each algorithm-instance • Statistical tests for significancedifferences (95%) • Representation: integer matrix + real vector + integer vector Global Parameters Experiments: Global Parameters
- 35. 35 / 47First International Summer School on SBSE, Cádiz, june/july 2016 Introduction Basic Formulation Multi-Objective Formulation Robust Formulation Preference-Based Formulation Conclusions & Future Work • NSGA-II and MOCell are the best algorithms • NSGA-II is specially good in robust versions of the problem • MOCell is good in the non-robust version • PAES is the worst algorithm in the comparison • Running time between 2.5 and 5 minutes in NR and around 5 hours in OTR and STR Hypervolume (HV) Results: Hypervolume Comparison have been evaluated using the HV indicator which values are summarised in Ta- ble 2. The best performances are highlighted in a dark grey background whereas second to best are shown in light grey. We also mark with ⇤ the results hav- ing statistically significant di↵erences with the best result. Several conclusions can be drawn from these values. Both NSGA-II and MOCell obtained the best (largest) values for the two instances (as well as many of the second to best values). NSGA-II resulted in the best performance when tackling the robust versions of the instances (in 3 out of the 4 scenarios the approximated Pareto front with best HV indicator was returned). On the other hand, MOCell seems to be specially well suited for the non-robust setting, yielding the higher HV indicator for the two instances. PAES seems to be clearly the worst algorithm with respect to this indicator, specially for the robust versions. The uncertainty in the objective functions could be the main reason behind this fact. Regarding the runtime, all the algorithms require between 2.5 and 5 minutes in the NR scenario, while they require around 5 hours in the OTR and STR scenarios. Table 2. Median and IQR of the HV value for the two instances. NSGAII SPEA2 PAES MOCell NSGAII SPEA2 PAES MOCell Rob. ms1 ms2 NR 0.943⇤ 0.000 0.943⇤ 0.000 0.518⇤ 0.065 0.9440.000 0.904⇤ ±0.000 0.905⇤ ±0.001 0.543⇤ ±0.031 0.905±0.000 OTR 0.829⇤ 0.027 0.807⇤ 0.030 0.328⇤ 0.039 0.8160.032 0.738±0.025 0.730±0.018 0.287⇤ ±0.020 0.695⇤ ±0.043 STR 0.7460.028 0.688⇤ 0.063 0.345⇤ 0.036 0.7420.025 0.764±0.025 0.717⇤ ±0.030 0.387⇤ ±0.032 0.769±0.022 3.2 Analysis of solutions Median and interquartile range
- 36. 36 / 47First International Summer School on SBSE, Cádiz, june/july 2016 Introduction Basic Formulation Multi-Objective Formulation Robust Formulation Preference-Based Formulation Conclusions & Future Work Results: Comparison with a (Human) Base Solution 1000 1200 1400 1600 1800 2000 2200 2400 2600 2800 3000 95000 100000 105000 110000 115000 120000 125000 130000 135000 140000 145000 150000 Makespan Cost Sample solutions Instance ms1 Instance ms2 Base Solution ms1 Base Solution ms2 0 5000 10000 15000 20000 25000 11500 Makespan NSGA-II
- 37. 37 / 47First International Summer School on SBSE, Cádiz, june/july 2016 Introduction Basic Formulation Multi-Objective Formulation Robust Formulation Preference-Based Formulation Conclusions & Future Work Results: 50%-Attainment Surface 150000 0 5000 10000 15000 20000 25000 115000 120000 125000 130000 135000 140000 145000 150000 155000 160000 165000 Makespan Cost NSGA-IINSGA-II ms1 instance STR approach
- 38. 38 / 47First International Summer School on SBSE, Cádiz, june/july 2016 Introduction Basic Formulation Multi-Objective Formulation Robust Formulation Preference-Based Formulation Conclusions & Future Work Results: Analysis of the Solution Features • Spearman rank correlation coefficients of the solutions in an approximated Front – : positive correlation – : negative correlation – Gray scale: absolute value of correlation • An example for an approximated Pareto front of MOCell using the NR approach in the ms2 instance mak e1 e2 e3 e4 e5 e6 e7 t1 t2 t3 t4 t5 t6 t7 t8 t9 t10 t11 t12 t13 t14 t15 t16 t17 t18 t19 t20 t21 t22 t23 t24 t25 t26 t27 t28 t29 cost mak e1 e2 e3 e4 e5 e6 e7 t1 t2 t3 t4 t5 t6 t7 t8 t9 t10 t11 t12 t13 t14 t15 t16 t17 t18 t19 t20 t21 t22 t23 t24 t25 t26 t27 t28 Fig. 3. Correlations between cost, duration Correlation between average team sizes for the different tasks Correlation between objectives and average team sizes Correlation between average employee parallelization and average team sizes Correlation between average employee parallelization for different employees Correlation between objectives and average employee parallelization
- 39. 39 / 47First International Summer School on SBSE, Cádiz, june/july 2016 Introduction Basic Formulation Multi-Objective Formulation Robust Formulation Preference-Based Formulation Conclusions & Future Work Results: Analysis of the Solution Features mak e1 e2 e3 e4 e5 e6 e7 t1 t2 t3 t4 t5 t6 t7 t8 t9 t10 t11 t12 t13 t14 t15 t16 t17 t18 t19 t20 t21 t22 t23 t24 t25 t26 t27 t28 t29 cost mak e1 e2 e3 e4 e5 e6 e7 t1 t2 t3 t4 t5 t6 t7 t8 t9 t10 t11 t12 t13 t14 t15 t16 t17 t18 t19 t20 t21 t22 t23 t24 t25 t26 t27 t28 Fig. 3. Correlations between cost, duration • Increasing the size of the working teams the makespan is reduced • Employee e3 is the only one able to perform a task in the critical path • No correlation is observed in tasks for which only one employee can do the work
- 40. 40 / 47First International Summer School on SBSE, Cádiz, june/july 2016 Introduction Basic Formulation Multi-Objective Formulation Robust Formulation Preference-Based Formulation Conclusions & Future Work
- 41. 41 / 47First International Summer School on SBSE, Cádiz, june/july 2016 Introduction Basic Formulation Multi-Objective Formulation Robust Formulation Preference-Based Formulation Conclusions & Future Work Introducción Propuesta Estudio experimental Conclusión Problema de optimización multiobjetivo Optimización multiobjetivo: metodologías Enfoque basado en punto de referencia Planificación de proyectos software ominancia de Pareto ficiente o Pareto óptima si @x0 2 S tal que x0 x Z=f (S) Z = f(S)f2 f1 o de soluciones Pareto óptimas ! frente óptimo de Pareto • Sometimes the decision maker is not interested in the whole Pareto front… Introducción Propuesta Estudio experimental Conclusión Problema de optimización multiobjetivo Optimización multiobjetivo: metodologías Enfoque basado en punto de referencia Planificación de proyectos software Preferencias mediante punto de referencia Punto de referencia alcanzable Punto de referencia inalcanzable … only in a region of the objective space The algorithm can save computational effort if it focuses on the region of interest Expressing Preferences in Objective Space
- 42. 42 / 47First International Summer School on SBSE, Cádiz, june/july 2016 Introduction Basic Formulation Multi-Objective Formulation Robust Formulation Preference-Based Formulation Conclusions & Future Work • The region of interest can be determined by a single point in the objective space: the reference point (a)q alcanzable rection based NSGA-II (RD-NSGA-II) [9] incorpora a NSGA-II una metodolog´ıa extra´ıda de MCDM de- nominada direcci´on de referencia [24]. A partir de un punto del espacio de objetivos y un punto de re- ferencia proporcionado por el decisor se define una direcci´on de referencia, considerando la diferencia entre ambos. Sobre ´esta se definen puntos de re- ferencia equidistantes que son proyectados sobre el frente ´optimo de Pareto mediante la funci´on escala- rizada de logro. Otra idea extra´ıda de MCDM, de- nominada Light Beam Search [25], ha sido utilizada en [10] con su integraci´on en NSGA-II. interactive MOEA/D (iMOEA/D) es un enfoque interactivo de MOEA/D propuesto en [11]. Tras un n´umero deter- minado de generaciones se muestra un conjunto de soluciones al decisor, que especifica sus preferencias sobre ´estas. El conjunto de pesos usado en MOEA/D para optimizar m´ultiples funciones de logro es aco- tado al vecindario de las soluciones determinadas como preferidas. As´ı, el proceso de b´usqueda se orienta progresivamente hacia la regi´on de inter´es (a)q alcanzable (b)q inalcanzable Fig. 1. Ilustraci´on de la m´etrica HVq. un punto ferencia p direcci´on entre amb ferencia e frente ´opt rizada de nominada en [10] co MOEA/D MOEA/D minado d soluciones sobre ´esta para optim tado al v como pre orienta pr del frente La form planificaci presentad sonas invo cada empl salario po junto de t son defini riando j d cada tarea que se co tas preced dencia de G(T, A) cu Reachable reference point Unreachable reference point Hypervolume restricted to the interest region Expressing Preferences in Objective Space
- 43. 43 / 47First International Summer School on SBSE, Cádiz, june/july 2016 Introduction Basic Formulation Multi-Objective Formulation Robust Formulation Preference-Based Formulation Conclusions & Future Work • Some algorithms to solve the problem – WASF-GA – g-NSGA-II (based on g-dominance) – P-MOGA (similar to WASF-GA) Algorithms
- 44. 44 / 47First International Summer School on SBSE, Cádiz, june/july 2016 Introduction Basic Formulation Multi-Objective Formulation Robust Formulation Preference-Based Formulation Conclusions & Future Work • If the decision maker is available, he can interactively guide the search by defining different reference points Introducción Propuesta Estudio experimental Conclusión Preferencias en el problema SPS Un enfoque interactivo en el problema SPS Interactive SPS Un enfoque interactivo en el problema SPS Inicialmente se aproxima el frente óptimo de Pareto. En la interacción con el decisor (DM), éste determina q. Con el enfoque interactivo, el DM adquiere conocimiento sobre el problema. Rubén Saborido y Francisco Chicano MAEB 2015, Mérida, España, Febrero de 2015 q Introducción Propuesta Estudio experimental Conclusión Preferencias en el problema SPS Un enfoque interactivo en el problema SPS Interactive SPS Un enfoque interactivo en el problema SPS Inicialmente se aproxima el frente óptimo de Pareto. En la interacción con el decisor (DM), éste determina q. Con el enfoque interactivo, el DM adquiere conocimiento sobre el problema. Rubén Saborido y Francisco Chicano MAEB 2015, Mérida, España, Febrero de 2015 Interaction with Decision Maker
- 45. 45 / 47First International Summer School on SBSE, Cádiz, june/july 2016 Introduction Basic Formulation Multi-Objective Formulation Robust Formulation Preference-Based Formulation Conclusions & Future Work • We developed a tool for interactive preference-based resolution Introducción Propuesta Estudio experimental Conclusión Preferencias en el problema SPS Un enfoque interactivo en el problema SPS Interactive SPS Interfaz gráfica de usuario de iSPS Rubén Saborido y Francisco Chicano MAEB 2015, Mérida, España, Febrero de 2015 Demo Software Tool
- 46. 46 / 47First International Summer School on SBSE, Cádiz, june/july 2016 Introduction Basic Formulation Multi-Objective Formulation Robust Formulation Preference-Based Formulation Conclusions & Future Work • Search algorithms are useful to take decisions at the management level • Some published ideas have been shown in this presentation… • ...but much more opportunities are waiting for us – New algorithmic proposals – More realistic models – ... – … and real data Concluding Remarks
- 47. 47 / 47First International Summer School on SBSE, Cádiz, june/july 2016 Thanks for your attention !!! Search-based Software Project Scheduling
- 48. 48 / 47First International Summer School on SBSE, Cádiz, june/july 2016 Introduction Basic Formulation Multi-Objective Formulation Robust Formulation Preference-Based Formulation Conclusions & Future Work Employees Hit rate Duration E*pdur 5 87 21,880,91 109,404,54 10 65 11,270,32 112,743,17 15 49 7,730,20 115,902,95 20 51 5,880,14 117,562,74 • Duration decreases as number of employee increases First instances group Resultados
- 49. 49 / 47First International Summer School on SBSE, Cádiz, june/july 2016 Introduction Basic Formulation Multi-Objective Formulation Robust Formulation Preference-Based Formulation Conclusions & Future Work Tareas Tasa éxito Coste Duración pcost / pdur 10 73 9800000,00 21,840,87 44944,341720,76 20 33 26000000,00 58,293,76 44748,122265,24 30 0 - - - • La duración aumenta con el número de tareas • La duración disminuye al aumentar el número de empleados Second group of instances Resultados
- 50. 50 / 47First International Summer School on SBSE, Cádiz, june/july 2016 Introduction Basic Formulation Multi-Objective Formulation Robust Formulation Preference-Based Formulation Conclusions & Future Work Tareas Tasa éxito Coste Duración pcost / pdur 10 73 9800000,00 21,840,87 44944,341720,76 20 33 26000000,00 58,293,76 44748,122265,24 30 0 - - - • La duración aumenta con el número de tareas • La duración disminuye al aumentar el número de empleados Segundo grupo de instancias Resultados E.Alba & F. Chicano, Software Project Managementwith GAs, InformationSciences 177,pp. 2380-2401,2007 Conclusiones y trabajo futuro Metodología y resultados FundamentosIntroducción Planif. de proyectos sw Generación de casos de prueba Búsqueda de errores de seguridad 26 / 57
- 51. 51 / 47First International Summer School on SBSE, Cádiz, june/july 2016 Introduction Basic Formulation Multi-Objective Formulation Robust Formulation Preference-Based Formulation Conclusions & Future Work Habilidades Tasa éxito Duración pcost / pdur 2 39 21,710,97 45230,221957,89 4 53 21,770,75 45068,661535,53 6 77 21,980,84 44651,291593,47 8 66 22,000,87 44617,011717,67 10 75 22,111,15 44426,932051,03 • Asignación más eficiente con plantilla especializada • La duración aumenta con el número de tareas • La duración disminuye al aumentar el número de empleados Tercer grupo de instancias Resultados E.Alba & F. Chicano, Software Project Managementwith GAs, InformationSciences 177,pp. 2380-2401,2007 Conclusiones y trabajo futuro Metodología y resultados FundamentosIntroducción Planif. de proyectos sw Generación de casos de prueba Búsqueda de errores de seguridad 26 / 57
- 52. 52 / 47First International Summer School on SBSE, Cádiz, june/july 2016 Introduction Basic Formulation Multi-Objective Formulation Robust Formulation Preference-Based Formulation Conclusions & Future Work La duración del proyecto se reduce con más empleados 84 97 100 76 El coste del proyecto aumenta con las tareas Resultados Cuarto grupo de instancias E.Alba & F. Chicano, Managementof Software Projectswith GAs, MIC 2005,pp. 13-18 6-7 habilidades por empleado Conclusiones y trabajo futuro Metodología y resultados FundamentosIntroducción Planif. de proyectos sw Generación de casos de prueba Búsqueda de errores de seguridad 27 / 57
- 53. 53 / 47First International Summer School on SBSE, Cádiz, june/july 2016 Introduction Basic Formulation Multi-Objective Formulation Robust Formulation Preference-Based Formulation Conclusions & Future Work Algorithms: NSGA-II procedure and a density estimator known as crowding distance ( see [19]). Finally, the population is updated with the best ind are repeated until the termination condition is fulfilled. Algorithm 1 Pseudocode of NSGA-II. 1: proc Input:(nsga-II) //Algorithm parameters in ‘nsga-II’ 2: P Initialize Population() // P = population 3: Q // Q = auxiliary population 4: while not Termination Condition() do 5: for i to (nsga-II.popSize / 2) do 6: parents Selection(P) 7: offspring Recombination(nsga-II.Pc,parents) 8: offspring Mutation(nsga-II.Pm,offspring) 9: Evaluate Fitness(offspring) 10: Insert(offspring,Q) 11: end for 12: R P Q 13: Ranking And Crowding(nsga-II, R) 14: P Select Best Individuals(nsga-II, R) 15: end while 16: end proc
- 54. 54 / 47First International Summer School on SBSE, Cádiz, june/july 2016 Introduction Basic Formulation Multi-Objective Formulation Robust Formulation Preference-Based Formulation Conclusions & Future Work Algorithms: PAES Algorithm 4 Pseudocode of PAES. 1: proc Input:(paes) //Algorithm parameters in ‘paes’ 2: archive 3: currentSolution Create Solution(paes) // Creates an initial solution 4: while not Termination Condition() do 5: mutatedSolution Mutation(currentSolution) 6: Evaluate Fitness(mutatedSolution) 7: if IsDominated(currentSolution, mutatedSolution) then 8: currentSolution mutatedSolution 9: else 10: if Solutions Are Nondominated(currentSolution, mutatedSolution) then 11: Insert(archive, mutatedSolution) 12: currentSolution Select(paes, archive) 13: end if 14: end if 15: end while 16: end proc