Sayyad slides ase13_v4
0 likes478 views
The document discusses scalable product line configuration, emphasizing the challenges in managing complex feature models and the necessity of multi-objective optimization to meet user preferences while respecting domain constraints. It outlines methodologies for feature model representation, optimization algorithms like NSGA-II and IBEA, and presents experimental results demonstrating the effectiveness of incorporating feature fixing and leveraging solvers like Z3 to enhance solution quality. The research is funded by the Qatar National Research Fund and aims to advance understanding in the field of software product lines.
![Feature
Model
Repository
Size
?
290
Features,
421
Rules
cardinal
=
2.26x1049
[Pohl
‘11]
6](https://image.slidesharecdn.com/sayyadslidesase13v4-131115194956-phpapp01/85/Sayyad-slides-ase13_v4-6-320.jpg)
![Two
Algorithms
1) NSGA-‐II
[Deb
et
al.
‘02]
Non-‐dominated
Sor4ng
Gene4c
Algorithm
Binary
Dominance
2)
IBEA
[Zitzler
and
Kunzli
‘04]
Indicator-‐Based
Evolu4onary
Algorithm
Con4nuous
Dominance
21](https://image.slidesharecdn.com/sayyadslidesase13v4-131115194956-phpapp01/85/Sayyad-slides-ase13_v4-21-320.jpg)
Sayyad slides ase13_v4
- 1. Scalable Product Line Configura4on: A Straw to Break the Camel’s Back Abdel Salam Sayyad Joseph Ingram Tim Menzies Hany Ammar
- 2. Sound bites Real SoGware Product Lines… … are big, complex … representa4ve feature models are now available for research … scale up or die ! What does it take? Mul4-‐objec4ve problem formula4on An op4mizer that relies heavily on user preferences Respect domain constraints, and... … A Straw to Break the Camel’s Back 2
- 3. Roadmap ① State of the Art ② Experiment & Results ③ Conclusion
- 4. Roadmap ① State of the Art ② Experiment & Results ③ Conclusion
- 5. Feature Models • Feature models = a lightweight method for defining a space of op4ons • De facto standard for modeling variability Cross-‐Tree Constraints Size ? 10 Features, 8 Rules Cross-‐Tree Constraints 5
- 6. Feature Model Repository Size ? 290 Features, 421 Rules cardinal = 2.26x1049 [Pohl ‘11] 6
- 7. Feature Models of Real SoGware Projects Size ? 6888 Features; 344,000 Rules cardinal? Forgeddabou4t! 7
- 8. Size isn’t all… Property Size Constraints Feature Groups Leaves SPLOT Linux Significantly smaller Significantly larger Significantly less Significantly more High ra4o Low ra4o Deeper Shallower T. Berger, S. She, R. Lotufo, A. Wasowski, and K. Czarnecki, "A Study of Variability Models and Languages in the Systems SoGware Domain," IEEE Tran So< Eng. 2013 8
- 9. A Mul4-‐Objec4ve Formula4on Mul4-‐objec4ve op4miza4on = naviga4ng compe4ng concerns – Success criteria = choose features that achieve the user preferences! Suppose each feature had the following metrics: 1. Boolean USED_BEFORE? 2. Integer DEFECTS 3. Real COST Show me the space of “best op4ons” according to the objec4ves: 1. 2. 3. 4. 5. That sa4sfies most domain constraints (0 ≤ #viola4ons ≤ 100%) That offers most features That we have used most before Using features with least known defects Using features with least cost 9
- 10. No single “op4mum” solu4on The Pareto Front Higher-‐level Decision Making The Chosen Solu4on 10
- 11. Features Single-‐obj Mul4-‐obj Linux (LVAT) 6888 State of the Art 544 SPLOT 290 9 Objec4ves 11
- 12. Features 6888 State of the Art Mul4-‐obj Linux (LVAT) Single-‐obj 544 SPLOT 290 Single-‐obj, CSP 9 Up to 25 features Exponen[al [me Benavides ‘05 Objec4ves 12
- 13. Features State of the Art 6888 Mul4-‐obj Linux (LVAT) Single-‐obj 544 SPLOT 290 9 Pohl ‘11 Benavides ‘05 BDD, SAT, CSP, 90 models “All products” not a`empted if cardinal > 3x106 Objec4ves 13
- 14. Features State of the Art 6888 Mul4-‐obj Linux (LVAT) Single-‐obj 544 SPLOT 290 9 Pohl ‘11 Benavides ‘05 Lopez-‐ Herrejon ‘11 Fixing model inconsistencies 27 min. for 94 features Objec4ves 14
- 15. Features State of the Art 6888 Mul4-‐obj Linux (LVAT) Single-‐obj 544 SPLOT 290 9 Pohl ‘11 Benavides ‘05 Lopez-‐ Herrejon ‘11 Sayyad ’13a Mul[-‐obj configura[on, Z3 >15 days for E-‐Shop Objec4ves 15
- 16. State of the Art Features 6888 Mul4-‐obj Linux (LVAT) Single-‐obj 544 White ‘07, ‘08, 09a, 09b, Shi ‘10, Guo ‘11 SPLOT 290 9 Pohl ‘11 Benavides ‘05 Lopez-‐ Herrejon ‘11 Scale up with randomly-‐ generated feature models Sayyad ’13a Velazco ‘13 Objec4ves 16
- 17. State of the Art Features Linux (LVAT) 6888 544 SPLOT 290 9 Single-‐obj Mul4-‐obj Johansen ‘11 Test covering arrays, [med out on all ops for Linux 6888 features (and some ops on smaller FMs) Pohl ‘11 Benavides ‘05 Lopez-‐ Herrejon ‘11 Sayyad ’13a Velazco ‘13 Objec4ves 17
- 18. State of the Art Features Linux (LVAT) 6888 Single-‐obj Johansen ‘11 Mul4-‐obj Henard ‘12 Test covering arrays 20 hours for Linux 6888 features 6-‐wise coverage 544 SPLOT 290 9 Pohl ‘11 Benavides ‘05 Lopez-‐ Herrejon ‘11 Sayyad ’13a Velazco ‘13 Objec4ves 18
- 19. State of the Art Features Linux (LVAT) 6888 Single-‐obj Johansen ‘11 Mul4-‐obj Henard ‘12 Sayyad ’13b Mul[-‐obj configura[on, IBEA 30 configs in 30 minutes for Linux 6888 features 544 SPLOT 290 9 Pohl ‘11 Benavides ‘05 Lopez-‐ Herrejon ‘11 Sayyad ’13a Velazco ‘13 Objec4ves 19
- 20. Roadmap ① State of the Art ② Experiment & Results ③ Conclusion
- 21. Two Algorithms 1) NSGA-‐II [Deb et al. ‘02] Non-‐dominated Sor4ng Gene4c Algorithm Binary Dominance 2) IBEA [Zitzler and Kunzli ‘04] Indicator-‐Based Evolu4onary Algorithm Con4nuous Dominance 21
- 22. 7 Feature Models 22
- 23. Feature Fixing • Look for mandatory or dead features. • Fix those feature in the evolu4on. • Skip rules with only those features. 23
- 24. Experiment Low Crossover & Muta[on rates Stopping Criteria 24
- 25. HV = Hypervolume of dominated region %Correct = % of solu4ons that are fully correct (out of 300 solu4ons) 25
- 26. Comparing %Correct i.e. % of solu4ons that are fully correct (out of 300 solu4ons) NSGA-‐II No FF 100% NSGA-‐II with FF 80% IBEA No FF IBEA with FF 60% 40% 20% 0% ToyBox axTLS eCos FreeBSD Fiasco uClinux Linux X86 26
- 27. How to crack the nut? • What if we told IBEA what good solu4ons look like? • Good solu4on? … A correct solu4on, with: • Minimal skeleton? • Maximum features? • Then what? • Seed the ini4al popula4on with a pre-‐calculated “good” solu4on. 27
- 28. • How to generate good (correct) solu4ons? • Z3 SMT (Sa4sfiability Module Theory) Solver • Two-‐objec4ve IBEA, min viola4ons, max features • Z3: super fast, low on selected features • 2-‐obj IBEA: slow, but feature-‐rich 28
- 29. Correct solu4ons aGer 30 minutes for Linux Kernel feature model 29
- 30. Correct solu4ons aGer 30 minutes for Linux Kernel feature model … A Straw to Break the Camel’s Back 30
- 31. More recent result • added the “PULL” trick (give more objec4ve weight to constraint viola4ons) 31
- 32. Why is this interes4ng? • 20 correct solu4ons to choose from within 2 minutes 32
- 33. Do I have the user’s aven4on? • 2 seconds… • to stay focused • 10 seconds… • to stay on task • few minutes? • To provide 4mely input at a mee4ng 33
- 34. Roadmap ① State of the Art ② Experiment & Results ③ Conclusion
- 35. Sound bites Scalability Method innova[on is key Problem Formula4on Give the best (comprehensive yet concise) picture to the user. IBEA with Feature Fixing Con[nuous dominance Respect domain constraints Enter Z3? Acknowledgment This research work was funded by the Qatar Na[onal Research Fund (QNRF) under the Na[onal Priori[es Research Program (NPRP) Grant No.: 09-‐1205-‐2-‐470. One day we will build that bridge! 35