A Research on Measuring and Reducing Problem Complexity to Increase System Affordability: From Theory to Practice
Download as PPTX, PDF1 like413 views
The research presented at the 13th Annual Conference on Systems Engineering Research focuses on measuring and reducing problem complexity to enhance system affordability. It discusses methodologies to improve requirements engineering and explores the impact of requirement excess and categorization on system design and performance. The findings include empirical results demonstrating how structured approaches can reduce unnecessary requirements and improve system outcomes.
A Research on Measuring and Reducing Problem Complexity to Increase System Affordability: From Theory to Practice
- 1. CSER 2015 March 18-19, 2015 1 A Research on Measuring and Reducing Problem Complexity to Increase System Affordability From Theory to Practice By Alejandro Salado and Roshanak Nilchiani 13th Annual Conference on Systems Engineering Research (CSER) March 18, 2015 Stevens Institute of Technology Hoboken, NJ www.stevens.edu/sse/CSER2015org
- 2. CSER 2015 March 18-19, 2015 2 System affordbility AFFORDABILITY PERCEIVED VALUE BUDGET Perceived benefit Actual benefit Cost Business Marketing Technology push Process improvement Labor Outsourcing Procurement strategies Investment System development & operation DfE DfA Psychology-based design Spiral Context/ Environment Awareness Lobby Market change Economic situation Laws and regulations Awareness Lobby Competition Design&Develop.Tradespacedefinition Use cases Pugh matrices Spiral, agile, lean Other tradit. SE De-scope needs Use cases Agile, lean Pugh matrices Other tradit. SE NbC Tradespace maximization/adaptation Certification Quality Process improvement 6-sigma Servicing Customer service Evidence-based deliverable Skunkworks Evidence-based deliverable Skunkworks DfR, DfA, DfE,DfS, DfF, Value-driven design DtC, DfC, TCO, Cost reviews, DfM, Complexity control Pareto Set Tradespace exploration Manuf.Ops.PM Manuf.: Manufacturing - Ops.: Operation - PM: Project Management
- 3. CSER 2015 March 18-19, 2015 3 Research design: An end-to-end approach
- 4. CSER 2015 March 18-19, 2015 4 Research design: An end-to-end approach
- 5. CSER 2015 March 18-19, 2015 5 Foundations: Theorems A. Salado, R. Nilchiani, and D. Verma, “A formal theory of requirements engineering: stakeholder needs, system requirements, solution spaces, and requirements’ qualities,” unpublished. A. Salado and R. Nilchiani, “A mathematical justification for increasing the size of the solution space to improve the probabilities of designing compliant and affordable systems,” unpublished. VARIABLES RELATION System requirements #Solution space #Solution space #Solution space #Solution space Conflicting requirements Difficulty of compliance Difficulty of affordability Monotonic Monotonic Monotonic Monotonic
- 6. CSER 2015 March 18-19, 2015 6 Excess requirements need to be compensated with effort... A. Salado and R. Nilchiani, “Increasing the probability of developing affordable systems by maximizing and adapting the solution space,” Procedia Computer Science, Vol. 28, 2014, pp. 547-554.
- 7. CSER 2015 March 18-19, 2015 7 Consequence of foundations: Problem complexity A. Salado and R. Nilchiani, “The Concept of Problem Complexity,” Procedia Computer Science, Vol. 28, 2014, pp. 537-546.
- 8. CSER 2015 March 18-19, 2015 8 From foundations to practice Traditionally… Requirement categories employed to facilitate completeness of a req set Written from a designer perspective or procurement perspective Sets of more tan 1,000 requirements, where each requirement COSTS 1. Because of their individual management 2. Because of their effect in unnecessarily reducing the solution space …but feel for completeness (not completeness) is paid at a high price Mix product and process requirements Facilitate redundant and overlapping requirements Facilitate elicitation of solution constraints, not problem definition Facilitate elicitation of requirements to enabling systems, not to the system
- 9. CSER 2015 March 18-19, 2015 9 An excerpt from an actual space system 35 requirements! A. Salado and R. Nilchiani,“A categorization model of requirements based on Max-Neef‘s model of human needs,“ Syst. Eng., 17:348-360,2014
- 10. CSER 2015 March 18-19, 2015 10 A method for practitioners Value level Functions (Do) Performance (Being) Resources (Have) Interaction (Interact) Break- event Req. 1 Req. 2 Req. 3 Req. 4 Req. 5 Req. 6 Req. 7 Goal Req. 8 Req. 9 Wish Req. 10 Req. 11 Req. 12 Functional requirements (Do) What the system does in essence, which includes what it accepts and what it delivers Performance requirements (Being): How well the system does it, which includes performance related to functions the system performs or characteristics of the system on its own, such as –ilities Resource requirements (Have): What the system uses to transform what it accepts in what it delivers Interaction requirements (Interact): Where the system does it, which includes any type of operation during its life-cycle. A partition! No overlap of categories Full system definition Requirements as subsets Avoids flaw of individual prioritization Reflects value sets to stakeholders Bigger picture understsanding Facilitates relating functions to their performance, resources, and environment Resources is more Facilitates allocation of unnecessary constraints for easy identification A. Salado and R. Nilchiani,“A categorization model of requirements based on Max-Neef‘s model of human needs,“ Syst. Eng., 17:348- 360,2014
- 11. CSER 2015 March 18-19, 2015 11 Measuring its effectiveness: an experiment with practitioners INCLUSION Practicing SE Non-practicing SE >5y SE exp Researcher SE EXCLUSION Students Not large-scale systems Non SE Non-parametric Mann-Whitney U Test 95% confidence / alfa = 0.05 ORGANIZATION 13 test + 8 control + 3 discarded Random assignment Isolation NON-MANIPULATED INDEP. Experience/Competence Specific knowledge DEPENDENT Amount of constraints Amount of inapplicable reqs Completeness INDEPENDENT Categorization method A. Salado and R. Nilchiani,“Reducing excess requirements through orthogonal categorizations: results of a factorial experiment,“ unpublsihed, 2014.
- 12. CSER 2015 March 18-19, 2015 12 Measuring its effectiveness: an experiment with practitioners MOE Problem statement Self-perception on req quality Experience in systems engineering 1.H0 0.220 0.038 -0.055 2.H0 0.459* -0.166 -0.129 3.H0 -0.408 0.579 0.570 NULL HYPOTHESES SECOND H0 Both groups elicit the same amount of inapplicable reqs. THIRD H0 Both groups elicit the same amount of net requirements. FIRST H0 Both groups elicit the same amount of constraints. p = 0.001 0.025 0.804 Statistical power > 97% median = 26% vs 5% 18% vs 1% 24 vs 25 26% vs 0% Completeness Same level of completeness Unnecessary constraints Less unnecessary constraints Inapplicable requirements Less inapplicable requirements ! A. Salado and R. Nilchiani,“Reducing excess requirements through orthogonal categorizations: results of a factorial experiment,“ unpublsihed, 2014.
- 13. CSER 2015 March 18-19, 2015 13 Identifying conflicting requirements Expert assessment ? MBSE - Low effectiveness - Low effort - Before architecture - High effectiveness - High effort - Design exists
- 14. CSER 2015 March 18-19, 2015 14 Tension matrix and elemental decomposition Reqs. Resources Phases of matter Elemental decomposition Laws of physics Laws of society Logical r7 r8 r9 S L G V T P v L1 L2 L3 F r1 X ↑ Methods in chapter 2 r2 X ↑ ↓ r3 X P r4 ↓ r5 X ↑ ↓ ↑ r6 X ↑ R r7 r8 ↓ r9 I r10 X r11 X ↓ ↓ r12 X Heuristics Targeted modeling Elemental decomposition
- 15. CSER 2015 March 18-19, 2015 15 Identifying conflicting requirements NEEDS - IR band imaging - X band imaging - PtP secure coms r31 r32 S L G V Pc Temp OpTime Rfout Rfnoise R0 Data The satellite shall perform X-band operations at 30° incidence angle. The satellite shall operate for more than 5 X X L I The satellite shall have a reliability higher L L The satellite shall transmit communication services at 7.75 GHz. I The satellite shall transmit image data at I The satellite shall receive communication services at 400 MHz. L The satellite shall transmit image data with EIRP higher than 30 dBW. I I The satellite shall transmit communication services with EIRP higher than 35 dBW. I I The satellite shall have a G/T higher than - 9 dB/K for communication services. L The satellite shall transmit telemetry data I The satellite shall receive command data L The satellite shall transmit telemetry data with EIRP higher than 13 dBW. I I The satellite shall have a G/T for receiving command data higher than -49 L The satellite shall image the Earth with spatial resolution better than 30 m. Note: applicable to IR band. X X L I The satellite shall image the Earth with a field of view higher than 1.22°. Note: applicable to IR band. I The satellite shall image the Earth with SNR higher than 125 Note: applicable to IR band. L The satellite shall have a radiometric accuracy better than 2 unit. Note: applicable to IR band. The satellite shall image the Earth with spatial resolution better than 4 m. Note: applicable to X band in range and azimuth directions. I The satellite shall image the Earth with a swath of no less than 40 km. X I The satellite shall have a lower than -18 I I The satellite shall store image data for up to 4 h. X X I Req ID Requirement Elementary decomposition Laws of physics Resources Phases of matter PerformanceCategory 0 1 2 0 20 40 60 Utility IR spatial resolution (m) COMPARATIVE ANALYSIS Benchmark vs Tension matrix 1.5 1.55 1.6 1.65 1.7 1.75 1.8 x 10 6 0.5 0.52 0.54 0.56 0.58 0.6 0.62 0.64 0.66 0.68 Cost (k$) Utility
- 16. CSER 2015 March 18-19, 2015 16 Wrapping up! Continuous satisfaction levels? Absolute calibration? Empirical metrics for improvement?