Expert mining compsac-2014
0 likes491 views
The document outlines a workshop procedure for assessing risk indicators in open source software adoption, focusing on the RISCOSS analytics platform and tactical workshops involving domain experts. It details the steps involved in preparing for these workshops, managing risks, and evaluating scenarios using statistical tools and expert judgments. Additionally, it highlights applications and lessons learned from past workshops conducted in various countries.
![10
Tacticalworkshop
protocol
Workshop procedure
Pre-TasksPart II: Community dinamicsPart I : Community dataTacticalWorkshop OrganizerDomainExpert6.Make presentation of RISCOSS project summary. 7.Explain the RISCOSS analytics platform. 8.Explain the tactical workshop Part I and Part II9.Explain the risk driver selection WS Excel File Excel File: risk driver selection WSTabsTimelinessActivenessCommunity10.Study the use case scenario 11.Assess the use case scenario overall state 12.Determine the level of the risk indicator 13.Study the use case scenario 14.Assess the use case scenario overall state 15.Determine the level of the community risk indicatorPDF File:KPA RISCOSS Analytics [more risk drivers] [not more risk drivers] RISCOSS Analytics Team1.Determine drivers and risk indicators2.Construct Bayesian networks3.Define scenarios4.Identify and contact experts5.Workshops planning and preparation16.Send the scenarios judgementPopulate BN nodes](https://image.slidesharecdn.com/expertmining-compsac-2014-140825162147-phpapp02/85/Expert-mining-compsac-2014-10-320.jpg)
![25
Related Work
Related Work
Delphi method
(QUELCE)
QuantifyingUncertaintyinEarlyCostEstimation[2]
Reliableconsensusofopinionofagroupofexperts[1]
[1]N.DalkeyandO.Helmer,“AnexperimentalapplicationoftheDelphimethodtotheuseofexperts,”.
[2]R.W.Ferguson,D.Goldenson,J.M.McCurley,R.W.Stoddard,andD.Zubrow,“QuantifyingUncertaintyinEarlyLifecycleCostEstimation(QUELCE),”](https://image.slidesharecdn.com/expertmining-compsac-2014-140825162147-phpapp02/85/Expert-mining-compsac-2014-25-320.jpg)
![27
Domain
Expert
Conclusions and further work
Conclusions and further work
Pre-Tasks
Part II:
Community
dinamics
Part I :
Community data
Tactical
Workshop
Organizer
Domain
Expert
6.Make
presentation
of RISCOSS
project
summary.
7.Explain the
RISCOSS
analytics
platform.
8.Explain the
tactical
workshop
Part I and
Part II
9.Explain the
risk driver
selection WS
Excel File
Excel File: risk driver
selection WS
Tabs
Timeliness
Activeness
Community
10.Study the
use case
scenario
11.Assess the
use case
scenario
overall state
12.Determine
the level of
the risk
indicator
13.Study the
use case
scenario
14.Assess the
use case
scenario
overall state
15.Determine
the level of
the
community
risk indicator
PDF File:KPA
RISCOSS
Analytics
[more
risk drivers]
[not more
risk drivers]
RISCOSS
Analytics
Team
1.Determine
drivers and
risk indicators
2.Construct
Bayesian
networks
3.Define
scenarios
4.Identify and
contact
experts
5.Workshops
planning and
preparation
16.Send the
scenarios
judgement
Populate BN
nodes
Domain
Expert SNA
Project Site
Code Version
Repository
Bug Tracker Mailing List IRC
Step-by-step
protocol
Data used to
construct
Bayesian
networks
Future work
Combines opinion
of domain experts
with OSS raw data
Empirical
application
&
lessons](https://image.slidesharecdn.com/expertmining-compsac-2014-140825162147-phpapp02/85/Expert-mining-compsac-2014-27-320.jpg)
Expert mining compsac-2014
- 1. Expert Mining for Evaluating Risk Indicators ScenariosOscar Franco-Bedoya, Dolors Costal, Soraya Hidalgo, Ron Ben-JacobMonday 21st July 2014
- 2. 2 Background Workshop procedure Generalization of the approach Applications and lessons learned Related work Conclusions and further work Outline
- 3. 3 Background
- 4. 4 Background
- 5. 5 Background Managing risk in open source adoption
- 6. 6 Identification Mitigation methods Management Advanced tools Provides Platform Methods OSS adoption projects To support a In Uses Ecosystem modeling Statistical tools Risk Management i.e. i.e. i.e. Bayesian Networks Social network analysis Expert scenarios assessment e.g. e.g. e.g. Background Risks Systematic protocol uses
- 7. 7 Background Project Site Code Version Repository Bug Tracker Mailing List IRC Ecosystem hubs Project indexes Social Media Twitter Facebook I Raw Data SNA Measures Risk Indicators: • Project • Community • Contextual II Indicators Scenario-based Assessment Domain Expert Business Analysis • Business goals III Business Goals 3-Layered RISCOSS approach Number of downloads Number of event references Centrality Number of open bugs
- 8. 8 Outline Background Workshop procedure Generalization of the approach Applications and lessons learned Related work Conclusions and further work Outline
- 9. 9 The RISCOSS tactical workshop is designed to permit experts to assess risk indicators Workshop procedure
- 10. 10 Tacticalworkshop protocol Workshop procedure Pre-TasksPart II: Community dinamicsPart I : Community dataTacticalWorkshop OrganizerDomainExpert6.Make presentation of RISCOSS project summary. 7.Explain the RISCOSS analytics platform. 8.Explain the tactical workshop Part I and Part II9.Explain the risk driver selection WS Excel File Excel File: risk driver selection WSTabsTimelinessActivenessCommunity10.Study the use case scenario 11.Assess the use case scenario overall state 12.Determine the level of the risk indicator 13.Study the use case scenario 14.Assess the use case scenario overall state 15.Determine the level of the community risk indicatorPDF File:KPA RISCOSS Analytics [more risk drivers] [not more risk drivers] RISCOSS Analytics Team1.Determine drivers and risk indicators2.Construct Bayesian networks3.Define scenarios4.Identify and contact experts5.Workshops planning and preparation16.Send the scenarios judgementPopulate BN nodes
- 11. 11 Pre-Tasks Tactical Workshop Organizer RISCOSS Analytics Team 1.Determine drivers and risk indicators 2.Construct Bayesian networks 3.Define scenarios 4.Identify and contact experts 5.Workshops planning and preparation Correspond to tasks that must be done before conducting the tactical workshop Workshop procedure
- 12. 12 Pre-Tasks Tactical Workshop Organizer RISCOSS Analytics Team 1.Determine drivers and risk indicators 2.Construct Bayesian networks 3.Define scenarios 4.Identify and contact experts 5.Workshops planning and preparation Workshop procedure Risk Driver Forum posts per day Forum messages per thread Mail per day Overall community size Number of developers involved Number of testers (individuals providing feedback) Number of companies using the software Companies supporting the project (adding to code) Risk Indicator Activeness
- 13. 13 Pre-Tasks Tactical Workshop Organizer RISCOSS Analytics Team 1.Determine drivers and risk indicators 2.Construct Bayesian networks 3.Define scenarios 4.Identify and contact experts 5.Workshops planning and preparation Workshop procedure
- 14. 14 Pre-Tasks Tactical Workshop Organizer RISCOSS Analytics Team 1.Determine drivers and risk indicators 2.Construct Bayesian networks 3.Define scenarios 4.Identify and contact experts 5.Workshops planning and preparation Workshop procedure
- 15. 15 Pre-Tasks Tactical Workshop Organizer RISCOSS Analytics Team 1.Determine drivers and risk indicators 2.Construct Bayesian networks 3.Define scenarios 4.Identify and contact experts 5.Workshops planning and preparation Workshop procedure
- 16. 16 The tactical workshops begin with an exposition about the main topics that will be covered Workshop procedure 6.Make presentation of RISCOSS project summary. 7.Explain the RISCOSS analytics platform. 8.Explain the tactical workshop Part I and Part II PDF File:KPA RISCOSS Analytics Tactical Workshop Organizer
- 17. 17 Workshop procedure 9.Explain the risk driver selection WS Excel File Excel File: risk driver selection WS Tabs Timeliness Activeness Community Tactical Workshop Organizer
- 18. 18 Workshop procedure Part I : Community data Domain Expert 10.Study the use case scenario 11.Assess the use case scenario overall state 12.Determine the level of the risk indicator
- 19. 19 Workshop procedure Part II: Community dinamics 13.Study the use case scenario 14.Assess the use case scenario overall state 15.Determine the level of the community risk indicator 16.Send the scenarios judgement Domain Expert
- 20. 20 Outline Outline Background Workshop procedure Generalization of the approach Applications and lessons learned Related work Conclusions and further work
- 21. 21 Generalization of the approach
- 22. 22 Outline Outline Background Workshop procedure Generalization of the approach Applications and lessons learned Related work Conclusions and further work
- 23. 23 Applications and lessons learned Applications and lessons learned We have conducted 10 technical workshops In private and official organizations and academic institutions. The experts were from different countries. France, Israel, Italy, Spain, and Netherlands There are some inconsistencies in the scenarios The scenarios were designed using random number generators While the domain experts are conducting the tactical workshops, The degree of "calibration" of their judgement improves
- 24. 24 Outline Outline Background Workshop procedure Generalization of the approach Applications and lessons learned Related work Conclusions and further work
- 25. 25 Related Work Related Work Delphi method (QUELCE) QuantifyingUncertaintyinEarlyCostEstimation[2] Reliableconsensusofopinionofagroupofexperts[1] [1]N.DalkeyandO.Helmer,“AnexperimentalapplicationoftheDelphimethodtotheuseofexperts,”. [2]R.W.Ferguson,D.Goldenson,J.M.McCurley,R.W.Stoddard,andD.Zubrow,“QuantifyingUncertaintyinEarlyLifecycleCostEstimation(QUELCE),”
- 26. 26 Outline Outline Background Workshop procedure Generalization of the approach Applications and lessons learned Related work Conclusions and further work
- 27. 27 Domain Expert Conclusions and further work Conclusions and further work Pre-Tasks Part II: Community dinamics Part I : Community data Tactical Workshop Organizer Domain Expert 6.Make presentation of RISCOSS project summary. 7.Explain the RISCOSS analytics platform. 8.Explain the tactical workshop Part I and Part II 9.Explain the risk driver selection WS Excel File Excel File: risk driver selection WS Tabs Timeliness Activeness Community 10.Study the use case scenario 11.Assess the use case scenario overall state 12.Determine the level of the risk indicator 13.Study the use case scenario 14.Assess the use case scenario overall state 15.Determine the level of the community risk indicator PDF File:KPA RISCOSS Analytics [more risk drivers] [not more risk drivers] RISCOSS Analytics Team 1.Determine drivers and risk indicators 2.Construct Bayesian networks 3.Define scenarios 4.Identify and contact experts 5.Workshops planning and preparation 16.Send the scenarios judgement Populate BN nodes Domain Expert SNA Project Site Code Version Repository Bug Tracker Mailing List IRC Step-by-step protocol Data used to construct Bayesian networks Future work Combines opinion of domain experts with OSS raw data Empirical application & lessons
- 28. 28 Outline Outline Background Workshop procedure Generalization of the approach Applications and lessons learned Related work Conclusions and further work
- 29. 29 SEeD4FI
- 30. Thanks for your attention Comments and Questions
- 31. 31 SEeD4FI