Final Survey On Rule Base Development Slideshare

Development and Verification of Rule Based Systems - a Survey of Developers Valentin Zacharias FZI Karlsruhe, Germany

Results from an Online Survey about Rule Base Development Agenda Participants Development Methodology Tools Development Debugging V&V Comparison to OO Development & Issues Hindering Dev. Comparison to last surveys (from 18 years ago) Hamilton, D., Kelley, K.: State-of-the-practice in knowledge-based system veri cation and validation. O'Leary, D. In: Design, Development and Validation of Expert Systems: A Survey of Developers.

Size Mean Median Standard Deviation PM for entire software 59 15 148 PM for rule base 9 5,5 15 Number of rules 1969 120 8693 Size of average rule 9,3 5 17 Size of largest rule 24 11 39 Rule developers 3 2 4 Other software developers 3 1 8 Domain experts that created rules 1,5 1 2 Domain experts as consultants 1,9 1 2,5 Domain experts for V&V 1,7 1 2,4 Others 0,6 0 1,6

Size 65000 learned rules for ‘disease event analysis’ 100 PM to build a system that determines the input parameters for a medical imaging system Mean Median Standard Deviation PM for entire software 59 15 148 PM for rule base 9 5,5 15 Number of rules 1969 120 8693 Size of average rule 9,3 5 17 Size of largest rule 24 11 39 Rule developers 3 2 4 Other software developers 3 1 8 Domain experts that created rules 1,5 1 2 Domain experts as consultants 1,9 1 2,5 Domain experts for V&V 1,7 1 2,4 Others 0,6 0 1,6

Methodologies “ CommonKADS is *the* methodology to build rule based systems”

Methodology – only 10PM+ Medical Best Practices Integration of Car Information Scheduling (Logistics) Credit Allocation

Methodology – only 10PM+ 18 Years Ago: “ The most frequent (40%) life cycle model used was the cyclic model [...]. However 22% of the respondents stated that no model was followed”

Tools used for Development “ I doubt that manually created rule bases will serve as the basis for the semantic web or other application areas”

Development Tools IDE without debugging Hard coded rules Schema aware XML Editor ‘ this [Text Editor] is only for viewing the results SWRL Tab

Verification & Debugging “ The correctness of rule bases is ensured with formal verification”

Debugging Paradigms for Rule Based Systems Procedural Debugging Stepwise execution of inference engine Algorithmic Debugging System identifies fault by asking user (oracle) about correctness of results of subcomputations Explanations Concise NL or graphical representation of justification for a result Why-Not Explanations Explanation also for missing conclusions Automatic Theory Revision Automatic correction of rule base Valentin Zacharias: The Debugging of Rule Bases, to appear in Handbook of Research on Emerging Rule-Based Languages and Technologies, IGI Global, Hershey (USA) 2009.

Debugging Tools Fair Isaac Blaze Advisor Gnu Prolog SWI Prolog Jboss Rules / Drools Jboss Rules / Drools F-Logic (SmartWeb) SWI Prolog? Visual Prolog BizTalk Jboss Rules / Drools Jess

Comparison To ‘Conventional Programming’ and Issues Hindering Development “ Runtime Performance and Expressivity are the problems hindering the development of rule based systems”

Comparison – Question How does the rule base and its development process compare to a ‘conventional’ program (created with procedural/object oriented languages) of similar size? Ease of change and maintenance Rule base superior; Comparable; Conventional program superior; Don’t know [...] Computer average with: * Rule base superior = 1 * Comparable = 0 * Conventional program sup= -1 * Ignore ‘Don’t know’

Issues Hindering Development – Question What were the most important issues in the development of the rule base? Rule expressivity – could not (easily) represent what was needed Not an issue; Annoyance; Hindered development [...] Computer average with: * Not an issue = 0 * Annoyance = 1 * Hindered development = 2

Issues Hindering Development 18 Years Ago: Completeness of Knowledge Base Correctness of Knowledge Base System does not present possible opportunities to the user System is hard to use Results difficult to interpret Systems presents incorrect opportunities to the user Difficult to sequence rules correctly

„ This paper is not a (normal) scientific paper, it summarizes a survey [...]“ Little empirical data about the challenges facing actual rule base developers Little interest in the academic rule community at identifying and tackling practical problems? Meta Conclusion

Little academic interest in relevant (in particular agile) rule base development methodologies Debugging and finding faults as *the challenge* Tool support found wanting, possible motivation for rule interchange still most rules created manually, text editors widespread Conclusion

Attribution Pool Rules by Joe Shlabotnik on Flickr Waterfall by Sachman75 on Flickr Tools by docman on Flickr Bug by Chewy Chua on Flickr Obstacle by Guillaume Lemoine on Flickr Sun Conclusion by ecstaticist on Flickr End by bondywhat on Flickr Thanks for your Attention Valentin Zacharias [email_address] http://vzach.de

Final Survey On Rule Base Development Slideshare

More Related Content

What's hot (9)

Similar to Final Survey On Rule Base Development Slideshare (20)

Recently uploaded (20)

Final Survey On Rule Base Development Slideshare