CommonKADS knowledge modelling process
1 like3,154 views
The document discusses guidelines for constructing a knowledge model. It describes a process model with stages including knowledge identification, specification, and refinement. The stages involve activities like exploring information sources, choosing a task template, constructing a domain schema, and completing the knowledge model specification before validating and refining the knowledge bases. The goal is to support inexperienced knowledge modelers through a prescriptive yet flexible process.
CommonKADS knowledge modelling process
- 1. Knowledge Model Construction Process model & guidelines Knowledge elicitation techniques
- 2. Knowledge-model construction 2 Process & Product ■ so far: focus on knowledge model as product ■ bottleneck for inexperienced knowledge modelers ➤ how to undertake the process of model construction. ■ solution: process model ➤ as prescriptive as possible ➤ process elements: stage, activity, guideline, technique ■ but: modeling is constructive activity ➤ no single correct solution nor an optimal path ■ support through a number of guidelines that have proven to work well in practice. ■ knowledge modeling is specialized form of requirements specification ➤ general software engineering principles apply
- 3. Knowledge-model construction 3 Stages in Knowledge-Model Construction knowledge identification knowledge specification -‐ choose task template (provides initial task decomposition) -‐ construct initial domain conceptualization (main domain information types) -‐ complete know ledge-‐model specification (know ledge model w ith partial know ledge bases) -‐ domain familiarization (information sources, glossary, scenarios) -‐ list potential model components for reuse (task-‐ and domain-‐related components) -‐ validate know ledge model (paper simulation, prototype of reasoning system) -‐ know ledge-‐base refinement (complete the know ledge bases) knowledge refinement TYP IC AL AC TIVITIESS TAGES
- 4. Knowledge-model construction 4 Stage 1: Knowledge identification ■ goal ➤ survey the knowledge items ➤ prepare them for specification ■ input ➤ knowledge-intensive task selected ➤ main knowledge items identified. ➤ application task classified – assessment, configuration, combination of task types ■ activities ➤ explore and structure the information sources ➤ study the nature of the task in more detail
- 5. Knowledge-model construction 5 Exploring information sources ■ Factors ➤ Nature of the sources – well-understood?, theoretical basis? ➤ Diversity of the sources – no single information source (e.g. textbook or manual) – diverse sources may be conflicting – multiple experts is a risk factor. ■ Techniques ➤ text marking in key information sources ➤ some structured interviews to clarify perceived holes in domain ■ main problem: ➤ find balance between learning about the domain without becoming a full
- 6. Knowledge-model construction 6 Guidelines ■ Talk to people in the organization who have to talk to experts but are not experts themselves ■ Avoid diving into detailed, complicated theories unless the usefulness is proven ■ Construct a few typical scenarios which you understand at a global level ■ Never spend too much time on this activity. Two person weeks should be maximum.
- 7. Knowledge-model construction 7 Exploring the housing domain ■ Reading the two-weekly magazine in detail ➤ organizational goal of transparent procedure makes life easy ■ Reading the original report of the local government for setting up the house assignment procedure ➤ identification of detailed information about handling urgent cases ■ Short interviews/conversations ➤ staff member of organization ➤ two applicants (the “customers”)
- 8. Knowledge-model construction 8 Results exploration ■ Tangible ➤ Listing of domain knowledge sources, including a short characterization. ➤ Summaries of selected key texts. ➤ Glossary/lexicon ➤ Description of scenarios developed. ■ Intangible ➤ your own understanding of the domain – most important result
- 9. Knowledge-model construction 9 List potential components ■ goal: pave way for reusing components ■ two angles on reuse: ➤ Task dimension – check task type assigned in Task Model – build a list of task templates ➤ Domain dimension – type of the domain: e.g. technical domain – look for standardized descriptions AAT for art objects ontology libraries, reference models, product model libraries
- 10. Knowledge-model construction 10 Available components for the “housing” application ■ Task dimension: assessment templates ➤ CK book: single template ➤ assessment library of Valente and Loeckenhoff (1994) ■ Domain dimension ➤ data model of the applicant database ➤ data model of the residence database ➤ CK-book: generic domain schema
- 11. Knowledge-model construction 11 Stage 2: Knowledge specification ■ goal: complete specification of knowledge except for contents of domain models ➤ domain models need only to contain example instances ■ activities ➤ Choose a task template. ➤ Construct an initial domain conceptualization. ➤ Specify the three knowledge categories
- 12. Knowledge-model construction 12 Choose task template ■ baseline: strong preference for a knowledge model based on an existing application. ➤ efficient, quality assurance ■ selection criteria: features of application task ➤ nature of the output: fault category, plan ➤ nature of the inputs: kind of data available ➤ nature of the system: artifact, biological system ➤ constraints posed by the task environment: – required certainty, costs of observations.
- 13. Knowledge-model construction 13 Guidelines for template selection ■ prefer templates that have been used more than once ➤ empirical evidence ■ construct annotated inference structure (and domain schema) ■ if no template fits: question the knowledge-intensity of the task
- 14. Knowledge-model construction 14 Annotated inference structure “housing” application case abstracted case norms norm value decision abstract select match specify evaluate norm case-‐specific norms raw data about a residence and an applicant e.g. age, income, rent abstractions such as age category are added to the case criteria such as "rent fits income," "correct household size" a single criterion e.g. rent fits income e.g rent-‐fits income = true (or fals e) rules that only apply to one particular residence applicant is either eligible or not eligible for the residence
- 15. Knowledge-model construction 15 Construct initial domain schema ■ two parts in a schema: ➤ domain-specific conceptualization – not likely to change ➤ method-specific conceptualizations – only needed to solve a certain problem in a certain way. ■ output: schema should cover at least domain-specific conceptualizations
- 16. Knowledge-model construction 16 Initial housing schema res idence number: natural categroy: {starter-‐residence, followup-‐residence} build-‐type: {house, apartment} street-‐address: string city: string num-‐rooms: natural rent: number min-‐num-‐inhabitants: natural max-‐num-‐inhabitants: natural subsidy-‐type: subsidy-‐type-‐value surface-‐in-‐square-‐meters: natural floor: natural lift-‐available: boolean applicant registration-‐number: string applicant-‐type: {starter, existing-‐resident} name: string street-‐address: string city: string birth-‐date: date age: natural age-‐category: age-‐category-‐value gross-‐yearly-‐income: natural household-‐size: natural household-‐type: household-‐type-‐value res idence application application-‐date: string residence criterion truth-‐value: boolean correct house category correct household siz e rent fits income residence-‐specific constraints
- 17. Knowledge-model construction 17 Guidelines ■ use as much as possible existing data models: ➤ useful to use at least the same terminology basic constructs ➤ makes future cooperation/exchange easier ■ limit use of the knowledge-modeling language to concepts, sub- types and relations ➤ concentrate on "data" ➤ similar to building initial class model ■ If no existing data models can be found, use standard SE techniques for finding concepts and relations ➤ use “pruning” method ■ Constructing the initial domain conceptualization should be done in parallel with the choice of the task template ➤ otherwise: fake it
- 18. Knowledge-model construction 18 Complete model specification ■ Route 1: Middle-out ➤ Start with the inference knowledge ➤ Preferred approach ➤ Precondition: task template provides good approximation of inference structure. ■ Route 2: Middle-in ➤ Start in parallel with task decomposition and domain modeling ➤ More time-consuming ➤ Needed if task template is too coarse-grained
- 19. Knowledge-model construction 19 Middle-in and Middle-out D OMAI N S C H E MA concepts rela tions rule types Expressions in knowledge bases K NOW L E D G E BAS E D E FI NI T I ONS role mapping defines control over middle in middle out tasks and methods inference structure
- 20. Knowledge-model construction 20 Guidelines ■ inference structure is detailed enough, if the explanation it provides is sufficiently detailed ■ inference structure is detailed enough if it is easy to find for each inference a single type of domain knowledge that can act as a static role for this inference
- 21. Knowledge-model construction 21 Approach “housing” application ■ Good coverage by assessment template ➤ one adaptation is typical ■ Domain schema appears also applicable ➤ can also be annotated ■ Conclusion: middle-out approach
- 22. Knowledge-model construction 22 Task decomposition “housing” assess case abstract case match case assess trough abstract & match abstract method match method abstract specify select evaluate match tas k tas k method inference tas k tas k method
- 23. Knowledge-model construction 23 Completed domain schema “housing” residence application residence criterion residence decision residence abstraction residence decision rule residence requirement indicates has abstraction implies 1+ 1+ 1+
- 24. Knowledge-model construction 24 Guidelines for specifying task knowledge ■ begin with the control structure ➤ "heart" of the method ■ neglect details of working memory ➤ design issue ■ choose role names that clearly indicate role ➤ "modeling is naming" ■ do not include static knowledge roles ■ real-time applications: consider using a different representation than pseudo code ➤ but: usage of "receive"
- 25. Knowledge-model construction 25 Guidelines for specifying inference knowledge ■ Start with the graphical representation ■ Choose names of roles carefully ➤ dynamic character ➤ hypothesis, initial data, finding ■ Use as much as possible a standard set of inferences ➤ see catalog of inferences in the book
- 26. Knowledge-model construction 26 Guidelines for specifying domain knowledge ■ domain-knowledge type used as static role not required to have exactly the “right’” representation ➤ design issue; ➤ key point: knowledge is available. ■ scope of domain knowledge is typically broader than what is covered by inferences ➤ requirements of communication, explanation
- 27. Knowledge-model construction 27 Stage 3: Knowledge Refinement ■ Validate knowledge model ■ Fill contents of knowledge bases
- 28. Knowledge-model construction 28 Fill contents of knowledge bases ■ schema contains two kinds of domain types: ➤ information types that have instances that are part of a case ➤ knowledge types that have instances that are part of a domain model ■ goal of this task: find (all) instances of the latter type ■ case instances are only needed for a scenario
- 29. Knowledge-model construction 29 Guidelines for filling contents ■ filling acts as a validation test of the schema ■ usually not possible to define full, correct knowledge base in the first cycle ■ knowledge bases need to be maintained ➤ knowledge changes over time ■ techniques: ➤ incorporate editing facilities for KB updating, trace transcripts, structured interview, automated learning, map from existing knowledge bases
- 30. Knowledge-model construction 30 Validate knowledge model ■ internally and externally ■ verification = internal validation ➤ “is the model right?” ■ validation = validation against user requirements ➤ "is it the right model?"
- 31. Knowledge-model construction 31 Validation techniques ■ Internal ➤ structured walk-troughs ➤ software tools for checking the syntax and find missing parts ■ External ➤ usually more difficult and/or more comprehensive. ➤ main technique: simulation – paper-based simulation – prototype system
- 32. Knowledge-model construction 32 Paper-based simulation the user says: “the car does not start” DIAGNOSIS: Complaint: engine- behavior.status = does-not-start Complaint is received, for which a diagnostic task is started a possible cause is that the fuel tank is empty COVER: hypothesis; fuel- tank.status = empty One of the three possible causes is produced. in that case we would expect the gas indicator to be low PREDICT: expected-finding: gas-dial.value = zer The expected finding provides us with a way of getting supporting evidence for hypothesis
- 34. Knowledge-model construction 34 Maintenance ■ CK view: not different from development ■ model development is a cyclic process ■ models act as information repositories ➤ continuously updated ■ but: makes requirements for support tools stronger ➤ transformation tools
- 35. Knowledge-model construction 35 Domain Documentation Document (KM-1) ■ Knowledge model specification ■ list of all information sources used. ■ list of model components that we considered for reuse. ■ scenarios for solving the application problem. ■ results of the simulations undertaken during validation ■ Elicitation material (appendices)
- 36. Knowledge-model construction 36 Summary process ■ Knowledge identification ➤ familiarization with the application domain ■ Knowledge specification ➤ detailed knowledge analysis ➤ supported by reference models ■ Knowledge refinement ➤ completing the knowledge model ➤ validating the knowledge model ■ Feedback loops may be required ➤ simulation in third stage may lead to changes in specification ➤ Knowledge bases may require looking for additional knowledge sources. ➤ general rule: feedback loops occur less frequently, if the application problem is well-understood and similar problems have been tackled
- 37. Knowledge-model construction 37 Elicitation of expertise ■ Time-consuming ■ Multiple forms ➤ e.g. theoretical, how-to-do-it ■ Multiple experts ■ Heuristic nature ➤ distinguish empirical from heuristic ■ Managing elicitation efficiently ➤ knowledge about when to use particular techniques
- 38. Knowledge-model construction 38 Expert types ■ Academic ➤ Regards domain as having a logical structure ➤ Talks a lot ➤ Emphasis on generalizations and laws ➤ Feels a need to present a consistent “story”: teacher ➤ Often remote from day-to-day problem solving ■ Practitioner ➤ Heavily into day-to-day problem solving ➤ Implicit understanding of the domain ➤ Emphasis on practical problems and constraints ➤ Many heuristics
- 39. Knowledge-model construction 39 Human limitations and biases ■ Limited memory capacity ■ Context may be required for knowledge recollection ■ Prior probabilities are typically under-valued ■ Limited deduction capabilities
- 40. Knowledge-model construction 40 Elicitation techniques ■ Interview ■ Self report / protocol analysis ■ Laddering ■ Concept sorting ■ Repertory grids ■ Automated learning techniques ➤ induction
- 41. Knowledge-model construction 41 Session preparation ■ Establish goal of the session ■ Consider added value for expert ■ Describe for yourself a profile of the expert ■ List relevant questions ■ Write down opening and closing statement ■ Check recording equipment ➤ audio recording is usually sufficient ■ Make sure expert is aware of session context: goal, duration, follow-up, et cetera
- 42. Knowledge-model construction 42 Start of the session ■ Introduce yourself (if required) ■ Clarify goal and expectations ■ Indicate how the results will be used ■ Ask permission for tape recording ■ Privacy issues ■ Check whether the expert has some questions left ■ Create as much as possible a mutual trust
- 43. Knowledge-model construction 43 During the session ■ Avoid suggestive questions ■ Clarify reason of question ■ Phrase questions in terms of probes ➤ e.g, “why …” ■ Pay attention to non-verbal aspects ■ Be aware of personal biases ■ Give summaries at intermediate points
- 44. Knowledge-model construction 44 End of the session ■ Restate goal of the session ■ Ask for additional/qualifying ■ Indicate what will be the next steps ■ Make appointments for the next meetings ■ Process interview results ASAP. ■ Organize feedback round with expert ■ Distribute session results
- 45. Knowledge-model construction 45 Unstructured interview ■ No detailed agenda ■ Few constraints ■ Delivers diverse, incomplete data ■ Used in early stages: feasibility study, knowledge identification ■ Useful to establish a common basis with expert ➤ s/he can talk freely
- 46. Knowledge-model construction 46 Structured interview ■ Knowledge engineer plans and directs the session ■ Takes form of provider-elicitor dialogue ■ Delivers more focused expertise data ■ Often used for “filling in the gaps” in the knowledge base ➤ knowledge refinement phase ■ Also useful at end of knowledge identification or start of knowledge specification ■ Always create a transcript
- 47. Knowledge-model construction 47 Interview structure for domain- knowledge elicitation ■ Identify a particular sub-task ➤ should be relatively small task, e.g. an inference ■ Ask expert to identify “rules” used in this task ■ Take each rule, and ask when it is useful and when not ■ Use fixed set of probes: ➤ “Why would you do that?” ➤ “How would you do that?” ➤ “When would you do that?” ➤ “What alternatives are there for this action?” ➤ “What if …?” ➤ “Can you tell me more about ..?”
- 48. Knowledge-model construction 48 Interview pitfalls ■ Experts can only produce what they can verbalize ■ Experts seek to justify actions in any way they can ➤ “spurious justification” ■ Therefore: supplement with techniques that observe expertise “in action” ➤ e.g. self report
- 49. Knowledge-model construction 49 Self report ■ Expert performs a task while providing a running commentary ➤ expert is “thinking aloud” ■ Session protocol is always transcribed ➤ input for protocol analysis ■ Variations: ➤ shadowing: one expert performs, a second expert gives a running commentary ➤ retrospection: provide a commentary after the problem- solving session ■ Theoretical basis: cognitive psychology
- 50. Knowledge-model construction 50 Requirements for self-report session ■ Knowledge engineer must be sufficiently acquainted with the domain ■ Task selection is crucial ➤ only a few problems can be tackled ➤ selection typically guided by available scenario’s and templates ■ Expert should not feel embarrassed ➤ consider need for training session
- 51. Knowledge-model construction 51 Analyzing the self-report protocol ■ Use a reference model as a coding scheme for text fragments ➤ Task template ■ Look out for “when”-knowledge ➤ Task-control knowledge ■ Annotations and mark-ups can be used for domain- knowledge acquisition ■ Consider need for tool support
- 52. Knowledge-model construction 52 Example transcript
- 53. Knowledge-model construction 53 Guidelines and pitfalls ■ Present problems in a realistic way ■ Transcribe sessions as soon as possible ■ Avoid long sessions (maximum = 20 minutes) ■ Presence of knowledge engineer is important ■ Be aware of scope limitations ■ Verbalization may hamper performance ■ Knowledge engineer may lack background knowledge to notice distinctions
- 54. Knowledge-model construction 54 Use of self reports ■ Knowledge specification stage ■ Validation of the selection of a particular reference model ■ Refining / customizing a task template for a specific application ■ If no adequate task template model is available: use for bottom-up reasoning model construction ➤ but: time-consuming
- 55. Knowledge-model construction 55 Laddering ■ Organizing entities in a hierarchy ■ Hierarchies are meant as pre-formal structures ■ Nodes can be of any type ➤ class, process, relation, …. ■ Useful for the initial phases of domain-knowledge structuring ➤ in particular knowledge identification ■ Can be done by expert ➤ tool support
- 56. Knowledge-model construction 56 Example ladder
- 57. Knowledge-model construction 57 Concept sorting ■ Technique: ➤ present expert with shuffled set of cards with concept names ➤ expert is asked to sort cards in piles ■ Helps to find relations among a set of concepts ■ Useful in case of subtle dependencies ■ Simple to apply ■ Complementary to repertory grids ➤ concept sort: nominal categories ➤ repertory grid: ordinal categories
- 58. Knowledge-model construction 58 Card sort tool
- 59. Knowledge-model construction 59 Repertory grid ■ Based on personal construct theory (Kelly, 1955) ■ Subject: discriminate between triads of concepts ➤ Mercury and Venus versus Jupiter ■ Subject is asked for discriminating feature ➤ E.g. “planet size” ■ Re-iterate until no new features are found ■ Rate all concepts with respect to all features ■ Matrix is analyzed with cluster analysis ■ Result: suggestions for concept relations ■ Tool support is required
- 61. Knowledge-model construction 61 When to use? ■ Knowledge identification ➤ Unstructured interview, laddering ■ Knowledge specification ➤ Domain schema: concept sorting, repertory grid ➤ Template selection: self report ➤ Task & inference knowledge: self report ■ Knowledge refinement ➤ Structured interview, reasoning prototype