Validation and verification
Download as PPT, PDF14 likes11,124 views
The document discusses the concepts of model verification and validation, emphasizing the distinction between ensuring the model is built correctly and ensuring it accurately represents the real system. It outlines techniques for verification, such as flow diagrams and documentation, as well as an iterative calibration process to refine models. Various validation approaches are presented, ranging from using expert comparisons to statistical tests, highlighting cost-value ratios in method selection.
Validation and verification
- 1. Validation and Verification Adapted from Jerry Banks
- 2. Verification Concerned with building the model right Comparison of conceptual model and computer representation Is the model implemented correctly in the computer? Are the inputs and logical parameters represented properly?
- 3. Validation Concerned with building the right model Accurate representation of the real system This is achieved through the calibration of the model Iterative process until accuracy is acceptable
- 4. Model Building, Verification, and Validation REAL SYSTEM Conceptual Model 1Assumptions on system components 2Structural Assumptions (defines the interactions between the system components) 3Input parameters and data assumptions Operational Model (Computer Representation) Conceptual Validation Model Verification Calibration and Validation
- 5. Common sense suggestions for verification Have someone check the computerized model Make a flow diagram (with logical actions for each possible event) Examine model output for reasonableness Print the input parameters at the end of the simulation
- 6. Common sense suggestions for verification Make the computerized representation as self documenting as possible If animated, verify what is seen Use IRC or debuggers Use graphical interface
- 7. Three Classes of Techniques for Verification Common sense techniques Thorough documentation Traces
- 8. Calibration and Validation Validation is the overall process of comparing the model and its behavior to the real system and its behavior Calibration is the iterative process of comparing the model to the real system and making adjustments to the model, and so on.
- 9. Iterative Process of Calibration REAL SYSTEM Initial Model Second Revision of Model First Revision of Model Compare Model to Reality Compare Revised Model to Reality Compare second Revised Model to Reality
- 10. 3 Step Approach by Naylor and Finger (1967) Build a model with high face validity Validate model assumptions Compare the model input-output transformations to corresponding input-output transformations of the real system
- 11. Possible validation techniques in order of increasing cost-value ratio by Van Horn (1971) High face validity. Use previous research/ studies/observation/experience Conduct statistical test for data homogeneity, randomness, and goodness of fit test Conduct Turing test. Have a group of experts compare model output versus system output and detect the difference Compare model output to system output using statistical tests
- 12. Possible validation techniques in order of increasing cost-value ratio by Van Horn (1971) After model development, collect new data and apply previous 3 tests Build a new system or redesign the old one based on simulation results and use this data to validate the model Do little or no validation. Implement results without validating