Human computer learning interaction in a virtual laboratory by Vasilis Zafeiropoulos (HOU)
0 likes145 views
The document discusses the development and implementation of a virtual laboratory platform named OnLabs, designed to enhance biology lab education by simulating real lab experiences in a 3D environment. It highlights various operational modes like experimentation, instruction, and evaluation, which utilize machine learning techniques to improve user performance and ensure safe, efficient training. The ultimate goal of OnLabs is to complement traditional lab education, reduce costs, and effectively elevate remote learning outcomes.
![Scoring Mechanism (Evaluation Mode)
Consists of two parts:
Success Rate [0-100%] (how “close to” or “far from” the user is from the
experiment’s final state)
For the i-th action performed, a score xi ∈ [0,1] is assigned, e.g.:
― if the microscope is connected to the socket (1st action), score x1 = 1; otherwise x1 = 0.
― if the microscope switch is turned on (2nd action), score x2 = 1; otherwise x2 = 0.
― if the microscope light intensity knob is set to 18 (3rd action), score x3 = 1; for any other
case x3 lies within (0,1)
Each action has different significance, so we intuitively define a particular
weight wi for each one of them
Success rate is the weighted average of the various scores xi:
𝑠𝑢𝑐𝑐𝑒𝑠𝑠 𝑟𝑎𝑡𝑒 ←
σ 𝑖=1
𝑛
𝑤 𝑖∙𝑥 𝑖
σ 𝑖=1
𝑛 𝑤 𝑖
∙ 100
From weights wi, we create a weight vector 𝑤 = 𝑤1, 𝑤2, … , 𝑤 𝑛
𝑇
(useful for
Machine Learning later…)
Penalty points (they are received whenever the user performs actions in
wrong order)
The Aggregate Score [0-100%] is calculated combining the Success
Rate and the Penalty Points](https://image.slidesharecdn.com/human-computerlearninginteractioninavirtuallaboratory1-200616123845/85/Human-computer-learning-interaction-in-a-virtual-laboratory-by-Vasilis-Zafeiropoulos-HOU-12-320.jpg)
Human computer learning interaction in a virtual laboratory by Vasilis Zafeiropoulos (HOU)
- 1. Human-Computer Learning Interaction in a Virtual Laboratory Vasilis Zafeiropoulos PhD Candidate HELLENIC OPEN UNIVERSITY School of Science and Technology
- 3. Οn-site laboratory education - no plain sailing Large number of trainees Short time for training Lab equipment not really completely available (sensitive, wear and tear, consumables) Risks of accidents (health and maintenance) 3
- 4. About Onlabs Simulation of a real Biology Lab Main characteristics: 3D Realism Interactivity Genre: Modern Adventure Games Development under: Hive3D by Eyelead Software (2012-2015) Unity (2016-now) 4
- 6. Purposes Decongestion of real labs Familiarizing oneself with equipment - safely Protection of lab equipment from damages Capability of infinite trial & error testing (more) Pleasant training process (more) Effective learning Virtual lab education is to complement on- site lab education, not substitute it! 6
- 7. Experiments simulated in Onlabs (1) Α. Use of Optical Microscope 1. Configuration of the microscope (e.g. adjusting light, iris, lenses) 2. Creation of a test specimen (a slide with a piece of paper and water on it) 3. Microscoping with all objectives lenses 7
- 8. Experiments simulated in Onlabs (2) Electrophoresis 1. Use of the electric scale for scaling the various powders 2. Use of electronic pipette for drawing and pouring precise quantities of fluids 3. Use of magnetic stirrer for fully dissolving and stirring the components of a dilution 4. Use of automatic pipette for extracting DNA from PRC tubes 5. Use of microwave oven for dissolving agarose powder in electrophoresis buffer 6. Use electrophoresis device for separating nucleic acid samples by size 7. Use UV viewer for visualizing nucleic acid bands 8
- 9. Operation Modes Experimentation Mode The human user makes free use of all the lab equipment Instruction Mode The computer guides the human user to complete an experiment Evaluation Mode The computer evaluates the performance of a human user on conducting an experiment Computer Training Mode A human expert (expert ≠ user) teaches the computer with Machine Learning to: rate properly (Rater Training Sub-Mode), i.e. provide an accurate score for the user’s performance) play properly (Bot Training Sub-Mode), i.e. conduct an experiment
- 10. Instruction Mode
- 11. Evaluation Mode
- 12. Scoring Mechanism (Evaluation Mode) Consists of two parts: Success Rate [0-100%] (how “close to” or “far from” the user is from the experiment’s final state) For the i-th action performed, a score xi ∈ [0,1] is assigned, e.g.: ― if the microscope is connected to the socket (1st action), score x1 = 1; otherwise x1 = 0. ― if the microscope switch is turned on (2nd action), score x2 = 1; otherwise x2 = 0. ― if the microscope light intensity knob is set to 18 (3rd action), score x3 = 1; for any other case x3 lies within (0,1) Each action has different significance, so we intuitively define a particular weight wi for each one of them Success rate is the weighted average of the various scores xi: 𝑠𝑢𝑐𝑐𝑒𝑠𝑠 𝑟𝑎𝑡𝑒 ← σ 𝑖=1 𝑛 𝑤 𝑖∙𝑥 𝑖 σ 𝑖=1 𝑛 𝑤 𝑖 ∙ 100 From weights wi, we create a weight vector 𝑤 = 𝑤1, 𝑤2, … , 𝑤 𝑛 𝑇 (useful for Machine Learning later…) Penalty points (they are received whenever the user performs actions in wrong order) The Aggregate Score [0-100%] is calculated combining the Success Rate and the Penalty Points
- 13. Computer Training Mode (1) Rater Training Sub-Mode The student plays various sessions The human expert evaluates each session The computer scoring mechanism is adjusted according to human expert’s feedback Machine learning techniques used: Genetic Algorithm Artificial Neural Network
- 15. Computer Training Mode (2) Bot Training Sub-Mode The computer plays a session by itself The human expert evaluates each session The computer learns how to play correctly Machine Learning technique used: Reinforcement Learning (Under development…)
- 17. Genetic Algorithm (Rater Training Mode) A GA simulates biological evolution Our GA is interactive (a human supervisor contributes to the learning process) In our GA, weight vectors are the chromosomes Our first generation consists of 30 randomly produced weight vectors. Weight vectors, like chromosomes, compete against each other with respect to each one’s fitness (expressed by a fitness function) For each play session, the score produced by a weighted vector and the score provided by the human expert are compared by our fitness function According to weight vectors’ fitness: A fixed percentage of them are selected (directly copied) to the new generation A fixed percentage of them are chosen for crossover (reproduction) with each other and their offspring are put into the new generation A fixed percentage of the chromosomes in the produced generation are mutated The GA stops after a termination condition is satisfied; our GA’s termination condition is 50 generations The fittest weight vector of the final generation is the training result with our GA
- 18. Artificial Neural Network (Rater Training Mode) An ANN simulates neural networks in the brain Our ANN consists of 3 layers of neurons To each neuron of an ANN come weights from the neurons of the previous layer Our ANN has different weights and provides a different scoring mechanism from the ones in Evaluation Mode! Our ANN’s initial weights wj→i are randomly produced For each play session, our ANN produces a single value (score) as output through its rightmost neuron For each play session, the human expert provides their own score, too The error between the ANN’s score and the human expert’s score is back-propagated through the ANN and its weights are re-configured An ANN is re-trained several times (epochs); our ANN is being trained for 1000 epochs The training result is the ANN’s final weights
- 19. Training and Testing Train on set A, test on set B Train on all sets, test on all sets (biased) Train on Experts E1, E2, E3, test on E4 (cross-validation among experts) Train on Expert E1, test on E1 (cross-validation within the same expert) Train on Classifications C1, C2, test on C3 (cross-validation among performance classifications*) Train on Groups G1, G2, G3, G4, test on G5 (cross-validation among various groups of training sets**) Calculation of Mean Squared Error (MSE) on each testing set Comparison of GA’ fittest weight vector to intuitive weight vector of evaluation mode * Classifications (of user’s performance): Low, Medium, High ** Groups of 12 training sets (one classification from each expert in each group)
- 21. Artificial Neural Network Training Results (1)
- 22. Artificial Neural Network Training Results (2) Mean Squared Error to Epochs graph (train on all, test on all):
- 23. Testing & Evaluation by Students Focus group: biology-oriented students with minimum or zero previous knowledge in science topics Written conceptual pre-tests and post-tests Questionnaires to express opinion of satisfaction, motivation, engagement, etc. Practice examinations in the wet lab The Virtual Lab Group (educated with Onlabs) and the Control Group (educated without Onlabs) conducted a 22 steps microscopy experiment Results: ▪ a. I completed this step easily ▪ b. I completed this step on difficulty ▪ c. I couldn’t complete this step by myself – I asked for help
- 24. Future Work Simulation of the rest of lab instruments and biology experiments Simulation of instruments and experiments at: Chemistry lab Physics lab Any other place with instruments and procedures Insertion of more than one bots in the virtual lab Let them interact, co-operate and compete with the human user and each other Evaluate learning outcomes in terms of: speed accuracy use of resources etc. 24
- 25. The grand goal Reduce the cost of laboratory education Elevate lab education at-a-distance to at least the same level of learning effectiveness with on-site lab education 25
- 26. Development Team Vasilis Zafeiropoulos Dimitris Kalles Argyro Sgourou Achilles Kameas Evgenia Paxinou Kostas Mitropoulos http://onlabs.eap.gr/ Thank you!