![#1 Psychophysical Experiment
One Sequence [11sec]
Vibrate!
③
Vibrate!
①
Vibrate!
⑥
Vibrate!
⑤
Vibrate!
④
Vibrate!
②
time [sec]
1. Present Target Stimulus [5sec] 2. Random Stimulus Trial [6sec]
Vibrate!
④
Vibrate
Example
[1sec]
Break Time
[2sec]
6 7 8 9 101 2 3 4 50
Vibration Each Stimulus [1sec × 6]
"12
Present Target
[2sec]
❖ The detail flow of one sequence
press!](https://image.slidesharecdn.com/publother4presen-160719053127/85/Spatial-tactile-brain-computer-interface-paradigm-by-applying-vibration-stimulus-to-large-body-areas-12-320.jpg)
![❖ Boxplots of all subjects’ response
− Average and distribution of the response time for each target stimulus
"18
#1 Psychophysical Experiment Results
Command
Responsetime[ms]
300 ms
200300400500600700800
Median value
Minimum value
Maximum value
1/4 value
3/4 value
1 2 3 4 5 6](https://image.slidesharecdn.com/publother4presen-160719053127/85/Spatial-tactile-brain-computer-interface-paradigm-by-applying-vibration-stimulus-to-large-body-areas-18-320.jpg)
![❖ EEG experiment SWLDA the accuracy for each of the 7 subjects
0%
20%
40%
60%
80%
#1 #2 #3 #4 #5 #6 #7Ave.
42.7%
50.0%
73.4%
16.7%
46.7%
43.3%
30.0%
38.9%
"19
#2 EEG Experiment Results
0%
20%
40%
60%
80%
100%
#1 #2 #3 #4 #5 #6 #7 Ave.
76.2%
100%100%
50%
66.7%
83.3%
50%
83.3%
Accuracy[%]
Average accuracy rates Max accuracy rates
Users Users
Chance level:
16.7%](https://image.slidesharecdn.com/publother4presen-160719053127/85/Spatial-tactile-brain-computer-interface-paradigm-by-applying-vibration-stimulus-to-large-body-areas-19-320.jpg)
![❖ EEG experiment ITR results of 7 uses
"22
Subject Max accuracy [%] ITR [bit/min]
#1 83.3 4.30
#2 50 1.18
#3 83.3 4.30
#4 66.7 2.48
#5 50 1.18
#6 100 7.18
#7 100 7.18
#2 EEG Experiment Result](https://image.slidesharecdn.com/publother4presen-160719053127/85/Spatial-tactile-brain-computer-interface-paradigm-by-applying-vibration-stimulus-to-large-body-areas-23-320.jpg)
Spatial tactile brain-computer interface paradigm by applying vibration stimulus to large body areas
- 1. Spatial Tactile Brain-Computer Interface Paradigm by Applying Vibration Stimulus to Large Body Areas 201213082 Multimedia Laboratory ! ! Takumi Kodama! Supervisors:! ! Tomasz M. Rutkowski, Takeshi Yamada, Shigeki Miyabe, Shoji Makino
- 2. Research Background "2 ❖ Healthy and non-disabled people − Can communicate their intentions with others easily ❖ Amyotrophic lateral sclerosis (ALS) patients − Can’t move their muscle by themselves − Thus they can’t communicate their intentions So they need device not using muscle like “BCI” ・・・
- 3. About “Brain-Computer Interface” BCI could be a communicating tool for ALS patients "3 Brain waves are captured Machine learning estimates user intention Thought based application control ❖ Brain Computer Interface (BCI) − Control the device based on brain signals (EEG) ❖ To input BCI commands: − Not necessary to use your body movements
- 4. ❖ Stimulus-driven BCI: − Use brain activity only generated in response to external stimuli ❖ Stimulus-driven BCI has five modes (five senses of human): − Difference is how to evoke P300 response Type of Brain-Computer Interface: I use “tactile (touch) BCI” in my research project "4 Auditory BCI Gustatory BCI Olfactory BCI Tactile BCI Visual BCI Stimulus-driven BCI Imagery BCI BCI
- 5. About “P300 response” ❖ P300 response − Positive signal that occurs 300 ms after receiving a stimulus P300 response evoked by applying tactile stimulus to human body "5 : Target : non-Target
- 6. Research Approach ❖ Development of a new tactile BCI paradigm − Propose new communication option by using tactile BCI "6 Give user tactile stimuli of body parts with larger distances Tactile stimuliTactile stimuli
- 7. About a “gaming pad” ❖ Specification of the gaming pad −Used to deliver tactile stimuli to any body parts "7 Only the vibrotactile output to stimulate the users is used Specification Name “ZEUS VYBE” Developer Disney Research Input Audio signal Output Sound and vibration Sound Two speakers inside Vibration 12 vibration motors inside Input: ! Audio signal ♪ ♫ Output:! Sound & Vibration ~ ~~ ~
- 8. ❖ Define six vibrotactile stimulus positions in gaming pad − Stimulus positions are located in spatial body parts "8 Users identify six vibrotactile positions individually How to deliver the tactile stimulus No. Pos. name 1 Left shoulder (LS) 2 Right shoulder (RS) 3 Left waist (LW) 4 Right waist (RW) 5 Left leg (LL) 6 Right leg (RL)
- 9. How to use the gaming pad ❖ Gaming pad is used in “laying-position” − Because ALS patients are often bedridden "9 Users lie on the gaming pad and receive vibrotactile stimuli “Sitting position” “Laying position” ▲Experiment in laying position
- 10. ❖ The experiments were carried out in two steps "10 First: Psychophysical Experiment #1 Psychophysical Experiment Purpose: ! ! ! ! ! Experimental Procedure ! ? ! ! #2 EEG Experiment ! Purpose: ! ! ! ! ・To check whether the P300 response could be evoked after vibrotactile stimuli ・To check whether the users could perceive the tactile stimulation correctly
- 11. ❖ Flow of the experiment in each sequences "11 #1 Psychophysical Experiment 3. Users press the button when feel the target stimulus ! ! ! ! ! ! ! press! 1. Present the target stimulus ! ! ! ! ! ! Target! Right Waist Measure the reaction time from stimulus onset to button press 2. Give six random stimulus ! ! ! ! ! 3 1 6 5 2 non-Target non-Target non-Target non-Target non-Target Target!4
- 12. #1 Psychophysical Experiment One Sequence [11sec] Vibrate! ③ Vibrate! ① Vibrate! ⑥ Vibrate! ⑤ Vibrate! ④ Vibrate! ② time [sec] 1. Present Target Stimulus [5sec] 2. Random Stimulus Trial [6sec] Vibrate! ④ Vibrate Example [1sec] Break Time [2sec] 6 7 8 9 101 2 3 4 50 Vibration Each Stimulus [1sec × 6] "12 Present Target [2sec] ❖ The detail flow of one sequence press!
- 13. ❖ Experimental condition "13 #1 Psychophysical Experiment Condition Detail Subjects 7 persons (3 males and 4 females) Type of stimulus Only tactile (vibration) Use device Gaming pad “ZEUS VYBE” Stimulus positions 6 (Left Shoulder,Right Shoulder , Left Waist, Right Waist, Left Leg, Right Leg) Number of sequences 120 Stimulus duration 300 ms Inter-stimulus interval (ISI) 700 ms
- 14. "14 Next: EEG Experiment Experimental Procedure ❖ The experiments were carried out in two steps #1 Psychophysical Experiment Purpose: ! ! ! ! ! ! ? ! ! #2 EEG Experiment ! Purpose: ! ! ! ! ・To check whether the P300 response could be evoked after vibrotactile stimuli ・To check whether the users could perceive the tactile stimulation correctly
- 15. ❖ Flow of the experiment in each sessions 1. Present the target stimulus ! ! ! ! ! ! "15 Classify the user’s brain responses in each target stimulus #2 EEG Experiment 2. Give 60 random stimulus (10 Targets + 50 non-Targets) ! ! ! ! ! 3. Users pay attention to the target stimulus (Generate P300 response) ! ! ! ! ! ！ 1 6 5 2 - - - - 6 3 5 - - - - 3 5 2 1 - - - - … - Target! Right Waist 4 ! 4 ! 4 ! 4 ! 4 ! 2 3
- 16. ❖ Experimental condition (additional & changed item) "16 #2 EEG Experiment Condition Detail EEG recording system g.USBamp active EEG electrodes system Number of channels 16 EEG electrode position Cz Pz P3 P4 C3 C4 CP5 CP6 P1 P2 POz C1 C2 FC1 FC2 FCz Number of sessions 5 Target stimulus length 250 ms Inter-stimulus interval (ISI) 350~370 ms
- 17. ❖ Confusion matrix of all users’ response − Correct rate of response for each target stimulus "17 #1 Psychophysical Experiment Results Response Target 1 2 3 4 5 6 123456
- 18. ❖ Boxplots of all subjects’ response − Average and distribution of the response time for each target stimulus "18 #1 Psychophysical Experiment Results Command Responsetime[ms] 300 ms 200300400500600700800 Median value Minimum value Maximum value 1/4 value 3/4 value 1 2 3 4 5 6
- 19. ❖ EEG experiment SWLDA the accuracy for each of the 7 subjects 0% 20% 40% 60% 80% #1 #2 #3 #4 #5 #6 #7Ave. 42.7% 50.0% 73.4% 16.7% 46.7% 43.3% 30.0% 38.9% "19 #2 EEG Experiment Results 0% 20% 40% 60% 80% 100% #1 #2 #3 #4 #5 #6 #7 Ave. 76.2% 100%100% 50% 66.7% 83.3% 50% 83.3% Accuracy[%] Average accuracy rates Max accuracy rates Users Users Chance level: 16.7%
- 20. ❖ EEG mean result in Cz channels "20 : Target : non-Target #2 EEG Experiment Results
- 21. ❖ EEG mean result in Cz channels "20 : Target : non-Target #2 EEG Experiment Results
- 22. ❖ Information transfer rate (ITR) − Commonly used to assess the performance of BCI "21 #2 EEG Experiment Result R: the number of bits / selection; N: the number of classes (N=6 in this experiment); P: the classifier accuracy (each best result in this experiment); V: the classification speed in selections / minute (V=5.56 in this experiment);
- 23. ❖ EEG experiment ITR results of 7 uses "22 Subject Max accuracy [%] ITR [bit/min] #1 83.3 4.30 #2 50 1.18 #3 83.3 4.30 #4 66.7 2.48 #5 50 1.18 #6 100 7.18 #7 100 7.18 #2 EEG Experiment Result
- 24. ❖ The effectiveness of proposed method was proven as follows: − This approach could be applicable easily for ALS patients − The BCI command identification performance of each target stimulus has been very high − The P300 response appears very clearly in the all tested electrode channels "23 Conclusions