Brain-computer interface based on SSVEP

1. In 3rd game level player needs to concentrate on a specific object displayed on screen to get a point. The game is able to detect, from player’s brainwaves, if he is currently gazing on this object, and then to reward him or punish, depending on the result. This section contains detailed description of detection process.Detection method employed in the game is based on fact that brain reacts very directly to flickering: if player looks at the object which is blinking with some frequency, then his brain starts to generate specific brainwaves containing the same frequency – SSVEP [1]. SSVEP is produced by occipital parts of the brain – those which are in the back and responsible for visual processing. So to detect if player is currently looking at flickering object these steps are needed:Capture occipital signals with EEG equipment

2. Check if object’s flickering frequency is present in these signals. If this frequency is pronounced, then SSVEP is present, and it means that user is currently watching the object.But brain has its own tastes: it can react very intensively to some types of objects, and to other types of objects reaction can be much weaker. Intensity of reaction depends on flickering frequency too. So selection of objects & frequencies that elicit strongest response is required for reliable SSVEP detection. Selection of object that is least annoying to human eye is also needed if this object is supposed to be used in game.So different objects: checkerboard pattern (most commonly used in SSVEP experiments), horizontal and vertical stripes, big and small circles, and different frequencies: 7.5, 8.57, 10, 12 and 30 Hz, - were checked for suitability. Frequencies were chosen from frequencies that can be displayed by LCD with 60Hz refresh rate, which is used in most of computers today [2]. Honestly speaking, during our experiments we set stimulation frequencies to be 10.63, 12.14, 14.17, 17 and 21.25 Hz, because we thought that refresh rate was 80 Hz, but despite of what we set, first frequencies were actually produced by LCD display.Different stimuli used to elicit SSVEP responseThe EEG was recorded continuously from two subjects while they were looking on different objects flickering with different frequencies (only one object was presented at the same time). Then SSVEP detection method was applied to each N seconds of EEG, or to each window of length N. Windows were taken with one second step (i.e. each window started one second later after previous window).Each window was analyzed for SSVEP presence. This analysis consisted of:Applying Common Average Reference

3. Selecting one occipital electrode – OZ, O2 or O1

4. Fast Fourier Transform of the EEG signal from that channel

5. Detection if SSVEP is present in the spectrum obtained in previous stepIn third step amplitudes of different frequencies (spectrum) that are presented in EEG signal are received. FFT plot of EEG that was recorded while player was watching the blinking object, will contain a peak at the blinking frequency:Frequency spectrum of 30-second long EEG signal recorded from the subject while he was stimulated visually by object flickering with frequency 12Hz. A clear peak at this frequency and at its second harmonic (24Hz) can be seenSo when SSVEP is present, a peak in the spectrum can be detected. But detection of that peak is a bit tricky task - because of properties of Fourier Transform, sometimes amplitude of flickering frequency may not be present in the spectrum, but only of frequencies that are close to it. Surely, it can always be approximated by increasing resolution of FFT, or calculated from known values for neighboring frequencies, but result can be inaccurate. Another problem is that flickering frequency may be unstable on LCD display. Because of these problems, SSVEP in some cases might not be detected correctly:Peak in frequency spectrum located next to the place (yellow) where it supposed to beTaking all these problems in consideration, following method for detecting peaks in the spectrum was developed:Look for frequency with maximal amplitude (FM) among all frequencies neighboring to frequency we look for (FL). The furthest neighbors we take is 1.5Hz below and 1.5Hz above FL.

6. If FL is not presented in spectrum we got after FFT, find frequency in the spectrum nearest to it and set F = this frequency, or F = frequency we look for if it is present.

7. If FM is next to FL in the spectrum, set FL = FM

8. Take the amplitude of FL and divide it by FM. If resulting value is above some threshold, then we decide that peak is present. During experiments it was found that threshold which gives best results is 0.8.This method is quite simple because game does not include training session.SSVEP detection methodAfter applying method described above following results (using OZ channel) were obtained:Subject 1 (Danny)Frequencyobject typecheckerboardCircleStipessmallbighorizontalverticalBoth7.583.61-----8.5793.33-----1098.36-----129056.4592.0678.6976.2796.553090.16-----Subject 2 (Lasse)Frequencyobject typecheckerboardCircleStipessmallbighorizontalverticalboth7.587.1-----8.5767.74-----1082.26-----1252.3144.1283.6148.3963.4973.443080.33-----These tables show successful peak detection rates for different objects & stimulation frequencies (window length here is 4 seconds). For some type of stimuli rates are very low – it means that reaction to them was very weak, so there is no point in using it in the game. Those stimuli that were detected well in both subjects are:Checkerboard pattern blinking with frequencies 7.5, 10 and 30

9. Big circle flickering with frequency 12

10. Vertical and horizontal stripes with frequency 12 These stimulations are worthy of further investigation: it’s necessary to make sure that there will be no peak detected when player doesn’t look to the object. Two of stimulations will be investigated:Big-circle stimulus, because from all stimuli above it is the least annoying one, so it is the top candidate for using in our game

11. Checkerboard pattern with frequency 7.5, because it was detected better than any other stimulusFor first stimulus, method detected SSVEP with following accuracies:On this plot, horizontal axis is different window length and vertical is percentage of windows in which SSVEP was detected. The lowest graph, False Detection Rate (FDR), shows percentage of windows in which it was detected, despite of the fact that no SSVEP was present (e.g. there was no blinking object on the screen). The highest graph, True Detection Rate (TDR), shows percentage of windows in which SSVEP was present and was detected correctly. The median graph, Classification Rate, shows overall performance of our SSVEP detection method (e.g. ((100-FDR) +TDR)/2)As can be seen on the graph, to achieve good performance (79% correct detections and more) windows at least 3 seconds long is needed. It means that player needs to concentrate for at least 3 seconds on flickering object before computer can detect it. For shorter windows, SSVEP is also detected very well (80% and more), but there is high rate of false positives – in instance, for window length 2, SSVEP was detected in half of the windows not containing it.Here are plots amplitudes of lookup frequency divided by maximal amplitude in bin (FL/FM) for individual windows for one of subjects (window length here is 3 seconds). In first plot, player was stimulated with big circle flickering with frequency of 12Hz, and SSVEP for this stimulation was successfully detected in 85% of windows. In second plot there is no stimulation, but despite of this, SSVEP was wrongly detected in 31% of all windows.Percentage of windows in which SSVEP was detected. Horizontal axis: time (in seconds) of window start, vertical: FL/FM value. Values above threshold (0.8) are detected as SSVEP. Top plot: windows containing SSVEP, bottom plot: windows without SSVEP.For checkerboard pattern flickering with 7.5Hz frequency these accuracies were obtained:FDR, TDR and Classification rate for checkerboard 7.5Hz stimulusAgain, it shows that for good performance windows of length 3 seconds and more are needed, and we have same trend for FDR.DiscussionResults of this preliminary analysis show that method for SSVEP detection described here does not require training but gives quite a good performance, so it can be used it in game. Currently in the big circle flickering with frequency of 7.5Hz is used to stimulate the player. This stimulus was not tested in preliminary analysis, but it works in the game. The only problem encountered is that for some subjects, SSVEP detection does not work very well. But there is no surprise in it - this analysis and in preliminary studies with 30-second windows show that classification rate depends on subject: for first subject – Danny – true detection rate was almost perfect for every stimulus, but in the same time for second subject – Lasse – it was random for some stimuli. But in future SSVEP detection will be made more generalizable.Future workMethod which uses signal from only one EEG channel to detect SSVEP was developed. But results of offline analysis show that SSVEP can be detected in all three occipital channels. Information from these channels can be combined to make detection better and more generalizable.Also method relies only on detection of one peak in frequency spectrum – at the stimulation frequency. However, as offline analysis shows, there is also a peak at the second harmonics when SSVEP is present. Future versions will employ usage of this peak too.ReferencesHerrmann, C.S.V., Human EEG responses to 1–100 Hz flicker: resonance phenomena in visual cortex and their potential correlation to cognitive phenomena. Exp Brain Res, 2001. 137: p. 346-353.

12. Ivan Volosyak and Hubert Cecotti and Axel Gräser Impact of Frequency Selection on LCD Screens for SSVEP Based Brain-Computer Interfaces. IWANN 1, volume 5517 of Lecture Notes in Computer Science, page706-713. Springer, 2009

Brain-computer interface based on SSVEP

More Related Content

Recently uploaded (20)

Featured (20)

Brain-computer interface based on SSVEP