Approximately 900,000 Americans have little to no control over their hands, which causes daily tasks to become difficult and decreases quality of life. Our goal is to help improve quality of life with a brain computer interface that allows subjects to read an electronic book (eBook), using affordable, commercially available components. Concentrating on a blinking square induces SSVEP (steady-state visually evoked potentials) signals in the brain’s occipital lobe at the same frequency as the stimulus. The SSVEP signal is used as a control signal to flip pages of an eBook. EEG signals from the occipital lobe are acquired with the Emotiv EPOC headset while the subject concentrates on two squares blinking at different frequencies. Frequency domain features detect if the subject is concentrating on one of the blinking squares, and the control signal is used to turn eBook pages.
BACKGROUND AND PROBLEM STATEMENT
In the US, motor neuron diseases (MNDs) such as multiple sclerosis (MS) and amyotrophic lateral sclerosis (ALS) affect 7 out of every 100,000 people (MNDA, 2013). Roughly 900,000 Americans have little or no control over their hands (Bureau, 2005). Such individuals may have difficulty interacting with their environment. Many individuals with advanced-stage MNDs cannot use existing assistive-living technologies that require coordinated muscle control, such as voice recognition, puff and sip, and tongue interfaces. As a result, these individuals experience reduced quality of life due to a decreased ability to interact with their environments and act independently. Hiring an aide may be cost-prohibitive, and in an assisted-living environment, the aide may only have a limited amount of time for each resident. Moreover, noninvasive interfaces that require minimal voluntary motion are expensive, have a bulky physical setup, and are still in the development phase.
We propose an assistive technology, the eyeReader, which is a completely hands-free eBook reader that the user controls with a noninvasive EEG-based brain computer interface system. We offer that the eyeReader will improve the quality of life of MND patients by providing entertainment and the satisfaction of performing the task of turning pages independently. The eyeReader allows the user to read comfortably from a laptop or tablet. On the screen, an eBook page is flanked by two rectangles that blink at controlled rates. When the user wants to turn the page forward or backward, they will concentrate on one of the blinking rectangles (blinker). Concentrating on a blinking light produces steady-state visually evoked potentials (SSVEPs) in the brain’s occipital lobe. SSVEPs range between 3.5Hz and 75Hz and are well-characterized, robust signals with a relatively high signal to noise ratio (Vialatte, Maruice, Dauwels, & Cichocki, 2010). These brain signals are accessible via EEG. We used an inexpensive, easy-to-use EEG headset available to consumers to record from the occipital lobe. Depending on the user’s choice to focus on one of the two blinkers, the eBook page will turn forwards or backwards. The ability for individuals with advanced MNDs to navigate a computer interface with their brain signals will improve quality of life.
Our goal is to enable people affected by severe loss of voluntary motion due to motor neuron diseases and other conditions, by providing an easy-to-use and inexpensive eBook interface.
The eyeReader allows users to use an eBook via an EEG headset and a laptop or tablet (Figure 1). Our system has four components: the interface, signal acquisition, signal processing, and calibration.
Concentrating on a continuously flashing rectangle (blinker) induces SSVEP signals in the brain’s occipital lobe at the same frequency as the stimulus. Our user interface has two windows flashing at 7Hz and 10Hz. When the software detects that the user is looking at either window, the eBook pages turn forward or backward, respectively. If neither frequency is detected, the eBook stays at the current page. When the user sees the page turned correctly, the visual feedback improves the user’s ability to concentrate on the stimuli and increases future user accuracy (Middendorf, McMillan, Calhoun, & Jones, 2000).
The Emotiv EPOC Headset collects EEG data from 14 electrodes and outputs a signal digitized at 128 samples/second wirelessly to the laptop. Our software only uses EEG data from two electrodes, O1 and O2 electrodes, which correspond to the left and right occipital regions (Emotiv, 2013). The Emotiv headset’s O1 and O2 electrodes can be easily adjusted to sit right above the occipital protuberance.
Data collection and processing are performed in C++ by a module named EEGLogger (Figure 3) . EEGLogger fills a ten-second long first-in-first-out buffer with data from the Emotiv headset. Every second EEGLogger updates its buffer, performs signal averaging (Figure 4A and B), and performs the Fourier Transform (FFTW library) on the full ten seconds available. Since the SSVEP signals have the same frequency as the stimuli frequency, EEGLogger looks for the largest peak in the vicinity of the stimuli frequency and records its magnitude. If there is not a distinct peak, EEGLogger takes the magnitude of the frequency corresponding to the stimuli frequency. The magnitude at the stimuli frequency is compared to the magnitudes of two frequencies nearby, one a little above and one a little below the stimuli frequency to calculate two frequency domain features (Figure 4C). The user concentrating on one of the stimuli frequencies is detected when both frequency domain features exceed predetermined thresholds.
To improve detection accuracy, a training interface uses audio commands to guide the user to look at the blinkers or the eBook page, set the detection algorithm thresholds.
The eyeReader was tested on two healthy subjects to validate our method for detecting which of two blinkers in the visual field the subject was concentrating on.
- Brain signals over the occipital lobe acquired using the commercially available Emotiv EPOC ™ headset have distinct peaks in the frequency domain corresponding to the stimuli frequency (Figure 5 and Figure 6).
- Our detection algorithm successfully uses SSVEP signals to interface with an eBook reader (video above).
- Accuracy may be improved by customizing the stimuli frequency to the user (Figure 7A). JD was tested using 7.2Hz and 16Hz, while JL was tested using 12.8Hz and 16Hz.
- Subject accuracy may improve with training (Figure 7).
Concentrating on one of the two blinkers produces brain signals at the stimuli frequency evidenced by the peaks in Figure 5(B). Distinct peaks in the frequency domain are used to detect which blinker the subject is concentrating on Figure 5(C). In the study, the subject was screened for a variety of stimuli frequencies. Figure 7 shows three of ten frequencies tested on JL. The 8Hz stimulus was not used due to low accuracy. The subject was randomly directed to look at one of the two blinkers using recorded verbal commands. The detection accuracies (listed in Table 1) range from 80% and 99% and average to 86.2%. At least one control signal can be detected every 10 seconds, for an approximately 5.2 bits/minute information transfer rate as calculated by Kronegg et al (Kronegg, 2005). We expect the detection accuracy to improve with better signal detection algorithms and subject training because JL was able to achieve 99% detection accuracy with 16Hz after training for 8 sessions over a month (Figure 7).Raw EEG signal from the O1 electrode. The last ten seconds is highlighted in magenta, and analyzed in the frequency domain [B] which shows a strong peak at 10Hz (black circle) and no peak at 7.2Hz (red circle). [C]: The frequency detected by the algorithm over time (blue lines) follow the frequency the user was commanded verbally to look at (green line).”]
SSVEP-based brain computer interfaces such as the eyeReader can provide a wealth of control signals that can be used to make assistive-living technologies. Since the individual only needs to concentrate on one of several blinkers in the visual field, individuals with advanced motor neuron diseases and other conditions that restrict voluntary muscular control may benefit from this non-invasive interface, especially if they do not have sufficient muscle control for speech recognition, puff-and-sip, and chin/tongue lever-based technologies. Specifically, the eyeReader is a system with commercially available components that allows users with limited hand control to read eBooks. The user can read up to twelve hours straight, which improves quality of living by providing a source of entertainment and the satisfaction of performing a task independently. The eyeReader can be used by anyone capable of concentrating on one of two blinkers in the visual field. This provides an easy-to-use, inexpensive interface for individuals with advanced-stage MNDs.
For individuals who already own a laptop or computer, the cost of the Emotiv EEG Headset is $299 retail. This is comparable to the $200-$700 cost of switch-based interfaces we found online.
Current interfaces for patients with motor neuron diseases (MNDs) and quadriplegia use a variety of control signals derived from EOG (Mark Sagar, 2007), voice recognition (Anschutz, Luther-Krug, Seel, & Jones, 2012), sip-and-puff (Koerlin, 2011), or tongue and chin position (Caltenco, Struijk, & Breidegard, 2010). SSVEP-based brain computer interfaces only require the user to concentrate on a blinker in the visual field, so this technology is uniquely suited for patients who are physically unable to operate those devices. While SSVEP-based computer interfaces have been implemented in various research contexts (Zhu, 2010), the eyeReader uses the Emotiv EPOC EEG headset as an inexpensive and accessible platform.
Based on the recommendation of a physical therapist with two patients who have MS, we will make improvements to the prototype. We plan to create a mount for the laptop to interface with the patients’ beds. Moreover, now that we have a reasonable detection algorithm with simple commands, our next goal is to use this control signal to accept and terminate a phone call, using a web-based telephone service such as Google Voice and a smartphone, the first item on a patient’s ‘wish list’. Afterwards, we will work on a physical interface that opens and closes the patient’s room door. The eyeReader is a first step towards expanding the choice of interfaces available to individuals with advanced MNDs to improve quality of life.
We would like to thank Professor Arye Nehorai for providing the BCI student group with workspace. Our graduate student advisers Elad Gilboa and Chuck Holmes, and Professors Ed Richter and Robert Morley gave us good advice about signal processing. Special thanks goes to the two test subjects who helped us gather data. We would like to thank physical therapist Arlene Goldberg for providing guidance on applications. This work is supported by the McKelvey Undergraduate Research Fellowship, the Washington University Electrical and Systems Engineering Department, and the Washington University Engineering Project Review Board. We would like to thank Emotiv Systems for donating a headset.
Anschutz, J. R., Luther-Krug, M. V., Seel, R. T., & Jones, M. L. (2012, Nov 20). Patent No. US8314691 B2. US.
Arnaud Delorme, A. L. (2012, April 10). Patent No. US8155736 B2. US.
Bureau, U. C. (2005). Retrieved from Prevalence of Disability Among Individuals 15 Years and Older by Specific Measures of : http://www.census.gov/hhes/www/disability/sipp/disab05/d05tb1.pdf
Caltenco, H. A., Struijk, L. N., & Breidegard, B. (2010). TongueWise: Tongue-computer interface software for people with tetraplegia. 32nd Annual International Conference of the IEEE EMBS. Buenos Aires.
Collura, T. F. (2007, September 11). Patent No. US7269456 B2. US.
Emotiv. (2013). EEG Features. Retrieved April 5, 2013, from http://www.emotiv.com/eeg/features.php
Koerlin, J. M. (2011, Jan 26). Patent No. EP1774944 B1. Europe.
Kronegg, J. S. (2005). Information-transfer rate modeling of EEG-based synchronized brain-computer interfaces.
Mark Sagar, R. S. (2007, June 20). Patent No. EP1797537 A2. Europe.
Middendorf, M., McMillan, G., Calhoun, G., & Jones, K. S. (2000). Brain-computer interfaces based on the steady-state visual-evoked response. IEEE Transactions on Rehabilitation Engineering, 8(2), 211-214.
MNDA. (2013). About MND. Retrieved April 5, 2013, from http://www.mndassociation.org/what-is-mnd/Brief+guide+to+MND
Vialatte, F.-B., Maruice, M., Dauwels, J., & Cichocki, A. (2010). Steady-state visually evoked potentials: focus on essential paradigms and future perspectives. Progress in Neurobiology, 90, 418-438.
Zhu, D. (2010). A Survey of Stimulation Methods Used in SSVEP-Based BCIs. Computational Intelligence and Neuroscience, 1-12.