Rahul Sharma Enrollment no. Any accomplished requires the effort of many people and this work is no different. This seminar difficult due to numerous reasons some of error correction was beyond my control. Sometimes, I was like rudderless boat without knowing what to do next. It was then the timely guidance of that has seen us through all these odds. I would be very grateful to them for their inspiration, encouragement and guidance in all phases of the endeavor.
It is my great pleasure to thank Dr. Soni Changlani, HOD of Electronics and Communication for her constant encouragement and valuable advice for this seminar.
I also wish to express my gratitude towards all other staff members for their kind help. Finally, I would thank Pro.
Prachi Parashar who was tremendously contributed to this seminar directly as well as indirectly; gratitude from the depths of my heart is due to her.
Regardless of source I wish to express my gratitude to those who may contribute to this work, even though anonymously. These ideas have captured the imagination of humankind in the form of ancient myths and modern science fiction stories.
However, it is only recently that advances in cognitive neuroscience and brain imaging technologies have started to provide us with the ability to interface directly with the human brain. In these systems, users explicitly manipulate their brain activity instead of using motor movements to produce signals that can be used to control computers or communication devices.
The impact of this work is extremely high, especially to those who suffer from devastating neuromuscular injuries and neurodegenerative diseases such as amyotrophic lateral sclerosis, which eventually strips individuals of voluntary muscular activity while leaving cognitive function intact.
This ability is made possible through the use of sensors that can monitor some of the physical processes that occur within the brain that correspond with certain forms of thought. In Berger was the first to record human brain activity by means of EEG. Berger was able to identify oscillatory activity in the brain by analyzing EEG traces. One wave he identified was the alpha wave 8—13 Hz , also known as Berger's wave.
Berger's first recording device was very rudimentary. He inserted silver wires under the scalps of his patients. These were later replaced by silver foils attached to the patients' head by rubber bandages.
Berger connected these sensors to a Lippmann capillary electrometer, with disappointing results. More sophisticated measuring devices, such as the Siemens double-coil recording galvanometer, which displayed electric voltages as small as one ten thousandth of a volt, led to success.
Berger analyzed the interrelation of alternations in his EEG wave diagrams with brain diseases. The papers published after this research also mark the first appearance of the expression brain—computer interface in scientific literature.
Rather, it is a complex assemblage of competing sub-systems, each highly specialized for particular tasks Carey By studying the effects of brain injuries and, more recently, by using new brain imaging technologies, neuroscientists have built detailed topographical maps associating different parts of the physical brain with distinct cognitive functions.
The brain can be roughly divided into two main parts: 2. Since this is the largest and most complex part of the brain in the human, this is usually the part of the brain people notice in pictures.
This is the region that current BCI work has largely focused on. For instance, most language functions lie primarily in the left hemisphere, while the right hemisphere controls many abstract and spatial reasoning skills.
Also, most motor and sensory signals to and from the brain cross hemispheres, meaning that the right brain senses and controls the left side of the body and vice versa.
The brain can be further divided into separate regions specialized for different functions. For example, occipital regions at the very back of the head are largely devoted to processing of visual information. Areas in the temporal regions, roughly along the sides and lower areas of the cortex, are involved in memory, pattern matching, language processing, and auditory processing.
Still other areas of the cortex are devoted to diverse functions such as spatial representation and processing, attention orienting, arithmetic, voluntary muscle movement, planning, reasoning and even enigmatic aspects of human behavior such as moral sense and ambition.
We should emphasize that our understanding of brain structure and activity is still fairly shallow. These topographical maps are not definitive assignments of location to function. In fact, some areas process multiple functions, and many functions are processed in more than one area.
Although invasive technologies provide high temporal and spatial resolution, they usually cover only very small regions of the brain. Additionally, these techniques require surgical procedures that often lead to medical complications as the body adapts, or does not adapt, to the implants.
Furthermore, once implanted, these technologies cannot be moved to measure different regions of the brain. While many researchers are experimenting with such implants, we will not review this research in detail as we believe these techniques are unsuitable for human-computer interaction work and general consumer use.
We summarize and compare the many non-invasive technologies that use only external sensors. While the list may seem lengthy, only Electroencephalography EEG and Functional Near Infrared Spectroscopy fNIRS present the opportunity for inexpensive, portable, and safe devices, properties we believe are important for brain-computer interface applications in HCI work.
A BCI must have four components. It must record activity directly from the brain invasively or non-invasively. It must provide feedback to the user, and must do so in realtime.
Finally, the system must rely on intentional control. Devices that only passively detect changes in brain activity that occur without any intent, such as EEG activity associated with workload, arousal, or sleep, are not BCIs.
Donoghue et al. Known as cortical neural prostheses CNPs , devices based on this technology are a subset of neural prosthetics, a larger category that includes stimulating, as well as recording, electrodes. In fact, there is no difference between the first three terms. Neuroprostheses also called neural prostheses are devices that cannot only receive output from the ner- vous system, but can also provide input. Moreover, they can interact with the peripheral and the central nervous systems.
Figure 2 presents examples of neu- roprostheses, such as cochlear implants auditory neural prostheses and retinal implants visual neural prostheses. BCIs are a special category of neuroprostheses. One of the most important reasons that this is significant is that current BCI systems aim to provide assis- tive devices for people with severe disabilities that can render people unable to perform physical movements.
Radiation accidents like the one in the Star Trek episode described above are unlikely today, but some diseases can actually lead to the locked-in syndrome. Amyotrophic lateral sclerosis ALS is an example of such a disease.
The exact cause of ALS is unknown, and there is no cure. ALS starts with muscle weakness and atrophy. Usually, all voluntary movement, such as walking, speaking, swallow- ing, and breathing, deteriorates over several years, and eventually is lost completely. The disease, however, does not affect cognitive functions or sensations. People can still see, hear, and understand what is happening around them, but cannot control their muscles. This is because ALS only affects special neurons, the large alpha motor neurons, which are an integral part of the motor pathways.
Death is usually caused by failure of the respiratory muscles. Life-sustaining measures such as artificial respiration and artificial nutrition can considerably prolong the life expectancy. However, this leads to life in the locked- in state. Once the motor pathway is lost, any natural way of communication with the environment is lost as well.
BCIs offer the only option for communication in such cases. So, a BCI is an artificial output channel, a direct interface from the brain to a computer or machine, which can accept voluntary commands directly from the brain without requiring physical movements. A technology that can listen to brain activ- ity that can recognize and interpret the intent of the user? This misconception is quite common among BCI new- comers, and is presumably also stirred up by science fiction and poorly researched articles in popular media.
In the following section, we explain the basic principles of BCI operation. It should become apparent that BCIs are not able to read the mind. To understand BCI operation better, one has to understand how brain activity can be measured and which brain signals can be utilized.
In this chapter, we focus on the most important recording methods and brain signals. Therefore, sensors can detect different types of changes in electrical or magnetic activity, at different times over different areas of the brain, to study brain activity.
Most BCIs rely on electrical measures of brain activity, and rely on sensors placed over the head to measure this activity. Electroencephalography EEG refers to recording electrical activity from the scalp with electrodes. It is a very well estab- lished method, which has been used in clinical and research settings for decades. EEG equipment is inexpensive, lightweight, and comparatively easy to apply.
Temporal resolution, meaning the ability to detect changes within a certain time interval, is very good.
However, the EEG is not with- out disadvantages: The spatial topographic resolution and the frequency range are limited. The EEG is susceptible to so-called artifacts, which are contamina- tions in the EEG caused by other electrical activities. Examples are bioelectrical activities caused by eye movements or eye blinks electrooculographic activity, EOG and from muscles electromyographic activity, EMG close to the recording sites.
External electromagnetic sources such as the power line can also contaminate the EEG. Furthermore, although the EEG is not very technically demanding, the setup pro- cedure can be cumbersome. To achieve adequate signal quality, the skin areas that are contacted by the electrodes have to be carefully prepared with special abrasive Fig.
Because gel is required, these electrodes are also called wet elec- trodes. The number of electrodes required by current BCI systems range from only a few to more than electrodes. Most groups try to minimize the number of elec- trodes to reduce setup time and hassle.
Since electrode gel can dry out and wearing the EEG cap with electrodes is not convenient or fashionable, the setting up pro- cedure usually has to be repeated before each session of BCI use. A possible solution is a technology called dry electrodes.
Dry electrodes do not require skin prepara- tion nor electrode gel. This technology is currently being researched, but a practical solution that can provide signal quality comparable to wet electrodes is not in sight at the moment. A BCI analyzes ongoing brain activity for brain patterns that originate from spe- cific brain areas. To get consistent recordings from specific regions of the head, scientists rely on a standard system for accurately placing electrodes, which is called the International 10—20 System [6].
The other electrodes are placed at similar fractional distances. The inter-electrode distances are equal along any transverse from left to right and antero-posterior from front to back line and the placement is symmetrical. The labels of the electrode positions are usually also the labels of the recorded channels. That is, if an electrode is placed at site C3, the recorded signal from this electrode is typically also denoted as C3.
The first letters of the labels give a hint of the brain region over which the electrode is located: Fp — pre-frontal, F — frontal, C — central, P — parietal, O — occipital, T — temporal. Figure 4 depicts the electrode placement according to the 10—20 system. While most BCIs rely on sensors placed outside of the head to detect electrical activity, other types of sensors have been used as well [7]. Magnetoencephalography MEG records the magnetic fields associated with brain activity.
Functional mag- netic resonance imaging fMRI measures small changes in the blood oxygenation level-dependent BOLD signals associated with cortical activation.
Like fMRI also near infrared spectroscopy NIRS is a hemodynamic based technique for assess- ment of functional activity in human cortex. Different oxygen levels of the blood result in different optical properties which can be measured by NIRS.
All these methods have been used for brain—computer communication, but they all have draw- backs which make them impractical for most BCI applications: MEG and fMRI are very large devices and prohibitively expensive. That is, there is no need to perform surgery or even break the skin. The nasion is the intersection of the frontal and nasal bones at the bridge of the nose. The inion is a small bulge on the back of the skull just above the neck sensors. This surgery includes opening the skull through a surgical procedure called a craniotomy and cutting the membranes that cover the brain.
When the electrodes are placed on the surface of the cortex, the signal recorded from these electrodes is called the electrocorticogram ECoG. ECoG does not damage any neurons because no electrodes penetrate the brain. The signal recorded from electrodes that penetrate brain tissue is called intracortical recording. Invasive recording techniques combine excellent signal quality, very good spatial resolution, and a higher frequency range. Artifacts are less problematic with invasive recordings.
Further, the cumbersome application and re-application of electrodes as described above is unnecessary for invasive approaches. Intracortical electrodes can record the neural activity of a single brain cell or small assemblies of brain cells. The ECoG records the integrated activity of a much larger number of neurons that are in the proximity of the ECoG electrodes. However, any invasive technique has better spatial resolution than the EEG.
Clearly, invasive methods have some advantages over non-invasive methods. However, these advantages come with the serious drawback of requiring surgery. Ethical, financial, and other considerations make neurosurgery impractical except for some users who need a BCI to communicate. Even then, some of these users may find that a noninvasive BCI meets their needs. It is also unclear whether both ECoG and intracortical recordings can provide safe and stable recording over years.
Long- term stability may be especially problematic in the case of intracortical recordings. Electrodes implanted into the cortical tissue can cause tissue reactions that lead to deteriorating signal quality or even complete electrode failure.
For ethical reasons, some invasive research efforts rely on patients who undergo neurosurgery for other reasons, such as treatment of epilepsy. Studies with these patients can be very infor- mative, but it is impossible to study the effects of training and long term use because these patients typically have an ECoG system for only a few days before it is removed.
Figure 5 summarizes the different methods for recording bioelectrical brain activity. However, measuring activity is not enough, because a BCI cannot read the mind or decipher thoughts in general.
A BCI can only detect and classify spe- cific patterns of activity in the ongoing brain signals that are associated with specific tasks or events. What the BCI user has to do to produce these patterns is determined by the mental strategy sometimes called experimental strategy or approach the BCI system employs.
The mental strategy is the foundation of any brain—computer communication. The mental strategy determines what the user has to do to volition- ally produce brain patterns that the BCI can interpret. The amount of training required to successfully use a BCI also depends on the mental strategy. The most common mental strategies are selective focused attention and motor imagery [2, 10—12].
In the following, we briefly explain these different BCIs. The stimuli can be auditory [13] or somatosensory [14]. Most BCIs, however, are based on visual stimuli. That is, the stimuli could be different tones, different tactile stimulations, or flashing lights with different frequencies. In order to select a command, the users have to focus their attention to the corresponding stimulus. A BCI based on selective attention could rely on five stimuli. Four stimuli are associated with the commands for cursor movement: left, right, up, and down.
The fifth stimulus is for the select command. This system would allow two dimensional navigation and selection on a computer screen. Users operate this BCI by focusing their attention on the stimulus that is associated with the intended command. Assume the user wants to select an item on the computer screen which is one position above and left of the current cursor position. The user would first need to focus on the stimulus that is associated with the up command, then on the one for the left command, then select the item by focusing on the stimulus associated with the select command.
The items could represent a wide variety of desired messages or commands, such as letters, words, instructions to move a wheelchair or robot arm, or signals to control a smart home. A 5-choice BCI like this could be based on visual stimuli. In fact, visual attention can be implemented with two different BCI approaches, which rely on somewhat different stimuli, mental strategies, and signal processing.
These approaches are named after the brain patterns they produce, which are called P potentials and steady-state visual evoked potentials SSVEP. A P based BCI relies on stimuli that flash in succession. These stimuli are usually letters, but can be symbols that represent other goals, such as controlling a cursor, robot arm, or mobile robot [15, 16].
The BCI can detect this P In contrast to the P approach, however, these stim- uli do not flash successively, but flicker continuously with different frequencies in the range of about 6—30 Hz. The BCI knows the flickering frequencies of all light sources, and when an SSVEP is detected, it can determine the corresponding light source and its associated command. BCI approaches using selective attention are quite reliable across different users and usage sessions, and can allow fairly rapid communication.
Moreover, these approaches do not require significant training. Almost all subjects can learn the simple task of focusing on a target letter or symbol within a few minutes.
This is relevant because completely locked-in patients are not able to shift gaze anymore. Another concern is that some people may dislike the external stimuli that are necessary to elicit P or SSVEP activity.
In fact, already the preparation of movement or the imagination of movement also change the so-called sensorymotor rhythms. Sensorimotor rhythms SMR refer to oscillations in brain activity recorded from somatosensory and motor areas see Fig. Alpha activity recorded from sensorimotor areas is also called mu activity.
The decrease of oscillatory activ- ity in a specific frequency band is called event-related desynchronization ERD. Correspondingly, the increase of oscillatory activity in a specific frequency band is called event-related synchronization ERS.
The frequency bands that are most important for motor imagery are mu and beta in EEG signals. Invasive BCIs often also use gamma activity, which is hard to detect with electrodes mounted outside the head. Activity invoked by right hand movement imagery is most prominent over electrode location C3 see Fig.
The central sulcus divides the frontal lobe from the parietal lobe. It also separates the precen- tral gyrus indicated in red and the postcentral gyrus indicated in blue. The temporal lobe is separated from the frontal lobe by the lateral fissure. The occipital lobe lies at the very back of the cerebrum. The following cortical areas are particularly important for BCIs are: motor areas, somatosensory cortex, posterior parietal cortex, and visual cortex over C4.
That is, activity invoked by hand movement imagery is located on the contralateral opposide side. Foot movement imagery invokes activity over Cz. A distinction between left and right foot movement is not possible in EEG because the corresponding cortical areas are too close. To produce patterns that can be detected, the cortical areas involved have to be large enough so that the resulting activity is sufficiently prominent compared to the remaining EEG background EEG.
Hand areas, foot areas, and the tongue area are comparatively large and topographically different. Therefore, BCIs have been controlled by imagining moving the left hand, right hand, feet, and tongue [19]. And since these patterns originate from motor and somatosensory areas, which are directly connected to the normal neuromuscular output pathways, motor imagery is a par- ticularly suitable mental strategy for BCIs. For example, some BCIs can tell if the users are thinking of moving your left hand, right hand, or feet.
This can lead to a BCI that allows 3 signals, which might be mapped on to commands to move left, right, and select. Another type of motor imagery BCI relies on more abstract, subject-specific types of movements.
Over the course of several training sessions with a BCI, people can learn and develop their own motor imagery strategy. In a cur- sor movement task, for instance, people learn which types of imagined movements are best for BCI control, and reliably move a cursor up or down. Some subjects can learn to move a cursor in two [20] or even three [21] dimensions with further training.
However, motor imagery is a skill that has to be learned. BCIs based on motor imagery usually do not work very well during the first session. Instead, unlike BCIs on selective attention, some training is necessary. However, longer training is often necessary to achieve sufficient control.
Therefore, training is an important component of many BCIs. Users learn through a process called operant conditioning, which is a fundamental term in psychology. In operant conditioning, people learn to associate a certain action with a response or effect. For example, people learn that touching a hot stove is painful, and never do it again.
Nonetheless, if imagined actions produce effects, then conditioning can still occur. During BCI use, operant conditioning involves training with feedback that is usually presented on a computer screen. Positive feedback indicates that the brain signals are modulated in a desired way. Negative or no feedback is given when the user was not able to perform the desired task. BCI learning is a type of feedback called neurofeedback.
The feedback indicates whether the user performed the mental task well or failed to achieve the desired goal through the BCI. Users can utilize this feedback to optimize their mental tasks and improve BCI performance. The feedback can be tactile or auditory, but most often it is visual. This signal processing can have three stages: preprocessing, feature extraction, and detection and classification.
Preprocessing aims at simplifying subsequent processing operations without los- ing relevant information. An important goal of preprocessing is to improve signal quality by improving the so-called signal-to-noise ratio SNR.
A bad or small SNR means that the brain patterns are buried in the rest of the signal e. Transformations combined with filtering techniques are often employed during preprocessing in a BCI. Scientists use these techniques to transform the signals so unwanted signal components can be eliminated or at least reduced.
These techniques can improve the SNR. The brain patterns used in BCIs are characterized by certain features or proper- ties. The firing rate of individual neurons is an important feature of invasive BCIs using intracortical recordings. The feature extraction algorithms of a BCI calculate extract these features. Feature extraction can be seen as another step in preparing the signals to facilitate the subsequent and last signal processing stage, detection and classification.
Detection and classification of brain patterns is the core signal processing task in BCIs. The user elicits certain brain patterns by performing mental tasks according to mental strategies, and the BCI detects and classifies these patterns and translates them into appropriate commands for BCI applications.
This detection and classification process can be simplified when the user com- municates with the BCI only in well defined time frames. Such a time frame is indicated by the BCI by visual or acoustic cues. During this time, the user is supposed to perform a specific mental task. The BCI tries to classify the brain signals recorded in this time frame. This mode of operation is called synchronous or cue-paced. Although these BCIs are relatively easy to develop and use, they are impracti- cal in many real-world settings.
A cue-pased BCI is somewhat like a keyboard that can only be used at certain times. In an asynchronous or self-paced BCI, users can interact with a BCI at their leisure, without worrying about well defined time frames [23].
Users may send a signal, or choose not to use a BCI, whenever they want. This mode of operation is technically more demanding, but it offers a more natural and conve- nient form of interaction with a BCI.
A simple measure is classification performance also termed classification accuracy or classification rate. It is the ratio of the number of correctly classified trials successful attempts to perform the required mental tasks and the total number of trials.
The error rate is also easy to calculate, since it is just the ratio of incorrectly classified trials and the total number of trials.
Although classification or error rates are easy to calculate, application dependent measures are often more meaningful. For instance, in a mental typewriting applica- tion the user is supposed to write a particular sentence by performing a sequence of mental tasks. Again, classification performance could be calculated, but the number of letters per minute the users can convey is a more appropriate measure. Letters per minute is an application dependent measure that assesses indirectly not only the classification performance but also the time that was necessary to perform the required tasks.
A more general performance measure is the so-called information transfer rate ITR [25]. It depends on the number of different brain patterns classes used, the time the BCI needs to classify these brain patterns, and the classification accu- racy.
ITR is measured in bits per minute. Since ITR depends on the number of brain patterns that can be reliably and quickly detected and classified by a BCI, the information transfer rate depends on the mental strategy employed. A major reason is that BCIs based on selective attention usually provide a larger number of classes e.
Motor imagery, for instance, is typically restricted to four or less motor imagery tasks. More imagery tasks are possible but often only to the expense of decreased classification accuracy, which in turn would decrease in the information transfer rate as well.
Such performance, however, is not typical for most users in real world settings. In fact, these record values are often obtained under laboratory conditions by indi- vidual healthy subjects who are the top performing subjects in a lab. Of course, it is interesting to push the limits and learn the best possible performance of current BCI technology, but it is no less important to estimate realistic performance in more practical settings.
Unfortunately, there is currently no study available that investigates the average information transfer rate for various BCI systems over a larger user population and over a longer time period so that a general estimate of average BCI performance can be derived. Furthermore, a minority of subjects exhibit little or no control [11, 26, 31, ]. In any case, a BCI provides an alternative communication channel, but this channel is slow.
It certainly does not provide high-speed interaction. It can- not compete with natural communication such as speaking or writing or traditional man-machine interfaces in terms of ITR. However, it has important applications for the most severely disabled. There are also new emerging applications for less severely disabled or even healthy people, as detailed in the next section.
Proportional output could be a continuous value within the range of a certain minimum and maximum. Depending on the mental strategy and on the brain patterns used, some BCIs are more suitable for providing discrete output values, while others are more suitable for allowing proportional control [32]. A P BCI, for instance, is par- ticularly appropriate for selection applications. SMR based BCIs have been used for discrete control, but are best suited to proportional control applications such as 2-dimensional cursor control.
In fact, the range of possible BCI applications is very broad — from very simple to complex. BCIs have been validated with many applications, includ- ing spelling devices, simple computer games, environmental control, navigation in virtual reality, and generic cursor control applications [26, 33, 34].
Most of these applications run on conventional computers that host the BCI sys- tem and the application as well. Typically, the application is specifically tailored for a particular type of BCI, and often the application is an integral part of the BCI system.
BCIs that can connect and effectively control a range of already exist- ing assistive devices, software, and appliances are rare. An increasing number of systems allow control of more sophisticated devices, including orthoses, prosthe- ses, robotic arms, and mobile robots [35—40]. Figure 7 shows some examples of BCI applications, most of which are described in detail in this book corresponding references are given in the figure caption.
There is little argument that such an interface would be a boon to BCI research. BCIs can control any application that other interfaces can control, provided these applications can function effectively with the low information throughput of BCIs.
On the other hand, BCIs are normally not well suited to controlling more demanding and complex applications, because they lack the necessary information transfer rate.
Complex tasks like rapid online chatting, grasping a bottle with a robotic arm, or playing some computer games require more information per second than a BCI can provide. However, this problem can sometimes be avoided by offering short cuts.
For instance, consider an ALS patient using a speller application for communi- cation with her caregiver. The patient is thirsty and wants to convey that she wants to drink some water now. Since this is a wish the patient may have quite often, it would be useful to have a special symbol or command for this message.
In this way, the patient can convey this particular mes- sage much faster, ideally with just one mental task. Many more short cuts might allow other tasks, but these short cuts lack the flexibility of writing individual mes- sages. In other words, the BCI should allow a combination of process-oriented or low- level control and goal-oriented or high level control [41, 42].
Low-level control means the user has to manage all the intricate interactions involved in achieving a task or goal, such as spelling the individual letters for a message. In contrast, goal- oriented or high-level control means the users simply communicate their goal to the application.
Such applications need to be sufficiently intelligent to autonomously perform all necessary sub-tasks to achieve the goal. In any interface, users should not be required to control unnecessary low-level details of system operation. This is especially important with BCIs.
Allowing low-level control of a wheelchair or robot arm, for example, would not only be slow and frustrating but potentially dangerous.
Figure 8 presents two such examples of very complex applications. The semi-autonomous wheelchair Rolland III can deal with different input mod- alities, such as low-level joystick control or high-level discrete control. Autonomous and semi-autonomous navigation is supported. In this scenario, the system detects the bottle and the glass both located at arbitrary positions on the tray , grabs the bottle, moves the bottle to the glass while automatically avoiding any obstacles on the tray, fills the glass with liquid from the bottle while avoiding pouring too much, and finally puts the bottle back in its original position — again avoiding any possible collisions.
These assistive devices offload much of the work from the user onto the system. The wheelchair provides safety and high-level control by continuous path planning and obstacle avoidance.
The rehabilitation robot offers a collection of tasks which are performed autonomously and can be initiated by single commands. Without this device intelligence, the user would need to directly control many aspects of device operation. Consequently, controlling a wheelchair, a robot arm, or any com- plex device with a BCI would be almost impossible, or at least very difficult, time consuming, frustrating, and in many cases even dangerous.
Such complex BCI applications are not broadly available, but are still topics of research and are being evaluated in research laboratories. The success of these applications, or actually of any BCI application, will depend on their reliability and on their acceptance by users. Another factor is whether these applications provide a clear advantage over conventional assistive technologies.
In the case of completely locked-in patients, alternative control and communication methodologies do not exist. BCI control and communication is usually the only possible practical option. However, the situation is different with less severely disabled or healthy users, since they may be able to communicate through natural means like speech and gesture, and alternative control and communication technologies based on movement are available to them such as keyboards or eye tracking systems.
Until recently, it was assumed that users would only use a BCI if other means of communication were unavailable. More recent work showed a user who preferred a BCI over an eye tracking system [43]. Although BCIs are gaining acceptance with broader user groups, there are many scenarios where BCIs remain too slow and unreliable for effective control.
For example, most prostheses cannot be effectively controlled with a BCI. Typically, prostheses for the upper extremities are controlled by electromyo- graphic myoelectric signals recorded from residual muscles of the amputation stump.
In the case of transradial amputation forearm amputation , the muscle activ- ity recorded by electrodes over the residual flexor and extensor muscles is used to open, close, and rotate a hand prosthesis.
Controlling such a device with a BCI is not practical. For higher amputations, however, the number of degrees-of-freedom of a prostheses i. In the extreme case of an amputation of the entire arm shoulder disarticulation , conventional myoelectric control of the prosthetic arm and hand becomes very difficult.
Controlling such a device by a BCI may seem to be an option. In fact, several approaches have been investigated to con- trol prostheses with invasive and non-invasive BCIs [39, 40, 44]. Ideally, the control of prostheses should provide highly reliable, intuitive, simultaneous, and propor- tional control of many degrees-of-freedom. Proportional control in this case means the user can modulate speed and force of the actuators in the prosthesis.
That is, for instance, the prosthetic hand can be closed while the wrist of the hand is rotated at the same time. None of the BCI approaches that have been currently suggested for controlling prostheses meets these criteria. Non-invasive approaches suffer from limited band- width, and will not be able to provide complex, high-bandwidth control in the near future.
Invasive approaches show considerable more promise for such control in the near future. However, then these approaches will need to demonstrate that they have clear advantages over other methodologies such as myoelectric control combined with targeted muscle reinnervation TMR.
TMR is a surgical technique that transfers residual arm nerves to alternative muscle sites. After reinnervation, these target muscles produce myoelectric signals electromyographic signals on the surface of the skin that can be measured and used to control prosthetic devices [45]. Figure 9 shows a prototype of a prosthesis with 7 degrees-of-freedom 7 joints controlled by such a system.
Today, there is no BCI that can allow independent control of 7 different degrees of freedom, which is necessary to duplicate all the movements that a natural arm could make. On the other hand, sufficiently indepen- dent control signals can be derived from the myoelectric signals recorded from the Fig. Moreover, control is largely intuitive, since users invoke muscle activity in the pectoralis in a similar way as they did to invoke movement of their healthy hand and arm.
Because of this intuitive control feature, TMR based prosthetic devices can also be seen as thought-controlled neuroprostheses. Clearly, TMR holds the promise to improve the operation of complex prosthetic systems.
BCI approaches non-invasive and invasive will need to demonstrate clinical and commercial advantages over TMR approaches in order to be viable. The example with prostheses underscores a problem and an opportunity for BCI research. The problem is that BCIs cannot provide effective control because they cannot provide sufficient reliability and bandwidth information per second.
Similarly, the bandwidth and reliability of modern BCIs is far too low for many other goals that are fairly easy with conventional interfaces. Rapid communication, most computer games, detailed wheelchair navigation, and cell phone dialing are only a few examples of goals that require a regular interface. Does this mean that BCIs will remain limited to severely disabled users? We think not, for several reasons. Second, BCIs are advancing very rapidly. Third, some people may use BCIs even though they are slow because they are attracted to the novel and futuristic aspect of BCIs.
Many research labs have demonstrated that computer games such as Pong, Tetris, or Pacman can be con- trolled by BCIs [46] and that rather complex computer applications like Google Earth can be navigated by BCI [47]. Users could control these systems more effec- tively with a keyboard, but may consider a BCI more fun or engaging. Motivated by the advances in BCI research over the last years, companies have started to consider BCIs as possibility to augment human—computer interaction.
This interest is under- lined by a number of patents and new products, which are further discussed in the concluding chapter of this book. We are especially optimistic about BCIs for new user groups for two reasons. First, BCIs are becoming more reliable and easier to apply.
New users will need a BCI that is robust, practical, and flexible. All applications should function outside the lab, using only a minimal number of EEG channels ideally only one channel , a simple and easy to setup BCI system, and a stable EEG pattern suitable for online detection.
The Graz BCI lab developed an example of such a system. It uses a spe- cific motor imagery-based BCI designed to detect the short-lasting ERS pattern in the beta band after imagination of brisk foot movement in a single EEG channel [48].
Most present-day BCI applications focus on communication or control. New user groups might adopt BCIs that instead focus on neurorehabilitation. A BCI for neurorehabilitation is a new concept that uses neurofeedback and oper- ant conditioning in a different way than a conventional BCI. For communication and control applications, neurofeedback is necessary to learn to use a BCI.
The ultimate goal for these applications is to achieve the best possible control or communica- tion performance. Neurofeedback is only a means to that end. In neurofeedback and neuro-rehabilitation applications, the situation is different. In these cases, the training itself is the actual application. BCIs are the most advanced neurofeedback systems available. It might be the case that modern BCI technology used in neu- rofeedback applications to treat neurological or neuropsychological disorders such as epilepsy, autism or ADHD is more effective than conventional neurofeedback.
Neuro-rehabilitation of stroke is another possible BCI neurorehabilitation appli- cation. Here, the goal is to apply neuro-physiological regultion to foster cortical reorganization and compensatory cerebral activation of brain regions not affected by stroke [54].
Chapter Brain—Computer Interface in Neurorehabilitation of this book discusses this new direction in more detail. A conventional BCI monitors brain activity and detects certain brain patterns that are interpreted and translated to commands for communication or control tasks.
BCIs may rely on different tech- nologies to measure brain activity. BCIs also vary in other ways, including the mental strategy used for control, interface parameters such as the mode of opera- tion synchronous or asynchronous , feedback type, signal processing method, and application.
Figure 10 gives a comprehensive overview of BCI components and how they relate to each other. BCI research over the last 20 years has focused on developing communication and control technologies for people suffering from severe neuromuscular disor- ders that can lead to complete paralysis or the locked-in state.
The objective is to provide these users with basic assistive devices. Although the bandwidth of present- days BCIs is very limited, BCIs are of utmost importance for people suffering from complete locked-in syndrome, because BCIs are their only effective means of communication and control. BCI systems may provide communication and control to users with less severe dis- abilities, and even healthy users in some situations. BCIs may also provide new means of treating stroke, autism, and other disorders.
As BCIs become more popular with different user groups, increasing commercial possibilities will likely encourage new applied research efforts that will make BCIs even more practical. Consumer demand for reduced cost, increased performance, and greater flexibility and robustness may contribute substantially to making BCIs into more mainstream tools. Our goal in this chapter was to provide a readable, friendly overview to BCIs. We also wanted to include resources with more information, such as other chapters in this book and other papers.
While most BCIs portrayed in science fiction are way beyond modern technology, there are many significant advances being made today, and reasonable progress is likely in the near future. We hope this chapter, and this book, convey not only some important information about BCIs, but also the sense of enthusiasm that we authors and most BCI researchers share about our promising and rapidly developing research field.
References 1.
0コメント