Eye gaze collaboration with Brain-Computer Interfaces – using both modalities for more robust interaction

Gaye Lightbody, Chris Brennan, Paul McCullagh, Leo Galway

Research output: Chapter in Book/Report/Conference proceedingChapterpeer-review


In this chapter, we discuss the motivation for the hybrid Brain-Computer Interface (BCI), and review progress toward more robust user interaction from existing studies. In addition, we discuss the design and development of a hybrid Brain-Computer Interface (hBCI) example that combines two symbiotic modalities: Steady State Visual Evoked Potential and eye gaze technology. By adopting a modular design, we show that it has been possible to implement such hybridisation by integrating mostly existing software components and, indeed, facilitate future updates to the system that will be necessary as hardware, software and interfaces continue to evolve.
Original languageEnglish
Title of host publicationBRAIN-COMPUTER INTERFACES HANDBOOK Technological and Theoretical Advances
EditorsChang S. Nam, Anton Nijholt, Fabien Lotte
PublisherTaylor & Francis
ISBN (Print)9781498773430
Publication statusPublished - 24 Jan 2018

Bibliographical note

Reference text: Abdulkader, S. N., Atia, A. and Mostafa, M. S. M. (2015) ‘Brain computer interfacing: Applications and challenges’, Egyptian Informatics Journal, pp. 213–230. doi: 10.1016/j.eij.2015.06.002.
Ahn, M. and Jun, S. C. (2015) ‘Performance variation in motor imagery brain-computer interface: A brief review’, Journal of Neuroscience Methods, 243, pp. 103–110. doi: 10.1016/j.jneumeth.2015.01.033.
Allison, B. (2011) ‘Trends in BCI research’, XRDS: Crossroads, The ACM Magazine for Students, 18(1), p. 18. doi: 10.1145/2000775.2000784.
Allison, B., Dunne, S., Leeb, R., Mill, R. and Nijholt, A. (2013) ‘Towards Practical Brain-Computer Interfaces’. Edited by B. Z. Allison, S. Dunne, R. Leeb, J. Del R. Millán, and A. Nijholt. Berlin, Heidelberg: Springer Berlin Heidelberg (Biological and Medical Physics, Biomedical Engineering), pp. 1–13. doi: 10.1007/978-3-642-29746-5.
Allison, B., Jin, J., Zhang, Y. and Wang, X. (2014) ‘A four-choice hybrid P300/SSVEP BCI for improved accuracy’, Brain-Computer Interfaces, 1(1), pp. 17–26. doi: 10.1080/2326263X.2013.869003.
Allison, B., Leeb, R., Brunner, C., Müller-Putz, G., Bauernfeind, G., Kelly, J. W. and Neuper, C. (2012) ‘Toward smarter BCIs: extending BCIs through hybridization and intelligent control.’, Journal of neural engineering, 9(1), p. 13001. doi: 10.1088/1741-2560/9/1/013001.
Allison, B., Luth, T., Valbuena, D., Teymourian, A., Volosyak, I. and Graser, A. (2010) ‘BCI demographics: How many (and what kinds of) people can use an SSVEP BCI?’, IEEE Transactions on Neural Systems and Rehabilitation Engineering, 18(2), pp. 107–116. doi: 10.1109/TNSRE.2009.2039495.
Allison, B., McFarland, D. J., Schalk, G., Zheng, S. D., Jackson, M. M. and Wolpaw, J. R. (2008) ‘Towards an independent brain-computer interface using steady state visual evoked potentials’, Clinical Neurophysiology, 119(2), pp. 399–408. doi: 10.1016/j.clinph.2007.09.121.
Allison, B., Millan, J. del R., Nijholt, A., Dunne, S., Leeb, R., Whitmer, D., Poel, M. and Neuper, C. (2012) Future BNCI: A Roadmap for Future Direction in Brain / Neuronal Computer Interaction Research, Future Directions in Brain/Neuronal Computer Interaction (Future BNCI). doi: 10.1080/2326263X.2015.1008956.
Amiri, S., Fazel-Rezai, R. and Asadpour, V. (2012) ‘A review of hybrid brain-computer interface systems.’, Advances in Human-Computer Interaction, 2013, pp. 1–8. doi: 10.1155/2013/187024.
Amiri, S., Rabbi, A., Azinfar, L. and Fazel-Rezai, R. (2013) ‘A Review of P300, SSVEP, and Hybrid P300/SSVEP Brain- Computer Interface Systems’, Brain-Computer Interface Systems - Recent Progress and Future Prospects, 2013, pp. 1–8. doi: 10.5772/56135.
Baccino, T. and Manunta, Y. (2005) ‘Eye-fixation-related potentials: Insight into parafoveal processing’, Journal of Psychophysiology, 19(3), pp. 204–215. doi: 10.1027/0269-8803.19.3.204.
Blankertz, B., Acqualagna, L., Dähne, S., Haufe, S., Schultze-Kraft, M., Sturm, I., Ušćumlić, M., Wenzel, M., Curio, G., Mueller, K. R., Cumlí C, M. U. and Müller, K.-R. (2016) ‘The Berlin Brain-Computer Interface: Progress Beyond Communication and Control’, Front. Neurosci, 10(November). doi: 10.3389/fnins.2016.00530.
Blankertz, B., Tangermann, M., Vidaurre, C., Fazli, S., Sannelli, C., Haufe, S., Maeder, C., Ramsey, L., Sturm, I., Curio, G. and Müller, K.-R. (2010) ‘The Berlin Brain–Computer Interface: Non-Medical Uses of BCI Technology’, Frontiers in Neuroscience. Frontiers, 4, p. 198. doi: 10.3389/fnins.2010.00198.
BRAIN Project (2011) EU project BRAIN (ICT-2007-224156), Project web site. Available at: http://www.brain-project.org/ (Accessed: 5 April 2017).
Brennan, C., McCullagh, P., Lightbody, G. and Galway, L. (2017) ‘Evaluation of an SSVEP and eye gaze hybrid BCI’, in 7th Graz Brain-Computer Interface Conference. Graz.
Brunner, C., Andreoni, G., Bianchi, L., Blankertz, B., Breitwieser, C., Kanoh, S., Kothe, C. A., Lécuyer, A., Makeig, S., Mellinger, J., Perego, P., Renard, Y., Schalk, G., Susila, I. P., Venthur, B. and Müller-Putz, G. R. (2012) ‘BCI Software Platforms’, in. Springer Berlin Heidelberg, pp. 303–331. doi: 10.1007/978-3-642-29746-5_16.
Brunner, C., Birbaumer, N., Blankertz, B., Guger, C., Kübler, A., Mattia, D., Millán, J. del R., Miralles, F., Nijholt, A., Opisso, E., Ramsey, N., Salomon, P. and Müller-Putz, G. R. (2015) ‘BNCI Horizon 2020: towards a roadmap for the BCI community’, Brain-Computer Interfaces, 2(1), pp. 1–10. doi: 10.1080/2326263X.2015.1008956.
Brunner, P., Joshi, S., Briskin, S., Wolpaw, J. R., Bischof, H. and Schalk, G. (2010) ‘Does the “P300” speller depend on eye gaze?’, Journal of neural engineering, 7(5), p. 56013. doi: 10.1088/1741-2560/7/5/056013.
Buttfield, A., Ferrez, P. W. and Del R. Millan, J. (2006) ‘Towards a Robust BCI: Error Potentials and Online Learning’, IEEE Transactions on Neural Systems and Rehabilitation Engineering, 14(2), pp. 164–168. doi: 10.1109/TNSRE.2006.875555.
Cavrini, F., Quitadamo, L. R., Bianchi, L. and Saggio, G. (2014) ‘Combination of Classifiers using the Fuzzy Integral for Uncertainty Identification and Subject Specific Optimization - Application to Brain-Computer Interface’, in Proceedings of the International Conference on Fuzzy Computation Theory and Applications. SCITEPRESS - Science and and Technology Publications, pp. 14–24. doi: 10.5220/0005035900140024.
Cecotti, H. (2011) ‘Spelling with non-invasive Brain-Computer Interfaces--current and future trends.’, Journal of physiology, Paris. Elsevier Ltd, 105(1–3), pp. 106–14. doi: 10.1016/j.jphysparis.2011.08.003.
Cheng, M., Gao, X., Gao, S. and Xu, D. (2001) ‘Multiple color stimulus induced steady state visual evoked potentials’, in 2001 Conference Proceedings of the 23rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE, pp. 1012–1014. doi: 10.1109/IEMBS.2001.1020359.
Choi, B. and Jo, S. (2013) ‘A Low-Cost EEG System-Based Hybrid Brain-Computer Interface for Humanoid Robot Navigation and Recognition’, PLoS ONE, 8(9). doi: 10.1371/journal.pone.0074583.
Dong, X., Wang, H., Chen, Z. and Shi, B. E. (2015) ‘Hybrid Brain Computer Interface via Bayesian Integration of EEG and Eye Gaze’, 7th Annual International IEEE EMBS Conference on Neural Engineering, 1, pp. 22–24. doi: 10.1109/NER.2015.7146582.
Durka, P., Kus, R. and Zygierewicz, J. (2009) ‘High-frequency SSVEP responses parametrized by multichannel matching pursuit’, … Abstract: 2nd INCF …. doi: 10.3389/conf.neuro.11.2009.08.055.
Van Erp, J. B. F., Lotte, F. and Tangermann, M. (2012) ‘Brain-computer interfaces: Beyond medical applications’, Computer, 45(4), pp. 26–34. doi: 10.1109/MC.2012.107.
Eugster, M. J. A., Ruotsalo, T., Spapé, M. M., Barral, O., Ravaja, N., Jacucci, G. and Kaski, S. (2016) ‘Natural brain-information interfaces: Recommending information by relevance inferred from human brain signals’, Scientific Reports. Nature Publishing Group, 6(arXiv:1607.03502), p. 38580. doi: 10.1038/srep38580.
Faller, J., Torrellas, S., Miralles, F., Holzner, C., Kapeller, C., Guger, C., Bund, J., Muller-Putz, G. R. and Scherer, R. (2012) ‘Prototype of an auto-calibrating, context-aware, hybrid brain-computer interface’, Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS, pp. 1827–1830. doi: 10.1109/EMBC.2012.6346306.
Farwell, L. A. and Donchin, E. (1988) ‘Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials.’, Electroencephalography and Clinical Neurophysiology, 70, pp. 510–523. doi: 10.1016/0013-4694(88)90149-6.
Finke, A., Essig, K., Marchioro, G. and Ritter, H. (2016) ‘Toward FRP-Based Brain-Machine Interfaces Single-Trial Classification of Fixation- Related Potentials’, PLoS ONE, 11(1), p. e0146848. doi: 10.1371/ journal.pone.0146848.
Fisher, R. S., Harding, G., Erba, G., Barkley, G. L. and Wilkins, A. (2005) ‘Photic- and pattern-induced seizures: A review for the epilepsy foundation of america working group’, Epilepsia. Blackwell Science Inc, pp. 1426–1441. doi: 10.1111/j.1528-1167.2005.31405.x.
Friman, O., Luth, T., Volosyak, I. and Graser, A. (2007) ‘Spelling with Steady-State Visual Evoked Potentials’, in Proceedings of International IEEE EMBS Conference on Neural Engineering. IEEE, pp. 354–357. doi: 10.1109/CNE.2007.369683.
Galway, L., Brennan, C., McCullagh, P. and Lightbody, G. (2015) ‘BCI and eye gaze: Collaboration at the interface’, in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Springer, Cham, pp. 199–210. doi: 10.1007/978-3-319-20816-9_20.
Gao, S., Wang, Y., Gao, X. and Hong, B. (2014) ‘Visual and auditory brain-computer interfaces’, IEEE Transactions on Biomedical Engineering, 61(5), pp. 1436–1447. doi: 10.1109/TBME.2014.2300164.
Gao, X., Xu, D., Cheng, M. and Gao, S. (2003) ‘A bci-based environmental controller for the motion-disabled’, IEEE Transactions on Neural Systems and Rehabilitation Engineering, 11(2), pp. 137–140. doi: 10.1109/TNSRE.2003.814449.
Garcia-Molina, G. and Mihajlovic, V. (2010) ‘Spatial filters to detect steady-state visual evoked potentials elicited by high frequency stimulation: BCI application’, Biomedizinische Technik/Biomedical Engineering, 55(3), pp. 173–182. doi: 10.1515/bmt.2010.013.
Garcia-Molina, G., Zhu, D. and Abtahi, S. (2010) ‘Phase Detection in a Visual-Evoked-Potential Based Brain–Computer Interface’, in 18th European Signal Processing Conference (EUSIPCO), pp. 949–953.
Gartner (2016) Gartner’s 2016 Hype Cycle for Emerging Technologies. Available at: http://www.gartner.com/newsroom/id/3412017 (Accessed: 5 April 2017).
Gembler, F., Stawicki, P. and Volosyak, I. (2015a) ‘Autonomous parameter adjustment for SSVEP-Based BCIs with a novel BCI wizard’, Frontiers in Neuroscience. Frontiers, 9(DEC), p. 474. doi: 10.3389/fnins.2015.00474.
Gembler, F., Stawicki, P. and Volosyak, I. (2015b) ‘How many targets can be used in a SSVEP-based BCI-system’, in Volosyak, I. (ed.) EBCI 2015 Workshop. Aachan: Shakler-Verlag, pp. 53–62.
Gerson, A. D., Parra, L. C. and Sajda, P. (2006) ‘Cortically Coupled Computer Vision for Rapid Image Search’, IEEE Transactions on Neural Systems and Rehabilitation Engineering, 14(2), pp. 174–179. doi: 10.1109/TNSRE.2006.875550.
Gollee, H., Volosyak, I., McLachlan, A. J., Hunt, K. J. and Gräser, A. (2010) ‘An SSVEP-Based Brain–Computer Interface for the Control of Functional Electrical Stimulation’, IEEE Transactions on Biomedical Engineering, 57(8), pp. 1847–1855. doi: 10.1109/TBME.2010.2043432.
Guger, C., Daban, S., Sellers, E., Holzner, C., Krausz, G., Carabalona, R., Gramatica, F. and Edlinger, G. (2009) ‘How many people are able to control a P300-based brain-computer interface (BCI)?’, Neuroscience letters, 462(1), pp. 94–8. doi: 10.1016/j.neulet.2009.06.045.
Hardoon, D. R., Szedmak, S. and Shawe-Taylor, J. (2004) ‘Canonical Correlation Analysis: An Overview with Application to Learning Methods’, Neural Computation. MIT Press 238 Main St., Suite 500, Cambridge, MA 02142-1046 USA journals-info@mit.edu, 16(12), pp. 2639–2664. doi: 10.1162/0899766042321814.
Henshaw, J., Liu, W. and Romano, D. (2014) ‘Problem solving using hybrid brain-computer interface methods: A review’, in 2014 5th IEEE Conference on Cognitive Infocommunications (CogInfoCom). IEEE, pp. 215–219. doi: 10.1109/CogInfoCom.2014.7020448.
Hild, J., Putze, F., Kaufman, D., Kühnle, C., Schultz, T. and Beyerer, J. (2014) ‘Spatio-Temporal Event Selection in Basic Surveillance Tasks using Eye Tracking and EEG’, Proceedings of the 2014 Workshop on Eye Gaze in Intelligent Human Machine Interaction, pp. 3–8. doi: 10.1145/2666642.2666645.
Hinterberger, T., Schmidt, S., Neumann, N., Mellinger, J., Blankertz, B., Curio, G. and Birbaumer, N. (2004) ‘Brain-Computer Communication and Slow Cortical Potentials’, IEEE Transactions on Biomedical Engineering, 51(6), pp. 1011–1018. doi: 10.1109/TBME.2004.827067.
Holmqvist, K., Nyström, M., Andersson, R., Dewhurst, R., Jarodzka, H. and Van de Weijer, J. (2011) Eye tracking: A comprehensive guide to methods and measures. 1st Editio. OUP Oxford.
Huang, B., Lo, A. H. P. and Shi, B. E. (2013) ‘Integrating EEG information improves performance of gaze based cursor control’, International IEEE/EMBS Conference on Neural Engineering, NER, pp. 415–418. doi: 10.1109/NER.2013.6695960.
Hwang, H. J., Kim, S., Choi, S. and Im, C. H. (2013) ‘EEG-Based Brain-Computer Interfaces: A Thorough Literature Survey’, International Journal of Human-Computer Interaction. Taylor & Francis, 29(12), pp. 814–826. doi: 10.1080/10447318.2013.780869.
Hwang, H. J., Lim, J. H., Jung, Y. J., Choi, H., Lee, S. W. and Im, C. H. (2012) ‘Development of an SSVEP-based BCI spelling system adopting a QWERTY-style LED keyboard’, Journal of Neuroscience Methods, 208(1), pp. 59–65. doi: 10.1016/j.jneumeth.2012.04.011.
ISO (1998) ISO 9241-11:1998(en), Ergonomic requirements for office work with visual display terminals (VDTs) — Part 11: Guidance on usability. Available at: https://www.iso.org/obp/ui/#iso:std:iso:9241:-11:ed-1:v1:en (Accessed: 15 June 2017).
Jarosiewicz, B., Masse, N., Bacher, D. and Sarma, A. (2015) ‘Context-Aware Self-Calibration’.
Jin, J., Allison, B., Zhang, Y., Wang, X. and Cichocki, A. (2014) ‘An ERP-based BCI using an oddball paradigm with different faces and reduced errors in critical functions.’, International journal of neural systems, 24(8), p. 1450027. doi: 10.1142/S0129065714500270.
Kelly, S. P., Lalor, E. C., Finucane, C., McDarby, G. and Reilly, R. B. (2005) ‘Visual spatial attention control in an independent brain-computer interface.’, IEEE transactions on bio-medical engineering, 52(9), pp. 1588–96. doi: 10.1109/TBME.2005.851510.
Kim, D.-W., Lee, J.-C., Park, Y.-M., Kim, I.-Y. and Im, C.-H. (2012) ‘Auditory brain-computer interfaces (BCIs) and their practical applications’, Biomedical Engineering Letters, 2(1), pp. 13–17. doi: 10.1007/s13534-012-0051-1.
Kluge, T. and Hartmann, M. (2007) ‘Phase Coherent Detection of Steady-State Evoked Potentials: Experimental Results and Application to Brain-Computer Interfaces’, in 2007 3rd International IEEE/EMBS Conference on Neural Engineering. IEEE, pp. 425–429. doi: 10.1109/CNE.2007.369700.
Lalor, E. C., Kelly, S. P., Finucane, C., Burke, R., Smith, R., Reilly, R. B. and Mcdarby, G. (2005) ‘E. C. Lalor et al; Steady-State VEP-Based Brain-Computer Interface Control in an Immersive 3D Gaming Environment’, EURASIP Journal on Applied Signal Processing, 19, pp. 3156–3164.
Leeb, R., Sagha, H., Chavarriaga, R. and Millán, J. del R. (2011) ‘A hybrid brain–computer interface based on the fusion of electroencephalographic and electromyographic activities’, Journal of Neural Engineering. IOP Publishing, 8(2), p. 25011. doi: 10.1088/1741-2560/8/2/025011.
Lin, K., Cinetto, A., Wang, Y., Chen, X., Gao, S. and Gao, X. (2016) ‘An online hybrid BCI system based on SSVEP and EMG.’, Journal of neural engineering. IOP Publishing, 13(2), p. 26020. doi: 10.1088/1741-2560/13/2/026020.
Lin, Z., Zhang, C., Wu, W. and Gao, X. (2007) ‘Frequency recognition based on canonical correlation analysis for SSVEP-based BCIs’, IEEE Transactions on Biomedical Engineering, 54(6), pp. 1172–1176. doi: 10.1109/TBME.2006.889197.
Manyakov, N. V, Chumerin, N., Robben, A., Combaz, A., van Vliet, M. and Van Hulle, M. M. (2013) ‘Sampled sinusoidal stimulation profile and multichannel fuzzy logic classification for monitor-based phase-coded SSVEP brain–computer interfacing’, Journal of Neural Engineering, 10(3), p. 36011. doi: 10.1088/1741-2560/10/3/036011.
Meena, Y. K., Cecotti, H., Wong-Lin, K. and Prasad, G. (2015) ‘Towards increasing the number of commands in a hybrid brain-computer interface with combination of gaze and motor imagery’, Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS, 2015–Novem, pp. 506–509. doi: 10.1109/EMBC.2015.7318410.
Millán, J. D. R., Rupp, R., Müller-Putz, G. R., Murray-Smith, R., Giugliemma, C., Tangermann, M., Vidaurre, C., Cincotti, F., Kübler, A., Leeb, R., Neuper, C., Müller, K. R. and Mattia, D. (2010) ‘Combining brain-computer interfaces and assistive technologies: State-of-the-art and challenges’, Frontiers in Neuroscience, 4(SEP), pp. 1–15. doi: 10.3389/fnins.2010.00161.
Morash, V., Bai, O., Furlani, S., Lin, P. and Hallett, M. (2008) ‘Classifying EEG signals preceding right hand, left hand, tongue, and right foot movements and motor imageries’, Clinical Neurophysiology. International Federation of Clinical Neurophysiology, 119(11), pp. 2570–2578. doi: 10.1016/j.clinph.2008.08.013.
Mukesh, T. M. S., Jaganathan, V. and Reddy, M. R. (2006) ‘A novel multiple frequency stimulation method for steady state VEP based brain computer interfaces’, Physiological Measurement. IOP Publishing, 27(1), pp. 61–71. doi: 10.1088/0967-3334/27/1/006.
Müller-Putz, G. R., Scherer, R., Brauneis, C. and Pfurtscheller, G. (2005) ‘Steady-state visual evoked potential (SSVEP)-based communication: impact of harmonic frequency components’, Journal of Neural Engineering, 2(4), pp. 123–130. doi: 10.1088/1741-2560/2/4/008.
Nielsen, J. (1995) ‘10 Usability Heuristics for User Interface Design’, Conference companion on Human factors in computing systems CHI 94. Nielsen Norman Group, pp. 152–158. doi: 10.1145/191666.191729.
Nijboer, F., Allison, B., Dunne, S., Bos, D., Nijholt, A. and Haselager, P. (2011) ‘A preliminary survey on the perception of marketability of Brain-Computer Interfaces (BCI) and initial development of a repository of BCI companies’, 5th International Brain-Computer Interface Conference, BCI 2011, pp. 344–347.
Nikolaev, A. R., Meghanathan, R. N. and van Leeuwen, C. (2016) ‘Combining EEG and eye movement recording in free viewing: Pitfalls and possibilities’, Brain and Cognition, 107, pp. 55–83. doi: 10.1016/j.bandc.2016.06.004.
Pasqualotto, E., Matuz, T., Federici, S., Ruf, C. A., Bartl, M., Olivetti Belardinelli, M., Birbaumer, N. and Halder, S. (2015) ‘Usability and Workload of Access Technology for People With Severe Motor Impairment: A Comparison of Brain-Computer Interfacing and Eye Tracking.’, Neurorehabilitation and neural repair, 29(10), pp. 950–7. doi: 10.1177/1545968315575611.
Pastor, M. A., Artieda, J., Arbizu, J., Valencia, M. and Masdeu, J. C. (2003) ‘Human cerebral activation during steady-state visual-evoked responses.’, The Journal of neuroscience : the official journal of the Society for Neuroscience, 23(37), pp. 11621–7. doi: 23/37/11621 [pii].
Pfurtscheller, G., Allison, B., Brunner, C., Bauernfeind, G., Solis-Escalante, T., Scherer, R., Zander, T. O., Mueller-Putz, G., Neuper, C. and Birbaumer, N. (2010) ‘The hybrid BCI.’, Frontiers in neuroscience, 4(April), p. 30. doi: 10.3389/fnpro.2010.00003.
Pfurtscheller, G. and Neuper, C. (2001) ‘Motor imagery and direct brain- computer communication’, Proceedings of the IEEE, 89(7), pp. 1123–1134. doi: 10.1109/5.939829.
Pfurtscheller, G., Solis-escalante, T., Member, S., Ortner, R., Linortner, P. and Müller-putz, G. R. (2010) ‘Self-Paced Operation of an SSVEP-Based Orthosis With and Without an Imagery-Based “ Brain Switch :” A Feasibility Study Towards a Hybrid BCI’, 18(4), pp. 409–414.
Putze, F., Amma, C. and Schultz, T. (2015) ‘Design and Evaluation of a Self-Correcting Gesture Interface based on Error Potentials from EEG’, in Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems - CHI ’15. New York, New York, USA: ACM Press, pp. 3375–3384. doi: 10.1145/2702123.2702184.
Putze, F., Popp, J., Hild, J., Beyerer, J. and Schultz, T. (2016) ‘Intervention-free selection using EEG and eye tracking’, in Proceedings of the 18th ACM International Conference on Multimodal Interaction - ICMI 2016. New York, New York, USA: ACM Press, pp. 153–160. doi: 10.1145/2993148.2993199.
Rama, P. and Baccino, T. (2010) ‘Eye fixation–related potentials (EFRPs) during object identification’, Visual Neuroscience, 27(5–6), pp. 187–192. doi: 10.1017/S0952523810000283.
Regan, D. (1988) ‘Human visual evoked potentials’, in Handbook of electroencephalography and clinical neurophysiology. Vol 3 ed T. Elsevier, pp. 159–244.
Riccio, A., Mattia, D., Simione, L., Olivetti, M. and Cincotti, F. (2012) ‘Eye-gaze independent EEG-based brain-computer interfaces for communication.’, Journal of neural engineering, 9(4), p. 45001. doi: 10.1088/1741-2560/9/4/045001.
Rutkowski, T. M., Mori, H., Matsumoto, Y., Cai, Z., Chang, M., Nishikawa, N., Makino, S. and Mori, K. (2012) ‘Haptic bci paradigm based on somatosensory evoked potential’, pp. 1–2.
Saproo, S., Faller, J., Shih, V., Sajda, P., Waytowich, N. R., Bohannon, A., Lawhern, V. J., Lance, B. J. and Jangraw, D. (2016) ‘Cortically Coupled Computing: A New Paradigm for Synergistic Human-Machine Interaction’, Computer, 49(9), pp. 60–68. doi: 10.1109/MC.2016.294.
Schettini, F., Aloise, F., Aricò, P., Salinari, S., Mattia, D. and Cincotti, F. (2014) ‘Self-calibration algorithm in an asynchronous P300-based brain-computer interface.’, Journal of neural engineering. IOP Publishing, 11(3), p. 35004. doi: 10.1088/1741-2560/11/3/035004.
Schmidt, N. M., Blankertz, B. and Treder, M. S. (2012) ‘Online detection of error-related potentials boosts the performance of mental typewriters.’, BMC neuroscience, 13(1), p. 19. doi: 10.1186/1471-2202-13-19.
Sellers, E. W., Krusienski, D. J., McFarland, D. J., Vaughan, T. M. and Wolpaw, J. R. (2006) ‘A P300 event-related potential brain-computer interface (BCI): The effects of matrix size and inter stimulus interval on performance’, Biological Psychology, 73(3), pp. 242–252. doi: 10.1016/j.biopsycho.2006.04.007.
Sellers, E. W., Vaughan, T. M. and Wolpaw, J. R. (2010) ‘A brain-computer interface for long-term independent home use.’, Amyotrophic lateral sclerosis : official publication of the World Federation of Neurology Research Group on Motor Neuron Diseases, 11(5), pp. 449–55. doi: 10.3109/17482961003777470.
Sereno, S. and Rayner, K. (2003) ‘Measuring word recognition in reading: eye movements and event-related potentials’, Trends in Cognitive Sciences, 7(11), pp. 489–493. doi: 10.1016/j.tics.2003.09.010.
Shahid, S., Prasad, G. and Sinha, R. K. (2011) ‘On fusion of heart and brain signals for hybrid BCI’, in 2011 5th International IEEE/EMBS Conference on Neural Engineering. IEEE, pp. 48–52. doi: 10.1109/NER.2011.5910486.
Shishkin, S. L., Nuzhdin, Y. O., Svirin, E. P., Trofimov, A. G., Fedorova, A. A., Kozyrskiy, B. L. and Velichkovsky, B. M. (2016) ‘EEG Negativity in Fixations Used for Gaze-Based Control: Toward Converting Intentions into Actions with an Eye-Brain-Computer Interface’, Frontiers in Neuroscience, 10(November). doi: 10.3389/fnins.2016.00528.
Spüler, M., Bensch, M., Kleih, S., Rosenstiel, W., Bogdan, M. and Kübler, A. (2012) ‘Online use of error-related potentials in healthy users and people with severe motor impairment increases performance of a P300-BCI.’, Clinical neurophysiology : official journal of the International Federation of Clinical Neurophysiology, 123(7), pp. 1328–37. doi: 10.1016/j.clinph.2011.11.082.
Treder, M. S. and Blankertz, B. (2010) ‘(C)overt attention and visual speller design in an ERP-based brain-computer interface.’, Behavioral and brain functions : BBF, 6, p. 28. doi: 10.1186/1744-9081-6-28.
Ušćumlić, M. and Blankertz, B. (2016) ‘Active visual search in non-stationary scenes: coping with temporal variability and uncertainty.’, Journal of neural engineering. IOP Publishing, 13(1), p. 16015. doi: 10.1088/1741-2560/13/1/016015.
Valbuena, D., Volosyak, I., Malechka, T. and Gräser, A. (2011) ‘A novel EEG acquisition system for Brain Computer Interfaces’, Journal of Bioelectromagnetism, 13(2), pp. 74–75.
Vaughan, T. M., McFarland, D. J., Schalk, G., Sarnacki, W. a, Krusienski, D. J., Sellers, E. W. and Wolpaw, J. R. (2006) ‘The Wadsworth BCI Research and Development Program: at home with BCI.’, IEEE transactions on neural systems and rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society, 14(2), pp. 229–33. doi: 10.1109/TNSRE.2006.875577.
Vilimek, R. and Zander, T. O. (2009) ‘BC(eye): Combining Eye-Gaze Input with Brain-Computer Interaction’, in. Springer Berlin Heidelberg, pp. 593–602. doi: 10.1007/978-3-642-02710-9_66.
Volosyak, I. (2011) ‘SSVEP-based Bremen-BCI interface--boosting information transfer rates.’, Journal of neural engineering, 8(3), p. 36020. doi: 10.1088/1741-2560/8/3/036020.
Volosyak, I., Cecotti, H., Valbuena, D. and Graser, A. (2009) ‘Evaluation of the Bremen SSVEP based BCI in real world conditions’, in 2009 IEEE International Conference on Rehabilitation Robotics. IEEE, pp. 322–331. doi: 10.1109/ICORR.2009.5209543.
Volosyak, I., Gembler, F. and Stawicki, P. (2017) ‘Age-related Differences in SSVEP-based BCI performance’, Neurocomputing. doi: 10.1016/j.neucom.2016.08.121.
Volosyak, I., Valbuena, D., Lüth, T., Malechka, T. and Gräser, A. (2011) ‘BCI demographics II: how many (and what kinds of) people can use a high-frequency SSVEP BCI?’, IEEE transactions on neural systems and rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society, 19(3), pp. 232–9. doi: 10.1109/TNSRE.2011.2121919.
Wang, Y., Gao, X., Hong, B., Jia, C. and Gao, S. (2008) ‘Brain-computer interfaces based on visual evoked potentials: Feasibility of practical system designs’, IEEE Engineering in Medicine and Biology Magazine, pp. 64–71. doi: 10.1109/MEMB.2008.923958.
Ware, M. P., Lightbody, G., McCullagh, P. J., Mulvenna, M. D., Martin, S. and Thomson, E. (2014) ‘A method for assessing the usability of an on screen display for a brain-computer interface’, International Journal of Computers in Healthcare, 2(1), p. 43. doi: 10.1504/IJCIH.2014.065811.
Ware, M. P., McCullagh, P. J., McRoberts, A., Lightbody, G., Nugent, C., McAllister, G., Mulvenna, M. D., Thomson, E. and Martin, S. (2010) ‘Contrasting levels of accuracy in command interaction sequences for a domestic brain-computer interface using SSVEP’, 2010 5th Cairo International Biomedical Engineering Conference. IEEE, pp. 150–153.
Wenzel, M. A., Golenia, J.-E. and Blankertz, B. (2016) ‘Classification of Eye Fixation Related Potentials for Variable Stimulus Saliency’, Frontiers in Neuroscience, 10(FEB), pp. 1–14. doi: 10.3389/fnins.2016.00023.
Wilson, J. J. and Palaniappan, R. (2009) ‘Augmenting a SSVEP BCI through single cycle analysis and phase weighting’, in 2009 4th International IEEE/EMBS Conference on Neural Engineering. IEEE, pp. 371–374. doi: 10.1109/NER.2009.5109310.
Wolpaw, J. R., Ramoser, H., McFarland, D. J. and Pfurtscheller, G. (1998) ‘EEG-based communication: Improved accuracy by response verification’, IEEE Transactions on Rehabilitation Engineering, 6(3), pp. 326–333. doi: 10.1109/86.712231.
Yin, E., Zhou, Z., Jiang, J., Chen, F., Liu, Y. and Hu, D. (2014) ‘A speedy hybrid BCI spelling approach combining P300 and SSVEP’, IEEE Transactions on Biomedical Engineering, 61(2), pp. 473–483. doi: 10.1109/TBME.2013.2281976.
Yong, X., Fatourechi, M., Ward, R. K. and Birch, G. E. (2011) ‘The Design of a Point-and-Click System by Integrating a Self-Paced Brain–Computer Interface With an Eye-Tracker’, IEEE Journal on Emerging and Selected Topics in Circuits and Systems, 1(4), pp. 590–602. doi: 10.1109/JETCAS.2011.2175589.
Zander, T., Gaertner, M., Kothe, C. and Vilimek, R. (2010) ‘Combining Eye Gaze Input With a Brain–Computer Interface for Touchless Human–Computer Interaction’, International Journal of Human-Computer Interaction, 27(1), pp. 38–51. doi: 10.1080/10447318.2011.535752.
Zander, T. O. and Kothe, C. (2011) ‘Towards passive brain-computer interfaces: applying brain-computer interface technology to human-machine systems in general.’, Journal of neural engineering, 8(2), p. 25005. doi: 10.1088/1741-2560/8/2/025005.
Zhang, Y. U., Zhou, G., Jin, J., Wang, X. and Cichocki., A. (2014) ‘Frequency recognition in SSVEP-based BCI using multiset Canonical Correlation Analysis’, International Journal of Neural Systems. World Scientific Publishing Company, 24(4), p. 1450013. doi: 10.1142/S0129065714500130.
Zhu, D., Bieger, J., Garcia Molina, G. and Aarts, R. M. (2010) ‘A survey of stimulation methods used in SSVEP-based BCIs’, Computational Intelligence and Neuroscience. Hindawi Publishing Corp., 2010, pp. 1–12. doi: 10.1155/2010/702357.


  • Brain-computer interfaces
  • eye gaze
  • hybrid BCI
  • hBCI


Dive into the research topics of 'Eye gaze collaboration with Brain-Computer Interfaces – using both modalities for more robust interaction'. Together they form a unique fingerprint.

Cite this