visual stimuli essay

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Review Article
• Published: 05 August 2019

The human imagination: the cognitive neuroscience of visual mental imagery

• Joel Pearson   ORCID: orcid.org/0000-0003-3704-5037 1  

Nature Reviews Neuroscience volume  20 ,  pages 624–634 ( 2019 ) Cite this article

33k Accesses

266 Citations

218 Altmetric

Metrics details

• Object vision
• Sensory systems
• Working memory

Mental imagery can be advantageous, unnecessary and even clinically disruptive. With methodological constraints now overcome, research has shown that visual imagery involves a network of brain areas from the frontal cortex to sensory areas, overlapping with the default mode network, and can function much like a weak version of afferent perception. Imagery vividness and strength range from completely absent (aphantasia) to photo-like (hyperphantasia). Both the anatomy and function of the primary visual cortex are related to visual imagery. The use of imagery as a tool has been linked to many compound cognitive processes and imagery plays both symptomatic and mechanistic roles in neurological and mental disorders and treatments.

This is a preview of subscription content, access via your institution

Access options

Access Nature and 54 other Nature Portfolio journals

Get Nature+, our best-value online-access subscription

24,99 € / 30 days

cancel any time

Subscribe to this journal

Receive 12 print issues and online access

176,64 € per year

only 14,72 € per issue

Buy this article

• Purchase on Springer Link
• Instant access to full article PDF

Prices may be subject to local taxes which are calculated during checkout

Similar content being viewed by others

Microdosing with psilocybin mushrooms: a double-blind placebo-controlled study

Federico Cavanna, Stephanie Muller, … Enzo Tagliazucchi

The language network as a natural kind within the broader landscape of the human brain

Evelina Fedorenko, Anna A. Ivanova & Tamar I. Regev

brainlife.io: a decentralized and open-source cloud platform to support neuroscience research

Soichi Hayashi, Bradley A. Caron, … Franco Pestilli

Zeman, A., Dewar, M. & Della Sala, S. Lives without imagery — congenital aphantasia. Cortex 73 , 378–380 (2015). This article documents and coins the term aphantasia, described as the complete lack of visual imagery ability .

Article   PubMed   Google Scholar  

Pearson, J. & Westbrook, F. Phantom perception: voluntary and involuntary non-retinal vision. Trends Cogn. Sci. 19 , 278–284 (2015). This opinion paper proposes a unifying framework for both voluntary and involuntary imagery .

Pearson, J., Naselaris, T., Holmes, E. A. & Kosslyn, S. M. Mental imagery: functional mechanisms and clinical applications. Trends Cogn. Sci. 19 , 590–602 (2015).

Article   PubMed   PubMed Central   Google Scholar  

Egeth, H. E. & Yantis, S. Visual attention: control, representation, and time course. Annu. Rev. Psychol. 48 , 269–297 (1997).

Article   CAS   PubMed   Google Scholar  

Dijkstra, N., Zeidman, P., Ondobaka, S., Gerven, M. A. J. & Friston, K. Distinct top-down and bottom-up brain connectivity during visual perception and imagery. Sci. Rep. 7 , 5677 (2017).

Article   CAS   PubMed   PubMed Central   Google Scholar  

Dentico, D. et al. Reversal of cortical information flow during visual imagery as compared to visual perception. Neuroimage 100 , 237–243 (2014).

Schlegel, A. et al. Network structure and dynamics of the mental workspace. Proc. Natl Acad. Sci. USA 110 , 16277–16282 (2013).

Ranganath, C. & D’Esposito, M. Directing the mind’s eye: prefrontal, inferior and medial temporal mechanisms for visual working memory. Curr. Opin. Neurobiol. 15 , 175–182 (2005).

Yomogida, Y. Mental visual synthesis is originated in the fronto-temporal network of the left hemisphere. Cereb. Cortex 14 , 1376–1383 (2004).

Ishai, A., Ungerleider, L. G. & Haxby, J. V. Distributed neural systems for the generation of visual images. Neuron 28 , 979–990 (2000).

Goebel, R., Khorram-Sefat, D., Muckli, L., Hacker, H. & Singer, W. The constructive nature of vision: direct evidence from functional magnetic resonance imaging studies of apparent motion and motion imagery. Eur. J. Neurosci. 10 , 1563–1573 (1998).

Mellet, E. et al. Functional anatomy of spatial mental imagery generated from verbal instructions. J. Neurosci. 16 , 6504–6512 (1996).

O’Craven, K. M. & Kanwisher, N. Mental imagery of faces and places activates corresponding stimulus- specific brain regions. J. Cogn. Neurosci. 12 , 1013–1023 (2000).

Kosslyn, S. M., Ganis, G. & Thompson, W. L. Neural foundations of imagery. Nat. Rev. Neurosci. 2 , 635–642 (2001).

Hassabis, D., Kumaran, D. & Maguire, E. A. Using imagination to understand the neural basis of episodic memory. J. Neurosci. 27 , 14365–14374 (2007).

Bird, C. M., Capponi, C., King, J. A., Doeller, C. F. & Burgess, N. Establishing the boundaries: the hippocampal contribution to imagining scenes. J. Neurosci. 30 , 11688–11695 (2010).

Hassabis, D., Kumaran, D., Vann, S. D. & Maguire, E. A. Patients with hippocampal amnesia cannot imagine new experiences. Proc. Natl Acad. Sci. USA 104 , 1726–1731 (2007).

Kreiman, G., Koch, C. & Fried, I. Imagery neurons in the human brain. Nature 408 , 357–361 (2000).

Maguire, E. A., Vargha-Khadem, F. & Hassabis, D. Imagining fictitious and future experiences: evidence from developmental amnesia. Neuropsychologia 48 , 3187–3192 (2010).

Kim, S. et al. Sparing of spatial mental imagery in patients with hippocampal lesions. Learn. Mem. 20 , 657–663 (2013).

Pearson, J. & Kosslyn, S. M. The heterogeneity of mental representation: ending the imagery debate. Proc. Natl Acad. Sci. USA 112 , 10089–10092 (2015). This paper proposes an end to the ‘imagery debate’ based on the discussed evidence that imagery can be represented in the brain in a depictive manner .

D’Esposito, M. et al. A functional MRI study of mental image generation. Neuropsychologia 35 , 725–730 (1997).

Knauff, M., Kassubek, J., Mulack, T. & Greenlee, M. W. Cortical activation evoked by visual mental imagery as measured by fMRI. Neuroreport 11 , 3957–3962 (2000).

Trojano, L. et al. Matching two imagined clocks: the functional anatomy of spatial analysis in the absence of visual stimulation. Cereb. Cortex 10 , 473–481 (2000).

Wheeler, M. E., Petersen, S. E. & Buckner, R. L. Memory’s echo: vivid remembering reactivates sensory-specific cortex. Proc. Natl Acad. Sci. USA 97 , 11125 (2000).

Formisano, E. et al. Tracking the mind’s image in the brain I: time-resolved fMRI during visuospatial mental imagery. Neuron 35 , 185–194 (2002).

Sack, A. T. et al. Tracking the mind’s image in the brain II: transcranial magnetic stimulation reveals parietal asymmetry in visuospatial imagery. Neuron 35 , 195–204 (2002).

Le Bihan, D. et al. Activation of human primary visual cortex during visual recall: a magnetic resonance imaging study. Proc. Natl Acad. Sci. USA 90 , 11802–11805 (1993).

Sabbah, P. et al. Functional magnetic resonance imaging at 1.5T during sensorimotor and cognitive task. Eur. Neurol. 35 , 131–136 (1995).

Chen, W. et al. Human primary visual cortex and lateral geniculate nucleus activation during visual imagery. Neuroreport 9 , 3669–3674 (1998).

Ishai, A. Visual imagery of famous faces: effects of memory and attention revealed by fMRI. Neuroimage 17 , 1729–1741 (2002).

Ganis, G., Thompson, W. L. & Kosslyn, S. M. Brain areas underlying visual mental imagery and visual perception: an fMRI study. Cogn. Brain Res. 20 , 226–241 (2004).

Article   Google Scholar  

Klein, I., Paradis, A. L., Poline, J. B., Kossly, S. M. & Le Bihan, D. Transient activity in the human calcarine cortex during visual-mental imagery: an event-related fMRI study. J. Cogn. Neurosci. 12 (Suppl. 2), 15–23 (2000).

Lambert, S., Sampaio, E., Scheiber, C. & Mauss, Y. Neural substrates of animal mental imagery: calcarine sulcus and dorsal pathway involvement — an fMRI study. Brain Res. 924 , 176–183 (2002).

Cui, X., Jeter, C. B., Yang, D., Montague, P. R. & Eagleman, D. M. Vividness of mental imagery: individual variability can be measured objectively. Vision Res. 47 , 474–478 (2007).

Amedi, A., Malach, R. & Pascual-Leone, A. Negative BOLD differentiates visual imagery and perception. Neuron 48 , 859–872 (2005).

Reddy, L., Tsuchiya, N. & Serre, T. Reading the mind’s eye: decoding category information during mental imagery. Neuroimage 50 , 818–825 (2010).

Dijkstra, N., Bosch, S. E. & van Gerven, M. A. J. Vividness of visual imagery depends on the neural overlap with perception in visual areas. J. Neurosci. 37 , 1367–1373 (2017).

Kosslyn, S. M. & Thompson, W. L. When is early visual cortex activated during visual mental imagery? Psychol. Bull. 129 , 723–746 (2003).

Albers, A. M., Kok, P., Toni, I., Dijkerman, H. C. & de Lange, F. P. Shared representations for working memory and mental imagery in early visual cortex. Curr. Biol. 23 , 1427–1431 (2013). This paper shows that both imagery and visual working memory can be decoded in the brain based on training on either, showing evidence of a common brain representation .

Koenig-Robert, R. & Pearson, J. Decoding the contents and strength of imagery before volitional engagement. Sci. Rep. 9 , 3504 (2019). This paper shows that the content and vividness of a mental image can be decoded in the brain up to 11 seconds before an individual decides which pattern to imagine .

Article   PubMed   PubMed Central   CAS   Google Scholar  

Naselaris, T., Olman, C. A., Stansbury, D. E., Ugurbil, K. & Gallant, J. L. A voxel-wise encoding model for early visual areas decodes mental images of remembered scenes. Neuroimage 105 , 215–228 (2015). This study shows that mental imagery content can be decoded in the early visual cortex when the decoding model is based on depictive perceptual features .

Fox, M. D. et al. The human brain is intrinsically organized into dynamic, anticorrelated functional networks. Proc. Natl Acad. Sci. USA 102 , 9673–9678 (2005).

Smith, S. M. et al. Correspondence of the brain’s functional architecture during activation and rest. Proc. Natl Acad. Sci. USA 106 , 13040–13045 (2009).

Østby, Y. et al. Mental time travel and default-mode network functional connectivity in the developing brain. Proc. Natl Acad. Sci. USA 109 , 16800–16804 (2012).

Andrews-Hanna, J. R., Reidler, J. S., Sepulcre, J., Poulin, R. & Buckner, R. L. Functional-anatomic fractionation of the brain’s default network. Neuron 65 , 550–562 (2010).

Hassabis, D. & Maguire, E. A. Deconstructing episodic memory with construction. Trends Cogn. Sci. 11 , 299–306 (2007).

Gerlach, K. D., Spreng, R. N., Gilmore, A. W. & Schacter, D. L. Solving future problems: default network and executive activity associated with goal-directed mental simulations. Neuroimage 55 , 1816–1824 (2011).

Levine, D. N., Warach, J. & Farah, M. Two visual systems in mental imagery. Neurology 35 , 1010 (1985).

Keogh, R. & Pearson, J. The blind mind: no sensory visual imagery in aphantasia. Cortex 105 , 53–60 (2017).

Sakai, K. & Miyashita, Y. Neural organization for the long-term memory of paired associates. Nature 354 , 152–155 (1991).

Messinger, A., Squire, L. R., Zola, S. M. & Albright, T. D. Neuronal representations of stimulus associations develop in the temporal lobe during learning. Proc. Natl Acad. Sci. USA 98 , 12239–12244 (2001).

Schlack, A. & Albright, T. D. Remembering visual motion: neural correlates of associative plasticity and motion recall in cortical area MT. Neuron 53 , 881–890 (2007).

Bannert, M. M. & Bartels, A. Decoding the yellow of a gray banana. Curr. Biol. 23 , 2268–2272 (2013).

Hansen, T., Olkkonen, M., Walter, S. & Gegenfurtner, K. R. Memory modulates color appearance. Nat. Neurosci. 9 , 1367–1368 (2006).

Meng, M., Remus, D. A. & Tong, F. Filling-in of visual phantoms in the human brain. Nat. Neurosci. 8 , 1248–1254 (2005).

Sasaki, Y. & Watanabe, T. The primary visual cortex fills in color. Proc. Natl Acad. Sci. USA 101 , 18251–18256 (2004).

Kok, P., Failing, M. F. & de Lange, F. P. Prior expectations evoke stimulus templates in the primary visual cortex. J. Cogn. Neurosci. 26 , 1546–1554 (2014).

Bergmann, J., Genc, E., Kohler, A., Singer, W. & Pearson, J. Smaller primary visual cortex is associated with stronger, but less precise mental imagery. Cereb. Cortex 26 , 3838–3850 (2016). This study shows that stronger but less precise imagery is associated with a smaller primary and secondary visual cortex .

Stensaas, S. S., Eddington, D. K. & Dobelle, W. H. The topography and variability of the primary visual cortex in man. J. Neurosurg. 40 , 747–755 (1974).

Song, C., Schwarzkopf, D. S. & Rees, G. Variability in visual cortex size reflects tradeoff between local orientation sensitivity and global orientation modulation. Nat. Commun. 4 , 1–10 (2013).

Google Scholar  

Dorph-Petersen, K.-A., Pierri, J. N., Wu, Q., Sampson, A. R. & Lewis, D. A. Primary visual cortex volume and total neuron number are reduced in schizophrenia. J. Comp. Neurol. 501 , 290–301 (2007).

Sack, A. T., van de Ven, V. G., Etschenberg, S., Schatz, D. & Linden, D. E. J. Enhanced vividness of mental imagery as a trait marker of schizophrenia? Schizophr. Bull. 31 , 97–104 (2005).

Maróthi, R. & Kéri, S. Enhanced mental imagery and intact perceptual organization in schizotypal personality disorder. Psychiatry Res. 259 , 433–438 (2018).

Morina, N., Leibold, E. & Ehring, T. Vividness of general mental imagery is associated with the occurrence of intrusive memories. J. Behav. Ther. Exp. Psychiatry 44 , 221–226 (2013).

Chao, L. L., Lenoci, M. & Neylan, T. C. Effects of post-traumatic stress disorder on occipital lobe function and structure. Neuroreport 23 , 412–419 (2012).

Tavanti, M. et al. Evidence of diffuse damage in frontal and occipital cortex in the brain of patients with post-traumatic stress disorder. Neurol. Sci. 33 , 59–68 (2011).

Kavanagh, D. J., Andrade, J. & May, J. Imaginary relish and exquisite torture: the elaborated intrusion theory of desire. Psychol. Rev. 112 , 446–467 (2005).

Ersche, K. D. et al. Abnormal brain structure implicated in stimulant drug addiction. Science 335 , 601–604 (2012).

Song, C., Schwarzkopf, D. S., Kanai, R. & Rees, G. Reciprocal anatomical relationship between primary sensory and prefrontal cortices in the human brain. J. Neurosci. 31 , 9472–9480 (2011).

Panizzon, M. S. et al. Distinct genetic influences on cortical surface area and cortical thickness. Cereb. Cortex 19 , 2728–2735 (2009).

Winkler, A. M. et al. Cortical thickness or grey matter volume? The importance of selecting the phenotype for imaging genetics studies. Neuroimage 53 , 1135–1146 (2010).

Bakken, T. E. et al. Association of common genetic variants in GPCPD1 with scaling of visual cortical surface area in humans. Proc. Natl Acad. Sci. USA 109 , 3985–3990 (2012).

Pearson, J., Rademaker, R. L. & Tong, F. Evaluating the mind’s eye: the metacognition of visual imagery. Psychol. Sci. 22 , 1535–1542 (2011).

Rademaker, R. L. & Pearson, J. Training visual imagery: improvements of metacognition, but not imagery strength. Front. Psychol. 3 , 224 (2012).

Pearson, J. New directions in mental-imagery research: the binocular-rivalry technique and decoding fMRI patterns. Curr. Dir. Psychol. Sci. 23 , 178–183 (2014).

Keogh, R., Bergmann, J. & Pearson, J. Cortical excitability controls the strength of mental imagery. Preprint at bioRxiv https://www.biorxiv.org/content/10.1101/093690v1 (2016).

Terhune, D. B., Tai, S., Cowey, A., Popescu, T. & Kadosh, R. C. Enhanced cortical excitability in grapheme-color synesthesia and its modulation. Curr. Biol. 21 , 2006–2009 (2011).

Chiou, R., Rich, A. N., Rogers, S. & Pearson, J. Exploring the functional nature of synaesthetic colour: dissociations from colour perception and imagery. Cognition 177 , 107–121 (2018).

Arieli, A., Sterkin, A., Grinvald, A. & Aertsen, A. Dynamics of ongoing activity: explanation of the large variability in evoked cortical responses. Science 273 , 1868–1871 (1996).

Wassell, J., Rogers, S. L., Felmingam, K. L., Bryant, R. A. & Pearson, J. Biological psychology. Biol. Psychol. 107 , 61–68 (2015).

Kraehenmann, R. et al. LSD increases primary process thinking via serotonin 2A receptor activation. Front. Pharmacol. 8 , 418–419 (2017).

Article   CAS   Google Scholar  

Pearson, J., Clifford, C. W. G. & Tong, F. The functional impact of mental imagery on conscious perception. Curr. Biol. 18 , 982–986 (2008). This study shows that the content of visual imagery can bias or prime subsequent binocular rivalry; this paper was the basis for using binocular rivalry as a measurement tool for imagery .

Ishai, A. & Sagi, D. Common mechanisms of visual imagery and perception. Science 268 , 1772–1774 (1995).

Tartaglia, E. M., Bamert, L., Mast, F. W. & Herzog, M. H. Human perceptual learning by mental imagery. Curr. Biol. 19 , 2081–2085 (2009). This study shows that training with a purely imaged visual stimulus transfers to improve performance in perceptual tasks .

Lewis, D. E., O’Reilly, M. J. & Khuu, S. K. Conditioning the mind’s eye associative learning with voluntary mental imagery. Clin. Psychol. Sci. 1 , 390–400 (2013).

Laeng, B. & Sulutvedt, U. The eye pupil adjusts to imaginary light. Psychol. Sci. 25 , 188–197 (2014).

Brascamp, J. W., Knapen, T. H. J., Kanai, R., van Ee, R. & van den Berg, A. V. Flash suppression and flash facilitation in binocular rivalry. J. Vis. 7 , 12 (2007).

Tanaka, Y. & Sagi, D. A perceptual memory for low-contrast visual signals. Proc. Natl Acad. Sci. USA 95 , 12729–12733 (1998).

Chang, S., Lewis, D. E. & Pearson, J. The functional effects of color perception and color imagery. J. Vis. 13 , 4 (2013).

Slotnick, S. D., Thompson, W. L. & Kosslyn, S. M. Visual mental imagery induces retinotopically organized activation of early visual areas. Cereb. Cortex 15 , 1570–1583 (2005).

Thirion, B. et al. Inverse retinotopy: inferring the visual content of images from brain activation patterns. Neuroimage 33 , 1104–1116 (2006).

Horikawa, T. & Kamitani, Y. Generic decoding of seen and imagined objects using hierarchical visual features. Nat. Commun. 8 , 15037 (2017).

Keogh, R. & Pearson, J. Mental imagery and visual working memory. PLOS ONE 6 , e29221 (2011).

Keogh, R. & Pearson, J. The sensory strength of voluntary visual imagery predicts visual working memory capacity. J. Vis. 14 , 7 (2014).

Aydin, C. The differential contributions of visual imagery constructs on autobiographical thinking. Memory 26 , 189–200 (2017).

Schacter, D. L. et al. The future of memory: remembering, imagining, and the brain. Neuron 76 , 677–694 (2012).

Tong, F. Imagery and visual working memory: one and the same? Trends Cogn. Sci. 17 , 489–490 (2013).

Berger, G. H. & Gaunitz, S. C. Self-rated imagery and encoding strategies in visual memory. Br. J. Psychol. 70 , 21–24 (1979).

Harrison, S. A. & Tong, F. Decoding reveals the contents of visual working memory in early visual areas. Nature 458 , 632–635 (2009).

Borst, G., Ganis, G., Thompson, W. L. & Kosslyn, S. M. Representations in mental imagery and working memory: evidence from different types of visual masks. Mem. Cognit. 40 , 204–217 (2011).

Kang, M.-S., Hong, S. W., Blake, R. & Woodman, G. F. Visual working memory contaminates perception. Psychon Bull. Rev. 18 , 860–869 (2011).

Keogh, R. & Pearson, J. The perceptual and phenomenal capacity of mental imagery. Cognition 162 , 124–132 (2017). This study shows a new method to measure the capacity function of visual imagery and shows that it is quite limited .

Luck, S. J. & Vogel, E. K. Visual working memory capacity: from psychophysics and neurobiology to individual differences. Trends Cogn. Sci. 17 , 391–400 (2013).

Pearson, J. & Keogh, R. Redefining visual working memory: a cognitive-strategy, brain-region approach. Curr. Dir. Psychol. Sci. 28 , 266–273 (2019).

Greenberg, D. L. & Knowlton, B. J. The role of visual imagery in autobiographical memory. Mem. Cognit. 42 , 922–934 (2014).

Sheldon, S., Amaral, R. & Levine, B. Individual differences in visual imagery determine how event information is remembered. Memory 25 , 360–369 (2017).

D’Argembeau, A. & Van der Linden, M. Individual differences in the phenomenology of mental time travel: the effect of vivid visual imagery and emotion regulation strategies. Conscious Cogn. 15 , 342–350 (2006).

Vannucci, M., Pelagatti, C., Chiorri, C. & Mazzoni, G. Visual object imagery and autobiographical memory: object Imagers are better at remembering their personal past. Memory 24 , 455–470 (2015).

Galton, F. Statistics of mental imagery. Mind 5 , 301–318 (1880). This paper was the first formal empirical paper investing imagery vividness, including the first report of what is now called aphantasia .

Holmes, E. A. & Mathews, A. Mental imagery in emotion and emotional disorders. Clin. Psychol. Rev. 30 , 349–362 (2010).

Hackmann, A., Bennett-Levy, J. & Holmes, E. A. Oxford Guide to Imagery in Cognitive Therapy (Oxford Univ. Press, 2011).

Blackwell, S. E. et al. Positive imagery-based cognitive bias modification as a web-based treatment tool for depressed adults: a randomized controlled trial. Clin. Psychol. Sci. 3 , 91–111 (2015).

Crane, C., Shah, D., Barnhofer, T. & Holmes, E. A. Suicidal imagery in a previously depressed community sample. Clin. Psychol. Psychother. 19 , 57–69 (2011).

Hales, S. A., Deeprose, C., Goodwin, G. M. & Holmes, E. A. Cognitions in bipolar affective disorder and unipolar depression: imagining suicide. Bipolar Disord. 13 , 651–661 (2011).

Holmes, E. A. et al. Mood stability versus mood instability in bipolar disorder: a possible role for emotional mental imagery. Behav. Res. Ther. 49 , 707–713 (2011).

Tiggemann, M. & Kemps, E. The phenomenology of food cravings: the role of mental imagery. Appetite 45 , 305–313 (2005).

Connor, J. P. et al. Addictive behaviors. Addict. Behav. 39 , 721–724 (2014).

May, J., Andrade, J., Panabokke, N. & Kavanagh, D. Visuospatial tasks suppress craving for cigarettes. Behav. Res. Ther. 48 , 476–485 (2010).

Michael, T., Ehlers, A., Halligan, S. L. & Clark, D. M. Unwanted memories of assault: what intrusion characteristics are associated with PTSD? Behav. Res. Ther. 43 , 613–628 (2005).

Holmes, E. A., James, E. L., Kilford, E. J. & Deeprose, C. Key steps in developing a cognitive vaccine against traumatic flashbacks: visuospatial tetris versus verbal pub quiz. PLOS ONE 5 , e13706 (2010).

Shine, J. M. et al. Imagine that: elevated sensory strength of mental imagery in individuals with Parkinson’s disease and visual hallucinations. Proc. R. Soc. B 282 , 20142047 (2014).

Foa, E. B., Steketee, G., Turner, R. M. & Fischer, S. C. Effects of imaginal exposure to feared disasters in obsessive-compulsive checkers. Behav. Res. Ther. 18 , 449–455 (1980).

Hunt, M. & Fenton, M. Imagery rescripting versus in vivo exposure in the treatment of snake fear. J. Behav. Ther. Exp. Psychiatry 38 , 329–344 (2007).

Holmes, E. A. & Mathews, A. Mental imagery and emotion: a special relationship? Emotion 5 , 489–497 (2005).

Zeman, A. Z. J. et al. Loss of imagery phenomenology with intact visuo-spatial task performance: a case of ‘blind imagination’. Neuropsychologia 48 , 145–155 (2010).

Ungerleider, L. G. & Haxby, J. V. ‘What’ and ‘where’ in the human brain. Curr. Opin. Neurobiol. 4 , 157–165 (1994).

Jacobs, C., Schwarzkopf, D. S. & Silvanto, J. Visual working memory performance in aphantasia. Cortex 105 , 61–73 (2017).

Gray, C. R. & Gummerman, K. The enigmatic eidetic image: a critical examination of methods, data, and theories. Psychol. Bull. 82 , 383–407 (1975).

Stromeyer, C. F. & Psotka, J. The detailed texture of eidetic images. Nature 225 , 346–349 (1970).

Haber, R. N. Twenty years of haunting eidetic imagery: where’s the ghost? Behav. Brain Sci. 2 , 616–617 (1979).

Allport, G. W. Eidetic imagery. Br. J. Psychol. 15 , 99–120 (1924).

Kwok, E. L., Leys, G., Koenig-Robert, R. & Pearson, J. Measuring thought-control failure: sensory mechanisms and individual differences. Psychol. Sci. 57 , 811–821 (2019). This study shows that, even when people think they have successfully suppressed a mental image, it is still actually there and biases subsequent perception (a possible candidate for unconscious imagery) .

Kosslyn, S. M. Image and Mind (Harvard Univ. Press, 1980).

Kosslyn, S. M. Mental images and the brain. Cogn. Neuropsychol. 22 , 333–347 (2005).

Pylyshyn, Z. W. What the mind’s eye tells the mind’s brain: a critique of mental imagery. Psychol. Bull. 80 , 1–24 (1973).

Pylyshyn, Z. Return of the mental image: are there really pictures in the brain? Trends Cogn. Sci. 7 , 113–118 (2003). This review provides an updated summary of the imagery debate .

Chang, S. & Pearson, J. The functional effects of prior motion imagery and motion perception. Cortex 105 , 83–96 (2017).

Stokes, M., Thompson, R., Cusack, R. & Duncan, J. Top-down activation of shape-specific population codes in visual cortex during mental imagery. J. Neurosci. 29 , 1565–1572 (2009).

Amit, E. & Greene, J. D. You see, the ends don’t justify the means: visual imagery and moral judgment. Psychol. Sci. 23 , 861–868 (2012).

Dobson, M. & Markham, R. Imagery ability and source monitoring: implications for eyewitness memory. Br. J. Psychol. 84 , 111–118 (1993).

Gonsalves, B. et al. Neural evidence that vivid imagining can lead to false remembering. Psychol. Sci. 15 , 655–660 (2004).

Bird, C. M., Bisby, J. A. & Burgess, N. The hippocampus and spatial constraints on mental imagery. Front. Hum. Neurosci. 6 , 142 (2012).

Jones, L. & Stuth, G. The uses of mental imagery in athletics: an overview. Appl. Prev. Psychol. 6 , 101–115 (1997).

Dils, A. T. & Boroditsky, L. Visual motion aftereffect from understanding motion language. Proc. Natl Acad. Sci. USA 107 , 16396–16400 (2010).

Christian, B. M., Miles, L. K., Parkinson, C. & Macrae, C. N. Visual perspective and the characteristics of mind wandering. Front. Psychol. 4 , 699 (2013).

Palmiero, M., Cardi, V. & Belardinelli, M. O. The role of vividness of visual mental imagery on different dimensions of creativity. Creat. Res. J. 23 , 372–375 (2011).

Download references

Acknowledgements

The author thanks R. Keogh, R. Koenig-Robert and A. Dawes for helpful feedback and discussion on this paper. This paper, and some of the work discussed in it, was supported by Australian National Health and Medical Research Council grants APP1024800, APP1046198 and APP1085404, a Career Development Fellowship APP1049596 and an Australian Research Council discovery project grant DP140101560.

Author information

Authors and affiliations.

School of Psychology, The University of New South Wales, Sydney, Australia

Joel Pearson

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Joel Pearson .

Ethics declarations

Competing interests.

The author declares no competing interests.

Additional information

Peer review information Nature Reviews Neuroscience thanks D. Kavanagh, J. Hohwy and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The reverse direction of neural information flow, for example, from the top-down, as opposed to the bottom-up.

Magnetic resonance imaging and functional magnetic resonance imaging decoding methods that are constrained by or based on individual voxel responses to perception, which are then used to decode imagery.

Transformations in a spatial domain.

The conscious sense or feeling of something, different from detection.

A mental disorder characterized by social anxiety, thought disorder, paranoid ideation, derealization and transient psychosis.

Rights and permissions

Reprints and permissions

About this article

Cite this article.

Pearson, J. The human imagination: the cognitive neuroscience of visual mental imagery. Nat Rev Neurosci 20 , 624–634 (2019). https://doi.org/10.1038/s41583-019-0202-9

Download citation

Published : 05 August 2019

Issue Date : October 2019

DOI : https://doi.org/10.1038/s41583-019-0202-9

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

This article is cited by

Predicting the subjective intensity of imagined experiences from electrophysiological measures of oscillatory brain activity.

• Derek H. Arnold
• Blake W. Saurels
• Dietrich S. Schwarzkopf

Scientific Reports (2024)

Visual hallucinations induced by Ganzflicker and Ganzfeld differ in frequency, complexity, and content

• Oris Shenyan
• Matteo Lisi
• Tessa M. Dekker

Neural signatures of imaginary motivational states: desire for music, movement and social play

• Giada Della Vedova
• Alice Mado Proverbio

Brain Topography (2024)

Brain state and dynamic transition patterns of motor imagery revealed by the bayes hidden markov model

• Yunhong Liu

Cognitive Neurodynamics (2024)

Combinatorial Language parent-report Scores Differ Significantly Between Typically Developing Children and Those with Autism Spectrum Disorders

• Matthew Arnold
• Rebecca Netson
• Andrey Vyshedskiy

Journal of Autism and Developmental Disorders (2024)

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

• Skip to primary navigation
• Skip to main content
• Skip to primary sidebar
• Skip to footer

Bringing the World's Best Biology to You

Exploring the Visual Brain

• Duration: 24:05

00:00:15.17 I'm Torsten Wiesel. 00:00:17.24 I'm a neurobiologist 00:00:20.23 and I'd like to talk about some discoveries 00:00:23.28 that David Hubel and I made 00:00:26.18 starting out sixty years ago. 00:00:32.11 And this picture is from when we started out. 00:00:35.04 David Hubel is to the left and me to the right, 00:00:38.27 and we had just finished our medical studies 00:00:43.00 and decided to leave the clinic 00:00:45.13 and try to see if it could not advance our understanding of the brain 00:00:51.22 by doing basic research in neuroscience. 00:00:56.09 And the... we were very... 00:01:00.12 our attitude was very much the one of an explorer. 00:01:07.08 We just had tools in our hands 00:01:11.05 and we were interested to understand 00:01:15.07 how we visualize images on the retina, 00:01:20.05 how they are being processed by the brain, 00:01:23.07 so we can see things in detail and color and that. 00:01:29.17 So that was... and no one really had, at the time, 00:01:32.28 studied cells in the visual cortex. 00:01:36.26 So, that was a virgin territory. 00:01:40.02 And so we had no hypothesis, 00:01:43.14 no real theory of how it should work. 00:01:51.29 We just wanted to, 00:01:55.03 using the method of recording from single nerve cells in the brain, 00:01:57.16 then try to see how the system works. 00:02:01.18 So, a simple-minded idea. 00:02:05.28 And we were very fortunate 00:02:09.01 in that we were able to come to the laboratory 00:02:13.03 of Stephen Kuffler, 00:02:16.14 who was already at the time... 00:02:20.26 he was working... 00:02:23.12 his lab was in the basement of the Wilmer Institute 00:02:25.23 at Johns Hopkins Medical School, 00:02:28.02 and it was a simple setup 00:02:30.21 and we got a room there. 00:02:33.27 And the reason why we were happy in the basement 00:02:37.09 was that Stephen Kuffler was a brilliant person 00:02:41.21 in his work 00:02:43.28 and his main interest was in synaptic transmission, 00:02:46.25 how the synapse, the connection between nerve fibers 00:02:50.11 and nerve cells, 00:02:53.13 work. 00:02:55.10 It's a very fine... and that was his passion, 00:02:58.27 but since he was at an eye institute, he also started... 00:03:02.07 did some work on the retina. 00:03:04.03 He recorded from single retinal ganglion cells. 00:03:06.23 And this was a fundamental work 00:03:11.27 that I will describe in the bit, 00:03:14.07 that really laid the foundation 00:03:18.07 for the work that David and I carried out in... 00:03:21.24 started out in the basement. 00:03:23.24 So, David Hubel, he... 00:03:28.21 he was Canadian and he had... 00:03:31.29 and he had already shown his talent in neuroscience. 00:03:43.15 He was working in [unknown] 00:03:48.09 and then during this period 00:03:53.11 he developed the micro-electrode, 00:03:55.21 the tungsten micro-electrode, 00:03:57.28 which was really a major advance. 00:04:00.20 And I had heard him giving talks about having 00:04:09.05 recorded from wide-awake cats 00:04:12.05 using a very elegant technique that he developed himself, 00:04:14.26 but actually later on was the model for 00:04:18.28 the work that is now still going on, 00:04:22.13 recording from wide-awake monkeys, in particular. 00:04:26.13 And he was very skillful in terms of instrumentation. 00:04:31.07 He had a lathe in the lab 00:04:33.16 and could build his own equipment. 00:04:35.19 So, I think, for me, 00:04:38.14 to have not only such a skillful technical ability 00:04:43.23 but also a brilliant mind for a person to work with 00:04:49.02 was, I think, my great fortune in my scientific career. 00:04:55.09 So, the combination of Kuffler and David 00:04:58.16 really set me off in a very good start. 00:05:02.19 The ummm... the ummm... 00:05:09.03 this is a picture of the micro-electrode 00:05:14.01 -- you can see the thing coming in from the right -- 00:05:16.01 and coming close to a nerve cell. 00:05:19.07 The nerve cell is stained here with a silver stain 00:05:25.03 to show the cell body and the processes. 00:05:28.05 And it turns out, when you put the electrode close, 00:05:31.22 as we can see here, to the cell body, 00:05:34.08 you can record the all-or-none action potentials 00:05:38.16 that cells, nerve cells, 00:05:42.01 use to communicate with each other. 00:05:44.07 So, this all-or-none action potential 00:05:47.03 travels down a long process called an axon, 00:05:50.06 that then goes to another nerve cell 00:05:52.25 and communicates with that. 00:05:56.22 So, the frequency... 00:05:59.03 the action potential is an all-or-none signal 00:06:02.24 that changes in frequency and depending... 00:06:06.10 the information that's conveyed from one cell to another 00:06:11.04 is in the changes in the frequency of the firing of the cell. 00:06:15.27 And once the signal comes to the end of the axon, 00:06:21.06 the terminal, 00:06:24.15 the transmitter is released 00:06:27.16 and you then have an analog transformation 00:06:30.10 of the digital signal. 00:06:32.08 So, that's how the nervous system works. 00:06:35.09 The digital signaling over long or short distances, 00:06:39.00 and then an analog, 00:06:42.19 who would process the signal from, perhaps, thousands and thousands 00:06:49.10 of individual inputs to the cell. 00:06:52.21 So, this sounds very complicated, 00:06:54.20 but it turns out that, as we will see in the recording 00:06:57.15 that I will show, 00:06:59.18 that you actually can receive information, 00:07:02.12 useful information, by this recording. 00:07:05.04 And just to give you a little bit of a background, 00:07:07.26 let me show you ummm... the ummm... 00:07:12.12 system that we used. 00:07:14.27 In our studies, initially, we were in the... 00:07:18.28 as I said, in the basement, 00:07:22.04 and we had... 00:07:25.00 we had very little money 00:07:27.08 and the only thing we had available was this blackboard 00:07:30.18 and also an old slide projector 00:07:34.06 that we could put in, 00:07:36.21 and then we had an amplifier 00:07:39.07 and the animal was sitting behind David 00:07:41.22 and could look at the screen, 00:07:44.09 and then we'd project images onto the small papers 00:07:47.28 that you see tagged onto the blackboard. 00:07:50.22 And this was then our method of 00:07:56.23 stimulating the receptive field, 00:07:59.06 the light-sensitive area of single cells. 00:08:02.16 It might be easier to understand this 00:08:05.27 by looking at this graph. 00:08:08.19 This is a human head... 00:08:11.20 so, we only worked on anesthetized cats and monkeys, 00:08:14.19 but the system is very similar in man 00:08:19.10 and, in particular, in the monkey. 00:08:22.18 So, here you can see the eye, 00:08:27.27 and the person here is looking at the screen, 00:08:30.24 and you can see the squares on the screen, 00:08:34.13 and the eye then projects to the relay nucleus, 00:08:38.16 the retinal ganglion project, 00:08:40.18 and then this relay nucleus 00:08:44.22 projects to the visual cortex, 00:08:49.00 which is the two circles. 00:08:51.10 So, the first circle is the relay nucleus 00:08:53.12 and the big circle is the primary visual cortex. 00:08:58.29 And David and I, we spent about 20 years 00:09:02.03 trying to understand how this beautiful structure worked. 00:09:06.23 Now, the ummm... 00:09:10.16 one thing it's important to remember is that... 00:09:13.19 even if there are a million fibers 00:09:16.11 coming in from the eye to the relay nucleus 00:09:20.18 and then there are hundreds of millions of cells 00:09:24.02 in the primary visual cortex, 00:09:26.05 but even if there are these large numbers of cells, 00:09:28.06 you can still, by looking at one cell at a time, 00:09:30.24 you will, as you can see, 00:09:33.10 you will learn a great deal about how the system works. 00:09:36.06 And some people have said that, David and I, 00:09:39.19 we broke the code about how visual information 00:09:43.03 was processed from the eye and then by the brain. 00:09:46.19 So, we... or, I... I'd now like to... 00:09:53.24 so, keep in mind that a given cell 00:09:58.02 can only see one of the squares, 00:10:01.27 what we call the light-sensitive area 00:10:05.20 or the receptive field of a cell. 00:10:07.16 And you can see that here... 00:10:12.01 you see the horizontal bar... 00:10:14.00 so, a single cell can see that, 00:10:16.13 which we call, then, the receptive field, 00:10:19.06 the light-sensitive area. 00:10:21.11 The ummm... 00:10:24.13 the film you're going to hear, it sounds... 00:10:27.27 when you put the electrode close to the nerve cell 00:10:31.13 for the action potential, 00:10:34.21 the all-or-none signal, 00:10:36.07 there's an amplifier connected, so you can hear a plop 00:10:38.23 when the cell sends the signal. 00:10:42.13 And it's the frequency of these plops 00:10:45.03 that you will hear that indicate the signal. 00:10:48.14 So, you're now going to hear the signal 00:10:51.21 from one single nerve cell, 00:10:54.14 first from the relay nucleus 00:10:56.22 and then from the cortical cell. 00:10:59.12 And another thing you need to know is that 00:11:02.18 when Stephen Kuffler recorded from retinal ganglion cells, 00:11:07.05 he showed that they responded 00:11:11.03 very well to small spots of light, 00:11:13.06 but not to bit spots, 00:11:15.10 because they have a sensitive center 00:11:17.24 and an antagonistic surround. 00:11:21.13 So, by stimulating a bit spot, you antagonize a cell as well as exciting it, 00:11:27.22 so it's not as effective as just stimulating the center. 00:11:31.26 Now, David and I found that 00:11:34.18 the cells in the relay nucleus, here, 00:11:36.29 have the same properties: they like small spots, 00:11:39.07 but not big spots. 00:11:41.10 So, so... and then you will see, 00:11:44.26 when you record from cells, here, in the primary visual cortex, 00:11:49.28 we found that they don't respond 00:11:53.09 to small spots alike. 00:11:55.17 It was very frustrating, 00:11:57.22 and it was only by chance that 00:12:00.07 we discovered what you will see in the movie in a few... 00:12:02.23 in a second. 00:12:05.18 So, first, in the movie, you will hear the recording 00:12:08.04 from a single cell 00:12:11.07 in the lateral geniculate nucleus body, 00:12:13.08 and then, without transitioning, you'll hear a difference in the pitch of the sound 00:12:16.19 when you record from the cells 00:12:19.10 in the primary visual cortex. 00:12:21.01 So... so, here we stimulate, 00:12:26.23 now, in the center of that square, 00:12:29.14 which is the receptive field/sensitive area of the cell, 00:12:32.11 and you can hear a strong response. 00:12:36.03 Okay, now... here you can see, then, 00:12:38.22 the cell responds very well to small spots of light... 00:12:41.15 we make the spot large 00:12:44.15 and it's less effective 00:12:46.25 -- it's not as crisp as before. 00:12:49.01 And the reason is, as you can show here, 00:12:51.06 that you're stimulating... the surround is antagonistic. 00:12:53.26 The cell stops firing and when you turn it off it fires. 00:12:59.10 Okay. 00:13:01.29 And so you can see this antagonism 00:13:05.15 functions by contracting 00:13:08.13 -- have a big spot that you contract to a small spot, and then cell fires, 00:13:11.02 because you take away the inhibition. 00:13:13.12 It's just showing the center and surround arrangement 00:13:17.12 that Kuffler was the first to show in retinal ganglion cells, 00:13:21.00 but this is a geniculate cell. 00:13:22.26 Now, we go... just to show that this is 00:13:26.29 a circular and symmetric kind of organization 00:13:29.04 of the receptive field, 00:13:30.26 and to show that we have this bar 00:13:33.03 move across the receptive field 00:13:35.11 in different orientations. 00:13:37.07 And you can hear that 00:13:40.19 there's no real difference in the frequency of the impulses from... 00:13:43.03 we're still recording from a single nerve cell 00:13:45.19 in the relay nucleus. 00:13:48.02 Now, we have suddenly a pitch change, 00:13:49.26 and you can hear that, 00:13:52.08 and now we are recording from a single cell 00:13:54.28 in the primary visual cortex. 00:13:56.27 And we map out the light-sensitive area, 00:13:59.11 the receptive field, 00:14:01.24 of a single cortical cell. 00:14:04.01 And you can see... 00:14:06.18 the blackboard we had before... 00:14:08.19 it's very simple 00:14:11.20 and you can draw on this piece of paper the outline, 00:14:13.16 and the size of the field 00:14:16.28 is a little bit bigger than indicated here 00:14:19.21 and the orientation is a little bit counterclockwise, 00:14:22.21 but in principle this is actually 00:14:24.29 an experiment that David and I carried out 00:14:28.26 some 50-60 years ago. 00:14:30.16 We just wanted to have it documented, 00:14:32.17 because at first people didn't believe us 00:14:34.18 that it was true that cells had this property 00:14:37.28 that you now would see. 00:14:39.20 Some cells... 00:14:41.25 this cell responded a little bit better to leftward movement than right, 00:14:44.15 and some cells responded 00:14:47.04 only to one direction, 00:14:49.00 and other cells prefer one over the other. 00:14:54.15 So, this is the critical point coming up now. 00:14:59.11 Keep in mind, we are stimulating the same visual area 00:15:02.28 in the cortex, 00:15:04.28 but this cell doesn't see a horizontal bar 00:15:07.07 -- it can only see a bar along the vertical. 00:15:11.10 So, this is nature's amazing picture, 00:15:16.02 and they... 00:15:18.18 by asking and exploring nature, 00:15:21.13 the brain told us about this trick. 00:15:25.12 Next, we just show a simple paintbrush, 00:15:29.24 just to show you how 00:15:33.21 strong and simple this system is, 00:15:36.21 to detect any orientation of any contour. 00:15:44.17 So, the... I'd like to go back, now, 00:15:47.15 to this diagram, 00:15:49.20 just to point out that, 00:15:52.04 within the receptive... 00:15:55.11 the area of the receptive field, the light-sensitive area, 00:16:00.24 the cell you just saw 00:16:08.12 prefers sort of close to vertical orientation. 00:16:10.11 Now, when David and I first discovered this... 00:16:13.25 by chance, actually, it wasn't because we were clever... 00:16:17.19 it just happened that the contour of a... 00:16:21.01 of this orientation happened to pass by and we said, 00:16:23.21 "Oh, what does this mean?" 00:16:25.12 We couldn’t activate the cell with spots of light, 00:16:28.10 so David, who was very... 00:16:31.01 he rushed down the hall and called on our colleagues 00:16:33.14 to come and say, "Look what we have found!" 00:16:36.29 And so everybody came and were amazed, like we were, 00:16:39.22 about the specificity with which this cell responded 00:16:43.02 -- preferring one orientation over any other. 00:16:45.26 But within this square, here, 00:16:48.29 there are many... 00:16:53.02 there are other neighboring cells who prefer a different orientation. 00:16:55.24 In the drawing, here, we have a horizontal, 00:16:59.04 and there are other weak orientations. 00:17:01.08 Perhaps there is a representation of 20 different orientations, 00:17:04.14 but the cells... 00:17:06.20 all these cells see the same little area, 00:17:09.23 so you have to think about our visual system 00:17:12.14 as partialized into small light-sensitive areas, 00:17:19.19 and then it's not until you get into higher visual areas 00:17:21.29 when these areas come together, 00:17:24.01 and you have bigger areas of light sensitivity, 00:17:26.01 so you can see the whole world. 00:17:29.09 But the initial step in the processing of visual information 00:17:33.00 is this partialization of images. 00:17:35.23 And the idea, here, 00:17:38.15 is that these cells sensitive for an orientation of a contour 00:17:43.02 are the instrument used by the brain to visualize... 00:17:46.21 to rec... 00:17:48.22 to make it possible for us to perceive any object. 00:17:52.09 This slide is from a book by the late David Marr, 00:17:58.04 who was a... 00:18:01.15 he liked the idea about... 00:18:04.20 he was more a model builder, and he said that... 00:18:10.25 if you look at a teddy bear, 00:18:13.21 and you have these cells sensitive to orientation or contour, 00:18:17.17 and he drew, here, to the right, the... 00:18:22.26 here is a vertically oriented cell 00:18:27.08 preferring the top of the head... 00:18:30.19 so, you can actually draw the teddy bear 00:18:34.11 using these very simple cells. 00:18:36.21 So, that was the concept that... 00:18:46.06 it's like having a Lego set in your camp, 00:18:49.10 and you can look upon these cells as individual pieces, 00:18:53.03 that then the brain can use 00:18:55.10 in order to detect any object of interest. 00:18:58.05 Now, the funny thing happened 00:19:01.10 when we published this paper in the spring of 1969, 00:19:05.02 not this figure but the fact that the visual... 00:19:10.12 primary visual cortex is sensitive... 00:19:12.20 the cells there are sensitive to orientation of contours. 00:19:16.00 I received a note, or we received a note, 00:19:20.06 from John Z Young, 00:19:22.29 who was a very prominent neurobiologist in Britain, 00:19:26.05 and he said, 00:19:29.11 "Torsten, this is very nice, but it's nothing new," 00:19:33.03 and then he sent me this picture of Van Gogh's self-portrait. 00:19:40.25 So, you can... as you can see, 00:19:43.04 Van Gogh gives oriented lines in order to draw his picture, 00:19:47.01 and other artists have also used similar ways, 00:19:49.24 and so you can ask, 00:19:53.11 maybe artists have a more intuitive understanding of how our brain works, 00:19:57.11 and they obviously don't realize it, 00:20:00.15 and we don't either, necessarily, 00:20:03.10 but this is one way, perhaps, that we should... 00:20:07.04 like to bring the arts and sciences together, 00:20:10.05 that by working together we may learn from each other. 00:20:13.12 So, the ummm... so, this... this was ummm... 00:20:18.19 the one discovery... 00:20:21.23 when we started out in September 1958, 00:20:30.07 we discovered the orientational cell sensitivity... 00:20:35.09 specificity of cortical cell orientation 00:20:38.20 after three months, 00:20:40.26 and so we quickly wrote a paper 00:20:44.05 and then published in the spring. 00:20:45.29 The point to come back to this slide is that... 00:20:51.23 to remind you that this discovery 00:20:55.09 is critically important for our understanding how vision... 00:20:59.18 how the visual information is coded, 00:21:02.13 but done with very simple... 00:21:04.29 two young guys using very simple tools. 00:21:08.05 And that... 00:21:10.21 like to inspire young people to, 00:21:12.22 if you have an idea, 00:21:14.20 just find out what you can do with it, 00:21:16.29 and then go out and try it out 00:21:20.07 and then once you find something that works, 00:21:22.20 you can get the resources and become... 00:21:25.20 build the proper laboratory, which we did. 00:21:29.23 So, David and I continued to work. 00:21:33.00 This, as I said, took a very short time, 00:21:35.23 and later we spent 20 years together. 00:21:39.26 David was supposed to come, as I mentioned, one month... 00:21:47.07 one year while his lab in Mountcastle was renovated, 00:21:51.26 but we got along so well 00:21:54.20 and so we stayed together and collaborated for 20 years, 00:21:57.14 and during those 20 years we did... 00:22:00.12 studied the cortical architecture, 00:22:04.13 we studied... showed that the orient... 00:22:09.05 cells with similar orientation 00:22:11.25 are grouped in columnar organization 00:22:14.09 in a very beautiful way, 00:22:16.11 and that the two eyes work together in a very precise way, 00:22:18.26 also in a columnar fashion. 00:22:20.29 And we could show, also, that the development of the visual system, 00:22:23.20 what we are both with 00:22:26.06 and how that can be modified by the environment. 00:22:28.00 So, this is along story, 00:22:30.07 but the fact that we had the resources and time, 00:22:37.16 but it just started out very simply... 00:22:40.13 I think this is... we'll remember. 00:22:43.14 Of course, we then, in 1981, 00:22:48.00 we were told that we had won the Nobel Prize, 00:22:50.20 so this is just a press conference in that context 00:22:54.21 and we both looked happy. 00:22:57.27 Now, David continued to stay at Harvard until 2013, 00:23:07.11 when he died, 00:23:12.07 and I still feel strongly that... 00:23:17.18 I used to call... 00:23:19.28 we used to call each other scientific brothers, and that's true. 00:23:23.18 We became, scientifically, very close 00:23:27.01 and enjoyed each other scientifically, 00:23:30.20 whereas at the same time we had our own personal lives. 00:23:34.21 But it was a real brotherhood-type... 00:23:40.08 scientific brotherhood situation. 00:23:43.07 So, I still miss the guy. 00:23:46.00 So, thank you.

• Educators of H. School / Intro Undergrad
• Educators of Adv. Undergrad / Grad

Talk Overview

Dr. Torsten Wiesel tells us the story of the Nobel Prize-winning discovery of how the brain processes information through the visual cortex. Together with David Hubel, Wiesel used micro-electrodes to monitor changes to a single neuron’s action potential with visual stimuli. With this simple, but innovative experiment, they were able to map each visual stimulus to a specific region of the brain, providing insight on how the brain generates an image from visual inputs.

Speaker Bio

Torsten wiesel.

Wiesel received his medical degree from the Karolinska Institute in Sweden and continued his training at Johns Hopkins University Medical School. There he began a collaboration with Dr. David Hubel, which led to a major breakthrough in the understanding of the visual cortex and ultimately resulted in them be awarded the Nobel Prize in Physiology… Continue Reading

More Talks in Neuroscience

Reader Interactions

Evelyn Blount says

November 3, 2020 at 5:35 pm

Highly enjoyed this video! Thank you!

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Find a Video

• Search All Videos
• Search YouTube
• Short Films
• All Playlists
• All Speakers
• Videos in Other Languages
• Biochemistry
• Bioengineering
• Cell Biology
• CRISPR-Cas Technology
• Development and Stem Cells
• Genetics and Gene Regulation
• Human Disease
• Microbiology
• Neuroscience
• Plant Biology
• Professional Development

For Educators

• Educator Newsletter
• Educator Resources
• Flipped Courses
• Online Courses Resources
• All Educator Resources

Educators Recommend

• Cells of the Immune System
• Discovering Cell Cycle Regulators
• Genome Engineering with CRISPR-Cas9
• Hardy-Weinberg Equilibrium
• Next Generation Sequencing
• Semi-Conservative Replication of DNA
• Toll-Like Receptors in Adaptive Immunity

Free Courses

• Free Online Courses
• Take a Free Course
• Learn Microscopy
• Intro to Microscopy

Career Development

• Professional Development Courses
• Professional Development Talks
• Career Exploration
• Biomedical Workforce
• NRMN Resources

Best of iBiology

• Short Science Films
• Meselson and Stahl
• Doudna & Charpentier’s Nobel Prize
• Decoding Ancestral Knowledge
• Microscopy Series
• Short Microscopy Series
• Famous Discoveries
• Bench to Bedside
• Share Your Research
• Our Mission
• Science Communication Lab
• Financial Conflict of Interest Policy

If you're seeing this message, it means we're having trouble loading external resources on our website.

If you're behind a web filter, please make sure that the domains *.kastatic.org and *.kasandbox.org are unblocked.

To log in and use all the features of Khan Academy, please enable JavaScript in your browser.

Middle school biology - NGSS

Course: middle school biology - ngss   >   unit 1, sensory processing and the brain.

• Understand: sensory processing and the brain

Key points:

• The nervous system helps humans and other animals sense and respond to their environments.
• Some sensory receptors detect mechanical stimuli . These include the receptors involved with our sense of touch.
• Other sensory receptors detect chemical stimuli . These include the receptors involved with our senses of taste and smell.
• Others detect electromagnetic stimuli . These include the receptors involved with our sense of sight.
• Information from sensory receptors is transmitted , or passed along, nerve cells to the brain .
• The brain processes , or organizes, information from different sensory receptors. The brain can then trigger a response or store the information as a memory.

Want to join the conversation?

• Upvote Button navigates to signup page
• Downvote Button navigates to signup page
• Flag Button navigates to signup page

Academia.edu no longer supports Internet Explorer.

To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to  upgrade your browser .

Enter the email address you signed up with and we'll email you a reset link.

• We're Hiring!
• Help Center

Analyzing the Effects of Visual and Verbal Stimuli in Students’ Production of Written Narratives

The aim of this research was to investigate the effects of two types of stimuli, visual and verbal stimuli, in triggering students’ interpretations and expressions of these interpretation in English language which was their second language. The subjects were 40 secondary school students whose English proficiency was limited and therefore classified as novice learners. These students were from a vocational school located in the interior region of the island state, Sabah, Malaysia. They tended to merely recite and rewrite their answers to textual prompts which did little in improving their skills in producing ideas of their own in their own words. In this research, the students were presented with a list of vocabulary (verbal stimulus) and a still image (visual stimulus) and they were required to write a short narrative based on these two types of stimuli. Their responses were analyzed through holistic rubric which encompassed image, characterization, voice and story. The findings showed that the written responses produced by the students towards visual stimulus contained richer imagination, stronger plot and complex sentences than the written responses produced towards verbal stimuli. This suggested that visuals might be more effective in encouraging students to explore and express their creativity in the target language by using their own words.

Related Papers

Listyani Listyani

Purpose: Second language writing as an inherent part of ELT is no exception. One specific part of second language writing in which visual images can be used is narrative essay writing. Visual images, in this case, comic series and pictures, can be a very useful aid in stimulating students' ideas, creativity, as well as interest and ability in narrative essay writing. Students' writing activities in using pictures and comic strips are discussed in this study, including students' responses towards this particular topic, as well as the procedures of how these visual images are used. This study is therefore aimed at disclosing how pictures and comic strips, as forms of visual images, can be used to enhance students' narrative writing ability as well as creativity. In writing narrative essays with the help of visual images, students were helped in terms of generating ideas, developing logical and critical thinking, and improving reasoning skills. Research Methods: The design of this study was qualitative in nature. The participants were comprised of 19 Professional Narrative Writing students. The data for this research was taken from documents, that is, students' essays and journals which were written after the writing activities were done, interviews with two students, and students' scores. There were also pre-tests and post-tests given at the beginning and the end of the semester, but the scores were descriptively presented. Interviews with two students were also conducted to validate the findings. These students, whose essays were used as analyzed documents, were enrolled in the Professional Narrative Writing class of the English Language Education Program, the Faculty of Language and Arts (FLA), Universitas Kristen Satya Wacana (UKSW), Salatiga, Indonesia. The class was conducted in Semester I of the 2017-2018 academic year. The students were fourth-semester students. The Professional Narrative Writing course taught students how to be professional in writing narratives. Findings: The findings showed that pictures, as well as comic strips, were very useful in helping students to write narratives. They helped students generate ideas, delve into more creativity, as well as develop their imagination and motivation in writing, though some students experienced difficulties in some aspects like ideas, grammar, diction, and plot. Implications for Research and Practice: The results of the study hopefully can inspire other narrative writing lecturers all over the globe to maximize the use of visual images, including pictures and comic strips. These visual aids can enhance students' writing abilities as well as their creativity. Students taking writing courses can hopefully be motivated to write better narratives.

International Journal of Language and Linguistics

Andi Asrifan

Redite Kurniawan

Indri Rahayu

cicih nuraeni

The objective of this study is to find out whether the use of picture series significantly improves the achievement on writing narrative text of the eighth grade students of Junior High School. The subject of this study is the eighth grade students of SMP Negeri 76 Jakarta. The sample of this study is the students VIII-4 consisting 30 students. The method of this research is descriptive qualitative method.. The technique for collecting the qualitative data was gathered through students’ writing text and observation sheet. Qualitative data showed that the students were interested in using pictures as the learning media in writing narrative text. From the findings can be seen that student’s narrative writing is found improved through picture series. Students’ responses toward the use of picture series activity in EFL writing classroom were mostly positive. It can be concluded that picture series was an interesting alternative. Under certain condition, it was said more intensive than o...

Ferdian Achmad

irfan cahya

Writing is one of skills in english language. Writing is very important in learning foreign language, because the objective of teaching and learning English is that students are able to use English both in spoken and writen form. However the writer knew that students are still difficult to express or tell their ideas and their feeling in writen English. The objective of this research is to know and describe the influence of using visual to focus description toward students’ ability in writing descriptive text. The population of the research was taken from students at the second semester of the tenth class of SMAN 16 Bandar Lampung. In taking the sample of the research, the writer used stratified Propotional Random Sampling technique because the students’ achievement was heterogeneous. In collecting the data, the writer use free writing test or essay test, the students given some pictures and titles and then the students chose one of the titles and describe it at leas is 150 words. I...

pipih setiawati

This research investigated “Using Picture as Teaching Media in Writing Descriptive Text”. It enhanced the ability to write descriptive in one of junior high school in Garut. The purpose of the research was to know the effect of using pictures on students’ writing ability in descriptive text. The research population was the eighth grade students. The researcher used a written test to collect data. The research design was quantitative experimental. Data were collected through pre-test and post-test. The technique used in analyzing the data was t-test formula. It was used to determine whether there was a significance difference between the pre-test and post-test. The result of the test between pre-test and post-test, the mean of pre-test was 59.63 and the mean of post-test was 75.17 The obtained ttest was tobserved = 12.51 and tcritical = 2.0452. The result showed that tobserved greater than tcritical (12.51 > 2.0452). The tobserved was in the area H0 rejected it means H0 was reject...

Edukasi Lingua Sastra

Penelitian ini bertujuan untuk mengetahui peranan gambar berseri sebagai media pembelajaran terhadap kemampuan menulis paragraf narasi siswa. Data dikumpulkan melalui dua test, yakni pre-test dan post-test terhadap dua kelompok siswa. Subyek penelitian adalah siswa kelas delapan (VIII) di SMP Negeri 22 Bandar Lampung yang terdiri dari 38 siswa dari kelas VIII-H sebagai kelompok kontrol dan 38 siswa dari kelas VIII-G sebagai kelompok eksperimen. Kelompok eksperimen dikenakan perlakuan dengan media gambar berseri sedangkan kelompok control tidak diberi perlakuan. Data diolah dengan menggunakan uji-tterhadap dua sample yang kenakan treatment yang berbeda. Hasil penelitian menunjukan bahwa tingkat signifikansi 1% = 2.64, 5% =1.99, dan t-hitung = 12.16, dapat dikatakan bahwa tingkat signifikan lebih kecil dari t-hitung. Dapat disimpulkan bahwa terdapat peranan gambar berseri terhadap kemampuan siswa dalam menulis paragraf narasi. Oleh karena itu disarankan kepada guru untuk menggunakan ...

RELATED PAPERS

GOING CONCERN : JURNAL RISET AKUNTANSI

Erill Tulangow

Patricia Propersi

Janette Gongora

Fish & Shellfish Immunology

Sasimanas UNAJAK

Human Molecular Genetics

Darcie Hastings

Proceedings of the Combustion Institute

anna ciajolo

Journal of Organometallic Chemistry

Jean-francois Halet

Matheus Trevizam

AIDS Research and Therapy

alberto castillo

Filippo Capolino

Biotechnology Research International

glaucia Pastore

Community Empowerment

firman faradisi

Kunal Bhatnagar

Chris Lisle

angie vargas

Rafael Reis P B de Mello

Perisai : Islamic Banking and Finance Journal

Theresia sthng

039ณัติยาภรณ์ ชัยรัตน์

Dusan Drbohlav

IEEE Access

germain garcia

The American Journal of Surgical Pathology

Cheryl Coffin

Lecture notes in electrical engineering

Vojin Tošić

Prof. Nimali Singh

Joaquín Gaspar

RELATED TOPICS

•   We're Hiring!
•   Help Center
• Find new research papers in:
• Health Sciences
• Earth Sciences
• Cognitive Science
• Mathematics
• Computer Science
• Academia ©2024

Sample details

• Words: 3132
• Views: 1,130

Related Topics

• Self Reliance
• Microorganisms
• Microbiology
• Biomolecules
• Homeostasis
• Photosynthesis

The effect of visual stimuli on heart rate

Green ass and Multicultural 5/s) on the effect of the fight or flight response measured by the earth rate of the viewer? Hypothesis If the speed and the color brightness of the light is Increased, then the heart rate of the viewer will be Increased. The highest heart rate measured will be from the multicultural stimulus and the lowest will be at the black and white stimulus. (Due to the visual stimuli causing different amounts of adrenaline being released from the adrenal glands, referred to as the flight or flight’ response) Background Information/ Theory The Heart is a vital organ within the human body (C.

J. College). Its main purpose within he human body is to pump blood through the entire human body (Seymour, 2007). Blood contains white and red blood cells, which the human body uses for different reasons (Seymour, 2007). As the heart is quiet a small organ In relation to the body size, it would be difficult for just the heart to pump all the blood around the body, but, within many different places within the human body, there are smaller organs called pulses which helps push the blood through the veins (Seymour, 2007).

ready to help you now

Without paying upfront

These pulses are completely In sync with the heart, beating at the same rate at which the heart does (Seymour, 2007). The Pulses are In some locations (specifically the wrist) where they are very close to the skin, thus enabling a measurement of the heart rate to be possible (Race, 2007). Heart rate can be measured by many different methods, but the popular way of measuring heart rate Is with a heart rate monitor strapped to the wrist (Kumar, Bass, & Faust, 2005). Light, in which humans are able to perceive, is referred to as “visible light” which has sun in which Earth orbits (ICE, 1987).

The reason behind Earth having so many colors is due to the absorption spectrum, where different colored materials absorb ND radiate different wavelengths, for example, the color blue absorbs the suns light at wavelengths of Mann-?Mann and radiates the rest of the light shined upon (Loafer, 2013). When humans are placed in different situations where they have to make an immediate decision, the heart rate of the particular individual will increase due to the adrenalin secreted from the adrenal glands within the body, this is referred to as the fight or flight response (C. J. College).

This response is caused by the visual perception of the person experiencing the situation and their reaction to it (C. J. College). Some types of lights, when flashed in a certain tempo, can cause this response to occur in a slight amount (Kumar, Bass, & Faust, 2005). This is due to certain colors having a connection to certain emotions that the person has experienced (Kumar, Bass, & Faust, 2005). From this information, I have gathered the hypothesis of this investigation “If the speed and the color brightness of the light is increased, then the heart rate of the viewer will be increased”.

Table 1. Variables for Visual Stimuli on Heart Rate Investigation Type of Variable Variable Identified Independent The types of light being projected to the viewers. The lights have different colors and different flashing speeds (measured in flashed per second (e. G. 1 Is)). (White Los (control), White and Black 1 Is, Blue 2/s, Red 3/s, Green ass and Multicultural 5/s) Dependent The heart rate of the viewer after they have viewed the visual stimuli measured using a heart rate monitor. Controlled The light source of the visual stimuli.

The time of day that the experiment is conducted on. The method of measuring heart rate of the group of viewers. The colors of light being projected to the viewers for the visual stimuli. The visual stimuli duration. The speed of light flash (flashes per second). The type of light of the visual stimuli. The temperature of the room were the experiment will be conducted. Uncontrolled Previous factors that may alter heart rate, this includes any activities the viewer may have done previous to entering the experiment.

Any heart conditions the viewer may have that could affect their reaction to the visual stimuli. As heat increases heart rate, the clothing that the viewer chooses to wear into the experiment will be unaltered. 1. 2 Controlling Variables Treatment of Controlled Variables Variables Control Treatment The Light Source Conduct the experiment using only the projector in the experimentation room as the light source for the visual stimuli. Method of Measuring Heart Rate Only measure heart rates using a heart rate monitor. This will be given to the viewers prior to entering the experiment.

The Experimentation Room Temperature The experiment is to be conducted within a room with a constant temperature of ICC. This temperature will be kept constant through the use of an air conditioner. Environmental Factors Conduct experiment in the same room (Bal 52) at all times. The Duration of the Visual Stimuli Each trial of flashing lights are to be kept at a 45 second duration. Control Experiment The control for this experiment was a plain white light. For the control experiment for this investigation, the white light had no flashes (remained a constant solid light for 45 seconds).

To keep the control experiment the same as all the other experiments, the same controlled variables were implemented. To ensure this experiment did not have any environmental factor that could potentially affect the conduction of the experiment, the control experiment remained in the same room that all the experiments were conducted within. Participants Within this experiment, the participants involved were in the ages of 14 to 17 and there will be at minimum 10 participants. The participants were both female and male.

The participants had a variety of nationalities. The target population of this experiment was participants between the ages of 14 to 17 and females and males of varied nationalities. 1. 3 Experimental Method Materials Table 3 The Materials Used Within the Conduction of the Experiment of Visual Stimuli on Heart Rate Apparatus Required Quantity Projector Computer with Visual Stimuli and Microsoft Excel Chairs 10 Risk Assessment Refer to Appendix A – SHAHS STUDENT ACTIVITY RISK ASSESSMENT and PARA ORDER FORM. Ethics: Human research guidelines were followed.

Before the experiment, all the participants were given a consent form that clearly stated: any information about the study is obtainable, their rights within the experiment and the availability to withdraw any information they have included within the experiment. These consent forms were approved by the biology teacher and the schools principal prior to the experiment. Method 1 . Turn on the air conditioning to the experimentation room for ICC. 2. Power on the Projector. 3. Power on the computer. 4. Connect the computer to the projector. 5.

On the computer, open the visual stimuli, but don’t play it. 6. Open a new Microsoft excel spread sheet. 7. Invite the 10 participants in. 8. Seat the participants in no random order. 9. Initiate briefing statement 10. Equip the participants with their heart rate monitors and test that they are working correctly. 1 1 . Start the first visual stimulus of the experiment (control, White O flashes per second). 12. After the visual stimulus is finished, gather all of the heart rates again. 13. Place the new heart rates data within the spread sheet. 14.

Repeat until all of the visual stimuli (White Los (control), White and Black Xi’s, Blue 2/s, Red 3/s, Green ass and Multicultural 5/s) have been played in front of the participants and all of the heart rates of the participants are collected. 15. Initiate debriefing statement. Diagram Diagram 1: Experimental Set up for Visual Stimuli on Heart Rate Investigation. Visual Stimulus – The visual stimulus that will be played to the participants involved within this investigation are White Los (control), White and Black Xi’s, Blue 2/s, Red 3/s, Green ass and Multicultural 5/s.

Where the light flashes are measured in flashes per second, for example, the third trial of this experiment, blue, will have two flashes of blue light per second. Heart Rate Monitor – The heart rate monitor will be used within the experiment as the measurement of the heart rate of the participant. The data Participant – The participants involved within this experiment will be within the age group of 14-17 and will be students within Queensland Academy for Health Sciences. 2 Data Collecting and Processing 2. 1 Recording Raw Data Quantitative Data Table 4: Experimental Raw Data for Visual Stimuli on Heart Rate Qualitative Raw Data

Picture 1: Qualitative Observations During the Visual Stimuli on Heart Rate Experiment The photo above was taken during the briefing statement of the experiment, prior to the first visual stimulus. Black boxes have been used to keep the participants anonymous. 2. 2 Processing Raw Data Statistical Processing – Calculations Average To determine the average of the sets of data recorded, the following calculations were needed to be made. The sum of the change of heart rate for the repeats of the trials divided by the number of results. E. G. / n +68)/ 10 This data was obtained from Trial 1 (control) of this experiment. 0. 8 Standard Deviation is a statistical measure of the precision for a series of repetitive measurements. It is an effective way to show uncertainty in an average taken from a set of results. Standard Deviation has been used to show error bars on the graphed results. S = standard deviation x = each individual value = mean of all measurements = deviation from mean = degrees of freedom These calculations were made using Microsoft excel. The above picture is the standard deviation formula for the control trial of this experiment. All data was processed using this formula. . 3 Presenting Processed Data Results Table Table 5: Processed Data for Visual Stimuli on Heart Rate Conditions Mean Heart Rate (BPML В± 1. 0) Standard Deviation Control White Los 70. 8 2. 485513584 Black and White Xi’s 79. 6 1 . 712697677 Blue 75. 3 Red 81 . 5 2. 549509757 Green ass 82. 2 1 . 813529401 Multicultural ass 76. 9 2. 558211181 Results Graph Graph 1: The Relationship Between Different Visual Stimuli and Heart Rate This graph is a representation of the relationship between heart rate and different types of visual stimuli (of many flashing colors).

As the experiment contained a total of 6 trails, the trail number 1 within this graph is a representation of the control of his investigation. The data presented shows that the highest average heart rate was recorded after watching the fourth (trial number 5) trial of this investigation. This graph also showed that the increase of flashing colors has no effect on heart rate. This is evident in the graph with trial numbers 1, 3 and 6 as these have a lower average heart rate recording. A further explanation of this trend will be discussed within the conclusion aspect of this investigation report.

Small error bars indicate a small standard deviation and therefore greater degree of precision, whereas, large error bars indicates a large standard deviation / low degree of precision. 3. Conclusion and Evaluation 3. 1 Conclusion Conclusion Statement The focus question for this investigation is as follows: “What is the effect of a timed visual stimuli (45 seconds), in the form of flashing colored lights (White Los (control), White and Black 1 Is, Blue 2/s, Red 3/s, Green ass and Multicultural 5/s) on the effect of the fight or flights response on the heart rate of the viewer? From the data gained within this investigation, it is evident to say that there is, to some extent, a allegations between visual stimuli and heart rate However, the data from this experiment does make the hypothesis of this experiment: “If the speed and the color brightness of the light is increased, then the heart rate of the viewer will be and the lowest will be at the black and white stimulus (due to flight or fight response) prove to be incorrect. Conclusion Explanation The results formed from this investigation show that in an extent there is a relationship between visual stimuli and heart rate.

Within graph 1 of this investigation report, the line of best fit has a positive trend. This is a visual representation of overall trend of the averaged data. As this representation is positive, it is evident to say that there is a relationship. It is also evident to say that the colors used within this experiment for the visual stimuli had a large effect on the heart rate. This is evident within table 5 where trial numbers 2 and 5 had data that was significantly lower from the rest of the data. This data is visually represented in graph 1 of this investigation report (numbers 3 and 6 on the graph).

The cause for the increase in heart rate is due to the fight or flight response after exposure to the visual stimuli. The fight or flight response is a reaction to an external stimuli, this can be in the visual, auditory or sensory (touch) form (Martin, 2013). From this response, the body releases adrenaline, which is a hormone that increases the heart rate very rapidly (Martin, 2013). Within this investigation, the fight or flight response is present as the visual stimuli contains many colors that can affect the emotions of the viewer.

The types of colors used within the experiment have affected the heart rate collected within the experiment of this investigation. This can be noticed within rape 1 of this experiment where the data’s average heart rates are fluctuating. A low point observed in graph 1 is trial number 3, which involved a blue light to be flashed two times per second for forty-five seconds. The color blue has an effect on human emotions as it can be related to sad and depressing themes, causing the participants in this experiment to feel a negative emotion when exposed to this stimuli (C. J.

College). A negative emotion can cause a lowered heart rate (Mandrill, 2011). This theory can also be Justified with the opposite emotion; anger, which was used in this experiment s anger is associated with the color red (trial 3 of the experiment and number 4 on graph 1) (Mandrill, 2011). This trial had the second highest reading of the investigation, which shows the correlation between the color red and angers effect on heart rate. However, the highest recorded average heart rate was recorded after the fourth trial of this experiment with an average heart rate of 82. Bomb.

This trial involved a green flashing light that was flashed four times per second. The color green has an affect on the heart rate as it correlates to fast pace themes, as a green eight is a common indicator for the commencing of many things (Mandrill, 2011). Examples of greens lights that we are exposed too everyday are: traffic lights, electrical appliances and external media such as videotapes. As these green lights are exposed to the average person everyday numerous times it is a learned behavior to react quickly to it, thus giving a higher heart rate measurement (C. J. College). Rates measured.

This is evident within graph 1 where, as the data points increased, so did the number of flashes per second. As can be observed, the heart rates measured don’t increase with the flashes per second. This is most observable with number 6 on graph 1 . This point represented trial number 5 which contained the colors: red blue green purple and white all flashing within a second, had one of the lowest averages in the graph. This is due to the brain not having enough time to react to the stimuli and thus causing very little reaction to the adrenaline release equaling small heart rate increase from control (Martin, 2013) and (Mandrill, 2011).

Upon observations of the data tables and graphs, one might observe the data for the red 3 flashes per second stimulus (trial 3 and number 4 on graph) and green 4 lashes per second (trial 4 and number 5 on graph) to be close in numerical terms. This may be caused by possible errors within the conduction of the experiment and will be discussed later on within the report in the reliability section. 3. 2 Evaluating Procedures Reliability The experiment that took place can be portrayed as valid data as many factors were made to keep accuracy and precision of the data at the most highest capability.

Some of these factors include increased number of repetition of the trials. This was done by using 10 participants, rather than using 3. An increased number of petition of the trials was a necessity as repeating the visual stimulus to a smaller group of participants numerous times would have been both boring to the participants, causing inaccurate data and will drag out the experiment by an extra few minutes, which isn’t appealing to the participants.

By using more trials (10 rather than the usual 5) within the experiment, there is less of a chance for a significant error to occur within conduction. Although the experiment has ten repeats of each trial one more trial was added for a comparison within the processing data aspect of the experiment. This comparative trial is called the control trial because it is in a controlled environment. Within the set of data acquired from the experiment, a very noticeable data point stands out form the rest. This data being the standard deviation from the graph 1.

The third trial on the graph (blue 2 flashes per second) has a large standard deviation; this is due to the data collected from the investigation having a wide range. The cause of a high standard deviation may also be from being incorrectly measured or from the heart rate monitors being incorrectly used within conduction on the experiment. Limitations and Weaknesses Throughout the conduction and data processing aspects of this experiment, the conductors observed many significant limitations. Such observations have been made into a table for organizational purposes.

Limitation Observed Evidence of Limitation Possible Solution Setting of the Experiment As the classroom was quite small and compact, a lot of the participants got distracted whilst watching the stimulus due to their friends being close to them and the availability of looking away from the stimulus. Have each participant an individual viewing window that is free from possible distractions, such as friends or other callousness classroom items Time Taken to Measure the Heart Rate After the stimulus was presented to the participants, it was their role to pump the heart rate monitors enough so that an accurate reading could be read.

Some participants started to pump the heart rate monitor a few seconds after the stimulus had finished, this may have caused an inaccurate reading as after the stimulus has finished, the heart rate will lower to adapt to the original environment. Whilst the stimulus is taking place, have the heart rate monitored pumped so that when the stimulus was finished the heart rate could be measured instantaneously. Social Aspects As the experiment was conducted within a group environment, there is a social factor in the heart rate of the participants.

Conduct the experiment individually with the participants to minimalism the social factor on the heart rate. Previous Activities Affecting Heart Rate The experiment was conducted late afternoon (2 pm) during a regular school day. This means that the participants could have done many things throughout the day that could have potentially affected their heart rate prior to entering the experiment thus affecting the data within the experiment. Have the participants undergo a alleging activity for 5 minutes prior to visual stimuli of the investigation.

Cite this page

https://graduateway.com/the-effect-of-visual-stimuli-on-heart-rate/

You can get a custom paper by one of our expert writers

• Natural Selection
• Endocrine system
• Female Genital Mutilation
• Organic compounds
• Reproduction
• Cardiovascular system
• Regeneration

Check more samples on your topics

Investigate the effect of caffeine on the heart rate of daphnia.

Investigating the effects of Caffeine on the heart rate of Daphnia Shrimp. Aim: In this experiment we are trying to find out how varying the concentration of caffeine can affect the heart rate of Daphnia shrimp. Prediction: I predict that caffeine will increase the heart rate of the shrimp . Introduction: I believe the results

Effect of Caffeine on the Heart Rate of Daphnia

Caffeine is made by plants as a way of getting rid of insects. Cocoa is produced in South America, coffee in Africa and tea is produced in Asia have all been used for a very long time to give us a little rush in order to keep us going. Now caffeine is also used as

Visual Advertising Research Paper Visual AdvertisingThe

Advertising

Communication

Ocular Advertising Essay, Research Paper Ocular Ad The universe of advertisement is the foundation for how a company communicates its merchandise to the consumers. Here a complex battle takes topographic point continually to derive the attending, even for a 2nd, of a possible purchaser. The footwear industry is no exclusion, and when combined with the semiologies of

Adaptation to Sensory Stimuli: Texture, Temperature, and Light

Temperature

            The demonstrations reported below illustrated the process of sensory adaptation, where we become less sensitive to an unchanging stimulus over time (Davis & Palladino, 2006). Demonstrations             Texture.  In a demonstration of adaptation to a texture, an example of the sense of touch, I rubbed each of my index fingers back and forth several times over

From Stimuli to Long Term Memory

In general, memory is taken as a process in the brain, wherein encoding, storage and retrieving of data happens. Basically there are two general types of memory: explicit and implicit. As it names denotes explicit memory refers to clear information and usually to ‘first hand statements or declarations’. Thus, commonly explicit memory involves autobiographical information

Internal Rate of Return Vs. Modified Internal Rate of Return

Both IRR and MIRR are methods used for valuation.Proper firm valuation is essential for efficient planning and allocation. The decisions made by managers, in both the short and long term, greatly affect a firm's value. Accurate valuation methods are crucial in assessing the impact of projects and business activities on this value. The internal rate

Daphnia Heart rate – Lab Report

Introduction: Caffeine is a stimulant which is derived from leaves, flowers and seeds of plants. The plant uses the caffeine as a way to repel insects as it is used as a natural pesticide by paralyzing and killing certain insects. It is used as a stimulant to human the central nervous system by increasing the stimulatory

Daphnia Heart Rate Experiment

The purpose of the experiment is to examine the impact of the drug Caffeine on the heartbeat of a Daphnia. This investigation can offer valuable information about its effects on human heart rates. The use of Daphnia guarantees an impartial and secure experiment. These organisms, also known as water fleas, are crustaceans that belong to

Heart Rate Investigation

The heart rate of a human varies from 150 beats per minute in young children, to about 60 beats per minute in the aged, however I am going to investigate whether my heart rate increases or decreases during physical exercise and why it does this. PredictionFrom my general knowledge I predict that my respiration rate will

Hi, my name is Amy 👋

In case you can't find a relevant example, our professional writers are ready to help you write a unique paper. Just talk to our smart assistant Amy and she'll connect you with the best match.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• v.7(3); 2021 Mar

Exploration of illusory visual motion stimuli: An EEG-based brain-computer interface for practical assistive communication systems

Yunyong punsawad.

a School of Informatics, Walailak University, Nakhon Si Thammarat, 80160 Thailand

b Informatics Innovative Center of Excellence, School of Informatics, Walailak University, Nakhon Si Thammarat, 80160 Thailand

Nannaphat Siribunyaphat

Yodchanan wongsawat.

c Department of Biomedical Engineering, Faculty of Engineering, Mahidol University, Nakhon Pathom, 73170 Thailand

Associated Data

Data included in article/supplementary material/referenced in article.

This paper presents an illusory visual motion stimulus-based brain-computer interface (BCI). We aim to use the proposed system to enhance the motor imagery (MI) modality. Since motor imagery requires a long time for training, a stimulation method with external stimuli through the sensory system is an alternative method for increasing efficiency. The research is divided into two parts. First, we observed the visual motion illusion pattern based on brain topographic maps for the novel BCI modality. Second, we implemented the illusory visual motion stimulus-based BCI system. Arrow and moving-arrow patterns were used to modulate alpha rhythms at the visual and motor cortex. The arrow pattern had an average classification accuracy of approximately 78.5%. Additionally, illusory visual motion stimulus-based BCI systems are proposed using the proposed feature extraction and decision-making algorithm. This proposed BCI system can control the cursor moving in the left or right direction with the designed algorithm to create five commands for assistive communication. Ten volunteers participated in the experiment, and a brain-computer interface system with motor imagery and an illusory visual motion stimulus were used to compare efficiencies. The results showed that the proposed method achieved approximately 4% higher accuracy than motor imagery. The accuracy of the proposed illusory visual motion stimulus and algorithm was approximately 80.3%. Therefore, an illusory visual motion stimulus hybrid BCI system can be incorporated into the MI-based BCI system for beginner motor imagery. Based on the results, the proposed assistive communication system can be used to enhance communication in people with severe disabilities.

Biomedical engineering, Electroencephalogram, Brain-computer interface, Illusory motion, Assistive communication devices.

1. Introduction

The brain-computer interface (BCI) is a modern technology used for communication between humans and external devices via brain signals [ 1 , 2 ]. BCIs are widely and continuously used in many kinds of applications, such as biometric, prevention, economic, education, sports, and medical applications, including diagnosis, assistive technology, and rehabilitation [ 3 , 4 , 5 , 6 , 7 , 8 , 9 , 10 ]. BCIs can be divided into invasive and noninvasive BCIs according to the acquisition technique. Many researchers prefer to develop noninvasive BCIs by using an electroencephalogram (EEG) signal. EEG signals are electric brain signals obtained by placing electrodes on the scalp following the international 10–20 system to measure the summation of neuron potentials. EEG machines are small, flexible and portable, employing a dry electrode with a wireless system. A new EEG machine has high efficiency in measuring and recording signals with a high resolution. Popular EEG features include event-related desynchronization/synchronization (ERD/ERS) via a mental motor imagery (MI) task and visual evoked potentials (VEPs), which are direct responses to visual stimuli through the optic nerve (1). Examples of VEP-BCIs, transient VEPs or P300, and steady-state visual evoked potentials (SSVEPs) [ 11 , 12 ] can achieve high accuracy and require less time for training. However, like natural thinking, the motor imagery paradigm is still a favorite and challenging topic for BCI research. Mobility enhancements, such as electric wheelchairs and robots controlled via EEG during motor imagery [ 13 , 14 ], are a popular application for people who have a severe disability. We can collect brain data during imagery tasks, which requires practice to create EEG features, such as ERD/ERS. The MI-based BCI system communicates with devices using imaging of physical movement to generate the signal and convert it into a command to operate the machine. However, the MI-based BCI system is still not suitable for all users. The system requires a training session. To enhance the performance of motor imagery-based BCIs [ 15 , 16 ], previous research proposed a novel MI paradigm and integrated motor imagery with other EEG features to create a hybrid BCI system [ 17 , 18 , 19 , 20 , 21 , 22 , 23 , 24 ]. For example, Horki et al. proposed using ERD and SSVEP for MI-based BCIs and combining a multisensory approach with visual and somatosensory stimulation. Event-related potentials (ERPs) were experimentally determined by placing arms on the table, putting 3 LED lights between the arms at the same distance and applying somatosensory stimulation to both wrists. A flashing LED that approaches the wrist had a higher incidence of stimulation. The results show that visual stimulation is an automatic condition of somatosensory stimulation [ 17 ]. Moreover, Allison and team introduced two BCI modalities, motor imagery and steady-state visual evoked potentials (SSVEPs) [ 18 ]. The EEG signals were recorded under three conditions: 1) imaging the movement of the left hand or right hand, 2) stimulation with visual attention, and 3) the use of both methods simultaneously. Switching among all three methods can improve accuracy for some of the subjects. The result of this experiment can explain why the hybrid has 81.0% accuracy, while ERD has 74.8% and SSVEP has 76.9%. However, some subjects can improve the accuracy when using the proposed technique because some of them cannot combine two different BCI approaches or one of the signals has high accuracy. Furthermore, Ma et al. [ 19 ] explained the combination of motor imagery (MI) and motion-onset visual evoked potential (mVEP) for the new hybrid brain-computer interface system to improve the efficient 2D movement control of a cursor. The results from seven subjects show that the proposed system could evoke the MI and mVEP signals simultaneously, and both were very close to the single-modality BCI task in the offline experiment; the single-modality MI 75% mVEP 85%, and the Multimodality MI 77% mVEP 84%. The online experiment provided more efficient and natural control commands. However, subjects needed to perform multiple tasks simultaneously, and they allocated more attention to MI while gazing at visual stimuli. In addition, Xiaokang et al. [ 20 ] proposed unilateral tactile stimulation (Uni-TS) for motor imagery, which was employed in an experiment with two groups of participants: a control group performing motor imagery with both hands and an enhanced group contacting the tactile stimulator with disabled hands and motor imagery. The results indicated that using Uni-TS significantly affects contralateral cortical activation during MI of hand stimulation. The accuracy of this proposed system improved from 72.5% to 84.7%. Moreover, stroke patients in the enhanced group achieved an accuracy of more than 80%. However, this experiment had a small sample of stroke patients, which may not represent patients with tactile sensation problems. Additionally, Sangtae et al. [ 18 ] presented a new hybrid brain-computer interface that integrates two different EEG tasks: tactile selective attention using steady-state somatosensory evoked potentials (SSSEPs) and motor imagery using event-related desynchronization (ERD). They divided the experiment into 4 methods: 1) motor imagery paradigm, 2) tactile selective attention paradigm, 3) hybrid paradigm-simultaneous approach, and 4) hybrid paradigm-consecutive approach. The consecutive approach achieved the best performance compared to the other methods. The classification accuracy of this method improved by approximately 10% compared with motor imagery. Moreover, the enhancement paradigm with a moving rubber hand illusion system [ 22 ] presented a method for improving ERD using body illusion, also known as the rubber hand illusion, using a motor-driven mechanical hand. In the first experiment, subjects attempted to move the right wrist when a green light appeared on the screen randomly. In the second experiment, the subjects performed the task with the mechanical hand while the real right hand was covered with a blanket to prevent visual feedback. Then, the subjects touched the mechanical and real hands in the same position. When the subjects felt that the mechanical hand was a real hand, they imagined that it was similar to being touched by a real hand. The results improved the ERD method but were not sufficient for motor execution. Therefore, we attempted to investigate a new visual illusion stimulus for motor imagery enhancement. Additionally, BCI system evaluation is influenced by human cognition when performing a motor imagery paradigm. The subjects performed a conventional motor imagery paradigm. After that, they generated an estimate by themselves without receiving feedback and then compared the two values. The subjects accurately predicted the effectiveness of motor imagery-based BCIs [ 8 ].

Previous research on the enhancement of motor imagery had four primary purposes: 1) developing feature extraction and classification methods, 2) investigating novel MI paradigms and feedback, 3) developing an approach comprising a user training system, and 4) proposing a hybrid BCI system. In this work, we consider a novel paradigm from human perception to induce motor activity. We employ a phenomenon from illusory motion visual stimulation involving the wheel pattern, arrow pattern, and moving-arrow pattern for an MI-based BCI. The proposed system represents a novel BCI system, and we hope that the proposed system can enhance the communication of people with severe disabilities. Moreover, illusory visual motion stimuli are further integrated with motor imagery for user training sessions. The methods can be divided into two main parts: the investigation of the visual illusory motion stimulus paradigm and brain activity, and the design of illusory motion visual stimulus-based BCIs and evaluation.

2. Proposed methods

2.1. proposed assumption and visual illusory motion stimulation paradigm.

For brain stimulation, depending on the kind of stimulus or paradigm, a brain cortex that responds with a decrease in alpha power (8–12 Hz) is called event-related desynchronization (ERD), or an alpha power increase is called event-related synchronization (ERS). For example, visual attention can generate alpha rhythm at prefrontal and occipital areas. Motor imagery can generate alpha rhythms in frontal and central areas. Furthermore, previous studies observed a flickering wheel illusion pattern by Sokoliuk and VanRullen [ 25 ]. They reported that the flickering wheel frequency of visual illusion correlated with the EEG frequency in the alpha band.

We repeated the exploration by using our paradigm and a wheel pattern based on EEG frequency analysis [ 26 ]. The wheel pattern consisted of 24 pieces (angle between propeller = 5°), 48 pieces (4°), and 96 pieces (3°) of the propeller with a 5 cm diameter and was used to stimulate the visual and motor cortex. We also found that the number of pieces in the wheel pattern affected the alpha band, and 96 pieces quickly induced a response in the occipital and central areas of the brain. Therefore, we proposed a new illusory motion stimulation paradigm for activating a response in the visual and motor cortexes by using the arrow pattern shown in Figure 1 (a), consisting of an illusory motion arrow pattern (picture) and a moving-arrow pattern (video: 5 frames per second) to stimulate the visual cortex and induce the motor to compare both EEG responses. For this assumption, we assumed that by focusing on the illusory motion stimulus pattern such that the eyes are stimulated and the motors are induced in the left/right direction, alpha asymmetry [ 27 ] in the central and occipital areas can be observed. We set up a new experiment to explore the paradigm and illusory motion stimulus. The results were used to design the feature extraction method and algorithms to extend the use of our proposed visual illusory motion stimulus patterns and paradigm in a real-time motor imagery-based BCI system.

(a) Arrow pattern for illusory motion stimulation. (b) Focusing paradigm on the left arrow for inducing the right visual and motor cortex. (c) Focusing paradigm on the right arrow for inducing the left visual and motor cortex.

Using the paradigm shown in Figures  3 (a) and (b), two commands were generated using the left and right directions of the arrow by looking at the proposed illusory motion stimulator according to the focusing paradigm. The two commands were performed as 1) both eyes looking at the center of the left-direction arrow for left command and 2) both eyes looking at the center of the right-direction arrow for the right command.

Example brain topographic maps for the visual illusory motion stimulus in representative subject 1. (a) Illusory motion stimulation with the left arrow, (b) illusory motion stimulation with the right arrow, (c) illusory motion stimulation with the left-moving arrow, and (d) illusory motion stimulation with the right-moving arrow.

2.2. Preliminary study

In this preliminary experiment, we used a 19-channel Brainmaster Discovery 24E for EEG signal acquisition at a sampling rate of 256 Hz. During preprocessing, the recorded signals were filtered for power line noise by a 50-Hz notch filter, and a 2-Hz to 40-Hz bandpass digital filter was used for motion artifact removal. Seven healthy subjects (mean age 22 ± 3.4 years) participated in the experiment by following the task sequence, as illustrated in Figure 2 . A single trial consisted of four events, starting with focusing at fixation (+) for EEG baseline collection over 5 s. Then, the subject rested for 3 s. After that, the subject stared at the left or right arrow illusory motion visual stimulator for 5 s. Finally, the subject rested for 3 s. The subjects performed the same sequences in Figure 2 for moving the arrow illusory motion visual stimulator (video). Each subject randomly performed left and right commands, with 20 trials per stimulus pattern and 80 trials for each subject.

Task experiment for visual illusory motion stimulation.

NeuroGuide software was employed to visualize the brain response during alpha rhythm (including sensorimotor rhythm and visual attention response) signal analysis. This software provides dynamic normative EEG comparisons in real time during editing and automatic artifact rejection and has been used for clinical and research purposes [ 28 ]. According to the grand-averaged brain topographic mapping of the FFT absolute power for all subjects, we visually observed the feature pattern for each illusory motion pattern ( Figure 3 and Figure 4 ). The brain areas of interest were the central, parietal, and occipital areas. We found that the brain response occurred through visual attention and motor intention.

Example brain topographic maps for the visual illusory motion stimulus in representative subject 2. (a) Illusory motion stimulation with the left arrow, (b) illusory motion stimulation with the right arrow, (c) illusory motion stimulation with the left-moving arrow, and (d) illusory motion stimulation with the right-moving arrow.

Nonmoving-arrow pattern: For the left-direction stimulus, we observed that the right central (C4), right parietal (P4), and left occipital (O2) regions exhibited a greater response in the alpha band (8–12 Hz), as shown in Figure 3 (a) and Figure 4 (a) for subjects 1 and 2, respectively. In contrast, for the right-direction stimulus, we observed that the left central (C3), left parietal (P3), and left occipital (O1) regions exhibited a greater response in the alpha band (8–12 Hz), as shown in Figure 3 (b) and Figure 4 (b) for subjects 1 and 2, respectively.

Moving-arrow pattern: For the left-direction stimulus, we observed that only the left occipital (O2) exhibited a greater response in the alpha band (8–12 Hz), as shown in Figure 3 (c) and Figure 4 (c) for subjects 1 and 2, respectively. For the right-direction stimulus, we also observed that only the left occipital (O1) exhibited a greater response in the alpha band (8–12 Hz), as shown in Figure 3 (d) and Figure 4 (d) for subjects 1 and 2, respectively.

2.3. Proposed BCI system based on an illusory motion stimulus

We employed the results in section 2.2 to propose an illusory motion stimulation-based BCI system for enhancing motor imagery, as shown in Figure 5 . We selected the arrow illusory motion pattern and paradigm ( Figure 1 ). According to the preliminary results, we designed an algorithm to extract features to classify EEG signals into left/right commands to control the cursor of the proposed assistive communication system ( Figure 6 (b)). We also used the visual-based bar graph level indicator to feedback the response and guided the user to achieve high efficiency for user practice.

The proposed BC system based on an illusory motion stimulus.

The real-time illusory visual motion stimulus-based BCI system for controlling the cursor of the assistive communication system. (a) Decision flowchart for the direction of cursor movement. (b) Graphic user interface (GUI) consisting of a 1) left arrow illusory motion stimulator, 2) right arrow illusory motion stimulator, 3) bar graph for visual feedback, and 4) the communication panel contains one message box for the question by typing and four choices with pictures or words for answering that were defined by the caretaker. The patient performed choice selection with cursor control.

2.4. Illusory visual motion stimulus-based BCI for a practical assistive communication system

Using the proposed system with the arrow pattern and paradigm in Figure 1 (b) and (c), we implemented a real-time illusory visual motion stimulus-based BCI system for cursor control via a left/right arrow illusory motion stimulator ( Figure 6 (b)). Five choices were selected by moving the cursor in the left or right direction by following the diagram in Figure 6 (a).

Before commanding the system, the red circle in the middle was activated as the default command. Then, the user moved the cursor in the left or right direction to make a request. Additionally, the caretaker could input other pictures or word questions and answers to communicate with the user. The user instructions were summarized as follows:

• 1) User calibration was performed by following section 3.3.
• 2) The user stared at the left or right arrow illusory motion stimulator to move the cursor in the left or right direction to stop at the desired command.
• 3) The caretaker responded to the requirement and presses the red button to reset.

3. The real-time illusory visual motion stimulus-based BCI system

3.1. eeg acquisition.

Based on the results from section 2 , two bipolar channels, O1–C3 and O2–C4, were acquired using a BIOPAC™ system EEG amplifier. The electrode positions followed the international 10–20 electrode placement system. The acquired signals were filtered by an analog bandpass filter with cutoff frequencies at 1 and 35 Hz to avoid artifacts. A 50 Hz analog notch filter was used to remove power line noise. For analog-to-digital A/D conversion, a National Instrument (NI) USB 6009 multifunction data acquisition card was used with a sampling rate of 256 Hz to convert the analog signals to digital data. A 3–35 Hz digital bandpass filter was used to remove power line noise and motion artifacts. This study protocol was approved by the Institutional Review Board (IRB) of Mahidol University in compliance with the Declaration of Helsinki, The Belmont Report, CIOM Guidelines and the International Conference on Harmonization in Good Clinical Practice (ICH-GCP): MU-CIRB 2017/037.2802. Written informed consent was obtained from all participants prior to participating in this study.

3.2. Feature extraction and decision-making algorithms

For the proposed real-time BCI system, a simple signal processing and decision-making algorithm for visual attention and motor intention detection using Welch's periodogram method algorithm was performed for fast computations [ 29 , 30 ].

• 1) Calibration: Before using the proposed system, baseline parameters were collected while the user looked at a blank screen for 4 s five times.

B O1 was defined as the baseline relative alpha power in EEG channels O1–C3, and B O2 was defined as the baseline relative alpha power in EEG channels O2–C4, which were calculated as:

• 2) Feature Extraction: D O 1 are the differences in relative alpha power between RP O1 and B O1 , and D O 2 are the differences in relative alpha power between RP O2 and B O2 . According to our assumption, the alpha band should increase. Hence, the index was defined to allow the difference level to be greater than 0.25, to multiply B O 1 and B O 2 by 1.25 as the threshold for making the decision to calculate the parameters D O 1 and D O 2 , and by the following: D O 1 = { ( R P O 1 − B O 1 ) , R P O 1 − B O 1 > 0 0 , R P O 1 − B O 1 < 0 (3) D O 2 = { ( R P O 2 − B O 2 ) , R P O 2 − B O 2 > 0 0 , R P O 2 − B O 2 < 0 (4)

Of the EEG features acquired during stimulation, RPO1 is the relative power spectral density (PSD) of the alpha band of the EEG signals from the O1 position, and RPO2 is the relative PSDs of the alpha bands of the EEG signals from the O2 position, which are calculated as:

• if D O 1 < D O 2 , Decision is “Left”
• if D O 1 > D O 2 , Decision is “Right”
• if D O 1 = D O 2 = 0 , No Decision.

4. Experiments

4.1. experiment i: performance evaluation of the arrow and moving-arrow illusory motion stimulation pattern and the proposed algorithms.

Ten healthy subjects (mean age 24 ± 3.8 years) without any BCI experiences were enrolled. The experimental cue was defined by randomly asking the subject to stare at the center of each stimulator (left arrow or right arrow) for 5 s to create an output command using the proposed algorithm to detect and automatically calculate the accuracy. Each subject performed 40 trials (20 trials for the left stimulus and 20 trials for the right stimulus) by following the paradigm in Figure 1 (b) and (c).

4.2. Experiment II: performance of illusory visual motion stimulus-based BCI for a practical assistive communication system

Ten healthy subjects from the previous experiment, seven of whom had no experience with real-time BCIs, participated in this experiment ( Figure 7 ). All subjects were trained for 20 min on how to perform the illusory visual motion stimulus-based BCI system. Each subject performed three trials per day. For each trial, there were 10 commands. Each subject performed motor imagery and visual motion stimulation to control the cursor following the sequence in Table 1 . The experiment consisted of two days for user performance verification. The classification accuracy is reported in Table 3 .

Experimental setup.

Table 1

The task for testing the system.

Note: R = right and L = left.

Table 3

Results of the real-time illusory visual motion stimulus-based BCI system for the assistive communication system.

5. Results and discussions

According to the results in Table 2 , by using our proposed algorithm, the average classification accuracy of the proposed system for individual commands ranged from 65% to 90%, the maximum accuracy was achieved by the arrow pattern, and the average accuracy of the arrow pattern was 80% and 77% for the individual left and right commands, respectively. Compared with previous works, the performance of the arrow pattern ranged from the previous visual modality to combined MI as combining motor imagery and moving onset, 77%–84% [ 19 ]. The moving-arrow stimulus pattern yielded slightly lower accuracy. Moreover, using the moving-arrow pattern can easily induce eye fatigue in the subject. Therefore, we employed the arrow pattern for illusory motion visual stimulus-based BCI for a practical assistive communication system.

Table 2

Results of the left/right commands of the arrow and moving-arrow illusory motion stimulation pattern.

According to the results in Table 3 , two issues were listed. The first is the efficiency of BCI methods. The maximum average accuracy of the motor imagery was 76.0%, and the average accuracy of the proposed method was 80.3% from the second time of the experiment (the next day). The proposed illusory visual motion stimulus can yield higher maximum accuracy than motor imagery, at approximately 4%.

The second issue considers user performance. In the first experiment, the average accuracy of motor imagery ranged from 60% to 80%, and the average accuracy of the proposed method ranged from 66.7% to 86.7%. In the second experiment, the average accuracy of motor imagery ranged from 73.3% to 83.3%, and the average accuracy of the proposed method ranged from 73.3% to 90%. The proposed method achieved high accuracy for the first time. However, with user progression, motor imagery can increase the accuracy to exceed that of the proposed methods. Individual subjects reported equivalent results between the motor imagery and illusory visual motion stimulus methods.

Following the preliminary study to verify our assumption of using illusory visual motion stimulation for brain-computer interfaces, brain topographic maps of the illusory visual motion stimulus indicated an asymmetry of central and occipital areas. Following the preliminary result, we intend to verify the assumption by including additional participants. In addition, we generate a protocol for the users of the proposed illusory visual motion stimulus-based BCI systems. Experiment I indicated that the arrow illusory motion stimulation pattern (picture) provides a better stimulus than the moving-arrow illusory motion stimulation pattern (video) and for the proposed feature extraction and decision-making algorithm. With the use of motor imagery, some users may have difficulty performing and need time for training. Hence, using illusory visual motion stimulation can induce motor areas and can create two commands. This can be added to additional motor imagery-based BCI systems. Finally, an application in real-time assistive communication manipulation was demonstrated in Experiment II, and acceptable accuracies were obtained.

6. Conclusions

In this study, we proposed an illusory motion visual stimulus pattern for a practical assistive communication system. To explore our proposed illusory visual motion stimulus for brain-computer interfaces in section 2 , brain topographic maps for the visual and motor intention in the left and right directions initially displayed asymmetry of the occipital and central areas on the opposite side of the stimulus direction. The central area (C3 and C4), which is a motor function area, is an efficient reference electrode that can acquire bipolar EEG signals on each side of the visual area (occipital area) for visual motor intention detection. Section 3 shows that the nonmoving-arrow pattern (picture) is more user-friendly than the moving-arrow illusory motion stimulus pattern (video) for verifying the proposed feature selection and decision-making algorithm. We also generated a protocol for users of the proposed illusory visual motion stimulus-based BCI. Finally, the real-time practical assistive communication system was demonstrated in section 4 , and user-friendliness was obtained. The proposed application can be used for communication enhancement in people with severe disabilities. Furthermore, we conclude that the illusory visual motion stimulus method can be used for a real-time BCI system and can also be further employed to increase the efficiency of motor imagery.

7. Limitations

Some limitations of illusory visual motion stimulus-based BCI systems when applied to assistive communication systems should be reported:

• 1) Following the initial verification of illusory visual motion stimuli with a small number of subjects, we aimed to further examine additional subjects.
• 2) To use the proposed system, some subjects could not see the illusion movement within the arrow every time within a short period of time, which can cause a low iteration transfer rate (ITR).
• 3) For multicommand BCIs, the proposed system yielded lower efficiency than VEP-based BCIs (P300 and SSVEP). Nevertheless, natural actions such as the thought of moving the left hand or right hand and left or right visual spatial attention are still popular for a BCI system. With two commands creation, "yes" or "no," the proposed achieved acceptable efficiency for assistive technology for communication.

Declarations

Author contribution statement.

Yunyong Punsawad: Conceived and designed the experiments; Performed the experiments; Analyzed and interpreted the data; Wrote the paper.

Nannaphat Siribunyaphat: Performed the experiments; Analyzed and interpreted the data; Wrote the paper.

Yodchanan Wongsawat: Conceived and designed the experiments; Analyzed and interpreted the data; Wrote the paper.

Funding statement

This work was supported by Walailak University.

Data availability statement

Declaration of interests statement.

The authors declare no conflict of interest.

Additional information

No additional information is available for this paper.

Acknowledgements

The authors would like to acknowledge all volunteers in this research. This work involved an EEG machine and experimental setting provided by the Brain-Computer Interface Laboratory (BCI LAB), Department of Biomedical Engineering, Mahidol University, Thailand.