Virtual reality has been a part of our culture for decades. This technology began in the late 1950s, with a first device called “Sensorama” that consisted of a booth with a swivel chair that projected stereoscopic images. Today, Virtual Reality has been massively developed as a promising technology increasingly present in our lives. But, how does the use of this technology influence the way the brain perceives information? Based on the latest neuroscientific research, this article addresses the key changes our brain implements to learn and handle information virtual environments.
The real experience in a virtual world begins with the sense of presence
The sense of presence refers to how much a person feels being in the virtual environment rather than the physical one. In other words, the sense of presence is the subjective psychological response of “being there” fully inmerse in the virtual reality experience (Berkman and Akan, 2018).
There are three common features that characterize presence in the context of VR (Gaggioli et al. 2003).:
(1) High sensory engagement into the virtual environment
(2) A feeling or mental state (e.g. motivation) allowing the subjective experience
(3) The interplay of perceptual, cognitive and affective components eliciting physiological responses.
The virtual reality-based technology are well known to incur visual, auditory, and haptic induced perceptions in a whole inmersive experience that mimics the real-world (sensory engagement). In virtual reality your senses behave identically to when they percieve “real” enviroments making difficult to distinguish the real experience from an illusion or simulation (subjective component). For example, when you navigate in a VR space, you see things in 3D, just as you would if it was actually there. This such a robust immersive experience makes your brain to respond as though it is really happening (multidimensional experience).
The only difference between real and virtual “worlds” is the prior knowledge you have that what you see in a virtual environment is not real, allowing you to act as if it really is but without fear.
For example, consider a person with phobia to spiders inmerse in a VR enviroment were many insects are shown. Knowledge of not being hurt in this virtual world will change the way the person face up to the stimuli that terrify him. Such interaction teaches the person´s brain that the threatening event can be experienced without disaster, which ultimately prepares the person for facing them outside the virtual environment.
VR technology synergies with neuroimaging
To comprehend how virtual reality experience actually affects human brain, researchers have used different neuroimaging techniques allowing to assess the brain activity in response with external stimulations (Mishra et al., 2021).
Commonly used virtual reality-based devices produce visual, auditory, and haptic induced perceptions that generate specific neurophysiological signals. These VR devices can be for instance screen-based or head-mounted systems. Effectively integrating these virtual reality-based devices with real-time neuroimaging recording systems are useful in correlating human perception with brain responses in a virtually designed environment.
For example, there are various brain recording systems, such as electroencephalography (EEG), magnetoencephalography (MEG), and magnetic resonance imaging (MRI) that are well-known for registering both, resting-state and active brain states at a very accurate level (Padmanabhan et al., 2018). An optimal combination of VR-based elements with these neuroimaging techniques enables to study human behavior, spatial navigation performance, and spatial presence, to name a few.
In a typical experiment set up, ongoing brain data is registered while the user is being stimulated within the VR inmmersive experience. The recorded brain signal can be decompose in different frequency bands (e.g., Neural Oscillations), or map into brain areas (e.g. hemodynamic response) that are active during VR stimulation.
Researchers are currently working with such integrative approaches in biomedicine, education, healthcare, marketing and gaming by combining human brain signal systems with modern VR-based technology. The effective blending of all these elements is key to obtaining valuable insights on users cognitive states, their perceptions and other neurophysiological correlates that emerge during the immersive experience.
Does my brain behave differently in a virtual world?
Partly yes, although the research in this regard is not yet conclusive.
The study of the brain undepinings of VR experience is a topic of great interest for researchers. A main goal include investigating the brain changes after exposure to VR. Scientific evidence suggest that different kinds of experiences lead to different brain structures and neural functioning (Dr Bruce D. Berry, Baylor College of Medicine). It is likely, therefore, that experience using virtual reality technologies modifies the brain at different levels. This idea is based on the assumption that the brains of early adopters of virtual reality devices may be hardwired differently from non-adopters brain, as a results of experience. We know that this is possible thanks to the neuroplasticity property that characterizes our brain.
To better understand how VR affects brain functioning, we first need to understand how the brain makes sense of the world around us. Considering that we experience reality based on information captured by sensory organs such as the eyes, ears, and skin, the realistic sensation driven by VR experiences, relies on the following processes:
- Sensation: involves sensory stimulation of our senses
- Transmition: is the act of sending the information captured by our sensory organs to the brain.
- Perception: is the mental representation of stimuli and involves a sequence of steps such as organization, and interpretation of selected stimuli.
- Experience: the interplay of sensation and perception procesess (our brain’s interpretation of this information) creates our experience of reality.
The VR-based devices are well-known to incur visual, auditory, and haptic induced sensations, enhancing perception, motor behavior, and sensorimotor adaptation (Wright, 2014). It has the potential to positively impact cognition augmenting the brain executive functioning in many different ways (e.g., learning, memory, motor control).
Part of such brain capabilities enhacement relies on the aforedmentioned sense of presence. As shown by a group of researchers from the University of Maryland, brain becomes more accurate at encoding memories when we experience the vividness of presence in VR. Specifically, their study showed 9% improvement in memory accuracy when learning in VR compared to learning using a desktop computer (Krokos, E., Plaisant, C. & Varshney, 2018).
In addition to the perception of the external world, virtual reality changes how we perceive ourselves. Researchers at the Virtual Human Interaction Laboratory in Stanford demonstrated that after a few minutes in virtual reality embeded with the avatar of an elderly person, the brain begins to adapt, adopting the avatar’s body as its own. Such rapid internalization resulted in a significant impact on participants attitude towards eldery (Yee and Bailenson, 2006). In another study, researchers show that people with a distorted representation of the size of their body, when placed in a healthy sized avatar resulted in an improvement of their self-body image (Keizer et al., 2016).
Although these are fascinating findings, there’s still much about the underlying neurological mechanisms that remain unknown in this field. The study of brainwaves in the context of VR is an interesting field of research that is bringing new insights in this regard.
Are brainwaves tuned for VR? Insights from scientific research
A person “moving” through a VR space experiences very different types of stimulation. One function the brain does while navigating in VR is to gather all types of information of the enviroment throught activation of differenct brain regions connectivity. How neurons retune and boost brain rhythms to discern between a simulated or real environment is a current topic of strong research. Using EEG, this type of questions can be explored to figure out how the neurons response to VR stimulation (Baumgartner et al., 2006).
The role of Theta (4-8Hz) and Alpha brainwaves in VR inmersion
An interesting finding in current research is a positive relationship between sense of presence in VR and higher activation of Alpha and Theta bands (Clemente et al., 2014). In memory-based studies, a significant role of both types of brainwaves have been shown, likely mirroing visual-spatial memory processes emerging during virtual navigation. This is what a group of researchers at the University of Pennsylvania investigated using an integrated VR and EEG system. The participants were instructed to navigate a virtual corridor and remember the routes to access to different destination rooms while their brain activity was recorded. EEG markers of theta band activity appeared more predominant during the encoding of visuospatial information, while the Alpha band activity increased during the short-term memory retrieval phase (Jaiswal, Ray and Slobounov, 2010) .
Theta rythm also appears critical on perception processes involved in VR navigation. In a exploratory study in rats, researchers demostrated that the brain responds differently in immersive virtual reality environments versus the real world (Safaryan and Mehta, 2021). They found that stronger theta rhythm enhanced the brain’s ability to learn and retain sensory information. Such novel finding is a revealing groundbased to further understand how the human brain integrates sensory information from different sources to build a united picture of the percieved world around us.
Applications and research fields
The specific knowledge of the neural mechanisms that arise during virtual reality immersion is paving the way for “virtual reality therapy” approaches. For example, benefits of inmersive VR in the treatment of neurodevelopmental and neurodegenerative disorders including ADHD, autism, Alzheimer’s disease and epilepsy, has been confirmed in multiple studies ( Albani et al., 2010; Bashiri, Ghazisaeedi, & Shahmoradi, 2017; Choo and May, 2014; Moseley, 2017; Mühlberger et al., 2020) .
The use of virtual reality technology to improve cognitive performance via brainwave entrainment is also a promising field of research. This approach is becoming more and more prevalent in medical rehabilitation of different pathologies like motor or neurological damage (Gaetano et al., 2018; Georgiev et al., 2021; Krokos and Varshney, 2021) as well as in other non-clinical fields such as neuromarketing (e.g., advertising, percepción).
The current trend in research is precisely directed towards a deeper understanding of the brain mechanisms that operate in virtual reality environments. This will allow to promote of a wide range of virtual reality applications based on brain wave training protocols better adapted to the specific needs of users and compatible with their neurobiology.
Conclusions
With the emergence of virtualization in contexts increasingly close to people’s daily lives (supermarkets, educational centers, hospitals, leisure, etc.) the human brain has been forcibly adapting to handle a large amount of information from a very different way than it did just 20 years ago. VR technology allows immediate access to many more sources of information, which makes the brain adapt, changing thanks to plasticity the way neurons communicate and organize themselves to manage it.
While it certainly sounds advantageous, there are questions yet to be investigated such as the effects that this immersive technology would have on cognition and brain organization in future generations of natives of virtual reality technology.
References
Argento, E., Papagiannakis, G., Baka, E. et al. Augmented Cognition via Brainwave Entrainment in Virtual Reality: An Open, Integrated Brain Augmentation in a Neuroscience System Approach. Augment Hum Res 2, 3 (2017). https://doi.org/10.1007/s41133-017-0005-3
Bashiri, A., Ghazisaeedi, M., & Shahmoradi, L. (2017). The opportunities of virtual reality in the rehabilitation of children with attention deficit hyperactivity disorder: a literature review. Korean journal of pediatrics, 60(11), 337–343. https://doi.org/10.3345/kjp.2017.60.11.337
Baumgartner T, Valko L, Esslen M, Jäncke. Neural correlate of spatial presence in an arousing and noninteractive virtual reality: an EEG and psychophysiology study. LCyberpsychol Behav. 2006 Feb; 9(1):30-45.
Berkman M.I., Akan E. (2019) Presence and Immersion in Virtual Reality. In: Lee N. (eds) Encyclopedia of Computer Graphics and Games. Springer, Cham. https://doi.org/10.1007/978-3-319-08234-9_162-1
Baumgartner T, Valko L, Esslen M, Jäncke L. Neural correlate of spatial presence in an arousing and noninteractive virtual reality: an EEG and psychophysiology study. Cyberpsychol Behav. 2006 Feb;9(1):30-45. doi: 10.1089/cpb.2006.9.30. PMID: 16497116.
A. Choo and A. May, “Virtual mindfulness meditation: Virtual reality and electroencephalography for health gamification,” 2014 IEEE Games Media Entertainment, 2014, pp. 1-3, doi: 10.1109/GEM.2014.7048076.
Clemente, M., Rodríguez, A., Rey, B., Alcañiz, M. Assessment of the influence of navigation control and screen size on the sense of presence in virtual reality using EEG,
Expert Systems with Applications. Volume 41, Issue 4, Part 2. 2014. Pages 1584-1592,
ISSN 0957-4174. https://doi.org/10.1016/j.eswa.2013.08.055.
Eric Krokos, Amitabh Varshney. Quantifying VR cybersickness using EEG. Virtual Reality, 2021; DOI: 10.1007/s10055-021-00517-2
Gaetano Tieri, Giovanni Morone, Stefano Paolucci & Marco Iosa (2018) Virtual reality in cognitive and motor rehabilitation: facts, fiction and fallacies, Expert Review of Medical Devices, 15:2, 107-117, DOI: 10.1080/17434440.2018.1425613
Georgiev DD, Georgieva I, Gong Z, Nanjappan V, Georgiev GV. Virtual Reality for Neurorehabilitation and Cognitive Enhancement. Brain Sciences. 2021; 11(2):221. https://doi.org/10.3390/brainsci11020221
Jaiswal, N., Ray, W., & Slobounov, S. (2010). Encoding of visual-spatial information in working memory requires more cerebral efforts than retrieval: Evidence from an EEG and virtual reality study. Brain research, 1347, 80–89. https://doi.org/10.1016/j.brainres.2010.05.086
Keizer A, van Elburg A, Helms R, Dijkerman HC (2016) A Virtual Reality Full Body Illusion Improves Body Image Disturbance in Anorexia Nervosa. PLoS ONE 11(10): e0163921. https://doi.org/10.1371/journal.pone.0163921
Krokos, E., Plaisant, C. & Varshney, A. Virtual memory palaces: immersion aids recall. Virtual Reality 23, 1–15 (2019). https://doi.org/10.1007/s10055-018-0346-3
Mishra, S.; Kumar, A.; Padmanabhan, P.; Gulyás, B. Neurophysiological Correlates of Cognition as Revealed by Virtual Reality: Delving the Brain with a Synergistic Approach. Brain Sci. 2021, 11, 51. https://doi.org/10.3390/brainsci11010051
R. Moseley, “Deep immersion with Kasina: An exploration of meditation and concentration within virtual reality environments,” 2017 Computing Conference, 2017, pp. 523-529, doi: 10.1109/SAI.2017.8252146.
Mühlberger A, Jekel K, Probst T, Schecklmann M, Conzelmann A, Andreatta M, Rizzo AA, Pauli P, Romanos M. The Influence of Methylphenidate on Hyperactivity and Attention Deficits in Children With ADHD: A Virtual Classroom Test. J Atten Disord. 2020 Jan;24(2):277-289. doi: 10.1177/1087054716647480. Epub 2016 May 13. PMID: 27178061.
Safaryan, K., Mehta, M.R. Enhanced hippocampal theta rhythmicity and emergence of eta oscillation in virtual reality. Nat Neurosci 24, 1065–1070 (2021). https://doi.org/10.1038/s41593-021-00871-z
Wright WG (2014) Using virtual reality to augment perception, enhance sensorimotor adaptation, and change our minds. Front. Syst. Neurosci. 8:56. doi: 10.3389/fnsys.2014.00056
Yee, N., & Bailenson, J.N. (2006). Walk a mile in digital shoes: The impact of embodied perspective-taking on the reduction of negative stereotyping in immersive virtual environments. Proceedings of PRESENCE 2006: The 9th Annual International Workshop on Presence. August 24 – 26, Cleveland, Ohio, USA.
Cristina Gil López, PhD 