The neural basis of crossmodal attention presents a significant challenge to. Aug 23, 2012 crossmodal attention attending to a sensory modality. Exogenous orienting of crossmodal attention in 3d space. Applications of crossmodal relationships in interfaces for. Crossmodal attentionthe contribution of eventrelatedpotential. Crossmodal links between vision and touch in spatial. Nevertheless, recent research on multimodal issues already shows that crossmodal links in attention are substantial and numerous. Crossmodal links have been observed either when attention is directed voluntarily endogenous or involuntarily exogenous. Crossmodal interactions during affective picture processing vera ferrari1, serena mastria2, nicola bruno1 1department of neuroscience, university of parma, parma, italy, 2department of psychology, university of bologna, bologna, italy abstract natural crossmodal correspondences, such as the spontaneous tendency to associate high pitches with. By investigating ways to map from a tactile location on the body to an audio location in a. Degree of masters of science, 2008 zainab fatima institute of medical science, university of toronto abstract attentional cueing modulated neural processes differently depending on input modality. Applications of crossmodal relationships in interfaces for complex systems.
A standard crossmodal exogenous spatial cuing effect was observed, with. Selective attention appears to gate crossmodal coupling between sensory systems. Many studies have revealed that attention operates across different sensory modalities, to facilitate the selection of relevant information in the multimodal situations of everyday life. Conclusions the study of crossmodal selective attention is in its infancy, when compared with the vast body of previous at tention research that has only considered single modalities in isolation. Here we show that crossmodal processing in peripersonal space is reduced for perceptual events that another person acts upon. I abstract this thesis explores the design of crossmodal display, a new kind of displaybased interface that aims to help prevent information overload and support information pres. The final series of studies to be described underlines this general point.
Crossmodal space and crossmodal attention charles spence. Selective attention gates the interactive crossmodal coupling. Unimodal and crossmodal gradients of spatial attention. These can be used in interfaces as a means of nonvisual output. Request pdf crossmodal space and crossmodal attention many organisms possess multiple sensory systems, such as vision, hearing, touch, smell, and. These approaches can be used where crossmodal translation is required e. Separating targets from distractors in crossmodal space. There has been a rapid growth of interest in the study of crossmodal attentional capture in recent years, as more and more researchers have started to address the question of whether or not the. In recent years there has been dramatic progress in understanding how information from different sensory modalities gets integrated in order to construct useful representations of external space. Natural crossmodal correspondences, such as the spontaneous tendency to associate high pitches with high spatial locations, are often hypothesized to occur preattentively and independently of task instructions topdown attention. A study of temporal synchrony by wayne chi wei giang a thesis presented to the university of waterloo in fulfillment of the thesis requirement for the degree of master of applied science in systems design engineering waterloo, ontario, canada, 2011.
According to the crossmodal attention perspective, attention often occurs simultaneously through multiple sensory modalities. Multimodal and crossmodal representation learning from. Importantly, fieldindependent participants demonstrated a high efficiency for identifying the properties of audiovisual events and applying the crossmodal correspondence in crossmodal interaction. Cognitive styles differentiate crossmodal correspondences. However, crossmodal correspondences between various unisensory features. An eventrelated brain potential study of crossmodal links in spatial attention. Request pdf crossmodal space and crossmodal attention many organisms possess multiple sensory systems, such as vision, hearing, touch, smell, and taste. Comparing attended versus unattended visual stimuli when.
The spatial resolution of crossmodal attention purdue engineering. Thinking about the existence of crossmodal mental imagery in the various sensory modalities does, at the very least, seem to draw attention away from the controver. One limitation of previous research on the distribution of attention is that it has primarily considered only visual cueing of visual attention. Crossmodal space and crossmodal attention request pdf. Jan 03, 2018 crossmodal attention refers to the distribution of attention to different senses. However, combining information from different senses also poses many challenges for the. This research has emphatically shown that even early sensory processing within a single sense is modulated by information in, and attention towards, the other senses. Investigating the crossmodal spatial cuing of driver attention cristy ho, hong z. One possibility is that lipreading interacts with speech at a perceptual level, while reading interacts with speech at a decision stage. Attention and the crossmodal construction of space. Visual attention priming based on crossmodal expectations carlos beltr.
Preservation of crossmodal selective attention in healthy aging. Traditional studies of spatial attention consider only a single sensory modality at a time e. These benefits arise not only because each modality can sense different aspects of the environment, but also because different senses can respond jointly to the same external object or event, thus enriching the. A crossmodal icon is an abstract icon that can be instantiated in one of two equivalent forms auditory or tactile. Performing the task together reduced the crossmodal interference effect on tactile judgments, but only if the partner occupied the participants peripersonal space experiment 1 and if she responded to all, rather than only a subset, of the visual distractors experiment 2. The results suggest cognitive styles could differentiate crossmodal correspondences in crossmodal interaction.
Crossmodal interactions during affective picture processing. Crossmodal icons for information display abstract this paper describes a novel form of display using crossmodal output. Others actions reduce crossmodal integration in peripersonal. Abstractnatural crossmodal correspondences, such as the spontaneous tendency to associate high pitches with high. In addition, neurophysiological and neuroimaging methods have been combined with these psychophysical approaches to identify changes in neural activity that might underlie the cross. Experience with crossmodal statistics reduces the sensitivity for audiovisual temporal asynchrony. To date, most of the research on spatial attention has focused on probing peoples responses to stimuli presented in frontal space. These modalities process information from the different sensory fields, such as. The possession of such multiple ways of sensing the world offers many benefits. In daily life, however, our spatial attention often has to be coordinated across several modalities.
Crossmodal space and crossmodal attention oxford scholarship. Crossmodal space and crossmodal attention edited by charles spence and jon driver. Dolan 2001 crossmodal binding of fear in voice and face. One of the most fundamental questions in crossmodal attention research concerns the extent to which people can selectively direct their attention toward a particular sensory modality such as, for example, audition, at the expense of the processing of stimuli presented in the other modalities. Pdf in daily life, our attention has to be coordinated across several different sensory.
See also the multisensory perception of flavour by charles spence, in the september 2010 issue of the. The nonspatial task was designed to closely replicate the crossmodal attention demands of the spatial task, but with a design that did not use. Participants performed the task both alone and with a partner who responded to the visual distractors. The possession of multiple ways of sensing the world offers many benefits. Pdf attention and the crossmodal construction of space. Structure of human processing resources are there any costs of monitoring more than one sensory channel. The results suggest that the influence of rightwardbiased attention is more likely to be observed when the crossmodal signals interact at later stages of information processing and under.
Attentional modulation of eventrelated potentials elicited by crossmodal stimuli. Many organisms possess multiple sensory systems, such as vision. The crossmodal associations between sound and taste rachel guetta1, psyche loui1,2 1 department of psychology, wesleyan university, middletown, connecticut, united states of america. Over 10 million scientific documents at your fingertips. Oct 10, 2009 behavioral and eventrelated potential erp studies have shown that spatial attention is gradually distributed around the center of the attentional focus. Multisensory interplay reveals crossmodal influences on sensory. Many organisms possess multiple sensory systems, such as vision, hearing, touch, smell, and taste. The aim of the present study was to investigate exogenous crossmodal orienting of attention in threedimensional 3d space. Crossmodal perception, crossmodal integration and cross modal plasticity of the human brain are increasingly studied in neuroscience to gain a better understanding of the largescale and longterm properties of the brain.
This is a nontrivial problem, given that each modality initially codes space in entirely different ways. Participants performed a visualtactile interference task 6 in which spatially incongruent visual distractors in the peripersonal space are known to interfere with judging the location of a tactile stimulus 710. Synesthesia is a condition in which stimulation in one modality also gives rise to a perceptual experience in a second modality. The renewed interest that has emerged around the topic of crossmodal correspondences in recent years has demonstrated that crossmodal matchings and mappings exist between the majority of sensory dimensions, and across all combinations of sensory modalities. Over the past several years, that has been a growing interest in crossmodal attention i. Modulation of human visual cortex by crossmodal spatial attention. Investigating the crossmodal spatial cuing of driver attention.
Finally, crossmodal studies of attention seem a particularly fruitful topic for interdisciplinary, exchange, as recent developments in the neuroscience of spatial representation in the brain may be closely related to the psychology of crossmodal attention. A related research theme is the study of multisensory perception and multisensory integration. However, in the last few years, numerous examples of crossmodal interactions have been documented. The present study compared uni and crossmodal gradients of spatial attention to investigate whether the orienting of auditory and visual spatial attention is based on modality specific or supramodal representations of space. Most studies in which the orienting of attention has been examined in 3d space concerned either exogenous intramodal or endogenous crossmodal attention. Evidence for exogenous crossmodal orienting of attention in depth is lacking.
Audiovisual crossmodal cuing effects in front and rear space. Python code for the crossmodal retrieval system proposed at acm mm 10 in a new approach to crossmodal multimedia retrieval emanuetrecrossmodal. In the last five years, there has been a spate of studies on crossmodal attention. Crossmodal attention refers to the distribution of attention to different senses. Using a set of earcons and tactons as crossmodal icons, the work presented here investigates the use of spatial location as another parameter for use in mobile crossmodal auditory and tactile displays. Implications for the design of multimodal interfaces rob gray and rayka mohebbi arizona state university and hong z. Investigating the neural correlates of crossmodal facilitation as a result of attentional cueing. Attention is the cognitive process of selectively emphasizing and ignoring sensory stimuli. Visual attention priming based on crossmodal expectations. They are able to map from one modality to another and back, as well as representing them in a joint representation space.