Combining sensory cues for spatial updating: The minimal sensory context to enhance mental rotations Manuel Vidal , Alexandre Lehmann and Heinrich H. Bülthoff 1, 2
2
1
1 MPI
for Biological Cybernetics, Tübingen, Germany 2 LPPA, CNRS / Collège de France, Paris, France
Introduction Mental rotation is the capacity to predict the outcome of spatial relationships after a change in viewpoint. These changes arise either from the rotation of the test object/array [1] or from the rotation of the observer [2]. Previous studies showed that the cognitive cost of mental rotations is reduced if viewpoint
changes result from the observer's motion [3, 4]. This finding was explained by a spatial updating mechanism that is involved during self-motion. However, little is known about the minimal requirements of such process, namely how the various sensory cues available during self-motion combine to improve the updating performance.
Is the contribution of these sensory cues additive? Some studies claim that extra-retinal cues are necessary for the updating [5, 6] though it was not clearly assessed. One study reported a cost reduction in RT with only visual stimulation [7]. Could layout-motion cues also improve mental rotations, without involving extra-retinal cues?
General methods Conditions
12 naïve subjects for each of the 3 experiments.
Visual rotations (room environment)
Task description:
Test object layout (touch screen)
Sound rotations (church bell)
Motion sensory cue
Test Rotated view (mental rotation)
Virtual room
Virtual room
Darkness
Darkness
Same view (control) Virtual room
None (Exp. 1)
Darkness
Body (Exp. 2)
al
al
Virtual room
rtu
Vi
rtu
Vi
m
o ro
m
o ro
The test layout with 5 objects ( ) was presented for 3s, and then disappeared for 7s. In the hidden phase, the table with the objects and/or the subjects was rotated of 50º in one of the five self-motion sensory contexts: None, Body, Vision, Body & Audition, Body & Sound. The task was to touch which of the 5 objects had moved (random translation of 5cm).
Observer position
Unchanged
Participants:
Apparatus:
Learning
Vision (Exp. 3)
· Blocks of 5 trials with the same condition · Nested Latin Square (block order, condition, subject) · 20 repetitions for each condition
Changed
Protocol:
Darkness
Darkness
Virtual room
Virtual room
Darkness
Body & Audition (Exp. 2)
Data analyzed: Body rotations (motion platform)
In a given sensory context, the cost of the mental rotation is defined as the difference between the same view and the rotated view performance.
Virtual room
Body & Vision (Exp. 1)
Results and discussion A) Sensory combinations:
General discussion:
40%
When only one modality is stimulated during the observer change in viewpoint (Body or Vision) there is no significant improve. In turn, when two modalities are consistently co-activated (Body & Audition or Body & Vision), there is a significant cost reduction. We conclude that the richer the sensory context is the more mental rotation performance improves.
Cost of the mental rotation
35% 30%
20%
References: [1] Shepard, R. N. & Metzler, J. (1971), Science, 171, 701-703 [2] Diwadkar, V. A. & McNamara, T. P. (1997), Psychological Science, 8, 302-307 [3] Simons, D. J. & Wang, R. F. (1998), Psychological Science, 9, 315-320
N.S.
15%
t(22) = 2.81, p < 0.01
10% 5%
t(11) = 2.89, p < 0.02
0% None
Body
Vision
Body & Audition Body & Vision
Motion sensory cues
40%
B) Continuous external rotation cue:
35%
Cost of the mental rotation
Enhancing the spatial binding possibilities by displaying the table texture rotating during the hidden phase was sufficient to significantly reduce the mental rotation cost without observer motion (only the layout rotates). It provided a better cue than the handle in [4], possibly enabling the spatial updating with an external cue.
N.S.
25%
30% 25% 20% 15%
t(22) = 2.13; p < 0.05
10%
N.S.
5% 0% Unchanged
Unchanged with table cue
Changed
Observer position
[4] Wang, R. X. F. & Simons, D. J. (1999), Cognition, 70, 191-210 [5] Simons, D. J., Wang, R. X. F., & Roddenberry, D. (2002), Perception & Psychophysics, 64, 521-530 [6] Christou, C. & Bülthoff, H. H. (1999), MPI Technical Report, No. 75 [7] Burgess, N., Spiers, H. J., & Paleologou, E. (2004), Cognition, 94,
Our findings suggest a sensory-independent triggering of the spatial updating mechanism during self-motion, with non-linear additive effects when sensory modalities are co-activated. The fact that significance was not reached with a single modality should not be interpreted as not contributing. More natural unimodal cues than that of our setup or more subjects could in fact reveal a significant cost reduction. This multisensory approach allows gathering the very debated differences reported in the mental rotation literature. It also provides a new interpretation of the allocentric cue card influence introduced in [7]. In our view, stable cues are needed for egocentric motion perception in the visual and auditory modalities, and therefore both the cue card and the body motion-cues in [7] just contribute in an additive fashion to improve the spatial updating performance.
Poster presented at the 9th International Multisensory Research Forum Hamburg 2008
