Meeting ConGas Workgroup 1 Gualtiero Volpe InfoMus Lab – Laboratorio di Informatica Musicale DIST – University of Genova www.infomus.dist.unige.it [email protected] Marseille, November 27-28, 2003

Expressive Gesture • Gesture as “a movement of the body that contains information” (Kurtenbach and Hulteen, 1990) • In artistic contexts, gesture is often not intended to denote things or to support speech, but the information it contains is related to the affective/emotional domain.

1

Expressive Gesture • Expressive Gesture as gesture conveying information related to the emotional/affective sphere: – what Cowie et al. (2001) call “implicit messages”; – what Hashimoto (1997) calls “KANSEI” • We call such information expressive content

Expressive Gesture • Expressive content is different and usually independent from, even if often superimposed to, possible denotative meaning. • Example: gait analysis • Example: everyday actions (Pollick, 2001) • Example: same action performed in several ways, by stressing different qualities of movement

2

Expressive Gesture • Intended in a wide perspective: – expressive gesture in movement, music, visual media (e.g., computer animation) – gesture not directly related to body movement (e.g., gesture in visual media) • Carrier of a set of temporal/spatial features (expressive cues) responsible of conveying expressive content.

Objectives • To understand the role of expressive gesture in non-verbal communication: – which expressive cues are important – how they can be measured – how expressive cues are related to expressive content

• To develop novel interactive multimedia systems: – enabling novel interaction paradigms – allowing deeper engagement of the user

3

Expressive Gesture in Human Movement • Human full-body movement as a main nonverbal communication channel • Dance, the artistic expression of movement, as a testbed for studying expressive gesture mechanisms • Development of – a multimodal conceptual framework – a collection of algorithms for gesture processing – experiments for testing and validation

Conceptual Framework Concepts

High-level expressive information

Gestures

Motion segmentation and representation

Cues

E.g., energy, contraction, stability, kinematics

Physical Signals

Motion detection and tracking

4