Creating Interactive Poly-Artistic Works: The ConceptMove Project Olivier Delerue1, Stéphane Donikian2, and Gildas Clénet2 1

IRCAM, 1 place Igor Stravinsky, 75004 Paris [email protected] 2 IRISA/INRIA, Campus de Beaulieu, 35042 Rennes {donikian,gclenet}@irisa.fr

Abstract. Poly-artistic works often come up against two kinds of problems: heterogeneity of technologies on the one hand, and lack of abstraction to write the global structure of a piece on the other hand. To avoid the endless start from scratch approach, we present our project ConceptMove that addresses these issues. ConceptMove is a unified paradigm for describing interactive art pieces. This paradigm led us to code generation in order to automatically solve common issues. Last but not least, we propose an intuitive and user-friendly authoring tool that makes it possible for authors to easily write down their ideas. Keywords: Interactivity, poly-artistic works, unified paradigms, code generation, temporal constraints.

1 Introduction Interactive poly-artistic works is a type of expression becoming increasingly common nowadays. Consequently, users, specta(c)tors, expect more and more to play an active part in these works. Such works include for instance the Virtual Museum of Contemporary photography by Ahad Yari Rad, Topologies de l’instant or Seule avec loup by N+N Corsino, Schlag! by Roland Auzet, or Waves by Andrea Cera and Herve Robbe… Such creations always require the use of a wide range of technologies (3D video and audio display, video and audio synthesis, body tracking…): a large number of computer environments, software and frameworks have been created to fulfill these needs. These include for instance Max/MSP for audio processing, OpenMask [1] for 3D graphics, Life Forms [2] for the animation of human movements… Standards as well have been studied such as VRPN [3] or VRML for virtual realities as well as communication protocols between applications (OSC for instance). However, despite this important profusion in terms of technical tools, several issues remain unsolved when realizing such artistic works. First, in the context of collaborative arts, existing frameworks do not provide means for conceptualizing art pieces for contributors coming from different artistic areas (composition, choreography, video, 3D graphics…). Second, establishing communications between software or hardware components is often complicated. Finally, the communication process and its language have to be redefined from scratch for each new realization. M. Cavazza and S. Donikian (Eds.): ICVS 2007, LNCS 4871, pp. 211 – 215, 2007. © Springer-Verlag Berlin Heidelberg 2007

212

O. Delerue, S. Donikian, and G. Clénet

2 The Concept Move Project We address these issues in ConceptMove, a project funded by the French ANR research program RIAM. Our objective is to facilitate the realization of interactive poly-artistic art pieces in several ways. First, the communication process should be as transparent as possible to the users. Second, the artists should be in a position to use their favorite environments or software components in order to fulfill their ideas. And last, we wish to provide a symbolic shared area to specify the relationships between the different artistic worlds so that artists coming from various domains have a common language to start working on. Thus, the key idea is to describe poly-artistic interactive art pieces in a most generic way, regardless of the various software, hardware and environments that will be used to implement concretely the functionalities. Such description is represented in Fig.1 where the art piece has been formalized with temporal entities organized in time using the Allen’s relations.

Fig. 1. Simple sketch of an interactive art piece using Allen’s relations

In this simple sketch, the art piece is composed of six temporal objects (including the art piece overall container) and a number of temporal relations established between the temporal objects: when the piece starts, the Introduction is immediately started and followed right away by the central section composed of the tracking activity, the drummer and the video sequence linked together by « equals » relations. During this section the tracking module transmits listener position events to be mapped to the drummer module tempo parameter. The central section finishes exactly when the expected « interaction event » occurs. This leads to the Ending section witch finishes the piece. In the following section we show how this description is used in order to generate automatically computer code for implementation in the selected environments.

3 Transformations and Code Generation Once the overall description of the piece is achieved with the meta-language, including interactive specifications and temporal constraints, it is time to choose what environment or programming language the different people involved wish to use. As

Creating Interactive Poly-Artistic Works: The ConceptMove Project

213

this XML description will be the source of different plug-ins (currently implemented are Max/MSP, Pure Data, Listenspace, OpenMASK, Java), there is technically no limitation in the range of software one can use. Next development may for instance consider addressing OpenMusic, Flash, and Eyesweb [4]. This architecture can also communicate with existing material, assuming they already had an OSC communication: the generated code will automatically match the particular protocol.

Fig. 2. Overall transformation process

Fig. 3. Illustration of the generated code in several environments

The code generator (Fig. 2), with given IP address and port information, will then generate a low level interface for each requested environment. This interface will then be easily used at a high level in this environment (Fig. 3). For instance, a MAX/MSP developer will be able to establish a communication with a Java application, simply by using its send / receive common objects. This part, handling the communication, relies on an existing standard, WSDL, which allows describing the way an application communicates with the rest of the world. But there are also complementary temporal constraints that enable hierarchical and complex relations between temporal objects. This makes it possible for the code generator to create the “engine” of the art piece, e.g. a simple application that will schedule events in real time. This application will send start and stop events on time, but will also receive requests to start or stop temporal objects from the outside, enabling this way dynamic modification of temporal structures, through interaction for instance. Such constraint requests will of course be ignored if they turn to be inconsistent with written script of the piece. The written script is and stays the main reference and score for generated material and structural events.

214

O. Delerue, S. Donikian, and G. Clénet

4 Authoring Authoring remains an important issue since the whole description of the art piece has to be specified in an XML format, which is not usable directly by the artists. A prototype of authoring tool has been created and can be seen on Fig.4.



Fig. 4. View of the XML authoring tool (left) and equivalent XML code (right)

This tool is implemented as an eclipse plug-in and allows specifying the art piece from the most abstract level down to the implementation details. Once the piece is fully described, the authoring tool generates and equivalent XML description that will then be used for the transformation and code generation described previously.

5 Conclusion After analyzing the brake that slow down artistic creation in poly-artistic works, we developed, within the ConceptMove project, a meta-language with the aim of solving some of these problems. This language, a XML dialect, describes communications and time constraints of a piece, in a unified paradigm. Even if this approach may not be suitable for some situations needing fine grained interactivity (score following to

Creating Interactive Poly-Artistic Works: The ConceptMove Project

215

name one), it proposes a convenient formalism for lots of other situations like virtual and augmented realities, as well as interactive narration. Finally, after addressing the main issues through code generation, we proposed an intuitive graphical authoring tool that places the XML meta-language at a comprehensive level.

References 1. Margery, D., Arnaldi, B., Chauffaut, A., Donikian, S., Duval, T.: Openmask: Multi-threaded or modular animation and simulation kernel or kit: a general introduction. In: Richir, S., Richard, P., Taravel, B. (eds.) VRIC 2002, Laval, France (2002) 2. Landis, J., Chapman, J., Calvert, T.: New Directions in Dance. In: Notation of Dance with Computer Assistance, pp. 169–178. Pergamon Press, Oxford (1979) 3. Taylor, R.M., Hudson, T.C., Seeger, A., Weber, H., Juliano, J., Helser, A.T.: Vrpn: a device independent, network-transparent VR peripheral system. In: ACM Symposium on Virtual reality software and technology, pp. 55–61. ACM Press, New York (2001) 4. Camurri, A., Hashimoto, S., Ricchetti, M., Trocca, R., Suzuki, K., Volpe, G.: Eyesweb toward gesture and affect recognition in interactive dance and music systems. Computer Music Journal 24(1) (2000)