La Maschera dell’Anima was born as a collaboration with the talented visual artist Federico Gaggero and my long-known teammate Dario Mazzanti. It consists of a site-specific installation designed to be part of Cortilissima 2012, an exposition within a theater festival held in Turin. The work has been meant to physically and emotionally guide the audience from everyday life to the world of theater and actors, handling the complex transition from reality to fiction. Thanks to tracking and projection technologies, each member of the audience, while walking through the aisle leading to the stalls, is accompanied by an avatar, called maschera [“mask” in our beloved Italian], that follows all her/his movements. The closer the spectator gets to the entrance, the more detailed the maschera becomes. Eventually, a complete fusion with it occurs as soon as the entrance is reached; the audience “wears” the mask and enters the theater.
La Maschera dell’Anima
2012 - Interactive audio/visual installation
The installation becomes part of the physical structure of the aisle that connects the hall gate of the venue [Arena San Filippo, Turin] and the entrance to the theater built in the inner courtyard. On the opposite side of the main doorway, a large curtain works both as projection screen and entrance to the stalls. The sensor is placed on the side of the curtain, together with a hidden laptop and a couple of speakers. On the opposite side, just above the main doorway, the projector is attached to the ceiling, and draws the maschere each time a new spectator walks through the aisle.
Behind each maschera there is a 2D avatar, composed of several graphic elements attached onto a multi-joint rigged skeleton. Similarly to puppeteering, the data coming from the 3D tracking of the spectators are processed and adapted to animate the avatar’s skeleton. Since up to 6 users are discriminated by the sensor, each spectator is assigned a different set of elements; each set forms a maschera, a user-driven puppet that resembles the appearance of recurrent theater characters, but reinterpreted in a modern way.
In La Maschera dell’Anima, an audio/visual connection between the avatar and the audience is birth, grows, and reaches its final state when the curtain is crossed. This process mimics the actor that steps onto the stage, and capitalizes on 2 interactive paradigms. First of all, the set of elements that compose the maschera is ever-changing; initially, they just combine into a simple sticky man. When getting closer to the curtain, the avatar’s elements change, gaining details and showing animations. Furthermore, the detected body movements produce real-time sounds and music, whose mappings get more and more refined while approaching the entrance to the theater.
The sensor used in La Maschera dell’Anima is a Kinect; using a simple hack strategy, it streams raw data to a Mac OS X laptop via USB. A C++ application embeds the OpenNI library to acquire and process the sensor data, obtaining the skeletal information.
The projected scene is a Processing sketch. It combines the set of elements – images made in Photoshop and loaded into the sketch – with the tracking data coming from the parallel C++ application. The two applications communicate via OSC.
An Ableton Live Set wrapping a MaxForLive patch works as real-time audio engine, accessing the Live devices, parameters and clips through the LiveAPI’s embedded in MaxForLive. This engine is driven by the Processing sketch via OSC as well.
Teaser trailer for the festival installation: