Performing in Flatland is a high tech-based performance inspired by the physics of sound and, in particular, by the theory of pressure propagation in 2D spaces. Loosely based on the satirical novel Flatland: A Romance of Many Dimensions by Edwin Abbott Abbott, Performing in Flatland depicts a two-dimensional world with no depth, solely populated by simple geometrical shapes. In this world, sound becomes a fundamental means to observe reality. In the absence of a third dimension, each shape is bound to a specific sound, a single tone that helps distinguish objects and perceive the otherwise unclear surrounding flat space. Throughout the piece, while these 2D shapes morph and overlap, state-of-the-art physical modelling computes the propagation of their intrinsic sounds in space; sound pressure is visualized as colored waves that slowly gain spectral complexity as they interact with the shapes, eventually turn into flat virtual instruments and transform the voices of Flatland into an evolving musical and visual performance.
Performing in Flatland has been designed and is performed by Arvind Vasudevan, Sidney Fels and myself. It was premiered as invited performance at the 2016 meeting of the Canadian Acoustical Association, hosted in Vancouver, BC.
On stage, two performers populate with shapes and sounds the initially empty 2D world of Flatland, which entirely resides on the computer screen on their desk. A graphics tablet is used to draw shapes directly on the screen, while a MIDI interface controls the sound radiating from special sound source pixels. A third performer uses a camera to shoot the development of the scene and the physical interaction with the interfaces, and feeds it to a big projected screen for the audience to see it.
The piece starts with the introduction of three main “characters” that work as three basic musical building blocks, each represented by a geometric feature [curved, angled and irregular line] and associated to a distinctive sound [sine, square wave and noise]. Through the drawing interface, the performers combine the characters, creating reflecting walls, resonant chambers and sound sources that blend their initial properties. The realistic interaction between these elements produces phase and spectral sound modifications; these are used by the performers as primary means of composition and are visualized on the screen as clear changes of waves’ trajectories and recursive motion patterns, generating a strong bimodal representation of the involved acoustic phenomena.
As the piece progresses, shapes and sounds become more complex. Different self-oscillating models are coupled with the main characters, excited by their sound and resonating in their geometries, to create and play 2D virtual instruments inspired by real aerophones. The piece then climaxes, when all the playing techniques explored so far in Flatland are layered on top of an EDM track that evolves for a few minutes. Before reverting back to silence/void, the performers directly act on the parameters of the underlying simulation to disrupt the physical laws that govern acoustics in Flatland, and produce sounds and visual wave patterns otherwise infeasible in reality.
Performing in Flatland is the first work of art designed using the technology developed in the DIGITAL_DUETS project.
The performance revolves around an audio/visual application that uses GPU parallel computing to simulate pressure wave propagation in real-time, both sonically and visually. Pressure and velocity values are calculated in the 2D domain using an FDTD implicit solver, implemented in GLSL. Each FDTD cell corresponds to a pixel of a dynamic texture, that is updated on the GPU and rendered on screen. The simulation runs at audio rate and is managed by a custom C++ audio engine that I built around the ALSA library; at each cycle, the pressure values of a chosen position on the texture are sampled and packed into a buffer, which is then sent to the audio interface to provide a continuous sound stream.
Both the solver and the bidirectional data exchange routines between GPU and CPU memory are highly optimized, allowing for relatively low latency control of simulation spanning over large domains, depending on the hardware employed. The core of the instrument used in the performance is a workstation running Ubuntu and equipped with a Nvidia GTX Titan X graphics card and an M-Audio Eight Tracks audio card [of which only ONE CHANNEL was actually used]. It also includes an interactive interface, composed of a Wacom graphics tablet, a Novation Launch Control XL MIDI controller and a simple GUI. For the premiere, Performing in Flatland ran over a domain of 148×108 cells at 44.1 kHz, with a buffer size of 128 samples [~4 ms latency]. These numbers allow for precise sound synthesis in space and a fairly detailed wave-front visualization; however, at such a resolution solid lines look very aliased. This gives quite a charming old-fashioned look to the drawing interface, similar to 70s/early 80s videogame graphics, dramatically contrasting the fact that the system is equipped with one of the most powerful GPUs available in 2016.
Basically this means sound over graphics, even when talking about GPUs!
We would like to thank Dr. Andrew Allen and Dr. Nikunj Raghuvanshi for their help and support. Without their contribution this work would not be possible.