This post is an introduction to a 3-episode tutorial that explains how to implement real-time physical modelling synthesis on the GPU, using OpenGL shaders.
It is part of a full paper titled “Shader-based Physical Modelling for the Design of Massive Digital Musical Instruments“, by Victor Zappi [me], Andrew Allen and Sidney Fels, that will be presented in May at the international conference on New Interfaces for Musical Expression [NIME] 2017.
Here is the video abstract of the paper:
The example chosen for the tutorial is a simple real-time simulation of a drum head hit by a series of regular strokes.
The used physical model is based on a quite common Finite-Difference Time-Domain [FDTD] solver, but implemented directly within the graphics pipeline, as a GLSL shader program. This allows to scale up the model to a massive size and permits the integration of real-time controls, a fundamental feature for DMI design and quite unlikely in typical physical models based on numerical analysis [like the FDTD]. The related NIME paper is really the best way to fully understand what we are talking about.
The application also includes a code infrastructure written in C/C++ that initializes the simulation and manages the step-advancement of the solver. It is very simple, despite all the OpenGL functions with scary names. Actually, I use C++ for my personal convenience only, the whole program can be easily turned into a pure C project, since no strict C++ features are used.
When we run the application, a virtual drum head is automatically excited for 5 seconds, after which the simulation automatically ends. It includes a simple 2D visualization of the excited surface, but audio is not sent to the audio card. The samples are simply saved in the buffer to do as you please. Indeed, the structure of the code mimics a buffer-based audio card callback setup, which makes it easy to port it into any C++ real-time audio programming environment [e.g., Alsa, PortAudio]. The entire code is composed of 9 files, including headers. The overall simplicity and “general purpose-ness”of this example aims at clarifying the approach that we propose, in the hope that people out there can build their own terrific and amazingly complex DMIs.
The code has been tested on several Linux machines [all Ubuntu 14.04/16.04 LTS, equipped with different Nvidia GPUs], but it *should* run on Windows as well, since extensive testing has been done before for another project.
Unfortunately for Mac users, Apple has its own OpenGL implementation, that is a bit different/weird/outdated, you name it, and so far I could not be bothered to port the code.
Regardless of the OS flavor of choice, the only strict requirement is a graphics card that supports OpenGL 4.5 [maybe 4 works too, let me know!].
The specs of the used machine are not very relevant, since most of the work is gladly done by the GPU.
Here is how the full tutorial is structured [I’m gonna get sued, I know…]:
- Episode 1: Simulation Initialization
I’ll start from a brief introduction to the physical model [FDTD] and how we need set up an OpenGL context that matches its requirements. This part will cover C++ files only:
- [old friend] main.cpp
- Episode 2: Simulation Cycle and Shader Programs
Here I will show the actual rendering loop [again in the main], then I will go through the source of the 2 GLSL programs that actually implement the GPU FDTD solver and visualize the result.
The programs are composed of the following files [two couples of vertex and fragment shaders]:
- Episode 3: Generic OpenGL Stuff and Compilation [in progress]
Finally, I will briefly introduce the source of the support functions we used in Episode 1 [from the header file] to properly set up the OpenGL context in C++.
I will also share a step-by-step guide on how to install and run everything in the least painful way possible [on my friend Linux].
In case you are super excited about the idea of finally synthesizing real-time sound using your GPU [best usage of such an expensive piece of hardware IMO]
you’re a bit rusty in OpenGL, graphic pipeline and non-sensical C-like syntax [that was me when I started],
I strongly recommend to skim through this set of amazing C-OpenGL tutorials exquisitely crafted by Dr. Anton Gerdelan. Trying to compile one of these much simpler examples is also a much less stressing starting point.
If you’re a master already,
[feel free to comment/contact me if you spot any errror, or have questions, doubts or feedback (: ][/vc_column_text][/vc_column][/vc_row]