Application screenshot - click to view full size
This web-page documents work for the MAT 240D Fall 2005 course.
Application screenshot - click to view full size
My final project was an application to perform granular processing of audio files, under the control of a visualised grain scheduler with a user interface for scheduling parameters.
The core of audio generation is performed in the Granulator and GrainMaker classes (granulator.h & .cpp).
Each individual grain is represented by a grain struct, defined as follows:
typedef struct
{
// scheduling parameters - to be moved to a superclass (at a future date)
int id; // maybe unnecessary, but helps in debugging!
long time; // timestamp of the grain's onset in samples
long duration; // duration of grain in samples
// granular specific parameters - to remain in a subclass (at a future date)
double amp; // 'loudness'
double pos; // stereo pan position
double slope; // slope of grain triangle, where 0 = zero attack, 1 = zero decay
double phase; // phase of the grain event
double pincr; // phase increment, i.e. rate of playback
long onset; // onset time in the delay buffer
} grain;
The grains exist within a pre-allocated array in the Granulator (currently fixed at 400 grains in length). However for efficient management of grain scheduling, the Granulator also contains two STL lists of grain pointers, for active and inactive grains respectively. The granulator also contains an audio buffer (mRingBuffer, currently mono and just under 6 seconds long) as the source sound to be granulated.
During the audio callback, the Granular::nextBuffer() method zeroes the output buffer and then iterates through the active grain list (mActiveGrains), checking each grain's time and duration properties to ascertain the appropriate samples in the current block to calculate (if any). For those samples to be calculated, the granulator performs the appropriate scaling, interpolation and enveloping of the ring buffer according to the grain parameters, and updates the grain's phase property accordingly. Finally, the nextBuffer method copies a block of audio from the source file to the internal ring buffer, updating the ring buffer's write cursor (mRingCursor) and internal sample clock (mNow) accordingly.
The Granulator's audio callback and buffer management comes from the CREATE Signal Library (CSL 4 alpha), which is currently in development at CREATE, UCSB. For more information please see www.create.ucsb.edu/CSL.
The GrainMaker class contains a pointer to the Granulator, and thus can reference the Granulators mGrains array and mActiveGrains and mDeadGrains lists. Grains are scheduled (moved from the mDeadGrains to the mActiveGrains lists with changes of parameters) in the GrainMaker's scheduleGrains() method. This method first iterates through the mActiveGrains list and removes any grains whose time+duration is before the Granulator's current notion of time (mNow). Then the GrainMaker attempts to refill the mActiveGrains list up to the required number of active grains by taking grains from the mDeadGrains list, adjusting the time (relative to the Granulator's mNow), duration and other parameters according to the current user settings, and adding them to the mActiveGrains list.
The user settings for the GrainMaker (e.g. number of grains, grain duration, stereo witdth etc) are retrieved and set using the getParameter() and setParameter() methods, which scales values from 0..1 to an appropriate range. The scheduleGrains method applies some degree of randomisation from these parameters to individual grains as appropriate. The parameters include:
long mDensity; // no. of grains to schedule at once, 0-400
double mDiffusion; // no. of samples to schedule ahead by, 0-10s
double mDuration; // grain duration in samples, 0-1s
double mAmp; // global amplitude control, 0-100%
double mWidth; // stereo pan width, 0-100%
double mRate; // grain playback rate, fixed at 1 currently
double mDelay; // no. of samples back in history to copy from, 0-buffer size
double mFeedback; // amount of granular output fed back into ring buffer, fixed at 0 currently
double mFadeTime; // for cheap triangle envelope, 0% = no attack, 100% = no decay
long mQTime; // quantise time size, 0-2s
double mQTimeAmount; // quantise time amount, 0-100%
long mQDelay; // quantise delay size, 0-buffer size
double mQDelayAmount; // quantise delay amount, 0-100%
Labelled screenshot - click to view full size
I was keen to develop a graphical representation and interface for the granulator as part of a larger effort to produce real-time graphical interfaces for composition and performance, however at this stage the project's interface is just a prototype. The main region of the window displays a somewhat typical horizontal 'timeline' view of the scheduled grains. Each grain in the mActiveGrains list is represented by a triangle where horizontal length relates to duration, colour relates to stereo position, transparency relates to amplitude, and shape represents the shape of the grain envelope. Horizontal position indicates the time that the grain is scheduled to be sonified, and vertical position indicates the delay into the ring buffer (mRingBuffer) from which the grain will draw audio to granulate. The contents of the ring buffer itself are displayed in an aligned vertical waveform view to the right of this main view. The current audio block is represented by the vertical dark grey line (whose width relates to the block size), and the quantisation grids of grain time and buffer delay are represented by the vertical and horizontal lines respectively, where transparency relates to the degree of quantisation applied.
Beneath the main view is a slider for each of the GrainMaker's parameters (currently unlabelled), and a preset crossfader. I stored 10 presets in the application for demonstration purposes, selected using the number keys 0 through 9, and the crossfader can be used to interpolate the parameter settings between the current and previously selected preset.
The graphical interface was built using an OpenGL based GUI library that I recently began to build, and which will be continued in a joint project with similar work simultaneously being done by Lance Putnam & Eric Newman at MAT. In the source folder, the graphics library can be found in the /glvu folder. The graphics library hierarchy is as follows:
The 'main' appliaction code therefore looks as follows (Test_glv.cpp):
#include <stdlib.h>
#include <GLUT/glut.h> // GLUT & Open GL
#include <OpenGL/OpenGL.h>
#include "glv.h"
#include "sliderView.h"
#include "cslWindow.h"
namespace glv {}
using namespace glv;
int main(int argc, char** argv)
{
GLV_INIT; // REQUIRED first call for GL, also resets timer
cslWindow * window = new cslWindow("G R A N U L A T O R",10,10,800,490);
window->run();
return EXIT_SUCCESS; // quit politely - actually just for the compiler
}
The cslWindow constructor builds the CSL graph and sets all application parameters accordingly. The inherited Window constructor registers this window instance as the target of the static window pointer in the Window class (MAIN_WINDOW), and registers the GLUT callbacks to the static window callback methods. Each callback method in turn then calls the static main window pointer's target methods, which in turn call all appropriate subview methods cascading down to all gui elements, evident for example in the static draw callback below:
void Window::CallBackDisplayFunc(void)
{
frameDuration = glutGet(GLUT_ELAPSED_TIME) - frameTimeStamp;
frameTimeStamp += frameDuration;
// clear the OpenGL buffers
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
MAIN_WINDOW->drawFrame(); // draw it's background, border etc
MAIN_WINDOW->draw(); // draw it's own graphics routine
MAIN_WINDOW->drawViews(); // draw all it's child nodes (recursive)
// FINALLY swap back buffer to the screen (double-buffering)
glutSwapBuffers();
}
Quicktime capture at half resolution (please excuse the poor capture quality - it was the best my aging computer could manage).
I chose a well-recorded, dry and recognisable audio source to granulate (voice and acoustic guitar) in order that the effects of granulation are more apparent. For the curious, the source is Nick Drake, "Black Eyed Dog", and comes highly recommended.
Currently a large part of the implementation of feedback granulation (mixing the grain output stream with the input stream when writing back into the ring buffer) and variable grain playback rate is complete, but not incorporated into the GUI. Of course, textual lablelling of the GUI elements is yet to be completed.
The GrainMaker functionality was separated out from the main Granulator functionality in order that it would be possible to have several schedulers with different parameter sets acting upon the same Granulator resource, however it remains to implement the framework and GUI for mulitple GrainMakers.
At present the scheduler view simply loops, and no history of expired grains is maintained, but it would be reasonably simple to enable such functionality, bringing a score-like potential to the application. I am keen to take further advantage of the graphical representation, for example giving the user a variety of 'drawing' tools in order that grains can be scheduled by drawing into the scheduler canvas.
The quantising grids are a powerful feature of the application but currently act globally for all grains across all parameter ranges - it would be interesting to be able to specifiy not only different quantisation grids and shapes for different GrainMaker parameter sets, but also for different regions of parameters (e.g. relating time quantisation to stereo position, or delay quantisation to amplitude etc.), in order to lend more complex coherence to the grain streams. Furthermore incorporating this functionality to a storable score-like data structure would be a powerful addition.
Finally, in the longer term I am considering methods of specifying grain parameters and grain scheduler parameters according to inheritable hierarchies, such that grain 'gestures' made up of many grains can be globally or locally transformed, and various avenues can be explored rapidly and compared, for example.
Besides the granulator engine and gui controller, the code in the archive includes a beta version of CSL 4, and an extremely alpha version of an OpenGL based gui library that will be being developed at MAT by Lance Putnam, Eric Newman and myself over the coming months.
Note that for copyright reasons no audio source file is included in the archive. If compiling from the Xcode 2.1 project included, insert an audio file of your choosing (renamed to "source.aif") into the main folder before running.
For the 240D course I also learned the Max/MSP SDK and built several audio synthesis and processing objects for Max/MSP, including a wavetable oscillator (with a Max/MSP gui for drawing waveforms) with linear and cubic interpolation, an interpolating decimator, a filter to compare interpolated filter coefficients with interpolated filter output (the conclusion being that there is no difference, clear in retrospect since the calculations are linear), an attempt at a waveguide mesh, and a basic granulator that formed the origin of my main project. Some of these externals may become part of a library of Max/MSP externals I hope to release to the general public (with source) in the near future.