December 27, 2017
I recently read Stephen Boyer’s post titled “My unusual hobby” and resonated with much of the expressed thought - the pleasure of programming as a recreational activity. As I grow up as a software engineer, I’ve observed for most of the fellow engineers that programming appears like a stressful activity and writing code that barely anyone cares much about doesn’t really fit the general idea of having fun.
I’m another one who loves writing code for a variety of hobby projects. These aren’t projects aiming to make world a better place or anything, but just trying to recreate or create something that fascinates me. One class of such hacks is creating audio visualizations.
There is history to why I derive fun out of things moving with sound. The first set of audio visualizations I worked on were part of an audio player I wrote a while ago in Visual Basic. That was an attempt to recreate something like Winamp back in the day, and I have to say it’s still one of my most memorable projects ever. I did not understand much about sine waves, audio frequency response, and everything else but those were the days of scavenging the internet for code snippets, trying to understand them enough, and tweak them to make them work. That part though, is still much the same.
Although, I attempted to create audio visualizations a bunch of times in recent years but almost always the lack of concept of an apt application to contain my visualizations has shot the motivation down. Reading about recent WebAudio developments gave me some ideas. When a powerful platform is present everywhere, it gives it’s developers great power when they think of building an application. Friction produced by worrying about the platform capabilities and distributing the applications is gone. That’s what I love about web as a platform.
Noisetab is the project that got me back into the visualizations game again. The fact that I could use WebAudio to build visualizations meant I could build web apps or browser extensions around audio visualizations. Noisetab is a Chrome extension with two basic audio visualizations - spectrum analyzer and the frequency domain oscilloscope. I could not contain my itch to build something different so over the next few days, I worked on another visualization which I called circular. This might look like a variation.
The code can pretty much explain how Circular is rendered. In the scope of this post, I’ll set the context on how audio parameters affect the rendering. In the visualization, you’ll notice a bunch of things reacting to the audio response - length and color of the bars along the circle and the radius of the circle. The length of each bar along the circle changes proportionally with the audio response at a particular frequency and red channel of the color changes similarly. The radius of the circle is the average of audio response over a range of low frequencies. That was the closest I could get to very basic beat detection.
Circular turned out great. This was when I decided I’m going to try out 3D next. This was also an opportune moment to try out 3D stuff on web. Three.js came out as a nice way to try this out as you don’t need to understand OpenGL/WebGL to use the GPU for playing with the pixel geometry and the colors. The effect I wanted was like particles being pushed out of an energetic core, which is as energetic as the average of audio responses at certain frequencies. In my head, I must have visualized a supernova explosion.
Since emission is a particle system, there are bunch of parameters which are based on the audio input - particle velocity, spawn rate, color, etc. All of them changing proportionally with the average of audio response over a range of low frequencies. To get a better understanding, these parameters can be played around with in the three.js example of a particle system here. This is the example project which I forked, and tweaked to make those parameters react to the audio response.