Dissociative Fugue video

This was an attempt to visualize some of the abstract noise that goes on inside my head when listening to music. At first blush, it may not seem that different from a software visualizer. Unfortunately, visualizers are bound by the analysis of a stereo mix. While a it may extract information from certain frequency ranges, it can't really tell a clarinet from a duduk and represent each discretely. Here, each element has its own visual counterpart.

I read that some people lack the ability to distinguish the individual parts of a piece of music clearly. This blew my mind. Perhaps it will help others hear music in a different way, or grasp some of the more subtle interplay of the individual parts.



The youtube compressor makes hamburger out of this type of computer animation. If you're inclined, I've stashed a higher-quality version here.

Technical stuff about the video

I constructed the video with Adobe After Effects.

First, I calculated a frame rate to work at that aligns with the tempo of the project so beats will fall precisely on a frame.

Next, I exported all the individual audio tracks as guide tracks. Most of the parts have easily-detected transients. That way, I can see each note onset for each individual part in After Effects.

For each part, I created a simple composition to represent the travel and relative pitch of the notes. Using the audio guide track, I can match the travel of an abstract shape across the screen. I composited all the parts into one 3D composition. Four copies of this composition were nested inside another composition at right angles, creating a box.

For some additional elements, I created expressions based on the audio track to automate effects transformation. For example, the shakers track is using the audio amplitude to modulate opacity, scaling, and color. The guitar tracks at the end and the duduk track at the beginning also use this technique.

Finally, I choreographed the camera, taking care to align the movements to musically-sensible metric units.