Dancing Photo Fractals
I didn't set out to make photographic fractals that dance back at you.
It all started in 2001 or so, when Paul Kosek bought a portable GPS receiver. He had a plan: to read the location data into a laptop mounted to his car dashboard, estimate velocity, and play the appropriate roller coaster sounds, which he had liberated from a video game. We were both fresh out of college — Paul for engineering and me for programming — and I offered to help him learn enough Python and C to get it done.
During one of those sessions, we got to talking about complex and imaginary numbers. He told me the sine and cosine functions were merely slices of the complex exponential, \(e^{i \cdot t}\), and mentioned Euler's astonishing identity: \(e^{i \cdot \pi} + 1 = 0\). I had seen \(i\) in math classes, and used it to program the famous Mandelbrot fractal, but I hadn't given it much thought. Now I started wondering: what does it mean, geometrically, to have an imaginary exponent? Or to multiply or divide complex numbers?
My next Mandelbrot program was different: instead of running each pixel through N iterations and coloring them in place, I started with a small sampling of oversized pixels (particles?) and animated them across the screen with each iteration. To accomplish an iteration of \(z = z^2 + c\) (\(c\) being the initial \(z\) value/pixel coord), each pixel \(z\) (\(= x + y \cdot i\)) moved first from \(z\) to \(z^2\) (\( = x^2 - y^2 + x \cdot y \cdot i\)), then to \(z^2 + c\) (\(= (x^2 - y^2 + x) + (y + x \cdot y) \cdot i\)).
The particles flew around the screen, each completing its own fraction of a spiral, then making an idiosyncratic diagonal correction. The two-step motion looked goofy to me, and make me wonder if there is a continuous formula for the \(n\)th iteration of \(z\). I once saw a math professor derive a closed, continuous formula for the nth Fibonacci number (1, 1, 2, 3, 5, 8, ...) which is more commonly calculated by adding the previous two. Appropriately for this discussion, it was a quadratic formula of complex numbers, and had a non-imaginary value only for whole number inputs.
To this day, I wonder if there is an equivalent continuous Mandelbrot formula. Stymied, I crossed my fingers and found a pleasant surprise: For any two complex points, let's say \(z_1\) and \(z_2\), there's a (complex) exponent \(p\) for which \(z_1^p = z_2\). Rewriting slightly: \(z_1^{1+q} = z_2\). Considering that \(z_1^{1 + 0 \cdot q} = z_1^1 = z_1\), we can define a curve between \(z_1\) and \(z_2\) as \(f(r) = z_1^{1+r \cdot q}\), for \(r\) between 0 and 1 This is a totally arbitrary way of animating them, but it's at least simple enough to make the endpoints obvious, and pleasantly curved. I'd draw the classical Mandelbrot set in the background, and animate a few particles in the foreground.
For a while, I only noticed the first few iterations, when a cacophony of particles flock across the screen, and many of them fly entirely out of the picture. Later, after they settle, I'd see a remnant, each flying a regular schedule between a handful of distant points. I didn't give much thought to the rest of the particles, which seemed to become stationary across the interior of the Mandelbrot bulbs. Finally, I caught one of them coming to a rest, and realized it was making smaller and smaller triangles until they became imperceptible.
This inspired a new plan: draw just one particle, centered at its limit point, and draw all iterations \(z_1, z_2, z_3,\) ... connected by lines. It became clear that the Mandelbrot interior is filled with trajectories that spiral inward. I thought I'd just animate it by moving the camera down into the spiral. Then I started adjusting the initial point \(z_1\) and found that it morphed the trajectory curve smoothly. By generating random numbers to accelerate \(z_1\), and integrating the equations of motion, I had my first dancing fractal, albeit dancing to its own random beat. My wife was captivated, and suggested that it would be a fantastic toy for people dancing at live concerts if only the fractal could move with the music.
My friends were intrigued and wanted to play with it, but how to share it? I started with Windows apps, but half my friends had Macs. I made it into a screen saver, but power-save mode became the norm. Even when it was compatible, I'd have to convince people to install it. In desperation, I compiled it for the Raspberry Pi and salvaged some old TVs so it could have a dependable physical home. Then I read that web browsers were adding 2D and 3D graphics functions. Finally, there was a device-independent graphics platform with no installation required. I ported the dancing spiral, and was tickled to see that it even ran on my cell phone.
("Wow these things are faster than I thought! What else can they do?") How about drawing the classical Mandelbrot set in real time — iterating each pixel dozens of times to compute its final color. Awesome! Interactive zoom and pan, to the limits of single-precision floating point. The Julia set formula is almost the same as the Mandelbrot formula, only \(c\) is a global constant. I made it adjust \(c\) as you drag along the picture, and had a lot of fun morphing between nearby \(c\) values.
Learning web graphics, I found out they were adding all sorts of functions to the web browser. For example, I could read the phone's GPS, which immediately reminded me of Paul's roller coaster. They also added detailed audio playback functions, allowing me to adjust the "playback rate" of a media clip. The obvious one was Sammy Hagar's "I Can't Drive 55," with playback rate keyed to velocity so that it only sounds right when you're driving exactly 55.
Fun art, right? But how would you exhibit it? It's such a personal and vehicular experience. There was a technical issue as well: Apple hadn't yet implemented the playback rate function for iPhones. To reach iPeople, I made a sequencer (beat machine) with tempo controlled by velocity. I called it "The Beat Governor" because you could govern your speed just by listening. To make it gallery-friendly, I separated the sequencer and velocity reading functions. Velocity readings can come from a remote cell phone, relayed through the cloud.
Beyond GPS, cell phones have several sensors that can be read by web apps, including accelerometers and gyroscopes. The gyroscopes (a.k.a. "Device Orientation") are particularly easy and fun, giving the angles of rotation about three axes. Now I could adjust the \(c\) coordinate without blocking the screen with my finger, by adding an x,y displacement proportional to the change in orientation on two axes. I've since expanded "tilt" control to a choice of several variables.
Mobile web pages can also ask for video from the camera. I had read that the classical definition of "escaped" was somewhat arbitrary. That instead of testing whether a point has left the circle about the origin, you can use any shape. I'd even experimented with a rudimentary "paint" style program to draw different escape shapes, but the webcam took it to the next level. I just had to convert the picture to black and white — no gray — it's either escaped or it hasn't. Now I could incorporate details from nature and architecture (and Scrabble), and monochrome soon felt limiting.
Remembering the individual pixel trajectories, I took another leap of faith: what if we forget about escaping, and just pick up a little bit of color from every spot the trajectory visits? After \(n\) iterations, take the average visited color and apply it to the source pixel, and we get full-color Julia-style fractals. Connect local and/or remote gyroscopes and it's interactive.
It feels like a new kind of camera, but it's more accurately a new kind of darkroom. Even now, when most cameras are fully automatic, and the analog darkroom is almost extinct. A picture can be taken once, then developed into dozens of different animated artifacts. It's a fundamental shift in the nature of photography, much like when Photoshop inaugurated an era in which a picture is never set and finished, but can be repeatedly transformed into new discrete images. My new photographs are in constant motion as they respond to their surroundings.
It has often been remarked that an artwork's meaning is not fixed, but arises from the dialogue between artifact and audience, evolving with them. Here, we facilitate and dramatize that relationship, and give new agency to the viewer.