Norbert Herber  wave terrain synthesis, ambisonic diffusion, harmonic system design

Jeffrey A. Wolin & Zach Norman  plenoptic (light field) photography, anaglyphic video, dye sublimation & 3D prints

Andrew Lumsdaine & Georgi Chunev  camera and software engineering

This is a montage of the videos featured on three separate LCD screens in the installation. Light Field was allowed to play freely as visitors moved about the gallery to examine the videos. See the YouTube link below for our previsualization of the installation space. Headphones required; anaglyph 3D glasses (red-blue lenses) recommended.

This recording was captured directly from an optional binaural output. Headphones required.

Description

Light Field was created for Re-Imag(in)ing Science, an exhibition of fifteen teams of researchers in the arts and sciences from Indiana University, Bloomington. As a traveling exhibit, it contains a variety of projects by artists and scientists that illustrate scientific principles and articulate various methods of inquiry. Light Field was first exhibited as a collaborative installation in the fall of 2016 alongside the plenoptic (light field) photography, anaglyphic video, and 3D prints of Jeffrey A. Wolin and Zachary Norman. Additional technical and conceptual support was provided by Dr. Andrew Lumsdaine and Georgi Chunev.

As a sound work, Light Field extends from an examination of the technical and artistic potential of light field photography. Working with Wolin and Norman I was challenged to consider the idea of "solid space." A light field image not only shows what is in the frame. It documents an entire scene, including the space between the elements that make up the composed photograph. As with white space in visual design or silence in music, so-called "empty" material cannot be minimized or taken for granted. Plenoptic photographs show so much more than the subjects that appear within the frame. The camera captures the depth of an entire visual field and allows the artist to work with an entire scene rather than a solitary image.

The plenoptic image places viewers at the edge of a visual field and allows them to traverse its depth. Similarly, Light Field is meant to envelop listeners in orbiting planes of sound. Sounds are derived directly from the metadata of plenoptic images and diffused over an 8.1 Ambisonic system to convey the visual depth of the plenoptic photograph as an aural space. While ones listening attention reaches outward, the affect is like a gravitational force, creating both torque and motion along axes of rotation. The listener becomes the center of a sonic solar system where solid space is realized in washes of tone and timbre.

Technical

Light field cameras are unique due to their array of micro lenses. The Lytro Illum, for instance, has over 200,000 hexagonal lenses that capture the direction, color, and luminosity of millions of light rays. This means that the camera captures not only light and its character, but its origin as well. Access to this data allows the photographer to manipulate focus, aperture, and a variety of other parameters in postproduction. A scene can be completely reconstructed after the exposure has been made.

One such manipulation can be performed with metadata that is much like a topographic map. A Depth Map (see below) is a grayscale rendering of the space captured in a plenoptic image. It collapses the recorded depth of a shot into a 2D field where shades of white, black and gray signify the fore-, back-, and middle ground (respectively) of the image. Using a procedural algorithm for Wave Terrain synthesis, Light Field continuously generates its core sound by tracing trajectories over the depth maps of the images shown in the installation.

While the sound of Light Field is derived from the varying depths of each image, an actual sense of depth and space is rendered within the installation. An 8.1-speaker array uses the HoaLibrary (High Order Ambisonics Library) by CICM to spatialize the Wave Terrain sounds across ten unique depth planes. Using this same system, Light Field can also be realized for 2-channel, binaural playback.

Everything concerning the sound of Light Field is procedural and is the result of a deliberate artistic choice. While the plenoptic photographs comprising the visual portion of the installation are static, the data each image contains speaks to a dynamic space that would be ever-changing should you find yourself within the original scene. My collaborators did not describe "solid space" as "living space" but I imagine it that way. To traverse the plenoptic image is to step inside solid space as if it were actual, physical space. It is not and can never be this kind of space, but through processes of mediation it becomes its own space, as vital and real as if it were part of our physical reality.

—Norbert Herber, 2016

Previsualization

Rough, 3D fly-fhrough to help the gallery install crew hang our work.