Kinect-ing the Dots: How Artists Are Using Hacked Infrared Cameras to Stretch the Limits of Visual Art

A still from "Clouds," a documentary by James George and Jonathan Minard
(Courtesy the artists)

Media artist James George and photographer Alexander Porter want to turn photography into a desk job. Well, even more of a desk job. As digital cameras and ever-larger memory cards have negated the need to think about scarcity while shooting pictures, the pair’s work to combine digital SLR photography with the Microsoft Xbox 360’s Kinect infrared camera have made the medium even less immediate, turning Henri Cartier-Bresson’s decisive moment into a prolonged sifting process performed at a keyboard instead of behind a lens or in a darkroom.

The end results of this is a new hybrid form of photography, ghostly collections of data points that fragment the image and destabilize the role of the photographer. This new format is highlighted in “Wired Frames,” a group exhibition open today at Eyebeam


The details of George and Porter's process are fascinating. Instead of snapping a photo and awaiting the outcome on film or on a screen, they take their camera-combine out to collect data rather than visible light, gathering spatial information from the Kinect’s infrared sensor and color from the more traditional camera. Using a custom software kit called RGB+D (red, green, and blue, plus depth), the pair process the information they collect, rendering the Kinect data as three-dimensional clouds of dots and lines that can be “painted” with the SLR data, adding washes of relevant color.

For their “DepthEditorDebug” series, George and Porter took their setup to a subway station platform and came back with a series of images depicting wavering visualizations of human figures, architectural emptinesses, and subway cars turned into clouds of points. The images, also on display in “Wired Frames,” are all created from the same set of data — because of the depth sensing, the camera has no fixed viewpoint; through the software, the computer can parse the information and map relevant color onto depth from any angle.

Though it may be surprising to find a traditionally trained photographer dismantling his own medium, Porter has embraced the RGB+D format. The process of creating images changes, becoming a search for “photographic moments in the software,” explained Porter in a recent conversation at Eyebeam. “It doesn’t ruin the essential things for me.” Through the software, the serendipity of photography persists, with some added twists: There are new variables to play with, like altering the way color is mapped onto spatial points or how the depth points are distributed. “The impulse behind it is the same,” Porter concluded.

George, who is currently an artist in residence at Eyebeam, identifies with the New Aesthetic label that recently went viral with novelist and futurist Bruce Sterling's Wired essay, but worries about making work that is simply “about technology,” rather than using the technology to make innovative art. With this work, they are “appropriating technology made for machine consumption for expressive capability,” he said, using the tools “in ways not intended by the people who made them.”

The other artists in “Wired Frames” are equally adept at twisting technology to their own creative ends. Kyle McDonald, who is famous for hacking into a Mac store full of computers and hijacking their webcams, collaborated with Arturo Castro to create “Faces,” an installation that uses a webcam and McDonald’s FaceOSC software to track viewers’ facial movements and overlay a real-time virtual mask of a different face on top their own. For the exhibition, McDonald has also curated a selection of projects that other people have created using FaceOSC, from artists to developers and hobbyists.

By including these other projects, “Wired Frames” exposes the greater community behind technology-based art and underlines the participating artists’ status not just as makers of objects, images, and videos, but creators of public tools and resources and members of an extremely active online community. Both RGB+D and FaceOSC are open source, meaning that anyone can use or adapt the software without licensing restrictions. One issue that technology-based art confronts is that the idea of the single author is increasingly irrelevant — all work is based on the machines, tools, and software already created by other artists and developers. Unlike the contemporary art norm, the key here is to acknowledge your immediate peers and influences properly, and participate in the community by keeping your own creations accessible and open.

“There’s no conflict of interest there, it’s only supportive,” said George. According to the artist, a creation like RGB+D might not be an art object in itself, but it is nevertheless “validated when work from other people comes out of it… There’s [original] work made with it, but it also enables other people.”

George and documentarian Jonathan Minard further trace the outlines of their artistic community in a new documentary, “Clouds,” filmed with Kinect and SLR, which will make its debut in the Eyebeam exhibition. Advancing the aesthetic of the earlier “DepthEditorDebug” images, the film is made up of interviews of new media luminaries including artist and professor Golan Levin, curator and critic Regine Debatty, and computational designer Karsten Schmidt, with each video portrait composed of colored point clouds that whirl and zoom as the figures speak. Their dialogues, ranging from musings on the future of data to explanations of the Kinect technology, are fascinating, but the effects created by the artists heighten what would be a straight talking-head documentary into a computational art piece, a new media philosophy made visual.

“Clouds” looks like the future, with its swirling data points and polygonated, holographic gurus of digital life. But what’s most surprising about the work is that it’s heavily invested in the present tense — not some distant, utopian future, but our own near future, the coming visual aesthetic. “We’re using the language of our time to talk about our time,” one of the film’s subjects, artist Sonia Yuditskaya, pronounces. Her face is an etched series of data points against a blank black ground.