Out of Reach
2D interactive project with the Kinect
Sara Fisher
This art piece aims to create an interactive, illusion of depth with the use of the Kinect and 2D artwork.
Abstract:
The intent of this project is to give the viewer a sense of discovery. The viewer interacts with the animations displayed on a monitor, creating a disconnect between the flat image and the space they are engaging with. Depending on the position of the viewer, they are able to see different aspects of the art piece, little secrets hidden under layers.
Using the Kinect to track the motion of the viewer, the layers of the piece will move and change in scale, mimicking the effect of 3D art and creating the illusion of depth. The project is focused on the sunken city of Baia, which was chosen to create a visual connection with the concept of a place that you would like to interact with, but that is out of physical reach. The ocean lends itself to an exploration of many layers, secrets hidden in its depths.
Background:
When developing the concept of this project, I was interested in how physical data of a viewer could be used to create in interactive experience in an art piece using TouchDesigner. In my research for how to accomplish that, I had considered using FaceOSC, however, the Kinect was able to give much more reliable data that was easy to implement into TouchDesigner. I researched examples of different parallax images, to see how others had used multiple flat layers to create an illusion of depth.
For the concept of this project, I was inspired by the work of Markin Tomsky, who uses layers of laser-cut wood to create a 3D effect with 2D layers, and Dustin Yellin, who creates 3D art with 2D images by encasing them in resin. I also, researched examples of parallax in web design, such as the project Jess&Russ, which uses scrolling to prompt movement of layers.
Project execution:
The setup of the project requires a Kinect to be placed on a tripod and set behind a computer. While the Kinect is detecting the viewer’s motion in relationship to itself, since the viewer will be directly interacting with the computer monitor, it will give the illusion of their motions being related to their interaction to the screen.
In TouchDesigner, the Kinect is linked to the project with the Kinect CHOP, and then using the Select CHOP, I extracted the data from the Kinect that was relevant to the user’s head movement:
head:depthu (side to side)
head:depthv (up/down)
head:tz (foreward/backward motion)
This project relied on the Math, Limit, and Lag CHOPs to adjust the number that was given by the Kinect.
The Math Chop was helpful both to multiply the number, and also to set the range of the calculation. By adjusting the range of each image, the layers would have a different field of motion depending on how far into the background they were intended to be.
The Limit CHOP kept the image from stretching/shrinking to the point of looking unnatural.
The Lag CHOP was able to create an illusion of depth by creating a subtle difference in the motions between layers. This worked most effectively on images that were moving from the foreground to the background, particularly when combined with opacity layers.
With the calculations made from the CHOPs, I was then able to adjust the opacity, scale, and position of each layer based on the distance of the viewer from the screen. This was accomplished by feeding the number into the Transform TOP, to adjust the Scale and Pivot, and a Level TOP for the opacity.
Different CHOPs and TOPs were used for each layer of the image, so that each could be adjusted based on where it was placed in the image and how far into the background the image was intended to be.
Results and conclusion:
The project was able to create a sense of depth and interactivity for an individual user, but does not have that impact when viewed as a video, or from the perspective of someone watching another viewer interact with the Kinect. I would like to develop this concept further by considering the experience of an outsider. Not necessarily to make the effect work for multiple users, but to allow outside perspectives to have their own experience with discovering the work. I am interested in the relationship built between the user and the piece when it is directly reacting to their perspective.
