2010-2011-Touchless Earth

Touchless Earth Project

by Marcel Jillings and Sijmon Heitmeijer

The objective for Touchless Earth is to create an application which is able to interact with Google Earth using a webcam. The webcam is well known for video conferences but what if it’s possible to use the webcam as an input device?

 

Touchless Earth allows the user to control Google Earth completely with gestures. These gestures are made using four different markers. The markers are detected by using a separate color for each marker and are best placed on the index finger and the thumb of each hand. There are gestures to be made for navigation, zooming and rotating. Each gesture consists of a basic gesture, pinching. For the pinching gesture to work the user needs to pinch two fingers of the same hand together. Each gesture ends when both fingers are not pinched anymore.

For the navigation gesture only two fingers of one hand need to be pinched. The user can drag the world to a new position while dragging and pinching the fingers together. The rotation and zoom gesture requires both fingers on each hand to be pinched. The user is able to zoom the world in and out when both markers of each hand are pinched on the same line horizontally. Pinching and moving both hands closer together will zoom out on the world. Pinching and moving both hands farther apart will zoom in on the world. With both markers of each hand pinched above each other the application will be able to rotate the world in Google Earth. Rotating the pinched fingers clockwise will rotate the world to the right, whereas rotating the pinched fingers counter-clockwise will rotate the world to the left.

Touchless Earth is using the open source Touchless SDK for marker detection. The SDK is able to detect the color of each marker in every frame and with this information the center position of the marker. The detection of the markers works very well in surroundings where the light intensity hardly changes. If the light intensity changes, the colors of the markers are different (e.g. going from daylight to night darkens the colors) than the colors stored originally. If this difference is too much the markers will not be detected by the SDK anymore, requiring the user to add the markers again.