Our objective was to create a game similar to the iPhone/iPod Touch game 'Flight Control.' The player looks at a map of an airport and the surroundings. From time to time new aircraft appear on the map. The objective of the game is to land aircraft by drawing a route to a runway. The challenge is to avoid crashes between airplanes.
In our version of the game, the player can use both hands to control the game. The hands are captured by a webcam. The player should hold his hands on top of table. The player can see his or her own hands on the screen. The background of the webcam image is replaced by the map. The player can use one hand to 'grab' the map and scroll through the map. The other hand is used to 'grab' airplanes and draw a route to the runway.
Before the player can start the game he or she has to tell the program what the background (usually an empty table) is. We implemented our own thresholding algorithm to replace the background with the map in the game. The player can adjust some parameters to optimize the thresholding.
The hands are recognized by a blob-detection algorithm we implemented ourselves. The player should wear gloves with a distinctive colors. The player has to tell the program what the gloves are and when the hand is opened or 'grabbing' something.
We discovered that the program is very sensitive to differences in light. The program works best when using a tube light. The media framework we used to capture images (Java Media Framework) is very slow. It adds a delay of a second. This makes the game hard to control. Unfortunately we didn't have time to address these issues.