Touch Interaction with 3D Environments

Touch interaction is increasingly becoming an important means of human-computer interaction. However, often we have to deal with 3D data, visualizations, or scenes for which regular 2D input as derived from touch surfaces is not sufficient. Here we have to develop means of mapping the 2D input to interactions in 3D space, such that the touch exploration of the 3D environments remains intuitive.

The solutions to this 2D to 3D mapping problem differ depending on the objects or data with which one wants to interact. For example, information visualizations placed in a 3D scene can be interacted with by means of dedicated panels which need to allow both 3D interaction as well as manipulation of the data itself. In the other hand, some data such as 3D particle clouds does not provide large enough elements to serve as means to constrain the interaction, for such cases we need to use ways to interact with the data space itself, e.g., by means of an interaction frame placed around the data view. The following videos illustrate the different approaches to the 2D to 3D interaction mapping problem.