Hybrid Touch/Tangible Spatial Selection in Augmented Reality

Description:

We study tangible touch tablets combined with Augmented Reality Head-Mounted Displays (AR-HMDs) to perform spatial 3D selections. We are primarily interested in the exploration of 3D unstructured datasets such as cloud points or volumetric datasets. AR-HMDs immerse users by showing datasets stereoscopically, and tablets provide a set of 2D exploration tools. Because AR-HMDs merge the visualization, interaction, and the users’ physical spaces, users can also use the tablets as tangible objects in their 3D space. Nonetheless, the tablets' touch displays provide their own visualization and interaction spaces, separated from those of the AR-HMD. This raises several research questions compared to traditional setups. In this paper, we theorize, discuss, and study different available mappings for manual spatial selections using a tangible tablet within an AR-HMD space. We then study the use of this tablet within a 3D AR environment, compared to its use with a 2D external screen.

Paper download: (29.6 MB)

Software:

The software is available at https://github.com/MickaelSereno/SciVis_Server/tree/CHI2020.

Study materials and data:

Our study materials and data can be found in the following OSF repositories: osf.io/pwauq, osf.io/rvpuc, and osf.io/qu634.

Videos:

presentation at EuroVis 2022:

Get the videos:

Pictures:

(these images as well as others in the paper are available under a CC-BY 4.0 license, see the license statement at the end of the paper)

Main Reference:

Mickael Sereno, Stéphane Gosset, Lonni Besançon, and Tobias Isenberg (2022) Hybrid Touch/Tangible Spatial Selection in Augmented Reality. Computer Graphics Forum, 41(3):403–415, June 2022.

BibTeX entry:



@ARTICLE{Sereno:2022:HTT,
  author      = {Mickael Sereno and St{\'e}phane Gosset and Lonni Besan{\c{c}}on and Tobias Isenberg},
  title       = {Hybrid Touch/Tangible Spatial Selection in Augmented Reality},
  journal     = {Computer Graphics Forum},
  year        = {2022},
  volume      = {41},
  number      = {3},
  month       = jun,
  pages       = {403--415},
  doi         = {10.1111/cgf.14550},
  doi_url     = {https://doi.org/10.1111/cgf.14550},
  oa_hal_url  = {https://hal.science/hal-03699232},
  osf_url     = {https://osf.io/qu634/},
  osf_url2    = {https://osf.io/pwauq},
  osf_url3    = {https://osf.io/rvpuc},
  github_url  = {https://github.com/MickaelSERENO/SciVis_Server/tree/CHI2020},
  github_url2 = {https://github.com/MickaelSERENO/TangibleBrushAR_GRSI},
  url         = {https://tobias.isenberg.cc/p/Sereno2022HTT},
  pdf         = {https://tobias.isenberg.cc/personal/papers/Sereno_2022_HTT.pdf},
}

Other References:

Stéphane Gosset, Mickael Sereno, Lonni Besançon, and Tobias Isenberg (2020) Tangible Volumetric Brushing in Augmented Reality. In Alex Endert, Steffen Koch, Christophe Hurter, Michael Sedlmair, Daniel Jönsson, and Filip Sadlo, eds., Posters at the IEEE Conference on Visualization (IEEE VIS, October 25–30, (virtually in) Salt Lake City, Utah). 2020. Extended abstract and poster.

BibTeX entry:



@INPROCEEDINGS{Gosset:2020:TVB,
  author      = {St{\'e}phane Gosset and Mickael Sereno and Lonni Besan{\c{c}}on and Tobias Isenberg},
  title       = {Tangible Volumetric Brushing in Augmented Reality},
  booktitle   = {Posters at the IEEE Conference on Visualization (IEEE VIS, October 25--30, (virtually in) Salt Lake City, Utah)},
  OPTeditor   = {Alex Endert and Steffen Koch and Christophe Hurter and Michael Sedlmair and Daniel J{\"o}nsson and Filip Sadlo},
  year        = {2020},
  oa_hal_url  = {https://hal.science/hal-02938756},
  url         = {https://tobias.isenberg.cc/p/Sereno2022HTT},
  pdf         = {https://tobias.isenberg.cc/personal/papers/Gosset_2020_TVB.pdf},
}

Mickaël Sereno (2021) Collaborative Data Exploration and Discussion with Augmented Reality Support. PhD thesis, Université Paris-Saclay, France, December 2021.

BibTeX entry:



@PHDTHESIS{Sereno:2021:CDE,
  author      = {Micka{\"e}l Sereno},
  title       = {Collaborative Data Exploration and Discussion with Augmented Reality Support},
  year        = {2021},
  school      = {Universit{\'e} Paris-Saclay},
  month       = dec,
  address     = {France},
  oa_hal_url  = {https://hal.science/tel-03550513},
  url2        = {https://tobias.isenberg.cc/VideosAndDemos/Sereno2022PSC},
  pdf         = {https://tobias.isenberg.cc/personal/papers_students/Sereno_2021_CDE.pdf},
}

This work was done at the AVIZ project group of Inria, France.