Challenges of Intracellular Visualization using Virtual and Augmented Reality

Description:

Microscopy image observation is commonly performed on 2D screens, which limits human capacities to grasp volumetric, complex, and discrete biological dynamics. With the massive production of multidimensional images (3D + time, multi-channels) and derived images (e.g., restored images, segmentation maps, and object tracks), scientists need appropriate visualization and navigation methods to better apprehend the amount of information in their content. New modes of visualization have emerged, including virtual reality (VR)/augmented reality (AR) approaches which should allow more accurate analysis and exploration of large time series of volumetric images, such as those produced by the latest 3D + time fluorescence microscopy. They include integrated algorithms that allow researchers to interactively explore complex spatiotemporal objects at the scale of single cells or multicellular systems, almost in a real time manner. In practice, however, immersion of the user within 3D + time microscopy data represents both a paradigm shift in human-image interaction and an acculturation challenge, for the concerned community. To promote a broader adoption of these approaches by biologists, further dialogue is needed between the bioimaging community and the VR&AR developers.

Paper download: (1.8 MB)

Software (from the sub-project that I was heading):

The AR software for exploring the HoloTracks visualization of live RPE1 cell stacks can be found in InraE's GitLab server: https://forgemia.inra.fr/nathan.van-hille/mrtk.

Pictures (from the sub-project that I was heading):

(these images as well as others from the paper, just like the whole paper, can be used under a CC-BY 4.0 license, see the license statement in the paper or at the digital library page of the publisher)

Main Reference:

Cesar Augusto Valades-Cruz, Ludovic Leconte, Gwendal Fouché, Thomas Blanc, Nathan van Hille, Kévin Fournier, Tao Laurent, Benjamin Gallean, François Deslandes, Bassam Hajj, Emmanuel Faure, Ferran Argelaguet, Alain Trubuil, Tobias Isenberg, Jean-Baptiste Masson, Jean Salamero, and Charles Kervrann (2022) Challenges of Intracellular Visualization using Virtual and Augmented Reality. Frontiers in Bioinformatics, 2, article no. 997082, 9 pages, September 2022.

BibTeX entry:



@ARTICLE{Valades-Cruz:2022:CIV,
  author      = {Valades-Cruz, Cesar Augusto and Ludovic Leconte and Gwendal Fouch{\'e} and Thomas Blanc and van Hille, Nathan and K{\'e}vin Fournier and Tao Laurent and Benjamin Gallean and Fran{\c{c}}ois Deslandes and Bassam Hajj and Emmanuel Faure and Ferran Argelaguet and Alain Trubuil and Tobias Isenberg and Jean-Baptiste Masson and Jean Salamero and Charles Kervrann},
  title       = {Challenges of Intracellular Visualization using Virtual and Augmented Reality},
  journal     = {Frontiers in Bioinformatics},
  year        = {2022},
  volume      = {2},
  month       = sep,
  pages       = {997082:1--997082:9},
  articleno   = {997082},
  numpages    = {9},
  doi         = {10.3389/fbinf.2022.997082},
  doi_url     = {https://doi.org/10.3389/fbinf.2022.997082},
  oa_gold_url = {https://doi.org/10.3389/fbinf.2022.997082},
  oa_hal_url  = {https://hal.science/hal-03790274},
  gitlab_url  = {https://forgemia.inra.fr/nathan.van-hille/mrtk},
  url         = {https://tobias.isenberg.cc/p/Valades-Cruz2022CIV},
  pdf         = {https://tobias.isenberg.cc/personal/papers/Valades-Cruz_2022_CIV.pdf},
}

This work was done as a collaboration between the AVIZ project group of Inria, France, the MaiAGE team of InraE Jouy-en-Josas, and several other teams.

This work was supported through the Naviscope project, funded by Inria, France.