LineageD: An Interactive Visual System for Plant Cell Lineage Assignments based on Correctable Machine Learning

Description:

We describe LineageD—a hybrid web-based system to predict, visualize, and interactively adjust plant embryo cell lineages. Currently, plant biologists explore the development of an embryo and its hierarchical cell lineage manually, based on a 3D dataset that represents the embryo status at one point in time. This human decision-making process, however, is time-consuming, tedious, and error-prone due to the lack of integrated graphical support for specifying the cell lineage. To fill this gap, we developed a new system to support the biologists in their tasks using an interactive combination of 3D visualization, abstract data visualization, and correctable machine learning to modify the proposed cell lineage. We use existing manually established cell lineages to obtain a neural network model. We then allow biologists to use this model to repeatedly predict assignments of a single cell division stage. After each hierarchy level prediction, we allow them to interactively adjust the machine learning based assignment, which we then integrate into the pool of verified assignments for further predictions. In addition to building the hierarchy this way in a bottom-up fashion, we also offer users to divide the whole embryo and create the hierarchy tree in a top-down fashion for a few steps, improving the ML-based assignments by reducing the potential for wrong predictions. We visualize the continuously updated embryo and its hierarchical development using both 3D spatial and abstract tree representations, together with information about the model's confidence and spatial properties. We conducted case study validations with five expert biologists to explore the utility of our approach and to assess the potential for LineageD to be used in their daily workflow. We found that the visualizations of both 3D representations and abstract representations help with decision making and the hierarchy tree top-down building approach can reduce assignments errors in real practice.

Paper download: (12.0 MB)

Software:

The software to set up a local copy of the server tool is available at https://gitlab.inria.fr/jhong/lineaged.

Demo:

We also have an online demo of LineageD. For getting the password, please contact us via e-mail.

Study materials and data:

Our study materials and data can be found in the following OSF repository: osf.io/rhyg4.

Videos:

presentation at EuroVis 2022:

Get the videos:

Pictures:

(these images as well as others in the paper are available under a CC-BY 4.0 license, see the license statement at the end of the paper)

Cross-Reference:

We continued this project, which led to our work on LineageD+. Please check this out as well.

Main Reference:

Jiayi Hong, Alain Trubuil, and Tobias Isenberg (2022) LineageD: An Interactive Visual System for Plant Cell Lineage Assignments based on Correctable Machine Learning. Computer Graphics Forum, 41(3):195–207, June 2022.

BibTeX entry:



@ARTICLE{Hong:2022:CST,
  author      = {Jiayi Hong and Alain Trubuil and Tobias Isenberg},
  title       = {{LineageD}: An Interactive Visual System for Plant Cell Lineage Assignments based on Correctable Machine Learning},
  journal     = {Computer Graphics Forum},
  year        = {2022},
  volume      = {41},
  number      = {3},
  month       = jun,
  pages       = {195--207},
  doi         = {10.1111/cgf.14533},
  doi_url     = {https://doi.org/10.1111/cgf.14533},
  oa_hal_url  = {https://hal.science/hal-03699221},
  osf_url     = {https://osf.io/rhyg4/},
  gitlab_url  = {https://gitlab.inria.fr/jhong/lineaged},
  url         = {https://tobias.isenberg.cc/p/Hong2022CST},
  pdf         = {https://tobias.isenberg.cc/personal/papers/Hong_2022_CST.pdf},
}

Other Reference:

Jiayi Hong (2023) Machine Learning Supported Interactive Visualization of Hybrid 3D and 2D Data for the Example of Plant Cell Lineage Specification. PhD thesis, Université Paris-Saclay, France, February 2023.

BibTeX entry:



@PHDTHESIS{Hong:2023:MLS,
  author      = {Jiayi Hong},
  title       = {Machine Learning Supported Interactive Visualization of Hybrid 3D and 2D Data for the Example of Plant Cell Lineage Specification},
  year        = {2023},
  school      = {Universit{\'e} Paris-Saclay},
  month       = feb,
  address     = {France},
  oa_hal_url  = {https://hal.science/tel-04016701},
  url         = {https://tobias.isenberg.cc/VideosAndDemos/Hong2021DET},
  url2        = {https://tobias.isenberg.cc/VideosAndDemos/Hong2022CST},
  pdf         = {https://tobias.isenberg.cc/personal/papers_students/Hong_2023_MLS.pdf},
}

Jiayi Hong, Ferran Argelaguet, Alain Trubuil, and Tobias Isenberg (2020) An Interactive System for Analyzing Plant Embryo Cell Division. In Alex Endert, Steffen Koch, Christophe Hurter, Michael Sedlmair, Daniel Jönsson, and Filip Sadlo, eds., Posters at the IEEE Conference on Visualization (IEEE VIS, October 25–30, (virtually in) Salt Lake City, Utah). 2020. Extended abstract and poster.

BibTeX entry:



@INPROCEEDINGS{Hong:2020:ISA,
  author      = {Jiayi Hong and Ferran Argelaguet and Alain Trubuil and Tobias Isenberg},
  title       = {An Interactive System for Analyzing Plant Embryo Cell Division},
  booktitle   = {Posters at the IEEE Conference on Visualization (IEEE VIS, October 25--30, (virtually in) Salt Lake City, Utah)},
  OPTeditor   = {Alex Endert and Steffen Koch and Christophe Hurter and Michael Sedlmair and Daniel J{\"o}nsson and Filip Sadlo},
  year        = {2020},
  oa_hal_url  = {https://hal.science/hal-02942953},
  url         = {https://tobias.isenberg.cc/p/Hong2021DET},
  pdf         = {https://tobias.isenberg.cc/personal/papers/Hong_2020_ISA.pdf},
}

This work was done at the AVIZ project group of Inria, France, in collaboration with the MaiAGE team of InraE Jouy-en-Josas. There is also a separate project webpage on the Aviz server.

This work was supported through the Naviscope project, funded by Inria, France.