Visualizing and Comparing Machine Learning Predictions to Improve Human-AI Teaming on the Example of Cell Lineage

Description:

We visualize the predictions of multiple machine learning models to help biologists as they interactively make decisions about cell lineage—the development of a (plant) embryo from a single ovum cell. Based on a confocal microscopy dataset, traditionally biologists manually constructed the cell lineage, starting from this observation and reasoning backward in time to establish their inheritance. To speed up this tedious process, we make use of machine learning (ML) models trained on a database of manually established cell lineages to assist the biologist in cell assignment. Most biologists, however, are not familiar with ML, nor is it clear to them which model best predicts the embryo’s development. We thus have developed a visualization system that is designed to support biologists in exploring and comparing ML models, checking the model predictions, detecting possible ML model mistakes, and deciding on the most likely embryo development. To evaluate our proposed system, we deployed our interface with six biologists in an observational study. Our results show that the visual representations of machine learning are easily understandable, and our tool, LineageD+, could potentially increase biologists’ working efficiency and enhance the understanding of embryos.

Paper download: (13.0 MB)

Study materials and data:

Our study materials and data can be found in the following OSF repository: osf.io/dcek9. Our study pre-registration is at osf.io/2f6uc.

Software:

The software is available at github.com/JiayiHong/LineageD_Plus.

Demo:

We also have an online demo of LineageD+. For getting the password, please contact us via e-mail.

Videos:

paper video:

30s preview video for IEEE :

pre-recorded presentation video for IEEE :

presentation at IEEE :

Get the videos:

Pictures:

(these images as well as others from the paper are available under a CC-BY 4.0 license, see the license statement at the end of the paper)

Cross-Reference:

This project is based on our earlier work on LineageD. Please check this out as well.

Main Reference:

Jiayi Hong, Ross Maciejewski, Alain Trubuil, and Tobias Isenberg (2024) Visualizing and Comparing Machine Learning Predictions to Improve Human-AI Teaming on the Example of Cell Lineage. IEEE Transactions on Visualization and Computer Graphics, 30(4):1956–1969, April 2024.

BibTeX entry:



@ARTICLE{Hong:2024:CVM,
  author      = {Jiayi Hong and Ross Maciejewski and Alain Trubuil and Tobias Isenberg},
  title       = {Visualizing and Comparing Machine Learning Predictions to Improve Human-{AI} Teaming on the Example of Cell Lineage},
  journal     = {IEEE Transactions on Visualization and Computer Graphics},
  year        = {2024},
  volume      = {30},
  number      = {4},
  month       = apr,
  pages       = {1956--1969},
  doi         = {10.1109/TVCG.2023.3302308},
  doi_url     = {https://doi.org/10.1109/TVCG.2023.3302308},
  oa_hal_url  = {https://hal.science/hal-04212205},
  preprint    = {https://osf.io/c8bez/},
  osf_url     = {https://osf.io/dcek9/},
  osf_url2    = {https://osf.io/2f6uc/},
  github_url  = {https://github.com/JiayiHong/LineageD_Plus},
  url         = {https://tobias.isenberg.cc/p/Hong2024CVM},
  pdf         = {https://tobias.isenberg.cc/personal/papers/Hong_2024_CVM.pdf},
}

Other Reference:

Jiayi Hong (2023) Machine Learning Supported Interactive Visualization of Hybrid 3D and 2D Data for the Example of Plant Cell Lineage Specification. PhD thesis, Université Paris-Saclay, France, February 2023.

BibTeX entry:



@PHDTHESIS{Hong:2023:MLS,
  author      = {Jiayi Hong},
  title       = {Machine Learning Supported Interactive Visualization of Hybrid 3D and 2D Data for the Example of Plant Cell Lineage Specification},
  year        = {2023},
  school      = {Universit{\'e} Paris-Saclay},
  month       = feb,
  address     = {France},
  oa_hal_url  = {https://hal.science/tel-04016701},
  url         = {https://tobias.isenberg.cc/VideosAndDemos/Hong2021DET},
  url2        = {https://tobias.isenberg.cc/VideosAndDemos/Hong2022CST},
  pdf         = {https://tobias.isenberg.cc/personal/papers_students/Hong_2023_MLS.pdf},
}

This work was done at the AVIZ project group of Inria, France.

This work was supported through the Naviscope project, funded by Inria, France.