Research Overview

Touch-sensitive displays make it possible to explore alternative ways of presenting and interacting with scientific visualizations. In particular, they can be advantageous with respect to immersion, feeling in control, somesthetic feedback, collaboration, and speed of input. We explore touch interfaces that allow people to benefit from these advantages, in particular, for scientific visualization and exploration of data.

Non-photorealistic rendering is inspired by a long tradition of illustration. We applied NPR techniques to illustration and visualization problems in medical, technical, and other domains. We also worked on combining NPR methods with other rendering techniques to create hybrid rendering styles for illustration and visualization.

Many traditional artistic and illustrative depiction techniques have now been simulated withing the context of non-photorealistic rendering. However, many of these techniques can still be visually distinguished from their originals by people looking at such images. We work on identifying the differences between the two groups of images and to understand what this means for a viewer.

I am not only myself actively following open science practices but also investigate how we as a community do it overall and what we can do to improve things even further.

The world is flat and rectangular when it comes to the types of physical screens that we use for representing data and making decisions. Display technology, however, is already evolving quickly: curved, bendable, and highly flexible displays, spherical displays, cubed displays, and even drone-based displays have emerged and are commercially available. These novel types of displays offer new ways to represent and explore data embedded in everyday environments, to communicate it, and share it. For a possible future in which non-planar displays will be ubiquitous, however, there are open questions about what visualizations should look like on these displays, how we would interact with them, and how people would engage with them. Non-planar displays, therefore, not only pose perceptual challenges for data visualization, but it is also yet largely unexplored which visualization types work on them and how to create effective and appealing interactive data visualization experiences. As such, the potential and the challenges of these displays for visual data representation remain unexplored. This project aims to escape from the “display flatland” that characterizes today’s research in visualization. It will establish foundations for how to engage with a future in which physical displays take on several different form factors and become truly embedded in our environments.

Nowadays, the detection and visualization of important localized events and process in multidimensional and multi-valued images, especially in cell and tissue imaging, is tedious and inefficient. Specialized scientists can miss key events due to complexity of the data and the lack of computer guidance. In this proposal, we plan to develop original and cutting-edge visualization and navigation methods to assist scientists, enabling semi-automatic analysis, manipulation, and investigation of temporal series of multi-valued volumetric images, with a strong focus on live cell imaging and microscopy application domains. We will build Naviscope upon the strength of scientific visualization and machine learning methods in order to provide systems capable to assist the scientist to obtain a better understanding of massive amounts of information. Such systems will be able to recognize and highlight the most informative regions of the dataset by reducing the amount of information displayed and guiding the observer attention. In the Naviscope project, we will address the three following challenges and issues: (1) Novel machine learning methods able to detect the main regions of interest, and automatic quantification of sparse sets of molecular interactions and cell processes during navigation to save memory and computational resources. (2) Novel visualization methods able to encode 3D motion/deformation vectors and dynamics features with color/texture-based and non-sub-resolved representations, abstractions, and discretization, as used to show 2D motion and deformation vectors and patterns. (3) Effective machine learning-driven navigation and interaction techniques for complex functional 3D+Time data enabling the analysis of sparse sets of localized intra-cellular events and cell processes (migration, division, etc.). Finally, we will have also to overcome the technological challenge of gathering up the software developed in each team to provide a unique original tool for users in biological imaging, and potentially in medical imaging.

The essential building block of visualization is the phenomenon of visual abstraction. While visual abstraction is intuitively understood, there is no scientific theory associated with it that would be useful in the visualization synthesis process. Our central aim of this project is thus to gain better understanding of the visual abstraction characteristics. We lay down a hypothetical initial basis of theoretical foundations of visual abstractions in the proposal. We hypothesize that visual abstraction is a multidimensional phenomenon that can be spanned by axes of abstraction. Besides abstractions associated with a static structure we take a closer look at abstractions related to dynamics, procedures, and emergence of the structure. We also study abstraction characteristics related to multi-scale phenomena defined both in space and in time. This hypothetical basis is either supported or rejected by means of exemplary evidence from specific application domain of structural biology. Structural biology data is very complex, it includes the aspect of emergence and it is defined over multiple scales. Furthermore, abstraction has led to key discoveries in biology, such as the organization of the DNA. We study the multiscale visual abstraction characteristics on the visualization of long nucleic strands and the abstractions that convey emerging phenomena on visualization of molecular machinery use cases. From these two fields we work toward a theory of visual abstraction in a bottom-up manner, investigating the validity of the theory in other application domains as well.

Touch interaction is increasingly becoming an important means of human-computer interaction. However, often we have to deal with 3D data, visualizations, or scenes for which regular 2D input as derived from touch surfaces is not sufficient. Here we have to develop means of mapping the 2D input to interactions in 3D space, such that the touch exploration of the 3D environments remains intuitive.

With the growing resolution on today's interactive displays it becomes increasingly difficult to maintain responsiveness of the system. We work on applying techniques from computer graphics to address the complexity issues on such interfaces. In addition, large displays require completely new interaction techniques, including the need to accommodate multiple people, multiple interaction points per person or in general, dealing with orientations and awareness.

Many NPR techniques are encapsulated algorithms that allow little influence on the final product other than global parameter changes. In addition, interfaces for NPR are often cluttered with an enormous number of parameters whose influence is difficult to understand. We work on giving people understandable and direct local control over the rendering process.

Hand-drawn pen-and-ink drawings have fascinated me for a long time. Using non-photorealistic rendering we have worked on generating such drawings consisting of silhouette lines and other strokes with the computer.

When doing research on line rendering and pen-and-ink NPR, in particular if print is the medium that one aims for, one needs to produce high quality images. Therefore, we attempt to produce vector images of my NPR techniques wherever possible.