Research Overview

Touch-sensitive displays make it possible to explore alternative ways of presenting and interacting with scientific visualizations. In particular, they can be advantageous with respect to immersion, feeling in control, somesthetic feedback, collaboration, and speed of input. We explore touch interfaces that allow people to benefit from these advantages, in particular, for scientific visualization and exploration of data.

Non-photorealistic rendering is inspired by a long tradition of illustration. We applied NPR techniques to illustration and visualization problems in medical, technical, and other domains. We also worked on combining NPR methods with other rendering techniques to create hybrid rendering styles for illustration and visualization.

Many traditional artistic and illustrative depiction techniques have now been simulated withing the context of non-photorealistic rendering. However, many of these techniques can still be visually distinguished from their originals by people looking at such images. We work on identifying the differences between the two groups of images and to understand what this means for a viewer.

The essential building block of visualization is the phenomenon of visual abstraction. While visual abstraction is intuitively understood, there is no scientific theory associated with it that would be useful in the visualization synthesis process. Our central aim of this project is thus to gain better understanding of the visual abstraction characteristics. We lay down a hypothetical initial basis of theoretical foundations of visual abstractions in the proposal. We hypothesize that visual abstraction is a multidimensional phenomenon that can be spanned by axes of abstraction. Besides abstractions associated with a static structure we take a closer look at abstractions related to dynamics, procedures, and emergence of the structure. We also study abstraction characteristics related to multi-scale phenomena defined both in space and in time. This hypothetical basis is either supported or rejected by means of exemplary evidence from specific application domain of structural biology. Structural biology data is very complex, it includes the aspect of emergence and it is defined over multiple scales. Furthermore, abstraction has led to key discoveries in biology, such as the organization of the DNA. We study the multiscale visual abstraction characteristics on the visualization of long nucleic strands and the abstractions that convey emerging phenomena on visualization of molecular machinery use cases. From these two fields we work toward a theory of visual abstraction in a bottom-up manner, investigating the validity of the theory in other application domains as well.

Nowadays, the detection and visualization of important localized events and process in multidimensional and multi-valued images, especially in cell and tissue imaging, is tedious and inefficient. Specialized scientists can miss key events due to complexity of the data and the lack of computer guidance. In this proposal, we plan to develop original and cutting-edge visualization and navigation methods to assist scientists, enabling semi-automatic analysis, manipulation, and investigation of temporal series of multi-valued volumetric images, with a strong focus on live cell imaging and microscopy application domains. We will build Naviscope upon the strength of scientific visualization and machine learning methods in order to provide systems capable to assist the scientist to obtain a better understanding of massive amounts of information. Such systems will be able to recognize and highlight the most informative regions of the dataset by reducing the amount of information displayed and guiding the observer attention. In the Naviscope project, we will address the three following challenges and issues: (1) Novel machine learning methods able to detect the main regions of interest, and automatic quantification of sparse sets of molecular interactions and cell processes during navigation to save memory and computational resources. (2) Novel visualization methods able to encode 3D motion/deformation vectors and dynamics features with color/texture-based and non-sub-resolved representations, abstractions, and discretization, as used to show 2D motion and deformation vectors and patterns. (3) Effective machine learning-driven navigation and interaction techniques for complex functional 3D+Time data enabling the analysis of sparse sets of localized intra-cellular events and cell processes (migration, division, etc.). Finally, we will have also to overcome the technological challenge of gathering up the software developed in each team to provide a unique original tool for users in biological imaging, and potentially in medical imaging.

Touch interaction is increasingly becoming an important means of human-computer interaction. However, often we have to deal with 3D data, visualizations, or scenes for which regular 2D input as derived from touch surfaces is not sufficient. Here we have to develop means of mapping the 2D input to interactions in 3D space, such that the touch exploration of the 3D environments remains intuitive.

With the growing resolution on today's interactive displays it becomes increasingly difficult to maintain responsiveness of the system. We work on applying techniques from computer graphics to address the complexity issues on such interfaces. In addition, large displays require completely new interaction techniques, including the need to accommodate multiple people, multiple interaction points per person or in general, dealing with orientations and awareness.

Many NPR techniques are encapsulated algorithms that allow little influence on the final product other than global parameter changes. In addition, interfaces for NPR are often cluttered with an enormous number of parameters whose influence is difficult to understand. We work on giving people understandable and direct local control over the rendering process.

Hand-drawn pen-and-ink drawings have fascinated me for a long time. Using non-photorealistic rendering we have worked on generating such drawings consisting of silhouette lines and other strokes with the computer.

When doing research on line rendering and pen-and-ink NPR, in particular if print is the medium that one aims for, one needs to produce high quality images. Therefore, we attempt to produce vector images of my NPR techniques wherever possible.