Responsive Interaction on Large Displays

With the resolution on today's interactive displays growing and multiple users concurrently interacting with them, it becomes increasingly difficult to maintain responsiveness of the system. I work on applying techniques from computer graphics to address the complexity issues on such interfaces. Specifically, I use the idea of image buffers to store information about the interactive environment.

The concept of image buffers has previously been used, for example, to achieve hidden surface removal (z-buffer) and, in particular in non-photorealistic rendering for the creation of expressive renditions and to steer agents that, e.g., place strokes. Using such buffers to store properties of an interface, I have been able to regain interactivity and responsiveness on large displays and an increase in rendering speed of over a magnitude over previous implementations. The image below shows a screenshot of an application that was created based on this new approach and that can maintain high frame-rates while users are interacting with it.

Interactive multi-user tabletop application.

Publications

• Jens Grubert, Sheelagh Carpendale, and Tobias Isenberg. Interactive Stroke-Based NPR using Hand Postures on Large Displays. In Katerina Mania and Erik Reinhard, editors, Short Papers at Eurographics 2008 (EG 2008, April 14–18, 2008, Crete, Greece), pages 279–282, Aire-la-Ville, Switzerland, 2008. Eurographics Association. Short paper and poster. (PDF, 3874332 bytes)
• Jens Grubert, Sheelagh Carpendale, and Tobias Isenberg. Interactive Stroke-Based NPR using Hand Postures on Large Displays. Technical Report 2007-883-35, Department of Computer Science, University of Calgary, Canada, December 2007. Also see the short paper at Eurographics 2008. (PDF, 3888599 bytes) (doi:1880/45787)
• Jens Grubert, Mark Hanckock, Sheelagh Carpendale, Edward Tse, and Tobias Isenberg. Interacting with Stroke-Based Rendering on a Wall Display. Technical Report 2007-882-34, Department of Computer Science, University of Calgary, Canada, October 2007. (PDF, 6913049 bytes) (doi:1880/45786)
• Tobias Isenberg, Simon Nix, Martin Schwarz, André Miede, Stacey D. Scott, and Sheelagh Carpendale. Mobile Spatial Tools for Fluid Interaction. Technical Report 2007-872-24, Department of Computer Science, University of Calgary, Canada, July 2007. (PDF, 3155509 bytes) (doi:1880/45785)
• Martin Schwarz, Tobias Isenberg, Katherine Mason, and Sheelagh Carpendale. Modeling with Rendering Primitives: An Interactive Non-Photorealistic Canvas. In Maneesh Agrawala and Oliver Deussen, editors, Proceedings of the Fifth International Symposium on Non-Photorealistic Animation and Rendering (NPAR 2007, August 4–5, 2007, San Diego, California, USA), pages 15–22, New York, 2007. ACM Press. (PDF, 7648955 bytes) (doi:10.1145/1274871.1274874)
• Martin Schwarz, Tobias Isenberg, Katherine Mason, and Sheelagh Carpendale. Modeling with Rendering Primitives: An Interactive Non-Photorealistic Canvas. Technical Report 2007-851-03, Department of Computer Science, University of Calgary, Canada, February 2007. Also see the paper at NPAR 2007. (PDF, 7205657 bytes) (doi:1880/45781)
• Sheelagh Carpendale, Tobias Isenberg, Stacey D. Scott, Uta Hinrichs, André Miede, Russel Kruger, Stefan Habelski, and Kori Inkpen. Collaborative Interaction on Large Tabletop Displays. In Amy Voida, editor, Conference Supplement of the 2006 ACM Conference on Computer Supported Collaborative Work (CSCW 2006, November 4–8, 2006, Banff, Alberta, Canada), pages 57–58, New York, 2006. ACM Press. Extended abstract and interactive demo. (PDF, 3750323 bytes)
• Tobias Isenberg, André Miede, and Sheelagh Carpendale. A Buffer Framework for Supporting Responsive Interaction in Information Visualization Interfaces. In Hajime Kita, Kim Rose, Benay Dara-Abrams, Rieko Kadobayashi, Rick McGeer, and Hideyuki Takada, editors, Proceedings of the Fourth International Conference on Creating, Connecting and Collaborating through Computing (C⁵ 2006, January 26–27, 2006, Berkeley, California, USA), pages 262–269, Los Alamitos, CA, 2006. IEEE Computer Society. (PDF, 4915484 bytes) (doi:10.1109/C5.2006.4)
• Tobias Isenberg, Petra Neumann, Sheelagh Carpendale, Simon Nix, and Saul Greenberg. Interactive Annotations on Large, High-Resolution Information Displays. In 2006 Conference Compendium of IEEE VIS, IEEE InfoVis, and IEEE VAST (October 29–November 3, 2006, Baltimore, Maryland, USA), pages 124–125, Los Alamitos, CA, 2006. IEEE Computer Society. Extended abstract and poster. (PDF, 6416705 bytes)