Neural Style Transfer: A Paradigm Shift for Image-based Artistic Rendering?
In this meta paper we discuss image-based artistic rendering (IB-AR) based on neural style transfer (NST) and argue, while NST may represent a paradigm shift for IB-AR, that it also has to evolve as an interactive tool that considers the design aspects and mechanisms of artwork production. IB-AR received significant attention in the past decades for visual communication, covering a plethora of techniques to mimic the appeal of artistic media. Example-based rendering (EBR) represents one the most promising paradigms in IB-AR to (semi-)automatically simulate artistic media with high fidelity, but so far has been limited because it relies on pre-defined image pairs for training or informs only low-level image features for texture transfers. Recent advancements in deep convolutional neural networks showed to alleviate these limitations by separating image contents from style, thus making a generalized style transfer practicable. We categorize style transfers within the taxonomy of IB-AR, then propose a semiotic structure to derive a technical research agenda for NSTs with respect to the grand challenges of NPAR. We finally discuss the potentials of NSTs, thereby identifying applications such as casual creativity and art production.
|Amir Semmo, Tobias Isenberg, and Jürgen Döllner (2017) Neural Style Transfer: A Paradigm Shift for Image-based Artistic Rendering? In Holger Winnemöller and Lyn Bartram, eds., Proceedings of the International Symposium on Non-Photorealistic Animation and Rendering (NPAR as part of Expressive, July 29–30, Los Angeles, CA, USA). New York. ACM, pages 5:1–5:13, 2017.|| doi|