Hasso-Plattner-Institut
Prof. Dr. Jürgen Döllner
  
 

Publications

Neural Style Transfer: A Paradigm Shift for Image-based Artistic Rendering?

Semmo, Amir; Isenberg, Tobias; Döllner, Jürgen in Proceedings International Symposium on Non-Photorealistic Animation and Rendering Winnemöller, Holger; Bartram, Lyn ( Hrsg. ), Seite 5:1--5:13 . New York , ACM , 2017 .

In this meta paper we discuss image-based artistic rendering (IB-AR) based on neural style transfer (NST) and argue, while NST may represent a paradigm shift for IB-AR, that it also has to evolve as an interactive tool that considers the design aspects and mechanisms of artwork production. IB-AR received significant attention in the past decades for visual communication, covering a plethora of techniques to mimic the appeal of artistic media. Example-based rendering represents one the most promising paradigms in IB-AR to (semi-)automatically simulate artistic media with high fidelity, but so far has been limited because it relies on pre-defined image pairs for training or informs only low-level image features for texture transfers. Advancements in deep learning showed to alleviate these limitations by matching content and style statistics via activations of neural network layers, thus making a generalized style transfer practicable. We categorize style transfers within the taxonomy of IB-AR, then propose a semiotic structure to derive a technical research agenda for NSTs with respect to the grand challenges of NPAR. We finally discuss the potentials of NSTs, thereby identifying applications such as casual creativity and art production.
[ DOI ]
Neural Style Transfer: A ... - Download
Weitere Informationen
Herausgeber Winnemöller, Holger; Bartram, Lyn
Tags cgs