Prof. Dr. Jürgen Döllner


Contribution accepted for Appy Hour @ SIGGRAPH 2017

The contribution "Pictory: Combining Neural Style Transfer and Image Filtering" by Amir Semmo, Matthias Trapp, Jürgen Döllner and Mandy Klingbeil has been accepted as a contribution to SIGGRAPH 2017's Appy Hour program. The 44rd International Conference and Exhibition on Computer Graphics & Interactive Techniques will be held in Los Angeles, CA, USA from 30th July to 3th August. Members of the research group will present the App at SIGGRAPH 2017 in cooperation with Digital Masterpieces GmbH, a spin-off of the Computer Graphics Systems Group at the Hasso Plattner Institute.



Pictory is a mobile app that empowers users to transform photos into artistic renditions by using a combination of neural style transfer with user-controlled state-of-the-art nonlinear image filtering. The combined approach features merits of both artistic rendering paradigms: deep convolutional neural networks can be used to transfer style characteristics at a global scale, while image filtering is able to simulate phenomena of artistic media at a local scale. Thereby, the proposed app implements an interactive two-stage process: first, style presets based on pre-trained feed-forward neural networks are applied using GPU-accelerated compute shaders to obtain initial results. Second, the intermediate output is stylized via oil paint, watercolor, or toon filtering to inject characteristics of traditional painting media such as pigment dispersion (watercolor) as well as soft color blendings (oil paint), and to filter artifacts such as fine-scale noise. Finally, on-screen painting facilitates pixel-precise creative control over the filtering stage, e. g., to vary the brush and color transfer, while joint bilateral upsampling enables outputs at full image resolution suited for printing on real canvas.