Hasso-Plattner-Institut
  
Hasso-Plattner-Institut
Prof. Dr. Jürgen Döllner
  
 

Coherence-Enhancing Filtering

Directional features and flow-like structures are considered pleasant, harmonic, or at least interesting by most humans [Wei99]. They are also a highly sought-after property in many of the traditional art forms, such as paintings and illustrations. Enhancing directional coherence in the image helps to clarify region boundaries and features. As exemplified by Expressionism, it also helps to evoke mood or ideas and even elicit emotional response from the viewer. Particular examples include van Gogh and Munch, who have emphasized these features in their paintings. In this work, we present a new image and video abstraction technique that places emphasis on enhancing the directional coherence of features. The most notable related work in this category is image abstraction and stylization based on partial differential equations (PDE), in particular, shape-simplifying image abstraction by Kang and Lee [KL08] and Weickert’s coherence-enhancing shock filter [Wei03]. However, such PDE-based techniques may require a large number of iterations and tend to be unstable when used for video processing [Par08].

We build upon the idea of combining diffusion with shock filtering for image abstraction, but our approach is, in a sense, contrary to that of [KL08], which our technique outperforms in terms of speed, temporal coherence and stability. Instead of simplifying the shape of the image features, we aim to preserve the shape by using a curvature preserving smoothing method that enhances coherence. More specifically, our approach performs smoothing, in the direction of the smallest change, and sharpening, in the orthogonal direction. Instead of modeling this process by a PDE and solving it, we use approximations that operate as local filters on a neighborhood of a pixel. Therefore, good abstraction results are already achieved in a few iterations. This makes it possible to process images and video at real-time rates on a GPU. It also results in a much more stable algorithm that enables temporallycoherent video processing. Compared to the conventional abstraction approaches [WOG06,OBBT07,KKD09], we provide a good balance between the enhancement of directional features and the smoothing of isotropic regions. Our technique preserves and enhances directional features better and creates stronger contrast, which helps to clarify boundaries and features. Furthermore, our approach facilitates easy control over the level of abstractions.

Publications

Image and Video Abstraction by Coherence-Enhancing Filtering

Kyprianidis, Jan Eric and Kang, Henry
Computer Graphics Forum, vol. 30(2):593–602 2011 Proceedings Eurographics 2011

Abstract:

In this work, we present a non-photorealistic rendering technique to create stylized abstractions from color images and videos. Our approach is based on adaptive line integral convolution in combination with directional shock filtering. The smoothing process regularizes directional image features while the shock filter provides a sharpening effect. Both operations are guided by a flow field derived from the structure tensor. To obtain a high-quality flow field, we present a novel smoothing scheme for the structure tensor based on Poisson's equation. Our approach effectively regularizes anisotropic image regions while preserving the overall image structure and achieving a consistent level of abstraction. Moreover, it is suitable for per-frame filtering of video and can be efficiently implemented to process content in real-time.

BibTeX file

@article{KK11,
author = { Kyprianidis, Jan Eric and Kang, Henry },
title = { Image and Video Abstraction by Coherence-Enhancing Filtering },
journal = { Computer Graphics Forum },
year = { 2011 },
volume = { 30 },
number = { 2 },
pages = { 593--602 },
month = { 0 },
abstract = { In this work, we present a non-photorealistic rendering technique to create stylized abstractions from color images and videos. Our approach is based on adaptive line integral convolution in combination with directional shock filtering. The smoothing process regularizes directional image features while the shock filter provides a sharpening effect. Both operations are guided by a flow field derived from the structure tensor. To obtain a high-quality flow field, we present a novel smoothing scheme for the structure tensor based on Poisson's equation. Our approach effectively regularizes anisotropic image regions while preserving the overall image structure and achieving a consistent level of abstraction. Moreover, it is suitable for per-frame filtering of video and can be efficiently implemented to process content in real-time. },
note = { Proceedings Eurographics 2011 },
priority = { 0 }
}

Copyright Notice

last change: Wed, 04 May 2011 21:53:03 +0200

Links

  • Open source demo application of coherence-enhancing image and video abstraction.