-
A Point-Based and Image-Based Multi-Pass Rendering Technique for Visualizing Massive 3D Point Clouds in VR Environments. Discher, Sören; Masopust, Leon; Schulz, Sebastian; Richter, Rico; Döllner, Jürgen in Proceedings of WSCG 2018 (2018).
Real-time rendering for 3D point clouds allows for interactively exploring and inspecting real-world assets, sites, or regions on a broad range of devices but has to cope with their vastly different computing capabilities. Virtual reality (VR) applications rely on high frame rates (i.e., around 90 fps as opposed to 30 - 60 fps) and show high sensitivity to any kind of visual artifacts, which are typical for 3D point cloud depictions (e.g., holey surfaces or visual clutter due to inappropriate point sizes). We present a novel rendering system that allows for an immersive, nausea-free exploration of arbitrary large 3D point clouds on state-of-the-art VR devices such as HTC Vive and Oculus Rift. Our approach applies several point-based and image-based rendering techniques that are combined using a multipass rendering pipeline. The approach does not require to derive generalized, mesh-based representations in a preprocessing step and preserves precision and density of the raw 3D point cloud data. The presented techniques have been implemented and evaluated with massive real-world data sets from aerial, mobile, and terrestrial acquisition campaigns containing up to 2.6 billion points to show the practicability and scalability of our approach.
-
A Scalable WebGL-based Approach for Visualizing Massive 3D Point Clouds using Semantics-Dependent Rendering Techniques. Discher, Sören; Richter, Rico; Döllner, Jürgen in Proceedings of Web3D ’18 (2018).
3D point cloud technology facilitates the automated and highly detailed digital acquisition of real-world environments such as assets, sites, cities, and countries; the acquired 3D point clouds represent an essential category of geodata used in a variety of geoinformation applications and systems. In this paper, we present a web-based system for the interactive and collaborative exploration and inspection of arbitrary large 3D point clouds. Our approach is based on standard WebGL on the client side and is able to render 3D point clouds with billions of points. It uses spatial data structures and level-of-detail representations to manage the 3D point cloud data and to deploy out-of-core and web-based rendering concepts. By providing functionality for both, thin-client and thick-client applications, the system scales for client devices that are vastly different in computing capabilities. Different 3D point-based rendering techniques and post-processing effects are provided to enable task-specific and data-specific filtering and highlighting, e.g., based on per-point surface categories or temporal information. A set of interaction techniques allows users to collaboratively work with the data, e.g., by measuring distances and areas, by annotating, or by selecting and extracting data subsets. Additional value is provided by the system's ability to display additional, context-providing geodata alongside 3D point clouds and to integrate task-specific processing and analysis operations. We have evaluated the presented techniques and the prototype system with different data sets from aerial, mobile, and terrestrial acquisition campaigns with up to 120 billion points to show their practicality and feasibility.
-
Approaches for Local Artistic Control of Mobile Neural Style Transfer. Reimann, Max; Klingbeil, Mandy; Pasewaldt, Sebastian; Semmo, Amir; Döllner, Jürgen; Trapp, Matthias (2018).
This work presents enhancements to state-of-the-art adaptive neural style transfer techniques, thereby providing a generalized user interface with creativity tool support for lower-level local control to facilitate the demanding interactive editing on mobile devices. The approaches are implemented in a mobile app that is designed for orchestration of three neural style transfer techniques using iterative, multi-style generative and adaptive neural networks that can be locally controlled by on-screen painting metaphors to perform location-based filtering and direct the composition. Based on first user tests, we conclude with insights, showing different levels of satisfaction for the implemented techniques and user interaction design, pointing out directions for future research.
-
Combined Visual Exploration of 2D Ground Radar and 3D Point Cloud Data for Road Environments. Wolf, Johannes; Discher, Sören; Masopust, Leon; Schulz, Sebastian; Richter, Rico; Döllner, Jürgen in Proceedings of 3D GeoInfo 2018 (2018).
Ground-penetrating 2D radar scans are captured in road environments for examination of pavement condition and below-ground variations such as lowerings and developing pot-holes. 3D point clouds captured above ground provide a precise digital representation of the road’s surface and the surrounding environment. If both data sources are captured for the same area, a combined visualization is a valuable tool for infrastructure maintenance tasks. This paper presents visualization techniques developed for the combined visual exploration of the data captured in road environments. Main challenges are the positioning of the ground radar data within the 3D environment and the reduction of occlusion for individual data sets. By projecting the measured ground radar data onto the precise trajectory of the scan, it can be displayed within the context of the 3D point cloud representation of the road environment. We show that customizable overlay, filtering, and cropping techniques enable insightful data exploration. A 3D renderer combines both data sources. To enable an inspection of areas of interest, ground radar data can be elevated above ground level for better visibility. An interactive lens approach enables to visualize data sources that are currently occluded by others. The visualization techniques prove to be a valuable tool for ground layer anomaly inspection and were evaluated in a real-world data set. The combination of 2D ground radar scans with 3D point cloud data improves data interpretation by giving context information (e. g., about manholes in the street) that can be directly accessed during evaluation.
-
EvoCells – A Treemap Layout Algorithm for Evolving Tree Data. Scheibel, Willy; Weyand, Christopher; Döllner, Jürgen (2018).
-
Hierarchical Spatial Aggregation for Level-of-Detail Visualization of 3D Thematic Data. Vollmer, Jan Ole; Trapp, Matthias; Schumann, Heidrun; Döllner, Jürgen in ACM Transactions on Spatial Algorithms and Systems (2018). 4(3) 9:1--9:23.
Thematic maps are a common tool to visualize semantic data with a spatial reference. Combining thematic data with a geometric representation of their natural reference frame aids the viewer’s ability in gaining an overview, as well as perceiving patterns with respect to location; however, as the amount of data for visualization continues to increase, problems such as information overload and visual clutter impede perception, requiring data aggregation and level-of-detail visualization techniques. While existing aggregation techniques for thematic data operate in a 2D reference frame (i.e., map), we present two aggregation techniques for 3D spatial and spatiotemporal data mapped onto virtual city models that hierarchically aggregate thematic data in real time during rendering to support on-the-fly and on-demand level-of-detail generation. An object-based technique performs aggregation based on scene-specific objects and their hierarchy to facilitate per-object analysis, while the scene-based technique aggregates data solely based on spatial locations, thus supporting visual analysis of data with arbitrary reference geometry. Both techniques can apply different aggregation functions (mean, minimum, and maximum) for ordinal, interval, and ratio-scaled data and can be easily extended with additional functions. Our implementation utilizes the programmable graphics pipeline and requires suitably encoded data, i.e., textures or vertex attributes. We demonstrate the application of both techniques using real-world datasets, including solar potential analyses and the propagation of pressure waves in a virtual city model.
-
Interaction and Locomotion Techniques for the Exploration of Massive 3D Point Clouds in VR Environments. Thiel, Felix; Discher, Sören; Richter, Rico; Döllner, Jürgen in Proceedings of ISPRS Technical Commission IV Symposium 2018 (2018).
Emerging virtual reality (VR) technology allows immersively exploring digital 3D content on standard consumer hardware. Using in-situ or remote sensing technology, such content can be automatically derived from real-world sites. External memory algorithms allow for the non-immersive exploration of the resulting 3D point clouds on a diverse set of devices with vastly different rendering capabilities. Applications for VR environments raise additional challenges for those algorithms as they are highly sensitive towards visual artifacts that are typical for point cloud depictions (i.e., overdraw and underdraw), while simultaneously requiring higher frame rates (i.e., around 90 fps instead of 30 - 60 fps). We present a rendering system for the immersive exploration and inspection of massive 3D point clouds on state-of-the-art VR devices. Based on a multi-pass rendering pipeline, we combine point-based and image-based rendering techniques to simultaneously improve the rendering performance and the visual quality. A set of interaction and locomotion techniques allows users to inspect a 3D point cloud in detail, for example by measuring distances and areas or by scaling and rotating visualized data sets. All rendering, interaction and locomotion techniques can be selected and configured dynamically, allowing to adapt the rendering system to different use cases. Tests on data sets with up to 2.6 billion points show the feasibility and scalability of our approach.
-
MaeSTrO: A Mobile App for Style Transfer Orchestration using Neural Networks. Reimann, Max; Klingbeil, Mandy; Pasewaldt, Sebastian; Semmo, Amir; Trapp, Matthias; Döllner, Jürgen (2018). 9-16.
Mobile expressive rendering gained increasing popularity amongst users seeking casual creativity by image stylization and supports the development of mobile artists as a new user group. In particular, the neural style transfer has advanced as a core technology to emulate characteristics of manifold artistic styles and media without deep prior knowledge of photo processing or editing. However, when it comes to creative expression, the technology still faces inherent limitations in providing low-level controls for localized image stylization, e.g., with respect to image feature semantics or the user's ideas and interest. The goal of this work is to implement and enhance state-of-the-art neural style transfer techniques, providing a generalized user interface with interactive tools for local control that facilitate a creative editing process on mobile devices. At this, we first propose a problem characterization consisting of three goals that represent a trade-off between visual quality, run-time performance and ease of control. We then present MaeSTrO, a mobile app for orchestration of three neural style transfer techniques using iterative, multi-style generative and adaptive neural networks that can be locally controlled by on-screen painting metaphors to direct a semantics-based composition and perform location-based filtering. Based on first user tests, we conclude with insights, showing different levels of satisfaction for the implemented techniques and user interaction design, pointing out directions for future research.
-
MaeSTrO: Mobile-Style Transfer Orchestration Using Adaptive Neural Networks. Reimann, Max; Semmo, Amir; Pasewaldt, Sebastian; Klingbeil, Mandy; Döllner, Jürgen (2018).
We present MaeSTrO, a mobile app for image stylization that empowers users to direct, edit and perform a neural style transfer with creative control. The app uses iterative style transfer, multi-style generative and adaptive networks to compute and apply flexible yet comprehensive style models of arbitrary images at run-time. Compared to other mobile applications, MaeSTrO introduces an interactive user interface that empowers users to orchestrate style transfers in a two-stage process for an individual visual expression: first, initial semantic segmentation of a style image can be complemented by on-screen painting to direct sub-styles in a spatially-aware manner. Second, semantic masks can be virtually drawn on top of a content image to adjust neural activations within local image regions, and thus direct the transfer of learned sub-styles. This way, the general feed-forward neural style transfer is evolved towards an interactive tool that is able to consider composition variables and mechanisms of general artwork production, such as color, size and location-based filtering. MaeSTrO additionally enables users to define new styles directly on a device and synthesize high-quality images based on prior segmentations via a service-based implementation of compute-intensive iterative style transfer techniques.
-
MNPR: A Framework for Real-Time Expressive Non-Photorealistic Rendering of 3D Computer Graphics. Montesdeoca, Santiago; Seah, Hock Soon; Semmo, Amir; Bénard, Pierre; Vergne, Romain; Thollot, Joëlle; Benvenuti, Davide T. Aydın, Sýkora, D. (eds.) (2018).
We propose a framework for expressive non-photorealistic rendering of 3D computer graphics. Our work focuses on enabling stylization pipelines with a wide range of control, thereby covering the interaction spectrum with real-time feedback. In addition, we introduce control semantics that allow cross-stylistic art-direction, which is demonstrated through our implemented watercolor, oil and charcoal stylizations. Our generalized control semantics and their style-specific mappings are designed to be extrapolated to other styles, by adhering to the same control scheme. We then share our implementation details by breaking down the framework and elaborating on its inner workings. Finally, we evaluate the usefulness of each level of control through a user study involving 20 experienced artists and engineers in the industry, who have collectively spent over 245 hours using our system. Our framework is implemented in Autodesk Maya and open-sourced through this publication, to facilitate adoption by artists and further development by the expressive research and development community.
-
Reducing Affective Responses to Surgical Images through Color Manipulation and Stylization. Besançon, Lonni; Semmo, Amir; Biau, David; Frachet, Bruno; Pineau, Virginie; Ariali, El Hadi; Taouachi, Rabah; Isenberg, Tobias; Dragicevic, Pierre T. Aydın, Sýkora, D. (eds.) (2018).
We present the first empirical study on using color manipulation and stylization to make surgery images more palatable. While aversion to such images is natural, it limits many people's ability to satisfy their curiosity, educate themselves, and make informed decisions. We selected a diverse set of image processing techniques, and tested them both on surgeons and lay people. While many artistic methods were found unusable by surgeons, edge-preserving image smoothing gave good results both in terms of preserving information (as judged by surgeons) and reducing repulsiveness (as judged by lay people). Color manipulation turned out to be not as effective.
-
Service-based Processing and Provisioning of Image-Abstraction Techniques. Richter, Marvin; Söchting, Maximilian; Semmo, Amir; Döllner, Jürgen; Trapp, Matthias (2018).
Digital images and image streams represent two major categories of media captured, delivered, and shared on the Web. Techniques for their analysis, classification, and processing are fundamental building blocks in today's digital media applications ranging from mobile image transformation apps to professional digital production suites. To efficiently process such digital media (1) independent of hardware requirements, (2) at different data complexity scales, (3) to yield high-quality results, poses several challenges for software frameworks and hardware systems, in particular for mobile devices. With respect to these aspects, using service-based architectures are a common approach to strive for. However, unlike geodata, there is currently no standard approach for service definition, implementation, and orchestration in the domain of digital images and videos. This paper presents an approach for service-based image processing and provisioning of processing techniques by the example of image-abstraction techniques. The generality and feasibility of the proposed system is demonstrated by different client applications that have been implemented for the Android Operation System, for Google's G-Suite Software-as-a-Service Infrastructure, as well as for Desktop systems. The performance of the system is discussed at the example of complex, resource-intensive image abstraction techniques, such as watercolor rendering.
-
Service-Oriented Processing and Analysis of Massive Point Clouds in Geoinformation Management. Discher, Sören; Richter, Rico; Trapp, Matthias; Döllner, Jürgen in Service Oriented Mapping: Changing Paradigm in Map Production and Geoinformation Management, J. Döllner, Jobst, M., Schmitz, P. (eds.) (2018).
-
Teaching Image-Processing Programming for Mobile Devices: A Software Development Perspective. Trapp, Matthias; Pasewaldt, Sebastian; Dürschmid, Tobias; Semmo, Amir; Döllner, Jürgen F. Post, Žára, J. (eds.) (2018).
In this paper we present a concept of a research course that teaches students in image processing as a building block of mobile applications. Our goal with this course is to teach theoretical foundations, practical skills in software development as well as scientific working principles to qualify graduates to start as fully-valued software developers or researchers. The course includes teaching and learning focused on the nature of small team research and development as encountered in the creative industries dealing with computer graphics, computer animation and game development. We discuss our curriculum design and issues in conducting undergraduate and graduate research that we have identified through four iterations of the course. Joint scientific demonstrations and publications of the students and their supervisors as well as quantitative and qualitative evaluation by students underline the success of the proposed concept. In particular, we observed that developing using a common software framework helps the students to jump start their course projects, while industry software processes such as branching coupled with a three-tier breakdown of project features helps them to structure and assess their progress.
-
Towards The Generation of Digital Twins for Facility Management Based on 3D Point Clouds. Stojanovic, Vladeta; Trapp, Matthias; Richter, Rico; Hagedorn, Benjamin; Döllner, Jürgen (2018).
-
Attributed Vertex Clouds. Scheibel, Willy; Buschmann, Stefan; Trapp, Matthias; Döllner, Jürgen in GPU Zen, W. Engel, Oat, C. (eds.) (2017). (8 ed.)
In todays computer graphics applications, large 3D scenes are rendered which consist of polygonal geometries such as triangle meshes. Using state- of-the-art techniques, this geometry is often represented on the GPU using vertex and index buffers, as well as additional auxiliary data such as tex- tures or uniform buffers. For polygonal meshes of arbitrary complexity, the described approach is indispensable. However, there are several types of simpler geometries (e.g., cuboids, spheres, tubes, or splats) that can be generated procedurally. We present an efficient data representation and render- ing concept for such geometries, denoted as attributed vertex clouds (AVCs). Using this approach, geometry is generated on the GPU during execution of the programmable rendering pipeline. Each vertex is used as the argument for a function that procedurally generates the target geometry. This function is called a transfer function, and it is implemented using shader programs and therefore executed as part of the rendering process. This approach allows for compact geometry representation and results in reduced memory footprints in comparison to traditional representations. By shifting geometry generation to the GPU, the resulting volatile geometry can be controlled flexibly, i.e., its position, parameteri- zation, and even the type of geometry can be modified without requiring suggests improved rendering times and reduced memory transmission through the rendering pipeline.
-
Attributed Vertex Clouds. Scheibel, Willy; Buschmann, Stefan; Trapp, Matthias; Döllner, Jürgen in GPU Zen, W. Engel, Oat, C. (eds.) (2017). (8 ed.)
In todays computer graphics applications, large 3D scenes are rendered which consist of polygonal geometries such as triangle meshes. Using state- of-the-art techniques, this geometry is often represented on the GPU using vertex and index buffers, as well as additional auxiliary data such as tex- tures or uniform buffers. For polygonal meshes of arbitrary complexity, the described approach is indispensable. However, there are several types of simpler geometries (e.g., cuboids, spheres, tubes, or splats) that can be generated procedurally. We present an efficient data representation and render- ing concept for such geometries, denoted as attributed vertex clouds (AVCs). Using this approach, geometry is generated on the GPU during execution of the programmable rendering pipeline. Each vertex is used as the argument for a function that procedurally generates the target geometry. This function is called a transfer function, and it is implemented using shader programs and therefore executed as part of the rendering process. This approach allows for compact geometry representation and results in reduced memory footprints in comparison to traditional representations. By shifting geometry generation to the GPU, the resulting volatile geometry can be controlled flexibly, i.e., its position, parameteri- zation, and even the type of geometry can be modified without requiring suggests improved rendering times and reduced memory transmission through the rendering pipeline.
-
Challenges in User Experience Design of Image Filtering Apps. Klingbeil, Mandy; Pasewaldt, Sebastian; Semmo, Amir; Döllner, Jürgen (2017).
Photo filtering apps successfully deliver image-based stylization techniques to a broad audience, in particular in the ubiquitous domain (e.g., smartphones, tablet computers). Interacting with these inherently complex techniques has so far mostly been approached in two different ways: (1) by exposing many (technical) parameters to the user, resulting in a professional application that typically requires expert domain knowledge, or (2) by hiding the complexity via presets that only allows the application of filters but prevents creative expression thereon. In this work, we outline challenges of and present approaches for providing interactive image filtering on mobile devices, thereby focusing on how to make them usable for people in their daily life. This is discussed by the example of BeCasso, a user-centric app for assisted image stylization that targets two user groups: mobile artists and users seeking casual creativity. Through user research, qualitative and quantitative user studies, we identify and outline usability issues that showed to prevent both user groups from reaching their objectives when using the app. On the one hand, user-group-targeting has been improved by an optimized user experience design. On the other hand, multiple level of controls have been implemented to ease the interaction and hide the underlying complex technical parameters. Evaluations underline that the presented approach can increase the usability of complex image stylization techniques for mobile apps.
-
Comparative Visualization of BIM Geometry and Corresponding Point Clouds. Stojanovic, Vladeta; Richter, Rico; Döllner, Jürgen; Trapp, Matthias in International Journal of Sustainable Development and Planning (2017). 13(1)
-
Demo: Pictory - Neural Style Transfer and Editing with CoreML. Pasewaldt, Sebastian; Semmo, Amir; Klingbeil, Mandy; Döllner, Jürgen (2017).
This work presents advances in the design and implementation of Pictory, an iOS app for artistic neural style transfer and interactive image editing using the CoreML and Metal APIs. Pictory combines the benefits of neural style transfer, e.g., high degree of abstraction on a global scale, with the interactivity of GPU-accelerated stateof-the-art image-based artistic rendering on a local scale. Thereby, the user is empowered to create high-resolution, abstracted renditions in a two-stage approach. First, a photo is transformed using a pre-trained convolutional neural network to obtain an intermediate stylized representation. Second, image-based artistic rendering techniques (e.g., watercolor, oil paint or toon filtering) are used to further stylize the image. Thereby, fine-scale texture noise—introduced by the style transfer—is filtered and interactive means are provided to individually adjust the stylization effects at run-time. Based on qualitative and quantitative user studies, Pictory has been redesigned and optimized to support casual users as well as mobile artists by providing effective, yet easy to understand, tools to facilitate image editing at multiple levels of control.
-
Hinter den Bildern von Andreas Schiller. Döllner, Jürgen in Andreas Schiller: global backup II, J. Penzel (ed.) (2017). 75-79.
-
Hybrid-Treemap Layouting. Hahn, Sebastian; Döllner, Jürgen (2017).
This paper presents an approach for hybrid treemaps, which applies and combines several different layout principles within a single tree map in contrast to traditional treemap variants based on a single layout concept. To this end, we analyze shortcomings of state-of-the-art treemap algorithms such as Moore, Voronoi and Strip layouts. Based on a number of identified edge cases, we propose a combination of these different layout algorithms, individually selected for and applied on each sub-hierarchy of the given treemap data. The selection decision is based on the number of items to be layouted as well as the aspect-ratio of the containing visual elements. Futhermore, a layout quality score based on existing treemap layout metrics (e.g., average distance change, relative direction change, average aspect ratio) has been used to evaluate the results of the proposed hybrid layout algorithm and to demonstrate its usefulness applied on representative hierarchical data sets. © The Authors 2017. This is the authors' version of the work. It is posted here for your personal use. Not for redistribution. The definitive version will be published in Proceedings of the EuroVis2017.
-
Improving Layout Quality by Mixing Treemap-Layouts Based on Data-Change Characteristics. Bethge, Joseph; Hahn, Sebastian; Döllner, Jürgen M. Hullin, Klein, R., Schultz, T., Yao, A. (eds.) (2017). (Vol. 2017)
This paper presents a hybrid treemap layout approach that optimizes layout-quality metrics by combining state-of-the-art treemap layout algorithms. It utilizes machine learning to predict those metrics based on data metrics describing the characteristics and changes of the dataset. For this, the proposed approach uses a neural network which is trained on artificially generated dataset,s containing a total of 15.8 million samples. The resulting model is integrated into an approach called Smart-Layouting. This approach is evaluated on real-world data from 100 publicly available software repositories. Compared to other state-of-the-art treemap algorithms it reaches an overall better result. Additionally, this approach can be customized by an end user’s needs. The customization allows for specifying weights for the importance of each layout-quality metric. The results indicate, that the algorithm is able to adapt successfully towards a given set of weights.
-
Mixed-Projection Treemaps: A Novel Approach Mixing 2D and 2.5D Treemaps. Limberger, Daniel; Scheibel, Willy; Trapp, Matthias; Döllner, Jürgen (2017).
-
Neural Style Transfer: A Paradigm Shift for Image-based Artistic Rendering? Semmo, Amir; Isenberg, Tobias; Döllner, Jürgen H. Winnemöller, Bartram, L. (eds.) (2017). 5:1--5:13.
In this meta paper we discuss image-based artistic rendering (IB-AR) based on neural style transfer (NST) and argue, while NST may represent a paradigm shift for IB-AR, that it also has to evolve as an interactive tool that considers the design aspects and mechanisms of artwork production. IB-AR received significant attention in the past decades for visual communication, covering a plethora of techniques to mimic the appeal of artistic media. Example-based rendering represents one the most promising paradigms in IB-AR to (semi-)automatically simulate artistic media with high fidelity, but so far has been limited because it relies on pre-defined image pairs for training or informs only low-level image features for texture transfers. Advancements in deep learning showed to alleviate these limitations by matching content and style statistics via activations of neural network layers, thus making a generalized style transfer practicable. We categorize style transfers within the taxonomy of IB-AR, then propose a semiotic structure to derive a technical research agenda for NSTs with respect to the grand challenges of NPAR. We finally discuss the potentials of NSTs, thereby identifying applications such as casual creativity and art production.
-
Pictory: Combining Neural Style Transfer and Image Filtering. Semmo, Amir; Trapp, Matthias; Döllner, Jürgen; Klingbeil, Mandy (2017). 5:1--5:2.
This work presents Pictory, a mobile app that empowers users to transform photos into artistic renditions by using a combination of neural style transfer with user-controlled state-of-the-art nonlinear image filtering. The combined approach features merits of both artistic rendering paradigms: deep convolutional neural networks can be used to transfer style characteristics at a global scale, while image filtering is able to simulate phenomena of artistic media at a local scale. Thereby, the proposed app implements an interactive two-stage process: first, style presets based on pre-trained feed-forward neural networks are applied using GPU-accelerated compute shaders to obtain initial results. Second, the intermediate output is stylized via oil paint, watercolor, or toon filtering to inject characteristics of traditional painting media such as pigment dispersion (watercolor) as well as soft color blendings (oil paint), and to filter artifacts such as fine-scale noise. Finally, on-screen painting facilitates pixel-precise creative control over the filtering stage, e. g., to vary the brush and color transfer, while joint bilateral upsampling enables outputs at full image resolution suited for printing on real canvas.
-
Progressive High-Quality Rendering for Interactive Information Cartography using WebGL. Limberger, Daniel; Pursche, Marcel; Klimke, Jan; Döllner, Jürgen in Web3D '17 (2017). 4.
Information cartography services provided via web-based clients using real-time rendering do not always necessitate a continuous stream of updates in the visual display. This paper shows how progressive rendering by means of multi-frame sampling and frame accumulation can introduce high-quality visual effects using robust and straightforward implementations. For it, (1) a suitable rendering loop is described, (2) WebGL limitations are discussed, and (3) an adaption of THREE.js featuring progressive anti-aliasing, screen-space ambient occlusion, and depth of field is detailed. Furthermore, sampling strategies are discussed and rendering performance is evaluated, emphasizing the low per-frame costs of this approach.
-
ProsumerFX: Mobile Design of Image Stylization Components. Dürschmid, Tobias; Söchting, Maximilian; Semmo, Amir; Trapp, Matthias; Döllner, Jürgen (2017).
With the continuous advances of mobile graphics hardware, high-quality image stylization—e.g., based on image filtering, stroke-based rendering, and neural style transfer—is becoming feasible and increasingly used in casual creativity apps. The creative expression facilitated by these mobile apps, however, is typically limited with respect to the usage and application of pre-defined visual styles, which ultimately do not include their design and composition—an inherent requirement of prosumers. We present ProsumerFX, a GPU-based app that enables to interactively design parameterizable image stylization components on-device by reusing building blocks of image processing effects and pipelines. Furthermore, the presentation of the effects can be customized by modifying the icons, names, and order of parameters and presets. Thereby, the customized visual styles are defined as platform-independent effects and can be shared with other users via a web-based platform and database. Together with the presented mobile app, this system approach supports collaborative works for designing visual styles, including their rapid prototyping, A/B testing, publishing, and distribution. Thus, it satisfies the needs for creative expression of both professionals as well as the general public.
-
Reducing Visual Complexity in Software Maps using Importance-based Aggregation of Nodes. Limberger, Daniel; Scheibel, Willy; Hahn, Sebastian; Döllner, Jürgen in Proceedings of the 8th International Conference on Information Visualization Theory and Applications (2017).
Depicting massive software system data using software maps can result in visual clutter and increased cognitive load. This paper introduces an adaptive level-of-detail (LoD) technique that uses scoring for interactive aggregation on a per-node basis. The scoring approximates importance by degree-of-interest measures as well as screen and user-interaction scores. The technique adheres to established aggregation guidelines and was evaluated by means of two user studies. The first user study investigates task completion time in visual search. The second evaluates the readability of the presented nesting level contouring for aggregates. With the adap- tive LoD technique software maps allow for multi-resolution depictions of software system information. It facilitates efficient identification of important nodes and allows for additional annotation. © The Authors 2017. This is the authors' version of the work. It is posted here for your personal use. Not for redistribution. The definitive version will be published in Proceedings of the 8th International Conference on Information Visualization Theory and Applications (IVAPP 2017).
-
Relative Direction Change: A Topology-based Metric for Layout Stability in Treemaps. Hahn, Sebastian; Bethge, Joseph; Döllner, Jürgen (2017).
This paper presents a topology-based metric for layout stability in treemaps—the Relative Direction Change (RDC). The presented metric considers the adjacency and arrangement of single shapes in a treemap, and allows for a rotation-invariant description of layout changes between two snapshots of a dataset depicted with treemaps. A user study was conducted that shows the applicability of the Relative Direction Change in comparison and addition to established layout metrics, such as Average Distance Change (ADC) and Average Aspect Ratio (AAR), with respect to human perception of treemaps. This work contributes to the establishment of a more precise model for the replicable and reliable comparison of treemap layout algorithms © The Authors 2017. This is the authors' version of the work. It is posted here for your personal use. Not for redistribution. The definitive version will be published in Proceedings of the 8th International Conference on Information Visualization Theory and Applications (IVAPP 2017).
-
Towards Architectural Styles for Android App Software Product Lines. Dürschmid, Tobias; Trapp, Matthias; Döllner, Jürgen (2017).
-
Vom Fotorealismus zum Nichtfotorealismus. Döllner, Jürgen in Hands on: Kunstgeschichte: Methodik und Unterrichtsbeispiele der gestaltungspraktischen Kunstrezeption, J. Penzel (ed.) (2017).
-
BeCasso: Artistic Image Processing and Editing on Mobile Devices. Pasewaldt, Sebastian; Semmo, Amir; Döllner, Jürgen; Schlegel, Frank (2016).
BeCasso is a mobile app that enables users to transform photos into high-quality, high-resolution non-photorealistic renditions, such as oil and watercolor paintings, cartoons, and colored pencil drawings, which are inspired by real-world paintings or drawing techniques. In contrast to neuronal network and physically-based approaches, the app employs state-of-the-art nonlinear image filtering. For example, oil paint and cartoon effects are based on smoothed structure information to interactively synthesize renderings with soft color transitions. BeCasso empowers users to easily create aesthetic renderings by implementing a two-fold strategy: First, it provides parameter presets that may serve as a starting point for a custom stylization based on global parameter adjustments. Thereby, users can obtain initial renditions that may be fine-tuned afterwards. Second, it enables local style adjustments: using on-screen painting metaphors, users are able to locally adjust different stylization features, e.g., to vary the level of abstraction, pen, brush and stroke direction or the contour lines. In this way, the app provides tools for both higher-level interaction and low-level control [Isenberg 2016] to serve the different needs of non-experts and digital artists. References: Isenberg, T. 2016. Interactive NPAR: What Type of Tools Should We Create? In Proc. NPAR, The Eurographics Association, Goslar, Germany, 89–96
-
BeCasso: Image Stylization by Interactive Oil Paint Filtering on Mobile Devices. Semmo, Amir; Döllner, Jürgen; Schlegel, Frank (2016).
BeCasso is a mobile app that enables users to transform photos into an oil paint look that is inspired by traditional painting elements. In contrast to stroke-based approaches, the app uses state-of-the-art nonlinear image filtering techniques based on smoothed structure information to interactively synthesize oil paint renderings with soft color transitions. BeCasso empowers users to easily create aesthetic oil paint renderings by implementing a two-fold strategy. First, it provides parameter presets that may serve as a starting point for a custom stylization based on global parameter adjustments. Second, it introduces a novel interaction approach that operates within the parameter spaces of the stylization effect to facilitate creative control over the visual output: on-screen painting enables users to locally adjust the appearance in image regions, e.g., to vary the level of abstraction, brush and stroke direction. This way, the app provides tools for both higher-level interaction and low-level control [Isenberg 2016] to serve the different needs of non-experts and digital artists. References: Isenberg, T. 2016. Interactive NPAR: What Type of Tools Should We Create? In Proc. NPAR, The Eurographics Association, Goslar, Germany, 89–96
-
Design and Implementation of Non-Photorealistic Rendering Techniques for 3D Geospatial Data. Technical Report (PhD dissertation), Semmo, Amir (2016).
Geospatial data has become a natural part of a growing number of information systems and services in the economy, society, and people's personal lives. In particular, virtual 3D city and landscape models constitute valuable information sources within a wide variety of applications such as urban planning, navigation, tourist information, and disaster management. Today, these models are often visualized in detail to provide realistic imagery. However, a photorealistic rendering does not automatically lead to high image quality, with respect to an effective information transfer, which requires important or prioritized information to be interactively highlighted in a context-dependent manner. Approaches in non-photorealistic renderings particularly consider a user's task and camera perspective when attempting optimal expression, recognition, and communication of important or prioritized information. However, the design and implementation of non-photorealistic rendering techniques for 3D geospatial data pose a number of challenges, especially when inherently complex geometry, appearance, and thematic data must be processed interactively. Hence, a promising technical foundation is established by the programmable and parallel computing architecture of graphics processing units. This thesis proposes non-photorealistic rendering techniques that enable both the computation and selection of the abstraction level of 3D geospatial model contents according to user interaction and dynamically changing thematic information. To achieve this goal, the techniques integrate with hardware-accelerated rendering pipelines using shader technologies of graphics processing units for real-time image synthesis. The techniques employ principles of artistic rendering, cartographic generalization, and 3D semiotics—unlike photorealistic rendering—to synthesize illustrative renditions of geospatial feature type entities such as water surfaces, buildings, and infrastructure networks. In addition, this thesis contributes a generic system that enables to integrate different graphic styles—photorealistic and non-photorealistic—and provide their seamless transition according to user tasks, camera view, and image resolution. Evaluations of the proposed techniques have demonstrated their significance to the field of geospatial information visualization including topics such as spatial perception, cognition, and mapping. In addition, the applications in illustrative and focus+context visualization have reflected their potential impact on optimizing the information transfer regarding factors such as cognitive load, integration of non-realistic information, visualization of uncertainty, and visualization on small displays.
-
Dynamic 2.5D Treemaps using Declarative 3D on the Web. Limberger, Daniel; Scheibel, Willy; Lemme, Stefan; Jürgen, Döllner (2016). 33--36.
The 2.5D treemap represents a general purpose visualization technique to map multi-variate hierarchical data in a scalable, interactive, and consistent way used in a number of application fields. In this paper, we explore the capabilities of Declarative 3D for the web-based implementation of 2.5D treemap clients. Particularly, we investigate how X3DOM and XML3D can be used to implement clients with equivalent features that interactively display 2.5D treemaps with dynamic mapping of attributes. We also show a first step towards a glTF-based implementation. These approaches are benchmarked focusing on their interaction capabilities with respect to rendering and speed of dynamic data mapping. We discuss the results for our representative example of a complex 3D interactive visualization technique and summerize recommendations for improvements towards operational web clients.
-
Dynamic 2.5D Treemaps using Declarative 3D on the Web. Limberger, Daniel; Scheibel, Willy; Lemme, Stefan; Jürgen, Döllner (2016). 33--36.
The 2.5D treemap represents a general purpose visualization technique to map multi-variate hierarchical data in a scalable, interactive, and consistent way used in a number of application fields. In this paper, we explore the capabilities of Declarative 3D for the web-based implementation of 2.5D treemap clients. Particularly, we investigate how X3DOM and XML3D can be used to implement clients with equivalent features that interactively display 2.5D treemaps with dynamic mapping of attributes. We also show a first step towards a glTF-based implementation. These approaches are benchmarked focusing on their interaction capabilities with respect to rendering and speed of dynamic data mapping. We discuss the results for our representative example of a complex 3D interactive visualization technique and summerize recommendations for improvements towards operational web clients.
-
Evaluation of Sketchiness as a Visual Variable for 2.5D Treemaps. Limberger, Daniel; Fiedler, Carolin; Hahn, Sebastian; Trapp, Matthias; Döllner, Jürgen (2016).
Interactive 2.5D treemaps serve as an effective tool for the visualization of attributed hierarchies, enabling exploration of non-spatial, multi-variate, hierarchical data. In this paper the suitability of sketchiness as a visual variable, e.g., for uncertainty, is evaluated. Therefore, a design space for sketchy rendering in 2.5D and integration details for real-time applications are presented. The results of three user studies indicate, that sketchiness is a promising candidate for a visual variable that can be used independently and in addition to others, e.g., color and height. © The Authors 2016. This is the authors' version of the work. It is posted here for your personal use. Not for redistribution. The definitive version will be published in Proceedings of the 20th International Conference on Information Visualization (IV'16).
-
Image Stylization by Interactive Oil Paint Filtering. Semmo, Amir; Limberger, Daniel; Kyprianidis, Jan Eric; Döllner, Jürgen in Computers & Graphics (2016). 55 157--171.
This paper presents an interactive system for transforming images into an oil paint look. The system comprises two major stages. First, it derives dominant colors from an input image for feature-aware recolorization and quantization to conform with a global color palette. Afterwards, it employs non-linear filtering based on the smoothed structure adapted to the main feature contours of the quantized image to synthesize a paint texture in real-time. Our filtering approach leads to homogeneous outputs in the color domain and enables creative control over the visual output, such as color adjustments and per-pixel parametrizations by means of interactive painting. To this end, our system introduces a generalized brush-based painting interface that operates within parameter spaces to locally adjust the level of abstraction of the filtering effects. Several results demonstrate the various applications of our filtering approach to different genres of photography.
-
Interactive and View-Dependent See-Through Lenses for Massive 3D Point Clouds. Discher, Sören; Richter, Rico; Döllner, Jürgen (2016).
-
Interactive GPU-based Image Deformation for Mobile Devices. Vollmer, Jan Ole; Trapp, Matthias; Döllner, Jürgen (2016).
Interactive image deformation is an important feature of modern image processing pipelines. It is often used to create caricatures and animation for input images, especially photos. State-of-the-art image deformation techniques are based on transforming vertices of a mesh, which is textured by the input image, using affine transformations such as translation, and scaling. However, the resulting visual quality of the output image depends on the geometric resolution of the mesh. Performing these transformations on the CPU often further inhibits performance and quality. This is especially problematic on mobile devices where the limited computational power reduces the maximum achievable quality. To overcome these issue, we propose the concept of an intermediate deformation buffer that stores deformation information at a resolution independent of the mesh resolution. This allows the combination of a high-resolution buffer with a low-resolution mesh for interactive preview, as well as a high-resolution mesh to export the final image. Further, we present a fully GPU-based implementation of this concept, taking advantage of modern OpenGL ES features, such as compute shaders.
-
Interactive Image Filtering with Multiple Levels-of-Control on Mobile Devices. Semmo, Amir; Dürschmid, Tobias; Trapp, Matthias; Klingbeil, Mandy; Döllner, Jürgen; Pasewaldt, Sebastian (2016).
With the continuous development of mobile graphics hardware, interactive high-quality image stylization based on nonlinear filtering is becoming feasible and increasingly used in casual creativity apps. However, these apps often only serve high-level controls to parameterize image filters and generally lack support for low-level (artistic) control, thus automating art creation rather than assisting it. This work presents a GPU-based framework that enables to parameterize image filters at three levels of control: (1) presets followed by (2) global parameter adjustments can be interactively refined by (3) complementary on-screen painting that operates within the filters' parameter spaces for local adjustments. The framework provides a modular XML-based effect scheme to effectively build complex image processing chains-using these interactive filters as building blocks-that can be efficiently processed on mobile devices. Thereby, global and local parameterizations are directed with higher-level algorithmic support to ease the interactive editing process, which is demonstrated by state-of-the-art stylization effects, such as oil paint filtering and watercolor rendering.
-
Interactive Multi-scale Oil Paint Filtering on Mobile Devices. Semmo, Amir; Trapp, Matthias; Dürschmid, Tobias; Döllner, Jürgen; Pasewaldt, Sebastian (2016).
This work presents an interactive mobile implementation of a filter that transforms images into an oil paint look. At this, a multi-scale approach that processes image pyramids is introduced that uses flow-based joint bilateral upsampling to achieve deliberate levels of abstraction at multiple scales and interactive frame rates. The approach facilitates the implementation of interactive tools that adjust the appearance of filtering effects at run-time, which is demonstrated by an on-screen painting interface for per-pixel parameterization that fosters the casual creativity of non-artists.
-
Interactive Oil Paint Filtering On Mobile Devices. Semmo, Amir; Trapp, Matthias; Pasewaldt, Sebastian; Döllner, Jürgen (2016).
Image stylization enjoys a growing popularity on mobile devices to foster casual creativity. However, the implementation and provision of high-quality image filters for artistic rendering is still faced by the inherent limitations of mobile graphics hardware such as computing power and memory resources. This work presents a mobile implementation of a filter that transforms images into an oil paint look, thereby highlighting concepts and techniques on how to perform multi-stage nonlinear image filtering on mobile devices. The proposed implementation is based on OpenGL ES and the OpenGL ES shading language, and supports on-screen painting to interactively adjust the appearance in local image regions, e.g., to vary the level of abstraction, brush, and stroke direction. Evaluations of the implementation indicate interactive performance and results that are of similar aesthetic quality than its original desktop variant.
-
Interactive Revision Exploration using Small Multiples of Software Maps. Scheibel, Willy; Trapp, Matthias; Döllner, Jürgen in IVAPP 2016 (2016). 131-138.
To explore and to compare different revisions of complex software systems is a challenging task as it requires to constantly switch between different revisions and the corresponding information visualization. This paper proposes to combine the concept of small multiples and focus+context techniques for software maps to facilitate the comparison of multiple software map themes and revisions simultaneously on a single screen. This approach reduces the amount of switches and helps to preserve the mental map of the user. Given a software project the small multiples are based on a common dataset but are specialized by specific revisions and themes. The small multiples are arranged in a matrix where rows and columns represents different themes and revisions, respectively. To ensure scalability of the visualization technique we also discuss two rendering pipelines to ensure interactive frame-rates. The capabilities of the proposed visualization technique are demonstrated in a collaborative exploration setting using a high-resolution, multi-touch display.
-
Interactive Web-based Visualization for Accessibility Mapping of Transportation Networks. Schoedon, Alexander; Trapp, Matthias; Hollburg, Henning; Döllner, Jürgen (2016).
Accessibility is a fundamental aspect in transportation, routing, and spare-time activity planning concerning traveling in modern cities. In this context, interactive web-based accessibility-map visualization techniques and systems are important tools for provisioning, exploration, analysis, and assessment of multi-modal and location-based travel time data and routing information. To enable their effective application, such interactive visualization techniques demands for flexible mappings with respect to user-adjustable parameters such as maximum travel times, the types of transportation used, or used color schemes. However, traditional approaches for web-based visualization of accessibility-maps do not allow this degree of parametrization without significant latencies introduced by required data processing and transmission between the routing server and the visualization client. This paper presents a novel web-based visualization technique that allows for efficient client-side mapping and rendering of accessibility data onto transportation networks using WebGL and the OpenGL transmission format. A performance evaluation and comparison shows the superior performance of the approach over alternative implementations.
-
Konzepte und Techniken zur servicebasierten Visualisierung von geovirtuellen 3D-Umgebungen. Technical Report (PhD dissertation), Hagedorn, Benjamin PhD thesis, Potsdam. (2016).
Im Allgemeinen liegen Geodaten und georeferenzierte Daten verteilt vor, sind heterogen in ihrem Inhalt und ihrer Form, umfassen sehr große Datenmengen und müssen in verschiedenen IT-Informationssystemen integriert und in verschiedenen Anwendungskontexten genutzt werden. Im Mittelpunkt dieser Arbeit stehen deshalb Konzepte und Techniken für die Integration, Visualisierung, Analyse, Bereitstellung und Nutzung von 2D- und 3D-Geodaten sowie georeferenzierten Daten. Hierbei wird ein Ansatz verfolgt, der zum einen virtuelle 3D-Umgebungen als konzeptionellen und technischen Rahmen nutzt und zum anderen auf serviceorientierten Softwarearchitekturen und Geo-Standards basiert. Die vorgestellten Konzepte und Verfahren stellen damit Schlüsselbausteine dar, um neuartige IT-Lösungen und Anwendungen für 3D-Geoinformationen, z.B. als Bestandteile von Geodateninfrastrukturen, zu realisieren. Im Bereich der servicebasierten 3DGeovisualisierung beschreibt diese Arbeit, wie virtuelle 3D-Stadtmodelle für die Integration heterogener und verteilter Geodatenquellen genutzt werden können. Dazu werden Anforderungen für die Integration identifiziert und ein Konzept für die Integration auf Datenebene und die Integration auf Visualisierungsebene entworfen und deren Umsetzung am Beispiel komplexer 3D-Bauwerksinformationsmodelle beschrieben und demonstriert. Im Bereich servicebasierter, bildbasierter 3DPortrayal-Services wird mit dem Web View Service (WVS) ein spezialisierter SoftwareService für die Visualisierung, Interaktion und Analyse von geovirtuellen 3D-Umgebungen konzipiert und entwickelt. Kernkonzept dieses Services sind die serverseitige Datenintegration und -verwaltung sowie die ebenfalls serverseitige Bilderzeugung. Mit diesem durchgehend serverseitigen Ansatz können sehr große Mengen an 3D-Geodaten auch auf solchen Endgeräte bereitgestellt werden, die nicht über ausreichend Speicher und Rechenleistung für die Vorhaltung, die Verarbeitung und das Rendering von 3D-Modellen verfügen. Der entwickelte WVS ermöglicht so u.a. auch auf Tablet-PCs und im Webbrowser die interaktive Erkundung und Analyse von 3D-Geodaten. Anhand einer Referenzimplementierung des WVS und einer Client-Anwendung wird die praktische Anwendung des WVS demonstriert. Im Bereich der Komposition von Web View Services wird untersucht, wie der WVS als Baustein einer komplexen, verteilten Visualisierungs- und Renderingpipeline eingesetzt werden kann. Durch die Komposition des WVS mit anderen Darstellungs- und Verarbeitungsservices können z. B. komplexe Renderingeffekte erzielt oder einzelne 3D-Objekte nachträglich in eine 3D-Ansicht integriert werden. Hierzu wird ein Konzept zur servicebasierten, Tiefenbildbasierten Bildkomposition beschrieben und am Beispiel der verdeckungsfreien Annotation von 3D-Ansichten umgesetzt und demonstriert. Im Bereich der Interaktion mit Web View Services liefert diese Arbeit Grundlagen, Konzept und Umsetzung für intelligente, assistierende und automatisierte Interaktions- und Navigationstechniken, die auf dem Angebotscharakter (der Affordanz) von 3D-Szenenobjekten sowie auf der skizzen- und gestenbasierten Eingabe von Nutzerintentionen basiert. Diese Eingaben werden hinsichtlich ihrer Form und unter Berücksichtigung der Semantik der 3DSzenenobjekte ausgewertet und interpretiert und anschließend in anwendungsspezifische Navigationskommandos übersetzt, aus denen teilautomatische Kamerafahrten abgeleitet werden.
-
Animated visualization of spatial-temporal trajectory data for air-traffic analysis. Buschmann, Stefan; Trapp, Matthias; Döllner, Jürgen in The Visual Computer (2015). 32(3) 371-381.
With increasing numbers of flights worldwide and a continuing rise in airport traffic, air-traffic management is faced with a number of challenges. These include monitoring, reporting, planning, and problem analysis of past and current air traffic, e.g., to identify hotspots, minimize delays, or to optimize sector assignments to air-traffic controllers. To cope with these challenges, cyber worlds can be used for interactive visual analysis and analytical reasoning based on aircraft trajectory data. However, with growing data size and complexity, visualization requires high computational efficiency to process that data within real-time constraints. This paper presents a technique for real-time animated visualization of massive trajectory data. It enables (1) interactive spatio-temporal filtering, (2) generic mapping of trajectory attributes to geometric representations and appearance, and (3) real-time rendering within 3D virtual environments such as virtual 3D airport or 3D city models. Different visualization metaphors can be efficiently built upon this technique such as temporal focus+context, density maps, or overview+detail methods. As a general-purpose visualization technique, it can be applied to general 3D and 3+1D trajectory data, e.g., traffic movement data, geo-referenced networks, or spatio-temporal data, and it supports related visual analytics and data mining tasks within cyber worlds.
-
Automatic Detection and Large-Scale Visualization of Trees for Digital Landscapes and City Models based on 3D Point Clouds. Oehlke, Christoph; Richter, Rico; Döllner, Jürgen (2015). 151-160.
-
Cartography-Oriented Design of 3D Geospatial Information Visualization - Overview and Techniques. Semmo, Amir; Trapp, Matthias; Jobst, Markus; Döllner, Jürgen in The Cartographic Journal (2015). 52(2) 95--106.
In economy, society and personal life map-based, interactive geospatial visualization becomes a natural element of a growing number of applications and systems. The visualization of 3D geospatial information, however, raises the question how to represent the information in an effective way. Considerable research has been done in technology-driven directions in the fields of cartography and computer graphics (e.g., design principles, visualization techniques). Here, non-photorealistic rendering represents a promising visualization category–situated between both fields–that offers a large number of degrees for the cartography-oriented visual design of complex 2D and 3D geospatial information for a given application context. Still today, however, specifications and techniques for mapping cartographic design principles to the state-of-the-art rendering pipeline of 3D computer graphics remain to be explored. This paper revisits cartographic design principles for 3D geospatial visualization and introduces an extended 3D semiotic model that complies with the general, interactive visualization pipeline. Based on this model, we propose non-photorealistic rendering techniques to interactively synthesize cartographic renditions of basic feature types, such as terrain, water, and buildings. In particular, it includes a novel iconification concept to seamlessly interpolate between photorealistic and cartographic representations of 3D landmarks. Our work concludes with a discussion of open challenges in this field of research, including topics such as user interaction and evaluation.
-
Echtzeit-Rendering-Techniken für 3D-Punktwolken basierend auf semantischen und topologischen Attributen. Discher,; Sören, (2015).
-
Geometry Batching Using Texture-Arrays. Trapp, Matthias; Döllner, Jürgen (2015). 239-246.
High-quality rendering of 3D virtual environments typically depends on high-quality 3D models with significant geometric complexity and texture data. One major bottleneck for real-time image-synthesis represents the number of state changes, which a specific rendering API has to perform. To improve performance, batching can be used to group and sort geometric primitives into batches to reduce the number of required state changes, whereas the size of the batches determines the number of required draw-calls, and therefore, is critical for rendering performance. For example, in the case of texture atlases, which provide an approach for efficient texture management, the batch size is limited by the efficiency of the texture-packing algorithm and the texture resolution itself. This paper presents a pre-processing approach and rendering technique that overcomes these limitations by further grouping textures or texture atlases and thus enables the creation of larger geometry batches. It is based on texture arrays in combination with an additional indexing schema that is evaluated at run-time using shader programs. This type of texture management is especially suitable for real-time rendering of large-scale texture-rich 3D virtual environments, such as virtual city and landscape models.
-
Image Stylization by Oil Paint Filtering using Color Palettes. Semmo, Amir; Limberger, Daniel; Kyprianidis, Jan Eric; Döllner, Jürgen (2015). 149--158.
This paper presents an approach for transforming images into an oil paint look. To this end, a color quantization scheme is proposed that performs feature-aware recolorization using the dominant colors of the input image. In addition, an approach for real-time computation of paint textures is presented that builds on the smoothed structure adapted to the main feature contours of the quantized image. Our stylization technique leads to homogeneous outputs in the color domain and enables creative control over the visual output, such as color adjustments and per-pixel parametrizations by means of interactive painting. © The Authors 2015. This is the authors' version of the work. It is posted here for your personal use. Not for redistribution. The definitive version will be published in Proceedings of the International Symposium on Computational Aesthetics in Graphics, Visualization, and Imaging (CAe'15).
-
Interactive Image Filtering for Level-of-Abstraction Texturing of Virtual 3D Scenes. Semmo, Amir; Döllner, Jürgen in Computers & Graphics (2015). 52 181--198.
Texture mapping is a key technology in computer graphics. For the visual design of 3D scenes, in particular, effective texturing depends significantly on how important contents are expressed, e.g., by preserving global salient structures, and how their depiction is cognitively processed by the user in an application context. Edge-preserving image filtering is one key approach to address these concerns. Much research has focused on applying image filters in a post-process stage to generate artistically stylized depictions. However, these approaches generally do not preserve depth cues, which are important for the perception of 3D visualization (e.g., texture gradient). To this end, filtering is required that processes texture data coherently with respect to linear perspective and spatial relationships. In this work, we present an approach for texturing 3D scenes with perspective coherence by arbitrary image filters. We propose decoupled deferred texturing with (1) caching strategies to interactively perform image filtering prior to texture mapping and (2) for each mipmap level separately to enable a progressive level of abstraction, using (3) direct interaction interfaces to parameterize the visualization according to spatial, semantic, and thematic data. We demonstrate the potentials of our method by several applications using touch or natural language inputs to serve the different interests of users in specific information, including illustrative visualization, focus+context visualization, geometric detail removal, and semantic depth of field. The approach supports frame-to-frame coherence, order-independent transparency, multitexturing, and content-based filtering. In addition, it seamlessly integrates into real-time rendering pipelines, and is extensible for custom interaction techniques.
-
Interactive Rendering and Stylization of Transportation Networks Using Distance Fields. Trapp, Matthias; Semmo, Amir; Döllner, Jürgen (2015). 207-219.
Transportation networks, such as streets, railroads or metro systems, constitute primary elements in cartography for reckoning and navigation. In recent years, they have become an increasingly important part of 3D virtual environments for the interactive analysis and communication of complex hierarchical information, for example in routing, logistics optimization, and disaster management. A variety of rendering techniques have been proposed that deal with integrating transportation networks within these environments, but have so far neglected the many challenges of an interactive design process to adapt their spatial and thematic granularity (i.e., level-of-detail and level-of-abstraction) according to a user's context. This paper presents an efficient real-time rendering technique for the view-dependent rendering of geometrically complex transportation networks within 3D virtual environments. Our technique is based on distance fields using deferred texturing that shifts the design process to the shading stage for real-time stylization. We demonstrate and discuss our approach by means of street networks using cartographic design principles for context-aware stylization, including view-dependent scaling for clutter reduction, contour-lining to provide figure-ground, handling of street crossings via shading-based blending, and task-dependent colorization. Finally, we present potential usage scenarios and applications together with a performance evaluation of our implementation.
-
Natural Phenomena as Metaphors for Visualization of Trend Data in Interactive Software Maps. Würfel, Hannes; Trapp, Matthias; Limberger, Daniel; Döllner, Jürgen (2015).
Software maps are a commonly used tool for code quality monitoring in software-development projects and decision making processes. While providing an important visualization technique for the hierarchical system structure of a single software revision, they lack capabilities with respect to the visualization of changes over multiple revisions. This paper presents a novel technique for visualizing the evolution of the software system structure based on software metric trends. These trend maps extend software maps by using real-time rendering techniques for natural phenomena yielding additional visual variables that can be effectively used for the communication of changes. Therefore, trend data is automatically computed by hierarchically aggregating software metrics. We demonstrate and discuss the presented technique using two real world data sets of complex software systems.
-
Out-of-Core Visualization of Classified 3D Point Clouds. Richter, Rico; Discher, Sören; Döllner, Jürgen (2015). 227-242.
3D point clouds represent an essential category of geodata used in a variety of geoinformation applications and systems. We present a novel, interactive out-of-core rendering technique for massive 3D point clouds based on a layered, multi-resolution kd-tree, whereby point-based rendering techniques are selected according to each point's classification (e.g., vegetation, buildings, terrain). The classification-dependent rendering leads to an improved visual representation, enhances recognition of objects within 3D point cloud depictions, and facilitates visual filtering and highlighting. To interactively explore objects, structures, and relations represented by 3D point clouds, our technique provides efficient means for an instantaneous, ad-hoc visualization compared to approaches that visualize 3D point clouds by deriving mesh-based 3D models. We have evaluated our approach for massive laser scan datasets of urban areas. The results show the scalability of the technique and how different configurations allow for designing task and domain-specific analysis and inspection tools. © The Authors 2014. This is the authors' version of the work. It is posted here for your personal use. Not for redistribution. The definitive version will be published in 3D Geoinformation Science: The Selected Papers of the 3D GeoInfo 2014 by Springer International Publishing. http://dx.doi.org/10.1007/978-3-319-12181-9.
-
Real-Time Visualization of Massive Movement Data in Digital Landscapes. Buschmann, Stefan; Trapp, Matthias; Döllner, Jürgen (2015). 213-220.
Due to continuing advances in sensor technology and increasing availability of digital infrastructure that allows for acquisition, transfer, and storage of big data sets, large amounts of movement data (e.g., road, naval, or air-traffic) become available. In the near future, movement data such as traffic data may even be available in real-time. In a growing number of application fields (e.g., landscape planning and design, urban development, and infrastructure planning), movement data enables new analysis and simulation applications. In this paper, we present an interactive technique for visualizing massive 3D movement trajectories. It is based on mapping massive movement data to graphics primitives and their visual variables in real-time, supporting a number of visualization schemes such as sphere, line, or tube-based trajectories, including animations of direction and speed. This generic technique enhances the functionality of VR and interactive 3D systems using virtual environments such as digital landscape models, city models, or virtual globes by adding support for this important category of spatio-temporal data.
-
Review: visual analytics of climate networks. Nocke, Thomas; Buschmann, Stefan; Donges, Jonathan; Marwan, Norbert; Schulz, Hans-Jörg; Tominski, Christian in Nonlinear Processes in Geophysics (2015). 22(5) 545-570.
Network analysis has become an important approach in studying complex spatiotemporal behaviour within geophysical observation and simulation data. This new field produces increasing amounts of large geo-referenced networks to be analysed. Particular focus lies currently on the network analysis of the complex statistical interrelationship structure within climatological fields. The standard procedure for such network analyses is the extraction of network measures in combination with static standard visualisation methods. Existing interactive visualisation methods and tools for geo-referenced network exploration are often either not known to the analyst or their potential is not fully exploited. To fill this gap, we illustrate how interactive visual analytics methods in combination with geovisualisation can be tailored for visual climate network investigation. Therefore, the paper provides a problem analysis, relating the multiple visualisation challenges with a survey undertaken with network analysts from the research fields of climate and complex systems science. Then, as an overview for the interested practitioner, we review the state-of-the-art in climate network visualisation and provide an overview of existing tools. As a further contribution, we introduce the visual network analytics tools CGV and GTX, providing tailored solutions for climate network analysis, including alternative geographic projections, edge bundling, and 3-D network support. Using these tools, the paper illustrates the application potentials of visual analytics for climate networks based on several use cases including examples from global, regional, and multi-layered climate networks.
-
ThreadCity: Combined Visualization of Structure and Activity for the Exploration of Multi-threaded Software Systems. Hahn, Sebastian; Trapp, Matthias; Wuttke, Nikolai; Döllner, Jürgen (2015).
This paper presents a novel visualization technique for the interactive exploration of multi-threaded software systems. It combines the visualization of static system structure based on the EvoStreets approach with an additional traffic metaphor to communicate the runtime characteristics of multiple threads simultaneously. To improve visual scalability with respect to the visualization of complex software systems, we further present an effective level-of-detail visualization based on hierarchical aggregation of system components by taking viewing parameters into account. We demonstrate our technique by means of a prototypical implementation and compare our result with existing visualization techniques. © The Authors 2015. This is the authors' version of the work. It is posted here for your personal use. Not for redistribution. The definitive version will be published in Proceedings of the 19th International Conference on Information Visualization (IV'15).
-
VideoMR: A Map and Reduce Framework for Real-time Video Processing. Meier, Benjamin-Heinz; Trapp, Matthias; Döllner, Jürgen (2015).
This paper presents VideoMR: a novel map and reduce framework for real-time video processing on graphic processing units (GPUs). Using the advantages of implicit parallelism and bounded memory allocation, our approach enables developers to focus on implementing video operations without taking care of GPU memory handling or the details of code parallelization. Therefore, a new concept for map and reduce is introduced, redefining both operations to fit to the specific requirements of video processing. A prototypical implementation using OpenGL facilitates various operating platforms, including mobile development, and will be widely interoperable with other state-of-the-art video processing frameworks.
-
2D and 3D Presentation of Spatial Data: A Systematic Review. Dübel, Steve; Röhlig, Martin; Schumann, Heidrun; Trapp, Matthias (2014).
The question whether to use 2D or 3D for data visualization is generally difficult to decide. Two-dimensional and three-dimensional visualization techniques exhibit different advantages and disadvantages related to various perceptual and technical aspects such as occlusion, clutter, distortion, or scalability. To facilitate problem understanding and comparison of existing visualization techniques with regard to these aspects, this report introduces a systematization based on presentation characteristics. It enables a categorization with respect to combinations of static 2D and 3D presentations of attributes and their spatial reference. Further, it complements ex-isting systematizations of data in an effort to formalize a common terminology and theoretical framework for this problem domain. We demonstrate our approach by reviewing different visualization techniques of spatial data according to the presented systematization.
-
An Interaction Framework for Level-of-Abstraction Visualization of 3D Geovirtual Environments. Semmo, Amir; Döllner, Jürgen (2014). 43--49.
3D geovirtual environments constitute effective media for the analysis and communication of complex geospatial data. Today, these environments are often visualized using static graphical variants (e.g., 2D maps, 3D photorealistic) from which a user is able to choose from. To serve the different interests of users in specific information, however, the spatial and thematic granularity at which model contents are represented (i.e., level of abstraction) should be dynamically adapted to the user's context, which requires specialized interaction techniques for parameterization. In this work, we present a framework that enables interaction interfaces to parameterize the level-of-abstraction visualization according to spatial, semantic, and thematic data. The framework is implemented in a visualization system that provides image-based rendering techniques for context-aware abstraction and highlighting. Using touch and natural language interfaces, we demonstrate its versatile application to geospatial tasks, including exploration, navigation, and orientation.
-
Hardware-Accelerated Attribute Mapping for Interactive Visualization of Complex 3D Trajectories. Buschmann, Stefan; Trapp, Matthias; Lühne, Patrick; Döllner, Jürgen (2014). 355-363.
The visualization of 3D trajectories of moving objects and related attributes in 3D virtual environments represents a fundamental functionality in various visualization domains. Interactive rendering and visual analytics of such attributed trajectories involves both conceptual questions as well as technical challenges. Specifically, the mapping of trajectory attributes to rendering primitives and appearance represents a challenging task in the case of large data sets of high geometric complexity. There are various visualization approaches and rendering techniques considering specific aspects of these mappings to facilitate visualization and analysis of this kind of data. To solve the underlying general mapping problem efficiently, we developed an approach that uses and combines diverse types of visualizations, rather than being tailored to a specific use case. This paper describes an interactive rendering system for the visualization of 3D trajectories that enables the combinations of different mappings as well as their dynamic configuration at runtime. A fully hardware-accelerated implementation enables the processing of large sets of attributed 3D trajectories in real-time.
-
High-quality video generation for thin clients - An application for image-based 3D portrayal services. Klimke, Jan (2014). (Vol. 83)
-
Image Filtering for Interactive Level-of-Abstraction Visualization of 3D Scenes. Semmo, Amir; Döllner, Jürgen (2014). 5--14.
Texture mapping is a key technology in computer graphics for visual design of rendered 3D scenes. An effective information transfer of surface properties, encoded by textures, however, depends significantly on how important information is highlighted and cognitively processed by the user in an application context. Edge-preserving image filtering is a promising approach to address this concern while preserving global salient structures. Much research has focused on applying image filters in a post-process stage to foster an artistically stylized rendering, but these approaches are generally not able to preserve depth cues important for 3D visualization (e.g., texture gradient). To this end, filtering that processes texture data coherently with respect to linear perspective and spatial relationships is required. In this work, we present a system that enables to process textured 3D scenes with perspective coherence by arbitrary image filters. We propose decoupled deferred texturing with (1) caching strategies to interactively perform image filtering prior to texture mapping, and (2) for each mipmap level separately to enable a progressive level of abstraction. We demonstrate the potentials of our methods on several applications, including illustrative visualization, focus+context visualization, geometric detail removal, and depth of field. Our system supports frame-to-frame coherence, order-independent transparency, multitexturing, and content-based filtering. © The Authors 2014. This is the authors' version of the work. It is posted here for your personal use. Not for redistribution. The definitive version will be published in Proceedings of the International Symposium on Computational Aesthetics in Graphics, Visualization, and Imaging (CAe'14). http://dx.doi.org/10.1145/2630099.2630101.
-
Konzepte für eine Service-basierte Systemarchitektur zur Integration, Prozessierung und Analyse von massiven 3D-Punktwolken. Discher, Sören; Richter, Rico; Döllner, Jürgen (2014).
Die Nutzung von hoch aufgelösten, räumlich überlappenden und multitemporalen 3D-Punktwolken im Kontext von Geoinformationssystemen stellt hohe Anforderungen an die Leistungsfähigkeit der zugrundeliegenden Software- und Hardwaresysteme. Um angesichts eines weiter zunehmenden Datenaufkommens ein effizientes und wirtschaftliches Arbeiten mit solchen Daten zu ermöglichen, schlagen wir die Nutzung einer service-basierten Software- und Geodateninfrastruktur vor, die eine Erfassung, Aktualisierung und Bereitstellung von 3D-Punktwolken im Sinne eines kontinuierlichen Prozesses ermöglicht. In diesem Beitrag erläutern wir die grundlegenden Anforderungen und den konzeptionellen Aufbau einer entsprechenden Infrastruktur, die unter anderem die bedarfsgerechte Bereitstellung ausgewählter Bereiche einer 3D Punktwolke anhand von semantischen oder temporalen Attributen unterstützt.
-
Multi-Perspective 3D Panoramas. Pasewaldt, Sebastian; Semmo, Amir; Trapp, Matthias; Döllner, Jürgen in International Journal of Geographical Information Science (IJGIS) (2014). 28(10) 2030-2051.
This article presents multi-perspective 3D panoramas that focus on visualizing 3D geovirtual environments (3D GeoVEs) for navigation and exploration tasks. Their key element, a multi-perspective view, seamlessly combines what is seen from multiple viewpoints into a single image. This approach facilitates thepresentation of information for virtual 3D city and landscape models, particularly by reducing occlusions, increasing screen-space utilization, and providing additional context within a single image. We complement multi-perspective views with cartographic visualization techniques to stylize features according to their semantics and highlight important or prioritized information. When combined, both techniques constitute the core implementation of interactive, multi-perspective 3D panoramas. They offer a large number of effective means for visual communication of 3D spatial information, a high degree of customization with respect to cartographic design, and manifold applications in different domains. We discuss design decisions of 3D panoramas for the exploration of and navigation in 3D GeoVEs. We also discuss a preliminary user study that indicates that 3D panoramas are a promising approach for navigation systems using 3D GeoVEs.
-
Nichtfotorealistische Visualisierung virtueller 3D-Stadtmodelle. Semmo, Amir in HPImgzn (2014). (15) 32-35.
Die nichtfotorealistische Bildsynthese stellt ein umfangreiches, innovatives Repertoire zur grafischen Gestaltung bereit, die eine wirkungsvolle Visualisierung komplexer raumbezogener Informationen ermöglicht. Der Fachbereich für computergrafische Systeme beschäftigt sich u.a. mit dem Design, der Implementierung und Evaluierung von nichtfotorealistischen Visualisierungstechniken für virtuelle 3D-Umgebungen – ein Forschungsbericht von Amir Semmo.
-
Oil Paint Filtering Using Color Palettes For Colorization. Semmo, Amir; Döllner, Jürgen (2014).
We present a novel technique for oil paint filtering that uses color palettes for colorization. First, dominant feature-aware colors are derived from the input image via entropy-based metrics. Seed pixels are then determined and propagated to the remaining pixels by adopting the optimization framework of Levin et al. [2004] for feature-aware colorization. Finally, the quantized output is combined with flow-based highlights and contour lines to simulate paint texture. Our technique leads to homogeneous outputs in the color domain and enables interactive control over color definitions.
-
Painting Per-Pixel Parametrization for Interactive Image Filtering. Limberger, Daniel; Döllner, Jürgen (2014).
We present a photo-editing method that enables per-pixel parameter manipulation of image filtering by means of interactive painting. Predefined as well as custom image filters are exposed to the user, as a parametrizable composition of image operations. Brushes, as a sequences of actions mapping user inputs (in terms of brush shape, flow, pressure, etc.) to arbitrary functions or convolution operators, are used to draw within the parameter space. Our tool demonstrates that interactive painting can be used to, e.g., locally tweak inadequate parametrization and, furthermore, provides a blueprint for an open, collaborative photo-editing platform.
-
Real-Time Animated Visualization of Massive Air-Traffic Trajectories. Buschmann, Stefan; Trapp, Matthias; Döllner, Jürgen (2014). 172-181.
With increasing numbers of flights world-wide and a continuing rise in airport traffic, air-traffic management is faced with a number of challenges. These include monitoring, reporting, planning, and problem analysis of past and current air traffic, e.g., to identify hotspots, minimize delays, or to optimize sector assignments to air-traffic controllers. Interactive and dynamic 3D visualization and visual analysis of massive aircraft trajectories, i.e., analytical reasoning enabled by interactive cyber worlds, can be used to approach these challenges. To facilitate this kind of analysis, especially in the context of real-time data, interactive tools for filtering, mapping, and rendering are required. In particular, the mapping process should be configurable at run-time and support both static mappings and animations to allow users to effectively explore and realize movement dynamics. However, with growing data size and complexity, these stages of the visualization pipeline require computational efficient implementations to be capable of processing within real-time constraints. This paper presents an approach for real-time animated visualization of massive air-traffic data, that implements all stages of the visualization pipeline based on GPU techniques for efficient processing. It enables (1) interactive spatio-temporal filtering, (2) generic mapping of trajectory attributes to geometric representations and appearances, as well as (3) real-time rendering within 3D virtual environments, such as virtual 3D airport and city models. Based on this pipeline, different visualization metaphors (e.g., temporal focus+context, density maps, and overview+detail visualization) are implemented and discussed. The presented concepts and implementation can be generally used as visual analytics and data mining techniques in cyber worlds, e.g., to visualize movement data, geo-referenced networks, or other spatio-temporal data.
-
Scalable Multi-Platform Distribution of Spatial 3D Contents. Klimke, Jan; Hagedorn, Benjamin; Döllner, Jürgen in International Journal of 3-D Information Modeling, (U. Isikdag, ed.) (2014). 3(3) 35-49.
Virtual 3D city models provide powerful user interfaces for communication of 2D and 3D geoinformation. Providing high quality visualization of massive 3D geoinformation in a scalable, fast, and cost efficient manner is still a challenging task. Especially for mobile and web-based system environments, software and hardware configurations of target systems differ significantly. This makes it hard to provide fast, visually appealing renderings of 3D data throughout a variety of platforms and devices. Current mobile or web-based solutions for 3D visualization usually require raw 3D scene data such as triangle meshes together with textures delivered from server to client, what makes them strongly limited in terms of size and complexity of the models they can handle. This paper introduces a new approach for provisioning of massive, virtual 3D city models on different platforms namely web browsers, smartphones or tablets, by means of an interactive map assembled from artificial oblique image tiles. The key concept is to synthesize such images of a virtual 3D city model by a 3D rendering service in a preprocessing step. This service encapsulates model handling and 3D rendering techniques for high quality visualization of massive 3D models. By generating image tiles using this service, the 3D rendering process is shifted from the client side, which provides major advantages: a The complexity of the 3D city model data is decoupled from data transfer complexity b the implementation of client applications is simplified significantly as 3D rendering is encapsulated on server side c 3D city models can be easily deployed for and used by a large number of concurrent users, leading to a high degree of scalability of the overall approach. All core 3D rendering techniques are performed on a dedicated 3D rendering server, and thin-client applications can be compactly implemented for various devices and platforms.
-
Scalable Multi-Platform Distribution of Spatial 3D Contents. Klimke, Jan; Hagedorn, Benjamin; Döllner, Jürgen in International Journal of 3-D Information Modeling (2014). 3(3) 35--49.
Virtual 3D city models provide powerful user interfaces for communication of 2D and 3D geoinformation. Providing high quality visualization of massive 3D geoinformation in a scalable, fast, and cost efficient manner is still a challenging task. Especially for mobile and web-based system environments, software and hardware configurations of target systems differ significantly. This makes it hard to provide fast, visually appealing ren-derings of 3D data throughout a variety of platforms and devices. Current mobile or web-based solutions for 3D visualization usually require raw 3D scene data such as triangle meshes together with textures delivered from server to client, what makes them strongly limited in terms of size and complexity of the models they can handle. This paper introduces a new approach for provisioning of massive, virtual 3D city models on different platforms namely web browsers, smartphones or tablets, by means of an interactive map assembled from artificial oblique image tiles. The key concept is to synthesize such images of a virtual 3D city model by a 3D rendering service in a preprocessing step. This service encapsulates model handling and 3D rendering techniques for high quality visualization of massive 3D models. By generating image tiles using this service, the 3D rendering process is shifted from the client side, which provides major advantages: (a) The complexity of the 3D city model data is decoupled from data transfer complexity (b) the implementation of client applica-tions is simplified significantly as 3D rendering is encapsulated on server side (c) 3D city models can be easily deployed for and used by a large number of concurrent users, leading to a high degree of scalability of the overall approach. All core 3D rendering techniques are performed on a dedicated 3D rendering server, and thin-client applications can be compactly implemented for various devices and platforms.
-
Scalable Multi-Platform Distribution of Spatial 3D Contents. Klimke, Jan; Hagedorn, Benjamin; Döllner, Jürgen in International Journal of 3-D Information Modeling (2014). 3(3) 35--49.
Virtual 3D city models provide powerful user interfaces for communication of 2D and 3D geoinformation. Providing high quality visualization of massive 3D geoinformation in a scalable, fast, and cost efficient manner is still a challenging task. Especially for mobile and web-based system environments, software and hardware configurations of target systems differ significantly. This makes it hard to provide fast, visually appealing ren-derings of 3D data throughout a variety of platforms and devices. Current mobile or web-based solutions for 3D visualization usually require raw 3D scene data such as triangle meshes together with textures delivered from server to client, what makes them strongly limited in terms of size and complexity of the models they can handle. This paper introduces a new approach for provisioning of massive, virtual 3D city models on different platforms namely web browsers, smartphones or tablets, by means of an interactive map assembled from artificial oblique image tiles. The key concept is to synthesize such images of a virtual 3D city model by a 3D rendering service in a preprocessing step. This service encapsulates model handling and 3D rendering techniques for high quality visualization of massive 3D models. By generating image tiles using this service, the 3D rendering process is shifted from the client side, which provides major advantages: (a) The complexity of the 3D city model data is decoupled from data transfer complexity (b) the implementation of client applica-tions is simplified significantly as 3D rendering is encapsulated on server side (c) 3D city models can be easily deployed for and used by a large number of concurrent users, leading to a high degree of scalability of the overall approach. All core 3D rendering techniques are performed on a dedicated 3D rendering server, and thin-client applications can be compactly implemented for various devices and platforms.
-
Service-Based 3D Rendering and Interactive 3D Visualization. Hagedorn, Benjamin; Döllner, Jürgen in HPI Future SOC Lab: Proceedings 2013, C. Meinel, Polze, A., Oswald, G., Strotmann, R., Seibold, U., Schulzki, B. (eds.) (2014). 3-5.
-
Service-Based 3D Rendering and Interactive 3D Visualization (2012). Hagedorn, Benjamin; Döllner, Jürgen in HPI Future SOC Lab: Proceedings 2012, C. Meinel, Polze, A., Oswald, G., Strotmann, R., Seibold, U., Bernhard, S. (eds.) (2014). 7-10.
This report describes the subject and preliminary results of our work in the context of the HPI Future SOC Lab, which generally aims on how to exploit high performance computing (HPC) capabilities for service-based 3D rendering and service‐based, interactive 3D visualization. A major focus is on the application of HPC technologies for the creation, management, analysis, and visualization of and interaction with virtual 3D environments, especially with complex 3D city models.
-
Visualization of Varying Hierarchies by Stable Layout of Voronoi Treemaps. Hahn, Sebastian; Trümper, Jonas; Moritz, Dominik; Döllner, Jürgen (2014). 50-58.
Space-restricted techniques for visualizing hierarchies generally achieve high scalability and readability (e.g., tree maps, bundle views, sunburst). However, the visualization layout directly depends on the hierarchy, that is, small changes to the hierarchy can cause wide-ranging changes to the layout. For this reason, it is difficult to use these techniques to compare similar variants of a hierarchy because users are confronted with layouts that do not expose the expected similarity. Voronoi treemaps appear to be promising candidates to overcome this limitation. However, existing Voronoi treemap algorithms do not provide deterministic layouts or assume a fixed hierarchy. In this paper we present an extended layout algorithm for Voronoi treemaps that provides a high degree of layout similiarity for varying hierarchies, such as software-system hierarchies. The implementation uses a deterministic initial-distribution approach that reduces the variation in node positioning even if changes in the underlying hierarchy data occur. Compared to existing layout algorithms, our algorithm achieves lower error rates with respect to node areas in the case of weighted Voronoi diagrams, which we show in a comparative study.
-
Web-based and Mobile Provisioning of Virtual 3D Reconstructions. Klimke, J.; Hagedorn, B.; Trapp, M.; Döllner, J. (2014). 17--28.
-
Web-based and Mobile Provisioning of Virtual 3D Reconstructions. Klimke, Jan; Hagedorn, Benjamin; Trapp, Matthias; Döllner, Jürgen R. Franken-Wendelstorf, Lindinger, E., Sieck, J. (eds.) (2014). 17--28.
Communication of cultural heritage by means of digital information systems has been gaining more and more importance over recent years. Interactive virtual 3D applications enable users to explore 3D virtual reconstructions in real-time, to directly interact with the contained digital cultural heritage artifacts, and to obtain insights into this data. Nevertheless, these artifacts are usually very detailed and complex 3D models that are hard to handle for end-user systems. This paper presents the concept and a prototypical implementation of an image-based, web-based approach for the communication of digital cultural heritage and its provisioning for the Web and mobile devices by the example of the project Colonia3D – a high-detail, virtual reconstruction and high-detail 3D city model of Roman Cologne. Through this web-based and mobile provisioning, complex digital reconstructions can be used, e.g., on-site to match local findings and reconstructions.
-
Web-based and Mobile Provisioning of Virtual 3D Reconstructions. Klimke, J.; Hagedorn, B.; Trapp, M.; Döllner, J. (2014). 17--28.
-
2.5D Clip-Surfaces for Technical Visualization. Trapp, Matthias; Döllner, Jürgen in Journal of WSCG, (V. Skala, ed.) (2013). 21(1) 89-96.
The concept of clipping planes is well known in computer graphics and can be used to create cut-away views. But clipping against just analytical defined planes is not always suitable for communicating every aspect of such visualization. For example, in hand-drawn technical illustrations, artists tend to communicate the difference between a cut and a model feature by using non-regular, sketchy cut lines instead of straight ones. To enable this functionality in computer graphics, this paper presents a technique for applying 2.5D clip-surfaces in real-time. Therefore, the clip plane equation is extended with an additional offset map, which can be represented by a texture map that contains height values. Clipping is then performed by varying the clip plane equation with respect to such an offset map. Further, a capping technique is proposed that enables the rendering of caps onto the clipped area to convey the impression of solid material. It avoids a re-meshing of a solid polygonal mesh after clipping is performed. Our approach is pixel precise, applicable in real-time, and takes fully advantage of graphics accelerators.
-
3D geovisualization services for efficient distribution of 3D geodata. Klimke, J. in Technische Berichte der Hasso-Plattner-Instituts an der Universität Potsdam (2013). (Vol. 76) 63-72.
Design, implementation, and operation of services for interactive 3D geovisualization are faced with a large number of challenges including (a) processing and integration of massive amounts of heterogeneous and distributed 2D and 3D geodata such as terrain models, buildings models, and thematic georeferenced data, (b) assembling, styling, and rendering 3D map contents according to application requirements and design principles, and (c) interactive provisioning of created 3D maps on mobile devices and thin clients as well as their integration as third-party components into domainspecific web and information systems. This report presents a concept and implementation of a service-oriented platform that addresses these major requirements of 3D web mapping systems. It is based on a separation of concerns for data management, 3D rendering, application logic, and user interaction. The main idea is to divide 3D rendering process into two stages. In the first stage, at the server side, an imagebased representation of the 3D scene is created by means of multi-layered virtual 3D panoramas; in the second stage, at the client side, the 3D scene is interactively reconstructed based on these panoramas. Beside the interactive client introduced earlier, scalable variant for provisioning of virtual 3D city models is described, that utilizes the capabilities of our rendering service for data preparation.
-
A Service-Based Concept for Camera Control in 3D Geovirtual Environments. Klimke, Jan; Hagedorn, Benjamin; Döllner, Jürgen in Lecture Notes in Geoinformation and Cartography, J. Pouliot, Daniel, S., Hubert, F., Zamyadi, A. (eds.) (2013). 101--118.
-
A Service-Based Concept for Camera Control in 3D Geovirtual Environments. Klimke, Jan; Hagedorn, Benjamin; Döllner, Jürgen in Lecture Notes in Geoinformation and Cartography, J. Pouliot, Daniel, S., Hubert, F., Zamyadi, A. (eds.) (2013). 101--118.
-
Concepts and techniques for integration, analysis and visualization of massive 3D point clouds. Richter, Rico; Döllner, Jürgen in Computers, Environment and Urban Systems (2013). 45 114-124.
Remote sensing methods, such as LiDAR and image-based photogrammetry, are established approaches for capturing the physical world. Professional and low-cost scanning devices are capable of generating dense 3D point clouds. Typically, these 3D point clouds are preprocessed by GIS and are then used as input data in a variety of applications such as urban planning, environmental monitoring, disaster management, and simulation. The availability of area-wide 3D point clouds will drastically increase in the future due to the availability of novel capturing methods (e.g., driver assistance systems) and low-cost scanning devices. Applications, systems, and workflows will therefore face large collections of redundant, up-to-date 3D point clouds and have to cope with massive amounts of data. Hence, approaches are required that will efficiently integrate, update, manage, analyze, and visualize 3D point clouds. In this paper, we define requirements for a system infrastructure that enables the integration of 3D point clouds from heterogeneous capturing devices and different timestamps. Change detection and update strategies for 3D point clouds are presented that reduce storage requirements and offer new insights for analysis purposes. We also present an approach that attributes 3D point clouds with semantic information (e.g., object class category information), which enables more effective data processing, analysis, and visualization. Out-of-core real-time rendering techniques then allow for an interactive exploration of the entire 3D point cloud and the corresponding analysis results. Web-based visualization services are utilized to make 3D point clouds available to a large community. The proposed concepts and techniques are designed to establish 3D point clouds as base datasets, as well as rendering primitives for analysis and visualization tasks, which allow operations to be performed directly on the point data. Finally, we evaluate the presented system, report on its applications, and discuss further research challenges.
-
Evaluating the Perceptual Impact of Rendering Techniques on Thematic Color Mappings in 3D Virtual Environments. Engel, Juri; Semmo, Amir; Trapp, Matthias; Döllner, Jürgen J. F. Michael Bronstein, Hormann, K. (eds.) (2013). 25-32.
Using colors for thematic mapping is a fundamental approach in visualization, and has become essential for 3D virtual environments to effectively communicate multidimensional, thematic information. Preserving depth cues within these environments to emphasize spatial relations between geospatial features remains an important issue.A variety of rendering techniques have been developed to preserve depth cues in 3D information visualization, including shading, global illumination, and image stylization. However, these techniques alter color values, which may lead to ambiguity in a color mapping and loss of information. Depending on the applied rendering techniques and color mapping, this loss should be reduced while still preserving depth cues when communicating thematic information. This paper presents the results of a quantitative and qualitative user study that evaluates the impact of rendering techniques on information and spatial perception when using visualization of thematic data in 3D virtual environments. We report the results of this study with respect to four perception-related tasks, showing significant differences in error rate and task completion time for different rendering techniques and color mappings.
-
Increasing the Usability of Pedestrian Navigation Interfaces by means of Landmark Visibility Analysis. Delikostidis, Ioannis; Engel, Juri; Retsios, Bas; van Elzakker, Corné P.J.M.; Kraak, Menno-Jan; Döllner, Jürgen in The Journal of Navigation (2013). 66(04) 523--537.
Communicating location-specific information to pedestrians is a challenging task which can be aided by user-friendly digital technologies. In this paper, landmark visibility analysis, as a means for developing more usable pedestrian navigation systems is discussed. Using an algorithmic framework for image-based 3D analysis, this method integrates a 3D city model with identified landmarks and produces raster visibility layers for each one. This output enables an Android phone prototype application to indicate the visibility of landmarks from the user’s actual position. Tested in the field, the method achieves sufficient accuracy for the context of use and improves navigation efficiency and effectiveness.
-
Interactive Projective Texturing for Non-Photorealistic Shading of Technical 3D Models. Lux, Roland; Trapp, Matthias; Semmo, Amir; Döllner, Jürgen S. Czanner, Tang, W. (eds.) (2013). 101--108.
This paper presents a novel interactive rendering technique for creating and editing shadings for man-made objects in technical 3D visualizations. In contrast to shading approaches that use intensities computed based on surface normals (e.g., Phong, Gooch, Toon shading), the presented approach uses one-dimensional gradient textures, which can be parametrized and interactively manipulated based on per-object bounding volume approximations. The fully hardware-accelerated rendering technique is based on projective texture mapping and customizable intensity transfer functions. A provided performance evaluation shows comparable results to traditional normal-based shading approaches. The work also introduce simple direct-manipulation metaphors that enables interactive user control of the gradient texture alignment and intensity transfer functions.
-
Interactive Rendering of Complex 3D-Treemaps. Trapp, Matthias; Hahn, Sebastian; Döllner, Jürgen S. Coquillart, Andujar, C., Laramee, R. S., Kerren, A., Braz, J. (eds.) (2013). 165-175.
3D-Treemaps are an important visualization technique for hierarchical views. In contrast to 2D-Treemaps, height can be used to map one additional attribute of the data items. Using the Treemap technique in combination with large datasets (more than 500k) a fast rendering and interaction techniques that are beyond collapsing/uncollapsing nodes is still one of the main challenges. This paper presents a novel rendering technique that enables the image synthesis of geometrical complex 3D-Treemaps in real-time. The fully hardware accelerated approach is based on shape generation using geometry shaders. This approach offers increased rendering performance and low update latency compared to existing techniques and through it enables new real-time interaction techniques to large datasets.
-
Interactive Rendering Techniques for Focus+Context Visualization of 3D Geovirtual Environments. Technical Report (PhD dissertation), Trapp, Matthias PhD thesis, Hasso-Plattner-Institut für Softwaresystemtechnik GmbH, Prof.-Dr.-Helmert-Str. 2-3, D-14482 Potsdam. (2013).
This thesis introduces a collection of new real-time rendering techniques and applications for focus+context visualization of interactive 3D geovirtual environments such as virtual 3D city and landscape models. These environments are generally characterized by a large number of objects and are of high complexity with respect to geometry and textures. For these reasons, their interactive 3D rendering represents a major challenge. Their 3D depiction implies a number of weaknesses such as occlusions, cluttered image contents, and partial screen-space usage. To overcome these limitations and, thus, to facilitate the effective communication of geo-information, principles of focus+context visualization can be used for the design of real-time 3D rendering techniques for 3D geovirtual environments (see Figure). In general, detailed views of a 3D geovirtual environment are combined seamlessly with abstracted views of the context within a single image. To perform the real-time image synthesis required for interactive visualization, dedicated parallel processors (GPUs) for rasterization of computer graphics primitives are used. For this purpose, the design and implementation of appropriate data structures and rendering pipelines are necessary. The contribution of this work comprises the following five real-time rendering methods: The rendering technique for 3D generalization lenses enables the combination of different 3D city geometries (e.g., generalized versions of a 3D city model) in an single image in real time. The method is based on a generalized and fragment-precise clipping approach, which uses a compressible, raster-based data structure. It enables the combination of detailed views in the focus area with the representation of abstracted variants in the context area. The rendering technique for the interactive visualization of dynamic raster data in 3D geovirtual environments facilitates the rendering of 2D surface lenses. It enables a flexible combination of different raster layers (e.g., aerial images or videos) using projective texturing for decoupling image and geometry data. Thus, various overlapping and nested 2D surface lenses of different contents can be visualized interactively. The interactive rendering technique for image-based deformation of 3D geovirtual environments enables the real-time image synthesis of non-planar projections, such as cylindrical and spherical projections, as well as multi-focal 3D fisheye-lenses and the combination of planar and non-planar projections. The rendering technique for view-dependent multi-perspective views of 3D geovirtual environments, based on the application of global deformations to the 3D scene geometry, can be used for synthesizing interactive panorama maps to combine detailed views close to the camera (focus) with abstract views in the background (context). This approach reduces occlusions, increases the usage the available screen space, and reduces the overload of image contents. The object-based and image-based rendering techniques for highlighting objects and focus areas inside and outside the view frustum facilitate preattentive perception. The concepts and implementations of interactive image synthesis for focus+context visualization and their selected applications enable a more effective communication of spatial information, and provide building blocks for design and development of new applications and systems in the field of 3D geovirtual environments.
-
Interactive Software Maps for Web-Based Source Code Analysis. Limberger, Daniel; Wasty, Benjamin; Trümper, Jonas; Döllner, Jürgen (2013). 8.
Software maps -- linking rectangular 3D-Treemaps, software system structure, and performance indicators -- are commonly used to support informed decision making in software-engineering processes. A key aspect for this decision making is that software maps provide the structural context required for correct interpretation of these performance indicators. In parallel, source code repositories and collaboration platforms are an integral part of today's software-engineering tool set, but cannot properly incorporate software maps since implementations are only available as stand-alone applications. Hence, software maps are 'disconnected' from the main body of this tool set, rendering their use and provisioning overly complicated, which is one of the main reasons against regular use. We thus present a web-based rendering system for software maps that achieves both fast client-side page load time and interactive frame rates even with large software maps. We significantly reduce page load time by efficiently encoding hierarchy and geometry data for the net transport. Apart from that, appropriate interaction, layouting, and labeling techniques as well as common image enhancements aid evaluation of project-related quality aspects. Metrics provisioning can further be implemented by predefined attribute mappings to simplify communication of project specific quality aspects. The system is integrated into dashboards to demonstrate how our web-based approach makes software maps more accessible to many different stakeholders in software-engineering projects.
-
Multi-Perspective Detail+Overview Visualization for 3D Building Exploration. Pasewaldt, Sebastian; Trapp, Matthias; Döllner, Jürgen S. Czanner, Tang, W. (eds.) (2013). 57--64.
This paper presents a multi-perspective rendering technique that enables detail+overview visualization and interactive exploration of virtual 3D building model. Virtual 3D building models, as main elements of virtual 3D city models, are used in a growing number of application domains, such as geoanalysis, disaster management and architectural planning. Visualization systems for such building models often rely on perspective or orthogonal projections using a single viewpoint. Therefore, the exploration of a complete model requires a user to change the viewpoint multiple times and to memorize the content of each view to obtain a comprehensive mental model. Since this is usually a time-consuming task, which implies context switching, current visualization systems use multiple viewports to simultaneously depict an object from different perspectives. Our approach extends the idea of multiple viewports by combining two linked views for the interactive exploration of virtual 3D buildings model and their facades. In contrast to traditional approaches, we automatically generate a multi-perspective view that simultaneously depicts all facades of the building in one overview image. This facilitates the process of obtaining overviews and supports fast and direct navigation to various points-of-interest. We describe the concept and implementations of our Multiple-Center-of-Projection camera model for real-time multi-perspective image synthesis. Further, we provide insights into different interaction techniques for linked multi-perspective views and outline approaches of future work.
-
Multiscale Visual Comparison of Execution Traces. Trümper, Jonas; Döllner, Jürgen; Telea, Alexandru (2013). 53-62.
Understanding the execution of programs by means of program traces is a key strategy in software comprehension. An important task in this context is comparing two traces in order to find similarities and differences in terms of executed code, execution order, and execution duration. For large and complex program traces, this is a difficult task due to the cardinality of the trace data. In this paper, we propose a new visualization method based on icicle plots and edge bundles. We address visual scalability by several multiscale visualization metaphors, which help users navigating from the main differences between two traces to intermediate structural-difference levels, and, finally fine-grained function call levels. We show how our approach, implemented in a tool called TraceDiff, is applicable in several scenarios for trace difference comprehension on real-world trace datasets.
-
Object class segmentation of massive 3D point clouds of urban areas using point cloud topology. Richter, Rico; Behrens, Markus; Döllner, Jürgen in International Journal of Remote Sensing (2013). 34(23) 8408-8424.
A large number of remote-sensing techniques and image-based photogrammetric approaches allow an efficient generation of massive 3D point clouds of our physical environment. The efficient processing, analysis, exploration, and visualization of massive 3D point clouds constitute challenging tasks for applications, systems, and workflows in disciplines such as urban planning, environmental monitoring, disaster management, and homeland security. We present an approach to segment massive 3D point clouds according to object classes of virtual urban environments including terrain, building, vegetation, water, and infrastructure. The classification relies on analysing the point cloud topology, it does not require per-point attributes or representative training data. The approach is based on an iterative multi-pass processing scheme, where each pass focuses on different topological features and considers already detected object classes from previous passes. To cope with the massive amount of data, out-of-core spatial data structures and graphics processing unit (GPU)-accelerated algorithms are utilized. Classification results are discussed based on a massive 3D point cloud with almost 5 billion points of a city. The results indicate that object-class-enriched 3D point clouds can substantially improve analysis algorithms and applications as well as enhance visualization techniques.
-
Out-of-Core GPU-based Change Detection in Massive 3D Point Clouds. Richter, Rico; Kyprianidis, Jan Eric; Döllner, Jürgen in Transactions in GIS (2013). 17(5) 724-741.
If sites, cities, and landscapes are captured at different points in time using technology such as LiDAR, large collections of 3D point clouds result. Their efficient storage, processing, analysis, and presentation constitute a challenging task because of limited computation, memory, and time resources. In this work, we present an approach to detect changes in massive 3D point clouds based on an out-of-core spatial data structure that is designed to store data acquired at different points in time and to efficiently attribute 3D points with distance information. Based on this data structure, we present and evaluate different processing schemes optimized for performing the calculation on the CPU and GPU. In addition, we present a point-based rendering technique adapted for attributed 3D point clouds, to enable effective out-of-core real-time visualization of the computation results. Our approach enables conclusions to be drawn about temporal changes in large highly accurate 3D geodata sets of a captured area at reasonable preprocessing and rendering times. We evaluate our approach with two data sets from different points in time for the urban area of a city, describe its characteristics, and report on applications.
-
Photorealistic Rendering of Day and Night Sky Phenomena in Interactive 3D Geovirtual Environments. Limberger,; Daniel, (2013).
-
Real-Time Rendering of Water Surfaces with Cartography-Oriented Design. Semmo, Amir; Kyprianidis, Jan Eric; Trapp, Matthias; Döllner, Jürgen in Proceedings International Symposium on Computational Aesthetics in Graphics, Visualization, and Imaging (CAe) (2013). 5--14.
More than 70% of the Earth's surface is covered by oceans, seas, and lakes, making water surfaces one of the primary elements in geospatial visualization. Traditional approaches in computer graphics simulate and animate water surfaces in the most realistic ways. However, to improve orientation, navigation, and analysis tasks within 3D virtual environments, these surfaces need to be carefully designed to enhance shape perception and land-water distinction. We present an interactive system that renders water surfaces with cartography-oriented design using the conventions of mapmakers. Our approach is based on the observation that hand-drawn maps utilize and align texture features to shorelines with non-linear distance to improve figure-ground perception and express motion. To obtain local orientation and principal curvature directions, first, our system computes distance and feature-aligned distance maps. Given these maps, waterlining, water stippling, contour-hatching, and labeling are applied in real-time with spatial and temporal coherence. The presented methods can be useful for map exploration, landscaping, urban planning, and disaster management, which is demonstrated by various real-world virtual 3D city and landscape models. © ACM, 2013. This is the authors' version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published in Proceedings of the International Symposium on Computational Aesthetics in Graphics, Visualization, and Imaging (CAe'13). http://dx.doi.org/10.1145/2487276.2487277.
-
Scalable Multi-Platform Distribution of Spatial 3D Contents. Klimke, Jan; Hagedorn, Benjamin; Döllner, Jürgen in ISPRS Annals of Photogrammetry, Remote Sensing and Spatial Information Sciences (2013). II-2/W1 193--200.
-
State of the ‘Art’: A Taxonomy of Artistic Stylization Techniques for Images and Video. Kyprianidis, Jan Eric; Collomosse, John; Wang, Tinghuai; Isenberg, Tobias in IEEE Transactions on Visualization and Computer Graphics (2013). 19(5) 866--885.
This paper surveys the field of non-photorealistic rendering (NPR), focusing on techniques for transforming 2D input (images and video) into artistically stylized renderings. We first present a taxonomy of the 2D NPR algorithms developed over the past two decades, structured according to the design characteristics and behavior of each technique. We then describe a chronology of development from the semi-automatic paint systems of the early nineties, through to the automated painterly rendering systems of the late nineties driven by image gradient analysis. Two complementary trends in the NPR literature are then addressed, with reference to our taxonomy. First, the fusion of higher level computer vision and NPR, illustrating the trends toward scene analysis to drive artistic abstraction and diversity of style. Second, the evolution of local processing approaches toward edge-aware filtering for real-time stylization of images and video. The survey then concludes with a discussion of open challenges for 2D NPR identified in recent NPR symposia, including topics such as user and aesthetic evaluation.
-
SyncTrace: Visual Thread-Interplay Analysis. Karran, Benjamin; Trümper, Jonas; Döllner, Jürgen (2013). 10.
In software comprehension, program traces are important to gain insight into certain aspects of concurrent runtime behavior, e.g., thread-interplay. Here, key tasks are finding usages of blocking operations, such as synchronization and I/O operations, assessing temporal order of such operations, and analyzing their effects. This is a hard task for large and complex program traces due to their size and number of threads involved. In this paper, we present SyncTrace, a new visualization technique based on (bended) activity diagrams and edge bundles that allows for parallel analysis of multiple threads and their inter-thread correspondences. We demonstrate how the technique, implemented as a tool, can be applied on real-world trace datasets to support understanding concurrent behavior.
-
Towards Visualizing Geo-Referenced Climate Networks. Buschmann, Stefan; Nocke, Thomas; Tominski, Christian; Döllner, Jürgen (2013).
In the last few years, network construction from climate data has developed to a promising analysis method. We discuss challenges for the interactive visual analysis of such geo-referenced networks and present first ideas for addressing the challenges. In particular, we present visualizations of 3D geo-spatial networks and of large networks within cartographic contexts.
