-
Döllner, Jürgen. Geospatial Artificial Intelligence: Potentials Of Machine Learning For 3D Point Clouds And Geospatial Digital Twins. Hasso Plattner Institute, 2019.
Artificial Intelligence (AI) is changing fundamentally the way how IT solutions are built and operated across all application domains, including the geospatial domain. In this article, we briefly reflect on the term “AI” and outline the factors such as Machine Learning (ML) and Deep Learning (DL) that contribute to applying AI successfully for IT solutions. In the main part we discuss AI for the geospatial domain (GeoAI) focussing on 3D point clouds as a key category of geodata, describe their properties and discuss its suitability for ML and DL. In particular, we conclude that 3D point clouds constitute a corpus with similar properties than natural language corpora and formulate a naturalness hypothesis for 3D point clouds. We then outline concepts and examples of ML-based interpretation approaches that compute domain-specific and application-specific semantics for 3D point clouds without having to create explicit spatial models or explicit rule sets. Finally, we will show how ML enables us to efficiently build and maintain base data for digital twins of our environment such as virtual 3D city models, indoor models, or building information models.
-
Klimke, Jan. “Web-Based Provisioning And Application Of Large-Scale Virtual 3D City Models”, 2019. https://publishup.uni-potsdam.de/42805.
-
Scheibel, Willy, Judith Hartmann, and Jürgen Döllner. “Design And Implementation Of Web-Based Hierarchy Visualization Services”. In 10Th International Conference On Information Visualization Theory And Applications (Ivapp), 2019.
There is a rapidly growing, cross-domain demand for interactive, high-quality visualization techniques as components of web-based applications and systems. In this context, a key question is how visualization services can be designed, implemented, and operated based on Software-as-a-Service as software delivery model. In this paper, we present concepts and design of a SaaS framework and API of visualization techniques for tree-structured data, called HiViSer. Using representational state transfer (REST), the API supports different data formats, data manipulations, visualization techniques, and output formats. In particular, the API defines base resource types for all components required to create an image or a virtual scene of a hierarchy visualization. We provide a treemap visualization service as prototypical implementation for which subtypes of the proposed API resources have been created. The approach generally serves as a blue-print for fully web-based, high-end visualization services running on thin clients in a standard browser environment.
-
Shekhar, Sumit, Amir Semmo, Matthias Trapp, Okan Tarhan Tursun, Sebastian Pasewaldt, Karol Myszkowski, and Jürgen Döllner. “Consistent Filtering Of Videos And Dense Light-Fields Without Optic-Flow”. In , H.-J. Schulz, Teschner, M., and Wimmer, M., 2019.
-
Semmo, Amir, Max Reimann, Mandy Klingbeil, Sumit Shekhar, Matthias Trapp, and Jürgen Döllner. “Vivid: Depicting Dynamics In Stylized Live Photos”. In Proceedings Siggraph Appy Hour, 8:1-8:2. New York: ACM, 2019.
We present ViVid, a mobile app for iOS that empowers users to express dynamics in stylized Live Photos. This app uses state-of-the-art computer-vision techniques based on convolutional neural networks to estimate motion in the video footage that is captured together with a photo. Based on these analytics and best practices of contemporary art, photos can be stylized as a pencil drawing or cartoon look that includes design elements to visually suggest motion, such as ghosts, motion lines and halos. Its interactive parameterizations enable users to filter and art-direct composition variables, such as color, size and opacity, of the stylization process. ViVid is based on Apple's CoreML, Metal and PhotoKit APIs for optimized on-device processing. Thus, the motion estimation is scheduled to utilize the dedicated neural engine and GPU in parallel, while shading-based image stylization is able to process the video footage in real-time. This way, the app provides a unique tool for creating lively photo stylizations with ease.
-
Stojanovic, Vladeta, Matthias Trapp, Rico Richter, and Jürgen Döllner. “Generation Of Approximate 2D And 3D Floor Plans From 3D Point Clouds”. In 14Th International Conference On Computer Graphics Theory And Applications (Grapp 2019), 2019.
-
Limberger, Daniel, Matthias Trapp, and Jürgen Döllner. “In-Situ Comparison For 2.5D Treemaps”. In 10Th International Conference On Information Visualization Theory And Applications (Ivapp 2019), 2019.
-
Trapp, Matthias, Frank Schlegel, Sebastian Pasewaldt, and Jürgen Döllner. “Rendering Procedural Textures For Visualization Of Thematic Data In 3D Geovirtual Environments”. In 10Th International Conference On Information Visualization Theory And Applications (Ivapp 2019), 2019.
-
Listemann, Marc, Matthias Trapp, and Jürgen Döllner. “Lens-Based Focus+Context Visualization Techniques For Interactive Exploration Of Web-Based Reachability Maps”. In 27Th International Conference In Central Europe On Computer Graphics, Visualization And Computer Vision (Wscg 2019), 2019.
Reachability maps are powerful means to help making location-based decisions, such as choosing convenient sites for subsidiaries or planning vacation trips. Existing visualization approaches, however, introduce drawbacks concerning an effective acquisition of relevant information and an efficient data processing. In this paper, we introduce the first approach known so far to apply focus+context techniques to web-based reachability maps. We therefore propose a real-time enabled combination of an isochrone-based and a network-based representation of travel times obtained from multi-modal routing analysis using interactive lenses. We furthermore present a GPU-accelerated image-based method to compute isochrones based on a travel time-attributed network on client-side and thus achieve reduced data transmission efforts while yielding isochrones of higher precision, compared to generalized geometry-based approaches.
-
Trapp, Matthias, Sebastian Pasewaldt, and Jürgen Döllner. “Techniques For Gpu-Based Color Quantization”. In 27Th International Conference In Central Europe On Computer Graphics, Visualization And Computer Vision (Wscg 2019), 2019.
This paper presents a GPU-based approach to color quantization by mapping of arbitrary color palettes to input images using LUTs. For it, different types of LUTs, their GPU-based generation, representation, and respective mapping implementations are described and their run-time performance is evaluated and compared.
-
Wegen, Ole, Matthias Trapp, Jürgen Döllner, and Sebastian Pasewaldt. “Performance Evaluation And Comparison Of Service-Based Image Processing Based On Software Rendering”. In 27Th International Conference In Central Europe On Computer Graphics, Visualization And Computer Vision (Wscg 2019), 2019.
This paper presents an approach and performance evaluation of performing service-based image processing using software rendering implemented using Mesa3D. Due to recent advances in cloud computing technology (with respect to both, hardware and software) as well as increased demands of image processing and analysis techniques, often within an eco-system of devices, it is feasible to research and quantify the impact of service-based approaches in this domain with respect to cost-performance relation. For it, we provide a performance comparison for service-based processing using GPU-accelerated and software rendering.
-
Florio, Alessando, Matthias Trapp, and Jürgen Döllner. “Semantic-Driven Visualization Techniques For Interactive Exploration Of 3D Indoor Models”. In 23Rd International Conference Information Visualisation (Iv 2019), 2019.
The availability of detailed virtual 3D building models including representations of indoor elements, allows for a wide number of applications requiring effective exploration and navigation functionality. Depending on the application context, users should be enabled to focus on specific object-of-interests or important building elements. This requires approaches to filtering building parts and techniques to visualize important building objects and their relations. For it, this paper explores the application and combination of interactive rendering techniques as well as their semantically-driven configuration in the context of 3D indoor models.
-
Trapp, Matthias, and Jürgen Döllner. “Interactive Close-Up Rendering For Detail+Overview Visualization Of 3D Digital Terrain Models”. In 23Rd International Conference Information Visualisation (Iv 2019), 2019.
This paper presents an interactive rendering technique for detail+overview visualization of 3D digital terrain models using interactive close-ups. A close-up is an alternative presentation of input data varying with respect to geometrical scale, mapping, appearance, as well as level-of-detail and level-of-abstraction used. The presented 3D close-up approach enables in-situ comparison of multiple region-of-interest simultaneously. We present a GPU-based rendering technique for the image-synthesis of multiple close-ups in real-time.
-
Trapp, Matthias, and Jürgen Döllner. “Real-Time Screen-Space Geometry Draping For Digital Terrain Models”. In 23Rd International Conference Information Visualisation (Iv 2019), 2019.
A fundamental task in 3D geovisualization and GIS applications is the visualization of vector data that can represent features such as transportation networks or land use coverage. Mapping or draping vector data represented by geometric primitives (e.g., polylines or polygons) to 3D digital elevation or terrain models is a challenging task. We present an interactive GPU-based approach that performs geometry-based draping of vector data on per-frame basis using an image-based representation of a 3D digital elevation or terrain model only.
-
Discher, Sören, Rico Richter, Matthias Trapp, and Jürgen Döllner. “Service-Oriented Processing And Analysis Of Massive Point Clouds In Geoinformation Management”. In Service-Oriented Mapping: Changing Paradigm In Map Production And Geoinformation Management, Jürgen Döllner, Jobst, Markus, and Schmitz, Peter, 43--61. Cham: Springer International Publishing, 2019. https://doi.org/10.1007/978-3-319-72434-8_2.
Today, landscapes, cities, and infrastructure networks are commonly captured at regular intervals using LiDAR or image-based remote sensing technologies. The resulting point clouds, representing digital snapshots of the reality, are used for a growing number of applications, such as urban development, environmental monitoring, and disaster management. Multi-temporal point clouds, i.e., 4D point clouds, result from scanning the same site at different points in time and open up new ways to automate common geoinformation management workflows, e.g., updating and maintaining existing geodata such as models of terrain, infrastructure, building, and vegetation. However, existing GIS are often limited by processing strategies and storage capabilities that generally do not scale for massive point clouds containing several terabytes of data. We demonstrate and discuss techniques to manage, process, analyze, and provide large-scale, distributed 4D point clouds. All techniques have been implemented in a system that follows service-oriented design principles, thus, maximizing its interoperability and allowing for a seamless integration into existing workflows and systems. A modular service-oriented processing pipeline is presented that uses out-of-core and GPU-based processing approaches to efficiently handle massive 4D point clouds and to reduce processing times significantly. With respect to the provision of analysis results, we present web-based visualization techniques that apply real-time rendering algorithms and suitable interaction metaphors. Hence, users can explore, inspect, and analyze arbitrary large and dense point clouds. The approach is evaluated based on several real-world applications and datasets featuring different densities and characteristics. Results show that it enables the management, processing, analysis, and distribution of massive 4D point clouds as required by a growing number of applications and systems.
-
Trapp, Matthias, Fabian Dumke, and Jürgen Döllner. “Occlusion Management Techniques For The Visualization Of Transportation Networks In Virtual 3D City Models”. In 12Th International Symposium On Visual Information Communication And Interaction (Vinci 2019). ACM, 2019.
-
Stojanovic, Vladeta, Matthias Trapp, Rico Richter, Benjamin Hagedorn, and Jürgen Döllner. “Semantic Enrichment Of Indoor Point Clouds: An Overview Of Progress Towards Digital Twinning”. In 37Th Ecaade Conference, 2019.
-
Limberger, Daniel, Willy Scheibel, Jürgen Döllner, and Matthias Trapp. “Advanced Visual Metaphors And Techniques For Software Maps”. In 12Th International Symposium On Visual Information Communication And Interaction (Vinci 2019). ACM, 2019.
-
Schoedon, Alexander, Matthias Trapp, Henning Hollburg, Daniel Gerber, and Jürgen Döllner. “Web-Based Visualization Of Transportation Networks For Mobility Analytics”. In 12Th International Symposium On Visual Information Communication And Interaction (Vinci 2019). ACM, 2019.
-
Stojanovic, Vladeta, Matthias Trapp, Rico Richter, and Jürgen Döllner. “Classification Of Indoor Point Clouds Using Multiview”. In 24Th International Acm Conference On 3D Web Technology (Web3D 2019), 2019. https://www.web3d.org/event/web3d-conference-2019.
-
Stojanovic, Vladeta, Matthias Trapp, Benjamin Hagedorn, Jan Klimke, Rico Richter, and Jürgen Döllner. “Sensor Data Visualization For Indoor Point Clouds”. In 15Th International Conference On Location Based Services (Lbs 2019), 2019. https://lbsconference.org.
-
Reimann, Max, Mandy Klingbeil, Sebastian Pasewaldt, Amir Semmo, Matthias Trapp, and Jürgen Döllner. “Locally Controllable Neural Style Transfer On Mobile Devices”. The Visual Computer (2019).
-
Stojanovic, Vladeta, Matthias Trapp, Rico Richter, and Jürgen Döllner. “Service-Oriented Semantic Enrichment Of Indoor Point Clouds Using Octree-Based Multiview Classification”. Jörg Peters. Graphical Models (2019). https://www.sciencedirect.com/science/article/pii/S152407031930030X.
The use of Building Information Modeling (BIM) for Facility Management (FM) in the Operation and Maintenance (O&M) stages of the building life-cycle is intended to bridge the gap between operations and digital data, but lacks the functionality of assessing the state of the built environment due to non-automated generation of associated semantics. 3D point clouds can be used to capture the physical state of the built environment, but also lack these associated semantics. A prototypical implementation of a service-oriented architecture for classification of indoor point cloud scenes of office environments is presented, using multiview classification. The multiview classification approach is tested using a retrained Convolutional Neural Network (CNN) model - Inception V3. The presented approach for classifying common office furniture objects (chairs, sofas and desks), contained in 3D point cloud scans, is tested and evaluated. The results show that the presented approach can classify common office furniture up to an acceptable degree of accuracy, and is suitable for quick and robust semantics approximation — based on RGB (red, green and blue color channel) cubemap images of the octree partitioned areas of the 3D point cloud scan. Additional methods for web-based 3D visualization, editing and annotation of point clouds are also discussed. Using the described approach, captured scans of indoor environments can be semantically enriched using object annotations derived from multiview classification results. Furthermore, the presented approach is suited for semantic enrichment of lower resolution indoor point clouds acquired using commodity mobile devices.
-
Discher, Sören, Rico Richter, and Jürgen Döllner. “Concepts And Techniques For Web-Based Visualization And Processing Of Massive 3D Point Clouds With Semantics”. Graphical Models 104 (2019): 101036. http://www.sciencedirect.com/science/article/pii/S152407031930027X.
3D point cloud technology facilitates the automated and highly detailed acquisition of real-world environments such as assets, sites, and countries. We present a web-based system for the interactive exploration and inspection of arbitrary large 3D point clouds. Our approach is able to render 3D point clouds with billions of points using spatial data structures and level-of-detail representations. Point-based rendering techniques and post-processing effects are provided to enable task-specific and data-specific filtering, e.g., based on semantics. A set of interaction techniques allows users to collaboratively work with the data (e.g., measuring distances and annotating). Additional value is provided by the system’s ability to display additional, context-providing geodata alongside 3D point clouds and to integrate processing and analysis operations. We have evaluated the presented techniques and in case studies and with different data sets from aerial, mobile, and terrestrial acquisition with up to 120 billion points to show their practicality and feasibility.
-
Scheibel, Willy, Christopher Weyand, and Jürgen Döllner. “Evocells – A Treemap Layout Algorithm For Evolving Tree Data”. In 9Th International Conference On Information Visualization Theory And Applications (Ivapp), 2018.
-
Richter, Marvin, Maximilian Söchting, Amir Semmo, Jürgen Döllner, and Matthias Trapp. “Service-Based Processing And Provisioning Of Image-Abstraction Techniques”. In Proceedings International Conference On Computer Graphics, Visualization And Computer Vision (Wscg). Plzen, Czech Republic, 2018.
Digital images and image streams represent two major categories of media captured, delivered, and shared on the Web. Techniques for their analysis, classification, and processing are fundamental building blocks in today's digital media applications ranging from mobile image transformation apps to professional digital production suites. To efficiently process such digital media (1) independent of hardware requirements, (2) at different data complexity scales, (3) to yield high-quality results, poses several challenges for software frameworks and hardware systems, in particular for mobile devices. With respect to these aspects, using service-based architectures are a common approach to strive for. However, unlike geodata, there is currently no standard approach for service definition, implementation, and orchestration in the domain of digital images and videos. This paper presents an approach for service-based image processing and provisioning of processing techniques by the example of image-abstraction techniques. The generality and feasibility of the proposed system is demonstrated by different client applications that have been implemented for the Android Operation System, for Google's G-Suite Software-as-a-Service Infrastructure, as well as for Desktop systems. The performance of the system is discussed at the example of complex, resource-intensive image abstraction techniques, such as watercolor rendering.
-
Besançon, Lonni, Amir Semmo, David Biau, Bruno Frachet, Virginie Pineau, El Hadi Ariali, Rabah Taouachi, Tobias Isenberg, and Pierre Dragicevic. “Reducing Affective Responses To Surgical Images Through Color Manipulation And Stylization”. In Proceedings Of The Joint Symposium On Computational Aesthetics, Sketch-Based Interfaces And Modeling, And Non-Photorealistic Animation And Rendering (Expressive), Tunc Aydın and Sýkora, Daniel. Victoria, BC, Canada: ACM, 2018.
We present the first empirical study on using color manipulation and stylization to make surgery images more palatable. While aversion to such images is natural, it limits many people's ability to satisfy their curiosity, educate themselves, and make informed decisions. We selected a diverse set of image processing techniques, and tested them both on surgeons and lay people. While many artistic methods were found unusable by surgeons, edge-preserving image smoothing gave good results both in terms of preserving information (as judged by surgeons) and reducing repulsiveness (as judged by lay people). Color manipulation turned out to be not as effective.
-
Montesdeoca, Santiago, Hock Soon Seah, Amir Semmo, Pierre Bénard, Romain Vergne, Joëlle Thollot, and Davide Benvenuti. “Mnpr: A Framework For Real-Time Expressive Non-Photorealistic Rendering Of 3D Computer Graphics”. In Proceedings Of The Joint Symposium On Computational Aesthetics, Sketch-Based Interfaces And Modeling, And Non-Photorealistic Animation And Rendering (Expressive), Tunc Aydın and Sýkora, Daniel. Victoria, BC, Canada: ACM, 2018.
We propose a framework for expressive non-photorealistic rendering of 3D computer graphics. Our work focuses on enabling stylization pipelines with a wide range of control, thereby covering the interaction spectrum with real-time feedback. In addition, we introduce control semantics that allow cross-stylistic art-direction, which is demonstrated through our implemented watercolor, oil and charcoal stylizations. Our generalized control semantics and their style-specific mappings are designed to be extrapolated to other styles, by adhering to the same control scheme. We then share our implementation details by breaking down the framework and elaborating on its inner workings. Finally, we evaluate the usefulness of each level of control through a user study involving 20 experienced artists and engineers in the industry, who have collectively spent over 245 hours using our system. Our framework is implemented in Autodesk Maya and open-sourced through this publication, to facilitate adoption by artists and further development by the expressive research and development community.
-
Stojanovic, Vladeta, Matthias Trapp, Rico Richter, Benjamin Hagedorn, and Jürgen Döllner. “Towards The Generation Of Digital Twins For Facility Management Based On 3D Point Clouds”. In Arcom 2018, 2018.
-
Trapp, Matthias, Sebastian Pasewaldt, Tobias Dürschmid, Amir Semmo, and Jürgen Döllner. “Teaching Image-Processing Programming For Mobile Devices: A Software Development Perspective”. In Proceedings Eurographics Education Papers, Frits Post and Žára, Jirí. Delft, Netherlands: The Eurographics Association, 2018.
In this paper we present a concept of a research course that teaches students in image processing as a building block of mobile applications. Our goal with this course is to teach theoretical foundations, practical skills in software development as well as scientific working principles to qualify graduates to start as fully-valued software developers or researchers. The course includes teaching and learning focused on the nature of small team research and development as encountered in the creative industries dealing with computer graphics, computer animation and game development. We discuss our curriculum design and issues in conducting undergraduate and graduate research that we have identified through four iterations of the course. Joint scientific demonstrations and publications of the students and their supervisors as well as quantitative and qualitative evaluation by students underline the success of the proposed concept. In particular, we observed that developing using a common software framework helps the students to jump start their course projects, while industry software processes such as branching coupled with a three-tier breakdown of project features helps them to structure and assess their progress.
-
Reimann, Max, Mandy Klingbeil, Sebastian Pasewaldt, Amir Semmo, Matthias Trapp, and Jürgen Döllner. “Maestro: A Mobile App For Style Transfer Orchestration Using Neural Networks”. In Proceedings International Conference On Cyberworlds, 9-16. IEEE, 2018.
Mobile expressive rendering gained increasing popularity amongst users seeking casual creativity by image stylization and supports the development of mobile artists as a new user group. In particular, the neural style transfer has advanced as a core technology to emulate characteristics of manifold artistic styles and media without deep prior knowledge of photo processing or editing. However, when it comes to creative expression, the technology still faces inherent limitations in providing low-level controls for localized image stylization, e.g., with respect to image feature semantics or the user's ideas and interest. The goal of this work is to implement and enhance state-of-the-art neural style transfer techniques, providing a generalized user interface with interactive tools for local control that facilitate a creative editing process on mobile devices. At this, we first propose a problem characterization consisting of three goals that represent a trade-off between visual quality, run-time performance and ease of control. We then present MaeSTrO, a mobile app for orchestration of three neural style transfer techniques using iterative, multi-style generative and adaptive neural networks that can be locally controlled by on-screen painting metaphors to direct a semantics-based composition and perform location-based filtering. Based on first user tests, we conclude with insights, showing different levels of satisfaction for the implemented techniques and user interaction design, pointing out directions for future research.
-
Reimann, Max, Mandy Klingbeil, Sebastian Pasewaldt, Amir Semmo, Jürgen Döllner, and Matthias Trapp. “Approaches For Local Artistic Control Of Mobile Neural Style Transfer”. In Proceedings Of The Joint Symposium On Computational Aesthetics, Sketch-Based Interfaces And Modeling, And Non-Photorealistic Animation And Rendering (Expressive). New York: ACM, 2018.
This work presents enhancements to state-of-the-art adaptive neural style transfer techniques, thereby providing a generalized user interface with creativity tool support for lower-level local control to facilitate the demanding interactive editing on mobile devices. The approaches are implemented in a mobile app that is designed for orchestration of three neural style transfer techniques using iterative, multi-style generative and adaptive neural networks that can be locally controlled by on-screen painting metaphors to perform location-based filtering and direct the composition. Based on first user tests, we conclude with insights, showing different levels of satisfaction for the implemented techniques and user interaction design, pointing out directions for future research.
-
Reimann, Max, Amir Semmo, Sebastian Pasewaldt, Mandy Klingbeil, and Jürgen Döllner. “Maestro: Mobile-Style Transfer Orchestration Using Adaptive Neural Networks”. In Proceedings Siggraph Appy Hour. New York: ACM, 2018.
We present MaeSTrO, a mobile app for image stylization that empowers users to direct, edit and perform a neural style transfer with creative control. The app uses iterative style transfer, multi-style generative and adaptive networks to compute and apply flexible yet comprehensive style models of arbitrary images at run-time. Compared to other mobile applications, MaeSTrO introduces an interactive user interface that empowers users to orchestrate style transfers in a two-stage process for an individual visual expression: first, initial semantic segmentation of a style image can be complemented by on-screen painting to direct sub-styles in a spatially-aware manner. Second, semantic masks can be virtually drawn on top of a content image to adjust neural activations within local image regions, and thus direct the transfer of learned sub-styles. This way, the general feed-forward neural style transfer is evolved towards an interactive tool that is able to consider composition variables and mechanisms of general artwork production, such as color, size and location-based filtering. MaeSTrO additionally enables users to define new styles directly on a device and synthesize high-quality images based on prior segmentations via a service-based implementation of compute-intensive iterative style transfer techniques.
-
Discher, Sören, Rico Richter, Matthias Trapp, and Jürgen Döllner. “Service-Oriented Processing And Analysis Of Massive Point Clouds In Geoinformation Management”. In Service Oriented Mapping: Changing Paradigm In Map Production And Geoinformation Management, Jürgen Döllner, Jobst, Markus, and Schmitz, Peter. Springer, 2018.
-
Stojanovic, Vladeta, Matthias Trapp, Rico Richter, and Jürgen Döllner. “A Service-Oriented Approach For Classifying 3D Points Clouds By Example Of Office Furniture Classification”. In 23Rd International Acm Conference On 3D Web Technology (Web3D 2018). ACM, 2018.
-
Thiel, Felix, Sören Discher, Rico Richter, and Jürgen Döllner. “Interaction And Locomotion Techniques For The Exploration Of Massive 3D Point Clouds In Vr Environments”. Proceedings Of Isprs Technical Commission Iv Symposium 2018 (2018).
Emerging virtual reality (VR) technology allows immersively exploring digital 3D content on standard consumer hardware. Using in-situ or remote sensing technology, such content can be automatically derived from real-world sites. External memory algorithms allow for the non-immersive exploration of the resulting 3D point clouds on a diverse set of devices with vastly different rendering capabilities. Applications for VR environments raise additional challenges for those algorithms as they are highly sensitive towards visual artifacts that are typical for point cloud depictions (i.e., overdraw and underdraw), while simultaneously requiring higher frame rates (i.e., around 90 fps instead of 30 - 60 fps). We present a rendering system for the immersive exploration and inspection of massive 3D point clouds on state-of-the-art VR devices. Based on a multi-pass rendering pipeline, we combine point-based and image-based rendering techniques to simultaneously improve the rendering performance and the visual quality. A set of interaction and locomotion techniques allows users to inspect a 3D point cloud in detail, for example by measuring distances and areas or by scaling and rotating visualized data sets. All rendering, interaction and locomotion techniques can be selected and configured dynamically, allowing to adapt the rendering system to different use cases. Tests on data sets with up to 2.6 billion points show the feasibility and scalability of our approach.
-
Wolf, Johannes, Sören Discher, Leon Masopust, Sebastian Schulz, Rico Richter, and Jürgen Döllner. “Combined Visual Exploration Of 2D Ground Radar And 3D Point Cloud Data For Road Environments”. Proceedings Of 3D Geoinfo 2018 (2018).
Ground-penetrating 2D radar scans are captured in road environments for examination of pavement condition and below-ground variations such as lowerings and developing pot-holes. 3D point clouds captured above ground provide a precise digital representation of the road’s surface and the surrounding environment. If both data sources are captured for the same area, a combined visualization is a valuable tool for infrastructure maintenance tasks. This paper presents visualization techniques developed for the combined visual exploration of the data captured in road environments. Main challenges are the positioning of the ground radar data within the 3D environment and the reduction of occlusion for individual data sets. By projecting the measured ground radar data onto the precise trajectory of the scan, it can be displayed within the context of the 3D point cloud representation of the road environment. We show that customizable overlay, filtering, and cropping techniques enable insightful data exploration. A 3D renderer combines both data sources. To enable an inspection of areas of interest, ground radar data can be elevated above ground level for better visibility. An interactive lens approach enables to visualize data sources that are currently occluded by others. The visualization techniques prove to be a valuable tool for ground layer anomaly inspection and were evaluated in a real-world data set. The combination of 2D ground radar scans with 3D point cloud data improves data interpretation by giving context information (e. g., about manholes in the street) that can be directly accessed during evaluation.
-
Discher, Sören, Rico Richter, and Jürgen Döllner. “A Scalable Webgl-Based Approach For Visualizing Massive 3D Point Clouds Using Semantics-Dependent Rendering Techniques”. Proceedings Of Web3D ’18 (2018).
3D point cloud technology facilitates the automated and highly detailed digital acquisition of real-world environments such as assets, sites, cities, and countries; the acquired 3D point clouds represent an essential category of geodata used in a variety of geoinformation applications and systems. In this paper, we present a web-based system for the interactive and collaborative exploration and inspection of arbitrary large 3D point clouds. Our approach is based on standard WebGL on the client side and is able to render 3D point clouds with billions of points. It uses spatial data structures and level-of-detail representations to manage the 3D point cloud data and to deploy out-of-core and web-based rendering concepts. By providing functionality for both, thin-client and thick-client applications, the system scales for client devices that are vastly different in computing capabilities. Different 3D point-based rendering techniques and post-processing effects are provided to enable task-specific and data-specific filtering and highlighting, e.g., based on per-point surface categories or temporal information. A set of interaction techniques allows users to collaboratively work with the data, e.g., by measuring distances and areas, by annotating, or by selecting and extracting data subsets. Additional value is provided by the system's ability to display additional, context-providing geodata alongside 3D point clouds and to integrate task-specific processing and analysis operations. We have evaluated the presented techniques and the prototype system with different data sets from aerial, mobile, and terrestrial acquisition campaigns with up to 120 billion points to show their practicality and feasibility.
-
Vollmer, Jan Ole, Matthias Trapp, Heidrun Schumann, and Jürgen Döllner. “Hierarchical Spatial Aggregation For Level-Of-Detail Visualization Of 3D Thematic Data”. Acm Transactions On Spatial Algorithms And Systems 4, no. 3 (2018): 9:1--9:23.
Thematic maps are a common tool to visualize semantic data with a spatial reference. Combining thematic data with a geometric representation of their natural reference frame aids the viewer’s ability in gaining an overview, as well as perceiving patterns with respect to location; however, as the amount of data for visualization continues to increase, problems such as information overload and visual clutter impede perception, requiring data aggregation and level-of-detail visualization techniques. While existing aggregation techniques for thematic data operate in a 2D reference frame (i.e., map), we present two aggregation techniques for 3D spatial and spatiotemporal data mapped onto virtual city models that hierarchically aggregate thematic data in real time during rendering to support on-the-fly and on-demand level-of-detail generation. An object-based technique performs aggregation based on scene-specific objects and their hierarchy to facilitate per-object analysis, while the scene-based technique aggregates data solely based on spatial locations, thus supporting visual analysis of data with arbitrary reference geometry. Both techniques can apply different aggregation functions (mean, minimum, and maximum) for ordinal, interval, and ratio-scaled data and can be easily extended with additional functions. Our implementation utilizes the programmable graphics pipeline and requires suitably encoded data, i.e., textures or vertex attributes. We demonstrate the application of both techniques using real-world datasets, including solar potential analyses and the propagation of pressure waves in a virtual city model.
-
Discher, Sören, Leon Masopust, Sebastian Schulz, Rico Richter, and Jürgen Döllner. “A Point-Based And Image-Based Multi-Pass Rendering Technique For Visualizing Massive 3D Point Clouds In Vr Environments”. Proceedings Of Wscg 2018 (2018).
Real-time rendering for 3D point clouds allows for interactively exploring and inspecting real-world assets, sites, or regions on a broad range of devices but has to cope with their vastly different computing capabilities. Virtual reality (VR) applications rely on high frame rates (i.e., around 90 fps as opposed to 30 - 60 fps) and show high sensitivity to any kind of visual artifacts, which are typical for 3D point cloud depictions (e.g., holey surfaces or visual clutter due to inappropriate point sizes). We present a novel rendering system that allows for an immersive, nausea-free exploration of arbitrary large 3D point clouds on state-of-the-art VR devices such as HTC Vive and Oculus Rift. Our approach applies several point-based and image-based rendering techniques that are combined using a multipass rendering pipeline. The approach does not require to derive generalized, mesh-based representations in a preprocessing step and preserves precision and density of the raw 3D point cloud data. The presented techniques have been implemented and evaluated with massive real-world data sets from aerial, mobile, and terrestrial acquisition campaigns containing up to 2.6 billion points to show the practicability and scalability of our approach.
-
Semmo, Amir, Tobias Isenberg, and Jürgen Döllner. “Neural Style Transfer: A Paradigm Shift For Image-Based Artistic Rendering?”. In Proceedings International Symposium On Non-Photorealistic Animation And Rendering, Holger Winnemöller and Bartram, Lyn, 5:1--5:13. Los Angeles, California: ACM, 2017.
In this meta paper we discuss image-based artistic rendering (IB-AR) based on neural style transfer (NST) and argue, while NST may represent a paradigm shift for IB-AR, that it also has to evolve as an interactive tool that considers the design aspects and mechanisms of artwork production. IB-AR received significant attention in the past decades for visual communication, covering a plethora of techniques to mimic the appeal of artistic media. Example-based rendering represents one the most promising paradigms in IB-AR to (semi-)automatically simulate artistic media with high fidelity, but so far has been limited because it relies on pre-defined image pairs for training or informs only low-level image features for texture transfers. Advancements in deep learning showed to alleviate these limitations by matching content and style statistics via activations of neural network layers, thus making a generalized style transfer practicable. We categorize style transfers within the taxonomy of IB-AR, then propose a semiotic structure to derive a technical research agenda for NSTs with respect to the grand challenges of NPAR. We finally discuss the potentials of NSTs, thereby identifying applications such as casual creativity and art production.
-
Semmo, Amir, Matthias Trapp, Jürgen Döllner, and Mandy Klingbeil. “Pictory: Combining Neural Style Transfer And Image Filtering”. In Proceedings Siggraph Appy Hour, 5:1--5:2. Los Angeles, California: ACM, 2017.
This work presents Pictory, a mobile app that empowers users to transform photos into artistic renditions by using a combination of neural style transfer with user-controlled state-of-the-art nonlinear image filtering. The combined approach features merits of both artistic rendering paradigms: deep convolutional neural networks can be used to transfer style characteristics at a global scale, while image filtering is able to simulate phenomena of artistic media at a local scale. Thereby, the proposed app implements an interactive two-stage process: first, style presets based on pre-trained feed-forward neural networks are applied using GPU-accelerated compute shaders to obtain initial results. Second, the intermediate output is stylized via oil paint, watercolor, or toon filtering to inject characteristics of traditional painting media such as pigment dispersion (watercolor) as well as soft color blendings (oil paint), and to filter artifacts such as fine-scale noise. Finally, on-screen painting facilitates pixel-precise creative control over the filtering stage, e. g., to vary the brush and color transfer, while joint bilateral upsampling enables outputs at full image resolution suited for printing on real canvas.
-
Limberger, Daniel, Marcel Pursche, Jan Klimke, and Jürgen Döllner. “Progressive High-Quality Rendering For Interactive Information Cartography Using Webgl”. In Proceedings Of The 22Nd International Conference On 3D Web Technology, 4. Web3D '17. ACM, 2017. https://dl.acm.org/citation.cfm?id=3075951.
Information cartography services provided via web-based clients using real-time rendering do not always necessitate a continuous stream of updates in the visual display. This paper shows how progressive rendering by means of multi-frame sampling and frame accumulation can introduce high-quality visual effects using robust and straightforward implementations. For it, (1) a suitable rendering loop is described, (2) WebGL limitations are discussed, and (3) an adaption of THREE.js featuring progressive anti-aliasing, screen-space ambient occlusion, and depth of field is detailed. Furthermore, sampling strategies are discussed and rendering performance is evaluated, emphasizing the low per-frame costs of this approach.
-
Limberger, Daniel, Willy Scheibel, Matthias Trapp, and Jürgen Döllner. “Mixed-Projection Treemaps: A Novel Approach Mixing 2D And 2.5D Treemaps”. In Proceedings Of The International Conference On Information Visualization 2017, 2017.
2D treemaps are a space-filling visualization technique that facilitate exploration of non-spatial, attributed, tree-structured data using the visual variables size and color. In extension thereto, 2.5D treemaps introduce height for additional information display. This extension entails challenges such as increased rendering effort, occlusion, or the need for navigation techniques that counterbalance the advantages of 2D treemaps to a certain degree. This paper presents a novel technique for combining 2D and 2.5D treemaps using multi-perspective views to leverage the advantages of both treemap types. It enables a new form of overview+detail visualization for complex treemaps and contributes new concepts for real-time rendering of and interaction with mixed-projection treemaps. The technique operates by tilting up inner nodes using affine transformations and animated state transitions. The mixed use of orthogonal and perspective projections is discussed and application examples that facilitate exploration of multi-variate data and benefit from the reduced interaction overhead are demonstrated.
-
Bethge, Joseph, Sebastian Hahn, and Jürgen Döllner. “Improving Layout Quality By Mixing Treemap-Layouts Based On Data-Change Characteristics”. In Vision, Modeling & Visualization, Matthias Hullin, Klein, Reinhard, Schultz, Thomas, and Yao, Angela. Vol. 2017. The Eurographics Association, 2017.
This paper presents a hybrid treemap layout approach that optimizes layout-quality metrics by combining state-of-the-art treemap layout algorithms. It utilizes machine learning to predict those metrics based on data metrics describing the characteristics and changes of the dataset. For this, the proposed approach uses a neural network which is trained on artificially generated dataset,s containing a total of 15.8 million samples. The resulting model is integrated into an approach called Smart-Layouting. This approach is evaluated on real-world data from 100 publicly available software repositories. Compared to other state-of-the-art treemap algorithms it reaches an overall better result. Additionally, this approach can be customized by an end user’s needs. The customization allows for specifying weights for the importance of each layout-quality metric. The results indicate, that the algorithm is able to adapt successfully towards a given set of weights.
-
Hahn, Sebastian, Joseph Bethge, and Jürgen Döllner. “Relative Direction Change: A Topology-Based Metric For Layout Stability In Treemaps”. In Proceedings Of The 8Th International Conference Of Information Visualization Theory And Applications (Ivapp 2017), 2017.
This paper presents a topology-based metric for layout stability in treemaps—the Relative Direction Change (RDC). The presented metric considers the adjacency and arrangement of single shapes in a treemap, and allows for a rotation-invariant description of layout changes between two snapshots of a dataset depicted with treemaps. A user study was conducted that shows the applicability of the Relative Direction Change in comparison and addition to established layout metrics, such as Average Distance Change (ADC) and Average Aspect Ratio (AAR), with respect to human perception of treemaps. This work contributes to the establishment of a more precise model for the replicable and reliable comparison of treemap layout algorithms © The Authors 2017. This is the authors' version of the work. It is posted here for your personal use. Not for redistribution. The definitive version will be published in Proceedings of the 8th International Conference on Information Visualization Theory and Applications (IVAPP 2017).
-
Dürschmid, Tobias, Maximilian Söchting, Amir Semmo, Matthias Trapp, and Jürgen Döllner. “Prosumerfx: Mobile Design Of Image Stylization Components”. In Proceedings Siggraph Asia Mobile Graphics And Interactive Applications (Mgia). Bangkok, Thailand: ACM, 2017.
With the continuous advances of mobile graphics hardware, high-quality image stylization—e.g., based on image filtering, stroke-based rendering, and neural style transfer—is becoming feasible and increasingly used in casual creativity apps. The creative expression facilitated by these mobile apps, however, is typically limited with respect to the usage and application of pre-defined visual styles, which ultimately do not include their design and composition—an inherent requirement of prosumers. We present ProsumerFX, a GPU-based app that enables to interactively design parameterizable image stylization components on-device by reusing building blocks of image processing effects and pipelines. Furthermore, the presentation of the effects can be customized by modifying the icons, names, and order of parameters and presets. Thereby, the customized visual styles are defined as platform-independent effects and can be shared with other users via a web-based platform and database. Together with the presented mobile app, this system approach supports collaborative works for designing visual styles, including their rapid prototyping, A/B testing, publishing, and distribution. Thus, it satisfies the needs for creative expression of both professionals as well as the general public.
-
Klingbeil, Mandy, Sebastian Pasewaldt, Amir Semmo, and Jürgen Döllner. “Challenges In User Experience Design Of Image Filtering Apps”. In Proceedings Siggraph Asia Mobile Graphics And Interactive Applications (Mgia). Bangkok, Thailand: ACM, 2017.
Photo filtering apps successfully deliver image-based stylization techniques to a broad audience, in particular in the ubiquitous domain (e.g., smartphones, tablet computers). Interacting with these inherently complex techniques has so far mostly been approached in two different ways: (1) by exposing many (technical) parameters to the user, resulting in a professional application that typically requires expert domain knowledge, or (2) by hiding the complexity via presets that only allows the application of filters but prevents creative expression thereon. In this work, we outline challenges of and present approaches for providing interactive image filtering on mobile devices, thereby focusing on how to make them usable for people in their daily life. This is discussed by the example of BeCasso, a user-centric app for assisted image stylization that targets two user groups: mobile artists and users seeking casual creativity. Through user research, qualitative and quantitative user studies, we identify and outline usability issues that showed to prevent both user groups from reaching their objectives when using the app. On the one hand, user-group-targeting has been improved by an optimized user experience design. On the other hand, multiple level of controls have been implemented to ease the interaction and hide the underlying complex technical parameters. Evaluations underline that the presented approach can increase the usability of complex image stylization techniques for mobile apps.
-
Pasewaldt, Sebastian, Amir Semmo, Mandy Klingbeil, and Jürgen Döllner. “Demo: Pictory - Neural Style Transfer And Editing With Coreml”. In Proceedings Siggraph Asia Mobile Graphics And Interactive Applications (Mgia). Bangkok, Thailand: ACM, 2017.
This work presents advances in the design and implementation of Pictory, an iOS app for artistic neural style transfer and interactive image editing using the CoreML and Metal APIs. Pictory combines the benefits of neural style transfer, e.g., high degree of abstraction on a global scale, with the interactivity of GPU-accelerated stateof-the-art image-based artistic rendering on a local scale. Thereby, the user is empowered to create high-resolution, abstracted renditions in a two-stage approach. First, a photo is transformed using a pre-trained convolutional neural network to obtain an intermediate stylized representation. Second, image-based artistic rendering techniques (e.g., watercolor, oil paint or toon filtering) are used to further stylize the image. Thereby, fine-scale texture noise—introduced by the style transfer—is filtered and interactive means are provided to individually adjust the stylization effects at run-time. Based on qualitative and quantitative user studies, Pictory has been redesigned and optimized to support casual users as well as mobile artists by providing effective, yet easy to understand, tools to facilitate image editing at multiple levels of control.
-
Hahn, Sebastian, and Jürgen Döllner. “Hybrid-Treemap Layouting”. In Proceedings Of Eurovis 2017 - Short Papers, 2017.
This paper presents an approach for hybrid treemaps, which applies and combines several different layout principles within a single tree map in contrast to traditional treemap variants based on a single layout concept. To this end, we analyze shortcomings of state-of-the-art treemap algorithms such as Moore, Voronoi and Strip layouts. Based on a number of identified edge cases, we propose a combination of these different layout algorithms, individually selected for and applied on each sub-hierarchy of the given treemap data. The selection decision is based on the number of items to be layouted as well as the aspect-ratio of the containing visual elements. Futhermore, a layout quality score based on existing treemap layout metrics (e.g., average distance change, relative direction change, average aspect ratio) has been used to evaluate the results of the proposed hybrid layout algorithm and to demonstrate its usefulness applied on representative hierarchical data sets. © The Authors 2017. This is the authors' version of the work. It is posted here for your personal use. Not for redistribution. The definitive version will be published in Proceedings of the EuroVis2017.
-
Limberger, Daniel, Willy Scheibel, Sebastian Hahn, and Jürgen Döllner. “Reducing Visual Complexity In Software Maps Using Importance-Based Aggregation Of Nodes”. In Proceedings Of The 8Th International Conference On Information Visualization Theory And Applications. Ivapp 2017, 2017.
Depicting massive software system data using software maps can result in visual clutter and increased cognitive load. This paper introduces an adaptive level-of-detail (LoD) technique that uses scoring for interactive aggregation on a per-node basis. The scoring approximates importance by degree-of-interest measures as well as screen and user-interaction scores. The technique adheres to established aggregation guidelines and was evaluated by means of two user studies. The first user study investigates task completion time in visual search. The second evaluates the readability of the presented nesting level contouring for aggregates. With the adap- tive LoD technique software maps allow for multi-resolution depictions of software system information. It facilitates efficient identification of important nodes and allows for additional annotation. © The Authors 2017. This is the authors' version of the work. It is posted here for your personal use. Not for redistribution. The definitive version will be published in Proceedings of the 8th International Conference on Information Visualization Theory and Applications (IVAPP 2017).
-
Döllner, Jürgen. “Hinter Den Bildern Von Andreas Schiller”. In Andreas Schiller: Global Backup Ii, Joachim Penzel, 75-79. Prestel Verlag, 2017.
-
Döllner, Jürgen. “Vom Fotorealismus Zum Nichtfotorealismus”. In Hands On: Kunstgeschichte: Methodik Und Unterrichtsbeispiele Der Gestaltungspraktischen Kunstrezeption, Joachim Penzel. copaed Verlag, 2017.
-
Scheibel, Willy, Stefan Buschmann, Matthias Trapp, and Jürgen Döllner. “Attributed Vertex Clouds”. In Gpu Zen, Wolfgang Engel and Oat, Christopher. 8 ed.. Gpu Pro, 2017.
In todays computer graphics applications, large 3D scenes are rendered which consist of polygonal geometries such as triangle meshes. Using state- of-the-art techniques, this geometry is often represented on the GPU using vertex and index buffers, as well as additional auxiliary data such as tex- tures or uniform buffers. For polygonal meshes of arbitrary complexity, the described approach is indispensable. However, there are several types of simpler geometries (e.g., cuboids, spheres, tubes, or splats) that can be generated procedurally. We present an efficient data representation and render- ing concept for such geometries, denoted as attributed vertex clouds (AVCs). Using this approach, geometry is generated on the GPU during execution of the programmable rendering pipeline. Each vertex is used as the argument for a function that procedurally generates the target geometry. This function is called a transfer function, and it is implemented using shader programs and therefore executed as part of the rendering process. This approach allows for compact geometry representation and results in reduced memory footprints in comparison to traditional representations. By shifting geometry generation to the GPU, the resulting volatile geometry can be controlled flexibly, i.e., its position, parameteri- zation, and even the type of geometry can be modified without requiring suggests improved rendering times and reduced memory transmission through the rendering pipeline.
-
Limberger, Daniel, Willy Scheibel, Matthias Trapp, and Jürgen Döllner. “Mixed-Projection Treemaps: A Novel Approach Mixing 2D And 2.5D Treemaps”. In Proceedings Of The International Conference On Information Visualization 2017, 2017.
-
Dürschmid, Tobias, Matthias Trapp, and Jürgen Döllner. “Towards Architectural Styles For Android App Software Product Lines”. In 2017 Ieee/acm 4Th International Conference On Mobile Software Engineering And Systems (Mobilesoft),, 2017.
-
Stojanovic, Vladeta, Rico Richter, Jürgen Döllner, and Matthias Trapp. “Comparative Visualization Of Bim Geometry And Corresponding Point Clouds”. International Journal Of Sustainable Development And Planning 13, no. 1 (2017).
-
Hagedorn, Benjamin. “Konzepte Und Techniken Zur Servicebasierten Visualisierung Von Geovirtuellen 3D-Umgebungen”. PhD dissertation, Potsdam, 2016. https://publishup.uni-potsdam.de/opus4-ubp/frontdoor/index/index/docId/9286.
Im Allgemeinen liegen Geodaten und georeferenzierte Daten verteilt vor, sind heterogen in ihrem Inhalt und ihrer Form, umfassen sehr große Datenmengen und müssen in verschiedenen IT-Informationssystemen integriert und in verschiedenen Anwendungskontexten genutzt werden. Im Mittelpunkt dieser Arbeit stehen deshalb Konzepte und Techniken für die Integration, Visualisierung, Analyse, Bereitstellung und Nutzung von 2D- und 3D-Geodaten sowie georeferenzierten Daten. Hierbei wird ein Ansatz verfolgt, der zum einen virtuelle 3D-Umgebungen als konzeptionellen und technischen Rahmen nutzt und zum anderen auf serviceorientierten Softwarearchitekturen und Geo-Standards basiert. Die vorgestellten Konzepte und Verfahren stellen damit Schlüsselbausteine dar, um neuartige IT-Lösungen und Anwendungen für 3D-Geoinformationen, z.B. als Bestandteile von Geodateninfrastrukturen, zu realisieren. Im Bereich der servicebasierten 3DGeovisualisierung beschreibt diese Arbeit, wie virtuelle 3D-Stadtmodelle für die Integration heterogener und verteilter Geodatenquellen genutzt werden können. Dazu werden Anforderungen für die Integration identifiziert und ein Konzept für die Integration auf Datenebene und die Integration auf Visualisierungsebene entworfen und deren Umsetzung am Beispiel komplexer 3D-Bauwerksinformationsmodelle beschrieben und demonstriert. Im Bereich servicebasierter, bildbasierter 3DPortrayal-Services wird mit dem Web View Service (WVS) ein spezialisierter SoftwareService für die Visualisierung, Interaktion und Analyse von geovirtuellen 3D-Umgebungen konzipiert und entwickelt. Kernkonzept dieses Services sind die serverseitige Datenintegration und -verwaltung sowie die ebenfalls serverseitige Bilderzeugung. Mit diesem durchgehend serverseitigen Ansatz können sehr große Mengen an 3D-Geodaten auch auf solchen Endgeräte bereitgestellt werden, die nicht über ausreichend Speicher und Rechenleistung für die Vorhaltung, die Verarbeitung und das Rendering von 3D-Modellen verfügen. Der entwickelte WVS ermöglicht so u.a. auch auf Tablet-PCs und im Webbrowser die interaktive Erkundung und Analyse von 3D-Geodaten. Anhand einer Referenzimplementierung des WVS und einer Client-Anwendung wird die praktische Anwendung des WVS demonstriert. Im Bereich der Komposition von Web View Services wird untersucht, wie der WVS als Baustein einer komplexen, verteilten Visualisierungs- und Renderingpipeline eingesetzt werden kann. Durch die Komposition des WVS mit anderen Darstellungs- und Verarbeitungsservices können z. B. komplexe Renderingeffekte erzielt oder einzelne 3D-Objekte nachträglich in eine 3D-Ansicht integriert werden. Hierzu wird ein Konzept zur servicebasierten, Tiefenbildbasierten Bildkomposition beschrieben und am Beispiel der verdeckungsfreien Annotation von 3D-Ansichten umgesetzt und demonstriert. Im Bereich der Interaktion mit Web View Services liefert diese Arbeit Grundlagen, Konzept und Umsetzung für intelligente, assistierende und automatisierte Interaktions- und Navigationstechniken, die auf dem Angebotscharakter (der Affordanz) von 3D-Szenenobjekten sowie auf der skizzen- und gestenbasierten Eingabe von Nutzerintentionen basiert. Diese Eingaben werden hinsichtlich ihrer Form und unter Berücksichtigung der Semantik der 3DSzenenobjekte ausgewertet und interpretiert und anschließend in anwendungsspezifische Navigationskommandos übersetzt, aus denen teilautomatische Kamerafahrten abgeleitet werden.
-
Semmo, Amir. “Design And Implementation Of Non-Photorealistic Rendering Techniques For 3D Geospatial Data”, 2016.
Geospatial data has become a natural part of a growing number of information systems and services in the economy, society, and people's personal lives. In particular, virtual 3D city and landscape models constitute valuable information sources within a wide variety of applications such as urban planning, navigation, tourist information, and disaster management. Today, these models are often visualized in detail to provide realistic imagery. However, a photorealistic rendering does not automatically lead to high image quality, with respect to an effective information transfer, which requires important or prioritized information to be interactively highlighted in a context-dependent manner. Approaches in non-photorealistic renderings particularly consider a user's task and camera perspective when attempting optimal expression, recognition, and communication of important or prioritized information. However, the design and implementation of non-photorealistic rendering techniques for 3D geospatial data pose a number of challenges, especially when inherently complex geometry, appearance, and thematic data must be processed interactively. Hence, a promising technical foundation is established by the programmable and parallel computing architecture of graphics processing units. This thesis proposes non-photorealistic rendering techniques that enable both the computation and selection of the abstraction level of 3D geospatial model contents according to user interaction and dynamically changing thematic information. To achieve this goal, the techniques integrate with hardware-accelerated rendering pipelines using shader technologies of graphics processing units for real-time image synthesis. The techniques employ principles of artistic rendering, cartographic generalization, and 3D semiotics—unlike photorealistic rendering—to synthesize illustrative renditions of geospatial feature type entities such as water surfaces, buildings, and infrastructure networks. In addition, this thesis contributes a generic system that enables to integrate different graphic styles—photorealistic and non-photorealistic—and provide their seamless transition according to user tasks, camera view, and image resolution. Evaluations of the proposed techniques have demonstrated their significance to the field of geospatial information visualization including topics such as spatial perception, cognition, and mapping. In addition, the applications in illustrative and focus+context visualization have reflected their potential impact on optimizing the information transfer regarding factors such as cognitive load, integration of non-realistic information, visualization of uncertainty, and visualization on small displays.
-
Semmo, Amir, Matthias Trapp, Sebastian Pasewaldt, and Jürgen Döllner. “Interactive Oil Paint Filtering On Mobile Devices”. In Expressive 2016 - Posters, Artworks, And Bridging Papers, 2016.
Image stylization enjoys a growing popularity on mobile devices to foster casual creativity. However, the implementation and provision of high-quality image filters for artistic rendering is still faced by the inherent limitations of mobile graphics hardware such as computing power and memory resources. This work presents a mobile implementation of a filter that transforms images into an oil paint look, thereby highlighting concepts and techniques on how to perform multi-stage nonlinear image filtering on mobile devices. The proposed implementation is based on OpenGL ES and the OpenGL ES shading language, and supports on-screen painting to interactively adjust the appearance in local image regions, e.g., to vary the level of abstraction, brush, and stroke direction. Evaluations of the implementation indicate interactive performance and results that are of similar aesthetic quality than its original desktop variant.
-
Vollmer, Jan Ole, Matthias Trapp, and Jürgen Döllner. “Interactive Gpu-Based Image Deformation For Mobile Devices”. In Computer Graphics And Visual Computing (Cgvc). The Eurographics Association, 2016.
Interactive image deformation is an important feature of modern image processing pipelines. It is often used to create caricatures and animation for input images, especially photos. State-of-the-art image deformation techniques are based on transforming vertices of a mesh, which is textured by the input image, using affine transformations such as translation, and scaling. However, the resulting visual quality of the output image depends on the geometric resolution of the mesh. Performing these transformations on the CPU often further inhibits performance and quality. This is especially problematic on mobile devices where the limited computational power reduces the maximum achievable quality. To overcome these issue, we propose the concept of an intermediate deformation buffer that stores deformation information at a resolution independent of the mesh resolution. This allows the combination of a high-resolution buffer with a low-resolution mesh for interactive preview, as well as a high-resolution mesh to export the final image. Further, we present a fully GPU-based implementation of this concept, taking advantage of modern OpenGL ES features, such as compute shaders.
-
Limberger, Daniel, Carolin Fiedler, Sebastian Hahn, Matthias Trapp, and Jürgen Döllner. “Evaluation Of Sketchiness As A Visual Variable For 2.5D Treemaps”. In Proceedings Of The 20Th International Conference Of Information Visualization (Iv'16), 2016.
Interactive 2.5D treemaps serve as an effective tool for the visualization of attributed hierarchies, enabling exploration of non-spatial, multi-variate, hierarchical data. In this paper the suitability of sketchiness as a visual variable, e.g., for uncertainty, is evaluated. Therefore, a design space for sketchy rendering in 2.5D and integration details for real-time applications are presented. The results of three user studies indicate, that sketchiness is a promising candidate for a visual variable that can be used independently and in addition to others, e.g., color and height. © The Authors 2016. This is the authors' version of the work. It is posted here for your personal use. Not for redistribution. The definitive version will be published in Proceedings of the 20th International Conference on Information Visualization (IV'16).
-
Schoedon, Alexander, Matthias Trapp, Henning Hollburg, and Jürgen Döllner. “Interactive Web-Based Visualization For Accessibility Mapping Of Transportation Networks”. In Proceedings Of Eurovis 2016 - Short Papers, 2016.
Accessibility is a fundamental aspect in transportation, routing, and spare-time activity planning concerning traveling in modern cities. In this context, interactive web-based accessibility-map visualization techniques and systems are important tools for provisioning, exploration, analysis, and assessment of multi-modal and location-based travel time data and routing information. To enable their effective application, such interactive visualization techniques demands for flexible mappings with respect to user-adjustable parameters such as maximum travel times, the types of transportation used, or used color schemes. However, traditional approaches for web-based visualization of accessibility-maps do not allow this degree of parametrization without significant latencies introduced by required data processing and transmission between the routing server and the visualization client. This paper presents a novel web-based visualization technique that allows for efficient client-side mapping and rendering of accessibility data onto transportation networks using WebGL and the OpenGL transmission format. A performance evaluation and comparison shows the superior performance of the approach over alternative implementations.
-
Semmo, Amir, Matthias Trapp, Tobias Dürschmid, Jürgen Döllner, and Sebastian Pasewaldt. “Interactive Multi-Scale Oil Paint Filtering On Mobile Devices”. In Proceedings Acm Siggraph Posters, 2016.
This work presents an interactive mobile implementation of a filter that transforms images into an oil paint look. At this, a multi-scale approach that processes image pyramids is introduced that uses flow-based joint bilateral upsampling to achieve deliberate levels of abstraction at multiple scales and interactive frame rates. The approach facilitates the implementation of interactive tools that adjust the appearance of filtering effects at run-time, which is demonstrated by an on-screen painting interface for per-pixel parameterization that fosters the casual creativity of non-artists.
-
Pasewaldt, Sebastian, Amir Semmo, Jürgen Döllner, and Frank Schlegel. “Becasso: Artistic Image Processing And Editing On Mobile Devices”. In Proceedings Acm Siggraph Asia Symposium On Mobile Graphics And Interactive Applications (Demo), 2016.
BeCasso is a mobile app that enables users to transform photos into high-quality, high-resolution non-photorealistic renditions, such as oil and watercolor paintings, cartoons, and colored pencil drawings, which are inspired by real-world paintings or drawing techniques. In contrast to neuronal network and physically-based approaches, the app employs state-of-the-art nonlinear image filtering. For example, oil paint and cartoon effects are based on smoothed structure information to interactively synthesize renderings with soft color transitions. BeCasso empowers users to easily create aesthetic renderings by implementing a two-fold strategy: First, it provides parameter presets that may serve as a starting point for a custom stylization based on global parameter adjustments. Thereby, users can obtain initial renditions that may be fine-tuned afterwards. Second, it enables local style adjustments: using on-screen painting metaphors, users are able to locally adjust different stylization features, e.g., to vary the level of abstraction, pen, brush and stroke direction or the contour lines. In this way, the app provides tools for both higher-level interaction and low-level control [Isenberg 2016] to serve the different needs of non-experts and digital artists. References: Isenberg, T. 2016. Interactive NPAR: What Type of Tools Should We Create? In Proc. NPAR, The Eurographics Association, Goslar, Germany, 89–96
-
Semmo, Amir, Jürgen Döllner, and Frank Schlegel. “Becasso: Image Stylization By Interactive Oil Paint Filtering On Mobile Devices”. In Proceedings Acm Siggraph Appy Hour, 2016.
BeCasso is a mobile app that enables users to transform photos into an oil paint look that is inspired by traditional painting elements. In contrast to stroke-based approaches, the app uses state-of-the-art nonlinear image filtering techniques based on smoothed structure information to interactively synthesize oil paint renderings with soft color transitions. BeCasso empowers users to easily create aesthetic oil paint renderings by implementing a two-fold strategy. First, it provides parameter presets that may serve as a starting point for a custom stylization based on global parameter adjustments. Second, it introduces a novel interaction approach that operates within the parameter spaces of the stylization effect to facilitate creative control over the visual output: on-screen painting enables users to locally adjust the appearance in image regions, e.g., to vary the level of abstraction, brush and stroke direction. This way, the app provides tools for both higher-level interaction and low-level control [Isenberg 2016] to serve the different needs of non-experts and digital artists. References: Isenberg, T. 2016. Interactive NPAR: What Type of Tools Should We Create? In Proc. NPAR, The Eurographics Association, Goslar, Germany, 89–96
-
Semmo, Amir, Tobias Dürschmid, Matthias Trapp, Mandy Klingbeil, Jürgen Döllner, and Sebastian Pasewaldt. “Interactive Image Filtering With Multiple Levels-Of-Control On Mobile Devices”. In Proceedings Acm Siggraph Asia Symposium On Mobile Graphics And Interactive Applications, 2016.
With the continuous development of mobile graphics hardware, interactive high-quality image stylization based on nonlinear filtering is becoming feasible and increasingly used in casual creativity apps. However, these apps often only serve high-level controls to parameterize image filters and generally lack support for low-level (artistic) control, thus automating art creation rather than assisting it. This work presents a GPU-based framework that enables to parameterize image filters at three levels of control: (1) presets followed by (2) global parameter adjustments can be interactively refined by (3) complementary on-screen painting that operates within the filters' parameter spaces for local adjustments. The framework provides a modular XML-based effect scheme to effectively build complex image processing chains-using these interactive filters as building blocks-that can be efficiently processed on mobile devices. Thereby, global and local parameterizations are directed with higher-level algorithmic support to ease the interactive editing process, which is demonstrated by state-of-the-art stylization effects, such as oil paint filtering and watercolor rendering.
-
Discher, Sören, Rico Richter, and Jürgen Döllner. “Interactive And View-Dependent See-Through Lenses For Massive 3D Point Clouds”. In Advances In 3D Geoinformation, 2016.
-
Limberger, Daniel, Willy Scheibel, Stefan Lemme, and Döllner Jürgen. “Dynamic 2.5D Treemaps Using Declarative 3D On The Web”. In Proceedings Of The 21St International Conference On Web3D Technology (Web3D), 33--36, 2016.
The 2.5D treemap represents a general purpose visualization technique to map multi-variate hierarchical data in a scalable, interactive, and consistent way used in a number of application fields. In this paper, we explore the capabilities of Declarative 3D for the web-based implementation of 2.5D treemap clients. Particularly, we investigate how X3DOM and XML3D can be used to implement clients with equivalent features that interactively display 2.5D treemaps with dynamic mapping of attributes. We also show a first step towards a glTF-based implementation. These approaches are benchmarked focusing on their interaction capabilities with respect to rendering and speed of dynamic data mapping. We discuss the results for our representative example of a complex 3D interactive visualization technique and summerize recommendations for improvements towards operational web clients.
-
Scheibel, Willy, Matthias Trapp, and Jürgen Döllner. “Interactive Revision Exploration Using Small Multiples Of Software Maps”. In 7Th International Conference On Information Visualization Theory And Applications, 131-138. Ivapp 2016, 2016.
To explore and to compare different revisions of complex software systems is a challenging task as it requires to constantly switch between different revisions and the corresponding information visualization. This paper proposes to combine the concept of small multiples and focus+context techniques for software maps to facilitate the comparison of multiple software map themes and revisions simultaneously on a single screen. This approach reduces the amount of switches and helps to preserve the mental map of the user. Given a software project the small multiples are based on a common dataset but are specialized by specific revisions and themes. The small multiples are arranged in a matrix where rows and columns represents different themes and revisions, respectively. To ensure scalability of the visualization technique we also discuss two rendering pipelines to ensure interactive frame-rates. The capabilities of the proposed visualization technique are demonstrated in a collaborative exploration setting using a high-resolution, multi-touch display.
-
Semmo, Amir, Daniel Limberger, Jan Eric Kyprianidis, and Jürgen Döllner. “Image Stylization By Interactive Oil Paint Filtering”. Computers & Graphics 55 (2016): 157--171.
This paper presents an interactive system for transforming images into an oil paint look. The system comprises two major stages. First, it derives dominant colors from an input image for feature-aware recolorization and quantization to conform with a global color palette. Afterwards, it employs non-linear filtering based on the smoothed structure adapted to the main feature contours of the quantized image to synthesize a paint texture in real-time. Our filtering approach leads to homogeneous outputs in the color domain and enables creative control over the visual output, such as color adjustments and per-pixel parametrizations by means of interactive painting. To this end, our system introduces a generalized brush-based painting interface that operates within parameter spaces to locally adjust the level of abstraction of the filtering effects. Several results demonstrate the various applications of our filtering approach to different genres of photography.
-
Trapp, Matthias, and Jürgen Döllner. “Geometry Batching Using Texture-Arrays”. In Proceedings Of The 10Th International Conference On Computer Graphics Theory And Applications (Grapp 2015), 239-246, 2015.
High-quality rendering of 3D virtual environments typically depends on high-quality 3D models with significant geometric complexity and texture data. One major bottleneck for real-time image-synthesis represents the number of state changes, which a specific rendering API has to perform. To improve performance, batching can be used to group and sort geometric primitives into batches to reduce the number of required state changes, whereas the size of the batches determines the number of required draw-calls, and therefore, is critical for rendering performance. For example, in the case of texture atlases, which provide an approach for efficient texture management, the batch size is limited by the efficiency of the texture-packing algorithm and the texture resolution itself. This paper presents a pre-processing approach and rendering technique that overcomes these limitations by further grouping textures or texture atlases and thus enables the creation of larger geometry batches. It is based on texture arrays in combination with an additional indexing schema that is evaluated at run-time using shader programs. This type of texture management is especially suitable for real-time rendering of large-scale texture-rich 3D virtual environments, such as virtual city and landscape models.
-
Würfel, Hannes, Matthias Trapp, Daniel Limberger, and Jürgen Döllner. “Natural Phenomena As Metaphors For Visualization Of Trend Data In Interactive Software Maps”. In Computer Graphics And Visual Computing (Cgvc). The Eurographics Association, 2015.
Software maps are a commonly used tool for code quality monitoring in software-development projects and decision making processes. While providing an important visualization technique for the hierarchical system structure of a single software revision, they lack capabilities with respect to the visualization of changes over multiple revisions. This paper presents a novel technique for visualizing the evolution of the software system structure based on software metric trends. These trend maps extend software maps by using real-time rendering techniques for natural phenomena yielding additional visual variables that can be effectively used for the communication of changes. Therefore, trend data is automatically computed by hierarchically aggregating software metrics. We demonstrate and discuss the presented technique using two real world data sets of complex software systems.
-
Discher,, and Sören. “Echtzeit-Rendering-Techniken Für 3D-Punktwolken Basierend Auf Semantischen Und Topologischen Attributen”. In Shortlist Karl Kraus Young Scientists Award 2015, 35. Wissenschaftlich-Technische Jahrestagung Der Dgpf, 2015.
-
Meier, Benjamin-Heinz, Matthias Trapp, and Jürgen Döllner. “Videomr: A Map And Reduce Framework For Real-Time Video Processing”. In International Conference In Central Europe On Computer Graphics, Visualization And Computer Vision (Wscg 2015), 2015.
This paper presents VideoMR: a novel map and reduce framework for real-time video processing on graphic processing units (GPUs). Using the advantages of implicit parallelism and bounded memory allocation, our approach enables developers to focus on implementing video operations without taking care of GPU memory handling or the details of code parallelization. Therefore, a new concept for map and reduce is introduced, redefining both operations to fit to the specific requirements of video processing. A prototypical implementation using OpenGL facilitates various operating platforms, including mobile development, and will be widely interoperable with other state-of-the-art video processing frameworks.
-
Hahn, Sebastian, Matthias Trapp, Nikolai Wuttke, and Jürgen Döllner. “Threadcity: Combined Visualization Of Structure And Activity For The Exploration Of Multi-Threaded Software Systems”. In Proceedings Of The 19Th International Conference Of Information Visualization (Iv'15), 2015.
This paper presents a novel visualization technique for the interactive exploration of multi-threaded software systems. It combines the visualization of static system structure based on the EvoStreets approach with an additional traffic metaphor to communicate the runtime characteristics of multiple threads simultaneously. To improve visual scalability with respect to the visualization of complex software systems, we further present an effective level-of-detail visualization based on hierarchical aggregation of system components by taking viewing parameters into account. We demonstrate our technique by means of a prototypical implementation and compare our result with existing visualization techniques. © The Authors 2015. This is the authors' version of the work. It is posted here for your personal use. Not for redistribution. The definitive version will be published in Proceedings of the 19th International Conference on Information Visualization (IV'15).
-
Oehlke, Christoph, Rico Richter, and Jürgen Döllner. “Automatic Detection And Large-Scale Visualization Of Trees For Digital Landscapes And City Models Based On 3D Point Clouds”. In 16Th Conference On Digital Landscape Architecture (Dla 2015), 151-160, 2015.
-
Buschmann, Stefan, Matthias Trapp, and Jürgen Döllner. “Real-Time Visualization Of Massive Movement Data In Digital Landscapes”. In 16Th Conference On Digital Landscape Architecture (Dla 2015), 213-220, 2015.
Due to continuing advances in sensor technology and increasing availability of digital infrastructure that allows for acquisition, transfer, and storage of big data sets, large amounts of movement data (e.g., road, naval, or air-traffic) become available. In the near future, movement data such as traffic data may even be available in real-time. In a growing number of application fields (e.g., landscape planning and design, urban development, and infrastructure planning), movement data enables new analysis and simulation applications. In this paper, we present an interactive technique for visualizing massive 3D movement trajectories. It is based on mapping massive movement data to graphics primitives and their visual variables in real-time, supporting a number of visualization schemes such as sphere, line, or tube-based trajectories, including animations of direction and speed. This generic technique enhances the functionality of VR and interactive 3D systems using virtual environments such as digital landscape models, city models, or virtual globes by adding support for this important category of spatio-temporal data.
-
Trapp, Matthias, Amir Semmo, and Jürgen Döllner. “Interactive Rendering And Stylization Of Transportation Networks Using Distance Fields”. In Proceedings Of The 10Th International Conference On Computer Graphics Theory And Applications (Grapp 2015), 207-219, 2015.
Transportation networks, such as streets, railroads or metro systems, constitute primary elements in cartography for reckoning and navigation. In recent years, they have become an increasingly important part of 3D virtual environments for the interactive analysis and communication of complex hierarchical information, for example in routing, logistics optimization, and disaster management. A variety of rendering techniques have been proposed that deal with integrating transportation networks within these environments, but have so far neglected the many challenges of an interactive design process to adapt their spatial and thematic granularity (i.e., level-of-detail and level-of-abstraction) according to a user's context. This paper presents an efficient real-time rendering technique for the view-dependent rendering of geometrically complex transportation networks within 3D virtual environments. Our technique is based on distance fields using deferred texturing that shifts the design process to the shading stage for real-time stylization. We demonstrate and discuss our approach by means of street networks using cartographic design principles for context-aware stylization, including view-dependent scaling for clutter reduction, contour-lining to provide figure-ground, handling of street crossings via shading-based blending, and task-dependent colorization. Finally, we present potential usage scenarios and applications together with a performance evaluation of our implementation.
-
Semmo, Amir, Daniel Limberger, Jan Eric Kyprianidis, and Jürgen Döllner. “Image Stylization By Oil Paint Filtering Using Color Palettes”. In Proceedings International Symposium On Computational Aesthetics In Graphics, Visualization, And Imaging (Cae), 149--158, 2015.
This paper presents an approach for transforming images into an oil paint look. To this end, a color quantization scheme is proposed that performs feature-aware recolorization using the dominant colors of the input image. In addition, an approach for real-time computation of paint textures is presented that builds on the smoothed structure adapted to the main feature contours of the quantized image. Our stylization technique leads to homogeneous outputs in the color domain and enables creative control over the visual output, such as color adjustments and per-pixel parametrizations by means of interactive painting. © The Authors 2015. This is the authors' version of the work. It is posted here for your personal use. Not for redistribution. The definitive version will be published in Proceedings of the International Symposium on Computational Aesthetics in Graphics, Visualization, and Imaging (CAe'15).
-
Richter, Rico, Sören Discher, and Jürgen Döllner. “Out-Of-Core Visualization Of Classified 3D Point Clouds”. In 3D Geoinformation Science: The Selected Papers Of The 3D Geoinfo 2014, 227-242. Cham: Springer International Publishing, 2015.
3D point clouds represent an essential category of geodata used in a variety of geoinformation applications and systems. We present a novel, interactive out-of-core rendering technique for massive 3D point clouds based on a layered, multi-resolution kd-tree, whereby point-based rendering techniques are selected according to each point's classification (e.g., vegetation, buildings, terrain). The classification-dependent rendering leads to an improved visual representation, enhances recognition of objects within 3D point cloud depictions, and facilitates visual filtering and highlighting. To interactively explore objects, structures, and relations represented by 3D point clouds, our technique provides efficient means for an instantaneous, ad-hoc visualization compared to approaches that visualize 3D point clouds by deriving mesh-based 3D models. We have evaluated our approach for massive laser scan datasets of urban areas. The results show the scalability of the technique and how different configurations allow for designing task and domain-specific analysis and inspection tools. © The Authors 2014. This is the authors' version of the work. It is posted here for your personal use. Not for redistribution. The definitive version will be published in 3D Geoinformation Science: The Selected Papers of the 3D GeoInfo 2014 by Springer International Publishing. http://dx.doi.org/10.1007/978-3-319-12181-9.
-
Semmo, Amir, Matthias Trapp, Markus Jobst, and Jürgen Döllner. “Cartography-Oriented Design Of 3D Geospatial Information Visualization - Overview And Techniques”. The Cartographic Journal 52, no. 2 (2015): 95--106.
In economy, society and personal life map-based, interactive geospatial visualization becomes a natural element of a growing number of applications and systems. The visualization of 3D geospatial information, however, raises the question how to represent the information in an effective way. Considerable research has been done in technology-driven directions in the fields of cartography and computer graphics (e.g., design principles, visualization techniques). Here, non-photorealistic rendering represents a promising visualization category–situated between both fields–that offers a large number of degrees for the cartography-oriented visual design of complex 2D and 3D geospatial information for a given application context. Still today, however, specifications and techniques for mapping cartographic design principles to the state-of-the-art rendering pipeline of 3D computer graphics remain to be explored. This paper revisits cartographic design principles for 3D geospatial visualization and introduces an extended 3D semiotic model that complies with the general, interactive visualization pipeline. Based on this model, we propose non-photorealistic rendering techniques to interactively synthesize cartographic renditions of basic feature types, such as terrain, water, and buildings. In particular, it includes a novel iconification concept to seamlessly interpolate between photorealistic and cartographic representations of 3D landmarks. Our work concludes with a discussion of open challenges in this field of research, including topics such as user interaction and evaluation.
-
Buschmann, Stefan, Matthias Trapp, and Jürgen Döllner. “Animated Visualization Of Spatial-Temporal Trajectory Data For Air-Traffic Analysis”. The Visual Computer 32, no. 3 (2015): 371-381.
With increasing numbers of flights worldwide and a continuing rise in airport traffic, air-traffic management is faced with a number of challenges. These include monitoring, reporting, planning, and problem analysis of past and current air traffic, e.g., to identify hotspots, minimize delays, or to optimize sector assignments to air-traffic controllers. To cope with these challenges, cyber worlds can be used for interactive visual analysis and analytical reasoning based on aircraft trajectory data. However, with growing data size and complexity, visualization requires high computational efficiency to process that data within real-time constraints. This paper presents a technique for real-time animated visualization of massive trajectory data. It enables (1) interactive spatio-temporal filtering, (2) generic mapping of trajectory attributes to geometric representations and appearance, and (3) real-time rendering within 3D virtual environments such as virtual 3D airport or 3D city models. Different visualization metaphors can be efficiently built upon this technique such as temporal focus+context, density maps, or overview+detail methods. As a general-purpose visualization technique, it can be applied to general 3D and 3+1D trajectory data, e.g., traffic movement data, geo-referenced networks, or spatio-temporal data, and it supports related visual analytics and data mining tasks within cyber worlds.
-
Semmo, Amir, and Jürgen Döllner. “Interactive Image Filtering For Level-Of-Abstraction Texturing Of Virtual 3D Scenes”. Computers & Graphics 52 (2015): 181--198.
Texture mapping is a key technology in computer graphics. For the visual design of 3D scenes, in particular, effective texturing depends significantly on how important contents are expressed, e.g., by preserving global salient structures, and how their depiction is cognitively processed by the user in an application context. Edge-preserving image filtering is one key approach to address these concerns. Much research has focused on applying image filters in a post-process stage to generate artistically stylized depictions. However, these approaches generally do not preserve depth cues, which are important for the perception of 3D visualization (e.g., texture gradient). To this end, filtering is required that processes texture data coherently with respect to linear perspective and spatial relationships. In this work, we present an approach for texturing 3D scenes with perspective coherence by arbitrary image filters. We propose decoupled deferred texturing with (1) caching strategies to interactively perform image filtering prior to texture mapping and (2) for each mipmap level separately to enable a progressive level of abstraction, using (3) direct interaction interfaces to parameterize the visualization according to spatial, semantic, and thematic data. We demonstrate the potentials of our method by several applications using touch or natural language inputs to serve the different interests of users in specific information, including illustrative visualization, focus+context visualization, geometric detail removal, and semantic depth of field. The approach supports frame-to-frame coherence, order-independent transparency, multitexturing, and content-based filtering. In addition, it seamlessly integrates into real-time rendering pipelines, and is extensible for custom interaction techniques.
-
Nocke, Thomas, Stefan Buschmann, Jonathan Donges, Norbert Marwan, Hans-Jörg Schulz, and Christian Tominski. “Review: Visual Analytics Of Climate Networks”. Nonlinear Processes In Geophysics 22, no. 5 (2015): 545-570.
Network analysis has become an important approach in studying complex spatiotemporal behaviour within geophysical observation and simulation data. This new field produces increasing amounts of large geo-referenced networks to be analysed. Particular focus lies currently on the network analysis of the complex statistical interrelationship structure within climatological fields. The standard procedure for such network analyses is the extraction of network measures in combination with static standard visualisation methods. Existing interactive visualisation methods and tools for geo-referenced network exploration are often either not known to the analyst or their potential is not fully exploited. To fill this gap, we illustrate how interactive visual analytics methods in combination with geovisualisation can be tailored for visual climate network investigation. Therefore, the paper provides a problem analysis, relating the multiple visualisation challenges with a survey undertaken with network analysts from the research fields of climate and complex systems science. Then, as an overview for the interested practitioner, we review the state-of-the-art in climate network visualisation and provide an overview of existing tools. As a further contribution, we introduce the visual network analytics tools CGV and GTX, providing tailored solutions for climate network analysis, including alternative geographic projections, edge bundling, and 3-D network support. Using these tools, the paper illustrates the application potentials of visual analytics for climate networks based on several use cases including examples from global, regional, and multi-layered climate networks.
-
Klimke, Jan. “High-Quality Video Generation For Thin Clients - An Application For Image-Based 3D Portrayal Services”. In Technische Berichte Des Hasso-Plattner-Instituts Fur Softwaresystemtechnik An Der Universitat Potsdam. Vol. 83, 2014.
-
Semmo, Amir, and Jürgen Döllner. “An Interaction Framework For Level-Of-Abstraction Visualization Of 3D Geovirtual Environments”. In Proceedings 2Nd Acm Sigspatial Workshop On Mapinteraction, 43--49, 2014.
3D geovirtual environments constitute effective media for the analysis and communication of complex geospatial data. Today, these environments are often visualized using static graphical variants (e.g., 2D maps, 3D photorealistic) from which a user is able to choose from. To serve the different interests of users in specific information, however, the spatial and thematic granularity at which model contents are represented (i.e., level of abstraction) should be dynamically adapted to the user's context, which requires specialized interaction techniques for parameterization. In this work, we present a framework that enables interaction interfaces to parameterize the level-of-abstraction visualization according to spatial, semantic, and thematic data. The framework is implemented in a visualization system that provides image-based rendering techniques for context-aware abstraction and highlighting. Using touch and natural language interfaces, we demonstrate its versatile application to geospatial tasks, including exploration, navigation, and orientation.
-
Semmo, Amir, and Jürgen Döllner. “Oil Paint Filtering Using Color Palettes For Colorization”. In Expressive Poster Session, 2014.
We present a novel technique for oil paint filtering that uses color palettes for colorization. First, dominant feature-aware colors are derived from the input image via entropy-based metrics. Seed pixels are then determined and propagated to the remaining pixels by adopting the optimization framework of Levin et al. [2004] for feature-aware colorization. Finally, the quantized output is combined with flow-based highlights and contour lines to simulate paint texture. Our technique leads to homogeneous outputs in the color domain and enables interactive control over color definitions.
-
Semmo, Amir, and Jürgen Döllner. “Image Filtering For Interactive Level-Of-Abstraction Visualization Of 3D Scenes”. In Proceedings International Symposium On Computational Aesthetics In Graphics, Visualization, And Imaging (Cae), 5--14, 2014.
Texture mapping is a key technology in computer graphics for visual design of rendered 3D scenes. An effective information transfer of surface properties, encoded by textures, however, depends significantly on how important information is highlighted and cognitively processed by the user in an application context. Edge-preserving image filtering is a promising approach to address this concern while preserving global salient structures. Much research has focused on applying image filters in a post-process stage to foster an artistically stylized rendering, but these approaches are generally not able to preserve depth cues important for 3D visualization (e.g., texture gradient). To this end, filtering that processes texture data coherently with respect to linear perspective and spatial relationships is required. In this work, we present a system that enables to process textured 3D scenes with perspective coherence by arbitrary image filters. We propose decoupled deferred texturing with (1) caching strategies to interactively perform image filtering prior to texture mapping, and (2) for each mipmap level separately to enable a progressive level of abstraction. We demonstrate the potentials of our methods on several applications, including illustrative visualization, focus+context visualization, geometric detail removal, and depth of field. Our system supports frame-to-frame coherence, order-independent transparency, multitexturing, and content-based filtering. © The Authors 2014. This is the authors' version of the work. It is posted here for your personal use. Not for redistribution. The definitive version will be published in Proceedings of the International Symposium on Computational Aesthetics in Graphics, Visualization, and Imaging (CAe'14). http://dx.doi.org/10.1145/2630099.2630101.
-
Hahn, Sebastian, Jonas Trümper, Dominik Moritz, and Jürgen Döllner. “Visualization Of Varying Hierarchies By Stable Layout Of Voronoi Treemaps”. In Proceedings Of The 5Th International Conference On Information Visualization Theory And Applications (Ivapp 2014), 50-58. SCITEPRESS – Science and Technology Publications, 2014.
Space-restricted techniques for visualizing hierarchies generally achieve high scalability and readability (e.g., tree maps, bundle views, sunburst). However, the visualization layout directly depends on the hierarchy, that is, small changes to the hierarchy can cause wide-ranging changes to the layout. For this reason, it is difficult to use these techniques to compare similar variants of a hierarchy because users are confronted with layouts that do not expose the expected similarity. Voronoi treemaps appear to be promising candidates to overcome this limitation. However, existing Voronoi treemap algorithms do not provide deterministic layouts or assume a fixed hierarchy. In this paper we present an extended layout algorithm for Voronoi treemaps that provides a high degree of layout similiarity for varying hierarchies, such as software-system hierarchies. The implementation uses a deterministic initial-distribution approach that reduces the variation in node positioning even if changes in the underlying hierarchy data occur. Compared to existing layout algorithms, our algorithm achieves lower error rates with respect to node areas in the case of weighted Voronoi diagrams, which we show in a comparative study.
-
Dübel, Steve, Martin Röhlig, Heidrun Schumann, and Matthias Trapp. “2D And 3D Presentation Of Spatial Data: A Systematic Review”. In Ieee Vis International Workshop On 3Dvis, 2014.
The question whether to use 2D or 3D for data visualization is generally difficult to decide. Two-dimensional and three-dimensional visualization techniques exhibit different advantages and disadvantages related to various perceptual and technical aspects such as occlusion, clutter, distortion, or scalability. To facilitate problem understanding and comparison of existing visualization techniques with regard to these aspects, this report introduces a systematization based on presentation characteristics. It enables a categorization with respect to combinations of static 2D and 3D presentations of attributes and their spatial reference. Further, it complements ex-isting systematizations of data in an effort to formalize a common terminology and theoretical framework for this problem domain. We demonstrate our approach by reviewing different visualization techniques of spatial data according to the presented systematization.
-
Klimke, Jan, Benjamin Hagedorn, Matthias Trapp, and Jürgen Döllner. “Web-Based And Mobile Provisioning Of Virtual 3D Reconstructions”. In Tagungsband Der 12. Konferenz Kultur Und Informatik: Reality And Virtuality, R. Franken-Wendelstorf, Lindinger, E., and Sieck, J., 17--28. Werner Hülsbusch Verlag, 2014.
Communication of cultural heritage by means of digital information systems has been gaining more and more importance over recent years. Interactive virtual 3D applications enable users to explore 3D virtual reconstructions in real-time, to directly interact with the contained digital cultural heritage artifacts, and to obtain insights into this data. Nevertheless, these artifacts are usually very detailed and complex 3D models that are hard to handle for end-user systems. This paper presents the concept and a prototypical implementation of an image-based, web-based approach for the communication of digital cultural heritage and its provisioning for the Web and mobile devices by the example of the project Colonia3D – a high-detail, virtual reconstruction and high-detail 3D city model of Roman Cologne. Through this web-based and mobile provisioning, complex digital reconstructions can be used, e.g., on-site to match local findings and reconstructions.
-
Klimke, J., B. Hagedorn, M. Trapp, and J. Döllner. “Web-Based And Mobile Provisioning Of Virtual 3D Reconstructions”. In Kultur Und Informatik: Reality And Virtuality, 17--28. Werner Hülsbusch Verlag, 2014.
-
Klimke, J., B. Hagedorn, M. Trapp, and J. Döllner. “Web-Based And Mobile Provisioning Of Virtual 3D Reconstructions”. In Kultur Und Informatik: Reality And Virtuality, 17--28. Werner Hülsbusch Verlag, 2014.
-
Buschmann, Stefan, Matthias Trapp, Patrick Lühne, and Jürgen Döllner. “Hardware-Accelerated Attribute Mapping For Interactive Visualization Of Complex 3D Trajectories”. In Proceedings Of The 5Th International Conference On Information Visualization Theory And Applications (Ivapp 2014), 355-363. SCITEPRESS – Science and Technology Publications, 2014.
The visualization of 3D trajectories of moving objects and related attributes in 3D virtual environments represents a fundamental functionality in various visualization domains. Interactive rendering and visual analytics of such attributed trajectories involves both conceptual questions as well as technical challenges. Specifically, the mapping of trajectory attributes to rendering primitives and appearance represents a challenging task in the case of large data sets of high geometric complexity. There are various visualization approaches and rendering techniques considering specific aspects of these mappings to facilitate visualization and analysis of this kind of data. To solve the underlying general mapping problem efficiently, we developed an approach that uses and combines diverse types of visualizations, rather than being tailored to a specific use case. This paper describes an interactive rendering system for the visualization of 3D trajectories that enables the combinations of different mappings as well as their dynamic configuration at runtime. A fully hardware-accelerated implementation enables the processing of large sets of attributed 3D trajectories in real-time.
-
Discher, Sören, Rico Richter, and Jürgen Döllner. “Konzepte Für Eine Service-Basierte Systemarchitektur Zur Integration, Prozessierung Und Analyse Von Massiven 3D-Punktwolken”. In Tagungsbände Der 34. Wissenschaftlich-Technischen Jahrestagung Der Dgpf, 2014.
Die Nutzung von hoch aufgelösten, räumlich überlappenden und multitemporalen 3D-Punktwolken im Kontext von Geoinformationssystemen stellt hohe Anforderungen an die Leistungsfähigkeit der zugrundeliegenden Software- und Hardwaresysteme. Um angesichts eines weiter zunehmenden Datenaufkommens ein effizientes und wirtschaftliches Arbeiten mit solchen Daten zu ermöglichen, schlagen wir die Nutzung einer service-basierten Software- und Geodateninfrastruktur vor, die eine Erfassung, Aktualisierung und Bereitstellung von 3D-Punktwolken im Sinne eines kontinuierlichen Prozesses ermöglicht. In diesem Beitrag erläutern wir die grundlegenden Anforderungen und den konzeptionellen Aufbau einer entsprechenden Infrastruktur, die unter anderem die bedarfsgerechte Bereitstellung ausgewählter Bereiche einer 3D Punktwolke anhand von semantischen oder temporalen Attributen unterstützt.
-
Limberger, Daniel, and Jürgen Döllner. “Painting Per-Pixel Parametrization For Interactive Image Filtering”. In Expressive Poster Session, 2014.
We present a photo-editing method that enables per-pixel parameter manipulation of image filtering by means of interactive painting. Predefined as well as custom image filters are exposed to the user, as a parametrizable composition of image operations. Brushes, as a sequences of actions mapping user inputs (in terms of brush shape, flow, pressure, etc.) to arbitrary functions or convolution operators, are used to draw within the parameter space. Our tool demonstrates that interactive painting can be used to, e.g., locally tweak inadequate parametrization and, furthermore, provides a blueprint for an open, collaborative photo-editing platform.
-
Hagedorn, Benjamin, and Jürgen Döllner. “Service-Based 3D Rendering And Interactive 3D Visualization (2012)”. In Hpi Future Soc Lab: Proceedings 2012, Christoph Meinel, Polze, Andreas, Oswald, Gerhard, Strotmann, Rolf, Seibold, Ulrich, and Bernhard, Schulzki, 7-10. Technische Berichte Des Hasso-Plattner-Instituts Für Softwaresystemtechnik An Der Universität Potsdam. Universitätsverlag Potsdam, 2014. https://publishup.uni-potsdam.de/opus4-ubp/files/6986/tbhpi85.pdf.
This report describes the subject and preliminary results of our work in the context of the HPI Future SOC Lab, which generally aims on how to exploit high performance computing (HPC) capabilities for service-based 3D rendering and service‐based, interactive 3D visualization. A major focus is on the application of HPC technologies for the creation, management, analysis, and visualization of and interaction with virtual 3D environments, especially with complex 3D city models.
-
Hagedorn, Benjamin, and Jürgen Döllner. “Service-Based 3D Rendering And Interactive 3D Visualization”. In Hpi Future Soc Lab: Proceedings 2013, Christoph Meinel, Polze, Andreas, Oswald, Gerhard, Strotmann, Rolf, Seibold, Ulrich, and Schulzki, Bernhard, 3-5. Technische Berichte Des Hasso-Plattner-Instituts Für Softwaresystemtechnik An Der Universität Potsdam. Universitätsverlag Potsdam, 2014. https://publishup.uni-potsdam.de/opus4-ubp/files/6982/tbhpi88.pdf.
-
Buschmann, Stefan, Matthias Trapp, and Jürgen Döllner. “Real-Time Animated Visualization Of Massive Air-Traffic Trajectories”. In Proceedings Of Cyberworlds 2014, 172-181. IEEE Computer Society, 2014.
With increasing numbers of flights world-wide and a continuing rise in airport traffic, air-traffic management is faced with a number of challenges. These include monitoring, reporting, planning, and problem analysis of past and current air traffic, e.g., to identify hotspots, minimize delays, or to optimize sector assignments to air-traffic controllers. Interactive and dynamic 3D visualization and visual analysis of massive aircraft trajectories, i.e., analytical reasoning enabled by interactive cyber worlds, can be used to approach these challenges. To facilitate this kind of analysis, especially in the context of real-time data, interactive tools for filtering, mapping, and rendering are required. In particular, the mapping process should be configurable at run-time and support both static mappings and animations to allow users to effectively explore and realize movement dynamics. However, with growing data size and complexity, these stages of the visualization pipeline require computational efficient implementations to be capable of processing within real-time constraints. This paper presents an approach for real-time animated visualization of massive air-traffic data, that implements all stages of the visualization pipeline based on GPU techniques for efficient processing. It enables (1) interactive spatio-temporal filtering, (2) generic mapping of trajectory attributes to geometric representations and appearances, as well as (3) real-time rendering within 3D virtual environments, such as virtual 3D airport and city models. Based on this pipeline, different visualization metaphors (e.g., temporal focus+context, density maps, and overview+detail visualization) are
