Interactive Techniques for Visual Media Analysis and Processing (Wintersemester 2020/2021)

Lecturer: Dr. Matthias Trapp (Computergrafische Systeme) , Sebastian Pasewaldt (Computergrafische Systeme) , Sumit Shekhar (Computergrafische Systeme) , Max Reimann (Computergrafische Systeme) , Dr. Amir Semmo (Computergrafische Systeme)

General Information

• Weekly Hours: 4
• Credits: 6
• Graded: yes
• Enrolment Deadline: 01.10.2020 - 20.11.2020
• Teaching Form: Seminar / Project
• Enrolment Type: Compulsory Module
• Course Language: English

Programs, Module Groups & Modules

Description

This project seminar aims at master students who want to extend their image processing, computer vision, and computer graphics skills regarding the analysis, planning, and development of GPU-accelerated image and video processing techniques for mobile, desktop, and server systems. A short video showcasing results from last semester can be found here: https://youtu.be/YNgGWarBFEY

The course has mainly a project character and is subdivided into two parts:The course has mainly a project character and is subdivided into two parts:

The first part of the course is organized as a lecture series. The lecture topics are specified together with the seminar students and can include an introduction to the following basic concepts and foundations to:

• A short introduction into the field of image and video analytics,
• Techniques for image and video processing,
• Application development for mobile and Desktop/Server systems

Using specific image and video processing operations, the course teaches how fundamental processing techniques can be designed, developed, and tested.

In the second part of the course, participants will work individually, or as a team (max. 2 members), to implement assigned topics in the field of interactive image and video processing. For all target systems, we offer middleware for development, which can be used. For example, a C++ Framework for Desktop applications, an Android and iOS framework for mobile applications, and JS (Angular, Node framework) for service-based browser-applications will be provided. Topics for this project seminar cover the following domains (not limited to):

• Convolutional Neural Networks for image analysis and transformation*.
• Image and video processing for VR (Virtual Reality) and AR (Augmented Reality) applications.
• Web-based image processing using WebGPU or WebGL
• Integration of interactive rendering techniques in 3rd party applications (e.g., Photoshop plugin).
• Implementation of interactive image stylization and editing tools for desktop systems.
• Service-based image and video-processing*
• Automated video summarization approaches to efficiently and effectively shorten videos*.
• Web-app development for service-based image- and video processing
• Integration of deep learning frameworks into visual computing pipelines for videos
• Implementing effects for visual media abstraction*

 

Topics marked by a * are related to a joint research project of the Hasso Plattner Institute with Digital Masterpieces and the German Federal Ministry of Education, which investigates new concepts and techniques for multidimensional video processing and automatic video abstraction.

Requirements

• Successful completion of the lectures Computer Graphics I and/or II
• Basic knowledge of OpenGL (ES) Shading Languages or Metal Shading Language for image and video processing topics
• Basic knowledge/understanding of Neural Networks and/or ComputerVision algorithms for image and video analysis topics
• For Service/WebApps development: basic knowledge/understanding of Angular, Node.js, JavaScript, and Docker
• For Android mobile development: basic knowledge of Java programming language
• For iOS development: basic knowledge of Swift development
• For Desktop development: basic knowledge of C++ development   

Literature

Topic-specific material will be provided throughout the course

Learning

Project seminar (4 SWS/6 ECTS)

Examination

The final grade will be determined as follows:

• 50% Documented source code & prototypical application
• 15% Concept presentation (approx. 10 minutes)
• 25% Final presentation (approx. 25 minutes)
• 10 % Projectmanagement 

Dates

The kick-off meeting as well as the lecture sessions are implemented virtually via Zoom.us. The kick-off meeting will be on Monday, 02.11.2020, 13.30. 

The rest of the seminar is organized as follows: 

• The lectures will take place in the first 4 weeks (02.11. -27.11.2020)
• The project part will start in a self-organized way. Appointments with the supervisor are coordinated with the individual supervisors.
• The midterm presentation will take place in the week from 04.01.-15.01.2021.
• Based on student’s voting, the final presentation will take place March 2021.

Please note: In order to participate in one of the zoom meetings, please register in the respective moodle lecture: https://moodle.hpi3d.de/course/view.php?id=141

Zurück