One of the most important functionalities of a tele-teaching portal is the search for data. Since recording technology has become more and more inexpensive and easy to use, the amount of content produced for these portals has become huge. Therefore it is nearly impossible for students to find the required content without a search function. But even when the user has found the right content, he still needs to find the piece of information he requires within the 90 minutes of a lecture. Therefore, the development of an automated solution for lecture video indexing is hightly desirable and especially useful for e-learning and tele-teaching. In our study, we have applied Video OCR and ASR technologies described as follows:

• Video OCR: The text displayed in a lecture video is closely related to the lecture content. Therefore, it provides a valuable source for indexing and retrieving lecture video contents. Textual content can be detected, extracted and analyzed automatically by video Optical Character Recognition (OCR) techniques. In this project, we have developed an approach for automated lecture video indexing based on video OCR technology: Firstly, we developed a novel video segmenter to capture the real slide transition. Having adopted a localization-verification scheme we perform text detection secondly. We employ Stroke Width Transform (SWT) not only to remove false alarms from the text detection, but also to analyze the slide structure further. Unlike other OCR-based lecture video indexing approaches we utilize the geometrical information of detected text bounding boxes and stroke width value of texts, so that the summerized lecture outline can be extracted automatically from the OCR transcripts. The video indexing could be performed by using both, segmented slide shots and extracted lecture outlines (cf. Fig. 1).

• Automated Speech Recognition (ASR) for lecture videos: Speech is the most natural way of communication and also the main carrier of information in nearly all lectures. Therefore, it is of distinct advantage that the speech information can be used for automated indexing of lecture videos. However, most of existing lecture speech recognition systems have only low recognition accuracy, the Word Error Rates (WERs) having been reported from many research publications are approximately 40%–85%. The poor recognition results limit the quality of the later indexing process. Compared to the English language, German has a much higher lexical variety. A German recognition vocabulary is several times larger than a corresponding English one. In addition, German lecture videos in specific domains, as e.g., computer science are more difficult to recognize than common contents such as TV news due to several reasons such as,  poor recording quality, speaker‘s dialects, interfering noise or out of vocabulary problem by which many topic-related technical terms are not in the standard dictionary of a ASR-software. In this project, we have developed a solution that enables a continued improvement of recognition rate by creating and refining new speech training data. It is important that the involved tasks can be performed efficiently and even fully automated, if possible. For this reason, we have implemented an automated procedure for generating a phonetic dictioary and a method for splitting raw audio lecture data into small pieces which are used in the speech model training process. For the manual transcription step we have implemented a simple software-tool, which provides a Sphinx-Trainer comfortable output-format that can be directly used in speech model training process.

• Key-phrases Extraction from OCR and ASR transcripts: We have developed a method for automated extraction of indexable key-phrases from OCR and ASR transcripts. The high accurate lecture outline which were extracted from OCR texts can be used as a cue for the correct speech context. In this way, a more accurate refinement and extraction of key-phrases for ASR transcript could take place. This may solve the issue of building search indices for highly imperfect ASR transcripts. The extracted key-phrase collection could provides the summerized information about a lecture video or each video segment. On the other hand, the time-based distribution of key-phrases during a video or a segment can also be demonstrated that help to guide the user with navigation within the lecture video.

We have developed a novel lecture video segmenter for slide transition detection, video text detection and recognition methods. In addition, we have also applied automated speech recognition engine for speech text extraction. The question is then how to integrate such multimedia analysis engines into a lecture video portal, so that the analysis process could be easily handled and the efficient indexing functionalities could be provided to the users. In this project, we have designed and implemented an architecture (cf. Fig. 2), in which a set of management services have been created, i.e., network video analysis management, data transmission, result storage and visualization etc. Using this architecture, we can make the multimedia analysis engine applicable for a lecture video portal. The implementation of our approach can be found on teleTASK lecture video portal.