Development of a software system for automatic scene and person indexing in scientific video archives

点击翻译按钮获取中文摘要

The German Broadcasting Archive (DRA) is a non-profit foundation under civil law, with offices in Frankfurt am Main and Potsdam-Babelsberg. The collection priorities of the archive at the Frankfurt location are audio recordings of contemporary history and music since the beginning of recording and historical recording media. In 1994, the DRA was extended to include the radio and television broadcasting archives of the former German Democratic Republic (GDR) initially at a location in Berlin, today in Potsdam-Babelsberg. In joint previous work, selected special GDR television broadcasts were digitized, and using innovative methods of content-based image and video analysis, have been made searchable. The material consists of approximately 3,000 hours of video footage, including the newscasts "Aktuelle Kamera", magazine broadcasts and 220 hours of the East German television film tradition. Through the use and development of automated methods for content-based video analysis, scientists have obtained new possibilities to carry out their searches for desired scenes, camera shots and persons, or for similar images. The sustainability of the achieved very good results of scene classification (detection of visual concepts) is intended to be further developed for future use. In the proposed project, a software system usable by archive staff will be developed to enable the DRA and other archives to easily integrate automatic video analysis methods for content-based image search. In this software system, deep learning methods will be employed, thereby making it possible at the same time to improve person and visual concept detection and expand them to other research-intensive parts of the television broadcasts. In particular, the project has the following objectives: 1. development of a sustainable software system for user-friendly expansion of two lexicons (concepts and persons) by archive staff, 2. integration of the software system in the digitalization workflows of the DRA to make automatic video analysis methods applicable to the total stock of television broadcasts in the archive, 3. improvement of the detection rates for concepts and persons by applying deep learning methods, 4. expansion of the visual concept lexicon by about 100 further concepts, 5. expansion of the person lexicon to about 100 persons of the GDR history, 6. improvement of the detection rates for concepts and persons through user feedback and similarity search, 7. development of appropriate visualizations for effective search. In this way, is it not only possible for scientists to carry out search queries on the basis of pre-defined concepts and persons, but they can also easily expand the extensive lexicons for visual concepts and persons for their own research tasks. The developed software tools will be made available to other scientific institutions as open source software.