Deep Spatiotemporal Models for Video Representation Learning

点击翻译按钮获取中文摘要

The area of video representation learning is of interest to the Artificial Intelligence community, as it aims to foster the field of computer vision by using the dynamics of a scene to make critical decisions. Compared to using still images, video data gives more context to the information present, and this improves the quality of decisions made by systems built on the framework of video representation learning. The research aims to add to the field of video representation learning by developing a graph-based machine learning model that would outperform current state-of-the-art models. Although video data do not have a naturally occurring graph structure (unlike social networks), using a graph-based architecture significantly reduces the number of parameters needed in the model [1]. And this makes the proposed model suitable for memory-constrained devices like mobile phones.[1] Shirian, A., Tripathi, S., & Guha, T. (2021). Dynamic Emotion Modeling with Learnable Graphs and Graph Inception Network. IEEE Transactions on Multimedia.The aims and objectives of the research *The objective of the research is to:1. develop spatiotemporal graphs for modeling videos;2. learn the adjacency such that it is data dependent; and3. extend to heterogeneous graphs where data modalities can be multiple (e.g.,video with audio).The novelty of the research methodology (if any) *The research would contribute to the existing body of knowledge in video representation learning. The research aims to design a new graph-based machine learning architecture, new loss functions to better penalize our model and optimization techniques to speed up model training.The potential impact, applications, and benefits *The research would:1. improve the current autonomous navigation systems;2. be applied for object detection and action prediction in surveillance systems;3. be used to improve computer vision in robots;4. be suitable for memory-constrained devices like mobile phones;5. be able to narrate events happening in a scene. And this is useful for visuallyimpaired individuals etc.How the research relates to the remit *The research cuts across the field of Artificial Intelligence and robotics, mathematical science, and Information and communication technologies (ICT). And these are key areas of interest for the EPSRC, as this research wouldimprove visual perception in robotics, broaden the knowledge on the applicability of graph theory beyond social networks and the future of self driving cars would not be far from reach. Research Category; ICT [Information and Communication Technologies], Mathematical SciencesExternal Partner - Intel Labs, San Diego.