Deep Spatiotemporal Models for Video Representation Learning

用于视频表示学习的深度时空模型

• 批准号: 2431426
• 金额: --
• 依托单位: University of Warwick
• 依托单位国家: 英国
• 项目类别: Studentship
• 财政年份: 2020
• 资助国家: 英国
• 起止时间: 2020 至 无数据
• 项目状态: 未结题
• 来源: https://gtr.ukri.org/projects?ref=studentship-2431426
• 关键词: Deep; Spatiotemporal; Models; Video; Representation

The area of video representation learning is of interest to the Artificial Intelligence community, as it aims to foster the field of computer vision by using the dynamics of a scene to make critical decisions. Compared to using still images, video data gives more context to the information present, and this improves the quality of decisions made by systems built on the framework of video representation learning. The research aims to add to the field of video representation learning by developing a graph-based machine learning model that would outperform current state-of-the-art models. Although video data do not have a naturally occurring graph structure (unlike social networks), using a graph-based architecture significantly reduces the number of parameters needed in the model [1]. And this makes the proposed model suitable for memory-constrained devices like mobile phones.[1] Shirian, A., Tripathi, S., & Guha, T. (2021). Dynamic Emotion Modeling with Learnable Graphs and Graph Inception Network. IEEE Transactions on Multimedia.The aims and objectives of the research *The objective of the research is to:1. develop spatiotemporal graphs for modeling videos;2. learn the adjacency such that it is data dependent; and3. extend to heterogeneous graphs where data modalities can be multiple (e.g.,video with audio).The novelty of the research methodology (if any) *The research would contribute to the existing body of knowledge in video representation learning. The research aims to design a new graph-based machine learning architecture, new loss functions to better penalize our model and optimization techniques to speed up model training.The potential impact, applications, and benefits *The research would:1. improve the current autonomous navigation systems;2. be applied for object detection and action prediction in surveillance systems;3. be used to improve computer vision in robots;4. be suitable for memory-constrained devices like mobile phones;5. be able to narrate events happening in a scene. And this is useful for visuallyimpaired individuals etc.How the research relates to the remit *The research cuts across the field of Artificial Intelligence and robotics, mathematical science, and Information and communication technologies (ICT). And these are key areas of interest for the EPSRC, as this research wouldimprove visual perception in robotics, broaden the knowledge on the applicability of graph theory beyond social networks and the future of self driving cars would not be far from reach. Research Category; ICT [Information and Communication Technologies], Mathematical SciencesExternal Partner - Intel Labs, San Diego.

视频表示学习领域引起了人工智能界的兴趣，因为它旨在通过利用场景的动态做出关键决策来促进计算机视觉领域的发展。与使用静态图像相比，视频数据为所呈现的信息提供了更多上下文，这提高了基于视频表示学习框架的系统做出的决策质量。该研究旨在通过开发基于图的机器学习模型来拓展视频表示学习领域，该模型的性能将优于当前最先进的模型。尽管视频数据不具有自然发生的图结构（与社交网络不同），但使用基于图的架构可以显着减少模型中所需的参数数量[1]。这使得所提出的模型适用于手机等内存受限的设备。[1] Shirian, A.、Tripathi, S. 和 Guha, T. (2021)。使用可学习图和图初始网络进行动态情感建模。 IEEE Transactions on Multimedia。研究的目的和目标*研究的目标是：1。开发用于视频建模的时空图；2。学习邻接关系，使其依赖于数据；和3。扩展到数据模态可以是多种的异构图（例如，视频和音频）。研究方法的新颖性（如果有）*该研究将有助于视频表示学习的现有知识体系。该研究旨在设计一种新的基于图的机器学习架构、新的损失函数以更好地惩罚我们的模型和优化技术以加速模型训练。潜在的影响、应用和好处 *研究将：1。改进现有的自主导航系统；2．应用于监控系统中的目标检测和动作预测；3．用于改善机器人的计算机视觉；4．适用于手机等内存受限的设备；5．能够叙述场景中发生的事件。这对于视障人士等很有用。该研究与职权范围的关系如何*该研究跨越人工智能和机器人、数学科学以及信息和通信技术 (ICT) 领域。这些都是 EPSRC 感兴趣的关键领域，因为这项研究将改善机器人技术的视觉感知，拓宽图论在社交网络之外的适用性知识，自动驾驶汽车的未来也将指日可待。研究类别； ICT [信息和通信技术]，数学科学外部合作伙伴 - 圣地亚哥英特尔实验室。