UMPIRE: United Model for the Perception of Interactions in visuoauditory REcognition
裁判:视觉听觉识别中交互感知的联合模型
基本信息
- 批准号:EP/T004991/1
- 负责人:
- 金额:$ 127.65万
- 依托单位:
- 依托单位国家:英国
- 项目类别:Fellowship
- 财政年份:2020
- 资助国家:英国
- 起止时间:2020 至 无数据
- 项目状态:未结题
- 来源:
- 关键词:
项目摘要
Humans interact with tens of objects daily, at home (e.g. cooking/cleaning) or outdoors (e.g. ticket machines/shopping bags), during working (e.g. assembly/machinery) or leisure hours (e.g. playing/sports), individually or collaboratively. When observing people interacting with objects, our vision assisted by the sense of hearing is the main tool to perceive these interactions. Let's take the example of boiling water from a kettle. We observe the actor press a button, wait and hear the water boil and the kettle's light go off before water is used for, say, preparing tea. The perception process is formed from understanding intentional interactions (called ideomotor actions) as well as reactive actions to dynamic stimuli in the environment (referred to as sensormotor actions). As observers, we understand and can ultimately replicate such interactions using our sensory input, along with our underlying complex cognitive processes of event perception. Evidence in behavioural sciences demonstrates that these human cognitive processes are highly modularised, and these modules collaborate to achieve our outstanding human-level perception.However, current approaches in artificial intelligence are lacking in their modularity and accordingly their capabilities. To achieve human-level perception of object interactions, including online perception when the interaction results in mistakes (e.g. water is spilled) or risks (e.g. boiling water is spilled), this fellowship focuses on informing computer vision and machine learning models, including deep learning architectures, from well-studied cognitive behavioural frameworks.Deep learning architectures have achieved superior performance, compared to their hand-crafted predecessors, on video-level classification, however their performance on fine-grained understanding within the video remains modest. Current models are easily fooled by similar motions or incomplete actions, as shown by recent research. This fellowship focuses on empowering these models through modularisation, a principle proven since the 50s in Fodor's Modularity of the Mind, and frequently studied by cognitive psychologists in controlled lab environments. Modularity of high-level perception, along with the power of deep learning architectures, will bring a new understanding to videos analysis previously unexplored.The targeted perception, of daily and rare object interactions, will lay the foundations for applications including assistive technologies using wearable computing, and robot imitation learning. We will work closely with three industrial partners to pave potential knowledge transfer paths to applications.Additionally, the fellowship will actively engage international researchers through workshops, benchmarks and public challenges on large datasets, to encourage other researchers to address problems related to fine-grained perception in video understanding.
人类每天在家里(例如做饭/清洁)或户外(例如售票机/购物袋)、在工作期间(例如组装/机械)或休闲时间(例如玩/运动)单独或协作地与数十个物体交互。在观察人与物体的互动时,我们的视觉辅以听觉是感知这些互动的主要工具。让我们以烧开水壶里的水为例。我们观察演员按下按钮,等着听到水开了,茶壶的灯熄灭了,然后水才被用来泡茶。知觉过程是通过理解有意的相互作用(称为意念运动动作)以及对环境中动态刺激的反应动作(称为感觉运动动作)而形成的。作为观察者,我们理解并最终可以使用我们的感官输入以及我们对事件感知的潜在复杂认知过程来复制这种互动。行为科学中的证据表明,这些人类认知过程是高度模块化的,这些模块协作实现了我们出色的人类水平的感知。然而,目前人工智能领域的方法缺乏模块化,因此缺乏能力。为了实现人类对对象交互的感知,包括当交互导致错误(例如,水溢出)或风险(例如,沸水溢出)时的在线感知,该奖学金专注于从经过充分研究的认知行为框架中通知计算机视觉和机器学习模型,包括深度学习体系结构。与其手工制作的前身相比,深度学习体系结构在视频级别分类方面取得了更好的性能,但它们在视频中的细粒度理解方面的表现仍然不是很好。最近的研究表明,当前的模型很容易被类似的动作或不完整的动作所愚弄。该奖学金专注于通过模块化来增强这些模型的能力,这一原则自50年代以来在Fodor的《大脑模块化》中得到了验证,并经常被认知心理学家在受控实验室环境中研究。高层感知的模块化,以及深度学习架构的能力,将为以前未开发的视频分析带来新的理解。对日常和稀有对象交互的有针对性的感知,将为使用可穿戴计算的辅助技术和机器人模仿学习等应用奠定基础。我们将与三个行业伙伴密切合作,为应用程序铺平潜在的知识转移道路。此外,该奖学金将通过研讨会、基准和大型数据集的公开挑战积极吸引国际研究人员,以鼓励其他研究人员解决与视频理解中的细粒度感知相关的问题。
项目成果
期刊论文数量(10)
专著数量(0)
科研奖励数量(0)
会议论文数量(0)
专利数量(0)
Rescaling Egocentric Vision
重新调整以自我为中心的愿景
- DOI:10.48550/arxiv.2006.13256
- 发表时间:2020
- 期刊:
- 影响因子:0
- 作者:Damen D
- 通讯作者:Damen D
Action Modifiers: Learning From Adverbs in Instructional Videos
- DOI:10.1109/cvpr42600.2020.00095
- 发表时间:2019-12
- 期刊:
- 影响因子:0
- 作者:Hazel Doughty;I. Laptev;W. Mayol-Cuevas;D. Damen
- 通讯作者:Hazel Doughty;I. Laptev;W. Mayol-Cuevas;D. Damen
Computer Vision - ACCV 2022 - 16th Asian Conference on Computer Vision, Macao, China, December 4-8, 2022, Proceedings, Part IV
计算机视觉 - ACCV 2022 - 第十六届亚洲计算机视觉会议,中国澳门,2022 年 12 月 4-8 日,会议记录,第四部分
- DOI:10.1007/978-3-031-26316-3_27
- 发表时间:2023
- 期刊:
- 影响因子:0
- 作者:Fragomeni A
- 通讯作者:Fragomeni A
Learning Temporal Sentence Grounding From Narrated EgoVideos
- DOI:10.48550/arxiv.2310.17395
- 发表时间:2023-10
- 期刊:
- 影响因子:0
- 作者:Kevin Flanagan;D. Damen;Michael Wray
- 通讯作者:Kevin Flanagan;D. Damen;Michael Wray
Epic-Sounds: A Large-scale Dataset of Actions That Sound
史诗般的声音:声音动作的大规模数据集
- DOI:10.48550/arxiv.2302.00646
- 发表时间:2023
- 期刊:
- 影响因子:0
- 作者:Huh J
- 通讯作者:Huh J
{{
item.title }}
{{ item.translation_title }}
- DOI:
{{ item.doi }} - 发表时间:
{{ item.publish_year }} - 期刊:
- 影响因子:{{ item.factor }}
- 作者:
{{ item.authors }} - 通讯作者:
{{ item.author }}
数据更新时间:{{ journalArticles.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ monograph.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ sciAawards.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ conferencePapers.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ patent.updateTime }}
Dima Damen其他文献
Correspondence, Matching and Recognition
- DOI:
10.1007/s11263-015-0827-8 - 发表时间:
2015-05-14 - 期刊:
- 影响因子:9.300
- 作者:
Tilo Burghardt;Dima Damen;Walterio Mayol-Cuevas;Majid Mirmehdi - 通讯作者:
Majid Mirmehdi
Cognitive Robotics Systems
- DOI:
10.1007/s10846-015-0244-9 - 发表时间:
2015-06-03 - 期刊:
- 影响因子:2.800
- 作者:
Lazaros Nalpantidis;Renaud Detry;Dima Damen;Gabriele Bleser;Maya Cakmak;Mustafa Suphi Erden - 通讯作者:
Mustafa Suphi Erden
Explaining Activities as Consistent Groups of Events
- DOI:
10.1007/s11263-011-0497-0 - 发表时间:
2011-10-05 - 期刊:
- 影响因子:9.300
- 作者:
Dima Damen;David Hogg - 通讯作者:
David Hogg
Dima Damen的其他文献
{{
item.title }}
{{ item.translation_title }}
- DOI:
{{ item.doi }} - 发表时间:
{{ item.publish_year }} - 期刊:
- 影响因子:{{ item.factor }}
- 作者:
{{ item.authors }} - 通讯作者:
{{ item.author }}
{{ truncateString('Dima Damen', 18)}}的其他基金
LOCATE: LOcation adaptive Constrained Activity recognition using Transfer learning
LOCATE:使用迁移学习的位置自适应约束活动识别
- 批准号:
EP/N033779/1 - 财政年份:2016
- 资助金额:
$ 127.65万 - 项目类别:
Research Grant
相似海外基金
The Model United Nations as a Tool for Global Competency Development
模拟联合国作为全球能力发展的工具
- 批准号:
23K00666 - 财政年份:2023
- 资助金额:
$ 127.65万 - 项目类别:
Grant-in-Aid for Scientific Research (C)
Collaborative Research: Assessing the timing and characteristics of deglacial Laurentide Ice Sheet thinning in the northeast United States through paired data-model analyses
合作研究:通过配对数据模型分析评估美国东北部冰消劳伦太德冰盖变薄的时间和特征
- 批准号:
2133699 - 财政年份:2022
- 资助金额:
$ 127.65万 - 项目类别:
Continuing Grant
Collaborative Research: Assessing the timing and characteristics of deglacial Laurentide Ice Sheet thinning in the northeast United States through paired data-model analyses
合作研究:通过配对数据模型分析评估美国东北部冰消劳伦太德冰盖变薄的时间和特征
- 批准号:
2133698 - 财政年份:2022
- 资助金额:
$ 127.65万 - 项目类别:
Continuing Grant
Collaborative Research: P2C2--Western United States Hydroclimate during the Last Interglacial: Developing Proxy Records and Using Model Intercomparison to Glimpse the Future
合作研究:P2C2——末次间冰期美国西部水文气候:开发代理记录并利用模型比对展望未来
- 批准号:
2102884 - 财政年份:2021
- 资助金额:
$ 127.65万 - 项目类别:
Standard Grant
Collaborative Research: P2C2--Western United States Hydroclimate during the Last Interglacial: Developing Proxy Records and Using Model Intercomparison to Glimpse the Future
合作研究:P2C2——末次间冰期美国西部水文气候:开发代理记录并利用模型比对展望未来
- 批准号:
2102901 - 财政年份:2021
- 资助金额:
$ 127.65万 - 项目类别:
Standard Grant
Collaborative Research: P2C2--Western United States Hydroclimate during the Last Interglacial: Developing Proxy Records and Using Model Intercomparison to Glimpse the Future
合作研究:P2C2——末次间冰期美国西部水文气候:开发代理记录并利用模型比对展望未来
- 批准号:
2102885 - 财政年份:2021
- 资助金额:
$ 127.65万 - 项目类别:
Standard Grant
Conducting an energy study and model of the Ross-Island Energy Grid and McMurdo Power Plant, in support of the National Science Foundation and the United States Antarctic Program.
在国家科学基金会和美国南极计划的支持下,对罗斯岛能源网和麦克默多发电厂进行能源研究和模型。
- 批准号:
2034195 - 财政年份:2020
- 资助金额:
$ 127.65万 - 项目类别:
Contract Interagency Agreement
Investigating the potential for West Nile virus outbreaks in the United Kingdom using a mathematical epidemiological model
使用数学流行病学模型调查西尼罗河病毒在英国爆发的可能性
- 批准号:
2427778 - 财政年份:2020
- 资助金额:
$ 127.65万 - 项目类别:
Studentship
Evaluating Portfolio Interventions for HIV Incidence Reduction in the United States: Development of a Novel Agent-Based Decision-Analytic Model for Dynamic Evaluations of Interventions
评估美国减少艾滋病毒发病率的组合干预措施:开发基于代理的新型决策分析模型,用于干预措施的动态评估
- 批准号:
9411456 - 财政年份:2017
- 资助金额:
$ 127.65万 - 项目类别:
Evaluating Portfolio Interventions for HIV Incidence Reduction in the United States: Development of a Novel Agent-Based Decision-Analytic Model for Dynamic Evaluations of Interventions
评估美国减少艾滋病毒发病率的组合干预措施:开发基于代理的新型决策分析模型,用于干预措施的动态评估
- 批准号:
10217960 - 财政年份:2017
- 资助金额:
$ 127.65万 - 项目类别: