RI:Small:Collaborative Research: Understanding Human-Object Interactions from First-person and Third-person Videos
RI:Small:协作研究:从第一人称和第三人称视频中理解人与物体的交互
基本信息
- 批准号:2204808
- 负责人:
- 金额:$ 15万
- 依托单位:
- 依托单位国家:美国
- 项目类别:Standard Grant
- 财政年份:2021
- 资助国家:美国
- 起止时间:2021-11-01 至 2022-07-31
- 项目状态:已结题
- 来源:
- 关键词:
项目摘要
Ubiquitous cameras, together with ever increasing computing resources, are dramatically changing the nature of visual data and their analysis. Cities are adopting networked camera systems for policing and intelligent resource allocation, and individuals are recording their lives using wearable devices. For these camera systems to become truly smart and useful for people, it is crucial that they understand interesting objects in the scene and detect ongoing activities/events, while jointly considering continuous 24/7 videos from multiple sources. Such object-level and activity-level awareness in hospitals, elderly homes, and public places would provide assistive and quality-of-life technology for disabled and elderly people, provide intelligent surveillance systems to prevent crimes, and allow smart usage of environmental resources. This project will investigate novel computer vision algorithms that combine 1st-person videos (from wearable cameras) and 3rd-person videos (from static environmental cameras) for joint recognition of humans, objects, and their interactions. The key idea is to combine the two views' complementary and unique advantages for joint visual scene understanding. To this end, it will create a new dataset, and develop new algorithms that learn to recognize objects jointly across the views, learn human-object and human-human relationships through the two views, and anonymize the videos to preserve users' privacies. The project will provide new algorithms that have the potential to benefit applications in smart environments, security, and quality-of-life assistive technologies. The project will also perform complementary educational and outreach activities that engage students in research and STEM.This project will develop novel algorithms that learn from joint 1st-person videos (from wearable cameras) and 3rd-person videos (from static environmental cameras) for joint recognition of humans, objects, and their interactions. The 1st-person view is ideal for object recognition, while the 3rd-person view is ideal for human activity recognition. Thus, this project will investigate unique solutions to challenging problems that would otherwise be difficult to overcome when analyzing each viewpoint in isolation. The main research directions will be: (1) creating a benchmark 1st-person and 3rd-person video dataset to investigate this new problem; and developing algorithms that (2) learn to establish object and human correspondences between the two views; (3) learn object-action relationships across the views; and (4) anonymize the visual data for privacy-preserving visual recognition.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
无处不在的相机以及不断增加的计算资源正在极大地改变视觉数据及其分析的性质。城市正在采用网络摄像头系统进行警务和智能资源分配,个人正在使用可穿戴设备记录他们的生活。为了使这些摄像头系统变得真正智能且对人们有用,至关重要的是它们能够理解场景中有趣的物体并检测正在进行的活动/事件,同时共同考虑来自多个来源的连续 24/7 视频。医院、养老院和公共场所的这种对象级和活动级意识将为残疾人和老年人提供辅助和生活质量技术,提供智能监控系统以预防犯罪,并允许智能利用环境资源。 该项目将研究新颖的计算机视觉算法,将第一人称视频(来自可穿戴相机)和第三人称视频(来自静态环境相机)相结合,以联合识别人类、物体及其交互。其关键思想是结合两种视图的互补和独特优势来进行联合视觉场景理解。为此,它将创建一个新的数据集,并开发新的算法,学习跨视图共同识别对象,通过两个视图学习人与对象和人与人的关系,并对视频进行匿名化以保护用户的隐私。该项目将提供新的算法,有可能使智能环境、安全和生活质量辅助技术中的应用受益。该项目还将开展互补的教育和外展活动,让学生参与研究和 STEM。该项目将开发新颖的算法,从联合第一人称视频(来自可穿戴摄像机)和第三人称视频(来自静态环境摄像机)中学习,以联合识别人类、物体及其交互。第一人称视图非常适合物体识别,而第三人称视图非常适合人类活动识别。因此,该项目将研究独特的解决方案来解决具有挑战性的问题,否则在单独分析每个观点时很难克服这些问题。主要研究方向是:(1)创建基准第一人称和第三人称视频数据集来研究这个新问题;开发算法:(2) 学习在两种视图之间建立物体和人类的对应关系; (3) 学习跨视图的对象-动作关系; (4) 对视觉数据进行匿名处理,以保护隐私的视觉识别。该奖项反映了 NSF 的法定使命,并通过使用基金会的智力价值和更广泛的影响审查标准进行评估,认为值得支持。
项目成果
期刊论文数量(6)
专著数量(0)
科研奖励数量(0)
会议论文数量(0)
专利数量(0)
Seeing the Unseen: Predicting the First-Person Camera Wearer’s Location and Pose in Third-Person Scenes
- DOI:10.1109/iccvw54120.2021.00384
- 发表时间:2021-10
- 期刊:
- 影响因子:0
- 作者:Yangming Wen;Krishna Kumar Singh;Markham H. Anderson;Wei-Pang Jan;Yong Jae Lee
- 通讯作者:Yangming Wen;Krishna Kumar Singh;Markham H. Anderson;Wei-Pang Jan;Yong Jae Lee
The Two Dimensions of Worst-case Training and Their Integrated Effect for Out-of-domain Generalization
- DOI:10.1109/cvpr52688.2022.00941
- 发表时间:2022-04
- 期刊:
- 影响因子:0
- 作者:Zeyi Huang;Haohan Wang;Dong Huang;Yong Jae Lee;Eric P. Xing
- 通讯作者:Zeyi Huang;Haohan Wang;Dong Huang;Yong Jae Lee;Eric P. Xing
Toward Learning Human-aligned Cross-domain Robust Models by Countering Misaligned Features
- DOI:
- 发表时间:2021-11
- 期刊:
- 影响因子:0
- 作者:Haohan Wang;Zeyi Huang;Hanlin Zhang;Eric P. Xing
- 通讯作者:Haohan Wang;Zeyi Huang;Hanlin Zhang;Eric P. Xing
Equine Pain Behavior Classification via Self-Supervised Disentangled Pose Representation
- DOI:10.1109/wacv51458.2022.00023
- 发表时间:2021-08
- 期刊:
- 影响因子:0
- 作者:M. Rashid;S. Broomé;K. Ask;Elin Hernlund;P. Andersen;H. Kjellström;Yong Jae Lee
- 通讯作者:M. Rashid;S. Broomé;K. Ask;Elin Hernlund;P. Andersen;H. Kjellström;Yong Jae Lee
Delving Deeper into Anti-Aliasing in ConvNets
- DOI:10.1007/s11263-022-01672-y
- 发表时间:2020-08
- 期刊:
- 影响因子:19.5
- 作者:Xueyan Zou;Fanyi Xiao;Zhiding Yu;Yong Jae Lee
- 通讯作者:Xueyan Zou;Fanyi Xiao;Zhiding Yu;Yong Jae Lee
{{
item.title }}
{{ item.translation_title }}
- DOI:
{{ item.doi }} - 发表时间:
{{ item.publish_year }} - 期刊:
- 影响因子:{{ item.factor }}
- 作者:
{{ item.authors }} - 通讯作者:
{{ item.author }}
数据更新时间:{{ journalArticles.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ monograph.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ sciAawards.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ conferencePapers.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ patent.updateTime }}
Yong Jae Lee其他文献
Who Moved My Cheese? Automatic Annotation of Rodent Behaviors with Convolutional Neural Networks
谁动了我的奶酪?
- DOI:
- 发表时间:
2017 - 期刊:
- 影响因子:0
- 作者:
Zhongzheng Ren;Adriana Noronha Annie;Vogel Ciernia;Yong Jae Lee - 通讯作者:
Yong Jae Lee
Pancytopenia Associated with Hypopituitarism in an Acromegaly Patient: A Case Report and a Review of the Literature
肢端肥大症患者全血细胞减少症与垂体机能减退相关:病例报告及文献综述
- DOI:
- 发表时间:
2012 - 期刊:
- 影响因子:0
- 作者:
J. Koh;Yong Jae Lee;J. Kang;B. Choi;Y. Jeon;Sang Soo Kim;B. Kim;I. Kim - 通讯作者:
I. Kim
Mo1763 - Fecal Calprotectin Versus Fecal Immunochemical Test for the Prediction of Mucosal Healing and Endoscopic Activity in Ulcerative Colitis
- DOI:
10.1016/s0016-5085(17)32683-5 - 发表时间:
2017-04-01 - 期刊:
- 影响因子:
- 作者:
Dae Gon Ryu;Hyung Wook Kim;Cheol Woong Choi;Dae Hwan Kang;Su Bum Park;Su Jin Kim;Hyeong Seok Nam;Jeong Seok Lee;Hyeong Jin Kim;Il Eok Jo;Yong Jae Lee - 通讯作者:
Yong Jae Lee
Sa1264 - Location Features of Early Gastric Cancer Treated with Endoscopic Submucosal Dissection
- DOI:
10.1016/s0016-5085(17)31162-9 - 发表时间:
2017-04-01 - 期刊:
- 影响因子:
- 作者:
Dae Gon Ryu;Cheol Woong Choi;Dae Hwan Kang;Hyung Wook Kim;Su Bum Park;Su Jin Kim;Hyeong Seok Nam;Hyeong Jin Kim;Jeong Seok Lee;Il Eok Jo;Yong Jae Lee - 通讯作者:
Yong Jae Lee
The efficacy of serum CA125 and HE4 as a prognostic marker for the germline BRCA-affected patients in high-grade serous carcinoma (1175)
血清 CA125 和 HE4 作为生殖系 BRCA 影响的高级别浆液性癌患者预后标志物的疗效(1175)
- DOI:
10.1016/j.ygyno.2023.06.095 - 发表时间:
2023-09-01 - 期刊:
- 影响因子:4.100
- 作者:
Young Joo Lee;Soo Min Hong;Yong Jae Lee;Jung-Yun Lee;Sang Wun Kim;Sunghoon Kim;Young Tae Kim;Eun Ji Nam - 通讯作者:
Eun Ji Nam
Yong Jae Lee的其他文献
{{
item.title }}
{{ item.translation_title }}
- DOI:
{{ item.doi }} - 发表时间:
{{ item.publish_year }} - 期刊:
- 影响因子:{{ item.factor }}
- 作者:
{{ item.authors }} - 通讯作者:
{{ item.author }}
{{ truncateString('Yong Jae Lee', 18)}}的其他基金
CAREER: Weakly-Supervised Visual Scene Understanding: Combining Images and Videos, and Going Beyond Semantic Tags
职业:弱监督视觉场景理解:结合图像和视频,超越语义标签
- 批准号:
2150012 - 财政年份:2021
- 资助金额:
$ 15万 - 项目类别:
Continuing Grant
CAREER: Weakly-Supervised Visual Scene Understanding: Combining Images and Videos, and Going Beyond Semantic Tags
职业:弱监督视觉场景理解:结合图像和视频,超越语义标签
- 批准号:
1751206 - 财政年份:2018
- 资助金额:
$ 15万 - 项目类别:
Continuing Grant
RI:Small:Collaborative Research: Understanding Human-Object Interactions from First-person and Third-person Videos
RI:Small:协作研究:从第一人称和第三人称视频中理解人与物体的交互
- 批准号:
1812850 - 财政年份:2018
- 资助金额:
$ 15万 - 项目类别:
Standard Grant
EAGER: Leveraging Synthetic Data for Visual Reasoning and Representation Learning with Minimal Human Supervision
EAGER:在最少的人类监督下利用合成数据进行视觉推理和表示学习
- 批准号:
1748387 - 财政年份:2017
- 资助金额:
$ 15万 - 项目类别:
Standard Grant
相似国自然基金
昼夜节律性small RNA在血斑形成时间推断中的法医学应用研究
- 批准号:
- 批准年份:2024
- 资助金额:0.0 万元
- 项目类别:省市级项目
tRNA-derived small RNA上调YBX1/CCL5通路参与硼替佐米诱导慢性疼痛的机制研究
- 批准号:n/a
- 批准年份:2022
- 资助金额:10.0 万元
- 项目类别:省市级项目
Small RNA调控I-F型CRISPR-Cas适应性免疫性的应答及分子机制
- 批准号:32000033
- 批准年份:2020
- 资助金额:24.0 万元
- 项目类别:青年科学基金项目
Small RNAs调控解淀粉芽胞杆菌FZB42生防功能的机制研究
- 批准号:31972324
- 批准年份:2019
- 资助金额:58.0 万元
- 项目类别:面上项目
变异链球菌small RNAs连接LuxS密度感应与生物膜形成的机制研究
- 批准号:81900988
- 批准年份:2019
- 资助金额:21.0 万元
- 项目类别:青年科学基金项目
肠道细菌关键small RNAs在克罗恩病发生发展中的功能和作用机制
- 批准号:31870821
- 批准年份:2018
- 资助金额:56.0 万元
- 项目类别:面上项目
基于small RNA 测序技术解析鸽分泌鸽乳的分子机制
- 批准号:31802058
- 批准年份:2018
- 资助金额:26.0 万元
- 项目类别:青年科学基金项目
Small RNA介导的DNA甲基化调控的水稻草矮病毒致病机制
- 批准号:31772128
- 批准年份:2017
- 资助金额:60.0 万元
- 项目类别:面上项目
基于small RNA-seq的针灸治疗桥本甲状腺炎的免疫调控机制研究
- 批准号:81704176
- 批准年份:2017
- 资助金额:20.0 万元
- 项目类别:青年科学基金项目
水稻OsSGS3与OsHEN1调控small RNAs合成及其对抗病性的调节
- 批准号:91640114
- 批准年份:2016
- 资助金额:85.0 万元
- 项目类别:重大研究计划
相似海外基金
Collaborative Research: RI: Small: Foundations of Few-Round Active Learning
协作研究:RI:小型:少轮主动学习的基础
- 批准号:
2313131 - 财政年份:2023
- 资助金额:
$ 15万 - 项目类别:
Standard Grant
Collaborative Research: RI: Small: Deep Constrained Learning for Power Systems
合作研究:RI:小型:电力系统的深度约束学习
- 批准号:
2345528 - 财政年份:2023
- 资助金额:
$ 15万 - 项目类别:
Standard Grant
Collaborative Research: RI: Small: Motion Fields Understanding for Enhanced Long-Range Imaging
合作研究:RI:小型:增强远程成像的运动场理解
- 批准号:
2232298 - 财政年份:2023
- 资助金额:
$ 15万 - 项目类别:
Standard Grant
Collaborative Research: RI: Small: End-to-end Learning of Fair and Explainable Schedules for Court Systems
合作研究:RI:小型:法院系统公平且可解释的时间表的端到端学习
- 批准号:
2232055 - 财政年份:2023
- 资助金额:
$ 15万 - 项目类别:
Standard Grant
Collaborative Research: RI: Small: End-to-end Learning of Fair and Explainable Schedules for Court Systems
合作研究:RI:小型:法院系统公平且可解释的时间表的端到端学习
- 批准号:
2232054 - 财政年份:2023
- 资助金额:
$ 15万 - 项目类别:
Standard Grant
Collaborative Research: RI: Small: Motion Fields Understanding for Enhanced Long-Range Imaging
合作研究:RI:小型:增强远程成像的运动场理解
- 批准号:
2232300 - 财政年份:2023
- 资助金额:
$ 15万 - 项目类别:
Standard Grant
Collaborative Research: RI: Small: Motion Fields Understanding for Enhanced Long-Range Imaging
合作研究:RI:小型:增强远程成像的运动场理解
- 批准号:
2232299 - 财政年份:2023
- 资助金额:
$ 15万 - 项目类别:
Standard Grant
Collaborative Research: RI: Small: Foundations of Few-Round Active Learning
协作研究:RI:小型:少轮主动学习的基础
- 批准号:
2313130 - 财政年份:2023
- 资助金额:
$ 15万 - 项目类别:
Standard Grant
RI: Small: Collaborative Research: Evolutionary Approach to Optimal Morphology and Control of Transformable Soft Robots
RI:小型:协作研究:可变形软机器人的最佳形态和控制的进化方法
- 批准号:
2325491 - 财政年份:2023
- 资助金额:
$ 15万 - 项目类别:
Standard Grant
Collaborative Research: RI: Small: End-to-end Learning of Fair and Explainable Schedules for Court Systems
合作研究:RI:小型:法院系统公平且可解释的时间表的端到端学习
- 批准号:
2334936 - 财政年份:2023
- 资助金额:
$ 15万 - 项目类别:
Standard Grant