CPS: Synergy: MONA LISA - Monitoring and Assisting with Actions

CPS:协同作用:蒙娜丽莎 - 监控和协助行动

基本信息

  • 批准号:
    1544787
  • 负责人:
  • 金额:
    $ 80万
  • 依托单位:
  • 依托单位国家:
    美国
  • 项目类别:
    Standard Grant
  • 财政年份:
    2015
  • 资助国家:
    美国
  • 起止时间:
    2015-09-01 至 2018-08-31
  • 项目状态:
    已结题

项目摘要

Cyber-physical systems of the near future will collaborate with humans. Such cognitive systems will need to understand what the humans are doing. They will need to interpret human action in real-time and predict the humans' immediate intention in complex, noisy and cluttered environments. This proposal puts forward a new architecture for cognitive cyber-physical systems that can understand complex human activities, and focuses specifically on manipulation activities. The proposed architecture, motivated by biological perception and control, consists of three layers. At the bottom layer are vision processes that detect, recognize and track humans, their body parts, objects, tools, and object geometry. The middle layer contains symbolic models of the human activity, and it assembles through a grammatical description the recognized signal components of the previous layer into a representation of the ongoing activity. Finally, at the top layer is the cognitive control, which decides which parts of the scene will be processed next and which algorithms will be applied where. It modulates the vision processes by fetching additional knowledge when needed, and directs the attention by controlling the active vision system to direct its sensors to specific places. Thus, the bottom layer is the perception, the middle layer is the cognition, and the top layer is the control. All layers have access to a knowledge base, built in offline processes, which contains the semantics about the actions.The feasibility of the approach will be demonstrated through the development of a smart manufacturing system, called MONA LISA, which assists humans in assembly tasks. This system will monitor humans as they perform assembly task. It will recognize the assembly action and determine whether it is correct and will communicate to the human possible errors and suggest ways to proceed. The system will have advanced visual sensing and perception; action understanding grounded in robotics and human studies; semantic and procedural-like memory and reasoning, and a control module linking high-level reasoning and low-level perception for real time, reactive and proactive engagement with the human assembler. The proposed work will bring new tools and methodology to the areas of sensor networks and robotics and is applicable, besides smart manufacturing, to a large variety of sectors and applications. Being able to analyze human behavior using vision sensors will have impact on many sectors, ranging from healthcare and advanced driver assistance to human robot collaboration. The project will also catalyze K-12 outreach, new courseware (undergraduate and graduate), publication and open-source software.
在不久的将来,网络物理系统将与人类合作。这样的认知系统需要理解人类在做什么。它们需要实时解释人类的行为,并预测人类在复杂、嘈杂和混乱的环境中的直接意图。该提案提出了一种新的认知网络物理系统架构,可以理解复杂的人类活动,并特别关注操纵活动。所提出的架构,生物感知和控制的动机,由三层组成。在底层是视觉过程,检测,识别和跟踪人类,他们的身体部位,物体,工具和物体几何形状。中间层包含人类活动的符号模型,它通过语法描述将前一层识别的信号成分组装成正在进行的活动的表示。最后,在顶层是认知控制,它决定场景的哪些部分将被处理,以及哪些算法将被应用于何处。它通过在需要时获取额外的知识来调节视觉过程,并通过控制主动视觉系统将其传感器引导到特定位置来引导注意力。因此,底层是感知,中间层是认知,顶层是控制。所有层都可以访问离线过程中构建的知识库,其中包含有关操作的语义。该方法的可行性将通过开发一个名为莫纳丽莎的智能制造系统来证明,该系统可帮助人类完成装配任务。该系统将在人类执行装配任务时进行监控。它将识别装配动作并确定其是否正确,并将向人类传达可能的错误并建议继续进行的方法。该系统将具有先进的视觉传感和感知;基于机器人和人类研究的动作理解;语义和类似程序的记忆和推理,以及一个连接高级推理和低级感知的控制模块,用于真实的时间,与人类装配人员的反应和主动参与。拟议的工作将为传感器网络和机器人领域带来新的工具和方法,除了智能制造外,还适用于各种行业和应用。能够使用视觉传感器分析人类行为将对许多领域产生影响,从医疗保健和高级驾驶员辅助到人类机器人协作。该项目还将促进K-12推广,新课件(本科生和研究生),出版物和开源软件。

项目成果

期刊论文数量(0)
专著数量(0)
科研奖励数量(0)
会议论文数量(0)
专利数量(0)

数据更新时间:{{ journalArticles.updateTime }}

{{ item.title }}
{{ item.translation_title }}
  • DOI:
    {{ item.doi }}
  • 发表时间:
    {{ item.publish_year }}
  • 期刊:
  • 影响因子:
    {{ item.factor }}
  • 作者:
    {{ item.authors }}
  • 通讯作者:
    {{ item.author }}

数据更新时间:{{ journalArticles.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ monograph.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ sciAawards.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ conferencePapers.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ patent.updateTime }}

Cornelia Fermuller其他文献

Interactive-FAR: Interactive, Fast and Adaptable Routing for Navigation Among Movable Obstacles in Complex Unknown Environments
Interactive-FAR:交互式、快速且适应性强的路由,用于复杂未知环境中可移动障碍物之间的导航
  • DOI:
    10.48550/arxiv.2404.07447
  • 发表时间:
    2024
  • 期刊:
  • 影响因子:
    0
  • 作者:
    Botao He;Guofei Chen;Wenshan Wang;Ji Zhang;Cornelia Fermuller;Y. Aloimonos
  • 通讯作者:
    Y. Aloimonos
A 3D shape constraint on video
视频的 3D 形状约束
MorphEyes: Variable Baseline Stereo For Quadrotor Navigation
MorphEyes:用于四旋翼导航的可变基线立体
AcTExplore: Active Tactile Exploration on Unknown Objects
AcTExplore:对未知物体的主动触觉探索
  • DOI:
  • 发表时间:
    2023
  • 期刊:
  • 影响因子:
    0
  • 作者:
    A. Shahidzadeh;Seong Jong Yoo;P. Mantripragada;Chahat Deep Singh;Cornelia Fermuller;Y. Aloimonos
  • 通讯作者:
    Y. Aloimonos
Border ownership assignment in real images.
真实图像中的边界所有权分配。
  • DOI:
    10.1167/15.12.763
  • 发表时间:
    2015
  • 期刊:
  • 影响因子:
    1.8
  • 作者:
    Cornelia Fermuller;C. L. Teo;Y. Aloimonos
  • 通讯作者:
    Y. Aloimonos

Cornelia Fermuller的其他文献

{{ item.title }}
{{ item.translation_title }}
  • DOI:
    {{ item.doi }}
  • 发表时间:
    {{ item.publish_year }}
  • 期刊:
  • 影响因子:
    {{ item.factor }}
  • 作者:
    {{ item.authors }}
  • 通讯作者:
    {{ item.author }}

{{ truncateString('Cornelia Fermuller', 18)}}的其他基金

Emphasizing Explanation in AI Augmented String Instrumental Education
强调人工智能增强弦乐教育中的解释
  • 批准号:
    2318255
  • 财政年份:
    2023
  • 资助金额:
    $ 80万
  • 项目类别:
    Standard Grant
AccelNet: Accelerating Research on Neuromorphic Perception, Action, and Cognition
AccelNet:加速神经形态感知、行动和认知的研究
  • 批准号:
    2020624
  • 财政年份:
    2021
  • 资助金额:
    $ 80万
  • 项目类别:
    Standard Grant
Research Coordination Network: Cognitive Functions in the Learning of Symbolic Signals & Systems
研究协调网络:符号信号学习中的认知功能
  • 批准号:
    1824198
  • 财政年份:
    2018
  • 资助金额:
    $ 80万
  • 项目类别:
    Standard Grant
SL-CN: Cortical Architectures for Robust Adaptive Perception and Action
SL-CN:用于鲁棒自适应感知和行动的皮质架构
  • 批准号:
    1540916
  • 财政年份:
    2015
  • 资助金额:
    $ 80万
  • 项目类别:
    Standard Grant
INSPIRE: Signals to Symbols: From Bio-inspired Hardware to Cognitive Systems
INSPIRE:信号到符号:从仿生硬件到认知系统
  • 批准号:
    1248056
  • 财政年份:
    2012
  • 资助金额:
    $ 80万
  • 项目类别:
    Standard Grant
POWRE: Enhanced Vision---Eyes from Eyes
POWRE:增强视觉——眼睛来自眼睛
  • 批准号:
    9973428
  • 财政年份:
    1999
  • 资助金额:
    $ 80万
  • 项目类别:
    Standard Grant

相似海外基金

Leveraging the synergy between experiment and computation to understand the origins of chalcogen bonding
利用实验和计算之间的协同作用来了解硫族键合的起源
  • 批准号:
    EP/Y00244X/1
  • 财政年份:
    2024
  • 资助金额:
    $ 80万
  • 项目类别:
    Research Grant
Multiple Representations of Learning in Dynamics and Control: Exploring the Synergy of Low-Cost Portable Lab Equipment, Virtual Labs, and AI within Student Learning Activities
动力学和控制中学习的多重表示:探索低成本便携式实验室设备、虚拟实验室和人工智能在学生学习活动中的协同作用
  • 批准号:
    2336998
  • 财政年份:
    2024
  • 资助金额:
    $ 80万
  • 项目类别:
    Standard Grant
Development of a novel oral vaccine for fish: Synergy of chitosan nano particle and complement-mediated opsonization
新型鱼类口服疫苗的开发:壳聚糖纳米颗粒与补体介导的调理作用的协同作用
  • 批准号:
    24K17960
  • 财政年份:
    2024
  • 资助金额:
    $ 80万
  • 项目类别:
    Grant-in-Aid for Early-Career Scientists
Synergy between future 21-cm experiments and physical cosmology
未来 21 厘米实验与物理宇宙学之间的协同作用
  • 批准号:
    DE240101129
  • 财政年份:
    2024
  • 资助金额:
    $ 80万
  • 项目类别:
    Discovery Early Career Researcher Award
Investigation of upper limb synergy in hemiplegic patients and development of neurorehabilitation.
偏瘫患者上肢协同作用的调查和神经康复的发展。
  • 批准号:
    23K10410
  • 财政年份:
    2023
  • 资助金额:
    $ 80万
  • 项目类别:
    Grant-in-Aid for Scientific Research (C)
BRITE Synergy: Chemically Resilient, Fouling Resistant Separation Membranes Manufactured Using Aqueous Phase Inversion
BRITE Synergy:采用水相转化技术制造的化学弹性、防污分离膜
  • 批准号:
    2227307
  • 财政年份:
    2023
  • 资助金额:
    $ 80万
  • 项目类别:
    Standard Grant
Contralesional Corticobulbospinal Structural and Functional Changes Post Stroke: Biomarkers for the upper limb flexion synergy
中风后对侧皮质球脊髓结构和功能变化:上肢屈曲协同作用的生物标志物
  • 批准号:
    10741103
  • 财政年份:
    2023
  • 资助金额:
    $ 80万
  • 项目类别:
BRITE Synergy: Seismic Cracking of Embankments and Earth Dams
BRITE Synergy:路堤和土坝的地震开裂
  • 批准号:
    2226154
  • 财政年份:
    2023
  • 资助金额:
    $ 80万
  • 项目类别:
    Standard Grant
Mechanisms of Synergy between Oncolytic Herpes Simplex Virus and Trabectedin in Pediatric Bone Sarcomas
溶瘤单纯疱疹病毒与曲贝替定治疗小儿骨肉瘤的协同作用机制
  • 批准号:
    10607503
  • 财政年份:
    2023
  • 资助金额:
    $ 80万
  • 项目类别:
Sex-dependent synergy between O3 exposure, APOE4 e4 genotype, and aging in the onset of Alzheimer's disease
O3 暴露、APOE4 e4 基因型和衰老在阿尔茨海默病发病过程中的性别依赖性协同作用
  • 批准号:
    10584765
  • 财政年份:
    2023
  • 资助金额:
    $ 80万
  • 项目类别:
{{ showInfoDetail.title }}

作者:{{ showInfoDetail.author }}

知道了