CRII: HCC: 3D Hand & Full-Body Pose Estimation in Telehealth for Children with Autism

CRII：HCC：3D 手

• 批准号: 2153249
• 负责人: Kevin Desai
• 金额: $ 17.44万
• 依托单位: University of Texas at San Antonio
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-06-01 至 2025-05-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2153249&HistoricalAwards=false
• 关键词: CRII; HCC; 3D; Hand; Full

This award is funded in whole or in part under the American Rescue Plan Act of 2021 (Public Law 117-2). The overall objective of this project is to provide efficient full-body interaction in virtual reality systems that do not use head-mounted displays. This project aims to create accurate and real-time 3D hand and body pose estimation, in the highly significant application area of children with autism. A novel synthetic hand data generation framework will generate 3D hand poses with increased diversity in terms of hand distance from camera, hand size, camera viewpoint, occlusion, background, and skin color. The outcome will be a novel 3D synthetic hand dataset consisting of realistic and kinematically accurate hand models with articulated poses that will advance current and future research endeavors in 3D hand pose estimation research. The project will advance the state-of-the-art in 3D body pose estimation for humans present further away from the camera at room-scale distances. The synthetic dataset, algorithms, and programming libraries will be made publicly available for wide-spread adoption, thereby advancing pose estimation research.This research will have broad societal impact because it will improve the usability and interaction in human centered telehealth applications, initially helping with the applied behavior analysis for children with autism. Existing systems that employ head-mounted displays or wearable sensors for tracking the user's hand and body movements are not suitable for children with autism, and have disadvantages in many other application areas. Therefore, by enabling 3D hand and full-body pose estimation, this project will advance a plethora of 3D immersive applications such as education, virtual STEM laboratories, tele-rehabilitation, tele-operation, military training, entertainment, and communication. The need for real-time, remote and interactive human motion sensing exists now more than ever, considering the increase in virtual activities because of the recent pandemic.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该奖项全部或部分根据2021年美国救援计划法案（公法117-2）资助。该项目的总体目标是在不使用头戴式显示器的虚拟现实系统中提供有效的全身交互。该项目旨在创建准确和实时的3D手部和身体姿势估计，在自闭症儿童的高度重要的应用领域。一种新的合成手部数据生成框架将生成3D手部姿势，其在与相机的手部距离、手部大小、相机视点、遮挡、背景和肤色方面具有增加的多样性。其结果将是一个新的3D合成手数据集，包括现实和运动学上准确的手模型与关节姿势，将推进当前和未来的研究工作在3D手姿态估计研究。该项目将推进最先进的3D人体姿势估计，以使人类在房间尺度的距离上远离相机。合成数据集、算法和编程库将被公开以供广泛采用，从而推进姿势估计研究。这项研究将产生广泛的社会影响，因为它将提高以人为中心的远程医疗应用的可用性和交互性，最初有助于自闭症儿童的应用行为分析。采用头戴式显示器或可穿戴传感器来跟踪用户的手和身体运动的现有系统不适合患有自闭症的儿童，并且在许多其他应用领域中具有缺点。因此，通过实现3D手部和全身姿势估计，该项目将推动大量3D沉浸式应用，如教育、虚拟STEM实验室、远程康复、远程操作、军事训练、娱乐和通信。考虑到由于最近的大流行而增加的虚拟活动，现在比以往任何时候都更需要实时，远程和交互式的人体运动传感。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。