Collaborative Research: Teaching Human Motion Tasks at Population Scale

合作研究：在人群规模上教授人体运动任务

• 批准号: 1822819
• 负责人: Devin Balkcom
• 金额: $ 63.61万
• 依托单位: Dartmouth College
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-09-15 至 2022-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1822819&HistoricalAwards=false
• 关键词: Collaborative; Research; Teaching; Human; Motion

The project will develop technology and study methods for teaching motion tasks, with the teaching of sign language as a first application. Simultaneous placement or quick movement of parts of the body is hard to observe, explain, and execute. While tactile-sensing and augmented-reality systems have been developed to enable machine-human communication of physical processes, the focus has largely been on execution, rather than on teaching and learning. The initial focus of the project will be on teaching sign language, but principles and techniques discovered will be generalized to research the learning of increasingly complex physical motions, ranging from simple posing tasks to high-speed fine manipulation tasks. The proposed work is transformative in that it will directly address the scientific question of how to use technology to understand correct or incorrect human motion and provide constructive guidance, leading to a better understanding of human motion learning. The project will disseminate findings and resources through traditional scientific publications. In addition, models, algorithms, and designs for rapidly-prototyped tools for manipulation will be made available on the online. Results will also be communicated broadly through collaborations with local high schools and museums, and through participation in events such as the USA Science and Engineering Festival.The task of teaching motion motivates the research of three fundamental challenges. First, closed-loop control is a core feature of cyber-physical systems. With a human participant in the system, how can the loop be closed around slow and low-bandwidth human attention? Actuation that guides the human must be easily communicated and sufficient to stabilize the human-suit system. Second, due to limitations in how much information may be communicated, complex human motions must be broken down, and components taught in isolation. How can these component motions be discovered, taught, and re-integrated? Third, algorithms and systems must be developed to measure the accuracy and retention of the learner during the teaching process, guiding repetition and selection of practice material. The project will design and build a lightweight sensing and guidance system that allows interactive communication about motion between human and computer. This technology will allow the investigators to address fundamental research questions in cyber-learning about how to better teach and learn human motion tasks. Research questions include how to measure and evaluate human motion with respect to the task, how to select sensory input to use as guidance, and how to selectively apply or remove training aids, until the learner can complete the motion task with no assistance.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目将开发技术和研究方法，用于教学动作任务，手语教学作为第一个应用。身体各部分的同时放置或快速移动很难观察、解释和执行。虽然已经开发了战斗感测和增强现实系统来实现物理过程的人机通信，但重点主要是执行，而不是教学和学习。该项目最初的重点将是教授手语，但发现的原则和技术将被推广到研究越来越复杂的身体动作的学习，从简单的姿势任务到高速精细操作任务。拟议的工作是变革性的，因为它将直接解决如何使用技术来理解正确或不正确的人体运动的科学问题，并提供建设性的指导，从而更好地理解人体运动学习。该项目将通过传统的科学出版物传播研究结果和资源。此外，模型，算法和设计的快速原型工具的操纵将在网上提供。通过与当地高中和博物馆的合作，以及参加美国科学与工程节等活动，将成果广泛传播。教学动作的任务激发了三个基本挑战的研究。首先，闭环控制是信息物理系统的核心特征。如果系统中有人类参与者，那么如何才能围绕缓慢且低带宽的人类注意力闭合循环？引导人类的致动必须容易传达，并且足以稳定人类服装系统。其次，由于信息量的限制，复杂的人体动作必须被分解，并单独教授组件。如何发现、教授和重新整合这些组成运动？第三，必须开发算法和系统来衡量学习者在教学过程中的准确性和保持力，指导重复和选择练习材料。该项目将设计和建造一个轻量级的传感和引导系统，允许人机之间的互动交流。这项技术将使研究人员能够解决网络学习中的基本研究问题，即如何更好地教授和学习人体运动任务。研究问题包括如何测量和评估人类运动方面的任务，如何选择感官输入使用作为指导，以及如何选择性地应用或删除培训aids，直到学习者可以完成运动task没有assistance.This奖项反映了NSF的法定使命，并已被认为是值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估的支持。

项目成果

Teaching American Sign Language in Mixed Reality

• DOI: 10.1145/3432211
• 发表时间: 2020-12
• 期刊: Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
• 作者: Qijia Shao;A. Sniffen;Julien Blanchet;Megan E. Hillis;Xinyu Shi;Themistoklis K. Haris;Jason Liu;Jason Lamberton;Melissa Malzkuhn;Lorna C. Quandt;James Mahoney;David J. M. Kraemer;Xia Zhou;Devin J. Balkcom

PolarTag: Invisible Data with Light Polarization

• DOI: 10.1145/3376897.3377854
• 发表时间: 2020-02
• 期刊: Proceedings of the 21st International Workshop on Mobile Computing Systems and Applications
• 作者: Tian Zhao;Charles J. Carver;Qijia Shao;Monika Roznere;Alberto Quattrini Li;Xia Zhou

Overlapping semantic representations of sign and speech in novice sign language learners

• 发表时间: 2022
• 作者: Megan E. Hillis;Brianna Aubrey;Julien Blanchet;Qijia Shao;Xia Zhou;Devin J. Balkcom;David J. M. Kraemer