CDI Type-II: Language Models for Human Dexterity

该奖项由2009年《美国复苏和再投资法案》(公法111-5)资助。这项基于网络的发现和创新提案的目标是开发新的技术，用于根据运动和视频数据对人类操纵活动进行建模和分析。特别是，这项研究是基于这样一个假设，即操纵活动可以用时间序列分析方法来分析，类似于那些革命性地改变了语音和语言处理的方法。将研究三个重点领域。第一个重点领域将是基于使用直觉的外科达芬奇机器人获得的大量且不断增长的手术记录数据集的“外科语言”。这些记录在实验室和真实世界的外科环境中都提供高保真运动和立体视频数据。第二个重点领域将是基于在人类操纵活动中获得的运动数据的“精细操纵语言”。最终的焦点领域将是基于举起和搬运等全身操作的运动和视频数据的“身体运动语言”。这些研究将开发分析时间序列运动和视频数据的新方法，对人体运动结构建模的新视角，以及对人体运动中技能、风格和内容之间关系的新见解。对人体运动进行建模和分析的概念将对基础科学和许多具有社会意义的应用产生巨大影响。这些结果将对医学教学和培训以及康复等领域产生直接影响。它们将使人们能够更基本地理解人类操纵活动的建模和自动化。该项目还将振兴一个将联合多个科学学科的研究领域。该项目的私人投资机构还积极参与了许多多元化的外联活动，包括REU方案、妇女参与科学和工程(WISE)方案以及K-12外联夏令营。这些项目的参与者将被纳入拟议的研究。

This award is funded under the American Recovery and Reinvestment Act of 2009 (Public Law 111-5).The objective of this Cyber-enabled Discovery and Innovation proposal is to develop new techniques for modeling and analyzing human manipulative activities from motion and video data. In particular, this research is based on the hypothesis that manipulative activities can be analyzed using time-series analysis methods similar to those that have revolutionized speech and language processing. Three focus areas will be examined. The first focus area will be a "Language of Surgery" based on a large and growing data set of surgical recordings acquired using the Intuitive Surgical da Vinci robot. These recordings provide high-fidelity motion and stereo video data in both laboratory and real-world surgical settings. The second focus area will be a "Language of Fine Manipulation" based on motion data acquired during human manipulation activities. The final focus area will be a "Language of Body Motion" based on motion and video data of whole-body manipulations such as lifting and carrying. These investigations will develop new methods for analyzing time-series motion and video data, new perspectives on modeling the structure of human motion, and new insights into the relationships between skill, style, and content in human motion.The notion of modeling and analyzing human motion will have enormous impact for both basic science and many societally relevant applications. These results will have immediate implications for teaching and training in medicine, and on areas such as rehabilitation. They will enable more basic understanding for modeling and automating human manipulative activities. The project will also invigorate an area of research that will unite multiple scientific disciplines. The PIs of this project are also active in many diversity outreach activities, including REU programs, the Women in Science and Engineering (WISE) program, and summer camps for K-12 outreach. Participants in these programs will be incorporated into the proposed research.