CDI Type-II: Language Models for Human Dexterity

点击翻译按钮获取中文摘要

This award is funded under the American Recovery and Reinvestment Act of 2009 (Public Law 111-5).The objective of this Cyber-enabled Discovery and Innovation proposal is to develop new techniques for modeling and analyzing human manipulative activities from motion and video data. In particular, this research is based on the hypothesis that manipulative activities can be analyzed using time-series analysis methods similar to those that have revolutionized speech and language processing. Three focus areas will be examined. The first focus area will be a "Language of Surgery" based on a large and growing data set of surgical recordings acquired using the Intuitive Surgical da Vinci robot. These recordings provide high-fidelity motion and stereo video data in both laboratory and real-world surgical settings. The second focus area will be a "Language of Fine Manipulation" based on motion data acquired during human manipulation activities. The final focus area will be a "Language of Body Motion" based on motion and video data of whole-body manipulations such as lifting and carrying. These investigations will develop new methods for analyzing time-series motion and video data, new perspectives on modeling the structure of human motion, and new insights into the relationships between skill, style, and content in human motion.The notion of modeling and analyzing human motion will have enormous impact for both basic science and many societally relevant applications. These results will have immediate implications for teaching and training in medicine, and on areas such as rehabilitation. They will enable more basic understanding for modeling and automating human manipulative activities. The project will also invigorate an area of research that will unite multiple scientific disciplines. The PIs of this project are also active in many diversity outreach activities, including REU programs, the Women in Science and Engineering (WISE) program, and summer camps for K-12 outreach. Participants in these programs will be incorporated into the proposed research.