Modeling the Interaction Between Speech and Gesture (Linguistics)

将通过在三维互动动画模型上对这两种媒体之间的互动进行动画制作，来研究叙事话语中言语和手势之间的关系。三维互动动画模型是计算机屏幕上的一个图形人物，最终将能够与听者“交流”，在产生语音的同时做出适当的手势。研究表明，在叙事语篇的产生和理解过程中，手势与言语有着密切的联系。这种互动存在于组织的许多不同级别。语音和手势之间的关系可以在手势相对于从句的韵律轮廓的定时的水平上指定。它可以在话语的语义和语用描述层面上指定，也可以在叙事结构层面上根据叙事中不同点的手势类型来具体说明。主办机构的互动活动包括：每两周举行一次会议，向研究生和初级教师开放，讨论完成工作中的特殊问题，并为实现专业目标营造一个有利的环境；每月隔几周与本科生举行一次长达一小时的会议，讨论类似的主题；每学期赞助两名来访的认知科学家，这些科学家将是做出了非标准职业选择和/或撰写过关于女性在学术界面临的专业问题的女性；与艾伦·普林斯共同教授关于话语处理和手势的计算语言学研讨会。所有部分都将在计算机科学系的支持下进行，但将向宾夕法尼亚大学认知科学社区的其他成员开放。该项目进一步推动了VPW计划的目标，即：(1)为妇女提供机会，在NSF资助的工程和科学学科中发展她们的职业生涯；(2)通过提高受雇于工业、政府和学术机构的女科学家和工程师的知名度，鼓励妇女追求科学和工程事业。通过鼓励妇女参与科学，这是对国家未来科学活力的宝贵投资。

The relationship between speech and gesture in narrative discourse will be investigated by animating the interaction between the two media on a three-dimensional interactive animated model -- a graphic figure on a computer screen that would ultimately be able to "communicate" with a listener, gesturing appropriately while producing speech. Research demonstrates that gesture is intimately linked to speech both in the production and the comprehension of narrative discourse. This interaction exists at many different levels of organization. The relationship between speech and gesture can be specified at the level of the timing of gestures with respect to the prosodic contour of a clause. It can be specified at the level of semantic and pragmatic descriptions of an utterance, and at the level of narrative structure with respect to the types of gestures found at different points in a narrative. Interactive activities at the host institution include: meetings once every two weeks, open to graduate students and junior faculty, to discuss particular problems in accomplishing work and constructing a supportive environment for achieving professional goals; hour-long meetings the alternate weeks of the month with undergraduate students, to discuss similar topics; sponsorship of two visiting cognitive scientists per semester, where these will be women who have made 'non-standard' career choices and/or have written about professional issues facing women in academia; co- teaching a seminar in Computational Linguistics with Ellen Prince on discourse processing and gesture. All of the parts will be carried out under the aegis of the Computer Science Department, but will be open to other members of the Cognitive Science community at the University of Pennsylvania. This project furthers VPW program objectives which are (1) to provide opportunities for women to advance their careers in engineering and in the disciplines of science supported by NSF and (2) to encourage women to pursue careers in science and engineering by providing greater visibility for women scientists and engineers employed in industry, government, and academic institutions. By encouraging the participation of women in science, it is a valuable investment in the Nation's future scientific vitality.