Modeling the Interaction Between Speech and Gesture (Linguistics)

点击翻译按钮获取中文摘要

The relationship between speech and gesture in narrative discourse will be investigated by animating the interaction between the two media on a three-dimensional interactive animated model -- a graphic figure on a computer screen that would ultimately be able to "communicate" with a listener, gesturing appropriately while producing speech. Research demonstrates that gesture is intimately linked to speech both in the production and the comprehension of narrative discourse. This interaction exists at many different levels of organization. The relationship between speech and gesture can be specified at the level of the timing of gestures with respect to the prosodic contour of a clause. It can be specified at the level of semantic and pragmatic descriptions of an utterance, and at the level of narrative structure with respect to the types of gestures found at different points in a narrative. Interactive activities at the host institution include: meetings once every two weeks, open to graduate students and junior faculty, to discuss particular problems in accomplishing work and constructing a supportive environment for achieving professional goals; hour-long meetings the alternate weeks of the month with undergraduate students, to discuss similar topics; sponsorship of two visiting cognitive scientists per semester, where these will be women who have made 'non-standard' career choices and/or have written about professional issues facing women in academia; co- teaching a seminar in Computational Linguistics with Ellen Prince on discourse processing and gesture. All of the parts will be carried out under the aegis of the Computer Science Department, but will be open to other members of the Cognitive Science community at the University of Pennsylvania. This project furthers VPW program objectives which are (1) to provide opportunities for women to advance their careers in engineering and in the disciplines of science supported by NSF and (2) to encourage women to pursue careers in science and engineering by providing greater visibility for women scientists and engineers employed in industry, government, and academic institutions. By encouraging the participation of women in science, it is a valuable investment in the Nation's future scientific vitality.