Collaborative Research: Multimethod Investigation of Articulatory and Perceptual Constraints on Natural Language Evolution

点击翻译按钮获取中文摘要

Languages change over time, such that the way we speak English now is very different than the speech patterns of elder generations and our distant ancestors. This project will exploit the visual nature of sign languages--where the body parts producing language are highly visible--to determine whether languages change so that they are easier to produce or so that they are easier to understand. In doing so, the project will address fundamental theoretical questions about language change that cannot be addressed by analyzing historical samples of spoken languages. To this end, the researchers will develop computational tools that allow 3D human body poses to be automatically extracted from 2D video. Such tools will be useful for the development of automated sign language recognition, promoting accessibility for deaf and hard-of-hearing people, and for developing automated systems for recognizing and classifying human gestures. The research will involve deaf and hard-of-hearing students, helping to increase diversity in the nation's scientific workforce.It is well documented that sign languages change over time, and it is a commonly held belief that those changes have resulted from successive generations making language easier to perceive. However, most of this evidence has been anecdotal and descriptive and has not quantified changes in the ease of perception and production of ASL over time. The research team will take advantage of the fully visible articulators of sign languages to develop novel pose estimation algorithms that are able to automatically extract information contained in 2D video to create accurate 3D models of articulator movement during language production. The recent birth and rapid evolution of Nicaraguan Sign Language (NSL) has allowed researchers to study language change, from the beginning, on a compressed time-scale. By leveraging an existing NSL database--comprised of 2D videos from four generations of Nicaraguan signers--and utilizing these novel pose estimation algorithms, the researchers will be able to empirically assess the extent to which linguistic changes are driven by perceptual constraints imposed by the human visual system and/or articulatory constraints imposed by the musculoskeletal system. The researchers will also query lexical databases of American Sign Language to test predictions about the perceptual form of modern day ASL, and conduct behavioral studies with deaf and hearing users of ASL to test hypotheses regarding the allocation of visual attention as a result of both deafness and acquisition of a sign language. In doing so, the research will provide valuable information about how the human brain changes the tools we use (in this case, language) and the way that those tools in turn shape the function of the human brain. This will provide a more complex understanding of language change that illuminates the complex interaction between languages and the human beings that use them.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.