Multimodal Influences on Perception and Action in Computer-Mediated Environments

我的研究项目结合了新型多模态硬件和软件平台的开发，以解决与用户需求相关的特定问题，并支持旨在提高我们对人类对多模态刺激感知的潜在因素的理解的实验。这项研究的目标包括补偿感官或身体限制，改善沟通和信息传递的质量，并在游戏玩法和模拟环境中实现引人注目的计算机媒介体验。

My research program combines the development of novel multimodal hardware and software platforms that solve specific problems related to user needs and support experimentation aimed at improving our understanding of the factors underlying human perception of multimodal stimuli. The objectives of this research include compensation for sensory or physical limitations, improved communication and quality of information delivery, and achieving compelling computer-mediated experiences in gameplay and simulation environments. The interdisciplinary nature of my research brings together a core of HQP from electrical, mechanical, and computer engineering (and computer science), who often collaborate with colleagues from psychology, medicine, physical and occupational therapy, neuroscience, vision research, music, and new media art. The opportunities for novel research topics and advances to the state of the art at the intersections of engineering with these disciplines, in particular for projects that offer a benefit to society, help inspire my lab's HQP, who work extensively on their research projects with other lab members, our collaborators, undergraduate research interns, and visitors. The activities of of the proposed DG are expected to train a total of 12 graduate HQP and 10 undergraduate HQP (2 per year), funded by a combination of the DG grant and scholarships. The methodology by which the HQP will be carrying out their research is heavily user-centric, in which the focus throughout is on actual user needs. These, in turn, drive initial exploration of a space of possible solutions, and typically inspire further research questions beyond the original ones. My approach involves iterative cycles of design, development with prototypes of increasing capability, significant experimentation with human subjects, and artifacts that include research publications, open source software, deployed tools, and licensed technologies. Immediate applications of my research range from medical treatment technologies, information displays for populations with sensory deficits, and training environments for complex or challenging activities. Longer term benefits of this research include informing the design of next-generation interaction technologies and enabling greater human capabilities and performance. As one example of anticipated impact from the proposed DG activities, we will design and produce a new architecture of in-shoe sensors and haptic actuators, significantly more capable than existing commercial systems (e.g., from Lechal) at rendering information about the wearer's environment, that is responsive to user activity, suitable for applications in rehabilitation therapy, and capable of providing richly textured patterns that result in exciting mobile gameplay activities.