Multimodal Influences on Perception and Action in Computer-Mediated Environments

点击翻译按钮获取中文摘要

My research program combines the development of novel multimodal hardware and software platforms that solve specific problems related to user needs and support experimentation aimed at improving our understanding of the factors underlying human perception of multimodal stimuli. The objectives of this research include compensation for sensory or physical limitations, improved communication and quality of information delivery, and achieving compelling computer-mediated experiences in gameplay and simulation environments.******The interdisciplinary nature of my research brings together a core of HQP from electrical, mechanical, and computer engineering (and computer science), who often collaborate with colleagues from psychology, medicine, physical and occupational therapy, neuroscience, vision research, music, and new media art. The opportunities for novel research topics and advances to the state of the art at the intersections of engineering with these disciplines, in particular for projects that offer a benefit to society, help inspire my lab's HQP, who work extensively on their research projects with other lab members, our collaborators, undergraduate research interns, and visitors.******The activities of of the proposed DG are expected to train a total of 12 graduate HQP and 10 undergraduate HQP (2 per year), funded by a combination of the DG grant and scholarships. The methodology by which the HQP will be carrying out their research is heavily user-centric, in which the focus throughout is on actual user needs. These, in turn, drive initial exploration of a space of possible solutions, and typically inspire further research questions beyond the original ones. My approach involves iterative cycles of design, development with prototypes of increasing capability, significant experimentation with human subjects, and artifacts that include research publications, open source software, deployed tools, and licensed technologies.******Immediate applications of my research range from medical treatment technologies, information displays for populations with sensory deficits, and training environments for complex or challenging activities. Longer term benefits of this research include informing the design of next-generation interaction technologies and enabling greater human capabilities and performance. As one example of anticipated impact from the proposed DG activities, we will design and produce a new architecture of in-shoe sensors and haptic actuators, significantly more capable than existing commercial systems (e.g., from Lechal) at rendering information about the wearer's environment, that is responsive to user activity, suitable for applications in rehabilitation therapy, and capable of providing richly textured patterns that result in exciting mobile gameplay activities.