Multimodal Influences on Perception and Action in Computer-Mediated Environments

我的研究项目结合了新型多模态硬件和软件平台的开发，以解决与用户需求相关的特定问题，并支持旨在提高我们对人类对多模态刺激感知的潜在因素的理解的实验。这项研究的目标包括补偿感官或身体限制，改善沟通和信息传递的质量，并在游戏玩法和模拟环境中实现引人注目的计算机媒介体验。******我研究的跨学科性质汇集了来自电气，机械和计算机工程（以及计算机科学）的HQP核心，他们经常与心理学，医学，物理和职业治疗，神经科学，视觉研究，音乐和新媒体艺术的同事合作。在与这些学科交叉的工程领域，特别是那些对社会有益的项目，有机会获得新颖的研究课题和先进的技术，这有助于激励我实验室的HQP，他们与其他实验室成员、合作者、本科生研究实习生和访客广泛地开展研究项目。******拟任总干事的活动预计将共培养12名研究生和10名本科生（每年2名），由总干事补助金和奖学金共同资助。HQP开展研究的方法是高度以用户为中心的，其重点始终放在实际用户需求上。这些反过来又推动了对可能解决方案空间的初步探索，并通常激发了超出原始问题的进一步研究问题。我的方法涉及到设计的迭代周期，使用增加能力的原型进行开发，对人类受试者进行重要的实验，以及包括研究出版物、开源软件、部署工具和许可技术在内的工件。******我的研究的直接应用范围包括医疗技术，为有感觉缺陷的人群提供信息显示，以及复杂或具有挑战性活动的训练环境。这项研究的长期好处包括为下一代交互技术的设计提供信息，并使人类的能力和性能得到提高。作为提议的DG活动预期影响的一个例子，我们将设计和生产一种新的鞋内传感器和触觉执行器架构，比现有的商业系统（例如Lechal）在呈现有关穿戴者环境的信息方面能力更强，能够响应用户活动，适合康复治疗的应用，并能够提供丰富的纹理图案，从而产生令人兴奋的移动游戏活动。

My research program combines the development of novel multimodal hardware and software platforms that solve specific problems related to user needs and support experimentation aimed at improving our understanding of the factors underlying human perception of multimodal stimuli. The objectives of this research include compensation for sensory or physical limitations, improved communication and quality of information delivery, and achieving compelling computer-mediated experiences in gameplay and simulation environments.******The interdisciplinary nature of my research brings together a core of HQP from electrical, mechanical, and computer engineering (and computer science), who often collaborate with colleagues from psychology, medicine, physical and occupational therapy, neuroscience, vision research, music, and new media art. The opportunities for novel research topics and advances to the state of the art at the intersections of engineering with these disciplines, in particular for projects that offer a benefit to society, help inspire my lab's HQP, who work extensively on their research projects with other lab members, our collaborators, undergraduate research interns, and visitors.******The activities of of the proposed DG are expected to train a total of 12 graduate HQP and 10 undergraduate HQP (2 per year), funded by a combination of the DG grant and scholarships. The methodology by which the HQP will be carrying out their research is heavily user-centric, in which the focus throughout is on actual user needs. These, in turn, drive initial exploration of a space of possible solutions, and typically inspire further research questions beyond the original ones. My approach involves iterative cycles of design, development with prototypes of increasing capability, significant experimentation with human subjects, and artifacts that include research publications, open source software, deployed tools, and licensed technologies.******Immediate applications of my research range from medical treatment technologies, information displays for populations with sensory deficits, and training environments for complex or challenging activities. Longer term benefits of this research include informing the design of next-generation interaction technologies and enabling greater human capabilities and performance. As one example of anticipated impact from the proposed DG activities, we will design and produce a new architecture of in-shoe sensors and haptic actuators, significantly more capable than existing commercial systems (e.g., from Lechal) at rendering information about the wearer's environment, that is responsive to user activity, suitable for applications in rehabilitation therapy, and capable of providing richly textured patterns that result in exciting mobile gameplay activities.