Multimodal Influences on Perception and Action in Computer-Mediated Environments

我的研究项目结合了新型多模态硬件和软件平台的开发，这些平台可以解决与用户需求相关的特定问题，并支持旨在提高我们对人类感知多模态刺激的因素的理解的实验。这项研究的目标包括补偿感官或身体限制，改善沟通和信息传递的质量，并在游戏和模拟环境中实现引人注目的计算机介导体验。我的研究的跨学科性质汇集了HQP的核心，从电气，机械和计算机工程（和计算机科学），他们经常与来自心理学，医学，物理和职业治疗，神经科学，视觉研究，音乐和新媒体艺术的同事合作。在工程与这些学科的交叉点，特别是那些对社会有益的项目，有助于激励我实验室的HQP，他们与其他实验室成员、我们的合作者、本科生研究实习生和来访者一起广泛地研究他们的研究项目。拟议总干事的活动预计将培训12名研究生HQP和10名本科生HQP（每年2名），由总干事赠款和奖学金共同资助。HQP开展研究的方法论是以用户为中心的，始终关注实际用户需求。这些反过来又推动了对可能解决方案空间的初步探索，并通常激发了超越原始问题的进一步研究问题。 我的方法包括设计的迭代循环，使用增加功能的原型进行开发，对人类受试者进行重要的实验，以及包括研究出版物，开源软件，部署工具和许可技术的工件。我的研究的直接应用范围从医疗技术，为感官缺陷的人群提供信息显示，以及复杂或具有挑战性的活动的训练环境。这项研究的长期好处包括为下一代交互技术的设计提供信息，并提高人类的能力和表现。作为拟议DG活动的预期影响的一个例子，我们将设计和生产一种鞋内传感器和触觉致动器的新架构，其能力明显高于现有的商业系统（例如，来自Lechal的）呈现关于佩戴者环境的信息，其响应于用户活动，适合于康复治疗中的应用，并且能够提供导致令人兴奋的移动的游戏活动的丰富纹理图案。

My research program combines the development of novel multimodal hardware and software platforms that solve specific problems related to user needs and support experimentation aimed at improving our understanding of the factors underlying human perception of multimodal stimuli. The objectives of this research include compensation for sensory or physical limitations, improved communication and quality of information delivery, and achieving compelling computer-mediated experiences in gameplay and simulation environments.******The interdisciplinary nature of my research brings together a core of HQP from electrical, mechanical, and computer engineering (and computer science), who often collaborate with colleagues from psychology, medicine, physical and occupational therapy, neuroscience, vision research, music, and new media art. The opportunities for novel research topics and advances to the state of the art at the intersections of engineering with these disciplines, in particular for projects that offer a benefit to society, help inspire my lab's HQP, who work extensively on their research projects with other lab members, our collaborators, undergraduate research interns, and visitors.******The activities of of the proposed DG are expected to train a total of 12 graduate HQP and 10 undergraduate HQP (2 per year), funded by a combination of the DG grant and scholarships. The methodology by which the HQP will be carrying out their research is heavily user-centric, in which the focus throughout is on actual user needs. These, in turn, drive initial exploration of a space of possible solutions, and typically inspire further research questions beyond the original ones. My approach involves iterative cycles of design, development with prototypes of increasing capability, significant experimentation with human subjects, and artifacts that include research publications, open source software, deployed tools, and licensed technologies.******Immediate applications of my research range from medical treatment technologies, information displays for populations with sensory deficits, and training environments for complex or challenging activities. Longer term benefits of this research include informing the design of next-generation interaction technologies and enabling greater human capabilities and performance. As one example of anticipated impact from the proposed DG activities, we will design and produce a new architecture of in-shoe sensors and haptic actuators, significantly more capable than existing commercial systems (e.g., from Lechal) at rendering information about the wearer's environment, that is responsive to user activity, suitable for applications in rehabilitation therapy, and capable of providing richly textured patterns that result in exciting mobile gameplay activities.