CAREER: Augmenting Passive Physical Interfaces into Adaptive Interfaces

世界上到处都是被动的物理接口，比如电灯开关和墙壁插座。智能功能的集成，如驱动、传感和能量收集，有望将被动接口转变为智能自适应接口，可以帮助残疾人，自动化家庭任务，并为数百万无处不在的计算机提供动力。虽然用智能设备完全取代所有遗留接口目前是不可行的，并且可能会导致重大的环境影响，但增加它们是一种有希望的方法，可以经济有效地重新配置日常接口。然而，许多终端用户，例如护工、建筑经理，甚至是对家庭自动化感兴趣的个人，都忽视了通过计算创新独特生活模式和风格的机会。日常设计问题可能很难注意到，因为基于先前经验的熟悉。即使对于具有特定目标的用户，例如，减少公用事业费用，在现实生活中采用最新的科学进展也需要专业知识，因为缺乏最终用户支持工具。该项目旨在(1)提高最终用户对使用三维（3D）打印增强件的日常设计机会的认识，实现(2)以高效和准确的方式捕获复杂增强件的关键制造参数，以及(3)在多个应用领域以最小或无障碍的方式制造智能增强件。提供了用于制造和（重新）编程的库和工具包，最终用户可以处理已知的模块化编程策略，例如混合匹配、单元测试、模型-视图-控制器（MVC）模型，并派生现有的增强功能以重新使用它们。该项目涉及数字制造、终端用户编程、深度学习、机器人和设计等多方面的跨学科方法，为未来奠定了基础，在未来，每个人都可以在辅助计算设备、智能家居和绿色建筑中创造日常创新。该项目将围绕三个主要研究目标展开。首先，将制定一个理论框架，使用数据驱动的方法对物理接口进行概念化。研究者将通过描述它们的交互属性和用户上下文来帮助理解重构需求、交互障碍和自适应/期望的交互，通过部件级分割来增强和扩展日常对象的大量图像数据集。其次，将开发新的计算技术来捕获物理：感兴趣的部分（POI）、感兴趣的维度（DOI）和感兴趣的运动（MOI），它们是增强被动接口提取智能功能的关键参数。最后，集成数据集和技术，模块化制造管道将与可组合的构建工具包和智能增强库一起开发，建立捕获-定制/制造-（重新）程序范式。通过教育目标，该项目推进了妇女、学生和少数民族教育工作者的教学、培训和学习计算，并通过德克萨斯人机交互（TxHCI）系列研讨会促进了多学科对话。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The world is full of physical interfaces, such as light switches and walls sockets, that are passive. The integration of smart capabilities such as actuating, sensing, and energy-harvesting is expected to transform passive interfaces into smart adaptive interfaces, that can assist people with disabilities, automate domestic tasks, and power millions of ubiquitous computers. While the complete replacement of all legacy interfaces with smart devices is not currently feasible and may lead to significant environmental impacts, augmenting them is a promising approach to cost-effectively reconfiguring daily interfaces. Yet, many end users, e.g., caretakers, building managers, even individuals who are interested in home-automation, overlook opportunities to innovate unique life patterns and styles via computation. Daily design issues can be difficult to notice because of familiarity based on prior experiences. Even for users with a specific goal, e.g., reducing utility bills, adopting the latest scientific advances in real-life demands expertise because end-user support tools are lacking. This project aims to (1) increase end-users’ awareness about daily design opportunities using three-dimensional (3D) printed augmentations, enabling (2) capturing critical fabrication parameters for complex augmentations with efficiency and accuracy, and (3) fabricating smart augmentations with minimal to no barriers across multiple application domains. Provided with the libraries and toolkits for fabrication and (re)programming, end-users are allowed to tackle known modular-programming strategies, such as mix-and-match, unit-testing, model-view-controller (MVC) model, and fork the existing augmentations to repurpose them.Tackling multifaceted, interdisciplinary approaches across Digital Fabrication, End-user Programming, Deep Learning, Robotics, and Design, this project lays the foundation for a future where every individual creates daily innovations in assistive computing devices, smart homes, and green buildings. This project will center around three main research objectives. Firstly, a theoretical framework will be formulated to conceptualize physical interfaces using a data-driven approach. The investigator will augment and expand massive image dataset of daily objects presented with part level segmentation, by characterizing their interaction properties and user context to help understand the reconfiguration needs, interaction barriers, and adaptive/desired interactions. Secondly, novel computational techniques will be developed to capture physicality: parts of interest (POI), dimensions of interest (DOI), and motions of interest (MOI), which are critical parameters to augment passive interfaces distilling smart capabilities. Lastly, integrating the dataset and techniques, a modular fabrication pipeline will be developed along with a composable construction toolkit and libraries of smart augmentations, establishing capture—customize/fabricate--(re)program paradigm. Through educational objectives, this project advances teaching, training, and learning computing for women, students and educators of minorities, and promotes multi-disciplinary conversations via the Texas Human-Computer Interaction (TxHCI) seminar series.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.