CAREER: Augmenting Passive Physical Interfaces into Adaptive Interfaces

世界上充斥着被动的物理接口，如电灯开关和墙壁插座。驱动、传感和能量收集等智能功能的集成有望将被动接口转变为智能自适应接口，可以帮助残疾人、自动化家庭任务，并为数百万无处不在的计算机供电。虽然用智能设备完全替换所有传统接口目前还不可行，可能会导致重大的环境影响，但增强这些接口是以经济高效的方式重新配置日常接口的一种有前途的方法。然而，许多最终用户，例如管理员、大厦管理人员，甚至对家庭自动化感兴趣的个人，都忽视了通过计算来创新独特生活模式和方式的机会。日常设计问题可能很难注意到，因为基于以前的经验而熟悉。即使对于有特定目标的用户，例如减少公用事业费用，在现实生活中采用最新的科学进步也需要专业知识，因为缺乏最终用户支持工具。该项目旨在(1)提高终端用户对使用三维(3D)打印扩充物的日常设计机会的认识，使(2)能够高效和准确地捕捉复杂扩充物的关键制造参数，以及(3)跨多个应用领域以最小甚至无障碍地制造智能增强物。有了用于制造和(重新)编程的库和工具包，最终用户可以处理已知的模块化编程策略，如混合匹配、单元测试、模型-视图-控制器(MVC)模型，并扩展现有的扩展以重新利用它们。该项目将多方面、跨学科的方法贯穿数字制造、最终用户编程、深度学习、机器人学和设计，为每个人都在辅助计算设备、智能家居和绿色建筑中创造日常创新的未来奠定了基础。该项目将围绕三个主要研究目标展开。首先，将制定一个理论框架，以使用数据驱动的方法概念化物理接口。研究人员将通过表征日常对象的交互属性和用户上下文来扩充和扩展日常对象的海量图像数据集，以帮助理解重新配置需求、交互障碍和自适应/期望的交互。其次，将开发新的计算技术来捕获物理性：感兴趣部分(POI)、感兴趣维度(DOI)和感兴趣运动(MOI)，这些是增强被动接口提取智能能力的关键参数。最后，集成数据集和技术，将开发一个模块化制造管道，以及一个可组合的构建工具包和智能增强库，建立捕获-定制/制造-(重新)程序范式。通过教育目标，该项目促进了妇女、学生和少数族裔教育工作者的教学、培训和学习计算，并通过德克萨斯州人机交互(TxHCI)研讨会系列促进了多学科对话。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

The world is full of physical interfaces, such as light switches and walls sockets, that are passive. The integration of smart capabilities such as actuating, sensing, and energy-harvesting is expected to transform passive interfaces into smart adaptive interfaces, that can assist people with disabilities, automate domestic tasks, and power millions of ubiquitous computers. While the complete replacement of all legacy interfaces with smart devices is not currently feasible and may lead to significant environmental impacts, augmenting them is a promising approach to cost-effectively reconfiguring daily interfaces. Yet, many end users, e.g., caretakers, building managers, even individuals who are interested in home-automation, overlook opportunities to innovate unique life patterns and styles via computation. Daily design issues can be difficult to notice because of familiarity based on prior experiences. Even for users with a specific goal, e.g., reducing utility bills, adopting the latest scientific advances in real-life demands expertise because end-user support tools are lacking. This project aims to (1) increase end-users’ awareness about daily design opportunities using three-dimensional (3D) printed augmentations, enabling (2) capturing critical fabrication parameters for complex augmentations with efficiency and accuracy, and (3) fabricating smart augmentations with minimal to no barriers across multiple application domains. Provided with the libraries and toolkits for fabrication and (re)programming, end-users are allowed to tackle known modular-programming strategies, such as mix-and-match, unit-testing, model-view-controller (MVC) model, and fork the existing augmentations to repurpose them.Tackling multifaceted, interdisciplinary approaches across Digital Fabrication, End-user Programming, Deep Learning, Robotics, and Design, this project lays the foundation for a future where every individual creates daily innovations in assistive computing devices, smart homes, and green buildings. This project will center around three main research objectives. Firstly, a theoretical framework will be formulated to conceptualize physical interfaces using a data-driven approach. The investigator will augment and expand massive image dataset of daily objects presented with part level segmentation, by characterizing their interaction properties and user context to help understand the reconfiguration needs, interaction barriers, and adaptive/desired interactions. Secondly, novel computational techniques will be developed to capture physicality: parts of interest (POI), dimensions of interest (DOI), and motions of interest (MOI), which are critical parameters to augment passive interfaces distilling smart capabilities. Lastly, integrating the dataset and techniques, a modular fabrication pipeline will be developed along with a composable construction toolkit and libraries of smart augmentations, establishing capture—customize/fabricate--(re)program paradigm. Through educational objectives, this project advances teaching, training, and learning computing for women, students and educators of minorities, and promotes multi-disciplinary conversations via the Texas Human-Computer Interaction (TxHCI) seminar series.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.