EAGER: Prototyping Touch with FLS-Matter

该项目研究使用成群的飞行光斑(FLS)实现身临其境和交互的3D显示的技术和算法。FLS是一种带有一个或多个光源的微米无人机，可以产生亮度可调的不同颜色和纹理。同步的FLS群组将在预先指定的3D体积(FLS显示)中渲染对象。身临其境的FLS显示屏使用户能够触摸和操纵对象。例如，用户可以拿起FLS照明的茶壶或投掷FLS照明的石头。这些对象的构建块是使用具有可扩展行为的FLSS的FLS物质、2D(例如，正方形)和3D(例如，金字塔)几何形状。自由光物质可能会从根本上改变研究人员对人机交互的看法。其概念验证原型的成功实现将为实现身临其境的交互式3D显示提供必要的见解。这种显示器的大小可以是桌面长方体、电话亭，甚至是房间。这些展示有可能改变人类交流和感知的未来，以及公众与信息和数据的互动，可能会改变人们的工作、学习、接受护理和社交的方式。该项目包括K-12 STEM推广计划，以及通过这项跨学科研究指导和培训本科生、研究生和高中生的计划。将推出一个新的研究生级别的全息甲板研讨会课程。除了在洛杉矶组织第一次全息甲板国际会议外，研究团队还将参加推广活动，如南加州大学的机器人开放参观活动。FLS-Matter是一个新的变革性想法，需要一个概念验证原型。该项目专注于创建一个原型，其中包括设计和实现物理启发的框架，使一群FLS物质能够对人类触摸施加的力做出反应，而不会伤害用户或损坏FLSS。由于目前还没有微米级的无人机，模拟研究将与评估大型现成无人机的行为和用户交互的物理研究结合使用。该项目将研究定位系统的可行性，以识别FLSS的位置，进而识别用户触摸的位置。这些研究将包括集中式和分散式算法的设计和实施，以及评估它们的准确性和可伸缩性特征作为FLSS数量的函数。这项研究还将使用模拟和人类主题研究中的定量模型与商用无人机相结合，根据蜂群的速度、飞行模式、FLSS的质量以及人类触摸在扰动点使用的FLSS数量来量化蜂群施加的力量。这些多个FLSS一起工作的用户研究将确定用户的接触方法和一组用于渲染对象的合理刚度值。还将对测试的FLSS进行校准，以确定特定于单个用户的阈值。通过FLS-Matter概念的设计和实现，该项目解决了用户与一群飞行机器人交互在空中触觉方面的挑战，包括故障、电池充电之间的飞行时间有限、无碰撞飞行路径和协调的力量输出。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

This project investigates techniques and algorithms that enable immersive and interactive 3D displays using swarms of flying light specks (FLS). An FLS is a micrometer drone with one or more light sources to generate different colors and textures with adjustable brightness. Synchronized swarms of FLSs will render objects in a pre-specified 3D volume, an FLS display. An immersive FLS display enables a user to touch and manipulate objects. For example, a user may pick up an FLS illuminated teapot or throw an FLS illuminated rock. The building block of these objects is FLS-matter, 2D (e.g., a square) and 3D (e.g., a pyramid) geometric shapes using FLSs with extendable behavior. FLS-matter may fundamentally transform how researchers think about human-machine interaction. A successful realization of its proof-of-concept prototype will provide essential insights into implementing an immersive and interactive 3D display. The size of such a display may be a tabletop cuboid, a telephone booth, or even a room. With the potential to transform the future of human communication and perception, and the public's interactions with information and data, these displays can potentially change how people work, learn, receive care, and socialize. The project includes plans for K-12 STEM outreach and for mentoring and training undergraduate, graduate, and high school students through this interdisciplinary research. A new graduate level seminar course on holodecks will be introduced. Apart from organizing the First international Conference on Holodeck at Los Angeles, the research team will participate in outreach activities such as Robotics Open House Events in the University of Southern California.FLS-matter is a novel transformative idea that requires a proof-of-concept prototype. This project focuses on creating a prototype that includes design and implementation of physics inspired frameworks that enable a swarm of FLS-matter to react to the force exerted by human touch without causing injury to the user or damaging the FLSs. Since micrometer-sized drones are not currently available, simulation studies will be used in conjunction with physical studies evaluating behavior of and user interaction with macro-sized off-the-shelf drones. The project will investigate the feasibility of a positioning system to identify the location of the FLSs, which in turn identifies the location of user touch. These studies will include design and implementation of both centralized and decentralized algorithms, and an evaluation of their accuracy and scalability characteristics as a function of the number of FLSs. This research will also use a combination of quantitative models in simulation and human subject studies with commercially available drones to quantify the amount of force exerted by a swarm based on its speed, flight pattern, mass of its FLSs, and the number of FLSs used at the perturbation point by a human touch. These user studies with multiple FLSs working together will determine the user’s contact method and a set of reasonable stiffness values for rendering an object. The tested FLSs will also be calibrated to determine thresholds specific to an individual user. Through the design and implementation of the FLS-matter concepts, this project addresses the challenges in mid-air haptics for user interactions with a swarm of flying robots, including failures, limited flight time between battery charges, collision-free flight paths, and coordinated force output.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.