EAGER: Prototyping Touch with FLS-Matter

该项目研究了利用飞行光斑群（FLS）实现沉浸式交互式3D显示的技术和算法。FLS是一种带有一个或多个光源的微米无人机，可以产生不同的颜色和纹理，亮度可调。同步的FLS群将在预先指定的3D体积中渲染对象，这是一个FLS显示。身临其境的FLS显示器使用户能够触摸和操纵物体。例如，用户可以拿起一个FLS照明的茶壶或扔一个FLS照明的石头。这些对象的构建块是fls物质，2D（例如，正方形）和3D（例如，金字塔）几何形状，使用具有可扩展行为的fls。fls物质可能会从根本上改变研究人员对人机交互的看法。其概念验证原型的成功实现将为实现沉浸式交互式3D显示提供重要见解。这种显示器的大小可以是桌面的长方体，电话亭，甚至是一个房间。这些展示有可能改变人类沟通和感知的未来，以及公众与信息和数据的互动，它们有可能改变人们工作、学习、接受照顾和社交的方式。该项目包括K-12 STEM推广计划，以及通过这项跨学科研究指导和培训本科生、研究生和高中生。一个新的研究生水平的研讨会课程将介绍全息甲板。除了在洛杉矶组织第一届全息甲板国际会议外，研究团队还将参加南加州大学的机器人开放日活动等外展活动。FLS-matter是一个新颖的变革性想法，需要一个概念验证原型。这个项目的重点是创建一个原型，包括设计和实现受物理启发的框架，使一群fls物质能够对人类触摸施加的力做出反应，而不会对用户造成伤害或损坏fls。由于微米尺寸的无人机目前还不可用，模拟研究将与物理研究结合使用，评估宏观尺寸的现成无人机的行为和用户交互。该项目将研究一种定位系统的可行性，以识别fls的位置，从而识别用户触摸的位置。这些研究将包括集中式和分散式算法的设计和实现，以及作为FLSs数量函数的准确性和可扩展性特征的评估。本研究还将结合模拟中的定量模型和商用无人机的人类受试者研究，根据其速度、飞行模式、fls质量以及人类触摸摄动点使用的fls数量，量化蜂群施加的力的大小。这些与多个FLSs一起工作的用户研究将确定用户的接触方法和一组用于呈现对象的合理刚度值。还将对测试的FLSs进行校准，以确定特定于个人用户的阈值。通过FLS-matter概念的设计和实现，该项目解决了用户与一群飞行机器人交互的空中触觉挑战，包括故障、电池充电之间的有限飞行时间、无碰撞飞行路径和协调力输出。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This project investigates techniques and algorithms that enable immersive and interactive 3D displays using swarms of flying light specks (FLS). An FLS is a micrometer drone with one or more light sources to generate different colors and textures with adjustable brightness. Synchronized swarms of FLSs will render objects in a pre-specified 3D volume, an FLS display. An immersive FLS display enables a user to touch and manipulate objects. For example, a user may pick up an FLS illuminated teapot or throw an FLS illuminated rock. The building block of these objects is FLS-matter, 2D (e.g., a square) and 3D (e.g., a pyramid) geometric shapes using FLSs with extendable behavior. FLS-matter may fundamentally transform how researchers think about human-machine interaction. A successful realization of its proof-of-concept prototype will provide essential insights into implementing an immersive and interactive 3D display. The size of such a display may be a tabletop cuboid, a telephone booth, or even a room. With the potential to transform the future of human communication and perception, and the public's interactions with information and data, these displays can potentially change how people work, learn, receive care, and socialize. The project includes plans for K-12 STEM outreach and for mentoring and training undergraduate, graduate, and high school students through this interdisciplinary research. A new graduate level seminar course on holodecks will be introduced. Apart from organizing the First international Conference on Holodeck at Los Angeles, the research team will participate in outreach activities such as Robotics Open House Events in the University of Southern California.FLS-matter is a novel transformative idea that requires a proof-of-concept prototype. This project focuses on creating a prototype that includes design and implementation of physics inspired frameworks that enable a swarm of FLS-matter to react to the force exerted by human touch without causing injury to the user or damaging the FLSs. Since micrometer-sized drones are not currently available, simulation studies will be used in conjunction with physical studies evaluating behavior of and user interaction with macro-sized off-the-shelf drones. The project will investigate the feasibility of a positioning system to identify the location of the FLSs, which in turn identifies the location of user touch. These studies will include design and implementation of both centralized and decentralized algorithms, and an evaluation of their accuracy and scalability characteristics as a function of the number of FLSs. This research will also use a combination of quantitative models in simulation and human subject studies with commercially available drones to quantify the amount of force exerted by a swarm based on its speed, flight pattern, mass of its FLSs, and the number of FLSs used at the perturbation point by a human touch. These user studies with multiple FLSs working together will determine the user’s contact method and a set of reasonable stiffness values for rendering an object. The tested FLSs will also be calibrated to determine thresholds specific to an individual user. Through the design and implementation of the FLS-matter concepts, this project addresses the challenges in mid-air haptics for user interactions with a swarm of flying robots, including failures, limited flight time between battery charges, collision-free flight paths, and coordinated force output.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.