EAGER: Prototyping Touch with FLS-Matter

EAGER：使用 FLS-Matter 制作触摸原型

• 批准号: 2232382
• 负责人: Shahram Ghandeharizadeh
• 金额: $ 25万
• 依托单位: University of Southern California
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-01-01 至 2024-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2232382&HistoricalAwards=false
• 关键词: EAGER; Prototyping; Touch; FLS; Matter

This project investigates techniques and algorithms that enable immersive and interactive 3D displays using swarms of flying light specks (FLS). An FLS is a micrometer drone with one or more light sources to generate different colors and textures with adjustable brightness. Synchronized swarms of FLSs will render objects in a pre-specified 3D volume, an FLS display. An immersive FLS display enables a user to touch and manipulate objects. For example, a user may pick up an FLS illuminated teapot or throw an FLS illuminated rock. The building block of these objects is FLS-matter, 2D (e.g., a square) and 3D (e.g., a pyramid) geometric shapes using FLSs with extendable behavior. FLS-matter may fundamentally transform how researchers think about human-machine interaction. A successful realization of its proof-of-concept prototype will provide essential insights into implementing an immersive and interactive 3D display. The size of such a display may be a tabletop cuboid, a telephone booth, or even a room. With the potential to transform the future of human communication and perception, and the public's interactions with information and data, these displays can potentially change how people work, learn, receive care, and socialize. The project includes plans for K-12 STEM outreach and for mentoring and training undergraduate, graduate, and high school students through this interdisciplinary research. A new graduate level seminar course on holodecks will be introduced. Apart from organizing the First international Conference on Holodeck at Los Angeles, the research team will participate in outreach activities such as Robotics Open House Events in the University of Southern California.FLS-matter is a novel transformative idea that requires a proof-of-concept prototype. This project focuses on creating a prototype that includes design and implementation of physics inspired frameworks that enable a swarm of FLS-matter to react to the force exerted by human touch without causing injury to the user or damaging the FLSs. Since micrometer-sized drones are not currently available, simulation studies will be used in conjunction with physical studies evaluating behavior of and user interaction with macro-sized off-the-shelf drones. The project will investigate the feasibility of a positioning system to identify the location of the FLSs, which in turn identifies the location of user touch. These studies will include design and implementation of both centralized and decentralized algorithms, and an evaluation of their accuracy and scalability characteristics as a function of the number of FLSs. This research will also use a combination of quantitative models in simulation and human subject studies with commercially available drones to quantify the amount of force exerted by a swarm based on its speed, flight pattern, mass of its FLSs, and the number of FLSs used at the perturbation point by a human touch. These user studies with multiple FLSs working together will determine the user’s contact method and a set of reasonable stiffness values for rendering an object. The tested FLSs will also be calibrated to determine thresholds specific to an individual user. Through the design and implementation of the FLS-matter concepts, this project addresses the challenges in mid-air haptics for user interactions with a swarm of flying robots, including failures, limited flight time between battery charges, collision-free flight paths, and coordinated force output.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目研究的技术和算法，使沉浸式和交互式的3D显示使用成群的飞行光点（FLS）。 FLS是一种具有一个或多个光源的微米无人机，可以产生不同的颜色和纹理，亮度可调。同步的FLS群将在预先指定的3D体积（FLS显示）中渲染对象。 沉浸式FLS显示器使用户能够触摸和操纵对象。 例如，用户可以拿起FLS照明的茶壶或投掷FLS照明的岩石。 这些对象的构建块是FLS物质，2D（例如，正方形）和3D（例如，金字塔）几何形状。 FLS物质可能会从根本上改变研究人员对人机交互的看法。其概念验证原型的成功实现将为实现沉浸式和交互式3D显示提供重要的见解。这种显示器的大小可以是桌面长方体，电话亭，甚至是房间。这些显示器有可能改变人类沟通和感知的未来，以及公众与信息和数据的互动，从而有可能改变人们工作、学习、接受护理和社交的方式。该项目包括K-12 STEM外展计划，以及通过跨学科研究指导和培训本科生，研究生和高中生的计划。一个新的研究生水平的研讨会课程全息甲板将被介绍。除了在洛杉矶举办第一届Holodeck国际会议外，研究团队还将参加南加州大学的机器人开放日活动等外展活动。FLS-matter是一个新颖的变革性想法，需要一个概念验证原型。该项目的重点是创建一个原型，其中包括设计和实施物理启发的框架，使一群FLS物质能够对人类触摸施加的力做出反应，而不会对用户造成伤害或损坏FLS。 由于微米级无人机目前不可用，模拟研究将与物理研究结合使用，评估宏观尺寸现成无人机的行为和用户交互。该项目将研究定位系统的可行性，以确定FLSs的位置，从而确定用户触摸的位置。这些研究将包括集中式和分散式算法的设计和实现，以及作为FLS数量的函数的准确性和可扩展性特征的评估。这项研究还将使用模拟和人类受试者研究中的定量模型与商用无人机相结合，以量化基于其速度，飞行模式，FLS质量以及在扰动点使用的FLS数量的群体施加的力。这些用户研究与多个FLS一起工作，将确定用户的接触方法和一组合理的刚度值渲染对象。 还将校准测试的FLS，以确定特定于单个用户的阈值。通过FLS-matter概念的设计和实施，该项目解决了用户与飞行机器人群交互的空中触觉挑战，包括故障，电池充电之间的有限飞行时间，无碰撞飞行路径，该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的评估被认为值得支持。影响审查标准。

项目成果

Virtual Reality, Augmented Reality, Mixed Reality, Holograms and Holodecks

虚拟现实、增强现实、混合现实、全息图和全息甲板

• 发表时间: 2023
• 期刊: First International Conference on Holodecks
• 作者: Ghandeharizadeh S.;Oria V.
• 通讯作者: Oria V.

Modeling Illumination Data with Flying Light Specks

• DOI: 10.1145/3587819.3592544
• 发表时间: 2023-06
• 期刊: Proceedings of the 14th Conference on ACM Multimedia Systems
• 作者: Hamed Alimohammadzadeh;Daryon Mehraban;Shahram Ghandeharizadeh

Towards Enabling Complex Touch-based Human-Drone Interaction

实现复杂的基于触摸的人机交互

• 发表时间: 2023
• 期刊: Human Multi-Robot Interaction at the 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2023
• 作者: Chen Y.;Alimohammadzadeh H.;Ghandeharizadeh S.;Culbertson H.
• 通讯作者: Culbertson H.

Dronevision: An Experimental 3D Testbed for Flying Light Specks

• DOI: 10.48550/arxiv.2308.10121
• 发表时间: 2023-08
• 期刊: ArXiv
• 作者: Hamed Alimohammadzadeh;Rohit Bernard;Yang Chen;Trung Phan;Prashant Singh;Shuqin Zhu;Heather Culbertson-He

A Conceptual Model of Intelligent Multimedia Data Rendered using Flying Light Specks

使用飞行光斑渲染的智能多媒体数据的概念模型

• 发表时间: 2023
• 期刊: First International Conference on Holodecks
• 作者: Yazdani N.;Alimohammadzadeh H.;Ghandeharizadeh S.
• 通讯作者: Ghandeharizadeh S.