I-Corps Teams: Invisible Light Field Messaging

I-Corps 团队：隐形光场消息传递

• 批准号: 1907550
• 负责人: Kristin Dana
• 金额: $ 5万
• 依托单位: Rutgers University New Brunswick
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-12-01 至 2020-11-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1907550&HistoricalAwards=false
• 关键词: Corps; Teams; Invisible; Light; Field

The broader impact/commercial potential of this I-Corps project is significant in the area of light field messaging. The goal of this technology is invisible communication to a machine, and not subliminal communication to users who may observe the image. Information is embedded within images on a display so that the messages are machine-readable but invisible to humans. The electronic display, such as a billboard or kiosk, retains its purpose to convey visual information such as maps, advertisements and schedules. Simultaneously, the display conveys a time-varying message to a camera-equipped computational system, such as a smartphone or robot. The near-term target application is interactive televisions, computer displays, and electronic billboards using existing cameras on smartphones. With light field messaging, a phone-based mobile app could: retrieve product information from a television ad, obtain nearby traffic conditions from a roadside billboard, or obtain walking directions within a large airport from a kiosk. This approach is expected to provide significant new avenues of interactive marketing and media. Additionally, the technology will provide novels methods of indoor localization, where GPS is not precise. Light field messaging for sending navigation cues to robotic systems and self-driving cars is a promising future application. This I-Corps project uses an innovative deep learning architecture to embed a new message in each frame of video. The goal of this technology is invisible communication to a machine, and not subliminal communication to users who may observe the image. The approach is comprised of software-based algorithms with two main components: 1) Embedding method that embeds/codes hidden messages into photos and videos together and 2) Recovery method on the camera side that retrieves the message. Application software will run on display systems for embedding and on smartphones or other camera systems for obtaining messages from embedded imagery on electronic displays. Unlike older methods such as digital steganography, this approach uses modern neural networks to learn a robust coding method that overcomes the distortion of the light field transmission channel. This messaging paradigm also improves on prior QR-codes because the code does not distract from the visible image and the code is dynamic, allowing significantly more information to be transmitted. The message is fully contained within a single frame of video so that issues of time synchronization are avoided.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

这个I-Corps项目的广泛影响/商业潜力在轻型外地信息领域具有重要意义。 这项技术的目标是与机器进行隐形通信，而不是与可能观察图像的用户进行潜意识通信。信息被嵌入显示器上的图像中，因此这些信息是机器可读的，但对人类是不可见的。电子显示器，如广告牌或信息亭，保留其传达视觉信息，如地图，广告和时间表的目的。 同时，显示器将时变消息传递到配备相机的计算系统，例如智能手机或机器人。近期目标应用是使用智能手机上现有摄像头的交互式电视、计算机显示器和电子广告牌。 通过光场消息传递，基于手机的移动的应用程序可以：从电视广告中检索产品信息，从路边的广告牌中获取附近的交通状况，或者从信息亭中获取大型机场内的步行方向。 预计这种方法将为互动营销和媒体提供重要的新途径。此外，该技术将提供室内定位的新方法，其中GPS不精确。用于向机器人系统和自动驾驶汽车发送导航提示的光场消息传递是一种有前途的未来应用。这个I-Corps项目使用创新的深度学习架构在每帧视频中嵌入新消息。这项技术的目标是与机器进行隐形通信，而不是与可能观察图像的用户进行潜意识通信。该方法由基于软件的算法组成，具有两个主要组成部分：1）将隐藏消息嵌入/编码到照片和视频中的嵌入方法，以及2）在相机侧检索消息的恢复方法。应用软件将在显示系统上运行，用于嵌入，并在智能手机或其他相机系统上运行，用于从电子显示器上的嵌入图像中获取信息。与数字隐写术等旧方法不同，这种方法使用现代神经网络来学习一种鲁棒的编码方法，该方法克服了光场传输通道的失真。 这种消息传递模式还改进了先前的QR码，因为代码不会分散可见图像的注意力，并且代码是动态的，允许传输更多的信息。 该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。