I-Corps Teams: Invisible Light Field Messaging

这个I-Corps项目的更广泛的影响/商业潜力在光场信息领域是重要的。这项技术的目标是与机器进行无形的交流，而不是与可能观察图像的用户进行潜意识的交流。信息被嵌入在显示器上的图像中，这样信息是机器可读的，但人类是看不见的。电子显示器，如广告牌或售货亭，保留其目的，以传达视觉信息，如地图，广告和时间表。同时，显示器将时变信息传递给配备摄像头的计算系统，如智能手机或机器人。近期目标应用是使用智能手机上现有摄像头的交互式电视、计算机显示器和电子广告牌。有了光场信息，基于手机的移动应用程序可以：从电视广告中检索产品信息，从路边的广告牌上获取附近的交通状况，或者从报亭上获取大型机场内的步行方向。这种方法有望为互动营销和媒体提供重要的新途径。此外，该技术将提供新的室内定位方法，而GPS并不精确。向机器人系统和自动驾驶汽车发送导航提示的光场信息是一个很有前途的未来应用。这个I-Corps项目使用创新的深度学习架构在每一帧视频中嵌入新的信息。这项技术的目标是与机器进行无形的交流，而不是与可能观察图像的用户进行潜意识的交流。该方法由基于软件的算法组成，主要包括两个部分：1)嵌入方法，将隐藏信息嵌入/编码到照片和视频中；2)在相机侧检索信息的恢复方法。应用软件将运行在用于嵌入的显示系统和智能手机或其他相机系统上，用于从电子显示器上嵌入的图像中获取信息。与数字隐写等旧方法不同，该方法使用现代神经网络学习一种鲁棒编码方法，克服了光场传输通道的畸变。这种消息传递范式还改进了先前的qr码，因为代码不会分散可见图像，并且代码是动态的，允许传输更多的信息。信息被完全包含在一帧视频中，这样就避免了时间同步的问题。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The broader impact/commercial potential of this I-Corps project is significant in the area of light field messaging. The goal of this technology is invisible communication to a machine, and not subliminal communication to users who may observe the image. Information is embedded within images on a display so that the messages are machine-readable but invisible to humans. The electronic display, such as a billboard or kiosk, retains its purpose to convey visual information such as maps, advertisements and schedules. Simultaneously, the display conveys a time-varying message to a camera-equipped computational system, such as a smartphone or robot. The near-term target application is interactive televisions, computer displays, and electronic billboards using existing cameras on smartphones. With light field messaging, a phone-based mobile app could: retrieve product information from a television ad, obtain nearby traffic conditions from a roadside billboard, or obtain walking directions within a large airport from a kiosk. This approach is expected to provide significant new avenues of interactive marketing and media. Additionally, the technology will provide novels methods of indoor localization, where GPS is not precise. Light field messaging for sending navigation cues to robotic systems and self-driving cars is a promising future application. This I-Corps project uses an innovative deep learning architecture to embed a new message in each frame of video. The goal of this technology is invisible communication to a machine, and not subliminal communication to users who may observe the image. The approach is comprised of software-based algorithms with two main components: 1) Embedding method that embeds/codes hidden messages into photos and videos together and 2) Recovery method on the camera side that retrieves the message. Application software will run on display systems for embedding and on smartphones or other camera systems for obtaining messages from embedded imagery on electronic displays. Unlike older methods such as digital steganography, this approach uses modern neural networks to learn a robust coding method that overcomes the distortion of the light field transmission channel. This messaging paradigm also improves on prior QR-codes because the code does not distract from the visible image and the code is dynamic, allowing significantly more information to be transmitted. The message is fully contained within a single frame of video so that issues of time synchronization are avoided.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.