Frameless video system: emulating the human visual system

在最实际的意义上，现实世界是模拟的和连续的。我们听到的声音有一个幅度随时间变化，以一种连续和模拟的方式。我们通过眼睛感受到的光在幅度和时间上也是连续的，因为眼睛的视网膜接收到持续的光子流，通过化学反应将信息发送到大脑。 然而，传统的摄像机无法捕捉人类看到的光信息，因为它们仅限于随着时间的推移捕捉离散的帧。电影和视频是聪明和复杂的发明，给人的眼睛带来了运动和现实感的错觉，但却愚弄了他们，让他们相信他们看到的是真实世界的真实复制品。 在离散帧中捕捉视频的挑战可以追溯到1892年，当时爱迪生和迪克森开发了一种名为电影放映机的设备，通过每秒大约15帧的回放来观看产生的运动图像。直到后来，声音的添加才迫使帧增加到每秒24帧。最近，这些速率显著增加，但我们继续将视频视为离散的图像序列，对像素信息、动态范围和精度施加了不必要的限制，导致运动伪影、压缩挑战和整体有限的体验质量。更重要的是，这种频闪观察对人类来说肯定是令人厌倦的，因为眼睛和大脑必须提供感知连续运动所需的时间整合。 我们认为，考虑到视频捕获和处理技术的最新进展，我们捕获、处理和显示视觉信息的方式进行范式转变的时机已经成熟，密切模仿人类的视觉系统及其感知现实世界连续性的能力。我们的长期目标是更好地理解连续时间视频表示，并开发框架和必要的技术，以满足无帧视频的视觉质量、存储、传输和显示要求，从而实现更真实、更悦目的真实世界的呈现。 这对数字媒体世界的影响是非常重要的。如果无框架视频显示出希望，广播和娱乐业以及消费电子行业将不得不抓住机会，转向向消费者提供娱乐、新闻和社交媒体内容的新方式。我们怀疑计算机视觉行业将不得不做出类似的反应，推动数字媒体行业的重大新市场机遇，这些行业代表着世界上增长最快的一些行业，市场增长率为25%，预计到2021年全球销售额将达到2150亿美元。

The real world is analog and continuous in the most practical sense. The sound we hear has an amplitude variation over time, in a continuous and analog fashion. The light we sense through our eyes is also continuous both in amplitude and time, as the eye's retina receives a continuous flow of photons, sending that information to the brain through a chemical reaction. However, conventional video cameras cannot capture light information the way the human sees it, as they are limited to capturing discrete frames over time. Movies and videos are clever and sophisticated inventions that give the illusion of motion and realism to the human eyes, yet fooling them to believe they are seeing a faithful replica of the real world. The challenge of capturing video in discrete frames goes back to 1892, when Edison and Dickson developed a device called the Kinetoscope to view the resulting motion picture by playing back about 15 frames per second. It was until later that the addition of sound forced the frames to increase to 24 per second. Recently, these rates have significantly increased, but we continue treating video as a discrete sequence of images, imposing an unnecessary constraint on pixel information, dynamic range and precision, causing motion artifacts, compression challenges and an overall limited quality of experience. More importantly, this stroboscopic viewing is definitely tiresome to humans, as the eye and the brain have to provide the temporal integration necessary to perceive continuous motion. We believe that, considering the recent advances in video capturing and processing technologies, the time is ripe for a paradigm shift in the way we capture, process and display visual information, closely emulating the human visual system and its ability to perceive the continuity of the real world. Our long-term objective is to embark on a better understanding of the continuous time video representation and develop the framework and necessary technologies that will address visual quality, storage, delivery, and display requirements for frameless video, thus arriving at a more accurate rendition of the real world that is authentic and more pleasing to the eye. The implication to the digital media world is very significant. If frameless video shows promise, the broadcasting and entertainment industry, as well as the consumer electronics industry will have to embrace the opportunity to migrate to new ways to deliver entertainment, news and social media content to consumers. We suspect a similar reaction will have to occur for the computer vision community, driving significant new market opportunities across the digital media industries which represent some of the fastest growing sectors in the world, with a market growth rate of 25% and global sales forecast to hit US$215 billion by 2021.