CAREER: Theory, Software Tools, and Applications for Algorithmic Visual Information Representation

提出了一种新的媒体表示算法框架的研究和开发，重点是数字图像和视频。 特别是，一个新的体系结构的视觉信息表示的可编程终端的概念的基础上，详细说明，基本上考虑作为图灵机的解码器。 在这样的框架中，算法本身成为内容，而不仅仅是用来处理内容。 这种方法的好处是多方面的，并且可以通过考虑计算机使用从用于数字处理大型机的简单文本终端到今天托管Web浏览器的强大图形用户界面的转变所带来的革命性好处来理解。 计算机最初被认为是计算设备，但并不主要用于生成基本形式的算法内容，如窗口、菜单、图标、字体等，该框架使用户能够执行高级文本文档的创建，分发，访问等。建议的框架将这种范例推广到数字图像和视频的情况下，并允许其在广泛的应用中使用。 向算法表示的转变需要理论和应用创新。 从理论的角度来看，它表明，可以开发一个新的数学理论（称为复杂性失真理论），提供了一个通用的数学框架，所有的编码技术（从传统的熵编码到基于模型的视频编码）可以适当地分析。 该理论建立在可编程终端的概念上，并且对于在时间和空间约束下操作的真实的系统，或者当压缩仅是表示要求之一时，承诺显著的结果。 从实用的，面向应用的角度来看，这种新的方法可以消除几十年的滞后视听信息表示相对于现代软件设计。 描述了一种基于软件的阿尔奇结构，围绕一种新的编程语言范式（FLAVOR -视听对象表示的形式语言），并开发为真实的功能系统提供基础。 几个应用程序和应用领域进行了讨论，在其中所提出的方法可以产生根本性的影响，使用户能够无缝创建，操纵和访问视觉信息。

The investigation and development of a new algorithmic framework for media representation is proposed, with emphasis on digital images and video. In particular, a new architecture for visual information representation is detailed based on the notion of programmable terminals, essentially considering the decoder as a Turing machine. In such a framework, the algorithm becomes itself the content, rather than just being used to process content. The benefits of such an approach are numerous, and can be appreciated by considering the revolutionary benefits derived from the transition of computer usage from the days of simple text terminals for number-crunching mainframes to today's powerful graphical user interfaces that host Web browsers. Computers were initially considered as calculation devices, but are not mostly used to generate elementary forms of algorithmic content, such as windows, menus, icons, fonts, etc., that empower users to perform advanced text document creation, distribution, access, etc. The proposed framework generalizes this paradigm to the case of digital images and video, and allows its use in a wide range of applications. The shift towards algorithmic representations requires both theoretical and applied innovations. From a theoretical standpoint, it is shown that a new mathematical theory can be developed (called Complexity Distortion Theory), providing a universal mathematical framework in which all coding techniques (from traditional entropy coding to model-based video coding) can be appropriately analyzed. This theory builds on the notion of a programmable terminal and promises significant results for real systems that operate under time and space constraints, or when compression is only one of the representation requirements. From a practical, applications-oriented standpoint, this new approach allows the elimination of decades of lag of audio-visual information representation with respect to modern software design. A software-based archi tecture is described, centered around a new programming language paradigm (FLAVOR - Formal Language for Audio-Visual Object Representation), and developed to provide the basis for real, functioning systems. Several applications and application domains are discussed in which the proposed approach can have a fundamental impact, empowering users with seamless creation, manipulation, and access to visual information.