CAREER: Representing, Understanding, and Enhancing Scenes at the Internet-Scale

职业：在互联网规模上呈现、理解和增强场景

• 批准号: 1641340
• 负责人: James Hays
• 金额: $ 9.53万
• 依托单位: Georgia Tech Research Corporation
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2016
• 资助国家: 美国
• 起止时间: 2016-03-01 至 2017-12-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1641340&HistoricalAwards=false
• 关键词: CAREER; Representing; Understanding; Enhancing; Scenes

CAREER: Understanding, Representing, and Enhancing Scenes at the Internet-scalePhotography has an enormous impact on society -- it is our primary visual history and a medium for storytelling, entertainment, and art. But our visual world is extraordinarily complex which makes it difficult for computer vision to understand photos and for computer graphics to synthesize visual content. However, the emergence of Internet-scale photo collections in recent years enables new research directions. We use scene-based representations to leverage Internet-scale data. Scenes (places or environments) are the context in which all other visual phenomena exist and it seems possible to brute-force the space of scenes -- with millions of scenes, we find qualitatively similar scenes and create massively data-driven algorithms with capabilities that are complementary to typical bottom-up graphics and vision pipelines. The underlying principle of this study is that joint investigations of scene representations and large image databases will advance the state-of-the-art in graphics and vision. First, we are investigating detail synthesis tasks which alleviate camera shake, motion blur, defocus, atmospheric scattering, or low resolution. Scene representations are robust enough to find matching scenes in Internet-scale photo collections even in the presence of dramatic blurring. These matching scenes provide a context-specific statistical model which can be used to insert convincing texture and object detail. Second, we are studying attribute-based representations of scenes. We use crowdsourcing to discover attributes and build large databases for the community. Attributes are a powerful intermediate representation for the next generation of big data imaging research which can have broad societal impact through applications such as robotics, security, assistance to vision-impaired, and vehicle safety. The investigators also are developing a new introductory course for Brown students to explore big data computing across scientific disciplines and are creating an online community for visual computing education to benefit students interested in photography and programming.

职业生涯：理解、表现和增强互联网规模的场景摄影对社会有着巨大的影响--它是我们主要的视觉历史，也是讲故事、娱乐和艺术的媒介。但我们的视觉世界非常复杂，这使得计算机视觉很难理解照片，计算机图形学也很难合成视觉内容。然而，近年来互联网规模的照片收藏的出现为研究提供了新的方向。我们使用基于场景的表示来利用互联网规模的数据。场景(地方或环境)是所有其他视觉现象存在的背景，似乎有可能强行占用场景的空间--通过数百万个场景，我们找到了定性上相似的场景，并创建了大量数据驱动的算法，其功能与典型的自下而上的图形和视觉管道互补。这项研究的基本原理是，对场景表示和大型图像数据库的联合调查将推动图形和视觉的最新发展。首先，我们正在研究细节合成任务，以减轻相机抖动、运动模糊、散焦、大气散射或低分辨率。场景表示足够强大，即使在存在戏剧性模糊的情况下，也可以在互联网规模的照片集中找到匹配的场景。这些匹配场景提供了特定于上下文的统计模型，该模型可用于插入令人信服的纹理和对象细节。第二，我们正在研究场景的基于属性的表示。我们使用众包来发现属性，并为社区建立大型数据库。属性是下一代大数据成像研究的强大中间表示，这些研究可以通过机器人、安全、帮助视力受损和车辆安全等应用产生广泛的社会影响。调查人员还为布朗大学的学生开发了一门新的入门课程，让他们探索科学学科中的大数据计算，并正在为视觉计算教育创建一个在线社区，以造福于对摄影和编程感兴趣的学生。