EAPSI: Developing a Semantic Attributes Learner through Machine Learning Approaches

该项目旨在证明计算机视觉中的机器学习方法可以发现美术分类的两个关键组成部分：第一，人类如何识别和分类目标对象的不同视觉风格，第二，他们使用什么语义视觉属性来完成分类决策。该项目将使用大量的美术绘画数据集，研究一种计算程序，以确定一系列不同视觉风格的文字描述，这些文字描述对人类来说是可解释的，并且对所有绘画风格的编码都是有效的。很难提供客观的理由来确定一种视觉风格：即使是艺术专家，也不容易解释为什么是克劳德·莫奈？的罂粟花被归类为印象派的基础上，其属性。如果计算算法自动找到确定人类观察者可识别的视觉风格的语义属性，则结果将提供对已知指定起来复杂的人类视觉感知过程的科学分析。稳定后，该算法将生成描述视觉风格的大量图像数据集，而无需昂贵的人工工作。该数据集将是未来计算机视觉研究的有用数据集。该项目将与韩国延世大学数据智能实验室的Seung Wan Hwang教授合作进行。黄教授设计了通过数据模式分析来寻找语义属性的定性和定量方法。该奖项支持了一项研究，即从数据集设计属性学习器算法，以实现美术绘画风格的分类，并生成包含艺术作品有价值特征的扩展数据集，这些数据集可以通过学习的属性生成器自动注释。PI将设计一个学习器，而不是一个昂贵的注释训练集来学习感兴趣的属性，该学习器可以在没有人类监督的情况下自动收获属性。这种方法消除了对需要专家注释的语义属性的预定义（和潜在的主观）词汇表的需要。该项目将使用通过深度人工神经网络（ANN）模型获得的数值高维数据。人工神经网络模型是通过一个大的图像数据训练的艺术风格推理。随着无监督的深层架构，相关的图像和纹理数据通过一个共享的隐藏层，预计隐藏层？的正或负变量将被解释为信息属性。数据集将包括一些冗余和难以破译的信息，因此需要压缩并转换为人类可解释的概念。由于数据集的可用地面实况信息仅限于作者、年份和艺术风格，因此与主办研究人员的合作工作将集中在地面实况信息和数字数据之间的模式信息的提取上。学术界已有类似的研究工作涉及特征提取，但对于美术风格和无监督属性学习这一新课题，本文的研究具有一定的创新性。东亚和太平洋夏季研究所计划下的这个奖项支持美国研究生的夏季研究，由NSF和韩国国家研究基金会共同资助。

This project aims to prove that machine learning approaches in computer vision can discover two key components of fine art classification: first, how humans recognize and classify different visual styles for a target object, and second, what semantic visual attributes they use to finalize their classification decision. Working with a large data set of fine art paintings, the project will investigate a computational procedure to identify a list of word descriptions of different visual styles that is interpretable to humans and is further valid to encode all styles of painting. It can be difficult to provide objective grounds that necessarily determine a visual style: even for the art expert, it is not easy to explain why Claude Monet?s Poppies is classified as impressionist based on its attributes. If the computational algorithm automatically finds semantic attributes determining visual styles that are recognizable to human observers, the result will provide scientific analysis of the human visual perceptual process which is known to be complex to specify. After stabilization, the algorithm will generate annotations describing visual styles for a massive image data set without expensive human work. This data set will be useful data set for future computer vision research. This project will be conducted in collaboration with Professor Seung Wan Hwang in the Data Intelligence Lab at Yonsei University in Korea. Professor Hwang has devised qualitative and quantitative methods to find semantic attributes through data pattern analysis.This award supports a research study to design an attributes learner algorithm from datasets that will enable classification of fine art painting styles, and produce extended datasets containing valuable features of the art work that can be annotated automatically via learned attributes generators. Rather than an expensive training set of annotations to learn the attributes of interest, the PI will design a learner which automatically harvests attributes without human supervision. This approach eliminates the need for a predefined (and potentially subjective) vocabulary of semantic attributes which require expert annotation. The project will use numeric high dimensional data gotten through a Deep Artificial Neural Net (ANN) model. The ANN model is trained through a big image data targeting art style inference. With the unsupervised deep architecture that correlates images and textural data through a shared hidden layer, it is expected that the hidden layer?s positive or negative variables will be interpreted as informative attributes. The data set will include some amount of redundant and hard-to-decipher information, so it requires compression and translation to human-interpretable concepts. Since available ground truth information of the data set is limited to authors, year, and art style, cooperative work with the hosting researcher will focus on the extraction of pattern information between the ground truth information and numeric data. There has been similar research work related to feature extraction in academia, but regarding the new subject of Fine Art style and unsupervised attributes learning, this research will be innovative. This award under the East Asia and Pacific Summer Institutes program supports summer research by a U.S. graduate student and is jointly funded by NSF and the National Research Foundation of Korea.