EAPSI: Developing a Semantic Attributes Learner through Machine Learning Approaches

该项目旨在证明计算机视觉中的机器学习方法可以发现美术分类的两个关键组成部分：第一，人类如何识别和分类目标对象的不同视觉风格，第二，他们使用什么语义视觉属性来完成分类决策。该项目将使用大量美术绘画数据集，研究一种计算程序，以识别人类可解释的不同视觉风格的单词描述列表，并进一步有效地编码所有风格的绘画。很难提供客观的依据来确定一种视觉风格：即使对艺术专家来说，也不容易解释为什么克劳德·莫奈？根据其特点，《罂粟》被归类为印象派作品。如果计算算法自动找到确定人类观察者可识别的视觉样式的语义属性，则结果将为人类视觉感知过程提供科学分析，这是已知的复杂的具体说明。稳定后，该算法将为大量图像数据集生成描述视觉样式的注释，而无需昂贵的人工工作。该数据集将为今后的计算机视觉研究提供有用的数据集。该项目将与韩国延世大学数据智能实验室的Hwang Seung Wan教授合作进行。黄禹锡教授设计了通过数据模式分析寻找语义属性的定性和定量方法。该奖项支持一项从数据集设计属性学习算法的研究，该算法将实现美术绘画风格的分类，并产生包含艺术作品有价值特征的扩展数据集，这些特征可以通过学习属性生成器自动注释。PI将设计一个无需人工监督即可自动获取属性的学习器，而不是使用昂贵的注释训练集来学习感兴趣的属性。这种方法消除了对需要专家注释的预定义（可能是主观的）语义属性词汇表的需求。该项目将使用通过深度人工神经网络（ANN）模型获得的数值高维数据。该人工神经网络模型通过大图像数据进行训练，目标是艺术风格推断。通过共享隐藏层将图像和纹理数据关联起来的无监督深度架构，可以预期隐藏层？S个正或负变量将被解释为信息属性。数据集将包含一定数量的冗余和难以破译的信息，因此需要压缩和翻译为人类可解释的概念。由于数据集的可用真实信息仅限于作者、年份和艺术风格，因此与托管研究人员的合作工作将侧重于提取真实信息和数字数据之间的模式信息。学术界已经有了与特征提取相关的类似研究工作，但对于美术风格和无监督属性学习这一新课题，本研究将具有创新性。该奖项是由美国国家科学基金会（NSF）和韩国国立科学研究财团共同资助的东亚太平洋暑期研究所项目，旨在支持美国研究生的暑期研究。

This project aims to prove that machine learning approaches in computer vision can discover two key components of fine art classification: first, how humans recognize and classify different visual styles for a target object, and second, what semantic visual attributes they use to finalize their classification decision. Working with a large data set of fine art paintings, the project will investigate a computational procedure to identify a list of word descriptions of different visual styles that is interpretable to humans and is further valid to encode all styles of painting. It can be difficult to provide objective grounds that necessarily determine a visual style: even for the art expert, it is not easy to explain why Claude Monet?s Poppies is classified as impressionist based on its attributes. If the computational algorithm automatically finds semantic attributes determining visual styles that are recognizable to human observers, the result will provide scientific analysis of the human visual perceptual process which is known to be complex to specify. After stabilization, the algorithm will generate annotations describing visual styles for a massive image data set without expensive human work. This data set will be useful data set for future computer vision research. This project will be conducted in collaboration with Professor Seung Wan Hwang in the Data Intelligence Lab at Yonsei University in Korea. Professor Hwang has devised qualitative and quantitative methods to find semantic attributes through data pattern analysis.This award supports a research study to design an attributes learner algorithm from datasets that will enable classification of fine art painting styles, and produce extended datasets containing valuable features of the art work that can be annotated automatically via learned attributes generators. Rather than an expensive training set of annotations to learn the attributes of interest, the PI will design a learner which automatically harvests attributes without human supervision. This approach eliminates the need for a predefined (and potentially subjective) vocabulary of semantic attributes which require expert annotation. The project will use numeric high dimensional data gotten through a Deep Artificial Neural Net (ANN) model. The ANN model is trained through a big image data targeting art style inference. With the unsupervised deep architecture that correlates images and textural data through a shared hidden layer, it is expected that the hidden layer?s positive or negative variables will be interpreted as informative attributes. The data set will include some amount of redundant and hard-to-decipher information, so it requires compression and translation to human-interpretable concepts. Since available ground truth information of the data set is limited to authors, year, and art style, cooperative work with the hosting researcher will focus on the extraction of pattern information between the ground truth information and numeric data. There has been similar research work related to feature extraction in academia, but regarding the new subject of Fine Art style and unsupervised attributes learning, this research will be innovative. This award under the East Asia and Pacific Summer Institutes program supports summer research by a U.S. graduate student and is jointly funded by NSF and the National Research Foundation of Korea.