Predicting and Grouping Digitized Paintings by Style using Unsupervised Feature Learning.
Predicting and Grouping Digitized Paintings by Style using Unsupervised Feature Learning.
复制标题
DOI：
10.1016/j.culher.2017.11.008
复制
发表时间：
2018-05
影响因子：
3.1
通讯作者：
Makrehchi M
中科院分区：
文献类型：
--
作者：
Gultepe E;Conturo TE;Makrehchi M
创建一个系统，通过根据自动学习的风格特征对数字化绘画进行分类和分组，帮助分析艺术史，而无需事先了解。来自8种不同艺术风格（新艺术、巴洛克、表现主义、印象派、现实主义、浪漫主义、文艺复兴和后印象派）的6776幅数字化画作被用于根据风格对画作进行分类（预测）和聚类（分组）。利用受深度学习启发的无监督特征学习方法（unsupervised feature learning with K-means， UFLK）从画作中提取特征。然后将这些特征用于：(1)基于具有已知风格标签的绘画训练集的支持向量机算法对新测试绘画的风格进行分类；(2)光谱聚类算法将画作分成不同的风格组（匿名，不使用任何已知的风格标签）。分类效果由准确率和f分决定。聚类性能由以下因素决定：1)恢复原始风格分组的能力（使用8组标签分配的所有可能组合的成本分析）；2) f值;3)可靠性分析。后一种分析使用了两种新颖的方法来确定零假设的分布：(a)在原始数据主成分上的均匀分布；(b)随机加权邻接矩阵。通过对聚类结果进行语义分析，测试了获得艺术洞察力的能力。为此，我们用一个n维特征向量来表示每幅画的特征特征，并绘制出向量端点之间的距离（即画之间的相似性）。然后，我们用分配的最低成本样式标签对端点进行颜色编码。散点图被目视检查绘画的分离，其中颜色簇之间的分离量提供了风格之间相互关系的语义信息。uflk提取的特征与绘画中的边缘/线条/颜色相似。对于基于特征的绘画分类，宏观平均f值为0.469。与使用更复杂的特征学习模型（例如，卷积神经网络，一种监督算法）的其他分类方法相比，分类精度和f分数相似或更高。通过uflk提取的特征聚类产生了8个未标记的样式组。在8个簇中的6个簇中，最常见的真实绘画风格与成本分析分配的簇风格相匹配。聚类的f值为0.212。（目前还没有对绘画进行分类的比较方法。）在语义分析中，发现巴洛克风格和新艺术风格的特征特征是相似的，这表明了这两种风格之间的关系。UFLK方法可以从数字化绘画中提取特征。我们能够在没有任何关于特征性质或绘画风格指定的事先信息的情况下提取艺术特征。本文的方法可以为艺术研究人员提供最新的计算技术，用于记录、解释和鉴定艺术。这些工具有助于为子孙后代保存具有文化敏感性的艺术作品，并为艺术作品和创作它们的艺术家提供新的见解。
To create an system to aid in the analysis of art history by classifying and grouping digitized paintings based on stylistic features automatically learned without prior knowledge. 6,776 digitized paintings from 8 different artistic styles (Art Nouveau, Baroque, Expressionism, Impressionism, Realism, Romanticism, Renaissance, and Post-Impressionism) were utilized to classify (predict) and cluster (group) paintings according to style. The method of unsupervised feature learning with K-means (UFLK), inspired by deep learning, was utilized to extract features from the paintings. These features were then used in: (1) a support vector machine algorithm to classify the style of new test paintings based on a training set of paintings having known style labels; and (2) a spectral clustering algorithm to group the paintings into distinct style groups (anonymously, without employing any known style labels). Classification performance was determined by accuracy and F-score. Clustering performance was determined by: 1) the ability to recover the original stylistic groupings (using a cost analysis of all possible combinations of 8 group label assignments); 2) F-score; and 3) a reliability analysis. The latter analysis used two novel ways to determine the distribution of the null-hypothesis: (a) a uniform distribution projected onto the principal components of the original data; and (b) a randomized, weighted adjacency matrix. The ability to gain insights into art was tested by a semantic analysis of the clustering results. For this purpose, we represented the featural characteristics of each painting by an N-dimensional feature vector, and plotted the distance between vector endpoints (i.e., similarity between paintings). Then, we color-coded the endpoints with the assigned lowest-cost style labels. The scatterplot was visually inspected for separation of the paintings, where the amount of separation between color clusters provides semantic information on the interrelatedness between styles. The UFLK-extracted features resembled the edges/lines/colors in the paintings. For feature-based classification of paintings, the macro-averaged F-score was 0.469. Classification accuracy and F-score were similar/higher compared to other classification methods using more complex feature learning models (e.g., convolutional neural networks, a supervised algorithm). The clustering via UFLK-extracted features yielded 8 unlabeled style groupings. In 6 of 8 clusters, the most common true painting style matched the cluster style assigned by cost analysis. The clustering had an F-score of 0.212. (There are no comparison methods for clustering paintings.) For the semantic analysis, the featural characteristics of Baroque and Art Nouveau were found to be similar, indicating a relationship between these styles. The UFLK method can extract features from digitized paintings. We were able to extract characteristics of art without any prior information about the nature of the features or the stylistic designation of the paintings. The methods herein may provide art researchers with the latest computational techniques for the documentation, interpretation, and forensics of art. The tools could assist the preservation of culturally sensitive works of art for future generations, and provide new insights into works of art and the artists who created them.
登录
查看更多内容
影响因子：
32.8
作者：
Bengio, Yoshua
通讯作者：
Bengio, Yoshua
影响因子：
4.6
作者：
GARDNER, H
通讯作者：
GARDNER, H
影响因子：
2.9
作者：
Ciresan, Dan Claudiu;Meier, Ueli;Schmidhuber, Juergen
通讯作者：
Schmidhuber, Juergen
影响因子：
3.4
作者：
Halkidi, M;Batistakis, Y;Vazirgiannis, M
通讯作者：
Vazirgiannis, M
影响因子：
5.1
作者：
Berezhnoy, Igor;Postma, Eric;van den Herik, Jaap
通讯作者：
van den Herik, Jaap