Search by Screenshots for Universal Article Clipping in Mobile Apps
Search by Screenshots for Universal Article Clipping in Mobile Apps
复制标题
DOI：
10.1145/3091107
复制
发表时间：
2017-06
期刊：
影响因子：
--
通讯作者：
Kazutoshi Umemoto;Ruihua Song;Jian-Yun Nie;Xing Xie;Katsumi Tanaka;Y. Rui
中科院分区：
文献类型：
--
作者：
Kazutoshi Umemoto;Ruihua Song;Jian-Yun Nie;Xing Xie;Katsumi Tanaka;Y. Rui
为了解决从各种移动的应用程序（应用程序）中剪切文章的困难，我们提出了一种新的框架，称为UniClip，它允许用户捕捉文章的屏幕保存在一个地方的整篇文章。该框架的关键任务是通过屏幕截图进行搜索，这有三个挑战：（1）如何表示屏幕截图;（2）如何制定查询以进行有效的文章检索;以及（3）如何从搜索结果中识别文章。我们通过以下方式解决这些问题：（1）将屏幕截图分割成称为块的结构单元，（2）通过考虑每个块的作用来制定有效的搜索查询，以及（3）聚合多个查询的搜索结果列表。为了提高效率，我们还扩展了我们的方法与学习排名技术，使我们可以找到所需的文章，只有一个查询。实验结果表明，我们的方法取得了很高的检索性能（F1 = 0.868），优于基线的关键字提取和分块方法。学习排名模型在没有学习的情况下将我们的方法改进了约6%。一项旨在调查UniClip可用性的用户研究显示，22名参与者中有21人因其简单性和有效性而首选我们的产品。
To address the difficulty in clipping articles from various mobile applications (apps), we propose a novel framework called UniClip, which allows a user to snap a screen of an article to save the whole article in one place. The key task of the framework is search by screenshots, which has three challenges: (1) how to represent a screenshot; (2) how to formulate queries for effective article retrieval; and (3) how to identify the article from search results. We solve these by (1) segmenting a screenshot into structural units called blocks, (2) formulating effective search queries by considering the role of each block, and (3) aggregating the search result lists of multiple queries. To improve efficiency, we also extend our approach with learning-to-rank techniques so that we can find the desired article with only one query. Experimental results show that our approach achieves high retrieval performance (F1 = 0.868), which outperforms baselines based on keyword extraction and chunking methods. Learning-to-rank models improve our approach without learning by about 6%. A user study conducted to investigate the usability of UniClip reveals that ours is preferred by 21 out of 22 participants for its simplicity and effectiveness.