Ousting ivory tower research: towards a web framework for providing experiments as a service
Ousting ivory tower research: towards a web framework for providing experiments as a service
复制标题
DOI：
10.1145/2348283.2348501
复制
发表时间：
2012-08
期刊：
影响因子：
--
通讯作者：
Tim Gollub;Benno Stein;Steven Burrows
中科院分区：
文献类型：
--
作者：
Tim Gollub;Benno Stein;Steven Burrows
由于与Web的紧密联系，IR社区注定要利用Web今天提供的传播和协作功能。特别是随着软件作为服务原则的出现，IR社区可以通过Web发布任何人都可以执行的实验。回顾SIGIR最近的论文，我们离这种合作的愿景还很远。将IR实验作为一种服务发布的好处对整个社区来说是惊人的，包括提高研究概况和声誉的潜力。然而，额外的工作必须保持在最低限度，敏感数据必须保持私密，以使这种模式成为一种可接受的做法。为了促进实验作为一种服务，在IR，我们提出了一个Web框架的实验，解决了所概述的挑战，并拥有一套独特的引人注目的功能相比，现有的解决方案。我们还介绍了我们的参考实现已经正式作为一个既定的国际剽窃检测竞争的评估平台。
With its close ties to the Web, the IR community is destined to leverage the dissemination and collaboration capabilities that the Web provides today. Especially with the advent of the software as a service principle, an IR community is conceivable that publishes experiments executable by anyone over the Web. A review of recent SIGIR papers shows that we are far away from this vision of collaboration. The benefits of publishing IR experiments as a service are striking for the community as a whole, and include potential to boost research profiles and reputation. However, the additional work must be kept to a minimum and sensitive data must be kept private for this paradigm to become an accepted practice. To foster experiments as a service in IR, we present a Web framework for experiments that addresses the outlined challenges and possesses a unique set of compelling features in comparison to existing solutions. We also describe how our reference implementation is already used officially as an evaluation platform for an established international plagiarism detection competition.