Systems and Methods for Crowdsourcing Domain Expertise

越来越多的软件专业人员正在开发应用程序，以支持复杂领域的领域专家的工作，如生物信息学、金融分析、医疗保健等。然而，这些软件专业人员主要受过计算机科学、工程或信息技术方面的培训，如果没有领域专家的帮助，他们可能很难理解不熟悉的复杂领域的细微差别。由于财务专家、医生、药剂师和其他领域专家已经在有限的时间表内工作，他们无法做出一对一的承诺来回答特定于领域的问题，这通常会导致软件项目延迟或失败。 拟议的工作将通过发明解决方案来减少对领域专家的一对一依赖，从而提高在复杂领域工作的软件专业人员的生产率。这项研究将通过设计和评估利用领域专家知识的新工具和方法来探索众包领域专业知识的想法。众包是指通过将任务分解成几个微任务，由在线众包工作者或志愿者社区完成的过程(例如，维基百科、雅虎答案和亚马逊的机械土耳其人是众包系统)。与在很大程度上为大众分解任务的现有服务不同，拟议的研究将为拥有深厚主题知识、专家洞察力以及在许多情况下拥有博士学位或数十年经验的领域专家创造微任务环境。此外，这项研究还将发明将这些众包服务嵌入软件专业人员使用的工具中的方法，以促进特定领域知识的上下文检索。 借鉴人机交互和软件工程领域，以及我开发众包系统的经验，拟议的研究将有三个主要目标： 1)开发将复杂的、特定于领域的问题分解为微任务的框架：我们将建立一个框架，说明如何在微任务环境中将任务分解和分配给领域专家小组。我们将调查哪些类型的专家任务适合众包，并经验地确定在众包服务中聘请领域专家所需的激励措施。 2)开发支持特定领域众包的软件服务和基础设施：我们将设计和实施利用领域专业知识的众包服务。我们将专门研究如何为任务分配开发有效的系统和算法，以及如何将这些众包服务嵌入软件专业人员使用的工具中。 3)通过实证研究验证众包技术和服务：虽然在整个研究过程中将使用迭代评估方法，但核心重点将是与软件专业人员验证拟议的系统设计。通过不同领域的实证调查，我们将能够确定长期使用众包服务如何影响软件专业人员的生产力和领域专家的工作流程。 这项研究的工具和发现将提高加拿大软件专业人员在复杂领域工作的生产率，更广泛地说，将促进众包领域专业知识的科学。除了支持软件专业人员，这项研究还将为转变研究和行业环境中其他类型的多学科合作奠定关键基础，并使加拿大成为众包领域专业知识研究的领先者。

Software professionals are increasingly developing applications to support the work of domain experts in complex domains, such as bioinformatics, financial analysis, healthcare, among others. However, these software professionals are primarily trained in computer science, engineering, or information technology and it can be challenging for them to understand the nuances of an unfamiliar complex domain without the help of a domain expert. Since financial experts, physicians, pharmacists, and other domain experts already work within constrained schedules, they are not able to make one-on-one commitments for answering domain-specific questions, often leading to delayed or failed software projects. The proposed work will improve the productivity of software professionals working in complex domains by inventing solutions that reduce the one-on-one dependency on domain experts. This research will explore the idea of crowdsourcing domain expertise by designing and evaluating new tools and methods for harnessing knowledge from domain experts. Crowdsourcing is the process of accomplishing a task by de-composing into several micro-tasks to be completed by a community of online crowd workers or volunteers (e.g., Wikipedia, Yahoo Answers, and Amazon’s Mechanical Turk are crowdsourcing systems). Unlike existing services that largely decompose tasks for the masses, the proposed research will invent micro-task environments for domain experts who have deep subject-matter knowledge, expert insights, and in many cases, doctoral degrees or decades of experience. Furthermore, this research will also invent approaches for embedding these crowdsourcing services within the tools used by software professionals to facilitate the contextual retrieval of domain-specific knowledge. Drawing upon the fields of human-computer interaction (HCI), and software engineering, and my experience in developing crowdsourcing systems, the proposed research will have three key objectives: 1) Develop a framework for decomposing complex, domain-specific problems into micro-tasks: We will establish a framework for how we can decompose and distribute tasks to groups of domain experts in a micro-task environment. We will investigate what types of expert tasks are amenable to crowdsourcing and also empirically determine the incentives necessary for engaging domain experts in a crowdsourcing service. 2) Develop software services and infrastructure to support domain-specific crowdsourcing: We will design and implement crowdsourcing services that harness domain expertise. We will specifically look at how to develop effective systems and algorithms for task distribution and how to embed these crowdsourcing services within the tools used by software professionals. 3) Validate the crowdsourcing techniques and services through empirical studies: Although an iterative evaluation approach will be used throughout this research, a core focus will be on validating the proposed system designs with software professionals. Through empirical investigations in different domains, we will be able to determine how the long-term usage of crowdsourcing services affects the productivity of software professionals and workflows of domain experts. The tools and findings from this research will improve the productivity of Canadian software professionals working in complex domains and, more broadly, advance the science of crowdsourcing domain expertise. Beyond supporting software professionals, this research will lay the critical groundwork for transforming other types of multidisciplinary collaborations in research and industry settings and make Canada a leader in research on crowdsourcing domain expertise.