Systems and Methods for Crowdsourcing Domain Expertise

点击翻译按钮获取中文摘要

Software professionals are increasingly developing applications to support the work of domain experts in complex domains, such as bioinformatics, financial analysis, healthcare, among others. However, these software professionals are primarily trained in computer science, engineering, or information technology and it can be challenging for them to understand the nuances of an unfamiliar complex domain without the help of a domain expert. Since financial experts, physicians, pharmacists, and other domain experts already work within constrained schedules, they are not able to make one-on-one commitments for answering domain-specific questions, often leading to delayed or failed software projects. The proposed work will improve the productivity of software professionals working in complex domains by inventing solutions that reduce the one-on-one dependency on domain experts. This research will explore the idea of crowdsourcing domain expertise by designing and evaluating new tools and methods for harnessing knowledge from domain experts. Crowdsourcing is the process of accomplishing a task by de-composing into several micro-tasks to be completed by a community of online crowd workers or volunteers (e.g., Wikipedia, Yahoo Answers, and Amazon’s Mechanical Turk are crowdsourcing systems). Unlike existing services that largely decompose tasks for the masses, the proposed research will invent micro-task environments for domain experts who have deep subject-matter knowledge, expert insights, and in many cases, doctoral degrees or decades of experience. Furthermore, this research will also invent approaches for embedding these crowdsourcing services within the tools used by software professionals to facilitate the contextual retrieval of domain-specific knowledge. Drawing upon the fields of human-computer interaction (HCI), and software engineering, and my experience in developing crowdsourcing systems, the proposed research will have three key objectives: 1) Develop a framework for decomposing complex, domain-specific problems into micro-tasks: We will establish a framework for how we can decompose and distribute tasks to groups of domain experts in a micro-task environment. We will investigate what types of expert tasks are amenable to crowdsourcing and also empirically determine the incentives necessary for engaging domain experts in a crowdsourcing service. 2) Develop software services and infrastructure to support domain-specific crowdsourcing: We will design and implement crowdsourcing services that harness domain expertise. We will specifically look at how to develop effective systems and algorithms for task distribution and how to embed these crowdsourcing services within the tools used by software professionals. 3) Validate the crowdsourcing techniques and services through empirical studies: Although an iterative evaluation approach will be used throughout this research, a core focus will be on validating the proposed system designs with software professionals. Through empirical investigations in different domains, we will be able to determine how the long-term usage of crowdsourcing services affects the productivity of software professionals and workflows of domain experts. The tools and findings from this research will improve the productivity of Canadian software professionals working in complex domains and, more broadly, advance the science of crowdsourcing domain expertise. Beyond supporting software professionals, this research will lay the critical groundwork for transforming other types of multidisciplinary collaborations in research and industry settings and make Canada a leader in research on crowdsourcing domain expertise.