Enhancing Online User-Facing Systems through Dynamic Experimentation

点击翻译按钮获取中文摘要

People are continually interacting with online software systems, for a wide range of reasons. For example, people browse websites to gain information or make decisions, and apps to communicate or improve their health. Even when designers put careful thought into how to create such software, it often fails to meet user needs - any user of a website or app can attest to frustrations in software not helping them achieve their goals. A constant challenge in human-computer interaction is that it is difficult to know how effective a piece of technology is until it is deployed with real users, and data is being collected. I aim to understand: How can we build computational systems so that they can be perpetually changed and enhanced, even after deployment to users? For example, if a non-profit organization created a website describing its work and inviting donations, how could it deploy the website to the public, and then keep enhancing the information and appeals it makes to users? ******My research program will explore methods for better using randomized A/B experiments in improving the design of user-facing software, in scalable and efficient ways. I will explore how to draw on the human intelligence of users and designers to generate new versions of technology components that can be experimentally evaluated (e.g. different messages to get users to donate to a charity). I will investigate how to help people conduct randomized A/B experiments to create alternative versions of a product/interface/experience, and measure their relative efficacy. I will also investigate how to pool human intelligence in encouraging collaborative design of experiments between designers, between designers and users, and for crowdsourcing ideas and experimental conditions from users. To make such experimentation an engine for continual improvement, I will explore how best to apply algorithms that automatically discover which conditions are effective and dynamically provide better versions to future users (such as multi-armed bandits and other algorithms for trading off exploration and exploitation from reinforcement learning). This research can explore how to use experimentation to bring together work on human computation and reinforcement learning, identifying computational principles for perpetually improving real-world systems that help people.**