Mining Software Repositories to Support the Development and Operation of Ultra Large Scale Software Systems

点击翻译按钮获取中文摘要

The Mining Software Repositories (MSR) field explores innovative approaches to recover useful information from static record-keeping software repositories like source control and bug tracking systems. The recovered information is used to gain an empirically based understanding of software development practices. This knowledge helps practitioners predict, plan and understand various aspects of their project. Over the past decade, the international MSR community has explored innovative approaches to recover useful and actionable information from such static record-keeping repositories. MSR techniques are now integrated into developer toolkits at some of the world's largest and most successful organizations: Microsoft, BlackBerry, AT&T Labs, CISCO, Mozilla and Avaya.As one of the founders of the MSR field, I believe that the full potential of MSR can only be achieved by taking MSR results and techniques beyond software development challenges. My ultimate goal is to establish MSR as a strategic decision-making instrument throughout the lifetime of software systems. However, the focus of research today remains on traditional software development where a clear separation exists between the development of the software and its operation, with development cycles lasting months or years. This separation is a key challenge to Ultra Large Scale (ULS) software services where fast and frequent releases are the norm. There is a clear need to leverage knowledge from both the development and the operation of software services. Two examples of benefits arising from this knowledge integration are: i) developers can use operation knowledge (e.g., logs) to better understand how their code is used in the field so they can optimize it, and fix or prioritize the fixing of bugs more effectively; and ii) operators can make use of source code documentation to rapidly understand, triage and resolve field problems.The long-term vision of the proposed program is to systematically uncover the potential and to empirically demonstrate the benefits of MSR in delivering significant improvements in the reliability and cost-effectiveness of today's long-lived rapidly-evolving software. Short term goals over the next five years will focus on realizing this vision in collaboration with industry partners like BlackBerry, SAP and Avaya. Over the next five years the research program will progress along the following two themes: (1) the recovery, representation and linking of development and operation knowledge in a simple format in order to ease access to such knowledge; and (2) the development of methodologies, tools and approaches that explore and demonstrate the benefits of MSR in understanding and supporting software development and operation activities.In particular, the proposed research program will focus on the use of system logs as a mechanism to bridge the gap between software development and operation. Logs are messages generated from statements inserted by developers in the source code and are often used by operators for monitoring the field operation of a system. However, the rich knowledge in logs has not yet been fully-used because of their non-structured nature, their large scale, and the use of ad-hoc log analysis techniques.The research program proposed will train five highly qualified personnel (HQP): 3 PhD and 2 Master's students. All HQP will collaborate closely with Canadian companies, gaining an appreciation of the challenges facing practitioners on a daily basis and the potential of software data in addressing those challenges.