Mining Software Repositories to Support the Development and Operation of Ultra Large Scale Software Systems

挖掘软件库(MSR)领域探索从静态记录保存软件库(如源代码控制和错误跟踪系统)中恢复有用信息的创新方法。恢复的信息用于获得基于经验的对软件开发实践的理解。这些知识可以帮助实践者预测、计划和理解项目的各个方面。在过去的十年里，国际MSR界探索了创新的方法，从这种静态的记录保存库中恢复有用和可操作的信息。现在，微软、黑莓、AT&T Labs、思科、Mozilla和Avaya等世界上最大和最成功的组织都将MSR技术集成到开发人员工具包中。作为MSR领域的创始人之一，我相信只有将MSR的结果和技术超越软件开发挑战，才能充分发挥MSR的潜力。我的最终目标是将MSR确立为贯穿软件系统生命周期的战略决策工具。然而，今天的研究重点仍然是传统的软件开发，在传统的软件开发和操作之间存在明显的分离，开发周期持续数月或数年。这种分离是对超大规模(ULS)软件服务的关键挑战，在ULS服务中，快速而频繁的发布是常态。显然有必要利用软件服务的开发和运作方面的知识。这种知识集成带来的两个好处的例子是：i)开发人员可以使用操作知识(例如，日志)来更好地了解他们的代码在现场是如何使用的，以便他们可以对其进行优化，并更有效地修复错误或确定修复错误的优先顺序；以及ii)操作员可以利用源代码文档来快速了解、分类和解决现场问题。拟议计划的长期愿景是系统地发现潜力，并经验地证明MSR在显著改进当今长期存在的快速发展的软件的可靠性和成本效益方面的好处。未来五年的短期目标将集中在与黑莓、SAP和Avaya等行业合作伙伴合作实现这一愿景。在接下来的五年里，研究计划将沿着以下两个主题前进：(1)以简单的格式恢复、表示和链接开发和运营知识，以便于获取这些知识；(2)开发方法、工具和途径，探索和展示MSR在理解和支持软件开发和运营活动方面的好处。特别是，拟议的研究计划将侧重于使用系统日志作为弥合软件开发和运营之间差距的机制。日志是由开发人员在源代码中插入的语句生成的消息，操作员经常使用日志来监控系统的现场操作。然而，由于日志的非结构化、规模性和使用自组织的日志分析技术，其丰富的知识还没有得到充分的利用，提出的研究计划将培养5名高素质人才：3名博士和2名硕士。所有HQP将与加拿大公司密切合作，了解从业者每天面临的挑战，以及软件数据在应对这些挑战方面的潜力。

The Mining Software Repositories (MSR) field explores innovative approaches to recover useful information from static record-keeping software repositories like source control and bug tracking systems. The recovered information is used to gain an empirically based understanding of software development practices. This knowledge helps practitioners predict, plan and understand various aspects of their project. Over the past decade, the international MSR community has explored innovative approaches to recover useful and actionable information from such static record-keeping repositories. MSR techniques are now integrated into developer toolkits at some of the world's largest and most successful organizations: Microsoft, BlackBerry, AT&T Labs, CISCO, Mozilla and Avaya.As one of the founders of the MSR field, I believe that the full potential of MSR can only be achieved by taking MSR results and techniques beyond software development challenges. My ultimate goal is to establish MSR as a strategic decision-making instrument throughout the lifetime of software systems. However, the focus of research today remains on traditional software development where a clear separation exists between the development of the software and its operation, with development cycles lasting months or years. This separation is a key challenge to Ultra Large Scale (ULS) software services where fast and frequent releases are the norm. There is a clear need to leverage knowledge from both the development and the operation of software services. Two examples of benefits arising from this knowledge integration are: i) developers can use operation knowledge (e.g., logs) to better understand how their code is used in the field so they can optimize it, and fix or prioritize the fixing of bugs more effectively; and ii) operators can make use of source code documentation to rapidly understand, triage and resolve field problems.The long-term vision of the proposed program is to systematically uncover the potential and to empirically demonstrate the benefits of MSR in delivering significant improvements in the reliability and cost-effectiveness of today's long-lived rapidly-evolving software. Short term goals over the next five years will focus on realizing this vision in collaboration with industry partners like BlackBerry, SAP and Avaya. Over the next five years the research program will progress along the following two themes: (1) the recovery, representation and linking of development and operation knowledge in a simple format in order to ease access to such knowledge; and (2) the development of methodologies, tools and approaches that explore and demonstrate the benefits of MSR in understanding and supporting software development and operation activities.In particular, the proposed research program will focus on the use of system logs as a mechanism to bridge the gap between software development and operation. Logs are messages generated from statements inserted by developers in the source code and are often used by operators for monitoring the field operation of a system. However, the rich knowledge in logs has not yet been fully-used because of their non-structured nature, their large scale, and the use of ad-hoc log analysis techniques.The research program proposed will train five highly qualified personnel (HQP): 3 PhD and 2 Master's students. All HQP will collaborate closely with Canadian companies, gaining an appreciation of the challenges facing practitioners on a daily basis and the potential of software data in addressing those challenges.