Mining Software Repositories to Support the Development and Operation of Ultra Large Scale Software Systems

挖掘软件存储库（MSR）领域探索从静态记录保存软件存储库（如源代码控制和bug跟踪系统）中恢复有用信息的创新方法。恢复的信息用于获得基于经验的对软件开发实践的理解。这些知识有助于从业者预测、计划和理解项目的各个方面。在过去十年中，国际MSR社区探索了从这种静态记录保存库中恢复有用和可操作信息的创新方法。MSR技术现在已经集成到一些世界上最大和最成功的组织的开发人员工具包中：微软、黑莓、AT&T实验室、思科、Mozilla和Avaya。作为MSR领域的创始人之一，我相信MSR的全部潜力只能通过超越软件开发挑战的MSR结果和技术来实现。我的最终目标是将MSR建立为贯穿软件系统整个生命周期的战略决策工具。然而，今天研究的焦点仍然是传统的软件开发，在传统的软件开发中，软件的开发和它的操作之间存在着明确的分离，开发周期持续数月或数年。这种分离是超大规模（ULS）软件服务的一个关键挑战，因为快速和频繁的发布是常态。显然需要利用来自软件服务开发和操作的知识。这种知识集成带来的好处有两个例子：i)开发人员可以使用操作知识（例如，日志）来更好地了解他们的代码在该领域是如何使用的，这样他们就可以对其进行优化，并更有效地修复或优先修复bug；ii)操作员可以利用源代码文档来快速理解、分类和解决现场问题。所提议的计划的长期愿景是系统地揭示潜力，并以经验证明MSR在交付当今长寿命快速发展的软件的可靠性和成本效益方面的重大改进方面的好处。未来五年的短期目标将集中在与黑莓、SAP和Avaya等行业合作伙伴合作实现这一愿景。在未来五年内，该研究计划将沿着以下两个主题进行：(1)以一种简单的格式恢复、表示和链接开发和运营知识，以便于获取这些知识；(2)开发方法、工具和方法，探索和证明MSR在理解和支持软件开发和操作活动方面的好处。特别地，拟议的研究计划将侧重于使用系统日志作为一种机制，以弥合软件开发和操作之间的差距。日志是由开发人员在源代码中插入的语句生成的消息，操作员经常使用它来监视系统的现场操作。然而，由于日志的非结构化性质、大规模以及使用特殊的日志分析技术，日志中的丰富知识尚未得到充分利用。本项目拟培养5名高素质人才（HQP）： 3名博士研究生和2名硕士研究生。所有HQP将与加拿大公司密切合作，了解从业者每天面临的挑战，以及软件数据在解决这些挑战方面的潜力。

The Mining Software Repositories (MSR) field explores innovative approaches to recover useful information from static record-keeping software repositories like source control and bug tracking systems. The recovered information is used to gain an empirically based understanding of software development practices. This knowledge helps practitioners predict, plan and understand various aspects of their project. Over the past decade, the international MSR community has explored innovative approaches to recover useful and actionable information from such static record-keeping repositories. MSR techniques are now integrated into developer toolkits at some of the world's largest and most successful organizations: Microsoft, BlackBerry, AT&T Labs, CISCO, Mozilla and Avaya.As one of the founders of the MSR field, I believe that the full potential of MSR can only be achieved by taking MSR results and techniques beyond software development challenges. My ultimate goal is to establish MSR as a strategic decision-making instrument throughout the lifetime of software systems. However, the focus of research today remains on traditional software development where a clear separation exists between the development of the software and its operation, with development cycles lasting months or years. This separation is a key challenge to Ultra Large Scale (ULS) software services where fast and frequent releases are the norm. There is a clear need to leverage knowledge from both the development and the operation of software services. Two examples of benefits arising from this knowledge integration are: i) developers can use operation knowledge (e.g., logs) to better understand how their code is used in the field so they can optimize it, and fix or prioritize the fixing of bugs more effectively; and ii) operators can make use of source code documentation to rapidly understand, triage and resolve field problems.The long-term vision of the proposed program is to systematically uncover the potential and to empirically demonstrate the benefits of MSR in delivering significant improvements in the reliability and cost-effectiveness of today's long-lived rapidly-evolving software. Short term goals over the next five years will focus on realizing this vision in collaboration with industry partners like BlackBerry, SAP and Avaya. Over the next five years the research program will progress along the following two themes: (1) the recovery, representation and linking of development and operation knowledge in a simple format in order to ease access to such knowledge; and (2) the development of methodologies, tools and approaches that explore and demonstrate the benefits of MSR in understanding and supporting software development and operation activities.In particular, the proposed research program will focus on the use of system logs as a mechanism to bridge the gap between software development and operation. Logs are messages generated from statements inserted by developers in the source code and are often used by operators for monitoring the field operation of a system. However, the rich knowledge in logs has not yet been fully-used because of their non-structured nature, their large scale, and the use of ad-hoc log analysis techniques.The research program proposed will train five highly qualified personnel (HQP): 3 PhD and 2 Master's students. All HQP will collaborate closely with Canadian companies, gaining an appreciation of the challenges facing practitioners on a daily basis and the potential of software data in addressing those challenges.