SaTC: CORE: Small: SOFIA: Finding and profiling malware source-code in public archives at scale

该项目开发了从公共软件档案中识别和分析恶意软件源代码的方法和工具。该项目的动机是以下见解：软件存档，如GitHub，托管了大量可公开访问的恶意软件存储库，其中“恶意软件存储库”是指提供用于编译工作恶意软件二进制文件的源代码的存储库（实际上是恶意软件作者的软件项目）。这构成了一个巨大的错失机会：仅GitHub就有超过3200万个公共存储库，还有许多类似的软件平台。该项目的创新之处是：（a）有效地和大规模地提取和剖析恶意软件源代码的新颖方法，以及（B）最大的注释恶意软件源代码数据库。最终，该项目为打击恶意软件提供了一个关键的构建块：安全研究可以从目前不可用的广泛的恶意软件源代码数据库中受益匪浅。该项目更广泛的意义和重要性在于它可以帮助建立一个针对恶意软件的预防系统。这一功能可以在黑客生态系统中早期检测恶意软件及其创建，从而提前（可能在感染或攻击发生之前）提供对黑客活动的重要见解。从技术角度来看，该项目开发了以下方法：（a）系统地挖掘这些存储库，（B）研究和分析恶意软件和相关的黑客生态系统。初步研究已经确定了7000多个GitHub恶意软件源代码存储库和许多高度协作的社区，其中有数百名恶意软件作者。一个关键的新奇是，知识库是用一套全面的功能沿着三个维度来描述的：（a）元数据，如标题和描述，（B）源代码及其结构，（c）社会背景，它捕捉作者和知识库之间的互动。第二个关键的新奇是算法，因为该项目开发了新的方法，并评估和适应：（a）现有技术的数据挖掘技术，例如词嵌入，（B）特定于代码的剖析技术，以及（c）用于描述作者的交互的技术，该奖项反映了NSF的法定使命，并通过评估被认为值得支持使用基金会的知识价值和更广泛的影响审查标准。

This project develops methods and tools for identifying and profiling malware source code from public software archives. The project is motivated by the following insight: software archives, like GitHub, host a surprisingly large number of publicly-accessible malware repositories, where a "malware repository" refers to repositories that provide the source-code for compiling a working malware binary (effectively the malware author's software project). This constitutes a huge missed opportunity: GitHub alone has more than 32 million public repositories and there are many similar software platforms. The project’s novelties are: (a) novel methods to extract and profile malware source code effectively and at scale, and (b) the largest annotated malware source code database. Ultimately, the project provides a key building block for fighting malware: security research could greatly benefit from an extensive database of malware source code, which is currently unavailable. The project's broader significance and importance are it can help build a preventive system against malware. This capability enables an early detector of malware and its creation within the hacker ecosystem, which can provide critical insights into the activities of the hackers in advance, and possibly before an infection or an attack takes place.From a technical point of view, the project develops methods to: (a) systematically mine these repositories, and (b) study and profile the malware and the related hacking ecosystem. Preliminary research has identified more than 7000 GitHub malware source code repositories and many highly-collaborative communities with hundreds of malware authors. A key novelty is that repositories are described with a comprehensive set of features along three dimensions: (a) metadata, such as title and description, (b) the source code and its structure, and (c) the social context, which captures the interactions among authors and repositories. The second key novelty is algorithmic, as the project develops new approaches and also evaluates and adapts: (a) state of the art data-mining techniques, such as word embedding, (b) code-specific profiling techniques, and (c) techniques for the describing the interactions of authors, such as a novel recursive tensor decomposition.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.