Machine Learning Methods for Malware Detection

恶意软件检测是一个重要的问题，因为它可能造成严重的损害和经济损失。这一威胁继续增加，呈指数级增长。尽管使用机器学习算法的解决方案可以解决这个问题，但仍有几个关键问题尚未解决，从而影响了其在现实世界环境中的应用。目前，利用机器学习进行恶意软件检测的关键空白包括处理恶意软件案例的稀有性和时间趋势。为能够处理这些漏洞的恶意软件检测提供健壮的解决方案是极其重要的。该研究计划的长期目标是开发特殊用途，可靠的机器学习算法，用于检测包含相关领域知识的恶意软件。为了实现这一目标，我们定义了以下短期目标：(i)开发一个用于恶意软件检测的特征工程和选择框架；（ii）开发新的专用预处理方法来处理该域的典型不平衡；（iii）开发新的恶意软件检测算法，该算法能够考虑到恶意软件事件的时间演变性质及其稀有性；（iv）开发集成主动学习和预处理方法的解决方案，用于恶意软件检测。机器学习算法使用的特征必须是信息性的和多样化的，以捕获数据的相关特征。我的目标是定义一个能够动态提取和选择特征的框架，使其能够适应恶意软件攻击中发生的不同场景和时间变化。我们的研究假设是，该框架将有助于以更有效和健壮的方式检测不同的恶意软件类型。恶意软件案例通常很少，这给机器学习算法带来了额外的挑战。我们将通过探索在处理恶意软件检测问题中发生的自然不平衡时提供高性能增益的预处理方法来解决这个问题。我们将以最有趣的方法为基础，开发针对这一特定领域特征量身定制的新型专用预处理策略。恶意软件攻击的时间演化是一个关键问题。由于新开发的恶意软件更难检测，我们将开发考虑恶意软件时间趋势的方法，以便在现实场景中有效。手动标记可疑恶意软件案例的过程非常耗时。为此，我们将探索特殊用途的主动学习解决方案，从大量数据中仔细选择有代表性的案例，同时考虑到恶意软件的稀有性。该研究计划的意义源于开发有效的恶意软件检测解决方案的重要性，该解决方案将通过探索关键和未解决问题的机器学习解决方案来推进最先进的恶意软件检测。

Malware detection is an important issue due to its potential of causing severe damage and financial losses. This threat continues to increase, exhibiting an exponential growth. Although solutions using machine learning algorithms exist to address this problem, several critical issues remain unsolved, compromising its application in a real world environment. Currently, critical gaps in malware detection using machine learning include dealing with the rarity and temporal trend of malware cases. Providing robust solutions for malware detection that are able to deal with these gaps is of extreme importance. The long term goal of this research program is to develop special purpose, reliable machine learning algorithms for detecting malware that incorporate relevant domain knowledge. To achieve this goal we defined the following short term goals: (i) develop a framework for feature engineering and selection for malware detection; (ii) develop new special-purpose pre-processing methods for dealing with the typical imbalance of this domain; (iii) develop novel algorithms for malware detection that are capable of taking into account the time-evolving nature of the malware events and their rarity; and (iv) develop solutions that integrate active learning and pre-processing methods for malware detection. The features used by machine learning algorithms must be informative and diversified to capture relevant characteristics of the data. My goal is to define a framework that enables the dynamic extraction and selection of features, enabling its adaptation to different scenarios and time changes occurring in malware attacks. Our research hypothesis is that this framework will help to detect different malware types in a more efficient and robust way. Malware cases are typically rare which creates an additional challenge for machine learning algorithms. We will address this issue by exploring pre-processing methods that provide a high performance gain when tackling the natural imbalance occurring in malware detection problems. We will build upon the most interesting methods to develop novel special-purpose pre-processing strategies specially tailored for the characteristics of this specific domain. The time evolution of malware attacks is a critical issue. Because newly developed malware is more difficult to detect, we will develop methods that take into account the malware's temporal trend in order to be effective in real-world scenarios. The process of manually labeling suspicious malware cases is time consuming. To this end, we will explore special purpose active learning solutions that carefully select representative cases from large collections of data while taking into account the malware rarity. The significance of the research program results from the importance of developing efficient solutions for malware detection that will advance the state-of-the-art on malware detection by exploring machine learning solutions for critical and unsolved issues.