Machine Learning Methods for Malware Detection

恶意软件检测是一个重要的问题，因为它可能造成严重的破坏和经济损失。这一威胁继续增加，呈现指数增长。尽管已经有使用机器学习算法的解决方案来解决这个问题，但仍有几个关键问题尚未解决，这影响了它在现实世界环境中的应用。目前，在使用机器学习检测恶意软件方面的关键差距包括处理恶意软件案例的稀有性和时间趋势。提供能够处理这些漏洞的强大的恶意软件检测解决方案是极其重要的。该研究计划的长期目标是开发专用的、可靠的机器学习算法，用于检测包含相关领域知识的恶意软件。为了实现这一目标，我们定义了以下短期目标：(I)开发用于恶意软件检测的特征工程和选择的框架；(Ii)开发用于处理该领域典型的不平衡的新的专用预处理方法；(Iii)开发能够考虑到恶意软件事件及其稀有性随时间演变的新的恶意软件检测算法；以及(Iv)开发集成了主动学习和恶意软件检测的预处理方法的解决方案。机器学习算法使用的特征必须是信息性的和多样化的，以捕获数据的相关特征。我的目标是定义一个框架，使其能够动态提取和选择特征，使其能够适应恶意软件攻击中发生的不同场景和时间变化。我们的研究假设是，该框架将有助于以更高效和更健壮的方式检测不同类型的恶意软件。恶意软件案例通常很少，这给机器学习算法带来了额外的挑战。我们将通过探索在解决恶意软件检测问题中出现的自然不平衡时提供高性能收益的预处理方法来解决这个问题。我们将以最有趣的方法为基础，开发专门为这一特定领域的特征量身定做的新颖的特殊用途的预处理策略。恶意软件攻击的时间演变是一个关键问题。由于新开发的恶意软件更难检测，我们将开发考虑恶意软件的时间趋势的方法，以便在真实场景中有效。手动标记可疑恶意软件案例的过程非常耗时。为此，我们将探索特殊目的的主动学习解决方案，从大量数据中仔细选择具有代表性的案例，同时考虑到恶意软件的稀有性。研究计划的意义在于开发有效的恶意软件检测解决方案的重要性，该解决方案将通过探索针对关键和未解决问题的机器学习解决方案来推动恶意软件检测的最先进水平。

Malware detection is an important issue due to its potential of causing severe damage and financial losses. This threat continues to increase, exhibiting an exponential growth. Although solutions using machine learning algorithms exist to address this problem, several critical issues remain unsolved, compromising its application in a real world environment. Currently, critical gaps in malware detection using machine learning include dealing with the rarity and temporal trend of malware cases. Providing robust solutions for malware detection that are able to deal with these gaps is of extreme importance. The long term goal of this research program is to develop special purpose, reliable machine learning algorithms for detecting malware that incorporate relevant domain knowledge. To achieve this goal we defined the following short term goals: (i) develop a framework for feature engineering and selection for malware detection; (ii) develop new special-purpose pre-processing methods for dealing with the typical imbalance of this domain; (iii) develop novel algorithms for malware detection that are capable of taking into account the time-evolving nature of the malware events and their rarity; and (iv) develop solutions that integrate active learning and pre-processing methods for malware detection. The features used by machine learning algorithms must be informative and diversified to capture relevant characteristics of the data. My goal is to define a framework that enables the dynamic extraction and selection of features, enabling its adaptation to different scenarios and time changes occurring in malware attacks. Our research hypothesis is that this framework will help to detect different malware types in a more efficient and robust way. Malware cases are typically rare which creates an additional challenge for machine learning algorithms. We will address this issue by exploring pre-processing methods that provide a high performance gain when tackling the natural imbalance occurring in malware detection problems. We will build upon the most interesting methods to develop novel special-purpose pre-processing strategies specially tailored for the characteristics of this specific domain. The time evolution of malware attacks is a critical issue. Because newly developed malware is more difficult to detect, we will develop methods that take into account the malware's temporal trend in order to be effective in real-world scenarios. The process of manually labeling suspicious malware cases is time consuming. To this end, we will explore special purpose active learning solutions that carefully select representative cases from large collections of data while taking into account the malware rarity. The significance of the research program results from the importance of developing efficient solutions for malware detection that will advance the state-of-the-art on malware detection by exploring machine learning solutions for critical and unsolved issues.