Advanced Malware Detection Techniques based on Artificial Intelligence and Distributed Machine Learning

现有的防病毒系统依赖于基于签名、行为或沙盒的解决方案，这些解决方案在当今快速变化的互联网中是不够的。基于签名和启发式的方法对目标威胁和新的恶意软件无效。沙箱可以检测到以前未知的威胁，但不能有效地预防，因为可疑程序必须在沙箱中执行，这通常需要几分钟甚至几个小时。在这个项目中，我们的目标是研究机器学习和自动化大数据处理在恶意软件检测和分析中的应用，重点关注四个子项目：1)通过广泛的特征工程和分解机器检测Android恶意软件；2)基于图卷积网络（一种新兴的基于图的深度学习技术）的Android恶意软件检测；基于深度神经网络和遗传算法的PC恶意软件检测；4)开发用于多方模型训练的分布式算法和体系结构。第一个子项目侧重于通过反编译APK文件将Android应用程序的功能提取到稀疏数组中，然后通过使用Factorization Machine检测恶意软件。第二个子项目的目标是通过使用调用图生成应用程序的内部结构来检测Android恶意软件。然后将这些图输入到图卷积神经网络中。第三个子项目将首先致力于从PC文件中提取特征为通用格式，然后通过使用遗传算法训练和优化深度神经网络。最终子项目的核心是允许不同的反病毒客户端在不共享原始数据或遇到隐私泄露风险的情况下为训练机器学习模型做出贡献。该项目将强调开发用于恶意软件检测的可操作智能及其在现实中基于分散数据集的大规模实施。

Existing antivirus systems rely on signature-based, behavioural or sandbox-based solutions, which are insufficient in today's fast-changing Internet. Signature and heuristic based approaches are ineffective against targeted threats and new malware. Sandboxes can detect previously unknown threats, but are not effective at prevention, since the suspicious program must be executed in a sandbox, which often requires minutes or even hours. In this project, we aim to investigate the application of machine learning and automated big data processing to malware detection and analysis, focusing on four sub-projects: 1) Android Malware detection via extensive feature engineering and Factorization Machines; 2) Android Malware detection via Graph Convolutional Networks, a emerging deep learning technique executed on graphs; 3) PC Malware detection based on Deep Neural Networks and Genetic Algorithms; 4) the development of distributed algorithms and architectures for multiparty model training. The first subproject focuses on extracting Android app features into sparse arrays through the decompilation of APK files and then detecting malware via the use of a Factorization Machine. The the aim of the second subproject is to detect Android malware by generating an app's internal structure using call graphs. These graphs are then fed into Graph Convolutional Neural Networks. The third subproject will first aim to extract features from PC files into a common format and then both train and optimize Deep Neural Networks via the use of Genetic Algorithms. At its core, the final subproject is meant to allow different anti-virus clients to contribute to training machine learning models without sharing raw data or running into privacy leakage risks. The project will emphasize the development of actionable intelligence for malware detection and its large-scale implementation based on decentralized datasets in reality.