SaTC: CORE: Medium: Guarding Noisy Neighborhoods with Weak Detectors

恶意程序(“恶意软件”)代价高昂，可能会危及人们的生命安全。遗憾的是，恶意软件的自动检测很困难，许多自动检测系统会产生大量的错误警报。在大型企业中，检测器每天可能会创建数百万个安全日志条目，给人类分析师带来大量错误警报。该项目正在开发算法和统计技术，以自动分析这些安全日志，并将人工分析师每天必须审查的数量从数百万减少到仅数十或数百。研究人员的关键见解是，攻击模式会在时间和节点(用户或设备)之间产生瞬时关联。该项目探索了这样的假设，即编码和算法利用这种瞬时相关性可以导致多尺度警报数据的巨大降维，并允许对恶意软件活动进行统计上的重大洞察。该团队正在使用两种想法来探索这一假说。首先，他们定义了“邻居”的概念，允许过滤警报。邻居是共享动作属性的一组节点，例如在特定时间窗口内访问过公共网站或从相同来源接收到电子邮件。因此，邻居是可能暴露于类似攻击媒介的节点的动态集合，例如，受危害的网络服务器或恶意网络钓鱼电子邮件。第二种思想是一种用于组成局部检测器的统计方法，其使用导致局部检测器告警的特征向量(FV)，而不是仅对来自节点的原始告警标志进行操作。这可能允许全局检测器通过比较来自每个邻区的警报FV的分布形状来更好地区分真阳性邻区和假阳性邻区。研究小组将进行实验，使用来自一家大型商业企业和德克萨斯大学信息安全办公室的生产规模实时警报日志来评估他们的技术。

Malicious programs ("malware") are expensive and can put people's lives at risk. Unfortunately, automatic malware detection is difficult and many automated detection systems produce a large number of false alarms. In large enterprises, detectors may create millions of security log entries per day, deluging the human analysts with false alarms. This project is developing algorithmic and statistical techniques to automatically analyze these security logs and reduce the number that human analysts must review from millions to only tens or hundreds per day.The researcher's key insight is that attack patterns induce transient correlations across time and nodes (users or devices). The project explores the hypothesis that encoding and algorithmically exploiting such transient correlations can lead to tremendous dimensionality reductions of the multi-scale alert data, and allow statistically significant insights into malware activity. The team is exploring this hypothesis using two ideas. First, they have defined the idea of a "neighborhood" that allows filtering of alerts. A neighborhood is a set of nodes that shares an action attribute such as having visited a common website or received emails from the same source within a specific time window. Thus, neighborhoods are dynamic collections of nodes that are likely to be exposed to a similar attack vector, e.g., a compromised web-server or a malicious phishing email. The second idea is a statistical approach for composing local detectors that uses the feature vectors (FVs) leading to local detector alerts ("alert-FVs") instead of operating only on the original alert flags from nodes. This may allow a global detector to better separate true positive neighborhoods from false positive neighborhoods by comparing the distributional shape of alert-FVs from each neighborhood. The research team will perform experiments to evaluate their techniques using production-scale, real-time alert logs from a large commercial enterprise and the University of Texas' Information Security Office.