SaTC: CORE: Medium: Guarding Noisy Neighborhoods with Weak Detectors

恶意程序（“恶意软件”）是昂贵的，可以把人们的生命置于危险之中。不幸的是，自动恶意软件检测是困难的，许多自动检测系统产生大量的假警报。在大型企业中，检测器每天可能创建数百万个安全日志条目，使人工分析人员被虚假警报淹没。该项目正在开发算法和统计技术，以自动分析这些安全日志，并将人工分析师每天必须审查的数量从数百万减少到数十或数百。研究人员的关键见解是，攻击模式会导致跨时间和节点（用户或设备）的短暂相关性。该项目探索了一种假设，即编码和算法利用这种瞬态相关性可以导致多尺度警报数据的巨大维数降低，并允许对恶意软件活动进行统计上的重大洞察。该团队正在用两个想法来探索这一假设。首先，他们定义了允许过滤警报的“邻居”概念。邻居是一组节点，它们共享一个行为属性，比如在特定的时间窗口内访问过一个共同的网站或收到来自同一来源的电子邮件。因此，社区是节点的动态集合，这些节点很可能暴露于类似的攻击向量，例如，受损的web服务器或恶意网络钓鱼电子邮件。第二个想法是使用统计方法组成局部检测器，使用特征向量（FVs）导致局部检测器警报（“alert-FVs”），而不是仅对来自节点的原始警报标志进行操作。这可能允许全局检测器通过比较每个邻域的alert- fv的分布形状来更好地分离真阳性邻域和假阳性邻域。研究小组将使用来自一家大型商业企业和德克萨斯大学信息安全办公室的实时警报日志进行实验，以评估他们的技术。

Malicious programs ("malware") are expensive and can put people's lives at risk. Unfortunately, automatic malware detection is difficult and many automated detection systems produce a large number of false alarms. In large enterprises, detectors may create millions of security log entries per day, deluging the human analysts with false alarms. This project is developing algorithmic and statistical techniques to automatically analyze these security logs and reduce the number that human analysts must review from millions to only tens or hundreds per day.The researcher's key insight is that attack patterns induce transient correlations across time and nodes (users or devices). The project explores the hypothesis that encoding and algorithmically exploiting such transient correlations can lead to tremendous dimensionality reductions of the multi-scale alert data, and allow statistically significant insights into malware activity. The team is exploring this hypothesis using two ideas. First, they have defined the idea of a "neighborhood" that allows filtering of alerts. A neighborhood is a set of nodes that shares an action attribute such as having visited a common website or received emails from the same source within a specific time window. Thus, neighborhoods are dynamic collections of nodes that are likely to be exposed to a similar attack vector, e.g., a compromised web-server or a malicious phishing email. The second idea is a statistical approach for composing local detectors that uses the feature vectors (FVs) leading to local detector alerts ("alert-FVs") instead of operating only on the original alert flags from nodes. This may allow a global detector to better separate true positive neighborhoods from false positive neighborhoods by comparing the distributional shape of alert-FVs from each neighborhood. The research team will perform experiments to evaluate their techniques using production-scale, real-time alert logs from a large commercial enterprise and the University of Texas' Information Security Office.