CICI: RSSD:Massive Internal System Traffic Research Analysis and Logging

该项目创建了一个供网络安全研究人员和网络运营商使用的数据集(西北风数据集)，用于识别威胁，从而更好地保护与研究相关的资源。数据集中包含的数据来源反映了若干科学应用程序及其相关网络基础设施之间的实际网络活动。这些数据被安全地捕获、安全地存储，并通过相关网络安全研究人员的授权访问进行访问，以便发现可能对已确定的科学应用程序和网络基础设施构成威胁的异常或恶意活动。由于数据是通过自动化手段连续收集的，因此西北风数据集提供了一段时间内威胁的真实和相关的特征。该项目还制作了数据集的公共版本。Mistral项目包括基础设施、数据集和一组概念验证分析工作。该基础设施包括一个数据存储管道，用于处理存储在本地和/或云中的估计为1TB/天的数据，一个参考监测框架，以及用于收集、分析和共享数据和相关元数据的工具，这些数据和相关元数据是南北(面向互联网)和东西(横向)数据流的特征。数据集包括使用生产网络流(例如，源/目标IP、端口、协议、日期/时间、连接数量和大小)以及数据中心和研究实验室安全捕获的域科学工作流行为，以及来自DNS、身份验证日志、入侵检测警报和其他安全事件警报的补充数据(例如，详细说明危害指标的威胁情报数据)。最初的概念验证分析包括各种研究人员和学生(研究生和本科课程项目)的数据分析工作，以设计检测异常或恶意活动或研究该活动的技术；这些合作者还测试西北区环境和数据集，以建议基础设施和数据收集过程的改进。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

This project creates a dataset (the MISTRAL Dataset) for cybersecurity researchers and network operators to use in identifying threats and thereby better protect research-related resources. The sources of data contained in the Dataset reflect actual network activity to and from several scientific applications and their related cyberinfrastructure. These data are safely captured, securely stored and accessible through authorized access to associated cybersecurity researchers for in the purpose of detecting abnormal or malicious activities that could represent threats to the identified science applications and cyberinfrastructure. Because the data are collected continuously and through automated means, the MISTRAL Dataset provides a realistic and relevant characterization of threats over time. The project also produces a public version of the Dataset.The MISTRAL project encompasses an Infrastructure, the Dataset and a set of proof-of-concept analytic endeavors. The Infrastructure includes a data storage pipeline for handling an estimated 1TB/day of data stored on-premises and/or in the cloud, a reference monitoring framework, and tools for collecting, analyzing, and sharing the data and relevant metadata that characterize both north-south (Internet-facing) and east-west (lateral) data flows. The Dataset consists of safely captured domain science workflow behavior using production network flows (e.g., source/destination IP, port, protocol, date/time, number, and size of connections) and data centers and research labs, as well as supplemental data from DNS, authentication logs, intrusion detection alerts and other security event alerts (e.g., threat intelligence data detailing Indicators of Compromise). The initial proof-of-concept analytics comprise various researcher and student (graduate and undergraduate course project) data analysis efforts to devise techniques for detecting abnormal or malicious activity or to study that activity; these collaborators also test the MISTRAL environment and Dataset to recommend refinement of the Infrastructure and the data collection process.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.