CICI: RSSD:Massive Internal System Traffic Research Analysis and Logging

该项目为网络安全研究人员和网络运营商创建了一个数据集（MISTRAL数据集），用于识别威胁，从而更好地保护研究相关资源。数据集中包含的数据源反映了几个科学应用及其相关网络基础设施之间的实际网络活动。这些数据被安全捕获、安全存储，并通过相关网络安全研究人员的授权访问进行访问，以检测可能对已识别的科学应用和网络基础设施构成威胁的异常或恶意活动。由于数据是通过自动化手段连续收集的，因此MISTRAL数据集提供了随时间推移的现实和相关的威胁特征。该项目还生成了数据集的公共版本。MISTRAL项目包括基础设施、数据集和一组概念验证分析工作。基础设施包括一个数据存储管道，用于处理存储在本地和/或云中估计每天1TB的数据，一个参考监控框架，以及用于收集、分析和共享数据和相关元数据的工具，这些数据和元数据具有南北（面向互联网）和东西（横向）数据流的特征。该数据集包括使用生产网络流（例如，源/目的IP，端口，协议，日期/时间，连接数量和大小）和数据中心和研究实验室安全捕获的领域科学工作流行为，以及来自DNS，身份验证日志，入侵检测警报和其他安全事件警报的补充数据（例如，详细说明妥协指标的威胁情报数据）。最初的概念验证分析包括各种研究人员和学生（研究生和本科生课程项目）的数据分析工作，以设计检测异常或恶意活动或研究该活动的技术；这些合作者还测试了MISTRAL环境和数据集，以建议改进基础设施和数据收集过程。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This project creates a dataset (the MISTRAL Dataset) for cybersecurity researchers and network operators to use in identifying threats and thereby better protect research-related resources. The sources of data contained in the Dataset reflect actual network activity to and from several scientific applications and their related cyberinfrastructure. These data are safely captured, securely stored and accessible through authorized access to associated cybersecurity researchers for in the purpose of detecting abnormal or malicious activities that could represent threats to the identified science applications and cyberinfrastructure. Because the data are collected continuously and through automated means, the MISTRAL Dataset provides a realistic and relevant characterization of threats over time. The project also produces a public version of the Dataset.The MISTRAL project encompasses an Infrastructure, the Dataset and a set of proof-of-concept analytic endeavors. The Infrastructure includes a data storage pipeline for handling an estimated 1TB/day of data stored on-premises and/or in the cloud, a reference monitoring framework, and tools for collecting, analyzing, and sharing the data and relevant metadata that characterize both north-south (Internet-facing) and east-west (lateral) data flows. The Dataset consists of safely captured domain science workflow behavior using production network flows (e.g., source/destination IP, port, protocol, date/time, number, and size of connections) and data centers and research labs, as well as supplemental data from DNS, authentication logs, intrusion detection alerts and other security event alerts (e.g., threat intelligence data detailing Indicators of Compromise). The initial proof-of-concept analytics comprise various researcher and student (graduate and undergraduate course project) data analysis efforts to devise techniques for detecting abnormal or malicious activity or to study that activity; these collaborators also test the MISTRAL environment and Dataset to recommend refinement of the Infrastructure and the data collection process.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.