CCSS: Small: Universal Feature Selection in Integrated Monitoring of Large Networks

监控和管理大型网络的主要挑战来自两个方面。首先，网络测量数据是多模态的，具有不同形式、质量、意义的数据，并且总体上比当前技术可以联合处理的维度高几个数量级。其次，这种网络监控结果通常需要用于多种目的，以回答有关网络行为的不同方面的查询，以在微观层面上了解互连组件如何交互。该项目的目标是应用一种新的几何特征选择方法，将高维网络测量数据减少到一个更易于管理的低维特征空间中，在此空间中，可以应用有效的算法来理解网络行为，识别可疑活动，并优化网络资源分配。该项目的成功可以带来网络状态的新接口，更好的网络运营以及改善网络安全的新方法。该项目的主要技术贡献是基于一种新的“通用特征选择”公式，其目标是从数据中找到对可能的查询集合具有普遍信息性的特征。一个基于几何的分析方法的开发，以定量解决这类问题。相对于传统的信息度量仅量化信息量，几何框架有助于产生一个新的概念“信息向量”，它可以用来定量地衡量一条信息的含义和与特定查询的相关性。因此，基于这种几何结构的特征选择可以将信息处理操作与这些新的信息度量联系起来，为优化提供指导，并为性能提供理论保证。因此，该项目是基于一个一般性的理论框架，连接几个不同的统计分支。网络监控问题是一个特别重要的问题，

The key challenges of monitoring and managing large networks comes from two sources. First, the network measurement data is multi-modal, with data of different forms, qualities, meanings, and overall several orders of magnitudes higher dimensionality than the current techniques can jointly processing. Second, such network monitoring results are often needed to serve multiple purposes, to answer queries about different aspects of the network behaviors, to understand at a microscopic level of how the interconnected components interact. The goal of this project is to apply a new geometric approach of feature selection to reduce the high dimensional network measurement data into a much more manageable lower dimensional feature space, where efficient algorithms can be applied to understand network behaviors, identify suspicious activities, and optimize network resource allocations. The success of this project can lead to new interfaces to network status, better network operations, as well as new approaches to improve cyber security. The main technical contribution of this project is based on a novel formulation of "universal feature selection", with the goal of finding features from the data that is universally informative to a collection of possible queries. A geometric-based analysis method is developed to give quantitative solutions to such problems. Comparing to the tradition information metrics that only quantify the volume of information, the geometric framework helps to give rise to a new notion of "information vector", which can be used to quantitatively measure the meaning of a piece of information and the relevance to a specific query. Feature selection based on such a geometric structure can thus connect the information processing operations to these new information metrics, providing guidance of optimization and theoretical guarantees of the performance. The project is thus based on a theoretic framework that is general, connecting several different branches of statistics. The network monitoring problem is a particularly