Effective and Efficient Data Analysis Techniques for Emerging Data Intensive Applications

从手持设备到传感器到监视到微阵列再到网络的广泛可用性（例如，社交媒体、社交网络、物联网、点击流等）允许电子捕获，收集和维护大量的结构化和非结构化数据，从而形成大数据存储库。示例数据源包括与经由Web访问的信息检索相关的日志和事务、与诸如点击流和零售购买的电子商务相关的日志和事务、与诸如天气和股票市场状态的持续事件相关的日志和事务、与诸如社交媒体和社交网络的社交交互相关的日志和事务、与诸如交通控制、欺诈检测和国土安全的监控系统相关的日志和事务、与诸如交通控制、欺诈检测和国土安全的监控系统相关的日志和事务。这样的数据被认为是知识发现的宝贵资源，从而导致有效的决策。然而，只有当这些大量的数据能够被有效地处理和分析时，这才能发生，而通过传统手段来处理和分析这些数据越来越成问题。有时，数据可能会出现偏差，其中数据实例的数量与大量的特征（例如具有数百个样本和数千个基因）相比很小;这可以通过数据丰富来解决。此外，真实的数据通常遭受各种其他相关的问题，这些问题需要处理降维、缺失值和噪声、有限计算能力下的可扩展性（即，使用通常可用的计算机），数据建模和集成以揭示特定的语义（例如，捕获和监视社交媒体数据中的行为和趋势以允许更好的推荐）等。这些问题不能被孤立地处理，因此需要一种能够在数据中存在这些问题时处理这些问题的任何组合的集成框架。

Recent developments in the technology from handheld devices to sensors to surveillance to microarrays to the wide availability of the Web (e.g., social media, social networking, internet of things, click stream, etc.) allow for electronically capturing, collecting and maintaining huge volumes of structured and unstructured data leading to big data repositories. Example data sources include logs and transactions related to information retrieval via Web access, to e-commerce such as click streams and retail purchases, on-going events such as weather and stock market status, social interactions such as social media and social networking, monitoring systems such as traffic control, fraud detection, and homeland security, and so on. Such data is recognized as a valuable resource for knowledge discovery leading to effective decision making. Yet this can happen only if these large volumes of data can be processed and analyzed effectively, which is increasingly problematic to do by conventional means. Sometimes the data could be skewed where the number of data instances is small compared to the huge number of features like having hundreds of samples and thousands of genes; this could be fixed by data enrichment. In addition, real data generally suffers from various other interrelated problems that require dealing with dimensionality reduction, missing values and noise, scalability under limited computing power (i.e., using commonly available computers), data modeling and integration to uncover specific semantics (e.g., capturing and monitoring behavior and trend in social media data to allow for better recommendations), etc. These problems cannot be treated in isolation and hence there is a need for an integrated framework capable of handling any combination of these problems when present in the data.