Effective and Efficient Data Analysis Techniques for Emerging Data Intensive Applications

点击翻译按钮获取中文摘要

Recent developments in the technology from handheld devices to sensors to surveillance to microarrays to the wide availability of the Web (e.g., social media, social networking, internet of things, click stream, etc.) allow for electronically capturing, collecting and maintaining huge volumes of structured and unstructured data leading to big data repositories. Example data sources include logs and transactions related to information retrieval via Web access, to e-commerce such as click streams and retail purchases, on-going events such as weather and stock market status, social interactions such as social media and social networking, monitoring systems such as traffic control, fraud detection, and homeland security, and so on. Such data is recognized as a valuable resource for knowledge discovery leading to effective decision making. Yet this can happen only if these large volumes of data can be processed and analyzed effectively, which is increasingly problematic to do by conventional means. Sometimes the data could be skewed where the number of data instances is small compared to the huge number of features like having hundreds of samples and thousands of genes; this could be fixed by data enrichment. In addition, real data generally suffers from various other interrelated problems that require dealing with dimensionality reduction, missing values and noise, scalability under limited computing power (i.e., using commonly available computers), data modeling and integration to uncover specific semantics (e.g., capturing and monitoring behavior and trend in social media data to allow for better recommendations), etc. These problems cannot be treated in isolation and hence there is a need for an integrated framework capable of handling any combination of these problems when present in the data.