Information Theoretic Research on Big Data Compression and Analytics: Theory, Algorithms, and Applications

大数据压缩与分析的信息论研究：理论、算法与应用

• 批准号: RGPIN-2016-03871
• 负责人: Yang, Enhui
• 金额: $ 3.28万
• 依托单位: University of Waterloo
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2016
• 资助国家: 加拿大
• 起止时间: 2016-01-01 至 2017-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=614686
• 关键词: Information; Theoretic; Research; Big; Data

With an explosion in data sets in our society, we are at the beginning of a big data revolution. Big data has a potential to accelerate the pace of discovery in science, engineering, and medicine, improve healthcare, finance, business, and our lives, and ultimately transform our society. To tap the opportunities afforded by the big data revolution, however, many challenges of big data have to be carefully addressed. For example, to fast access remotely, and reduce the on-going cost of maintaining, huge volumes of digital data, it is desirable to compress data as much as possible. Likewise, to glean knowledge and discoveries from huge volumes of digital data, tools and techniques based on context analytics have to be developed to organize, visualize, and manage huge volumes of digital data. Efficient data compression and accurate context analytics are of vital importance to turning big data to knowledge and discoveries to actions while achieving resource (bandwidth and power) efficiency. In the past, data compression and data analytics have been largely investigated separately in different disciplines. Although data compression has been around for many years, its existing theories and techniques, especially source coding theory from information theory, have been largely developed for stationary and/or well-structured data (such as text, web pages, etc.). In the context of big data, however, most data types are nonstationary and unstructured/semi-structured; applying existing compression techniques directly to these data types often leads to unsatisfied compression performance. Therefore, it is imperative to develop novel compression theories and techniques by incorporating data analytics into data compression to handle the lack of structure and stationarity in an elegant way. In the opposite direction, it is also beneficial to investigate how to apply information theoretic ideas, particularly source coding theory and techniques, to data analytics to develop better solutions for data analytics such as cognitive distance, clustering, and organization. Building on our early success, in this research, we will investigate and explore the interactions of data compression and analytics to advance knowledge in both fields. Our approach will be information theoretic. Three theme areas will be focused on: (1) interactions of data compression and analytics to improve lossy compression performance for nonstationary and unstructured data; (2) interactions of data compression and analytics to develop better cognitive distances and hence provide better tools for data clustering and organization; and (3) compression of and pattern discovery in large bipartite graphs. In the process of achieving our scientific objectives, we will maintain and enhance our leadership in the related areas, and train highly qualified personnel in information and communications technology for Canada.

随着我们社会中数据集的爆炸式增长，我们正处于大数据革命的开端。大数据有可能加快科学、工程和医学领域的发现步伐，改善医疗保健、金融、商业和我们的生活，并最终改变我们的社会。