Big Data platforms for science automation

点击翻译按钮获取中文摘要

The objective of my research program is to automate Big Data analyses from data processing to knowledge publication. In the next years, I will focus on the following three objectives of this immense challenge. (1) Interoperability among Big Data platforms: Big Data platforms used in science currently operate in silos, which hinders open science, reduces the overall quality of platforms by limiting competition, and creates technological dependence on particular software projects. I will design the building blocks of a decentralized network connecting platforms so that data, processing pipelines and analyses can be uniformly found, accessed and reused in various platforms. (2) Reproducibility of Big Data analyses over time and space: science is going through a severe reproducibility crisis, which, if not properly addressed, will prevent various disciplines from leveraging the tremendous wealth of data that is acquired: Big Data has to be converted into trusted knowledge. I will develop methods to identify, quantify and correct reproducibility issues, focusing on the challenges that originate in the computing infrastructure. (3) Performance optimization of Big Data computations: while they have been transformational for industry, Big Data technologies remain underused in science, due to the lack of appropriate benchmarks and performance optimization studies. I will design and create an optimized, easy-to-use Big Data processing environment for large scientific datasets. These three objectives echo the notorious V’s of Big Data: interoperability addresses data Variety, reproducibility targets Veracity, and performance provides Velocity and manages data Volume.