Earthcube Building Blocks: Collaborative Proposal: Polar Data Insights and Search Analytics for the Deep and Scientific Web

点击翻译按钮获取中文摘要

This project develops an NSF EarthCube Building Block focused on Polar Data Science. The system will build upon work in Information Retrieval and Data Science and upon existing investment from NSF Polar, EarthCube, and from DARPA and NASA in this area. The system will collect, analyze, and make interactive the wealth of textual and scientific Polar data collected to date across the Deep web of scientific information -- scientific journals, multimedia information, scientific data, web pages, etc. The system builds upon fundamental research in text analysis, search, and visualization. Its primary goal is to unlock unstructured scientific data from 90+ data formats and to scale to 10s-100s of millions of records using the NSF XSEDE supercomputing resources. The system will perform information retrieval and machine learning on data crawled from the Polar Deep and Scientific web. Crawling will be informed by science questions crowdsourced through the EarthCube and Polar communities. The project is a collaboration with NSIDC, Ronin Institute, and the broader community including the newly funded Arctic Data Center led by NCEAS, to build our proposed system.The result of periodic and regular crawling will be a Crawl Data Repository (CDR) of raw textual data e.g., web pages containing richly curated dataset abstract descriptions, news stories tied to datasets, ASCII note files and dataset descriptions, and other textual data available on or pointed to by Polar repositories as well as scientific data (HDF, Grib, NetCDF, Matlab, etc.). The CDR will be made available for historical and future analysis by the broader EarthCube and Polar communities. In addition, an extraction pipeline will generate an Extraction Data Repository (EDR) of machine learning features not previously present (geospatial, temporal, people, places, scientific publications and topics, etc.) that will be the basis of interactive, visual analytics over the Polar data resources. Information collected will assist in answering scientific questions such as these derived from the President?s National Strategy for the Arctic Region. To date, the team has also crowd sourced 30+ questions from the Polar community represented on CRYOLIST https://goo.gl/4dDyIS and will continue to solicit this feedback and use the information collected to aid science as prioritized by the community. They will also engage the community to assist in validating our system. This is not a predictive tool per-se ? though it can help to enable such predictions. Its focus is on building an operational and core capability for textual scientific data analysis, both retrospective, and prospective.