Earthcube Building Blocks: Collaborative Proposal: Polar Data Insights and Search Analytics for the Deep and Scientific Web

该项目开发了一个专注于极地数据科学的NSF EarthCube构建块。该系统将建立在信息检索和数据科学方面的工作，以及NSF Polar、EarthCube、DARPA和NASA在该领域的现有投资的基础上。该系统将收集、分析和交互迄今为止在科学信息深网（科学期刊、多媒体信息、科学数据、网页等）中收集的丰富的文本和科学极地数据。该系统建立在文本分析、搜索和可视化的基础研究之上。它的主要目标是从90多种数据格式中解锁非结构化科学数据，并使用NSF XSEDE超级计算资源扩展到数万-上亿条记录。该系统将对从极地深度和科学网络抓取的数据进行信息检索和机器学习。爬行将通过EarthCube和Polar社区众包的科学问题获得信息。该项目是与NSIDC、Ronin研究所和更广泛的社区合作，包括由NCEAS领导的新资助的北极数据中心，以建立我们提出的系统。定期和定期抓取的结果将是原始文本数据的抓取数据存储库（CDR），例如，包含丰富的数据集抽象描述的网页，与数据集相关的新闻故事，ASCII注释文件和数据集描述，以及极地存储库上可用或指向的其他文本数据以及科学数据（HDF, Grib, NetCDF， Matlab等）。CDR将用于更广泛的EarthCube和Polar社区的历史和未来分析。此外，提取管道将生成一个提取数据存储库（EDR），其中包含以前不存在的机器学习特征（地理空间、时间、人员、地点、科学出版物和主题等），这将成为对Polar数据资源进行交互式可视化分析的基础。所收集的信息将有助于回答诸如总统提出的这些科学问题。中国的北极地区国家战略。到目前为止，该团队还从cryyolist https://goo.gl/4dDyIS上的极地社区代表那里收集了30多个问题，并将继续征求这些反馈，并使用收集到的信息来帮助社区优先考虑的科学研究。他们还将参与社区协助验证我们的系统。这本身并不是一个预测工具？尽管它可以帮助实现这样的预测。它的重点是为文本科学数据分析建立一个可操作的核心能力，包括回顾性和前瞻性。

This project develops an NSF EarthCube Building Block focused on Polar Data Science. The system will build upon work in Information Retrieval and Data Science and upon existing investment from NSF Polar, EarthCube, and from DARPA and NASA in this area. The system will collect, analyze, and make interactive the wealth of textual and scientific Polar data collected to date across the Deep web of scientific information -- scientific journals, multimedia information, scientific data, web pages, etc. The system builds upon fundamental research in text analysis, search, and visualization. Its primary goal is to unlock unstructured scientific data from 90+ data formats and to scale to 10s-100s of millions of records using the NSF XSEDE supercomputing resources. The system will perform information retrieval and machine learning on data crawled from the Polar Deep and Scientific web. Crawling will be informed by science questions crowdsourced through the EarthCube and Polar communities. The project is a collaboration with NSIDC, Ronin Institute, and the broader community including the newly funded Arctic Data Center led by NCEAS, to build our proposed system.The result of periodic and regular crawling will be a Crawl Data Repository (CDR) of raw textual data e.g., web pages containing richly curated dataset abstract descriptions, news stories tied to datasets, ASCII note files and dataset descriptions, and other textual data available on or pointed to by Polar repositories as well as scientific data (HDF, Grib, NetCDF, Matlab, etc.). The CDR will be made available for historical and future analysis by the broader EarthCube and Polar communities. In addition, an extraction pipeline will generate an Extraction Data Repository (EDR) of machine learning features not previously present (geospatial, temporal, people, places, scientific publications and topics, etc.) that will be the basis of interactive, visual analytics over the Polar data resources. Information collected will assist in answering scientific questions such as these derived from the President?s National Strategy for the Arctic Region. To date, the team has also crowd sourced 30+ questions from the Polar community represented on CRYOLIST https://goo.gl/4dDyIS and will continue to solicit this feedback and use the information collected to aid science as prioritized by the community. They will also engage the community to assist in validating our system. This is not a predictive tool per-se ? though it can help to enable such predictions. Its focus is on building an operational and core capability for textual scientific data analysis, both retrospective, and prospective.