A frame work for virtual integration of distributed heterogeneous data of breeding/crop experiments by Data Grid technology

通过育种和作物试验积累了大量的数据集。这些数据集是异质的，并且分散在不同的实验站之间，很少合并以进行整体研究，导致有价值的数据死灰复燃。这项研究的目的是开发一个框架，它可以为我们提供一个虚拟集成这些分布的异质数据集的环境，从而实现对完全合并的数据集的分析。作物数据库合并分布式和异质数据集：通过分析在多个地点通过育种和作物试验记录的几种类型的数据集，我们定义了标准的作物数据格式。用户上传的数据集将根据吸收异质性的样式表定义自动转换为标准格式，从而将这些数据统一存储在作物数据库中。用户只需将本地定义的有序数据集上传到数据库，并轻松合并来自不同位置的数据集。作物数据与气象数据的集成：利用MetBroker提供的对分布式、异构性气象数据库的一致访问，开发了作物数据与位置、周期对应气象数据的半自动集成系统。使用从几个农业试验站收集的15万多条水稻数据记录对该框架进行了测试。提供了日文和英文的用户手册。该框架使得能够对广泛合并的数据集进行作物建模和数据挖掘，集成分散的作物数据集和相应的天气数据，这在以前是不容易实现的。

Huge amount of data sets have been being accumulated through breeding and crop experiments. The data sets are heterogeneous and separated among different experimental stations and are rarely merged for total researches, causing dead storage of valuable data. This study was conducted to develop a frame work that can provide us an environment to virtually integrate such distributed heterogeneous data sets so as to realize analyses on totally merged data sets.1. Crop database to merge distributed and heterogeneous data sets : Analyzing several types of data sets recorded through breeding and crop experiments in several locations, we defined standard crop data format. Uploaded data sets from users are automatically transformed to the standard format based on style sheet definitions that absorbs heterogeneity so that these are stored in the crop database uniformly. Users can simply upload their ordinal data sets that are locally defined to the database and easily merge data sets from different locations.2. Integration of crop data and meteorological data : Utilizing MetBroker that can provide us consistent access to distributed and heterogeneous weather databases, a system to semi-automatically integrate crop data with positionally and periodically corresponding weather data was developed.3. The frame work was tested using more than 150,000 records of rice data collected from several agricultural experimental stations.4. User manuals in both Japanese and English were provided.5. The frame work makes it possible to conduct crop modeling and datamining of widely merged data sets, integrating distributed crop data sets and corresponding weather data, which was not easily realized before.