SEMAGROW - Data intensive techniques to boost the real-time performance of global agricultural data infrastructures

As the trend to open up data and provide them freely on the Internet intensifies, the opportunities to create added value by combining and cross-indexing heterogeneous data at a large scale increase. To seize these opportunities we need infrastructure that is not only efficient, real-time responsive and scalable but is also flexible and robust enough to welcome data in any schema and form and to transparently relegate and translate queries from a unifying end-point to the multitude of data services that make up the open data cloud.

Transparent relegation and translation relies on detailed and accurate data summaries and other data source annotations, and with increased data volumes and heterogeneity managing these annotations, it becomes by itself a challenging data problem. SemaGrow will (a) develop scalable and robust semantic storage and indexing algorithms that can take advantage of resource naming conventions and other natural groupings of URIs to compress data source annotations about extremely large datasets; (b) develop query decomposition, source selection, and distributed querying methods that take advantage of such algorithms to implement a scalable and robust infrastructure for data service federation; and (c) rigorously test its components and overall architecture over real, complex, interconnected datasets comprising data and document collections, sensor data, and GIS data.

SemaGrow will be rigorously tested on the large-scale and complex agricultural data service ecosystem, comprising more than 20 currently operating data services providing today Gigatriples of RDF data, projected to double before SemaGrow ends and to reach Teratriples by 2020. Being able to query across these datasets is a real and present need in order to, for example, apply financial models to predict crop yield using a combination of historical crop yield data, precipitation forecast models and historical data, live data streams of precipitation sensors, and relevant scientific documentation.

Besides satisfying its current use cases, SemaGrow envisages to develop the scalable, efficient, and robust data services needed to take full advantage of the data-intensive and inter-disciplinary Science of 2020 and to re-shape the way that data analysis techniques are applied to the heterogeneous data cloud.

Cooperating with the NSCR-D team that will serve as the Scientific & Technical Manager and lead two core WPs that will research novel intelligent information management algorithms/methods.

SemaGrow website: http://www.semagrow.eu