Analysis and classification of data sets for calibration and validation of agro-ecosystem models

Highlights

• A software is presented to classify and label data suitability for modelling.

• Data requirements for modelling are specific and vary with model purpose.

• Quantitative classification of data sets facilitates their use for modelling.

• Test of model and data consistency improves data usability.

• Guidelines for experimentalist to improve data suitability for modelling.

Abstract

Experimental field data are used at different levels of complexity to calibrate, validate and improve agro-ecosystem models to enhance their reliability for regional impact assessment. A methodological framework and software are presented to evaluate and classify data sets into four classes regarding their suitability for different modelling purposes. Weighting of inputs and variables for testing was set from the aspect of crop modelling. The software allows users to adjust weights according to their specific requirements. Background information is given for the variables with respect to their relevance for modelling and possible uncertainties. Examples are given for data sets of the different classes. The framework helps to assemble high quality data bases, to select data from data bases according to modellers requirements and gives guidelines to experimentalists for experimental design and decide on the most effective measurements to improve the usefulness of their data for modelling, statistical analysis and data assimilation.

Introduction

Soil–crop–atmosphere interactions play a central role in the multiple functions of agro-ecosystems and rural landscapes such as food and energy production, carbon sequestration, soil properties, biodiversity or conservation of water resources. Emissions from agriculture are also seen as a threat to the global climate system, which makes agriculture one of the key handles for climate change mitigation. There is an increasing need to better understand these complex systems, and to develop and utilize reliable process-based models for scenario analyses as a basis for policy and management decisions. Agro-ecosystem models are increasingly applied beyond the point and field scales to support decision-making (Van Ittersum et al., 2003, Jones et al., 2003, Brisson et al., 2003, Stockle et al., 2003, Keating et al., 2003), assess the impact of climate change (Holzworth et al., 2015, position paper of thematic issue), and to derive adaptation and mitigation strategies for the sustainable use and management of land and other natural resources (Hammer et al., 2002, White et al., 2011). Integrated Assessment and Modelling as suggested by Parker et al. (2002) requires the integration of dispersed data sources in a consistent and spatially and temporarily complete data set to provide necessary model inputs for decision making (Janssen et al., 2009) and to transfer site-based knowledge to regions and continents. With increasing size of the area under investigation, input data tend to become more uncertain relative to the point data of experimental sites, which were the original basis of development for the majority of agro-ecosystem models. Hence, model uncertainty also increases with the area under investigation since data of relevant state variables for testing and evaluation are not commonly available.

Critical to the evaluation, improvement, and use of crop models is the availability of high quality data from field observations. There is a mismatch between the rising demand by users for tested models and research budgets for suitable experimental research and monitoring, which tend to be decreasing (Rötter et al., 2011).

Since field experimental data sets are usually not recorded for modelling purposes, their level of detail, quality of records, variables considered as well as their number of spatial and temporal replicates vary enormously (Nix, 1985, Groot and Verberne, 1991) Therefore, their suitability for modelling is often insufficient for different reasons. White et al. (2013) proposed a standard approach for describing and identifying variables of management, environmental conditions, soils, and crop measurements, all for the purpose of developing, testing, and applying crop simulation models. In general, datasets used for model calibration and validation consist of data describing a) the initial soil conditions, b) the crop-specific management and c) the seasonal weather conditions (Palosuo et al., 2011, Rötter et al., 2012, White et al., 2013). Additionally, data on phenology of the crop, yields and nutrient contents from intermediate harvests, intra-seasonal soil conditions and measurements of fluxes of energy, water and CO₂ may be provided.

The international community of agricultural system modellers, e.g., in the Agricultural Model Intercomparison and Improvement Project AgMIP (Rosenzweig et al., 2013) or the European MACSUR (Modelling European Agriculture with Climate Change for Food SecURity) project (Rötter et al., 2013) are currently building harmonized data bases for the purpose of model testing and improvement including the opportunity to create model-specific interfaces for various models (Porter et al., 2014). In order to find suitable experimental data for specific applications in the context of modelling out of the vast offer of available data sets, a transparent method of screening and pre-selection is demanded, which highlights specific positive and negative features of a data set with respect to the intended application. To evaluate and select data sets, Rosenzweig et al. (2013) proposed different classes of data for so-called “Sentinel Sites”, which represent specific sites with experimental data suitable for different levels of model testing and improvement. However, specific for a transparent classification were not provided. A joint community effort lead to the development of a qualitative (Boote et al., 2015) and a quantitative framework (this publication) to evaluate the quality of field experimental data sets for crop modelling according to robust and accepted criteria.

The aim of this paper is to provide a quantitative classification framework by which the consistency and quality of agricultural datasets can be evaluated. Variables under consideration are weighted according to both their importance and their quality, and justified by literature describing variance and errors of the different state variables and measurement methods. The objective of such a classification framework of data evaluation and labelling is (i) to allow data base managers to pre-check the quality of data sets before integrating them into their data base, (ii) to support the creation and use of international publicly available benchmarked data sets for model evaluation, inter-comparison and improvement, (iii) to enable modellers to select appropriate data according to their requirements, (iv) to give guidance to experimentalists for designing their experiments with respect to aspects that go beyond their primary research question, allowing for a broader use of experimental data for systems analysis and modelling.