Validating land change models based on configuration disagreement: Fid-Bau Portal

Validating land change models based on configuration disagreement

Pickard, Brian R. / Meentemeyer, Ross K.

Abstract Land change models are increasingly being employed to predict future landscapes and influence policy and decision-making. To ensure the highest model accuracy, validation methods have become commonplace following a land change simulation. The most common validation method employed uses quantity and allocation disagreement. However, these current measures may not account for differences in the configurations of land change, placing them in potential conflict with the principals of heterogeneity and spatial patterning of landscape ecology. We develop a new metric, termed configuration disagreement, designed to focus on the size, shape, and complexity of land change simulations. Using this metric, we demonstrate the value of including errors of configuration disagreement – in addition to quantity and allocation error – in the assessment of land change models. Four computational experiments of land change that vary only in spatial pattern are developed using the FUTURES land change model. For each experiment, configuration disagreement and the traditional validation metrics are computed simultaneously. Results indicate that models validated only with consideration of quantity and allocation error may misrepresent, or not fully account for, spatial patterns of landscape change. The research objective will ultimately guide which component, or components, of model disagreement are most critical for consideration. Yet, our work reveals why it may be more helpful to validate simulations in terms of configuration accuracy. Specifically, if a study requires accurately modeling the spatial patterns and arrangements of land cover. Configuration disagreement could add critical information with respect to a model's simulated changes in size, shape, and spatial arrangements, and possibly enhance ecologically meaningful land change science.

Highlights • We detail a new metric for assessing accuracy of land change models, called configuration disagreement, designed to be complimentary to the existing validation methods of quantity and allocation. • Simulation results indicate that models validated using currently accepted approaches may misrepresent the accuracy of land change models with respect to spatial patterns of landscape change. • By calculating current validation methods and configuration disagreement simultaneously, we demonstrate the need for considering configuration to more holistically understand model accuracy. • The research objective will typically guide which components of model accuracy are most critical for consideration, however ecologically meaningful land change science could be enhanced by considering configuration disagreement.