An imperative for soil spectroscopic modelling is to think global but fit local with transfer learning

Raphael A. Viscarra Rossel*, Zefang Shen, Leonardo Ramirez Lopez, Thorsten Behrens, Zhou Shi, Johanna Wetterlind, Kenneth A. Sudduth, Bo Stenberg, Cesar Guerrero, Asa Gholizadeh, Eyal Ben-Dor, Mervin St Luce, Claudio Orellano

*Corresponding author for this work

Research output: Contribution to journalReview articlepeer-review

Abstract

Soil spectroscopy with machine learning (ML) can estimate soil properties. Extensive soil spectral libraries (SSLs) have been developed for this purpose. However, general models built with those SSLs do not generalize well on new ‘unseen’ local data. The main reason is the different characteristics of the observations in the SSL and the local data, which cause their conditional and marginal distributions to differ. This makes the modelling of soil properties with spectra challenging. General models developed using large ‘global’ SSLs offer broad, systematic information on the soil-spectra relationships. However, to accurately generalize in a local situation, they must be adjusted to capture the site-specific characteristics of the local observations. Most current methods for ‘localizing’ spectroscopic modelling report inconsistent results. An understanding of spectroscopic ‘localization’ is lacking, and there is no framework to guide further developments. Here, we review current localization methods and propose their reformulation as a transfer learning (TL) undertaking. We then demonstrate the implementation of instance-based TL with RS-LOCAL 2.0 for modelling the soil organic carbon (SOC) content of 12 sites representing fields, farms and regions from 10 countries on the seven continents. The method uses a small number of instances or observations (measured soil property values and corresponding spectra) from the local site to transfer relevant information from a large and diverse global SSL (GSSL 2.0) with more than 50,000 records. We found that with ≤30 local observations, RS-LOCAL 2.0 produces more accurate and stable estimates of SOC than modelling with only the local data. Using the information in the GSSL 2.0 and reducing the number of samples for laboratory analysis, the method improves the cost-efficiency and practicality of soil spectroscopy. We interpreted the transfer by analysing the data, models, and soil and environmental relationships of the local and the ‘transferred’ data to gain insight into the approach. Transferring instances from the GSSL 2.0 to the local sites helped to align their conditional and marginal distributions, making the spectra-SOC relationships in the models more robust. Finally, we propose directions for future research. The guiding principle for developing practical and cost-effective spectroscopy should be to think globally but fit locally. By reformulating the localization problem within a TL framework, we hope to have acquainted the soil science community with a set of methodologies that can inspire the development of new, innovative algorithms for soil spectroscopic modelling.

Original languageEnglish
Article number104797
JournalEarth-Science Reviews
Volume254
DOIs
StatePublished - Jul 2024

Funding

FundersFunder number
Government of Western Australia
Australian Government
Australian Government's Australia-China Science and Research Fund-Joint Research CentresACSRIV000077
Australian Research CouncilDP210100420
Australian Research Council

    Keywords

    • Localization
    • Machine learning
    • Multivariate statistics
    • Soil organic carbon
    • Soil spectral library
    • Spectroscopic modelling
    • Transfer learning
    • vis–NIR spectra

    Fingerprint

    Dive into the research topics of 'An imperative for soil spectroscopic modelling is to think global but fit local with transfer learning'. Together they form a unique fingerprint.

    Cite this