Robustness of portable and laboratory-based visible/near- and mid-infrared spectroscopy for optimized determination of temporally and spatially variable soil properties

Application of visible/near- (VISNIRS) and mid-infrared (MIRS) spectroscopy to the field of soil science is promising, as it allows information about a range of properties to be collected simultaneously and rapidly, thereby providing high spatial and temporal resolution data for use in, e.g., soil mapping, precision agriculture, and long-term monitoring. However, investigation of the conditions under which visNIRS and MIRS can replace traditional lab methods is required, as the accuracy, robustness, and efficiency of these methods depend on a wide range of factors that are not sufficiently understood. The objective of this dissertation was therefore to i) compare the performance of field vs lab visNIRS and MIRS for prediction of key soil properties using partial least squares regression; ii) investigate the spectral prediction mechanisms for these soil properties using a) loadings of PLSR components, b) variable importance in the projection scores, c) principal component analysis, and d) model robustness in independent validation; iii) determine the effects of disturbance factors, including a) soil moisture (and its interaction with soil texture) and b) changes in the crop residue quantity (via residue incorporation or decomposition) and quality (clover vs wheat straw incorporation) in soil; iv) compare the performance of various sizes of local calibrations and regional calibrations with and without the addition of local soils (spiking); v) compare the accuracy of spectral models for prediction of soil organic carbon (OC) fractions of variable residence time to prediction with covariates using multiple linear regressions; and vi) determine if spectroscopy can accurately predict the effects of tillage treatments on soil OC contents using analysis of variance. The three studies composing this dissertation utilized surface soils from several sites in Germany. The soil properties under investigation included total and fraction OC contents, total nitrogen (TN) content, pH, and texture. These studies demonstrated the excellent accuracy of lab MIRS OC and TN estimations, while the accuracy of visNIRS and MIRS was lower and more comparable for texture predictions. We found spectral estimation of OC fractions may not have an advantage compared to estimation with covariates since prediction mechanisms are likewise partially indirect (i.e. both organic and mineral spectral signatures were important). The loss of prediction accuracy from lab to field measurement was greater for MIRS than visNIRS, but in situ performance rankings of visNIRS vs MIRS were moisture dependent. Soil moisture more negatively affected OC prediction than clay prediction. No simple trend was established for the performances of soil subsets with low, high or variable moisture content, but accuracy was most negatively affected by moisture for the site with the highest sand content. The independence of the validation soils had a marked effect on model performance, and calculation of bias was essential to describing calibration suitability and hinted at indirect prediction mechanisms. We demonstrated the performance of lab vs field MIRS models for small local and regional calibrations with and without spiking, and the diminishing marginal returns to accuracy from using ever-larger calibration sets. While purely regional lab-MIRS models could accurately predict changes to OC content in response to tillage, field-MIRS models required local or spiked regional calibration to achieve accurate estimations. Thus, the higher efficiency of field measurement is counterbalanced by a more arduous calibration process to achieve satisfactory models.