Convolutional neural networks (CNN) for spectroscopic modelling are currently tuned manually, and the effects of their hyperparameters are not analysed. These can result in sub-optimal models. ...Here, we propose an approach to tune one-dimensional CNN (1D-CNNs) automatically. It consists of a parametric representation of 1D-CNNs and an optimisation of hyperparameters to maximise a model's performance. We used a large European soil spectroscopic database to demonstrate our approach for estimating soil organic carbon (SOC) contents. To assess the optimisation, we compared it to random search, and to understand the effects of the hyperparameters, we calculated their importance using functional Analysis of Variance. Compared to random search, the optimisation produced better final results and showed faster convergence. The optimal model produced the most accurate estimates of SOC with Formula: see text (s.d.) and Formula: see text (s.d.). The hyperparameters associated with model training and architecture critically affected the model's performance, while those related to the spectral preprocessing had little effect. The optimisation searched through a complex hyperparameter space and returned an optimal 1D-CNN. Our approach simplified the development of 1D-CNNs for spectroscopic modelling by automatically selecting hyperparameters and preprocessing methods. Hyperparameter importance analysis shed light on the tuning process and increased the model's reliability.
Soil information is needed for environmental monitoring to address current concerns over food, water and energy securities, land degradation, and climate change. We developed the Soil Condition ...ANalysis System (SCANS) to help address these needs. It integrates an automated soil core sensing system (CSS) with statistical analytics and modeling to characterize soil at fine depth resolutions and across landscapes. The CSS’s sensors include a γ-ray attenuation densitometer to measure bulk density, digital cameras to image the measured soil, and a visible–near-infrared (vis–NIR) spectrometer to measure iron oxides and clay mineralogy. The spectra are also modeled to estimate total soil organic carbon (C), particulate, humus, and resistant organic C (POC, HOC, and ROC, respectively), clay content, cation exchange capacity (CEC), pH, volumetric water content, available water capacity (AWC), and their uncertainties. Measurements of bulk density and organic C are combined to estimate C stocks. Kalman smoothing is used to derive complete soil property profiles with propagated uncertainties. The SCANS provides rapid, precise, quantitative, and spatially explicit information about the properties of soil profiles with a level of detail that is difficult to obtain with other approaches. The information gained effectively deepens our understanding of soil and calls attention to the central role soil plays in our environment.
We can effectively monitor soil condition—and develop sound policies to offset the emissions of greenhouse gases—only with accurate data from which to define baselines. Currently, estimates of soil ...organic C for countries or continents are either unavailable or largely uncertain because they are derived from sparse data, with large gaps over many areas of the Earth. Here, we derive spatially explicit estimates, and their uncertainty, of the distribution and stock of organic C in the soil of Australia. We assembled and harmonized data from several sources to produce the most comprehensive set of data on the current stock of organic C in soil of the continent. Using them, we have produced a fine spatial resolution baseline map of organic C at the continental scale. We describe how we made it by combining the bootstrap, a decision tree with piecewise regression on environmental variables and geostatistical modelling of residuals. Values of stock were predicted at the nodes of a 3‐arc‐sec (approximately 90 m) grid and mapped together with their uncertainties. We then calculated baselines of soil organic C storage over the whole of Australia, its states and territories, and regions that define bioclimatic zones, vegetation classes and land use. The average amount of organic C in Australian topsoil is estimated to be 29.7 t ha⁻¹ with 95% confidence limits of 22.6 and 37.9 t ha⁻¹. The total stock of organic C in the 0–30 cm layer of soil for the continent is 24.97 Gt with 95% confidence limits of 19.04 and 31.83 Gt. This represents approximately 3.5% of the total stock in the upper 30 cm of soil worldwide. Australia occupies 5.2% of the global land area, so the total organic C stock of Australian soil makes an important contribution to the global carbon cycle, and it provides a significant potential for sequestration. As the most reliable approximation of the stock of organic C in Australian soil in 2010, our estimates have important applications. They could support Australia's National Carbon Accounting System, help guide the formulation of policy around carbon offset schemes, improve Australia's carbon balances, serve to direct future sampling for inventory, guide the design of monitoring networks and provide a benchmark against which to assess the impact of changes in land cover, land management and climate on the stock of C in Australia. In this way, these estimates would help us to develop strategies to adapt and mitigate the effects of climate change.
This paper compares predictions of soil organic carbon (SOC) using visible and near infrared reflectance (vis–NIR) hyperspectral proximal and remote sensing data. Soil samples were collected in the ...Narrabri region, dominated by Vertisols, in north western New South Wales (NSW), Australia. Vis–NIR spectra were collected over this region proximally with an AgriSpec portable spectrometer (350–2500 nm) and remotely from the Hyperion hyperspectral sensor onboard satellite (400–2500 nm). SOC contents were predicted by partial least-squares regression (PLSR) using both the proximal and remote sensing spectra. The spectral resolution of the proximal and remote sensing data did not affect prediction accuracy. However, predictions of SOC using the Hyperion spectra were less accurate than those of the Agrispec data resampled to similar resolution as the Hyperion spectra. Finally, the SOC map predicted using Hyperion data shows similarity with field observations. There is potential for the use of hyperspectral remote sensing for predictions of soil organic carbon. The use of these techniques will facilitate the implementation of digital soil mapping.
There are reflectance spectra in the visible and near infrared wavelengths from some 20 000 archived samples of soil in Australia. Their particular forms depend on absorbances at specific wavelengths ...characteristic of components in the soil such as water, iron oxides, clay minerals and carbon compounds, and so one might expect to be able to predict soil properties from the spectra. We tested a tree‐based technique for the prediction of 24 soil properties. A tree is first constructed by the definition of rules that separate the data into fairly homogeneous groups for any given property on both the absorptions at specified wavelengths and other, categoric, variables. Then within each group the property is predicted from the absorptions at those wavelengths by ordinary least‐squares regression. The spectroscopic predictions of the soil properties were compared with actual values in a subset of sample data separated from the whole data for validation. The criteria of success that we used were the root mean squared error (RMSE) to measure the inaccuracy of our predictions, the mean error (ME) to measure their bias and the standard deviation of the error (SDE) to measure their imprecision. We also used the ratio of performance to deviation (RPD), which is the ratio of the standard deviation of the observed values to the RMSE of the predictions; the larger it is the better does the technique perform. We found good predictions (RPD>2) for clay and total sand content, for total organic carbon and total nitrogen, pH, cation exchange capacity, and exchangeable calcium, magnesium and sodium. Several other properties were moderately well predicted (1.5 ≤ RPD < 2); they included air‐dry water content, volumetric water content at field capacity and wilting point, bulk density, the contents of silt, fine sand and coarse sand, total and exchangeable potassium, total phosphorus and extractable iron. Properties that were poorly predicted (RPD < 1.5) include the carbon‐to‐nitrogen ratio, available phosphorus and exchangeable acidity. We conclude that even though the predictions are less accurate than direct measurements, the spectra are cheap yet valuable sources of information for predicting values of individual soil properties when large numbers of analyses are needed, for example, for soil mapping.
Clay minerals are the most reactive inorganic components of soils. They help to determine soil properties and largely govern their behaviors and functions. Clay minerals also play important roles in ...biogeochemical cycling and interact with the environment to affect geomorphic processes such as weathering, erosion and deposition. This paper provides new spatially explicit clay mineralogy information for Australia that will help to improve our understanding of soils and their role in the functioning of landscapes and ecosystems. I measured the abundances of kaolinite, illite and smectite in Australian soils using near infrared (NIR) spectroscopy. Using a model‐tree algorithm, I built rule‐based models for each mineral at two depths (0–20 cm, 60–80 cm) as a function of predictors that represent the soil‐forming factors (climate, parent material, relief, vegetation and time), their processes and the scales at which they vary. The results show that climate, parent material and soil type exert the largest influence on the abundance and spatial distribution of the clay minerals; relief and vegetation have more local effects. I digitally mapped each mineral on a 3 arc‐second grid. The maps show the relative abundances and distributions of kaolinite, illite and smectite in Australian soils. Kaolinite occurs in a range of climates but dominates in deeply weathered soils, in soils of higher landscapes and in regions with more rain. Illite is present in varied landscapes and may be representative of colder, more arid climates, but may also be present in warmer and wetter soil environments. Smectite is often an authigenic mineral, formed from the weathering of basalt, but it also occurs on sediments and calcareous substrates. It occurs predominantly in drier climates and in landscapes with low relief. These new clay mineral maps fill a significant gap in the availability of soil mineralogical information. They provide data to for example, assist with research into soil fertility and food production, carbon sequestration, land degradation, dust and climate modeling and paleoclimatic change.
Key Points
New insights into factors affecting formation and distribution of clay minerals
New methodology for measuring and high‐resolution mapping of clay minerals
Can be used by various geoscientists for dust modeling, C seq., food production
Information on the geographic variation in soil has traditionally been presented in polygon (choropleth) maps at coarse scales. Now scientists, planners, managers and politicians want quantitative ...information on the variation and functioning of soil at finer resolutions; they want it to plan better land use for agriculture, water supply and the mitigation of climate change land degradation and desertification. The GlobalSoilMap project aims to produce a grid of soil attributes at a fine spatial resolution (approximately 100m), and at six depths, for the purpose. This paper describes the three-dimensional spatial modelling used to produce the Australian soil grid, which consists of Australia-wide soil attribute maps. The modelling combines historical soil data plus estimates derived from visible and infrared soil spectra. Together they provide a good coverage of data across Australia. The soil attributes so far include sand, silt and clay contents, bulk density, available water capacity, organic carbon, pH, effective cation exchange capacity, total phosphorus and total nitrogen. The data on these attributes were harmonised to six depth layers, namely 0–0.05m, 0.05–0.15m, 0.15–0.30m, 0.30–0.60m, 0.60–1.00m and 1.00–2.00m, and the resulting values were incorporated simultaneously in the models. The modelling itself combined the bootstrap, a decision tree with piecewise regression on environmental variables and geostatistical modelling of residuals. At each layer, values of the soil attributes were predicted at the nodes of a 3 arcsecond (approximately 90m) grid and mapped together with their uncertainties. The assessment statistics for each attribute mapped show that the models explained between 30% and 70% of their total variation. The outcomes are illustrated with maps of sand, silt and clay contents and their uncertainties. The Australian three-dimensional soil maps fill a significant gap in the availability of quantitative soil information in Australia.
This review addresses the applicability of visible (Vis), near-infrared (NIR), and mid-infrared (MIR) reflectance spectroscopy for the prediction of soil properties. We address (1) the properties ...that can be predicted and the accuracy of the predictions, (2) the most suitable spectral regions for specific soil properties, (3) the number of predictions reported for each property, and (4) in-field versus laboratory spectral techniques.
We found the following properties to be successfully predicted: soil water content, texture, soil carbon (C), cation exchange capacity, calcium and magnesium (exchangeable), total nitrogen (N), pH, concentration of metals/metalloids, microbial size, and activity. Generally, MIR produced better predictions than Vis-NIR, but Vis-NIR outperformed MIR for a number of properties (e.g., biological). An advantage of Vis-NIR is instrument portability although a new range of MIR portable devices is becoming available. In-field predictions for clay, water, total organic C, extractable phosphorus, total C and N appear similar to laboratory methods, but there are issues regarding, for example, sample heterogeneity, moisture content, and surface roughness.
The nature of the variable being predicted, the quality and consistency of the reference laboratory methods, and the adequate representation of unknowns by the calibration set must be considered when predicting soil properties using reflectance spectroscopy.
Proximal soil sensing (PSS) using portable visible–near infrared (vis–NIR: 400–2500 nm) spectrophotometers can be used to measure soil properties in situ. The objectives of this research were: (i) to ...compare field spectra collected in situ to spectra collected in the laboratory, (ii) to estimate soil colour and mineral composition from the spectra, and (iii) to make predictions of clay content using a spectral library that contains mostly spectra collected in the laboratory but also a smaller number of field spectra that were collected in situ. The evaluation was conducted using 10 soil profiles derived from different parent materials. Spectroscopic measurements were collected both in the field and in the laboratory at different depths, in triplicate. These spectra were compared multivariately using principal component analysis and by using wavelength specific
t-tests. Except in the water absorption regions around 1400 nm and 1900 nm and in regions that are not primarily used to characterise soil mineral composition, field-collected spectra were not significantly different to spectra collected in the laboratory. Estimates of soil colour and mineral composition were made from the spectra using a continuum-removal technique and by targeting characteristic absorption features. Estimates of soil colour were derived from the spectra of each profile using the Munsell HVC and CIE
Lab colour models. These were compared to qualitative estimates of Munsell colour made in the field. Spectroscopic estimates of soil colour were in fair agreement with Munsell book estimates, although the vis–NIR estimates tended to be somewhat darker and more yellow. Quantitative estimates of mineral composition were derived by comparing soil spectra to the spectra of pure minerals. These estimates were assessed using qualitative X-ray diffraction (XRD) analysis. The characterisation of soil mineral composition by vis–NIR was effective, with good agreement between the results of this method and XRD analysis. The vis–NIR technique was less laborious than conventional XRD, did not require sample preparation and was better at detecting iron oxides. A spectral library containing 1287 laboratory-collected spectra and 74 spectra collected in situ at field conditions was used to develop partial least squares regression (PLSR) models to predict the clay content of both the field- and laboratory-collected spectra from the 10 soil profiles. Predictions of clay content from the field-collected spectra (RMSE
=
7.9%) were slightly more accurate than those from the laboratory-collected spectra (RMSE
=
8.3%). Extending the range of the PLSR calibrations by ‘spiking’ them with 74 field spectra improved the generalisation capacity of the models. PLSR with bootstrap aggregation, or bagging-PLSR (bPLSR), produced predictions of clay content for each profile with a measure of their uncertainty.
Soil properties, such as organic carbon, pH and clay content, are critical indicators of ecosystem function. Visible-near infrared (vis-NIR) reflectance spectroscopy has been widely used to ...cost-efficiently estimate such soil properties. Multivariate modelling, such as partial least squares regression (PLSR), and machine learning are the most common methods for modelling soil properties with spectra. Often, such models do not account for the multiresolution information presented in the vis-NIR signal, or the spatial variation in the data. To address these potential shortcomings, we used wavelets to decompose the vis-NIR spectra of 226 soils from agricultural and forested regions in south-western Western Australia and developed a wavelet geographically weighted regression (WGWR) for estimating soil organic carbon content, clay content and pH. To evaluate the WGWR models, we compared them to linear models derived with multiresolution data from a wavelet decomposition (WLR) and PLSR without multiresolution information. Overall, validation of the WGWR models produced more accurate estimates of the soil properties than WLR and PLSR. Around 3.5-49.1% of the improvement in the estimates was due to the multiresolution analysis and 1.0-5.2% due to the integration of spatial information in the modelling. The WGWR improves the modelling of soil properties with spectra.
PDF
