This study aims to develop a deep learning based classification framework for remotely sensed time series. The experiment was carried out in Yolo County, California, which has a very diverse ...irrigated agricultural system dominated by economic crops. For the challenging task of classifying summer crops using Landsat Enhanced Vegetation Index (EVI) time series, two types of deep learning models were designed: one is based on Long Short-Term Memory (LSTM), and the other is based on one-dimensional convolutional (Conv1D) layers. Three widely-used classifiers were also tested for comparison, including a gradient boosting machine called XGBoost, Random Forest, and Support Vector Machine. Although LSTM is widely used for sequential data representation, in this study its accuracy (82.41%) and F1 score (0.67) were the lowest among all the classifiers. Among non-deep-learning classifiers, XGBoost achieved the best result with 84.17% accuracy and an F1 score of 0.69. The highest accuracy (85.54%) and F1 score (0.73) were achieved by the Conv1D-based model, which mainly consists of a stack of Conv1D layers and an inception module. The behavior of the Conv1D-based model was inspected by visualizing the activation on different layers. The model employs EVI time series by examining shapes at various scales in a hierarchical manner. Lower Conv1D layers of the optimized model capture small scale temporal variations, while upper layers focus on overall seasonal patterns. Conv1D layers were used as an embedded multi-level feature extractor in the classification model which automatically extracts features from input time series during training. The automated feature extraction reduces the dependency on manual feature engineering and pre-defined equations of crop growing cycles. This study shows that the Conv1D-based deep learning framework provides an effective and efficient method of time series representation in multi-temporal classification tasks.
•Deep neural networks were developed for crop classification.•Deep neural network achieved 85.54% accuracy and an F1 score of 0.73.•The best non-deep-learning classifier achieved 84.17% accuracy and an F1 score of 0.69.•One-dimensional convolutional neural network was used as automated temporal feature extractor.•One-dimensional convolutional neural network identifies complex seasonal dynamics of economic crops.
As an effective approach to obtaining agricultural information, the remote sensing technique has been applied in the classification of crop types. The unmanned aerial vehicle (UAV)-manned ...hyperspectral sensors provide imagery with high spatial and high spectral resolutions. Moreover, the detailed spatial information, as well as abundant spectral properties of UAV-manned hyperspectral imagery, opens a new avenue to the fine classification of crops. In this manuscript, multiscale superpixel-based approaches are proposed for the fine identification of crops in the UAV-manned hyperspectral imagery. Specifically, the multiscale superpixel segmentation is performed and a series of superpixel maps can be obtained. Then, the multiscale information is integrated into image classification by two strategies, namely pre-processing and post-processing. For the pre-processing strategy, the superpixel is regarded as the minimum unit for image classification, whose feature is obtained by using the average of spectral values of pixels within it. At each scale, the classification is performed on the basis of the superpixel. Then, the multiscale classification results are combined to generate the final map. For the post-processing strategy, the pixel-wise classification is implemented to obtain the label and posterior probabilities of each pixel. Subsequently, the superpixel-based voting is conducted at each scale, and these obtained voting results are fused to generate the multiscale voting result. To evaluate the effectiveness of the proposed approaches, three open-sourced hyperspectral UAV-manned datasets are employed in the experiments. Meanwhile, seven training sets with different numbers of labeled samples and two classifiers are taken into account for further analysis. The results demonstrate that the multiscale superpixel-based approaches outperform the single-scale approaches. Meanwhile, the post-processing strategy is superior to the pre-processing strategy in terms of higher classification accuracies in all the datasets.
The globe's population is increasing day by day, which causes the severe problem of organic food for everyone. Farmers are becoming progressively conscious of the need to control numerous essential ...factors such as crop health, water or fertilizer use, and harmful diseases in the field. However, it is challenging to monitor agricultural activities. Therefore, precision agriculture is an important decision support system for food production and decision-making. Several methods and approaches have been used to support precision agricultural practices. The present study performs a systematic literature review on hyperspectral imaging technology and the most advanced deep learning and machine learning algorithm used in agriculture applications to extract and synthesize the significant datasets and algorithms. We reviewed legal studies carefully, highlighted hyperspectral datasets, focused on the most methods used for hyperspectral applications in agricultural sectors, and gained insight into the critical problems and challenges in the hyperspectral data processing. According to our study, it has been found that the Hyperion hyperspectral, Landsat-8, and Sentinel 2 multispectral datasets were mainly used for agricultural applications. The most applied machine learning method was support vector machine and random forest. In addition, the deep learning-based Convolutional Neural Networks (CNN) model is mainly used for crop classification due to its high performance with hyperspectral datasets. The present review will be helpful to the new researchers working in the field of hyperspectral remote sensing for agricultural applications with a machine and deep learning methods.
•Hyperspectral image data and deep/machine learning methods have been synthesized.•The most used hyper and multispectral data is EO-1 Hyperion, Landsat and Sentinel.•The most used machine learning methods are SVM, random forest, and neural networks.•The most widely used deep learning methods are convolutional neural networks.•Python is the most preferred language for developing deep learning models.
This study describes a novel three-dimensional (3D) convolutional neural networks (CNN) based method that automatically classifies crops from spatio-temporal remote sensing images. First, 3D kernel ...is designed according to the structure of multi-spectral multi-temporal remote sensing data. Secondly, the 3D CNN framework with fine-tuned parameters is designed for training 3D crop samples and learning spatio-temporal discriminative representations, with the full crop growth cycles being preserved. In addition, we introduce an active learning strategy to the CNN model to improve labelling accuracy up to a required threshold with the most efficiency. Finally, experiments are carried out to test the advantage of the 3D CNN, in comparison to the two-dimensional (2D) CNN and other conventional methods. Our experiments show that the 3D CNN is especially suitable in characterizing the dynamics of crop growth and outperformed the other mainstream methods.
Unmanned aerial vehicle (UAV)-borne hyperspectral systems can acquire hyperspectral imagery with a high spatial resolution (which we refer to here as H2 imagery). As a result of the low operating ...cost, high flexibility, and the ability to achieve real-time data acquisition, UAV-borne hyperspectral systems have become an important data source for remote sensing based agricultural monitoring. However, precise crop classification based on UAV-borne H2 imagery is a challenging task when faced with a number of different crop classes. The traditional hyperspectral classification methods, such as the spectral-based and object-oriented classification methods, have difficulty in classifying H2 imagery, faced with the problems of salt-and-pepper (SP) noise and scale selection. In this article, the deep convolutional neural network with a conditional random field classifier (CNNCRF) framework is proposed for precise crop classification with UAV-borne H2 imagery. In the proposed method, a deep convolutional neural network (CNN) is designed to extract and fuse in-depth spectral and local spatial features, and the conditional random field (CRF) model further incorporates the spatial-contextual information to improve the problem of holes and isolated regions in the classification map. Meanwhile, virtual sample augmentation based on the hyperspectral imaging mechanism is used to lessen the issue of the limited labeled samples. To validate the results, a new dataset—the Wuhan UAV-borne hyperspectral image (WHU-Hi) dataset—has been built for precise crop classification. The experimental results obtained using the WHU-Hi dataset confirm the accuracy and visualization performance of the proposed CNNCRF classification method, which outperforms the previous methods. In addition, the WHU-Hi dataset could serve as a benchmark dataset for hyperspectral image classification studies.
•AV-borne datasets with high spectral and spatial resolution (H2) is acquired.•A deep convolutional neural network with CRF classifier (CNNCRF) is proposed.•UAV H2 datasets is classified by CNNCRF for precise crop identification.•Wuhan UAV-borne hyperspectral image (WHU-Hi) has been built as the benchmark.•Effective results were achieved from the accuracy and visualization performance.
When using microwave remote sensing for land use/land cover (LULC) classifications, there are a wide variety of imaging parameters to choose from, such as wavelength, imaging mode, incidence angle, ...spatial resolution, and coverage. There is still a need for further study of the combination, comparison, and quantification of the potential of multiple diverse radar images for LULC classifications. Our study site, the Qixing farm in Heilongjiang province, China, is especially suitable to demonstrate this. As in most rice growing regions, there is a high cloud cover during the growing season, making LULC from optical images unreliable. From the study year 2009, we obtained nine TerraSAR-X, two Radarsat-2, one Envisat-ASAR, and an optical FORMOSAT-2 image, which is mainly used for comparison, but also for a combination. To evaluate the potential of the input images and derive LULC with the highest possible precision, two classifiers were used: the well-established Maximum Likelihood classifier, which was optimized to find those input bands, yielding the highest precision, and the random forest classifier. The resulting highly accurate LULC-maps for the whole farm with a spatial resolution as high as 8 m demonstrate the beneficial use of a combination of x- and c-band microwave data, the potential of multitemporal very high resolution multi-polarization TerraSAR-X data, and the profitable integration and comparison of microwave and optical remote sensing images for LULC classifications.
•Crop classification based on time series of Sentinel-1 images using SAR polarimetry.•Multi-temporal descriptors of crops phenology derived from coherence matrices and H/α decomposition.•Fast ...object-oriented classification approach with the use of Random Forest classifier.•County-wide testing and mapping.•Use of administrative data as a source of reference data.
Crop classification is a crucial prerequisite for the collection of agricultural statistics, efficient crop management, biodiversity control, the design of agricultural policy, and food security. Crops are characterized by significant change during the growing season, and this information can be used to improve classification accuracy. However, capturing variation in vegetation cover requires a reliable source of valid data. Sentinel-1 radar images are a good candidate, as they supply information about Earth’s surface every six days, independent of weather and light conditions. In this paper, we present a method for crop classification based on radar polarimetry. We propose a set of multi-temporal indices derived from time series Sentinel-1 images that aim to characterize crop phenology. A big data, object-oriented classification technique is developed and tested on 16 crop types for the whole of Poland. Our analysis found that overall accuracy varied (regionally) from 86.36 to 89.13% in 2019, and from 85.95 to 89.81% in 2020. F1 scores for individual crops varied from 0.73 to 0.99, and the use of our multi-temporal phenological indices increased F1 scores by about 0.14 compared to calculations using only basic parameters. Results obtained for the whole country demonstrate the efficacy of the method and its resistance to environmental conditions.
•Overall accuracy of crop classification reaches 85 %–90 % by using full year UAVSAR.•Polarimetric parameters contribute more than linear polarizations to crop mapping.•The CP parameters are much ...more important than the FD parameters for crop mapping.•The combined use of four acquisitions is adequate to achieve a nearly optimal accuracy.
Accurate and timely information on the distribution of crop types is vital to agricultural management, ecosystem services valuation and food security assessment. Synthetic Aperture Radar (SAR) systems have become increasingly popular in the field of crop monitoring and classification. However, the potential of time-series polarimetric SAR data has not been explored extensively, with several open scientific questions (e.g. the optimal combination of image dates for crop classification) that need to be answered. In this research, the usefulness of full year (both 2011 and 2014) L-band fully-polarimetric Uninhabited Aerial Vehicle Synthetic Aperture Radar (UAVSAR) data in crop classification was fully investigated over an agricultural region with a heterogeneous distribution of crop categories. In total, 11 crop classes including tree crops (almond and walnut), forage crops (grass, alfalfa, hay, and clover), a spring crop (winter wheat), and summer crops (corn, sunflower, tomato, and pepper), were discriminated using the Random Forest (RF) algorithm. The SAR input variables included raw linear polarization channels as well as polarimetric parameters derived from Cloude-Pottier (CP) and Freeman-Durden (FD) decompositions. Results showed clearly that the polarimetric parameters yielded much higher classification accuracies than linear polarizations. The combined use of all variables (linear polarizations and polarimetric parameters) produced the maximum overall accuracy of 90.50 % and 84.93 % for 2011 and 2014, respectively, with a significant increase of approximately 8 percentage points compared with linear polarizations alone. The variable importance provided by the RF illustrated that the polarimetric parameters had a far greater influence than linear polarizations, with the CP parameters being much more important than the FD parameters. The most important acquisitions were the images dated during the peak biomass stage (July and August) when the differences in structural characteristics between most crops were the largest. At the same time, the images in spring (April and May) and autumn (October) also contributed to the crop classification since they respectively provided unique information for discriminating fruit crops (almond and walnut) as well as summer crops (corn, sunflower, and tomato). As a result, the combined use of only four acquisitions (dated May, July, August, and October for 2011 and April, June, August, and October for 2014) was adequate to achieve a nearly-optimal overall accuracy. In light of the promising classification accuracies demonstrated in this research, it becomes increasingly viable to provide accurate and up-to-date crops inventories over large areas based solely on multitemporal polarimetric SAR.
Deep learning (DL) is a powerful state-of-the-art technique for image processing including remote sensing (RS) images. This letter describes a multilevel DL architecture that targets land cover and ...crop type classification from multitemporal multisource satellite imagery. The pillars of the architecture are unsupervised neural network (NN) that is used for optical imagery segmentation and missing data restoration due to clouds and shadows, and an ensemble of supervised NNs. As basic supervised NN architecture, we use a traditional fully connected multilayer perceptron (MLP) and the most commonly used approach in RS community random forest, and compare them with convolutional NNs (CNNs). Experiments are carried out for the joint experiment of crop assessment and monitoring test site in Ukraine for classification of crops in a heterogeneous environment using nineteen multitemporal scenes acquired by Landsat-8 and Sentinel-1A RS satellites. The architecture with an ensemble of CNNs outperforms the one with MLPs allowing us to better discriminate certain summer crop types, in particular maize and soybeans, and yielding the target accuracies more than 85% for all major crops (wheat, maize, sunflower, soybeans, and sugar beet).
PDF
