•Cross-modal data retrieval.•Novel framework for designing a unified representation for different modalities of data.•Improved performance using the unified representation for both cross and ...uni-modal retrieval.•Extensive experiments performed on the benchmark DSRSID and UC-Merced with audio samples datasets.
We address the problem of cross-modal information retrieval in the domain of remote sensing. In particular, we are interested in two application scenarios: i) cross-modal retrieval between panchromatic (PAN) and multispectral imagery, and ii) multi-label image retrieval between very high resolution (VHR) images and speech-based label annotations. These multi-modal retrieval scenarios are more challenging than the traditional uni-modal retrieval approaches given the inherent differences in distributions between the modalities. However, with the increasing availability of multi-source remote sensing data and the scarcity of enough semantic annotations, the task of multi-modal retrieval has recently become extremely important. In this regard, we propose a novel deep neural network-based architecture that is considered to learn a discriminative shared feature space for all the input modalities, suitable for semantically coherent information retrieval. Extensive experiments are carried out on the benchmark large-scale PAN - multispectral DSRSID dataset and the multi-label UC-Merced dataset. Together with the Merced dataset, we generate a corpus of speech signals corresponding to the labels. Superior performance with respect to the current state-of-the-art is observed in all the cases.
We address the problem of multi-label scene classification from Very High Resolution (VHR) satellite remote sensing (RS) images in this paper by exploring the deep graph convolutional network (GCN). ...Since a given VHR RS scene contains several local features, the traditional single-label classification frameworks do not convey the true semantics of the scene. The multi-label classification approaches, on the other hand, is expected to aid in better characterization of the area under consideration. Under the multi-label setup, we find it intuitive to represent a given image as a region adjacency graph (RAG) of the respective local regions. In order to extract discriminative features from such irregular structures for enhanced classification, we model the subsequent supervised learning problem in terms of a novel multi-label deep GCN. We validate the efficacy of the proposed technique on the popular UC-Merced dataset where our framework outperforms with respect to the literature.
Altering the morphology of electrochemically active nanostructured materials could fundamentally influence their subsequent catalytic as well as oxygen evolution reaction (OER) performance. Enhanced ...OER activity for mixed-metal spinel-type sulfide (CuCo2S4) nanorods is generally done by blending the material that has high conductive supports together with those having a high surface volume ratio, for example, graphitic carbon nitrides (g-C3N4). Here, we report a noble-metal-free CuCo2S4 nanorod-based electrocatalyst appropriate for basic OER and neutral media, through a simple one-step thermal decomposition approach from its molecular precursors pyrrolidine dithiocarbamate-copper(II), CuPDTC2, and pyrrolidine dithiocarbamate-cobalt(II), CoPDTC2 complexes. Transmission electron microscopy (TEM) images as well as X-ray diffraction (XRD) patterns suggest that as-synthesized CuCo2S4 nanorods are highly crystalline in nature and are connected on the g-C3N4 support. Attenuated total reflectance–Fourier-transform infrared (ATR-FTIR), X-ray photoelectron spectroscopy (XPS), and Raman spectroscopy studies affirm the successful formation of bonds that bridge (Co–N/S–C) at the interface of CuCo2S4 nanorods and g-C3N4. The kinetics of the reaction are expedited, as these bridging bonds function as an electron transport chain, empowering OER electrocatalytically under a low overpotential (242 mV) of a current density at 10 mA cm–2 under basic conditions, resulting in very high durability. Moreover, CuCo2S4/g-C3N4 composite nanorods exhibit a high catalytic activity of OER under a neutral medium at an overpotential of 406 mV and a current density of 10 mA cm–2.
Palladium nanoparticles (Pd NPs) have been grafted at the surfaces of a Co-containing metal-organic framework material MCoS-1. The Pd NPs grafted material Pd(0)/MCoS-1 has been characterized by HR ...TEM, XPS and EPR spectroscopic analyses. Pd(0)/MCoS-1 showed excellent catalytic activities in Suzuki-Miyaura cross-coupling reaction for the synthesis of a diverse range of biaryl organics in water. Sonogashira cross-coupling between terminal alkynes and aryl halides has also been carried out over this Pd-nanocatalyst in water without the addition of any Cu co-catalyst. In both reactions the catalyst is easily recoverable and can be reused for several times without appreciable loss of catalytic activity. No evidence of the leaching of Pd from the catalyst during the course of reaction has been observed, suggesting true heterogeneity in our catalytic systems.
•Use of Kennaugh matrix representation for agricultural crop classification.•Implemented a new day-wise and time series approach for classification.•Comparison of day-wise and time-series approaches ...using RF and XGB classifiers.
Accurate spatio-temporal classification of crops is of prime importance for in-season crop monitoring. Synthetic Aperture Radar (SAR) data provides diverse physical information about crop morphology. In the present work, we propose a day-wise and a time-series approach for crop classification using full-polarimetric SAR data. In this context, the 4 × 4 real Kennaugh matrix representation of a full-polarimetric SAR data is utilized, which can provide valuable information about various morphological and dielectric attributes of a scatterer. The elements of the Kennaugh matrix are used as the parameters for the classification of crop types using the random forest and the extreme gradient boosting classifiers.
The time-series approach uses data patterns throughout the whole growth period, while the day-wise approach analyzes the PolSAR data from each acquisition into a single data stack for training and validation. The main advantage of this approach is the possibility of generating an intermediate crop map, whenever a SAR acquisition is available for any particular day. Besides, the day-wise approach has the least climatic influence as compared to the time series approach. However, as time-series data retains the crop growth signature in the entire growth cycle, the classification accuracy is usually higher than the day-wise data.
Within the Joint Experiment for Crop Assessment and Monitoring (JECAM) initiative, in situ measurements collected over the Canadian and Indian test sites and C-band full-polarimetric RADARSAT-2 data are used for the training and validation of the classifiers. Besides, the sensitivity of the Kennaugh matrix elements to crop morphology is apparent in this study. The overall classification accuracies of 87.75% and 80.41% are achieved for the time-series data over the Indian and Canadian test sites, respectively. However, for the day-wise data, a ∼6% decrease in the overall accuracy is observed for both the classifiers.
Conventional remote sensing data analysistechniques have a significant bottleneck of operating on a selectively chosen small-scale dataset. Availability of an enormous volume of data demands handling ...large-scale, diverse data, which have been made possible with neural network-based architectures. This article exploits the contextual information capturing ability of deep neural networks, particularly investigating multispectral band properties from Sentinel-2 image patches. Besides, an increase in the spatial resolution often leads to nonlinear mixing of land-cover types within a target resolution cell. We recognize this fact and group the bands according to their spatial resolutions, and propose a classification and retrieval framework. We design a representation learning framework for classifying the multispectral data by first utilizing all the bands and then using the grouped bands according to their spatial resolutions. We also propose a novel triplet-loss function for multilabeled images and use it to design an interband group retrieval framework. We demonstrate its effectiveness over the conventional triplet-loss function. Finally, we present a comprehensive discussion of the obtained results. We thoroughly analyze the performance of the band groups on various land-cover and land-use areas from agro-forestry regions, water bodies, and human-made structures. Experimental results for the classification and retrieval framework on the benchmarked BigEarthNet dataset exhibit marked improvements over existing studies.
This paper addresses the problem of land-cover classification of remotely sensed image pairs in the context of domain adaptation. The primary assumption of the proposed method is that the training ...data are available only for one of the images (source domain), whereas for the other image (target domain), no labeled data are available. No assumption is made here on the number and the statistical properties of the land-cover classes that, in turn, may vary from one domain to the other. The only constraint is that at least one land-cover class is shared by the two domains. Under these assumptions, a novel graph theoretic cross-domain cluster mapping algorithm is proposed to detect efficiently the set of land-cover classes which are common to both domains as well as the additional or missing classes in the target domain image. An interdomain graph is introduced, which contains all of the class information of both images, and subsequently, an efficient subgraph-matching algorithm is proposed to highlight the changes between them. The proposed cluster mapping algorithm initially clusters the target domain data into an optimal number of groups given the available source domain training samples. To this end, a method based on information theory and a kernel-based clustering algorithm is proposed. Considering the fact that the spectral signature of land-cover classes may overlap significantly, a postprocessing step is applied to refine the classification map produced by the clustering algorithm. Two multispectral data sets with medium and very high geometrical resolution and one hyperspectral data set are considered to evaluate the robustness of the proposed technique. Two of the data sets consist of multitemporal image pairs, while the remaining one contains images of spatially disjoint geographical areas. The experiments confirm the effectiveness of the proposed framework in different complex scenarios.
In this article, we address class incremental learning (IL) in remote sensing image analysis. Since remote sensing images are acquired continuously over time by Earth's observation sensors, the ...land-cover/land-use classes on the ground are likely to be found in a gradational manner. This process restricts the deployment of stand-alone classification approaches, which are trained for all the classes together in one iteration. Therefore, for every new set of categories discovered, the entire network consisting of old and new classes requires retraining. This procedure is often impractical, considering vast volumes of data, limited resources, and the complexity of learning models. In this respect, we propose a convolutional-neural-network-based framework (called CILEA-NET, curriculum-based incremental learning framework for remote sensing image classification) to efficiently resolve the difficulties associated with incremental learning paradigm. The framework includes new classes in the already trained model to avoid catastrophic forgetting for the old while ensuring improved generalization for the newly added classes. To manage the IL's stability-plasticity dilemma, we introduce a novel curriculum learning-based approach where the order of the new classes is devised based on their similarity to the already trained classes. We then perform the training in that given order. We notice that the curriculum learning setup distinctly enhances the training time for the new classes. Experimental results on several optical datasets: PatternNet and NWPU-RESISC45, and a hyperspectral dataset: Indian Pines, validate the robustness of our technique.
•A hybrid attention network for hyperspectral image classification is proposed.•1D and 2D CNNs are used to learn spectral and spatial attention masks.•Adaptive learning is used to combine the ...attention features before classification.•A 3D CNN is used for final classification.•Classwise Wasserstein loss is added to increase the intraclass compactness.
The task of land cover classification from hyperspectral images (HSI) has recently witnessed sharp improvements with the applications of deep convolutional networks (CNN). This is mainly attributed to the hierarchical spectral-spatial feature learning capabilities of the CNN models. However, it is important to encode the short to long range spatial dependencies to predict the pixelwise labels without inducing additional redundancy at the feature level. The standard CNN based land cover classification models for HSI overlook this aspect in general. To overcome this, we propose a hybrid attention based 3D classification model for hyperspectral images. Our model comprises of 1D and 2D CNNs to individually generate the attention masks that respectively highlight the spectral and spatial characteristics of our input image. For optimal learning the spectral and spatial features in our model are combined adaptively with the original input and sent to the 3D classification module. To enhance the classification performance, we incorporate classwise Wasserstein loss alongwith the crossentropy loss. Our methods are evaluated on three widely used hyperspectral datasets: Houston datasets (DFC-2013 and DFC-2018) and Salinas dataset, and has satisfactorily outperformed all the prior benchmark methods.
Display omitted
•Dense ‘self-looping’ connections in CNNs to classify hyperspectral images.•Contains feedback connections (using shared weights) for effective information flow.•Efficient information ...flow helps in reducing parameters without affecting accuracy.•Fewer parameters allow including wider filters while maintaining computation cost.
Hyperspectral image (HSI) classification using convolutional neural networks (CNNs) has always been a hot topic in the field of remote sensing. This is owing to the high level feature extraction offered by CNNs that enables efficient encoding of the features at several stages. However, the drawback with CNNs is that for exceptional performance, they need a deeper and wider architecture along with humongous amount of training data, which is often impractical and infeasible. Furthermore, the reliance on just forward connections leads to inefficient information flow that further limits the classification. Hence, to mitigate these issues, we propose a self-looping convolution network for more efficient HSI classification. In our method, each layer in a self-looping block contains both forward and backward connections, which means that each layer is the input and the output of every other layer, thus forming a loop. These loopy connections within the network allow for maximum information flow, thereby giving us a high level feature extraction. The self-looping connections enable us to efficiently control the network parameters, further allowing us to go for a wider architecture with a multiscale setting, thus giving us abstract representation at different spatial levels. We test our method on four benchmark hyperspectral datasets: Two Houston hyperspectral datasets (DFC 2013 and DFC 2018), Salinas Valley dataset and combined Pavia University and Centre datasets, where our method achieves state of the art performance (highest percentage kappa of 87.28%, 71.08%, 99.24% and 68.44% respectively for the four datasets).
PDF
