This paper investigates the problem of change detection in multitemporal synthetic aperture radar (SAR) images. Our motivation is to avoid using a large-size dense neighborhood around each pixel to ...measure its change level, which is usually considered by classical methods in order to perform their accurate detectors. Therefore, we propose to develop a pointwise approach to detect land-cover changes between two SAR images employing the principle of signal processing on graphs. First, a set of characteristic points is extracted from one of the two images to capture the image's significant contextual information. A weighted graph is then constructed to encode the interaction among these keypoints and hence capture the local geometric structure of this first image. With regard to this graph, the coherence of the information carried by the two images is considered for measuring changes between them. In other words, the change level will depend on how much the second image still conforms to the graph structure constructed from the first image. Additionally, due to the presence of speckle noise in SAR imaging, the log-ratio operator will be exploited to perform the image comparison measure. Experimental results performed on real SAR images show the effectiveness of the proposed algorithm, in terms of detection performance and computational complexity, compared to classical methods.
Object detection from aerial and satellite remote sensing images has been an active research topic over the past decade. Thanks to the increase in computational resources and data availability, deep ...learning-based object detection methods have achieved numerous successes in computer vision, and more recently in remote sensing. However, the ability of current detectors to deal with (very) small objects still remains limited. In particular, the fast detection of small objects from a large observed scene is still an open question. In this work, we address this challenge and introduce an enhanced one-stage deep learning-based detection model, called You Only Look Once (YOLO)-fine, which is based on the structure of YOLOv3. Our detector is designed to be capable of detecting small objects with high accuracy and high speed, allowing further real-time applications within operational contexts. We also investigate its robustness to the appearance of new backgrounds in the validation set, thus tackling the issue of domain adaptation that is critical in remote sensing. Experimental studies that were conducted on both aerial and satellite benchmark datasets show some significant improvement of YOLO-fine as compared to other state-of-the art object detectors.
This article tackles the problem of detecting small objects in satellite or aerial remote sensing images by relying on super-resolution to increase image spatial resolution, thus the size and details ...of objects to be detected. We show how to improve the super-resolution framework starting from the learning of a generative adversarial network (GAN) based on residual blocks and then its integration into a cycle model. Furthermore, by adding to the framework an auxiliary network tailored for object detection, we considerably improve the learning and the quality of our final super-resolution architecture, and more importantly increase the object detection performance. Besides the improvement dedicated to the network architecture, we also focus on the training of super-resolution on target objects, leading to an object-focused approach. Furthermore, the proposed strategies do not depend on the choice of a baseline super-resolution framework, hence could be adopted for current and future state-of-the-art models. Our experimental study on small vehicle detection in remote sensing data conducted on both aerial and satellite images (i.e., ISPRS Potsdam and xView datasets) confirms the effectiveness of the improved super-resolution methods to assist with the small object detection tasks.
Deep learning methods have become an integral part of computer vision and machine learning research by providing significant improvement performed in many tasks such as classification, regression, ...and detection. These gains have been also observed in the field of remote sensing for Earth observation where most of the state-of-the-art results are now achieved by deep neural networks. However, one downside of these methods is the need for large amounts of annotated data, requiring lots of labor-intensive and expensive human efforts, in particular for specific domains that require expert knowledge such as medical imaging or remote sensing. In order to limit the requirement on data annotations, several self-supervised representation learning methods have been proposed to learn unsupervised image representations that can consequently serve for downstream tasks such as image classification, object detection or semantic segmentation. As a result, self-supervised learning approaches have been considerably adopted in the remote sensing domain within the last few years. In this article, we review the underlying principles developed by various self-supervised methods with a focus on scene classification task. We highlight the main contributions and analyze the experiments, as well as summarize the key conclusions, from each study. We then conduct extensive experiments on two public scene classification datasets to benchmark and evaluate different self-supervised models. Based on comparative results, we investigate the impact of individual augmentations when applied to remote sensing data as well as the use of self-supervised pre-training to boost the classification performance with limited number of labeled samples. We finally underline the current trends and challenges, as well as perspectives of self-supervised scene classification.
This paper introduces an extension of morphological attribute profiles (APs) by extracting their local features. The so-called local feature-based APs (LFAPs) are expected to provide a better ...characterization of each APs' filtered pixel (i.e., APs' sample) within its neighborhood, and hence better deal with local texture information from the image content. In this paper, LFAPs are constructed by extracting some simple first-order statistical features of the local patch around each APs' sample such as mean, standard deviation, and range. Then, the final feature vector characterizing each image pixel is formed by combining all local features extracted from APs of that pixel. In addition, since the self-dual APs (SDAPs) have been proved to outperform the APs in recent years, a similar process will be applied to form the local feature-based SDAPs (LFSDAPs). In order to evaluate the effectiveness of LFAPs and LFSDAPs, supervised classification using both the random forest and the support vector machine classifiers is performed on the very high resolution Reykjavik image as well as the hyperspectral Pavia University data. Experimental results show that LFAPs (respectively, LFSDAPs) can considerably improve the classification accuracy of the standard APs (respectively, SDAPs) and the recently proposed histogram-based APs.
Recently, in the computer vision and machine learning (ML) communities, a growing interest has been directed to similarity measures operating in hyperbolic spaces due to the geometric properties of ...these spaces which make them very suitable for embedding data with an underlying hierarchy. These hyperbolic spaces, although increasingly adopted, have received limited attention in the remote sensing (RS) community despite the hierarchical nature of RS data. The objective of this study is therefore to examine the relevance of hyperbolic embeddings of RS data, in particular when addressing the few-shot remote sensing scene classification problem. We adopt hyperbolic prototypical networks as a meta-learning approach to embed scene images along with a feature clipping technique to ensure a more numerically steady model. We then examine whether hyperbolic embeddings provide a better representation than Euclidean representations and better reflect the underlying structure of scene classes. Experimental results on the NWPU-RESISC45 RS dataset demonstrate the superiority of hyperbolic embeddings over their Euclidean counterparts. Our study provides a new perspective by suggesting that operating in hyperbolic spaces is an interesting alternative for the RS community.
To leverage the large amount of unlabelled data available in remote sensing datasets, self-supervised learning (SSL) methods have recently emerged as an ubiquitous tool to pre-train robust image ...encoder models from unlabelled images. However, when used in a downstream setting, these models often need to be finetuned for a specific task after their pre-training. This finetuning still requires labelling information in order to train a classifier on top of the encoder while also updating the encoder weights. In this paper, we investigate the specific task of multimodal scene classification where a sample is composed of multiple views from multiple heterogeneous satellite sensors. We propose a method to improve the categorical cross-entropy finetuning process which is often used to specify the model for this downstream task. Our approach, based on the Supervised Contrastive Learning, uses the label information available to train an image encoder in a contrastive manner from multiple modalities while also training the task-specific classifier online. Such a multimodal supervised contrastive loss helps to better align representations from samples coming from multiple sensors but having the same class labels, thus improving the performance of the finetuning process. Our experiments on two public datasets including DFC2020 and Meter-ML with Sentinel-1/Sentinel-2 images show a significant gain over the baseline multimodal cross-entropy loss. All codes and datasets will be made publicly available for reproducibility upon acceptance.
Biomaterials-associated infections (BAIs) are related to bacterial colonization on medical devices, which lead to a serious medical burden, such as increased healthcare cost, prolonged hospital ...stays, and high mortality and morbidity. To reduce the risk of infections, in this work, a new approach which makes use of a bioinspired coating with dual antimicrobial and antifouling functions was developed through rapid deposition of functional polydopamine (pDA) and antimicrobial copper ions, and subsequent conjugation of zwitterionic antifouling sulfobetaine (SB) moieties by the aza-Michael addition reaction. pDA permits surface-independent versatile functionalization on a variety of substrates, such as TiO2, SiO2, gold, plastics, and Nitinol alloy. The characterizations for chemical elemental compositions and hydrophilicity by X-ray photoelectron spectroscopy and contact angle goniometer, respectively, indicating the successful grafting of SB moieties and the presence of copper ions in the pDA adlayers. Ellipsometric thicknesses of the thin films were followed to monitor the formation of pDA films and the changes after the post conjugation. UV–vis spectroscopy and inductively coupled plasma-mass spectrometry revealed the coordination structure of catechol-Cu, and release profile of Cu2+ from the constructed functional coatings. The superhydrophilic and charge-balanced SB interface allowed effective resistance of bacterial adsorption. Intriguingly, we scrutinized that the release of bactericidal copper ions enables killing the residual amount of adsorbed bacteria. Moreover, viability tests for fibroblast cells indicate the excellent biocompatibility of the developed medical coatings. For real-world implementation, the antifouling and antimicrobial coatings were applied on commercially available silicone-based urinary catheters, and the existence of bacteria was evaluated by using the plate-counting assay. The results showed an undetectable level of living bacteria. Consequently, the dual functional medical coating offers a promising approach to eliminate BAIs for practical applications.
Human activities in the sea, such as intensive fishing and exploitation of offshore wind farms, may impact negatively on the marine mega fauna. As an attempt to control such impacts, surveying, and ...tracking of marine animals are often performed on the sites where those activities take place. Nowadays, thank to high resolution cameras and to the development of machine learning techniques, tracking of wild animals can be performed remotely and the analysis of the acquired images can be automatized using state-of-the-art object detection models. However, most state-of-the-art detection methods require lots of annotated data to provide satisfactory results. Since analyzing thousands of images acquired during a flight survey can be a cumbersome and time consuming task, we focus in this article on the weakly supervised detection of marine animals. We propose a modification of the patch distribution modeling method (PaDiM), which is currently one of the state-of-the-art approaches for anomaly detection and localization for visual industrial inspection. In order to show its effectiveness and suitability for marine animal detection, we conduct a comparative evaluation of the proposed method against the original version, as well as other state-of-the-art approaches on two high-resolution marine animal image datasets. On both tested datasets, the proposed method yielded better F1 and recall scores (75% recall/41% precision, and 57% recall/60% precision, respectively) when trained on images known to contain no object of interest. This shows a great potential of the proposed approach to speed up the marine animal discovery in new flight surveys. Additionally, such a method could be adopted for bounding box proposals to perform faster and cheaper annotation within a fully-supervised detection framework.
LiDAR point clouds are receiving a growing interest in remote sensing as they provide rich information to be used independently or together with optical data sources, such as aerial imagery. However, ...their nonstructured and sparse nature make them difficult to handle, conversely to raw imagery for which many efficient tools are available. To overcome this specific nature of LiDAR point clouds, the standard approach relies on converting the point cloud into a digital elevation model, represented as a 2-D raster. Such a raster can then be used similarly as optical images, e.g., with 2-D convolutional neural networks (CNNs) for semantic segmentation. In this letter, we show that LiDAR point clouds provide more information than only the digital elevation model and that considering alternative rasterization strategies helps to achieve better semantic segmentation results. We illustrate our findings on the IEEE Data Fusion Contest (DFC) 2018 data set.
PDF
