In high resolution remote sensing imagery (HRI), the sizes of different geo-objects often vary greatly, posing serious difficulties to their successful segmentation. Although existent segmentation ...approaches have provided some solutions to this problem, the complexity of HRI may still lead to great challenges for previous methods. In order to further enhance the quality of HRI segmentation, this paper proposes a new segmentation algorithm based on scale-variable region merging. Scale-variable means that the scale parameters (SP) adopted for segmentation are adaptively estimated, so that geo-objects of various sizes can be better segmented out. To implement the proposed technique, 3 steps are designed. The first step produces a coarse-segmentation result with slight degree of under segmentation error. This is achieved by segmenting a half size image with the global optimal SP. Such a SP is determined by using the image of original size. In the second step, structural and spatial contextual information is extracted from the coarse-segmentation, enabling the estimation of variable SPs. In the last step, a region merging process is initiated, and the SPs used to terminate this process are estimated based on the information obtained in the second step. The proposed method was tested by using 3 scenes of HRI with different landscape patterns. Experimental results indicated that our approach produced good segmentation accuracy, outperforming some competitive methods in comparison.
Synthetic aperture radar (SAR) is playing an important role in maritime domain awareness. As a fundamental ocean mission, SAR ship detection can offer high-quality services for marine surveillance. ...It receives widespread concern in the SAR remote sensing community. Recently, deep learning (DL) has greatly improved SAR ship detection accuracy. However, there are still some unperceived imbalance problems that seriously hinder further accuracy improvements among current DL-based SAR ship detectors. Therefore, we propose a novel concept of balance learning (BL) for high-quality SAR ship detection. We first point out the four unperceived imbalance problems, i.e., image sample scene imbalance, positive negative sample imbalance, ship scale feature imbalance, and classification regression task imbalance. Then, we offer some profound insights into these imbalances. Immediately, we propose four effective solutions to handle the above four imbalances correspondingly, i.e., balance scene learning mechanism (BSLM), balance interval sampling mechanism (BISM), balance feature pyramid network (BFPN), and balance classification regression network (BCRN). Finally, combined with the four solutions, a novel balance learning network (BL-Net) is proposed. Ablation studies can confirm each solution’s effectiveness. Experimental results on five open datasets (SSDD, Gaofen-SSDD, Sentinel-SSDD, SAR-Ship-Dataset, and HRSID) reveal BL-Net’s state-of-the-art SAR ship detection performance compared to the other 30 DL-based SAR ship detectors. Specifically, in contrast to the current most competitive method, the accuracy increase of BL-Net is 2.98% on SSDD, 1.97% on Gaofen-SSDD, 1.49% on Sentinel-SSDD, 0.55% on SAR-Ship-Dataset, and 4.95% on HRSID. Last but not least, the satisfactory ship detection results on another two large-scene Sentinal-1 SAR images confirm BL-Net’s strong migration capability. This indicates BL-Net’s potential value in marine surveillance.
Extracting building footprints from remotely sensed imagery has long been a challenging task and is not yet fully solved. Obstructions from nearby shadows or trees, varying shapes of rooftops, ...omission of small buildings, and varying scale of buildings hinder existing automated models for extracting sharp building boundaries. Different reasons account for these challenges. In convolutional neural network-based methods, the down-sampling operation loses spatial details of the input images; and small buildings are omitted from the high-level features. The sheltering trees and adjacent objects shadowing may cause errors since semantic information cannot be effectively preserved. Moreover, the insufficient use of multi-scale building features causes blurry edges in the predictions for buildings with complex shapes. To address these challenges, we propose a novel coarse-to-fine boundary refinement network (CBR-Net) that accurately extracts building footprints from remote sensing imagery. Unlike the existing semantic segmentation methods that directly generate building predictions at the highest level, we designed a module that progressively refines the building prediction in a coarse-to-fine manner. In this way, the advantages of both the high-level and low-level features can be retained. We also present a novel boundary refinement (BR) module that enhances the ability of the CBR-Net model to perceive and refine building edges. The BR module refines building prediction by perceiving the direction of each pixel in a remotely sensed optical image to the center of the nearest object to which it might belong. The refined results are used as pseudo labels in a self-supervision process that increases model robustness to noisy labels or obstructions. Experimental results on three public building datasets, including the WHU building dataset, the Massachusetts building dataset, and the Inria aerial image dataset, demonstrate the effectiveness of the proposed method. In evaluation tests, CBR-Net outperformed other state-of-the-art algorithms on the three datasets by maintaining both the continuous entities and accurate boundaries of buildings. The source code of the proposed CBR-Net is available at https://github.com/HaonanGuo/CBRNet.
In object-based image analysis, how to produce accurate segmentation is usually a very important issue that needs to be solved before image classification or target recognition. The study for ...segmentation evaluation method is key to solving this issue. Almost all of the existent evaluation strategies only focus on the global performance assessment. However, these methods are ineffective for the situation that two segmentation results with very similar overall performance have very different local error distributions. To overcome this problem, this paper presents an approach that can both locally and globally quantify segmentation incorrectness. In doing so, region-overlapping metrics are utilized to quantify each reference geo-object's over and under-segmentation error. These quantified error values are used to produce segmentation error maps which have effective illustrative power to delineate local segmentation error patterns. The error values for all of the reference geo-objects are aggregated through using area-weighted summation, so that global indicators can be derived. An experiment using two scenes of very different high resolution images showed that the global evaluation part of the proposed approach was almost as effective as other two global evaluation methods, and the local part was a useful complement to comparing different segmentation results.
With the increasing popularity of OBIA, many scholars advocate that image segmentation plays a significant role in remote sensing image processing. Numerous segmentation algorithms for remote sensing ...images are based on region merging. Although good improvement is achieved, their accuracy is still dependent on parameter settings, leading to a low level of automation. To overcome this issue, this work proposes a new region merging method by using a random forest (RF) classifier. Unlike the traditional region merging methods that all adopt a scale threshold to determine whether a merging can be conducted, the new algorithm relies on a trained RF to decide the result of a merging test. Various merging criteria are simultaneously employed as feature variables of the RF model, enhancing the quality of the proposed scheme. The major problem in this work is how to train the RF classifier since the merging test samples need to be obtained in the iterative steps of a region merging process, which involves a huge number of human–computer interactions even for a small image. To simplify it, a sample collection strategy based on a set of three-scale segmentation results is devised. Representative merging test samples can be obtained by using this method. To validate the proposed technique, four Gaofen-2 images are used for training sample collection, and the most interesting result is that the samples extracted from one image can apply to others. Some images captured by Orbview-3, GeoEye-1, and Worldview-2 further indicate the robust performance of the new algorithm and the samples acquired in this work.
Digital elevation models (DEMs) are essential to various applications in topography, geomorphology, hydrology, and ecology. The Shuttle Radar Topographic Mission (SRTM) DEM data set is one of the ...most complete and most widely used DEM data sets; it provides accurate information on elevations over bare land areas. However, the accuracy of SRTM data over vegetated mountain areas is relatively low as a result of the high relief and the penetration limitation of the C-band used for obtaining global DEM products. The objective of this study is to assess the performance of SRTM DEMs and correct them over vegetated mountain areas with small-footprint airborne Light Detection and Ranging (Lidar) data, which can develop elevation products and vegetation products e.g., vegetation height, Leaf Area Index (LAI) of high accuracy. The assessing results show that SRTM elevations are systematically higher than those of the actual land surfaces over vegetated mountain areas. The mean difference between SRTM DEM and Lidar DEM increases with vegetation height, whereas the standard deviation of the difference increases with slope. To improve the accuracy of SRTM DEM over vegetated mountain areas, a regression model between the SRTM elevation bias and vegetation height, LAI, and slope was developed based on one control site. Without changing any coefficients, this model was proved to be applicable in all the nine study sites, which have various topography and vegetation conditions. The mean bias of the corrected SRTM DEM at the nine study sites using this model (absolute value) is 89% smaller than that of the original SRTM DEM, and the standard deviation of the corrected SRTM elevation bias is 11% smaller.
Wetland contains various ground objects with high spectral similarity. How to accurately distinguish complex classes has become a challenge in wetland land cover classification. In this paper, ...low-rank representation with elastic net (ENLRR) and the kernel version of ENLRR (KENLRR) are proposed for coastal wetland land cover classification by using Gaofen-5 (GF-5) hyperspectral data of China. The main idea of ENLRR is to combine elastic net with low-rank representation, which replaces rank function with the combination of nuclear norm and Frobenius norm when constraining the coefficient matrix. The KENLRR method considers nonlinear characteristics of hyperspectral data, where a neighborhood filter (NF) kernel function is adopted to map the original data space into a higher dimensional feature space for better classification. In the experiments, three typical coastal wetlands in China: Yellow River Delta, Jiangsu Dafeng Natural Reserve, and Yangtze River Delta (Nantong) are adopted, and the proposed methods and seven comparison methods are used to conduct wetland land cover classification. The experimental results demonstrate that the proposed ENLRR and KENLRR are effective in accurately distinguishing wetland ground objects and reliably mapping their distribution. More specifically, the KENLRR method can provide the best performance, and the OAs are 96.63%, 96.76% and 87.67% for the three wetlands, respectively. The land cover distributions and spatial patterns of the three wetlands are studied as well. Yellow River Delta is a typical estuarine wetland with abundant landscapes, Dafeng Nature Reserve is a coastal wetland with the block regular feature distribution in spatial, and Yangtze River Delta (Nantong) mainly includes river and flood plain, whose ecological environment is deeply affected by human activities.
Unmanned Aerial Vehicle (UAV) has seen a dramatic rise in popularity for remote-sensing image acquisition and analysis in recent years. It has brought promising results in low-altitude monitoring ...tasks that require detailed visual inspections. Semantic segmentation is one of the hot topics in UAV remote sensing image analysis, as its capability to mine contextual semantic information from UAV images is crucial for achieving a fine-grained understanding of scenes. However, in the remote sensing field, recent reviews have not focused on combining “UAV remote sensing” and “semantic segmentation” to summarize the advanced works and future trends. In this study, we focus primarily on describing various recent semantic segmentation methods applied in UAV remote sensing images and summarizing their advantages and limitations. According to the distinction in modeling contextual semantic information, we have categorized and outlined the methods based on graph-based contextual models and deep-learning-based models. Publicly available UAV-based image datasets are also gathered to encourage systematic research on advanced semantic segmentation methods. We provide quantitative results of representative methods on two high-resolution UAV-based image datasets for fair comparisons and discussions in terms of semantic segmentation accuracy and model inference efficiency. Besides, this paper concludes some remaining challenges and future directions in semantic segmentation for UAV remote sensing images and points out that methods based on deep learning will become the future research trend.
•Summarize semantic segmentation methods and datasets for UAV remote sensing images.•Provide fair comparisons of representative methods in accuracy and efficiency.•Identify challenges and future directions for UAV image semantic segmentation.•Semantic segmentation methods based on deep learning will be the mainstream.
The aim of geometric matching is to extract the geometric transformation parameters between the corresponding images. That is useful for photogrammetric mapping, deformation detection, and flying ...platform's posture analyses, etc. It is different compare with ordinary feature based image matching succeed by selecting feature points correctly, the proposed method takes all the pixels within the corresponding images to participate the matching procedure for calculating the geometric parameters by least square criterion. The principle of the algorithm, such as the gray corresponding equation, the information quantity inequation and procedure of least square solution are introduced in detail. Particularly, the wavelet analyses for gray signal and calculating the information quantity by signal to noise ratio. Finally, a series of sequential images obtained by a low-altitude helicopter equipped with a video camera was used to test and verify the validity and reliability of the theory and algorithm in this paper. Two typical results are got according to the relative orientation elements model and parallax grid model. The conclusion is got in comparing APM with ordinary feature point method by the information quantity inequation.
PDF