Single image superresolution (SR) aims to construct a high-resolution version from a single low-resolution (LR) image. The SR reconstruction is challenging because of the missing details in the given ...LR image. Thus, it is critical to explore and exploit effective prior knowledge for boosting the reconstruction performance. In this paper, we propose a novel SR method by exploiting both the directional group sparsity of the image gradients and the directional features in similarity weight estimation. The proposed SR approach is based on two observations: 1) most of the sharp edges are oriented in a limited number of directions and 2) an image pixel can be estimated by the weighted averaging of its neighbors. In consideration of these observations, we apply the curvelet transform to extract directional features which are then used for region selection and weight estimation. A combined total variation regularizer is presented which assumes that the gradients in natural images have a straightforward group sparsity structure. In addition, a directional nonlocal means regularization term takes pixel values and directional information into account to suppress unwanted artifacts. By assembling the designed regularization terms, we solve the SR problem of an energy function with minimal reconstruction error by applying a framework of templates for first-order conic solvers. The thorough quantitative and qualitative results in terms of peak signal-to-noise ratio, structural similarity, information fidelity criterion, and preference matrix demonstrate that the proposed approach achieves higher quality SR reconstruction than the state-of-the-art algorithms.
Inshore ship detection from high-resolution satellite images is a useful yet challenging task in remote surveillance and military reconnaissance. It is difficult to detect the inshore ships with high ...precision because various interferences are present in the harbor scene. An inshore ship detection method based on the weighted voting and rotation-scale-invariant pose is proposed to improve the detection performance. The proposed method defines the rotation angle pose and the scaling factor of the detected ship to detect the ship with different directions and different sizes. For each pixel on the ship template, the possible poses of a detection window are estimated according to all possible pose-related pixels. To improve robustness to the shape-similar distractor and various interferences, the score of the detection window is obtained by designing a pose weighted voting method. Moreover, the values of some parameters such as similarity threshold and the weight of "V" are investigated. The experimental results on actual satellite images demonstrate that the proposed method is invariant to rotation and scale and robust in the inshore ship detection. In addition, better detection performance is observed in comparison with the existing inshore ship detection algorithms in terms of precision rate and recall rate. The target pose of the detected ship can also be obtained as a byproduct of the ship detection.
Visual Simultaneous Localization and Mapping (V-SLAM) plays a crucial role in the development of intelligent robotics and autonomous navigation systems. However, it still faces significant challenges ...in handling highly dynamic environments. The prevalent method currently used for dynamic object recognition in the environment is deep learning. However, models such as Yolov5 and Mask R-CNN require significant computational resources, which limits their potential in real-time applications due to hardware and time constraints. To overcome this limitation, this paper proposes ADM-SLAM, a visual SLAM system designed for dynamic environments that builds upon the ORB-SLAM2. This system integrates efficient adaptive feature point homogenization extraction, lightweight deep learning semantic segmentation based on an improved DeepLabv3, and multi-view geometric segmentation. It optimizes keyframe extraction, segments potential dynamic objects using contextual information with the semantic segmentation network, and detects the motion states of dynamic objects using multi-view geometric methods, thereby eliminating dynamic interference points. The results indicate that ADM-SLAM outperforms ORB-SLAM2 in dynamic environments, especially in high-dynamic scenes, where it achieves up to a 97% reduction in Absolute Trajectory Error (ATE). In various highly dynamic test sequences, ADM-SLAM outperforms DS-SLAM and DynaSLAM in terms of real-time performance and accuracy, proving its excellent adaptability.
•Comparison of 12 loss functions for road segmentation based on deep learning.•Performances of the deep learning model differs between the loss functions.•Consistent differences in model performances ...across 2 remote sensing road datasets.
Road extraction from remote sensing imagery is a fundamental task in the field of image semantic segmentation. For this goal, numerous supervised deep learning techniques have been created, along with the employment of various loss functions that play a crucial role in determining the performances of supervised learning models. However, there is a lack of comprehensive analysis of the performance differences between the loss functions for road segmentation in remote sensing imagery. Therefore, this study conducts a comparative study of 12 well-known loss functions used widely in the field of image segmentation by training and evaluating the representative D-LinkNet network for road segmentation tasks with two publicly available remote sensing road datasets consisting of very high-resolution aerial and satellite images. The results show that different loss functions could lead to very different outcomes using the D-LinkNet, with varying focuses such as on overall model performances, precision, or recall. By dividing the loss functions into the distribution-based, region-based, and compound ones, we found that the region-based loss function type led to generally better model performances than the distribution-based one in terms of F1, IoU, and the road segmentation maps, with the compound loss function type being comparable to the region-based one. This paper eventually tries to offer suggestions for choosing the loss function that best suits the purposes of road segmentation-related studies.
•A new CNN based instance segmentation method.•The first study for instance segmentation-based oil well sites identification and extraction.•A complete framework including data fusion, object ...detection and extraction, and postprocessing for oil well sites for the task.
Fine-scale land disturbances due to mining development modify the land surface cover and have cumulative detrimental impacts on the environment. Understanding the distribution of fine-scale land disturbances related to mining activities, such as oil well sites, in mining regions is of vital importance to sustainable mining development. For efficient mapping, automated identification and extraction of the oil well sites using high-resolution satellite images are required. In this work, we proposed the Oil Well Site extraction (OWS) Mask R-CNN based on the original Mask R-CNN (Region-based Convolutional Neural Networks), to accurately extract well sites using multi-sensor remote sensing images. For improvement of mapping efficiency, two modifications were made to Mask R-CNN: (1) replacing the backbone of Mask R-CNN with D-LinkNet, and (2) adding a semantic segmentation branch to Mask R-CNN to force the whole network to focus on the relationship between line objects and oil well sites. As imagery data were from multiple sensors (RapidEye 2/3 and WorldView 3), a pre-trained Residual Channel Attention Network (RCAN) was applied to super-resolve the images with different resolutions. Several key spatial features, such as nearby roads and area size, have also been used in the oil well site mapping process. The experimental results indicate that our OWS Mask R-CNN considerably improves the average precision (AP) and the F1 score of Mask R-CNN from 51.26% and 25.7% to 60.93% and 61.59%, respectively.
•Developing three novel deep learning frameworks for TWS reconstruction.•Learning statistical relationships between LSM-simulated TWSA and GRACE-observed TWSA.•Conducting a comprehensive comparative ...study for GARCE-based TWS reconstruction.•Indicating the superiority of the proposed LSM-based DL model in complex study areas.
Terrestrial water storage (TWS) is an essential part of the global water cycle. Long-term information of observed and modeled TWS is fundamental to analyze water resources, meteorological extreme events (e.g., droughts and floods), and the climate change impacts. Over the past several decades, hydrologists have been applying physically-based hydrological model (GHM) and land surface model (LSM) to simulate TWS and its components (e.g., groundwater storage). However, the reliability of these physically-based models is often affected by uncertainties in climatic forcing data, model parameters, model structure, and mechanisms for physical process representations. Launched in March 2002, the Gravity Recovery and Climate Experiment (GRACE) satellite mission exclusively applies remote sensing techniques to measure the variations in TWS on a global scale. The mission length of GRACE, however, is too short to meet the requirements for analyzing long-term TWS. Therefore, lots of effort have been devoted to the reconstruction of GRACE-like TWS data for the pre-GRACE era. Data-driven methods, such as multilinear regression and machine learning, exhibit a great potential to reconstruct TWS data by integrating GRACE observations and physically-based model simulations. The advances in artificial intelligence enable adaptive learning of correlations between variables in complex spatiotemporal systems. However, the applicability of various deep learning techniques has not been adequately studied for GRACE TWS reconstruction. In this study, three deep learning-based models are developed to reconstruct the historical TWS using LSM outputs for the Canadian landmass from 1979 to 2002. The performance of the models is evaluated against the GRACE-observed TWS in 2002–2004 and 2014–2016. The trained models achieve a mean correlation coefficient of 0.96, with a mean RMSE of 53 mm. The results show that the LSM-based deep learning models significantly improve the correlations between original LSM simulations and GRACE observations.
This paper proposes a color-direction patch-sparsity-based image inpainting method to better maintain structure coherence, texture clarity, and neighborhood consistence of the inpainted region of an ...image. The method uses super-wavelet transform to estimate the multi-direction features of a degraded image, and combines with color information to construct the weighted color-direction distance (WCDD) to measure the difference between two patches. Based on the WCDD, the color-direction structure sparsity is defined to obtain a more robust filling order and more suitable multiple candidate patches are searched. Then, the target patches are sparsely represented by the multiple candidate patches under neighborhood consistency constraints in both the color and the multi-direction spaces. Experimental results are presented to demonstrate the effectiveness of the proposed approach on tasks such as scratch removal, text removal, block removal, and object removal. The effects of super-wavelet transforms and direction features are also investigated.
In order to improve the security of reversible data hiding algorithm in encrypted images, Liu designed an image multi-permutation encryption algorithm. By combining bit-plane permutation with ...pixel-block permutation, the embedding capacity of the algorithm is improved, and the ability of the encryption algorithm to resist existing Ciphertext-Only and Known-Plaintext attack is effectively improved. To analyze the security performance of the encryption algorithm, a Known-plaintext attack method based on the Root Mean Square(RMS) of image block is proposed in this paper. Firstly, the permutation order of bit plane is estimated by using the invariant distribution ratio of 0 and 1 before and after image encryption, the original pixel values is restored. Then, according to the characteristics that the pixel values of the image remain unchanged during block permutation and intra-block pixel permutation, the image block RMS feature is defined to search and estimate the block permutation sequence. Image block RMS equivalence class is defined by the image block RMS value, the ratio of the maximum number of elements in the equivalence class to the total number of pixels is
p
, which determines the difficulty of known plaintext attack. For the common image test set, 96% of the images have a
p
-value of less than 0.1, and only 4% of the images have a
p
-value greater than 0.1. Experimental results show that, When
p
<0.1, under the condition of 2 × 2 block size, the attacker only needs to know one pair of plaintext-ciphertext to decode about 50% of the encrypted image content. For a few images with
p
> 0.1, the attacker can crack more than 80% of the image content by two pairs of plaintext and ciphertext.
•An efficiency-based lithium-ion battery scraping criterion for peak-shaving service.•A new battery degradation model considering scraping parameters.•Embedding the battery degradation model in ...energy storage operation optimization.•Benefit comparison between efficiency-based and capacity-based scraping criterion.
Lithium-ion battery systems have been used in practical power systems for peak-shaving and demand response. However, a lithium-ion battery is degrading while cycling and would be scrapped when the capacity reduces to a certain experience-based threshold (e.g. 80%). The scrapping criterion could significantly affect battery lifetime and operation scheduling for grid application. Thus, such scraping criterion may not explore the maximum benefit from a new or re-used lithium-ion battery. In this paper, we propose a novel efficiency-based scrapping criterion for peak-shaving energy storage. This criterion is suitable for both new and re-used battery in grid application. Using the scraping criterion as a parameter, the maximum cycle number of battery is derived as a function of the depth of discharge, which makes the life model easy to embed in battery operation model. The results of the case study show that the operation method could maximize the benefit of peak-shaving energy storage while delaying battery degradation. Compared with the traditional 80% capacity-based scrapping criterion, our efficiency-based scrapping criterion can significantly improve the lifetime benefit of the battery by 97%.
•Application for multimodality satellite image registration by a coarse-to-fine matching strategy.•A dual-branch network with self-attention for providing a larger range of receptive ...field.•Cross-fusion module to the interaction of information between reference and sensed images.
The registration of multimodal satellite images is essential for a prerequisite for accruing complementary observational data. Nevertheless, the differential imaging nuances amongst non-linear radiometric multimodal images precipitate a complexity in keypoint detection, rendering it a great challenge. This complexity exacerbates the difficulty encountered in matching multimodal satellite images. In this paper, a dual-branch cross fusion network (DF-Net) is proposed for the purpose of satellite image registration. DF-Net relies on the self-attention granted to a pair of images, thereby providing cross-modal fusion feature descriptions. Initially, reference and sensed images are deployed as inputs for the dual-branch network, which in turn engenders feature descriptions of both high and low resolution, respectively. Sequentially, the matching of individual feature descriptions is anchored on the low-resolution feature map, paving the way for the establishment of coarse matching correspondences. Subsequently, the outcomes of these coarse correspondences are transposed onto the feature map with a higher resolution, thereby generating fine matching results for each coarse correspondence. An exhaustive set of qualitative and quantitative assessments have been administered on three satellite image datasets encompassing a diverse range of scenarios. The average Repeatability (Rep.), Mean Matching Accuracy (MMA), and Root-Mean-Square Error (RMSE) of the DF-Net applied to three large-scale satellite images were recorded to be 0.71, 0.65, and 2.34, respectively. These findings buttress the proficiency of the proposed strategy in facilitating cross-modal matching and bear testimony to the sterling performance of the method proposed.
PDF
