In this paper, we propose a new no-reference (NR)/ blind sharpness metric in the autoregressive (AR) parameter space. Our model is established via the analysis of AR model parameters, first ...calculating the energy- and contrast-differences in the locally estimated AR coefficients in a pointwise way, and then quantifying the image sharpness with percentile pooling to predict the overall score. In addition to the luminance domain, we further consider the inevitable effect of color information on visual perception to sharpness and thereby extend the above model to the widely used YIQ color space. Validation of our technique is conducted on the subsets with blurring artifacts from four large-scale image databases (LIVE, TID2008, CSIQ, and TID2013). Experimental results confirm the superiority and efficiency of our method over existing NR algorithms, the state-of-the-art blind sharpness/blurriness estimators, and classical full-reference quality evaluators. Furthermore, the proposed metric can be also extended to stereoscopic images based on binocular rivalry, and attains remarkably high performance on LIVE3D-I and LIVE3D-II databases.
Learning algorithms motivated by minimizing the sharpness of loss surface is a hot research topic in improving generalization. The existing methods usually solve a constrained min–max problem to ...minimize sharpness and find flat minima. However, most constraints (i.e., the neighborhood of the sharpness) are inappropriate, leading to sub-optimal results. This paper theoretically explores the optimal neighborhood from the view of Probably Approximately Correct-Bayesian (PAC-Bayesian) framework. A closed form of the optimal neighborhood is provided. This neighborhood is determined by the Hessian matrix and the scales of parameters. Then a generalization bound is derived that serves as a guiding principle in the design of the sharpness minimization algorithm. The Dynamic neighborhood-based Sharpness-Aware Minimization algorithm is proposed, which can adaptively adjust the neighborhood during the training process to gain better performance. Also, the algorithm is proved can convergent at the rate O(logT/T). Experimental results demonstrate that the proposed algorithm outperforms the other methods (e.g., accuracy ＋2.86% over baseline on CIFAR-100 for VGG-16).
•A novel sharpness minimization algorithm is proposed to find flat minima.•The method can adaptively adjust the neighborhood of sharpness during the training.•Convergence analyses are deduced to guarantee the feasibility of the algorithm.•The algorithm outperforms other state-of-the-art baselines.•A new insight into the generalization theory and optimization algorithms is provided.
Full text
Available for:
GEOZS, IJS, IMTLJ, KILJ, KISLJ, NLZOH, NUK, OILJ, PNG, SAZU, SBCE, SBJE, UILJ, UL, UM, UPCLJ, UPUK, ZAGLJ
Video deblurring is a challenging task because only input blurry sequences are available. To further constrain the optimization process, existing methods explore various additional information, e.g., ...events, depth, and sharpness prior. However, they consume large computational costs or generate unpleasant visual results due to the insufficient exploitation of spatio-temporal information. In this work, we propose a novel spatio-temporal sharpness map learned by a prior-based generation network implicitly. The proposed generation network blends both spatial and temporal sharpness priors in a blurry sequence, while few extra parameters are added. We show that the proposed map has better spatial continuity and guidance for video deblurring than the previous methods. Furthermore, different from the simply concatenation in the previous work, we allow the sharpness map to accommodate to more effective video deblurring via a dual-stream network. Specifically, the network is decomposed by two branches, namely the inter-frame and intra-frame reconstructions. The inter-frame reconstruction obtains the sharp patches of consecutive frames from the sharpness map to restore textures well. Meanwhile, the intra-frame branch is responsible for recovering structures of the latent frame, where a novel histogram statistical method is developed to quantify and count textures in the features under the modulation of the sharpness map. Quantitative and qualitative experiments successfully validate the effectiveness of our proposed method.
Single image superresolution is a classic and active image processing problem, which aims to generate a high-resolution (HR) image from a low-resolution input image. Due to the severely ...under-determined nature of this problem, an effective image prior is necessary to make the problem solvable, and to improve the quality of generated images. In this paper, a novel image superresolution algorithm is proposed based on gradient profile sharpness (GPS). GPS is an edge sharpness metric, which is extracted from two gradient description models, i.e., a triangle model and a Gaussian mixture model for the description of different kinds of gradient profiles. Then, the transformation relationship of GPSs in different image resolutions is studied statistically, and the parameter of the relationship is estimated automatically. Based on the estimated GPS transformation relationship, two gradient profile transformation models are proposed for two profile description models, which can keep profile shape and profile gradient magnitude sum consistent during profile transformation. Finally, the target gradient field of HR image is generated from the transformed gradient profiles, which is added as the image prior in HR image reconstruction model. Extensive experiments are conducted to evaluate the proposed algorithm in subjective visual effect, objective quality, and computation time. The experimental results demonstrate that the proposed approach can generate superior HR images with better visual quality, lower reconstruction error, and acceptable computation efficiency as compared with state-of-the-art works.
Free viewpoint video (FVV) has received considerable attention owing to its widespread applications in several areas such as immersive entertainment, remote surveillance and distanced education. ...Since FVV images are synthesized via a depth image-based rendering (DIBR) procedure in the "blind" environment (without reference images), a real-time and reliable blind quality assessment metric is urgently required. However, the existing image quality assessment metrics are insensitive to the geometric distortions engendered by DIBR. In this research, a novel blind method of DIBR-synthesized images is proposed based on measuring geometric distortion, global sharpness and image complexity. First, a DIBR-synthesized image is decomposed into wavelet subbands by using discrete wavelet transform. Then, the Canny operator is employed to detect the edges of the binarized low-frequency subband and high-frequency subbands. The edge similarities between the binarized low-frequency subband and high-frequency subbands are further computed to quantify geometric distortions in DIBR-synthesized images. Second, the log-energies of wavelet subbands are calculated to evaluate global sharpness in DIBR-synthesized images. Third, a hybrid filter combining the autoregressive and bilateral filters is adopted to compute image complexity. Finally, the overall quality score is derived to normalize geometric distortion and global sharpness by the image complexity. Experiments show that our proposed quality method is superior to the competing reference-free state-of-the-art DIBR-synthesized image quality models.
In this letter, we present a simple, yet effective wavelet-based algorithm for estimating both global and local image sharpness (FISH, Fast Image Sharpness). FISH operates by first decomposing the ...input image via a three-level separable discrete wavelet transform (DWT). Next, the log-energies of the DWT subbands are computed. Finally, a scalar index corresponding to the image's overall sharpness is computed via a weighted average of these log-energies. Testing on several image databases demonstrates that, despite its simplicity, FISH is competitive with the currently best-performing techniques both for sharpness estimation and for no-reference image quality assessment.
The human visual system exhibits multiscale characteristic when perceiving visual scenes. The hierarchical structures of an image are contained in its scale space representation, in which the image ...can be portrayed by a series of increasingly smoothed images. Inspired by this, this paper presents a no-reference and robust image sharpness evaluation (RISE) method by learning multiscale features extracted in both the spatial and spectral domains. For an image, the scale space is first built. Then sharpness-aware features are extracted in gradient domain and singular value decomposition domain, respectively. In order to take into account the impact of viewing distance on image quality, the input image is also down-sampled by several times, and the DCT-domain entropies are calculated as quality features. Finally, all features are utilized to learn a support vector regression model for sharpness prediction. Extensive experiments are conducted on four synthetically and two real blurred image databases. The experimental results demonstrate that the proposed RISE metric is superior to the relevant state-of-the-art methods for evaluating both synthetic and real blurring. Furthermore, the proposed metric is robust, which means that it has very good generalization ability.
This paper presents a no-reference image blur metric that is based on the study of human blur perception for varying contrast values. The metric utilizes a probabilistic model to estimate the ...probability of detecting blur at each edge in the image, and then the information is pooled by computing the cumulative probability of blur detection (CPBD). The performance of the metric is demonstated by comparing it with existing no-reference sharpness/blurriness metrics for various publicly available image databases.
Wheat rust is one of the important factors leading to wheat yield decline. The traditional method of artificial identification of wheat rust remains inefficient. With the development of unmanned ...technology, unmanned aerial vehicles (UAVs) with advanced deep vision models can be used to monitor wheat growth, hence enabling timely disease detection and deployment of corresponding treatment measures. However, due to the limitation of embedded system hardware, it is a challenge to deploy a high‐accuracy and lightweight wheat rust detection model in an embedded system. In this paper, a training method based on transfer learning and sharpness‐aware minimization (SAM) is proposed to improve the accuracy of lightweight models. Specifically, the initial model is pretrained on an ImageNet dataset, and then the classifier of the model is fine‐tuned on the wheat rust training set; to prevent the risk of overfitting, the SAM method is applied to adjust the global parameters of the model. The experimental results on the Yellow‐Rust‐19 dataset show that the training method can effectively improve the accuracy of the wheat rust detection model, which is 3.56% higher than the existing methods. In addition, the parameter scale and computing resource requirements of four lightweight models are compared. The results indicate that MobileNetV3‐Small can achieve satisfactory accuracy with low requirements for storage and computing resource, and a detection frame rate of 101.36 frames per second in the Raspberry Pi, which is suitable for the unmanned detection of wheat rust.
Lightweight models of wheat rust detection were trained with transfer learning and SAM approaches then deployed in the embedded system. The MobileNetV3‐Small model achieved the highest detection FPS with satisfactory accuracy.
Full text
Available for:
BFBNIB, FZAB, GIS, IJS, KILJ, NLZOH, NUK, OILJ, SBCE, SBMB, UL, UM, UPUK
An ideal anti-counterfeiting technique has to be inexpensive, mass-producible, nondestructive, unclonable and convenient for authentication. Although many anti-counterfeiting technologies have been ...developed, very few of them fulfill all the above requirements. Here we report a non-destructive, inkjet-printable, artificial intelligence (AI)-decodable and unclonable security label. The stochastic pinning points at the three-phase contact line of the ink droplets is crucial for the successful inkjet printing of the unclonable security labels. Upon the solvent evaporation, the three-phase contact lines are pinned around the pinning points, where the quantum dots in the ink droplets deposited on, forming physically unclonable flower-like patterns. By utilizing the RGB emission quantum dots, full-color fluorescence security labels can be produced. A convenient and reliable AI-based authentication strategy is developed, allowing for the fast authentication of the covert, unclonable flower-like dot patterns with different sharpness, brightness, rotations, amplifications and the mixture of these parameters.
PDF