Perceptual quality assessment plays a vital role in the visual communication systems owing to the existence of quality degradations introduced in various stages of visual signal acquisition, ...compression, transmission and display. Quality assessment for visual signals can be performed subjectively and objectively, and objective quality assessment is usually preferred owing to its high efficiency and easy deployment. A large number of subjective and objective visual quality assessment studies have been conducted during recent years. In this survey, we give an up-to-date and comprehensive review of these studies. Specifically, the frequently used subjective image quality assessment databases are first reviewed, as they serve as the validation set for the objective measures. Second, the objective image quality assessment measures are classified and reviewed according to the applications and the methodologies utilized in the quality measures. Third, the performances of the state-of-the-art quality measures for visual signals are compared with an introduction of the evaluation protocols. This survey provides a general overview of classical algorithms and recent progresses in the field of perceptual image quality assessment.
Automatically learned quality assessment for images has recently become a hot topic due to its usefulness in a wide variety of applications, such as evaluating image capture pipelines, storage ...techniques, and sharing media. Despite the subjective nature of this problem, most existing methods only predict the mean opinion score provided by data sets, such as AVA and TID2013. Our approach differs from others in that we predict the distribution of human opinion scores using a convolutional neural network. Our architecture also has the advantage of being significantly simpler than other methods with comparable performance. Our proposed approach relies on the success (and retraining) of proven, state-of-the-art deep object recognition networks. Our resulting network can be used to not only score images reliably and with high correlation to human perception, but also to assist with adaptation and optimization of photo editing/enhancement algorithms in a photographic pipeline. All this is done without need for a "golden" reference image, consequently allowing for single-image, semantic- and perceptually-aware, no-reference quality assessment.
Deep learning methods for image quality assessment (IQA) are limited due to the small size of existing datasets. Extensive datasets require substantial resources both for generating publishable ...content and annotating it accurately. We present a systematic and scalable approach to creating KonIQ-10k, the largest IQA dataset to date, consisting of 10,073 quality scored images. It is the first in-the-wild database aiming for ecological validity, concerning the authenticity of distortions, the diversity of content, and quality-related indicators. Through the use of crowdsourcing, we obtained 1.2 million reliable quality ratings from 1,459 crowd workers, paving the way for more general IQA models. We propose a novel, deep learning model (KonCept512), to show an excellent generalization beyond the test set (0.921 SROCC), to the current state-of-the-art database LIVE-in-the-Wild (0.825 SROCC). The model derives its core performance from the InceptionResNet architecture, being trained at a higher resolution than previous models (512 × 384 ). Correlation analysis shows that KonCept512 performs similar to having 9 subjective scores for each test image.
Recent years have witnessed an explosion of user-generated content (UGC) videos shared and streamed over the Internet, thanks to the evolution of affordable and reliable consumer capture devices, and ...the tremendous popularity of social media platforms. Accordingly, there is a great need for accurate video quality assessment (VQA) models for UGC/consumer videos to monitor, control, and optimize this vast content. Blind quality prediction of in-the-wild videos is quite challenging, since the quality degradations of UGC videos are unpredictable, complicated, and often commingled. Here we contribute to advancing the UGC-VQA problem by conducting a comprehensive evaluation of leading no-reference/blind VQA (BVQA) features and models on a fixed evaluation architecture, yielding new empirical insights on both subjective video quality studies and objective VQA model design. By employing a feature selection strategy on top of efficient BVQA models, we are able to extract 60 out of 763 statistical features used in existing methods to create a new fusion-based model, which we dub the VIDeo quality EVALuator (VIDEVAL), that effectively balances the trade-off between VQA performance and efficiency. Our experimental results show that VIDEVAL achieves state-of-the-art performance at considerably lower computational cost than other leading models. Our study protocol also defines a reliable benchmark for the UGC-VQA problem, which we believe will facilitate further research on deep learning-based VQA modeling, as well as perceptually-optimized efficient UGC video processing, transcoding, and streaming. To promote reproducible research and public evaluation, an implementation of VIDEVAL has been made available online: https://github.com/vztu/VIDEVAL .
In this paper, we put forward the concept of comparative perceptual quality assessment (C-PQA), which refers to the judgment of relative qualities of two visual signals of the same content, but ...subject to different types and levels of distortions. While it is straightforward for human observers to fulfill the CPQA task in daily lives, it remains a difficult challenge for the current research of perceptual quality assessment (PQA). Among the existing PQA algorithms, the full-reference (FR) and reducedreference (RR) methods both need prior knowledge of the original images while the no-reference (NR) algorithms usually work with a single input image. C-PQA is inherently different from those existing methods in that it takes an image pair as input and predicts their relative quality without using any knowledge about the original image. In this paper, we propose a brain theory inspired approach to C-PQA that emulates the process of comparing the relative quality of two visual stimuli as performed by the human visual system (HVS) within the framework of free energy minimization. The brain's internal generative models initialized on the inputs are then used to explain both images. During the internal generative modeling, a group of features are extracted and then integrated to determine the relative quality of two images. We designed a dedicated image database to test the proposed C-PQA algorithm. Experimental results show that the proposed method achieves up to 98% prediction accuracy in line with the subjective ratings, outperforming many state of the art PQA algorithms.
Point clouds offer a novel 3D data representation that has proven pivotal in immersive visual media applications involving human perception. Developing objective point cloud quality assessment (PCQA) ...methods is imperative, as they can substantially reduce human evaluation costs and drive advancements for visual perceptual experiences in point cloud related applications. Point cloud quality assessment without reference remains challenging. Previous PCQA methods predominantly employ a fixed perceptual distance and often overlook the variability in quality perceived from different viewpoints, which impedes their effectiveness in multiscale or multi-granularity feature extraction and learning, particularly for deep neural networks. The single fixed observation distance fails to capture the multi-resolution characteristics intrinsic to human perception. Addressing this gap, in this paper, we introduce a novel no-reference PCQA method (MOD-PCQA) that integrates multiscale features to enhance point cloud quality perception across diverse scales and granularities. MOD-PCQA pioneers a viewpoint-aware feature learning framework, capable of capturing visual features across various granularity levels, from fine to coarse. Specifically, we process and project point clouds into images from different viewpoints. Then, we extract multi-scale features under corresponding perspectives through three branch networks. Finally, we design an alternate learning strategy to optimize the feature extraction network to continuously refine the learned feature information from both inter-scale and intra-scale perspectives. Comprehensive experiments conducted on the SJTU-PCQA and WPC databases validate the superiority of our proposed model over state-of-the-art PCQA methods. Our method achieves optimal performance on both benchmarks by a significant margin, which comprehensively validates its effectiveness for the challenging PCQA task. The source code will be available at https://openi.pcl.ac.cn/OpenPointCloud/MOD-PCQA Zhang et al.
Fully Deep Blind Image Quality Predictor Kim, Jongyoo; Lee, Sanghoon
IEEE journal of selected topics in signal processing,
02/2017, Letnik:
11, Številka:
1
Journal Article
Recenzirano
In general, owing to the benefits obtained from original information, full-reference image quality assessment (FR-IQA) achieves relatively higher prediction accuracy than no-reference image quality ...assessment (NR-IQA). By fully utilizing reference images, conventional FR-IQA methods have been investigated to produce objective scores that are close to subjective scores. In contrast, NR-IQA does not consider reference images; thus, its performance is inferior to that of FR-IQA. To alleviate this accuracy discrepancy between FR-IQA and NR-IQA methods, we propose a blind image evaluator based on a convolutional neural network (BIECON). To imitate FR-IQA behavior, we adopt the strong representation power of a deep convolutional neural network to generate a local quality map, similar to FR-IQA. To obtain the best results from the deep neural network, replacing hand-crafted features with automatically learned features is necessary. To apply the deep model to the NR-IQA framework, three critical problems must be resolved: 1) lack of training data; 2) absence of local ground truth targets; and 3) different purposes of feature learning. BIECON follows the FR-IQA behavior using the local quality maps as intermediate targets for conventional neural networks, which leads to NR-IQA prediction accuracy that is comparable with that of state-of-the-art FR-IQA methods.
PDF
