This work proposes a method for texture classification based on the successive application of a local transform presented here for the first time. Such transform comprises two steps: (1) We built a ...two-layer mapping relating each pixel with its neighborhood, with the weights in the first layer randomly assigned; (2) We use the parameters learned by such mapping to transform the original image. Finally, we extract local descriptors at different stages of the successive application of this transform to compose the texture descriptors. The performance of our method is verified in the classification of benchmark texture databases and compared with state-of-the-art approaches. We also present an application for plant species identification. The results confirm our expectation that a model that is not based on the classical learning-based approach can still be competitive in texture analysis.
•A local randomized projection is proposed for texture classification.•The projection is successively applied over the image and local descriptors are collected.•The performance of the proposed descriptors is verified on benchmark texture databases.•An application to the identification of Brazilian plant species is also presented.•Our approach outperforms state-of-the-art methods in terms of classification accuracy.
Local Binary Patterns (LBP) have emerged as one of the most prominent and widely studied local texture descriptors. Truly a large number of LBP variants has been proposed, to the point that it can ...become overwhelming to grasp their respective strengths and weaknesses, and there is a need for a comprehensive study regarding the prominent LBP-related strategies. New types of descriptors based on multistage convolutional networks and deep learning have also emerged. In different papers the performance comparison of the proposed methods to earlier approaches is mainly done with some well-known texture datasets, with differing classifiers and testing protocols, and often not using the best sets of parameter values and multiple scales for the comparative methods. Very important aspects such as computational complexity and effects of poor image quality are often neglected.
In this paper, we provide a systematic review of current LBP variants and propose a taxonomy to more clearly group the prominent alternatives. Merits and demerits of the various LBP features and their underlying connections are also analyzed. We perform a large scale performance evaluation for texture classification, empirically assessing forty texture features including thirty two recent most promising LBP variants and eight non-LBP descriptors based on deep convolutional networks on thirteen widely-used texture datasets. The experiments are designed to measure their robustness against different classification challenges, including changes in rotation, scale, illumination, viewpoint, number of classes, different types of image degradation, and computational complexity. The best overall performance is obtained for the Median Robust Extended Local Binary Pattern (MRELBP) feature. For textures with very large appearance variations, Fisher vector pooling of deep Convolutional Neural Networks is clearly the best, but at the cost of very high computational complexity. The sensitivity to image degradations and computational complexity are among the key problems for most of the methods considered.
•A taxonomy and comprehensive survey of LBP variants.•Characteristics of, and connections between LBP variants are provided.•A comprehensive experimental evaluation of 32 LBP methods.•Comparison of 32 LBP variants with 8 deep ConvNets features.•Evaluation of robustness to rotation, illumination, scale and noise changes.•Comparison of computational complexity of forty variants.
•Deep layers of Convolutional Neural Networks are used for feature extraction.•Handcrafted and learned features are used together to extract information.•Different architectures for combining ...handcrafted and learned features are proposed.•Combination of different features is used to solve image classification problems.
This work presents a generic computer vision system designed for exploiting trained deep Convolutional Neural Networks (CNN) as a generic feature extractor and mixing these features with more traditional hand-crafted features. Such a system is a single structure that can be used for synthesizing a large number of different image classification tasks. Three substructures are proposed for creating the generic computer vision system starting from handcrafted and non-handcrafter features: i)one that remaps the output layer of a trained CNN to classify a different problem using an SVM; ii) a second for exploiting the output of the penultimate layer of a trained CNN as a feature vector to feed an SVM; and iii) a third for merging the output of some deep layers, applying a dimensionality reduction method, and using these features as the input to an SVM. The application of feature transform techniques to reduce the dimensionality of feature sets coming from the deep layers represents one of the main contributions of this paper. Three approaches are used for the non-handcrafted features: deep transfer learning features based on convolutional neural networks (CNN), principal component analysis network (PCAN), and the compact binary descriptor (CBD). For the handcrafted features, a wide variety of state-of-the-art algorithms are considered: Local Ternary Patterns, Local Phase Quantization, Rotation Invariant Co-occurrence Local Binary Patterns, Completed Local Binary Patterns, Rotated local binary pattern image, Globally Rotation Invariant Multi-scale Co-occurrence Local Binary Pattern, and several others. The computer vision system based on the proposed approach was tested on many different datasets, demonstrating the generalizability of the proposed approach thanks to the strong performance recorded. The Wilcoxon signed rank test is used to compare the different methods; moreover, the independence of the different methods is studied using the Q-statistic. To facilitate replication of our experiments, the MATLAB source code will be available at (https://www.dropbox.com/s/bguw035yrqz0pwp/ElencoCode.docx?dl=0).
Texture is a fundamental characteristic of many types of images, and texture representation is one of the essential and challenging problems in computer vision and pattern recognition which has ...attracted extensive research attention over several decades. Since 2000, texture representations based on Bag of Words and on Convolutional Neural Networks have been extensively studied with impressive performance. Given this period of remarkable evolution, this paper aims to present a comprehensive survey of advances in texture representation over the last two decades. More than 250 major publications are cited in this survey covering different aspects of the research, including benchmark datasets and state of the art results. In retrospect of what has been achieved so far, the survey discusses open challenges and directions for future research.
•The paper presents a novel texture descriptor based on Local Binary Patterns.•We address the problem of rotation invariance and feature selections.•The experiments are performed on standard datasets ...and the results are compared with the state-of-the-art.•The source code to reproduce the results is made publicly available.
In this paper, we present a novel rotation-invariant and computationally efficient texture descriptor called Dominant Rotated Local Binary Pattern (DRLBP). A rotation invariance is achieved by computing the descriptor with respect to a reference in a local neighborhood. A reference is fast to compute maintaining the computational simplicity of the Local Binary Patterns (LBP). The proposed approach not only retains the complete structural information extracted by LBP, but it also captures the complimentary information by utilizing the magnitude information, thereby achieving more discriminative power. For feature selection, we learn a dictionary of the most frequently occurring patterns from the training images, and discard redundant and non-informative features. To evaluate the performance we conduct experiments on three standard texture datasets: Outex12, Outex 10 and KTH-TIPS. The performance is compared with the state-of-the-art rotation invariant texture descriptors and results show that the proposed method is superior to other approaches.
Light field can record the four-dimensional information of light rays, i.e. the position and direction information in which depth information is implied. To improve the depth estimation accuracy, we ...propose a depth estimation algorithm based on convolutional neural network (CNN). First, a single image super resolution algorithm is adopted to spatially super resolve the sub-aperture images (SAIs). Second, to adapt the texture complexity, the SAIs are partitioned into two regions, i.e., simple texture region and complex texture region, based on the texture analysis of the central SAI. Third, the epipolar plane images (EPIs) in horizontal, vertical, 45 degree diagonal, and 135 degree diagonal directions for both complex and simple texture regions are extracted, and the corresponding EPIs for the simple and complex texture regions are fed into the specified network branches. Finally, a fusion module is designed to generate the depth map. Experimental results show that the quality of the estimated depth maps by the proposed method is better than the state-of-the-art methods in terms of both objective quality and subjective quality. Moreover, the proposed method is more robust to noise.
•Light field can record the four-dimensional information of light rays.•A depth estimation algorithm based on convolutional neural network is proposed.•The sub-aperture images are super resolved by single image algorithm.•The sub-aperture images are partitioned into two regions by texture complexity.•The different texture regions are fed into the specified network branches.
Automated tissue characterization is one of the most crucial components of a computer aided diagnosis (CAD) system for interstitial lung diseases (ILDs). Although much research has been conducted in ...this field, the problem remains challenging. Deep learning techniques have recently achieved impressive results in a variety of computer vision problems, raising expectations that they might be applied in other domains, such as medical image analysis. In this paper, we propose and evaluate a convolutional neural network (CNN), designed for the classification of ILD patterns. The proposed network consists of 5 convolutional layers with 2 × 2 kernels and LeakyReLU activations, followed by average pooling with size equal to the size of the final feature maps and three dense layers. The last dense layer has 7 outputs, equivalent to the classes considered: healthy, ground glass opacity (GGO), micronodules, consolidation, reticulation, honeycombing and a combination of GGO/reticulation. To train and evaluate the CNN, we used a dataset of 14696 image patches, derived by 120 CT scans from different scanners and hospitals. To the best of our knowledge, this is the first deep CNN designed for the specific problem. A comparative analysis proved the effectiveness of the proposed CNN against previous methods in a challenging dataset. The classification performance ( ~ 85.5%) demonstrated the potential of CNNs in analyzing lung patterns. Future work includes, extending the CNN to three-dimensional data provided by CT volume scans and integrating the proposed method into a CAD system that aims to provide differential diagnosis for ILDs as a supportive tool for radiologists.
Abstract
Texture classification plays a vital role in the emerging research field of image classification. This paper approaches the texture classification problem using significant features ...extracted from pre-trained Convolutional Neural Network (CNN) like Alexnet, VGG16, Resnet18, Googlenet, MobilenetV2, and Darknet19. These features are classified by machine learning classifiers such as Support Vector Machine (SVM), Ensemble, K Nearest Neighbour (KNN), Naïve Bayes (NB), Decision Tree (DT), and Discriminant Analysis (DA). The performance of the work is evaluated with the texture databases namely KTH-TIPS, FMD, UMD-HR, and DTD. Among these CNN features derived from VGG16 classify by SVM provides better classification accuracy rather than using VGG16 with a softmax classifier.
•We adapt the CNN architecture to texture analysis.•We introduce an energy layer to discard the overall shape information and focus on texture features.•We evaluate the domain transferability and the ...depth of networks that are from scratch or pretrained.•Our network is simpler than a classic CNN and obtains better classification results on texture datasets.•We combine our texture CNN to a classic CNN (overall shape analysis) and further improve the results.
Deep learning has established many new state of the art solutions in the last decade in areas such as object, scene and speech recognition. In particular Convolutional Neural Network (CNN) is a category of deep learning which obtains excellent results in object detection and recognition tasks. Its architecture is indeed well suited to object analysis by learning and classifying complex (deep) features that represent parts of an object or the object itself. However, some of its features are very similar to texture analysis methods. CNN layers can be thought of as filter banks of complexity increasing with the depth. Filter banks are powerful tools to extract texture features and have been widely used in texture analysis. In this paper we develop a simple network architecture named Texture CNN (T-CNN) which explores this observation. It is built on the idea that the overall shape information extracted by the fully connected layers of a classic CNN is of minor importance in texture analysis. Therefore, we pool an energy measure from the last convolution layer which we connect to a fully connected layer. We show that our approach can improve the performance of a network while greatly reducing the memory usage and computation.
•A formal definition of Concave and Convex Binary Thresholding Functions are introduced.•Two new LBP-like descriptors: Local Concave and Convex Micro-Structures Pattern (LCvMSP and LCxMSP) ...descriptors are proposed.•LCvMSP and LCxMSP are concatened into a single vector feature to obtain the multiscale LCCMSP descriptor.•A statistical hypothesis testing based method for parameters optimization on several datasets is proposed.•The proposed methods demonstrate superior performance to 79 LBP variants and non-LBP methods over 13 texture datasets.
Motivated by researching new image texture modeling that improves state-of-the-art LBP variants and non-LBP descriptors, this paper proposes a novel approach for constructing local image descriptors, which are suitable for histogram based image representation. Instead of heuristic code constructions, the proposed approach is based on local concave-and-convex characteristics, which have high ability to extract discriminative and stable texture representation. Different from the majority of descriptors that only encode relationships between the pixels in doublets around central pixel (within 3 × 3 neighborhood), the proposed approach encodes relationships between the pixels in triplets by including the central pixel in the modeling. We build two distinct descriptors by dividing local features into two distinct groups, i.e., local concave and convex microstructure patterns (LCvMSP and LCxMSP), according to relationships between the pixels inside the triplets, formed along closed path around the central pixel of a 3 × 3-grayscale image patch. To make the descriptors more insensitive to noise and invariant to monotonic gray scale transformation, two supplementary triplets are added in the modeling. These triplets are formed using the central pixel and four virtual pixels set to the median of the grey-scale values of the 3 × 3 neighbourhood and the whole image and the average local and global gray levels respectively. The histograms obtained from the single scale descriptors LCvMSP and LCxMSP are concatenated together to build multi-scale histogram feature vector referred to as local concave-and-convex micro-structure pattern (LCCMSP), that is expected to better represent salient local texture structure. We evaluated the effectiveness of the proposed methods on thirteen challenging representative widely-used texture datasets, and found that the proposed LCvMSP, LCxMSP and LCCMSP operators achieve performances that are competitive or better than a large number of recent most promising state-of- the-art LBP variants and non-LBP descriptors. Statistical comparison based on Wilcoxon signed rank test demonstrated that the proposed methods are the top three over all the tested datasets.
PDF
