Elsevier

Pattern Recognition Letters

Volume 51, 1 January 2015, Pages 16-22
Pattern Recognition Letters

Compact color–texture description for texture classification

https://doi.org/10.1016/j.patrec.2014.07.020Get rights and content

Highlights

  • We show that combining multiple texture description methods significantly improves the performance compared to using the single best texture method alone.

  • We further propose to use information theoretic compression approach to compress high-dimensional multi-texture features into a compact heterogeneous texture representation.

  • We perform a comprehensive evaluation of color features, popular in object recognition, for the task of texture classification.

  • We show that late fusion of our compact texture descriptor with discriminative color feature outperforms state-of-the-art results on challenging texture recognition datasets.

Abstract

Describing textures is a challenging problem in computer vision and pattern recognition. The classification problem involves assigning a category label to the texture class it belongs to. Several factors such as variations in scale, illumination and viewpoint make the problem of texture description extremely challenging. A variety of histogram based texture representations exists in literature. However, combining multiple texture descriptors and assessing their complementarity is still an open research problem. In this paper, we first show that combining multiple local texture descriptors significantly improves the recognition performance compared to using a single best method alone. This gain in performance is achieved at the cost of high-dimensional final image representation. To counter this problem, we propose to use an information-theoretic compression technique to obtain a compact texture description without any significant loss in accuracy. In addition, we perform a comprehensive evaluation of pure color descriptors, popular in object recognition, for the problem of texture classification. Experiments are performed on four challenging texture datasets namely, KTH-TIPS-2a, KTH-TIPS-2b, FMD and Texture-10. The experiments clearly demonstrate that our proposed compact multi-texture approach outperforms the single best texture method alone. In all cases, discriminative color names outperforms other color features for texture classification. Finally, we show that combining discriminative color names with compact texture representation outperforms state-of-the-art methods by 7.8%, 4.3% and 5.0% on KTH-TIPS-2a, KTH-TIPS-2b and Texture-10 datasets respectively.

Introduction

Classifying textures is a difficult problem in computer vision and pattern recognition. The task is to associate a class label to its respective texture category. In recent years, a variety of texture description approaches have been proposed [30], [10], [20], [5], [9], [52], [12], [41]. These approaches can be divided into two categories, namely sparse and dense representations. The sparse representation works by detecting feature points either based on interest point or dense sampling strategy. Feature description is then performed on these sampling points [20], [49]. The second strategy, dense representations, involves extracting local features for each pixel in an image [30], [10], [5]. In this paper, we investigate the problem of texture classification using dense local texture representations.

A variety of texture description approaches exist in literature [30], [10], [20], [5], [9], [52]. One of the most successful approaches is that of Local Binary Patterns (LBP) [30] based image representations. Other than texture classification, LBP have been successfully employed to solve other vision problems as well, such as object detection [48], face recognition [1] and pedestrian detection [42]. LBP describes the neighbourhood of a pixel by its binary derivatives which are used to form a short code to describe the pixel neighbourhood. A variety of LBP variants have been proposed [10], [47], [45]. Combining multiple texture features, such as variants of LBP features, is still an open research problem. The work of Guo et al. [9] proposes a learning framework to combine variants of LBP features for texture classification. Tan and Triggs [38] propose to combine Gabor wavelets and LBP features for the problem of face recognition. In this paper, we propose to use a heterogeneous feature set by combining multiple texture description methods.

Combining multiple texture description methods have an inherent problem of high-dimensional final image representations. Recently, Elfiky et al. [7] proposed to use a divisive information theoretic clustering (DITC) method [6] to counter the problem of high-dimensionality of bag-of-words based spatial pyramid representations. The DITC compression was shown to reduce the dimensionality of image representations without any significant loss in accuracy. Similar to the work of Elfiky et al. [7], we propose to use the DITC approach to compress the high-dimensional multi-texture representation. However, different to the work of Elfiky et al. [7], here we investigate compressing a multi-texture histogram to obtain a single heterogeneous texture representation.

Generally, state-of-the-art texture descriptors operate on grey level images thereby ignoring the color information. Color in combination with shape features has been shown to yield excellent results for object recognition [32], [18], [19], object detection [16] and action recognition [15]. Color description is a challenging problem due to significant variations in color caused by changes in illumination, shadows and highlights. Recent works have shown that an explicit color representation improves the performance for object recognition [18], [19], object detection [16] and action recognition [15]. In this paper, we perform a comprehensive evaluation of pure color descriptors, popular in object recognition, for the task of texture classification.

Contributions: We first show that combining multiple texture description methods significantly improves the performance compared to using the single best texture method alone. We further propose to use information theoretic compression approach to compress high-dimensional multi-texture features into a compact heterogeneous texture representation. Finally, we provide a comprehensive evaluation of color features, popular in object recognition, for the task of texture classification. This paper extends our earlier work [17] for texture classification that only evaluated the contribution of color for texture recognition. Beyond the work in [17], we here investigate the problem of combining multiple local texture descriptors for robust texture description. We perform extensive experiments on four challenging texture datasets namely, KTH-TIPS-2a, KTH-TIPS-2b, FMD and Texture-10.

The results of our experiments clearly demonstrate that combining multi-texture descriptors significantly improves the performance compared to the single best method alone. We further show that multi-texture representations can be compressed efficiently without any significant loss in accuracy. Finally, our comprehensive evaluation of color features suggest that discriminative color names outperforms other color descriptors for texture recognition. By combining the best color descriptor with our compact heterogenous texture representation provides state-of-the-art results on three of the four texture datasets.

The paper is organized as follows. In Section 3 we investigate the problem of combining multiple texture descriptors. A comprehensive evaluation of pure color descriptors for texture description is provided in Section 4. In Section 5 we provide experimental results. Section 6 finishes with concluding remarks.

Section snippets

Related work

A variety of texture description approaches have been proposed in recent years [30], [10], [20], [5], [9], [52], [47], [22]. Varma and Zisserman [41] propose a statistical approach for texture modeling using the joint probability distribution of filter responses. A multiresolution approach based on local binary patterns (LBP) is proposed by Ojala et al. [30] for gray-scale and rotation invariant texture classification. The LBP is one of the most successful approaches for texture classification

Combining multiple texture descriptors

Here we present our framework of combining multiple texture features and obtaining a compact heterogeneous texture representation. We combine five texture descriptors namely, completed local binary patterns [10], WLD descriptor [5], binary Gabor pattern [51], local phase quantization descriptor [31] and binarized statistical features [14]. We start by providing a brief overview of the five texture descriptors used in this work.

Completed local binary patterns [10]: The completed local binary

Combining color and texture

There exist two main strategies namely, early and late fusion, to combine color and texture information [29], [17]. Early fusion works by computing texture descriptor on the color channels. In this way, a joint color–texture representation is obtained that combines the two cues at the pixel-level. Early fusion based image representation has the advantage of being more discriminative since the two cues are combined at the pixel level. However, early fusion representations suffers from the

Experimental results

To validate the performance of the proposed framework, we use four challenging datasets, namely KTH-TIPS-2a, KTH-TIPS-2b, FMD and Texture-10. The KTH-TIPS-2a dataset consists of 11 texture categories with images at 9 different scales, 3 poses and 4 different illumination conditions. We use the standard protocol [4], [36], [5] by reporting the average classification performance over the 4 test runs. In each time, all the images from 1 sample are used for test while the images from the remaining

Conclusion

In this paper we investigated the problem of texture recognition in images. Firstly, we have shown that fusing different texture representations significantly improves the performance compared to the single best method. To counter the high-dimensionality problem of the image representation, we proposed to use the DITC approach. Additionally, we performed a comprehensive evaluation of pure color descriptors, popular in image classification, for the task of texture recognition.

The results show

Acknowledgements

This work has been supported by SSF through a grant for the project CUAS, by VR through a grant for the project ETT, through the Strategic Area for ICT research ELLIIT, CADICS and The Academy of Finland (Finnish Centre of Excellence in Computational Inference Research COIN, 251170). We also acknowledge the grants 255745 and 251170 of the Academy of Finland, SSP-14183 of EIT ICT Labs, and the D2I SHOK project. The project TIN2013-41751 of Spanish Ministry of Science. The calculations were

References (52)

  • A. Bosch, A. Zisserman, X. Munoz, Scene classification via plsa, in: ECCV,...
  • B. Caputo, E. Hayman, P. Mallikarjuna, Class-specific material categorisation, in: ICCV,...
  • J. Chen et al.

    Wld: a robust local image descriptor

    PAMI

    (2010)
  • I. Dhillon et al.

    A divisive information-theoretic feature clustering algorithm for text classification

    JMLR

    (2003)
  • N. Elfiky et al.

    Discriminative compact pyramids for object and scene recognition

    PR

    (2012)
  • Z. Guo et al.

    A completed modeling of local binary pattern operator for texture classification

    TIP

    (2010)
  • X. Hong et al.

    Combining lbp difference and feature correlation for texture description

    TIP

    (2014)
  • S. ul Hussain, B. Triggs, Visual recognition using local quantized patterns, in: ECCV,...
  • X. Jiang

    Asymmetric principal component and discriminant analyses for pattern classification

    PAMI

    (2009)
  • J. Kannala, E. Rahtu, Bsif: binarized statistical image features, in: ICPR,...
  • F.S. Khan et al.

    Coloring action recognition in still images

    IJCV

    (2013)
  • F.S. Khanm, R.M. Anwer, J. van de Weijer, A.D. Bagdanov, M. Vanrell, A.M. Lopez, Color attributes for object detection,...
  • F.S. Khan, J. van de Weijer, S. Ali, M. Felsberg, Evaluating the impact of color on texture recognition, in: CAIP,...
  • F.S. Khan et al.

    Modulating shape features by color attention for object recognition

    IJCV

    (2012)
  • R. Khan, J. van de Weijer, F.S. Khan, D. Muselet, C. Ducottet, C. Barat, Discriminative color descriptors, in: CVPR,...
  • S. Lazebnik et al.

    A sparse texture representation using local affine regions

    PAMI

    (2005)
  • Cited by (47)

    • Hierarchical ELM ensembles for visual descriptor fusion

      2018, Information Fusion
      Citation Excerpt :

      It has been employed to improve the performance of scene classification algorithms due to the complementary characteristics among the color channels [43]. Although, there is an increasing amount of work on combining texture and color descriptors [4,7,17,27,45], effective fusion of descriptors by assessing their complementarity is still an open research problem in computer vision. This motivated us to explore the complementary visual information in order to boost the scene classification performance.

    • Binary patterns encoded convolutional neural networks for texture recognition and remote sensing scene classification

      2018, ISPRS Journal of Photogrammetry and Remote Sensing
      Citation Excerpt :

      A variety of LBP variants have been proposed in literature, including Local Ternary Patterns (Tan and Triggs, 2010), Local Binary Pattern Variance (Guo et al., 2010a), Noise Tolerant Local Binary Patterns (Fathi and Nilchi, 2012), Completed Local Binary Patterns (Guo et al., 2010b), Extended Local Binary Patterns (Liu et al., 2012) and Rotation Invariant Local Phase Quantization (Ojansivu et al., 2009). In addition to the introduction of different LBP variants, the fusion of LBP descriptor with color features have also been investigated in previous studies (Maenpaa and Pietikainen, 2004; Khan et al., 2015). Deep learning: In recent years, Convolutional Neural Networks (CNNs) (LeCun et al., 1989) have shown to provide excellent performance for many computer vision tasks.

    • Distilling Local Texture Features for Colorectal Tissue Classification in Low Data Regimes

      2024, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
    • Salient Mask-Guided Vision Transformer for Fine-Grained Classification

      2023, Proceedings of the International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications
    View all citing articles on Scopus

    This paper has been recommended for acceptance by Y. Liu.

    View full text