Elsevier

Pattern Recognition

Volume 45, Issue 9, September 2012, Pages 3166-3182
Pattern Recognition

Towards automatic polyp detection with a polyp appearance model

https://doi.org/10.1016/j.patcog.2012.03.002Get rights and content

Abstract

This work aims at automatic polyp detection by using a model of polyp appearance in the context of the analysis of colonoscopy videos. Our method consists of three stages: region segmentation, region description and region classification. The performance of our region segmentation method guarantees that if a polyp is present in the image, it will be exclusively and totally contained in a single region. The output of the algorithm also defines which regions can be considered as non-informative. We define as our region descriptor the novel Sector Accumulation-Depth of Valleys Accumulation (SA-DOVA), which provides a necessary but not sufficient condition for the polyp presence. Finally, we classify our segmented regions according to the maximal values of the SA-DOVA descriptor. Our preliminary classification results are promising, especially when classifying those parts of the image that do not contain a polyp inside.

Highlights

► Definition of a general model of polyp appearance. ► Extension on the definition of the depth of valleys image. ► Development of an accurate region segmentation scheme. ► Integrate information from depth of valleys image in the novel SADOVA descriptor. ► Promising results on region-based and frame-based classification.

Introduction

Colon cancer's survival rate depends on the stage in which it is detected, decreasing from rates higher than 95% in the first stages to rates lower than 35% in stages IV and V [1]; hence the importance of detecting it on its early stages by using screening techniques, such as colonoscopy [2].

Although colonoscopy is considered nowadays as the gold standard for colon screening there are still open challenges to overcome, such as the reduction of the miss-rate [3]. During the last decades there is a trend that consists of developing intelligent systems for medical applications. Intelligent systems are currently being used to assist in other medical interventions. For instance, there are systems that can interpret medical data automatically, such as KARDIO [4], which was developed to interpret electrocardiograms. It is possible to find many examples of intelligent systems built to assist in cancer detection. The interested reader can consult some works in the field of breast cancer detection [5] or prostate cancer detection [6].

Our objective is to add significant value to the colonoscopy procedure by using methods based on computer vision or artificial intelligence. In the case of colonoscopy, there is a number of possible areas where an intelligent system can potentially help [7]. It is possible to think of an intelligent system that can assist in the diagnosis procedure, by highlighting parts of the colon that are likely to contain lesions or polyps as the physician progresses the instrumental through the patient. There is also a potential in the use of the information extracted from the analysis of a colonoscopy video in order to build up systems that can provide an objective assessment of the physician's skills. By doing so, training programs could be developed without the cost that a real intervention has. Another possible area of application could be to provide a whole description of what appears on the scene for automatic reporting. Finally, another possible domain of application could be the extension of the information that the colonoscopy intervention provides, by leading to a development of patient-specific models.

In our case, in the context of developing intelligent systems for colonoscopy, the main objective is to define a robust model of polyp appearance. This model can be potentially used to indicate which regions in the image are more likely to contain a polyp inside, which can be useful for several of the applications mentioned before. Particularly, our whole processing scheme is built on the fact that intensity valleys appear to surround polyps as the light of the colonoscope and the camera are in the same direction. For this reason, we propose to use valley and ridge information as cue to detect polyps. In our case, our method will not try to fit a certain model into the images but it will look for several appearance cues which will guide it.

In this paper we present our work on polyp detection, which extends the works on the depth of valleys image [8]. Our detection method consists of three stages: region segmentation, region description and region classification.

We present our first contribution, the region segmentation stage, in which an input colonoscopy image is segmented into a minimum number of informative regions, one of these regions containing the polyp in a complete way. Since all the non-informative regions are rejected, the size of the problem is reduced largely. The concept of informative and non-informative regions are used here in the context of assuring that no polyp is inside the given region and, therefore, there will be no need for further processing [9]. These results can be used later to classify the informative regions into polyp- vs. non-polyp-containing candidates.

Our second contribution consists of the introduction of the Sector Accumulation-Depth of Valleys Accumulation (SA-DOVA), which aims to find which points on the image are interior to objects, which are meant to be delimited by points with high value in the depth of valleys image. Finally, we classify the segmented regions into polyp-containing vs. the opposite, according to their maximum values of the DOVA descriptor.

The structure of the paper is as follows: in Section 2 we introduce previous approaches on polyp detection in colonoscopy videos. We present in Section 3 the theoretical model on which we base our polyp detection method, which is presented in Section 4. In Section 5 we show our experimental setup along with polyp detection results. In Section 6 we discuss in depth the performance of each stage. Finally we finish this paper in Section 7 with the main conclusions extracted from our approach and our proposals for future work.

Section snippets

Related work

The main objective of the colonoscopy procedures is to check the status of the colon of the patient, with the aim to find possible lesions and cancer polyps on it. The general appearance of the polyps has been covered widely by medical bibliographic sources [10]. However, there is a great variability in polyp appearance in colonoscopy videos, since there are some challenges that hinder polyp detection, namely: (1) the non-uniform appearance of polyps (see Fig. 1(a–d)); (2) their shape, flat or

Illumination

In order to define model of appearance for polyps in colonoscopy videos, we need both an a priori model about the polyp and a model of the illumination. For the sake of simplicity let us consider a polyp as a semi-spherical shape protruding from the colon wall plane. We will also consider that the polyp surface is regular and that its reflectance can be approximated by Phong's illumination model [22]. The colonoscope itself is modeled by a pinhole camera and a punctual illumination source

Region segmentation

The general scheme of the segmentation algorithm consists of four different stages which will be described next (see Fig. 4 for a graphical scheme).

  • 1.

    Image preprocessing: Before applying any segmentation algorithm there are some operations that should be done: (1) converting the image into gray-scale; (2) de-interleaving (as our images come from a high definition interleaved video source); (3) correction of the specular highlights, and (4) inverting the gray-scale image.

    To correct the specular

Database

We built a database from our data in order to test the performance of our segmentation and description methods. In Table 2 we present the key data of our database: length, number of frames and polyp shape (flat or peduncular).

We were provided with 15 random cases, in which the experts (physicians) annotated all the sequences showing polyps, and a random sample of 20 frames per sequence was obtained, with frame size of 500×574 pixels. The central portion of the images was cropped in order to

Discussion

In this section we discuss the performance of each stage of the processing scheme and we sketch how the performance of each of them could be improved to obtain better global detection results.

The region segmentation stage is based on the previously defined model of polyp appearance. The experimental results showed that segmentation results depend on the threshold that we apply to the DV image in a way such the higher the threshold, the lower the number of final regions. But, in this case, as

Conclusions and future work

In this paper we presented a polyp detection scheme based on a model of polyp appearance in the context of the analysis of colonoscopy images. This model is built on the appearance of valleys surrounding polyps as the light of the colonoscope and the camera are in the same direction, causing the apparition of shadows around prominent surfaces such as polyps. We also presented the concept of depth of valleys image, which combines the information of a ridges and valleys detector with the

Acknowledgments

The authors would like to acknowledge David Vázquez, Dr. Antonio López, Dr. Debora Gil and the GV2 group from Trinity College for their contribution in the discussions during the writing of this paper. This work was supported in part by a research grant from Universitat Autònoma de Barcelona 471-01-3/08, by the Spanish Government through the founded project “COLON-QA” (TIN2009-10435) and by research programme Consolider Ingenio 2010: MIPRCV (CSD2007-00018).

Jorge Bernal received the B.Sc. degree in Telecommunications Engineering from the Universidad de Valladolid, Spain, in 2008 and the M.Sc. degree in Computer Vision and Artificial Intelligence from the Universitat Autnoma de Barcelona, Spain, in 2009. Presently he is an assistant professor in the Universitat Autnoma de Barcelona and he is currently in his third year of his Ph.D. studies. His research interest include image processing, machine vision and medical imaging. He has worked in several

References (32)

  • Tresca, A., The Stages of Colon and Rectal Cancer, New York Times (About.com), 2010, p....
  • J.P. Hassinger et al.

    Effectiveness of a multimedia-based educational intervention for improving colon cancer literacy in screening colonoscopy patients

    Diseases of the Colon & Rectum

    (2010)
  • N. Segnan et al.

    European Guidelines for Quality Assurance in Colorectal Cancer Screening and Diagnosis

    (2011)
  • I. Bratko et al.

    KARDIO: A Study in Deep and Qualitative Knowledge for Expert Systems

    (1990)
  • J. Wei et al.

    Computer-aided detection of breast masses: four-view strategy for screening mammography

    Medical Physics

    (2011)
  • S. Viswanath, D. Palumbo, J. Chappelow, P. Patel, B. Bloch, N. Rofsky, R. Lenkinski, E. Genega, A. Madabhushi,...
  • J. Bernal et al.

    Colonoscopy Book 1: Towards Intelligent Systems for Colonoscopy

    (2011)
  • J. Bernal, J. Sánchez, F. Vilariño, A region segmentation method for colonoscopy images using a model of polyp...
  • J. Bernal, J. Sánchez, F. Vilariño, Reduction of pattern search area in colonoscopy images by merging non-informative...
  • T. Byers, B. Levin, D. Rothenberger, G.D. Dodd, R.A. Smith, F.T.A.C.S. Detection, T.A.G. on Colorectal Cancer, American...
  • J. Bernal, F. Vilariño, J. Sánchez, Feature Detectors and Feature Descriptors: Where We Are Now, Technical Report 154,...
  • S. Hwang, J.H. Oh, W. Tavanapong, et al., Polyp detection in colonoscopy video using elliptical shape feature, in: IEEE...
  • H. Zhu, Y. Fan, Z. Liang, Improved curvature estimation for shape analysis in computer-aided detection of colonic...
  • C. van Wijk et al.

    Detection and segmentation of colonic polyps on implicit isosurfaces by second principal curvature flow

    IEEE Transactions on Medical Imaging

    (2010)
  • P. Li, K. Chan, S. Krishnan, Learning a multi-size patch-based hybrid kernel machine ensemble for abnormal region...
  • S. Ameling et al.

    Texture-based polyp detection in colonoscopy

    Bildverarbeitung für die Medizin

    (2009)
  • Cited by (0)

    Jorge Bernal received the B.Sc. degree in Telecommunications Engineering from the Universidad de Valladolid, Spain, in 2008 and the M.Sc. degree in Computer Vision and Artificial Intelligence from the Universitat Autnoma de Barcelona, Spain, in 2009. Presently he is an assistant professor in the Universitat Autnoma de Barcelona and he is currently in his third year of his Ph.D. studies. His research interest include image processing, machine vision and medical imaging. He has worked in several projects during his Ph.D. studies such as reconstruction of 3-D images via backprojection and currently his research is focused on the development of intelligent systems for colonoscopy.

    Javier Sánchez received the B.Sc. degree in Computer Science in 1988 and his Ph.D. degree in Computer Science in 1996 from the Universitat Autnoma de Barcelona, Spain. Since 1988 he is a professor in the Universitat Autnoma de Barcelona, Spain, where he became an associated professor in 1997, working in the Computer Science Department. He currently develops his research and technological transfers activities in the machine vision and medical imaging fields. He has participated with success in more than 10 different funded research projects and developed more than 30 research, development and innovation projects for several enterprises and scientific publications.

    Fernando Vilariño received his B.Sc. degree in Physics from Univ. Santiago de Compostela, Spain (1996), M.Sc. in Computer Vision and Artificial Intelligence from Univ. Santiago de Compostela, Spain (1997) and Ph.D. from Universitat Autnoma de Barcelona, Spain (2006). His research interest is focused on feature extraction and machine learning. He has spent research stages in different institutions in the area of computer vision and machine learning, such as the Computer Vision Centre in Barcelona, Spain, the School of Informatics in Bangor Univ. Wales, UK, and the Computer Science Department in Trinity College Dublin, Ireland. He was granted the Ramon y Cajal Grant from the Spanish government in 2009, and he currently has an Associate Professor position at Univ. Autnoma de Barcelona. He is the PI of different projects related to Medical Imaging, and he has contributed with several publications in International Journals, Conferences and the co-authorship of a group of patents.

    View full text