Concept Based Intermedia Medical Indexing. Application on CLEF

pathologies that are similar to a patient's image(s). In teaching and research, visual retrieval methods could help researchers, lecturers, and student find rele-.
Télécharger le PDF
78KB taille 4 téléchargements 235 vues
commentaire
Report
Concept Based Intermedia Medical Indexing. Application on CLEF Medical Image with UMLS Metathesaurus Daniel Racoceanu1,2 , Caroline Lacoste1 , Joo-Hwee Lim1 , Jean-Pierre Chevallet1,3 , Le Thi Hoang Diem1,3 , and Xiong Wei1 1

IPAL-Image Perception Access & Language French National Research Center UMI-CNRS 2955, Institute for Infocomm Research, A*STAR, 21 Heng Mui Keng Terrace, Singapore 119613 [email protected] http://ipal.i2r.a-star.edu.sg/ 2 University of Franche-Comt´e, Besan¸con, France 3 University of Grenoble 1 - Joseph Fourier, France

Extended Abstract Content Based Medical Image Retrieval (CBMIR) has reached a very challenging threshold, related to the gap between low-level medical image features and the semantic highly specialized medical information and knowledge; to the important context-dependence of the query and navigation; and the wide distribution of the medical data and knowledge. Answers to questions concerning semantic descriptors, medical image analysis and report fusion and indexing, context-sensitive navigation and querying are thus still missing today. The medical imaging community has become increasingly aware of the potential benefit of using the new technologies in medical image analysis and retrieval, relating to diagnosis and prognosis assistance, evidence-based medicine and medical case-based reasoning. Besides the growing amount of medical data produced everyday, medical image retrieval systems have good potential in clinical decision making process, where it can be beneficial to find other images of the same modality, of the same anatomic region, and of the same disease. Hence, CBMIR systems can assist doctors in diagnosis by retrieving images with known pathologies that are similar to a patient’s image(s). In teaching and research, visual retrieval methods could help researchers, lecturers, and student find relevant images from large repositories. Visual features not only allow the retrieval of cases with patients having similar diagnoses but also cases with visual similarity but different diagnoses. Current Content Based Image Retrieval (CBIR) systems generally use primitive features such as color or texture, or logical features such as object and their relationships to represent images. We believe that the lack of medical knowledge, explains the poor results obtained in the medical domain. More specifically, the description of an image by low-level or medium-level features seems not sufficient

to express the semantic content of a medical image. This distance between user expectations and system automatic extractions is usually called semantic gap. We think that if this gap can be reduced, then it will lead to better retrieval results. Moreover, focusing on a specialized domain like medicine, enable us to use a very large set of precise knowledge as so giving us the chance to fill up the gap with knowledge. Among the limited research efforts of CBMIR, classification or clustering driven feature selection and weighting has received much attention as general visual cues often fail to be discriminative enough to deal with more subtle, domain-specific differences and more objective ground truth in the form of disease categories is usually available. In reality, pathology bearing regions tend to be highly localized. Hence, local features such as those extracted from segmented dominant image regions approximated by best fitting ellipses have been proposed. However, it has been recognized that pathology bearing regions cannot be segmented out automatically for many medical domains. Hence it is desirable to have a medical CBIR system that represents images in terms of semantic features, that can be learned from examples (rather than handcrafted with a lot of expert input) and do not rely on robust region segmentation. Intermedia is another way to enhance the correction and completeness of the index of a multimedia data collection. By intermedia, we mean producing, and fusing compatible information, in our case conceptual indexes, from several media. In particular, we have shown [7] that mixing text and image information increases the retrieval performance. In [1], statistical methods are used for modeling the occurrence of document keywords and visual characteristics. The proposed system is sensitive to the quality of the segmentation of the images. Other initiatives to combine image and text analysis study the use of Latent Semantic Analysis (LSA) techniques. In [8], the author applied the LSA method to features extracted from the two media. The conclusion of this study is that combining the image and the text through the LSA method is not always efficient. The usefulness of LSA is also not conclusive in [9]. Conversely, we have noticed that a simple late fusion of visual and textual indexes provides good results. In this paper, we present our work on medical image retrieval that is mainly based on the incorporation of medical knowledge in the system. This approach as been tested with success on CLEF [2, 5, 3]. For text, this knowledge comes from the Unified Medical Language System (UMLS) metathesaurus produced by NML4 . For images, this knowledge is in semantic features that are learned from examples and do not rely on robust region segmentation. In order to manage large and complex sets of visual entities (i.e., high content diversity) in the medical domain, we developed a structured learning framework that facilitates modular design and extraction of medical visual semantics. Two complementary visual indexing approaches within this framework have been proposed: a global indexing to access image modality, and a local indexing to access semantic local 4

National Library of Medicine - http://www.nlm.nih.gov/

features. This local indexing does not rely on region segmentation but builds upon patch-based semantic detector [4]. To benefit efficiently from both modalities, we have built an intermedia index where both information coming from image and text are represented. Moreover, this index is at a conceptual level using UMLS concepts. The use of UMLS concepts allows our system to standardize the index, facilitating the communication between visual and textual indexing. It is also a nice way to have a multilingual unique index has UMLS concepts can be expressed in 17 languages. We propose several fusion approaches [6] and a visual modality filtering designed to remove visually aberrant images according to the query modality concept(s). Using conceptual indexing is difficult because of the extra phase of concept extraction from text. Unfortunately, this approach often does not lead to better results compared to the classical bag of word solution. Our approach on query structure, thanks to UMLS semantic structure, has produced the best results on the medical task of CLEF20065 (i.e. imageCLEFmed) for IPAL 6 . A simple fusion between text and image also has given the best results. Future developments become very promising using homogeneous balanced semantic fusion to improve this fusion stage, for example, using local visual information derived from the proposed local patch classifier. Indeed, this method is complementary to the global medical image analysis. There is also some work to be done to extract more information from text and to improve image recognition. Appropriate clustering methods could bridge to medical multimedia datamining, opening the way to the evidence-based medicine and other advanced medical research applications and studies. On the other hand, the use of the incremental learning based on the initial database clustering, will facilitate the development of an efficient real-time medical case-based reasoning. Finally, a semantic query and case expansion scheme can be deployed using the symbolic and statistical relation available in UMLS as well as the contextual behavior information extracted from the real use of this type of retrieval system7 .

References 1. K. Barnard and D. Forsyth. Learning the semantics of words and pictures. In Proceedings of the International Conference on Computer Vision, volume 2, pages 408–415, 2001. 2. Jean-Pierre Chevallet, Joo-Hwee Lim, and Sa¨ıd Radhouani. Using ontology dimensions and negative expansion to solve precise queries in clef medical task. In CLEF Workhop, Working Notes Medical Image Track, Vienna, Austria, 21–23 September 2005. 5 6

7

Cross Evaluation Forum Language: http://www.clef-campaign.org/ IPAL is a French-Singaporean joint laboratory involving the Institute for Infocomm Research (I2R), the French National Research Center (CNRS) and the National University of Singapore (NUS): http://ipal.i2r.a-star.edu.sg/ This will constitute one of the main topic of the Multimedia Medical Conceptual Web for Intelligent Information Access (MMedWeb) project granted in July 2006 by A*STAR/SERC Singapore http://www.comp.nus.edu.sg/ leowwk/MMedWeb/

3. Caroline Lacoste, Jean-Pierre Chevallet, Joo-Hwee Lim, Xiong Wei, Daniel Raccoceanu, Diem Le Thi Hoang, Roxana Teodorescu, and Nicolas Vuillenemot. Ipal knowledge-based medical image retrieval in imageclefmed 2006. In Working Notes for the CLEF 2005 Workshop, 20-22 September , Medical Image Track, Alicante, Spain, 2006. 4. J. Lim and J.P. Chevallet. Vismed: a visual vocabulary approach for medical image indexing and retrieval. In Proceedings of the Asia Information Retrieval Symposium, pages 84–96, 2005. 5. Joo-Hwee Lim and Jean-Pierre Chevallet. A structured learning approach for medical image indexing and retrieval. In CLEF Workhop, Working Notes Medical Image Track, Vienna, Austria, 21–23 September 2005. 6. D. Racoceanu, C. Lacoste, R.Teodorescu, and N. Vuillemenot. A semantic fusion approach between medical images and reports using umls. In Proceedings of the Asia Information Retrieval Symposium, volume LNCS 4182, pages 460–475. SpringerVerlag, 2006. 7. Sa¨ıd Radhouani, Joo Hwee Lim, Jean-Pierre Chevallet, and Gilles Falquet. Combining textual and visual ontologies to solve medical multimodal queries. In International Conference on Multimedia & Expo IEEE ICME 2006, Toronto Canada, 2006. 8. T. Westerveld. Image retrieval : Content versus context. In Recherche d’Information Assistee par Ordinateur, 2000. 9. R. Zhao and W. Grosky. Narrowing the semantic gap - improved text-based web document retrieval using visual features. IEEE Transactions on Multimedia, 4(2):189– 200, 2002.