MESH DEFORMATION UNDER EXTERNAL AND INTERNAL

mise between time and segmentation efficiency. But recent fully-automatic methods have proven to be accurate and fast enough for clinical use. The Left Atrial ...
1MB taille 3 téléchargements 362 vues
MESH DEFORMATION UNDER EXTERNAL AND INTERNAL ENERGY ALGORITHM ; APPLICATION TO THE LEFT ATRIAL APPENDAGE Pol Grasland-Mongrain ∗

Olivier Ecabert† , J. Peters

ENS Cachan

Philips Research Aachen

ABSTRACT This paper present an algorithm which is based on the deformation of a mesh with a fixed topology. The principle was to inflate an high resolution mesh through the substructure we want to segment. It is made by minimizing two types of energy, external and internal. This algorithm has been applied on a part of the heart called Left Atrial Appendage . It has been tested on 17 patients. About 13 were satisfying enough, and there was only one major fail, due to previous modelization error. Index Terms— Left atrial appendage, LAA, Philips, segmentation, heart, cardiac model 1. INTRODUCTION The accurate segmentation of the heart, that is to say the process of assigning labels to region in the image, is an important goal today to detect diseases. In a model-based approach, this can be achieved by having a 3D model of the heart of the patient with annotated faces and vertices. Even if this segmentation could be manually done, it is practically almost impossible in a daily work. Most of the current techniques are using semi-automatic methods, which are a good compromise between time and segmentation efficiency. But recent fully-automatic methods have proven to be accurate and fast enough for clinical use. The Left Atrial Appendage (LAA), also known as left auricular appendix, is a substructure of the heart above the left ventricle, and connected to the left atrium. It is involved in various heart diseases, including thrombosis building, cardiac fibrillation. The LAA has an highly variable shape, often tubular, hooked and with a few lobes. Its size varies from 1 to 19 cm3 [?]. It is slightly different from the right atrium appendage which is more massive. Although the function of the LAA is not perfectly known, most physicians think today that it serves as a reservoir for the atrium, and helps the maintenance and the regulation of the heart function. It has been observed that its elimination causes various heart problem [?]. T. McInerney and D. Terzopoulos made a survey about deformable models in medical image analysis, in 1996 [?]. ∗ [email protected][email protected]

Deformable models, which are using bottom-up constraints (e.g. boundaries) with a top-down a priori knowledge (relative sizes, positions and orientations of parts of the heart, with expected variations), are consequently widely used for modeling heart. Y. Zheng et al. are describing an efficient method in [?] to modelize the four chambers of the heart (left and right ventricle, left and right atrium). This method is, unlike many others, fully automatic and quite fast. It has been tested on a very large dataset (323 volumes from 137 patients). The method presented in this article is included in a large software created by Philips. This software is based on an algorithm which segments the whole heart, including many substructures. The developed algorithm is widely described in [?], and has a segmentation success ratio of 138 over 140 patients. This method comes with a precomputed model of the heart made of vertices forming triangles. This model will be transformed in the way described aboved. 2. GLOBAL SEGMENTATION OF THE HEART The method comes with a precomputed model of the heart made of vertices forming triangles. This model will be transformed in the way described aboved. These steps, described in [?], are here only briefly reminded. Later we refer to this method simply as the current method. Segmentation Chain

New Image

1. Heart Detection

2. Parametric Adaptation (Similarity)

3. Parametric Adaptation (Piecewise Affine)

Segmented Image

4. Deformable Adaptation

Fig. 1. Segmentation chain in the current method

2.1. Heart Detection The Heart Detection algorithm uses a Generalized Hough Transform. The most probable position and scale of the heart are calculated. A few improvements like simplifying the image and not testing every position are done to make the process faster.

3. INFLATION ALGORITHM

2.2. Parametric Adaptation The steps 2 and 3 are quite similar: they consists in optimizing a parametric transformation (rotation, translation and scale). But while the transformation in the step 2 is computed and applied to the whole heart, the transformations in the step 3 are computed and applied independantly to each substructure (left and right ventricle, left and right atrium, etc). The interfaces have a special treatment which let the substructures disconnected. To find the transformation in each case, the following energy is computed : Eexternal =

T X

 wi

i=1

2 ) ∇I(xtarget target i · (xi − ci ) (1) )k k ∇I(xtarget i

where the sum is performed over the mesh triangles. This energy looks at each triangle centers ci ; these triangles centers detected at tends to be attracted towards target points xtarget i the object boundary. Some weights wi are large for reliably detected target points and small otherwise. The candidate points are detected in the image maximizing a boundary detection function Fi (ni , x), which is evaluated for each triangle along its normal vector ni . The point with the highest response is kept as the target point. Once all the target points are computed, a transformation which minimize the sum of the square of the distance between the centers of triangles and the target point is computed and applied. 2.3. Deformable Adaptation Each vertex is here allowed to move freely, and the mesh adaptation to a new image is there performed minimizing an energy function made of two contributions. The first one, called external energy, is the same as in the section 2.2 and attracts the model to the object boundaries in the image, whereas the internal energy, penalizes deviations of the deformed model: E = αEexternal + Einternal ,

(2)

with α a ponderation parameter. The internal energy will be responsible of (1) maintaining a suitable distribution of the vertices and (2) penalizing deviations between the current state of the mesh and the reference shape model. Einternal =

V X X

((v i − v j ) − (T [mi ] − T [mj ]))2 (3)

i=1 j∈N (i)

with N (i) the set of indices of the neighbor vertices of vertex v i , and mi the vertex coordinates of the reference model undergoing a geometric transformation T [.]. This transformation describes allowed global shape variations like e.g. rigid body motion or affine deformations. Mesh adaptation to a new image is thus performed by minimizing the equation (2).

The developed algorithm follows the previous steps described above. In this way, the position of the LAA is known before the beginning of the algorithm, and the gray value around this location are known. The principle of the algorithm is to have an high resolution mesh at the base of the LAA, which will inflate through the LAA. This inflation will be made by following two types of energy : one external, which adapts the mesh to the image, and one internal, which constrains the mesh to avoid odd shapes and behaviours, all with a fixed topology. In this way, “leakages” of the mesh in other organs than the LAA are avoided. We have tried two different external energies and four different internal. We could use one or any combination of the mentionned energies, with at least one external energy and one internal energy. 3.1. External Energies: Edge-based and Region-based The edge-based energy, described in the section [?] as the "external energy", couldn’t be properly used in our case. The borders of the LAA are indeed not a priori known, so the edgefeature detector was not reliable enough. That’s why we tried another energy, called region-based. This energy use a grayscale threshold between the LAA and the background. This threshold is done by minimizing the grayscale classification error, as evocated in [?], between the voxels of the left atrium (which have the same gray values as the LAA) and the myocardium (which have roughly the same gray values as the background around the LAA), these substructures being already segmented by the Philips algorithm. This energy will then look at each center of triangle. It looks at the gray value at this location, and determine thus if the center is inside or outside the LAA. If this center is inside, the algorithm will search for a target point outside the mesh (inflation of the mesh) ; if it is outside, the algorithm will search for a target point inside the mesh (deflation of the mesh). These target points are computed along the normal of the triangle. Then the algorithm searches for target points farther until either (1) it reaches the maximum number of target points allowed, (2) the gray value indicates that we have gone through the interface between LAA and background, or (3) the target point is already segmented, and belongs to another substructure. In our tests, we choose target points every millimeter along the normal of the triangle, and 3 target points tested maximum. 3.2. Internal Energies: Triangle Regularization, Curvature, N-Gon Regularization, Mesh Reference The mesh reference, described in the section [?] as the "internal energy" and that we called Mesh Reference, penalizes

any deformation of the model. As our mesh is intended to be highly deformed, we tried other internal energies that we called Triangle Regularization, Curvature and N-Gon Regularization.

where N(i) are the neighbors of the ith vertex and N (N (i)) is the vector normal to the plan passing through the neighbors vertices. 3.2.3. N-Gon Regularization Energy

3.2.1. Triangle Regularization Energy This energy tries to approximate all the triangles into equilateral triangles. The algorithm looks for all triangles. For each triangle, an equilateral triangle is created. It is then rotated and isotropically scaled to minimize the distance between the corners of the initial triangle and the corners of the equilateral triangle (see scheme 2). The vertices of the initial triangle are thus subjected to forces which “pull” them into an equilateral triangle (strings on the schemes). This energy can be written in

The principle is very similar to the triangle regularization energy. Instead of considering triangles, it looks at the vertices. We have indeed noticed that most of the loops, evocated in 3.2.2, are appearing in the interfaces, where more than 6 triangles are neighbouring, and cannot be all equilateral. We called here the vertices “N-Gon”, with N the number of neighbours of the reference vertex. For each N-Gon, we count the number of neighbours, and we create a regular N-Gon - that is to say, regularly distributed vertices at an equidistance from a reference vertex. This regular N-Gon is then rotated and scaled to minimize the distance between the neighbours of the initial N-Gon and the ones of the regular N-Gon (see scheme 3). The vertices of the neighbours of the initial N-Gon are thus subjected to forces which “pull” them into a regular N-Gon.

Fig. 2. Triangle Regularization Energy: some forces (strings on the schemes) are trying to approximate each triangle by an equilateral triangle. this way: Etrg reg =

V X 3 X

 vi,j − vi,(j+1)mod.3 − T ∗ tj − t(j+1)mod.3

i=1 j=1

(4) where T ∗ is the optimized transformation, vi,j the j th corners of the triangle i and the tj the 3 corners of the equilateral triangle. The “mod.3” is to indicated that the 4th corner of the triangle is identical to the 1st .

During the inflation, we often found some peaks, which can degenerate after into loops. Worse, once a loop is created, the region-based energy tends to make it worse: as we suppose that the triangle points outwards, the region-based energy transform it in the “wrong way”. We have thus thought about an energy which would remove peaks and smooth surface. For each vertex, the algorithm searches for the neighboors, and computes a plane which goes through them (if there are more than 3 neighboors, the algorithm computes the best fitting plane with a least-square method). The initial vertex will then be ”pulled” along the normal of the plane going through itself. This energy can be written this way: Ecurv =

X  2 N (N (i))T · (v j − v i )

i=1 j∈N (i)

This energy can be written in this way: EN-Gon reg =

V X X

((v i − v j ) − (T ∗ [g i − g j ]))2 (6)

i=1 j∈N (i)

where T ∗ is the optimized transformation (linear), gi the vertex in the center of the regular N-Gon and gj its neighbors.

3.2.2. Curvature Energy

V X

Fig. 3. N-Gon Regularization Energy: some forces (strings on the schemes) are trying to approximate each N-Gon by a 2regular N-Gon.

(5)

3.2.4. Mesh Reference This energy is the same as in the current current method (see equation 3): it has a reference mesh, and each deviation from this mesh is penalized. There one difference however: in the LAA-inflation algorithm, the reference mesh is regularly updated to the the current mesh. That is, at each iteration, the LAA grows, and the mesh created becomes the reference mesh. 3.3. Loop Repair During Growing The growing made appear some loops around the mesh, as said before. To deal with this problem, the intersecting faces are detected, with a method created by [?]. It searches then for its neighbours to the N-th order, and applies to them a

smoothing energy. Then, the intersecting faces are again detected, and further inflation are allowed only if this number is small enough. We determined that the best smoothing energy was the Mesh Reference energy. However, the repair is not efficient enough, and the loop are always reappearing at the same place at the next iteration.

4. MAIN RESULTS

The algorithm has been tested on a Intel Xeon at 2,4 Ghz, with 3,37 GB of RAM. The current method takes about 20 seconds, and the inflation method presented here about 30 seconds.

α=1 α=2 Numerical results for 17 patients are presented under. The first bar (specificity) represents the procentage of voxels belonging to the LAA which have been segmented by the algorithm, and the second bar (quality) the procentage of voxels segmented by the algorithm which really belong to the LAA. The main comment about these results is the difference beLeft Atrial Appendage Inflation Results

The region-based energy seems to be reliable though, and gives better results than the edge-based which doesn’t have specific edge detection features. For the internal energies, the mesh reference energy has been shown surprisingly efficient, for both of the growing and the loop correction. The step-bystep inflation, with the update of the reference mesh at each iteration, could explain this success.

Specificity = True Pos. / (True Pos. + False Neg.) Quality = True Pos. / (True Pos. + False Pos.) 100 80 60 40

20

The table 4 shows an example of inflation. During the growing, the weight of the external energy with regards to the internal energy was increasing every 5 iterations, from 0.2 to 2

0

Fig. 4. Numerical evaluation of the segmentation of the LAA by the algorithm. tween the two bars. The mesh doesn’t actually grow far enough, until the end of the LAA (low specificity), but almost all the voxels which are segmented really belong to the LAA (high quality). There is mainly one major fail, the patient 14 (very low specifity and quality), and three or four patient don’t show satisfying enough results. These fails are mainly due to segmentation errors in the current method. 5. CONCLUSION AND FUTURE WORK

Before growing

α = 0.2

The current main method in Philips is done in four steps, each one can repair small failures of the previous one: heart detection, global parametric adaptation, semi-global parametric adaptation, and deformable adaptation. The method is

quite efficient, but can have small modelization errors. The Left Atrial Appendage, highly deformable substructure of the heart, is segmented after applying the current method. The developed algorithm inflates a flat mesh through the LAA, following an internal energy called Mesh Reference and an external energy region-based. The results are satisfying in the majority of cases, but they can be improved, especially on the specificity. The algoritm could give better results with a better loop correction. 6. ACKNOWLEDGEMENTS

titative study of age-related changes in 500 autopsy hearts: implications for echocardiographic examination”", journal = "Circulation 96", volume = "9", pages = "3112–3115", year = "1997", url = "http://circ.ahajournals.org/cgi/content/full/96/9/3112"

@articlestollberger03, author = "Claudia Stöllberger and Birke Schneider and Josef Finsterer", title = "”Elimination of the Left Atrial Appendage To Prevent Stroke or Embolism?”", journal = "Chest 124", volume = "6", pages = "2356–2362", year = "dec 2003", url= "http://www.chestjournal.org/con

@articleblackshear96, author = "Joseph L. Blackshear and John A. Odell", title = "”Appendage Obliteration to This research was supported by Philips ForschungslaboratoReduce Stroke in Cardiac Surgical Patients With Atrial rien. We would like to thank the whole XIS team, especially Fibrillation”", journal = "Ann. Thorac. Surg.", volume H. Lehman and R. Kneser. In addition, we wish to thank P. = "61", pages = "755 - 759", year = "Feb 1996", url = Cignoni and his team for creating the MeshLabs software. Fi"http://ats.ctsnetjournals.org/cgi/content/full/61/2/755" nally, we wish to cite a teacher, H. Delingette, which created @articlemcinerney96, author = "T. McInerney and Demetri the connection between us ! Terzopoulos", title = "”Deformable models in medical image @articlepeters05, author = "Jochen Peters and Olivier analysis: a survey”", journal = "Medical Image Analysis", Ecabert and Jürgen Weese", title = "”Feature learning for volume = "1", number = "2", pages = "91-108", year = model-based image segmentation”", journal = "PFL-A Tech"1996" nical Note 2005-00582, Philips Research Laboratories, Aachen", @articlezheng08, author = "Y. Zheng and A. Barbu and month = "July", year = "2005" B. Georgescu and M. Scheuering and D. Comaniciu", title @articleecabert06, author = "Olivier Ecabert and Jochen = "”Four-Chamber Heart Modeling and Automatic SegmenPeters and Jürgen Weese", title = "”Modeling shape varitation for 3D Cardiac CT Volumes Using Marginal Space ability for full heart segmentation in cardiac computedLearning and Steerable Features”", journal = "IEEE Transtomography images”", journal = "Proc. of SPIE Medical actions on Medical Imaging", year = "2008" Imaging", year = "2006" @articlezheng09, author = "Y. Zheng and B. Georgescu @articleecabert08, author = "Olivier Ecabert and Jochen and F. Vega-Higuera and D. Comaniciu", title = "”Left VenPeters and H. Schramm and C. Lorenz and J. von Berg and tricle Endocardium Segmentation for Cardiac CT Columes M. Walker and M. Vembar and M. Olszewski and K. SubraUsing an Optimal Smooth Surface”", journal = "Medical manyan and G. Lavi and Jürgen Weese", title = "”Automatic Imaging 2009 : Image Processing Proc. of SPIE", volume = Model-Based Segmentation of the Heart in CT Images”", "7259, 72593V", year = "2009" journal = "IEEE Transactions on Medical Imaging", volume @articlelotjonen04, author = "J. Lötjönen and S. Kivistö = "27", year = "sept 2008" and J. Koikkalainen and D. Smutek and K. Laurema", title @articleweese00, author = " Jürgen Weese and V. Pekar = "”Statistical shape model of atria and ventricles and epiand M.R. Kaus and C. Lorenz and P. Rösch", title = "”3D cardium from short- and long-axis MR images”", journal = medicale image segmentation with shape constrained de"Medical Image Analysis", volume = "vol 8", pages = "371formable models”", journal = "PFL-H Report 875/00, Philips 286", year = "2004" Research Laboratories, Hamburg", year = "dec 2000" @articlecootes94, author = "T. F. Cootes and A. Hill and @articleberg03, author = " J von Berg and M.R. Kaus and C. J. Taylor and J. Haslam", title = "”The use of active shape V. Pekar and A. Franz", title = "”Modelling gray value apmodels for locating structures in medical images”", journal = prearance for object surface detection”", journal = "PFL-H "Image Vis. Comput.", volume = "12 and num. 7", pages = Report 1631/2003, Philips Research Laboratories, Hamburg", "355-366", year = "1994" year = "feb 2003" @articlestegmann05, author = "M. B. Stegmann and D. @articlekaus04, author = " M.R. Kaus and V. Pekar", title Pedersen", title = "”Bi-temporal 3-D active appearance mod= "”Automated organ delineation for radiation therapy planels with applications to unsupervised ejection fraction estimaning”", journal = "PFL-H Technical Note 42/03, Philips Retion”", journal = "Proc. SPIE Med. Imag.", volume = "5747", search Laboratories, Hamburg", year = "jan 2004" pages = "336-350", year = "2005" @articleveinot97, author = "John P. Veinot and Phillip J. Harrity and Federico Gentile and Bijoy K. Khandheria @articlevanassen08, author = "H. C. van Assen and M. G. and Kent R. Bailey and Jeffrey T. Eickholt and James B. Danilouchkine and M. S. Dirksen and J. H. C. Reiber and B. Seward and A. Jamil Tajik and William D. Edwards", title P. F. Lelieveldt", title = "”A 3-D Active Shape Model Driven = "”Anatomy of the normal left atrial appendage: a quanby Fuzzy Inference: Application to Cardiac CT and MR”",

journal = "IEEE Transactions on Information Technology in Biomedicine", volume = "12 and num. 5", year = "sept 2008" @articleyoung09, author = "A. A. Young and A. F. Frangi", title = "”Computational cardiac atlases: from patient to population and back”", journal = "Experimental Physiology", volume = "94.5", pages = "578-596", year = "april 2009" @articlehyvarinen00, author = "A. Hyvärinen and E. Oja", title = "”Independent Component Analysis: Algorithms and Applications”", journal = "Neural Networks", volume = "13 (4-5)", pages = "411-430", year = "2000" @articlekass88, author = "Michael Kass and Andrew P. Witkin and Demetri Terzopoulos", title = "”Snakes: Active contour models”", journal = "International Journal of Computer Vision", volume = "1(4)", pages = "321-331", year = "1988", @articlemeshlab, author = "Cignoni, Paolo and Corsini, Massimiliano and Ranzuglia, Guido", title = "MeshLab: an Open-Source 3D Mesh Processing System", journal = "ERCIM News", number = "73", pages = "45–46", month = "April", year = "2008", url = "http://vcg.isti.cnr.it/Publications/2008/CCR08" @articlecaselles95, author = "Vicent Caselles and Ron Kimmel and Guillermo Sapiro", title = "Geodesic Active Contours", journal = "International Journal of Computer Vision", year = "1995", volume = "22", pages = "61–79" @articleguigue03, author = "Philippe Guigue and Olivier Devillers", title = "”Fast and Robust Triangle-Triangle Overlap Test using Orientation Predicates”", journal = "Journal of graphics, gpu, and game tools", volume = "8", number = "1", pages = "25-42", year = "2003",