Topology and Medical Imaging
Introduction
June 15, 2010
Capital Fund Management
Manifold Surgery Geometrically-Accurate Topologically-Correct Cortical Segmentation
Florent Ségonne Athinoula Martinos Center (MIT, Harvard) - Imagine Laboratory (ENPC)
Manifold Surgery
Introduction
Topology and Medical Imaging
Manifold Surgery
Most macroscopic brain structures have the topology of a sphere.
Introduction
Topology and Medical Imaging
Manifold Surgery
Cortical surface can be considered to have the topology of a sphere.
Local functional organization of cortex is largely 2-dimensional [Sereno-95, Science].
Introduction
Topology and Medical Imaging
Manifold Surgery
Cortical surface can be considered to have the topology of a sphere.
Local functional organization of cortex is largely 2-dimensional [Sereno-95, Science].
Goal Achieve geometrically-accurate and topologically-correct segmentations of anatomical structures from medical images.
Introduction
Topology and Medical Imaging
Outline Introduction Why is a Model of the Cortical Surface Usefull Why Segmentation is Hard State of the Art Topology and Medical Imaging Topology and Differential Geometry Non-separating Loops Topology and Discrete Imaging Manifold Surgery Approach Location of Topological Defects Optimal Topological Correction
Manifold Surgery
Introduction
Topology and Medical Imaging
Manifold Surgery
The Segmentation Problem Given brain imaging data (T1-, T2-, PD-, DTI-weighted, atlases), locate accurately the anatomical structures & substructures so that each structure has the proper topology and correct relationships to its neighbors.
Introduction
Topology and Medical Imaging
Manifold Surgery
The Segmentation Problem Given brain imaging data (T1-, T2-, PD-, DTI-weighted, atlases), locate accurately the anatomical structures & substructures so that each structure has the proper topology and correct relationships to its neighbors. • Digital Image Segmentation ⇒ assign to each voxel a correct labeling.
Introduction
Topology and Medical Imaging
Manifold Surgery
The Segmentation Problem Given brain imaging data (T1-, T2-, PD-, DTI-weighted, atlases), locate accurately the anatomical structures & substructures so that each structure has the proper topology and correct relationships to its neighbors. • Digital Image Segmentation ⇒ assign to each voxel a correct labeling. • Surface Segmentation ⇒ generate geometrically accurate and topologically correct triangulations.
Manifold Surgery
Topology and Medical Imaging
Introduction
Why is a Model of the Cortical Surface Usefull • Shape Analysis • Presurgical Planning • Statistical analysis of morphometric properties • Aging • Neurodegenerative diseases • Longitudinal studies of structural changes • Hemispheric asymmetry
Visualization
Spherical atlas
Functional activity
Cortical Parcellation
Introduction
Topology and Medical Imaging
Manifold Surgery
Why Segmentation is hard!
• Partial voluming: a single voxel may
contain more than one tissue type. • Bias field: effective flip angle or
sensitivity of receive coil may vary accross space. • Tissue inhomogeneities: even within
tissue type (e.g. cortical gray matter), intrinsic properties such as T1, PD can vary (up to 20%). • subject motion. Distribution of voxel intensities in T1-weighted Images.
• susceptibility artifacts.
Introduction
Topology and Medical Imaging
Why Segmentation is hard! Assigning tissue classes to voxels can be difficult
Partial volume effect is often the cause for an incorrect topology
Manifold Surgery
Introduction
Topology and Medical Imaging
Manifold Surgery
Previous Work
Essentially two types of Approaches: 1. Segmentation under topological constraint: integrate a ‘hard’ segmentation constraint into the segmentation process
2. Retrospective topology correction of segmentations: identification of topological defects and retrospective correction of the segmentation
Topology and Medical Imaging
Introduction
Segmentation under topological constraint • Active contours
Triangulations [Dale-99, Davatzikos-96, MacDonald-00]
Level sets and Topologically constrained level-sets [Zeng-99, Han-03, Ségonne-08] • Homotopic digital deformations [Mangin-95, Poupon-98, Bazin-05] • Segmentation by registration and vector fields [Karacli-04, Christensen-97]
Main Drawback • Sensitivity to initial condition • Local decision to preserve topology often lead to large
geometrical errors
Manifold Surgery
Manifold Surgery
Topology and Medical Imaging
Introduction
Segmentation under topological constraint: example
No topological constraint
Topology-preserving [Han-02]
Genus-preserving [Ségonne-08]
Topology and Medical Imaging
Introduction
Retrospective Topology Correction of Segmentations
• Digital binary images [Shattuck-01, Han-02, Kriegeskorte-01, Ségonne-03] • Triangulations [Guskov-01, Fischl-01, Ségonne-07]
Main Drawback • Location of the topological defects is hard to control • Correction of the defects might not be optimal
Manifold Surgery
Manifold Surgery
Topology and Medical Imaging
Introduction
Retrospective Topology Correction of Segmentations Difficulty of Finding the Correct Solution 1. Necessity to integrate additional information
Topological Defect
Inaccurate correction
Accurate correction
2. Solution is not necessarily obvious
Topological Defect
Sagital view
Correct Defect
Sagital view
Introduction
Topology and Medical Imaging
Manifold Surgery
General Notions Topology Study of shape properties preserved through deformations, twistings, and stretchings, but no tearings [Massey 1967]. • Topology is a continuous notion.
Introduction
Topology and Medical Imaging
Manifold Surgery
General Notions Topology Study of shape properties preserved through deformations, twistings, and stretchings, but no tearings [Massey 1967]. • Topology is a continuous notion.
Sampling can break the topology or create self-intersections.
Continuity is not easy to define.
Introduction
Topology and Medical Imaging
Manifold Surgery
General Notions Topology Study of shape properties preserved through deformations, twistings, and stretchings, but no tearings [Massey 1967]. • Topology is a continuous notion. • Topology studies the number of holes, not their position.
Introduction
Topology and Medical Imaging
Manifold Surgery
General Notions Topology Study of shape properties preserved through deformations, twistings, and stretchings, but no tearings [Massey 1967]. • Topology is a continuous notion. • Topology studies the number of holes, not their position.
Two levels of topological equivalence • Intrinsic Topology: properties preserved by homeomorphisms
(ignore the embedding space). • Homotopy type: continuous transformations in the embedding
space (Algebraic Topology).
Introduction
Topology and Medical Imaging
Link between Topology and Differential Geometry Any compact connected orientable surface is homeomorphic to a sphere with some number of handles (i.e. the genus g):
Manifold Surgery
Introduction
Topology and Medical Imaging
Link between Topology and Differential Geometry Any compact connected orientable surface is homeomorphic to a sphere with some number of handles (i.e. the genus g): • Every compact surface C has a rectangular decomposition.
Manifold Surgery
Introduction
Topology and Medical Imaging
Link between Topology and Differential Geometry
Manifold Surgery
Introduction
Topology and Medical Imaging
Manifold Surgery
Link between Topology and Differential Geometry Any compact connected orientable surface is homeomorphic to a sphere with some number of handles (i.e. the genus g): • Every compact surface C has a rectangular decomposition. • If D is a rectangular decomposition of a compact surface C, let v,
e, and f be the number of vertices, edges, and faces in D. Then the Euler-Characteristic χC = v − e + f is the same for every rectangular decomposition of C.
Introduction
Topology and Medical Imaging
Manifold Surgery
Link between Topology and Differential Geometry Any compact connected orientable surface is homeomorphic to a sphere with some number of handles (i.e. the genus g): • Every compact surface C has a rectangular decomposition. • If D is a rectangular decomposition of a compact surface C, let v,
e, and f be the number of vertices, edges, and faces in D. Then the Euler-Characteristic χC = v − e + f is the same for every rectangular decomposition of C.
• If C and S are two compact orientable surfaces, χC = χS iff C and
S are diffeomorphic: The Euler-Characteristic is a topological invariant.
Manifold Surgery
Topology and Medical Imaging
Introduction
Link between Topology and Differential Geometry
v = 8, e = 12, f = 6
v = 8, e = 18, f = 12
v = 16, e = 32, f = 16
Introduction
Topology and Medical Imaging
Manifold Surgery
Link between Topology and Differential Geometry Any compact connected orientable surface is homeomorphic to a sphere with some number of handles (i.e. the genus g): • Every compact surface C has a rectangular decomposition. • If D is a rectangular decomposition of a compact surface C, let v,
e, and f be the number of vertices, edges, and faces in D. Then the Euler-Characteristic χC = v − e + f is the same for every rectangular decomposition of C.
• If C and S are two compact orientable surfaces, χC = χS iff C and
S are diffeomorphic: The Euler-Characteristic is a topological invariant.
• The total Gausian curvature of a compact orientable geometric surface C is 2πχC . [Gauss-Bonnet Theorem].
Introduction
Manifold Surgery
Topology and Medical Imaging
Euler characteristic C and genus g Euler characteristic χC = v − e + f
Table: Euler Characteristic
surface sphere torus sphere with n handles disk disk with n handles
χ 2 0 2 − 2n 1 1 − 2n
g 0 1 n 0 n
Introduction
Manifold Surgery
Topology and Medical Imaging
Euler characteristic C and genus g Euler characteristic χC = v − e + f • very useful to check the topology type of a triangulation.
Table: Euler Characteristic
surface sphere torus sphere with n handles disk disk with n handles
χ 2 0 2 − 2n 1 1 − 2n
g 0 1 n 0 n
Introduction
Manifold Surgery
Topology and Medical Imaging
Euler characteristic C and genus g Euler characteristic χC = v − e + f • very useful to check the topology type of a triangulation. • no localization of the topological defects.
Table: Euler Characteristic
surface sphere torus sphere with n handles disk disk with n handles
χ 2 0 2 − 2n 1 1 − 2n
g 0 1 n 0 n
Introduction
Manifold Surgery
Topology and Medical Imaging
Euler characteristic C and genus g Euler characteristic χC = v − e + f • very useful to check the topology type of a triangulation. • no localization of the topological defects. • related to the genus of a surface g = 1 − χ2 . The genus is
equivalent to the number of handles. Table: Euler Characteristic
surface sphere torus sphere with n handles disk disk with n handles
χ 2 0 2 − 2n 1 1 − 2n
g 0 1 n 0 n
Topology and Medical Imaging
Introduction
Topological Defects, Duality Foreground/Background Topological Defect = Deviation from Spherical Topology
• cavities • disconnected components • handles
Duality Foreground/Background: • cavity ⇔ background disconnected component • disconnected component ⇔ background cavity • foreground handle ⇔ background handle
Manifold Surgery
Introduction
Topology and Medical Imaging
Manifold Surgery
Topological Defects, Duality and Non-separating Loop • Presence of a handle is characterized by the existence of a
non-separating loop (Algebraic Topology). • At each foreground handle corresponds a background handle. • This provides a way to correct the topology:
- cutting the background handle. - cutting the associated foreground handle, i.e. filling the hole.
Introduction
Topology and Medical Imaging
Topology in Discrete Imaging
Adapt concept of continuity to a discrete framework
Surfaces and 3D images
Manifold Surgery
Introduction
Topology and Medical Imaging
Topology in Discrete Imaging
Adapt concept of continuity to a discrete framework ⇒ use the notion of connectivity instead Surfaces and 3D images
Manifold Surgery
Introduction
Topology and Medical Imaging
Topology in Discrete Imaging
Adapt concept of continuity to a discrete framework ⇒ use the notion of connectivity instead Surfaces and 3D images Tesselations ⇒ Euler-characteristic
3D digital images ⇒ Theory of Digital topology
Manifold Surgery
Introduction
Topology and Medical Imaging
Manifold Surgery
Manifold Surgery
Introduction
Topology and Medical Imaging
Approach: Retrospective Topology Correction
FreeSurfer Processing Pipeline
Manifold Surgery
Introduction
Topology and Medical Imaging
Manifold Surgery
Manifold Surgery Program
1. Identification of defects by Homeomorphic mapping
Shrink-Wrap Methods cannot reach deep folds
Expand and project the manifold onto the sphere
2. Optimally correct the topological defects - use all the available information (curvature, intensity profile) - probabilistic framework
Introduction
Topology and Medical Imaging
Identification of defects by Homeomorphic mapping
Manifold Surgery
Manifold Surgery
Topology and Medical Imaging
Introduction
Definition: Homeomorphism Homeomorphic mapping M : C → S • transformation that is continuous, one-to-one, and with a
continuous inverse M−1 • strictly positive Jacobian: ∀x JM (x) = dM dx (x) > 0 • Jacobian is related to the areal distortion dAS = JM dAC • For triangulations, the areal distortion is an approximation of the
Jacobian JM ≈
AS AC
Introduction
Topology and Medical Imaging
Quasi-homeomorphic mapping Cortical surface C with correct topology is homeomorphic to the sphere S: • continuous, one-to-one, continuous inverse • strictly positive Jacobian
Manifold Surgery
Topology and Medical Imaging
Introduction
Manifold Surgery
Quasi-homeomorphic mapping Cortical surface C with correct topology is homeomorphic to the sphere S: • continuous, one-to-one, continuous inverse • strictly positive Jacobian
In the presence of topological defects (χC < 2), no such mapping exists: • Search for a mapping that minimizes the regions with negative
Jacobian (quasi-homeomorphic mapping). • Topological defects will be located in regions with negative
Jacobian.
Introduction
Manifold Surgery
Topology and Medical Imaging
Identification of defects as a minimization problem
Identification of Topological Defects • Find a mapping Mo that is maximally homeomorphic:
Mo = arg min EM = arg min
Z C
f (JM (x))dx
with f a function that penalizes regions with negative jacobian. • Identify topological defects as non-homeomorphic regions:
D = M−1 o x s.t. JM (x) <
Topology and Medical Imaging
Introduction
Discrete Optimization on the Tesselation of C
• The mapping is defined on the vertices vi of C • Initialize the mapping by inflating and projecting onto S • Minimize EM by gradient descent: dvi = −∇S EM dt
EM =
#f X log 1 + ekRi i=1
with Ri =
Ati Aoi
=
k
− Ri
signed area at t of ith face in S area of ith face in M
Manifold Surgery
Topology and Medical Imaging
Introduction
Discrete Optimization on the Tesselation of C 1. Numerical Implementation : Computation of the Gradient ∂EM ∂Ati ∂EM −1 ∂EM = with = 0 ∂vk ∂Ati ∂vk ∂Ati Ai (1 + e−kRi ) ∂Ati ∂vk ∂Ati ∂ai
∂Ati ∂ai ∂Ati ∂bi + ∂ai ∂vk ∂bi ∂vk ∂Ati = bi ∧ ni , = ni ∧ ai ∂bi =
2. Implementation parameters • • • •
Cortical surface constains ≈ 100 topological defects Most defects are small (less than 100 faces) Currently, the whole identification process takes less 5mns. Less than 1% of negative semi-definite area.
Manifold Surgery
Introduction
Manifold Surgery
Topology and Medical Imaging
Minimization procedure different steps • inflation • projection • minimization of negative regions • defect = set of overlapping faces • back-projection
Introduction
Topology and Medical Imaging
Concrete example Topological defect D containing one handle, i.e. χD = −1
Manifold Surgery
Introduction
Topology and Medical Imaging
Optimally correct the topological defects
Manifold Surgery
Introduction
Topology and Medical Imaging
The retesselation problem
Finding a valid retessellation TD for each defect D
Manifold Surgery
Introduction
Topology and Medical Imaging
The retesselation problem
Finding a valid retessellation TD for each defect D • Topologically correct
Manifold Surgery
Introduction
Topology and Medical Imaging
The retesselation problem
Finding a valid retessellation TD for each defect D • Topologically correct
Use the concept of non-separating loops
Manifold Surgery
Topology and Medical Imaging
Introduction
The retesselation problem
Finding a valid retessellation TD for each defect D • Topologically correct
Use the concept of non-separating loops • Geometrically accurate, i.e. smoothness, location, no
self-intersection
Manifold Surgery
Topology and Medical Imaging
Introduction
The retesselation problem
Finding a valid retessellation TD for each defect D • Topologically correct
Use the concept of non-separating loops • Geometrically accurate, i.e. smoothness, location, no
self-intersection Use information from the cortical representation C
Manifold Surgery
Introduction
Topology and Medical Imaging
Manifold Surgery
Retesselating through non-separating loops • Generate ‘random’ non-separating loops on the graph of faces • Two complementary loops associated per handle • Discard the loop faces, and seal with appropriate patch • Local active contour optimization of the surface
Topology and Medical Imaging
Introduction
Measuring the accuracy of a potential retesselation Di Cortical surface:
Manifold Surgery
Topology and Medical Imaging
Introduction
Measuring the accuracy of a potential retesselation Di Cortical surface: • Smooth surface
Manifold Surgery
Topology and Medical Imaging
Introduction
Measuring the accuracy of a potential retesselation Di Cortical surface: • Smooth surface
⇒ Prior on the Surface p(Di |C)
Manifold Surgery
Topology and Medical Imaging
Introduction
Measuring the accuracy of a potential retesselation Di Cortical surface: • Smooth surface
⇒ Prior on the Surface p(Di |C)
• Clear separation between gray matter and white matter
Manifold Surgery
Topology and Medical Imaging
Introduction
Measuring the accuracy of a potential retesselation Di Cortical surface: • Smooth surface
⇒ Prior on the Surface p(Di |C)
• Clear separation between gray matter and white matter
⇒ Likelihood Term p(Di |C, I)
Manifold Surgery
Topology and Medical Imaging
Introduction
Measuring the accuracy of a potential retesselation Di Cortical surface: • Smooth surface
⇒ Prior on the Surface p(Di |C)
• Clear separation between gray matter and white matter
⇒ Likelihood Term p(Di |C, I) • No self-intersection
Manifold Surgery
Topology and Medical Imaging
Introduction
Measuring the accuracy of a potential retesselation Di Cortical surface: • Smooth surface
⇒ Prior on the Surface p(Di |C)
• Clear separation between gray matter and white matter
⇒ Likelihood Term p(Di |C, I) • No self-intersection
⇒ Search for the MAP estimate in a Bayesian Framework p(Di |C, I) ∝ p(Di |C, I)p(Di |C)
Manifold Surgery
Manifold Surgery
Topology and Medical Imaging
Introduction
Measuring the accuracy of a potential retesselation Di Definition of the likelihood term p(I|C, Di ) =
Y x∈C−
|
1
pw (I(x)|C, Di ) N
Y
1
pg (I(x)|C, Di ) N
x∈C+
{z volume-based information
Locally estimated from MRI around D
Vi Y
1
p(gi (v), wi (v)|C, Di ) Vi
v=1
{z } }| surface-based information
Estimed from non-defective portion of C
Topology and Medical Imaging
Introduction
Manifold Surgery
Measuring the accuracy of a potential retesselation Di Curvature information through the prior term p(Di |C)
p(Di |C) =
Q Vi
1
Vi v=1 p(κ1 (v), κ2 (v)|C)
Estimed from non-defective portion of C
Topology and Medical Imaging
Introduction
Manifold Surgery
Generation of nonseparating loop Nonseparating loop = connected set of faces that does not divide the rest of the triangulation into two components [Guskov-01]. Loop Generation • Select random seed faces • Front propagation by Fast-Marching on the graph of faces • Min-Heap data structure ⇒ Complexity of O(n log(n) • Loop-Generation at front intersection (check Euler Char.) • Complementary loop-generation by complementary front
propagation
Topology and Medical Imaging
Introduction
Manifold Surgery
Reducing the genus: cutting and sealing the open surface
Sealing the Cut • Discard the faces of a nonseparating loop. • Attach pressellated disks to each open side of defect. • Optimize locally the closed surface.
Introduction
Manifold Surgery
Topology and Medical Imaging
Optimization using active contour patches Active Contour Optimization • Deform each corrected patch so as to maximize p(Di |C, I). i |C,I) • Difficulty of deriving exact Euler-Lagrange Equations ∂p(D . ∂v k
• Approximation using a simple Euler-scheme:
vk (t + dt) = vk + FS (t) + λI FI (t)
µ w σ g + µg σ w FI (t) = −I(v ) ∇I(vk ) k + σw } | σ g {z local threshold
• Evaluate convergence using p(Di |C, I) every few steps.
Manifold Surgery
Topology and Medical Imaging
Introduction
Reducing the genus: cutting and sealing the open surface
opening & sealing
random corrections
optimal correction
Topological defect containing 3 handles χD = −5
Introduction
Topology and Medical Imaging
Manifold Surgery
Numerical Implementation
Implementation parameters • C contains ≈ 100 topological defects D with n faces. • ‘Semi-random’ generation of n3 loops ( 3n seed faces) per defect. • Loop generation: complexity of O( n3 × n log(n)). • Active contour optimization: quite fast. • Slowing factor: self-intersection check and fitness computation. • Implementation in C++∗ : timing is less than 20 minutes. • The whole process can be parallelized.
(*) source code available at http://florent.segonne.free.fr/publications.
Topology and Medical Imaging
Introduction
Manifold Surgery
Results
• Methodology has been tested on very significant number of
dataset. • Part of the free software FreeSurfer∗ , used in more than 1000
hospitals and research labs. • Publications:[Ségonne-05, Ségonne-07] (*) Freesurfer is available at http://surfer.nmr.mgh.harvard.edu/.
Introduction
Topology and Medical Imaging
Questions ?
Manifold Surgery
