Techniques to Correct for Soft Tissue Deformations during Image-Guided Brain Surgery

Michael I. Miga, Tuhin K. Sinha, David M. Cash Vanderbilt University, Department of Biomedical Engineering, Nashville, TN http://bmlweb.vuse.vanderbilt.edu

a contribution to a book entitled: Biomechanics Applied to Computer Assisted Surgery

ABSTRACT During image-guided surgery, patient-to-image misalignment becomes exacerbated by common surgical events such as retraction, resection, sag due to organ weight, or from the use of hyperosmotic drugs. One strategy to compensate for this mis-registration is to employ biomechanical models in conjunction with low-cost intraoperatively-acquired data (e.g. laser-range scanning, and co-registered ultrasonic data) to update the preoperative images and account for operating room (OR) actions. In this chapter, a framework for non-rigidly registering preoperative image-data to the patient during surgery is presented within the context of intra-cranial neurosurgery. The central tenets to the approach involve an organ-based registration framework driven by data acquired by a laser range scanner which is coupled to a biomechanical finite element model. Results are presented that demonstrate the techniques. INTRODUCTION The realization of co-registration, i.e. alignment, among patient, OR physical space, and preoperative image series such that surgical guidance feedback is provided by real-time visual displays has become somewhat commonplace in the neurosurgical theatre. As can be expected, improved computer-based surgical systems are increasingly on the horizon and their integration into treatment and reliance on for navigational assistance has made this technology essential. The major technical issues addressing image-guided surgery (IGS [1]) are the complexity and degree of feedback that characterize these advanced systems. In addition, the correct balance among cost, practicality, versatility, and feedback becomes an important concern in the development of these systems and numerous approaches are being investigated. By a considerable margin, IGS principles have found the most widespread adoption within the neurosurgical domain. This has been largely precipitated by several factors: (1) the critical need to guide surgery with respect to sensitive cortical and sub-cortical tissue structures, (2) the advantage of the constrained geometrical environment provided by the rigid cranium, (3) its peripheral location relative to the torso, and (4) the relative ease of identifying similar fiducials or landmarks (natural and/or synthetic) on the cranium and within preoperative image volumes. The foundation for many of the IGS systems developed in the 1980's and 1990's was the reliance on transformations between coordinate spaces that assumed objects were rigid structures. Although improvements in clinical outcomes have been afforded by these IGS systems (particularly within neurosurgery), in recent reports it has been shown that significant intra-operative soft-tissue deformations can ensue from common surgical loading conditions such as retraction, resection, gravity, etc. [2-4]. As a result, alignment degradation between patient and the MR/CT image volume can occur. The extent of this mis-registration can significantly compromise the guidance fidelity and in some cases lead to serious surgical error. In response to these data, there has been a concerted effort to account for intra-operative deformation during surgery for the improvement of surgical navigation systems. Several medical centers are deploying intra-operative magnetic resonance (iMR) imaging [3, 5, 6] and are developing sophisticated methods of visualization in the OR. Although conceptually appealing, the exorbitant cost and cumbersome nature of such systems have left their widespread adoption unclear at this time. Alternative imaging strategies such as co-registered intra-operative ultrasound (iUS) are also under investigation [7-9]. Although not capable of whole-volume imaging, the locally reconstructed image volumes generated by iUS can provide real-time guidance feedback during surgery. The quality of that feedback during surgery is still being assessed. One possible alternative to whole-brain MR imaging could be to integrate low-cost readily available intraoperative data in a framework that updates the preoperative image database via a biomechanical model. In this approach, the solution fidelity of computational models in conjunction with non-intrusive intra-operative data acquisition could serve as a means for updating all high resolution preoperative-based spatially encoded data (e.g. positron emission tomography, electroencephalography, functional MR imaging, and MR spectroscopy) to reflect

current OR conditions [10]. Preliminary studies with retrospective updating of gravity-induced brain shifts have shown that computational models can capture 70-80% of cortical surface deformations [11] as well as subsurface deformations [12]. Other more quantitative validations with subsurface data have been performed in porcine experiments and have demonstrated similar accuracy [13, 14]. In addition, other independent investigators have been developing similar approaches with comparable results [12, 15-19]. While the work in model-based correction strategies for deformation have been encouraging, it is unlikely that model-updating alone is sufficient to account for all forms of surgical shift to sub-millimetric accuracy. But rather, model-updating strategies can serve as one component of a new paradigm focused at improving surgical therapy. More specifically, novel localized imaging tools based on ultrasound, electrical impedance, and optical signatures of cancerous tissue are increasingly on the horizon and must warrant consideration when assessing the future of image-guided surgical treatment. For example, several studies have been performed using optical spectroscopy to assist in demarcating tumor borders during neurosurgery [20, 21]. This type of technique represents a minimally invasive localized imaging method focused at finding pathologic boundaries during surgery. In the new paradigm of technology-guided therapy, surgeons could use models in conjunction with minimally cumbersome spatial digitizers (e.g. laser range scanning, optical digitization probes, co-registered ultrasound, etc.) to account for large tissue movements and to assist in resecting bulk pathology while localized imaging units using techniques such as optical spectroscopy will provide the fine-scale resection precision. This vision of surgery is elegant, costeffective, unencumbered, and capable of wide-scale adoption by medical centers. In the future, this methodology may grow to be more accurate for the resection of pathology. For example, it is well-known that Gadolinium enhanced MR images show cancerous regions but as is often the case, the enhancement does not allow visualization of the complete tumor. With this realization, the need to move to a localized pathological imaging tool as outlined becomes more apparent. Model-based compensation algorithms fit nicely within this framework and provide a valuable component to surgical therapy. The use of computational models within surgical research and image analysis has a considerable history, yet the reliance on computational models for the real-time direction of surgical therapy is a relatively new concept. The dramatic increases in computational speed have yielded new possibilities for integrating these resources within the operating room environment (OR). To achieve these advances, several critical components are necessary: (1) preoperative imaging data that spatially characterizes pathology and other critical anatomical regions, (2) threedimensional digitization equipment that relates intra-operative changes in the patient’s spatial presentation, and (3) a computational platform that aligns the patient and compensates for changes as measured by digitization. In this chapter, techniques will be discussed that utilize a laser range scanner in conjunction with computational models to correct mis-registrations due to soft tissue deformation in neurosurgery. METHODS A. Intra-operative Brain Registration and Data Acquisition Surface-based registration has been an alternative to landmark-based registration for several years. With respect to the neurosurgical context, the predominant problem with surface registration is the lack of significant geometric features on the scalp or the tendency of those surfaces to deform. However, using the geometric patterns associated with the outer brain would provide a more unique surface to use for registration. Using features from the cortical surface to register does have some precedent. Nakajima et al. demonstrated an average of 2.3 +/- 1.3 mm registration error using cortical vessels for registration [22]. Some preliminary work using laser scanning based systems for cortical surface registration has been reported and has demonstrated encouraging results [23, 24]. Furthermore, the presence of such a feature-rich surface during surgery provides a strong rationale for using the unique presentation of the organ during surgery for registration purposes. To investigate this thesis statement, a laser-range scanner (LRS) system capable of capturing threedimensional topography as well surface texture mapping to sub-millimeter accuracy would have great utility for

brain surgery. One such commercial device is shown in Figure 1a (RealScan 3D, 3D Digital Corporation, Danbury, CT) and has been mounted to a monopod for the neurosurgical theatre in Figure 1b. This particular setup involves placing the monopod 1-2 feet from the patient’s cortical surface, acquiring a scan, and then moving the monopod out of the field for the continuation of surgery (with a modest a b amount of training, this chain of events Figure 1. (a) Laser range scanner, and (b) monopod setup for the operating takes approximately 0.5-1.5 minutes to complete). In addition to being fast, lightweight, and compact, this particular laser-range scanner system also has the unique capability of capturing feature-rich texture maps of the surface as well as the topographical characteristics (textured LRS or tLRS). In the neurosurgical setting, this added dimensionality of data is being used to assist in image-to-patient alignment and for the measuring of intra-operative brain deformations. Within other surgical contexts (e.g. intra-abdominal liver surgery), the texture map data has facilitated segmentation, i.e. extraction, of the surface of interest during surgery as well as for patient registration [25-27]. It should also be noted that we are currently monitoring the laser range scanner position with an optical tracking system (the infrared emitting diodes can be seen in Figure 1a atop the tLRS unit). Prior to surgery, a calibration phantom is digitized with the tLRS unit and with an optical stylus. This allows for the transformation between the scanner internal point cloud coordinate system and the optical tracking system to be determined [28]. Infrared Emitting Diodes

Within the neurosurgical setting, the exposed cortical surface is feature-rich with complex sulcal patterns, and a web of traversing vasculature. While the brain is often depicted as having deep invaginations which are generated from intensity voxel-averaging techniques, in practice, the cortical surface does not usually display this level of geometric feature intra-operatively (although under certain conditions such as with the use of hyperosmotic agents, more pronounced invaginations can occur). As a result, the use of geometric feature alone may be insufficient to constrain registration of the acquired range surface to its MR counterpart. One possibility is to further constrain and refine the alignment by using the exposed cortical pattern and the corresponding MR rendered surfaces as provided by intensity voxel-averaging techniques. This could be implemented using a rigid-body registration between intra-operative tLRS data and preoperative MR data with a combined texture/geometric-based alignment method. To allow for a simplified

a b c Figure 2. (a) Smooth radial-basis functions surface from a segmented MR tomogram, (b) surface normals from the refined mesh used to calculate surface texture, and (c) textured surface generated from preoperative MR tomograms.

a

b

Figure 3. (a) The gray-scale encoded cortical brain surface as segmented from a Gadolinium enhanced MR tomographic volume and corresponding OR surface as acquired by LRS. (b) Registered surfaces.

registration procedure, textured surfaces (similar to those gathered by tLRS) are generated from preoperative MR tomograms. This process begins with MR data where the cortical structure has been segmented either manually or automatically. A marching cubes process is then used to extract a polygonal surface representation from the segmented data. The extracted surface can be refined further to a smoother approximation and re-meshed to a more uniformly distributed polygon description (e.g. using radial-basis functions). Surface normals from each surface point in the mesh can then be used to project “rays” into the MR tomogram. The average intensity over 5-10 voxels along the normal direction can be recorded and used to texture each surface point. These textured-encoded representations of the brain's surface from the preoperative MR tomograms are then used for registration against tLRS datasets. Figure 2 illustrates this process. Texture based registration of two intensity encoded surfaces is provided by a constrained mutual information-based surface registration [29]. This process is initiated with a landmark-based registration of the two surfaces using identifiable features in each surface. A surfaced-based registration is then used to minimize surface to surface distance between the two clouds. At this point the two surfaces have been registered reasonably well and an intensity based registration is required to refine the registration based on the corresponding intensity features in each cloud. For this, each surface is transformed into a spherical representation by fitting a sphere to the preoperative MR surface and then projecting each surface point onto the surface of the fitted sphere. Surface points from the tLRS dataset are also projected to the fitted sphere. This process simplifies the usual 6 DOF registration for 3D surfaces to a 3 DOF registration (i.e. registration about the rotational axes: θx, θy, and θz). During the texturebased registration, point correspondence between clouds is determined using closest point measures. Mutual information between clouds is calculated using the correspondences and is optimized with respect to each DOF to provide the correct alignment of the two textured surfaces. Figure 3a illustrates a sample dataset of a MR grayscaleencoded tomogram and the respective tLRS cortical surface as reported by Miga et al. in IEEE Transactions on Medical Imaging vol. 22, no. 8, pp. 973-985, (© 2003 IEEE). Figure 3b represents the registration between patient and tLRS surface. In this initial realization, the closest point operator was used to determine correspondence between textures in lieu of the sphere-fitting process of the textured surfaces. The rigid-body registration provides a relationship between the preoperative state of the brain and any subsequent intraoperative brain measurements. The shift-tracking protocol developed for tLRS datasets uses the geometry and texture, and the unique mapping between the two to accurately determine cortical surface motion. The crux of the protocol is the accurate registration of the initial and serial tLRS spaces. This registration is

Optical emitter After Craniotomy Scan 1

During surgery Scan 2

Tracked Laser Scanner Deformation

3D Textured Point Cloud

3D Textured Point Cloud

Texture Image of FOV

Texture Image of FOV

Correspondence by Non-Rigid Image Registration of Textures

Figure 4. Illustration of cortical surface deformation measurement approach using a tracked laser range scanning system.

accomplished using a non-rigid 2D registration of the texture images from the initial and serial datasets. Once the two texture images have been aligned, the following steps are used to determine shift for a given point-of-interest in the initial tLRS dataset: (1) project from physical-space to texture-space in the initial tLRS dataset using texture coordinates, (2) transform from initial to serial texture-space using the non-rigid registration, and, (3) project from serial texture-space to serial physical-space using reverse texture coordinates. These steps provide 3D correspondence between serial acquisitions of the tLRS datasets, which allows for automatic cortical surface shift determination. Figure 4 conceptualizes the process whereby serial range scans are acquired producing threedimensional textured point cloud data. By non-rigidly aligning the two-dimensional texture images of the field of view (FOV), correspondence of cortical structures in each 3D textured point cloud is established which is subsequently used for the measurement of brain shift. By optically tracking the laser range scanner, the relationship of all scans to a consistent physical space relative to the patient and image volume can be easily established. This method of quantifying cortical surface shift, when tied to the preoperative MR data sets using the rigid registration described earlier, demonstrates a complete solution for cortical surface tracking and an attractive source of intraoperative data for the model-updating framework. B. Finite Element Modeling Central to the utilization of computational models to update image-guided navigation is the chosen mathematical representation for predicting realistic deformations of the brain (guided by sparse intra-operative measurements). One important aspect to be considered when selecting a model is the order and magnitude of deformations anticipated in relation to the inherent experimental error associated with the application of surgery. More specifically, the threshold of tolerable experimental discrepancies must be weighed against the level of model complexity. For example, if a linear elastic model proves effective within the tolerances of acceptable surgical error, extending constitutive laws to capture 2nd and 3rd order effects may be extraneous to the surgical system and overly cumbersome for application.

In this work, the brain is modeled as a biphasic continuum. The addition of another phase (interstitial fluid) creates a more complex solution procedure as well as a temporal dependence. Despite this complexity, the brain can be greatly affected by changes in hydraulic states. For example, non-communicating hydrocephalus is an excellent example of a hydraulic loading condition that causes tissue deformation. A second example can be found in focal edema region that is often associated with tumor growth. In addition, the drainage of cerebral spinal fluid or the administration of hyperosmotic drugs can cause dramatic deformations within the brain. Each of these conditions is relevant to neurosurgical interventions and requires the interaction with the hydrated nature of the brain. B.1 Brain Model One model that has been used for the brain that captures both the solid- and fluid-like behavior is based on consolidation physics [30] and was first presented in the context of finite element and brain deformation mechanics by Nagashima et al. [31, 32]. This work concentrated on modeling vasogenic edema within the brain, and was validated qualitatively in a feline experiment. This model was extended by Paulsen and Miga et al. to threedimensional calculations and validated in a series of repeat-experiments in a porcine system subjected to loads comparable to neurosurgical applications [33, 34]. The equations describing the model for brain deformation are: ∇ • G∇U + ∇(G + λ)∇ • U-α∇p = (ρ f − ρ t )g (1,2) ∂ (∇ • U) 1 ∂p ∇ • k ∇p − α − =0 ∂t S ∂t where G and λ are the Lame’ constants, α and 1/S are constants associated with fluid saturation, k is the hydraulic conductivity, ρ is density with f and t corresponding to density of the fluid and tissue respectively, and g is gravitational acceleration. The dependent variables are displacement in the x, y, and z direction represented by the vector U and the interstitial pressure p. Briefly describing equation (1) leads with: the first two terms are associated with mechanical equilibrium; the third term is concerned with the effect an interstitial pressure gradient could have on tissue deformation; the right-hand-side is concerned with modeling the weight of the brain as changes in buoyancy forces can occur. Briefly describing equation (2) leads with: the first two terms relating transport of fluid out of the porous media to the time rate of change of the dilatational strain; and the third term is related to an accumulation term as in the case of air within the porous media. This set of partial differential equations has been solved using a fully implicit time-stepping with the Galerkin finite element method [35]. Details of the weight residual equations, their solution and stability can be found elsewhere [33, 34, 36]. This model has been

a

b

c

e

d

f

Figure 5. (a) MR surface rendering of brain with (b) corresponding computational mesh with internal tumor and lateral ventricles visible. (c,d) MR axial slices at different locations within brain (e,f) with corresponding model material property distributions (gray-scale indicates regions of different material properties) .

quantitatively validated in repeat porcine experiments [13, 14, 37, 38] and in suggestive preliminary experiments with human data [11, 39, 40]. In addition, a model generation environment using the preoperative MR images to generate patient-specific computational grids is achievable and can be seen in Figure 5. These patient-specific models have been extensively refined such that computational domains can routinely incorporate intracranial structures such as white/gray matter, the falx and tentorium divisions, tumor tissue and the lateral ventricles [38, 40, 41] B.2 Boundary Conditions In practice, the conventional finite element method is relatively clear and has been implemented in many different software packages. With respect to modeling anatomical organs, there are some aspects to the application of boundary conditions that are particularly challenging to traditional representations. For example, in the application of displacement boundary conditions to the brain, it is often desirable to express the movement of the boundary in a coordinate system that is relative to the geometric shape, i.e., the coordinate system associated with directions that are normal and tangential to the organ surface. One strategy is to take the desired normal displacement and convert this to its Cartesian counterparts by using the relationship in equation (3),  dn   nx ny nz  dx       (3)  dt1  =  t1x t1y t1z  dy  dt   t    2   2 x t 2 y t 2 z  dz  r r r r r r r where nx = x • n, ny = y • n, etc. , and n, t1, t2 represent an orthogonal coordinate system with a normal unit vector (to the organ surface) and two tangential unit vector axes, respectively. The matrix above represents the transformation from Cartesian to normal-tangential space (n-t space). In this case, the inverse relationship in (3) could be used to take a normal-tangential displacement vector and transform it into Cartesian space; however, the ability to relate mixed boundary conditions within the n-t space framework is not possible. For example, in many instances it is desirable to allow an organ surface to slide along a supporting wall yet not deform in a direction normal to the wall, i.e. through the wall. This type of boundary condition requires stress free conditions tangent to the wall and restricted normal displacements, i.e.: (4) σ t1 = σ t 2 = 0, un = 0 . In this instance, the framework described above will not be able to provide this organ movement behavior. As a solution to this requirement, a second approach is to rotate the equations of equilibrium for nodes concerned with the boundary into an n-t space coordinate reference. This process involves the use of rotational matrices being applied at the local element assembly level: [R][i K ]i R T j {u}j = [R]i {b}i (5)

[ ]

where [K ]i is the local set of integrated weighted residual equations associated with equations (1), and (2); [R ]i is the local rotation matrix that rotates the equilibrium equation and body force components (note that [R ]i is the matrix shown in equation (3) and is expressed in the n-t space associated with node i), and [R ]j = [R ] T

−1

is the matrix that

th

rotates the displacement coefficients from Cartesian to n-t space (j refers to the j displacement coefficient). Careful attention must be paid to the determination of the local rotational matrix, [R], and to the arrangement of rotational multiplications. The form of this equation is dependent on the particular scalar functions of position and unknown coefficients involved with the FE integration process, specifically, with respect to the domain boundary. This approach to n-t space calculation has been reported by Engelman et al.[42]. C. Model-Updating C.1. Image Deforming Image deforming is performed using standard tri-linear interpolation with a back-propagating voxel fill-in strategy. More specifically, using image voxel dimension information, the finite element mesh and displacement

solution are scaled into image space. The displacement solution is used to deform the finite element mesh. Voxel coordinates are then identified within the deformed mesh volume. Using the natural interpolative basis associated with finite elements, a “reverse” voxel displacement value is calculated at each voxel location. Once determined, the voxel location can back-propagate to the un-deformed image volume, the intensity can be calculated using trilinear interpolation, and subsequently the voxel in the deformed volume can be “filled”. This strategy avoids possible discontinuities within the deformed image which can often occur when deforming the image in the forward sense. C.2. Brain Model Updating In some respects, the application of computer models to simulate surgical deformations has been a focus in recent surgical research. The next important question is how to integrate intra-operative measurement methods with these models to create an updating framework that is feasible for OR use. One approximation is to apply all measurements as displacement boundary conditions within the model and move forward. This approach has been utilized by Ferrant et al. [12] within the context of iMR and uses the computational model to interpolate updated images. Although intuitive, this approach may not work for a model-based compensation platform in medical centers without an iMR system. Ferrant et al. applied displacement conditions over the complete lateral ventricular and cortical surface to achieve updates. This extent of data cannot be achieved by standard IGS systems and tLRS technology only. Even with the addition of co-registered ultrasound, the same fidelity of data achieved with iMR (in respect to subsurface structures) cannot be anticipated. In addition, in this type of implementation, image interpolation is of primary importance at the cost to quantitative biomechanical information. For example, interpolative techniques routinely apply forces on the cortical surface and ventricular surfaces to simulate brain deformations due to gravity. This represents the utilization of a surface force to model what is a distributed force in mechanics. By modeling in this manner, the distribution of displacements may be adequate for guidance purposes but information regarding the stress distribution within the tissue may be inaccurate. Nevertheless, it should be noted that iMR-based model updates are of considerable importance as a means to non-rigidly register the wealth of pre-operative imaging information (e.g. functional MR, diffusion tensor images, MR spectroscopy, etc.) to the intraoperative environment. Outside of the iMR context, it is relatively clear that model-updating using interpolative models may be compromised due to the sparsity of intra-operative data. An alternative strategy is to use the growing understanding of brain deformation mechanics and intracranial boundary conditions to create a more biomechanically accurate model that only needs limited data to constrain solutions. One approach is to parse the update according biomechanical load categories such as: (1) distributed/body force load compensation, (2) followed by surface/interactive force load compensation. With respect to body loads, the primary finding among the majority of brain shift studies is that changes in the hydration level of the cranial contents is a major factor. More specifically, as cerebrospinal fluid (CSF) is drained from the cranial cavity and buoyancy forces are reduced, the brain begins to deform under its own weight, i.e. sags. In addition, certain hyperosmotic agents can be given which effectively shrink the brain by drawing interstitial fluid from the brain tissue. With respect to gravitation sag, the amount of intra-operative CSF drainage and patient’s orientation in the OR are two important variables in predicting intraoperative brain shift. Given its prominence, a strategy using tLRS technology and computer models is presented that corrects for this source of shift. In addition, the principles discussed could also be extended to incorporate other distributed loading conditions. In this first biomechanical category of loading, one possible approach, inspired by Davatzikos et al. [19, 43], is to employ a statistically-based model to compensate for the shift due to distributed loading conditions (e.g. currently, sag is a primary concern). As shown in Figure 6, the method begins by building a patient specific model using the patient’s preoperative MR images. The biomechanical computer model of the brain is solved repetitively assuming a range of patient orientations and degrees of intra-operative CSF drainage based on the preoperative surgical plan. In order to accomplish these calculations, a generalized form of the boundary conditions must be determined as a function of two variables (i.e. orientation and CSF drainage). One possible arrangement of

Intraoperative Data Acquisition tLRS Measurement of Cortical Shift

Surgical Plan Head Orientation & Craniotomy Region

Deformation Atlas Generation Atlas is a Function of Orientation & Cerebrospinal Fluid Level Patient Orientation

CSF Drainage Level

x measured !

shift

Statistical Model Cortical surface and volume deformation atlases

[E ]cortical , [A ]volume

Atlas of Boundary Conditions

[E],[A]

Compensation for Bulk Shift

x predicted shift = [E ]{α } min x predicted − x measured

2

x deform = [A ]{α} Deform Preoperative Image Volume

PREOPERATIVE INTRAOPERATIVE Figure 6. Stage 1 of a biomechanical load parsing strategy for model-updating.

conditions has been reported in [11]. By varying these variables, the geometric descriptions of the boundary conditions change, and with the collection of each subsequent solution, an atlas of deformations can be generated. One distinct advantage to this pre-computed atlas strategy is the range of solutions that can be accommodated at minimal OR computation cost. In addition to boundary conditions, the influence of other patient-specific variables may be factored into the atlas such as those arising from discrepancies in mechanical property measurements. Upon completion of the model solution series, a spatio-temporal atlas of deformation is generated which can then be used to characterize intra-operative deformations within the context of a statistical parameterization. In previous work [44], a least squares regression analysis was performed with non-negativity constraints. The results of a preliminary exploration of this approach are summarized below. One interesting note regarding this approach is vessel

Pre-retraction

Post-retraction

Direct Interactive Simulation Figure 7. Illustration of direct interaction with the computational volume due to retraction.

Figure 8. Texture based registration overlay. The gray brain surface is generated from preoperative MR tomograms. The color surface is from LRS during surgery.

that simulation information regarding tissue swelling (i.e. edema) and hyperosmotic drug influence could be incorporated quite easily. Physiologically, these mechanisms are hydrostatic in nature and result in the generation of distributed/body forces that would be a natural addition to atlas information. It should also be noted that Figure 6 has been cast in the context of surface measurement data acquisition only; but data points regarding subsurface movement as provided by a co-registered ultrasound system could be used in the method. The second biomechanical parsing category of loading is to apply any residual error remaining from inaccuracies with the statistical model as a direct boundary condition similar to Ferrant et al.’s strategy; but by applying our statistical approach first, the model should: (1) have more biomechanical relevance, and (2) require less driving data (perhaps tLRS data alone may be sufficient). With respect to these stage 2 loading conditions (e.g. retraction and resection), a great deal of work has been accomplished with respect to porcine experiments and some qualitative validation within humans [14, 40]. For example, retraction methods developed in [40] implement a localized element segmentation framework that allows the arbitrary insertion of a retractor (as represented by a description of triangular patches) and the separation of the element domain to allow separate boundary conditions to be designated on each side of the division. In addition, an approach to representing the resection and removal of tissue has been thoughtfully designed such that re-meshing is not required. Figure 7 is representative of a more direct interaction with the computational surgical model. This demonstration first appeared in Neurosurgery, vol. 49, no. 1, pp. 75-85, (© 2001 Neurosurgery). D. RESULTS D.1. Cortical Surface Registration and Brain Shift Measurement Results using the aforementioned methods show that the rigid registration and shift tracking protocol are capable of accurately characterizing surface motion using tLRS. In [29], the rigid registration method was validated using multimodal phantom data. The results from that work indicate that the texture based registration is capable of registering textured surfaces as accurately as existing registration protocols given identical registration conditions. The advantage of the texture based registration is its ability to incorporate all of the information presented by tLRS. That work indicated that the texture-based algorithm can resolve phantom deep-tissue targets more accurately than

Figure 9. Results of the shift tracking protocol on in vivo data. The red dots indicate target positions on the brain before resection. The blue dots indicate the same position after resection and brain sag. The positions indicated by the blue dots were calculated automatically using the shift-tracking protocol. The numbers annotated with each pairing indicate the magnitude of brain shift in millimeters. The surfaces above are the same; a rotation has been applied to the right surface to better see the displacements.

existing surface registration methods under the same registration conditions. An example alignment using textured based registration applied to intra-operative data is shown in Figure 8. Phantom experiments examining the shift-tracking protocol illustrated by Figure 4 show that the method is capable of accurately resolving shift [45]. For rigid-body motions of objects within the tLRS field-of-view, the shift-tracking protocol can resolve motion to within 1.0mm. Non-rigid motions, using the protocol, can be resolved to within 1.6mm. Shift tracking accuracy on preliminary in vivo datasets showed that the tracking accuracy was on the order of 1mm. Figure 9 shows results of the protocol when applied to in vivo human cortical surface data. The initial position is shown in red while the final position as predicted by our measurement method is shown in blue. The deformations shown are realistic and in preliminary results have demonstrated encouraging measurement accuracy. D.2. Model-Updated Image-Guided Neurosurgery One of the major sources for surgical error in applying image-guided principles to neurosurgery is inducement of brain shift due to gravity. Specifically, as cerebrospinal fluid is drained from the cranium, buoyancy forces are reduced and the brain begins to sag under its own weight. The right-hand-side equation (1) expresses a mechanism for modeling this type of brain deformation. The first results comparing model predictions to clinical data using this approach were reported by Miga et al. and appeared in IEEE Transactions on Medical Imaging vol. 18, no. 10, pp. 866-874, (© 1999 IEEE). Figure 10 shows one series of results from that preliminary study. Each column represents a different patient within that four-patient study. The first row represents the preoperative MR tomogram with the direction of gravity indicated by the white arrow for that patient’s orientation within the operating room (OR). The second row represents the simulated intra-operative update as provided by the model equations (1,2). The third row shows a difference image to demonstrate the shift of subsurface brain structure. It should be noted that although a single slice is shown, the models solved were three-dimensional. In these four

Figure 10. Examples of model-updates from gravity-induced shift (white arrow indicates the direction of gravity for particular patient configuration) reported by Miga et al. in IEEE Transactions on Medical Imaging, vol. 18, no. 10, pp. 866-874, (© 1999 IEEE).

Patient

Measured Shift (mm)

(TMI - 1999) Model Error (mm)

1

4.8+/-1.4

1.1 +/-0.8

(MICCAI - 2003) Statistical Model Error (mm) 1.0+/-0.8

2

7.5+/-2.5

1.7+/-2.6

2.0+/-1.0

3

6.2+/-1.3

1.3+/-0.3

1.4+/-0.2

4

4.0+/-0.6

0.4+/-0.1

0.1+/-0.0

Table 1. Comparison of model-updates to independent shift measurements in two model-updating studies.

cases, points on the cortical surface of each patient were independently tracked in the direction of gravity and then compared to model predictions. The results from that study are shown in Table 1, 3rd column. Since this initial study, more development on generating a more robust strategy for automatic brain shift compensation has been forthcoming and is shown in Figure 6. A comparison between the initial study and the deformation atlas approach

is shown in Table 1 [44]. The results above suggest that a pre-computed deformation atlas strategy as shown in Figure 6 may be an excellent way to compensate for brain shifts due to more distributed loading conditions. However, given the need to perform retraction and resection, a model-updating strategy would not be complete without enabling more direct interactions with the computational volume to reflect OR conditions. The illustrating case shown in Figures 11 and 12 involves the retraction of brain tissue along the falx cerebri to reveal a metastatic non-small cell carcinoma as represented by the hypointense region within the MR tomogram. This case was completely modeled from presentation to the resection of the tumor and was first reported by Miga et al. in Neurosurgery, vol. 49, no. 1, pp. 75-85, (© 2001 Neurosurgery). From that work, Figure 11 illustrates the progression of the surgery as predicted by the computational model and observed in the volume-rendered grayscale encoded tomogram with Figure: (11a) surgical orientation, (11b) brain presentation, (11c) post gravity-induced sag, (11d) post-retraction of the tissue, (11e) post-partial resection, and (11f) following complete resection. Figure 12 shows the same surgical progression from the perspective of a single sagittal slice within the image volume. Of note is the disappearance of tissue down to the tumor in Figures 13 c-e which represents the tissue in this region being pulled out of the image plane. Figure 13f shows the tissue after the retractor has been removed.

a

b

c

a

b

c

d

e

f

d

e

f

Figure 11. (a) Surgical orientation, (b) visualization of brain surface, (c) model-updated of brain due to sag, (d) model-update of retraction, (e) model-update after partial resection, and (f) modelupdate after total resection. Appeared in Neurosurgery, vol. 49, no. 1, pp. 75-85, (© 2001 Neurosurgery)

Figure 12 Corresponding sagittal slice to stages of operation in Figure 11. Appeared in Neurosurgery, vol. 49, no. 1, pp. 75-85, (© 2001 Neurosurgery).

DISCUSSION The methods described that use tLRS technology to provide initial patient-to-image alignment and measurements of cortical surface deformation represent a potentially automatic technique to quantify surface shift which is relatively accurate, fast, and minimally intrusive to the OR setting. In addition, the system shown in Figure 1 represents a relatively new technology in an early phase of development with respect to surgical research. At a moderate cost, this digitization technology affords detailed sub-millimetric capturing of cortical surface subtleties

which will ultimately lead to new registration approaches in the OR. The studies highlighted here demonstrate an impressive ability to capture surface topography and that registrations performed with these surfaces are visually compelling. The direct correlation seen in Figures 3 and 8 between the surgeon’s field of view as given by the textured point cloud and the preoperative MR grayscale encoded surface rendering will provide new anatomical cues that have not been available with other registration techniques. Figure 10 and Table 1 report that realistic brain shifts associated with the predominant mechanism of shift, i.e. gravitational sag, are achievable with the model shown in equations (1,2). The atlas-based results of Table 1 also demonstrate that pre-computation strategies may be very effective at alleviating concerns associated with computational overhead of finite element models. Figures 11 and 12 demonstrate that realistic interactive loading conditions such as from retraction and resection are also possible. CONCLUSIONS The improvement of image-guided surgical systems to account for surgically-induced brain deformations has important implications for the performance of difficult surgical techniques. Digitization technology, such as that provided by the laser range scanner, affords rich yet limited data sets regarding topographic features and progressive distortion due to intra-operative deformation. Given the increasing performance of computational resources, numerical modeling of complex intra-operative events is extending beyond its former predictive role to one of feedback thus providing fast, efficient, and valuable assistance to surgeons during surgery. REFERENCES [1] [2] [3] [4] [5] [6] [7] [8] [9] [10]

R. L. Galloway, "The process and development of image-guided procedures," Annual Review of Biomedical Engineering, vol. 3, pp. 83-108, 2001. D. W. Roberts, A. Hartov, F. E. Kennedy, M. I. Miga, and K. D. Paulsen, "Intraoperative brain shift and deformation: A quantitative analysis of cortical displacement in 28 cases," Neurosurgery, vol. 43, pp. 749758, 1998. C. Nimsky, O. Ganslandt, S. Cerny, P. Hastreiter, G. Greiner, and R. Fahlbusch, "Quantification of, visualization of, and compensation for brain shift using intraoperative magnetic resonance imaging," Neurosurgery, vol. 47, pp. 1070-9; discussion 1079-80., 2000. A. Nabavi, P. M. Black, D. T. Gering, C. F. Westin, V. Mehta, R. S. Pergolizzi, M. Ferrant, S. K. Warfield, N. Hata, R. B. Schwartz, W. M. Wells, R. Kikinis, and F. A. Jolesz, "Serial intraoperative magnetic resonance imaging of brain shift," Neurosurgery, vol. 48, pp. 787-797, 2001. P. M. Black, E. Alexander, C. Martin, and T. Moriarty, "Craniotomy for tumor treatment in an intraoperative magnetic resonance imaging unit," Neurosurgery, vol. 45, pp. 423-431, 1999. V. M. Tronnier, C. R. Wirtz, M. Knauth, G. Lenz, O. Pastyr, M. M. Bonsanto, F. K. Albert, R. Kuth, A. Staubert, W. Schlegel, K. Sartor, and S. Kunze, "Intraoperative diagnostic and interventional magnetic resonance imaging in neurosurgery," Neurosurgery, vol. 40, pp. 891-900; discussion 900-2., 1997. G. Unsgaard, S. Ommedal, T. Muller, A. Gronningsaeter, and T. A. N. Hernes, "Neuronavigation by Intraoperative three-dimensional ultrasound: Initial experience during brain tumor resection," Neurosurgery, vol. 50, pp. 804-812, 2002. R. M. Comeau, A. F. Sadikot, A. Fenster, and T. M. Peters, "Intraoperative ultrasound for guidance and tissue shift correction in image-guided neurosurgery," Medical Physics, vol. 27, pp. 787-800, 2000. R. D. Bucholz, D. D. Yeh, J. Trobaugh, L. L. McDurmont, C. D. Sturm, C. Baumann, J. M. Henderson, A. Levy, and P. Kessman, "The correction of stereotactic inaccuracy caused by brain shift using an intraoperative ultrasound device," Cvrmed-Mrcas'97, vol. 1205, pp. 459-466, 1997. D. W. Roberts, M. I. Miga, A. Hartov, S. Eisner, J. M. Lemery, F. E. Kennedy, and K. D. Paulsen, "Intraoperatively updated neuroimaging using brain modeling and sparse data," Neurosurgery, vol. 45, pp. 1199-1206, 1999.

[11] [12] [13] [14] [15] [16]

[17] [18] [19] [20] [21] [22] [23] [24] [25] [26] [27] [28]

M. I. Miga, K. D. Paulsen, J. M. Lemery, S. D. Eisner, A. Hartov, F. E. Kennedy, and D. W. Roberts, "Model-updated image guidance: Initial clinical experiences with gravity-induced brain deformation," IEEE Transactions on Medical Imaging, vol. 18, pp. 866-874, 1999. M. Ferrant, A. Nabavi, B. Macq, P. M. Black, F. A. Jolesz, R. Kikinis, and S. K. Warfield, "Serial registration of intraoperative MR images of the brain," Medical Image Analysis, vol. 6, pp. 337-359, 2002. M. I. Miga, K. D. Paulsen, P. J. Hoopes, F. E. Kennedy, A. Hartov, and D. W. Roberts, "In vivo quantification of a homogeneous brain deformation model for updating preoperative images during surgery," IEEE Transactions on Biomedical Engineering, vol. 47, pp. 266-273, 2000. L. A. Platenik, M. I. Miga, D. W. Roberts, K. E. Lunn, F. E. Kennedy, A. Hartov, and K. D. Paulsen, "In vivo quantification of retraction deformation modeling for updated image-guidance during neurosurgery," IEEE Transactions on Biomedical Engineering, vol. 49, pp. 823-835, 2002. P. J. Edwards, D. H. Hill, J. Little, and D. Hawkes, "A three component deformation model for image-guided surgery," Medical Image Analysis, vol. 2, pp. 355-367, 1998. O. Clatz, H. Delingette, E. Bardinet, D. Dormont, and N. Ayache, "Patient-specific biomechanical model of the brain: Application to Parkinson's disease procedure," in Surgery Simulation and Soft Tissue Modeling, Proceedings, vol. 2673, Lecture Notes in Computer Science. Berlin: SPRINGER-VERLAG BERLIN, 2003, pp. 321-331. A. Hagemann, K. Rohr, H. S. Stiehl, U. Spetzger, and J. M. Gilsbach, "Biomechanical modeling of the human head for physically based, nonrigid image registration," IEEE Transactions on Medical Imaging, vol. 18, pp. 875-884, 1999. O. Skrinjar, A. Nabavi, and J. Duncan, "Model-driven brain shift compensation," Medical Image Analysis, vol. 6, pp. 361-373, 2002. C. Davatzikos, D. G. Shen, A. Mohamed, and S. K. Kyriacou, "A framework for predictive modeling of anatomical deformations," IEEE Transactions on Medical Imaging, vol. 20, pp. 836-843, 2001. W. C. Lin, S. A. Toms, M. Johnson, E. D. Jansen, and A. Mahadevan-Jansen, "In vivo brain tumor demarcation using optical spectroscopy," Photochemistry and Photobiology, vol. 73, pp. 396-402, 2001. W. C. Lin, S. A. Toms, M. Motamedi, E. D. Jansen, and A. Mahadevan-Jansen, "Brain tumor demarcation using optical spectroscopy; an in vitro study," Journal of Biomedical Optics, vol. 5, pp. 214-220, 2000. S. Nakajima, H. Atsumi, R. Kikinis, T. M. Moriarty, D. C. Metcalf, F. A. Jolesz, and P. M. Black, "Use of cortical surface vessel registration for image-guided neurosurgery," Neurosurgery, vol. 40, pp. 1201-1208, 1997. M. A. Audette, K. Siddiqi, F. P. Ferrie, and T. M. Peters, "An integrated range-sensing, segmentation and registration framework for the characterization of intra-surgical brain deformations in image-guided surgery," Computer Vision and Image Understanding, vol. 89, pp. 226-251, 2003. M. A. Audette, K. Siddiqi, and T. M. Peters, "Level-set surface segmentation and fast cortical range image tracking for computing intrasurgical deformations," Lecture Notes in Computer Science: Medical Image Computing and Computer-Assisted Intervenstion - MICCAI'99, vol. 1679, pp. 788-797, 1999. M. I. Miga, D. M. Cash, Z. Cao, R. L. Galloway Jr., B. M. Dawant, and W. C. Chapman, "Intraoperative registration of the liver for image-guided surgery using laser range scanning and deformable models," presented at Medical Imaging 2003: Visualization, Image-guided Procedures, and Display, San Diego, 2003. D. M. Cash, T. K. Sinha, R. L. Chapman, R. L. Galloway, and M. I. Miga, "Fast, accurate surface acquisition using a laser range scanner for image-guided liver surgery," Medical Imaging 2002: Visualization, display, and image-guided procedures: Proc. of the SPIE 2002, vol. 4681, pp. 100-110, 2002. D. M. Cash, T. K. Sinha, W. C. Chapman, H. Terawaki, B. M. Dawant, R. L. Galloway, and M. I. Miga, "Incorporation of a laser range scanner into image-guided liver surgery: Surface acquisition, registration, and tracking," Medical Physics, vol. 30, pp. 1671-1682, 2003. T. K. Sinha, B. M. Dawant, V. Duay, D. M. Cash, R. J. Weil, R. C. Thompson, K. D. Weaver, and M. I. Miga, "A method to track cortical surface deformations using a laser range scanner," IEEE Transactions on Medical Imaging, (in press), 2005.

[29] [30] [31] [32] [33] [34] [35] [36] [37] [38] [39]

[40] [41] [42] [43] [44] [45]

M. I. Miga, T. K. Sinha, D. M. Cash, R. L. Galloway, and R. J. Weil, "Cortical surface registration for image-guided neurosurgery using laser-range scanning," IEEE Transactions on Medical Imaging, vol. 22, pp. 973-985, 2003. M. A. Biot, "General theory of three-dimensional consolidation," Journal of Applied Physics, vol. 12, pp. 155-164, 1941. T. Nagashima, S. Takayuki, and S. I. Rapoport, "A two-dimensional, finite element analysis of vasogenic brain edema," Neurol Med Chir (Tokyo), vol. 20, pp. 1-9, 1990. T. Nagashima, N. Tamaki, M. Takada, and Y. Tada, "Formation and Resolution of Brain Edema Associated with Brain- Tumors - a Comprehensive Theoretical-Model and Clinical Analysis," Acta Neurochirurgica, pp. 165-167, 1994. M. I. Miga, "Development and Quantification of a 3D Brain Deformation Model for Model-Updated ImageGuided Stereotactic Neurosurgery," in Engineering. Hanover: Dartmouth College, 1998, pp. 238. K. D. Paulsen, M. I. Miga, F. E. Kennedy, P. J. Hoopes, A. Hartov, and D. W. Roberts, "A computational model for tracking subsurface tissue deformation during stereotactic neurosurgery," IEEE Transactions on Biomedical Engineering, vol. 46, pp. 213-225, 1999. L. Lapidus and G. F. Pinder, Numerical Solution of Partial Differential Equations in Science and Engineering. New York: John Wiley & Sons, Inc., 1982. M. I. Miga, K. D. Paulsen, and F. E. Kennedy, "Von Neumann stability analysis of Biot's general twodimensional theory of consolidation," International Journal for Numerical Methods in Engineering, vol. 43, pp. 955-974, 1998. M. I. Miga, K. D. Paulsen, P. J. Hoopes, F. E. Kennedy, A. Hartov, and D. W. Roberts, "In vivo modeling of interstitial pressure in the brain under surgical load using finite elements," Journal of Biomechanical Engineering-Transactions of the Asme, vol. 122, pp. 354-363, 2000. M. I. Miga, K. D. Paulsen, F. E. Kennedy, P. J. Hoopes, A. Hartov, and D. W. Roberts, "In Vivo Analysis of Heterogeneous Brain Deformation Computations for Model-Updated Image Guidance," Comput Methods Biomech Biomed Engin, vol. 3, pp. 129-146, 2000. M. I. Miga, A. Staubert, K. D. Paulsen, F. E. Kennedy, V. M. Tronnier, D. W. Roberts, A. Hartov, L. A. Platenik, and K. E. Lunn, "Model-updated image-guided neurosurgery: Preliminary analysis using intraoperative MR," Medical Image Computing and Computer-Assisted Intervention - MICCAI 2000, vol. 1935, pp. 115-124, 2000. M. I. Miga, D. W. Roberts, F. E. Kennedy, L. A. Platenik, A. Hartov, K. E. Lunn, and K. D. Paulsen, "Modeling of retraction and resection for intraoperative updating of images," Neurosurgery, vol. 49, pp. 7584, 2001. M. I. Miga, K. D. Paulsen, F. E. Kennedy, A. Hartov, and D. W. Roberts, "Model-updated image-guided neurosurgery using the finite element method: Incorporation of the falx cerebri," Medical Image Computing and Computer-Assisted Intervention, MICCAI'99, Proceedings, vol. 1679, pp. 900-909, 1999. M. S. Engelman, R. L. Sani, and P. M. Gresho, "The implementation of normal and/or tangential boundary conditions in finite element codes for incompressible fluid flow," International Journal for Numerical Methods in Fluids, vol. 2, pp. 225-238, 1981. C. Davatzikos, "Measuring biological shape using geometry-based shape transformations," Image and Vision Computing, vol. 19, pp. 63-74, 2001. P. Dumpuri and M. I. Miga, "Model-updated image guidance: A statistical approach to gravity-induced brain shift," in Medical Image Computing and Computer-Assisted Intervention - MICCAI 2003, Pt 1, vol. 2878, Lecture Notes in Computer Science. Berlin: SPRINGER-VERLAG BERLIN, 2003, pp. 375-382. T. K. Sinha, V. Duay, B. M. Dawant, and M. I. Miga, "Cortical shift tracking using a laser range scanner and deformable registration methods," in Medical Image Computing and Computer-Assisted Intervention MICCAI 2003, Pt 2, vol. 2879, Lecture Notes in Computer Science. Berlin: SPRINGER-VERLAG BERLIN, 2003, pp. 166-174.