Multiresolution representation and deformation of very large volume datasets based on Haar wavelets Heurtebise Xavier, Thon Sébastien LSIS: Laboratoire des Sciences de l’Information et des Systèmes IUT de Provence, Rue Raoul Follereau, 13200 Arles – France {[email protected], [email protected]} Abstract In a virtual sculpture project, we would like to sculpt in real-time very large 3D objects sampled in volume elements (voxels). The drawback of this kind of representation is the important number of voxels required to represent very large and detailed objects. Consequently, that entails important memory cost and computation time issues. In order to allow real-time performance, we propose in this paper a new multiresolution model that combines octree and wavelet: a 3D object is roughly sampled in an octree, where each leaf containing data is thinly sampled thanks to a 3D Haar wavelet transform. Keywords--- Geometry and modelling ⋅ Virtual sculpture ⋅ Voxels ⋅ Levels of detail ⋅ 3D wavelets ⋅ Very large volume datasets

1. Introduction In this paper, we present an adaptive multiresolution model based on octree and 3D wavelets to represent a 3D object as a discrete set of volume elements (voxels). Such a discrete representation is of great use in medical imaging (to manage MRI or CT-scan data), in surgical simulations (bone surgery, dental surgery), in scientific simulations (in order to simulate inhomogeneous systems such as semiconductor device simulation, molecular dynamics, plasma physics, and fluid mechanics), and in a virtual sculpture context as it allows to simulate easily sculpture operations such as addition or subtraction of material by simply adding or removing voxels. However, the major problem of discrete representations of volumes is the huge memory cost when 3D objects are very large. To avoid a memory overload, we propose a new data structure which combines octree and wavelets: a 3D object is roughly sampled in an octree, where each leaf containing data is thinly sampled thanks to a 3D wavelet transform. A great advantage of our model is its multiresolution nature given by the use of 3D wavelets, that allows to accelerate display, interaction and sculpture operations thanks to levels of detail.

Another major advantage of our system is that we use this model to represent both objects and sculpting tools, so we can let the user design his or her own tools, by sculpting them from basic objects. This is an original approach to virtual sculpture, where most of other approaches use different models for objects and tools.

2. Previous works 2.1. Discrete spatial enumerations In this paper, we tackle the problem of virtual sculpture of a very large 3D object with a tool, both represented with spatial enumerations. Such a spatial enumeration is a set of volume elements called voxels, obtained by sampling the volume of a 3D object. It can be seen as a 3D image composed of voxels, where a 2D image is a bidimensional array composed of pixels. To make a spatial enumeration from a 3D object, several methods have already been suggested. The simplest way is a uniform discrete spatial enumeration, by regularly sampling the object into voxels with the same size. However, a major drawback of this representation is the large number of voxels needed to represent very large objects with detailed features (for example, a 3D image in 1024×1024×1024 has more than one billion voxels). This entails three main problems. The first one is the important memory cost to store this uniform spatial enumeration. The second one is that the display of these objects becomes slower. Finally, operations on these objects such as sculpture actions or displacements become less and less interactive. Ferley et al. [9] works on the binary search trees and hashing-tables to reduce the numbers of voxels stored in memory. The problem of this structure is that for a voxel, more information such as the position and the density of the voxel are stored in memory. So, the memory cost can be greater than a simple uniform discretization. Moreover, this kind of representation uses a sampling step fixed by the system, which is very restricted to represent the thinness of a 3D object. To prevent these inconveniences, adaptive sampling methods have been developed. Libes [18] uses an octree to gather groups of adjacent voxels having same values to reduce the number of elements stored in memory. This

method is very simple to use and to implement. Ferley et al. [10] also proposed a “n-tree” where each cell can be divided in 27 ones. This method looks like an octree and allows to reach a high level of detail. However, for an object with small details, the subdivision level of an octree or “n-tree” will be very high. So the processes (construction and use) will be slow. Several other sampling methods used in collision detection propose to modify properties of the voxels, such as the size, the orientation or even the shape. With the AABB method, Bergen [5] suggests to use voxels with different sizes. Gottschalk et al. [13] proposes to modify not only the size of the voxels but also their orientation with the OBB method. Hubbard [17] and Bradshaw and O’Sullivan [7] suggest to replace cubes by spheres to form a spheres tree, because spheres accelerate collision detection between objects. Thanks to these three methods, the object rendering is optimized because the original object shape can be approached with less voxels than with a simple uniform spatial enumeration, but to the cost of higher computation times.

2.2 Multiresolution representations To further improve the use of spatial enumeration, several methods of multiresolution representations have been proposed. So, processing and display times are adapted with the desired level of detail. Among these methods, there are trees and 3D wavelet decomposition. An octree can also be seen as a hierarchical representation of 3D object. The maximum level of subdivision of the octree defines the maximum level of detail of a multiresolution representation. Boada et al. [6] defines a section in an octree that determines the displayed nodes for a given level of detail. This method is extended to a “n-tree” by Ferley et al. [10]. The second multiresolution method uses wavelet decomposition. Wavelets are a mathematical tool for representing functions hierarchically. Wavelets are scaled and translated copies (daughter wavelets) of a finite-length oscillating waveform (mother wavelet). In our case, these functions are discrete 3D functions that define a set of voxels. Muraki [20] shows the use of 3D Haar wavelets [14] to represent a 3D object. Pinnamaneni et al. [21] build a 3D Haar wavelet decomposition from a sequence of 1D Haar wavelet decomposition in each direction of the 3D voxels grid. Wavelet decomposition allows to display a 3D object faster according to the level of detail. It also permits to drastically cut down the memory cost, because high compression ratio can be achieved on wavelets coefficients, especially if lossy compression schemes are used.

2.3. Virtual sculpture The previously cited methods are about discrete representation of 3D objects. Different methods have already been proposed to sculpt these kinds of objects.

A first volumetric method has been implemented by Galyean and Hughes [12]; their model is defined with a uniform discrete spatial enumeration which contains density value. 3D tools modify the discrete potentials which describe the object. Basic operations like addition or subtraction, and several tool definitions (heat gun, sand paper or colour modifier) are proposed. Ayasse and Müller [1] perform sculpture operations by the use of CSG (Constructive Solid Geometry). Complex objects are created by successive modifications of the material with a tool according to simple operations such as difference, union or intersection. However, the object and the tool are represented by simple uniform spatial enumerations. Moreover, voxels are limited to binary values (full or empty). Ayasse and Müller propose to reduce the computation time for each sculpture operation by using only the effective voxels according to a given displacement of the tool. However, they don’t use a multiresolution representation that could improve the display performance. In the Kizamu project, Frisken and Perry [11] use ADFs (Adaptively sampled Distance Fields) to model and to sculpt the material. A 3D object is sampled adaptively with a 3D grid according to the details of the object. Each grid cell contains a scalar specifying the minimum distance to the object shape. This distance is signed to distinguish the inside from the outside of the shape. Operations are CSG on the ADFs of the object, but the Euclidean distance field used instead of density raises discontinuity problems and increases the update computation time. Bærentzen and Christensen [2] propose the LevelSet method to deform the material. This method stores distance fields around the exterior of a 3D object. The single tool is a blob (a sphere) represented by an implicit function, which limits the sculpture capabilities. To represent an object to be sculpted, Ferley et al. [10] also use distance fields, stored in a “n-tree” hierarchical representation where the sampling rate depends on object’s details. The tool is limited to an ellipsoid defined by an implicit function discretized to perform sculpture actions on the object: this limitation is very restricted if the user wishes to use more complex tool shapes to perform specific sculpture actions. Raffin et al. [21] propose a model of virtual sculpture based on a multiresolution representation: the octree. The tools are defined as voxels sets, but they remain parallel to the axis, which limits the orientation of the tool and limits the sculpture operations. Heurtebise et al. [16] proposed a multiresolution sculpture system based on 3D Haar wavelets. A major advantage of this model is that sculpted objects can then be used as new tools, because the same model is used for both objects and tools. Moreover, any orientation of tools [15] is possible, which doesn’t limit the sculpture operations. In this paper, we will propose an extension of this model to manage very large volume datasets.

3. Original contribution Many models have already been proposed in order to represent 3D objects as discrete sets of voxels. However, these discrete representations face many problems when 3D objects are very large, like the huge memory cost, the computation time during sculpture operations and display. Consequently, to solve these limiting problems, the main contribution of this paper is to propose a new data structure that allows visualization and interactive modifications of very large volume datasets. This data structure combines octree and wavelets: a 3D object is roughly sampled in an octree, where each leaf containing data is thinly sampled thanks to a 3D wavelet transform. The remainder of the paper is organized as follows. In section 4, we describe our discrete multiresolution method combining the octree and the 3D wavelets. In section 5, we present dynamical and interactive modifications of 3D objects thanks to sculpting tools. Then, we conclude and present future works.

where the level of detail missing in the LoD structure must be created from a level already present in the cache, as less reconstruction steps are needed.

4. Our model for very large objects As the uniform spatial enumeration of a 3D object is expensive in processing and display times, we proposed in a previous paper [16] a multiresolution model based on 3D discrete Haar wavelets with real coefficients that overcomes these problems. However, there is a quality issue as real numbers on computers have a limited precision. Moreover, the process of real data is slower than integer data. Lastly, coding information with reals is more expensive in memory than with integers. So, to avoid these problems with real data, we choose the lifting integer Haar wavelet transform [8]. We propose now to enhance this model to tackle problems encountered when handling very large datasets.

4.1. Octree

(a)

Figure 2 LoD structure The results given in the paper have been obtained on a PC with an Intel Core 2 Duo 3 GHz with 512 Mb SDRam DDR2, a NVidia GeForce 7300 GT with 256 Mb video memory, a hard disk Western Digital Raptor SATA 34 Gb, 10000 rpm, and Windows XP Pro. 256x256x256 2 120 ms

128x128x128 330 ms

64x64x64 18 ms

32x32x32 7 ms

Table 1 Building times for lifting integer 3D Haar wavelet data in relation to the size in voxels of the initial 3D image.

(b)

256x256x256 1 590 ms

128x128x128 157 ms

64x64x64 15 ms

32x32x32 4 ms

Figure 1 Octree where (a) leaves are empty if there is no data and (b) leaves contain wavelet enumerations if there is material

Table 2 Extracting times for lifting integer 3D Haar wavelet data in relation to the size in voxels of the extracted 3D image.

First, the 3D object is roughly sampled in an octree (see figure 1), where each leaf containing data is thinly sampled thanks to a 3D wavelet transform. The extracted levels of detail for each block, presented in section 4.2, are stored in a cache, that we called LoD structure, presented in [16] (see figure 2), in order to avoid to extract the level each time we need to use it. The building and extracting times correspond only to the case

If the size is too small, the number of blocks will be too high and the processing times will increase. In order to ensure real-time performances, we found out that the more appropriate size of the wavelet enumeration for a leaf is 32×32×32 with less than 10 ms for the building and extracting times (see tables 1 and 2). Consequently, the maximum level of details of a 3D wavelet in this case is 5 (25 = 32).

The octree must satisfy the following conditions: • The root must contain the whole 3D object. • Only the leaves of the octree can contain a wavelet enumeration, if there is material. • A leaf can be empty if there is no data. Each node of the octree contains the following data: • Position in the scene. • Size of the block. • A pointer to its parent. • Eight pointers to its children if exist. • Wavelet data for leaves containing material.

4.2. Block structure

o

We can choose different level of detail to display each leaf of the octree to satisfy the conditions of real-time interaction between a user and the 3D object.

The limitation of the current display method is that we can see some joint problems if the Marching Cubes algorithm [19] is independently applied on each leaf of the octree, because the links between adjacent blocks are not treated (see figures 7 and 8). To avoid the apparition of joint problems, we could use another display method such as surfels [23].

5. Dynamical and interactive modifications Our model can easily handle sculpture operations by simply adding or removing voxels. A major advantage of our method is that the tool used for virtual sculpture has the same representation than the matter. So, the user can create his or her own tools to sculpt another 3D object. The tools can take any orientation and any position in relation to the object [15]. Other sculpture operations will be studied in future work.

5.1 Extension and reduction of the 3D object

Figure 3 A 3D object roughly sampled in an octree where leaves containing data are thinly sampled in 3D Haar wavelet enumerations. The figure 3 shows a 3D object represented by an octree where the leaves are wavelet enumerations when there is material. Each wavelet is a block which contains this information: • the wavelet enumeration; • a LoD structure (see figure 2) that caches extracted wavelet data and triangulated mesh of each level of detail of the wavelet enumeration. If a level of detail doesn’t appear in the LoD structure, this level of detail isn’t in memory.

4.3. Advantages and limitations The advantages of this extended model are multiple: • It takes the advantages of the octree: o The memory cost of an octree is smaller than the one of uniform enumeration. o An octree can be seen as a hierarchical representation of a 3D object. o An octree can be extended or reduced [3]. • It takes the advantages of wavelet enumeration: o Each leaf of the octree can be decomposed in wavelet enumeration with a different maximum level of detail. o The wavelet permits to compress efficiently the 3D objects.

During the sculpting process, there are two cases of sculpting operations: • When we add matter thanks to the tool, we determine which leaves of the octree of the object intersect the tool, and we create new leaves with 3D Haar wavelet if necessary. Then, each of these nodes is modified according to the algorithm described in section 5.2. • When we subtract matter thanks to the tool, the matter contained in a leaf of the octree can be entirely deleted. So, we can delete the wavelet data of this leaf, as well as the leaf itself and potentially many empty parent nodes. For that, we use a system of “Garbage collector” to update the octree and delete the useless nodes. 5.1.1 Creation of a new enumeration When we add matter to a 3D object, we add a new leaf containing wavelet data to the octree of the object. There are two cases: • The new data is inside the bounding box of the octree. In this case, we add a new leaf to the octree such as the new leaf is the smaller node containing the new data • The new data is outside the bounding box of the octree. In this case, we extend the octree by the root, by creating a new root including the previous one, in the direction given by the new data. We create a new root until the new data is inside the new root. While the new data is outside the bounding box of the octree, we create a new root, otherwise we extend the octree by the leaves.

(a) Initial octree.

(a) Initial octree on left and new data on right

(b) First pass: deleting of empty nodes without child.

Zone where the new data is

(c) Second pass: reduction of the root if it has only one useful child.

(d) Final octree. Position of new root (b) Root extension

Figure 5 Process of "garbage collector": the first pass (b) allows to delete the empty nodes without child, and the second pass (c) allows to reduce the root if it has only one useful child

5.2 Modification process (c) Leaves extension

Figure 4 (a) Position of the new data in relation to the initial octree. (b) Creation of a new root for the octree in relation to the position of the new data. (c) Creation of new leaves for the new data. The figure 4 shows a 2D case of root extension and leaves extension to add the new data into the initial quadtree. The principle is the same in 3D with the octree. 5.1.2. “Garbage collector” When we delete a node in the octree, some nodes can become useless. So, we propose a system of “Garbage Collector” (see figure 5) to update the octree and delete the useless nodes, in two passes: • First pass: we delete all the empty nodes of the octree, without child. If a node is not empty but the wavelet contains no data, i.e. all the wavelet coefficients are null, the wavelet is deleted. • Second pass: if the root has only one useful child, this child becomes the new root and the previous root is deleted. Nevertheless, in order to avoid slowing the performances of our system, we don’t perform this “garbage collector” after each sculpture operation, but only when no sculpture operation has been performed for a given time.

During the sculpture operations, we determine which blocks of the object intersect the blocks of the tool. We extend the octree representation if necessary. Then, for each block btool of the tool and each block bobject in intersection with btool, the following operations are performed: • In a first step, we perform the following tests of collision detection to determine if the tool and the object intersect each other: o We use the method of collision detection between two AABB [5] in order to determine roughly and rapidly if the bounding boxes of the tool intersect the one of the matter. o If this test is positive, we refine it by using the method of collision detection between two OBB [13] which is more precise than the previous one. o If the previous test is positive, we decompose the more little bounding box of the tool or the object into 8 boxes, like an octree decomposition. Then we perform again the first step. In order to avoid to make too many tests, which slow down the collision detection, we stop the decomposition at a given level of detail. • In a second step, we find the voxels of bobject and the tool which are in the intersection zone. • In a third step, we extract the voxels of btool thanks to the method of discrete rotation [15], only for the voxels in the intersection zone.

•

• •

In a fourth step, for each voxel vobject of bobject in the intersection zone, we find which voxels of btool intersect it, and we compute the new value of vobject according to the selected sculpting mode and the filling percentage of this voxel by those of btool. This voxel value is clamped to 32767 because we have density values between 0 and 32767 (see figure 6, where the values are given in percentage, i.e. 100% (respectively 0%) is used for the density value 32767 (respectively 0)): o In the “adding matter” mode, if the voxel of the tool isn’t null, the filling value (equals to the filling percentage multiplied by 32767) is given to the value of vobject only if the filling value is greater than the value of vobject. o In the “subtracting matter” mode, if the voxel of the tool isn’t null, the empty space value (32767 – filling value) is given to the value of vobject only if the empty space value is smaller than the value of vobject. In a fifth step, we modify the wavelet data according to this new voxel value. In a last step, for each level of detail, the triangulated surface is rebuilt by Marching Cubes only for the modified parts of the 3D object, to improve the computation time.

Figure 6 Adding (c) or subtracting (d) material to an object (a) with a tool (b).

6. Conclusions We have presented in this paper a multiresolution model for virtual sculpture of very large 3D objects with tools, using a combination of octree and 3D wavelet: a 3D object is roughly sampled in an octree, where each leaf containing data is thinly sampled thanks to a 3D wavelet transform. The advantages of this model are the following. First, this model allows to display and sculpt a discrete 3D object faster according to the level of detail. Then, it allows to reduce the memory cost. Indeed, empty zones of the 3D object aren’t stored in memory thanks the octree. Moreover, wavelets permits to compress efficiently each block of the 3D object. Lastly, the octree permits to extend or reduce the size of the 3D object. Consequently, thanks our model, we aren’t limited by the shape the resolution and the size of the 3D objects, because very fine details can be added by adding or deleting some blocks of the octree. To verify the applicability of our sculpting system, we have conducted many sculpting sessions which have resulted in numerous interesting sculptures. Some sculptures examples are shown on figures 7 and 8, and several other examples can be seen on http://www.iutarles.up.univ-mrs.fr/lxao/virsculpt/galerie.php.

Figure 7 Visualization with our model of a micro-CT scan of a human lumbar vertebral body L3 [4] (487×1024×1024 voxels). The multiresolution nature of our model permits to automatically show more and more details when getting closer to the data (from up to bottom), up to the spongy structure of the bone, at a constant framerate of 6 FPS.

[5]

[6]

[7]

[8]

Figure 8 A hole is dug through the previously presented vertebra with two spherical tools in 32×32×32 and 64×64×64 voxels. This operation can simulate a surgical operation by a specialist. The framerate during the sculpture operations and the display is about 6 FPS.

[9] [10]

[11]

7. Future work Many improvements of our sculpture system are possible, by investigating open issues such as memory cost, interaction with the object or computation time. Memory cost to store very large 3D objects can be very huge. So, we plan to create a cache system method on disk that will offer a compromise between memory usage and computation time. Interactive computation times will always be a challenging issue. In order to accelerate the sculpture actions, we plan to take more advantage of the levels of detail of the 3D Haar wavelet. In order to accelerate the display when the user interacts with the matter, different display methods will be investigated (surfels, point-based rendering…). We will also investigate the use of graphics hardware to speed up many parts of our system, such as a cache system in video memory to store the triangulated surface of wavelet data.

References [1]

[2]

[3] [4]

J. Ayasse and H. Müller. Interactive Manipulation of Voxel Volumes with Free-formed Voxel Tools. In Proceedings of the Vision Modelling and Visualization Conference 2001. 359-366. 2001. J.A. Bærentzen and N.J. Christensen. Volume sculpting using the Level-Set method. In Shape Modelling International 2002. IEEE Computer Society. 175-182. 2002. A. Bærentzen. Octree-based volume sculpting. Presented at IEEE Visualization ’98. 1998. G. Beller, M. Burkhart, D. Felsenberg, W. Gowin, H.C. Hege, B. Koller, S. Prohaska, P.I. Saparin and J.S. Thomsen. Vertebral Body Data Set ESA29-99-L3. url http://bone3d.zib.de/data/2005/ESA29-99-L3/

[12]

[13]

[14] [15]

[16]

[17]

[18] [19]

[20]

[21]

[22]

[23]

G.V.D. Bergen. Efficient collision detection of complex deformable models using AABB trees. In Journal of Graphic Tools. 2(4). 1-13. 1997. I. Boada, I. Navazo and R. Scopigno. Multiresolution volume visualization with a texturebased octree. In Visual Computer. 17. 185-197. 2001. G. Bradshaw and C. O’Sullivan. Adaptive medialaxis approximation for sphere-tree construction. In ACM Transactions on Graphics. 23(1). 1-26. 2004. I. Daubechies, R. Calderbank, W. Sweldens and B.L. Yeo. Wavelet transforms that map integers to integers. Applied and Computational Harmonic Analysis (ACHA). 5(5). 332-369. 1998. E. Ferley, M.P. Cani and J.D. Gascuel. Practical volumetric sculpting. Implicit Surfaces.1999. E. Ferley, M.P. Cani and J.D. Gascuel. Resolution adaptive volume sculpting. Graphical Models (GMOD). 63. 459-478. 2002. S.F. Frisken and R.N. Perry. Kizamu: a system for sculpting digital characters. In Proceedings of ACM SIGGRAPH 2001. 47-56. 2001. T. Galyean and J. Hughes. Sculpting: An interactive volumetric modelling technique. Computer Graphics. 25(4). 267-274. 1991. S. Gottschalk, M.C. Lin and D. Manocha. OBBTree: A hierarchical structure for rapid interference detection. In Proceedings of ACM SIGGRAPH’96. 171-180. 1996. A. Haar. Zur theorie der orthogonalen funktionensysteme. Thesis, Gottingen, Germany. 1909. X. Heurtebise and S. Thon. Discrete tools for virtual sculpture. In GRAPP 2006 proceedings. 415-422. 2006. X. Heurtebise, S. Thon and G. Gesquière. Multiresolution representation and deformation of wavelet-based 3D objects. In WSCG’06 short communication papers proceedings. Plzen, Czech Republic. 69-76. 2006. P. Hubbard. Collision detection for interactive graphics applications. Ph.D. Thesis, Dept. of Computer Science, Brown University. 1995. D. Libes. Modelling dynamic surfaces with octrees. In Computer & Graphics. 15(3). 1991. W.E. Lorensen and H.E. Cline. Marching Cubes: a high-resolution 3D surface construction algorithm. In Computer Graphics. 21(4). 163-169. 1987. S. Muraki. Approximation and rendering of volume data using wavelet transforms. In Proceedings of Visualization ’92. Boston. 21-28. 1992. P. Pinnamaneni, J. Meyer and S. Saladi. Remote transformation and local 3-D reconstruction and visualization of biomedical data sets in Java3D. In Proceedings of Electronic Imaging Science & Technology Visualization and Data Analysis Conference. San Jose, CA. 44-54. 2002. R. Raffin, G. Gesquière, E. Remy and S. Thon. VirSculpt: a virtual sculpting environment. In GraphiCon '04 Proceedings. 184-187. 2004. R. Szeliski and D. Tonnesen. Surface modelling with oriented particle systems. In Computer Graphics. 26(2). 185-194. 1992.