HELSINKI UNIVERSITY OF TECHNOLOGY Telecommunications Software and Multimedia Laboratory Tik-111.500 Seminar on computer graphics Spring 2003: Real-Time 3D Graphics

Potentially Visible Set (PVS) Mikko Laakso

45313V

24.4.2003

Potentially Visible Set (PVS) Mikko Laakso HUT, Telecommunications Software and Multimedia Laboratory [email protected]

Abstract Many graphics applications today have to cope with large amounts of data. A popular approach to reduce the amount of geometry to be rendered is to precompute visibility in order to render only objects that can be seen from the given viewpoint. This paper presents one technique to achieve this, potentially visible sets (PVS). These sets are used to avoid rendering hidden objects, especially geometry that is obscured by geometry closer to the viewer. Over the years many different techniques have been developed to perform this task. The idealized exact solution for general 3D scenes has often been regarded as computationally intractable. But recently a practical solution has been found. The essense of a new approach is to represent 3D polygons and the stabbing lines connecting them in a 5D Euclidean space derived from Plucker space and then to perform geometric subtractions of occluded lines from the set of potential stabbing lines. 1 INTRODUCTION Recent advances in graphics hardware have not been able to keep up with the increase in scene complexity. In order to support a new set of demanding applications, a multitude of rendering algorithms have been developed to both augment and optimize the use of hardware. Visibility determination is a key requirement in a wide range of graphics algorithms. An effective way to speed up rendering is to avoid rendering geometry that cannot be seen from the given viewpoint, such as geometry that is outside the view frustum, faces away from the viewer, or is obscured by geometry closer to the viewer. The hardest part of the visibility-culling problem is to avoid rendering geometry that cannot be seen due to its being occluded by closer geometry. This is called occlusion culling. Occlusion culling finds the visible parts of the scene without wasting time on the occluded parts, not even to determine that they are occluded. Occlusion culling is fundamentally different from view-frustum culling. In view-frustum culling, the determining relationship is that between the viewer and the polygon to be culled. To cull a polygon because of occlusion involves determining the spatial relationship between the viewer, the polygon, and other polygons in the scene. For this reason, occlusion culling algorithms are generally more complicated than view-frustum culling algorithms, and they are very sensitive to the scene being dynamic. 1

There had been some brief mentions about occlusion culling already in the 1970s, but the first practical algorithm for occlusion culling was described by Airey et al. (1990), who also coined the phrase potentially visible set – PVS. They suggested to partition the 3D model space into cells, and to compute (offline, e.g., by ray-casting) for each cell a PVS – a list of polygons that might be visible from at least one viewpoint in the cell. When rendering the scene from a given viewpoint, it suffices to render just those polygons in the PVS of the cell containing the viewpoint. This algorithm is obviously conservative, since many polygons will be rendered unnecessarily, but the set of rendered polygons will hopefully still be significant smaller than the total number of polygons in the scene, resulting in accelerated rendering. The conservatism may be reduced by increasing the number of cells, at the expense of more offline computing time and storage space for the PVSs. The power of the potentially visible sets can be seen in Figure 1. The top picture shows a rendered picture of a typical game level, in this case a castle corridor. The lower pictures show the same view with wireframe pictures. The left picture shows just a brute force rendering. The complete level part which intersects the view frustum is rendered, although only a very small part of it is really visible in the end. The white lump near the middle represents a major part of the complete level at a larger distance. Rendering this kind of view several times per second would be a demanding task for any computer. On the other hand, the right picture has been rendered using precalculated PVS. Only possibly visible objects are rendered and only they cost any performance. The difference is significant.

Figure 1: Example pictures showing the power of PVS. Pictures borrowed from Shark 3D technical article available at http://www.spinor.com/technology/render/pvs.html

2

2 PVS TECHNIQUES Calculating the potentially visible set is not always easy. Limited resolution and the question of sample point count and placement are just few of the problems. Over the years many different techniques have been developed to perform this task. These solutions can be loosely categorized (Nirenstein et al, 2002) into those applied either run-time of during a pre-process. With a few exceptions, the former determine the visibility from a single point, while the latter establish the subset of geometry visible from any point within a region. From-region visibility partitions the view-point space into regions of cells, rather than attempting to preprocess the infinite number of possible camera positions. The principal advantage of these techniques is that their considerable computation cost can be shifted to a pre-process, consequently removing significant runtime visibility computation. The results may then be saved to disk for future use. The disadvantage is that, as a pre-process, only static scenes can be fully treated. From-point visibility algorithms are less costly computationally than from-region approaches, allowing them to be applied on a per-frame basis at run-time. They are also better suited to unpredictably dynamic environments, where the share or position of objects may change between frames. Occlusion culling algorithms may be further categorized according to their accuracy in differentiating between visible and hidden polygons. Nirenstein et al. (2002) compare algorithms by image quality (correct or containing errors) and run-time performance (optimal or sub-optimal) and discriminate between conservative, aggressive, approximate and exact visibility algorithms.

• Conservative techniques consistently overestimate visibility and incur a false visibility error: hidden polygons are considered visible. This naturally gives correct images but results in sub-optimal run-time performance because some polygons are unnecessarily submitted to rendering pipeline. In decent conservative algorithms the conservativity should be no more than 30%. • Aggressive methods always underestimate the set of visible geometry and exhibit false invisibility, where visible polygons are erroneously excluded. Aggressive visibility causes image error but can be useful if the perceptual impact of the error is acceptably small or the algorithm handles scenes that cannot be solved effectively with conservative alternative due to excessive overestimation. However, in practise even rather small image errors are usually quite noticeable and thus unacceptable. • Approximate visibility techniques give both false visibility and false invisibility errors. They are thus useful usually only when the pre-processing efficiency is the overriding concern. • Exact visibility solutions provide both accurate images and optimal rendering performance. An exact visibility query will produce no more or less than the union of the polygons visible from all viewpoints within the region. There exists several surveys of different computer graphics occlusion culling techniques and algorithms. An interested reader may look for example Zhang (1998), SaonaVasquez et al. (1999), Durand (1999), Cohen-Or et al. (2001) and Bittner (2002). The following chapters of this paper present some of the most interesting solutions. 3

2.1 Cells and Portals Teller (1992) enchanced the basic PVS technique by partitioning the scene into cells, and defining portals between the cells. The visibility of one cell from another depends on the existence of a line of sight between them, intersected only by portals – the nonopaque boundaries between cells. Teller derived an analytic solution to this problem and further extended the cell-portal technique in order to determine exactly which parts of visible cells are indeed visible. These algorithms work well for scenes where the computational complexity of portal sequences is low. They are suited to indoor architectural scenes (rooms are cells, doorways and windows are portals), but aren’t a general visibility solution. Cell-portal rendering in general requires the model’s major occluders to be distinguished from its detail objects. Making this distinction manually precludes using these techniques in an automated batch process, and finding a general heuristic that makes this distinction automatically and properly for all architectural models is non-trivial. The cell intervisibility relationship can also occupy too much memory and take too long to calculate. Luebke and Georges (1995) used cells and portals too (see Figure 2). However, they avoid the time and space required to precalculate the entire cell intervisibility relationship by estimating the required part of the relationship at run time, based on the portals’ screen-space projections. Their scheme first projects the vertices of each portal into screen-space and takes the axial 2D bounding box of the resulting points. This 2D box, called the cull box, represents a conservative bound for the portal. As each successive portal is traversed, its box is intersected with the aggregate cull box using only a few comparisons. During traversal the contents of each cell are tested for visibility through the current portal sequence by comparing the screenspace projection of each object's bounding box against the intersected cull box of all portals in the sequence. If the projected bounding box intersects the aggregate cull box, the object is potentially visible through the portals and must be rendered. Unfortunately, the other drawbacks of cells-and-portals techniques remain, namely unsuitability to scenes other than indoor architecture and the difficulty in distinguishing major occluders.

Figure 2: Example pictures showing cells and portals. The left picture shows a view from a viewpoint inside a house. Cull boxes for portals are shown in white and mirror in red. The right picture shows an overhead view of the same viewpoint., showing the portal culling frustums (mirror frustum shown in red). Pictures from Luebke and Georges (1995).

4

Cohen-Or et al. (1998) and Saona-Vasquez et al. (1999) provide a more general, but more conservative solution. They only classify an object as hidden from a particular view-cell if a single polygon occludes it from every point inside that view-cell. For a given cell the method tests the visibility of all objects in the scene. For each object it searches for a strong occluder which guarantees that the object is not visible from any point within the cell. This is so conservative that it only achieves significant results if view-cells are small, relative to the size of scene polygons. For highly detailed scenes, this often requires a prohibitive number of view-cells and hence excessive runtimes. To counter this, it is necessary to consider the aggregated occlusion by a set of smaller occluders. Recent work has focused either on fusing occluders, or on the construction of larger virtual occluders, which represent the occlusion of multiple smaller occluders. 2.2 Virtual Occluders Given a scene and a viewcell, a virtual occluder is a view-dependent (simple) convex object, which is guaranteed to be fully occluded from any given point within the viewcell and which serves an effective occluder from the given viewcell. Figure 3 shows the viewcell in yellow and a set of occluders, The umbrae of individual occluders is illustrated in 3(a), showing that the occlusion of the individual objects is insignificant. Nevertheless, the aggregate occlusion of these objects is much larger, as can be seen in 3(b). Virtual occluders are a compact intermediate representation of the aggregate occlusion for a given cell. The introduction of such view-dependent virtual occluders enables applying an effective region-to-region or cell-to-cell culling technique and efficiently computing a PVS from a region/cell. Koltun et al. (2000) make use of separating lines to build larger more effective virtual occluders to represent many smaller occluders. They note that by building their virtual occluders as a pre-process and performing the occlusion culling at run-time (on a per cell basis), the cost of storing the visibility for each cell is removed. However, they only implement a 2½ D (height field) solution.

Figure 3: The union of the umbrae of the individual objects is insignificant, while their aggregate umbra is large and can be represented by a single virtual occluder. Pictures from Koltun et al. (2000).

5

Schaufler et al. (2000) presents a similar approach in 3D, where virtual occluders are generated from a volumetric discretisation of the interior of the scene objects. The method is conservative and classifies regions as occluded only when they are guaranteed to be hidden. It operates on a discrete representation of space and uses the opague interior of objects as occluders. This choice of occluders faciliates their extension into adjacent opaque regions of space, in essence maximizing their size and impact. The method efficiently detects and represents the regions of space hidden by such occluders. Wonka et al (2000) have a conservative 2½ D solution. They shrink a subset of occluders and then sample visibility (efficiently fusing occluders). Occluder shrinking allows the sampling process to maintain conservativity. The algorithm discretizes the scene into view cells, for which cell-to-object visibility is precomputed, making on-line overhead negligible. The method is able to conservatively compute all forms occluder interaction for an arbitrary number of occluders. There is a trade-off between samples required (and hence time) and the degree of shrinkage. This algorithm tends towards an exact solution as the number, and hence the amount of time required, tends towards infinity. 2.3 Aggressive Solutions The techniques surveyed above are all conservative in nature. They provide accurate images, but for many scenes there is a large margin between the size of the visibility sets they generate and those of an exact visibility solution. Recently, aggressive frompoint based solutions have been developed that admit false invisibility errors using approximate culling. These techniques aim to reduce rendering costs by removing objects that contribute little to the image. Andujar et al. (2000) use hardly visible sets to cull or simplify objects where only a small proportion of their geometry is visible. Their framework combines both occlusion culling and level-of-detail rendering techniques to improve rendering times. The main idea is to split each PVS into several subsets. More precisely, the PVS is split into two kinds of subsets. Completely unoccluded objects will be stored into completely visible set, and partially occluded objects will be arranged into several sets grouped by similar occlusion degree. Both kinds of subsets are named Hardly Visible Sets (HVS for short). At navigation time, the HVS is used to determine with a very simple computation those objects whose pixel contribution (measured as screen projection of the unoccluded part) is low enough to avoid its visualization while assuring that image losses are within a user-bounded threshold. The HVS with visibility levels lower than the user-defined tolerance are culled, and the remainder of HVS are candidates to be displayed. According to image accuracy and frame rate requirements, hardly visible sets provide a way to avoid sending to the graphics pipeline those objects whose pixel contribution is low due to partial occlusion. The image error can be bounded by the user at navigation time. On the other hand, as HVS offer a tighter selection of the level-of-detail at which objects are rendered. In general, hardly visible sets provide an easy way for selecting which objects can be culled by means of a user-defined threshold on the pixel contribution of their unoccluded part.

6

Klosowski and Silva (2000) present a prioritized layer projection algorithm that uses a heuristic priority ordering that tries to draw visible polygons first. It works by estimating the visible polygons of a scene from a given viewpoint incrementally, one primitive at a time. From a very high level, prioritized layer projection amounts to a modification of a simple view-frustum culling algorithm. However, it requires the computation of a special occupancy-based tessellation and the assignment to each cell of the tessellation a solidity value. This is used on computing, on demand, a priority order for the polygons that maximizes the likelihood of projecting visible polygons before occluded ones for any given scene. The algorithm does this in two steps: 1) As a preprocessing step, it computes an occupance-based tessellation of space, which tends to have smaller spatial cells where there are more geometric primitives, e.g., polygons; 2) at run-time, rendering is performed by traversing the cells in an order determined by their intrinsic solidity (likelihood of being occluded) and some other view-dependent information. As cells are projected, their geometry is scheduled for rendering (see Figure 4). Actual rendering is constained by a user-defined budget, e.g., time or number of triangles. This is a good time-critical solution, since one gets reasonable results even if rendering is prematurely terminated.

Figure 4: The prioritized-layered projection algorithm attempts to prioritize the rendering of geometry along layers of occlusion. Cells that have been projected by the algorithm are highlighted in red wireframe and their associated geometry is rendered, while cells that have not been projected are shown in green. Notice that the cells occluded by the desk are outlined in green, indicating that they have not been projected. Pictures from Klosowski and Silva (2000).

Gotsman et al. (1999) present a novel sample-based visibility solution. They use a 5D sub-division over three spatial and two angular dimensions. Each 5D cell maps to a beam in 3D space. The use of two angular divisions is intended to accelerate frustum culling at run-time. To determine from-region visibility, rays are cast from a random point in the cell to random points on an objects’s bounding box. A statistical model based on whether the rays hit the target object, is then used to decide if the objects is visible, hidden or whether more rays need to be cast. Error thresholding allows a tradeoff between pre-processing time and accuracy.

7

2.4 Exact Solutions Turning to exact visibility, various solutions have been developed for answering general visibility queries. These techniques typically build a structure in some form of dual line space that directly exposes visibility events (Nirenstein et al., 2002). A visibility event occurs when a topological change in visibility occurs in the scene (eg. the set of viewpoints from which a particular vertex of a previously occluded triangle may cross an edge of another and become visible). Structures representing this for general scenes are the aspect graph (changes in aspect are also encoded), the 3D visibility complex (Durand, 1999), and the visibility map. Earlier, Teller (1992) showed how the construction of the anti-penumbra through a set of portals may be computed via a similar structure, using the Plucker parametrization of line space. The usefulness of these algorithms is severely limited by combinatorial complexity and robustness issues. Koltun et al. (2001) build a representation of the rays between segments in a dual ray space. Occlusion is computed by determining whether the space of occluded rays contains all rays between the view-cell and the object in question. The visibility computation is represented in two-dimensional dual ray scape by discretising the unit square to bitmap, rendering all rays onto this bitmap (white pixels to black background) and then checking whether there are any black pixels left. Inaccuracy is minimized by shrinking particular polygons prior to rendering, ensuring conservativity. This algorithm is an efficient approximation of 2D exact visibility solution. It is robust and and can trade off between accuracy and speed. An example picture of the algorithm results is shown in Figure 4. Independently, Bittner et al. (2001) also developed an exact visibility solution in the plane, based on similar principles. The algorithm uses a subdivision of line space to analytically calculate a conservative PVS set for a given region in the scene.

Figure 5: The results of the dual ray algorithm for a viewcell that partly lies on London’s Oxford Street. The model is shown on the left and the results are visualized on the right. The algorithm has detected that the whole lenght of Oxford Street is visible (blue) from the viewcell (green), and was able to mark immediately the surrounding areas as occluded (red). Pictures from Koltun et al. (2001). 8

3 FIRST PRACTICAL 3D EXACT SOLUTION The previously discussed exact solutions were for 2D only. The idealised exact solution for general 3D scenes has often been regarded as computationally intractable. But recently, Nirenstein et al. (2002) presented the first exact 3D from-region visibility culling algorithm trackable for realistically sized scenes (as many as 1.5 million polygons). They follow Teller’s convention and use the Plucker coordinate line space parametrization. Plucker space is a special case of a Grassman coordinate system. The Plucker mapping of a directed line passing through the point (  ,   , 
) and then through (  ,   ,   ) , is defined as Π (l ) = (π 0 , π 1 , π 2 , π 3 , π 4 , π 5 )

where: q

p

π0 = o p − o p q q π3 = m n − n m

u v

π1 = t u − t u v v π4 = r s − s r

{

z

π2 = y z − y z { { π5 = x w − w w

This homogenous six-tuple of Ρ 5 is a unique representation of to within multiplication by a positive scale factor. Negating the scale factor flips the orientation of the line. Next issue to consider is duality mapping within Ρ 5 . Given π , ∈ Ρ 5 , one can define  5 to be the quantity π ( ): Ρ → π 0 | 3 + π 1| 4 + π 2 | 5 + π 3 | 0 + π 4 | 1 + π 5 | 2 which is a permuted inner product of π and  . The set of solutions  ∈ (Ρ 5 , of   so-called , -  !" #%$ π &!' Ρ 5 . Given lines 1 and ) π ( ) = 01 gives* rise to the 2 = Π (+ 2 ) ; 1 and 2 are incident if and only if π 1 lies on the 2 , let π = Π ( 1 ) and π dual hyperplane of π 2 (and vice versa). Formally, they are incident if and only if } π2 =0 π1 . / If this is not equal to zero, then the relative orientation of 1 and 2 is directly specified by the sign of the function. Figure 6 (in the next page) shows an illustration.

( )

0 1 Although all lines in 3 map to points in Ρ 5 , not all points in Ρ 5 map to lines in 3 . 2 Rather, Π is a bijection between the lines in 3 and a particular four-dimensional quadric surface embedded in Ρ 5 known as the 3547698:8;?6=@ ACBD!E , the FHG!IJLK>MNOPQ7JLR or the SUTLVWX9YZ\[]^YZ`_