10. Rasterization

Even primitives created by geometry shaders, such as the point and line shader, are all ultimately converted to triangle primitives and then generated (involving triangle generation and triangle setup). The generated triangles are culled, clipped, converted into window coordinates, and finally rasterized into a collection of fragments. Unlike in OpenGL ES 2.0, the scissor test is performed during the rasterization stage. Further processing affects the generated fragments.

Figure 10-1. Process From the Vertex and Geometry Shaders to Rasterization

Vertex Shader	Geometry Shader	Triangle Assembly	Culling	Clipping	Window Coordinate Conversion	Rasterization

All processing after rasterization is implemented as a fixed pipeline, so there are fixed vertex attributes that can be assigned to fragments. The following are the major vertex attributes.

  • Window coordinates
  • Depth values
  • Texture coordinates and partial differential values
  • Quaternions
  • View vectors
  • Vertex colors (rasterized from absolute values).

 

10.1. Culling

The order in which vertices are specified for the generated triangles (polygons) determines which face, front or back, is facing the viewer. Culling is a feature that determines whether to rasterize a polygon based on whether it is back-facing.

10.1.1. Determining the Front and Back Faces

A polygon's front and back faces are determined by the order in which its triangles' vertices are specified in window coordinates. Either a clockwise (CW) or a counterclockwise (CCW) winding can be considered to be front-facing. You can specify this using the glFrontFace() function.

Code 10-1. Definition of the glFrontFace Function
void glFrontFace(GLenum mode); 

For mode, specify GL_CW for clockwise winding or GL_CCW for counterclockwise winding. A counterclockwise winding (GL_CCW) is configured by default.

10.1.2. How to Use

Culling is used in the same way it is used in OpenGL

Enabling or Disabling Culling

To enable or disable culling, call glEnable or glDisable, respectively, and specify GL_CULL_FACE for cap. Call the glIsEnabled() function, and specify GL_CULL_FACE for cap to get the current setting. Culling is disabled by default.

Specifying the Culled Face

Use the glCullFace() function to specify which face is not rasterized (the culled face).

Code 10-2. Definition of the glCullFace Function
void glCullFace(GLenum mode); 

You can choose from the following mode values to specify the culled face.

Table 10-1. Specifying the Culled Face

Setting Value

Culled Face

GL_FRONT

Front

GL_BACK (default)

Back

GL_FRONT_AND_BACK

Both

10.2. Clipping

Clipping is a feature that clips (removes) primitives from the regions (clip volumes) at which the view volume intersects the half-spaces defined by the specified clipping planes. 3DS clipping features correspond to the clipping features in OpenGL ES 1.1, but they are all controlled using reserved uniforms. The clipping implementation also generates new vertices and triangles for each clipped triangle, splitting it into multiple triangles.

Because GPU vertex processing converts coordinates using 24-bit floating-point numbers, clipping may not be performed correctly at the far clipping plane, if the ratio of the near clipping plane to the far clipping plane is large. Either configure the clip volume to keep the ratio of the near clipping plane to the far clipping plane small, or avoid placing polygons near the far clipping plane, whenever possible.

10.2.1. Reserved Uniform

The following reserved uniforms are used for clipping.

Enabling or Disabling Clipping

To enable clipping, set the reserved uniform dmp_FragOperation.enableClippingPlane to GL_TRUE, using the glUniform1i() function. This setting is disabled (set to GL_FALSE) by default.

Clipping Plane

To specify the clipping plane, set four coefficients in the reserved uniform dmp_FragOperation.clippingPlane using the glUniform4f() function. If p1, p2, p3, and p4 are the four coefficients, the clip volume is a collection of points that satisfy the following equation.

Wc Zc Yc Xc p1 p2 p3 p4 ≥ 0

Because these coefficients must be defined in the clip coordinate system, you must specify values that have been through a modelview transformation and perspective projection for the clipping plane used by the OpenGL ES standard. Unlike the OpenGL ES specifications, the 3DS system clips z-coordinates between 0 and -Wc. OpenGL ES-compatible matrices cannot be used unchanged for perspective projections. All coefficients are set to 0.0 by default.

Note:

For cautions related to using OpenGL ES-compatible matrices with projection transformations, see 8.4. Notes for the Clip Coordinate System.

Table 10-2. Reserved Uniforms Used for Clipping

Reserved Uniform

Type

Value to Set

dmp_FragOperation.enableClippingPlane

bool

Enables or disables clipping.

GL_TRUE or GL_FALSE (default).

dmp_FragOperation.clippingPlane

vec4

Specifies the four coefficients for the clipping plane.

This is (0.0, 0.0, 0.0, 0.0) by default.

10.3. Transformation to Window Coordinates

A polygon model's vertices are transformed from object coordinates into window coordinates through the following four steps.

  • Modelview transformation:
    Transforms object coordinates into eye coordinates.
  • Projection transformation:
    Transforms eye coordinates into clip coordinates. Note that the converted values (Zc) are between -Wc and 0.
  • Perspective division:
    Transforms clip coordinates into normalized device coordinates using w values.
  • Viewport transformation:
    Transforms normalized device coordinates into window coordinates following the viewport settings.

The vertex shader (and geometry shader) must output vertex attributes in clip coordinates. As a result, the vertex shader generally performs modelview and projection transformations internally. Perspective division and viewport transformation determine the position of each generated triangle in the display region. These triangles are converted into fragments during rasterization, and are then used by processes such as fragment lighting.

The following figure illustrates the process of transforming coordinates.

Figure 10-2. Process of Transforming Vertices From Object Coordinates Into Window Coordinates

Model-View Transformation Object Coordinates VertexShader/GeometryShader Projection Transformation Eye Coordinates Clip Coordinates Perspective Division Normalized Device Coordinates Viewport Transformation Window Coordinates Triangle Setup

10.3.1. Configuring the Viewport

Normalized device coordinates are transformed into window coordinates by the following equation.

n f Zd py Yd oy px Xd ox Zw Yw Xw

Xw, Yw, and Zw are window coordinates.
Xd, Yd, and Zd are normalized device coordinates.
px and py are the viewport's width and height.
ox and oy are the viewport's center point.
n and f are the depth values of the near and far planes, respectively, in clip space.

You can use the glDepthRangef() function to set the values applied to n and f in this equation.

Code 10-3. Definition of the glDepthRangef Function
void glDepthRangef(GLclampf zNear, GLclampf zFar); 

The values set for zNear and zFar are clamped between 0.0 and 1.0. By default, zNear is 0.0 and zFar is 1.0.

px, py, ox, and oy can all be calculated from the viewport settings. The glViewport() function configures the viewport.

Code 10-4. glViewport Function
void glViewport(GLint x, GLint y, GLsizei width, GLsizei height); 

For x and y specify the coordinates of the viewport's starting point (lower-left corner). A GL_INVALID_VALUE error is generated if a negative value is specified. If the specified value is not a multiple of 4, processing efficiency drops (to half for an even number and to one third for an odd number). In this case, extend the viewport so that it is a multiple of 4, adjust the perspective projection matrix so that it renders correctly on the extended viewport, and then apply the scissor test to avoid rendering unnecessary regions.

For width and height, specify the width and height of the viewport. A GL_INVALID_VALUE error is generated if a negative value is specified. The maximum width and height are both 1024.

The value of width is applied to px, height is applied to py, (x + width) / 2 is applied to ox, and (y + height) / 2 is applied to oy.

10.3.1.1. A Viewport Larger Than 1023×1016 Causes Incorrect Rendering

A hardware bug prevents images from being rendered properly if the following conditions are met when the glViewport() function sets the viewport.

  • When width is greater than 1023, entire polygons are not rendered if they contain pixels whose window x-coordinate is at least 1023 greater than the x parameter of the glViewport() function (taking the left side of the window to have an x-coordinate of 0).
  • When height is greater than 1016, the GPU stops responding if any polygon contains pixels whose window y-coordinate is at least 1016 greater than the y parameter of the glViewport() function (taking the bottom of the window to have a y-coordinate of 0).

To work around this hardware bug, avoid rendering pixels at the problematic coordinates by setting the viewport’s width and height to values that are not greater than 1023 and 1016, respectively.

When using the render-to-texture technique with a 1024×1024 texture, you must only render to a 1023×1016 region, and you must adjust texture coordinates so that only a 1023×1016 texture region is valid for use.

If you want to render to the entire 1024×1024 region, you must keep the viewport’s width from exceeding 1023 and its height from exceeding 1016, while changing the viewport’s offset to render the texture in sections. For example, the bug described in this section does not occur if you split rendering into the following four function calls: glViewport(0, 0, 512, 512), glViewport(512, 0, 512, 512), glViewport(0, 512, 512, 512), and glViewport(512, 512, 512, 512).

You cannot work around this bug by using the scissor test to prevent the pixels at the problematic coordinates from being rendered.

10.3.2. Polygon Offset

Polygon offset is a feature that adds an offset to depth values when polygons are rasterized and converted into fragments. This resolves the situation in which a lack of depth value resolution prevents the fragments' front-to-back order from being determined, as for overlapping coplanar polygons. Polygon offset occurs during window coordinates conversion.

Enabling or Disabling Polygon Offset

To enable or disable polygon offset, call glEnable or glDisable, respectively, and specify GL_POLYGON_OFFSET_FILL for cap. Call the glIsEnabled() function, and specify GL_POLYGON_OFFSET_FILL for cap to get the current setting. Polygon offset is disabled by default.

Specifying Offset Values

You can use the glPolygonOffset() function to specify the offset for depth values when polygon offset is enabled.

Code 10-5. Definition of the glPolygonOffset Function
void glPolygonOffset(GLfloat factor, GLfloat units); 

OpenGL uses the values of factor and units to determine the offset, but the 3DS system uses only units. The value of factor is set, but it is irrelevant to the offset.

The offset value is set with the product of units and the minimum (fixed) value at which a difference in the depth value appears in window coordinates. Because the z-value for vertex coordinates is implemented as a 24-bit floating-point number after vertex processing, a units value that is not a multiple of 128 has no effect when a polygon’s z-value is close to 1.0. To get a definite effect, set units to a multiple of 128.

Depth values are written to the depth buffer after an offset is added to them.

10.3.3. W-Buffer

The w-buffer is a feature that calculates depth values in window coordinates without using a perspective projection. You can control the w-buffer with the following reserved uniforms. Settings made by the glDepthRangef() function are disabled when the w-buffer is enabled.

Depth Value Scale

Depth values are calculated by the following equation, when the w-buffer is enabled.

Zw = -scalew × Zc

Zw is the depth value in window coordinates, Zc is the z-value in clip coordinates, and scalew is the scale value. This scale value is a floating-point number set by the reserved uniform dmp_FragOperation.wScale. As long as it is not 0.0, the w-buffer is enabled. Set the scale such that Zw remains in the range from 0.0 to 1.0.

Table 10-3. Reserved Uniforms Used for the W-Buffer

Reserved Uniform

Type

Setting Value

dmp_FragOperation.wScale

float

Specifies the depth value scale.

0.0 by default.

When Polygon Offset Is Enabled

If both the w-buffer and polygon offset features are enabled, the value obtained by multiplying Wc (the w value in clip coordinates) times units, as specified by the glPolygonOffset() function, is applied to as the polygon offset value.

10.4. Scissor Test

The scissor test is a feature that rejects fragments outside the specified range of window coordinates to reduce the number of fragments handled by further processing.

10.4.1. How to Use

Although its position in the pipeline is different, the process has the same specifications as the OpenGL scissor test.

Enabling or Disabling the Scissor Test

To enable or disable the scissor test, call glEnable or glDisable respectively, and specify GL_SCISSOR_TEST for cap. Call the glIsEnabled() function and specify GL_SCISSOR_TEST for cap to get the current setting. The scissor test is disabled by default. When it is disabled, fragments are not rejected.

Specifying the Scissor Box

Use the glScissor() function to specify the range (scissor box) through which fragments are allowed to pass.

Code 10-6. Definition of the glScissor Function
void glScissor(GLint x, GLint y, GLsizei width, GLsizei height); 

x and y are coordinates that specify the starting point (lower-left corner) of the scissor box in the window coordinates. width and height specify the width and height of the scissor box. A GL_INVALID_VALUE error is generated if width or height is 0. Because there are no default values, always specify the scissor box when the scissor test is enabled.

The scissor box includes the fragment at the starting coordinates, but not the fragment with an x-coordinate of (x + width) or a y-coordinate of (y + height).

10.5. Rasterization

Polygon rasterization (fragment generation) by the PICA graphics core adheres to the following rules.

  • The center coordinates of the pixel (x + 0.5, y + 0.5; x and y are integers) must lie inside the polygon.
  • According to the lower-left rule, a fragment is generated if the lower or left edge of a polygon passes through the center coordinate of a pixel, but no fragment is generated if the upper or right edge passes through the center coordinate.
    Note:
    This assumes that the x-axis is negative toward the left and the y-axis is negative toward the lower edge.

Figure 10-2 shows how the lower-left rule is applied based on the rasterization results of two polygons. The two polygons are the one defined by the three vertices (5.5, 0.5), (5.5, 5.5), and (0.5, 5.5), and the one defined by the three vertices (5.5, 0.5), (0.5, 0.5), and (0.5, 5.5). The former is shown in red, while the latter is shown in blue.

Figure 10-3. Example Demonstrating Rasterization Rules (the Lower-Left Rule)

x y 0 1 2 3 4 5

The rasterization rules are applied as follows.

On the boundary of the two polygons—because the left edge of the red polygon passes through the center of the pixels, these pixels are colored red.
Because the blue polygon passes through the center of pixels with a center coordinate x value of 0.5, these pixels are colored blue.
Because the right side of the red polygon passes through the center of the pixels with a center coordinate x value of 5.5, these pixels are not colored.
Because the bottom of the blue polygon passes through the center of the pixels with a center coordinate y value of 0.5, these pixels are colored blue.
And, because the top of the red polygon passes through the center of the pixels with a center coordinate y value of 5.5, these pixels are not colored.


CONFIDENTIAL