Texture mapping is the process of adding detail to the surface of a 3D object. This can be likened to gift wrapping, where your wrapping paper is a 2D texture. Texture mapping is fundamental to modern rendering and is used for a multitude of interesting graphics techniques.
Generally, you’ll want more detail in your rendered 3D objects than the solid color produced by the HelloShader
and HelloStructs
effects from the last chapter. As you’ve learned, 3D models consist of vertices, typically organized into triangles. Those vertices are described, at a minimum, by a position. But they can contain additional information. A step toward additional surface detail is to provide a color for each vertex. Consider the 3D cubes in Figure 5.1.
On the cube to the left, each vertex is supplied a different color; on the right, each vertex has the same color. Clearly, the cube on the left has more detail. The colors add interest to the object and help define the faces of the cube. Furthermore, the colors of the individual pixels vary with respect to their positions relative to the vertices (recall the discussion of interpolation during the rasterizer stage from Chapter 1, “Introducing DirectX”). You gain more control over the color of a surface by increasing the number of colored vertices that make up your 3D objects. However, when viewed close up, you’ll never have enough vertices to produce a high-quality rendering, and any such attempt would quickly yield an overwhelming number of vertices. The solution is texture mapping.
To map a texture to a triangle, you include two-dimensional coordinates with each vertex. You use those coordinates to look up a color stored in a 2D texture. And you do this lookup in the pixel shader to find the color for each pixel of the triangle. The texture coordinates, then, are interpolated from the triangle vertices instead of the vertex color.
DirectX texture coordinates range from 0 to 1 (inclusive) on both the horizontal and vertical axes, where the origin is the top-left corner. These axes are commonly given the names u (horizontal) and v (vertical). Figure 5.2 shows a texture mapped to a quad (two triangles) and highlights the texture coordinates for each vertex.
Note
Direct3D supports 1D, 2D, and 3D textures along with texture arrays and texture cubes (discussed further in Chapter 8, “Gleaming the Cube”). The number of coordinates required for lookup matches the dimensionality of the texture.
Listing 5.1 presents the code for a texture mapping effect. As before, create a new effect/material in NVIDIA FX Composer and transpose the code from the listing. Then you can examine this code step by step.
/************* Resources *************/
#define FLIP_TEXTURE_Y 1
cbuffer CBufferPerObject
{
float4x4 WorldViewProjection : WORLDVIEWPROJECTION < string
UIWidget="None"; >;
}
RasterizerState DisableCulling
{
CullMode = NONE;
};
Texture2D ColorTexture <
string ResourceName = "default_color.dds";
string UIName = "Color Texture";
string ResourceType = "2D";
>;
SamplerState ColorSampler
{
Filter = MIN_MAG_MIP_LINEAR;
AddressU = WRAP;
AddressV = WRAP;
};
/************* Data Structures *************/
struct VS_INPUT
{
float4 ObjectPosition : POSITION;
float2 TextureCoordinate : TEXCOORD;
};
struct VS_OUTPUT
{
float4 Position : SV_Position;
float2 TextureCoordinate : TEXCOORD;
};
/************* Utility Functions *************/
float2 get_corrected_texture_coordinate(float2 textureCoordinate)
{
#if FLIP_TEXTURE_Y
return float2(textureCoordinate.x, 1.0 - textureCoordinate.y);
#else
return textureCoordinate;
#endif
}
/************* Vertex Shader *************/
VS_OUTPUT vertex_shader(VS_INPUT IN)
{
VS_OUTPUT OUT = (VS_OUTPUT)0;
OUT.Position = mul(IN.ObjectPosition, WorldViewProjection);
OUT.TextureCoordinate = get_corrected_texture_coordinate(IN.
TextureCoordinate);
return OUT;
}
/************* Pixel Shader *************/
float4 pixel_shader(VS_OUTPUT IN) : SV_Target
{
return ColorTexture.Sample(ColorSampler, IN.TextureCoordinate);
}
/************* Techniques *************/
technique10 main10
{
pass p0
{
SetVertexShader(CompileShader(vs_4_0, vertex_shader()));
SetGeometryShader(NULL);
SetPixelShader(CompileShader(ps_4_0, pixel_shader()));
SetRasterizerState(DisableCulling);
}
}
A number of language constructs come into play from this effect. First, notice the comments organizing the effect. HLSL supports C++-style single-line comments (// comments
) and multiline comments (/* comments */
).
Next, recognize the #define FLIP_TEXTURE_Y 1
macro at the top of the effect. This has behavior identical to the C/C++ #define
directive. Indeed, HLSL has several familiar preprocessor commands, including #if
, #else
, #endif
, and #include
.
Now examine the WorldViewProjection
shader constant, declared within the cbuffer
. This constant has the same behavior as in HelloShaders
and HelloStructs
, but you’ve added an annotation to the end of the declaration. Annotations are notes to the CPU-side application and are enclosed in angled brackets. These notes do not affect shader execution, but the application can use them. For example, the UIWidget
annotation, attached to WorldViewProjection
, controls how NVIDIA FX Composer treats that shader constant. With a value of None,
NVIDIA FX Composer hides the shader constant from the list of visible material properties. Figure 5.3 shows the NVIDIA FX Composer Properties panel both without the annotation (left) and with the annotation to hide the constant (right). Note that this only hides the value from the Properties panel; the CPU still updates the constant. Hiding the constant makes sense for the WorldViewProjection
matrix because you won’t be hand-editing that matrix.
Three steps are involved in using a texture within an HLSL effect. First, you must declare the texture object (see Listing 5.2). An HLSL texture declaration can use the explicit subtype (such as Texture2D
) or the more generic texture
data type. Texture objects cannot be declared within cbuffers
.
Texture2D ColorTexture <
string ResourceName = "default_color.dds";
string UIName = "Color Texture";
string ResourceType = "2D";
>;
Three annotations are associated with the ColorTexture
variable in Listing 5.2. As with all annotations, these are optional, but in the context of NVIDIA FX Composer, these annotations offer enhanced usability.
The UIName
annotation enables you to customize the displayed variable name within the Properties panel. ResourceType
identifies the type of texture that can be assigned, and ResourceName
allows a default texture to be used when the user has not supplied one.
Next, you must declare and initialize a texture sampler (see Listing 5.3). Samplers control how a color is retrieved from a texture. Direct3D 10 introduced the SamplerState
data type, and it directly maps to a corresponding Direct3D C struct with members for filtering and texture address modes. We discuss these topics shortly.
SamplerState ColorSampler
{
Filter = MIN_MAG_MIP_LINEAR;
AddressU = WRAP;
AddressV = WRAP;
};
The final step is to sample the texture using the declared sampler object. This step is performed in the pixel shader (see Listing 5.4).
float4 pixel_shader(VS_OUTPUT IN) : SV_Target
{
return ColorTexture.Sample(ColorSampler, IN.TextureCoordinate);
}
Note the C++-style method invocation of the Sample()
method against the ColorTexture
object. The first argument to the Sample()
method is the sampler object, followed by the coordinates to look up in the texture.
The vertex stream supplies the coordinates for sampling the texture, with corresponding members in the VS_INPUT
and VS_OUTPUT
data structures (see Listing 5.5). Observe the TEXCOORD
semantics attached to the 2D TextureCoordinate
members.
struct VS_INPUT
{
float4 ObjectPosition : POSITION;
float2 TextureCoordinate : TEXCOORD;
};
struct VS_OUTPUT
{
float4 Position : SV_Position;
float2 TextureCoordinate : TEXCOORD;
};
The vertex shader (see Listing 5.6) passes the input texture coordinates to the output of the stage, but it does so after invoking get_corrected_texture_coordinate()
. HLSL supports user-defined, C-style helper functions, and this one simply inverts the vertical texture coordinate if FLIP_TEXTURE_Y
is nonzero. This is necessary within NVIDIA FX Composer because it uses OpenGL-style texture coordinates for the built-in 3D models (the Sphere, Teapot, Torus, and Plane). The origin for OpenGL texture coordinates is the bottom-left corner instead of the top-left corner for DirectX. Therefore, when you’re visualizing your shaders in NVIDIA FX Composer against one of the built-in models, you need to flip the vertical texture coordinate. If you import a custom model into NVIDIA FX Composer and that model has DirectX-style texture coordinates, you can disable the FLIP_TEXTURE_Y
macro.
float2 get_corrected_texture_coordinate(float2 textureCoordinate)
{
#if FLIP_TEXTURE_Y
return float2(textureCoordinate.x, 1.0 - textureCoordinate.y);
#else
return textureCoordinate;
#endif
}
VS_OUTPUT vertex_shader(VS_INPUT IN)
{
VS_OUTPUT OUT = (VS_OUTPUT)0;
OUT.Position = mul(IN.ObjectPosition, WorldViewProjection);
OUT.TextureCoordinate = get_corrected_texture_coordinate(IN.
TextureCoordinate);
return OUT;
}
Figure 5.4 shows the output of the texture mapping effect applied to a sphere with a texture of Earth.
In our discussion of texture coordinates and sampling, you might have realized that you will rarely view a texture-mapped object at a one-to-one correspondence between the elements in the texture (texels) and the pixels rendered to the screen. Your camera will be at an arbitrary distance from the textured object and will view it at an arbitrary angle. Consider a scenario in which the camera is “zoomed in” to a textured object. The rasterizer stage determines which pixels to send to the pixel shader and interpolates their texture coordinates. But the coordinates represent locations that are in between authored texels in the texture. In other words, you’re trying to render the texture at a higher resolution than it was authored. The topic of texture filtering determines what color should be chosen for these pixels because they have no direct lookup in the texture.
The scenario just described is known as magnification—in this situation, there are more pixels to render than there are texels. Direct3D supports three types of filtering to determine the colors of these “in between” pixels: point filtering, linear interpolation, and anisotropic filtering.
Point filtering is the fastest of the filtering options, but it generally yields the lowest quality results. Also known as nearest-neighbor filtering, point filtering simply uses the color of the texel closest to the pixel center. Shown on the left side of Figure 5.5, you can see the blocky-looking image produced with point filtering.
A higher-quality filtering option is linear interpolation, in which the color is interpolated between neighboring texels. For 2D textures, this is more correctly named bilinear interpolation because the interpolation takes place both horizontally and vertically. In this filtering technique, the four texels surrounding the pixel center are identified and two 1D linear interpolations are performed for the pixels along the u-axis. Then a third interpolation is performed along the v-axis to produce the final color. The image on the right side of Figure 5.5 shows the same magnified texture using bilinear interpolation. Notice the smoother appearance of this image compared to point filtering.
Anistropy refers to the distortion of a texture projected onto a surface that is at an oblique angle to the camera. Anisotropic filtering reduces this distortion and improves the rendered output. This is the most expensive filtering option, but the results are compelling. Figure 5.6 compares a texture mapped object with and without anisotropic filtering.
Minification is the opposite of magnification and applies when a pixel covers more than one texel. This happens as the camera gets farther from a textured surface and Direct3D must choose the appropriate color. The same filtering options apply, but it’s a bit more complicated for minification. With linear interpolation, for example, the four texels surrounding the pixel center are identified just as they were for magnification. However, the in-between texels (the multiple colors occupying the same lookup for the pixel center) are ignored and don’t contribute to the final color. This can produce artifacts that decrease rendering quality.
You might envision an approach in which the texels are averaged to produce a better result. However, such computation is based on the resolution of the down-sampled texture, and an arbitrary number of “levels” can be computed. Building these levels on the fly would be impractical. Instead, this approach can be precomputed through a technique known as mipmapping.
Mipmaps are smaller versions of the original texture, typically precomputed and stored within the same file. Each mip-level is a progressive division by 2, down to a 1×1 texture. Figure 5.7 shows an example of the Earth texture at a size of 512×256 with nine mip-levels.
When employing mipmaps, two steps are required to produce a final color. First, the mip-level (or levels) must be selected for minification or magnification. Second, the selected mip-level(s) is filtered to derive a color.
Point or linear filtering can be used to select a mip-level. Point filtering simply selects the nearest mip-level, and linear filtering selects the two surrounding mipmaps. Then the selected mip-level is sampled through point, linear, or anisotropic filtering. If linear filtering was used to select the two neighboring mipmaps, both mip-levels are sampled and their results are interpolated. This technique produces the highest-quality results.
Finally, although mipmapping improves rendering quality, note that it also increases memory requirements by 33 percent.
Refer to the SamplerState
object from TextureMapping.fx
(reproduced in Listing 5.7), and you’ll notice the Filter
member and its assigned value of MIN_MAG_MIP_LINEAR
. This configures the sampler to use linear interpolation for minification, magnification, and mip-level sampling. Direct3D allows independent configurations for each of these elements. Table 5.1 provides examples of the various permutations. You can find a full listing in the Direct3D documentation on the Microsoft Developer Network (MSDN) website.
SamplerState ColorSampler
{
Filter = MIN_MAG_MIP_LINEAR;
AddressU = WRAP;
AddressV = WRAP;
};
You likely noticed the AddressU = WRAP
and AddressV = WRAP
settings from the SamplerState
object (see Listing 5.7). Known as addressing modes, these enable you to control what happens when a texture coordinate is outside the range [0, 1]. Direct3D supports four addressing modes: Wrap, Mirror, Clamp, and Border.
In wrap texture address mode, your texture is repeated as vertex coordinates go below 0 or above 1. For example, mapping a texture to a quad with UVs of (0.0, 0.0), (3.0, 0.0), (0.0, 3.0), and (3.3, 3.3) results in repeating the texture three times along both axes. Figure 5.8 illustrates this scenario.
Note
The black border surrounding the tiled textures in Figure 5.8 is part of the actual texture and is not created as part of the wrapping process. This is included to help illustrate the integer boundaries where wrapping occurs.
Mirror address mode is similar to wrap mode, except that the texture is mirrored along integer boundaries instead of duplicated. Figure 5.9 shows the results of the same textured quad, but with mirror address mode enabled.
In clamp address mode, the texture is not tiled. It is applied once, and all coordinates are clamped to the range [0, 1]. This has the effect of smearing the color of the texture’s edges, as in Figure 5.10.
Note
The mapped texture in Figure 5.10 is different than the texture in Figures 5.8 and 5.9. The black border has been removed and the image has been cropped so that the edge pixels vary in color. If the original texture had been used, the black border would be the color for every edge pixel and would be the “clamped” value.
As in clamp mode, border address mode applies the texture just once. But instead of smearing the edge pixels, a border color is used for all coordinates outside the range [0, 1]. Figure 5.11 shows the original, black-bordered cow texture, in border address mode with a blue border color.
In this chapter, you learned the details of texture mapping. You explored a complete texture mapping effect and uncovered additional HLSL syntax. You also learned about minification, magnification, mipmapping, and the filtering processes used to sample textures. Finally, you dove into the details of texture addressing modes and discovered how to produce various effects for geometry with texture coordinates that extend outside the range 0 to 1.
Texture mapping is fundamental to modern graphics, and future chapters build on this foundation.
1. Modify the SamplerState
object in TextureMapping.fx
to experiment with various permutations of filter modes. Use point, linear, and anisotropic filtering, and observe the results as you zoom in and out with a textured surface.
2. Import a quad with UVs that range outside [0, 1], and experiment with the texture addressing modes in the SamplerState
object of TextureMapping.fx
. Specifically, use wrap, mirror, clamp, and border, and observe the results. The book’s companion website has an example quad (in .obj
format, a 3D model format that NVIDIA FX Composer supports).
18.191.147.77