For this recipe, we will build on top of the shadow mapping recipe, Implementing shadow mapping with FBO. The code for this recipe is contained in the Chapter4/VarianceShadowMapping folder.

Let us start our recipe by following these simple steps:

The variance shadowmap technique tries to represent the depth data such that it can be filtered linearly. Instead of storing the depth, it stores the depth and depth*depth value in a floating point texture, which is then filtered to reconstruct the first and second moments of the depth distribution. Using the moments, it estimates the variance in the filtering neighborhood. This helps in finding the probability of a fragment at a specific depth to be occluded using Chebyshev's inequality. For more mathematical details, we refer the reader to the See also section of this recipe.

From the implementation point of view, similar to the shadow mapping recipe, the method works in two passes. In the first pass, we render the scene from the point of view of light. Instead of storing the depth, we store the depth and the depth*depth values in a floating point texture using the custom fragment shader (see Chapter4/VarianceShadowmapping/shaders/firststep.frag).

The vertex shader outputs the clip space position to the fragment shader using which the fragment depth value is calculated. To reduce self-shadowing, a small bias is added to the z value.

vec3 pos = clipSpacePos.xyz/clipSpacePos.w;
pos.z += 0.001;
float depth = (pos.z +1)*0.5;
float moment1 = depth;
float moment2 = depth * depth;
vFragColor = vec4(moment1,moment2,0,0);

After the first pass, the shadowmap texture is blurred using a separable Gaussian smoothing filter. First the vertical and then the horizontal filter is applied to the shadowmap texture by applying the shadowmap texture to a full-screen quad and alternating the filter FBO's color attachment. Note that the shadowmap texture is bound to texture unit 0 whereas the textures used for filtering are bound to texture unit 1 (attached to GL_COLOR_ATTTACHMENT0 on the filtering FBO) and texture unit 2 (attached to GL_COLOR_ATTACHMENT1 on the filtering FBO).

glBindFramebuffer(GL_FRAMEBUFFER,fboID);
  glViewport(0,0,SHADOWMAP_WIDTH, SHADOWMAP_HEIGHT);
  glDrawBuffer(GL_COLOR_ATTACHMENT0);
  glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT);
  DrawSceneFirstPass(MV_L, P_L);
  
glBindFramebuffer(GL_FRAMEBUFFER,filterFBOID);
glDrawBuffer(GL_COLOR_ATTACHMENT0);
glBindVertexArray(quadVAOID);
  gaussianV_shader.Use();
    glDrawElements(GL_TRIANGLES, 6, GL_UNSIGNED_SHORT, 0);
    
    glDrawBuffer(GL_COLOR_ATTACHMENT1);
  gaussianH_shader.Use();
  glDrawElements(GL_TRIANGLES, 6, GL_UNSIGNED_SHORT, 0);
glBindFramebuffer(GL_FRAMEBUFFER,0);
glDrawBuffer(GL_BACK_LEFT);
glViewport(0,0,WIDTH, HEIGHT);

In the second pass, the scene is rendered from the point of view of the camera. The blurred shadowmap is used in the second pass as a texture to lookup the sample value (see Chapter4/VarianceShadowmapping/shaders/VarianceShadowMap.{vert, frag}). The variance shadow mapping vertex shader outputs the shadow texture coordinates, as in the shadow mapping recipe.

#version 330 core
layout(location=0) in vec3 vVertex;
layout(location=1) in vec3 vNormal;
uniform mat4 MVP;   //modelview projection matrix
uniform mat4 MV;    //modelview matrix
uniform mat4 M;      //model matrix
uniform mat3 N;     //normal matrix
uniform mat4 S;     //shadow matrix
smooth out vec3 vEyeSpaceNormal;
smooth out vec3 vEyeSpacePosition;
smooth out vec4 vShadowCoords;
void main()
{
  vEyeSpacePosition = (MV*vec4(vVertex,1)).xyz;
  vEyeSpaceNormal   = N*vNormal;
  vShadowCoords     = S*(M*vec4(vVertex,1));
  gl_Position       = MVP*vec4(vVertex,1);
}

The variance shadow mapping fragment shader operates differently. We first make sure that the shadow coordinates are in front of the light (to prevent back projection), that is, shadowCoord.w>1. Next, the shadowCoords.xyz values are divided by the homogeneous coordinate, shadowCoord.w, to get the depth value.

if(vShadowCoords.w>1) {
  vec3 uv = vShadowCoords.xyz/vShadowCoords.w;
  float depth = uv.z;

The texture coordinates after homogeneous division are used to lookup the shadow map storing the two moments. The two moments are used to estimate the variance. The variance is clamped and then the occlusion probability is estimated. The diffuse component is then modulated based on the obtained occlusion probability.

  vec4 moments = texture(shadowMap, uv.xy);
  float E_x2 = moments.y;
  float Ex_2 = moments.x*moments.x;
  float var = E_x2-Ex_2;
  var = max(var, 0.00002);
  float mD = depth-moments.x;
  float mD_2 = mD*mD;
  float p_max = var/(var+ mD_2);
  diffuse *= max(p_max, (depth<=moments.x)?1.0:0.2);
}

To recap, here is the complete variance shadow mapping fragment shader:

#version 330 core
layout(location=0) out vec4 vFragColor;
uniform sampler2D  shadowMap;
uniform vec3 light_position;  //light position in object space
uniform vec3 diffuse_color;
uniform mat4 MV;
smooth in vec3 vEyeSpaceNormal;
smooth in vec3 vEyeSpacePosition;
smooth in vec4 vShadowCoords;
const float k0 = 1.0;  //constant attenuation
const float k1 = 0.0;  //linear attenuation
const float k2 = 0.0;  //quadratic attenuation
void main() {   
  vec4 vEyeSpaceLightPosition = (MV*vec4(light_position,1));
  vec3 L = (vEyeSpaceLightPosition.xyz-vEyeSpacePosition);
  float d = length(L);
   L = normalize(L);
  float attenuationAmount = 1.0/(k0 + (k1*d) + (k2*d*d));
  float diffuse = max(0, dot(vEyeSpaceNormal, L)) * attenuationAmount;
  if(vShadowCoords.w>1) {
    vec3 uv = vShadowCoords.xyz/vShadowCoords.w;
    float depth = uv.z;
    vec4 moments = texture(shadowMap, uv.xy);
    float E_x2 = moments.y;
    float Ex_2 = moments.x*moments.x;
    float var = E_x2-Ex_2;
    var = max(var, 0.00002);
    float mD = depth-moments.x;
    float mD_2 = mD*mD;
    float p_max = var/(var+ mD_2);
    diffuse *= max(p_max, (depth<=moments.x)?1.0:0.2);
  }
  vFragColor = diffuse*vec4(diffuse_color, 1);
}

Variance shadow mapping is an interesting idea. However, it does suffer from light bleeding artefacts. There have been several improvements to the basic technique, such as summed area variance shadow maps, layered variance shadow maps, and more recently, sample distribution shadow maps, that are referred to in the See also section of this recipe. After getting a practical insight into the basic variance shadow mapping idea, we invite the reader to try and implement the different variants of this algorithm, as detailed in the references in the See also section.

The demo application for this recipe shows the same scene (a cube and a sphere on a plane) lit by a point light source. Right-clicking the mouse button rotates the point light around the objects. The output result is shown in the following figure:

Comparing this output to the previous shadow mapping recipes, we can see that the output quality is much better if compared to the conventional shadow mapping and the PCF-based technique. When comparing the outputs, variance shadow mapping gives a better output with a significantly less number of samples. Obtaining the same output using PCF or any other technique would require a very large neighborhood lookup with more samples. This makes this technique well-suited for real-time applications such as games.