How WebGL Works

The most important thing to know about WebGL is that it is a state machine. The vast majority of the commands available will modify either the entire state of the rendering or the internal state of an object provided through WebGL. This knowledge is key to minimizing the number of calls made by the application.

To illustrate this point a simple scene can be profiled. One of the easiest examples, a “hello world” of sorts for WebGL, is the spinning cube rendered with a texture (see Listing 9-3). Although not a solid, real-world example, the cube showcases how calls can be reduced, owing to state being retained.

Listing 9-3. Rendering a Cube

function drawScene() {
  // Clear the buffers.
  gl.clearColor(0.0, 0.0, 0.0, 1.0);
  gl.clearDepth(1.0);
  gl.clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT);
   
  // Set the depth test
  gl.enable(gl.DEPTH_TEST);
  gl.depthFunc(gl.LEQUAL);
 
  // Create the perspective matrix
  perspectiveMatrix = makePerspective(45, 640.0/480.0, 0.1, 100.0);
   
  // Create the model view matrix
  loadIdentity();
  mvTranslate([-0.0, 0.0, -6.0]);
  mvPushMatrix();
  mvRotate(cubeRotation, [1, 0, 1]);
 
  // Set the program and its uniforms
  gl.useProgram(shaderProgram);
 
  var pUniform = gl.getUniformLocation(shaderProgram, "uPMatrix");
  gl.uniformMatrix4fv(pUniform, false, new Float32Array(perspectiveMatrix.flatten()));
 
  var mvUniform = gl.getUniformLocation(shaderProgram, "uMVMatrix");
  gl.uniformMatrix4fv(mvUniform, false, new Float32Array(mvMatrix.flatten()));
 
  gl.uniform1i(gl.getUniformLocation(shaderProgram, "uSampler"), 0);
 
  // Specify the texture.
  gl.activeTexture(gl.TEXTURE0);
  gl.bindTexture(gl.TEXTURE_2D, cubeTexture);
  gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.LINEAR);
  gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.LINEAR_MIPMAP_NEAREST);
 
  // Set the position for the vertices
  gl.bindBuffer(gl.ARRAY_BUFFER, cubeVerticesBuffer);
  gl.enableVertexAttribArray(gl.getAttribLocation(shaderProgram, "aVertexPosition"));
  gl.vertexAttribPointer(vertexPositionAttribute, 3, gl.FLOAT, false, 0, 0);
   
  // Set the texture coordinates attribute for the vertices
  gl.bindBuffer(gl.ARRAY_BUFFER, cubeVerticesTextureCoordBuffer);
  gl.enableVertexAttribArray(gl.getAttribLocation(shaderProgram, "aTextureCoord"));
  gl.vertexAttribPointer(textureCoordAttribute, 2, gl.FLOAT, false, 0, 0);
     
  // Set the index buffer
  gl.bindBuffer(gl.ELEMENT_ARRAY_BUFFER, cubeVerticesIndexBuffer);
 
  // Draw the elements
  gl.drawElements(gl.TRIANGLES, 36, gl.UNSIGNED_SHORT, 0);
   
  // Restore the original matrix
  mvPopMatrix();
   
  // Update the rotation of the cube
  updateRotation();
}

The naive implementation goes through the following steps:

1. Sets the values for clearing the screen’s frame buffer
2. Clears the screen’s frame buffer
3. Enables the depth test
4. Binds the shader program
5. Sets the uniforms for the matrices
6. Sets the texture unit for the sampler
7. Binds the texture to the specified unit
8. Sets the texture properties
9. Binds the vertex buffers
10. Sets up each vertex’s attributes
11. Binds the index buffer
12. Issues a draw call

This results in 25 calls to WebGL. In reality only three of these calls are needed with each frame: the clearing, setting the model-view-projection matrix (MVP matrix) uniform as the cube’s rotation is animating, and the draw call. The rest of the calls can be done once, during initialization. This is because the state of those calls is retained, so making them again with each frame results in redundant operations.

All calls contained in Table 9-1 will modify the state of an object or the rendering context as a whole.

WebGL also allows the state of itself or its objects to be queried. These calls should be avoided, as they can be very costly, and, at their worst, they can cause the pipeline to stall completely.

The only real exception to this rule, with regard to global state, is during the startup process, as the application can use this time to determine what features are available with the implementation, such as the number of texture units present.

With shader programs it’s perfectly valid to query the uniform information after creation; however, this should only happen once, as the value returned will be unchanged. The information for setting the value later can easily be stored by the application for future use.

During development it’s highly recommended that you check for any errors, using getError after every call. Yet, this should not go into production. The Khronos Group, the consortium that oversees the WebGL specification, provides a debug context for WebGL applications using JavaScript, and doing the same sort of wrapper is trivial in Dart through noSuchMethod invocations.

Any calls enumerated in Table 9-2 will query the state of WebGL.

Rather than relying on WebGL to enumerate the current state, the application itself should keep track of its status by mirroring WebGL’s own state machine. By so doing, redundant calls can be prevented.

Building the Renderer

When building a large application, it’s best to partition different subsystems into layers that handle specific functionality. This separates concerns and offers a singular component that can be tested and verified. Building the graphics subsystem involves wrapping the WebGL API into something friendlier to work with higher up within the program. This is also the level at which the API usage can be optimized.

Although the creation of a renderer that binds the whole of WebGL is a topic worthy of an entire tome, a smaller piece of the puzzle can be examined to learn how to go about building the larger whole. How the renderer deals with textures internally can provide that insight.

At a very high level, WebGL allows

• creation and deletion of objects
• manipulation of the underlying objects
• setting of the rendering pipeline

The code in Listing 9-4 begins to sketch out an implementation of this layer.

Listing 9-4. Graphics Library

function GraphicsDevice(gl) {
  this.gl_ = gl;
  this.context_ = new GraphicsContext(gl);
}
 
GraphicsDevice.prototype.createTexture2D = function() {
  var binding = this.gl_.createTexture(),
      texture = new Texture2D(this);
  texture.setBinding(binding);
 
  // The texture is not complete, and therefore not usable until
  // after the parameters have been set
  this.context_.initializeTexture(texture);
  return texture;
}
 
GraphicsDevice.prototype.deleteTexture = function(texture) {
  var binding = texture.getBinding();
 
  this.gl_.deleteTexture(binding);
  texture.setBinding(null);
}
//------------------------------------------------------------
function GraphicsResource(device) {
  this.device_ = device;
  this.binding_ = null;
}
 
GraphicsResource.prototype.getBinding = function() {
  return this.binding_;
}
 
GraphicsResource.prototype.setBinding = function(binding) {
  this.binding_ = binding;
}
//------------------------------------------------------------
Texture2D.prototype = new GraphicsResource();
Texture2D.constructor = Texture2D;
 
function Texture2D(device) {
  GraphicsResource.call(this, device);
}
 
Texture2D.prototype.getType = function() {
  // Corresponds to WebGLRenderingContext.TEXTURE_2D
  return 0x0DE1;
}

The layer is built to reflect those concerns. A GraphicsDevice class handles the creation and deletion of resources. Those resources are descended from the GraphicsResource class, which serves as a mechanism for binding the underlying objects WebGL provides. Resources such as textures and buffers would descend from the resource class and communicate an interface that allows the object’s internal state to be modified. The GraphicsContext class handles the state of the pipeline, containing a snapshot of its current condition, which is used to prevent redundant calls from being made.

The OpenGL is a C API, so it doesn’t have the clear separation of concerns that DirectX offers (see Listing 9-5). This means that the GraphicsContext class and GraphicsDevice class are more intertwined than is desirable, as initializing a texture requires calling to the context.

Listing 9-5. Texture Binding

function GraphicsContext(gl) {
  this.gl_ = gl;
 
  this.activeTexture_ = 0;
  this.boundTextures_ = new Array(8);
}
 
GraphicsContext.prototype.setTextureAt = function(index, texture) {
  // See if the texture is already bound
  if (this.boundTextures_[index] !== texture) {
    var gl = this.gl_;
 
    // See if the active texture unit is at the given index
    if (this.activeTexture_ !== index) {
      gl.activeTexture(gl.TEXTURE0 + index);
 
      this.activeTexture_ = index;
    }
 
    // Bind the texture
    gl.bindTexture(texture.getType(), texture.getBinding());
 
    this.boundTextures_[index] = texture;
  }
}
 
GraphicsContext.prototype.initializeTexture = function(texture) {
  // Bind the texture to the pipeline
  this.setTextureAt(0, texture);
 
  var gl = this.gl_,
      type = texture.getType();
 
  // Set the default sampler data
  gl.texParameteri(type, gl.TEXTURE_MAG_FILTER, gl.LINEAR);
  gl.texParameteri(type, gl.TEXTURE_MIN_FILTER, gl.TEXTURE_LINEAR_MIPMAP_NEAREST);
}

The GraphicsContext retains information that is contained within the underlying state machine. WebGL allows multiple textures to be bound to the pipeline at the same time, with each one residing in a separate texture unit. This information can be kept track of by holding an array of textures with the value held corresponding to the specific texture unit. The way to specify that texture unit is to make a call to the API, activeTexture, which specifies the index. This information also has to be retained.

When setting a texture to a given unit, the first thing to check is if that texture is already bound to the given location. This is done first, as the current texture unit only needs to be changed if the texture will actually be replaced. The check of the texture unit is performed when the texture is due to be replaced, because the texture unit may have been modified. During the process, whenever the internal WebGL state is modified, that same modification is applied to the context. This ensures that the underlying states are kept in lockstep with each other.

When building up the renderer further, it’s just a matter of extrapolating the same concepts to encompass other objects: only a single program can be bound to the pipeline at a time, the layout of the vertex buffer can be retained, and a uniform’s value should be changed only when a different input has been provided. All this, and more, needs to be reflected within the rendering context. Once these issues are handled, the interaction with WebGL is optimal at this level.

Debugging WebGL Usage

To verify that the API is being used properly, there are multiple techniques that can be implemented. Those interested in unit testing the graphics layer can do a mock of the WebGLRenderingContext and record the calls being sent. If a call passes through that shouldn’t have occurred, then the test should fail. Which framework to use is really dependent on developer choice, as there are a ton of frameworks available in JavaScript. The Closure Library provides a unit-testing and mocking framework, whereas QUnit is a popular unit-testing framework that can be coupled with Sinon.js for mocks. With Dart the choice is easier; it comes with a robust unit-testing library that also handles mocks. Writing unit tests is outside the scope of this chapter but is a great way to verify that no redundant calls are being made in an automated fashion.

What is more useful during the actual application development is to view the calls being made during a frame, by taking snapshots of what is actually happening. The most robust way to accomplish this is through the WebGL Inspector, a Chrome extension that allows the entire WebGL context to be viewed (see Figure 9-2). When WebGL content is available, a GL icon appears in the address bar, which is used to toggle the extension on and off.

The extension allows the entire state of a single frame to be examined. Each call made to the API is recorded and can be played back in the window, making it simple to see what the rendering looks like step by step. The extension can also show what redundant calls were made to the API, which can be used to diagnose how the renderer is doing. Another feature of the extension is its ability to observe the contents of various buffers within the context. This includes textures, vertex buffers, and index buffers. Shader programs, including the source code that makes up the vertex and fragment stages, can also bestudied. Finally, the state of the API, such as what is enabled or disabled on the pipeline, can be seen within the user interface (UI).

There is another option that just relies on Chrome itself, the Canvas Profiler, which introspects not only WebGL usage, but Canvas 2D Context API usage as well. At the time of writing, the profiler is not as robust as as WebGL Inspector, but it is likely to gain the same sort of functionality as its development continues. To signal that the tool is still under development, it is currently hidden behind the Experiments tab in Chrome DevTools and has to be enabled before it will appear under the Profiles section of the tools. If the Experiments tab is not present within the DevTools settings, then it needs to be enabled through chrome://flags by turning on the Developer Tools Experiments option. Once deemed ready for mass consumption, it should be available without any further configuration within the browser.

Once up and running, the profiler can capture calls to the API. The biggest piece of functionality available in Canvas Profiler that is not present in WebGL Inspector is the ability to capture multiple frames and thus a block of time. In terms of the UI, the profiler groups calls into batches, based on when the actual draw calls occur. This allows the individual geometries being drawn to be tracked in a simple manner.

To see how redundant calls can negatively affect the performance of an application, sample code is included that makes varying degrees of redundant calls. The number of objects being drawn can be modified, and these tools can be used to determine how the application is making use of WebGL. From there on, other three-dimensional content can be examined to see how different applications use the API.

Using Extensions

As graphics APIs evolve, they open up additional parts of the pipeline and offer new ways to perform more work with fewer calls. In the OpenGL world this functionality is exposed through extensions. OpenGL extensions allow developers to try new functionality and provide a path for promotion into the core specification. WebGL follows this same model and contains a number of extensions that can reduce the number of calls to the API.

It is important to note that there are no guarantees that an extension will be available, so relying on their presence can erode the number of potential users for the application. Unless the extension is widely supported, a fallback will be required for those users. A site such as WebGL Stats (http://webglstats.com), can give a good indication of what WebGL implementations in the wild have available. On the plus side all the extensions that will be examined here are present in the WebGL 2.0 specification.

// Create the instance data
gl.bindBuffer(gl.ARRAY_BUFFER, colorBuffer);
gl.enableVertexAttribArray(colorLocation);
gl.vertexAttribPointer(colorLocation, 4, gl.FLOAT, false, 16, 0);
ext.vertexAttribDivisorANGLE(colorLocation, 1);
 
// Draw the instanced meshes
ext.drawElementsInstancedANGLE(gl.TRIANGLES, indexCount, gl.UNSIGNED_SHORT, 0, instanceCount);

Rendering the Scene

With the renderer ensuring that no redundant calls are occurring, it’s time to focus on how the scene is being submitted. Even with the underlying optimizations present, the API usage may not be optimal when rendering the scene. For example, imagine that 100 cubes are visible and that each one can be one of three colors. Let’s also assume that there’s a uniform for the diffuse color and another uniform for the MVP matrix. The worst case scenario for the renderer diffuse value is that, modified for each cube, there are 200 uniform calls. This is almost 100 more than the best case scenario, in which the cubes are rendered by color, resulting in 3 calls to set the diffuse color and 100 calls setting the matrix. So, even though the underlying renderer is smart enough to prevent redundant calls, there can still be bottlenecks because of how the renderer is being fed the scene.

When optimizing, it’s important to think about the how the application works and to choose algorithms that fit the specific use. Generalization is usually in opposition to execution speed, so really think about how the scene should be rendered, and adjust accordingly. Additionally, profiling should be added from the start, as there is no way to prove that one algorithm is more effective than another without having any metrics to back that up. Also, remember that some optimizations may have a bigger effect on different hardware, so make sure to profile on a good sampling of devices that may run the program.

With graphics programming there are multitudes of published algorithms available to help speed the rendering. For this reason, it’s impossible to cover them all. However, there are a few optimization techniques that should be present in a software engineer’s toolbox.

/// Could be sped up by batching a number of spheres and removing branches.
Frustum.prototype.containsSphere = function(sphere) {
  var plane, dist, radius = sphere.getRadius();
 
  for (var i = 0; i < 6; ++i) {
    plane = this.planes_[i];
 
    dist = Vector3.dot(sphere.getCenter(), plane.getNormal()) + plane.getDistance();
 
    if (dist < -radius)
      return Frustum.Out;
    else if (Math.abs(dist) < radius)
      return Frustum.Intersect;
  }
 
  return Frustum.In;
}