CHAPTER 18

image

HTML5 Games in C++ with Emscripten

Chad Austin, Senior Technical Director, IMVU

If you had told me ten years ago that I’d someday compile real-time, 3D C++ games into JavaScript so I could run them in web browsers, I would have thought you were crazy. Since then, software has shifted from retail stores and optical discs to online app stores and web applications. Internet users have become increasingly security-conscious, and JavaScript engines have gotten faster by orders of magnitude.

The Web is the ultimate platform. It is secure: applications have no direct access to the local machine. It is seamless: users merely have to click a link to experience your game or web site. It is royalty-free: if you can host a web page, you can distribute a web app. It is capable: with the advent of HTML5 and WebGL, web applications have access to an increasing set of functionality, such as gamepads, full screen display, and local storage. And now, the web is fast too: with the advent of technologies like Mozilla’s asm.js and Google’s Portable Native Client, web applications can approach native performance.

Compiling code to JavaScript is not new. Google’s GWT compiles Java into JavaScript. Haxe is a game development language that targets many platforms, including JavaScript. JSIL compiles .NET programs into JavaScript. Mandreel makes it easy to port C, C++, and Objective C games to a variety of platforms, including Windows Phone and JavaScript.

More recently, an open source C++-to-JavaScript compiler called Emscripten has gained prominence and momentum. In the past, I’ve argued passionately that compiling C++ to JavaScript was a bad idea and that defining a portable bytecode was a better direction for the Web. However, the future is created by the people who invent it. Alon Zakai, with a vision in his head and persistence in his actions, created Emscripten as an open source project. Eventually Mozilla hired him and gave him funding to continue full-time development on Emscripten. Emscripten has since been proven viable and even Epic’s Unreal Engine 3 has been ported to HTML5 with it. While I would have loved to have seen Google’s Native Client technology gain adoption, I now believe that compiling C++ to JavaScript has strong survival genes: Emscripten-generated code runs on all recent browsers, including Internet Explorer 10, and Native Client is a far more complicated piece of technology.

In this chapter, I will describe how Emscripten works, why it’s fast, what’s possible, and then I will demonstrate what it’s like to port an existing C++ game to the browser. Being new, Emscripten is rough around the edges, but I will cover the common problems you may run into and how to address them.

What is Emscripten?

Emscripten relies heavily on LLVM, a set of open source compiler tools. The LLVM project provides both Clang, a compiler from C++ to platform-independent LLVM instructions (known as LLVM IR), and a set of platform-independent optimization passes and tools. The LLVM project is hosted at http://llvm.org/.  With the help of LLVM, Emscripten translates C or C++ into JavaScript so it can run directly from web browsers. You can think of Emscripten as the sum of three components:

  • A compiler from LLVM IR into a subset of JavaScript.
  • A set of convenience tools that make it easy to use LLVM and Clang to compile C++ into JavaScript.
  • A standard set of libraries and APIs, like libc, libc++, SDL, OpenGL, and zlib, to ease porting efforts.

Before we dive any deeper, I can hear you exclaim “But isn’t JavaScript much slower than native code? Why would I want to compile my fast native code into slow JavaScript? How could my game’s performance possibly be acceptable?”

We will dig into how compiling C++ to JavaScript has acceptable performance in more detail later, but let’s look at some numbers first. In 2011, C++ compiled to JavaScript ran at less than 10% of the speed of the equivalent native code—a 10x slowdown or more, depending on the code. That’s pretty terrible, but times have changed.

Table 18-1 shows the results from a software skeletal animation benchmark I ran in 2011, comparing vertex transform rate between a scalar floating point native implementation and the equivalent Emscripten-compiled program.

Table 18-1. Native vs. Emscripten Performance in 2011

Vertices/sec

Slowdown

Native gcc 4.2

63355501

1

Emscripten

5184815

12.2

Since then, JavaScript engines have learned to recognize and optimize the particular style of code generated by Emscripten and other C++-to-JavaScript compilers. Table 18-2 is the same benchmark run today, this time having Emscripten generate the asm.js subset of JavaScript.

Table 18-2. Native vs. Emscripten Performance in 2014

Vertices/sec

Slowdown

Native gcc 4.2

61215975

1

Firefox 27 asm.js

32282000

1.90

Chrome 32 asm.js

24036000

2.55

The absolute numbers differ from 2011’s as the benchmark was run on a different machine, so focus on the relative slowdown. With Emscripten compiler and JavaScript engine improvements, C++ compiled to JavaScript can run at 40-50% of native speed, a huge improvement from the 2011 numbers.

Emscripten-generated JavaScript, run in a browser, will likely never match native code performance, as the Web is expected to be secure, and security sandboxes generally impose some overhead. However, it’s conceivable that, in time, C++ code compiled into JavaScript for the browser could run with a mere 5-15% overhead relative to native, given that Google’s Native Client code performs within 5% of native. There is a fair amount of room for browser optimization technology to improve. You can track cross-browser asm.js benchmarks at http://arewefastyet.com/.

How Emscripten Works

Under the hood, Emscripten compiles C++ into JavaScript in two main phases.  First, Emscripten invokes LLVM’s Clang to compile your C and C++ source code into LLVM IR, which is a representation of a program somewhere between source code and compiled object code. Afterwards, Emscripten translates the LLVM IR into optimized JavaScript (see Figure 18-1).

9781430266976_Fig18-01.jpg

Figure 18-1. The Emscripten compiler workflow

Let’s walk through the steps one by one.

Clang

Clang is an open source C and C++ compiler. It translates your source code into LLVM IR, which is basically a typed, hardware-independent assembly language (see Figure 18-2). LLVM IR defines functions, and its typed local variables roughly correspond to CPU registers. For example, consider a C function named lerp to linearly interpolate two floating point numbers, as shown in Listing 18-1.

9781430266976_Fig18-02.jpg

Figure 18-2. The Clang stage

Listing 18-1. lerp in C

float lerp(float a, float b, float t) {
    return (1 - t) * a + t * b;
}

The same lerp function would be represented in LLVM IR as shown in Listing 18-2.

Listing 18-2. lerp in LLVM IR

define internal hidden float @_lerp(float %a, float %b, float %t) nounwind readnone inlinehint ssp {
  %1 = fsub float 1.000000e+00, %t
  %2 = fmul float %1, %a
  %3 = fmul float %t, %b
  %4 = fadd float %2, %3
  ret float %4
}

The LLVM IR captures the semantic meaning of the C code, but gives it a consistent structure so that Emscripten can translate it into JavaScript.

Emscripten

After Clang converts C or C++ source code into LLVM IR, Emscripten takes over (see Figure 18-3). Emscripten translates the LLVM IR into JavaScript operations. It leverages the fact that the JavaScript language has operators and expressions that match C semantics for signed and unsigned 32-bit integer math.

9781430266976_Fig18-03.jpg

Figure 18-3. The Emscripten stage

The aforementioned lerp function would get compiled and optimized into the JavaScript shown in Listing 18-3.

Listing 18-3. lerp Translated to JavaScript

function _lerp(a, b, t) {
  return(1 - t) * a + t * b
}

As you can see, for functions that operate only on arguments, the original C and resulting JavaScript often look similar. However, the vast majority of C and C++ functions involve some kind of memory access. To illustrate how Emscripten implements pointers and memory access, let’s compare C and JavaScript implementations of strlen;. The standard C strlen function, when translated to JavaScript, dereferences pointers by indexing into the HEAP8 array (Listings 18-4 and 18-5).

Listing 18-4. strlen in C

size_t strlen(const char *str) {
    const char *s = str;
    while (*s) ++s;
    return s - str;
}

Listing 18-5. strlen in JavaScript

function _strlen(str) {
  for(var s = str;0 != (HEAP8[s] | 0);) {
    s = s + 1 | 0
  }
  return s - str | 0
}

Note that these examples are simple C functions to illustrate basic code generation concepts. As C++ can be considered syntax sugar on top of C semantics, all the same principles apply to compiled C++ code.

In Listing 18-5, you can see that char* pointer dereferences are converted to HEAP8 array access. The next section shows how memory access works in general.

Memory Representation

To understand JavaScript strlen, you must know how Emscripten represents memory. The Typed Array specification, used in WebGL, HTML5 Canvas, and XMLHttpRequest Level 2, introduces a mechanism by which JavaScript can read from and write to contiguous blocks of binary data. An ArrayBuffer stores a sequence of contiguous bytes which can be accessed and interpreted through ArrayBufferView objects. For example, the Int8Array object exposes the ArrayBuffer as if it were signed 8-bit integers and Float32Array exposes the same memory as if it were IEEE 32-bit floats.

Emscripten’s memory space is a single JavaScript ArrayBuffer accessed through one of each of ArrayBufferView type. For example, HEAP8, used in _strlen above, is an Int8Array through which signed 8-bit integer values are read from and written to the program’s memory. Pointers, such as the variables str and s above, are simply numeric indices into the heap. Emscripten has ArrayBufferViews for 32-bit and 64-bit floats, as well as the full range of 8-, 16-, and 32-bit integers, both signed and unsigned.

Arithmetic

In JavaScript, all numbers are 64-bit IEEE floats. Using floating point numbers to represent pointers into the heap would be silly and slow, so to inform JavaScript optimizers that these variables can be made into integers, Emscripten sprinkles | 0 throughout the function. The x | 0 expression coerces the number x into a signed integer, allowing JavaScript engines to optimize, say, c = c + 1 | 0, into a fast, native increment instruction.

Through clever type analysis, JavaScript engines can even notice when it’s possible to reduce 64-bit precision floating point operations to 32-bit precision floating point operations without changing the semantics of the code.

You are beginning to discover why Emscripten-generated JavaScript is fast. It uses a compact ArrayBuffer for memory, so the garbage collector doesn’t have to do anything. Variables are always numbers, not objects, which means there are no dynamic method calls or hash table lookups. Clever use of JavaScript’s |, >>, and >>> operators indicate the code has integer semantics. All of this means that just-in-time optimizers can translate JavaScript directly into fast machine code.

These high-performance JavaScript conventions have been codified in a standard called asm.js.

What is asm.js?

asm.js is a subset of JavaScript: that is, it adds no new semantics to the existing JavaScript language. All asm.js code has identical behavior in browsers that do not specifically support it, though it runs much faster in browsers that recognize and optimize asm.js constructs.

The asm.js subset of JavaScript is restricted enough that it can be treated as a low-level compile target. It only exposes operations that can be directly translated into native machine instructions. Traditional JavaScript engines optimize code dynamically during execution with so-called just-in-time (JIT) compilation. asm.js, on the other hand, can be recognized, compiled, and optimized ahead-of-time (AOT), resulting in consistent, predictable, high performance, which is especially important for games.

asm.js sparked controversy in the web and game development communities. Is JavaScript the right way to specify what is effectively a virtual machine bytecode? Does asm.js break the “View Source” nature of the open web? Can native code performance really be achieved with this approach?

However, to us game developers, those discussions are somewhat academic. asm.js is real and it works well in practice, so if you want your game on the secure, cross-browser, open platform of the Web, targeting asm.js is a great option. You can read more about asm.js at http://asmjs.org/.

The Emscripten Toolchain

Emscripten consists of three parts: the compiler from LLVM to JavaScript, a set of library implementations that make it convenient to port existing codebases, and helpful tools and scripts for managing the entire compile process.

One of those tools, emcc, behaves much like gcc and can be dropped into most existing build systems. (For C++, em++ is the Emscripten replacement of g++.) Rather than producing native object files, emcc and em++ produce LLVM IR files. When linking, instead of generating a native executable, the final executable target is either an HTML page or a single JavaScript file, depending on whether the program stands alone or will be integrated into an existing web application.

Thus, compiling a simple Hello World program with Emscripten is as simple as Listing 18-6.

Listing 18-6. Compiling Hello World with emcc

$ cat helloworld.c
#include <stdio.h>
int main() {
    printf(“Hello world! ”);
}
$ emcc -o helloworld.html helloworld.c

emcc and em++ offer dozens of Emscripten-specific options, including custom optimizations and output format adjustments. They are not in scope for this chapter, but it’s worth reading through emcc --help.

Graphics Support

C++ programs in Emscripten have access to the entire range of browser capabilities by making direct calls to JavaScript with embind, which we explore in the “Audio Support” section below. Most programs, however, would take advantage of Emscripten’s convenient access to WebGL for 3D graphics or HTML5 Canvas for 2D graphics.

WebGL

Emscripten exposes WebGL to programs through one of two OpenGL modes: OpenGL ES 2 and OpenGL Legacy Emulation. By default, your code only has access to the OpenGL ES 2 functions, which are translated almost directly to WebGL calls, as WebGL is based on the OpenGL ES 2 specification. However, if your code uses legacy fixed function calls, you can enable the LEGACY_GL_EMULATION option which attempts to translate those calls into WebGL calls with some degree of accuracy.

For very basic OpenGL applications, LEGACY_GL_EMULATION will likely come close to the original OpenGL semantics, but for anything more complicated than a few textures and vertex buffers, the LEGACY_GL_EMULATION mode is likely inadequate. At the time of this writing, there is talk of replacing LEGACY_GL_EMULATION with a maintained and supported OpenGL emulation layer such as Regal (https://github.com/p3/regal), but I recommend limiting your games to the OpenGL ES 2 subset and not using an OpenGL emulation layer. Plus, if your game targets OpenGL ES 2, it can run natively on mobile platforms too!

Canvas

For games that only use 2D graphics, WebGL is not necessary. 2D games can render with the HTML5 Canvas API either through SDL or embind to access the 2D canvas context directly. To see an example of how to use embind, see the audio example in Listing 18-7.

Audio Support

For audio, Emscripten provides implementations of both SDL_audio and OpenAL.  If you need more direct access to the browser’s JavaScript audio APIs, there are several ways to access JavaScript from C++. One of them is embind, a Boost.Python-like interface to JavaScript objects. embind is included with Emscripten, but requires the --bind option passed to emcc. To illustrate how embind can be used to access browser APIs, the program in Listing 18-7 plays a two-second tone by using embind to directly manipulate Web Audio API JavaScript objects.

Listing 18-7. Accessing the Web Audio API from C++ with embind

#include <emscripten/bind.h>
#include <emscripten/val.h>
#include <math.h>
 
using namespace emscripten;
 
const double PI = atan(1) * 4;
 
int main() {
    val AudioContext = val::global("AudioContext");
    if (!AudioContext.as<bool>()) {
        AudioContext = val::global("webkitAudioContext");
    }
    val context = AudioContext.new_();
 
    int duration = 2;
    int sampleRate = 44100;
 
    int numberOfFrames = duration * sampleRate;
 
    val buffer = context.call<val>("createBuffer", 1, numberOfFrames, sampleRate);
    val data = buffer.call<val>("getChannelData", 0);
    
    for (int i = 0; i < numberOfFrames; ++i) {
        data.set(i, val(sin(440.0 * PI * i / sampleRate)));
    }
    
    auto source = context.call<val>("createBufferSource");
    source.set("buffer", buffer);
    source.call<void>("connect", context["destination"]);
    source.call<void>("start", 0);
}

There are a handful of constructs in this snippet worth calling out. The val::global function, given a string, returns the global JavaScript value with that name. val is a C++ type defined by embind that represents a handle to a JavaScript value.

Given a val, properties can be set with val::set and JavaScript methods can be called with val::call<ReturnValue>(arguments...).

In short, the code in Listing 18-7 is a C++ transliteration of the JavaScript in Listing 18-8.

Listing 18-8. Accessing the Web Audio API from JavaScript

function main() {
    var audioContext = window.AudioContext;
    if (!audioContext) {
        audioContext = window.webkitAudioContext;
    }
    var context = new audioContext();
 
    var duration = 2;
    var sampleRate = 44100;
 
    var numberOfFrames = duration * sampleRate;
 
    var buffer = context.createBuffer(1, numberOfFrames, sampleRate);
    var data = buffer.getChannelData(0);
    
    for (var i = 0; i < numberOfFrames; ++i) {
        data[i] = Math.sin(441.0 * Math.PI * i / sampleRate);
    }
    
    var source = context.createBufferSource();
    source.buffer = buffer;
    source.connect(context.destination);
    source.start(0);
}

With embind, anything you might write in JavaScript can be transliterated to C++, and thus the browser’s capabilities are exposed to your game.

image Note  When compiling the program in Listing 18-8, make sure to use the --bind emcc option to enable embind support.

Input Events

Emscripten provides three ways to access user input events: SDL input, glut, and direct access to browser events. If your application is built on SDL (www.libsdl.org/), using the SDL input events is an obvious choice. If all you need is quick access to an OpenGL context and keyboard and mouse events, glut, the OpenGL Utility Toolkit, is also available. Emscripten includes the open source FreeGLUT (http://freeglut.sourceforge.net/) implementation.

Finally, if your game benefits from direct access to browser events, use embind to connect your C++ code to JavaScript event callbacks such as canvas.onmousemove.

Performance

Current benchmarks show that Emscripten-compiled asm.js code is about a factor of two slower than the equivalent native code. This is possible because most native machine operations have direct analogues in JavaScript semantics. Remember that asm.js is restricted to JavaScript expressions that can be efficiently translated to machine code. Memory loads and stores are represented by ArrayBuffer reads and writes. Integer addition is represented by ((x|0)+(y|0))|0. Unsigned integer comparison is represented by ((x>>>0) < (y>>>0)). 32-bit integer multiplication is represented by Math.imul(x, y).

Math.imul is an interesting case, actually. Previous and current JavaScript specifications, ECMAScript 3 and ECMAScript 5, don’t have an expression that directly corresponds to C’s 32-bit integer multiplication. The function in Listing 18-9 implements C-like multiplication in JavaScript.

Listing 18-9. 32-bit Integer Multiplication Fallback

// courtesy of https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Math/imul
function imul(a, b) {
  var ah  = (a >>> 16) & 0xffff;
  var al = a & 0xffff;
  var bh  = (b >>> 16) & 0xffff;
  var bl = b & 0xffff;
  // the shift by 0 fixes the sign on the high part
  // the final |0 converts the unsigned value into a signed value
  return ((al * bl) + (((ah * bl + al * bh) << 16) >>> 0)|0);
}

Math.imul will be introduced in ECMAScript 6 (and is already available in Firefox and Chrome!) so that asm.js can efficiently translate it into a native multiply instruction. In addition, there are upcoming JavaScript proposals to expose other native instruction capabilities such as SIMD vector processing and 32-bit floating arithmetic. (All numbers in JavaScript are IEEE 64-bit floats.) Beyond instruction-level performance, as of ECMAScript 5, JavaScript has no access to shared-memory threads, though WebWorkers provide support for message-passing concurrency.

In short, the Web is a bit less capable than native platforms, but it’s certainly powerful enough to run a wide range of games, especially since the gap between desktop PCs and mobile phones is much larger than the gap between native code and asm.js. Moreover, we can expect improvements to performance over time. Emscripten, being a new compiler, still has room for improvement in its code generation. At the time of this writing, Alon Zakai is rewriting the compiler as a proper LLVM backend rather than a set of programs that parse LLVM IR directly. Emscripten as an LLVM backend is expected to enable further LLVM optimizations.

Concurrently with improvements to the Emscripten compiler itself, browsers are optimizing for asm.js. In November 2013, Google and Opera announced that their browsers have faster asm.js code generation (http://www.unrealengine.com/en/news/epic_citadel_cleared_for_chrome_and_opera_browsers/). I predict that the performance of asm.js will continue to improve.  It’s unrealistic to assume that the restricted, secure asm.js will equal native code performance, but if it reaches 80% to 90% of the performance of native code, then asm.js will be sufficiently fast that we can treat it like any other first-class target.

Debugging

Sadly, if you’re used to IDEs with wonderful debuggers like Visual Studio or even simpler tools like WinDbg or gdb, Emscripten is a step backwards. There are no Emscripten debuggers, though you can invoke emcc with the -g flag and use your web browser’s JavaScript debugger. With -g, emcc tries hard to preserve function and variable names, so it’s not terribly painful to figure out what’s happening at runtime from within a JavaScript debugger. Unfortunately, all current JavaScript debuggers are painfully slow given the large JavaScript files that Emscripten produces.

Thus, most of the time, it’s easiest to fall back on traditional printf debugging.

However, there is one trick that comes in handy. Much like Win32’s DebugBreak() function or x86’s __asm int 3, in Emscripten, you can cause the debugger to break on a line of code with asm(“debugger”);. In Emscripten, asm() allows insertion of arbitrary JavaScript into the generated code, and the JavaScript debugger statement causes the debugger to kick in when that code is reached.  I recommend inserting asm(“debugger”); into your assertion and fatal error functions, as shown in Listing 18-10.

Listing 18-10. Breaking into the Debugger on Fatal Errors

void fatal_error(const char* message) {
    fprintf(stderr, "fatal error: %s ", message);
#ifdef __EMSCRIPTEN__
    asm("debugger");
#elif defined(WIN32)
    DebugBreak();
#endif
    abort();
}

Since debugging Emscripten-generated code is not pleasant, I recommend maintaining a native Windows, Mac, or Linux build of your game and using that for active development. That way, you’ll know that any problems that occur in the Emscripten build are Emscripten-related.

A Game Port

Now that we’ve covered Emscripten from a high level, let’s take an existing game and port it to the Web. Alas, I can’t share any proprietary engine code, but there are a few open source 3D games that will provide an example of the kind of problems you might face when bringing a C++ game to the Web with Emscripten.

Choosing a Game

For this demonstration, I chose to port the game AstroMenace. It’s simple, 3D, has few dependencies, and sufficiently rich that it provides a good example of what’s possible on the Web. As you’ll see, a game that didn’t depend on the OpenGL fixed function pipeline would have been much easier, so when you port your game or engine to Emscripten/WebGL, it helps to have a pure OpenGL ES 2.0 renderer.

Getting Emscripten

To port a game to Emscripten, you need two things: the game’s source code and the latest Emscripten. As a point of detail, I am assuming you are using Linux or Mac OS X. The Emscripten compiler runs on Windows but it’s a little easier from the Linux or Mac command lines.

image Note  Many Emscripten tools are written in Python, and per PEP 0394 (www.python.org/dev/peps/pep-0394/), the Emscripten tools attempt to run the python2 command by default. On most Linux distributions, python2 is a symlink to whatever version of Python 2 is installed, but on Mac you may have to create this symlink yourself with sudo ln -s /usr/bin/python /usr/bin/python2.

At the time of this writing, the best way to get Emscripten is to download the Emscripten SDK from http://emscripten.org. However, you may want to maintain your own local fork of the Emscripten git repository, in case you need to modify anything during the porting effort. My local modifications for the purposes of the AstroMenace game port are available at https://github.com/chadaustin/emscripten.

You can follow the development process, in its unfiltered fits and starts, at my git repository at https://github.com/chadaustin/AstroMenaceEmscripten/.

Building the Game

After grabbing the source code, my first attempt was to write an emscripten-build shell script that uses Emscripten’s emcc and em++ commands to compile the source into HTML. Listing 18-11 shows the initial build command.

Listing 18-11. Initial Emscripten Build Attempt

em++ -Wall -o AstroMenace.html $(find AstroMenaceSource -iname '*.cpp')

As you can see, em++ is used similarly to g++. –Wall enables all warnings, and the remainder of the command says to compile all C++ source files into the AstroMenace.html program, which can then be loaded in a web browser.

The script in Listing 18-11 at least got me to the point where I could start using compile and link errors to drive the rest of the port. Well, almost. Because Emscripten is compiling C++ to JavaScript, unlike a traditional platform, it’s perfectly okay to reference symbols that aren’t defined.  Emscripten assumes that undefined functions refer to external JavaScript implementations. Sometimes having C++ directly call external JavaScript functions is a valid and useful technique. However, for a self-contained program, like a game written entirely in C++, you generally want an undefined function to fail the build. Passing -s ERROR_ON_UNDEFINED_SYMBOLS=1 to emcc or em++ will fail the build until all referenced symbols are defined.

Third-Party Dependencies and a Real Build System

The first round of undefined symbol errors, shown in Listing 18-12, were related to libogg and libvorbis, which were easy enough to integrate. I invoked emcc to produce an LLVM IR file from the libogg and libvorbis .c files, which was then added to the source list for AstroMenace.html. libogg and libvorbis can be downloaded from www.xiph.org/downloads/.

Listing 18-12. Undefined libogg and libvorbis Symbols

Error: unresolved symbol: ov_read
Error: unresolved symbol: ov_open_callbacks
Error: unresolved symbol: ov_info
Error: unresolved symbol: ov_pcm_total
Error: unresolved symbol: ov_clear
Error: unresolved symbol: ov_comment
Error: unresolved symbol: ov_pcm_seek

Next came freealut, which integrated much like libogg and libvorbis. The freealut source code is checked into Emscripten itself, so I just used that.

At this point I realized that using a shell script to compile AstroMenace was becoming 1) slow, as it caused a full rebuild each time, and 2) annoying, as I had to specify long lists of source files, so I switched the build system over to SCons, a Python-based build system. SCons is available at www.scons.org/. Emscripten does not require SCons: Make or any other build system is also suitable.

FreeType (an open source TrueType font renderer) was the hardest dependency because I had to be precise about specifically which FreeType source files to include in the build system. If you include every .c file, you will include multiple implementations of the same functions and see duplicate symbol definition errors. FreeType is also included in the Emscripten tree.

Progress! The code finally compiles and links. However, loading AstroMenace.html in a browser gave JavaScript errors.

Loading Game Content

Loading AstroMenace.html caused the browser to show the JavaScript error in Listing 18-13.

Listing 18-13. First Load Error in Browser

abort() at Error
    at stackTrace (AstroMenace.html:993:15)
    at abort (AstroMenace.html:510736:25)
    at __Z10FileDetectPKc [FileDetect(char*)] (AstroMenace.html:61135:121)
    at __Z8vw_fopenPKc [vw_fopen(char*)] (AstroMenace.html:61172:12)
    at __ZN12cXMLDocument4LoadEPKc [cXMLDocument::Load(char*)] AstroMenace.html:62962:13)
    at __Z16LoadXMLSetupFileb [LoadXMLSetupFile(bool)] (AstroMenace.html:275626:13)
    at Object._main (AstroMenace.html:87596:15)
    at Object.callMain (AstroMenace.html:510655:30)
    at doRun (AstroMenace.html:510695:25)
    at AstroMenace.html:510705:19

As you can see, the stack trace is fairly readable, making it possible to figure out the approximate location of the error. After stepping through the code in the Chrome developer tools JavaScript debugger, I noticed that a variable representing a function pointer from the SDL_RWops struct was completely bogus.

Further exploration uncovered that Emscripten’s current SDL_RWFromFile implementation does not match the header: it returns an opaque integer ID when the SDL API expects that the result of SDL_RWFromFile is a struct containing function pointers. Thus, attempting to dereference the pointer returned by SDL_RWFromFile returned data from bogus memory, so the function pointer was invalid, tripping an invalid virtual call assertion.

To work around this problem, I removed the SDL filesystem support from Emscripten’s SDL and replaced it with SDL_rwops.c from the SDL 1.2 source code itself. Note that, should you take an approach like this, you will need to satisfy the terms of SDL’s LGPL license in some way. Under the LGPL, the end user must be able to substitute their own implementation of any LGPL code, such as SDL, and since Emscripten does not have stable support for dynamic linking, you may struggle to satisfy the LGPL. In this case, because I’m porting an open source game anyway, there are no problems.

There are plans to replace Emscripten’s SDL 1.2 and 1.3 implementations with SDL 2.0, which is licensed under the permissive zlib license. SDL 2.0 would resolve all of these problems.

Finally I reached a point where the compiled AstroMenace.html produced errors about a missing game data pack, as shown in Listing 18-14.

Listing 18-14. Missing Game Assets

Can't find the file /home/emscripten/.astromenace/amconfig.xml AstroMenace.html:61
XML file not found: /home/emscripten/.astromenace/amconfig.xml AstroMenace.html:61
Can't open XML file for write /home/emscripten/.astromenace/amconfig.xml AstroMenace.html:61
*** Can't find VFS file /bin/gamedata.vfs AstroMenace.html:61 ***
*** gamedata.vfs file not found or corrupted. AstroMenace.html:61 ***

gamedata.vfs is a compiled “packfile” of art, generated from source art assets by the game itself when run with a special flag. Since the game isn’t running in Emscripten, I needed a pre-built gamedata.vfs. I could have built and compiled the game on a native platform to produce gamedata.vfs, but I found it easier to download a precompiled binary from the AstroMenace web site and use its gamedata.vfs file.

Linking a data file into an Emscripten program is accomplished with emcc’s --preload-file option. Specifically, I checked gamedata.vfs into AstroMenace/bin and added the build option --preload-file bin@/bin so the generated code has access to /bin/gamedata.vfs.

When using --preload-file, Emscripten-compiled code uses XMLHttpRequest to download the embedded filesystem image in advance of launching the game, but XMLHttpRequest doesn’t work with file:// URLs. Thus, a real HTTP server is needed to test the game. Python provides a simple HTTP server.

python -m SimpleHTTPServer

Since gamedata.vfs is over 100MB and Emscripten by default allocates 16 MiB for the application’s entire memory space, I also needed to increase the total memory size to 200 MiB with the emcc option -s TOTAL_MEMORY=209715200. I picked 200 MiB because 100 MiB is used for the game assets, and I surmised that 100 MiB was sufficient for all runtime allocations. If I turned out to be incorrect, the program would have told me with an out-of-memory error.

Getting the Game to Run

As web pages are single-threaded and event-based, it’s bad behavior for JavaScript to wait by spinning in a loop. Thus, the default Emscripten implementation of the SDL_Delay function crashes if you call it on the main Emscripten thread, so I removed it from Emscripten and replaced it with a function that does nothing. Instead, I could have also simply removed all calls to SDL_Delay from AstroMenace’s loading code. I will discuss the main loop in more depth later in this chapter.

Now all the data files load but the application crashes with the disappointing OpenGL error shown in Listing 18-15.

Listing 18-15. Emscripten Legacy Emulation OpenGL Error

WebGL: getParameter: parameter: invalid enum value 0x821b @http://localhost:8000/build/AstroMenace.js:5798
WebGL: getParameter: parameter: invalid enum value 0x821c @http://localhost:8000/build/AstroMenace.js:5798
WebGL: getParameter: parameter: invalid enum value 0xd31 @http://localhost:8000/build/AstroMenace.js:5798
WebGL: hint: invalid hint @http://localhost:8000/build/AstroMenace.js:5954
WebGL: texImage2D: format does not match internalformat @http://localhost:8000/build/AstroMenace.js:9603
uncaught exception: glMaterialfv: TODO

Emscripten’s OpenGL legacy emulation layer does not currently implement glMaterialfv or glLightfv . Instead, it throws an error with the message "TODO" whenever they are called. Thus, it’s time to pull in a real OpenGL emulation layer: Regal. The Regal project is available at https://github.com/p3/regal.

Integrating with OpenGL, Attempt #1: Regal

I ran into several gaps in Emscripten’s legacy GL emulation support. First, glLightfv and glMaterialfv simply throw an exception when called. Second, AstroMenace targets OpenGL 2.1 and GLSL 1.20, so its shaders start with #version 120. Since WebGL is based on OpenGL ES 2.0 and GLSL 1.00, its shaders must be marked with #version 100.

Rather than deal with all of this myself, I decided to try integrating Regal, a legacy OpenGL emulation on top of OpenGL ES 2. I turned off the -s LEGACY_GL_EMULATION option, and instead included Regal’s source code from within the build system. Then I spent several fruitless evenings attempting to get Regal to run. I ran into issues with illegal asm.js function pointer casts, null GL function pointers, infinite loops, and too many JavaScript local variables being generated when calling into the compiled Regal code. I was not able to clearly determine why Regal didn’t work, but I hope that in the near future the Emscripten implementation of LEGACY_GL_EMULATION is swapped out for something like Regal. I did port a very simple OpenGL demo (https://github.com/chadaustin/nehe-emscripten) to Emscripten and Regal and got it to work with some help from Emscripten’s community, so Regal is probably viable after applying some elbow grease.

In practice, any game that runs on Android or iOS would have an OpenGL ES 2 rendering pipeline and thus would not require any kind of OpenGL legacy emulation layer.

Integrating with OpenGL, Attempt #2: Simplifying the AstroMenace Renderer

For the purposes of this demonstration, I opted for something easier than getting Regal to work reliably. I disabled lighting and materials in the application and switched back to LEGACY_GL_EMULATION. To verify that my changes to AstroMenace’s source code weren’t breaking the game itself, I spun up a native development environment (specifically, an Ubuntu VM) so I could begin modifying and testing the original game code. After each change, I verified that the game still compiled and ran. After removing support for shadow maps, MSAA, occlusion queries, anisotropic filtering, texture compression, and various other fancier rendering capabilities, I finally had an Emscripten-compiled game that actually rendered 3D. The 3D output was obviously glitchy, with corrupted vertex buffers, largely due to the incomplete and buggy Emscripten legacy GL emulation layer. However, as I mentioned before, if a game’s rendering code is limited to the OpenGL ES 2 subset, porting it to Emscripten and WebGL will be dramatically easier.

The Main Loop

Finally, it’s worth discussing one significant change needed when most games are brought to Emscripten: the browser is an event-based platform. JavaScript runs with a single execution thread in the context of a web page. That means that, while JavaScript is running, the browser cannot update the page nor display anything to the screen. Many browsers can’t even process user input while JavaScript is running. This causes some problems when trying to integrate traditional game loops with a web page. A traditional game’s main loop is structured as shown in Listing 18-16.

Listing 18-16. Traditional Game Main Loop

while (running) {
    process_input();
    simulate_world();
    render_graphics();
    wait_for_next_frame();
}

On a web page, while such a loop is running, the browser cannot handle user input events or graphics updates. Thus, the browser will assume your script is hung and ask the user to kill it. Instead of waiting for the next frame in a blocking game loop, you must instead ask Emscripten to call your game when it’s time for the next frame, as shown in Listing 18-17.

Listing 18-17. Emscripten Game Main Loop

void main_loop();
int main() {
    // initialization code
    // ...
    // at game startup, usually at the end of main()
    emscripten_set_main_loop(main_loop, 60, false);
}
void main_loop() {
    process_input();
    simulate_world();
    render_graphics();
}

Note that, in Listing 18-17, the game loop is handled by the browser, allowing it to update the display and response to input events. The loop body, however, is implemented by the game in the main_loop() function, giving your game control over its simulation and rendering.

The Emscripten Platform

So far we’ve discussed Emscripten’s compiler, how to access the browser’s graphics, sound, and input APIs, and how to port a C++ game to the Web with Emscripten. Now let’s talk about releasing the same game on multiple platforms.

As with any multiplatform development strategy, simply consider Emscripten as just another platform, like iOS, Android, or Windows. Your engine will need some Emscripten-specific platform components, such as input device detection and configuration, the ability to invoke HTML5’s full screen and pointer lock APIs, and any other browser-specific APIs. The web’s capabilities come from the browser through JavaScript, but they need to be accessed through Emscripten.

At the time of this writing, Emscripten is not a complete product; rather, it’s more of a functional proof of concept. You will likely run into gaps in the platform, though they are being addressed over time. Various platform components may be missing, incomplete, unspecified, or subject to change. For example, even though Emscripten has built-in support for much of SDL and OpenGL fixed-function, I had to stub out or implement the functions in Listing 18-18 when porting AstroMenace.

Listing 18-18. Unimplemented Emscripten Functions Used by AstroMenace

glPushAttrib
glPopAttrib
gluErrorString
glLightf
glMaterialfv
glLightfv
SDL_GetWMInfo
SDL_GetGammaRamp
gluBuild2DMipmaps
alIsSource
alSourceRewind
SDL_WaitEvent

Don’t be intimidated, however! The rate at which Emscripten is improving is impressive. Over the period in which I have been using Emscripten, many of its bugs and limitations have been fixed.

If you discover a gap in the platform capabilities, you can either implement said gap in Emscripten itself or implement a stub in your own code. Of course, make sure you file a bug against Emscripten itself! The Emscripten community is helpful and responsive, so perhaps it will be fixed before you notice.

Predicting the Future

Compiling C++ to high-performance JavaScript is cutting edge. We can’t predict exactly how the next decade will play out, but since Emscripten and asm.js are very new, it’s easy to imagine significant performance and code generation improvements. However, the biggest room for improvement comes from new and richer web browser APIs. As of now, WebGL exposes the least common denominator of graphics APIs: OpenGL ES 2. When a version of WebGL that exposes OpenGL ES 3 is available, games can take advantage of increased graphical fidelity and capabilities.

When WebCL (a JavaScript API for GPU computing via OpenCL) is widespread, games can offload arbitrary computation to the GPU. WebCL could also make it possible to access any idle CPU cores and vector units, making better use of overall system resources.

I’m not sure that Emscripten itself will remain the dominant toolchain for compiling to JavaScript, but, if not, something else will take its place. Compiling to JavaScript isn’t going away anytime soon. The fact that you can run your compiled code in all browsers, even if they don’t specially optimize for asm.js, is too powerful.

If you have a C++ game and you want your players to download and play it securely, with no friction, in a web browser, give Emscripten a chance.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
18.117.186.46