What’s OpenGL ES

Open Graphics Library (OpenGL) came from Silicon Graphics (SGI); they were makers of high-end graphics workstations and mainframes. Initially, SGI had a proprietary graphics framework called IRIS GL (which grew to be an industry standard), but as competition increased, SGI opted to turn IRIS GL to an open framework. IRIS GL was stripped down of nongraphics-related functions and hardware-dependent features and became OpenGL.

OpenGL is a cross-language, cross-platform application programming interface (API) for rendering 2D and 3D graphics. It’s a lean mean machine for rendering polygons; it’s written in C as an API for interacting with a graphics processing unit (GPU) to achieve hardware accelerated rendering. It’s a very low-level hardware abstraction.

As small handheld devices became more and more common, OpenGL for Embedded Systems (OpenGL ES) was developed. OpenGL ES is a stripped-down version of the desktop version; it removed a lot of the more redundant API calls and simplified other elements to make it run efficiently on the less powerful CPUs in the market; as a result, OpenGL ES was widely adopted in many platforms such as HP webOS, Nintendo 3DS, iOS, and Android.

OpenGL ES is now an industry standard for (3D) graphics programming. It is maintained by the Khronos Group, which is an industry consortium whose members include, among others, ATI, NVIDIA, and Intel; together, these companies define and extend the standard.

There are still developers, especially those who focus on games that run on multiple platforms, who write for OpenGL ES 1.0; this is because of its simplicity, flexibility, and standard implementation. All Android devices support OpenGL ES 1.0, some devices support 2.0, and any device after Jelly Bean supports OpenGL ES 3.0. At the time of writing, more than half of activated Android devices already support OpenGL ES 3.0. Table 9-1 shows the distribution and Figure 9-1 shows a nice pie chart to go with it; this data was taken from https://developer.android.com/about/dashboards#OpenGL.

Table 9-1

OpenGL ES version distribution

OpenGL ES Version	Distribution
GL 1.1 only	0.0%
GL 2.0	14.5%
GL 3.0	18.6%
GL 3.1	9.8%
GL 3.2	57.2%

It’s important to note that OpenGL ES 2.0 breaks compatibility with the 1.x versions. You can use either 1.x or 2.0, but not both at the same time. The reason for this is that the 1.x versions use a programming model called fixed-function pipeline , while versions 2.0 and up let you programmatically define parts of the rendering pipeline via shaders.

What does OpenGL ES do

OpenGL ES just renders triangles. Not much else.

Think of OpenGL ES as working like a camera. To take a picture, you have to go to the scene you want to photograph. Your scene is composed of objects that all have a position and orientation relative to your camera as well as different materials and textures. Glass is translucent and reflective; a table is probably made out of wood; a magazine has some photo of a face on it; and so on. Some of the objects might even move around (e.g., cars or people). Your camera also has properties, such as focal length, field of view, image resolution, size of the photo that will be taken, and a unique position and orientation within the world (relative to some origin). Even if both the objects and the camera are moving, when you press the shutter release, you catch a still image of the scene. For that small moment, everything stands still and is well defined, and the picture reflects exactly all those configurations of position, orientation, texture, materials, and lighting. Figure 9-2 shows an abstract scene with a camera, light, and three objects with different materials.

Each object has a position and orientation relative to the scene’s origin. The camera, indicated by the eye, also has a position in relation to the scene’s origin. The pyramid in Figure 9-2 is called the view volume or view frustum , which shows how much of the scene the camera captures and how the camera is oriented. The little white ball with the rays is the light source in the scene, which also has a position relative to the origin.

We can map this scene to OpenGL ES, but to do so, we need to define (1) models or objects, (2) lights, (3) camera, and (4) viewport.

Models or Objects

OpenGL ES is a triangle rendering machine. OpenGL ES objects are a collection of points in 3D space; their location is defined by three values. These values are joined together to form faces, which are flat surfaces that look a lot like triangles. The triangles are then joined together to form objects or pieces of objects (polygons).

The resolution of your shapes can be improved by increasing the number of polygons in it. Figure 9-3 shows various shapes with varying number of polygons.

On the far left of Figure 9-3 is a simple sphere; it doesn’t really go over well as a sphere if you look closely at it. The shape next to it (right) is also a sphere but with more polygons. The shapes, as they progress to the right, form complex contours; this can be achieved by increasing the number of polygons in the shape.

Projections

OpenGL ES can construct a 2D bitmap of a scene from the camera’s point of view. While everything is defined in 3D space, OpenGL maps the 3D space to 2D via something called projections . A single triangle has three points defined in 3D space. To render such a triangle, OpenGL ES needs to know the coordinates of these 3D points within the pixel-based coordinate system of the framebuffer that are inside the triangle.

Matrices

There’s a lot more theories we need to absorb in OpenGL ES programming, but let’s explore some of those theories alongside a simple coding exercise.

Rendering a Simple Sphere

OpenGL ES APIs are built into the Android framework, so we don’t need to import any other libraries or include any other dependencies into the project.

The manifest entry is basically saying that the app expects the device to support OpenGL ES 2, which is practically all devices at the time of writing.

I removed the default TextView object and inserted a GLSurfaceView element with 400dp by 400dp size. Let’s keep it evenly square for now, so that our shape won’t skew. OpenGL assumes that drawing areas are always square.

Figure 9-4 shows the activity_main layout in design mode.

The GLSurfaceView is an implementation of the SurfaceView class that uses a dedicated surface for displaying OpenGL rendering; this object manages a surface, which is a special piece of memory that can be composited into the Android view system. The GLSurfaceView runs on a dedicated thread to separate the rendering performance from the main UI thread.

The onDrawFrame() method of the Renderer is where we tell OpenGL ES to draw something on the surface. We’ll do this by passing an array of numbers which represents positions, colors, and so on. In our case, we’re going to draw a sphere. We can hand-code the arrays of numbers—which represent X,Y,Z coordinates of the vertices—that we need to pass OpenGL ES, but that may not help us to envision what we’re trying to draw. So, instead, let’s use a 3D creation suite like Blender (www.blender.org) to draw a shape.

Blender is open source; you can use it freely. Once you’re done with the download and installation, you can launch Blender, then delete the default cube by pressing X; next, press Shift+A and select Mesh ➤ Ico Sphere, as shown in Figure 9-5.

Now we’ve got a moderately interesting object with a couple of vertices—it will be cumbersome to hand-code these vertices; that’s why we took the Blender route.

To use the sphere in our app, we must export it as a Wavefront object. A Wavefront object is a geometry definition file format. It’s an open format and is adopted by 3D graphics application vendors. This is a simple data format that represents 3D geometry, namely, the position of each vertex; the faces that make each polygon are defined as a list of vertices. For our purposes, we’re only interested in the position of the vertices and the faces.

In Blender, go to File ➤ Export Wavefront (.obj) as shown in Figure 9-6. In the following screen, give it a name (sphere.obj) and save it in a location of your choice. Don’t forget to note the export settings of Blender; check only the following:

• Export as OBJ object

• Triangulate faces

• Keep vertex order

These are the settings I found to be easy to work with, especially when you’re about to parse the exported vertex and faces data.

Notice how each line starts with either a “v” or an “f”. A line that starts with a “v” represents a single vertex, and a line that starts with an “f” represents a face. The vertex lines have the X, Y, and Z coordinates of a vertex, while the face lines have the indices of the three vertices (which together form a face).

To keep things organized, let’s create a class that will represent our sphere object—we don’t really want to write all the drawing code inside the onDrawFrame() method now, do we?

Let’s create a new class and add it to the project. You can do this by using Android Studio’s context menu; right-click the package name (as shown in Figure 9-7), then choose New ➤ Java Class.

In the screen that follows, provide the name of the class (Sphere), as shown in Figure 9-8.

The idea is to read the exported Wavefront OBJ file and load the vertices and faces data into their corresponding List objects. Before we can read the file, we need to add it to the project. We can do that by creating an assets folder. An assets folder gives us the ability to add external files to the project and make them accessible to our code. If your project doesn’t have an assets folder, you can create them. To do that, use the context menu; right-click the “app” in the Project tool window (as shown in Figure 9-9), then select New ➤ Folder ➤ Assets Folder.

In the window that follows, click Finish, as shown in Figure 9-10.

Gradle will perform a “sync” after you’ve added a folder to the project. Figure 9-11 shows the Project tool window with the newly created assets folder.

Next, right-click the assets folder, then choose Reveal in Finder (as shown in Figure 9-12)—this is the prompt I got because I’m using macOS. If you’re on Windows, you will see “Show in Explorer” instead.

You can now transfer the sphere.obj file to the assets folder of the project.

Alternatively, you can copy the sphere.obj file to the assets folder using the Terminal of Android Studio (as shown in Figure 9-13).

Use whichever way is more convenient for you. Some prefer the GUI way, and some prefer the command line. Use the tools you’re more familiar with.

We’re coding our app using the Java language, but you need to remember that OpenGL ES is actually a bunch of C APIs. We can’t simply pass our list of vertices and faces to OpenGL ES directly. We need to convert our vertices and faces data into something OpenGL ES will understand.

The next step is to create the shaders. We can’t render our 3D sphere if we don’t create the shaders; we need a vertex shader and a fragment shader. A shader is written in a C-like language called OpenGL Shading Language (GLSL for short).

A vertex shader is responsible for a 3D object’s vertices, while a fragment shader (also called a pixel shader) handles the coloring of the 3D object’s pixels.

To create the vertex shader, add a file to the project’s assets folder and name it vertex_shader.txt , as shown in Figure 9-14.

In the window that follows (Figure 9-15), enter the name of the file.

It’s a minimalistic fragment shader code; it basically assigns a light green color to all the pixels.

Before we can use the shaders, we need to link them to a program. We can’t use the shaders directly. This is what connects the output of the vertex shader with the input of the fragment shader. It’s also what lets us pass an input from our program and use the shader to draw our shapes.

At this point, you’re ready to run the app. Figure 9-16 shows the app at runtime.

After nearly 300 lines of code, all we got was a little green Icosphere without much definition. Welcome to OpenGL ES programming. This should give you an idea how involved and how much work goes into an OpenGL ES game.

Key Takeaways