15
Navigating the View Space
One of the most common uses of transformations in computer games is the movement of the camera viewing the scene. While the camera can be considered as another 3D object in the environment with respect to its movement and orientation, its current transformation is also used as the view matrix by which all virtual objects in the world are affected.
The transformations presented in this chapter for moving and rotating the camera within its environment are the exact same operations that would be performed on any object in the environment.
In this chapter we will explore:
- Flying maneuvers
- Understanding and fixing compound rotation quirks
- Improving camera orientations
By the end of this chapter, you will have a greater appreciation of the complexity of 3D rotations and the issues that arise from using them in mathematical operations to combine multiple orientations. The knowledge herein you will use over and over again as you work in graphics and games as the concepts are fundamental elements of all virtual environments in which you need to move objects or the camera.
Technical requirements
In this chapter, we will be using the Python, PyCharm and Pygame as used in previous chapters.
Before you begin coding, create a new folder in the PyCharm project for the contents of this chapter called Chapter_15.
The solution files containing the code can be found on GitHub at https://github.com/PacktPublishing/Mathematics-for-Game-Programming-and-Computer-Graphics/tree/main/Chapter15.
Flying maneuvers
Any object in 3D space can have transformations applied to it. It’s no different for the camera. In this section, we will explore how the same mathematics can be applied to the object to move and reorient it.
In Chapter 12, Mastering Affine Transformations, we discussed the three rotations that can take place in 3D space; namely, pitching, yawing, and rolling. These correspond with rotations around the x, y, and z axes respectively.
The 4 x 4 matrix that will perform a pitch around the x axis is:
The matrix that will perform a yaw around the y axis is:
The matrix to perform a roll around the z axis is:
We can use these matrices to rotate the camera and thus the view space. To do this, we multiply the view matrix by one or more of these.
The way the camera, or any other object transformed with these matrices, rotations can be likened to the movement of an aircraft as illustrated in Figure 15.1. In Figure 15.1 (a), the rotational movement around each axis is shown. In (b), a pitch rotation shows how the nose of the plane looks up and down pivoting around an x axis that extends to the left and right of the plane. Then (c) demonstrates how a roll tilts the plane from side to side and (d) demonstrates how a yaw rotates the plane on the spot around its up axis. This is how a first-person character in a game environment looks around:
Figure 15.1: The orientations of an aircraft
It is useful in 3D graphics environments to be able to orient the camera around the scene using these rotations. This is how a first-person character in a game environment looks around. Of course, the best way to experience these maneuvers is to put them into practice as we will do now.
Let’s do it…
In this practical exercise, we will implement roll, pitch, and yaw in the Camera.py class to enable you to move the camera around a scene:
- Make a copy of the Chapter_14 folder and name it Chapter_15.
The rotation matrices we need for the camera are in the Transform class. In fact, all the operations required for matrix manipulation to move and orient the camera are in the Transform class, and therefore rather than repeat code, it makes more sense to give the camera a transform component that can do all the work.
- To achieve this, go ahead and modify the Camera class as follows:
import pygame
import math
import numpy as np
from Transform import *
class Camera:
def __init__(self, fovy, aspect, near, far):..self.PPM = np.matrix([[a, 0, 0, 0],[0, b, 0, 0],[0, 0, c, -1],[0, 0, d, 0]])self.VM = np.identity(4)self.transform = Transform()self.pan_speed = 0.01def get_VM(self):return self.transform.MVMdef get_PPM(self):return self.PPMdef update(self):key = pygame.key.get_pressed()if key[pygame.K_w]:self.transform.update_position(self.transform.get_position()+ pygame.Vector3(0, 0, self.pan_speed),False)if key[pygame.K_s]:self.transform.update_position(self.transform.get_position()+ pygame.Vector3(0, 0,-self.pan_speed), False)
The key things to notice in this class are the addition of the transform property in the initializer and then its use to call the update_position() method in the update() method. Now that the Camera class is using the transform to store and manipulate the view matrix, in what’s called the MVM (ModelView matrix) in the Transform class, the get_VM() method is updated to return the MVM from the transform. The model view matrix in the Transform class is just a 4x4 matrix and you could say it holds the model view matrix that would be local to the camera itself (if the camera were just another object in the scene). Hence, we can use that matrix belonging to the camera as a view matrix. This is particularly useful as you might want to swap view matrices between a multitude of cameras. The camera’s new transform.MVM only becomes the view matrix if it is the camera being used to view the scene.
We are also passing a False parameter through the update_position() method, which we will now cater for.
- Because the movement of the camera means that the oppositive movement is added to any game objects in world coordinates, we must detail with its position update in the opposite order. To Transform.py make the following modifications:
def update_position(self, position: pygame.Vector3,
local=True):if local:self.MVM = self.MVM @ np.matrix([[1, 0, 0, 0],[0, 1, 0, 0],[0, 0, 1, 0],[position.x,position.y,position.z,1]])else:self.MVM = np.matrix([[1, 0, 0, 0],[0, 1, 0, 0],[0, 0, 1, 0],[position.x, position.y,position.z, 1]])@ self.MVM
Notice how the matrix multiplication is occurring in the opposite order if local is given a value of False. This is because we want to move the camera in world space and apply that to the entire scene.
With the modifications made to the camera, you can run the project from the main script file: TransformationMatrices.py and the scene will be as they were before where the camera can be moved in and out with the ‘W’ and ‘S’ keys.
Now as a challenge, let’s see whether you can program in the movements for the ‘A’ and ‘D’ keys.
Your turn...
Exercise A. Give the camera the ability to pan (slide) to the left when the ‘A’ key is pressed and to pan to the right when the ‘D’ key is pressed and to yaw when the ‘Q’ and ‘E’ keys are used.
- To make the scene more interesting to explore with the camera, I have made a low-polygon version of the teapot model called teapotSM.obj. You can download this from GitHub and add this to your models folder for the project and then load it in the main script of TransformationMatrics.py like this:
from LoadMesh import *
objects_2d = []
teapot = Object(“Teapot”)
teapot.add_component(Transform())
teapot.add_component(LoadMesh(GL_LINE_LOOP,
“models/teapotSM.obj”))trans: Transform = teapot.get_component(Transform)
trans.rotate_y(90)
trans.update_position(pygame.Vector3(0, -2, -3))
camera = Camera(60, (screen_width / screen_height),
0.1, 1000.0)objects_3d.append(teapot)
clock = pygame.time.Clock()
- If you still have print lines in your code printing out matrices in the console, such as the ones in the update() method of Object.py, you can remove them if you desire.
When first run, you’ll find a wireframe image of a teapot as shown in Figure 15.2. If you want of move the camera faster, increase the value of pan_speed in Camera.py.
Figure 15.2: Low-polygon teapot
- To color the teapot as in Figure 15.2, add some lights into the setup_3d() method as follows:
glEnable(GL_LIGHTING)
glLight(GL_LIGHT0, GL_POSITION, (5, 5, 5, 0))
glLightfv(GL_LIGHT0, GL_AMBIENT, (1, 0, 1, 1))
glLightfv(GL_LIGHT0, GL_DIFFUSE, (1, 1, 0, 1))
glLightfv(GL_LIGHT0, GL_SPECULAR, (0, 1, 0, 1))
glEnable(GL_LIGHT0)
- Now it’s time to add rotations to the camera. For this, we will first explore a yaw using the ‘Q’ and ‘E’ keys. To do this, add the following code to Camera.py:
def __init__(self, fovy, aspect, near, far):
..self.transform = Transform()self.pan_speed = 0.1self.rotate_speed = 10..
def update(self):
key = pygame.key.get_pressed()..if key[pygame.K_d]:self.transform.update_position(
self.transform.get_position()+ pygame.Vector3(-self.pan_speed, 0,0))if key[pygame.K_q]:self.transform.rotate_y(self.rotate_speed)if key[pygame.K_e]:self.transform.rotate_y(-self.rotate_speed)
In the new code, we have added a property to store the rotation speed, which you can change to rotate faster or slower, and key press tests for Q and E, which will make use of the rotate_y() method we wrote earlier.
- Run the program now and you will find that you can rotate the camera to the left and right around its y axis. If you would prefer the keys rotated in the opposite direction to the way I have them, you can remove the minus sign from the ‘E’ press line and add it to the Q press line.
From here the code to pitch and roll the camera is elementary. All you need to do is program extra keypresses and call the rotate_x() and rotate_z() methods. Try this for yourself! However, now we are going to continue by programming the mouse to rotate the camera.
- Before we continue, make a copy of TransformationMatrices.py and call it FlyCamera.py. Near the while loop you should make these changes to use the mouse:
pygame.event.set_grab(True)
pygame.mouse.set_visible(False)
while not done:
events = pygame.event.get()for event in events:if event.type == pygame.QUIT:done = Trueif event.type == KEYDOWN:if event.key == K_ESCAPE:pygame.mouse.set_visible(True)pygame.event.set_grab(False)if event.key == K_SPACE:pygame.mouse.set_visible(False)pygame.event.set_grab(True)glPushMatrix()
The first new lines use pygame to grab the mouse so it can be used to feed motion into the program. After that the mouse’s visibility is set to False so it is no longer visible in the window. Because at some time you might like to get the mouse back and stop it having motion control and switch back again, a KEYDOWN event is monitored. If the key is pressed, the mouse becomes visible and you can move it freely. This is very useful if you want to close the window or use in for any user interface elements such as buttons. However, if you want to go back into mousegrabbing mode, the spacebar can be used.
- To read the mouse movement and use the changes to rotate the camera, modify the Camera.py script like this:
def __init__(self, fovy, aspect, near, far):..self.rotate_speed = 10self.last_mouse = pygame.math.Vector2(0, 0)self.mouse_sensitivityX = 0.5self.mouse_sensitivityY = 0.5self.mouse_invert = -1
In the initialization method we add four new properties. last_mouse stores the last read position of the mouse. This is required for us to calculate how far the mouse has moved since the last frame. This value can then be used as a rotation amount. Following this, the mouse sensitivity is set for movements in x (horizontally across the screen) and y (vertically up and down the screen). And last, an invert setting is added in case you prefer the mouse movements to be in the opposite direction to which you are scrolling. This is a common optional setting in first-person shooter and flying games.
Next, in the update() method the mouse movements are captured and used to rotate the camera if the mouse is in the invisible mode set in the FlyCamera.py loop in step 5. Here, the mouse’s position is captured and taken away from the position of the mouse in the last frame. This gives the total movement of the mouse during the last frame:
def update(self):
key = pygame.key.get_pressed()
if key[pygame.K_w]:
..
if key[pygame.K_e]:
self.transform.rotate_y(
-self.rotate_speed)
if not pygame.mouse.get_visible():
mouse_pos = pygame.mouse.get_pos()
mouse_change = self.last_mouse -
pygame.math.Vector2(mouse_pos)
pygame.mouse.set_pos(
pygame.display.get_window_size()[0]
/ 2,
pygame.display.get_window_size()[1]
/ 2)
self.last_mouse = pygame.mouse.get_pos()
self.rotate_with_mouse(mouse_change.x *
self.mouse_sensitivityX,
mouse_change.y *
self.mouse_sensitivityY)Following this the position of the mouse is set to the center of the window using values stored by pygame. Even though the mouse can’t be seen, it’s still possible to scroll it outside the window. This would result in incorrect measurements of how far the mouse has moved. Therefore, for each frame, it is set back to the center. The next line of code resets the last_mouse value, which should be the center of the screen at this point.
Finally, the values recorded for the mouse’s change in position are used to rotate the camera inside a new method we will add next.
- To the Camera class add the following method:
def rotate_with_mouse(self, yaw, pitch):
self.transform.rotate_y(self.mouse_invert * yaw *self.mouse_sensitivityY)self.transform.rotate_x(self.mouse_invert *pitch * self.mouse_sensitivityX)
This method will rotate the mouse around the y axis using the mouse’s vertical movement and around the x axis using the mouse’s horizontal movement.
You can now run the program to test out the mouse movement. Note if it is too fast, turn down the mouse sensitivity values. You will also be able to use the keys for moving the camera in addition to turning it. At this point, it won’t work perfectly but it is a start.
In this exercise, we added code to move the camera around in its 3D environment as well as rotate it. The biggest issue you will have experienced is that the camera starts to tip roll around the z axis and may eventually turn upside down. This can easily be achieved by making small circular motions with the mouse. This might seem counterintuitive since you are only coding it to pitch and yaw. The reason why this is happening and a solution to the problem is the topic of the next section.
Understanding and fixing compound rotation quirks
In the previous section, you had the chance to experiment with moving the camera around in its 3D environment so you could explore looking at the environment from different angles. While moving the camera is elementary, adding rotations seems to introduce undesired results. In this section, you will discover the source of these issues and look at how to fix them.
Imagine you are holding a camera and looking through it toward the horizon. Now bend down at the hip by 90 degrees so that the camera is looking straight at the ground. Next, rotate your upper body 90 degrees upward to the left. You might be thinking along the steps shown in Figure 15.3. First rotating around the x axis turns the camera to face downward making its y axis horizontal and then a rotation around the y axis will face the camera sideways and result in the x axis sitting horizontally. In performing these orientations, we assume each axis can move independently of the others:
Figure 15.3: Pitching then yawing by 90 degrees
This might be the desired result – however, mathematically this doesn’t happen and it’s because of the way our matrices represent and calculate rotations. The axes are in fact connected to each other in a hierarchy of gimbals with y on the outside, x next on the inside, and z inside that. Look at Figure 15.4 where the default gimbal setup is shown on the left.
Figure 15.4: Rotating using codependent gimbals
Mathematically, when a 90-degree rotation around the x axis occurs, the x gimbal (shown in red) and the y gimbal (shown in green) become aligned. The x axis is stuck in this position such that if a y axis rotation occurs, the x gimbal has no other option than to rotate with it. Each of these gimbals represent a degree of freedom for orientations. In the image on the left there are three degrees of freedom. After an x rotation of 90 though, the x gimbal and y gimbal are aligned into the same dimension and therefore an entire dimension is lost.
To further exemplify this effect, we can look at it mathematically. Here’s the generic equation for performing a z rotation, y rotation, and x rotation, using Euler’s method, with the z rotation matrix being on the right and the x rotation on the left.
No matter what the angles used for the x and z rotations are, if 
is 90 degrees, the result will give:
Multiplying this out results in the compound matrix:
Without knowing the values for the other angles, the resulting matrix tells us that any rotation we wanted to include for the angle 
has been totally lost and therefore so has a dimension of freedom. This effect occurs in many instances of 90-degree rotations that can affect the gimbals. It’s known as gimbal lock and is a common issue with gimbal-based rotation systems as is documented as an issue with the famous Apollo 13 mission to The Moon (https://history.nasa.gov/afj/ap13fj/12day4-approach-moon.html ).
In this section we have explored how compound rotation mathematics we’ve implemented can break down. While it can work in many situations, the inherent flaw in Euler’s method can, at times, accidentally eliminate a degree of rotational freedom. Therefore, a better approach to working with compound rotations requiers a different form of rotational mathematics. It involves representing a rotation as vectors and angles in a format called a Quaternion. This is a very advanced mathematical concept that will be explored in the next chapter. For now though, we will fix our projects camera by restricting its rotations and making them far more intuitive with respect to a 3D flying camera.
Improving camera orientations
Regardless of the direction the camera is facing, when flying around in a 3D environment, being able to yaw around the world’s up axis is far less disorientating than yawing at an unexpected angle, no matter how mathematically correct it may be. Rolling around the forward-facing vector of the camera is however intuitive as it is the way the viewer is facing. Many first-person player controllers are set up to rotate around the world's up axis and the camera-forward axis as shown in Figure 15.5. It prevents the camera tipping accidentally upside down and experiencing gimbal lock.
Figure 15.5: Intuitive camera flying rotation axes
To instigate rotations using these axes, we need to be able to switch between local and world space. In this instance we want to rotate the camera in world space (where it yaws around the up axis) and roll around the local z axis, which is the axis that indicates the direction the camera is facing. Let’s now update the project to use this method of rotation.
Let’s do it…
In this exercise, we will modify the Transform class to work with local and world position systems and then modify the Camera class to rotate like a traditional first-person camera.
- The modifications to the rotation methods in the Transform class require the addition of a new parameter like this:
def rotate_x(self, amount, local=True):
amount = math.radians(amount)if local:self.MVM = self.MVM @ np.matrix([[1, 0, 0, 0],[0, math.cos(amount), math.sin(amount),0],[0, -math.sin(amount), math.cos(amount),0],[0, 0, 0, 1]])else:self.MVM = np.matrix([[1, 0, 0, 0],[0, math.cos(amount), math.sin(amount),0],[0, -math.sin(amount), math.cos(amount),0],[0, 0, 0, 1]]) @ self.MVMdef rotate_y(self, amount, local=True):
amount = math.radians(amount)if local:self.MVM = self.MVM @ np.matrix([[math.cos(amount), 0, -math.sin(amount),0],[0, 1, 0, 0],[math.sin(amount), 0, math.cos(amount),0],[0, 0, 0, 1]])else:self.MVM = np.matrix([[math.cos(amount), 0, -math.sin(amount),0],[0, 1, 0, 0],[math.sin(amount), 0, math.cos(amount),0],[0, 0, 0, 1]]) @ self.MVMdef rotate_z(self, amount, local=True):
amount = math.radians(amount)if local:self.MVM = self.MVM @ np.matrix([[math.cos(amount), math.sin(amount), 0,0],[-math.sin(amount), math.cos(amount), 0,0],[0, 0, 1, 0],[0, 0, 0, 1]])else:self.MVM = np.matrix([[math.cos(amount), math.sin(amount), 0,0],[-math.sin(amount), math.cos(amount), 0,0],[0, 0, 1, 0],[0, 0, 0, 1]]) @ self.MVM
Each of the rotation methods is given an extra parameter to indicate whether the operations should occur locally. Up until now we have been using local rotations and therefore setting this parameter to True will ensure the existing code doesn’t break.
Next, each one of the methods undergoes the same change. The if-else statement checks whether local is True. If it is, the same matrix multiplication that was performed before is used. If, however, local is False, the matrix multiplication occurs in reverse order. Performing the operation in reverse is the same as leaving the operation in the same order but inverting the rotation matrix. The inverse of a local matrix, in this case, puts it into world coordinates.
- Next, change the rotate_with_mouse() method to:
def rotate_with_mouse(self, yaw, pitch):
self.transform.rotate_y(self.mouse_invert * yaw *self.mouse_sensitivityY, False)self.transform.rotate_x(self.mouse_invert *pitch * self.mouse_sensitivityX, False)
The parts of this code doing all the work are the rotate_y() method, which operates in world space, and the rotate_x() method, which operates in local space.
With these changes made you can play the application. Note, the camera will still be able to go upside down, but it won’t end up at a strange angle.
Often with a first-person camera it makes sense to restrict the angles the yaw and roll can travel beyond to stop the camera from going upside down. We therefore need to keep track of the camera’s forward vector and the world up vector shown in Figure 15.6.
Figure 15.6: Angles between the world up and camera forward
Before we implement the code let’s pause to consider the mathematics.
Calculating the camera tilt
At some point in the camera’s pitch, you want to be able to stop it from moving any further. This means calculating the angle between the world up vector and the camera’s forward vector. As the camera pitches up, this angle gets smaller.
In this case, 
can be calculated as in Chapter 9, Practicing Vector Essentials, where we discussed the formula:
There will however be two angles that need to be considered – one where the camera is tilted up and the angle between the up vector and the camera’s forward vector is small and another when the camera is tilted down and the angle between the up vector and the camera’s forward vector is large. Anywhere in between the camera should be able to tilt freely.
- Rather than manually coding the entire rotation equations, we can use the built-in methods that come with pygame. Make the following changes to the rotate_with_mouse() method:
def rotate_with_mouse(self, yaw, pitch):
forward = pygame.Vector3(self.get_VM()[0, 2],self.get_VM()[1, 2], self.get_VM()[2, 2])up = pygame.Vector3(0, 1, 0)angle = forward.angle_to(up)self.transform.rotate_y(self.mouse_invert * yaw *self.mouse_sensitivityY,False)if angle < 170.0 and pitch > 0 or //angle > 30.0 and pitch < 0:self.transform.rotate_x(self.mouse_invert* pitch * self.mouse_sensitivityX, True)
Here, the forward vector for the camera is being taken from the third column in its transformation matrix, which contains its local z axis in world coordinates. Next, the up angle is constructed followed by the use of the pygame Vector3 class method, angle_to().
An if statement is then used to determine whether the pitch angle is between 170 and 30 degrees. If it is within this range, a pitch will still be possible. Note that a restriction of the pitch angle being greater or less than 0 is added as there’s no need to rotate around the x axis of the camera if there is no pitch angle.
- Now that you can rotate the camera, you’ll notice a small issue that occurs with the movement code. Previously we were using update_position() in the Transform.py class to add a vector to the position of an object to move it. However, this won’t work with an object you want to move in a forward direction no matter which way it is orientated. In Figure 15.5 the camera has a forward vector, and it is this vector we want to move it along when flying forward. The current code simply adds the world vector of (0, 0, 1), which moves the camera along the world z axis. But if the camera isn’t facing in this direction, this code won’t work. Instead, we need to find the forward-facing vector, as we have for the rotation and use that for movement as well. Modify the code in Camera.py like this:
def update(self):
key = pygame.key.get_pressed()forward = pygame.Vector3(self.get_VM()[0, 2],self.get_VM()[1, 2],self.get_VM()[2, 2])right = pygame.Vector3(self.get_VM()[0, 0],self.get_VM()[1, 0],self.get_VM()[2, 0])if key[pygame.K_w]:self.transform.update_position(self.transform.get_position()+ forward * self.pan_speed, False)if key[pygame.K_s]:self.transform.update_position(self.transform.get_position()+ forward * -self.pan_speed, False)if key[pygame.K_a]:self.transform.update_position(self.transform.get_position()+ right * self.pan_speed, False)if key[pygame.K_d]:self.transform.update_position(self.transform.get_position()+ right * -self.pan_speed, False)if key[pygame.K_q]:self.transform.rotate_y(self.rotate_speed)if key[pygame.K_e]:self.transform.rotate_y(-self.rotate_speed)
Now no matter what direction the camera is facing, it will always move in a consistent manner.
The code for this version of the camera is now complete and you are free to run the project and fly the camera around the teapot.
In this section, we investigated a common method used in first-person controllers in games to prevent the camera from going into gimbal lock as this causes erratic rotations. In addition, code was added to the project to implement this method of rotation as well as restrict the camera from pitching too steeply.
Summary
Rotations would have to be the single most cause of headaches for programmers working in 3D environments. By now you will have an appreciation for the complexity of their mathematics. It’s often not until you start using the mathematics that you find out what can go wrong. Only then can you really understand how the errors occur and what you can do to fix them. The issues that affect first-person controller rotations also impact the navigation systems of virtual moving objects such as simulated aircraft and spaceships.
In this chapter, we have explored the mathematics required to move and rotate a virtual camera. This knowledge will allow you to position and angle a camera anywhere in the world to view the objects being rendered. Though the use of several key presses, the camera in your project will now be able to move and rotate. This is essential understanding for any graphics or game programmer, as is being able to eye ball a transformation matrix and have an awareness for the way it will affect the movement of a virtual object.
One thing we did discover in this chapter that has had a monumental impact on graphics mathematics is the limitation of Euler’s representation of matrices, which can result in gimbal lock. The solution to removing this limitation is to work with Quaternions, a powerful mathematical concept that confounds the most learned of programmers working with 3D graphics and games. Even though it is an extremely advanced topic, I will give you a gentle introduction to the topic in the next chapter to the end of elucidating the subject matter and showing you how to work with them.
Answers
Exercise A:
In Camera.py add extra if statements to handle the ‘A’ and ‘D’ key presses and use the x axis to move the camera as follows:
def update(self):
key = pygame.key.get_pressed()
if key[pygame.K_w]:
self.transform.update_position(
self.transform.get_position()
+ pygame.Vector3(0, 0,
self.pan_speed),
False)
if key[pygame.K_s]:
self.transform.update_position
(self.transform.get_position()
+ pygame.Vector3(0, 0,
-self.pan_speed),
False)
if key[pygame.K_a]:
self.transform.update_position(
self.transform.get_position()
+ pygame.Vector3(self.pan_speed,
0, 0),
False)
if key[pygame.K_d]:
self.transform.update_position(
self.transform.get_position()
+ pygame.Vector3(-self.pan_speed,
0, 0),
False)