This recipe will operate not only with vertex position coordinates but also with normal vectors. Normal vectors are important as they specify the facing direction of polygons. These normal vectors are applied on triangles usually, but they can be used on vertices.

First, you'll need to obtain normal vectors. If you don't already have them, you can compute them for each triangle. Let's say that the triangle contains three vertices v1, v2, and v3. Each vertex has three coordinates: x, y, and z. A normal vector can be defined as a cross product of two vectors that define edges of the triangle. You can use triangle vertices v1, v2, and v3 to obtain these two vectors U and V. Finally, the U and V vectors can be used to compute the normal vector. The whole procedure is explained in the following lines, for which, first you need to obtain the U and V vectors:

U.x = v2.x - v1.x
U.y = v2.y - v1.y
U.z = v2.z - v1.z

V.x = v3.x - v1.x
V.y = v3.y - v1.y
V.z = v3.z - v1.z

Now you can use these vectors to compute the normal vector N:

N.x = U.y*V.z - U.z*V.y
N.y = U.z*V.x - U.x*V.z
N.z = U.x*V.y - U.y*V.x

The last thing you'll need to do is normal vector normalization. It means that the vector will always have a length of 1 unit. You can perform normalization by first obtaining the current length of the normal vector:

length = math.sqrt(N.x*N.x + N.y*N.y + N.z*N.z)

Now, you'll need to divide each normal vector coordinate with this length:

N.x = N.x/length
N.y = N.y/length
N.z = N.z/length

Finally, with normal vector N, you can draw a triangle:

gl.Begin(gl_enum.GL_TRIANGLES)
  gl.Normal3f(N.x, N.y, N.z)
  gl.Vertex3f(v1.x, v1.y, v1.z)
  gl.Vertex3f(v2.x, v2.y, v2.z)
  gl.Vertex3f(v3.x, v3.y, v3.z)
gl.End()

From now, this triangle will be able to reflect light because OpenGL knows in which direction the triangle is facing. In some cases, you'll need to change the direction of the normal vector to the opposite direction in relation to a different order of vertices, which is defined by winding. This may be important if you change the definition of the front-facing and back-facing polygons with the gl.FrontFace function. By default, OpenGL uses gl.FrontFace(gl_enum.GL_CCW), which means that the front face of the triangle is the one facing the normal vector that is computed from the vertices submitted in the counterclockwise direction.

A light source can be defined with three basic properties—diffuse color, specular color, and ambient color. These properties can be set with the following code:

local light_source = gl_enum.GL_LIGHT0
local light_diffuse =  {1, 1, 1, 1}
local light_specular = {1, 1, 1, 1}
local light_ambient =  {0, 0, 0, 1}

gl.Lightfv(light_source, gl_enum.GL_DIFFUSE, light_diffuse)
gl.Lightfv(light_source, gl_enum.GL_SPECULAR, light_specular)
gl.Lightfv(light_source, gl_enum.GL_AMBIENT, light_ambient)

This will produce a white light source. Each light source has its own identifier gl_enum.GL_LIGHTx, where x is a number from 0 to 7. You can turn the light source on or off with the following command:

gl.Enable(light_source) -- to turn on
gl.Disable(light_source) -- to turn off

You can also set other light source parameters such as light source position or direction, light spot size and direction, and light attenuation. These parameters can be set by using the following code:

local light_position = {0.2, 0.2, 0.4, 1.0}
local light_spot_direction = {0, 0, 0}
local light_attenuation_constant = 1
local light_attenuation_linear = 0.04
local light_attenuation_quadratic = 0.08
local light_spot_cutoff = 120

gl.Lightfv(light_source, gl_enum.GL_POSITION, light_position)
gl.Lightfv(light_source, gl_enum.GL_SPOT_DIRECTION, light_spot_direction)
gl.Lightf(light_source, gl_enum.GL_CONSTANT_ATTENUATION, light_attenuation_constant)
gl.Lightf(light_source, gl_enum.GL_LINEAR_ATTENUATION, light_attenuation_linear)
gl.Lightf(light_source, gl_enum.GL_QUADRATIC_ATTENUATION, light_attenuation_quadratic)
gl.Lightf(light_source, gl_enum.GL_SPOT_CUTOFF, light_spot_cutoff)

Special care should be taken to light position the vector. As you can see, it uses four coordinates x, y, z, and w. The last coordinate w can be set to 0 or 1. A zero value means that the light source is directional. This can be used for light panels, big screens, and so on. If the w coordinate equals to 1, the light source is positional. This is often viable for light bulbs, car lights, or spot lights.

The final vertex color is defined by this equation:

C = global_ambient * material_ambient + material_emission

for each light L
  C = C + light_attenuation * spotlight_factor * (light_ambient * material_ambient + light_diffuse * material_diffuse * normal . vectorLC + light_specular * material_specular * (normal . vectorRV) ^ shininess
  )

Every light on the scene contributes to the final vertex color. This equation uses the dot product with the vertex normal vector to determine the surface light intensity. In the first case, it's used with vectorLC, which contains direction of the light source to the vertex position. As you can see, camera orientation does not play any role in diffuse lighting.

In the second case, the dot product is used together with vectorRV, which contains the camera orientation vector. This is used to produce a shiny surface effect and it can be controlled with the shininess value. A lower shininess value will result in brighter light reflection.

The light attenuation value is based on the following equation:

light_attenuation = 1 / (a + b*r + c*r^2)

The distance between vertex and the light source is represented by the r variable. Other variables: a, b, and c are attenuation factors:

Attenuation determines how much will the light intensity decrease with increasing distance between the light source and object.

If your object looks too "blocky" with lighting, you might need to apply some kind of interpolation for normal vectors. This recipe applies them for whole polygons. You can use linear interpolation to obtain normal vectors for each vertex. The normal vector for the vertex can be computed in the following fashion. Let's assume that you want to compute the vertex normal vector for the V1 vertex, which is a part of the triangle with the normal vector N1. This vertex V1 is also a part of an other two triangles with normal vectors N2 and N3. A new normal vector N1V1 for vertex V1 will be computed as follows:

N1V1.x = (N1.x + N2.x + N3.x)/3
N1V1.y = (N1.y + N2.y + N3.y)/3
N1V1.z = (N1.z + N2.z + N3.z)/3

Now, when you compute normal vectors for each vertex, you can submit them for each vertex and you'll get smooth surface with lighting.