A very powerful and popular technique among real-time 3D applications is normal mapping. Normal mapping creates the illusion of highly detailed geometry on a low-poly model by storing surface normals in a texture map that can then be used to calculate the lighting of the objects. This method is especially popular in modern games, since this allows developers to strike a balance between high performance and scene detail.
Typically, lighting is calculated by using the surface normals of the triangles being rendered, meaning that the entire polygon will be lit as a continuous, smooth surface:
With normal mapping, the surface normals are replaced by normals that are encoded in a texture that give the appearance of a rough or bumpy surface. Note that the actual geometry is not changed when using a normal map – only how it's lit changes. If you look at a normal mapped polygon from the side, it will still appear to be perfectly flat:
The texture used to store the normals is called a normal map, and it's typically paired with a specific diffuse texture that complements the surface that the normal map is trying to simulate. For example, here is a diffuse texture of some flagstones and the corresponding normal map:
You can see that the normal map contains a similar pattern to the diffuse texture. Together, the two textures give the appearance that the stones are raised with a rough finish, while the grout is sunken in.
Although normal mapping is a powerful technique for efficiently adding more detail to assets, there are many other mapping techniques that follow the same line of reasoning. You can read about some of the other techniques that are available for use here: https://en.wikipedia.org/wiki/Category:Texture_mapping.
The normal map contains custom-formatted color information that can be interpreted by the shader at runtime as a fragment normal. A fragment normal is essentially the same as a vertex normal: it is a three-component vector that points away from the surface. The normal texture encodes the three components of the normal vector into the three channels of the texture's texel color. Red represents the x-axis, green represents the y-axis, and blue represents the z-axis.
The normals that have been encoded are typically stored in tangent space, as opposed to world or object space. Tangent space is the coordinate system for the texture coordinates of a face. Normal maps are commonly blue, since the normals they represent generally point away from the surface and thus have larger z components.