In order to detect faces, we need to understand what a face is, specifically what a human face is. Think about a typical human face. A typical human face has two eyes, a nose, and a mouth. But having these features isn't enough to define a human face. Dogs also have two eyes, a nose, and a mouth. We are, after all, products of mammalian evolution.

I encourage the reader to think more carefully about what makes a human face. We instinctively know what a face is, but to really quantify exactly what constitutes a face takes work. Often, it may lead to philosophical ruminations about essentialism.

If you watch terrible procedural TV shows, you might see faces being drawn with dots and lines when the detectives on TV are doing facial recognition across a database. These dots and lines are primarily due to the work of Woodrow Bledsoe, Helen Chan, and Charles Bisson in the 1960s. They were among the first people to study automated facial detection. One of the first things noticed is that the standard features of the face—hairline, browlines, gauntness of eyes, height of nose bridge, and so on—are all dynamically definable; that is to say that these features are measured relative to one another. This made automatically detecting features a little bit more challenging than expected.

Their solution was novel: using a device that is an ancestor to today's drawing tablets, annotate the location of eyes, nose, mouth, and other facial features. The distances between these annotations are then used as features for facial recognition. The process today is no different, except a lot more automatic. The works of Bledsoe, Chan, and gang led to an immense effort to quantify how pixels would co-occur to form facial features.

In order to understand the features that make up a face, abstract. What is the minimum possible number of dots and lines required to depict a face? It is instructive to note abstractions in the use of kaomoji. Consider the following kaomoji:

It's quite easy to see that these depict faces. Contrast them with kaomojis that depict other things (fish, spider, gun, and bomb respectively):

The process of abstraction—the act of removing details until only the ones that matter remain—allows one to think more clearly about a subject matter. This is true in art, as it is in mathematics. It is equally true of software engineering, though careful implementation of the abstractions needs to be made. Going back to the kaomojis, note that, even in their highly abstract form, they are capable of displaying emotions. In order of display, the kaomojis show happiness, indifference, love, dissatisfaction, and anger. These abstract depictions offer us a path to think about the facial features in pictures. To determine whether a face exists, we simply determine if those lines are there. The question now becomes how do we take a photo and draw lines?

Start with the facial structure and assume an evenly-lit room. Barring diseases such as Graves which cause proptosis, eyes are generally sunken. This causes the area of the eyes to be shadowed by the brow ridge of the face, as well as cheekbones. In pictures of an evenly-lit face, eyes would appear in shadow. Noses, on the other hand, would appear more brightly lit, because noses are raised compared with the rest of the face. Likewise, lips have a dark area and a bright area, separated by a dark line. These are all useful features to consider when thinking about detecting faces.