1.3.1. Uses of biometrics

Before going into detail concerning the operation of biometrics, it is interesting to consider its applications. The first objective of biometrics is identity verification, that is, to provide proof to corroborate an assertion of the type “I am Mr X”. A facial photograph or fingerprint acts in a similar way to a password; the system compares the image with a pre-recorded reference to ensure that the user is who they claim to be. The second application of biometrics concerns the identification of individuals in cases where their collaboration is not generally required (e.g. facial recognition based on video surveillance footage). Finally, biometrics is often used to secure access to places or tools (premises, smartphones and computers), for border control (automated border crossing systems), by police services (identity control) or for payment security (notably on smartphones), as shown in Figure 1.6.

1.3.2. Definitions

In order to recognize or identify an individual k, reference information R_k must be collected for the individual during an initial enrollment phase. During the authentication/identification phase, a new sample is captured, denoted as E. A biometric system will compare sample E to R_k in an attempt to authenticate an individual k, or to multiple references in a biometric database in cases of identification. A decision is then made (is this the right person?) by comparing the comparison score (in this case, taken as a distance) to a pre-defined threshold T:

The threshold T is defined by the application. In the case of distance, the lower the threshold, the stricter the system is, because it requires a small distance between the sample and the individual’s reference as proof of identity. A strict (high security) threshold will result in false rejections of legitimate users (measured by the FRR, false rejection rate). A looser threshold will result in an increased possibility of imposture (measured by the FAR, false acceptance rate). To set the threshold T for a given application, we consider the maximum permissible FAR for the system; the FRR results from this choice. As an example, consider a high security setting with an acceptable FAR rate of one in a million attempts. In this context, we expect an FRR of less than 2%. The equal error rate (EER) is the error obtained when the threshold is set so that the FRR is equal to the FAR. The EER is often used as an indicator of the performance of a biometric system, although using the associated threshold to parameterize a system is not of any particular practical use; it is simply easier to understand the performance of a system on the basis of a single EER value.

1.3.3. Biometric modalities

There are three main groups of biometric modalities (types of biometric information): morphology (part of the person’s body, such as the face or the iris), behavior (an individual action, such as the voice or the way of signing) and physiology (such as DNA). The first two modalities are the most widespread in transactional contexts due to processing time limitations. These three categories of biometric modalities are illustrated below, represented by DNA, signature dynamics and fingerprints.

Almost any morphological or behavioral characteristic may be considered as a biometric characteristic, as long as it satisfies the following properties (Prabhakar et al. 2003):

• – universality: all people to be identified must possess the characteristic;
• – uniqueness: the information should be as different as possible from one person to the next;
• – permanence: the collected information must remain present throughout the individual’s lifetime;
• – collectability: it must be possible to collect and measure the information in order to permit comparison;
• – acceptability: the system must respect certain criteria (ease of acquisition, rapidity, etc.) in order to permit use.

Not all biometric features have these properties, or they may have them, but to different degrees. Table 1.1, taken from Mahier et al. (2008), compares the main biometric modalities according to the properties listed above. As we see from this table, no characteristic is ideal; different modalities may be more or less suitable to particular applications. For example, DNA-based analysis is one of the most effective techniques for verifying an individual’s identity or for identification (Stolovitzky et al. 2002). However, it cannot be used for logical or physical access control, both due to the computation time and the fact that nobody would be willing to provide a sample of their blood for verification purposes. The choice of modality is thus based on a compromise between some or all of these properties according to the needs of each application. Note that the choice of the biometric modality may also depend on local cultures. In Asia, methods requiring physical contact, such as fingerprints, are not widely accepted for hygiene reasons; contactless methods are more widespread, and more readily accepted, in this setting.

1.4.1. Presentation attacks

There are many ways of attacking a biometric system (Ratha et al. 2001). An attacker may alter the storage of biometric credentials (e.g. replace a user’s biometric credentials in order to spoof the system), or replace a sub-module, such as the decision module, so that it returns a positive response to any attempt. In this section, we shall focus on presentation attacks, which consist of presenting the capture subsystem with biometric data intended to alter the operation of the biometric system. This type of attack can be quite easy to perform, for example by presenting a photo of the user’s face printed on paper. Impostors may also present biometric systems with falsified biometric data (e.g. a gelatin fingerprint), with or without the participation of the individual concerned. One particularly active area of research concerns the development of hardware or software mechanisms to detect this type of attack (Galbally et al. 2019).

The most common attack of this type is carried out on facial recognition systems. Facial recognition technology has come on in leaps and bounds since its invention in the 1970s, and is now the most “natural” of all biometric measures. By the same token, it has become a major focus for hackers. For example, Grigory Bakunov has developed a solution that can confuse facial recognition devices, by designing an algorithm that creates specific makeup arrangements to fool facial recognition software (see Figure 1.8(a)).

In late 2017, a Vietnamese company successfully bypassed the Face ID facial recognition feature of Apple’s iPhone X using a mask (see Figure 1.8(b)).

At the same time, researchers at a German company developed an attack technique to bypass Windows 10 Hello facial authentication. A key element of the attack appears to be taking a picture of the authenticated user with a near-infrared (IR) camera, since Windows Hello uses infrared imaging to unlock Windows devices (see Figure 1.8(c)).

In May 2018, Forbes magazine reported that researchers at the University of Toronto (Canada) had developed an algorithm (privacy filter) that confuses facial recognition software. The software changes the value of specific pixels in the image posted online. These changes, imperceptible to the human visual system (HVS), confuse the recognition algorithms.

One response to these types of attack is to use video rather than still images (Matta and Dugelay 2009). Some operators use interviews, via video conferencing software, to authenticate a person. Unfortunately, new attacks have already been developed for video authentication, and we can expect these attacks to become more sophisticated in the years to come. Video streams can now be manipulated in real time to show the facial reactivity of a counterfeiter on top of another person’s face (Thies et al. 2016), or through face swapping (Bitouk et al. 2008).

Numerous works have been published on this subject, mostly by researchers in the Image Forensics community (Redi et al. 2011; Yeap et al. 2018; Roy et al. 2020); the main approach involves looking for abnormalities in images or flows to identify locations where manipulations have occurred. Modifications are detected on the basis of inconsistencies or estimated abnormalities on image points, inconsistencies in sensor noise, recompressions, internal or external recopies and inconsistencies in terms of illumination or contours. Several technological challenges have been launched by DARPA, IEEE and NIST in the United States and by the DGA in France (including a DEFALS with participation from EURECO, UTT and SURYS) to measure the effectiveness of this type of method. It should be noted that significant progress has recently been made because of deep learning techniques. Passive detection can also draw on knowledge of the particularities of attacks, such as what is known to happen during morphing between two images (Raghavendra et al. 2016), or on the history of operations on the images in question (Ramachandra and Busch 2017).

However, the effectiveness of these countermeasures is beginning to be undermined by advances in inpainting technologies using deep learning, which create highly credible computer-generated images, produced in real time, using just a few photos of the person whose identity is being spoofed and a video stream of the spoofer responding (potentially) to all of the requests of the preceding tests. However, face spoofing can be detected in video streams by focusing on known features of the processed images, such as specific 3D characteristics of a face (Galbally et al. 2014). Evidently, more work is urgently needed in this area.

1.4.2. Acquisition of new biometric data or hidden biometrics

The objective here is to collect known biometric data by new capture methods (3D, multi-spectral (Venkatesh et al. 2019) and motion (Buriro et al. 2019)), or capture new biometric information (for example, the electrical signal from an individual’s body (Khorshid et al. 2020)). The goal is to propose new information which offers improved individual recognition, or which has a greater capacity to detect presentation attacks.

Elsewhere, considerable efforts have been made in recent years in exploring a specific form of biometrics, known as hidden biometrics. The principle consists of identifying or verifying people on the basis of physical characteristics, which are not accessible by traditional techniques, or which are not directly observable or perceivable by humans. This property makes systems particularly robust to attacks.

Hidden biometrics also concerns features that vary over time, that cannot be quantified at a given moment, and which can only be predicted (e.g. variations resulting from aging) or recovered (e.g. by rejuvenation). In this case, we speak of forward or backward prediction.

Certain modalities used in hidden biometrics rely on technologies developed in the fields of medicine or forensic science, particularly for data acquisition. Examples include the use of electrocardiograms (ECG), electroencephalograms (EEG) or electromyographs (EMG), involving a variety of imaging techniques (infrared, thermal, ultrasound, etc.) (Nait-Ali 2019a, 2019b).

In this section, we shall focus on three modalities used in hidden biometrics, namely human brain biometrics, hand biometrics and digital facial aging/rejuvenation.

In 2011, researchers showed that a biological signature, the “Braincode”, can be obtained from the human brain and used to distinguish between individuals. Both 2D and 3D processing approaches have been explored, using images obtained by magnetic resonance imaging (MRI). In the 2D approach, one idea is to extract biometric features from a single specific axial slice, as shown in Figure 1.9. Defining a region of interest (ROI) in the form of a crown, using an algorithm similar to the one used in iris biometrics, a recognition rate of around 98.25% can be achieved. In the 3D approach, the whole volume of the image obtained via the MRI scan is used in order to obtain a Braincode. The idea is to explore the whole volume image obtained by MRI to extract the Braincode. In an article published in Aloui et al. (2018), the envelope of the brain was estimated, highlighting the structure of convolutions, as shown in Figure 1.10.

While this modality cannot currently be used for practical applications, notably due to its technical complexity, cost and low level of user acceptability, future uses are not to be excluded.

Palm biometrics in the visible or infrared range (vein biometrics) are potentially vulnerable to attack. One reason for this relates to the superficiality of features extracted from the region of interest.

Technically, this risk can be considerably reduced by using a modality based on X-ray imaging. In this context, experiments have been carried out on many samples; researchers have shown that a biometric signature can be extracted by modeling the phalanges of the hand (see Figure 1.11 (Kabbara et al. 2013, 2015; Nait-Ali 2019a)).

In the algorithm in question, the image is segmented in order to highlight all of the phalanges. Each phalanx is then modeled using a number of parameters, which are then concatenated to create a biometric signature. Evidently, this approach raises questions concerning the impact of X-rays on user health. The study in question took the recommendations of the National Council on Radiation Protection and Measurements (NCRP) into account, limiting the radiation dose of the systems to 0.1 μSv/scan to ensure user safety.

1.4.3. Quality of biometric data

The quality of biometric data is not always easy to estimate. While quality metrics have been established for morphological modalities such as fingerprints (Yao et al. 2016b), much work is still needed in the case of behavioral modalities.

Work carried out in recent years has highlighted the importance of sample quality for recognition systems or comparison algorithms. The performance of a biometric system depends, to a great extent, on the quality of the sample image. Over the last decade, many research works have focused on defining biometric data quality metrics for the face (Nasrollahi and Moeslund 2008; Wasnik et al. 2017), vein networks (Qin and El Yacoubi 2017) and, especially, fingerprints (Tabassi et al. 2011; Yao et al. 2015a; Liu et al. 2016).

The development of a quality measurement for biometric data centers on an objective demonstration of the superiority of one indicator over others. In the case of image quality, the aim is to develop an algorithm that assigns quality ratings that correlate perfectly with human judgment; in biometrics, a quality measurement must combine elements of image quality with elements relating to the quality of the extracted biometric characteristics, ensuring that a system will perform well. In this case, the working framework is different and the real-world situation is not fully known, and this can prove problematic.

Yao et al. (2015a) have proposed a methodology for quantifying the performance of a quality metric for biometric data. Their approach is generic, and can be applied to any modality. The method estimates the proximity of a metric to an optimal judgment.

1.4.3.2. Metric behavior

Twelve biometric databases from the FVC competition (Maltoni et al. 2009) were used to study the behavior of metrics: FVC 2000 (DB1, DB2, DB3, DB4), FVC 2002 (DB1, DB2, DB3, DB4) and FVC 2004 (DB1, DB2, DB3, DB4). Five additional synthetic fingerprint databases of different qualities were also generated using SFINGE (Cappelli et al. 2004): SFINGE0 (containing fingerprints of varying quality), SFINGEA (excellent quality), SFINGEB (good quality), SFINGEC (average quality) and SFINGED (poor quality).

Seven current fingerprint quality metrics were tested:

1. 1) NFIQ: this metric classifies fingerprint images by five quality levels, based on a neural network (Tabassi et al. 2011). This metric has served as the industry standard for the past 15 years, and is included in all commercial biometric systems.
2. 2) NFIQ 2.0: Olsen et al. (2013) trained a two-layer self-organizing map (SOM neural network) to obtain a SOM unit activation histogram. The trained characteristic is then input into a random forest in order to estimate genuine matching scores. NFIQ 2.0 is the new ISO standard for measuring fingerprint quality.
3. 3) OCL: Lim et al. (2002) developed a quality measure based on a weighted combination of local and global quality scores, estimated as a function of several characteristics, such as the orientation certainty level.
4. 4) QMF: this metric is calculated by considering several different aspects, such as (1) the fingerprint image itself (using blind image quality evaluation and texture functions) and (2) the associated minutia model (Yao et al. 2015a). The quality metric is implemented via a linear combination of quality characteristics.
5. 5) NBIS: this quality indicator is a simple measure, based on the quality of minutia extracted by the NIST NBIS program (Ko 2007). The metric corresponds to the mean value of the quality of the minutia in the model.
6. 6) MSEG: this measure is based on cutting out the pixels from the foreground of a poor-quality image (Yao et al. 2016a).
7. 7) MQF: an original metric that calculates a quality score based on a minutia model (Yao et al. 2015b). Metrics of this type present a significant advantage for biometric systems integrated into chips or smart objects, where computation and storage constraints in the secure part of the chip mean that only the minutia model is available.

Table 1.2 shows the quality score obtained from the metrics tested on the first sample of four datasets. Note that, for NFIQ, a low value corresponds to high fingerprint quality, while for all other metrics a high score indicates good performance. The evaluation results seem to be effective. For example, almost all metrics identify a fingerprint from the SFINGEA (high quality) database as being of the best quality. We also note significant differences in scores between the metrics, especially between the image from the FVC2002DB3 database and the SFINGED database. However, since the metrics have values in very different ranges, these results are not directly comparable.

Dataset	NFIQ	NFIQ2	OCL	QMF	NBIS	MSEG	MQF
FVC2000DB1	2	65	0.73	83.81	14.16	0.44	59 802
FVC2000DB3	4	40	0.71	28.06	15.11	0.18	29 804
SFINGEA	1	69	0.90	76.09	57.46	0.83	55 720
SFINGED	3	28	0.47	91.19	10	0.006	43 546

Many further avenues remain to be explored in order to develop better quality metrics for fingerprint samples.

1.4.4. Efficient representation of biometric data

Once biometric data has been collected and authenticated, the best possible representation must be extracted in order to make comparisons. For many years, the search for relevant parameters to characterize biometric data focused on signal attributes and images (Wu et al. 2019). Recently, however, the use of convolutional neural networks has revolutionized the field, and statistical learning approaches are becoming increasingly widespread (Parkhi et al. 2015). Current research aims to generalize this type of approach to all biometric modalities, even in cases where large databases are not available.

The performance of biometric systems has progressively improved since the 2000s because of advancements in electronics (more powerful sensors), computing (more powerful computers) and algorithmics (more sophisticated data processing techniques). These three components are closely linked within the classical architecture of a biometric system; in other words, the failure of one of these three essential components can seriously compromise the overall performance of the system. Considering the algorithmic aspect, the last few years have seen significant changes due to the success of deep learning methods in various image processing and data analysis applications. The generic structure of biometric systems has been modified to include the parameter extraction phase within a multi-layered neural architecture, such as that shown in Figure 1.15. This architecture has resulted in considerable improvements in the performance of biometric systems, reliant on access to powerful computers equipped with GPU, for example, and on the availability of large databases for learning purposes. Learning mechanisms result in the creation of robust models that are easy to use in the test phase.

In this context, deep learning has been successfully applied to a range of biometric modalities:

1. 1) Facial recognition (performance ∼99%): DeepFace (AlexNet model), DeepID, DeepID2, DeepID3 (Modèle VGGNet-10), VGGface (VGGNet-16 model), FaceNet (googLeNet model).
2. 2) Fingerprint recognition (performance ∼95% to ∼98%): FingerNet, DeepCNN.
3. 3) Palmprint recognition (performance > 99%): Deep Scatering, MobileNetV2 + SVM, Deform-invariant.
4. 4) Iris recognition (performance ∼99%): DeepCNN, Deep Scatering, Deep Features.
5. 5) Signature recognition (performance 81–93%): Embending, SIGAN.
6. 6) Gait recognition (performance 68–95%) : Yan et al., Li et al., Zhang et al.

More details may be found in published “survey” articles, such as in Minaee et al. (2019).

1.4.5. Protecting biometric data

Biometric data are, by definition, personal data. The General Data Protection Regulation (GDPR) of 2018 reinforces the obligation to protect individual privacy (Voigt and Von dem Bussche 2017), and, evidently, covers biometric data. These data must be protected for both security and user privacy reasons. Solutions are therefore needed, both in terms of protection (Atighehchi et al. 2019) and for evaluating attack resistance (Gomez-Barrero and Galbally 2020).

1.4.5.1. State of the art

There are several possible approaches to protecting biometric data:

• – Classic encryption: classic approaches to encrypting biometric data usually rely on symmetric (e.g. AES) and asymmetric (e.g. RSA) cryptosystems. This approach has the disadvantage of needing to decrypt the biometric reference for comparison with a capture, making it vulnerable during this time. This approach is necessary (especially for storage in a biometric database) but not sufficient, considering the lifetime of a biometric data element (an algorithm which is secure at present may be broken during an individual’s lifetime). This is the minimum that is expected of a biometric system.
• – Storage in a secure element: this approach consists of storing clear or, preferably, encrypted biometric data in a hardware enclave such as a microcircuit chip (Wang et al. 2018). This method is classically used in smartphones, for example, where a micro-circuit chip may be linked to a fingerprint sensor. This chip stores the biometric reference and carries out the comparison with a new capture (see Figure 1.16). A hardware enclave guarantees very high resistance to physical and logical attacks (Maciej et al. 2019; Im et al. 2020). This type of approach also allows the individual to remain in possession of their own biometric data (in a smartphone or passport).
• – Cancelable biometrics: one major criticism of biometrics is the intrinsic non-revocability of biometric data. For instance, a fingerprint cannot be changed in the same way as a password. The aim of cancelable biometrics is to regenerate a biometric signature for an individual, even in cases of interception by an attacker, without impinging on their privacy. The general principle is to apply a non-invertible transformation of the biometric data, parameterized by a secret key (represented by a random number or a password (Lacharme and Plateaux 2011)), as illustrated in Figure 1.17. This type of approach is similar to two-factor authentication (combining biometrics and secret information). Note that knowledge of the secret information alone is not sufficient for an attacker to retrieve the initial biometric data, but may be sufficient to permit an attack (e.g. imposture) in certain cases (Lacharme et al. 2013). The verification step is relatively simple, requiring comparison with a distance (notably Hamming). Diversifying keys allows one individual to possess several biometric signatures using the same biometric data, preventing an attacker from accessing multiple digital services using the same individual identity.
• – Secure computation: this approach aims to offer protection for secure storage of biometric data (particularly in the Cloud) and permit identity verification without having to decrypt an individual’s biometric reference. Homomorphic encryption (Barrier 2016) is one possible approach used here. However, while biometric data elements are often small in size (such as a fingerprint, which may require less than 200 bytes), homomorphic encryption results in much larger signatures (several megabytes), and this limits its use for practical applications.

1.4.6. Aging biometric data

Biometric data do not always remain constant over the course of time. While fingerprints change little, an individual face can vary considerably. Behavioral biometric data are also subject to intrinsic intraclass variation that is difficult to manage in terms of recognition. This raises the need for new biometric systems with the capacity to manage data aging, in order to ensure consistently high performance, in terms of recognition, over the course of time (Pisani et al. 2019).

Despite the remarkable progress made in deep-learning based facial recognition approaches in recent years, in terms of both verification and identification performance, the neural architecture still has limitations. These limitations relate to the database used in the learning phase. If the selected database does not contain enough instances, the result may be systematically affected. For example, the performance of a facial biometric system may decrease if the person to be verified or identified was enrolled over 10 years ago. In adult individuals, aging results in changes to the texture of the face, notably with the appearance of wrinkles and skin sagging due to a loss of elasticity. These changes may be accentuated by weight gain or loss. In cases where enrollment is performed on a young child, verification/identification of the same individual’s identity during adolescence or even adulthood using a static or “invariant” facial recognition system may fail. To counteract this problem, researchers have developed models for facial aging or digital rejuvenation, and work in this area is still ongoing. Both GAN (Generative Adversarial Network) and statistical models have produced impressive results in terms of perception. Digital rejuvenation or aging is used to compensate for the differences in facial characteristics, which appear over a given time period. This feature can be incorporated into facial recognition systems in order to improve their performance.

Figure 1.18 shows virtual faces at different stages of adult life. Figure 1.19 shows changes in appearance from childhood through adolescence and into adulthood. As Farazdaghi and Nait-Ali (2017) have indicated, the appearance of a face at different times can be predicted using specific mathematical models, which combine a statistical element with a parametric model, of which the parameters are estimated using geometric transformations based on anthropometric measurements. The model can be extended to process 3D digitized faces, as shown in Figure 1.20. Initial results for applications in a facial verification context can be found in Heravi et al. (2019).