4.1.1 Pupil Dilation

There are several factors that negatively impact the matching accuracy of an iris recognition system. These include out-of-focus imaging, occlusions, specular reflections, image blur, non-uniform illumination, sensor noise, and pupil dilation. While most of these factors are a result of human–sensor interaction, pupil dilation is a result of ambient factors such as intensity of visible light entering the eye, psychological factors such as stress, and chemical factors such as alcohol intake [1] and drugs. Eye drops can also regulate the size of the pupil. Pupil dilation is measured in terms of pupil dilation ratio (ρ) which is the ratio of pupil radius over iris radius:

$\rho =\frac{\mathit{pupil}\mathit{radius}}{\mathit{iris}\mathit{radius}}.$

ρ typically varies from 0.2 to 0.8 in humans [5]. Smaller values of ρ indicate pupil constriction while larger values of ρ indicate pupil dilation. Fig. 4.3 shows two iris images of the same person at different visible illumination levels with a large difference in pupil dilation ratio.

Traditionally, the region between the pupillary and limbic boundaries is linearly sampled in order to unwrap it into a rectangular grid. This is referred to as the rubber sheet model [5]. It is often assumed, for the purpose of linear sampling, that the pupillary and limbic boundary are circular. One implementation of the rubber sheet model samples pixels radially along a fixed angular direction at regular intervals and maps them into a single column in the normalized image. This is repeated across multiple angular directions corresponding to different angular values. It was previously assumed that this type of linear sampling would be sufficient to handle changes in iris due to variation in pupil size. However, as can be seen in Fig. 4.4, employing the linear sampling of images in Fig. 4.3 results in different outputs.

The iris texture undergoes complex 3-D deformation when the pupil dilates or constricts. Hence, the linear normalization technique employed in Daugman [5] may not be a suitable method to match two images of the same iris with substantially different pupil sizes [7].

There are multiple ways to address this problem. These are pictorially represented in Fig. 4.5 and briefly discussed below.

Solutions to the problem of matching iris images with disparate pupil size may be incorporated at different stages of an iris recognition system, viz., image acquisition, normalization, encoding, and matching. Each of these stages could be modified either independently or in conjunction with the other stages to improve the iris matching performance.

Image Prediction. A deformation model based on the biology of the iris and the pupil could be used to predict how the iris texture would change as a function of pupil radius. Once the deformed iris pattern is predicted, subsequent processing steps such as normalization, encoding and matching can be applied without any modification. Examples of such an approach can be found in Wyatt [17] and Clark et al. [2].

Non-linear Sampling. A typical normalization technique radially samples the iris linearly in each angular position. The sampling scheme can be modified to better match the two normalized images [15]. Image prediction models could be potentially used to guide the non-linear sampling method.

New Encoding Method. A new feature encoding method can be developed to better match iris images with disparate pupil sizes. A corresponding matching method may have to be developed depending on the type of extracted features.

In this work, we propose a multi-scale fusion scheme where the IrisCodes generated using multiple Gabor filters are sequentially fused at the bit-level. The proposed scheme and its rationale is described in detail in Section 4.5.

4.1.2 Layout

This chapter is organized as follows. Section 4.2 discusses previous work on the topic of modeling the iris deformation due to pupil dilation and also on other types of IrisCode bit matching schemes. Section 4.3 describes the WVU Pupil Light Reflex (WVU-PLR) dataset that was assembled for the purposes of this research. Section 4.4 uses this dataset to demonstrate the negative influence of pupil dilation on iris recognition performance. The proposed method to mitigate the effect of pupil dilation is presented in Section 4.5. The experimental results are summarized in Section 4.6, while conclusions and future work are presented in Section 4.7.

4.2.1 Pupil Dilation

The previous work on this topic may be broadly divided into three categories based on their end goals. The first line of work tried to model the dynamics of iris deformation by deriving a theoretical model to understand the deformation process. The second line of work only focused on improving the iris matching performance in presence of pupil dilation without explicitly accounting for the biological basis for the associated deformation. The third category of work only focused on documenting the effects of pupil dilation.

Theoretical models include (i) minimum wear and tear model [17], where the iris tissue is assumed to be a network of orthogonal fiber like structures, and (ii) mechanical stress and strain model [2], where iris is assumed to be an elastic material enclosed between two concentric boundaries and the force is assumed to be exerted radially outwards from the pupillary boundary, while keeping the limbic boundary fixed. Wyatt's model [17] represented the iris deformation caused by pupil dilation as a sum of a linear term and a non-linear term, i.e.,

$R=R_{\mathit{linear}}+\mathrm{\Delta }R(p,r),$

where $R_{\mathit{linear}}$ is the solution to a linear deformation model and ΔR is the additional displacement of a point in the iris after application of the linear model. ΔR is approximated using a 6th order polynomial.

Several techniques have been published to improve the matching accuracy when comparing iris images with different pupil size. For example, Yuan and Shi [18] used the minimum wear and tear model to derive an equation that predicted the displaced location of a point in the iris region after dilation. In another work, Wei et al. [16] approximated the non-linear term in Wyatt's model, $\mathrm{\Delta }R(p,r)$, using a Gaussian distribution that in turn is learned from a training set. Thornton et al. [14] divided the normalized irises into a set of non-overlapping blocks, and computed transformation parameters between corresponding blocks of the images to be compared. Then the posterior probability of these parameters was maximized iteratively resulting in the optimal deformation parameter set. This information was used to compute block-wise similarity metrics that were averaged to produce a final score.

Pamplona et al. [10] acquired extremely dilated images of a few eyes by administering mydriatic drugs that dilated the pupil. Specific points in the iris region were then manually annotated and tracked across the images. It was observed that points are displaced predominantly in the radial direction and structures such as crypts deform also in the angular direction. Hollingsworth et al. [7] demonstrated that pupil dilation degrades iris matching performance by grouping the images based on pupil dilation ratio and comparing them against each other.

4.2.2 Bit Matching

There have been other methods developed that exploit the characteristics of IrisCode bits to improve the performance of iris recognition. Hollingsworth et al. [8] used a matching scheme where only the “best bits” in an IrisCode are used. These bits are chosen based on their stability, i.e., they retain their value across different samples of the same eye. Rathgeb et al. [11] employed a selective bit matching scheme by comparing only the most consistent bits in an IrisCode. These consistent bits were obtained by using different feature extractors. In another work, Rathgeb et al. [12] proposed a new distance measure based on Hamming distance values that are generated when matching two IrisCodes at different offsets. In Short Length IrisCode (SLIC) [6], IrisCodes were compared using one row at a time, thereby allowing for faster exclusion of potential non-matches.

4.5.1 IrisCode Generation

IrisCodes can be generated by applying multi-scale filters on a normalized iris image and quantizing the complex output. One such implementation by OSIRIS applies filters of three different sizes. Each filter produces two bits of IrisCode per pixel.

Let the ith image be denoted by $I_i$. Its normalized image is denoted as $N_i$. The size of normalized image is $r\times t$ where r is the radial resolution and t is the angular resolution. Three rectangular complex filters $F_{m_1\times n_1}^1$, $F_{m_2\times n_2}^2$, and $F_{m_3\times n_3}^3$ are applied on the normalized image. The resulting complex output is then converted to a binary IrisCode set ${(C_i^1,C_i^2,C_i^3)}_{r\times 2t}$ along with a mask $M_{i_{r\times 2t}}$. Fig. 4.10 pictorially shows an IrisCode set.

Normalized image with size $r=64$ and $t=512$ for filter sizes $9\times 15$, $9\times 27$, and $9\times 51$ are used in this work. Fig. 4.11 shows a normalized iris image and its corresponding IrisCode generated using the 3 complex filters. The smallest filter encodes smaller regions in the image and the largest filter encodes larger regions in the image. This is reflected in the smoothness of IrisCodes at different filter sizes. Larger filter results in a smoother IrisCode compared to smaller filter.

4.5.2 Typical IrisCode Matcher

Suppose that IrisCode sets generated from two normalized images $N_i$ and $N_j$ are being matched. The IrisCode sets are represented as $(C_i^1,C_i^2,C_i^3,M_i)$ and $(C_j^1,C_j^2,C_j^3,M_j)$, respectively. A common mask, $M_{ij}$, is computed to denote the location of common valid bits in both the IrisCodes.

$M_{ij}=M_i\bigcap M_j.$

Let the result of XOR operator, ⨂, for matching individual IrisCodes at filter F be $R^f$ given by

$R_{ij}^f=C_i^f⨂C_j^f,f=1,2,3.$

⨂ results in 0 if the corresponding bits are the same and 1 if they are not. Hamming distance between two IrisCodes at the fth filter scale is then given by

$H{D}_{ij}^f=\frac{\Vert R_{ij}^f\bigcap M_{ij}\Vert }{\Vert M_{ij}\Vert },f=1,2,3.$

Typically, the Hamming distances computed for each filter are fused using sum rule to produce a final matching score.

$D_{i{j}_{\mathit{sum}}}=H{D}_{ij}^1+H{D}_{ij}^2+H{D}_{ij}^3.$

The above described steps employed by a typical iris matcher are presented in the form of a flow chart in Fig. 4.12.

4.5.3 Multi-filter Matching Patterns

Three filter outputs are available at each pixel location in an iris image. Hence three filter matching results ($r^1,r^2,r^3$) are generated at every location when two IrisCode sets are matched. These three results at each location may be combined and represented as a single vector, R, which is hereby referred to as the matching bit pattern at every bit location. It can have values from $000,001,010,\dots ,111$. Here, 000 at a specific location would mean that the pixel matches at all the filter scales; 100 would mean that although the pixel is mismatched by filter 1, it is matched by filter 2 and filter 3. Similarly 111 would indicate that the pixel is mismatched at all filter scales.

Fig. 4.13 shows the distributions of these matching patterns for a few cases of intra-class matching corresponding to one subject. The legend in the plots denotes the size of pupil radius in pixels of the two images that are being matched. It is observed that the percentage of 000s (matched by all filters) decreases with increase in difference of pupil dilation ratios between the matched samples. Fig. 4.14 shows distributions of multi-filter matching patterns for a few randomly selected inter-class (impostor) pairs in the dataset. It is observed that the distribution of these decisions is roughly equal and similar across the decision patterns.

In a traditional sum rule matcher, the instances of $000,001,\dots ,111$ are summed up and divided by the total number of locations. However some interesting patterns are observed in the distribution of these multi-filter matching patterns for genuine and impostor matching cases. Fig. 4.15 shows the distribution of these matching results from three different filter scales for genuine and impostor matching cases.

It is observed from Fig. 4.15 that some matching patterns such as 000, 011, 101, 110, and 111 are much more discriminative compared to others. Hence, these filter decisions could be selectively fused to provide better performance.

4.5.4 Proposed IrisCode Matcher

The idea behind the proposed method is to make a matching decision at each pixel location based on information at multiple scales. The distribution of decision patterns shown in the previous section is exploited to come up with a better decision strategy. IrisCode bits generated from multiple filters are selectively matched to compute a final dissimilarity score. This is pictorially depicted in Fig. 4.16.

Multiple decision making strategies can be developed to allow for stricter or easier matching conditions. The proposed matching strategies are described below.

Method 1. Two iris images $(I_i,I_j)$ are first matched using IrisCodes generated by filter 1 at each bit location, $r^1=(c_i^1,c_j^1)$. If the images are not matched by filter 1, i.e., $r^1=1$, then the matching is extended to IrisCodes generated by larger filters 2 and 3. The bit location is deemed a match, if IrisCodes are at least matched by filters 2 and 3. This helps in handling local deformations since match is established at a larger scale for those bits that would have otherwise mismatched at smaller scales.

Method 2. This method relaxes the conditions for a match. If two IrisCodes are not matched at the lowest scale, an additional opportunity is provided at medium scale filter 2. In case IrisCodes are not matched by filter 2, then a final opportunity is afforded by larger filter 3.

Method 3. This method provides a stricter matching criterion by requiring the IrisCodes to match at filter 1 as well as either filter 2 or filter 3. This method removes the possibility of matching locally deformed regions. Only those regions that are matched at multiple scales are deemed a match.

The logical operations shown in Figs. 4.17, 4.18, and 4.19 are used in a sequential fusion step in Fig. 4.16 and can be implemented using a single Boolean expression. Corresponding truth tables are used to derive the Boolean expression that directly computes the final result based on the decisions at each scale. Hence, a single decision is made, i.e., $r=0$ (match) and $r=1$ (non-match), at each bit location in an IrisCode. The final decision will be equivalent to applying a single complex filter on the normalized image. Let the final matching decision bits be presented as a matrix R. Hamming Distance between two IrisCode sets $(C_i^1,C_i^2,C_i^3,M_i)$ and $(C_j^1,C_j^2,C_j^3,M_j)$ is then given by

$D_{ij}=\frac{\Vert r_{ij}\bigcap M_{ij}\Vert }{\Vert M_{ij}\Vert }.$

Table 4.3 shows the logical operations for these three methods along with the simple sum rule fusion.