20.2 Dimensionality Prioritization

DRT takes advantage of transformed components produced by a custom-designed transformation to represent the original data in a new transformed data space with a different data representation in which each data dimension is specified by a particular transformed component. The DR is then performed by retaining a prescribed number of transformed components, q. Accordingly, the effectiveness of DRT is determined by three key factors: the DR transformation being used to produce transformed components, significance of transformed components measured by a selected information criterion, and the value of q.

Using the commonly used PCA as an example, we can illustrate these three issues as follows. First of all, PCA transforms the original 2D spatial/1D spectral data coordinate system into a new data representation system formed by a set of PCs, each of which is characterized by a specific eigenvector that corresponds to a particular eigenvalue, that is, a sample data variance. Then, the significance of each PC is further measured by the magnitude of the eigenvalue corresponding to the eigenvector that specifies this particular PC. In other words, each dimension in a PCA- transformed data space is no longer a wavelength-specified spectral dimension in the original data space. That is, the original data represented by the wavelength-based spectral dimensionality can be reduced via PCA to a small number of PCs specified by eigenvectors corresponding to large eigenvalues in a PCA-transformed data space. Finally, the third issue of “how many PCs are required for such a PC-based data representation, that is, what is the value of q?” can be determined by the virtual dimensionality (VD) developed in Chapter 5.

While PCA enjoys all the nice and desired properties such as eigenvalues and eigenvectors described above, unfortunately other DR transformations do not have such luxury. For example, the independent component analysis (ICA) (Hyvarinen et al., 2001) does not have all the properties of PCA described above. Most importantly, ICA does not have an analytic equation similar to the characteristic polynomial equation obtained for PCA from the sample covariance matrix, which that can be used to find eigenvalues. To resolve this issue, it must find projection vectors directly from the data that serve as the same purpose of eigenvectors for PCA to produce PCs to produce ICs. In doing so, a general approach is to design an algorithm to generate the desired projection vectors. To initialize such an algorithm, a common practice to randomly generate a vector as an initial projection vector that will eventually lead to desired projection vectors. However, as a consequence of using a random vector as an initial projection vector, its final converged projection vector may not be repeatable. In other words, a final converged projection vector produced by one random initial vector may be different from that produced by another random initial vector. Such randomness issue has been addressed in endmember extraction (see Chapters 9 and 10) and is also encountered in the ISODATA (C-means) clustering method in Duda and Hart (1973). Secondly, once ICs are generated, the issue of how to measure the significance of information contained in each of ICs must be addressed, since there is no counterpart of eigenvalues used by PCA in ICA that can be used to rank the generated ICs in terms of information significance. So, it leads to two challenging problems for ICA to resolve. One is to find an appropriate approach that can produce desired projection vectors. The other is to rank the orders of ICs in accordance with significance of their provided information. The PIPP and DP are developed in this chapter exactly for this purpose where PIPP generalizes the concepts of PCA and ICA by introducing PI as a criterion to identify a direction of interestingness in which case PIPP is reduced to PCA and ICA when PI is specified by data variance and mutual information, respectively, and DP then uses an information criterion that can be the same PI or another PI to measure the significance of the PI-generated components for their priority ranking. By combining PIPP with DP, we can further derive a progressive spectral dimensionality process (PSDP) that implements two dual processes, PSDR-PIPP and PSDE-PIPP, to perform dimensionality reduction and dimensionality expansion, respectively.