4.1 Introduction

When a new algorithm is designed and developed, a frequently asked question is that “How does it perform compared to other algorithms?”. In other words, if one walks in with a new algorithm saying that his algorithm performs better than other existing algorithms, how do we substantiate his claim? This is particularly true for a new area such as hyperspectral imaging where new algorithms keep emerging and popping up in a fast pace and each algorithm claims to be better than others. Many users have been struggling and wrestling this issue when they come to select candidate algorithms for hardware architecture design and development such as field programmable gate array (FPGA). Despite the fact that computer-simulated data such as Monte Carlo simulations have been used for this purpose, on many occasions the computer simulations generally go far beyond reality and can be only used for proof-of-concept. In order to move to the next level, more realistic data are needed for further evaluation. The same dilemma occurs in medical community also where the so-called phantoms are designed and developed based on real data using controllable parameters to simulate real environments before experiments can be conducted for real data in vivo. One good example is the magnetic resonance (MR) brain web library provided by MR imaging simulator of McGill University, Montreal, Canada (available at www.bic.mni.mcgill.ca/brainweb/) (see Chapter 32). It seems natural that a similar approach to McGill's synthetic images can also be adopted for hyperspectral imaging. This chapter takes this challenge and designs so-called synthetic images that serve the same purpose as phantoms designed for medical imaging.

Before doing so several issues must be addressed. First, it is important to note that none of the algorithms can claim its superiority over other algorithms without specifying criteria to be used for optimality and applications for which they are designed. More specifically, in what sense of optimality and in what application can an algorithm perform better than other algorithms? Second, there should be a database or a spectral library available in the public domain that allows users to perform impartial assessments and objective comparative studies and analyses. Additionally, all the experiments conducted should be repeatable for validation so that any claimed algorithm can be verified by others using the same data set and identical environment under which the experiments are conducted. Only in this way it can prevent users from being accused of the use of their own data sets to make their own cases. Most important of all is how to design an effective and objective evaluation process to compare algorithms without controversy and subjectivity. Interestingly, to the author's best knowledge, no such effort has ever been made in the past. In this chapter, we investigate and focus on the first and third issues since the second issue can be resolved by many data sets available on website now. Besides, the third issue is also closely related to the first issue which is completely determined by applications. In this case, both issues will be investigated in a coherent manner when it comes to design of experiments.

This chapter first addresses the third issue of how to design and develop a creditable evaluation process. Unlike most computer simulations which generally use laboratory data sets via a set of parameters such as Monte Carlo simulations, here we use real data sets to design synthetic images that can simulate real images with certain properties that we would like to explore. Because hyperspectral imaging sensors can effectively capture targets that generally appear in a mixed form or at subpixel level, it is important to design a process of how such targets can be inserted into an image scene at our discretion. Two types of target insertion are considered, target implantation (TI) and target embeddedness (TE), where the former implants targets by removing the background pixels they intend to replace as opposed to the latter which embeds targets by superimposing the targets over the background pixels they intend to insert. While TI is of particular interest in endmember extraction since pure signatures can be simulated and inserted into the background with guaranteed 100% purity, TE is particularly useful in target detection where the noise and background are assumed to be additive. For each type of target insertion three scenarios can be designed to simulate different levels of target knowledge by adding a Gaussian noise. In order for the designed six scenarios (three scenarios for TI, TI1, TI2, TI3 and three scenarios for TE, TE1, TE2, TE3) to be used as standardized data sets for experiments, the data sets used to generate synthetic images can be chosen from existing databases or spectral libraries available in the public domain. Therefore, no particular data sets are required to simulate these synthetic images. By virtue of these particularly designed six scenarios, an algorithm can be assessed and evaluated objectively and compared fairly to other algorithms on the same setting.