33.9 Further Topics

It is nearly impossible to write a book to cover all interesting topics as it was originally planned. This is certainly the case for this book. In particular, real-time hyperspectral image processing has become an emerging and booming area in recent years, and many research efforts have been reported and published in the literature on a fast pace track. The following sections only provide a preview of this topic. More details can be found in Chang (2013).

33.9.1 Causal Processing

Causal processing is a prerequisite to real-time processing. It only uses the data samples that were already visited in the past but not those in the future for data processing. With this content, spatial domain-based literal techniques in traditional image processing are generally not causal because they are primarily developed to take advantage of spatial correlation among data samples. For example, texture-based and window-based image processing techniques are usually not applicable to real-time processing. This is also true for most anomaly detection techniques such as the commonly used anomaly detector, RX detector (Reed and Yu, 1990), and CEM detector (Harsanyi, 1993) which are not real-time detectors because they require the entire dataset to compute a global covariance or correlation matrix prior to detection. By contrast, the pixel-based nonliteral techniques developed for hyperspectral imagery are actually causal processing techniques due to the fact that no interpixel spatial correlation is involved in which case, the processing can be carried out pixel-by-pixel on a single pixel basis. A good representative of this type of processing is least-squares-based LSMA techniques, LSOSP, NCLS, and FCLS method, all of which can be implemented in real time since they process data pixel by pixel without using any interpixel spatial correlation.

33.9.2 Real-Time Processing

The importance of real-time processing has been recently realized and recognized in many applications. In some applications, for example, on-board spacecraft data processing system, it is very useful to have high levels of processing throughput. Specially, as data rate generated by spacecraft instruments is increasing at speed, on-board data processing has been largely absent from remote sensing missions. Many advantages can be benefited from real-time processing. One is detection of moving targets. This is critical and crucial in battlefield when moving targets such as tanks or missile launching vehicles pose real threat to ground troops. The real-time data processing provides timely intelligence information, which can help to reduce casualty. Another is on-board data processing. For space-borne satellites, real-time data processing can significantly reduce mass storage of data volume. A third advantage is chip design. Since it can be implemented in real time the computational load can be largely reduced. Furthermore, it can also provide a payload relief in aircrafts and satellites. Over the past years, many subpixel detection and mixed pixel algorithms have been developed and shown to be very versatile. However, their applicability to real-time processing problems is generally restricted by the very complex computational workloads.

One key issue in designing a real-time processing algorithm is that real-time processing must be performed in such a fashion that the output should be produced immediately at the same time as an input comes in. As a matter of fact, no such a real-time processing algorithm exists in real world applications since there is always a time lag resulting from data processing. However, form a practical appoint of view, such a time delay is determined by a specific application. For example, in surveillance and reconnaissance applications, finding moving targets is imminent for decision making and the responding time must be very short. In this case, very little time delay should be allowed. As another example, for applications in fire damage management/assessment, the time to respond can be minutes or hours in which case the allowable time delay can be longer. So, an algorithm that can meet such a time constraint can be considered as a real-time processing algorithm. Another key issue is that a real-time processing algorithm must be carried out causally in the sense that the data samples used for processing are only those up to the data sample vector currently being processed (Gelb, 1974). More specifically, no future data sample vectors after the current data sample vector can be allowed to be used for data processing. In many applications, the concept of such causality is not considered as a prerequisite to real-time processing as long as an algorithm can be fast processed with negligible time. In this case, it is generally considered as a real-time processing algorithm. Nevertheless, technically speaking, such an algorithm is neither a real-time processing algorithm nor a causal processing algorithm.

33.9.3 FPGA Designs for Hardware Implementation

Over the last few years, several research efforts have been directed toward the incorporation of specialized hardware for accelerating remote sensing-related calculations aboard airborne and satellite sensor platforms. Enabling on-board data processing introduces many advantages, such as the possibility to reduce the data downlink bandwidth requirements at the sensor by both preprocessing data and selecting data to be transmitted based upon predetermined content-based criteria. On-board processing also reduces the cost and the complexity of ground processing systems so that they can be affordable to a larger community. Other remote-sensing applications that will soon greatly benefit from onboard processing are future sensor missions as well as future Mars and planetary exploration missions, for which on-board processing would enable autonomous decisions to be taken on board.

In recent years, rapid advances in VLSI technology have large impact on modern digital signal processing. Over the past years, we have witnessed that the number of transistors per chip has doubled about once a year. Therefore, VLSI design of complex algorithms becomes more and more feasible. A major difficulty with implementing these algorithms in real time is computation of the inverse of a matrix. Systolic arrays provide a possibility. But it requires a series of Givens rotations to decompose a matrix into triangular matrices that can be implemented in real time. Unfortunately, such Givens rotations cannot be realized in hardware. To resolve this issue, the Givens rotations must be performed by operations such as adds, ORs, XORs, shifts that can be realized in hardware architecture. In order to do so, the COordinate Rotation DIgital Computer (CORDIC) algorithm is developed by Volder (1959), which allows us to convert a Givens rotation to a series of shifts-adds operations. Using systolic arrays architecture in conjunction with the CORDIC algorithm, we can implement a matrix inverse computation in a set of shifts-adds operations. As a result, it makes field programmable gate arrays (FPGA) design feasible to become a versatile technique for a wide range of applications in hyperspectral data exploitation because of its hardware advantages for fast data processing. The insights into FPGA design with real-time implementation have also been investigated in recent years. However, it also requires specific applications to realize its benefits such as application-specific integrated circuit (ASIC). One is to make use of orthogonal subspace projection (OSP) to design various hyperspectral imaging algorithms for real-time implementation. Its FPGA design was investigated in Wang (2003) and Chang and Wang (2008) where a QR-decomposition (QRD)-based matrix triangularization approach along with the well-known Feddeeva algorithm is used to calculate the pseudoinverse, for the OSP detector and a CORDIC, key to the QR-decomposition in FPGA, with systolic array architecture was also proposed. This FPGA design architecture has been successfully simulated with real hyperspectral imagery.

Another application is to design real-time implementation of CEM which has shown promise in hyperspectral data exploitation where its systolic array implementation was developed in Chang and Wang (2007). Unlike the OSP, CEM involves the computation of the inverse of the sample spectral correlation matrix instead of computation of the pseudo-inverse of a matrix, as does OSP. In this case, the CORDIC algorithm is readily applicable and there is no need of using the Faddeeva algorithm for conversion. Depending upon how to compute the input stream, two methods are of interest. Method 1 computes the input stream from image pixel vectors directly, while Method 2 computes the sample spectral correlation matrix R. These two module-based method were proposed in Chang and Wang (2007). For Method 1, five modules are required. The first module is to design an array of CORDIC circuits where the pixel stream is fed into the module and the upper triangular matrix is updated in real time. This is followed by the second module which applies backsubstitution to compute the inverse of , invR. Then Module 3 uses a distributed arithmetic to calculate where the d is the desired target signature. Next, Module 4 is developed to obtain the desired filter vector w by finding . Finally, Module 5 is to produce the results by applying an FIR filter to the current input pixel streams. Method 2 takes an alternative approach by first computing the auto-correlation matrix R. Four modules are proposed for this method. Module 1 is the design of auto-correlator that calculates the sample correlation matrix R. It is then followed by Module 2, which uses the CORDIC circuits to triangularize . Next, Module 3 applies the backsubstitution to obtain the desired filter vector w. Finally, Module 4 produces the filter output energy for target detection by applying an FIR filter to the current input pixel streams. Details of FPGA implementations of OSP and CEM can be found in Wang (2003), Chang and Wang (2007), Chang and Wang (2008), and Chang (2013).

33.9.4 Parallel Processing

Parallel processing is sometimes confused with real-time processing because it can also be implemented in real time. As a matter of fact, these two processes are completely different. The need of real-time processing arises from requirement of processing huge volumes of data and large data storage resulting from data communication. The real-time processing provides a great advantage of process-then-forget benefit in the sense that the processed data are immediately output and will not be stored during the entire process. All pixel-based LSMA techniques can be generally implemented to unmix data in real time. The need of parallel processing also arises from the demand of processing enormous volumes provided by hyperspectral data. Parallel processing implements a divide-and-conquer strategy in dividing the data to be processed into a number of small data subsets so that all the data subsets can be processed in parallel so as to increase processing speed. So, if a real-time processing algorithm does not have a parallel structure, it cannot be implemented as a parallel processing algorithm. On the other hand, a parallel processing algorithm does not require to be implemented in real time if it is not causal. While these two processes are not correlated with each other, it is certainly desirable to have both in algorithm design by taking advantage of their strengths to improve better data processing performance. As for designing and developing real-time hyperspectral imaging algorithms, readers can consult the book by Chang (2013). With regard to high performance computing via parallel processing, Plaza and Chang (2007a) provides a good reference for those who are interested in this topic.

33.9.5 Progressive Hyperspectral Processing

Progressive hyperspectral data processing is a new concept that has not been much explored in the past. Despite the fact that several chapters, Chapter 20–21, 23 and 26 in this book are devoted to this particular subject it seems still in its infancy in many applications, specifically, satellite data communication and transmission which must handle enormous data in an effective and efficient manner. A progressive process decomposes hyperspectral data processing into a number of sequential stages and processes data stage-by–stage progressively in the sense that the results obtained by previous stages are retained and also used to improve subsequent stages where only the new innovations information that is not available in previous stages is used for data updating. This is quite different from a sequential processing which processes data samples in a sequential manner where each data sample is fully processed not gradually processed. In addition to those discussed in this book several new developments for progressive processing have been recently explored and investigated. Two major trends are of interest. One is progressive band processing in various applications, for example, anomaly detection, endmemeber extraction, and linear spectral unmixing. It is different from progressive band/spectral dimensionality processing in Chapters 20–21, progressive band selection in Chapter 23 and progressive signature coding in Chapter 26 in the sense that progressive band processing is developed for operators particularly designed for specific applications where the operators work like a Kalman filter which can be updated by a recursive equation and innovations information. Another trend is real-time causal processing where data sample vectors are progressively processed in real time as well as causality in the sense that operators particularly designed for specific applications are updated by results obtained by those data sample vectors already visited and the information provided by the current being processed data sample vector via an iterative equation. These two trends pave the way of developing a new theory for progressive hyperspectral processing which is one of main themes in Chang (2013).