3.2.1.1. Cross-correlation in the spatial domain

Cross-correlation is a commonly used similarity measure in signal processing. Given a reference signal r(t) and a search signal s(t), the cross-correlation between the two signals can be defined as a function of the shift τ of s relative to r:

[3.3]

with r^∗(t) being the complex conjugate of r. Equation [3.3] expresses the cross-correlation for a 1D continuous signal. Considering images as discrete signals sampled in 2D space, equation [3.3] can be rewritten as:

[3.4]

with (x, y) being the image coordinates, (Δx, Δy) the image shifts and Ω the domain of the image templates, i.e. the size of the correlation window. Equation [3.4] would result in strongly varying values for γ for areas of different brightness in the images. To make the correlation values of different image areas comparable, the image templates need to be adjusted by their mean values μ_r, μ_s and normalized by their standard deviation, which results in the normalized cross-correlation (NCC):

[3.5]

The numerator of equation [3.5] is scaled by 1/N (N defines the total number of pixels in the template domain Ω) and thus cancels out the 1/N in the standard deviation of the templates. The values of the NCC range between and the maximum c_p of the cross-correlation peak is also called the normalized cross-correlation coefficient.

The location of the maximum of provides an estimate of the offset between the image templates. Oversampling of the input data r, s and interpolation or curve-fitting of the cross-correlation peak (a brief overview is given by Tong et al. (2019)) allows for tracking with sub-pixel accuracy with different levels of precision (Debella-Gilo and Kääb 2011). However, depending on the method used, “pixel-locking” or “peak-locking” can occur, i.e. the probability that integer offsets are determined by peak-fitting is higher than non-zero sub-pixel offsets (Chen and Katz 2005).

3.2.1.2. Confidence measure of offset estimation

An offset can only be reliably estimated by unambiguously identifying the cross-correlation peak. To quantify the confidence of offset estimation, two parameters are often used: the normalized cross-correlation value (height of the correlation peak) c_p and the signal-to-noise ratio (SNR) of the cross-correlation. Higher confidence is normally characterized by a higher correlation peak c_p, but multiple peaks can exist when periodic patterns are present in the images. The SNR is usually defined as the ratio between the height of the NCC peak and the average of the ambient correlation field

[3.6]

A high SNR indicates a high likelihood that a unique peak is present in the cross-correlation, whereas a low SNR can either be related to strong noise in the cross-correlation or to periodic patterns showing multiple peaks of high correlation c_p. Further methods exist to improve the confidence in the estimated offset (see Chapter 11, section 11.3.4.1).

3.2.1.3. Cross-correlation in the frequency domain

The direct calculation of the cross-correlation in the spatial domain is computationally expensive, especially for the relatively large templates required for SAR data as the computational time scales with the number of pixels per template. To reduce the computational complexity, we can calculate the cross-correlation in the frequency domain. For equation [3.3], flipping r^∗(t) as r^∗(−t) and letting u = −t, we have

which is a convolution integral. Using the convolution theorem, which states that the Fourier transform of a convolution is identical to the product of the two spectra

with · being element-wise multiplication, we can calculate the normalized cross-correlation in the frequency domain as

[3.7]

with F {r} and F {s} being the (fast) Fourier transformation of the two templates, and |F {. . . } |² the total of the power spectrum of r and s. Here, we used F {r^∗} = F {r}^∗. The frequency-domain correlation can be transformed back through the inverse Fourier transformation to track the cross-correlation peak in the spatial domain.

3.2.1.4. Sub-pixel estimation in the frequency domain

Displacements can not only be determined by sub-pixel localization of the cross-correlation peak but also directly from the cross-correlation in the frequency domain. When an integer shift is present and the phase-correlation matrix (see section 3.2.2.4) is transformed back to the spatial domain, the resulting correlation surface will be a unit impulse. However, to ensure the discrete nature of the Fourier transform, the impulse is diluted (or convoluted) by a 2D sinc, which smears most of the energy to immediate adjacent neighbors, but surrounded by steep dips (Foroosh et al. 2002). Thus, peak-locking effects appear (Chen and Katz 2005).

In order to estimate displacements through frequency methods, it is beneficial to directly estimate them in the phase plane. The phase values in the cross-spectrum have a saw-tooth pattern, where the ridges align perpendicular to the direction of the displacement, and the number of cycles corresponds to the magnitude of the displacement. Hence, many algorithms use unwrapping or work out a solution along one axis (see Balci and Foroosh (2006)). Along two axes, the Radon transform, random sample consensus (RANSAC) or single value decomposition (SVD) can also be used (see a review in Tong et al. (2019)).

Alternatively, a two-step procedure can be used, where the integer displacement is first estimated. This integer step is used to realign the template. Then, the displacement is below one pixel displacement and the saw tooth can be approximated by a plane, making direct estimation possible (Stone et al. 2001; Hoge 2003) or easing the convergence of iteration (Leprince et al. 2007). Phase-plane fitting is well-established using optical imagery. However, its application on SAR data is a current topic of research because the high-frequency noise from SAR speckle makes the application of phase correlation difficult.

3.2.2.1. Coherence tracking

For coherence tracking, the single-look-complex values Ae^jφ of a SAR image are used in the NCC (equation [3.5]). Coherence tracking is successful when the phase and the speckle pattern of two SAR images are correlated (top row in Figure 3.8). Finding the peak of the NCC then corresponds to a coherence maximization process. Derauw (1999) proposed this method to derive azimuth shifts in addition to interferometric estimates of line-of-sight displacements (see Chapter 4). For coherence tracking, relatively small correlation windows of, for example, 8×8 single-look pixels can be sufficient (Strozzi et al. 2002), so that very high spatial resolution in the offset field can be obtained. A disadvantage of coherence tracking is that phase gradients within the template (due to deformation gradients such as shear motions at the glacier edge) can reduce or even eliminate the correlation peak (Joughin 2002). These phase gradients can be corrected for by removing the topographic phase or locally estimating the phase gradient in the template’s interferogram (Trouvé et al. 1998; Werner et al. 2005). Coherence tracking can be used to complement InSAR displacement and obtain displacements in the azimuth direction. For example, coherence tracking was used by Funning et al. (2005) to derive 3D earthquake displacements.

3.2.2.2. Speckle tracking

For speckle tracking, the radar amplitude A or intensity I is used for equation [3.5]. Speckle tracking is successful when the speckle pattern is at least partially preserved between an image pair. Speckle tracking was suggested by Gray et al. (1998), who used the cross-correlation of detected radar images (intensity) to derive glacier velocity maps with both the range and azimuth direction. Similar to coherence tracking, speckle tracking allows for a very high spatial resolution in the offset field. For correlated speckle patterns, the speckle can be considered as correlated high-frequency noise, which generates a very sharp and narrow peak in the speckle cross-correlation function (Figure 3.8). A good description of speckle tracking combined with coherence tracking is given by Joughin (2002). Compared to coherence tracking, a slightly sharper peak can be obtained, as the intensity I = |A|² doubles spatial frequencies in the image (see section 3.1.3 and Figure 3.8).

3.2.2.3. Intensity tracking

When the speckle pattern is completely uncorrelated, only larger features extending over multiple intensity pixels can be used for cross-correlation (Figure 3.9). In this case, the speckle pattern can be considered as uncorrelated (or incoherent) noise. To distinguish between intensity tracking of speckle and intensity tracking of image features containing uncorrelated speckle, the latter is also called incoherent intensity tracking, whereas speckle tracking can be considered as coherent intensity tracking.

To average out the incoherent speckle noise in the cross-correlation, considerably larger templates are required for intensity tracking. Typical template sizes vary between 64 × 64 and 256 × 256 pixels (Strozzi et al. 2002; Werner et al. 2005; Paul et al. 2015). Incoherent intensity tracking can be compared to offset tracking in optical imagery, as in both cases physical features are tracked. However, in optical imagery, speckle are absent and much smaller template sizes (e.g. 16×16 or 32×32 pixels) can be used.

As mainly large-scale features contribute to intensity tracking, the correlation function shows a much broader peak (Figure 3.9). As larger templates are required, computation of the cross-correlation can become computationally expensive. In the early days of computers, Vesecky et al. (1988) applied a pyramidal approach for calculating the correlation field to estimate the motion of sea ice. Today, with the availability of the fast Fourier transform (FFT), calculation of the cross-correlation can be sped up significantly when calculated in the frequency domain (see section 3.2.1.3).

When considering that speckle represents multiplicative noise for incoherent intensity tracking, it can be of advantage to convert the intensity into dB before cross-correlation. Thus, speckle can be considered as additive noise that reduces outliers and makes the correlation more robust (lower row in Figure 3.9).

3.2.2.4. Filters for image pre- and post-processing

To better extract the relevant feature patterns from the input images, various filters can be applied to the images before the cross-correlation. Spatial high-pass filters were shown to be effective in improving the coverage of successful tracking and the SNR of the cross-correlation (equation [3.6]) by de Lange et al. (2007).

The “phase correlation” method can be considered as a special case of a (high-pass) frequency filter, in which the frequency spectrum of the input data is completely flattened (normalized). Due to the flattened spectrum, only the phase terms e^iφ of each frequency component are correlated, and hence this method is called “phase correlation” even though only real-valued intensity data are used as input. The normalization reduces the influence of long-wavelength structures and emphasizes high-frequency components in the image. For large enough templates and low enough noise, an extremely sharp correlation peak can be obtained (Schaum and McHugh 1991; Michel et al. 1999; Stone et al. 2001; Foroosh et al. 2002; Zitová and Flusser 2003). Nevertheless, emphasizing high-frequency image components makes the phase correlation method especially sensitive to speckle noise. A similar variant is “orientation correlation” where the orientation of intensity gradients is used as input for cross-correlation (Fitch et al. 2002). This method is often preferred for optical images with low noise, as it is less sensitive to illumination changes and shadow (Heid and Kääb 2012; Dehecq et al. 2015). However, due to its high-pass character, it is only of limited advantage for SAR images due to speckle.

Feature-based filters provide another method for pre-processing of SAR imagery. For example, stable point-like targets on mountain slopes can be discriminated from temporally variable speckle and their displacement can be tracked (Hu et al. 2013; Li et al. 2019). Edge-detection methods have been applied on sea ice followed by phase correlation on the speckle-free product (Karvonen et al. 2012).

Finally, before tracking results are used as input for geophysical models, post-processing can constrain the tracking results to the most probable or the most physical solution. An overview focused on glaciers is given in Chapter 11, sections 11.3.4 and 11.3.5.

3.2.3.1. Cross-correlation stacking

For a pair of non-identical image templates, their cross-correlation (or NCC) can be regarded as a noisy measurement of the offset between them. To enhance tracking, multiple NCCs can be stacked and averaged to reduce the noise and to improve the measurement accuracy. This principle, called the “average correlation method” by Meinhart et al. (2000), has been applied to estimate the velocity of quickly decorrelating ice falls using optical imagery, where it is referred to as “ensemble matching” (Altena and Kääb 2020). Similarly, Li et al. (2021) used this principle to estimate the velocity of alpine glaciers from SAR imagery strongly affected by speckle (incoherent intensity correlation, see section 3.2.2.3).

Let us assume that image templates are composed of a signal and noise as with ˆ indicating the Fourier transform. The NCC in the frequency domain [3.7] can be decomposed as:

[3.8]

Here, the signal is assumed to contain all the correlated content that contributes to the correct template matching, and the noise to represent all the uncorrelated content. In equation [3.8], the shifted auto-correlation generates the signal to track, and the remaining components add noise to the NCC and thus bring uncertainty to the measurement. In order to attain reliable offset estimates, a high SNR for the NCC must be achieved, and thus the term must generate a dominant peak that greatly surpasses the noise.

For NCC stacking, a time series of N coregistered images is collected with a revisit time of Δt, and an NCC is calculated from each pair of image templates with a temporal spacing of T = aΔt (a = 1, 2, 3... indicating integer multiples of the revisit time Δt). This results in a stack of N − a consecutive NCCs, in which each NCC captures a snapshot of the shift between the template pair. Assuming that image features move with constant velocities during acquisition of the image series, all NCCs in the stack represent identical offsets. Hence, the peaks of in all the NCCs should be located at the same position, whereas the noise term remains independent and can be reduced by averaging the NCC stack. The principle of NCC stacking can be generalized to multiple image time series, such as image series acquired in different years, with different spectral or polarimetric channels or even from different sensors (e.g. optical and SAR). NCC stacking is relatively simple to implement but is currently limited to NCC pairs of identical temporal spacing T .

Figure 3.10 shows that NCC stacking can effectively increase the spatial coverage of reliable velocity estimates for SAR offset tracking. In addition, Li et al. (2021) showed that NCC stacking produces reliable results already for a template size half as large as required for pairwise NCC. They also achieved a good performance gain for small stack sizes and thus the compromise on temporal resolution is limited.

3.2.3.2. Autofocusing motion blur in image stacks

Temporal multi-looking causes strong motion blur on moving surfaces [Figure 3.11a], which is why the averaging of image stacks before cross-correlation, studied by Meinhart et al. (2000) for PIV with optical imagery of low particle density, cannot be directly applied to SAR images. Instead, Leinss et al. (2021b) used an autofocus method to combine offset estimation with temporal multi-looking through stack averaging. With the aim of generating two locally coinciding image templates, they split a stack of N images into two sub-stacks each containing N₁ ≈ N₂ ≈ N/2 images. Then, they sampled different velocities by correcting for the time- and velocity-dependent displacement in every image. When the correct local velocity was sampled, temporal multi-looking of the sub-stacks created two speckle-reduced and motion-compensated SAR images revealing fine details that spatially coincided and generated a high cross-correlation score. For sampled velocities that did not match the true velocity, details in the averaged sub-stacks became blurred, resulting in a lower cross-correlation score. The method outperforms pairwise incoherent intensity tracking because the multi-looking process strongly reduces temporally uncorrelated speckle in SAR images.

The method requires a stack of N radar intensity images I_i, acquired at times t₁, ..., t_N, during which most surface features remain visible. The images must be coregistered with sub-pixel precision on stable terrain or the orbit accuracy must be precise enough to ensure sub-pixel registration. Varying time intervals Δt between acquisitions and minor gaps in the time series pose no problem.

To avoid motion blur in the time-averaged sub-stacks, the shift of surface features must be corrected with precisely known offset fields. In contrast, we can determine the offset field by trying to obtain a temporally multi-looked SAR image where motion blur is minimized. To achieve this, the (so far unknown) time-dependent offset field (δx, δy)(x, y, t) can be approximated by the stationary velocity model

[3.9]

where u, v describe the two components of the local velocity vector, is approximately the mean acquisition time of the stack and δt = t − t^∗ is the time interval during which the offset (δx, δy) occurs. Then, possible velocity values are sampled from a predefined distribution, for example a uniform distribution within a certain velocity range. The sampled velocity is used to estimate the time-dependent position (x′ , y′ )(δt) of features located at (x, y) at time t^∗. For the i-th image in the stack, the position at time δt_i = t_i − t^∗ follows from the warping equations

[3.10]

For a spatially constant velocity field, the warping equations describe a simple shear of the two image stacks. For an arbitrary but stationary velocity field, we can track image features that deviate for large δt from a linear trajectory using finite-step time integration of equation [3.10]. Using the warping equations for time-dependent interpolation of the original stacks, the offset-corrected image stacks can be obtained. Averaging each corrected sub-stack results in two offset-corrected temporally multi-looked intensity images in which speckle is strongly reduced.

To estimate the surface velocity field (u, v), the motion blur of the temporally multi-looked sub-stacks is iteratively evaluated for each sampled velocity until the blur is minimized. The motion blur in each iteration is quantified using a method adapted from the “passive phase detection autofocus” method of photographic cameras: two bundles of light rays, each entering the camera at opposite sides of the lens, generate two sub-images, and the focal-length is determined from the offset between the sub-image pair via cross-correlation. Similarly, we calculate the correlation score between the images obtained by averaging the offset-corrected sub-stack:

[3.11]

i=k₀

with (k₀, k_N) = (1, N₁) for k = 1, (k₀, k_N) = (N₁ + 1, N) for k = 2 and N_k = k_N − k₀ + 1. Already, with small N_k and a correctly chosen velocity, the images reveal surface structures that are hidden by speckle in single SAR images because the standard deviation of the intensity scales down with (section 3.1.2). Due to reduced speckle, a relatively small template size can be used for the NCC. To compress the dynamic range and focus on small-scale features, should be converted to dB and a high-pass filter can be applied before calculating the NCC.

Each sampled velocity (u, v) shifts the images in the sub-stacks according to equation [3.10] and generates to calculate an NCC value The NCC score is maximized when the sampled velocity eliminates the motion blur of features in the stack, and more importantly, when the features revealed in coincide. In practice, the sample velocity is applied to shift the entire image and the velocity that improves the local NCC score is stored. With sufficiently dense sampling, the optimal velocity for every image pixel can be found, which represents the estimated velocity field.

The velocity field can be iteratively refined, and thus the method is suitable to effectively track shear movements. The method requires that temporal variations in the velocity field be small, whereas there is no restriction on the spatial complexity of the observed flow patterns. In general, the uniform sampling of the velocity is computationally expensive as each iteration requires an interpolation of the entire image stack, the sum of the sub-stacks and the calculation of the NCC score. For effective velocity sampling, a pyramidal approach, as described by Vesecky et al. (1988), can be used.

As the method effectively correlates full-resolution speckle-reduced SAR imagery, the spatial resolution and also the coverage of the obtained velocity are significantly better compared to pairwise intensity cross-correlation, for which larger templates are required (compare Figure 3.11(c) vs. (d)). Compared to cross-correlation stacking, the method performs slightly better in robustness and accuracy. The reason is that in fact each image from one sub-stack is correlated with every image in the other sub-stack, resulting in (N/2)² cross-correlations. In contrast, for cross-correlation stacking, only a consecutive series of N − 1 cross-correlations are evaluated. As a side product, the methods generate temporally multi-looked radar images that are free of motion blur (Figure 3.11(b)) and that can even be smoothly animated by an arbitrary choice of t^∗ and a running average over a large image series (Leinss et al. 2021a).

3.2.4.1. Single-orbit tracking in slant-range coordinates

The projection of the 3D terrain into the SAR imaging geometry depends on the terrain and on the incidence angle (see Figure 3.3). As long as the incidence angle is constant between two acquisitions, offset tracking reveals only slant-range and azimuth displacement. However, small deviations from the reference orbit cause terrain-dependent distortions in the image content, resulting in terrain-dependent offset fields. To avoid such terrain-dependent offsets, the satellite orbit must be exactly repeated for every acquisition. As this is not always feasible, such geometric distortion can be corrected by DEM-based coregistration (Sansosti et al. 2006; Nitti et al. 2010). However, for SAR offset tracking, we should at least use imagery from the same orbit, as image features show different backscatter characteristics at different incidence angles. Speckle and coherence tracking are most sensitive to small deviations, as speckle can already decorrelate strongly for deviations of a few hundreds of meters from the reference orbit (see also the critical baseline described in Chapter 4, section 4.2.2).

Offset tracking in the native slant-range/azimuth geometry has the advantage that no resampling for orthorectification is required, providing slightly better image quality. Another interpolation can be avoided when tracking is done without resampling for coregistration and instead the offset field is corrected for global and terrain-induced shifts through post-processing. In the latter case, however, cross-correlation stacking and autofocusing motion blur (see sections 3.2.3.1 and 3.2.3.2) require geometry-dependent corrections.

To obtain displacements in a map geometry, the 2D projection of the displacement can be orthorectified (see also Chapter 11, section 11.3.3.3). However, it is crucial to specify that slant-range (line-of-slight) and azimuth displacements are shown or that additional, possibly model-based, transformation into other displacement geometries is applied.

3.2.4.2. Tracking in map coordinates

Offset tracking can also be applied to orthorectified radar images to obtain results directly in map coordinates. The oblique imaging geometry makes a precise orthorectification crucial to avoid image artifacts. For precise orthorectification, a SAR image simulated from an external DEM (Small et al. 1998; Small 2011) can be used as a reference to align the reference SAR image with the DEM used for orthorectification (Wegmüller 1999). DEM-based coregistration of a SAR image stack to a common reference scene is an advantage, in addition to coregistration of the reference scene to the simulated reference.

Working in map coordinates can simplify offset tracking because of the horizontally constant pixel spacing. However, tracking in map coordinates implies the implicit assumption of displacements parallel to the DEM slope. First, this is because the 3D displacement is projected to the slant range and azimuth, and then the orthorectification transforms the apparent slant-range displacement through the DEM-based orthorectification lookup-table into a horizontal displacement. While for slope-parallel displacements only the horizontal component is measured, non-slope-parallel displacements can introduce large biases into the obtained velocity maps.

Despite a constant pixel spacing in the orthorectified imagery, the slope-dependent effective resolution can vary strongly over the scene, which makes the tracking precision dependent on the terrain slope (see Figure 3.3). Another problem relates to small-scale features in the orthorectification DEM, which can distort or wrinkle the orthorectified images when the features move in time (e.g. moving glacier crevasses, growing forest).

3.2.4.3. Three-dimensional inversion

Three-dimensional displacement estimation (including east, north and up components) is an important subject in SAR displacement measurement exploration in relation to both offset tracking and SAR interferometry (Wright et al. 2004; Pathier et al. 2006). Displacement maps showing only line-of-slight and possibly also only azimuth displacements can make the comparison with other sources of information difficult. Furthermore, interpretation can be difficult for those not familiar with SAR geometries. The common approach is thus to estimate the 3D displacement, referred to as the common terrestrial coordinate, from InSAR or offset-tracking displacement measurements obtained from different directions (i.e. range and azimuth directions, different incidence angles, different orbit directions).

The problem can be resolved using a linear geometric model (Wright et al. 2004), as shown in Figure 3.12, and mathematically expressed as

[3.12]

where denotes the vector of N displacement measurements in different directions (LOS, azimuth) obtained from offset tracking or InSAR, and u is the vector containing the true 3D displacement components to estimate. The matrix P is the N × 3 projection matrix, where the rows define the projection either into the slant-range (pLOS,i) or azimuth direction (p_az,i). The projections depend on the incidence angle θ_i and the azimuth heading angle φ_i of each acquisition geometry used. φ_i is defined as between the direction of the satellite trajectory and north. The projections are given by the unit row vectors

[3.13]

To solve equation [3.12], the least-squares approach is commonly used:

[3.14]

with Σ_r being the error covariance of displacement measurements in r. Its main role consists of appropriately weighting displacement measurements of different uncertainties and propagating the uncertainty from r to u. In many cases, a diagonal matrix is used for the sake of simplicity, assuming independence of each displacement measurement. In the case of measurements sharing the same reference/secondary images, the latter assumption may not hold and improved estimations of the covariance matrix Σ_r may be necessary (Teunissen and Amiri-Simkooei 2008). Σ_r can also be approximated by other quality indicator parameters, such as interferometric coherence or cross-correlation coefficients.

Since u is a vector with three components, there should be displacement measurements in at least three directions in r. Tracking results from descending and ascending passes are therefore often combined. However, outside of high latitudes, the azimuth directions of the ascending and descending passes are nearly collinear. In practice, they are often considered as the same contribution to the least-square solution.

Through equation [3.14], a large number of displacement measurements can be transformed into a single 3D displacement result. Besides the ease of interpretation and comparison, another main feature of interest lies in the reduction of the uncertainty. The uncertainty of the 3D displacement obtained is smaller than that of any displacement measurement in r. For a random uncertainty, the uncertainty decreases with the number of displacement measurements used. Therefore, the use of all available displacement measurements is preferred, knowing that the eventual interdependence between different measurements can result in an underestimation of the uncertainty of the 3D displacement. However, when systematic uncertainty (e.g. from orbit uncertainties) is present, fuzzy logic allows us to better describe the uncertainty due to data fusion and provides an upper bound for the uncertainty (Yan et al. 2012). The extraction of 3D displacement fields also simplifies the comparison of results between different sensors, models and field measurements (Yan et al. 2013).