3.16.2.1 Parametric array model

Consider an array of sensors arranged in an arbitrary geometry that receives the waveforms generated by point sources (electromagnetic or acoustic). The output of each sensor is modeled as the response of a linear time-invariant bandpass system of bandwidth . The impulse response of each sensor to a signal impinging on the array depends on the physical antenna structure, the receiver electronics and other antennas in the array through mutual coupling. The complex amplitudes of these sources w.r.t. a carrier frequency are assumed to vary very slowly relative to the propagation time across the array (more precisely, the array aperture measured in wavelength, is much less than the inverse relative bandwidth ). This so-called narrowband assumption allows the time delays of the th source at the th sensor, relative to some fixed reference point, to be modeled as a simple phase-shift of the carrier frequency. If is the complex envelope of the additive noise, the complex envelope of the signals collected at the output of the sensors is given by applying the superposition principle for linear sensors by:

(16.1)

where and may include generally azimuth, elevation, range and polarization of the th source. However, we will here assume that there is only one parameter per source, referred as the direction of arrival (DOA) . is the steering vector associated with the th source. The array manifold, defined as the set for some region in DOA space, is perfectly known, either analytically or by measuring it in the field. It is further required for performance analysis that be continuously twice differentiable w.r.t. . is the steering matrix with .

To illustrate the parameterization of the steering vector , assume that the sources are in the far field of the array, and that the medium is non-dispersive, so that the waveforms can be approximated as planar. In this case, the th component of is simply where is the directivity gain of the th sensor, , represents the speed of propagation, is a unit vector pointing in the direction of propagation and is the position of the th sensor relative the origin of the different delays.

The by far most studied sensor geometry is that of uniform linear array (ULA), where the sensors are assumed to be identical and omnidirectional over the DOA range of interest. Referenced w.r.t. the first sensor that is used as the origin, and , where is the wavelength. To avoid any ambiguity, must be less than or equal to . The standard ULA has that ensures a maximum accuracy on the estimation of . In this case

(16.2)

3.16.2.2 Signal assumptions and problem formulation

Each vector observation is called a snapshot of the array output. Let the process be observed at time instants . is often sampled at a slow sampling frequency compared to the bandwidth of for which are independent. Temporal correlation between successive snapshots is generally not a problem, but implies that a larger number of snapshots is needed for the same performance. We will prove in Section 3.16.4.3 that the parameter that fixes the performance is not , but the observation interval . The signals and are assumed independent.¹ For well calibrated arrays, is often assumed to be dominated by thermal noise in the receivers, which can be well modeled as zero-mean temporally and spatially white circular Gaussian random process. In this case, and , for which the spatial covariance and spatial complementary covariance matrices are given by and , respectively. A common, alternative model assumes that is spatially correlated where is known up to a scalar multiplicative term , i.e., where is a known definite positive matrix. In this case, can be pre-multiplied by an inverse square-root factor of , which renders the resulting noise spatially white and preserves model (16.1) by replacing the steering vectors by .

Two kind of assumptions are used for . In the first one, called stochastic or unconditional model (see, e.g., [10,11]), are assumed to be zero-mean random variables for which the most commonly used distribution is the circular Gaussian one with spatial covariance and spatial complementary covariance . is nonsingular for not fully correlated sources (called also noncoherent) or near-singular for highly correlated sources. In the case of coherent sources (specular multipath or smart jamming, where some signals impinging on the array of sensors can be sums of scaled and delayed versions of the others), is singular. In this chapter is usually assumed nonsingular. For these assumptions, the snapshots are zero-mean complex circular Gaussian distributed with covariance matrix

(16.3)

This circular Gaussian assumption lies not only in the fact that circular Gaussian data are rather frequently encountered in applications, but also because optimal detection and estimation algorithms are much easier to deduce under this assumption. Furthermore, as will be discussed in Section 3.16.4, under rather general conditions and in large samples [21], the Gaussian CRB is the largest of all CRB matrices corresponding to different distributions of the sources of identical covariance matrix . This stochastic model can be extended by assuming that is arbitrarily distributed with finite fourth-order moments [20] including the case where associated with the second-order noncircular distributions.

A common alternative assumption, called deterministic or conditional model (see, e.g., [10,11]) is used when the distribution of is unknown or/and clearly non-Gaussian, for example in radar and radio communications. Here is nonrandom, i.e., the sequence is frozen in all realizations of the random snapshots . Consequently, is considered as a complex unknown parameter in . For this assumption, the snapshots are complex circular Gaussian distributed with mean and covariance matrix .

With these preliminaries, the main DOA problem can now be formulated as follows: Given the observations, and the described model (16.1), detect the number of incoming sources and estimate their DOAs .

3.16.2.3 Parameter identifiability

Once the distribution of the observations has been fixed, the question of the identifiability of the parameters (including the DOA ) must be raised. For example, under the assumption of independent, zero-mean circular Gaussian distributed observations, all information in the measured data is contained in the covariance matrix (16.3). The question of parameter identifiability is thus reduced to investigating under which conditions determines the unknown parameters. Thus, if no a priori information on is available, the unknown parameter of contains the following real-valued parameters:

(16.4)

and the parameter is identifiable if and only if . To ensure this identifiability, it is necessary that be full column rank for any collection of , distinct . An array satisfying this assumption is said to be unambiguous. Notice that this requirement is problem-dependent and, therefore, has to be established for the specific array under study. For example, due to the Vandermonde structure of in the ULA case (16.2), it is straightforward to prove that the ULA is unambiguous if . In the case where the rank of , that is the dimension of the linear space spanned by is known and equal to , different conditions of identifiability has been given in the literature. In particular, the condition

(16.5)

has been proved to be sufficient [22] and practically necessary [23].

When are not circularly Gaussian distributed, the identifiability condition is generally much more involved. For example, when is noncircularly Gaussian distributed, is noncircularly Gaussian distributed as well with complementary covariance

(16.6)

and the distribution of the observations are now characterized by both and . Consequently, the condition of identifiability will be modified w.r.t. the circular case given in (16.5). This condition has not been presented in the literature, except for the particular case of uncorrelated and rectilinear (called also maximally improper) sources impinging on a ULA for which, the augmented covariance matrix with is given by

(16.7)

where with is the second-order phase of noncircularity defined by

(16.8)

Due to the Vandermonde-like structure of the extended steering matrix , the condition of identifiability is now here .

Note that when is discrete distributed (for example when are symbols of a digital modulation taking different values), the condition of identifiability is nontrivial despite the distribution of is a mixture of circular Gaussian distributions of mean and covariance .

3.16.3.1 Performance analysis of a specific algorithm

3.16.3.2 Cramer-Rao bounds (CRB)

The accuracy measures of performance in terms of covariance and bias of any algorithm, described in the previous section may be of limited interest, unless one has an idea of what the best possible performance is. An important measure of how well a particular DOA finding algorithm performs is the mean square error (MSE) matrix of the estimation error . Among the lower bounds on this matrix, the celebrated Cramer-Rao bound (CRB) is by far the most commonly used. We note that this CRB is indeed deduced from the CRB on the complete unknown parameter of the parametrized DOA model, for example, given by (16.4) for the circular Gaussian stochastic model. Furthermore, rigorously speaking, this CRB ought to be only used for unbiased estimators and under sufficiently regular distributions of the measurements. Fortunately, these technical conditions are satisfied in practice and due to the property that the bias contribution is often weak w.r.t. the variance term in the mean square error (16.28) for , the CRB that lower bounds the covariance matrix of any unbiased estimators is used to lower bound the MSE matrix of any asymptotically unbiased estimator⁴

(16.29)

with is given under weak regularity conditions by:

(16.30)

where is the Fisher information matrix (FIM) given elementwise by

(16.31)

associated with the probability density function of the measurements .

The main reason for the interest of this CRB is that it is often asymptotically (when the amount of data is large) tight, i.e., there exist algorithms, such that the stochastic maximum likelihood (ML) estimator (see 3.16.4.2), whose covariance matrices asymptotically achieve this bound. Such estimators are said to be asymptotically efficient. However, at low SNR and/or at low number of snapshots, the CRB is not achieved and is overly optimistic. This is due to the fact that estimators are generally biased in such non-asymptotic cases. For these reasons, other lower bounds are available in the literature, that are more relevant to lower bound the MSE matrices. But unfortunately, their closed-form expressions are much more complex to derive and are generally non interpretable (see, e.g., the Weiss-Weinstein bound in [38]).

In practice, closed-form expressions of the FIM (16.31) are difficult to obtain for arbitrary distributions of the sources and noise. In general, the involved integrations of (16.31) are solved numerically by replacing the expectations by arithmetical averages over a large number of computer generated measurements. But for Gaussian distributions, there are a plethora of closed-form expressions of in the literature. And the reason of the popularity of this CRB is the simplicity of the FIM for Gaussian distributions of .

3.16.3.3 Asymptotically minimum variance bounds (AMVB)

To assess the performance of an algorithm based on a specific statistic built on , it is interesting to compare the asymptotic covariance (16.21) or (16.23) to an attainable lower bound that depends on the statistic only. The asymptotically minimum variance bound (AMVB) is such a bound. Furthermore, we note that the CRB appears to be prohibitive to compute for non-Gaussian sources and noise, except in simple cases and consequently this AMVB can be used as an useful benchmark against which potential estimates are tested. To extend the derivations of Porat and Friedlander [58] concerning this AMVB to complex-valued measurements, two additional conditions to those introduced in Section 3.16.3.1.1 must be satisfied:

Under these four conditions, the covariance matrix of the asymptotic distribution of any estimator built on the statistics is bounded below by :

(16.38)

where is the matrix .

Furthermore, this lowest bound is asymptotically tight, i.e., there exists an algorithm whose covariance of its asymptotic distribution satisfies (16.38) with equality. The following nonlinear least square algorithm is an AMV second-order algorithm:

(16.39)

where we have emphasized here the dependence of on the unknown DOA . In practice, it is difficult to optimize the nonlinear function (16.39), where it involves the computation of . Porat and Friedlander proved for the real case in [59] that the lowest bound (16.38) is also obtained if an arbitrary weakly consistent estimate of is used in (16.39), giving the simplest algorithm:

(16.40)

This property has been extended to the complex case in [60].

This AMVB and AMV algorithm have been applied to second-order algorithms that exploit both and in [24]. In this case, to fulfill the previously mentioned conditions (i)–(iv), the second-order statistics are given by

(16.41)

where denotes the operator obtained from by eliminating all supradiagonal elements of a matrix. Finally, note that these AMVB and AMV DOA finding algorithm have been also derived for fourth-order statistics by splitting the measurements and statistics into its real and imaginary parts in [60].

3.16.3.4 Relations between AMVB and CRB: projector statistics

The AMVB based on any statistics is generally lower bounded by the CRB because this later bound concerns arbitrary functions of the measurements . But it has been proved in [44], that the AMVB associated with the different estimated projectors , and introduced in Section 3.16.3.1.2, which are functions of the second-order statistics of the measurements, attains the stochastic CRB in the case of circular or noncircular Gaussian signals. Consequently, there always exist asymptotically efficient subspace-based DOA algorithms in the Gaussian context.

To prove this asymptotic efficiency, i.e.,

(16.42)

and

(16.43)

the condition (iv) of Section 3.16.3.3 that is not satisfied [61] for these statistics ought to be extended and consequently the results (16.38) and (16.39) must be modified as well, because here is singular.

In this singular case, it has been proved [61] that if the condition (iv) in the necessary conditions (i)–(iv) is replaced by the new condition , (16.38) and (16.39) becomes respectively

(16.44)

and

(16.45)

And it is proved that the three statistics , , and satisfy the conditions (i)–(iii) and (v) and thus satisfy results (16.44) and (16.45).

Finally, note that this efficiency property of the orthogonal projectors extends to the model of spatially correlated noise, for which where is a known positive definite matrix. In this case, for example, the orthogonal projector defined after whitening

satisfies

where is insensitive to the choice of the square root of , and is no longer a projection matrix.

3.16.4.1 Beamforming-based algorithms

Among the so-called beamforming-based algorithms, also referred to as low-resolution, compared to the parametric algorithms, the conventional (Bartlett) beamforming and Capon beamforming are the most referenced representatives of this family. These algorithms do not make any assumption on the covariance structure of the data, but the functional form of the steering vector is assumed perfectly known. These estimators are given by the highest (supposed isolated) maximizer and minimizer in of the respective following criteria:

(16.46)

where is the unbiased sample estimate and is either the biased estimate or the unbiased estimate (that both give the same estimate ). Note that these algorithms extend to parameters per source, where is replaced by in (16.46).

For arbitrary noise field (i.e., arbitrary noise covariance ) and/or an arbitrary number of sources, the estimate given by these two algorithms are non-consistent, i.e.,

and asymptotically biased. The asymptotic bias can be straightforwardly derived by a second-order expansion of the criterion around each true values (with [resp., ] for the conventional [resp. Capon] algorithm), but noting that is a maximizer or minimizer of or , respectively. The following value is obtained [62]

(16.47)

with and .

Following the methodology of Section 3.16.3.1.2, the additional bias for finite value of , that is of order can be derived, which gives

see, e.g., the involved expression of for the Capon algorithm [62, rel. (35)].

In the same way, the covariance which is of order can be derived. It is obtained with

see, e.g., the involved expression [50, rel. (24)] of associated with a source for several parameters per source. The relative values of the asymptotic bias, additional bias and standard deviation depend on the SNR, and , but in practice the standard deviation is typically dominant over the asymptotic bias and additional bias (see examples given in [62]).

Finally, note that in the particular case of a single source, uniform white noise () and an arbitrary number of parameters of the source (here ), it has been proved [63], that given by these two beamforming-based algorithms is asymptotically unbiased ( given by (16.47) is zero), if and only if is constant. Furthermore, based on the general expressions (16.48) of the FIM⁶

(16.48)

where is defined here by , for parameters associated with a single source, and expression [50, rel. (24)] of specialized to , it has been proved that , i.e., the conventional and Capon algorithms are asymptotically efficient, if and only if is constant.

3.16.4.2 Maximum likelihood algorithms

3.16.4.3 Second-order algorithms

Most of the narrowband DOA algorithms presented in the literature are second-order algorithms, i.e., are based on the sample covariance or more generally on . To prove common properties of this class of algorithm, it is useful to use the functional analysis presented in Section 3.16.3.1.1

(16.63)

in which any second-order algorithm is a mapping alg that generally satisfies

(16.64)

but not necessarily for all Hermitian positive semi-definite matrix . Depending on the a priori knowledge about , that is required by the second-order algorithms alg, different constraints are satisfied by the -differential matrix of the algorithm at the point (16.22). In particular, it has been proved the following main two constraints [20]:

(16.65)

(16.66)

Using these constraints, the general expression of the covariance of the asymptotic distribution of the sample covariance [31] obtained under mild conditions for non independent measurements with arbitrary distributed sources and noise of finite fourth-order moments, and the general relation (16.23), that links and to the covariance of the asymptotic distribution of , allows one to prove the following two results, that extend a robustness property presented in [71]:

Note that the majority of the second-order algorithms (e.g., the beamforming, ML, MUSIC, Min Norm, ESPRIT algorithms) does not require spatially uncorrelated sources. In contrast, second-order techniques based on state-space realizations (e.g., the Toeplitz approximation method (TAM), see [8]) and Toeplitzation or augmentation with ULA or uniform rectangular arrays, require this uncorrelation, and thus the asymptotic distribution of will be generally (except for a single source, for which the constraint (16.66) reduces to (16.65)) sensitive to the distribution, the second-order noncircularity or the temporal distribution of the sources, even when the noise is temporally uncorrelated.

To illustrate this sensitivity to the source distribution when the noise is temporally uncorrelated, we consider in Figure 16.2, the case of two equipowered and spatially uncorrelated sources impinging on a ULA of 10 sensors, and , where the DOAs are estimated by the standard MUSIC algorithm after Toeplization. The sources are either white Gaussian, ARMA Gaussian (generated by a (10, 10) Butterworth filter driven by a white circular Gaussian noise, where the bandwidth is fixed to 0.5) or harmonic. The centered frequencies of the ARMA and the frequencies of the harmonics are and . Figure 16.2 shows that the Toeplization improves the performance for very weak SNR only, whereas is very sensitive to the distribution of the sources for high SNR.

Usually, performance analyses are evaluated as a function of the number of observed snapshots without taking the sampling rate into account. In fact, depending on the value of this sampling rate, the collected samples are more or less temporally correlated and performance is affected. Thus, the interesting question arises as to how the asymptotic covariance of the DOA estimators (denoted here ) varies with this sampling rate for a fixed observation interval . This question has been investigated in [20], in which the continuous-time noise envelope is spatially white and temporally white in the bandwidth . It has been proved:

Consequently the array must be temporally oversampled, and the parameter of interest that characterizes performance ought not to be the number of snapshots, but rather the observation interval .

3.16.4.4 Subspace-based algorithms

We concentrate now on the family of second-order algorithms based on the orthogonal noise⁸ projector (16.10). These algorithms estimate , either by extrema-searching approaches (MUSIC, Min-Norm, etc.), by polynomial rooting approaches (Pisarenko, root MUSIC, and root Min-Norm for ULA), or by matrix shifting approaches (ESPRIT, TAM, Matrix pencil method). The most celebrated of these algorithms is the MUSIC algorithm, where is estimated as the deepest minima in a -dimensional (for parameters per source) of the following localization function :

(16.67)

of the so-called spatial null spectrum (or equivalently as the highest peaks (maxima) of its inverse). This algorithm has given a plethora of variants. For example, in the particular case of the ULA, this standard MUSIC algorithm have been favorably replaced by the root MUSIC algorithm. Using the general methodology presented in Section 3.16.3.1.2, the asymptotic distribution of given by any subspace-based algorithms alg is simply derived from the expression of the -differential matrix of the mapping evaluated at . For example, for the standard MUSIC algorithm, is straightforwardly obtained from the first-order expansion of that gives for one parameter per source

(16.68)

with and . Using (16.68) with (16.16) and (16.23) allow one to directly prove that the sequences converges in distribution to the zero-mean Gaussian distribution of covariance matrix given elementwise by and compactly by

(16.69)

where and has been defined in Section 3.16.3.1.2. Note that these expressions have been derived in [3] by much more involved derivations based on the asymptotic distribution of the eigenvectors of the sample covariance matrix . Finally, note that if the sample orthogonal noise projector is replaced by an adaptive estimator of , where is the step-size of an arbitrary constant step-size recursive stochastic algorithm (see e.g., [72,73]), it has been proved in [72] that converges in distribution to the zero-mean Gaussian distribution of covariance matrix given also by , where is an adaptive estimate of given by the MUSIC algorithm based on the specific adaptive estimate of studied in [72]. Using a similar approach [26], it has been proved that the Root MUSIC algorithm associated with the ULA, presents the same asymptotic distribution, but slightly outperforms the standard MUSIC algorithm outside the asymptotic regime. This analysis has been extended to MUSIC-like algorithms applied to the orthogonal noise projectors [resp. ] associated with the complementary sample covariance [the augmented sample covariance ] matrices for the DOA estimation of arbitrary noncircular [resp. rectilinear] sources [27]. Finally, note that with our general methodology, all the expressions of the covariance can be straightforward extended for several parameter per source.

The expression of the covariance (16.69) of the asymptotic distribution of given the standard MUSIC algorithm has been analyzed in detail (see, e.g., [2,3]). In particular it has been proved that the MUSIC algorithm is asymptotically efficient for a single source, an arbitrary number of parameters per source and depending on , e.g., for one parameter per source

For several sources, the MUSIC algorithm is in general asymptotically inefficient, in particular for correlated sources for which the efficiency degrades when the correlation between the sources increases. The degradation of performances are considerable for highly correlated sources for any value of the SNRs. In contrast, for uncorrelated sources, the MUSIC algorithm is asymptotically efficient when tends to zero, in the following sense . So, in practice, for uncorrelated sources, the MUSIC algorithm is asymptotically efficient for high SNRs of all the sources.

It is of utmost importance to investigate in what region of and SNR, the asymptotic theoretical results can predict actual performance. But unfortunately, only Monte Carlo simulations can specify this region. We illustrate in the following the SNR threshold region for the SML, DML, and MUSIC algorithm.

Consider two zero-mean circular Gaussian sources impinging on an ULA (16.2) with (for which the 3 dB bandwidth is about ) and a spatially uniform white noise (16.3). The source consist of a strong direct path at relative to array broadside and a weaker (multipath at ) at −3 dB w.r.t. . The correlation between and is giving thus the source covariance matrix . Figure 16.3 shows the root mean square error (RMSE) of the estimated DOA by the MUSIC algorithm w.r.t. the SNR defined by , compared with the theoretical standard deviation (TSD) and the square root of the stochastic CRB . We see from this figure that the MUSIC algorithm is not efficient at all for highly correlated sources. Furthermore, the domain of validity of the asymptotic regime is here very limited, i.e., for , SNR  > 30 dB is required.

With the same parameters, Figure 16.4 shows the RMSE of the estimated DOA by the SML and DML algorithms which are compared with the TSD and and the square roots of the CRBs and . We see from this figure that the numerical values of the four expressions of (16.57) are very close and the performance of the two ML algorithms are very similar except for the SNR threshold region for which the SML algorithm is efficient for SNR  > 0 dB with . Finally, comparing Figures 16.3 and 16.4, we see that both ML algorithms largely outperform the MUSIC algorithm for highly correlated sources.

3.16.4.5 Robustness of algorithms

We distinguish in this subsection, the robustness of the DOA estimation algorithms w.r.t. the narrowband assumption and to array modeling errors, because for the array modeling errors, the model (16.1) remains valid with a modified steering matrix, in contrast to the violation of narrowband assumption, for which (16.1) must be modified.

3.16.4.6 High-order algorithms

When the sources are non Gaussian distributed, they convey valuable statistical information in their moments of order greater than two (this is in particular true when considering communications signals). In these circumstances, it makes sense to consider DOA estimation techniques using this higher order information. Of particular interest are the algorithms based on higher order cumulants of the measurements due to their additivity property in the sums of independent components. Furthermore, these cumulants show the distinctive property of being in a certain sense, insensitive to additive Gaussian noise, making it possible to devise consistent DOA estimates without it being necessary to know, to model or to estimate the noise covariance . As generally, the distributions of the sources are even, their odd order moments are zero and thus to cope with these signals, only the even high-order cumulants of the measurements are used.

Computational considerations dictate using mainly fourth-order cumulants. To use these approaches, we consider the assumptions of Section 3.16.2.2, in which we add that the sources have nonvanishing fourth-order cumulants. Furthermore, we assume that their moments are finite up to the eighth-order, to study the statistical performance of these algorithms.

Of course, there are many more quadruples than pairs of indices, and consequently a very large number of cumulants , for circular sources (and more, and , for noncircular sources) can be exploited despite their redundancies, to identify the DOA parameters with unknown noise covariance. For example, for circular signals, the maximum set of nonredundant cumulants is

The asymptotically minimum variance (AMV) algorithm (see Section 3.16.3.3) based on a subset of fourth-order cumulants that can identify the DOA parameters, is the nonlinear least square algorithm (16.40) in which gathers the involved cumulants. To implement this AMV algorithm, one has to decide which cumulants should be included in . The best estimate would be obtained when all nonredundant cumulants are selected. This, however, may require excessive computations if is large. However it is sufficient to deal with a reduced set of cumulants, although there do not seem to be any simple guidelines in this matter [60]. In practice, a good tradeoff between computational complexity and accuracy is to devise suboptimal algorithms that require an overall computational effort similar to the second-order algorithms, while retaining a fourth-order cumulants subset, sufficient for DOA indentification. Such algorithms have been proposed in the literature such as the diagonal slice (DS), the contracted quadricovariance (CQ) and the so called 4-MUSIC [60] algorithms. The first two algorithms are fourth-order subspace-based algorithms built on the following rank defective matrices:

They require sources and their statistical performance has been analyzed in [19] with the general framework explained in Section 3.16.3.1. In particular, it is has been proved that for a single source and a ULA in spatially uniform white noise, these two fourth-order algorithms have similar performance to the MUSIC algorithm, except for low SNR, for which the MUSIC algorithm outperforms both fourth-order algorithms. The 4-MUSIC algorithm is built from the rank defective matrix

It is proved in [60] that

where , . is indefinite in general and its rank is where the sources are divided in groups, with in the th group. The sources in each group are assumed to be dependent, while sources belonging to different groups are assumed independent. Because the vectors , are columns of , the 4-MUSIC algorithm is obtained by searching the deepest minima of the following localization function :

(16.72)

where is now, the orthogonal projector onto the noise subspace of the sample estimate of . In practice the statistical dependence of the sources are unknown. Porat and Fiedlander [60] has proposed to retain only , rather eigenvectors corresponding to the smallest singular values of . We note that, to the best of our knowledge, no complete statistical performance analysis of this algorithm has yet appeared in the literature. Despite its higher variance (w.r.t. the MUSIC algorithm under the assumption of spatially uniform white noise), this fourth-order algorithm presents some advantages, aside from its capacity to deal with unknown Gaussian noise fields. Using the concept of virtual array, it is proved in [83] that this algorithm can identify up to sources when the sensors are identical and up to sources for different sensors. Furthermore, it is shown that its resolution for closely spaced sources and robustness to modeling errors is improved with respect to the MUSIC algorithm. To increase even more its number of sources to be processed, resolution and robustness to modeling errors, extensions of this 4-MUSIC algorithms, giving rise to the 2-MUSIC (with ) has been proposed [84].

3.16.5.1 MDL criterion

The information theoretic criteria approach is a general method for detecting the order of a statistical model. That is, given a parameterized probability density function for various order , detect such that , where is the ML estimate of and is a penalty function. For the MDL criterion which is based on a particular penalty function, is given for independent identically distributed measurements , by

(16.73)

where denotes the number of free real-valued parameters in . Depending on the distribution of the measurements and its parametrization , different implementations of the MDL criterion have been proposed.

The most often used assumption, is the zero-mean circular Gaussian distribution associated with the parametrization (16.1) in which all the elements of the steering matrix are assumed unknown with the only restriction that has full column rank with . For this modeling, the measurements can be parameterized by the parameter

where are the eigenvalues of and ,…,, the eigenvectors associated with the largest eigenvalues, and the general MDL criterion (16.73), which is referred to as the Gaussian MDL (GMDL), becomes [13]

(16.74)

with and , where are the eigenvalues of the sample covariance matrix , denoted here by .

3.16.5.2 Performance analysis of MDL criterion

This GMDL criterion has been analyzed in [16], and it has been shown to be a consistent estimator of the rank , i.e., the probability of error decreases to zero as the number of measurements increases to infinity. Moreover, under mild regularity conditions, like finite second moments, it is a consistent estimator of the rank , even if the measurements are non-Gaussian. This property contrasts with the Akaike information criterion (AIC) that yields an inconsistent estimate of that tends, asymptotically, to overestimate [13].

The GMDL criterion has been further analyzed by considering the events and , called underestimation and overestimation, respectively. Since are functions of the eigenvalues of , the derivation of the probabilities and needs the joint exact or asymptotic distribution of . This asymptotic distribution is available for circular complex Gaussian distribution [87] and more generally for arbitrary distributions with finite fourth-order moments [14], but unfortunately, the functional (16.74) is too complicated to infer its asymptotically distribution. Therefore, for simplifying the derivation of these probabilities, it has been argued [36,88,89] by extended Monte Carlo experiments (essentially for and ) that

(16.75)

As the probability of overestimation is concerned, exact and approximate asymptotic upper bound of this probability have been derived in [36] showing that generally . Therefore, only the probability of underestimation has been analyzed by many authors. In particular, using the refinement introduced by [90]

of the classical approximation and the asymptotic bias (16.18) and covariance (16.19), a closed-form expression of the probability of underestimation given by the GMDL criterion, used under arbitrary distributions with finite fourth-order moments, has been given in [14]. This expression has been analyzed for and for different distributions of the sources in [14]. Figure 16.5 illustrates the robustness of the MDL criterion to the distribution of the sources. We see from this figure that the probability of underestimation is sensitive to the distribution of the source, particularly for sources of large kurtosis and for weak values of the number of snapshots.

The general MDL criterion has been studied in [15], where using the approximation (16.75), a general analytical expression of has been given. This expression allows one to prove the consistency of the general MDL criterion when the number of snapshots tends to infinite and has been specialized to particular parameterized distributions. Among them, the Gaussian assumption associated with a parameterized steering matrix has been studied and some numerical illustrations show that the use of this prior information about the array geometry enables an improvement in performance of about 2 dB. Finally, note that the MDL criterion generally fails when the sample size is smaller than the number of sensors. In this situation a sample eigenvalue based detector has been proposed in [91].

3.16.6.1 Angular resolution limit based on mean null spectra

Based on the array beam-pattern , different resolution criteria have been defined from its main lobe w.r.t. a look direction , as the celebrated Rayleigh resolutions such as the half power beamwidth or the null to null beamwidth that depends solely on the antenna geometry, and consequently have the serious shortcoming of being independent of the SNR.

For specific so-called high resolution algorithms, such as different MUSIC-like algorithms, based on the search for two local minima of sample null spectra , two main criteria based on the mean null spectrum have been defined. These criteria are justified by the property that the standard deviation of the sample null spectrum associated with the conventional MUSIC and Min-Norm algorithms is small compared to its mean value in the vicinity of the true DOAs for for arbitrary SNR [1].

For the first criterion, introduced by Cox [92], two sources are resolved if the midpoint mean null spectrum is greater than the mean null spectrum in the two true source DOAs:

This criterion was first studied by Kaveh and Barabell [1] and Kaveh and Wang [93] in the resolution analysis of the conventional MUSIC and Min-Norm algorithms for two uncorrelated equal-powered sources and a ULA. This analysis has been extended to more general classes of situations, e.g., for two correlated or coherent equal-powered sources with the smoothed MUSIC algorithm [94], then for two unequal-powered sources impinging on an arbitrary array with the conventional and beamspace MUSIC algorithm [95]. A subsequent paper by Zhou et al. [96] developed a resolution measure based on the mean null spectrum and compared their results to Kaveh and Barabell’s work.

For the second criterion, introduced by Sharman and Durrani [97] and then studied by Forster and Villier [26] in the context of the conventional MUSIC and Min-Norm algorithms for two uncorrelated equal-powered sources and a ULA, two sources are resolved if the second derivative of the mean null spectrum at the midpoint is negative

Resorting to an analysis based on perturbations of the noise projector [6], instead of those of the eigenvectors (e.g., [1,95]), these two criteria have been studied for arbitrary distributions of the sources, for the conventional MUSIC algorithm. The following closed-form expressions of the approximation of the threshold ASNR given by these two criteria have been obtained in [27]:

(16.76)

where and are fractional expressions in specified in [27] for ULAs. These expressions (16.76) have been extended in [27] to a noncircular MUSIC algorithm adapted to rectilinear signals, introduced and analyzed in [98], for which (16.76) becomes

(16.77)

where is the second-order noncircularity phase separation (16.8) and where now and are expansions of without constant term, whose coefficients depend on , and the array configuration. Closed-form expressions of and are given in [27] for weak and large second-order noncircularity phase separations and ULAs, where it is proved that and are decreasing functions of and thus are minimum for .

Figure 16.6 illustrates these two threshold ASNRs for two independent equal-powered BPSK modulated signals impinging on a ULA with and . We clearly see in this figure that the noncircular MUSIC algorithm outperforms the conventional MUSIC algorithm except for very weak second-order noncircularity phase separations or which the ASNR thresholds of these two algorithms are very similar. Furthermore, we note that the behaviors of the ASNR threshold given by the two criteria are very similar although the ASNR thresholds are slightly weaker for the Sharman and Durrani criterion than for the Cox criterion.

Moreover, several authors have considered (e.g., [99–101]) the probability of resolution or an approximation of it, based on the Cox criterion applied to the null sample spectrum to circumvent the possible misleading results given by these two criteria. Finally note that the resolution capability of the conventional and Capon beamforming algorithms have been thoroughly analyzed (see, e.g., [102]). Thanks to the simple expression of their spatial null spectra (16.46), it is possible to derive an approximation of the probability of resolution defined as the probability that the dip in midway between the two sources is at least 3-dB less than the peak of either source as a function of the SNR and DOA separation. Thus, fixing a specific high confidence level, this allows one to predict the SNR required to resolve two closely spaced sources. The superiority of the Capon algorithm is proved in [102], as the resolving power increases with SNR; in contrast, the Bartlett algorithm cannot exceed the Fourier/Rayleigh limit no matter how strong the signals.

3.16.6.2 Angular resolution limit based on the CRB

Array resolution has been studied independently of any algorithm by using the CRB. Based on the observation that the standard MUSIC algorithm is unlikely to resolve closely spaced signals if the standard deviation of the DOA estimates exceed [3], Lee [103] has proposed to define the resolution limit as the DOA separation for which

(16.78)

for the two closely spaced sources, where is somewhat arbitrarily chosen. This criterion ignores the coupling between the estimates and . To overcome these drawbacks, Smith has proposed [18] to define the resolution limit as the source separation that equals the square root of its own CRB, i.e.,

(16.79)

with .⁹ This means that the angular resolution limit or the threshold ASNR are obtained by resolving the implicit equations (16.78) and (16.79). This latter criterion has been applied to the deterministic modeling of the sources in [18] and then extended to multiple parameters per source in [104]. For the stochastic modeling of the sources, the circular Gaussian distribution has been compared to the discrete one in [105]. In particular it has been proved that the threshold ASNR is inversely proportional to the number of snapshots and to the square of for the Gaussian case, in contrast to BPSK, MSK and QPSK case, for which it is inversely proportional to the fourth power of .

3.16.6.3 Angular resolution limit based on the detection theory

The previous two approaches to characterize the angular resolution have in fact two different purposes. The first one studies the capability of a specific algorithm to estimate the DOAs of two closely spaced sources when the number of sources is known. In contrast, the second one is aiming to define an absolute limit on resolution that depends only of the array configuration and parameters of interest as the number of sensors and SNR. But this latter approach based on the ad hoc relationships (16.78) and (16.79), essentially makes sense because the CRB indicates the parameter estimation accuracy and intuitively should be related to the resolution limit. But it suffers from two drawbacks. First, the resolution limit defined by this approach is not rigorously grounded in a statistical setting. Second, if the resolution limit is expressible by (16.78) or (16.79), can the translation factor , be analytically determined?

To solve these two problems, Liu and Nehorai have proposed to use a hypothesis test formulation [17]. This approach has been introduced in a 3D reference frame, but to be consistent with the notations of this section, it is briefly summarized in the following in the 2D framework, where the DOA of a source is the parameter . As the source localization accuracy may vary at different DOAs, consider the resolution limit at a specific DOA of interest. More precisely, assume there exists a source at a known DOA and we are interested in the minimum angular separation that the array can resolve between this source at and another source at a direction close to . Quite naturally, the resolution of the two sources can be achieved through the binary composite hypothesis test

To rigorously define the resolution limit , we fix the values of and for this test. Otherwise, could be arbitrary low, while the result of the test may be meaningless. Let be the unknown parameter of our statistical model, where is the parameter of interest and gathers all the unknown nuisance parameters. To conduct this test, the GLRT is considered due to the unknown nuisance parameters:

(16.80)

where and denote the probability density function of the measurement under the hypothesis and , respectively. and are respectively the ML estimate of and under , and is the ML estimate of under . The distribution of this GLRT is generally very involved to derive, but hopefully, approximations of the distribution of for large values of are available under and . First, under , Wilk’s theorem with nuisance parameters (see, e.g., [106, p. 132]) can be applied without having to know the exact form of . This theorem states the following convergence in distribution when tends to :

(16.81)

where denotes the central chi-square distribution with one degree of freedom (associated with the single parameter ). Under , the derivation of the asymptotic distribution of is much more involved. Using a theoretical result by Stroud [107], Stuart et al. [108, Chapter 14.7] have stated that when can take values¹⁰ near , is approximately distributed¹¹ as

(16.82)

where denotes the noncentral chi-squared distribution with one degree of freedom and noncentrality parameter given by (see [109, Section 6.5])

(16.83)

whose dependence on in the FIM of is emphasized, and where denotes the (1,1) th entry of . It is further shown [109, Appendix 6C] that as is large, (16.83) is approximated by

(16.84)

Based on these limit and approximate distributions of under and for which the GLRT in (16.81) can be rewritten as

(16.85)

the angular resolution limit (ARL) has been computed in [17] by using the two constraints

where the values of and are fixed and where and denote the right tail probability of the and distributions, respectively. It assumes the form

where the factor is analytically determined by the preassigned values of and . Note that the SNR is embedded in the expression of CRB that is proportional to . The dependence on the SNR of the CRB may vary according to the distribution of the sources. For example, Delmas and Abeida [105] proves that the CRB of the DOA separation of discrete sources is very different from those of Gaussian sources.