6.3.1. Equivalent energy bandwidth Δf_e of a low-pass filter

In practical applications, to facilitate the calculation, it is often interesting to replace a filter G(f) of a very large frequency band (tends towards infinity) by its equivalent G_e(f) (see Figure 6.2), by determining its energy bandwidth Δf_e, with the constraint of having the same gain at zero frequency:

|G_e(0)| = |G(0)|

In energy terms, one has:

[6.21]

hence:

[6.22]

6.4.1. Eye pattern

The eye diagram, called this because of its resemblance to the shape of a human eye, is the figure obtained by superimposing all traces or realizations of the signal s_c(t) noiseless. It is periodic, of period T.

The channel is of limited bandwidth B, let us then suppose that the signal p(t) extends over the interval [–L₁T, L₁T with L₁ and L₂ integers, and is zero or negligible outside this interval. Let us examine the set of ISI values contained in the signal s_c(t) over the interval [0, T[ for all the realizations of the sequence {a_n}.

On this interval, the signal s_c(t) is expressed (remember that it is assumed to be noise-free):

[6.23]

The possible number of traces of the signal s_c(t) is then equal to M^{L₁+L₂₊₁}, and 2^{L₁+L₂₊₁} in the case of binary symbols.

The superposition of all these traces constitutes the eye pattern associated with the signal s_c(t) on the interval [0, T[. This is, of course, valid for any interval [t₁, t₁ + T[ where t₁is an arbitrary instant.

The eye pattern can be displayed on an oscilloscope. The oscilloscope time base is synchronized to the sampling rate of 1/T frequency, so that at the instants t₀ + kT, the spot always occupies the same horizontal position on the screen. If the scanning velocity of the oscilloscope is sufficiently high compared to the persistence of the cathode ray tube or, even better, if the oscilloscope has a trace memory, a large number of traces of s_c (t) are visible simultaneously on the screen.

The eye pattern allows direct measurement of the ISI at any time, and, therefore, an assessment of the quality of a transmission link. It shows the importance of the distortion of the digital signal, and allows us to quickly and approximately adjust the equalization function.

In Figure 6.4 we show all the characteristics that an eye pattern can have in the case of binary symbols. In the general case of M-ary symbols, we will have (M – 1)

Consider the case where the ISI leads to decision errors in the absence of noise As the optimal decision thresholds without ISI (see formula [6.71]) are distant from 2p(t₀), errors due to the ISI may occur if the absolute value of the ISI is greater than p(t₀), that is to say:

[6.24]

However, the largest absolute value of a_n is (M – 1), which causes the maximum distortion brought by the ISI to be:

[6.25]

(denoted as |I_Max| in Figure 6.4).

When the relation [6.25] is verified, the eye pattern is completely closed at the sampling instants t_k, and decision errors are obvious, even in the absence of noise.

The distance e_v “vertical half-opening of the eye” represents the amplitude of noise corresponding to the threshold of appearance of the errors. In the absence of noise, it is sufficient that the vertical opening of the eye is greater than zero, so that it is possible to distinguish, without error, the values of two neighboring symbols.

The “horizontal half-opening of the eye” e_H represents the maximum tolerable offset of the sampling instant in the absence of noise. In the presence of noise, this tolerance is reduced.

We give two examples of eye diagrams in Figure 6.5: the first (Figure 6.5(a)) concerns the case of binary transmission M= 2, for a signal-to-noise ratio of 15 dB. The second (Figure 6.5(b)) concerns an M-ary transmission with M = 4, for a signal- to-noise ratio of 30 dB.

In both cases, the diagrams are represented for a roll-off factor α = 0.6 (the roll-off factor coefficient will be defined precisely later in the text).

6.5.1. Probability of error: case of binary symbols a_k = ±1

We must first note that when the symbols are binary, then we have T = T_b, where T_b is the duration of a transmitted binary symbol.

The sample s_c(t₀ + kT) at the input of the decision system is compared with a single threshold μ₀, and a decision â_k = d[s_c(t₀ + kT)] concerning the value of the symbol a_k is taken, according to the following rule:

[6.30]

Two types of wrong decision are possible: take the decision â_k = 1 knowing that a_k = –1, or vice versa.

Hence the two conditional probabilities of wrong decision:

[6.31]

[6.32]

If P_i = Pr{a_k = i} with i = [–1,1] is the probability to transmit the symbol a_k = i, and p_mι = Pr{m_ι} is the probability to have the interfering message , then, in the case of independent emitted symbols, the probability of error at time t_k is expressed by:

[6.33]

6.5.1.1. Calculation of conditional probabilities of erroneous decision

The signal sample s_c(t₀ + kT) in the relation [6.27] can then be considered as the realization of a random Gaussian variable of mean value:

and variance:

[6.34]

The reasoning and the calculation which follow are based on the fact that the probability of a random variable can be calculated starting from the knowledge of its density of probability and its support, indeed:

[6.35]

The noise b₁ (t₀ + kT) is written:

[6.36]

In general, the sample s_c (t₀ + kT) belongs to an interval [c, d [ depending on the decision thresholds. One can then write:

[6.37]

The decision rule [6.30] is set as follows:

1) One decides that:

[6.38]

hence:

[6.39]

and:

[6.40]

so:

[6.41]

or:

[6.42]

2) In the same way, one decides that:

[6.43]

hence:

[6.44]

and:

[6.45]

so:

[6.46]

or:

[6.47]

Recall that the probability density of the noise b₁ is given by:

[6.48]

and:

[6.49]

Functions erf(x) and erfc(x) are called the “error function” and the “complementary error function”.

Some authors prefer to use the function Q (x) defined by:

[6.50]

and, in relation with function erfc(x). by:

[6.51]

6.5.1.2. Connection with the detection theory

There is a direct connection with the statistical detection theory. Because the false alarm probability P_FA (false positive, FP) can be directly deduced with what was previously described:

[6.52]

It is the same for the probability of false non detection P_ND (false negative, FN):

[6.53]

We can then define the two following probabilities:

• – the probability of decision P_D:

[6.54]

• – the probability of correct decision P_c:

[6.55]

with:

[6.56]

6.5.1.3. Optimal decision threshold

The optimal decision threshold that minimizes the probability of error P_e is the one that makes null the derivative of P_e with μ₀ (I_mι(t₀ + kT) = 0).

Knowing that:

[6.57]

then:

[6.58]

This leads to the expression of the optimal threshold, denoted μ_0opt, given by:

[6.59]

The optimal threshold only depends on the probability distribution {p₁, p_–1}. For example, if p_± > p_–1. the threshold moves to negative values, so as to favor the decision â_k = 1, that is to say, the most likely a priori binary symbol.

In the case where the binary symbols transmitted are equiprobable, p₁ = p_–1 = 1/2, then the optimal threshold μ_0opt is zero, and the decision system tests only the sign of the sample s_c(t₀ + kT).

6.5.1.4. Probability of error in the simple case of equiprobability of symbols a_k

In this case, one has:

[6.60]

Hence the probability of error P_e:

[6.61]

6.5.2. Probability of error: case of binary RZ code

By adopting the same approach as above, the probability of error for a binary RZ code, according to the relation [6.33], is expressed by:

[6.62]

with:

[6.63]

and:

[6.64]

The optimal threshold is obtained by calculating the threshold value which cancels the derivative of the probability of error in optimal reception:

[6.65]

hence:

[6.66]

In the case of equiprobable symbols, the optimal threshold becomes:

[6.67]

6.5.2.1. Probability of error in the case of equiprobable symbols

In this case:

[6.68]

and then:

[6.69]

6.5.3. Probability of error: general case of M-ary symbols

Let us now consider the general case where the symbols are M-ary and take the following values:

[6.70]

The waveform x(t) is of duration T = kT_b. The structure of the receiver remains unchanged. It is composed of a reception filter (impulse response g_c(t)) followed by a sampler and a threshold comparator system with (M – 1) thresholds p_m. The thresholds p_m, in the case of optimal reception and equiprobable symbols, are optimum and located in the center of each interval, delimited by two consecutive values of the sample a_kp(t₀):

[6.71]

with:

[6.72]

where m is a relative integer.

In Figure 6.7 we show the different possible values of the sample a_kp(t₀) and the position of the optimal thresholds.

6.5.3.1. Decision rules (symbols a_k: â_k)

6.5.3.1.1. Case of intermediate values 2m+ 1, where and m ≠

The decision on symbol a_k is carried out by comparing the sample s_c(t₀ + kT) to the different thresholds μ_m, and the decision â_k = (2m + 1) is taken as soon as s_c(t₀ + kT) lies between the two consecutive thresholds μ_m and μ_m+1:

[6.73]

6.5.3.1.2. Case of extreme values +(M - 1)

To decide on extreme values +(M – 1), the sample s_c(t₀ + kT) is compared to a single threshold in each case:

[6.74]

Denoting by p_2i+1 the probability of emission of the symbols a_k:

[6.75]

and by P_e2i+1 the conditional probability of error which, conditionally to each value of the symbol a_k transmitted and of all the messages m¡ interfering with it, is written:

[6.76]

with:

[6.77]

The total average probability of error (wrong decision) is then expressed by:

[6.78]

with:

[6.79]

the probability of having the interfering message m_ι.

The expression [6.78] can be also written as:

[6.80]

In the case where M > 4 and non independent and non equiprobable symbols a_k are transmitted, the expression [6.78] of the probability of error is, in general, difficult to calculate analytically. In such cases, the computation is carried out by computer.

If the symbols a_k are independent and equiprobable, then one has:

[6.81]

The expression [6.78] of the error probability becomes:

[6.82]

6.5.3.2. Calculation of the conditional probabilities of error in the case of M-ary symbols (M is even)

Using the decision rule given by the relations [6.73] and [6.74], we can calculate the conditional probabilities of errors P_e2i+1 as follows (we develop the same approach as in the binary case).

In the receiver, the noise at the decision instant is written:

[6.83]

If the signal s_c(t₀ + kT) at the input of the decision system belongs to the interval [c,d[, consequently the amplitude of the noise b₁(t₀ + kT) belongs to the following interval:

[6.84]

The decision system operates according to the rules [6.73] and [6.74], so as follows.

6.5.3.2.1. Case of intermediate values of decision

It is decided that:

[6.85]

hence:

[6.86]

then:

[6.87]

Given a_k = (2i + 1), the conditional probability of error P_e2i+1 is written:

[6.88]

where:

[6.89]

so:

[6.90]

or:

[6.91]

6.5.3.2.2. Case of extreme value of decision â_k = (M – 1)

It is decided that:

[6.92]

hence:

[6.93]

then:

[6.94]

with:

[6.95]

Given a_k = (2i + 1) Ψ (M – 1), the conditional probability of error P_e2i+1 is written:

[6.96]

with:

[6.97]

or:

[6.98]

6.5.3.2.3. Case of the extreme value of decision â_k = –(M – 1)

It is decided that:

[6.99]

hence:

[6.100]

then:

[6.101]

with:

[6.102]

Given a_k = (2i + 1) ≠ –(M – 1), the conditional probability of error P_e2i+1 is written:

[6.103]

with:

[6.104]

or:

[6.105]

This makes it possible to then use the relation [6.78] to calculate the total probability of error.

6.5.4. Probability of error: case of bipolar code

The bipolar code is a code with M = 3 levels such as:

[6.106]

with the two threshold values:

[6.107]

As an example, to calculate the probability of error, we will assume that {b_k} are independent but not equiprobable, with:

[6.108]

Given the symbol a_k transmitted, the calculation of the conditional probabilities of the different messages of the form m_ι = (a_k–1, a_k+1) interfering with symbol a_k are given in Tables 6.2, 6.3 and 6.4.

In total, there are nine different messages, of probabilities:

[6.109]

with:

[6.110]

Example.-

etc.

The probability of error (made of 30 terms) is then:

[6.111]

If only adjacent decision errors are practically possible, i.e.:

[6.112]

then P_e will have only 22 terms.

6.6.1. Nyquist temporal criterion

The principle of cancellation of the intersymbol interference implies that, after equalization, at the sampling instants t_k = t₀ + kT and according to the expression [6.16], the signal p(t) fulfills the following temporal condition:

[6.113]

Taking t₀ as time origin, the previous temporal condition becomes (see also Figure 6.8):

[6.114]

6.6.2. Nyquist frequency criterion

The pulse p(t) can be of any length and shape, but all of its samples p[(k – n)T] with η ≠ k must be zero. To find the Nyquist frequency criterion, let us look at the transformation of the time condition [6.114] in the frequency domain.

For this, consider the sampled signal ρ_Δ(t) defined by:

[6.115]

Its Fourier transform P_Δ(f) is written:

[6.116]

Besides, by the definition of the Fourier transform, we have:

[6.117]

However, the relation [6.115] is also written by using the relation [6.114]:

[6.118]

thus:

[6.119]

Finally, from [6.116] and [6.119], we deduce the Nyquist frequency criterion:

[6.120]

Any function solution of [6.120] fulfills the Nyquist frequency criterion.

6.6.3. Interpretation of the Nyquist frequency criterion

The transmission channel (physical medium) has a bandwidth B. Its transfer function H(f) satisfies:

[6.121]

and, consequently, according to [6.12]:

[6.122]

We can distinguish three cases.

First case

The folded spectrum consists of replicas of P(f) separated by 1/T each other, which do not overlap, and their addition cannot give a constant function (see Figure 6.9). The Nyquist frequency criterion is not verified. This means that we cannot transmit a digital signal with symbol rate D_s (symbol/s) without an ISI in a channel with a bandwidth B < D_s/2.

Second case

In this case, the Nyquist criterion is only fulfilled for a given P(f), denoted N_m(f), (see Figures 6.10 and 6.11):

[6.123]

This means that the minimum bandwidth B of the channel required to transmit a digital signal with symbol rate D_s = 1/T and without ISI, is 1/2T, called the

Otherwise, in a channel of width B, one can transmit without ISI a maximum symbol rate of:

[6.124]

This corresponds to the first Nyquist criterion. It should be noted that the first Nyquist criterion sets a maximum limit at the symbol rate D_s, in symbols per second, and not at the bitrate D_b, in bits per second.

Indeed, according to the following relation:

[6.125]

At a given D_sMax , the bitrate D_bMax can be increased by increasing the number M of levels of the signal. However, the latter is limited by the signal-to-noise ratio.

In the temporal domain, the first Nyquist criterion is translated into the following relation:

[6.126]

This first Nyquist frequency criterion is only of theoretical interest. It has drawbacks which make it unusable in practice, since:

• – the perfectly rectangular template N_m(f) is impractical due to the gain discontinuity at the frequency 1/2T ;
• – function p(t) decreases as 1/|t|, and a small error ɛτ of the sampling instant t_k = kT leads to a maximum |I_Max| of ISI that is unbounded. Indeed, |I_Max| in this case is given, according to [6.25], by:

[6.127]

The arithmetic series of general term 1/|ɛ + m| being divergent, so |I_max| is not bounded, that means that the eye pattern is completely closed at time kT + ɛτ.

Note that taking into account t₀, the first Nyquist criterion is expressed in the frequency domain and in the time domain respectively by:

[6.128]

and by:

[6.129]

Third case

The folded spectrum consists of overlapping replicates of P(f) (see Figure 6.12) and it is possible to find many functions P(f) which satisfy the Nyquist frequency criterion called the '“extended Nyquist frequency criterion'” which do not have the drawbacks of the previous solution.

In the case where P(f) = 0 for |f| > 1/T real and positive raised cosine functions are widely used in practice in the transmitter and receiver filters of digital communications systems:

[6.130]

Parameter a, between 0 and 1, is called the “roll-offfactor”. Filters P_α(f) spread 1 + œ over the frequency bandwidth (by reasoning on the positive frequency domain), that is, an increase of the minimum bandwidth by a factor 1 – α. The impulse response of these filters is of the form:

[6.131]

Figures 6.13 and 6.14 show the normalized and delayed impulse response p_α(t) by 5T (so that the filter is causal) and the frequency response P_α(f) for different values of parameter α.

For α, one gets the rectangular filter N_m(f).

It can be seen that the amplitude of the oscillations of the function p_α(t) is smaller as the factor α is large, and therefore the bandwidth of the filter P_α (f) is large. This has an important practical consequence on the sensitivity of the receiver to inaccuracies in the sampling instants. Indeed, an offset ɛτ of the sampling instant will introduce an ISI which is all the more significant as the amplitude of the oscillations is large. At the limit, when a = 1, the amplitude of the oscillations is minimal (decay in 1/t² of p„(t)), to the point that the function p_α(t) is canceled at times (kT + T/2), for k ≠ [0, –1]. The horizontal opening of the eye e_H is double compared to the case where α = 0. This improvement comes at the expense of the necessary bandwidth of the channel, twice as much as that of Nyquist ideal.

In Figures 6.15, 6.16, 6.17, and 6.18, we represent the eye pattern in the case of binary symbols a_n = ±1 for several values of the parameter a. We notice that all the traces of the eye pattern pass through the same points at the sampling times in the absence of noise. We also show, in Figure 6.19, the eye pattern in the case of a quaternary symbol for α = 0.6. Finally, note that the Nyquist filters are not attainable because the function p_α(t) is non-causal, p_α(t) Ψ 0 for t <0, regardless of the value of the parameter α. However, in practice, the function p_α(t) is limited to a support fixed a priori with which we associate a delay line with a half-value delay in order to make causal the function p_α(t) as soon as a> 0.1.

These filters are commonly used in digital communications systems with values of a in the order of 0.2 to 0.6.

NOTE.– The Nyquist frequency criterion also applies when the function P(f) is complex. Condition [6.120] becomes:

[6.132]

Furthermore, in the following, we show that the raised cosine filter which cancels the ISI is distributed between the transmitter and receiver filters. Each impulse response is then given by:

[6.133]

6.7.1. Expression of the minimum probability of error for a low-pass channel satisfying the Nyquist criterion

The probability of error of M-ary symbols a_k is given by:

This probability of error P_e is minimal when is maximal. But, according to the expression [6.155], we recall:

The ratio is maximal if the reception filter is matched to the emission filter:

[6.159]

Moreover, if the noise b₀(t) is white noise, then:

[6.160]

where E_a is the average energy required to transmit a symbol a_k .

The probability of error is finally expressed by:

[6.161]

6.8.1. Transmission using the duobinary code

Let us consider the transmission of a series of binary symbols a_n over a noisy low-pass channel with bandwidth B. The sample s_c(t₀ + kT) at the input of the decision block is written in the usual form:

[6.164]

Let us consider that only the symbol a_k–1 interferes with the symbol a_k, thus producing an ISI equal to a_k–1p(t₀ + T).

Under these conditions, and assuming that:

[6.165]

The expression of s_c(t₀ + kT) is then:

[6.166]

The temporal condition of this particular form of ISI is:

[6.167]

The frequency function P(f) that leads to this same ISI should satisfy the following equality:

[6.168]

Or again:

[6.169]

The condition [6.169] is fulfilled by filter P(f):

[6.170]

that satisfies:

[6.171]

This filter covers a frequency band of 1/2T (by reasoning on the positive frequencies) but, unlike the filter verifying the first Nyquist criterion, it does not present discontinuities at frequencies +1/2T and it is therefore physically achievable (see Figure 6.22, section 6.8.3). Furthermore, the maximum distortion due to an offset ɛ of the optimal sampling instant (t₀ + kT) remains finite. The eye pattern is therefore opened in the vicinity of the optimal decision instant. Furthermore, by setting c_k = a_k + a_k–1, the expression [6.166] is written:

[6.172]

NOTE.– The expression [6.172] shows that the transmission of a sequence of independent a_n binary symbols, in the presence of a particular ISI, is seen as the transmission of a sequence of correlated ternary symbols c_n, but without ISI.

6.8.1.1. Demonstration of relation [6.168]

To demonstrate the relationship, we adopt the approach explained in section 6.6. The sampled signal p_Δ (t) is written:

[6.173]

Or considering relations [6.166] and [6.167]:

[6.174]

The respective Fourier transforms of expressions [6.173] and [6.174] are written:

[6.175]

[6.176]

By equalizing the two expressions of p_Δ(f), we obtain relation [6.168].

6.8.2. Transmission using 2nd order interleaved bipolar code

In this case, the ISI added to the useful signal at the instant (t₀ + kT) is:

–a_k–2p(t₀ + 2T)

The sample s_c(t₀ + kT) at the output of the reception filter is equal to:

[6.177]

again, assuming that p(t₀ + 2Γ) = p(t₀):

[6.178]

To obtain this amount of ISI, the signal p(t) must fulfill the following temporal condition:

[6.179]

By following the same approach as before, we easily show that the function P(f), which leads to the ISI mentioned above, verifies the following relation:

[6.180]

which is fulfilled by a filter P(f) such that:

[6.181]

The modulus | P (f) | of the frequency response is shown in Figure 6.22.

As in the case of the duobinary code, the frequency response occupies a frequency band equal to 1/2T (for positive frequencies) and has no discontinuity. In addition, the interleaved bipolar code of order 2 can be used for transmission over a cable, since |P(f)| at f = 0 is zero.

By setting c_k = a_k – a_k–2, the expression [6.178] is written like the expression [6.172]:

[6.182]

and the remark from the previous paragraph concerning the duobinary code applies.

6.8.3. Reception of partial response linear codes

In the previous section, it was shown that the transmission of binary information in the presence of controlled ISI is transformed, thanks to partial response coding, to the transmission of a series of correlated M-ary symbols without ISI.

Recall the relationship between the symbols c_k and a_k which depends on the type of ISI considered here, namely:

[6.183]

Decoding of binary symbols is obtained directly from the symbols c_k without propagation of error, thanks to the function of precoding on transmission.

6.8.3.1. Case of duobinary coding

The precoder generates a sequence such as:

[6.184]

The transcoder generates, at output, a symmetrical binary sequence {a_k}:

[6.185]

such that it generates:

[6.186]

The relation defining the operation of the partial response coder is expressed by:

[6.187]

with c_k ∈ [0, ±2].

Then, according to [6.187], we can write:

then:

[6.188]

So a direct estimate of the transmitted binary sequence from the estimated sequence of symbols received is:

[6.189]

6.8.3.2. Example of duobinary coding and decoding

In Figure 6.23, we give an example of duobinary coding and decoding.

We can easily see that the absence of precoding on transmission would lead to a propagation of decision errors on the binary symbols {b_k}. Indeed, in this case, according to [6.187], by replacing by b_k, one has:

[6.190]

So, if the binary element b_k–1 is badly decoded, the binary element b_k will also almost certainly be.

6.8.3.3. Case of 2nd order interleaved bipolar coding

In this case, the precoder is defined by the following rule:

[6.191]

The transcoding operation is the same whatever the code used, and it is therefore governed by relation [6.185].

The relation defining the operation of the partial response coder is written:

[6.192]

with c_k ∈ [0, ±2].

Hence:

[6.193]

This relation shows the instantaneous estimate of the sequence sent from the sequence received:

[6.194]

EXAMPLE.– 2nd order interleaved bipolar coding and decoding.

Figure 6.24 gives an example of 2nd order interleaved bipolar coding and decoding.

6.8.4. Probability of error P_e

Insofar as the binary elements b_k are independent and equiprobable on the alphabet {0,1}, the symbols c_k take the values {–2,0,2} with, respectively, the following probabilities: ļi, 2,.

The probability of error is then equal to:

[6.195]

where P_ek is the conditional probability that c_k = k with k = {–2,0,2}.

Let:

[6.196]

[6.197]

and:

[6.198]

By comparing the sample s_c(t₀ + kT) to a pair of thresholds –p(t₀) and p(t₀), we can easily calculate the different expressions of conditional probabilities.

6.8.4.1. Calculation of conditional probabilities

This now classic calculation is thus developed (see also Figure 6.25):

[6.199]

if:

[6.200]

where:

[6.201]

so:

[6.202]

Application of the decision rules

[6.203]

hence:

[6.204]

[6.205]

[6.206]

hence:

[6.207]

[6.208]

[6.209]

hence:

[6.210]

[6.211]

NOTE.– As soon as the signal-to-noise ratio E_mb/Γ_b0 is sufficiently large, one can make the reasonable assumption of neglecting the possibility of decoding when c_k =2 or vice versa. The probability of error on the binary digit P_eb is equal to the probability of error P_e on the symbols c.