5.1. Introduction

In recent decades, the pace at which technological advances have become available has sharply accelerated. In the last 30 years, the personal computer, the Internet, mobile networks as well as many other innovations have transformed human lives. Beyond science fiction-like visions, we have searched for substitutes for repetitive, physically challenging tasks, replacing them with machines or robots reaching beyond human capabilities or with computer programs under the supervision of human beings. Among these machines, the automobile has revolutionized the daily transport of millions of people and the number of cars has grown steadily since the democratization of the vehicle. This explosion of the number of cars gave rise to many problems, such as traffic jams, air pollution due to greenhouse gases and soil pollution associated with liquid and solid discharges (motor oils and heavy metals). Accidents are still one of the biggest road-related problems. The annual report of the French Road Safety Observatory (L'observatoire national interministériel de la sécurité routière – ONISR), estimated for 2015 [ONS 16], revealed that the number of deaths related to road traffic accidents had increased by 1.7% in comparison with the previous year, producing a total of 3,616 fatalities. Even though this number is lower compared to the 1980s, it is still unacceptable. At the global scale, more than 1.25 million people die in road accidents. The question then points to the main causes of these accidents and invites us to find the obstacles we should tackle in order to reduce the number of road accidents. In fact, the analysis of accident technique [ONS 16] revealed that the driver still represents the main cause of accidents: speed represents 32% of accidents (main or secondary), driving under the influence of alcohol causes 21% of accidents and illegal drugs cause 9% of accidents. Lack of attention, dangerous overtaking, sleepiness, lane changing and sickness accounted for 31% of the cause of accidents. Vehicle factors accounted for only 1% of accidents.

The causes of accidents were not the same for all age groups. In fact, novices and seniors mostly made mistakes related to not giving priority. On the other hand, alcohol and speed were the causes of the majority of accidents among younger people. The rates of accident causality per age group are shown on the histogram in Figure 5.1.

An accident is rarely triggered by a single cause: it is actually a conjunction of several causes, of which speed is often the aggravating factor. For example, driving under the influence of illegal drugs may induce high speed and lack of attention, followed by dangerous overtaking. That being said, the human factor was still the main cause for the majority of road traffic accidents in France in 2015. In fact, humans have to perform a variety of actions while driving: they have to perceive the environment, analyze and understand it, adopt and outline a driving strategy and carry it out through actuators. Tiredness, distraction, miscalculation, unconsciousness, etc. may provoke mistakes in each of these actions and eventually lead to accidents. In addition to road safety, fuel consumption is a concern for the human driver. In addition, Barth and Boriboonsoms in [BAR 09] proved that adopting an eco-driving system could help save between 10% and 20% of fuel. On another note, this economic process could help reduce the emission of carbon dioxide and nitrogen oxides into the atmosphere and thereby reduce atmospheric pollution. [BOU 10, HAJ 10].

In order to respond to these challenges, researchers and engineers have introduced various advanced driver-assistance systems (ADAS). This may explain the design of active safety systems, such as the collision avoidance system [HAL 01] and the lane departure warning system [JUN 04], in order to avoid accidents, or intelligent speed monitoring systems, with the aim of optimizing fuel consumption, as introduced by [BAR 08, AKH 16], or reducing risk on road curves [GLA 10, GAL 13]. However, active security systems struggle to achieve the expected results, due to the fact that they are limited to specific cases, and the interaction of these systems with the driver is still problematic and ambiguous. In fact, several recent incidents have been caused by a misunderstanding between human and machine, or at least by the latter’s limitations. Such technical limitations basically concern navigation systems, which may display non-existent sections of road, which is what happened to an American driver who was using the Waze app and found himself crossing a frozen lake, before the ice broke [AFP 18].

Automated driving seems to provide a global solution for the problems described above. Ideally, for an automated vehicle, every task involving perception, localization, planning and control should be executed by sensors, algorithms and actuators [JO 14], only allowing the driver to choose the destination to be reached. Based on this principle, several autonomous vehicle projects worldwide are going through the research phase, such as Google’s GoogleCar/Waymo. Others are interested in specific operation areas, such as Renault’s or General Motor’s prototypes. The goal is to achieve a high level of automation (i.e. level 4 or 5 according to the definition by SAE International [SAE 16], as shown in Figure 5.2), considering the driver as a fallback solution during failure scenarios or operating within specific environments (level 4), or in every possible situation (level 5), by providing autonomous monitoring of the vehicle’s environment, the performance of its sensors and algorithms, and the action to be taken in order to keep the vehicle in a safe state. The task of automation seems difficult to accomplish, given the wide range of road scenarios to consider, and the vehicle should be able to react to any damage, and especially to failures in its own means of perception. The diagnosis is therefore an essential step for detecting and isolating sensor faults as soon as these appear, in order to enable stable and safe driving, enhanced by fault-tolerant control. In this chapter, we will introduce a diagram of fault-tolerant control architecture based on voting algorithms so as to diagnose the faults of exteroceptive sensors.

This chapter is organized as follows: a formulation of the problem is illustrated in section 5.2; section 5.3 is devoted to the architecture of fault-tolerant control; voting algorithms are detailed in section 5.4; section 5.5 shows the results of digital simulation and a conclusion is reached in section 5.6.

5.2. Formulation of the problem

Given the diversity of situations and problems that a vehicle may encounter, this chapter will focus on the problem of having to regulate inter-vehicle distance. Nevertheless, the real problem in measuring the inter-vehicle distance lies in the inability of technologies to properly operate in all working modes. The use of several sensor technologies helps to overcome each sensor’s limitation, by merging data. However, data fusion may be affected if the sensors have hardware or software faults, which may lead to erroneous measurements. Therefore, it is necessary to manage the reliability of these measurements and to ensure fault tolerance, in order to guarantee safe operation in autonomous mode [MAR 09].

This topic was partly addressed in [MAR 09], where the authors suggested a simple switching strategy to choose between three control loops, each consisting of a given sensor (radar, lidar and a camera), a condition observer and a controller. The switching mechanism chooses the control loop that minimizes a quadratic criterion, from those that can be found in a robust, fault-free invariant set. However, this method shows the disadvantage of being restrictive in the case of longitudinal tracking, since it does not take into account the orientation of the front vehicle. In [REA 15, REA 16], multi-sensor fusion architecture was introduced, minimizing the influence of sensor faults. This technique was based on a fusion structure controlled by an SVM (Support Vector Machine) module for detecting faults. The fault detection block compared the divergences between the outputs in:

• – two local fusion structures, each using a Velodyne sensor and a vision sensor;
• – a master fusion structure.

The Velodyne sensor is considered as a reference sensor in both local fusion structures. Also, the data from the fusion are based on the weights generated by the SVM block. The purpose of this technique is to ensure that, were an obstacle to be detected, it would be possible to measure the distance between the obstacle and the autonomous vehicle. The disadvantage of this method lies in the fact that it employs cascade detectors, which may increase calculation time.

In this context, the purpose of our contribution is to suggest an approach based on voting algorithms so as to reduce the impact of sensor faults and to ensure safe autonomous driving. This method has been widely used for approaching fault tolerance problems. In fact, this technique has been broadly applied in the field of engineering, thanks to the ease of its design and its implementation. It has been applied in many areas such as graphology [ONA 16], medical monitoring [GAL 15], electric vehicles [BOU 13, RAI 16, SHA 17] and aeronautics [KAS 14].

5.3. Fault-tolerant control architecture

In order to ensure its reliability, the autonomous vehicle is equipped with three sensor technologies that may be used for longitudinal control: a long-range radar, a lidar and a Mobileye smart camera, which calculates the distance. Each of these sensors has a specific limitation, and does not work at its best under certain circumstances. For example, as the lidar is based on sending and receiving spectrum light waves, it can lead to aberrant detections in bad weather, such as in the presence of fog, heavy rain or dust particles.

In addition, the lidar cannot detect color or contrast, and does not guarantee a good identification of the nature of the objects. On the other hand, the radar may not detect the vehicle ahead, due to the difference in speed between the leading and following vehicles. The radar may also be unable to detect motorcyclists or other vehicles ahead when they are outside the center of the lane. In addition, the smart camera may not work thoroughly if the lens is partially or completely obstructed. The latter does not guarantee a 100% rate of detection of the vehicle ahead. In this way, weather conditions greatly influence the recognition and response capabilities of the smart camera.

As we mentioned earlier, each sensor has limitations that mainly depend on the technology employed and its operating area. Each sensor may be faulty in its operating area due to a software or hardware failure, leading to erroneous distance measurement. In order to solve these malfunctions, fault-tolerant control should manage all possible scenarios and keep autonomous driving stable and safe, as shown in Figure 5.3. Voting algorithms are intended to diagnose the faulty sensor by making a comparison between healthy signals, with the aim of ensuring stable and safe driving.

The monitoring speed is set by the reference speed generating block. The latter is deduced from the safety distance relation, resulting from the following relation [TOU 08]:

[5.1]

Then, from equation [5.1], we can deduce the reference speed as follows:

[5.2]

where d_stop represents the distance to be respected at the stop and h represents the inter-vehicle time (usually between one second and three seconds).

5.3.1. Vehicle dynamics modeling

Vehicle dynamics is known for being very complex and strongly interconnected. In order to illustrate the dynamics governing the behavior of a vehicle, it is necessary to resort to a number of simplifying assumptions, so as to reduce the model’s complexity [HED 97, BOU 17b]:

• –only one fault per sensor will be considered;
• –the road will be supposed to be flat (with no slopes, no inclination);
• –the lateral dynamics of the vehicle will not be taken into consideration;
• –rolling, lacing and pitching movements will not be taken into account.

Considering the assumptions made, the longitudinal dynamics of the vehicle can be expressed as follows:

[5.3]

When adopting a bicycle model [HED 97, BOU 17a]:

We obtain the following equation:

[5.4]

By substituting F_xf and F_xr in equation [5.3], we get:

[5.5]

Notation	Definition	Unit
m	Mass of the vehicle	kg
Vx	Vehicle speed	ms–1
Fxi	ith wheel–ground contact force	N
Fa	Aerodynamic force	N
	Overall inertia of the front axle	kgm2
	Inertia of the ith front wheel	kgm2
	Angular acceleration of the ith wheel	rad.s–1
Tm	Engine coupler	Nm
r	Wheel radius	m
Fri	Rolling friction of the ith wheel	N
Tbi	Braking torque of the ith wheel	Nm
T rf/r	Rolling resistance: front vs. rear axle	Nm
Jri	Inertia of the ith rear wheel	kgm2
Jr	Overall inertia of the rear axle	kgm2

Also, let us assume that the wheels do not slip throughout the maneuver. This assumption can be interpreted by the following equation:

which leads to rω_r = V_x, and then .

Replacing in equation [5.5], we obtain:

[5.6]

We then adopt the following notation:

Equation [5.6] becomes:

[5.7]

where: , and a as well as b are aerodynamic coefficients. T_eq represents the braking and accelerating torque. In order to overcome the delay problem related to the dynamics of braking and accelerating, we will assume that the accelerating/braking pair is governed by a first-order equation, written as follows:

[5.8]

Finally, the longitudinal dynamics of the vehicle is expressed by the following relation [PHA 97]:

[5.9]

5.4. Voting algorithms

As shown in Figure 5.3, a voting algorithm is able to choose a safe distance measure only by using sensor measurement signals. This distance is obtained by comparing the signals produced by the sensor/processing units, after which the voting logic chooses the most reliable signal. The voting techniques used for this task are: maximum likelihood, weighted averages and history-based weighted average.

5.4.1. Maximum likelihood voting (MLV)

The first work on MLV was published in [LEU 95]. The basic idea of this approach is to choose one of the input signals, ensuring the highest probability in terms of reliability. Thus, for each dynamic reliability dedicated to the sensors written as f_i, with i = 1,2,………N, where N represents the number of sensors, conditional probabilities Δ_i are calculated based on the equation below [KIM 96, RAI 16]:

[5.10]

In our case study, we use three sensors (intelligent camera, radar and lidar), so N = 3. D_max is a fixed real number, corresponding to the threshold (this is fixed at 10% of the reference range in order to apprehend the effect of noise on the voting algorithm), where x_k is the reference input.

The probability of each input sensor is calculated by the following formula:

[5.11]

The output signal is then chosen in such a way that it satisfies the maximum likelihood:

[5.12]

5.4.2. Weighted averages (WA)

The weighted averages method is based on the weight coefficients attributed to each sensor. Therefore, the output of the algorithm represents the average of the measured signals. Furthermore, the output signal is determined as a continuous function of the redundant inputs when we use weighted averages. In fact, this technique aims to reduce the transitory effect, in such a way that no switching is allowed, and the faulty sensor is isolated, with zero weight attributed to it (or a minimal weight, in the practical case) [BRO 75]. Therefore, considering an input signal x_i, with i = 1,2 … … … N, where N is the number of sensors, the output signal y is determined by the following equation:

[5.13]

where w_i, with i = 1,2, … … … N, are the weight coefficients. These coefficients correspond to the sum of the inverse distances of all input signals compared to the others, so that if the signals are close to the others, it implies a high weight.

In addition, the weight is obtained as follows:

[5.14]

where K_{i j} are the inverse of the distances between the input signals, which are represented by:

[5.15]

5.4.3. History-based weighted average (HBWA)

The underlying idea of this technique is to use a history that informs about the state of the sensors. Indeed, over time, a reliability index is accumulated for each sensor, so that the sensor that is chosen at the output of the algorithm has the highest reliability. Latif-Shabgahi, Bass and Bennett [LAT 01] presented two philosophies for this technique. The first one, called agreement state indicator weighted average, employs a dynamic weighting function based on the indicator’s status. Then, the weights are proportionally related to the indicator history. The second method, called module elimination weighted average, calculates an average of the indicator’s history, and then every entry with an historic indicator below this average is considered unreliable; therefore, its weight is set to zero. The HBWA technique is based on the algorithm introduced below:

1. 1) For input signals x_i and x_j, coefficients k_ij are calculated in the following form:

[5.16]

where i, j = 1,2,… … …N and i≠j.

1. 2) Using adjustable threshold parameters a and q, the agreement level for each input is given by:

[5.17]

1. 3) The level of consensus S_i calculated as:

[5.18]

1. 4) The w_i weighting for input i is calculated in the following form:
   [5.19]

where P_i(n) and P_avg(n) are the indicators of the i^th sensor’s status in the n^th calculation cycle and the average of the indicating status in the n^th calculation cycle, respectively. These will be discussed later.

1. 5) An initial output, denoted as , is then calculated using the input signals of sensor x_i and weights w_i:

[5.20]

where i = 1,2,… … … N.

1. 6) The distances between the initial output signal and the input signal for each sensor are calculated in order to determine the appropriate final output:

[5.21]

where i = 1,2, … … … N.

1. 7) The final output y is expressed by the following equation:

[5.22]

1. 8) The history H(n) of the i^th sensor is recorded in the n^th voting cycle, calculating the consensus h(n) level of the current cycle, based on the distance between each sensor x_i and the final output y:

[5.23]

Thereafter, the history is given by:

[5.24]

1. 9) The P_i(n) status indicator of the i^th input in the n^th voting cycle is calculated by the following equation:

[5.25]

1. 10) Finally, the indicator of P_avg(n) average status, in the n^th voting cycle, is determined as follows:

[5.26]

5.5. Simulation results

In this section, we will introduce the simulation results obtained from our fault-tolerant control architecture. The first step is to determine the MLV and HBWA threshold parameters. In order to achieve this, we employ an iterative method based on extended simulation. The threshold values are given as follows: a = 0.007, q = 700 and D_max = 1.37. Each sensor has a variable reliability with regards to its operating range.

On the other hand, the reliability of each sensor technology differs. Thus, the lidar is reputed to be more reliable than the other two, and its reliability is not affected by a variation regarding the obstacle’s distance. Also, the reliability of the lidar is fixed at f_Lidar = 99%, whereas the radar is more reliable over long distances when the reliability relationship f_Radar = 0.00067.d + 0.95 is respected. The smart camera has better precision over short distances with an expression of reliability equal to f_camera = - 0.0013.d + 0.97.

For this simulation, we adopted a scenario of following a leading vehicle while respecting a safety distance relative to the speed of the autonomous vehicle. This scenario is schematically illustrated in Figure 5.3. Thus, the leading vehicle starts at a speed of 20 m/s, and keeps it constant until t = 25 s. The autonomous vehicle initially drives at a speed of 25 m/s and quickly stabilizes and maintains 20 m/s in order to respect the safety distance of formula [5.2]. At t = 25 s, the leading vehicle rapidly puts on the brakes and reaches zero speed at t = 33 s, then remaining at rest.

By regulating the inter-vehicle distance, the autonomous vehicle also quickly puts the brakes on until it stops. When the cars are stopped, the distance between the leading vehicle and the autonomous vehicle is equal to 3 m, that is, distance d_stop defined in formula [5.2]; in our case, this stopping distance is considered when it reaches 3 m. At t = 40 s, the leading vehicle initiates an acceleration and becomes stable at a speed of 20 m/s until = 60 s, before putting on the brakes, in order to reach zero speed at t = 65 s and maintain its standstill until = 75 s, when the leading vehicle re-accelerates and reaches a speed of 12 m/s. As a result, the autonomous vehicle regulates and adapts its speed in accordance with that of the leader, always keeping a suitable safety distance.

In order to separately test the voting algorithms developed in section 5.4, we will emulate faults occurring on exteroceptive sensors (lidar, radar and camera). Figure 5.9 represents the emulation of faults on sensors, which are often associated with complete failures producing null signals. Also, the faults on sensors are emulated at moments t = [7 s 13 s] for the radar, t = [64 s 74 s] for the camera and t = [35 s 41 s [∪] 130 s 170 s] for the lidar.

In order to ensure a certain tolerance to the faults injected, the purpose of the voting algorithms is to detect the faulty sensor, choosing the most reliable measurement. Thus, the generated reference speed will be the most representative of reality, which would maximize the safety of the controlled vehicle.

The first algorithm we tested is that of maximum likelihood. Figure 5.11 shows the output of the MLV algorithm compared to the actual inter-vehicle distance (reference distance). The output of the MLV algorithm seems to provide a perfect picture of the true distance between the two vehicles (with a detection rate and an error rate for each sensor), the faults emulated on the sensors having been eliminated. This is related to the fact that the voting logic always chooses the most reliable sensor within the chosen distance range. Thus, Figure 5.12 shows the transitions between sensors at the output algorithm. Moreover, as the lidar has the best reliability, it will always be preferred over the others. However, as soon as this fails, the algorithm will choose the most reliable sensor depending on the range of distance measured. Therefore, the algorithm chooses the camera at t = [35 s 41 s], because the distance is short and the camera is more reliable in this case, whereas at t = [130 s 170 s], the algorithm chooses the radar, because the radar is more suitable at this distance.

For the second algorithm, the algorithm’s output compared to the reference distance (actual distance) is shown in Figure 5.13. The output of the WA algorithm seems to deviate a little from the actual distance. Such deviation may be explained by the fact that this algorithm offers an output weighted by the coefficients it calculates from sensor inputs. Thus, the output represents a fusion between the three signals admitted at the input. In addition, if we consider the evolution of the weights of the three sensors shown in Figure 5.14, then we will note that the weights do not completely disappear when the sensor fails, which influences the result offered by the algorithm. In addition, even when all the sensors are working under normal operating conditions, the weighting calculation mechanism favors the sensors providing the closest results, which can be detrimental to the proper functioning of the algorithm.

Unlike the WA algorithm, the history-based weighted average algorithm offers a much better result. The output of the HBWA algorithm is shown in Figure 5.15, where we can reckon that it accurately represents inter-vehicle distance.

By keeping a record of the state of each sensor (Figure 5.16), this technique favors the sensor which has worked under normal operating conditions (without failure) for the longest time, and contrary to the weighted averages (WA), where the output is an average of the three inputs, the HBWA algorithm chooses one sensor among three. This choice responds to the limitation set by the history, as well as by the closest sensor to a weighted average of the three sensors.

A comparison of the algorithm’s output inter-vehicle distance between the three voting techniques is shown in Figure 5.17, where we can note a clear advantage of the MLV and HBWA techniques over the WA technique, whereas there is a slight difference between the first two. The latter is still the most powerful algorithm, as long as it can provide a proper image of each sensor’s reliability. On the other hand, since the reference speed (the comparison in Figure 5.18) is directly calculated from the output inter-vehicle distance of voting algorithms, we can note the same appearance for the speed reference profile as for that of the inter-vehicle distance. This proves the dependence of the generation of the reference speed on the proper functioning of the voting algorithm.

The dynamic performance of the autonomous vehicle is also dependent on the performance of the algorithms, as we can infer from the comparison of the speed (Figure 5.19) and acceleration (Figure 5.20) profiles. The strong variations in the reference speed of the WA and the smaller variations of the MLV are clearly visible on the speed profile. These variations in speed influence the acceleration, even though this is still considered acceptable for the case studied.

5.6. Conclusion

Accident statistics on roads point to the human factor as one of the major causes. As a solution, replacing the driver in relation to different driving actions seems appropriate both in terms of accidentology and other societal problems, such as transport-related pollution, by optimizing consumption. Nonetheless, this requires sensor technologies that deliver robust and reliable information in relation to the variety of situations that a vehicle may come upon. In order to remedy the problem of exteroceptive sensor faults in the case of longitudinal autonomous driving, a comparative study of three voting algorithms (MLV, WA and HBWA) was featured. In fact, this approach proved its capacity and efficiency in the diagnosis of faulty sensors. In addition, the history-based weighted average (HBWA) algorithm offered better performance than the other two algorithms. The efficient performance of the HBWA algorithm is derived from the fact that it chooses one sensor among three, making sure that the selected sensor has not experienced failure on previous occasions. This ability represents a significant advantage in comparison with the MLV, which makes an immediate switch as soon as the fault appears. On the other hand, the setting of threshold parameters for MLV and HBWA is still difficult to obtain due to the chattering effect. On the other hand, the operating range must be properly studied because a fixed threshold cannot grant a good performance along a wide range of operation.

The results presented in this study have shown that voting algorithms can ensure fault tolerance for autonomous vehicles. Indeed, the ability to diagnose faulty sensors in order to use other sensors will ensure a safe and stable trajectory for an autonomous vehicle, as it will maximize safety for human passengers. On the one hand, the simplicity of the design and of the implementation of voting algorithms can guarantee this capability. On the other hand, these algorithms do not require much calculation time to identify faulty sensors, which makes us privilege them over data fusion, which requires a great amount of calculation time.