17.4.1 Classification by Axes

The capacity of a robot movement is determined by the movements in the axes that are possible to be controlled. It is very similar to movements in numerical control. Industrial robots are made in different types and sizes. They can make various arm movements and have different motion systems.

17.4.2 Classification of Robots by Coordinate Systems

1. Cartesian coordinate system: All robot movements; they are at right angles to each other. This is a robot design form with the most limited freedom of movement. The process for assembling some parts is done by Cartesian configuration robots. This robot shape has parts moving in three axes perpendicular to each other. Moving parts X, Y, and Z move parallel to the Cartesian coordinate system axes. The robot can move the lever to points within the three‐dimensional rectangular prism volume.
2. Cylindrical coordinate system: It can rotate around a basic bed and has a main body with other limbs. The movement is provided vertically and the main body is provided radially when the axis is accepted. Therefore, within the working volume, a zone is formed up to the volume of the main body, which the robot cannot reach. In addition, because of mechanical properties, the trunk cannot fully rotate 360°.
3. Spherical coordinate system: Mathematically, the spherical coordinate system has three axes, two circular axes and one linear axis. There are basically two movements. These are horizontal and vertical rotation. A third movement is the linear movement of the elongation arm. The linear motion behaves just like the movement of any coordinate from the Cartesian coordinates.
4. Rotating coordinate system: If a robot does the work in a circular motion, this type of robot is called a robot with a rotating coordinate system. The connections of the robot arm are mounted on the body so that it rotates around it and carries two separate sections with similar abutments. Rotating parts can be mounted horizontally and vertically.
5. Multiaxis (articulated) robots: Arm‐jointed robots, according to their abilities, were made to undertake the tasks that the human arm could fulfill. They move freely in six axes. Three are for arm movement and the other three are for wrist movement. The biggest advantage of this connection is the ability to reach every point in the work area.
6. SCARA Robotlar: The Selective Compliance Assembly stands for the Robotic Arm, i.e. the assembly robot arm that matches the selected. SCARA‐type robots are a type of robot with very high speed and best repetition capability. There are three general features: accuracy, high speed, and easy installation.

17.4.3 Other Robotic Classifications

There are many other classification methods such as classification by skill level, classification by control type, classification by technology level, classification by energy source, classification according to their work, holders, and limiters in robots.

• Robot holders: It is used very easily in the assembly process, welding operations, and painting operations. However, in an assembly line, it is seen that the same holder can do more than one job, or carry parts of various features, which can be seen as functional difficulties. In this case, it is a fact that a general purpose robot is needed. Robot hands have actuators based on various technologies that produce the power required for the movement of joints. The most widely used actuator technologies are electric motors, hydraulic actuators, and pneumatic actuators.
• Mechanical holders: They hold the parts between the mechanical holders and the fingers move mechanically.
• Vacuum holders: It is used to hold flat objects such as glass. The workpiece is held with the help of the vacuum formed between the holder and the holder.
• Magnetic holders: They are used to hold metal materials.
• Adhesive holders: It is used for carrying flexible materials such as adhesive materials and fabric.

The specifications of the programming languages of robots are listed below:

• The programming language is specific to the robot. (Example: Melfa Basic IV for Mitsubishi robot arms.)
• The language sets the electrical traffic for the robot components, processing the sensor data.
• It is simple to use and is limited by the capacity of the robot.

17.5.1 Artificial Intelligence in Image Processing

The information expected to be obtained by visual processing has gone far beyond the labeling of a cat or dog on the photograph. The factors leading to this increase in expectation are both the increase in the number of data that can be processed and the development of the algorithms used. We are now able to work in important areas such as visual processing techniques to diagnose cancer cells or to develop sensors to be used in military fields. This advanced technology in image processing is made possible by the use of side heads such as machine learning, deep learning, and artificial neural networks.

Face recognition technologies, driverless tools, or Google's translation mode are among the simplest examples of machine learning‐based image processing systems. Google Translate creates the image processing system in the translation application by machine learning and artificial neural network algorithms.

Google Translate's camera‐to‐camera mode allows you to save letters of any language with image processing and machine learning. When we scan the word with the camera, it combines the letters in the system database to reveal the words. After the emergence of the word or sentence as in the classic Google Translate application with its own language processing algorithm translates words. The important point for this mode to work is that the words you read with the camera are recognized by the system, which is thanks to the machine learning.

Artificial intelligence, in addition to providing advanced technology in image processing, also reduces processing time. The electro‐optical sensor systems used in the military field have been used for the determination of mobile or fixed targets for many years. Nowadays, these systems are strengthened by artificial intelligence to provide both faster and more precise target detection.

17.5.2 Image Processing Techniques

Recently, new development techniques have been developed to support classical image processing methods or to walk through completely new algorithms. Most of the techniques are designed to develop images from unmanned spacecraft, space probes, and military reconnaissance flights. However, the developments have shown the effect in all areas where image processing is used.

A digital image consists of the numbers that gives information about the image or color. The main advantages of digital image processing methods are multidirectional scanning capability, reproducibility, and original data accuracy. Major image processing techniques are image preprocessing, image enhancement, image separation, feature extraction, and image classification.

17.5.3 Artificial Intelligence Support in Digital Image Processing

All of these methods are the basic principles of image processing, which can help us to define the definition of image processing. The widespread use of image processing in the technological field and the expectation of increase in image processing applications necessitated the introduction of artificial intelligence into image processing.

17.6.1 Problem‐solving Process

The definition of the problem‐solving process depends on the training and experience of the researchers. For example, Bel et al. has proposed several approaches to problem solving and decision‐making that vary quantitatively according to people's intuition. Generally, six basic steps can be observed in the process: identifying and defining the problem, setting criteria for finding a solution, creating alternatives, searching for and evaluating solutions, making selections, and making recommendations and implementing them.

Some scientists use different classifications. Simon's classical approach, for example, has three phases: intelligence, design, and choice. Although the process in Figure 17.2 is shown as linear, it is rarely linear. In real life, some of these steps can be combined, some steps can be the basic steps, or revisions can be made in the initial steps. In short, this process is iterative. A brief description of each step is given below.

• Step 1 – Identifying and defining the problem: A problem (or opportunity) must be recognized first. The magnitude and significance of the problem (or opportunity) is specified and identified.
• Step 2 – Determining criteria: The solution of a problem depends on the criterion used to compare possible alternatives. For example, a good investment depends on criteria such as security, liquidity, and rejection rate. In this step, we determine the criteria and their relative importance to each other.
• Step 3 – Creating alternatives: According to the definition, there must be two or more chances to make a decision. Creating potential solutions requires creativity and intelligence.
• Step 4 – Finding and evaluating solutions: In this step, solution options are examined in the light of predetermined criteria. This step is basically a search process because we are trying to find the best or “good enough” solutions. Several step‐by‐step methodologies can be used in this step.
• Step 5 – Making choices and making recommendations: The result of the search is choosing a solution to recommend as a remedy to a problem.
• Step 6 – Implementation: To solve the problem, the recommended solution must be successfully implemented. In fact, this process is more complex because at each step, there may be several interim decisions, each following a similar process.

Applied Artificial Intelligence technologies can be used to support all of these six steps. However, most of the artificial intelligence movements take place in steps 4 and 5. In particular, expert systems are used to find solutions from the presumed alternatives. The role of artificial intelligence is mainly to manage search and evaluation by using some inference capabilities. Today, despite the limited role of artificial intelligence, it is hoped that after a certain time, technologies will play a greater role in the steps of the process. Nevertheless, artificial intelligence technologies have another great advantage. Although artificial intelligence effectively uses only two steps of the problem‐solving process, artificial intelligence is used in many other tasks that are not classified as problem solving and decision‐making. For example, expert systems help to develop computer commands. Expert systems are also used to simulate people's help centers, which provide information in catalogs and manuals, and for planning, complex scheduling, and information interpretation.

17.7.1 Optimization Techniques in Artificial Intelligence

Optimization techniques are called metaintuitive and research techniques. The aim of these algorithms is solving the problem. Some of the algorithms about the solution of the problem can guarantee the exact solution, but an appropriate solution can guarantee it.

Application results show that artificial intelligence optimization algorithms are problem‐dependent algorithms. In other words, they may be successful in one problem and not in the same way for another problem. The most important artificial intelligence optimization algorithms are artificial heat treatment algorithms, genetic algorithms, and taboo search algorithms [6].

• Artificial heat treatment algorithm: Artificial heat treatment algorithms, which are also mentioned in the literature as an annealing simulation method, are basically a different form of application of the best search methods by accepting the better solution encountered first. Unlike the search method with gradient, which aims to go to a better point at each step, the main disadvantage of this algorithm is that it may lead to a worse solution with a decreasing probability in the process.
• Genetic algorithm: Genetic algorithms were proposed by Holland based on the biological evolution process. Unlike the other two methods described here, the genetic algorithm evaluates multiple solutions at the same time instead of a single solution. This feature is described as the parallel search feature of the genetic algorithm. Problem solving is expressed by a chromosome consisting of genes.
• Taboo search: Taboo search is basically an adaptation of the best fit strategy around a solution that seeks the best local solution. The last steps taken by the algorithm are declared taboo so that the algorithm does not return to the same local best solution immediately after it leaves a local best solution. The taboo list is dynamic. Each time a new element enters the taboo list, the most remaining element in the taboo list is removed from the list. In this way, the algorithm is given a memory.

Applications of artificial intelligence optimization techniques are listed below:

• Scheduling: Production schedules, workforce planning, class schedules, machine scheduling, and business schedules
• Telecommunication: Search route problems, path assignment problems, network design for service, and customer account planning
• Design: Computer‐aided design, transport network design, and architectural area planning
• Production and finance: Manufacturing problems, material requirements planning (MRP) capacities, selection of production department, production planning, cost calculations, and stock market forecasts
• Location determination and distribution: Multiple trading problems, trade distribution planning, oil, and mineral exploration
• Route: Vehicle route problems, transportation routes problems, mobile salesman problems, computer networks, and airline route problems
• Logic and artificial intelligence: Maximum satisfaction problems, logic of probability, clustering, model classification and determination, and data storage problems
• Graphics optimization: Graphics partitioning problems, graphic color problems, graphics selection problems, P‐median problems, and image processing problems
• Technology: Seismic technology problems, electrical power distribution problems, engineering structure design, and robot movements
• General combinational optimization problems: 0–1 Programming problems, partial and multiple optimization, nonlinear programming, “all‐or‐none” networks, and general optimization

17.8.1 History of Autonomous System

After Turing, the father of computer science, John McCarthy is the father of artificial intelligence. He is also the one who indirectly brings us together with Siri. He established the Artificial Intelligence Laboratory at Stanford University and developed the list processing (LISP), which is considered the programming language of artificial intelligence based on lambda calculations. He first described the concept as science and engineering for the construction of intelligent machines, especially intelligent computer programs, at a conference in Dartmouth in 1956. The term reason is, of course, the subject of scientific and philosophical discussions. Some researchers believe that strong artificial intelligence can be done, and that we can develop an artificial intelligence that matches or exceeds human intelligence, and that it is possible to accomplish any intellectual task that a human can accomplish. Other researchers believe that weak artificial intelligence applications may be possible and that software can be used to perform certain tasks much better than humans, but can never cover all of human cognitive abilities. No one is yet sure which one is right.

When we look at the history of autonomous vehicles in 1920–1930, some prominent systems already gave the gospel of autonomous vehicles. The first models that could travel on their own were introduced in the 1980s. The first vehicle was built in 1984 by Carnegie Mellon University with Navlab and autonomous land vehicle (ALV) projects. This was followed in 1987 by the Eureka Prometheus project, jointly implemented by Mercedes‐Benz and the Bundeswehr University. After these two vehicles, countless companies have built thousands of autonomous cars, some of which are currently in traffic in several countries.

Autonomous automobiles seem to lead to a revolution in the passenger car sector, leading to a major transformation in the automotive sector. The rapid development of driver vehicle technology will soon change the rules of Henry Ford's game dramatically. Especially, the automotive industry will be one of the most affected units from this development. It is thought that driverless or autonomous cars will lead to an unprecedented economic, social, and environmental change in the near future. It is also clear that it will lead to an excellent equality of social status on the basis of citizens. Autonomous tools ensure that young people, the elderly, or physically disabled people have the same level of freedom to travel.

In addition to a few fundamental changes in the technological field, changes in the demands of consumers are being caused by the production of more powerful batteries, the provision of more environmentally friendly fuels, and the production of autonomous vehicles. These changes in the demands of consumers also lead to drastic changes in the planning of the manufacturing companies. In addition, the sharing economy has reached the automotive industry with BMW's diverse services, such as DriveNow in Paris or Autolib. Tesla has taken advantage of such developments so far, offering products that can be offered to the market among the most successful candidates. However, manufacturers in other companies such as BMW, Toyota, and General Motors in Silicon Valley are also making ambitious developments in this field. These developments seem to be one of the biggest steps toward the fourth industrial revolution in the automotive sector. Roland Berger, one of the largest strategic consulting firms based in Europe, says that the automotive, technology, and telecommunications sectors will unite at some point. Nowadays, considering the vehicles that have gained value in the market, we see that approximately 30% of the vehicles are based on electronics, and this ratio is expected to increase to 80% with the future innovations of the sector.

17.8.2 What Is an Autonomous Car?

Autonomous vehicles are automobiles that can go on the road without the intervention of the driver by detecting the road, traffic flow, and environment without the need of a driver thanks to the automatic control systems. Autonomous vehicles can detect objects around them by using technologies and techniques such as radar, light detection and ranging (LIDAR), GPS, odometry, and computer vision. Let us see detailed information about the sensors.

LIDAR, which stands for light detection and ranging, was developed in 1960s for the detection of airborne submarines and started to be used in 1970s. In the following years, there has been an increase in the usage area and types of sensors using LIDAR technique, both from air and land. LIDAR technology is currently used in architecture, archeology, urban planning, oil and gas exploration, mapping, and forest and water research. An LIDAR instrument principally consists of a laser, scanner, and specialized GPS receiver [7]. You can see an example of LIDAR view in Figure 17.3.

Both natural and manmade environments can be displayed by LIDAR. In a study, it is said to be very advantageous to use LIDAR for the emergency response operations [7].

GPS, which stands for global positioning system, is a system that helps to provide a location with satellites, receivers, and algorithms. It is generally used for traveling such as air, sea, and land locations. GPS works at all times and in almost all weather conditions [8].

In general, there are five important usage of GPS:

• Location: Detects the location and position.
• Navigation: Gets from a location to another.
• Tracking: Monitors objects or personal movement.
• Mapping: Creates maps of the world.
• Timing: Calculates the precise time measurements.

17.8.3 Literature of Autonomous Car

There are several studies describing the level of autonomy in the literature. However, the most common of these is suggested by Thomas B. Sheridan [9]. According to this scale, the order of increasing autonomy level from 1 to 10 is given below.

1. Computer will not help; human makes the entire task.
2. The computer provides a set of motion options that includes all options.
3. The computer reduces the options up to a few.
4. The computer recommends a single action.
5. The computer executes the movement if the person approves.
6. The computer gives the person the right to refuse for a limited time before automatic execution.
7. The computer automatically executes the task and informs the person necessarily.
8. The computer informs the person after the automatic execution if he wishes.
9. Computer automatically informs people if they want after the automatic execution.
10. The computer decides and executes everything and does not care about people.

The first developments in the field of advanced vehicle control systems were discovered by the General Motors Research Group in the early 1960s. This group developed and exemplified the automatic control of automobile steering, speed, and brakes on test tracks [10]. This research triggered other research groups. In the late 1960s, Ohio State University and Massachusetts University of Technology began working on the application of these techniques to address urban transport problems.

From the first days of intelligent transportation systems, we can see that the results of the studies carried out in the laboratory environment can be seen in advanced driving support systems in the vehicles on sale. Large vehicle manufacturers are constantly researching new and improved driving assistance systems [11]. The purpose of these systems is to help drivers rather than replace them. The most important developments in this field are cruise control [12], dynamic stability control [9], antilock brakes (ABs) [13], and pedestrian detection with night vision systems [14], collision avoidance [15], and semiautonomous parking system [16]. In these applications, longitudinal control (acceleration and braking actions) is mostly emphasized.

One of the leaders of lateral control studies on the steering wheel movement is Ackermann [17]. Ackermann's approach is used to combine active steering with feedback of the swing rate to strongly distinguish between skidding and lateral movements. It is possible to control nonlinear dynamic systems such as steering control thanks to techniques that allow fast, trouble‐free, and high‐quality control. In order for the vehicle to continue to follow the reference trajectory; fuzzy logic [18], optimization of linear matrix inequalities [19], and skew rate control [20].

Another important change in the steering system has been the replacement of hydraulic power steering system (HPS) with electric power steering system (EPS) in new‐generation vehicles. Such a system has advanced control simulations [17–21]. Sugeno and Murakami published a landmark publication in 1984. They have shown that any industrial process can be controlled by a simple process of human process or human experience [18]. The problem is reduced to finding the right control rules and fine‐tuning them based on a drive experience. Several studies have compared fuzzy control and classical control techniques in autonomous vehicles, and fuzzy controllers have shown good results [22].

The first serious step in fuzzy logic was demonstrated in an article written by Azerbaijani scientist Lütfi Aliasker Zade in 1965 [23]. In this study, Zade stated that a large part of human thought is blurred and not precise. Therefore, human reasoning system cannot be expressed with dual logic. Human logic uses expressions such as hot and cold, as well as expressions such as warm and cool.

After the introduction of fuzzy logic, it was not accepted for a long time in the Western world. The idea found its place in practice only nine years after its production, when Mamdani carried out the inspection of a steam engine [24]. As the 1990s approached, fuzzy logic attracted attention in Japan and began to be used in many applications. In 1987, the Japanese Sendai subway designed by Hitachi managed to reduce energy consumption by 10% while using fuzzy logic to provide more comfortable travel, smoother slowdown, and acceleration in the subway [25]. The products of fuzzy logic applications were introduced to consumers in Japan in the early 1990s. According to the degree of pollution in dishwashers and washing machines, washing technology has yielded fruitful results [26]. The Western world could not remain indifferent to these developments, and in 1993, IEEE Transactions on Fuzzy Systems magazine began to publish.

In order to develop autonomous vehicles, there are models developed on ATVs (all‐terrain vehicles). These models have a structure built on the acceleration and steering systems of the vehicle, as in almost all autonomous vehicle studies. Rather than developing a new vehicle from the start, it seems reasonable to make an already existing vehicle autonomous. One of the reasons for this is the opportunity to work with vehicles that have already been tested to work in all harsh conditions. Another reason is that the product that will be produced as a result of the studies will be independent of the vehicle to be integrated, and this will maximize the applicability. In this context, four teams whose names are Ensco [27], Spirit of Las Vegas [28], Overbot [29], and Cajunbot [30] that are built on the ATVs shown in Figure 17.4 produced for the DARPA Grand Challenge in 2004 and 2005 were examined. All the four teams controlled acceleration using a servo motor on the accelerator pedal. For steering, Cajunbot and Spirit of Las Vegas teams preferred DC Motors, while Ensco and Overbot teams preferred servo motors.

Researchers use DGPS (differential global positioning system) for positioning. In this way, the vehicle is able to determine its position with an average error of 10 cm instead of locating with an average of 15 m error with normal GPS [31]. The operating principle of DGPS is that in addition to the normal GPS system, an antenna with a location on the ground is taken as a reference. The satellites are detected with the reference antenna after the connection with the user and the reference antenna, and the location of the user is detected with very small errors. DGPS is the only solution yet available for positioning in outdoor applications.

In order to determine the exact movement of the vehicle, the researchers used a three‐axis gyroscope and an accelerometer. The three‐axis gyroscope is a key instrument in marine and aviation and is now being used in land vehicles. It works according to the basic physics rules and is based on the conservation of angular momentum. The gyroscope, which is also used in smartphones today, can easily be determined in which direction an object is inclined.

The accelerometer measures the acceleration to which a mass is exposed. The principle of operation is to measure the forces applied to the test mass from the mass in the reference axis. With this device, the forces applied to the mass such as acceleration, deceleration, and skidding can be measured.

Researchers have chosen a wide range of power sources for their projects. Power is an important issue because it is a major factor determining all dynamics in a drone. Although it is first considered that it meets the needs when supplying power to the system, the preferred power source should not place excessive volume and weight on the vehicle. In this context, internal combustion engines, high‐current batteries, generators, and uninterruptible power supplies are used as power supplies. When required, the required voltages are produced with DC/DC converters.

Processors, random access memory (RAM), and hard drives used to process data and control devices can be catered for a wide variety of projects. Because the vehicles will move, it is generally taken care not to use moving parts. In this context, it is seen that solid‐state disks are used for storage purposes. According to the needs of the project, wireless technologies are used to send the data to the computer.

Environment detection is the most important need of unmanned vehicles. The vehicles have to perform all their movements in the light of the data coming from the sensors. LIDAR, cameras, and ultrasonic sensors are the most commonly used sensors in unmanned land vehicle projects [32]. In the aforementioned studies, all teams used both LIDAR and camera. LIDAR is an optical distance sensing technology that can detect and measure objects by direction and distance, usually by laser‐transmitted rays. LIDAR systems used in unmanned land vehicles have laser range finders of the type, which are reflected by rotating mirrors [33]. The device is very high‐priced and helps to identify objects around the vehicle. Google's driverless vehicle can detect the perimeter of the vehicle.

The cameras used in the studies are two of types, mono and stereo. Mono cameras are generally used to monitor the vehicle's surrounding information, while stereo cameras are used to understand depth information. The images taken with all these sensors are combined, interpreted, and interpreted with advanced image processing methods. Open source software is widely used in the studies. Developed according to project‐specific requirements, these softwares are shaped according to the requirements of the hardware used.

17.8.4 How Does an Autonomous Car Work?

The autopilot drive of autonomous vehicles starts briefly with the ultrasonic sensors on its wheels detecting the positions of vehicles that are braking or parked, and data from a wide range of sensors are analyzed with a central computer system, and events such as steering control, braking, and acceleration are performed. This event can only be described as a beginning. With the technology becoming cheaper, the future of driverless vehicles becomes increasingly realizable.

In autonomous vehicles, systems such as sensor data, extensive data analysis, machine learning, and the M2M (machine‐to‐machine) communication system are essential for the successful implementation of the Internet of Things philosophy. The autonomous vehicle projects currently on the market, evaluates the sensor data, analyzes and provides machine movements as well as machine learning. At present, there are not enough autonomous vehicles in today's traffic to design a vehicle suitable for M2M communication system. However, when a certain threshold is reached in this issue, interesting developments such as autonomous traffic control systems and autonomous traffic control can also be allowed.

Such a system would have a very extraordinary impact on industry and society as a whole. At the level of community and living standards, the biggest impacts are the decrease in the number of accidents, the increase in the use of passenger cars that easily accompany, and the decrease in crowds in public transportation centers. Together with the increase in the use of vehicles, the increase in the amount of fuel consumption because of the increase in oil trade seems to cause a new activity in the country's economy.

In the next eight years, Korea plans to produce autonomous vehicles that will be developed entirely with its own technology. Currently in the monopoly of American‐ and European‐based companies in this sector in the future, market is expected to expand further into Asia. Until 2019, the Korean Government set aside a budget of approximately US$ 2 billion to produce the eight main components required for driverless vehicles, including visual sensors and radars. However, in 2024, 100% of Korean‐made autonomous vehicles are planned to hit the road. It is clear that autonomous vehicles have a very bright future around the world. Korean car manufacturers and technology giants such as Samsung have already been thrown into developments in the autonomous vehicle industry. The structure of an autonomous car is given in Figure 17.5.

There are two different types of autonomous car systems. The traditional approach divides the task into some basic sections, which is lane detection [34,35], path planning [36,37], and control logic [35,38], and is often investigated separately. Image processing techniques such as Hough Transform, edge detection algorithms are usually used for detecting lane marks. After detecting lane marks, path planning can be done, and control logic helps to check conditions. In this approach, feature extraction and interpretation of image data are done. On the other hand, the end‐to‐end learning approach to self‐driving tools is using convolutional neural networks (CNNs) [39]. End‐to‐end learning takes the raw image as the input and outputs the control signal automatically. The model optimizes itself based on training data, and there are no manually defined rules. These become two important advantages of end‐to‐end learning: better performance and less manual effort. Because the model is self‐optimizing based on data to give maximum overall performance, the intermediate parameters are automatically adjusted to be optimal. In addition, there is no need to design control logic based on identifying certain predefined objects, labeling these objects during training, or observing them. As a result, less manual effort is required.

17.8.5 Concept of Self‐driving Car

The concept of self‐driving car has a relationship between computer vision, sensor fusion, deep learning, path planning, actuator, and localization [40]. These are the main parts of the self‐driving car.

• Computer vision: Computer vision is an interdisciplinary field that can understand high‐level form of digital images or videos. Computer can see like from human's eye with the advanced computer vision algorithms [41,42].
• Sensor fusion: Sensor fusion is basically the combination of the data that is derived from the source [43,44].
• Deep learning: Deep learning is part of machine learning methods based on learning data representations. There are three types of methods, which are supervised, partially supervised, or unsupervised, for the learning [45,46].
• Path planning: Path planning is the most important field in the concept of robotics. In this field, shortest path can be calculated by the algorithm, and the calculated path helps autonomous mobile robots to move individually. Otherwise optimal paths could be paths that minimize the amount of turning, the amount of braking, or whatever a specific application requires [40].
• Actuator system: An actuator is the main component of the machine for controlling a machine or system. It is the heart of the system.
• Localization: Localization has a close relationship with navigation. For the navigation, robot or autonomous system needs a local or real‐time map. For generating a map of the robot or autonomous system, the system needs to know its position in the frame and plans a path toward the target. In order to navigate in its environment, the robot or any other mobility device requires representation, i.e. a map of the environment and the ability to interpret that representation [40].

An autonomous vehicle needs data that come from the different types of sensors such as camera, radar, laser, etc., to see the perspective of the car. This is used for creating a digital map that is very important for object detection [47].

17.8.5.1 Image Classification

Image classification is the main action of self‐driving car in the concept of object detection. The image is taken by the camera, and then computer vision algorithms are run on the image, and finally, items or objects are detected and labeled. There is an example of object detection in Figure 17.6.

Object detection consists of two processes, image classification and image localization. Image classification is used for determining the objects in the image, such as an animal or a plane, while image localization is used for determining the specific location of these objects. In the localization part, the objects are seen by the bounding boxes.

To perform image classification, a CNN is trained to recognize various objects, such as traffic lights and pedestrians. A CNN performs many operations on images in order to classify them.

17.8.6 Evaluation

Until now, all studies on autonomous vehicles have been tested mostly during daytime or in well‐lit environments. Uber accident that occurred in the night in recent years has frightened and upset everyone. After the accident, Uber interrupted the test drive on its driverless vehicles. For this reason, driverless vehicles continue to work in night/dark environments. Although there is a certain amount of illumination of the road and environment without a driver at night, the biggest problem of this issue is the lack of illumination. Apart from that, the current traffic is seen as a pattern of bright lights on a black background. To solve this problem, the fusion of the sensors is also required. Multiple sensors (LIDAR, GPS, advanced driver assistance systems – ADAS, etc.) and data from these sensors must be processed at the same time. At this point, testing this subject with deep learning will facilitate the procedures. When the previous studies are examined, the algorithm is generally followed as follows:

1. First, the data from the sensors used are collected and the collected data are simplified/extracted.
2. With a preferred algorithm, the object is detected and the objects around the vehicle are detected and labeled.
3. Lane detection is done by using deep learning architecture and image processing algorithms.
4. After all these determinations are made, decision‐making is carried out. (Gas/brake adjustment, updating steering angle, stop/start the car, etc.)

The above steps continue until the autonomous vehicle reaches the destination.

In another study, a leading vehicle was placed in front of the autonomous vehicle and the vehicle in front was followed on a dark and curvy road [60]. A real‐time autonomous car chaser operating optimally at night (RACCOON) solves these problems by creating an intermediate map structure that records the trajectory of the leading vehicle. The path is represented by points in a global reference frame and the computer‐controlled vehicle is routed from point to point. The autonomous vehicle follows this track at any speed and takes into account the taillights of the leading vehicle. Using this approach, the autonomous vehicle travels around corners and obstacles, not inside the vehicle. RACCOON was tested successfully applied on Carnegie Mellon Navlab I1. RACCOON consists of the following subsystems:

• Start
• Image acquisition
• Rear lamp (taillight) tracking
• Leader vehicle position calculation
• Consistency check and error recovery
• Output

In another study, object classification was successfully performed by using LIDAR, which is one of the most useful sensors for autonomous vehicles in night vision and image processing techniques. The collected samples were processed with CNN. The system works as follows:

1. Autonomous vehicle receives RGB image and LIDAR data from the camera.
2. These data are clipped by forming color and depth pairs.
3. The edited data are trained in the CNN architecture and the result is recorded on the driving cognitive map.