13.1 Introduction

Automatic system by capturing the data from the structure dynamically and the CAD system is the need of the day. Plenty of models based on image processing, statistical-based, and machine learning (ML) algorithms are available in literature. Each method is having its own limitations. This aforesaid case imbibes the real-time images from surfaces and train them using CNN and compare it with performance metrics of existing methods.

Structure surface are very prone to cyclic loading, stress in the concrete surface engineering which leads to hairline cracks at the initial stage. Microscopic level cracks are common in structures like concrete and beams but often results in cracks. This occurs because the structures are not resistant to serious stress, cyclic loading which happens due to long term exposure. This diminishes local strength and reduction in stiffness and material degradation [1, 2]. Early findings allow us to initiate preventive measures to reduce damage and can arrest the possible failure [3]. The crack detection can be made in two ways such as destructive testing (DT) and non-destructive testing (NDT). The dimensional analysis of the cracks on the structural surface shows the initial degradation level and holding strength of the concrete structures [4]. It also further helps the reader to focus on choosing the input, pre-processing and enhancing the model with much more sophistication incorporated with fine tuning parameter. Recently, deep convolutional neural networks (DCNN) have been conceived to bestow a remedy that involves intensity along with geometrical information. This procedure really yields effective results in conventional computer vision problems like semantic segmentation which occurs because of the various levels of abstraction in tracing out images. Such encouraging results envisages to boost the application of deep learning (DL) for vision-based surface auditing by availing advantage of the mathematical similarity between image segmentation and crack detection. This chapter discusses the automation of crack detection through the thermal and digital images using CNN. OpenCV is a python library which is designed to solve computer vision problems. It supports multiple platforms by which the vision based surface detection is achieved rapidly.

13.2 Related Work

DL is the latest technique which tends to replace all the previous techniques and it emphatically focuses toward the study of image data, DCNNs. DCNNs are neural network classifiers, having only the raw image as the input. Hence, image processing and cumbersome feature extraction will no longer be required. At the time of learning of a DCNN, the bit loads of different convolution network (neurons and boundaries) are iteratively evolved to naturally get familiar with the convolutional invariant highlights of the given picture. The major obstacle is that the training set does not restrain adequate examples. The loss function is generally learning to reduce the error in prediction. Its minimization performed with SGD (stochastic gradient descent) can be laborious even with GPUs. DCNNs are congenitally image or image patch segregators. That is the reason why, in the majority of the works, the information picture is examined with a fix, and for each fixes the DCNN figures if it contains breaks (fix grouping level) [5, 6]. In [7], fundamental calculations are utilized to recognize patches containing moderately clear breaks, yet a colossal number of incorrect positives are identified. Authors in [8] have additionally included the test of bogus positives at the CNN yield; however, they can be disposed of utilizing a few casings in post-preparing. Authors in [5, 9] examine imperfect learning techniques. The structure of DCNNs can be improved to guarantee expectation as a probabilistic break forecast map (pixel characterization level and all the pixels are obviously named in, based on the relationship between pixels), it is additionally brought in the writing “semantic division picture”. In [10], a completely convolutional network model (FCN) is encouraged to recognize breaks at the pixel level.

In initial DCNN application, the networks are a sequential model with fully connected layers at the end [11]. Such architecture needed ample of computational units, being most of the pixels in the image helps their weights on the speculation for every single pixel. The blurred output is due to loss function and the convolution layers are not related to the model parameters. This abstraction degrades the conservation of detailed patterns and, subsequently, may impact the accuracy of crack feature extraction. However, the upcoming hierarchical networks exhibited the improvement in arresting the degradation affect by the blurry effect. Thus, they have enormous scope of potentiality in all these tools for surface inspection and systematic health monitoring.

Researchers have developed an infrared (IR) thermography method on the basis of which IR image rectification happens [12, 13]. Crack description, size, and direction obtained from the extracted features. It further assists in evaluation of the structure of different cracks with a faster approach. Bahl et al. (1987) proposed the pre-processing technique and get geometric features of defect to enhance the classification accuracy [14]. The identification of cracks using the notches in the irregularities was propelled by [15]. It employs the IR thermography image rectification technique; cracks are being identified based on notches which differentiate as per the temperature.

Optically stimulated IR thermography is effectively used for the precise crack detection. PCA and pulse phase thermography are used for analysis [16]. Wavelet transform outperforms Fourier transform by removing more noise in the crack detection process.

13.3 Infrared Thermal Imaging Detection Method

The IR thermal imaging gentle testing technology is primarily handled by taking the peculiar conditions of the temperature variation at the crack level [17]. Cracks are depicted as hotspots or regions in the thermal camera, when the object temperature is higher than the surrounding temperature. When the area of interest has less temperature than the neighboring area, then cracks reveal themselves as cold spots. As per this approach, with the thermal images, the cracks will display vividly and can be able to magnify the variance high and low depth crack images [18, 19].

The heat is internally circulated when the temperature of the surface and the surrounding temperature are varied. In this chapter, focus will be mainly on cracks existing on the surface and the fabrication of heat buildup due to the pitfalls of crack defects. In local hot zone, the surface temperature is obviously high. The crack is found out by considering the thermal camera’s output and it is judged by the change in characteristics of the IR thermography [4, 7, 17]. If there are inconsistent emission levels on the surface, then analysis of real cracks will get seriously hampered due to unusual variations in temperature and alterations in the thermal image gathered by the IR camera [20]. Surface of on and inside cracks can be shown by IR imaging technology. Gradient difference is identified in the area with normal and abnormal region, when the area is cooled or heated. The variations are displayed by noticeable cracks in the thermal images. For identifying cracks, thermal image temperature gradient difference is analyzed. Variations are calculated in the cooling and heating stages, by considering the direction of heat.

13.4 Crack Detection Using CNN

The stages of crack detection are depicted in this section through the workflow as shown in Figure 13.1. Initially, retrieving the image of the structure pertinent to the crack detection process captured by thermal camera or any other allied sources. From thermal camera, both thermal image and digital image can be obtained. Training the classifier with the DL techniques uses the thermal images collecting from real-time. In addition, 300 numbers of crack surface thermal images and 600 numbers of non-crack surfaces are obtained for this work. The sample data from input data set used in CNN is showed in Figure 13.2. By argumentation techniques (rotation, horizontal flip, and vertical flip), imbalanced data set is converted to a balanced one with 3,000 images in each class. The dataset is segregated into train and test by the ratio of 80:20, by rescaling the images from [0, 255] range to [0, 1], training the classifier in Google Colab with open-source DL framework and Keras framework with TensorFlow backend.

13.4.1 Model Creation

With the advent of DL, separate feature extraction methods need not be applied, since the algorithm itself has the capacity to understand the data. Images obtained by the cameras are processed for getting useful insights. Convolutional neural network (CNN) extracts only those essential features, from the raw image data and learns to classify the status as shown in Figure 13.3. Using the trained model, CNN predicts the class that the images belong to, and if they are crack or non-crack. The images are loaded and inverted before they are fed into the model. In the model, the convolution layer is one of the most important primary layers that help in the feature extraction process. The convolution layer takes the image array and filter as input. The accuracy of the model is then improved by tuning the hyper parameters.

This combination of convolution and activation function such as ReLu, sigmoid, and SeLu is then passed on to the pooling layer where the dimensionality is reduced. The outputs are combined into a single neuron for the next layer. In the pooling layer, Max pooling is used to extract the largest element from the feature map. The convolution and pooling layer together form the hidden layers where the extraction of features from the image takes place. There are totally five hidden layers used in the model that acts as an important parameter for the accuracy obtained. Sometimes, a number of layers improve the feature extraction that helps obtain better performance of the model, while at times increasing the layers would lead to deterioration in the model performance. Using these features extracted from the hidden layers, the task of image classification is performed.

13.4.2 Activation Functions (AF)

An AF shown in Table 13.1 is an indispensable component in the context of DL, as it is accountable for a neuron to be activated. AF controls the amount of information to be passed through the networks using nonlinear transformations of an input signal that helps to learn the complex mapping between the input and the output.

S. no.	Activation function	Description	Formula
1.	Rectified Linear Unit (ReLU)	• A non-linear transformation function with an output range of [0, ∞). • It only activates a few neurons and computation is quicker if used.
2.	Tanh	• A non-linear function which is symmetric over the origin with an output range of (–1, 1). • Continuous and differentiable at all points and easy while performing backpropagation. • Approximately linear when x is very nearer to the origin.
3.	Linear (Identity activation function)	• A linear transformation function having the output directly proportional to the input. • Activates multiple neurons at once and can be used to predict continuous values in the output range of (-∞, ∞).
4.	Sigmoid	• A widely used nonlinear function, symmetric about the line y = 0.5 with an output range of (0, 1). • Continuous and differentiable at all points and easy while performing backpropagation. • When x is closer to 0, small changes in x translates to considerable changes in y, resulting in pushing values to the extreme ends.
5.	Softsign	• A less popular nonlinear function, with an output range of (–1, 1). • Similar to tanh except it converges in a polynomial style and the gradient descent will not suffer vanishing or exploding gradient problem.

S. no.	Optimizer	Description	Formula
1.	Gradient descent (GD)	• A parameter that was randomly initialized will be updated after every iteration until the minimum of the cost function is obtained. • Let a cost function be J(0), where 0 is the parameter, 0-learning rate
2.	Stochastic GD (SGD)	• In stochastic gradient descent, the parameter gets updated for every training sample in the dataset instead of every iteration.
3.	Momentum	• It is an augmented version of SGD, by accelerating the gradient descent in the relevant direction and diminishing the oscillations of parameters. • The new gradient is the weighted average of past gradients, resulting in better control of the optimization.
4.	Adagrad	• The adaptive gradient algorithm modifies each of the parameters with different learning rates. This varying learning speed will enable faster convergence.
5.	Adam	• The adaptive moment estimation algorithm, like Adagrad, adjusts different learning rates for every parameter. It has an additional term for moving average of second moments along with the average of the gradient, giving it a robust learning experience and rapid convergence.

13.5 Results and Discussion

Using the given data set, CNN is applied and the model is built to predict the status of given civil structure image. To figure out the performance of the model, it is tested with a combination of different optimizers and activation functions. To enhance the performance further, transfer learning is applied. The trained model uses 50 epochs with batch size of 16 and the same is maintained in testing time also. The transfer learning with VGG16 yields good accuracy. In this work, the fully connected network encoder uses the pre-trained weights from standard VGG16. The decoder is replacing the final fully connected layers. Total images are split into training, testing, and validation test by the function. Learning rate of 10³ and weight decay of 5 * 10^–4. The model was trained with an Adam with a learning rate of 1e–3 for 50 epochs. The weights loaded at the end of the block result from a good run (based on validation score). The results tabulated with different combinations in the Table 13.3. Of all the activation functions tested, ReLu was observed to have the highest accuracy. ReLu when tested with Adam as optimizer was observed to have an accuracy of 98.46%.

13.6 Conclusion

To ensure the safety of civil construction by detecting the cracks is a noteworthy challenge. From the state of the art, we inferred CNN achieves best performance in several applications like object recognition/detection and segmentation. The CNN has the special feature to aggregate visual levels in hierarchy. This motivates to apply CNN for crack detection. Hence, the challenges in civil structure are mitigated by designing a CNN framework to detect the cracks automatically. The conventional crack detection by experts with related instruments is time consuming procedure. Scarcity of experts and scanning huge and tall civil structures induced researchers to design automated crack detection system. It will assist users in early diagnosis and suggestion of precautionary maintenance for the civil structure in effective and efficient way. After detecting the crack, characteristics of cracks such as length, breadth, and depth can be measured.