Real‐time crack assessment using deep neural networks with wall‐climbing unmanned aerial system

Crack information provides important evidence of structural degradation and safety in civil structures. Existing inspection methods are inefficient and difficult to rapidly deploy. A real‐time crack inspection method is proposed in this study to address this difficulty. Within this method, a wall‐climbing unmanned aerial system (UAS) is developed to acquire detailed crack images without distortion, then a wireless data transmission method is applied to fulfill real‐time detection requirements, allowing smartphones to receive real‐time video taken from the UAS. Next, an image data set including 1,330 crack images taken by the wall‐climbing UAS is established and used for training a deep‐learning model. For increasing detection speed, state‐of‐the‐art convolutional neural networks (CNNs) are compared and employed to train the crack detector; the selected model is transplanted into an android application so that the detection of cracks can be undertaken on a smartphone in real time. Following this, images with cracks are separated and crack width is calculated using an image processing method. The proposed method is then applied to a building where crack information is acquired and calculated accurately with high efficiency, thus verifying the practicability of the proposed method and system.     

A Bayesian Probabilistic Approach for Crack Characterization in Plate Structures

Abstract: This article explores the possibility of using a Bayesian probabilistic approach for the detection of cracks in thin plate structures, utilizing measured dynamic responses at only a few points on the plate. Existing laser scanning or shearography based crack detection methods are applicable only when measurement at the region near the defect is possible. These types of techniques are important in providing information, in addition to that obtained through visual inspection, for the purpose of structural health monitoring. Because of the global nature of the vibration characteristics of structural systems, this article puts forward a crack detection approach that can be applicable with only a few sensors and when the sensor locations are not close to the crack. This kind of method is particularly valuable as it can be applied when visual inspection is not possible (e.g., part of the plate is obstructed and is not assessable by inspectors). Owing to the problems of measurement noise and incomplete measurement (i.e., only a limited number of measurement points are employed and high‐mode information is lost because of the digitization of the signal and measurement noise), the results of crack detection as an inverse problem contain uncertainties. To explicitly handle such uncertainty, the proposed crack detection method follows the Bayesian statistical system identification framework. Rather than pinpointing the crack parameters (i.e., the crack location, length, and depth), the posterior probability density function (PDF) of the crack parameters is calculated to quantify the confidence level of the identified results, which is extremely important for engineers when they make judgments about remedial works. This article reports the theoretical development of the modeling of a cracked plate and a crack detection method together with numerical verification of the proposed method. The case study results are very encouraging, and indicate that the proposed method is feasible.     

Image analysis method for crack distribution and width estimation for reinforced concrete structures

Crack observation is important for evaluating the structural performance and safety of reinforced concrete (RC) structures. Most of the existing image-based crack detection methods are based on edge detection algorithms, which detect cracks that are wide enough to present dark areas in the obtained images. Cracks initiate as thin cracks, generally having width less than the width of a pixel in images; such cracks are generally undetectable by edge detection-based methods.An image analysis method is proposed to observe the development and distribution of thin cracks on RC surfaces; it also allows estimation of crack widths. Image matching based on optical flow and subpixel is employed to analyze slight concrete surface displacements. Camera calibration is included to eliminate perspective effects and lens distortion. Geometric transformation is adopted so that cameras do not need to be perpendicular to the observed surface or specified positions. Formulas are proposed to estimate the width of shear crack opening. The proposed method was then applied to a cyclic test of an RC structure. The crack widths and their development analyzed by the image analysis were verified with human inspection in the test. In addition, concrete surface cracks that appeared at a very early stage of the test could be observed by the proposed method before they could be detected by the naked eye. The results thus demonstrate that the proposed image analysis method offers an efficient way applicable not only for structural tests but also for crack-based structural-health-monitoring applications.     

Image‐based crack assessment of bridge piers using unmanned aerial vehicles and three‐dimensional scene reconstruction

Crack assessment of bridge piers using unmanned aerial vehicles (UAVs) eliminates unsafe factors of manual inspection and provides a potential way for the maintenance of transportation infrastructures. However, the implementation of UAV‐based crack assessment for real bridge piers is hindered by several key issues, including the following: (a) both perspective distortion and the geometry distortion by nonflat structural surfaces usually appear on crack images taken by the UAV system from the pier surface; however, these two kinds of distortions are difficult to correct at the same time; and (b) the crack image taken by a close‐range inspection flight UAV system is partially imaged, containing only a small part of the entire surface of the pier, and thereby hinders crack localization. In this paper, a new image‐based crack assessment methodology for bridge piers using UAV and three‐dimensional (3D) scene reconstruction is proposed. First, the data acquisition of UAV‐based crack assessment is discussed, and the UAV flight path and photography strategy for bridge pier assessment are proposed. Second, image‐based crack detection and 3D reconstruction are conducted to obtain crack width feature pair sequences and 3D surface models, respectively. Third, a new method of projecting cracks onto a meshed 3D surface triangular model is proposed, which can correct both the perspective distortion and geometry distortion by nonflat structural surfaces, and realize the crack localization. Field test investigations of crack assessment of a real bridge pier using a UAV are carried out for illustration, validation, and error analysis of the proposed methodology.     

Zernike‐moment measurement of thin‐crack width in images enabled by dual‐scale deep learning

Although crack inspection is a routine practice in civil infrastructure management (especially for highway bridge structures), it is time‐consuming and safety‐concerning to trained engineers and costly to the stakeholders. To automate this in the near future, the algorithmic challenge at the onset is to detect and localize cracks in imagery data with complex scenes. The rise of deep learning (DL) sheds light on overcoming this challenge through learning from imagery big data. However, how to exploit DL techniques is yet to be fully explored. One primary component of practical crack inspection is that it is not merely detection via visual recognition. To evaluate the potential risk of structural failure, it entails quantitative characterization, which usually includes crack width measurement. To further facilitate the automation of machine‐vision‐based concrete crack inspection, this article proposes a DL‐enabled quantitative crack width measurement method. In the detection and mapping phase, dual‐scale convolutional neural networks are designed to detect cracks in complex scene images with validated high accuracy. Subsequently, a novel crack width estimation method based on the use of Zernike moment operator is further developed for thin cracks. The experimental results based on a laboratory loading test agree well with the direct measurements, which substantiates the effectiveness of the proposed method for quantitative crack detection.     

Quantitative road crack evaluation by a U-Net architecture using smartphone images and Lidar data

Road cracks are a major concern for administrators. Visual inspection is labor-intensive. The accuracy of previous algorithms for detecting cracks in images requires improvement. Further, the length and thickness of cracks must be estimated. Light detection and ranging (Lidar), a standard smartphone feature is used to develop a method for the completely automatic, accurate, and quantitative evaluation of road cracks. The two contributions of this study are as follows. To achieve the highest segmentation accuracy, U-Net is combined with data augmentation and morphology transform. To calculate the crack length and thickness, crack images are registered into Lidar color data. The proposed algorithm was validated using a public database of road cracks and those measured by the authors. The algorithm was 95% accurate in determining crack length. The coefficient of determination for thickness estimation accuracy was 0.98 addressing various crack shapes and asphalt pavement patterns.     

A Nonballasted Rail Track Slab Crack Identification Method Using a Level‐Set‐Based Active Contour Model

The nonballasted rail tracks have been extensively applied in the new high‐speed railway system development in China, that is, the China Railway Track System (CRTS). However, many defects have been identified during the operation of the CRTS, among which concrete slab crack is recognized as one of the most common, yet critical defects that require accurate identification and timely maintenance attention. Because of the unique outlook of the cracks in nonballasted rail track slabs captured in the survey imagery, direct adaptations of the existing crack extraction methods show dramatically degraded performance. A new automatic crack identification method is developed in this study by employing a region‐based active contour framework with the intensity cluster energy. The proposed method embodies three major contributions, including (1) a heavy penalization energy component that could effectively avoid both under‐ and overevolutions; (2) a multiphase level‐set function that effectively evolves the contours generated with different intensity clusters; and (3) a two‐step implementation of the framework that significantly improves the efficiency. The experimental test evaluates the performance of the proposed method using the data collected on high‐speed rail tracks in Hebei Province, China. The proposed method accurately identifies more than 93.0% of digitized cracks in different crack patterns and challenging backgrounds using a data set consisting of 1,500 synthetic images and 150 actual images. In addition, it shows promising performance in comparison with other popular state‐of‐the‐art crack detection algorithms in terms of accuracy and computational efficiency. The proposed method has demonstrated the promising capacity to support a reliable and efficient nonballasted rail track crack identification and to facilitate the subsequent maintenance cost‐effectively.     

Bridge inspection robot system with machine vision

A robotic system for inspecting the safety status of bridges is proposed in this paper. Currently, most bridge inspections have been done manually by counting the number of cracks, measuring their lengths and widths and taking pictures of them. Thus the quality and reliability of diagnosis reports on bridges are greatly dependent upon the diligence and education of inspection workers. The robotic inspection system to be proposed consists of three parts: a specially designed car, a robot mechanism and control system for mobility and a machine vision system for automatic detection of cracks. Ultimately, this robot system has been developed for gathering accurate data in order to record the biennial changes of the bridge's safety circumstances as well as check the safety status of bridges. We also demonstrate the effectiveness of the suggested crack detecting and tracing algorithms through experiments on a real bridge crack inspection.     

结构表面裂缝数字图像法识别研究综述与前景展望

裂缝反映结构受力状态与安全性、耐久性,是结构现场安全性检测监测以及结构模型试验研究的重要指标之一。现有的人工裂缝识别技术难以满足工程现场与实验室需求,操作中存在测不准、高空多、效率低、记不全等缺点。相比之下,采用数字图像法进行结构表面裂缝识别,具有便捷、自动、定量、准确等优势。文章对结构表面裂缝数字图像法识别研究进行系统综述,对裂缝识别中图像预处理、裂缝识别与提取、裂缝参数计算等重要环节的常见算法进行讨论,阐述采用多视角几何三维重建方法实现裂缝成像修正与拼接、裂缝表达输出的原理与流程,结合实桥案例报道了基于无人机平台的裂缝识别研究与应用,讨论国内首部数字图像法检测规程《工程结构数字图像法检测技术规程》征求意见稿中裂缝检测的相关规定。最后,对结构表面裂缝数字图像法识别研究进行前景展望。     

Encoder–decoder network for pixel‐level road crack detection in black‐box images

Timely monitoring of pavement cracks is essential for successful maintenance of road infrastructure. Accurate information concerning crack location and severity enables proactive management of the infrastructure. Black‐box cameras, which are becoming increasingly widespread at an affordable price, can be used as efficient road‐image collectors over a wide area. However, the cracks in these images are difficult to detect, because the images containing them often include objects other than roads. Thus, we propose a pixel‐level detection method for identifying road cracks in black‐box images using a deep convolutional encoder–decoder network. The encoder consists of convolutional layers of the residual network for extracting crack features, and the decoder consists of deconvolutional layers for localizing the cracks in an input image. The proposed network was trained on 427 out of 527 images extracted from black‐box videos and tested on the remaining 100 images. Compared with VGG‐16, ResNet‐50, ResNet‐101, ResNet‐200 with transfer learning, and ResNet‐152 without transfer learning, ResNet‐152 with transfer learning exhibited the best performance, achieving recall, precision, and intersection of union of 71.98%, 77.68%, and 59.65%, respectively. The experimental results prove that the proposed method is optimal for detecting cracks in black‐box images at the pixel level.     

Automated image processing technique for detecting and analysing concrete surface cracks

In the present work, an image processing technique that automatically detects and analyses cracks in the digital image of concrete surfaces is proposed. The image processing technique automates the measurement of crack characteristics including the width, length, orientation and crack pattern. In the proposed technique, a morphological technique was applied to correct the non-uniform brightness of the background, and enhanced binarisation and shape analysis were used to improve the detection performance; furthermore, detailed algorithms to calculate the crack width, length, orientation and an artificial neural network to recognise crack patterns including horizontal, vertical, diagonal (?45°), diagonal (+45°), and random cracks are proposed. An image processing program was developed for the proposed algorithm and a series of experimental and analytical investigations were performed to assess the validity of the algorithm. Then, the crack characteristics measured using the proposed technique were compared with those obtained using a conventional technique. The test results showed that the crack characteristics can be accurately measured and analysed using the proposed technique.     

A Pavement Crack Detection Method Combining 2D with 3D Information Based on Dempster‐Shafer Theory

Pavement cracking is one of the main distresses presented in the road surface. Objective and accurate detection or evaluation for these cracks is an important task in the pavement maintenance and management. In this work, a new pavement crack detection method is proposed by combining two‐dimensional (2D) gray‐scale images and three‐dimensional (3D) laser scanning data based on Dempster‐Shafer (D‐S) theory. In this proposed method, 2D gray‐scale image and 3D laser scanning data are modeled as a mass function in evidence theory, and 2D and 3D detection results for pavement cracks are fused at decision‐making level. The experimental results show that the proposed method takes advantage of the respective merits of 2D images and 3D laser scanning data and therefore improves the pavement crack detection accuracy and reduces recognition error rate compared to 2D image intensity‐based methods.     

一种改进的用于裂缝图像分割的Otsu方法

裂缝是隧道衬砌最常见的病害之一,基于近几年快速发展的工程检测系统与图像处理算法的研究,提出了一种CCD相机的衬砌裂缝快速检测系统采集裂缝图像。在提取裂缝特征之前,需要将裂缝区域与图像背景分离。采用Otsu法进行分割处理,然而传统的Otsu方法对裂缝区域过小的图像易产生欠分割;对背景不单一或光照度不均匀的裂缝图像易造成过分割的情况。根据Ostu方法分割特点对该方法进行改进,以达到更好的裂缝图像分割效果,从而为实现隧道裂缝的快速检测埋下基础。     

Automatic detection method of cracks from concrete surface imagery using two‐step light gradient boosting machine

Automated crack detection based on image processing is widely used when inspecting concrete structures. The existing methods for crack detection are not yet accurate enough due to the difficulty and complexity of the problem; thus, more accurate and practical methods should be developed. This paper proposes an automated crack detection method based on image processing using the light gradient boosting machine (LightGBM), one of the supervised machine learning methods. In supervised machine learning, appropriate features should be identified to obtain accurate results. In crack detection, the pixel values of the target pixels and geometric features of the cracks that occur when they are connected linearly should be considered. This paper proposes a methodology for generating features based on pixel values and geometric shapes in two stages. The accuracy of the proposed methodology is investigated using photos of concrete structures with adverse conditions, such as shadows and dirt. The proposed methodology achieves an accuracy of 99.7%, sensitivity of 75.71%, specificity of 99.9%, precision of 68.2%, and an F‐measure of 0.6952. The experimental results demonstrate that the proposed method can detect cracks with higher performance than the pix2pix‐based approach. Furthermore, the training time is 7.7 times shorter than that of the XGBoost and 2.3 times shorter than that of the pix2pix. The experimental results demonstrate that the proposed method can detect cracks with high accuracy.     

Fatigue Crack Initiation and Propagation in Piles of Integral Abutment Bridges

Abstract: A new continuum damage modeling approach of “successive initiation” is used to determine the location of a thermomechanical fatigue crack initiation and the propagation path and rate in piles of integral abutment bridges. A global‐local modeling approach is introduced to determine the critical location in the pile where a crack is initiated using a 3‐dimensional nonlinear finite element model and to implement “successive initiation.” A simulated case study is used to showcase the multistep procedure. The results indicate that for a pile subjected to the maximum stress, the first fatigue‐induced crack initiates in the tip of the flange at the element immediately below the abutment. Several other cracks at different locations form in the flange of the pile while the initial crack continues to propagate in the flange to the web. The crack propagation rate increases as more cracks initiate in the flange. The propagation rate decreases when the crack reaches the web. Based on the case study presented, a crack could initiate in the pile in as little as 6 years, but it may take about 20 years for it to reach the web; however, final failure of the pile may not take place for several decades. The method can also be used as a guide in bridge foundation inspection and in the determination of the remaining life of an existing bridge.     

Automated Pixel‐Level Pavement Crack Detection on 3D Asphalt Surfaces Using a Deep‐Learning Network

The CrackNet, an efficient architecture based on the Convolutional Neural Network (CNN), is proposed in this article for automated pavement crack detection on 3D asphalt surfaces with explicit objective of pixel‐perfect accuracy. Unlike the commonly used CNN, CrackNet does not have any pooling layers which downsize the outputs of previous layers. CrackNet fundamentally ensures pixel‐perfect accuracy using the newly developed technique of invariant image width and height through all layers. CrackNet consists of five layers and includes more than one million parameters that are trained in the learning process. The input data of the CrackNet are feature maps generated by the feature extractor using the proposed line filters with various orientations, widths, and lengths. The output of CrackNet is the set of predicted class scores for all pixels. The hidden layers of CrackNet are convolutional layers and fully connected layers. CrackNet is trained with 1,800 3D pavement images and is then demonstrated to be successful in detecting cracks under various conditions using another set of 200 3D pavement images. The experiment using the 200 testing 3D images showed that CrackNet can achieve high Precision (90.13%), Recall (87.63%) and F‐measure (88.86%) simultaneously. Compared with recently developed crack detection methods based on traditional machine learning and imaging algorithms, the CrackNet significantly outperforms the traditional approaches in terms of F‐measure. Using parallel computing techniques, CrackNet is programmed to be efficiently used in conjunction with the data collection software.     

Automatic recognition of asphalt pavement cracks using metaheuristic optimized edge detection algorithms and convolution neural network

Crack detection is a crucial task in periodic pavement survey. This study establishes and compares the performance of two intelligent approaches for automatic recognition of pavement cracks. The first model relies on edge detection approaches of the Sobel and Canny algorithms. Since the implementation of the two edge detectors require the setting of threshold values, Differential Flower Pollination, as a metaheuristic, is employed to fine-tune the model parameters. The second model is constructed by the implementation of the Convolution Neural Network (CNN) – a deep learning algorithm. CNN has the advantage of performing the feature extraction and the prediction of crack/non-crack condition in an integrated and fully automated manner. Experimental results show that the model based on CNN achieves a good prediction performance of Classification Accuracy Rate (CAR) = 92.08%. This performance is significantly better than the method based on the edge detection algorithms (CAR = 79.99%). Accordingly, the proposed CNN based crack detection model is a promising alternative to support transportation agencies in the task of periodic pavement inspection.     

Bayesian Modeling of External Corrosion in Underground Pipelines Based on the Integration of Markov Chain Monte Carlo Techniques and Clustered Inspection Data

In this study, a model is developed to assess external corrosion in buried pipelines based on the unification of Bayesian inferential structure derived from Markov chain Monte Carlo techniques using clustered inspection data. This proposed stochastic model combines clustering algorithms that can ascertain the similarity of corrosion defects and Monte Carlo simulation that can give an accurate probability density function estimation of the corrosion rate. The metal loss rate is chosen as the indicator of corrosion damage propagation, obeying a generalized extreme value (GEV) distribution. Bayesian theory was employed to update the probability distribution of metal loss rate as well as the GEV parameters in order to account for the model uncertainty. The proposed model was validated with direct and indirect inspection data extracted from a 110‐km buried pipeline system.     

Crack detection of the cantilever beam using new triple hybrid algorithms based on Particle Swarm Optimization

The presence of cracks in a concrete structure reduces its performance and increases in the size of cracks result in the failure of the structure. Therefore, the accurate determination of crack characteristics, such as location and depth, is one of the key engineering issues for assessment of the reliability of structures. This paper deals with the inverse analysis of the crack detection problems using triple hybrid algorithms based on Particle Swarm Optimization (PSO); these hybrids are Particle Swarm Optimization-Genetic Algorithm-Firefly Algorithm (PSO-GA-FA), Particle Swarm Optimization-Grey Wolf Optimization-Firefly Algorithm (PSO-GWO-FA), and Particle Swarm Optimization-Genetic Algorithm-Grey Wolf Optimization (PSO-GA-GWO). A strong correlation exists between the changes in the natural frequency of a concrete beam and the crack parameters. Thus, the location and depth of a crack in a beam can be predicted by measuring its natural frequency. Hence, the measured natural frequency can be used as the input parameter of the algorithm. In this paper, this is applied to identify crack location and depth in a cantilever beam using the new hybrid algorithms. The results show that among the proposed triple hybrid algorithms, the PSO-GA-FA and PSO-GWO-FA algorithms are much more effective than PSO-GA-GWO algorithm for the crack detection.     

Optical flow techniques for estimation of camera motion parameters in sewer closed circuit television inspection videos

This paper discusses a novel approach for automated analysis and tracking of camera motion in sewer inspection closed circuit television (CCTV) videos. This approach represents an important building block for any system that supports automated analysis and defect detection of CCTV videos. The proposed approach employs optical flow techniques to automatically identify, locate, and extract a limited set of video segments, called regions of interest (ROI), which likely include defects, thus reducing the time and computational requirements needed for video processing. Tracking the camera motion parameters is used to recover the operator actions during the inspection session, which would provide important clues about the location and severity of the ROI. Techniques for estimating the camera travelling distance, position inside the sewer, and direction of motion from optical flow vectors are discussed. The proposed techniques were validated using a representative set of sewer CCTV videos obtained from the cities of Regina and Calgary, Canada.