Real‐time crack assessment using deep neural networks with wall‐climbing unmanned aerial system

Crack information provides important evidence of structural degradation and safety in civil structures. Existing inspection methods are inefficient and difficult to rapidly deploy. A real‐time crack inspection method is proposed in this study to address this difficulty. Within this method, a wall‐climbing unmanned aerial system (UAS) is developed to acquire detailed crack images without distortion, then a wireless data transmission method is applied to fulfill real‐time detection requirements, allowing smartphones to receive real‐time video taken from the UAS. Next, an image data set including 1,330 crack images taken by the wall‐climbing UAS is established and used for training a deep‐learning model. For increasing detection speed, state‐of‐the‐art convolutional neural networks (CNNs) are compared and employed to train the crack detector; the selected model is transplanted into an android application so that the detection of cracks can be undertaken on a smartphone in real time. Following this, images with cracks are separated and crack width is calculated using an image processing method. The proposed method is then applied to a building where crack information is acquired and calculated accurately with high efficiency, thus verifying the practicability of the proposed method and system.     

基于计算机视觉技术的结构表面裂缝检测方法研究

计算机视觉技术用于混凝土结构表面裂缝检测,具有现场检测方便、效率高、客观性强的特点,但图像数据分析是该技术的核心,其中裂缝提取与定量测量较为复杂。为提高裂缝图像处理效率和准确率,将深度学习和数字图像处理技术相结合,提出一种裂缝检测方法。建立基于深度卷积神经网络的裂缝识别模型,在图像上自动定位裂缝并结合图像局域阈值分割方法提取裂缝。在裂缝宽度定量测量方面,采用双边滤波算法和三段线性变换对裂缝图像进行预处理,提高了裂缝边缘识别的精确度。通过改进边缘梯度法,实现裂缝最大宽度的定位和裂缝最大宽度的自动获取。该研究为全自动识别裂缝图像及高精度测量裂缝宽度提供了一种解决方法。     

A texture‐Based Video Processing Methodology Using Bayesian Data Fusion for Autonomous Crack Detection on Metallic Surfaces

Regular inspection of the components of nuclear power plants is important to improve their resilience. However, current inspection practices are time consuming, tedious, and subjective: they involve an operator manually locating cracks in metallic surfaces in the plant by watching videos. At the same time, prevalent automatic crack detection algorithms may not detect cracks in metallic surfaces because these are typically very small and have low contrast. Moreover, the existences of scratches, welds, and grind marks lead to a large number of false positives when state‐of‐the‐art vision‐based crack detection algorithms are used. In this study, a novel crack detection approach is proposed based on local binary patterns (LBP), support vector machine (SVM), and Bayesian decision theory. The proposed method aggregates the information obtained from different video frames to enhance the robustness and reliability of detection. The performance of the proposed approach is assessed by using several inspection videos. The results indicate that it is accurate and robust in cases where state‐of‐the‐art crack detection approaches fail. The experiments show that Bayesian data fusion improves the hit rate by 20% and the hit rate achieves 85% with only one false positive per frame.     

Automatic detection method of cracks from concrete surface imagery using two‐step light gradient boosting machine

Automated crack detection based on image processing is widely used when inspecting concrete structures. The existing methods for crack detection are not yet accurate enough due to the difficulty and complexity of the problem; thus, more accurate and practical methods should be developed. This paper proposes an automated crack detection method based on image processing using the light gradient boosting machine (LightGBM), one of the supervised machine learning methods. In supervised machine learning, appropriate features should be identified to obtain accurate results. In crack detection, the pixel values of the target pixels and geometric features of the cracks that occur when they are connected linearly should be considered. This paper proposes a methodology for generating features based on pixel values and geometric shapes in two stages. The accuracy of the proposed methodology is investigated using photos of concrete structures with adverse conditions, such as shadows and dirt. The proposed methodology achieves an accuracy of 99.7%, sensitivity of 75.71%, specificity of 99.9%, precision of 68.2%, and an F‐measure of 0.6952. The experimental results demonstrate that the proposed method can detect cracks with higher performance than the pix2pix‐based approach. Furthermore, the training time is 7.7 times shorter than that of the XGBoost and 2.3 times shorter than that of the pix2pix. The experimental results demonstrate that the proposed method can detect cracks with high accuracy.     

Concrete bridge surface damage detection using a single‐stage detector

Early and timely detection of surface damages is important for maintaining the functionality, reliability, and safety of concrete bridges. Recent advancement in convolution neural network has enabled the development of deep learning‐based visual inspection techniques for detecting multiple structural damages. However, most deep learning‐based techniques are built on two‐stage, proposal‐driven detectors using less complex image data, which could be restricted for practical applications and possible integration within intelligent autonomous inspection systems. In this study, a faster, simpler single‐stage detector is proposed based on a real‐time object detection technique, You Only Look Once (YOLOv3), for detecting multiple concrete bridge damages. A field inspection images dataset labeled with four types of concrete damages (crack, pop‐out, spalling, and exposed rebar) is used for training and testing of YOLOv3. To enhance the detection accuracy, the original YOLOv3 is further improved by introducing a novel transfer learning method with fully pretrained weights from a geometrically similar dataset. Batch renormalization and focal loss are also incorporated to increase the accuracy. Testing results show that the improved YOLOv3 has a detection accuracy of up to 80% and 47% at the Intersection‐over‐Union (IoU) metrics of 0.5 and 0.75, respectively. It outperforms the original YOLOv3 and the two‐stage detector Faster Region‐based Convolutional Neural Network (Faster R‐CNN) with ResNet‐101, especially for the IoU metric of 0.75.     

Image‐based crack assessment of bridge piers using unmanned aerial vehicles and three‐dimensional scene reconstruction

Crack assessment of bridge piers using unmanned aerial vehicles (UAVs) eliminates unsafe factors of manual inspection and provides a potential way for the maintenance of transportation infrastructures. However, the implementation of UAV‐based crack assessment for real bridge piers is hindered by several key issues, including the following: (a) both perspective distortion and the geometry distortion by nonflat structural surfaces usually appear on crack images taken by the UAV system from the pier surface; however, these two kinds of distortions are difficult to correct at the same time; and (b) the crack image taken by a close‐range inspection flight UAV system is partially imaged, containing only a small part of the entire surface of the pier, and thereby hinders crack localization. In this paper, a new image‐based crack assessment methodology for bridge piers using UAV and three‐dimensional (3D) scene reconstruction is proposed. First, the data acquisition of UAV‐based crack assessment is discussed, and the UAV flight path and photography strategy for bridge pier assessment are proposed. Second, image‐based crack detection and 3D reconstruction are conducted to obtain crack width feature pair sequences and 3D surface models, respectively. Third, a new method of projecting cracks onto a meshed 3D surface triangular model is proposed, which can correct both the perspective distortion and geometry distortion by nonflat structural surfaces, and realize the crack localization. Field test investigations of crack assessment of a real bridge pier using a UAV are carried out for illustration, validation, and error analysis of the proposed methodology.     

Encoder–decoder network for pixel‐level road crack detection in black‐box images

Timely monitoring of pavement cracks is essential for successful maintenance of road infrastructure. Accurate information concerning crack location and severity enables proactive management of the infrastructure. Black‐box cameras, which are becoming increasingly widespread at an affordable price, can be used as efficient road‐image collectors over a wide area. However, the cracks in these images are difficult to detect, because the images containing them often include objects other than roads. Thus, we propose a pixel‐level detection method for identifying road cracks in black‐box images using a deep convolutional encoder–decoder network. The encoder consists of convolutional layers of the residual network for extracting crack features, and the decoder consists of deconvolutional layers for localizing the cracks in an input image. The proposed network was trained on 427 out of 527 images extracted from black‐box videos and tested on the remaining 100 images. Compared with VGG‐16, ResNet‐50, ResNet‐101, ResNet‐200 with transfer learning, and ResNet‐152 without transfer learning, ResNet‐152 with transfer learning exhibited the best performance, achieving recall, precision, and intersection of union of 71.98%, 77.68%, and 59.65%, respectively. The experimental results prove that the proposed method is optimal for detecting cracks in black‐box images at the pixel level.     

Real-time automatic crack detection method based on drone

Real-time automated drone-based crack detection can be used for efficient building damage assessment. This paper proposes an automated real-time crack detection method based on a drone. Using a lightweight classification algorithm, a lightweight segmentation algorithm, a high-precision segmentation algorithm, and a crack width measurement algorithm, the cracks are classified, roughly segmented, finely segmented, and the maximum width is extracted. A crack information-assisted drone flight automatic control algorithm for automatic crack detection guides the drone toward the crack position. The effectiveness of the crack detection algorithm and the crack information-assisted drone flight automatic control algorithm was tested on two different datasets, a two-story building, and a 16-m-high shaking table test building. The results show that crack detection can be finished in real-time during the flight. Using the proposed method can significantly improve the MIoU of crack edge detection and the accuracy of maximum crack width measurement under the non-ideal shooting conditions of the actual inspection situation by automatically approaching the vicinity of the crack.     

Image analysis method for crack distribution and width estimation for reinforced concrete structures

Crack observation is important for evaluating the structural performance and safety of reinforced concrete (RC) structures. Most of the existing image-based crack detection methods are based on edge detection algorithms, which detect cracks that are wide enough to present dark areas in the obtained images. Cracks initiate as thin cracks, generally having width less than the width of a pixel in images; such cracks are generally undetectable by edge detection-based methods.An image analysis method is proposed to observe the development and distribution of thin cracks on RC surfaces; it also allows estimation of crack widths. Image matching based on optical flow and subpixel is employed to analyze slight concrete surface displacements. Camera calibration is included to eliminate perspective effects and lens distortion. Geometric transformation is adopted so that cameras do not need to be perpendicular to the observed surface or specified positions. Formulas are proposed to estimate the width of shear crack opening. The proposed method was then applied to a cyclic test of an RC structure. The crack widths and their development analyzed by the image analysis were verified with human inspection in the test. In addition, concrete surface cracks that appeared at a very early stage of the test could be observed by the proposed method before they could be detected by the naked eye. The results thus demonstrate that the proposed image analysis method offers an efficient way applicable not only for structural tests but also for crack-based structural-health-monitoring applications.     

A Bayesian Probabilistic Approach for Crack Characterization in Plate Structures

Abstract: This article explores the possibility of using a Bayesian probabilistic approach for the detection of cracks in thin plate structures, utilizing measured dynamic responses at only a few points on the plate. Existing laser scanning or shearography based crack detection methods are applicable only when measurement at the region near the defect is possible. These types of techniques are important in providing information, in addition to that obtained through visual inspection, for the purpose of structural health monitoring. Because of the global nature of the vibration characteristics of structural systems, this article puts forward a crack detection approach that can be applicable with only a few sensors and when the sensor locations are not close to the crack. This kind of method is particularly valuable as it can be applied when visual inspection is not possible (e.g., part of the plate is obstructed and is not assessable by inspectors). Owing to the problems of measurement noise and incomplete measurement (i.e., only a limited number of measurement points are employed and high‐mode information is lost because of the digitization of the signal and measurement noise), the results of crack detection as an inverse problem contain uncertainties. To explicitly handle such uncertainty, the proposed crack detection method follows the Bayesian statistical system identification framework. Rather than pinpointing the crack parameters (i.e., the crack location, length, and depth), the posterior probability density function (PDF) of the crack parameters is calculated to quantify the confidence level of the identified results, which is extremely important for engineers when they make judgments about remedial works. This article reports the theoretical development of the modeling of a cracked plate and a crack detection method together with numerical verification of the proposed method. The case study results are very encouraging, and indicate that the proposed method is feasible.     

Automated Pixel‐Level Pavement Crack Detection on 3D Asphalt Surfaces Using a Deep‐Learning Network

The CrackNet, an efficient architecture based on the Convolutional Neural Network (CNN), is proposed in this article for automated pavement crack detection on 3D asphalt surfaces with explicit objective of pixel‐perfect accuracy. Unlike the commonly used CNN, CrackNet does not have any pooling layers which downsize the outputs of previous layers. CrackNet fundamentally ensures pixel‐perfect accuracy using the newly developed technique of invariant image width and height through all layers. CrackNet consists of five layers and includes more than one million parameters that are trained in the learning process. The input data of the CrackNet are feature maps generated by the feature extractor using the proposed line filters with various orientations, widths, and lengths. The output of CrackNet is the set of predicted class scores for all pixels. The hidden layers of CrackNet are convolutional layers and fully connected layers. CrackNet is trained with 1,800 3D pavement images and is then demonstrated to be successful in detecting cracks under various conditions using another set of 200 3D pavement images. The experiment using the 200 testing 3D images showed that CrackNet can achieve high Precision (90.13%), Recall (87.63%) and F‐measure (88.86%) simultaneously. Compared with recently developed crack detection methods based on traditional machine learning and imaging algorithms, the CrackNet significantly outperforms the traditional approaches in terms of F‐measure. Using parallel computing techniques, CrackNet is programmed to be efficiently used in conjunction with the data collection software.     

Beamlet Transform‐Based Technique for Pavement Crack Detection and Classification

Abstract: This article presents a Beamlet transform‐based approach to automatically detect and classify pavement cracks in digital images. The proposed method uses a pavement distress image enhancement algorithm to correct the nonuniform background illumination by calculating the multiplicative factors that eliminate the background lighting variation. To extract linear features such as surface cracks from the pavement images, the image is partitioned into small windows and a Beamlet transform‐based algorithm is applied. The crack segments are then linked together and classified into four types: vertical, horizontal, transversal, and block. Simulation results show that the method is effective and robust in the extraction of cracks on a variety of pavement images.     

Rissbreitenberechnung für Elementwände und ‐decken aus Stahlbeton für Zwangbeanspruchungen rechtwinklig zu den Elementfugen

Calculation of crack widths of element ceilings and walls made of reinforced concrete due to imposed deformation perpendicular to the assembly joints. Prefabricated walls and floors have load bearing properties similar to geometrically identical monolithic members. This also applies to the effects of restraint strain caused by constrained shortening of members, if it occurs in the main bearing direction. In contrast to that load‐bearing structures of half‐prefabricated elements differ from monolithic structures in their behaviour at restraint strain orthogonal to the direction of span. The element joints form notches, in which tensile cracking already occurs at low strains. Crack spacing is determined by the modular grid. A phase of developing cracks, as it can be seen at monolithic components, does basically not occur. After a very short period, in which single cracks develop and which is finished when a crack occurs in the last joint, only small restraint strain is needed to reach a fully developed crack pattern. Due to larger crack spacing (element width) the concrete strain between the cracks is more significant than it is in monolithic members. Broader elements require more reinforcement across the joints to ensure the same crack widths. A method is suggested to calculate crack width and minimum reinforcement of notched cross‐sections. Minimum reinforcement orthogonal to the joint is higher than it is in comparable monolithic components.     

Automatic pixel‐level multiple damage detection of concrete structure using fully convolutional network

Deep learning‐based structural damage detection methods overcome the limitation of inferior adaptability caused by extensively varying real‐world situations (e.g., lighting and shadow changes). However, most deep learning‐based methods detect structural damage at the image level and grid‐cell level. To provide pixel‐level detection of multiple damages, a Fully Convolutional Network (FCN)‐based multiple damages detection method for concrete structure is proposed. To realize this method, a database of 2,750 images (with 504 × 376 pixels) including crack, spalling, efflorescence, and hole images in concrete structure is built, and the four damages included in those images are labeled manually. Then, the architecture of the FCN is modified, trained, validated, and tested using this database. A strategy of model‐based transfer learning is used to initialize the parameters of the FCN during the training process. The results show 98.61% pixel accuracy (PA), 91.59% mean pixel accuracy (MPA), 84.53% mean intersection over union (MIoU), and 97.34% frequency weighted intersection over union (FWIoU). Subsequently, the robustness and adaptability of the trained FCN model is tested and the damage is extracted, where damage areas are provided according to a calibrated relation between the ratio (the pixel area and true area of the detected object) and the distance from the smartphone to the concrete surface using a laser range finder. A comparative study is conducted to examine the performance of the proposed FCN‐based approach using a SegNet‐based method. The results show that the proposed method substantiates quite better performance and can indeed detect multiple concrete damages at the pixel level in realistic situations.     

一种改进的用于裂缝图像分割的Otsu方法

裂缝是隧道衬砌最常见的病害之一,基于近几年快速发展的工程检测系统与图像处理算法的研究,提出了一种CCD相机的衬砌裂缝快速检测系统采集裂缝图像。在提取裂缝特征之前,需要将裂缝区域与图像背景分离。采用Otsu法进行分割处理,然而传统的Otsu方法对裂缝区域过小的图像易产生欠分割;对背景不单一或光照度不均匀的裂缝图像易造成过分割的情况。根据Ostu方法分割特点对该方法进行改进,以达到更好的裂缝图像分割效果,从而为实现隧道裂缝的快速检测埋下基础。     

A Pavement Crack Detection Method Combining 2D with 3D Information Based on Dempster‐Shafer Theory

Pavement cracking is one of the main distresses presented in the road surface. Objective and accurate detection or evaluation for these cracks is an important task in the pavement maintenance and management. In this work, a new pavement crack detection method is proposed by combining two‐dimensional (2D) gray‐scale images and three‐dimensional (3D) laser scanning data based on Dempster‐Shafer (D‐S) theory. In this proposed method, 2D gray‐scale image and 3D laser scanning data are modeled as a mass function in evidence theory, and 2D and 3D detection results for pavement cracks are fused at decision‐making level. The experimental results show that the proposed method takes advantage of the respective merits of 2D images and 3D laser scanning data and therefore improves the pavement crack detection accuracy and reduces recognition error rate compared to 2D image intensity‐based methods.     

Performance of a BOTDR optical fibre sensing technique for crack detection in concrete structures

Strain distribution measurements and problematic crack detection are important issues in the damage detection and performance evaluation of concrete, or reinforced concrete, structures. In recent years, the Brillouin Optical Time Domain Reflectometry (BOTDR) based optical fibre sensing technique has attracted great attention as a distributed monitoring method. Current BOTDR instruments are suitable for strain measurements over a certain distance (termed spatial resolution), but damage such as cracks in concrete structures are local. It is crucial to find an effective method to detect local damage in concrete. In this study, two basic optical fibre installation methods, overall bonding (OB) installation and point fixation (PF) installation, are proposed. Then, several unique installation methods (one-round, one-round superposition and two-round superposition) are proposed and investigated experimentally for a reinforced concrete bending beam. The efficiency of the proposed installation methods and the effect of the length of the sensing region on the measurement accuracy are also discussed. Experimental results show that the n-round superposition installation method can effectively and correctly detect the total crack width within a relatively local region. The performance of the overall bonding and point fixation installation methods with different sensing region lengths, or gauge lengths, for local crack initiation and total crack width measurement is also discussed.     

Quantitative road crack evaluation by a U-Net architecture using smartphone images and Lidar data

Road cracks are a major concern for administrators. Visual inspection is labor-intensive. The accuracy of previous algorithms for detecting cracks in images requires improvement. Further, the length and thickness of cracks must be estimated. Light detection and ranging (Lidar), a standard smartphone feature is used to develop a method for the completely automatic, accurate, and quantitative evaluation of road cracks. The two contributions of this study are as follows. To achieve the highest segmentation accuracy, U-Net is combined with data augmentation and morphology transform. To calculate the crack length and thickness, crack images are registered into Lidar color data. The proposed algorithm was validated using a public database of road cracks and those measured by the authors. The algorithm was 95% accurate in determining crack length. The coefficient of determination for thickness estimation accuracy was 0.98 addressing various crack shapes and asphalt pavement patterns.     

A Nonballasted Rail Track Slab Crack Identification Method Using a Level‐Set‐Based Active Contour Model

The nonballasted rail tracks have been extensively applied in the new high‐speed railway system development in China, that is, the China Railway Track System (CRTS). However, many defects have been identified during the operation of the CRTS, among which concrete slab crack is recognized as one of the most common, yet critical defects that require accurate identification and timely maintenance attention. Because of the unique outlook of the cracks in nonballasted rail track slabs captured in the survey imagery, direct adaptations of the existing crack extraction methods show dramatically degraded performance. A new automatic crack identification method is developed in this study by employing a region‐based active contour framework with the intensity cluster energy. The proposed method embodies three major contributions, including (1) a heavy penalization energy component that could effectively avoid both under‐ and overevolutions; (2) a multiphase level‐set function that effectively evolves the contours generated with different intensity clusters; and (3) a two‐step implementation of the framework that significantly improves the efficiency. The experimental test evaluates the performance of the proposed method using the data collected on high‐speed rail tracks in Hebei Province, China. The proposed method accurately identifies more than 93.0% of digitized cracks in different crack patterns and challenging backgrounds using a data set consisting of 1,500 synthetic images and 150 actual images. In addition, it shows promising performance in comparison with other popular state‐of‐the‐art crack detection algorithms in terms of accuracy and computational efficiency. The proposed method has demonstrated the promising capacity to support a reliable and efficient nonballasted rail track crack identification and to facilitate the subsequent maintenance cost‐effectively.