Automated Pixel‐Level Pavement Crack Detection on 3D Asphalt Surfaces Using a Deep‐Learning Network

The CrackNet, an efficient architecture based on the Convolutional Neural Network (CNN), is proposed in this article for automated pavement crack detection on 3D asphalt surfaces with explicit objective of pixel‐perfect accuracy. Unlike the commonly used CNN, CrackNet does not have any pooling layers which downsize the outputs of previous layers. CrackNet fundamentally ensures pixel‐perfect accuracy using the newly developed technique of invariant image width and height through all layers. CrackNet consists of five layers and includes more than one million parameters that are trained in the learning process. The input data of the CrackNet are feature maps generated by the feature extractor using the proposed line filters with various orientations, widths, and lengths. The output of CrackNet is the set of predicted class scores for all pixels. The hidden layers of CrackNet are convolutional layers and fully connected layers. CrackNet is trained with 1,800 3D pavement images and is then demonstrated to be successful in detecting cracks under various conditions using another set of 200 3D pavement images. The experiment using the 200 testing 3D images showed that CrackNet can achieve high Precision (90.13%), Recall (87.63%) and F‐measure (88.86%) simultaneously. Compared with recently developed crack detection methods based on traditional machine learning and imaging algorithms, the CrackNet significantly outperforms the traditional approaches in terms of F‐measure. Using parallel computing techniques, CrackNet is programmed to be efficiently used in conjunction with the data collection software.     

Image‐based crack assessment of bridge piers using unmanned aerial vehicles and three‐dimensional scene reconstruction

Crack assessment of bridge piers using unmanned aerial vehicles (UAVs) eliminates unsafe factors of manual inspection and provides a potential way for the maintenance of transportation infrastructures. However, the implementation of UAV‐based crack assessment for real bridge piers is hindered by several key issues, including the following: (a) both perspective distortion and the geometry distortion by nonflat structural surfaces usually appear on crack images taken by the UAV system from the pier surface; however, these two kinds of distortions are difficult to correct at the same time; and (b) the crack image taken by a close‐range inspection flight UAV system is partially imaged, containing only a small part of the entire surface of the pier, and thereby hinders crack localization. In this paper, a new image‐based crack assessment methodology for bridge piers using UAV and three‐dimensional (3D) scene reconstruction is proposed. First, the data acquisition of UAV‐based crack assessment is discussed, and the UAV flight path and photography strategy for bridge pier assessment are proposed. Second, image‐based crack detection and 3D reconstruction are conducted to obtain crack width feature pair sequences and 3D surface models, respectively. Third, a new method of projecting cracks onto a meshed 3D surface triangular model is proposed, which can correct both the perspective distortion and geometry distortion by nonflat structural surfaces, and realize the crack localization. Field test investigations of crack assessment of a real bridge pier using a UAV are carried out for illustration, validation, and error analysis of the proposed methodology.     

Beamlet Transform‐Based Technique for Pavement Crack Detection and Classification

Abstract: This article presents a Beamlet transform‐based approach to automatically detect and classify pavement cracks in digital images. The proposed method uses a pavement distress image enhancement algorithm to correct the nonuniform background illumination by calculating the multiplicative factors that eliminate the background lighting variation. To extract linear features such as surface cracks from the pavement images, the image is partitioned into small windows and a Beamlet transform‐based algorithm is applied. The crack segments are then linked together and classified into four types: vertical, horizontal, transversal, and block. Simulation results show that the method is effective and robust in the extraction of cracks on a variety of pavement images.     

A grid-based classification and box-based detection fusion model for asphalt pavement crack

Crack identification is essential for the preventive maintenance of asphalt pavement. Through periodic inspection, the characteristics of existing and emerging cracks can be obtained, and the pavement condition index can be calculated to assess pavement health. The most common method to estimate the area of cracks in an image is to count the number of grid cells or boxes that cover the cracks in an image. Accurate and efficient crack identification is the premise of crack assessment. However, the current patch-based classification method is limited by the receptive field and cannot be used to directly classify cracks. Furthermore, the postprocessing algorithm in anchor-based detection is not explicitly optimized for crack topology. In this paper, a new model, which is the fusion of grid-based classification and box-based detection based on You Only Look Once version 5 (YOLO v5) is developed and described for the first time. The accuracy and efficiency of the model are high partly due to the implementation of a shared backbone network, multi-task learning, and joint training. The non-maximum suppression (NMS)–area-reduction suppression (ARS) algorithm is presented to filter redundant, overlapping prediction boxes in the postprocessing stage for the crack topology, and the mapping and matching algorithm is proposed to combine the advantages of both the grid-based and box-based models. A double-labeled dataset containing tens of thousands of asphalt pavement images is assembled, and the proposed method is verified on the test set. The fusion model has superior performance over the individual classification and detection models, and the proposed NMS-ARS algorithm further improves the detection accuracy. Experimental results demonstrate that the presented method effectively realizes automatic crack identification for asphalt pavement.     

Automated Pixel‐Level Pavement Crack Detection on 3D Asphalt Surfaces with a Recurrent Neural Network

A recurrent neural network (RNN) called CrackNet‐R is proposed in the article for fully automated pixel‐level crack detection on three‐dimensional (3D) asphalt pavement surfaces. In the article, a new recurrent unit, gated recurrent multilayer perceptron (GRMLP), is proposed to recursively update the internal memory of CrackNet‐R. Unlike the widely used long short‐term memory (LSTM) and gated recurrent unit (GRU), GRMLP is intended for deeper abstractions on the inputs and hidden states by conducting multilayer nonlinear transforms at gating units. CrackNet‐R implements a two‐phase sequence processing: sequence generation and sequence modeling. Sequence generation is specifically developed in the study to find the best local paths that are most likely to form crack patterns. Sequence modeling predicts timely probabilities of the input sequence being a crack pattern. In terms of sequence modeling, GRMLP slightly outperforms LSTM and GRU by using only one more nonlinear layer at each gate. In addition to sequence processing, an output layer is proposed to produce pixel probabilities based on timely probabilities predicted for sequences. The proposed output layer is critical for pixel‐perfect accuracy, as it accomplishes the transition from sequence‐level learning to pixel‐level learning. Using 3,000 diverse 3D images, the training of CrackNet‐R is completed through optimizing sequence modeling, sequence generation, and the output layer serially. The experiment using 500 testing pavement images shows that CrackNet‐R can achieve high Precision (88.89%), Recall (95.00%), and F‐measure (91.84%) simultaneously. Compared with the original CrackNet, CrackNet‐R is about four times faster and introduces tangible improvements in detection accuracy.     

Real‐time crack assessment using deep neural networks with wall‐climbing unmanned aerial system

Crack information provides important evidence of structural degradation and safety in civil structures. Existing inspection methods are inefficient and difficult to rapidly deploy. A real‐time crack inspection method is proposed in this study to address this difficulty. Within this method, a wall‐climbing unmanned aerial system (UAS) is developed to acquire detailed crack images without distortion, then a wireless data transmission method is applied to fulfill real‐time detection requirements, allowing smartphones to receive real‐time video taken from the UAS. Next, an image data set including 1,330 crack images taken by the wall‐climbing UAS is established and used for training a deep‐learning model. For increasing detection speed, state‐of‐the‐art convolutional neural networks (CNNs) are compared and employed to train the crack detector; the selected model is transplanted into an android application so that the detection of cracks can be undertaken on a smartphone in real time. Following this, images with cracks are separated and crack width is calculated using an image processing method. The proposed method is then applied to a building where crack information is acquired and calculated accurately with high efficiency, thus verifying the practicability of the proposed method and system.     

Encoder–decoder network for pixel‐level road crack detection in black‐box images

Timely monitoring of pavement cracks is essential for successful maintenance of road infrastructure. Accurate information concerning crack location and severity enables proactive management of the infrastructure. Black‐box cameras, which are becoming increasingly widespread at an affordable price, can be used as efficient road‐image collectors over a wide area. However, the cracks in these images are difficult to detect, because the images containing them often include objects other than roads. Thus, we propose a pixel‐level detection method for identifying road cracks in black‐box images using a deep convolutional encoder–decoder network. The encoder consists of convolutional layers of the residual network for extracting crack features, and the decoder consists of deconvolutional layers for localizing the cracks in an input image. The proposed network was trained on 427 out of 527 images extracted from black‐box videos and tested on the remaining 100 images. Compared with VGG‐16, ResNet‐50, ResNet‐101, ResNet‐200 with transfer learning, and ResNet‐152 without transfer learning, ResNet‐152 with transfer learning exhibited the best performance, achieving recall, precision, and intersection of union of 71.98%, 77.68%, and 59.65%, respectively. The experimental results prove that the proposed method is optimal for detecting cracks in black‐box images at the pixel level.     

Zernike‐moment measurement of thin‐crack width in images enabled by dual‐scale deep learning

Although crack inspection is a routine practice in civil infrastructure management (especially for highway bridge structures), it is time‐consuming and safety‐concerning to trained engineers and costly to the stakeholders. To automate this in the near future, the algorithmic challenge at the onset is to detect and localize cracks in imagery data with complex scenes. The rise of deep learning (DL) sheds light on overcoming this challenge through learning from imagery big data. However, how to exploit DL techniques is yet to be fully explored. One primary component of practical crack inspection is that it is not merely detection via visual recognition. To evaluate the potential risk of structural failure, it entails quantitative characterization, which usually includes crack width measurement. To further facilitate the automation of machine‐vision‐based concrete crack inspection, this article proposes a DL‐enabled quantitative crack width measurement method. In the detection and mapping phase, dual‐scale convolutional neural networks are designed to detect cracks in complex scene images with validated high accuracy. Subsequently, a novel crack width estimation method based on the use of Zernike moment operator is further developed for thin cracks. The experimental results based on a laboratory loading test agree well with the direct measurements, which substantiates the effectiveness of the proposed method for quantitative crack detection.     

基于图象子块分布特性的路面破损图象特征提取

由于路面破损形式的多种多样,造成路面破损分类[1]成为一大难题,这极大的限制了路面破损自动检测的普及和发展,使得路面破损自动检测即使在发达国家也普及得不够理想。本文在前文提出的破损密度因子的基础上,进一步设计了出方向密度因子,得到一种基于图象子块分布特性的路面破损识别算法。通过仿真,验证了其对常见的5种路面破损类型进行分类的可行性。为了进一步验证我们提出的识别算法,论文选择了另外一种路面破损分类算法,即PROXIMITY算法进行神经网络仿真对比。神经网络的训练样本是两组,测试样本也是两组,进行了四次仿真对比。四次仿真结果都显示方向密度因子算法优于PROXIMITY算法。     

基于计算机视觉技术的结构表面裂缝检测方法研究

计算机视觉技术用于混凝土结构表面裂缝检测,具有现场检测方便、效率高、客观性强的特点,但图像数据分析是该技术的核心,其中裂缝提取与定量测量较为复杂。为提高裂缝图像处理效率和准确率,将深度学习和数字图像处理技术相结合,提出一种裂缝检测方法。建立基于深度卷积神经网络的裂缝识别模型,在图像上自动定位裂缝并结合图像局域阈值分割方法提取裂缝。在裂缝宽度定量测量方面,采用双边滤波算法和三段线性变换对裂缝图像进行预处理,提高了裂缝边缘识别的精确度。通过改进边缘梯度法,实现裂缝最大宽度的定位和裂缝最大宽度的自动获取。该研究为全自动识别裂缝图像及高精度测量裂缝宽度提供了一种解决方法。     

The Fast Prefix Coding Algorithm (FPCA) for 3D Pavement Surface Data Compression

The enormous data inflow during three‐dimensional (3D) pavement surface data collection requires an efficient compression system for 3D data. However, with respect to the phase of lossless encoding, the commonly used Huffman Coding is inefficient in terms of speed and memory usage for encoding 3D pavement surfaces. The Fast Prefix Coding Algorithm (FPCA) is proposed in the article as an effective substitute of Huffman Coding at the stage of lossless encoding. It is demonstrated in the article that the FPCA is much faster and more memory efficient than Huffman Coding, while outperforming Shannon–Fano Coding in terms of both redundancy and time efficiency. The FPCA‐based coding approach is a modification of the baseline JPEG algorithm to support 3D pavement data whose dynamic range is more than 12 bits. The presented modifications include algorithms for Quantization, Run‐Length Encoding and Entropy Coding without limiting data depth in terms of dynamic range. Compared with the baseline JPEG approach, the proposed coding system is able to restrict the data loss more successfully and can achieve a significantly higher level of time efficiency and compression ratio (over than 30:1 for most of the evaluated 3D images). With parallel computing techniques, encoding full‐lane width pavement in 3D and at 1 mm resolution with an up‐to‐date desktop computer can be conducted at 150 MPH or even higher speed.     

Reconstruction of sub-mm 3D pavement images using recursive generative adversarial network for faster texture measurement

It is challenging to collect 3D pavement images with desired resolution for accurate texture measurement at driving speeds with current devices, particularly in the longitudinal direction. This paper presents a novel superresolution technique with recursive generative adversarial network, called Pavement Texture Super Resolution Generative Adversarial Network (PT-SRGAN), to reconstruct 0.1-mm pavement 3D image from low-resolution data for faster texture measurement. With the proposed pseudo-Laplacian pyramid and an array of learning strategies, the developed PT-SRGAN reconstructs 0.1-mm 3D texture images with multiple upscaling factors in longitudinal direction. Combined with the evaluation mask, the proposed method is substantially superior to other methods in terms of three metrics when comparing the image quality of reconstructed 0.1-mm 3D images against ground truth. The preliminary results indicate that the proposed method enables data collection at driving speeds up to 24 mph to collect 3D pavement images at sub-mm resolution for faster texture measurement.     

基于轨道移动激光扫描的圆形隧道正射影像纠正算法

高质量的隧道内壁影像是隧道病害检测的基础,而基于轨道移动激光扫描技术采集的圆形隧道内壁影像,在断面方向和运动方向上存在较大的影像长度误差.本文提出一种基于轨道移动激光扫描的圆形隧道正射影像纠正算法,利用速度纠正和正射投影纠正,插值激光扫描数据生成隧道内壁影像.经实验证明,该算法可有效纠正圆形隧道内壁影像在断面方向和运动...     

Computer vision‐based recognition of 3D relationship between construction entities for monitoring struck‐by accidents

Struck‐by accidents often cause serious injuries in construction. Monitoring of the struck‐by hazards in terms of spatial relationship between a worker and a heavy vehicle is crucial to prevent such accidents. The computer vision‐based technique has been put forward for monitoring the struck‐by hazards but there exists shortages such as spatial relationship distortion due to two‐dimensional (2D) image pixels‐based estimation and self‐occlusion of heavy vehicles. This study is aimed to address these problems, including the detection of workers and heavy vehicles, three‐dimensional (3D) bounding box reconstruction for the detected objects, depth and range estimation in the monocular 2D vision, and 3D spatial relationship recognition. A series of experiments were conducted to evaluate the proposed method. The proposed method is expected to estimate 3D spatial relationship between construction worker and heavy vehicle in a real‐time and view‐invariant manner, thus enhancing struck‐by hazards monitoring at the construction site.     

Quantitative road crack evaluation by a U-Net architecture using smartphone images and Lidar data

Road cracks are a major concern for administrators. Visual inspection is labor-intensive. The accuracy of previous algorithms for detecting cracks in images requires improvement. Further, the length and thickness of cracks must be estimated. Light detection and ranging (Lidar), a standard smartphone feature is used to develop a method for the completely automatic, accurate, and quantitative evaluation of road cracks. The two contributions of this study are as follows. To achieve the highest segmentation accuracy, U-Net is combined with data augmentation and morphology transform. To calculate the crack length and thickness, crack images are registered into Lidar color data. The proposed algorithm was validated using a public database of road cracks and those measured by the authors. The algorithm was 95% accurate in determining crack length. The coefficient of determination for thickness estimation accuracy was 0.98 addressing various crack shapes and asphalt pavement patterns.     

A Nonballasted Rail Track Slab Crack Identification Method Using a Level‐Set‐Based Active Contour Model

The nonballasted rail tracks have been extensively applied in the new high‐speed railway system development in China, that is, the China Railway Track System (CRTS). However, many defects have been identified during the operation of the CRTS, among which concrete slab crack is recognized as one of the most common, yet critical defects that require accurate identification and timely maintenance attention. Because of the unique outlook of the cracks in nonballasted rail track slabs captured in the survey imagery, direct adaptations of the existing crack extraction methods show dramatically degraded performance. A new automatic crack identification method is developed in this study by employing a region‐based active contour framework with the intensity cluster energy. The proposed method embodies three major contributions, including (1) a heavy penalization energy component that could effectively avoid both under‐ and overevolutions; (2) a multiphase level‐set function that effectively evolves the contours generated with different intensity clusters; and (3) a two‐step implementation of the framework that significantly improves the efficiency. The experimental test evaluates the performance of the proposed method using the data collected on high‐speed rail tracks in Hebei Province, China. The proposed method accurately identifies more than 93.0% of digitized cracks in different crack patterns and challenging backgrounds using a data set consisting of 1,500 synthetic images and 150 actual images. In addition, it shows promising performance in comparison with other popular state‐of‐the‐art crack detection algorithms in terms of accuracy and computational efficiency. The proposed method has demonstrated the promising capacity to support a reliable and efficient nonballasted rail track crack identification and to facilitate the subsequent maintenance cost‐effectively.     

Image analysis method for crack distribution and width estimation for reinforced concrete structures

Crack observation is important for evaluating the structural performance and safety of reinforced concrete (RC) structures. Most of the existing image-based crack detection methods are based on edge detection algorithms, which detect cracks that are wide enough to present dark areas in the obtained images. Cracks initiate as thin cracks, generally having width less than the width of a pixel in images; such cracks are generally undetectable by edge detection-based methods.An image analysis method is proposed to observe the development and distribution of thin cracks on RC surfaces; it also allows estimation of crack widths. Image matching based on optical flow and subpixel is employed to analyze slight concrete surface displacements. Camera calibration is included to eliminate perspective effects and lens distortion. Geometric transformation is adopted so that cameras do not need to be perpendicular to the observed surface or specified positions. Formulas are proposed to estimate the width of shear crack opening. The proposed method was then applied to a cyclic test of an RC structure. The crack widths and their development analyzed by the image analysis were verified with human inspection in the test. In addition, concrete surface cracks that appeared at a very early stage of the test could be observed by the proposed method before they could be detected by the naked eye. The results thus demonstrate that the proposed image analysis method offers an efficient way applicable not only for structural tests but also for crack-based structural-health-monitoring applications.     

Concrete Crack Detection by Multiple Sequential Image Filtering

Abstract: This article presents a new robust automated image processing method for detecting cracks in surface images of concrete structures. This method involves two steps: (1) development of an image filter for detecting major cracks using genetic programming (GP), and (2) elimination of residual noise after filtering and detection of indistinct cracks by iterative applications of the image filter to the local regions surrounding the cracks. The proposed method can be used for the accurate detection of cracks in surface images recorded under various conditions. Moreover, the widths of the detected cracks can be quantified on the basis of the spatial derivatives of the brightness patterns. The estimated crack widths are in good agreement with those measured manually.     

基于图像局部网格特征的隧道衬砌裂缝自动识别

 裂缝是隧道衬砌最常见也是最严重的病害之一。针对常规图像识别方法存在的问题,提出一种基于图像局部网格特征的隧道衬砌裂缝自动识别方法。首先将图像划分为8 Pixel×8 Pixel的局部网格,基于局部网格内不同方向之间的亮度差异和对比度差异特征设计十字形模板,通过模板计算将网格中灰度值最小的像素识别为潜在的裂缝种子,最后采用种子连接算法将离散的裂缝种子像素连接成为完整的连续裂缝,在连接过程中自动计算裂缝的走向、长度和宽度。通过接受者操作特征曲线估计算法的最优参数和最佳阈值,从定性和定量分析两方面验证其可靠性和准确性。工程应用实例表明,算法能取得良好的裂缝识别效果,特别是对细微裂缝和存在渗漏水的衬砌图像,算法的可靠性和识别率均高于常规的图像识别方法。