A Pavement Crack Detection Method Combining 2D with 3D Information Based on Dempster‐Shafer Theory

Pavement cracking is one of the main distresses presented in the road surface. Objective and accurate detection or evaluation for these cracks is an important task in the pavement maintenance and management. In this work, a new pavement crack detection method is proposed by combining two‐dimensional (2D) gray‐scale images and three‐dimensional (3D) laser scanning data based on Dempster‐Shafer (D‐S) theory. In this proposed method, 2D gray‐scale image and 3D laser scanning data are modeled as a mass function in evidence theory, and 2D and 3D detection results for pavement cracks are fused at decision‐making level. The experimental results show that the proposed method takes advantage of the respective merits of 2D images and 3D laser scanning data and therefore improves the pavement crack detection accuracy and reduces recognition error rate compared to 2D image intensity‐based methods.     

基于图象子块分布特性的路面破损图象特征提取

由于路面破损形式的多种多样,造成路面破损分类[1]成为一大难题,这极大的限制了路面破损自动检测的普及和发展,使得路面破损自动检测即使在发达国家也普及得不够理想。本文在前文提出的破损密度因子的基础上,进一步设计了出方向密度因子,得到一种基于图象子块分布特性的路面破损识别算法。通过仿真,验证了其对常见的5种路面破损类型进行分类的可行性。为了进一步验证我们提出的识别算法,论文选择了另外一种路面破损分类算法,即PROXIMITY算法进行神经网络仿真对比。神经网络的训练样本是两组,测试样本也是两组,进行了四次仿真对比。四次仿真结果都显示方向密度因子算法优于PROXIMITY算法。     

A grid-based classification and box-based detection fusion model for asphalt pavement crack

Crack identification is essential for the preventive maintenance of asphalt pavement. Through periodic inspection, the characteristics of existing and emerging cracks can be obtained, and the pavement condition index can be calculated to assess pavement health. The most common method to estimate the area of cracks in an image is to count the number of grid cells or boxes that cover the cracks in an image. Accurate and efficient crack identification is the premise of crack assessment. However, the current patch-based classification method is limited by the receptive field and cannot be used to directly classify cracks. Furthermore, the postprocessing algorithm in anchor-based detection is not explicitly optimized for crack topology. In this paper, a new model, which is the fusion of grid-based classification and box-based detection based on You Only Look Once version 5 (YOLO v5) is developed and described for the first time. The accuracy and efficiency of the model are high partly due to the implementation of a shared backbone network, multi-task learning, and joint training. The non-maximum suppression (NMS)–area-reduction suppression (ARS) algorithm is presented to filter redundant, overlapping prediction boxes in the postprocessing stage for the crack topology, and the mapping and matching algorithm is proposed to combine the advantages of both the grid-based and box-based models. A double-labeled dataset containing tens of thousands of asphalt pavement images is assembled, and the proposed method is verified on the test set. The fusion model has superior performance over the individual classification and detection models, and the proposed NMS-ARS algorithm further improves the detection accuracy. Experimental results demonstrate that the presented method effectively realizes automatic crack identification for asphalt pavement.     

Monitoring Crack Changes in Concrete Structures

Abstract:   This study proposes a crack-monitoring system to quantify the change of cracks from multitemporal images during the monitoring period. A series of images were taken from an off-the-shelf digital camera. Concrete cracks were extracted from the digital images by employing a series of image-processing techniques. The image coordinates and orientation of same cracks can be changed since the position and direction of the portable camera vary at every exposure time. To monitor the crack changes (width and length), it is critical to transform the image coordinates of cracks extracted from each image into the same object coordinates of the concrete surface. In this study, such a geometric relationship was automatically recovered using the two-dimensional (2D) projective transformation based on the modified iterated Hough transform (MIHT) algorithm, the result of which solved the transformation parameters. To improve the computational operation of MIHT, regions of parameter estimation were also investigated. The developed algorithms were applied to monitor the crack of the concrete specimen. As a result, the change of cracks on the concrete specimen was successfully detected and accurately quantified.     

Quantitative road crack evaluation by a U-Net architecture using smartphone images and Lidar data

Road cracks are a major concern for administrators. Visual inspection is labor-intensive. The accuracy of previous algorithms for detecting cracks in images requires improvement. Further, the length and thickness of cracks must be estimated. Light detection and ranging (Lidar), a standard smartphone feature is used to develop a method for the completely automatic, accurate, and quantitative evaluation of road cracks. The two contributions of this study are as follows. To achieve the highest segmentation accuracy, U-Net is combined with data augmentation and morphology transform. To calculate the crack length and thickness, crack images are registered into Lidar color data. The proposed algorithm was validated using a public database of road cracks and those measured by the authors. The algorithm was 95% accurate in determining crack length. The coefficient of determination for thickness estimation accuracy was 0.98 addressing various crack shapes and asphalt pavement patterns.     

Encoder–decoder network for pixel‐level road crack detection in black‐box images

Timely monitoring of pavement cracks is essential for successful maintenance of road infrastructure. Accurate information concerning crack location and severity enables proactive management of the infrastructure. Black‐box cameras, which are becoming increasingly widespread at an affordable price, can be used as efficient road‐image collectors over a wide area. However, the cracks in these images are difficult to detect, because the images containing them often include objects other than roads. Thus, we propose a pixel‐level detection method for identifying road cracks in black‐box images using a deep convolutional encoder–decoder network. The encoder consists of convolutional layers of the residual network for extracting crack features, and the decoder consists of deconvolutional layers for localizing the cracks in an input image. The proposed network was trained on 427 out of 527 images extracted from black‐box videos and tested on the remaining 100 images. Compared with VGG‐16, ResNet‐50, ResNet‐101, ResNet‐200 with transfer learning, and ResNet‐152 without transfer learning, ResNet‐152 with transfer learning exhibited the best performance, achieving recall, precision, and intersection of union of 71.98%, 77.68%, and 59.65%, respectively. The experimental results prove that the proposed method is optimal for detecting cracks in black‐box images at the pixel level.     

Foreground–background separation technique for crack detection

Current level‐2 condition assessment methods for critical infrastructure assets mostly rely on human visual investigation of visible damages and patterns at the structure surface, which can be a costly, time‐consuming, and subjective exercise in reality. In this article, a novel method for crack detection is proposed via salient structure extraction from textured background. This method first extracts strong edges and distinguishes them from strong textures in a local neighborhood. Then, the spatial distribution of texture features is estimated to detect cracks as salient structures that are not widely spread across the whole image. The outputs from these two key steps are fused to calculate the final structure saliency map for generation of the crack masks. This method was validated on a data set with 704 images and the outcome revealed an average f‐measure of 75% in detecting the concrete cracks that is significantly higher than two other baseline methods.     

Concrete crack detection using context‐aware deep semantic segmentation network

Computer‐vision and deep‐learning techniques are being increasingly applied to inspect, monitor, and assess infrastructure conditions including detection of cracks. Traditional vision‐based methods to detect cracks lack accuracy and generalization to work on complicated infrastructural conditions. This paper presents a novel context‐aware deep convolutional semantic segmentation network to effectively detect cracks in structural infrastructure under various conditions. The proposed method applies a pixel‐wise deep semantic segmentation network to segment the cracks on images with arbitrary sizes without retraining the prediction network. Meanwhile, a context‐aware fusion algorithm that leverages local cross‐state and cross‐space constraints is proposed to fuse the predictions of image patches. This method is evaluated on three datasets: CrackForest Dataset (CFD) and Tomorrows Road Infrastructure Monitoring, Management Dataset (TRIMMD) and a Customized Field Test Dataset (CFTD) and achieves Boundary F1 (BF) score of 0.8234, 0.8252, and 0.7937 under 2‐pixel error tolerance margin in CFD, TRIMMD, and CFTD, respectively. The proposed method advances the state‐of‐the‐art performance of BF score by approximately 2.71% in CFD, 1.47% in TRIMMD, and 4.14% in CFTD. Moreover, the averaged processing time of the proposed system is 0.7 s on a typical desktop with Intel^® Quad‐Core? i7‐7700 CPU@3.6 GHz Processor, 16GB RAM and NVIDIA GeForce GTX 1060 6GB GPU for an image of size 256 × 256 pixels.     

Automatic Road Defect Detection by Textural Pattern Recognition Based on AdaBoost

Abstract: The state of roads is continuously degrading due to meteorological conditions, ground movements, and traffic, leading to the formation of defects, such as grabbing, holes, and cracks. In this article, a method to automatically distinguish images of road surfaces with defects from road surfaces without defects is presented. This method, based on supervised learning, is generic and may be applied to all type of defects present in those images. They typically present strong textural information with patterns that show fluctuations at small scales and some uniformity at larger scales. The textural information is described by applying a large set of linear and nonlinear filters. To select the most pertinent ones for the current application, a supervised learning based on AdaBoost is performed. The whole process is tested both on a textural recognition task based on the VisTex image database and on road images collected by a dedicated road imaging system. A comparison with a recent cracks detection algorithm from Oliveira and Correia demonstrates the proposed method's efficiency.     

Wavelet‐Based Detection of Beam Cracks Using Modal Shape and Frequency Measurements

Abstract: When a structure is subjected to dynamic or static loads, cracks may develop and the modal shapes and frequencies of the cracked structure may change accordingly. Based on this, a new method is proposed to locate beam cracks and to estimate their depths. The fault‐induced modal shape and frequency changes of cracked structures are taken into account to construct a new hybrid crack detection method. The method includes two steps: crack localization and depth estimation. The locations of the cracks are determined by applying the wavelet transform to the modal shape. Using the measured natural frequencies as inputs, the depths of the cracks are estimated from a database established by wavelet finite element method. The effectiveness of the proposed hybrid two‐step method is demonstrated by numerical simulation and experimental investigation of a cantilever beam with two cracks. Our analyses also indicate that the proposed method performed reasonably well at certain level of noise.     

Concrete Crack Detection by Multiple Sequential Image Filtering

Abstract: This article presents a new robust automated image processing method for detecting cracks in surface images of concrete structures. This method involves two steps: (1) development of an image filter for detecting major cracks using genetic programming (GP), and (2) elimination of residual noise after filtering and detection of indistinct cracks by iterative applications of the image filter to the local regions surrounding the cracks. The proposed method can be used for the accurate detection of cracks in surface images recorded under various conditions. Moreover, the widths of the detected cracks can be quantified on the basis of the spatial derivatives of the brightness patterns. The estimated crack widths are in good agreement with those measured manually.     

Neural Network for Rapid Depth Evaluation of Shallow Cracks in Asphalt Pavements

Abstract:   Rapid and nondestructive evaluation of pavement crack depths is a major challenge in pavement maintenance and rehabilitation. This article presents a computer-based methodology with which one can estimate the actual depths of shallow, surface-initiated fatigue cracks in asphalt pavements based on rapid measurement of their surface characteristics. It is shown that the complex overall relationship among crack depths, surface geometrical properties of cracks, pavement properties, and traffic characteristics can be learnt effectively by a neural network (NN). The learning task is facilitated by a database that includes relevant traffic and pavement characteristics of Florida's state highway network. In addition, the specific data used for the NN model development also contained laser-scanned microscopic surface geometrical properties of cracks in 95 pavement sections and pavement core samples scattered within five counties of Florida. Relatively advanced training algorithms were investigated in addition to the Standard Backpropagation algorithm to determine the optimal NN architecture. In terms of optimizing the NN training process, the "early stopping method" was found to be effective. The crack depth evaluation model was validated based on an unused portion of the database and fresh core samples. The results indicate the promise of NN usage in nondestructive estimation of shallow crack depths based on crack-surface geometry and pavement and traffic characteristics .     

Automated Object Identification Using Optical Video Cameras on Construction Sites

Abstract: Visual recording devices such as video cameras, CCTVs, or webcams have been broadly used to facilitate work progress or safety monitoring on construction sites. Without human intervention, however, both real‐time reasoning about captured scenes and interpretation of recorded images are challenging tasks. This article presents an exploratory method for automated object identification using standard video cameras on construction sites. The proposed method supports real‐time detection and classification of mobile heavy equipment and workers. The background subtraction algorithm extracts motion pixels from an image sequence, the pixels are then grouped into regions to represent moving objects, and finally the regions are identified as a certain object using classifiers. For evaluating the method, the formulated computer‐aided process was implemented on actual construction sites, and promising results were obtained. This article is expected to contribute to future applications of automated monitoring systems of work zone safety or productivity.     

Automated Pixel‐Level Pavement Crack Detection on 3D Asphalt Surfaces Using a Deep‐Learning Network

The CrackNet, an efficient architecture based on the Convolutional Neural Network (CNN), is proposed in this article for automated pavement crack detection on 3D asphalt surfaces with explicit objective of pixel‐perfect accuracy. Unlike the commonly used CNN, CrackNet does not have any pooling layers which downsize the outputs of previous layers. CrackNet fundamentally ensures pixel‐perfect accuracy using the newly developed technique of invariant image width and height through all layers. CrackNet consists of five layers and includes more than one million parameters that are trained in the learning process. The input data of the CrackNet are feature maps generated by the feature extractor using the proposed line filters with various orientations, widths, and lengths. The output of CrackNet is the set of predicted class scores for all pixels. The hidden layers of CrackNet are convolutional layers and fully connected layers. CrackNet is trained with 1,800 3D pavement images and is then demonstrated to be successful in detecting cracks under various conditions using another set of 200 3D pavement images. The experiment using the 200 testing 3D images showed that CrackNet can achieve high Precision (90.13%), Recall (87.63%) and F‐measure (88.86%) simultaneously. Compared with recently developed crack detection methods based on traditional machine learning and imaging algorithms, the CrackNet significantly outperforms the traditional approaches in terms of F‐measure. Using parallel computing techniques, CrackNet is programmed to be efficiently used in conjunction with the data collection software.     

Automating the crack map detection process for machine operated crack sealer

Automated crack-sealing machinery must accurately locate continuous individual crack segments so that they can be processed and treated effectively. In this paper, we illustrate the use of a geodesic minimal path based method for generating the crack map suitable for the path planning process. The user can detect continuous cracks that extend over several miles just by providing the starting point of a crack as an input to the algorithm. The algorithm can also detect transverse cracks by giving a single point on the crack. The continuous crack map generated can be utilized very efficiently to generate the optimal path for the crack sealer. An extensive qualitative and quantitative evaluation on real pavement images was done to show the usefulness of the algorithm. The algorithm is computationally fast and efficient.     

Real‐time crack assessment using deep neural networks with wall‐climbing unmanned aerial system

Crack information provides important evidence of structural degradation and safety in civil structures. Existing inspection methods are inefficient and difficult to rapidly deploy. A real‐time crack inspection method is proposed in this study to address this difficulty. Within this method, a wall‐climbing unmanned aerial system (UAS) is developed to acquire detailed crack images without distortion, then a wireless data transmission method is applied to fulfill real‐time detection requirements, allowing smartphones to receive real‐time video taken from the UAS. Next, an image data set including 1,330 crack images taken by the wall‐climbing UAS is established and used for training a deep‐learning model. For increasing detection speed, state‐of‐the‐art convolutional neural networks (CNNs) are compared and employed to train the crack detector; the selected model is transplanted into an android application so that the detection of cracks can be undertaken on a smartphone in real time. Following this, images with cracks are separated and crack width is calculated using an image processing method. The proposed method is then applied to a building where crack information is acquired and calculated accurately with high efficiency, thus verifying the practicability of the proposed method and system.     

基于图像局部网格特征的隧道衬砌裂缝自动识别

 裂缝是隧道衬砌最常见也是最严重的病害之一。针对常规图像识别方法存在的问题,提出一种基于图像局部网格特征的隧道衬砌裂缝自动识别方法。首先将图像划分为8 Pixel×8 Pixel的局部网格,基于局部网格内不同方向之间的亮度差异和对比度差异特征设计十字形模板,通过模板计算将网格中灰度值最小的像素识别为潜在的裂缝种子,最后采用种子连接算法将离散的裂缝种子像素连接成为完整的连续裂缝,在连接过程中自动计算裂缝的走向、长度和宽度。通过接受者操作特征曲线估计算法的最优参数和最佳阈值,从定性和定量分析两方面验证其可靠性和准确性。工程应用实例表明,算法能取得良好的裂缝识别效果,特别是对细微裂缝和存在渗漏水的衬砌图像,算法的可靠性和识别率均高于常规的图像识别方法。     

Deep Learning‐Based Crack Damage Detection Using Convolutional Neural Networks

A number of image processing techniques (IPTs) have been implemented for detecting civil infrastructure defects to partially replace human‐conducted onsite inspections. These IPTs are primarily used to manipulate images to extract defect features, such as cracks in concrete and steel surfaces. However, the extensively varying real‐world situations (e.g., lighting and shadow changes) can lead to challenges to the wide adoption of IPTs. To overcome these challenges, this article proposes a vision‐based method using a deep architecture of convolutional neural networks (CNNs) for detecting concrete cracks without calculating the defect features. As CNNs are capable of learning image features automatically, the proposed method works without the conjugation of IPTs for extracting features. The designed CNN is trained on 40 K images of 256 × 256 pixel resolutions and, consequently, records with about 98% accuracy. The trained CNN is combined with a sliding window technique to scan any image size larger than 256 × 256 pixel resolutions. The robustness and adaptability of the proposed approach are tested on 55 images of 5,888 × 3,584 pixel resolutions taken from a different structure which is not used for training and validation processes under various conditions (e.g., strong light spot, shadows, and very thin cracks). Comparative studies are conducted to examine the performance of the proposed CNN using traditional Canny and Sobel edge detection methods. The results show that the proposed method shows quite better performances and can indeed find concrete cracks in realistic situations.     

Automatic Detection of Deficient Video Log Images Using a Histogram Equity Index and an Adaptive Gaussian Mixture Model

Abstract: Video log images are often used by transportation agencies to manually or automatically extract roadway infrastructure information, including roadway geometry, signs, etc. Poor‐quality images, especially those having illumination‐related deficiencies caused by color corruption with a plain‐like grayscale histogram, sun glare, or darkness problems, are unacceptable and need to be identified. Manually reviewing the tens of millions of video log images for quality control is labor intensive and time‐consuming, so there is a need to develop automatic video log image quality control procedures. The contribution of this article is that it formulates a new problem of roadway video log image quality control and then proposes a reasonable solution to address this problem in the hope that it will motivate the development of new algorithms by other researchers. For the first time, an algorithm using a Histogram Equity Index (HEI) and an adaptive Gaussian Mixture Model is proposed to address the video log image quality issue by automatically detecting illumination‐related deficiencies. The Alberta Department of Transportation provided 15,489 video log images to test the proposed algorithm. Test results show that the developed algorithm can detect illumination‐related video log image deficiencies with a false positive rate of 4%, 3%, and 12%; a false negative rate of 15%, 17%, and 19% for plain‐like color corruption, dark, and sun glare conditions, respectively; computation time is 0.1 second/image. The proposed algorithm could potentially be used to improve video log image data quality control.     

Automated crack evaluation of a high‐rise bridge pier using a ring‐type climbing robot

This article proposes a deep learning‐based automated crack evaluation technique for a high‐rise bridge pier using a ring‐type climbing robot. First, a ring‐type climbing robot system composed of multiple vision cameras, climbing robot, and control computer is developed. By spatially moving the climbing robot system along a target bridge pier with close‐up scanning condition, high‐quality raw vision images are continuously obtained. The raw vision images are then processed through feature control‐based image stitching, deep learning‐based semantic segmentation, and Euclidean distance transform–based crack quantification algorithms. Finally, a digital crack map on the region of interest (ROI) of the target bridge pier can be automatically established. The proposed technique is experimentally validated using in situ test data obtained from Jang–Duck bridge in South Korea. The test results reveal that the proposed technique successfully evaluates cracks on the entire ROI of the bridge pier with precision of 90.92% and recall of 97.47%.