复杂环境公路隧道裂缝快速识别与分割算法研究

随着我国公路隧道由建设为主朝建养并重转化,在运营里程快速增长与既有隧道劣化加剧的双重作用下,移动检测及结构安全快速诊断已成为目前公路隧道运营维养领域的研究热点之一。我国已研发了多种类型的隧道检测车,为裂缝、渗漏水等表观病害的快速检测提供了手段,然而公路隧道衬砌图像背景复杂、干扰因素多、裂缝占比小的特点,给检测数据的快速分析带来巨大挑战,已成为制约技术推广的主要瓶颈。基于深度学习算法,本文提出了一种将目标识别与语义分割进行组合的裂缝快速提取方法,首先采用Faster R-CNN网络对原始衬砌图像进行目标识别,判定所采集图片是否存在裂缝并智能框选出裂缝区域;随后对框选出的裂缝区域自动裁切,由此过滤不含裂缝的图片并去除含裂缝图片中的干扰背景,再利用U-Net语义分割网络对裂缝进行像素级分割。通过实际工程验证发现,单幅图像裂缝整体分割时间小于0.15 s,在常见各类干扰因素下,目标识别F1值可达到92%,语义分割像素准确度可达到98%以上。与阈值分割和同类智能分割算法相比,本方法显著提高了识别速度与精度,为从隧道检测车海量数据中进行快速准确的裂缝提取提供了良好手段。     

Building height calculation for an urban area based on street view images and deep learning

The building heights of an urban area are useful for space analysis, urban planning, and city management. To this end, a novel method for building height calculation for an urban area is proposed based on street view images and a deep learning model, that is, mask region-based convolutional neural network (Mask R-CNN). First, a spider of street view maps was developed, and an optimization model for observation locations was designed based on a genetic algorithm, by which the street view images of all buildings can be obtained with the minimum number of downloads. Subsequently, a deep learning workflow was designed based on the Mask R-CNN to detect buildings from the panorama images. Finally, an accurate height calculation model considering repeated detection of buildings was developed by mapping between detected buildings and actual buildings. Case studies indicate that the mean error of height calculation is 0.78 m, which achieves high precision for calculating building heights in urban areas, while the average calculation time is 4.57 s per building, which indicates that the proposed method is efficient for the application in urban areas.     

Hybrid semantic segmentation for tunnel lining cracks based on Swin Transformer and convolutional neural network

In the field of tunnel lining crack identification, the semantic segmentation algorithms based on convolution neural network (CNN) are extensively used. Owing to the inherent locality of CNN, these algorithms cannot make full use of context semantic information, resulting in difficulty in capturing the global features of crack. Transformer-based networks can capture global semantic information, but this method also has the deficiencies of strong data dependence and easy loss of local features. In this paper, a hybrid semantic segmentation algorithm for tunnel lining crack, named SCDeepLab, is proposed by fusing Swin Transformer and CNN in the encoding and decoding framework of DeepLabv3+ to address the above issues. In SCDeepLab, a joint backbone network is introduced with CNN-based Inverse Residual Block and Swin Transformer Block. The former is used to extract the local detailed information of the crack to generate the shallow feature layer, whereas the latter is used to extract the global semantic information to obtain the deep feature layer. In addition, Efficient Channel Attention enhanced Feature Fusion Module is proposed to fuse the shallow and deep features to combine the advantages of the two types of features. Furthermore, the strategy of transfer learning is adopted to solve the data dependency of Swin Transformer. The results show that the mean intersection over union (mIoU) and mean pixel accuracy (mPA) of SCDeepLab on the data sets constructed in this paper are 77.41% and 84.42%, respectively, which have higher segmentation accuracy than previous CNN-based and transformer-based semantic segmentation algorithms.     

Autonomous damage segmentation and measurement of glazed tiles in historic buildings via deep learning

Detecting and measuring the damage on historic glazed tiles plays an important role in the maintenance and protection of historic buildings. However, the current visual inspection method for identifying and assessing superficial damage on historic buildings is time and labor intensive. In this article, a novel two‐level object detection, segmentation, and measurement strategy for large‐scale structures based on a deep‐learning technique is proposed. The data in this study are from the roof images of the Palace Museum in China. The first level of the model, which is based on the Faster region‐based convolutional neural network (Faster R‐CNN), automatically detects and crops two types of glazed tile photographs from 100 roof images (2,488 × 3,264 pixels). The average precision values (AP) for roll roofing and pan tiles are 0.910 and 0.890, respectively. The cropped images are used to form a dataset for training a Mask R‐CNN model. The second level of the model, which is based on Mask R‐CNN, automatically segments and measures the damage based on the cropped historic tile images; the AP for the damage segmentation is 0.975. Based on Mask R‐CNN, the predicted pixel‐level damage segmentation result is used to quantitatively measure the morphological features of the damage, such as the damage topology, area, and ratio. To verify the performance of the proposed method, a comparative study was conducted with Mask R‐CNN and a fully convolutional network. This is the first attempt at employing a two‐level strategy to automatically detect, segment, and measure large‐scale superficial damage on historic buildings based on deep learning, and it achieved good results.     

Fire Recognition Based On Multi-Channel Convolutional Neural Network

In recent years, fire recognition methods have received more and more attention in the fields of academy and industry. Current sensor-based recognition methods rely heavily on the external physical signals, which will probably reduce the recognition precision if the external environment changes dramatically. With the rapid development of high-definition camera, the methods based on image feature extraction provide another solution which tries to conduct pattern recognition for the monitoring video. However, these methods couldn’t be widely and successfully applied to fire detection due to two deficiencies: (1) there are too many interference items like lamplight and car highlight in the room or tunnel, which will disturb the recognition performance largely; (2) The features depend on much prior knowledge about flame and smoke, and there lacks a universal and automatic extraction method for various fire scenes. As a breakthrough in pattern recognition, deep learning is capable of exploring the useful information from raw data, and can automatically provide accurate recognition results. Therefore, based on deep learning idea, a novel fire recognition method based on multi-channel convolutional neural network is proposed in this paper to overcome the deficiencies mentioned above. First, three channel colorful images are constructed as the input of convolutional neural network; Second, the hidden layers with multiple-layer convolution and pooling are constructed, and simultaneously, the model parameters are find tuned by using back propagation; Finally, softmax method is used to conduct the classification about fire recognition. To save the training time, we utilize GPU to construct training and test models. From public fire dataset and Internet, we collect 7000 images for training and 4494 images for test, and then run experiments with the comparison of four baseline methods including deep neural network, support vector machine based on scale-invariant feature transform feature, stack auto-encoder and deep belief network. The experimental results show that the proposed method is more capable of restoring the features of input image by means of hidden output figure, and for various flame scenes and types, the proposed method can reach 98% or more classification accuracy, getting improvement of around 2% than the traditional feature-based method. Also, the proposed method always outperforms other Deep Learning methods in terms of ROC curve, recall rate, precision rate and F1-score.     

Pavement defect detection with fully convolutional network and an uncertainty framework

Image segmentation has been implemented for pavement defect detection, from which types, locations, and geometric information can be obtained. In this study, an integration of a fully convolutional network with a Gaussian‐conditional random field (G‐CRF), an uncertainty framework, and probability‐based rejection is proposed for detecting pavement defects. First, a fully convolutional network is designed to generate preliminary segmentation results, and a G‐CRF is used to refine the segmentation. Second, epistemic and aleatory uncertainties in the model and database are considered to overcome the disadvantages of traditional deep‐learning methods. Last, probability‐based rejection is conducted to remove unreasonable segmentations. The proposed method is evaluated on a data set of images that were obtained from 16 highways. The proposed integration segments pavement distresses from digital images with desirable performance. It also provides a satisfactory means to improve the accuracy and generalization performance of pavement defect detection without introducing a delay into the segmentation process.     

Severe rail wear detection with rail running band images

Rail wear occurs continuously owing to the rolling contact load of trains and is fundamental for railway operational safety. A point-based manual rail wear inspection cannot satisfy the increasing demand for rapid, low-cost, and continuous monitoring. This paper proposes a depth-plus-region fusion network for detecting rail wear on a running band, which is a collection of wheel–rail interaction traces. The following steps are involved in the proposed method. (i) A depth map estimated by a modified MiDaS model is utilized as guidance for exploiting the depth information of the running band for rail wear detection. (ii) The running band of a rail is segmented and extracted from images using an improved mask region-based convolutional neural network that uses the scale and ratio information to perform instance segmentation of the running band images. (iii) A two-channel attention–fusion network that classifies rail wear is constructed. In this study, we collected real-world running band images and rail wear-related data to validate our approach using a high-accuracy rail-profile measurement tool. The case-study results demonstrated that the proposed method can rapidly and accurately detect rail wear under different ambient light conditions. Moreover, the recall rate of severe wear detection was 84.21%.     

Automated detection of sewer pipe defects in closed-circuit television images using deep learning techniques

Sanitary sewer systems are designed to collect and transport sanitary wastewater and stormwater. Pipe inspection is important in identifying both the type and location of pipe defects to maintain the normal sewer operations. Closed-circuit television (CCTV) has been commonly utilized for sewer pipe inspection. Currently, interpretation of the CCTV images is mostly conducted manually to identify the defect type and location, which is time-consuming, labor-intensive and inaccurate. Conventional computer vision techniques are explored for automated interpretation of CCTV images, but such process requires large amount of image pre-processing and the design of complex feature extractor for certain cases. In this study, an automated approach is developed for detecting sewer pipe defects based on a deep learning technique namely faster region-based convolutional neural network (faster R-CNN). The detection model is trained using 3000 images collected from CCTV inspection videos of sewer pipes. After training, the model is evaluated in terms of detection accuracy and computation cost using mean average precision (mAP), missing rate, detection speed and training time. The proposed approach is demonstrated to be applicable for detecting sewer pipe defects accurately with high accuracy and fast speed. In addition, a new model is constructed and several hyper-parameters are adjusted to study the influential factors of the proposed approach. The experiment results demonstrate that dataset size, initialization network type and training mode, and network hyper-parameters have influence on model performance. Specifically, the increase of dataset size and convolutional layers can improve the model accuracy. The adjustment of hyper-parameters such as filter dimensions or stride values contributes to higher detection accuracy, achieving an mAP of 83%. The study lays the foundation for applying deep learning techniques in sewer pipe defect detection as well as addressing similar issues for construction and facility management.     

Recognition of rust grade and rust ratio of steel structures based on ensembled convolutional neural network

Ensembled convolutional neural network (ECNN) was utilized to recognize the rust grade and rust ratio of steel structure to partially replace traditional visual inspection. The performance of ECNN was demonstrated by theoretical analysis and experimental verification, and the application scenarios of ECNN in the task of rust grade recognition and rust ratio recognition were discussed. The accuracy of ECNN classifier reached 93%, which improves upon the highest accuracy of 90% achieved by using a single classifier. By visualizing the misclassified images, it was found that the rust grade of misclassified image is indistinguishable and the classifiers show strong fault tolerance. The ensembled model is more robust than the single model in the task of rust ratio recognition. Gaussian blur was applied to the test images to study the effect of image blur on model performance, and the results show that the rust segmentation model was not susceptible to image blur.     

基于自注意力的隧道视频火灾识别技术研究

针对公路隧道火灾样本量少、深度学习效果不理想的问题，研究一种小样本学习技术，以提高对隧道火灾样本的利用率，并在此基础上利用成熟的机器学习方法，提出一种基于自注意力的隧道视频火灾识别技术。该技术采用自注意力机制结合SVM分类器搭建火焰识别模型，该模型针对各项特征对火焰识别的重要性分配不同的注意力权重，形成注意力矩阵，并将权重矩阵与特征向量的值相加权，通过SVM的Hinge Loss进行线性支持向量机分类，对公路隧道火灾进行识别和预警。在火灾识别训练过程中，通过对火焰疑似区域进行检测，并利用数据增强技术达到样本扩增的目的，随后采用多通道融合的特征提取方式构建特征向量，输入设计的自注意力火焰识别模型中，通过梯度下降优化器进行小批量模型训练，降低迭代次数，最终获得最优特征权重参数，得到最佳识别模型。试验结果表明，该方法在模型训练时收敛较快，在火焰识别时相比未使用小样本学习的传统SVM算法，准确率提高了5%，因此能在小样本环境下有效提高火灾识别的准确度。     

A Fast Detection Method via Region‐Based Fully Convolutional Neural Networks for Shield Tunnel Lining Defects

Tunnel lining defects are an important indicator reflecting the safety status of shield tunnels. Inspired by the state‐of‐the‐art deep learning, a method for automatic intelligent classification and detection methodology of tunnel lining defects is presented. A fully convolutional network (FCN) model for classification is proposed. Information about defects, collected using charge‐coupled device cameras, was used to train the model. The model's performance was compared to those of GoogLeNet and VGG. The best‐set accuracy of the proposed model was over 95% at a test‐time speed of 48 ms per image. For defects detection, image features were computed from large‐scale images by the FCN and then detected using a region proposal network and position‐sensitive region of interest pooling. Some indices (detection rate, detection accuracy, and detection efficiency, locating accuracy) were used to evaluate the model. The comparisons with faster R‐CNN and a traditional method were conducted. The results show that the model is very fast and efficient, allowing automatic intelligent classification and detection of tunnel lining defects.     

卷积神经网络在结构损伤诊断中的应用

对卷积神经网络(CNN)在工程结构损伤诊断中的应用进行了深入探讨; 以多层框架结构节点损伤位置的识别问题为研究对象,构建了可以直接从结构动力反应信号中进行学习并完成分类诊断的基于原始信号和傅里叶频域信息的一维卷积神经网络模型和基于小波变换数据的二维卷积神经网络模型; 从输入数据样本类别、训练时间、预测准确率、浅层与深层卷积神经网络以及不同损伤程度的影响等多方面进行了研究。结果表明:卷积神经网络能从结构动力反应信息中有效提取结构的损伤特征,且具有很高的识别精度; 相比直接用加速度反应样本,使用傅里叶变换后的频域数据作为训练样本能使CNN的收敛速度更快、更稳定,并且深层CNN的性能要好于浅层CNN; 将卷积神经网络用于工程结构损伤诊断具有可行性,特别是在大数据处理和解决复杂问题能力方面与其他传统诊断方法相比有很大优势,应用前景广阔。     

隧道掌子面前方低阻夹层的瞬变电磁探测研究

 实践表明,隧道掌子面前方含水、充泥等不良地质体对隧道施工构成潜在的危险和障碍,严重影响隧道施工速度。由于瞬变电磁法对低阻体反应灵敏,在进行瞬变电磁隧道超前地质预报时,这些含水、充泥等不良地质体通常又可以近似当成一种低阻夹层来处理,因此,利用瞬变电磁法对掌子面前方低阻夹层的探测研究就显得很有意义。采用建立在全空间理论上的瞬变电磁等效导电平面法,利用遗传算法进行计算,并结合视纵向电导及其微分成像对隧道掌子面前方低阻夹层展开相应的正演模拟及应用研究,重点讨论视纵向电导微分成像的电性界面响应特点及其与视电阻率断面等值线联合解释方法,总结出相应的结论和规律,以此来指导瞬变电磁法在隧道内的超前地质预报工作,不断提高瞬变电磁探测的精度及准确度。     

卷积神经网络在建筑消防信息化的应用——以城市综合管廊火灾监控为例

随着科学技术的进步,越来越多的行业和领域在朝着信息化方向发展。综合管廊已成为城市能源输送的重要保障,但是在大型市政基础设施集中化发展背景下,其火灾隐患问题逐渐凸显。利用YOLO V5建立的卷积神经网络能够对火焰进行高精度识别,进而实现从识别结果中提取实时火焰蔓延位置、蔓延速度和火焰宽度等重要火灾发展关键参数。通过设计12组发展速度不同的综合管廊电缆火灾试验,对卷积神经网络进行训练并验证其信息提取的准确性。结果表明：卷积神经网络提取的火焰前锋蔓延位置平均相对误差为5%～15%、火焰蔓延速度平均相对误差为6%～20%、火焰宽度平均相对误差为10%～27%,进而证实该方法能够保证良好的提取精度。对建筑消防信息化监控来说,该方法能为火灾现场制定灭火救援战术提供关键依据,并让实时研判火灾发展趋势、评估事故严重性和估计事故损失成为可能。     

Vision-aided three-dimensional damage quantification and finite element model geometric updating for reinforced concrete structures

This article presents a vision-aided framework to achieve three-dimensional (3D) concrete damage quantification and finite element (FE) model geometric updating for reinforced concrete structures. The framework can process images and point clouds to extract damage information and update it in an FE model. First, a mask region convolutional neural network was used to realize highly precise damage detection and segmentation based on images. Second, a 3D point cloud was adopted in conjunction with the processed images for 3D damage qualification. The model-updating method enables an FE model to delete concrete elements to update the variations in volume caused by structural damage. This framework supports interaction with mainstream FE software for further analysis. To demonstrate the efficiency of the proposed framework, it was used in an experiment on a reinforced-concrete shear wall.     

A unified convolutional neural network integrated with conditional random field for pipe defect segmentation

Semantic segmentation of closed‐circuit television (CCTV) images can facilitate automatic severity assessment of sewer pipe defects by assigning defect labels to each pixel on the image, from which defect types, locations, and geometric information can be obtained. In this study, a unified neural network, namely DilaSeg‐CRF, is proposed by fully integrating a deep convolutional neural network (CNN) with dense conditional random field (CRF) for improving the segmentation accuracy. First, DilaSeg is constructed with dilated convolution and multiscale techniques for producing feature maps with high resolution. The steps of the dense CRF inference algorithm are converted into CNN operations, which are then formulated as recurrent neural network (RNN) layers. The DilaSeg‐CRF is proposed by integrating DilaSeg with the RNN layers. Images containing three common types of sewer defects are collected from CCTV inspection videos and are annotated with ground truth labels, after which the proposed models are trained and evaluated. Experiments demonstrate that the end‐to‐end trainable DilaSeg‐CRF can improve the segmentation significantly, with an increase of 32% and 20% in mean intersection over union (mIoU) values compared with fully convolutional network (FCN‐8s) and DilaSeg, respectively. Our proposed DilaSeg‐CRF also achieves faster inference speed than FCN and eliminates the manual postprocessing for refining the segmentation results.     

Transformer-optimized generation,detection, and tracking network for images with drainage pipeline defects

Regular detection of defects in drainage pipelines is crucial. However, some problems associated with pipeline defect detection, such as data scarcity and defect counting difficulty, need to be addressed. Therefore, a Transformer-optimized generation, detection, and counting method for drainage-pipeline defects was established in this paper. First, a generation network called Trans-GAN-Cla was developed for data augmentation. A classification network was trained to improve the quality of the generated images. Second, a detection and tracking model called Trans-Det-Tra was developed to track and count the number of defects. Third, the feature extraction capability of the proposed method was improved by leveraging Transformers. Compared with some well-known convolutional neural network-based methods, the proposed network achieved the best classification and detection accuracies of 87.2% and 87.57%, respectively. Furthermore, the F₁ scores were 87.7% and 91.9%. Finally, two pieces of onsite videos were detected and tracked, and the numbers of misalignments and obstacles were accurately counted. The results indicate that the established Transformer-optimized method can generate high-quality images and realize the high-accuracy detection and counting of drainage pipeline defects.     

Concrete crack detection using context‐aware deep semantic segmentation network

Computer‐vision and deep‐learning techniques are being increasingly applied to inspect, monitor, and assess infrastructure conditions including detection of cracks. Traditional vision‐based methods to detect cracks lack accuracy and generalization to work on complicated infrastructural conditions. This paper presents a novel context‐aware deep convolutional semantic segmentation network to effectively detect cracks in structural infrastructure under various conditions. The proposed method applies a pixel‐wise deep semantic segmentation network to segment the cracks on images with arbitrary sizes without retraining the prediction network. Meanwhile, a context‐aware fusion algorithm that leverages local cross‐state and cross‐space constraints is proposed to fuse the predictions of image patches. This method is evaluated on three datasets: CrackForest Dataset (CFD) and Tomorrows Road Infrastructure Monitoring, Management Dataset (TRIMMD) and a Customized Field Test Dataset (CFTD) and achieves Boundary F1 (BF) score of 0.8234, 0.8252, and 0.7937 under 2‐pixel error tolerance margin in CFD, TRIMMD, and CFTD, respectively. The proposed method advances the state‐of‐the‐art performance of BF score by approximately 2.71% in CFD, 1.47% in TRIMMD, and 4.14% in CFTD. Moreover, the averaged processing time of the proposed system is 0.7 s on a typical desktop with Intel^® Quad‐Core? i7‐7700 CPU@3.6 GHz Processor, 16GB RAM and NVIDIA GeForce GTX 1060 6GB GPU for an image of size 256 × 256 pixels.     

基于卷积神经网络的结构损伤识别研究进展

为系统梳理基于卷积神经网络的工程结构损伤识别方法的发展脉络和研究现状,分别从结构损伤的识别目的和在不同类型结构中的应用两方面进行了归类、分析和评价。介绍了卷积神经网络的基本结构和评价指标,回顾了卷积神经网络的研究和应用历程。在损伤的识别目的方面,主要针对混凝土结构损伤的分类、定位和分割,详细介绍了基于不同类型卷积神经网络的结构损伤识别方法,即基于分类的方法、基于回归的方法和像素级的图像分割算法; 分析了各类方法所使用的卷积神经网络模型的结构特点、计算流程、训练方法和损伤识别性能。在不同类型结构的损伤识别方面,分析了卷积神经网络在砌体结构、钢结构桥梁和古建筑木结构裂缝识别中的应用。最后,基于对卷积神经网络优缺点的思考,提出了发展建议和展望。结果表明:训练样本中结构损伤的多样性对模型的损伤识别效果影响较大; 现有基于卷积神经网络的损伤分割方法模型参数较多,计算量大; 采用数据增广和迁移学习方法可有效防止模型过拟合,提高模型训练效率; 针对微小损伤和不同类型结构损伤的识别,此类方法的性能有待提高。     

Monocular 3D object detection for construction scene analysis

Three-dimensional (3D) object detection, that is, localizing and classifying all critical objects in a 3D space, is essential for downstream construction scene analysis tasks. However, accurate instance segmentation, few 2D object segmentation and 3D object detection data sets, high-quality feature representations for depth estimation, and limited 3D cues from a single red-green-blue (RGB) image pose significant challenges to 3D object detection and severely hinder its practical applications. In response to these challenges, an improved cascade-based network with a transformer backbone and a boundary-patch-refinement method is proposed to build hierarchical features and refine object boundaries, resulting in better results in 2D object detection and instance segmentation. Furthermore, a novel self-supervised monocular depth learning method is proposed to extract better feature representations for depth estimation from construction site video data with unknown camera parameters. Additionally, a pseudo-LiDAR point cloud method and a 3D object detection method with a density-based clustering algorithm are proposed to detect 3D objects in a construction scene without help from 3D labels, which will serve as a good foundation for other downstream 3D tasks. Finally, the proposed model is evaluated for object instance segmentation and depth estimation on the moving objects in construction sites (MOCS) and construction scene data sets. It brings a 9.16% gain in terms of mean average precision (mAP) for object detection and a 4.92% gain in mask mAP for object instance segmentation. The average order accuracy and relative mean error for depth estimation are improved by 0.94% and 60.56%, respectively. This study aims to overcome the challenges and limitations of 3D object detection and facilitate practical applications in construction scene analysis.