Static-based early-damage detection using symbolic data analysis and unsupervised learning methods

A large amount of researches and studies have been recently performed by applying statistical and machine learning techniques for vibration-based damage detection. However, the global character inherent to the limited number of modal properties issued from operational modal analysis may be not appropriate for early-damage, which has generally a local character. The present paper aims at detecting this type of damage by using static SHM data and by assuming that early-damage produces dead load redistribution. To achieve this objective a data driven strategy is proposed, consisting of the combination of advanced statistical and machine learning methods such as principal component analysis, symbolic data analysis and cluster analysis. From this analysis it was observed that, under the noise levels measured on site, the proposed strategy is able to automatically detect stiffness reduction in stay cables reaching at least 1%.     

Pruning deep convolutional neural networks for efficient edge computing in condition assessment of infrastructures

Health monitoring of civil infrastructures is a key application of Internet of things (IoT), while edge computing is an important component of IoT. In this context, swarms of autonomous inspection robots, which can replace current manual inspections, are examples of edge devices. Incorporation of pretrained deep learning algorithms into these robots for autonomous damage detection is a challenging problem since these devices are typically limited in computing and memory resources. This study introduces a solution based on network pruning using Taylor expansion to utilize pretrained deep convolutional neural networks for efficient edge computing and incorporation into inspection robots. Results from comprehensive experiments on two pretrained networks (i.e., VGG16 and ResNet18) and two types of prevalent surface defects (i.e., crack and corrosion) are presented and discussed in detail with respect to performance, memory demands, and the inference time for damage detection. It is shown that the proposed approach significantly enhances resource efficiency without decreasing damage detection performance.     

Machine learning based novelty detection using modal analysis

Among many structural assessment methods, the change of modal characteristics is considered a well‐accepted damage detection method. However, the presence of environmental or operational variations may pollute the baseline and prevent a dependable assessment of the change. In recent years, the use of machine learning algorithms gained interest within structural health community, especially due to their ability and success in the elimination of ambient uncertainty. This paper proposes an end‐to‐end architecture to detect damage reliably by employing machine learning algorithms. The proposed approach streamlines (a) collection of structural response data, (b) modal analysis using system identification, (c) learning model, and (d) novelty detection. The proposed system aims to extract latent features of accessible modal parameters such as natural frequencies and mode shapes measured at undamaged target structure under temperature uncertainty and to reconstruct a new representation of these features that is similar to the original using well‐established machine learning methods for damage detection. The deviation between measured and reconstructed parameters, also known as novelty index, is the essential information for detecting critical changes in the system. The approach is evaluated by analyzing the structural response data obtained from finite element models and experimental structures. For the machine learning component of the approach, both principal component analysis (PCA) and autoencoder (AE) are examined. While mode shapes are known to be a well‐researched damage indicator in the literature, to our best knowledge, this research is the first time that unsupervised machine learning is applied using PCA and AE to utilize mode shapes in addition to natural frequencies for effective damage detection. The detection performance of this pipeline is compared to a similar approach where its learning model does not utilize mode shapes. The results demonstrate that the effectiveness of the damage detection under temperature variability improves significantly when mode shapes are used in the training of learning algorithm. Especially for small damages, the proposed algorithm performs better in discriminating system changes.     

Self-training approach for crack detection using synthesized crack images based on conditional generative adversarial network

Urban infrastructure plays a crucial role in determining the quality of life for citizens. However, given the increasing number of aging infrastructures, regular inspections are essential to prevent accidents. Deep learning studies have been conducted to detect structural damage and ensure high accuracy and reliability of these inspections. However, these detection algorithms often face challenges due to scarcity of damage data. To overcome this issue, this paper proposes a method for synthesizing crack images and utilizing them for crack detection. Initially, crack images are synthesized from labeled images by using a conditional generative adversarial network. Subsequently, a new self-training method is implemented wherein the synthesized crack images from the prediction images were incorporated into the learning process to enhance data diversity. The proposed approach yields promising results with a mean intersection over union of 80.34% and F1-score of 76.31% on average. The proposed method can aid further research on virtual image generation for crack detection, seeking to reduce the reliance on extensive image collection.     

Dynamic reduction-based structural damage detection of transmission towers: Practical issues and experimental verification

Structural damage detection in large-scale three-dimensional spatial structures is a challenging problem. It is impractical to develop a general damage-detection method that is applicable to all types of structural systems and all kinds of damage. A practical and efficient structural damage detection method must consider the characteristics of the target structure and damage in the development stage. In 2009, Yin et al. [33] proposed a damage detection method for the health monitoring of transmission towers. The method was developed based on the dynamical finite element (FE) model reduction technique, which utilizes identified modal parameters, such as natural frequencies and mode shapes, with only a limited number of sensors. In Ref. [33], the proposed method was numerically verified by simulated noisy data from a three-dimensional transmission tower model for single and multiple damage cases. This paper discusses some practical issues related to the proposed method, such as sensor placement and computational efficiency. Rather than proposing a general sensor placement method, a set of preliminary sensor locations is determined based on engineering judgement. This set of sensor locations is then checked against the results of a sensitivity analysis to ensure that the measured data contain information for identifying all of the target damage scenarios. To reduce the required computational power, two simplified versions of the proposed method are presented. The proposed method is then verified with a scaled-down model of a transmission tower (2.4 m high) that was built at the Structural Vibration Laboratory (SVL) of the City University of Hong Kong. This paper reports the detailed experimental setup and the method of extracting the modal parameters from a series of free vibration tests with only a limited number of sensors. The verification results show that the proposed damage detection method identifies the damaged sub-structures in all of the experimental cases considered. It must be pointed out that the transmission tower structure, in its operating conditions, suffers from the effect of the forces transmitted from the cables it carries. The influence of this force on the damage identification result is great and can not be neglected in practice. In the present experimental case study, only a transmission tower-like structure without carrying the cables is investigated in laboratory conditions.     

Recording of bridge damage areas by 3D integration of multiple images and reduction of the variability in detected results

Machine learning models have been developed to perform damage detection using images to improve bridge inspection efficiency. However, in damage detection using images alone, the 3D coordinates of the damage cannot be recorded. Furthermore, the accuracy of the detection depends on the quality of the images. This paper proposes a method to integrate and record the damage detected from multiple images into a 3D model using deep learning to detect the damage from bridge images and structure from motion to identify the shooting position. The proposed method reduces the variability of the detection results between images and can assess the scale of damage or, conversely, where there is no damage and the extent of inspection omissions. The proposed method has been applied to a real bridge, and it has been shown that the actual damage locations can be recorded as a 3D model.     

Image‐based post‐disaster inspection of reinforced concrete bridge systems using deep learning with Bayesian optimization

Many bridge structures, one of the most critical components in transportation infrastructure systems, exhibit signs of deteriorations and are approaching or beyond the initial design service life. Therefore, structural health inspections of these bridges are becoming critically important, especially after extreme events. To enhance the efficiency of such an inspection, in recent years, autonomous damage detection based on computer vision has become a research hotspot. This article proposes a three‐level image‐based approach for post‐disaster inspection of the reinforced concrete bridge using deep learning with novel training strategies. The convolutional neural network for image classification, object detection, and semantic segmentation are, respectively, proposed to conduct system‐level failure classification, component‐level bridge column detection, and local damage‐level damage localization. To enable efficient training and prediction using a small data set, the model robustness is a crucial aspect to be taken into account, generally through its hyperparameters’ selection. This article, based on Bayesian optimization, proposes a principled manner of such selection, with which very promising results (well over 90% accuracies) and robustness are observed on all three‐level deep learning models.     

悬臂梁损伤的支持向量机识别研究

针对结构损伤识别中,损伤与其影响因素之间的复杂非线性关系,提出了结构损伤识别的支持向量机方法。支持向量机是一种基于统计学理论的机器学习算法,本文以模态频率作为损伤标识量,通过支持向量机建立了损伤程度和频率之间的支持向量机模型,并以悬臂梁的损伤为例进行了计算分析,结果表明提出的方法是科学,可行的。     

Fragility Assessment Model of Building Structures Using Characteristics of Artificial Aftershock Motions

Any additional loads applied to a damaged structure can aggravate its instability and thus, the impact of successive earthquakes need to be considered. This study proposed a quantitative assessment model for the fragility of a damaged structure subjected to aftershock. Mean period and the strong motion duration were considered as characteristics of earthquake motions. Simulation models of two reinforced concrete structures and one steel structure were selected to examine the applicability of the model. Based on the suggested fragility and residual deformation coefficients, critical earthquake sequences for each structure were identified. The proposed model was efficient in selecting critical earthquake sequences by using the limited number of aftershocks, and these sequences are expected to be useful indicators in the establishment of a retrofit plan according to the predicted structural response and target performance levels.     

Dynamic reduction-based structural damage detection of transmission tower utilizing ambient vibration data

This paper reports a feasibility study of utilizing ambient vibration data measured from a limited number of sensors in the structural damage detection of transmission towers, which are large-scaled three-dimensional spatial structures. To develop a practical and efficient structural damage detection methodology, the characteristics of transmission towers are considered in the development stage, including the most common types of damage, accessible locations for installing sensors, the technique needed to identify a reliable set of modal parameters utilizing ambient vibration data, a method to divide the transmission tower into sub-structures for structural damage detection, a way to formulate the damage detection problem, and the corresponding solution method. The proposed methodology is numerically verified by simulated noisy data from a three-dimensional transmission tower sample under both single and multiple damage cases. Very encouraging results are obtained, showing that the proposed methodology can identify the damaged sub-structure by estimating the ‘equivalent’ stiffness reduction even in the presence of both measurement noise and modeling error.     

Automatic pixel‐level multiple damage detection of concrete structure using fully convolutional network

Deep learning‐based structural damage detection methods overcome the limitation of inferior adaptability caused by extensively varying real‐world situations (e.g., lighting and shadow changes). However, most deep learning‐based methods detect structural damage at the image level and grid‐cell level. To provide pixel‐level detection of multiple damages, a Fully Convolutional Network (FCN)‐based multiple damages detection method for concrete structure is proposed. To realize this method, a database of 2,750 images (with 504 × 376 pixels) including crack, spalling, efflorescence, and hole images in concrete structure is built, and the four damages included in those images are labeled manually. Then, the architecture of the FCN is modified, trained, validated, and tested using this database. A strategy of model‐based transfer learning is used to initialize the parameters of the FCN during the training process. The results show 98.61% pixel accuracy (PA), 91.59% mean pixel accuracy (MPA), 84.53% mean intersection over union (MIoU), and 97.34% frequency weighted intersection over union (FWIoU). Subsequently, the robustness and adaptability of the trained FCN model is tested and the damage is extracted, where damage areas are provided according to a calibrated relation between the ratio (the pixel area and true area of the detected object) and the distance from the smartphone to the concrete surface using a laser range finder. A comparative study is conducted to examine the performance of the proposed FCN‐based approach using a SegNet‐based method. The results show that the proposed method substantiates quite better performance and can indeed detect multiple concrete damages at the pixel level in realistic situations.     

Structural Damage Detection Method Using Uncertain Frequency Response Functions

Abstract:   A structural damage detection method using uncertain frequency response functions (FRFs) is presented in this article. Structural damage is detected from the changes in FRFs from the original intact state. The measurements are always contaminated by noise, and sufficient data are often difficult to obtain; making it difficult to detect damage with a finite number of data. To surmount this, we introduce hypothesis testing based on the bootstrap method to statistically prevent detection errors due to measurement noise. The proposed method iteratively zooms in on the damaged elements by excluding the elements which were assessed as undamaged from among the damage candidates, step by step. The proposed approach was applied to numerical simulations using a 2D frame structure and its efficiency was confirmed.     

Interpretable deep learning for roof fall hazard detection in underground mines

Roof falls due to geological conditions are major hazards in the mining industry, causing work time loss, injuries, and fatalities. There are roof fall problems caused by high horizontal stress in several large-opening limestone mines in the eastern and midwestern United States. The typical hazard management approach for this type of roof fall hazards relies heavily on visual inspections and expert knowledge. In this context, we proposed a deep learning system for detection of the roof fall hazards caused by high horizontal stress. We used images depicting hazardous and non-hazardous roof conditions to develop a convolutional neural network (CNN) for autonomous detection of hazardous roof conditions. To compensate for limited input data, we utilized a transfer learning approach. In the transfer learning approach, an already-trained network is used as a starting point for classification in a similar domain. Results show that this approach works well for classifying roof conditions as hazardous or safe, achieving a statistical accuracy of 86.4%. This result is also compared with a random forest classifier, and the deep learning approach is more successful at classification of roof conditions. However, accuracy alone is not enough to ensure a reliable hazard management system. System constraints and reliability are improved when the features used by the network are understood. Therefore, we used a deep learning interpretation technique called integrated gradients to identify the important geological features in each image for prediction. The analysis of integrated gradients shows that the system uses the same roof features as the experts do on roof fall hazards detection. The system developed in this paper demonstrates the potential of deep learning in geotechnical hazard management to complement human experts, and likely to become an essential part of autonomous operations in cases where hazard identification heavily depends on expert knowledge. Moreover, deep learning-based systems reduce expert exposure to hazardous conditions.     

Large-scale structural health monitoring using composite recurrent neural networks and grid environments

The demand for resilient and smart structures has been rapidly increasing in recent decades. With the occurrence of the big data revolution, research on data-driven structural health monitoring (SHM) has gained traction in the civil engineering community. Unsupervised learning, in particular, can be directly employed solely using field-acquired data. However, the majority of unsupervised learning SHM research focuses on detecting damage in simple structures or components and possibly low-resolution damage localization. In this study, an unsupervised learning, novelty detection framework for detecting and localizing damage in large-scale structures is proposed. The framework relies on a 5D, time-dependent grid environment and a novel spatiotemporal composite autoencoder network. This network is a hybrid of autoencoder convolutional neural networks and long short-term memory networks. A 10-story, 10-bay, numerical structure is used to evaluate the proposed framework damage diagnosis capabilities. The framework was successful in diagnosing the structure health state with average accuracies of 93% and 85% for damage detection and localization, respectively.     

High-resolution model reconstruction and bridge damage detection based on data fusion of unmanned aerial vehicles light detection and ranging data imagery

Damage detection is essential for the maintenance of transportation infrastructure that experiences high daily traffic levels in potentially extreme environments and changes in use patterns. However, traditional physical inspection is always labor-intensive, subjective, and biased, lacking the objective perspective required for a comprehensive and reliable assessment. Recently, unmanned aerial vehicles (UAVs) combined with emerging high-performance sensor(s) have been intensively researched. Here, we present an aerial bridge surface survey method that can be used to assess damage. Existing damage detection methods focus on single types of damage and are limited in locating global damage, whereas our method detects two types of damage on the surface and marks them in a panoramic image. The workflow involves three steps: data acquisition using a meticulous UAV flight strategy that covers the entire surface, data processing using image-based and point-cloud models after polynomial rectification, and data output (i.e., damage detected by the combined models). To verify the method, a field test detected damage to two real bridges. A UAV equipped with a camera and light detection and ranging (LiDAR) equipment was employed. Experiments demonstrate the effectiveness of the proposed method, which is capable of producing accurate outputs and detecting damage with an average position error of 13.37 mm and a relative size error of 25.25%. Owing to the data fusion model taking advantage of two-dimensional (2D) images and 3D LiDAR data, it outputs a high-resolution 3D model and avoids environmental disturbances. After decision-making-level damage fusion, all position and size properties of damage information are computed into a panoramic damage image. This panoramic image showing all detecting damage helps technicians perform maintenance; the image can be zoomed to focus on any issue individually.     

Detection of damage locations and damage steps in pile foundations using acoustic emissions with deep learning technology

The aim of this study is to propose a new detection method for determining the damage locations in pile foundations based on deep learning using acoustic emission data. First, the damage location is simulated using a back propagation neural network deep learning model with an acoustic emission data set acquired from pile hit experiments. In particular, the damage location is identified using two parameters: the pile location (P_L) and the distance from the pile cap (D_S). This study investigates the influences of various acoustic emission parameters, numbers of sensors, sensor installation locations, and the time difference on the prediction accuracy of P_L and D_S. In addition, correlations between the damage location and acoustic emission parameters are investigated. Second, the damage step condition is determined using a classification model with an acoustic emission data set acquired from uniaxial compressive strength experiments. Finally, a new damage detection and evaluation method for pile foundations is proposed. This new method is capable of continuously detecting and evaluating the damage of pile foundations in service.     

Developing a Smart Structure Using Integrated Subspace‐Based Damage Detection and Semi‐Active Control

Damage detection in large building structures has always faced challenges due to analyzing the large amount of measured data. In this article, a new damage detection approach based on subspace method is proposed to identify damages using limited output data. Also, a new scheme is presented to develop a smart structure by integrating structural health monitoring with semi‐active control strategy. If damage occurs in such a structure under severe excitations, the proposed scheme has the capability to exert necessary control forces in order to compensate for damage and reduce simultaneously the dynamic response of the structure. The reliability and feasibility of the proposed method are demonstrated by implementing the technique to two shear building structures with semi‐active control devices. Results show that the damage could be identified accurately with saving time and cost due to less computation even under noise existence; and dynamic response is significantly reduced in the smart structure.     

Wavelet packet based damage detection in beam-like structures without baseline modal parameters

In this paper, a new approach for damage detection in beam-like structures is presented. The method can be used without the need for baseline modal parameters of the undamaged structure. Another advantage of the proposed method is that it can be implemented using a small number of sensors. In the proposed technique, the measured dynamic signals are decomposed into the wavelet packet decomposition (WPD) components, the power spectrum density (PSD) of each component is estimated and then a damage localisation indicator is computed to indicate the structural damage. The proposed method is firstly illustrated with a simulated beam and the identified damage is satisfactory with assumed damage. Then, the method is applied to a steel beam. The effect of damage location and the effects of wavelet type and the decomposition level are examined. The results show that the proposed method has great potential in crack detection of beam-like structures.     

Combination of neural network and wavelet transform for islanding detection of distributed generation in a small-scale network

In this paper, an optimum and intelligent method is proposed for islanding detection using wavelet transform. The suggested relay is based on neural network (NN) in which different heuristic algorithms are used for training the NN. In the proposed method, the appropriate signals for detection procedure as well as mother wavelet are selected optimally, based on the mean square error (MSE) concept. Lately, the desired relay is trained by the optimally selected signals using four different algorithms and the optimum condition of the fault detector is identified. Simulation results approved that non detection zone (NDZ) has a significant reduction utilising the proposed intelligent technique. The contributions of the proposed method include presenting an appropriate signal selection method based on MSE, selecting optimum number of relay input signals using the proposed technique, fast training of intelligent relay by using least information, solving threshold selection problem and reduction of NDZ approximately to zero.     

Active learning structural model updating of a multisensory system based on Kriging method and Bayesian inference

Model updating techniques are often applied to calibrate the numerical models of bridges using structural health monitoring data. The updated models can facilitate damage assessment and prediction of responses under extreme loading conditions. Some researchers have adopted surrogate models, for example, Kriging approach, to reduce the computations, while others have quantified uncertainties with Bayesian inference. It is desirable to further improve the efficiency and robustness of the Kriging-based model updating approach and analytically evaluate its uncertainties. An active learning structural model updating method is proposed based on the Kriging method. The expected feasibility learning function is extended for model updating using a Bayesian objective function. The uncertainties can be quantified through a derived likelihood function. The case study for verification involves a multisensory vehicle-bridge system comprising only two sensors, with one installed on a vehicle parked temporarily on the bridge and another mounted directly on the bridge. The proposed algorithm is utilized for damage detection of two beams numerically and an aluminum model beam experimentally. The proposed method can achieve satisfactory accuracy in identifying damage with much less data, compared with the general Kriging model updating technique. Both the computation and instrumentation can be reduced for structural health monitoring and model updating.