Leveraging Artificial Intelligence to unlock Next-Generation SHM software: Advanced Feature Extraction and Damage Identification
DOI:
https://doi.org/10.70465/ber.v2i1.16Keywords:
Artificial Intelligence, Digital Twin, Engineering Software, Machine Learning, Operational Modal Analysis, Structural Health MonitoringAbstract
The significant socio-economic impacts of aging infrastructure have driven the growing implementation of Structural Health Monitoring (SHM) worldwide as a key preventive maintenance strategy. However, scaling SHM systems nationwide presents significant hardware and software challenges, particularly in managing densely instrumented structures and deploying effective damage identification algorithms. This explains the circumstance that most SHM software tools are custom-built by specialized research groups, limiting technology transfer. In this context, although still in its early stages, the latest advances in Artificial Intelligence (AI) offer promising solutions to overcome these scalability issues. In this line, this work introduces the latest developments of MOVA/MOSS, a comprehensive SHM software platform developed by the authors that leverages AI to accelerate feature extraction in vibration-based systems and generate digital twins for quasi real-time damage identification. The potential of the developed platform is demonstrated through a real-world structure, the Mendez Nuñez Bridge in Spain, highlighting AI’s potential in facilitating the widespread adoption of SHM.
Downloads
Introduction
Tragic collapses, such as the Genoa Bridge in 20181 and the Nanfang’ao Bridge in 2019,2 are stark examples of the significant risks posed by aging infrastructure. Nonetheless, while considerable investments have been made in SHM since the 1970s, traditional methods relying on periodic visual inspections and corrective maintenance remain predominant.3 The widespread adoption of more effective SHM techniques is hindered by their highly specialized complexity and the lack of accessible tools for converting monitoring data into decision-making. This is evident in the limited availability of dedicated software solutions, with most existing platforms focusing on vibration testing and failing to offer a comprehensive framework for managing SHM systems. However, recent advances in AI may mark a turning point for SHM, potentially enabling its widespread adoption on a national scale.4
Effective management of long-term SHM systems requires integrating three key stages, treating the SHM process as a statistical pattern recognition problem:5 (i) feature extraction, (ii) data normalization, and (iii) damage identification. Feature extraction involves identifying damage-sensitive signatures from raw measurements. Vibration-based systems employing Operational Modal Analysis (OMA) are particularly popular owing to their ability to detect global damage and their minimal intrusiveness.6 However, due to their limited effectiveness in identifying local defects, it is often necessary to integrate other technologies with higher local detection capabilities, such as static sensors. Additionally, monitoring operational/environmental conditions (EOC) is often crucial, as they tend to induce daily/seasonal variations in the structural response.7 In this context, data normalization plays a critical role in eliminating variability in the selected features resulting from EOC variations.8 Finally, the normalized features can be used to tackle the damage identification problem through either data-driven (unsupervised) or model-driven (supervised) methods.9 Data-driven strategies rely solely on monitoring data to detect structural anomalies, while model-driven techniques involve inverse calibration of a machine learning or physics-based model of the monitored structure. Integrating all these steps into a comprehensive software platform presents significant challenges, particularly in managing vast monitoring databases from large-scale, densely instrumented structures within a Big Data framework, as well as providing the flexibility to fine-tune appropriate damage identification algorithms.10
In the feature extraction phase, OMA of large-scale, densely instrumented structures presents significant challenges from both software and hardware perspectives. Currently, available automated OMA algorithms, especially those based on Stochastic Subspace Identification (SSI),11 demand considerable hardware and computational resources, limiting or impeding on-site edge computing or requiring the transfer of large databases for offline processing. In this context, recent advancements in AI offer promising opportunities to enhance the computational efficiency of these algorithms, accelerating modal identification and reducing computational demands. Although still in the early stages of development, studies by Liu et al.12 and Shu et al.13 illustrate the potential of deep learning techniques for SSI and Blind Source Separation (BSS), achieving comparable accuracy to traditional OMA methods with significantly lower computational requirements. Similarly, recent work by some of the authors introduced an innovative Multitask Learning Deep Neural Network (MTL-DNN) model for fast second-order blind source identification (SOBI).14 Jian et al.15 also introduced a Graph Neural Network (GNN)-based deep learning scheme to identify modal properties of arbitrary truss structures (including simply-supported and cantilever configurations), achieving high computational efficiency and promising results in terms of accuracy. These encouraging results underscore the potential of AI-driven OMA techniques for rapid dynamic identification, paving the way for broader adoption of vibration-based SHM technologies for large-scale infrastructure systems.
A second key challenge in developing comprehensive SHM software is the integration of structural Digital Twins (DTs) for model-driven damage identification. In a broader sense, a DT is a physics-based or machine learning model that consistently utilizes monitoring data to infer and classify the health status of the physical asset, enabling timely interventions.16 Recent advances in surrogate models (SMs) or meta-models have greatly facilitated the development of DTs by replacing high-fidelity, computationally intensive numerical models (often finite element models [FEMs]) of large-scale structures.17 In this light, there has been a growing number of publications on the development of continuous deterministic18 and probabilistic19,20 DTs capable of providing fast damage identification by inverse calibration of local structural parameters, also referred to as model updating. In this context, recent research efforts, such as the works of Jayasinghe et al.21 and Huang et al.,22 showcase the potential of AI techniques for developing ultra-fast, physics-informed DTs with large potential for modeling complex non-linear responses with high accuracy. Nonetheless, despite these advancements, DT usage remains largely confined to academic environments and proprietary software scripts, with no existing commercial platforms currently offering a standardized environment for their development and implementation.
In view of the identified gap in the availability of widely implementable SHM software solutions, this work presents new updates to MOVA/MOSS, an in-house SHM software platform that integrates innovative AI solutions to address the scalability limitations of existing SHM methods. Originally developed by García-Macías and Ubertini,23 the latest update to the platform introduces new AI solutions in both the feature extraction and damage identification phases within the SHM process as a statistical pattern recognition problem. In the feature extraction phase, the platform supports the integration of any pre-trained AI model to extract modal properties (resonant frequencies, damping ratios, and mode shapes) from ambient acceleration records. As an example of this capability, the MTL-DNN model developed by Hernández-González et al.24 is incorporated. Furthermore, the platform facilitates the creation of structural DTs by generating a training population from any FEM developed in SAP2000, training any chosen AI model, and utilizing the resulting meta-model for continuous, supervised damage identification. Specifically, a novel feed-forward DNN model is introduced, capable of reproducing a set of resonant frequencies and mode shapes, demonstrating its integration potential within MOVA/MOSS. The remainder of this paper details the fundamentals of these modules and demonstrates their application on a real-world bridge: the Méndez-Núñez Bridge in Granada, Spain.
Modular Software Architecture for Long-Term SHM
MOVA and MOSS build on the research group’s decade-long experience in SHM of civil engineering structures. MOVA specializes in Ambient Vibration Testing (AVT), while MOSS is designed for the autonomous management of long-term integrated SHM systems. The primary purpose of MOSS is to enable fully autonomous oversight and management of permanent SHM systems, incorporating diverse sensing technologies, including dynamic, static, and environmental monitoring.
The typical workflow of an SHM system utilizing MOSS is outlined in Fig. 1, which materializes the paradigm of SHM as a statistical pattern recognition problem within a comprehensive software platform. The monitoring system consists of an integrated sensor network that continuously acquires monitoring data (both dynamic and non-dynamic) over a specified time duration, which are stored in separate computer files. After inputting the relevant directories (either locally or via cloud-based FTP protocols), MOSS automatically identifies and processes newly acquired records. Specifically, acceleration data are processed through a five-step sequence involving (i) signal pre-processing, (ii) automated OMA, (iii) modal tracking, (iv) elimination of environmental effects, and (v) control charts for online anomaly detection. Simultaneously, MOSS extracts features from static data using basic statistical metrics (e.g., mean, max/min, root mean square error), which serve as predictors (for removing EOCs) and/or damage-sensitive estimators, as defined by the user. Additional damage-sensitive features can be extracted by introducing a DT in MOSS, which extracts a time series of local fitting parameters by exploiting both static and/or dynamic data through inverse model calibration. Based on this, MOSS enables the definition of multiple control charts and performs simultaneous online damage detection of various damage-sensitive features. As mentioned above, the most computationally intensive operations in this workflow are the automated OMA and the assessment of the DT. To address this, two flexible AI platforms are introduced in MOSS, as described hereafter.
Figure 1. Schematic of a permanent SHM system managed by MOSS
The seamless integration of new modules into the general workflow of MOSS is enabled by its highly flexible modular architecture. Specifically, MOVA and MOSS are organized around two core standalone C++ graphical user interfaces (GUIs), originally developed using MATLAB (Fig. 2). These GUIs call independent GUIs or scripts corresponding to the previously indicated modules, passing information between them via standardized structure arrays. In this work, we focus on the inclusion of AI tools for automated OMA and the development of structural DTs for model-driven damage identification. For a detailed description of the remaining modules, interested readers can refer to García-Macías and Ubertini.23
Figure 2. Core GUIs of MOVA (a) and MOSS (b)
AI-Driven Operational Modal Analysis
The new module in MOVA/MOSS for implementing AI-driven algorithms is depicted in Fig. 3. The process begins by loading a standardized Python script (.py) to execute the prediction of a pre-trained Pytorch (.pt) or TensorFlow (.h5) neural network (NN) via a dedicated GUI (Fig. 3a). This Python script is designed to take as an input a standardized MATLAB file (.mat) containing the ambient vibration data previously processed by MOVA/MOSS along with the acquisition sampling frequency and outputs the dynamic identification parameters (i.e., frequencies, mode shapes, and damping) through a binary MATLAB MAT-file. The output file follows a standardized naming protocol for the dynamic identification variables, and it can include any additional variables of interest for the visualization of the identification results. For the latter, a MATLAB script (.m) can also be loaded, which contains a custom function designed to visualize the results generated by the NN in the main GUI for dynamic identification (Fig. 3b) using the additional variables of interest. Once loaded, MOVA/MOSS executes the AI algorithm by launching the Python interpreter through the Command Prompt in Windows, which is applicable to both MOVA for AVT and MOSS for continuous OMA. The architecture is designed with maximum flexibility, allowing for the incorporation of any AI-driven OMA model, as well as any general dynamic identification algorithm implemented in Python.
Figure 3. Module developed for the implementation of AI-driven OMA methods (a) and the dynamic identification GUI (b) in MOVA/MOSS
In this work, the MTL-DNN model presented in Hernández-González et al.24 is implemented for illustrative purposes. For completeness, the fundamentals of this model are concisely outlined herein. Assuming the structure behaves as a linear time-invariant system and that a number l of natural modes of vibration are excited, the system’s response x(t) monitored by m sensors deployed on the structure can be expanded using modal superposition as: where is the modal matrix composed by l mode shape vectors , and represents the modal displacement vector. To facilitate the extraction of complex-valued mode shapes, it is convenient to augment the measured data into its analytic form using the Hilbert transform (), as reported by McNeill and Zimmerman.25 The Hilbert transform introduces a phase shift of 90 degrees, that is , where and denotes the imaginary unit. The fundamental hypothesis of BSS states that can be expressed as a linear combination of a set of independent components as: where is the so-called mixing matrix, and represents a zero-mean temporally and spatially stationary white noise. Then, by comparing Eqs. (2) and (1), the dynamic identification problem reduces to estimating and as equivalent magnitudes for the modal displacements (which contain information about the resonant frequencies and damping ratios) and the modal matrix, respectively. The SOBI technique addresses this task by seeking the independent components that produce approximately diagonal time-shifted covariance matrices, relying on second-order statistics. To facilitate this, it is beneficial to first whiten the observed data using principal component analysis (PCA). This is achieved through eigenvalue decomposition of the covariance matrix of observations, that is , where and correspond to the diagonal eigenvalue and the orthogonal eigenvector matrices, respectively. Consequently, the whitened data is obtained as , with being referred to as the whitening matrix. Utilizing , the second step in the SOBI algorithm involves estimating an orthogonal matrix that approximately diagonalizes several time-shifted covariance matrices with time lags . The joint approximate diagonalization (JAD) algorithm proposed by Belouchrani et al.26 enables this to be expressed mathematically through an optimization problem as: where is the time-shifted covariance matrix of the whitened data, and the terms denote their autocovariances. Thus, the independent components and the mixing matrix can be obtained as: with denoting the de-mixing matrix. Given that both the mixture matrix and the independent components are complex-valued, one can write: where the and indexes correspond to the real and imaginary parts of the respective terms. Therefore, substituting Eq. (5) into (2), the system’s response x can be expressed as a linear combination of the real and imaginary parts of as:
Therefore, the system’s response can be estimated as a linear combination of the products of the real and the imaginary parts of the independent components multiplied by the corresponding real and complex parts of the modal matrix. This relationship can be represented using two fully connected layers with linear activation functions that model the system’s response and the independent components, where the weights correspond to the real and imaginary parts of the modal matrix. Following this intuition, the NN architecture depicted in Fig. 4 was originally proposed by Hernández-González et al.14 The NN takes the acceleration time series as input and outputs the reconstructed signal , generated after applying Eq. (6) with the estimated mixing matrix. Notably, the classical SOBI algorithm presented in McNeill and Zimmerman25 is primarily suited for the determined identification problem (i.e. ), meaning the number of identifiable modal properties is less than or equal to the number of sensors. To address the undetermined case, as it is common in practice, SOBI can be applied iteratively to the system’s response band-pass filtered within different frequency ranges of interest and used as input to the NN (see Fig. 4), allowing for the estimation of a maximum of independent components. Furthermore, to capture the sequential nature of the time series, a time lag is introduced between the observed data and the network’s outputs, namely and . After the input layer of neurons, a series of dense layers are employed to emulate both the whitening process and the JAD. The subsequent part of the network is split into two dense branches, J and K, of dense layers and number of neurons, except for the last layers of these two branches that contain neurons, equal to the number of independent components of interest. This division serves the primary purpose of separating the real and complex parts of the independent components into layers R and C, respectively. Finally, to improve generalization, the second and the second-to-last layers of the first set of dense layers, as well as the second-to-last layers of branches J and K, are regularization (dropout) layers.
Figure 4. Architecture of the MTL-DNN for multi-frequency band operational modal analysis
For the supervised training of the NN using the SOBI algorithm, both the reconstructed signals and the output of the branches and are monitored. Based on these outputs, the loss function for the MTL-DNN is defined as: where and represent the real and complex parts of the modal response obtained by SOBI, respectively; and denote the corresponding real and complex parts predicted by the MTL-DNN, respectively; and represent the input and reconstructed data, respectively. The subscript indicates that the corresponding terms are time-shifted to align with the time sequence of the network’s output sequence. The loss function is minimized using the Adam adaptive gradient descent backpropagation algorithm.
One trained, as mentioned earlier, the weights between the last two layers, representing the independent components and the reconstructed system’s response, directly yield the complex-valued mode shapes. Additionally, the outputs of both branches and can be used to extract the corresponding resonant frequencies and damping ratios. In this work, the single-degree-of-freedom (SDOF) Ibrahim Time-Domain (ITD) method, using the Natural Excitation Technique (NExT), is adopted for this purpose.
AI-Driven Surrogate Models for Continuous Supervised Damage Identification
The workflow for implementing a DT in MOSS includes two main steps: (i) initial calibration of a FEM of the structure, and (ii) creation of a SM. For the first step, a specific module in MOVA allows users to introduce and inverse-calibrate a SAP2000 model provided in the form of an Excel database (.xlsx) (Fig. 5a). In this module, users can introduce various fitting parameters through a simple configuration file created in a separate Excel spreadsheet. In this file, users can select a variety of fitting parameters such as material properties, stiffness of beams or shells, linear springs, added masses, etc., specifying their labels, nominal (reference uncalibrated) values, and maximum/minimum variation ranges. Afterward, the module enables users to conduct a sensitivity analysis of the model parameters (frequencies and mode shapes) by finite differences, offering the option to deselect low-sensitivity parameters or group them based on their sensitivities. This grouping is achieved through a hierarchical clustering approach using the Unweighed Pair Group Method with Arithmetic Mean (UPGMA) (see García-Macías and Ubertini19 for further details), which helps reduce the number of fitting parameters. In these analyses and in all subsequent evaluations of the SAP2000 model, the module directly executes the simulations via the SAP Open Application Programming Interface (OAPI).
Figure 5. Main window of the MOVA module for model updating of SAP2000 FEM models (a), screenshot of the model calibration process (b), and interface for model validation (c)
Afterward, the user can select from three different optimization algorithms: genetic algorithm (GA), particle swarm optimization (PSO), or gradient-based optimization. The user can specify their hyperparameters, including convergence tolerance, number of interactions, and number of genes/particles in case of GA or PSO (Fig. 5b). Moreover, the user can customize the objective function for inverse calibration. Let us consider fitting variables , used to parametrize the original FEM, constrained within the user-defined range . On this basis, the model parameters are estimated by solving the minimization problem , where represents the objective function that quantifies the discrepancy between experimental data and model predictions. If resonant frequencies and mode shapes have been identified in the AVT, the software defines a generic objective function as: where the terms and denote the i-th errors in terms of resonant frequencies and mode shapes, respectively, according to the user-defined settings. Specifically, for frequencies, users can choose between the mean/max squared error and mean/max relative error, while for mode shapes, they can select either the mean squared errors of the modal displacements or the mean/max value of . Here, denotes the modal assurance criterion between the mode shapes predicted by the model and the experimental mode shapes. The terms α, β, and η in Eq. (8) are weighting coefficients, while the last term, , serves as a regularization term to alleviate ill-conditioning in the optimization process. In MOSS, a modified version of the conventional Tikhonov regularization is implemented as: where denotes the reference value of the i-th parameter. Additionally, the software includes a mode matching algorithm to pair numerical and experimental mode shapes. The algorithm matches modes according to a metric involving the relative differences in frequency and values, respectively weighted by two factors and .
Once relevant hyperparameters are defined and the calibration is started, the module displays the convergence of the objective function, the mean relative errors in terms of resonant frequencies, and values (Fig. 5b). Upon completion, the user can access a validation interface (Fig. 5c), where the software presents a tabular comparison between the experimental and numerical modes in terms of relative errors in resonant frequencies and MAC values before and after the calibration, the auto-MAC matrix between the numerical and experimental mode shapes, and an interactive 3D graph that displays and animates the experimental and numerical mode shapes.
Once the FEM of the structure is calibrated, the user is ready for constructing the DT. In general, the FEMs of civil engineering structures are computationally intensive, which hinders their implementation for continuous SHM. Therefore, it often becomes imperative to develop an SM capable of bypassing the forward FEM with reduced computational demands. For this purpose, MOSS provides a specific module depicted in Fig. 6. This module first enables the generation of training and validation datasets for the construction of the SM. The user begins by defining the number of fitting parameters, their variation ranges, and the number of samples to construct both the training and the validation datasets. Assuming the user defines a population of N individuals for the training set, input samples are drawn uniformly over the design space , creating a matrix of design sites . To this aim, the user can choose between Latin Sampling Hypercube and Sobol quasi-random sequences. Fig. 6a shows a screenshot of the training samples generated for the case study presented hereafter in Section 5. Next, the corresponding outputs are obtained by direct Monte Carlo Simulations (MCS) using the forward FEM, producing an observation vector , with . If a SAP2000 model is provided as mentioned earlier, MOSS directly performs this operation. Alternatively, the software allows exporting the training/validation populations for externally running the FEM evaluations and inputting the observation vector through a MATLAB (.mat) file.
Figure 6. Module in MOSS for generating surrogate models (a) and sub-module for integrating AI-driven digital twins (b)
The resulting training population , also commonly referred to in the literature as Experimental Design (ED), is used to train the SM. MOSS offers a broad range of SM options, including second-order response surface models (RSM), Kriging, Polynomial Chaos Expansion (PCE), and general SMs defined through a custom MATLAB script (.m).
Additionally, the latest version of the software allows users to introduce pre-trained AI models to work as SMs. A dedicated interface, shown in Fig. 6b, facilitates this process. In this window, users can choose between a general Python script (.py) that calls a pre-trained NN, or a pre-trained NN in TensorFlow 2.17 (.pb), compatible with MATLAB. The former operates through standardized input/output MATLAB-MAT files and the Windows Command Prompt, similar to the approach previously introduced for AI-driven OMA. The latter option directly allows for transforming the pre-trained NN into a native MATLAB object, offering improved computational efficiency.
Once the SM is constructed, it is ready for continuous system identification. Users can utilize both static and dynamic experimental data to define the objective function. If only modal data are selected, users can customize the objective function following the same formulation introduced in Eq. (8). Alternatively, users can input a custom MATLAB script (.m) with a standardized structure, allowing for the definition of any function involving arbitrary combinations of static and/or dynamic data. Once set, the defined optimization problem is solved iteratively every time a new acquisition arrives in MOSS, and the fitting parameters are collected into an observation matrix . This matrix serves as a damage-sensitive estimator and can be processed like any other estimator (e.g., resonant frequencies), including EOC elimination and the construction of specific control charts. For a detailed explanation of all MOSS functionalities related to EOC elimination and control chart definitions, readers can refer to García-Macías and Ubertini.23 Unlike other features, such as modal signatures, persistent variations in any of the fitting parameters of the SM directly indicate the location and severity of the damage.
Finally, the GUI presents various quality assessment graphs for evaluating the accuracy of the SM, including FEM versus SM scatter plots, and error quality metrics such as coefficients of determination and mean squared errors.
Application Case Study: Méndez-Núñez Bridge
To illustrate the potential of the AI tools introduced in MOVA/MOSS for both automated OMA and DT definition, a real-world instrumented bridge is presented as a case study: the Méndez-Núñez Bridge in Granada, Spain.
Description of the Structure
The Méndez-Núñez Bridge is a continuous five-span, 122.5-meter-long post-tensioned concrete bridge with a variable section located in the Spanish city of Granada, in the Autonomous Community of Andalucía (Fig. 7). Built in March 1989 by the Dirección General de Carreteras (General Directorate of Highways) of the Province of Granada, the bridge currently supports traffic over Avenida de Andalucía between the municipalities of Jaén and Motril. The bridge deck is supported by four central reinforced concrete columns piers with sections measuring 3.40 m × 1.50 m, founded on piles and pile caps, and two abutments consisting of reinforced concrete retaining walls. All supports rest on elastomeric neoprene bearing pads with dimensions of 0.90 m × 0.80 m × 0.15 m. For visual reference, Fig. 7a shows two photographs of the current state of the structure.
Figure 7. Méndez-Núñez bridge: (a) general views of the bridge and monitoring system, (b) sensors layout (units in meters)
As part of a national R&D project, a permanent vibration-based SHM system was installed on September 27, 2023. The monitoring system comprises 10 uni-axial piezoelectric accelerometers model KB12VD (µ10% 10.0 V/g, broadband Resolution: 1 µg rms and ± 0.5 g pk) labeled as A1 to A10 as shown in Fig. 7b. Ambient vibration data are collected in separate 30-minute records at a sampling frequency of 100 Hz, recorded by a data acquisition system (DAQ) model cDAQ-9184, located on one of the piers. Additionally, temperature data are recorded by four probes model Pt 1000/3850 placed on the bridge deck, and humidity data through a hygrometer model AM2315 controlled using an independent Arduino Uno micro-controller, both with an acquisition frequency of 5 minutes. Note that the monitoring system generates approximately 19 GB of data per month, underscoring the management of this system as a Big Data problem. To handle this problem, automated OMA is performed using a standard on-site computer with MOVA/MOSS installed, and the identified modal properties are automatically transmitted via the Internet for data visualization and control. The raw time series are only stored locally and collected periodically. In this work, the monitoring records collected from September 27th, 2023, to October 18th, 2024, are analyzed to investigate the potential of the developed AI algorithms.
To generate a structural DT of the bridge, a 3D FEM of the bridge was developed using SAP2000, as shown in Fig. 8. The model’s geometry was constructed based on available structural drawings and in-situ inspections. Specifically, the deck of the bridge was modeled using frame elements with variable cross-sections and lumped torsional masses. Additionally, the inner piers were modeled using frame elements with variable-cross sections connected to the bridge deck through vertical and longitudinal springs with stiffness values extracted from the mechanical properties of the bearing pads and the Spanish standards. The initial material model was assumed isotropic with an elastic modulus of 35.0 GPa, Poisson’s ratio of 0.2, and a mass density of 2.35 t/m3. To extract the modal displacements at the points on the deck’s edges where the real sensors are located, infinitely rigid massless frame elements are also introduced in the model. Infinitely rigid massless elements are also included to simulate the actual eccentricities between the deck and the piers, abutments, and the foundations. Finally, the post-tensioning cables are modeled by means of internal forces following their actual parabolic profiles. Preliminary analyses considering P-delta effects confirmed that the prestressing forces play a negligible role in the modal properties of the bridge, as also discussed in the literature (see, e.g., Abdel‐Jaber and Glisic27). Therefore, their effect is disregarded in the subsequent inverse model calibration.
Figure 8. Méndez-Núñez Bridge FEM in in SAP2000
The dynamic properties of the Méndez-Núñez Bridge were identified using the covariance-driven SSI (CoV-SSI) method in MOVA. As hyperparameters, the covariance matrices were extracted considering a time lag of 3.2 s, and the list of stable poles were extracted for model orders ranging between 2 and 180, with tolerance criteria of 0.1% for resonant frequencies, 0.5% for damping ratios, and 0.005 for MAC values. The identification of physical poles was conducted through a hierarchical clustering approach, considering a cut-off distance of 0.01 (defined by the sum of the relative variations in frequency and 1-MAC values), and a minimum cluster size of 20 poles to consider a cluster as a physical mode. On this basis, a total of eleven clear global modes were identified in the frequency broadband up to 12 Hz, as depicted in the stabilization diagram in Fig. 9. Specifically, the identified modes include five global bending and six torsional modes, as reported hereafter in Fig. 10.
Figure 9. Stabilization diagram of the Méndez-Núñez Bridge extracted from CoV-SSI analysis of ambient accelerations recorded on November 30, 2023, at 10:00 AM
Figure 10. Comparison between experimental and numerical mode shapes after model updating
With these modal identification results, the FEM of the bridge was calibrated in MOVA, discretizing the bridge deck into eleven sections and considering their elastic moduli as fitting parameters (see Fig. 8). Assuming that the elastic moduli may range between 31.5 GPa and 45.5 GPa, the model was calibrated considering the previously reported eleven global modes using the PSO algorithm (60 particles and 65 generations). Maximum relative errors in terms of frequencies and the mean of 1-MAC values were included in the cost function ( = 1, ). The comparison between the experimental and numerical modal signatures before and after calibration is provided in Table 1, in terms of relative errors in frequencies and MAC values, and the corresponding mode shapes are depicted Fig. 10.
Uncalibrated | Calibrated | ||||||
---|---|---|---|---|---|---|---|
Mode Num. | [Hz] | [Hz] | Rel. Err. [%] | [Hz] | Rel. Err. [%] | ||
1 | 2.87 | 2.86 | −0.46 | 1.00 | 2.87 | −0.07 | 1.00 |
2 | 3.76 | 3.78 | 0.59 | 1.00 | 3.79 | 0.87 | 1.00 |
3 | 4.53 | 4.13 | −8.75 | 0.95 | 4.61 | 1.75 | 0.95 |
4 | 5.05 | 5.01 | 0.19 | 0.99 | 5.07 | 0.50 | 0.99 |
5 | 5.31 | 4.79 | −9.95 | 0.95 | 5.22 | −1.75 | 0.96 |
6 | 6.14 | 5.73 | −6.60 | 0.98 | 6.13 | −0.05 | 0.98 |
7 | 6.42 | 6.39 | −0.60 | 0.95 | 6.42 | −0.07 | 0.95 |
8 | 6.81 | 6.88 | 0.95 | 0.99 | 6.93 | 1.74 | 0.98 |
9 | 7.24 | 6.84 | −5.61 | 0.99 | 7.23 | −0.25 | 0.99 |
10 | 8.31 | 8.20 | −1.35 | 0.99 | 8.33 | 0.17 | 0.99 |
11 | 9.51 | 9.39 | −1.24 | 0.99 | 9.67 | 1.74 | 1.00 |
AI-Driven Modal Identification
Following the multi-frequency band SOBI approach previously overviewed in Section 3, two different frequency broadbands () were necessary, namely 1–6 Hz and 6–11 Hz, to identify 20 target modes using the 10 available accelerometers. Following the general architecture of the MTL-DNN previously introduced in Fig. 4, the specific architecture for this case study, furnished in Fig. 11a, has been implemented using the Python library PyTorch. To generate the training dataset, the independent modal components are extracted from the raw acceleration time series by standard SOBI applied to the two considered frequency broadbands independently.
Figure 11. MTL-DNN architecture for the dynamic identification of the Méndez-Núñez Bridge (a) and convergence analysis (b)
The raw acceleration responses (30-min-long record, 180,000 samples) and the estimated independent components are utilized as inputs and outputs to construct the training dataset, incorporating an input/output time lag of 10-time steps ( = 10 ms). The dataset is divided into training (80%), validation (10%), and test sub-sets (10%). The activation functions used for all the hidden layers are set to hyperbolic tangent (tanh), except for the last two layers, which use linear activation. The learning rate, the batch size, and the dropout rate are set to 0.001, 200, and 30%, respectively, and the weights of the network are estimated using the Adam back-propagation algorithm. The selection of these hyperparameters was conducted after parametric analyses, including different activation functions and optimization algorithms to update the network weights, until finding satisfactory results in terms of modal identification accuracy. For further details on the architecture and definition of the model hyperparameters, readers can refer to Hernández-González et al.14 The results of the training convergence up to 250 epochs are shown in Fig. 11b, indicating that optimal model convergence is approximately achieved by epoch 110. Notably, while the initial SOBI takes 9 minutes and 44 seconds, the source identification using the MTL-DNN during the prediction step only takes 4.44 seconds, representing a remarkable reduction in computational time.
The modal identification results of the first eleven modes and their comparison with the CoV-SSI results are reported in Table 2. The comparison reveals a close agreement between the estimates provided by CoV-SSI and the MTL-DNN model, with remarkably small relative differences in frequencies, the maximum being 1.97% and the average 0.75%. Additionally, the MAC values are all very close to 1.00, confirming an almost perfect correlation between the CoV-SSI mode shapes and those estimated by the MTL-DNN. However, the damping ratios estimated by the MTL-DNN model are slightly lower. Nevertheless, the damping ratios estimated by classical SOBI are very similar, as shown in Table 2, with an average relative difference with respect to the estimates by MTL-DNN of 3.24%. These differences are attributed to the large uncertainty in damping estimation, which is commonly observed in AVTs. To further assess the model’s performance, Fig. 12 illustrates the first six mode shapes estimated as the weights between the last two layers of the MTL-DNN. Note that the developed AI modal can effectively estimate complex-valued modes. This is particularly evident in the complexity plots of Modes 5 and 6, which exhibit Modal Phase Collinearity (MPC) values of 93.6%, and 93.2%, respectively.
Mode | Frequency [Hz] | Damping [%] | MAC | ||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
CoV-SSI | MTL-DNN | Rel.Diff. [%] | SOBI | Rel.Diff. [%] | CoV-SSI | MTL-DNN | Rel.Diff. [%] | SOBI | Rel.Diff. [%] | ||
1 | 2.87 | 2.88 | 0.14 | 2.86 | 0.13 | 1.19 | 0.92 | 22.57 | 0.93 | 21.69 | 1.00 |
2 | 3.76 | 3.75 | 0.09 | 3.76 | 0.07 | 1.19 | 1.21 | −2.12 | 1.22 | −2.34 | 1.00 |
3 | 4.52 | 4.47 | −0.62 | 4.53 | −0.61 | 1.86 | 1.08 | 42.07 | 1.04 | 44.22 | 1.00 |
4 | 5.05 | 5.05 | −0.30 | 5.03 | −0.23 | 1.29 | 1.12 | 13.14 | 1.10 | 14.88 | 1.00 |
5 | 5.31 | 5.37 | −0.56 | 5.36 | −0.58 | 2.50 | 1.30 | 48.07 | 1.34 | 46.14 | 1.00 |
6 | 6.13 | 6.10 | 0.36 | 6.12 | 0.21 | 1.28 | 1.08 | 15.44 | 1.06 | 16.95 | 1.00 |
7 | 6.42 | 6.37 | −0.25 | 6.38 | −0.42 | 2.12 | 2.03 | 4.49 | 1.45 | 31.73 | 1.00 |
8 | 6.81 | 6.74 | −0.49 | 6.85 | −0.32 | 2.50 | 1.94 | 22.16 | 2.05 | 17.85 | 1.00 |
9 | 7.24 | 7.22 | 0.07 | 7.24 | −0.24 | 1.43 | 1.63 | −13.92 | 1.31 | 8.72 | 1.00 |
10 | 8.30 | 8.15 | −0.09 | 8.32 | −0.09 | 1.39 | 1.05 | 24.18 | 1.07 | 22.84 | 1.00 |
11 | 9.51 | 9.56 | −0.97 | 9.53 | −0.88 | 1.57 | 1.09 | 30.29 | 1.05 | 32.84 | 1.00 |
Figure 12. Mode shapes of the Méndez-Núñez Bridge estimated by the developed MTL-DNN model
Finally, it is important to emphasize the critical need to assess the ability of the MTL-DNN model to maintain modal identification accuracy when damage affects the structural performance. In this case, since no real damage is observed, it would be necessary to generate synthetic damage data, possibly using the forward FEM illustrated in Fig. 8. These analyses are left for future work. Interested readers may refer to our previous work, Hernández-González et al.,14 where the ability of the proposed MTL-DNN model to generalize its modal identification capability beyond the observed healthy dataset was demonstrated on a real laboratory steel frame.
AI-Driven Digital Twin for Fast Structural Identification
To define the DT, a feedforward neural network (FNN) was designed in TensorFlow to replace the forward FEM of the bridge described in Section 5.1. Since the primary objective is its use for continuous system identification, the FEM is parameterized with fitting parameters, representing the elastic moduli of the sections obtained by discretizing the five spans into four equally long elements each. Specifically, 20 stiffness multipliers , , directly affecting the elastic moduli, are considered as fitting parameters. These multipliers serve as the input features of the FNN, while the output consists of the modal signatures (resonant frequencies and mode shapes).
The FNN architecture, shown in Fig. 13, is defined following a modular structure to simultaneously predict both the frequencies and mode shapes, while accounting for the unique characteristics of each mode shape. It begins with an input layer for the stiffness multipliers , followed by several fully connected (dense) layers with tangent hyperbolic activation functions to capture complex dependencies between the input parameters and the modal signatures. The network then branches into specialized paths: one for predicting the natural frequencies , where for this case study), and another set of independent blocks, each dedicated to predicting a specific mode shape. The output layers of the latter branches represent the modal displacements at the monitored degrees of freedom (DOFs) ( in this case study).
Figure 13. Architecture of the FNN for the generation of an AI-driven DT for the Méndez-Núñez Bridge
Assuming the stiffness multipliers can vary within the range [0.7, 1.1], the Latin Hypercube Sampling (LHS) approach implemented in MOVA was used to generate a training dataset of 2048, respectively, and the corresponding modal properties were extracted by Monte Carlo simulation of the forward FEM.
The neural network was trained for 1,000 epochs, with 15% of the 2048 training samples used for testing. A custom loss function was defined, accounting for the unweighted sum of the squared errors in frequencies and values. The hyperparameters of the model were selected after parametric analyses, controlling the convergence of the loss function in the training and testing datasets and the global accuracy in terms of relative errors in frequency and MAC values compared to the results obtained by the forward FEM. Based on these analyses, the Adam adaptative optimizer was used with an initial learning rate of 10−4 and a momentum value of 0.98. A batch size of 16 samples was used for training, with samples shuffled at each epoch. The convergence of the loss function is depicted in Fig. 14a, indicating rapid convergence beyond epoch 150, with strong performance both in the training and the testing sets. To further evaluate the FNN’s performance, an independent validation dataset of 1024 samples was used. For this dataset, Fig. 14b compares the predicted frequencies from the neural network with the FEM evaluations , while Fig. 2c presents the MAC values, , between the predicted modes and the FEM estimates . Notably, the Pearson coefficient of determination () for each mode’s frequencies exceeds 99%, and the MAC values exceed 0.999, demonstrating that the neural network accurately replicates the dynamic behavior of the FEM, effectively serving as an SM. It is important to emphasize the significant advantages in computational efficiency. While the FEM in SAP2000 requires approximately 3.5 seconds for a single linear modal analysis (4.5 seconds, including data post-processing and data transfer between MATLAB and SAP2000), the trained FNN completes the same task in just 0.01 seconds (a reduction of 99.71%).
Figure 14. Neural Network training and test results. (a) Training and validation loss over 1000 epochs. (b) Comparison between predicted frequencies and actual frequencies . (c) MAC between the estimated mode shapes by the DT, and the forward FEM,
Once trained, the AI-driven DT was used for continuous system identification of the Méndez-Núñez Bridge. For illustrative purposes, the modal properties extracted by CoV-SSI, using the same control parameters as those previously reported in Section 5.1, from September 27, 2023, to October 18, 2024, were used for inverse model calibration. The first eleven modes were consistently identified during this period, with identification success ratios above 95%, except for the fifth mode, which had a lower success ratio of 57%. On this basis, the inverse model calibration followed the formulation introduced earlier in Section 4, considering the resonant frequencies and mode shapes extracted from every 30-min ambient acceleration record (921 acquisitions in total). An autoregressive model using MATLAB’s fillgaps function and a previous interpolation model were applied to fill in the missing frequencies and mode shapes, respectively. For the cost function, the mean relative errors in frequency and values were considered, with weighting coefficients The resulting optimization problems were solved using PSO (40 particles and 90 iterations). Notably, despite the adopted large number of particles and iterations in the inverse calibration to ensure convergence to the global minimum, each system identification took only about 8 seconds, thanks to the high computational efficiency of the AI-driven DT.
The time series of stiffness multipliers identified from the AI-driven DT for the Méndez-Núñez Bridge, along with the mean environmental temperature, experimentally identified resonant frequencies, and predictions from the inverse-calibrated DT, are shown in Fig. 15 for the monitoring period from September 27, 2023, to October 18, 2024 (921 identifications). It is interesting to note that the daily oscillations observed in the time series of resonant frequencies translate into fluctuations in the stiffness multipliers. Specifically, similar to the resonant frequencies, all the stiffness multipliers exhibit negative correlations with environmental temperature (i.e., decreasing temperatures tend to make the structure respond in a stiffer manner), a behavior commonly observed in reinforced concrete bridges. Note that the DT can replicate the daily oscillations in the resonant frequencies with considerable accuracy, with error levels consistent with those reported in the initial FEM calibration in Table 1. Nevertheless, note in Fig. 15 that some parameters reach the upper definition limit (1.10) for some identifications despite the inclusion of the regularization term (). This may indicate ill-conditioning issues or suggest that the upper definition limit is insufficient to capture the full environmental variability of the modal signatures. Nevertheless, it is important to emphasize that these results are presented with the sole purpose of illustrating the potential of the AI-driven DT for fast system identification. When applied to damage identification, it may be preferable to minimize EOC-related variability in the modal properties prior to the system identification, for which a variation range of appears reasonable.
Figure 15. Continuous system identification of the Méndez-Núñez Bridge using an AI-driven digital twin (DT) from September 27th, 2023, to October 18th, 2024 (921 identifications)
To shed some light on the influence of EOC on the stiffness properties of the case study, Fig. 16 delves into the correlations between environmental temperature and the stiffness multipliers . Specifically, this figure presents the coefficients of variation (CVs) of the fitting parameters, along with the values between the fitting parameters and the environmental temperature. Notably, both the CVs and values exhibit certain patterns, with maximum values occurring in the elements closest to the abutments and piers 1 and 4. This suggests the influence of temperature on the joints at the abutments and possibly the bearings on the piers. Moreover, while the CV values exhibit considerable symmetry, as expected from a linear sensitivity analysis, the values are notably higher in the first pier. This asymmetry may point to uneven solar radiation, potentially due to the proximity of a twin bridge, which causes the south edge of the bridge to be more exposed than the north. Although further investigation is beyond the scope of this study, future work could consider including the stiffness properties of the supports as fitting parameters to better understand the temperature’s impact on the bridge’s structural behavior. Furthermore, the generation of synthetic damage data, possibly using the forward FEM in Fig. 8 or an independent, more realistic FEM, will be conducted to assess the ability of the developed AI-driven DT for damage localization and quantification.
Figure 16. Correlation analysis between stiffness multipliers ki and mean environmental temperature
Conclusions
This work has presented a comprehensive software platform called MOVA/MOSS, implemented in a modular architecture that integrates AI models with high flexibility. Specifically, AI is employed to alleviate the computational burden in the most computationally intensive stages of the SHM process as a statistical pattern recognition problem. This includes feature extraction using automated OMA and the generation of structural DTs for continuous supervised damage identification.
The potential of the developed AI functionalities has been illustrated through a real-world instrumented bridge: the Méndez-Núñez Bridge in Granada, Spain. In the feature extraction phase, a deep multi-task learning neural network embodying the principles of multi-frequency band SOBI has been implemented, demonstrating high accuracy in modal estimates with almost negligible computational burden. Additionally, a structural DT defined through a deep feedforward NN has been designed and implemented to bypass a computationally intensive FEM of the bridge. The developed DT has been used to conduct continuous system identification, as well as to interpret the influence of environmental temperature on the intrinsic local stiffness of the bridge. The developed AI tools achieve computational time reductions of 98.90% and 99.71% in the modal identification (from 9 minutes and 44 seconds using CoV-SSI to 4.44 seconds using the proposed MTL-DNN model) and the FEM evaluation (from 3.53 seconds using the forward FEM to 0.01 seconds using the proposed FNN model), respectively. Notably, the latter enables the inverse calibration of twenty fitting parameters in just 8 seconds. Overall, the presented results evidence the potential of AI to revolutionize SHM, offering vast potential for the widespread adoption of SHM techniques at regional or national scales.
References
Causes of the collapse of the Polcevera Viaduct in Genoa, Italy. Appl Sci. 2021;11(17). doi:10.3390/app11178098
Causes and statistical characteristics of bridge failures: a review. J Traffic Transp Eng (English Edition). 2022;9(3):388-406. doi:10.1016/j.jtte.2021.12.003
Structural health monitoring: closing the gap between research and industrial deployment. Struct Health Monitor. 2018;17(5):1225-1244. doi:10.1177/1475921717750047
Artificial intelligence in structural health management of existing bridges. Autom Construct. 2024;167(4). doi:10.1016/j.autcon.2024.105719
Structural Health Monitoring: A Machine Learning Perspective. John Wiley & Sons; 2012. doi:10.1002/9781118443118
Operational modal analysis on bridges: a comprehensive review. Infrastructures. 2023;8(12). doi:10.3390/infrastructures8120172
Effects of environmental and operational conditions on structural health monitoring and non-destructive testing: a systematic review. Buildings. 2023;13(4). doi:10.3390/buildings13040918
Eliminating environmental and operational effects on structural modal frequency: a comprehensive review. Struct Control Health Monitor. 2022;29(11). doi:10.1002/stc.3073
Towards vibration-based damage detection of civil engineering structures: overview, challenges, and future prospects. Int J Mech Mater Design. 2024;20(3):591-662. doi:10.1007/s10999-023-09692-3
Structural Health Monitoring Based on Data Science Techniques. Springer International Publishing; 2021. doi:10.1007/978-3-030-81716-9_12
Review and comparison of methods and benchmarks for automatic modal identification based on stabilization diagram. J Traffic Transp Eng. 2024;11(2):209-224. doi:10.1016/j.jtte.2023.05.007
Machine learning-based stochastic subspace identification method for structural modal parameters. Eng Struct. 2023;274. doi:10.1016/j.engstruct.2022.115178
A multi-task learning-based automatic blind identification procedure for operational modal analysis. Mech Syst Signal Process. 2023;187(1). doi:10.1016/j.ymssp.2022.109959
AI-driven blind source separation for fast operational modal analysis of structures. Mech Syst Signal Process. 2024;211(5). doi:10.1016/j.ymssp.2024.111267
Using graph neural networks and frequency domain data for automated operational modal analysis of populations of structures. arXiv preprint. Published online 2024. doi:10.48550/arXiv.2407.06492
A digital twin framework for civil engineering structures. Comput Methods Appl Mech Eng. 2024;418(5). doi:10.1016/j.cma.2023.116584
Recent advances and applications of surrogate models for finite element method computations: a review. Soft Comput. 2022;26(24):13709-13733. doi:10.1007/s00500-022-07362-8
From continuous vibration monitoring to FEM-based damage assessment: application on a stone-masonry tower. Construct Build Mater. 2017;156(5):252-265. doi:10.1016/j.conbuildmat.2017.08.160
Real-time Bayesian damage identification enabled by sparse PCE-Kriging meta-modelling for continuous SHM of large-scale civil engineering structures. J Build Eng. 2022;59(2). doi:10.1016/j.jobe.2022.105004
Enhancing structural damage evaluation of PC girder bridges through digital twinning Bayesian model updating. Eng Struct. 2024;321(3). doi:10.1016/j.engstruct.2024.118974
Innovative digital twin with artificial neural networks for real-time monitoring of structural response: a port structure case study. Ocean Eng. 2024;312. doi:10.1016/j.oceaneng.2024.119187
Damage identification of truss bridges based on feature transferable digital twins. Measurement. 2024;233. doi:10.1016/j.measurement.2024.114735
MOVA/MOSS: two integrated software solutions for comprehensive structural health monitoring of structures. Mech Syst Signal Process. 2020;143. doi:10.1016/j.ymssp.2020.106830
AI-driven automated operational modal analysis of bridges. Procedia Struct Integr. 2024;62(8):879-886. doi:10.1016/j.prostr.2024.09.118
A framework for blind modal identification using joint approximate diagonalization. Mech Syst Signal Process. 2008;22(7):1526-1548. doi:10.1016/j.ymssp.2008.01.010
A blind source separation technique using second-order statistics. IEEE Trans Signal Process. 1997;45(2):434-444. doi:10.1109/78.554307
Monitoring of prestressing forces in prestressed concrete structures—an overview. Struct Control Health Monitor. 2019;26(8). doi:10.1002/stc.2374
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Enrique Garcia-Macias, Elisa Tomassini, Israel Alejandro Hernández-González, Ubertini Filippo

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
© Authors under CC Attribution-NonCommercial-NoDerivatives 4.0.