Deep Learning–Based Time-Series Classification for Robotic Inspection of Pipe Condition Using Non-Contact Ultrasonic Testing

As energy demand expands due to population increases and electronic systems advancements, it is becoming more necessary to propel the creation, operation, and maintenance of energy systems into the next generation. Power plants provide a large amount of energy for residential and commercial communities, and components within these plants are subjected to extreme conditions. One of the most important components within these energy production systems is boiler tubes. Over time the tubular structures will degrade and show structural damage like cracks and corrosion [1]. This can lead to significant thinning of the tube causing inadequate strength to contain internal pressure, resulting in tubular failure [2]. Additional issues associated with tube failure are prolonged periods of oxidization, aging, and mechanical stress [3–5]. Failures of tubular components can result in power outages and costly repairs, therefore; researchers have presented a strong desire to avoid these sorts of complications through advancement in structural inspection processes.

Crawling and climbing mechanisms open possibilities for structure evaluation in situations where human access is not an option or is undesired [6]. To provide accurate and reliable inspection of structural tubes in unmanned areas, onboard inspection sensors and techniques are vital for collecting structural health data. Ultrasonic testing (UT) is a highly promising nondestructive testing/evaluation (NDT/E) method employed for routine inspection and Structural Health Monitoring (SHM) of infrastructures. Its effectiveness in harsh environments makes it a particularly auspicious approach for routine inspection applications. For tubular inspection, two locomotion methods are common in practice, in-pipe and out-pipe (external) [7–10]. Popular in-pipe robotic locomotion methods tend to have insect inspiration from terrestrial crawlers. Some techniques that have been successfully developed include a spider-inspired in-pipe mechanism by Chattopadhyay et al. and Neubauer [3,11], a snake-inspired robot by Xiao et al. [12], and an inchworm-inspired robot by Bekheit et al. and Qiao et al. [13,14]. An impediment facing in-pipe inspection is that the system cannot be operational while the inspection is being performed.

External locomotion is a more promising method for this application since it typically uses a prehensile design to transverse tubes. Mobility of out-pipe robots can be a challenging problem to overcome due to nonlinear tubular designs and obstacles, payload limitations, and gripper strength or surface adhesion of the robot's contact with tubes [1,6,15,16]. In reference to studies by Nemati et al. and Hills et al. [1,16], the authors presented a bio-inspired out-pipe technique that employed modular grippers and incorporated an NDE ultrasound sensing payload capable of traversing both vertical and horizontal exteriors of pipelines. The study demonstrated the successful generation of Lamb waves with satisfactory signal-to-noise ratio (SNR) levels for external tubular inspection by integrating electromagnetic acoustic transducers (EMATs) into the robotic system. However, the system's effectiveness in defect detection using traditional ultrasonic testing approaches was limited.

Considering these findings, the objective of this paper is to make a significant contribution by developing an advanced time-series classification (TSC) method that utilizes the collected ultrasound data to effectively detect anomalies in pipelines. Throughout this investigation, we have determined that artificial intelligence (AI) strategically enhances anomaly detection based on regionality. By introducing a region-focused system, AI empowers the identification of anomalies within specific pipe sections, achieving quicker predictions by strategically balancing accuracy. This innovative approach not only expedites the detection process but also ensures a thorough analysis of localized areas, underscoring the efficiency and precision of pipeline inspection. Finally, to ensure the robustness and generalizability of our proposed anomaly detection framework, we have conducted an in-depth exploration of model diversity and accuracy. We extended our analysis to encompass additional examination (training of 60%, 80%, and 100% of the training dataset) of the four distinct models by meticulously designing them to capture intricate patterns within the ultrasound data with simulated Gaussian noise.

We define the TSC task as a problem of building a classifier from a collection of temporal (time-series) and labeled training data. For this experiment, the univariate time-series task is an arrangement of ordered observations T = [t₁, t₂, …, t_n], where n is a set of time-series and T_i has m real-valued ordered observations T_i = 〈t_i,1, t_i,2, t_i,m〉, class label of c_i, in this case, binary (0,1). All classification techniques aim to establish a function that maps the space of time-series input to possible class labels [17].

TSC is a challenging task within the data mining community [18,19]. Over the last decade, the availability of temporal data has significantly increased leading to the advancement of TSC algorithms [20,21]. In general, many researchers agree that the benchmark method of TSC is the simple nearest neighbor classification and is considered difficult to beat for most applications [22]. Popular applications for TSC include Speech recognition, medical signal evaluation, and gesture recognition/classification [23]. When properly utilized, TSC algorithms hold great potential as a valuable approach for NDT/E, specifically in the context of ultrasonic testing methods. By leveraging TSC algorithms effectively, it is possible to enhance the accuracy and reliability of ultrasonic testing, enabling more effective detection and characterization of flaws or anomalies in inspected materials. The following subsections provide a summary of common TSC techniques. It is important to note that there may be overlapping and similarities among these techniques, making categorization a challenging task.

Distance-based TSC is a technique where a (dis)similarity measure amongst time series is defined, and the distances are used within a distance-based classification method. Popular techniques that have presented successful outcomes on TSC include k-nearest neighbor (k-NN), Support Vector Machines (SVM), distance features (global, local, etc.), and distance kernels [24]. Many of these techniques are quite sensitive to the input data (noise), as Bagnall et al. point out in Ref. [20], a difference of six–eight decimal places in the data can result in a 6% change in the classification performance. The 1-NN classifier has been utilized by most of the k-NN classifiers mainly due to how simple it is to utilize for TSC coupled with quality performance [20,24]. The distance is input into the 1-NN and the classifier predicts classes within the time series as a class of objects closest to it from the training dataset [24]. Distance feature methods vary from 1-NN in that time-series data are transformed into feature vectors 1. For global distance methods, the focus is on converting the time series into feature vectors by using vectors of distances to other series as the new representation and local distance methods utilize the distance amongst local patterns of the series as features [24].

The overall objective is to find the best possible shapelet, typically using information gain (IG) [25].

Ensemble methods are focused on transformation and collections of (ensembles) classification techniques as constituent classifier parts [26]. Separating the transformation from the classification via an ensemble approach is very important [27]. In 2015, Bagnall et al. introduced an ensemble method called Collective of Transformation-based Ensembles (COTE) consisting of eight heterogeneous ensemble methods (Random Forest (RF), rotation forest, and more) in the frequency, change, and shapelet transformation domains for TSC [26]. Lines et al. reported an evolution of the COTE algorithm, one with an additional three classifiers called the Hierarchical Vote Collective of Transformation-based Ensembles (HIVE-COTE) [17]. In Ref. [28], the Contract Random Interval Spectral Ensemble (c-RISE) is proposed for TSC that enables a check-point style progression and adaptively estimates the time for building trees during the ensemble. The contract classifier allows a user to specify the amount of computation time to build a model. The adaptive and dynamic nature of the algorithm has shown that even with faster training times, accuracy remains the same as compared to a non-adaptive technique reaching and exceeding 75% accuracy [28].

Dictionary approaches form counts based on the frequency of a subseries to determine patterns and build classifiers based on histograms of the results [29,30]. When data become highly dimensional and noisy, many data mining algorithms will degrade in performance [31]. The Bag-of-SFA-Symbols (BOSS) model is a structure-based similarity measure that attempts to reduce statistical noise to raw time-series data [30]. This method combines noise tolerance from Symbolic Fourier Approximations (SFA) and structure-based representations from the bag-of-words model [32]. Deng et al. proposed a time-series forest (TSF) for TSC in Ref. [33]. The algorithm randomly samples different features at every node and experiences computational complexity linearly to the length of the time-series dataset. The method uses mean, standard deviation, and slope to computationally be efficient. To do this, the authors describe a new measure called the Entrance (entropy and distance) gain that identifies high-quality splits. In addition, a temporal importance curve is utilized to record temporal features that are useful for TSC [33]. They demonstrated that the TSF has excellent error rates and average rank (2.48) was superior to that of competitive algorithms, at the time.

As it pertains to TSC, deep learning (DL) algorithms have demonstrated a potential proclivity toward the successful classification of temporal data. The recurrent neural network (RNN) is an advanced neural network tailored for prediction on sequential datasets [34]. One of the first investigations of RNNs was presented by Rumelhart et al. [35], where authors detail the learning procedure of RNNs including backpropagation where new features are distinguished by adjusting weights of the internal units. One popular DL technique used for both regression and classification with time-series data is Long-Short Term Memory (LSTM). This network is a specific type of RNN architecture developed to eliminate the vanishing gradient problem [36,37]. As a result, the LSTM does not experience long-term dependency issues that other RNN architectures have. The LSTM has showcased the ability to learn distant dependencies as compared to simple RNNs [38]. This network has garnered great interest from research spanning a wide variety of disciplines; Pham highlights the LSTM capabilities for TSC as it pertains to long sequential physiological signals [8]. Fawaz et al. introduced InceptionTime in 2020, an ensemble of five deep Convolutional Neural Network (CNN) models [39]. InceptionTime is created by cascading multiple Inception modules based on the work of Szegedy et al. in 2015 [40].

Wang et al. proposed a Fully Connected Network (FCN) for univariate TSC, consisting mainly of convolutional networks that do not alter the time-series data (no local pooling layers) [41]. The main factor in the algorithm’s success is the implementation of a final fully connected layer with global average pooling. By doing this, the network reduces the total number of parameters significantly. Karim et al. show promising capabilities of combining both the LSTM and FCN networks to create the LSTM–FCN [42]. The authors demonstrate that the LSTM–FCN achieves state-of-the-art performance as compared to current techniques via the Wilcoxon Signal Rank test, a statistical measure used to compare the median rank of the proposed LSTM–FCN against other networks.

Fauvel et al. present an eXplainable Convolutional neural network (XCM), for multivariate time-series (MTS) classification, an end-to-end explainable convolutional neural network for MTS classification. The algorithm supports predictions through reliable explanations [43]. The authors illustrate the performance of the algorithm by training/testing on a synthetic dataset. XCM uses a precise identification of regions from the time-series data that are significant and impactful during prediction. The XCM algorithm is shown to outperform most current state-of-the-art algorithms (up to 2021) on real-world data and includes explainability by providing informative explanations [43]. The authors show the versatility of the algorithm by testing it on small and large datasets, DL networks are notorious for poor performance on small datasets. During testing, the XCM competed with several other state-of-the-art TSC algorithms on 30 datasets and showcased the highest accuracy, therefore, winning 16 of the competitions (datasets), and the next highest was 7 [43].

In this study, the DL-TSC approach will be regarded as the primary methodology for defect detection in robotic inspection. The utilization of DL combined with TSC techniques will be the central focus of the research to develop an effective and reliable system for detecting defects in the inspected components.

Structural evaluation of pipes is at the center of this investigation. This section aims to present the methodology employed, encompassing aspects such as the robotic system, data collection methods, and the utilization of deep learning algorithms. However, before delving into the comprehensive explanations regarding these aspects, a concise overview of the experimental setup has been provided.

Using EMATs is a promising UT technique for the nondestructive evaluation of electrically conductive materials. EMATs offer advantages such as non-contact operation, suitability for rough surfaces and high-temperature environments, and generation of omnidirectional waves without the need for couplants [24]. Consequently, electromagnetic acoustic transducers (EMAT) enables the inspection of materials in scenarios where conventional UT techniques may not be practical or effective. This technology is adept at detecting flaws like cracks, corrosion, and voids, making it a versatile and valuable tool for nondestructive testing applications. In EMAT, various wave modes can be generated depending on the specific design and positioning of coils and magnets [44]. Our robotic system utilizes integrated EMATs to generate and receive Lamb waves. Due to their inherent multimodal and dispersive nature, it is important to plot the theoretical dispersion curves of Lamb waves to identify an appropriate mode. In this study, stainless-steel material (316 L) with a Young modulus of 200 GPa, Poisson’s ratio of 0.25, a density of 8000 kg/m³, and a thickness of 3.048 mm was used to generate dispersion curves [1]. Using a spiral coil, omnidirectional Lamb waves propagate in helical paths and cover the entire cross section of the tube between the two grippers. Figure 1 displays the phase velocity and group velocity dispersion curves related to Lamb waves propagating in a stainless-steel pipe with a thickness of 3.048 mm.

In an EMAT, achieving a matching condition between the wavelength of the Lamb wave and the coil configuration is typically essential. For a specific spiral coil, the approximate optimal wavelength (λ) generated by the coil is determined by the difference in the outer and inner diameters of the coil, which were 12.7 mm and 0 mm, respectively, in this study. As per the fundamental relation λ = c/f, the wavelength or period produced by the spiral coil is determined by the frequency and phase velocity. In Fig. 1(a), the sloped line denotes the activation line positioned at 12.7 mm, and the points where this line intersects with the dispersion curves indicate the preferential excitation points. When choosing excitation modes for Lamb wave testing, various approaches are at one's disposal. Typically, it is advisable to primarily employ the fundamental modes (A₀ and S₀). The simplest technique involves limiting the frequency to a level below the cut-off frequencies of higher modes, which here falls within the range of 1–650 kHz. In this case, the A₀ mode generated by the spiral coil demonstrates significant dispersion, leading to the selection of the S₀ mode for the inspection. In Fig. 1, the yellow dot represents the activation zone, which was achieved by applying a tone burst excitation with a frequency of approximately 400 kHz to the transducer. Notably, no amplifier was utilized during the experiment. We refer the reader to Ref. [1] for more details of the experimental setup. The following section is dedicated to describing the details of the robotic system presented in Refs. [1,16], the data collection methods and materials, and the DL algorithms used for TSC. Figure 2 is a detailed depiction of the three main phases of this investigation. In the first phase, we detail the robotic system, materials used, and methods for constructing the physical components of the lizard-inspired tube inspector (LTI) robot [16]. The second phase is dedicated to how the data were collected and the process of creating the time-series data. The last phase details the four DL algorithms used for TSC.

The subsequent sections delve into the process of data collection using integrated robotic systems and EMAT sensors. Additionally, the advantages of employing a DL-based TSC approach for detecting defects in pipeline systems are explored and discussed in detail.

A bio-inspired robot was designed to operate based on friction-based mobility like a lizard. The LTI robot can climb up a tubular structure and navigate along its outer surfaces [1,16].

This system is comprised of two modular grippers that grasp the tubular exterior surfaces and linear actuators that connect the two grippers and allow for linear motion [1,16]. Each gripper has four fingers with attached frictional pads made of polydimethylsiloxane (PDMS). The pads have curved grooves that mimic the frictional properties of the lizard's fingers. With the help of these gripper pads, the robotic system can readily attach to, hold on to, and detach from the tubular surfaces during the testing process. An EMAT sensor is designed to generate Lamb waves as described in Ref. [1]. As mentioned previously, the EMAT sensor comprises a spiral coil and a cylindrical magnet with dimensions of 12.7 mm in diameter and 28.5 mm in height. A transducer is attached to each finger, which makes direct contact with less than 2 mm lift-off from the tube surface, thereby enabling structural defect detection. Lamb waves are transmitted and received through the EMAT sensors to generate helical waves around the pipe and eliminate the necessity for point-by-point cross-section inspection of any tubular components [1,16]. The study assumes the pipeline structure to be akin to a wrapped plate, enabling the generation of Lamb waves within the pipe. This assumption facilitates the analysis of Lamb wave propagation and allows for the exploration of suitable techniques for detecting and assessing defects in the pipeline structure. The EMAT sensor and the fully assembled LTI robot are shown in Figs. 3(a) and 3(b).

Due to the modular design of the gripper, it can be seamlessly integrated with various types of robotic arms. In this experiment, the grippers were attached to two Universal Robots (URs) robotic arms, positioned at a fixed gripper distance of 110 mm from each other. To accommodate the gripper's weight, the UR5 robotic arms were selected, each supporting up to a 5-kg payload with a maximum reach of 1680 mm and six-degrees-of-freedom. The testing procedure involved implementing a synchronized pick-and-place algorithm, allowing the two UR5 robots to move simultaneously along the tube. This movement created direct contact points for the EMAT sensors to perform testing, as depicted in Fig. 4.

The tubes used in the experiment have diameters of 73.025, 88.9, and 114.3 mm and are made up of 316 L steel with a thickness of 3.048 mm. The tubes are shown in Fig. 2. Tests were conducted using the previously described EMAT sensors and a pitch–catch configuration. In total, there were eight sensors with one sensor transmitting Lamb waves while the other seven sensors received the signals. A schematic view of the location of the sensors is shown in Fig. 4(b).

Utilizing the designed gripper and configuration shown in Fig. 4(b), 56 raw time-series data were collected and stored at each position. As can be observed in Fig. 5, several helical waves with large SNR could be generated and received after the grippers become stable at their positions. For generating and receiving Lamb waves through the designed EMAT, a data acquisition system was employed. Throughout all measurements, the EMAT was driven by a five-cycle toneburst with a central frequency of 400 kHz. The waves were excited utilizing the computed burst frequency, and a receiver gain of 60 dB was applied. Figure 5 depicts how the data were collected from the (a) intact pipe, (b) pipe with corrosion, and (c) pipe with a crack along with the corresponding signals that were generated (sample for pipe type).

To simulate pitting corrosion, the surface of the pipe was subjected to grinding, resulting in a reduction of thickness by 30% within a specific area of 2.54 cm in diameter. To simulate a crack, a 2.54-cm notch was intentionally cut into the pipe, accompanied by a 30% reduction in pipe thickness. These intentional reductions in thickness mimic the effects of pitting corrosion and cracks and allow for the evaluation of detection and characterization techniques for such corrosion and crack defects.

The data are preprocessed and labeled according to the pipe classification. Two classifications are used for labeling during this study, crack, and corrosion are labeled with 0, and intact (no defect) data are labeled with 1. The univariate (amplitude) preprocessed dataset with labels has classes that are known in advance making it a supervised problem. The algorithms are trained on 75% of the full dataset and reshaped into an array of size (m, n, 1) using the NumPy library [45] before training with the main objective of learning what distinctive features separate classes from each other. Where m is the number of samples (168) and n is the length of each sample (3002), 25% of the data is withheld from training and used for testing.

The memory cell in an LSTM network has intermediate storage in the form of gates within each hidden layer. Figure 6 depicts an LSTM architecture that contains four parts within the memory block: an input gate (i), a forget gate (f), an output gate (o), and cell state (C_t−1). The forget gate is responsible for deciding which information is retained or discarded from the cell state. The input gate determines which values will be updated to a vector of new candidate values (C_t). Finally, the output gate decides what information will be passed to the cell state in the next time-step.

Details for model implementation can be seen in Fig. 8. Here, we showcase how time-series data are input into the network through the forward pass with the backpropagation pass responsible for adjusting the weights and bias vectors. We established a maximum of 100 epochs, meaning the network follows the forward and back propagation process 100 times. We utilize the popular “sparse categorical cross entropy” technique to calculate the loss on passes and the ADAM optimization which implements the AdaGrad and RMSProp algorithms creating an easy-to-implement algorithm capable of dealing with sparse gradients.

Another state-of-the-art model that we test in this investigation is the Long-Short Term Memory Fully Convolutional Network (LSTM–FCN), which is an FCN augmented with an LSTM for TSC task [42]. The model architecture is shown in Fig. 9, where the FCN is augmented by an LSTM block followed by dropout. Just like the FCN, the fully convolutional block consists of three stacked temporal convolutional blocks with filter sizes of 128, 256, and 128, respectively. This portion of the network is identical to the network proposed in Ref. [41]. Each block consists of a temporal convolutional layer followed by batch normalization and is activated via the ReLU activation function. The output of each layer utilizes Eq. (3). The final convolution block contains global average pooling. In parallel, raw time-series data that are input are fed into a dimension shuffle layer which transform the time series and pass the data to the LSTM block. At each time-step, the LSTM block uses Eqs. (2a)–(2f) to update the state condition. This is followed by an output layer of global pooling, and the LSTM block concatenates the feature maps to a softmax layer for classification.

Inception Time is composed of two different residual blocks, and each block contains several Inception modules shown in Fig. 10, as opposed to traditional fully connected layers [39].

The confusion matrix C_(i,j) is defined in Table 1. Here, C_(i,j) represents the ith actual observations and jth predictions. True Positive (TP) occurs when the sample is positive and predicted to be positive. True Negative (TN) takes place when a negative case occurs and is predicted to be negative. An occurrence of False Positive (FP) errors happens with an actual negative case predicted positive (Type I error) and False Negative (FN) error occurs when an actual case is positive and the prediction is negative (Type II error). The performance measure used in this investigation is accuracy which divides the number of correct predictions by the total number of predictions made, defined as

Figure 13 highlights the average signal for the two classes with Class 1 (P1, no defect) and Class 2 (P2 and P3, defect). The visual comparison of the two classes shows similar trends which may be problematic for model training. The classes do diverge around the 2000 index mark, suggesting that as time continues the signals differentiate from each other.

Figures 14–16 dive deeper into the data analysis by visualizing the distribution of data points based on each different class. This tactic is useful for determining if any patterns can be recognized as an initial step in determining if defects are present. The left column of Fig. 14 highlights the violin plots for the distribution of data in line with the transmitting coil whereas the right column shows results from the signal as it crosses the location of the defect. The data appear similarly distributed as transmitted by coil 1 in an inline fashion, received by coil 5 (Fig. 14(b)) and coil 6 (Fig. 14(c)) when compared to a cross-plane fashion in Fig. 15. Here, the time-series data distribution is shown for signal transmitting from coil number 1 and received by coil 8 (Fig. 15(b)) and coil 7 (Fig. 15(c)).

Figure 16 contains the same parameters as Fig. 14; the only difference is that the transmitting coil is number 2 instead of number 1. As a result, the distance is shortened between the transmitting EMAT and the defect. This appears to have an impact on the shape of the distribution of data when the signal is transmitted across the plane and interacts with a defect as seen in Figs. 17(b) and 17(c). In these mentioned plots, the shape of data distribution changes significantly when the pipe has no defect or contains corrosion, circled in red. Figures 18 and 19 demonstrate the statistical distributional characteristics of the time-series data that is transmitted and received crossing the pipe without primarily crossing the location of a defect. The distribution appears to be similar for all cases in Figs. 18 and 19, suggesting that even when a defect is present the signals will have difficulty showing this information unless the signal passes directly through the defect region.

After evaluating the time-series data, we can see that within a certain limit, state-of-the-art DL techniques utilized in this investigation potentially present a viable solution to TSC for the robotic inspection of pipes. Training plots for said models can be seen in Fig. 20. The first column of plots represents the entire loss during training, the middle plot provides a more detailed representation of what occurs during the final 50 epochs of training, and the third column highlights the learning process as it pertains to accuracy. The plot indicates that only 10 epochs are needed to reach optimal training of the LSTM.

The accuracy and time-to-prediction comparison for all four models can be seen in Fig. 21. Looking at the LSTM model closer, this algorithm utilizes 100 memory units and a binary cross-entropy loss function. The LSTM model achieved an accuracy of 78.81% with a time-to-prediction of roughly 3 s. The next most accurate model was the LSTM–FCN reaching an accuracy of 71.4% with roughly 1 s to classify the pipe condition. Rounding out the final two performances are the FCN and InceptionTime models with accuracies of 69 and 64.3% and time-to-predictions of 1.049 and 3.569 s, respectively.

Figure 23 depicts a visual representation of the accuracy comparison across different training data sizes for the LSTM–FCN network. For brevity purposes, we show the accuracies associated with the model and training data size, ranging from 60% to 100%, demonstrating that not only does the noise decrease accuracy prediction, but we can also expect a slight increase in model performance as the dataset grows in size. The corresponding y-axis represents the achieved accuracy in anomaly classification, the model trained on 100% of the dataset size reaches higher peak accuracies (∼70%). Distinct lines trace the accuracy trends for each training and testing, as expected a clear trend in decreased accuracy is present across all models compared to the model performance with signals without noise.

Complementing the visual analysis, Table 2 presents a tabulated summary of the accuracy scores obtained for each training data size. The table provides a concise overview of the model's discriminative capabilities. This tabulated presentation underscores the influence of data volume on accuracy, facilitating a direct comparison of results and aiding in the determination of optimal training data sizes for pipeline anomaly detection. We can see a trend with the noisy signals that as data size increases, so too does the accuracy of model classification. The highest accuracy (64%) was achieved with the LSTM–FCN model trained on 100% of the available data.

This study delved into Time-Series Classification (TSC) of ultrasonic testing data from Steel pipes, integrating cutting-edge AI algorithms within a robotic inspection framework. In this study, we aimed to lay the groundwork for pipeline anomaly detection using deep learning methodologies. While hyperparameter tuning and architectural optimization were not explicitly pursued in this work, the presented results provide an essential benchmark for assessing the performance of future schemes. The research addressed the intricate challenges posed by TSC in robotic inspections and introduced four Deep Learning (DL) algorithms for binary classification of pipe conditions. Notably, the LSTM model achieved a commendable 78.81% accuracy with a 3-second time-to-prediction, while the LSTM–FCN model provided 71.4% accuracy in just 1 s—a trade-off highlighting their distinct strengths.

By embedding a reliable DL network into robotics, this method emerges as a proactive tool, offering preemptive insights during routine inspections. It paves the way for early defect detection, enabling proactive maintenance and averting unexpected failures. This innovative approach presents a promising avenue for enhancing the longevity and reliability of vital infrastructure systems.

Although baseline DL performance showcases potential for tubular robotic inspections, the research underscores the need for tailored acoustic algorithms to heighten defect detection and characterization within the pipeline network. We believe this investigation showcases the potential of AI-powered TSC in robotic inspections, with the possibility of redefining the landscape of predictive maintenance during a robotic pipe inspection. This comprehensive approach not only addresses the need for increased model complexity in the field of machine learning but also emphasizes our commitment to achieving higher accuracy by investigating noise augmentation and utilizing different subsets of the dataset. By integrating these varied architectural variations, we aim to contribute to the advancement of pipeline inspection methodologies, providing a more thorough and reliable approach to anomaly detection.

The authors would like to thank DOE-NETL for its financial support. This research effort was funded under Award No. DE-FE0031649.

There are no conflicts of interest.

The datasets generated and supporting the findings of this article are obtainable from the corresponding author upon reasonable request.

DL =

deep learning

EMAT =

electromagnetic acoustic transducers

FCN =

fully connected network

LSTM =

long-short term memory

ML =

machine learning

MTS =

multivariate time-series

NDT =

nondestructive testing

NDE =

non-destructive evaluation

RF =

random forest

ReLU =

rectified linear unit

SHM =

structural health monitoring

TSC =

time-series classification

UR =

universal robot

UT =

ultrasonic testing