A general hierarchical ensemble-learning framework for structural reliability analysis

Highlights

• Propose a general hierarchical ensemble-learning framework for reliability analysis.

• Two-layer models cooperate with each other to fit the limit state function.

• The entire training process is divided into three different phases.

• A method called CESM-ELF is proposed to check the framework's effectiveness.

• The proposed framework can improve the prediction accuracy of ensemble-learning.

Abstract

Existing ensemble-learning methods for reliability analysis are usually developed by combining ensemble-learning with a learning function. A commonly used strategy is to construct the initial training set and the test set in advance. The training set is used to train the initial ensemble model, while the test set is adopted to allocate weight factors and check the convergence criterion. Reliability analysis focuses more on the local prediction accuracy near the limit state surface than the global prediction accuracy in the entire space. However, samples in the initial training set and the test set are generally randomly generated, which will result in the learning function failing to find the real “best” update samples and the allocation of weight factors may be suboptimal or even unreasonable. These two points have a detrimental impact on the overall performance of the ensemble model. Thus, we propose a general hierarchical ensemble-learning framework (ELF) for reliability analysis, which consists of two-layer models and three different phases. A novel method called CESM-ELF is proposed by embedding the classical ensemble of surrogate models (CESM) in the proposed ELF. Four examples are investigated to show that CESM-ELF outperforms CESM in prediction accuracy and is more efficient in some cases.

Introduction

Structural reliability analysis aims to assess the failure probability P_f of a structure under the current model inputs and their associated uncertainty. In general, the model inputs are considered as random variables, and the corresponding P_f can be formulated by means of the following equation [1,2],

$$P_f={\int }_{F}f\left(\mathit{x}\right)d\mathit{x}={\int }_{R^n}I\left(\mathit{x}\right)f_{\mathit{X}}\left(\mathit{x}\right)d\mathit{x}$$

where $\mathit{x}={(x_1,x_2,...,x_n)}^{\mathrm{T}}$ refers to the vector of n random variables, $f_{\mathit{X}}\left(\mathit{x}\right)$ denotes the joint probability density function (PDF) of x, and I(x) is the indicator function of the failure domain $F=\{\mathit{x}:g(\mathit{x})\le 0\}$, which can be expressed as

$$I\left(\mathbf{x}\right)=\{\begin{array}{c}1\\ 0\end{array}\begin{array}{c}\text{if}g\left(\mathbf{x}\right)\le 0\\ \text{otherwise}\end{array}$$

where g(x) is the performance function. The evaluation of Eq. (1) can be rather difficult, especially when the performance function is highly nonlinear or implicit [3]. Over the last few years, researchers have created a variety of methods to cope with such an issue, including approximation methods [4], sampling-based methods [5], and surrogate model methods [6], etc.

The first-order reliability method (FORM) [7] and second-order reliability method (SORM) [8] are two extensively utilized approximation methods. However, they rely heavily on the gradient information of the performance function, which is difficult to get when the function is implicit. Moreover, they tend to lose prediction accuracy in dealing with highly nonlinear problems. Monte Carlo simulation (MCS) [9], the basic sampling-based method, can not only overcome the above weaknesses, but also take any distribution type of random variables into consideration. As long as the computational budget is sufficient, MCS can obtain the results with the desired precision. Since evaluating a large number of samples by calling the performance function may be costly and time-consuming when complex models such as computational fluid dynamics (CFD) and finite element analysis (FEA) are involved, it is infeasible to directly apply MCS to handle practical engineering problems [10]. Surrogate model methods effectively relieve the computational burden by partly replacing computationally expensive model simulations with cheap surrogate models. Some widely used surrogate models include (but are not limited to) Kriging model [11], support vector machine (SVM) model [12], radial basis function (RBF) model [13] and artificial neural network (ANN) model [14]. Kaymaz [15] firstly applied Kriging to structural reliability problems and examined the effects of Kriging parameters on the results. Zhao et al. [16] adopted SVM to help obtain the design point and reliability index with higher accuracy and efficiency, and checked its application to rock engineering problems. Zhao et al. [17] proposed a reliability analysis method called RBF-GA, in which an RBF model was used to approximate the performance function and a genetic algorithm (GA) was adopted to find the “potential” most probable point (MPP) to refine the RBF model.

Although there are many surrogate model methods available in the literature, it is almost impossible to know which is the most suitable for approximating the performance function when one has only limited knowledge about it before the task of assessing structural reliability starts. An individual surrogate model may show excellent performance on one instance when the characteristic of the performance function to fit falls into its application domain, and may not work well in another instance [18]. To save the effort of selecting an appropriate individual model, the concept of ensemble-learning was proposed and has been developed [19]. An ensemble model tends to possess higher prediction accuracy than an individual surrogate model, as a result of taking advantage of the prediction abilities of multiple surrogate models [20]. Existing ensemble-learning methods can be divided into two strategies. One strategy is selecting the surrogate model with the highest accuracy among multiple surrogate models trained at the same time, and considering it alone for prediction. Teixeira et al. [21] proposed a measure of compatibility, which only activates the most suitable surrogate model for each iteration. The other strategy is combining multiple surrogate models through a weighted form, and the surrogate model with higher accuracy is given a higher weight factor. Yang et al. [22] trained several deep neural networks (DNN) and added them with linear weights to obtain a final forecast of the load uncertainties. Eamon et al. [23] assigned three stand-alone cumulative distribution functions (CDFs) weight factors based on their anticipated accuracy to build an ensemble CDF, which is the most useful for reliability estimation. In this paper, we select the second strategy as our research object since it appears more often in related studies.

One distinctive feature of reliability analysis by means of surrogate models is that prediction accuracy at samples near the limit state surface (g(x)=0) is of paramount importance compared to that of samples lying far from it. It implies that most of the computational budget should be spent on the evaluations of samples near the limit state surface. An appropriate learning function can effectively fulfill this requirement. Commonly used learning functions include (but are not limited to) efficient global optimization (EGO) method [24], the expected feasibility function (EFF) [25], and the U-learning function [26]. Jiang et al. [27] proposed a failure-pursuing sampling (FPS) strategy, which firstly partitions the sampling region into several Voronoi cells and then selects update points in the most relevant cell by the above-mentioned learning functions. Recently, Cheng and Lu [28] extended the idea of U-learning function and proposed an active learning function suitable for ensemble-learning, which seeks samples with high prediction error variance and close to the limit state surface to update the ensemble model.

Introducing the learning function can only ensure that the newly added training samples are near the limit state surface. However, initial training samples used to build the initial ensemble model tend to be selected randomly. These initial training samples cannot help the ensemble model fit the landscape near the limit state surface, which wastes computational resources to a certain extent. What's more, quite a number of ensemble-learning methods construct a test set in advance to evaluate weight factors and check the convergence criterion. Similarly, samples in the test set are randomly generated and may be anywhere in the entire space of associated random variables. This will lead to the fact that weight allocation may not be optimal or even unreasonable. In one word, existing ensemble-learning methods concentrate on global prediction accuracy in the entire space instead of local prediction accuracy near the limit state surface. Actually, the latter is the real concern of structural reliability analysis. In order to fill the vacancy of related researches and further improve prediction accuracy, a general hierarchical ensemble-learning framework (ELF) for reliability analysis is proposed. Two-layer models are adopted in the proposed ELF and cooperate with each other to obtain an approximation of the performance function. To be specific, a crude model in the outer layer is expected to feed an accurate model in the inner layer with samples near the limit state surface. There are three phases in total throughout the entire modeling process: the initialization phase, the supplementary phase and the stabilization phase. In different phases, both the number of samples provided by the outer model to the inner model and allocation modes of these samples are different. An accurate model in the inner layer is expected to explore the update points that are relevant and utilize these samples to refine the outer and inner models at the same time. The outer and inner models act alternately until a convergence criterion is satisfied. After the training is completed, the inner model is used instead of the real performance function to perform structural reliability analysis by the MCS method. Generally, an individual surrogate model can be chosen to be the outer model, and an ensemble model serves as the inner model.

In this paper, the classical ensemble of surrogate models (CESM) [28] based on global measures [29] is chosen to be embedded in the proposed ELF. We call this novel method CESM-ELF for short. Note that, in principle, any type of ensemble-learning method can be embedded in the proposed ELF. The remainder of this paper is organized as follows. Section 2 makes a brief review of CESM and two important components used in this work: the learning function and convergence criterion. In Section 3, ELF is proposed and the procedure of CESM-ELF is presented in detail. Four examples are investigated to test the effectiveness of the proposed ELF in Section 4. Finally, some conclusions are drawn in Section 5.