NIE Xiaobo(聂晓波)， LI Haibin(李海滨)

1 College of Mechanical Engineering， Inner Mongolia University of Technology， Hohhot 010051， China

2 College of Science， Inner Mongolia University of Technology， Hohhot 010051， China

3 Inner Mongolia Key Laboratory of Advanced Manufacturing Technology, Hohhot 010051, China

Abstract: Aiming at the reliability analysis of small sample data or implicit structural function， a novel structural reliability analysis model based on support vector machine (SVM) and neural network direct integration method (DNN) is proposed. Firstly， SVM with good small sample learning ability is used to train small sample data， fit structural performance functions and establish regular integration regions. Secondly， DNN is approximated the integral function to achieve multiple integration in the integration region. Finally， structural reliability was obtained by DNN. Numerical examples are investigated to demonstrate the effectiveness of the present method， which provides a feasible way for the structural reliability analysis.

Key words: support vector machine (SVM)； neural network direct integration method； structural reliability; small sample data; performance function

Introduction

Currently， structural reliability analysis calculation methods mainly include probability method (direct integration method[1-3])， approximate probability method (e.g.， first-order moment method， second-order moment method[4-5]) and numerical simulation methods (e.g.， Monte Carlo method[6]). With the development of mathematical statistics theory， the relatively accurate probability method-direct integration method， is gradually gaining more attention. The direct integral method for calculating structural reliability mainly includes three aspects of technical problems[1]， the regularization of the integral region， the constructing of the probability density function， and multiple definite integrals in the regular region. However， when the sample data are insufficient or the limit state function is implicit， regularization of the integration region is difficult to achieve. At the same time， the calculation of multiple integrals has not been effectively solved， which limits the direct integration method.

Reliability experimental data are often limited by many factors. When the experimental sample size is not large， it is difficult to obtain satisfactory results with classical reliability analysis methods. At present， the reliability analysis methods for small sample data mainly include the Bayes method[7]， the Bootstrap method[8]， and the support vector machine (SVM) method[9-10]. The Bayes method uses the prior distribution to obtain better probability estimates under small sample conditions. Whether the prior distribution is reasonable will directly affect the evaluation results of the system reliability. However， in the application process of the Bayes method， there is often little prior information or even no prior information. The Bootstrap method is essential a re-sampling process， which uses existing data to simulate unknown distributions. This method can perform interval estimation or statistical hypothesis testing. The small sample problem is transformed into a large sample problem by regenerating sampling， which is suitable for statistical inference under the condition of small sample.

SVM is a machine learning algorithm based on statistical learning theory that was proposed by Professor Vapnik. It adopts the principle of minimizing structural risk and has good generalization performance. It can solve small samples， nonlinear and high-dimensional problems， and is mainly used for pattern recognition and function fitting. The method of processing technology reliability evaluation based on Copula-SVM is presented in Ref.[11]. Yangetal.[12]proposed a multi-least squares recursive projection dual support vector machine (MLSPTSVM) for multi-classification problems. Tehranyetal.[13]used SVMS with different kernel functions for spatial prediction of flood occurrence. Mohammadpouretal.[14]predicted the water quality index of free constructed wetland using SVM and two artificial neural network(ANN) methods.

Neural networks have the ability to approximate arbitrary functions. Using this property， neural networks can be used in structural reliability calculations. Shu and Gong[15]presented an ANN-based response surface method that can be used to predict the failure probability of slopes with spatially variable soil. Pengetal.[16]proposed mixed uncertainties reliability analysis method based on back propagation neural networks. Concerning the issue of high-dimensions， hybrid uncertainties of randomness and intervals including implicit and highly nonlinear limit state function， reliability analysis based on the hybrid uncertainty reliability mode combining with back propagation neural network (HU-BP neural network) was proposed. Lietal.[1]proposed dual neural network direct integration method (DNN) for the calculation of multiple integrals. In this paper， the dual neural network combined with the function of regularization of the integration region overcame the difficulties of multiple integration and realized the accurate calculation of the reliability of the direct integration method.

This paper combines the advantages of SVM and DNN for small sample data reliability calculation. The basic idea of the proposed method is to use Latin hypercube sampling (LHS) to obtain the sample data， and then use the support vector regression machine to train the sample data， approach the limit state function， and obtain performance function of the direct integration method to regularize the integration region. Then the DNN[1]is used to perform multiple integration calculations to obtain the structural reliability and achieve a certain calculation accuracy. The proposed method is compared with traditional method such as Monte Carlo simulation method and first-order second-moment to verify the feasibility of the proposed method.

This paper is organized as follows. In section 1， direct integral method is introduced. In section 2， structure performance function is fitted by SVM and regularization of the integral area is obtained. In section 3， the reliability is solved with DNN. Examples are followed to demonstrate the proposed methods in section 4. The conclusions are shown in section 5.

1 Direct Integral Method

Assume that basic random variables of structure areX1，X2， …，Xnand the corresponding probability density function (PDF) isfX(x1，x2， …，xn). Limit state function (LSF) of these random variables isZ=g(X)=g(X1，X2， …Xn).Z>0 shows that the structure is safe；Z<0 shows that the structure falls in failure；Z=0 shows that the structure is in the critical state.

So reliability probability of the structure is expressed as

(1)

whereF={x|g(X)>0} indicates the structural failure domain andg(X) indicates the limit state function.

Direct integral method to calculate structural reliability mainly contains three parts. The first part is regularization of integral area. The second one is composition of PDF. The last one is multiple definite integral in regularization area.

(1) Regularization of integral area will be finished by SVM.

(2) For composition of PDF， this paper will adopt NATAF model.

Firstly define a set of standard normal variablesZ=(Z1，Z2， …，Zn) through transformation formulaZi=H-1[Fxi(xi)]，i=1， 2， …，n， where，H(x) is the standard normal cumulative distribution function，Fxi(xi) is the cumulative distribution function ofxi.

In the NATAF model， assume thatZis joint normal distribution， and then the joint PDF will be got by some transformation rules of the theory of probability， which is shown as

fX(X)=

fx1(x1)…fxn(xn)·hn(Z，R′)/h(z1)…h(zn)，

(2)

(3)

(3) Multiple definite integral in regularization area.

This paper will use DNN proposed in Ref.[1] to solve the multiple integration problem.

2 Regularization of the Integral Area

2.1 SVM fitting structure function

The SVM is formed by introducing a loss function that corrects the distance based on the support vector classifier. Its basic idea is to map the input space to the high-dimensional feature space through the nonlinear transformation of the inner product function. For non-linear problems， the original problem is mapped to a linear problem in a high-dimensional feature problem through non-linear transformation. In the high-dimensional feature space， the kernel function satisfying Mercer condition is used to replace the inner product in the linear problem， thus solving the problem of nonlinear SVM.

At present， there are four kernel functions commonly used.

(1) Linear kernel function

K(x，xi)=x·xi.

(5)

(2) Polynomial kernel function

K(x，xi)=[(x，xi)+1]q.

(6)

(3) Gaussian kernel function

(7)

(4) Sigmoid kernel function

K(x，xi)=tanh(v(x，xi)+c).

(8)

One of the key issues in using SVM to solve structural reliability is to choose a kernel function. The linear kernel function is suitable for the linearly separable cases. The input space and the feature space are the same dimension， the classification effect is obvious， and the parameters are less fast， so the linear kernel function is usually used in the support vector classifier. The polynomial kernel function is a global kernel function， which can map low-dimensional input space to high-dimensional feature space， but the polynomial kernel function has many parameters and the calculation is relatively complicated. When the polynomial indexqis too large， the learning complexity is too high， and it is prone to “over-fitting” phenomenon. The Gaussian kernel function is one of the most widely used kernel functions. It can also map samples into high-dimensional space. Compared with polynomial kernel functions， its parameters are few. It has good anti-interference ability to the noise existing in the data. When using the sigmoid kernel function， the SVM implements a multi-layer perceptron neural network， which has good generalization ability for unknown samples.

When the direct integral method based on SVM is used to calculate structural reliability， several problems should be processed as follows.

2.1.1LHSmethod

In LHS method， firstly determine the sampling frequency and then stratify the probability distribution of variables. Cumulative distribution curve is divided into equal interval on the cumulative probability scale [0， 1]. Then extract the sample from each interval or layer of probability distribution and represent the value of each interval.

Use LHS method to extractNsamplesxi=(xi1，xi2， …，xin)T(i=1， 2， …，N) from random vectorX=(X1，X2， …，Xn)T.

(1) The scope of each random variableXj(i=1， 2， …，n) can be divided into N equal probability interval.It means that value [0， 1] of cumulative distribution functionFXj(xj) can been divided into N intervals which don’t overlap each other [0， 1/N]，(1/N， 2/N]， …， (1-1/N， 1].

(2) For each variableXj， extract a sample from each interval in all. Every child interval only generates a random number， as a representative of the range value.

(3) For all sample values of each variableXj， make random arrangement according to interval number and then put them together according to the variable sequence.

2.1.2Datapreprocessing

The sample data have been obtained by sampling method. Then divide them into training set and test set. Carry out normalization preprocessing for both of them. Normalized mapping is

(4)

where，xin，yin∈Rn，xmin=min(xi)，xmax=max(xi). The result of data normalization is to normalize the original data to [0， 1]，yin∈[0， 1]，i=1， 2， …，n.

2.1.3Parameteroptimization

Reasonable selection of penalty parameterCand kernel function parameterεis essential for SVMs. In this paper， the K-fold Cross Validation (K-CV) method is used to perform parameter optimization. The specific methods are as follows.

Firstly， the original data is divided intoKgroups evenly， and then a validation set is performed on the data of each subset. At the same time， the data of the remainingK-1 groups of subsets are regarded as the training set， thereby obtainingKmodels. Finally， the average of the classification accuracy of the validation set of theKmodels is used as the performance index of the K-CV classifier. The K-CV classifier can effectively avoid the occurrence of under-learning or over-learning， and the selected parameters are more reasonable.

2.2 Regulation of integral area

Regulation of integral area can be achieved by introducing the indicator functionI[g(X)≤0]. The original irregular area is translated into an infinite area， and then the infinite area of integral is translated into limited area of integral according to certain accuracy， which is shown as

(5)

where，F(X)=I[g(X)>0]fX(X). Wheng(X)≤0，I[g(X)]=0； wheng(X)>0，I[g(X)]=1.

3 Reliability Solution

3.1 Dual neural network multiple integral method

In Ref.[1]， the DNN used the neural network method to approximate the integrand function to achieve multiple integrations in the integration region. The main idea is as follows.

The difficulty of multiple integration is to obtain the integrand function of the integrand. In this paper， the integrand function is obtained using dual neural networks. A single hidden layer BP neural networkBis constructed to establish the mapping relationship between the input variablesXand the original functionY. The network structure is shown in Fig. 1. The functional relationship between the network output and the input variables is

(6)

Perform the differentiation operation and obtain the neural networkAas

(7)

Fig. 1 Original function network structure

A pair of neural networks composed of neural networkAand neural networkBis called a dual neural network. Among them， neural networkAis called integrand function network， and neural networkBis called original function network. When networkAin the dual neural network approximates the integrand function in the integral， networkBapproximates the original function. Then， multiple integral calculations are performed.

3.2 Calculation process

Firstly， the structural performance function is fitted by support vector regression machine. Through LHS， sample data pre-processing， kernel function selection and optimization of SVM parameters， the SVM is used to train and test data， fit the structural performance function and regularize it； secondly construct a joint probability density function； finally， use DNN to perform multidimensional numerical integration to obtain the structural reliability. The calculation process is shown in Fig. 2.

Fig. 2 Flow chart of reliability calculation

4 Numerical Examples

In order to verify the rationality and feasibility of the proposed method， the two calculation examples are calculated using the proposed method—SVM and DNN (SVM-DNN) and traditional methods such as SVM， DNN and Monte Carlo simulation method(MC). The results of Monte Carlo method sampled 100 000 times are used as theoretical solutions， and the relative errors of different methods are given.

4.1 Example 1

Fig. 3 Flat frame structure

According to the above example， the joint probability density function of input variablesP，E，Ican be given by

whereδ=3.549 9，α=1.282 5.

Then use the proposed method and traditional methods to calculate the structural reliability， results are shown in Table 1.

Table 1 Reliability of Example 1

From Table 1， it can be seen that the result of MC method is 0.999 3 as theoretical solution； the result of DNN method is 0.984 5， relative error is 1.48%； the result of SVM method is 0.999 9， relative error is 0.06%； the result of the proposed method with 30 samples is 0.991 2， relative error is 0.8% which fully meets the requirements of engineering applications (relative error is less than 3%). The results show that the method is suitable for structural reliability analysis.

It can be seen from the results in the table that the calculation accuracy of SVM-DNN is higher than DNN. At the same time， SVM-DNN method also retains the high computational efficiency of DNN.

4.2 Example 2

The reliability analysis of the 10-bar truss structure is shown in Fig. 4. The lengths of the horizontal and vertical rods are bothL， assuming that the cross-sectional area of the horizontal rodA1， the cross-sectional area of the vertical rodA2， and the cross-sectional area of the inclined rodA3are normal distributed random variables， the average of these three random variables are 13， 2 and 9 Square inches respectively， and the coefficients of variation are all 0.1. It is assumed that other parameters of the structure， such as elastic modulus， material density， member length， and applied load， are all deterministic variables. LoadP=100 000 pounds， rod lengthL=9.14 m， elastic modulusE=107， and allowable displacement value of vertex 2,dallow=0.101 6 m.

Fig. 4 Diagram of 10-bar truss structure

Use the proposed method to approach the performance function with SVM and calculate the reliability with DNN. The reliability calculation results are listed in Table 2.

Table 2 Reliability of Example 2

From Table 2， it can be seen that the reliability calculated by MC method is 0.992 8 as theoretical solution； the reliability calculated by SVM method is 0.986 6， relative error is 0.6%； the reliability calculated by SVM-DNN is 0.994 7， relative error is 0.19%.

Because this example is an implicit structural function， the neural network method cannot be used to solve the structural reliability. Therefore， the proposed method is suitable for structural reliability calculation with implicit structural function.

5 Conclusions

In order to solve the structural reliability calculation of small sample data， this paper combines a support vector regression machine and a direct integration method of dual neural networks to propose a reliability calculation model. This method firstly fits structural performance function by support vector regression machines. Through LHS， sample data preprocessing， selection of kernel functions， and optimization of support vector machine parameters， the SVM is used to perform sample training and test set data to approach the performance function. Then construct a joint probability density function， and finally use neural network method to perform multi-dimensional numerical integration to calculate the structure reliability. Calculation examples show that the proposed method is a feasible and effective way to solve the structural reliability analysis under small sample data or implicit limit state function.

This method is based on DNN， so it inherits the computational efficiency of DNN and improves the computational accuracy at the same time. This method provides a new way to solve the structural reliability analysis of small sample data or implicit structural function.