Chuanqi Li, Jian Zhou*, Kun Du*, Daniel Dias

a School of Resources and Safety Engineering, Central South University, Changsha 410083, China

b Laboratory 3SR, CNRS UMR 5521, Grenoble Alpes University, Grenoble 38000, France

Keywords:Underground pillar stability Hard rock Support vector machine Metaheuristic algorithms

A B S T R A C T Hard rock pillar is one of the important structures in engineering design and excavation in underground mines.Accurate and convenient prediction of pillar stability is of great significance for underground space safety.This paper aims to develop hybrid support vector machine(SVM)models improved by three metaheuristic algorithms known as grey wolf optimizer (GWO), whale optimization algorithm (WOA)and sparrow search algorithm (SSA) for predicting the hard rock pillar stability.An integrated dataset containing 306 hard rock pillars was established to generate hybrid SVM models.Five parameters including pillar height, pillar width, ratio of pillar width to height, uniaxial compressive strength and pillar stress were set as input parameters.Two global indices, three local indices and the receiver operating characteristic (ROC) curve with the area under the ROC curve (AUC) were utilized to evaluate all hybrid models’ performance.The results confirmed that the SSA-SVM model is the best prediction model with the highest values of all global indices and local indices.Nevertheless, the performance of the SSASVM model for predicting the unstable pillar (AUC: 0.899) is not as good as those for stable (AUC:0.975) and failed pillars (AUC: 0.990).To verify the effectiveness of the proposed models, 5 field cases were investigated in a metal mine and other 5 cases were collected from several published works.The validation results indicated that the SSA-SVM model obtained a considerable accuracy,which means that the combination of SVM and metaheuristic algorithms is a feasible approach to predict the pillar stability.Ⓒ2023 Published by Elsevier B.V.on behalf of China University of Mining & Technology.This is an open

1.Introduction

The lack of non-renewable mineral resources not only leads to the green and efficient development of coal mining,but also forces most open-pit metal mines to gradually transfer to underground space[1].Although bolt support technology can improve the excavation stability, its failure mechanism is still unclear [2].On the other hand, the bolt cannot give full play to its anchoring performance in the pillar independent of the stable rock layer.In contrast, hard rock pillars can provide effective support to ensure a safe underground excavation environment.However,unstable pillars can lead to many major accidents, such as stope caving, collapse, and even casualties and equipment damage, resulting in significant economic losses [3].The development of effective and easy methods for predicting the hard rock pillar stability has great significance to ensure the underground space safety.

The traditional pillar stability analysis centered on the pillar safety factor (SF), which is defined as the ratio of pillar strength to pillar stress.When the SF is less than 1.0,it means that the pillar stress exceeds its ultimate compressive strength, and the pillar instability may occur[4].The pillar stress depends on the overburden depth and the distance between adjacent pillars.Generally,the tributary area theory and numerical modeling methods are widely used to estimate the bearing stress of pillars [5].Besides, some empirical formulas were generated to estimate the pillar strength based on a large number of stable and unstable pillars investigated in mines [6–8].For example, Hedley [9] first focused on the relationship between the strength parameter,width and height of pillar (w and h) and the pillar strength, and proposed an empirical formula to estimate the hard rock pillar strength at Quartzites Elliot Lake Uranium mine in Canada.Von Kimmelmann et al.[10]investigated 57 hard rock pillars and established an empirical formula to estimate their strength at Metasediments Selebi Phikwe mine.Esterhuizen et al.[11] created an empirical formula based on the joint information of pillars, and successfully analyzed the stone pillars’ stability in the United States.

Furthermore, this hotspot has been discussed using various numerical simulation methods [12–14].Mortazavi et al.[15] used FLAC3D to study the pillar stability and non-linear behavior of hard rock pillars under natural loading conditions.Renani and Martin[16] discussed the influence of pillar size and shape on the pillar stability using two-dimension and three-dimension (2D and 3D)finite difference analysis.Kim et al.[17] simulated the stability change of pillars during excavation based on the FLAC2D program at Nohyun limestone mine.Kumar et al.[18] estimated the pillar strength and studied the instability problem using a strain and softening model of a real environment.Deng et al.[5] proposed a hybrid analysis approach including the finite element method(FEM), neural networks (NN) and reliability analysis for pillar design.Recently, Li et al.[1] proposed a method combining finite difference method (FDM), neural network (NN) and Monte Carlo simulation (MCS) to assess pillar stability.Although empirical method and numerical simulation method have greatly enriched the study of hard rock pillar stability,the problem of low universality of empirical formula and the calculation distortion caused by simplified models in numerical simulations are always obstacles in the research of pillar stability prediction [19].

In recent years, artificial intelligence (AI) technology has become popular in the field of mining engineering [20–24].Compared with empirical formula and numerical simulation methods,the most unique characteristic of the AI method is the collection of ‘‘big data” of hard rock pillars from different mines to generate common prediction models.Therefore,various numbers of considered parameters can be used to predict the corresponding pillar stability.For example, Zhou et al.[25] used Fisher discriminant analysis and a support vector machine (SVM) model to predict the stability of 46 hard rock pillars by considering w, h, ratio of w to h (w/h), uniaxial compressive strength (UCS) and pillar stress(Ps).Wattimena[26]considered w/h and Psto predict 89 pillars stability by using the multinomial logistic regression (MLR) model.Ghasemi et al.[4] successfully analyzed the stability of 178 hard rock pillars by adopting J48 and SVM models.Ding et al.[27]used optimum stochastic gradient boosting (SGB), random forest (RF),SVM, and multilayer perceptron neural network (MLPNN) models to predict 205 pillars’ stability.Among machine learning models,SVM is frequently chosen to solve engineering problems,especially those with small samples, nonlinearity and high dimensions, who follows the structural risk minimization criterion and ensures that the dimension of the feature space does not affect the complexity of the problem[28,29].However,the prediction performance of AI models is limited by their hyper-parameters, e.g., number of hidden layers and neurons, trees, and smoothing factor [30–32].For instance, multiple hidden layers and neurons not only increase the running time but also reduce the learning performance of some neural network models[33].To achieve the best predictive performance,numerous metaheuristic algorithms were imposed to solve optimization problems based on the natural behavior, e.g., grey wolf optimizer (GWO) by Mirjalili et al.[34], whale optimization algorithm (WOA) by Mirjalili and Lewis [35], and sparrow search algorithm(SSA)by Xue and Shen[36].Moreover,numerous scholars have utilized one or more meta-heuristic algorithms to solve practical engineering problems [37–39].For the SVM model, an unsatisfactory choice of hyperparameters might trap the model in local optimization.Xu et al.[40] utilized differential evolution(DE), GWO and artificial bee colony (ABC) algorithms to optimize the SVM model for predicting the geo-mechanical properties of rock.The prediction results showed that the GWO could significantly improve the prediction performance of the SVM model with the optimal combination of hyperparameters.Zhou et al.[41]combined the WOA algorithm and the SVM model to predict the advance rate of a tunnel boring machine.The performance evaluation results indicated that the prediction accuracy of the WOASVM model is higher than the initial SVM model.Tuerxun et al.[42] utilized SSA to improve the predictive performance of the SVM model for estimating the wind turbine fault diagnosis.Although the combination and application of metaheuristic algorithms and SVM model in underground space is emerging, its research on the hard rock pillar stability prediction is insufficient and worth exploring.

The main objective of this paper is to predict the hard rock pillar stability by combining three metaheuristic algorithms and SVM model, i.e., GWO-SVM, WOA-SVM and SSA-SVM models.The rest of this paper is described as follows.Section 2 indicates the highlights and main contributions of this paper in analyzing the stability of hard rock pillars in underground mines.Section 3 introduces in detail the SVM model, GWO, WOA and SSA optimization algorithms.Section 4 presents the data sources of the integrated database and the corresponding data analyses.Section 5 displays two global indices and three local indices to evaluate the performance of the proposed models for predicting the hard rock pillar stability.Section 6 exhibits the development and evaluation performance of the proposed models.Section 7 describes the application and validation of the best-proposed model in a practical project,as well as the comparison with similar studies.Section 8 summarizes the main conclusions and gives the limitations of this paper and the possible research directions in future work.

2.Research significance

As a common large rock mass structure in the stope, hard rock pillars are often arranged by engineers to support the roof at the initial stage of design to facilitate the smooth development of mining work.However,the stability of irregular pillars under complex stress environments needs to be accurately captured.In the stability analysis of the pillar, geometric parameters are the most easily obtained external data of the pillar,and the pillar with too small w/h is prone to stress concentration, resulting in local collapse and instability.In other words, the pillar strength is not enough to withstand the load imposed by the upper rock mass.The increase in pillar width can reduce the load carried by each pillar per unit space, but it also means the ore loss.Therefore, the accurate prediction of the stability of hard rock pillars is of high significance since it is the necessary condition to ensure the reasonable and safe exploitation of resources.

Previous studies have tried to predict hard rock pillar stability using empirical and numerical simulation methods.However, the same empirical formula is difficult to apply in other mines and the pillar simulation model simplified by cylindrical or cuboid(i.e.,geometry difference,especially the w)cannot forecast the real stability of the hard rock pillar [43].Furthermore, the SF of unstable pillars is also greater than 1.0 in many actual mines [5,25].AI techniques, such as SVM, may be used to correctly predict pillar stability.This paper contributes a)to the integration of a database consisting of all published cases of hard rock pillars;b)to the prediction of the published hard rock pillars stability,which is significant to evaluate the overall stability of the stope and redesign the mining plan.Three optimized SVM models are proposed to predict the hard rock pillar stability, and these novel hybrid models are verified by using 5 true pillar cases from the previous studies and another 5 pillar cases from an actual mine named the Baishixi metal mine, Guizhou Province, China.

3.Methodologies

3.1.Support vector machine

Support vector machine(SVM)is a machine learning technique developed by Vapnik [44] and is widely used to solve regression and classification problems [45].SVM generalizes a limited number of samples based on the principle of minimizing structural risk.

Some important relationships between input (xi) and output (yi)parameters(i.e.,dependency,mapping,or function relations)need to be learned by the SVM model.If the output parameter is a certain label,the purpose of SVM is to generate a hyperplane to maximize the distance between samples of different classes as shown in Fig.1.Therefore, each object i satisfies the following relationship:

Fig.1.Schematic diagram of the SVM model for classification.

Fig.2.The overall flowchart of the GWO optimization.

Fig.3.The overall flowchart of the WOA optimization.

Fig.4.The overall flowchart of the SSA optimization.

where τ is the normal vector;P the penalty parameter;b the bias of the hyperplane; θithe distance between each object i to its corresponding margin hyperplane.And xiand yibelong to data set D:｛[x(i),y(i)]∊Rn∙R,i=1,...,n｝.

Noted that the search for the optimal process is only related to the inner product calculation between the training samples.Therefore, the kernel function is used to transform the linear regression hyperplane into nonlinearity [46].At present, the most applicable kernel function is the radial basis function(RBF)kernel,as follows:

where γ is a kernel parameter; and δ2the variance of the Gaussian kernel.

3.2.Grey wolf optimizer

Mirjalili et al.[34] proposed a novel metaheuristic algorithm named the grey wolf optimizer (GWO) based on the hunting behavior of grey wolves to solve the optimization problem, also known as the grey wolf intelligent optimization algorithm.The GWO has a simple structure and easy implementation, especially since it only needs to adjust the number of wolves to achieve optimization[47].The hunting behavior of grey wolves essentially follows the group cooperation mechanism, so there are strict social relations and hierarchies within wolves.All grey wolves are divided into four groups: alpha (aw), beta (bw), delta (dw) and omega (ow).Alpha is the sole leader, who is responsible for decision-making (hunting, resting, food distribution, etc.).Beta is responsible for assisting alpha to complete various affairs and is the link between alpha and other wolves.The main tasks of omega are reconnaissance, sentry and guard.If they behave badly, they will descend to the lowest rung of the pack, known as the delta.The wolves have grown into excellent hunting teams in nature under the action of such group relationships.Thus, grey wolves start hunting activities from an encircling, this behavior can be expressed mathematically:

Once the prey position is determined and surrounded, grey wolves except omega are divided into three small groups close to the prey.Meanwhile, omega is forced to change position to match that of the other wolves based on the Eqs.(9)–(11).After controlling the prey, grey wolves can choose to attack or leave it.If<1,grey wolves consider the prey to be the best prey and attack (i.e.,exploitation).On the contrary, if>1, grey wolves abandon the current prey and continue to look for a more suitable one (means exploration).

3.3.Whale optimization algorithm

The whale optimization algorithm(WOA)was proposed by Mirjalili and Lewis[35],it simulated the behavior of humpback whales preying on small fish.This whale-specific hunting behavior is also known as the bubble-net hunting.This behavior can be described in three steps.Firstly, whales dive to a depth of approximately 15 m.Then, whales create spiral bubbles around their prey in a path that resembles a circle or number ‘‘9”.Finally, whales follow the bubbles to the surface and catch their prey.In the WOA algorithm,whales need to find and encircle the prey,controlling it with the bubble-net method.After that, whales can choose to attack or move on to better prey.The behavior of searching and encircling prey can be done according to the following mechanism:

where R represents the distance between an individual whale and a prey; Zwthe best position of whales at the current iteration;Z(n ＋1) the updated position at the next iteration; n the current iteration time; and F and T the coefficient factors, which can be defined as follows:

where e and u are the coefficient factors.e is decreasing from 2 to 0 and u is changed directly at[0,1].The position of the spiral bubbles was updated to prevent the prey from moving, which can be expressed as Eq.(16):

where p is a random number between 0 and 1.l belongs to [0, 1]and v represents a constant to describe the spiral shape.

3.4.Sparrow search algorithm

The sparrow search algorithm (SSA) was proposed in 2020 and simulates the predatory and antipredatory behavior of sparrows[36].In the SSA algorithm,sparrows are divided into producers with large energy reserves, joiners finding food via producers and vigilantes who were responsible for warning.The identity of the discoverer and joiners is not fixed.Any sparrow that discovers a better food source during the search can become a producer, but that means another sparrow will become a joiner,because the ratio of producers to joiners is fixed in a group.During the foraging process,the producer is responsible for searching the area with abundant food and providing directions to other joiners,who always find the producer with the best food.Once vigilantes find a predator,they would send an alarm signal through song,and the producers take the joiners away to a safe area after the alarm signal reaches a certain threshold.Other sparrows at the edge of the group also quickly moved to safety, but sparrows in the middle of the group had to move randomly in the hope of getting closer to other sparrows.Assuming that the total number of sparrows is m, j represents the spatial distribution, the ratio of producers to joiners is 7 to 3 and the safety threshold of the warning signal is Ws, then Si,j=(S1,j,S2,j,...,Sm,j)is the position of the i-th sparrow in the swarm.

Therefore,the positions of the producer,joiner,and vigilante are updated according to Eqs.(17)–(19).In Eq.(17), R2≥Wsmeans that vigilantes detect a predator, all sparrows need quickly fly to safe places,and R2means that the i-th joiner with the worse fitness value is most likely the starving sparrow.In Eq.(19), fi>fbindicates that the sparrow is at the edge of swarm,and fi=fbindicates that the sparrow is in the middle of swarm.

3.5.SVM-based hybrid models

The purpose of this paper is to optimize the SVM model for predicting the hard rock pillar stability.To that end,three metaheuristic optimization algorithms named GWO, WOA and SSA were adopted to select the optimal hyperparameter combination of the SVM model, which are GWO-SVM, WOA-SVM and SSA-SVM models, respectively.Although the metaheuristic ideas of the three algorithms are very different, their core is related to swarm intelligent behavior.Therefore, these three algorithms are completely consistent in the process of optimizing the SVM model as follows:(i) Data preprocessing: Similar to the development of the initial SVM model,training and test sets are also utilized to generate classification models.It should be noted that both the training set and the test set remain the same when building different hybrid models; (ii) Parameters initialization: The metaheuristic optimizationbased swarm intelligence is an emerging method to determine the optimal or satisfactory solution by understanding or learning the hunting behavior [48].It does not depend on a lot of parameters,just the number of the population.For example, the only parameter that needs to be adjusted in the GWO algorithm is the number of grey wolves.Therefore, three hybrid models (GWO-SVM, WOASVM and SSA-SVM)were developed by considering a series of population values,while the range of population values is the same in each hybrid model.Furthermore, the upper and lower bounds of the searching space were determined by the range of hyperparameters of the SVM model.Besides, the computing time was also recorded to further select suitable models when the iteration was stopped state; (iii) Iterative loop: This step can be divided into three parts, including population initialization, fitness calculation and solution determination.Firstly,the initial population was randomly set in each optimization algorithm for searching targets.Subsequently, the fitness function is used to evaluate the predictive performance of each hybrid model with different populations.Finally, the best solution can be determined by resulting in the lowest fitness value.Meanwhile, the final solution includes the best hyperparameter combinations of the SVM model.The frameworks of using GWO, WOA and SSA to optimize the SVM model are shown in Figs.2–4, respectively.

4.Dataset and preparation

Over the past few decades, many engineers have investigated the stability of hard rock pillars around the world, including 31 cases from Westmin Resources Ltd.’s H-W mine in Canada by Lunder[7],28 cases from Eliot Lake uranium mines in Canada by Hedley [9], 47 cases from Selebi Phikwe mines in South Africa by Von Kimmelmann et al.[10], 18 cases from stone mines in USA by Esterhuizen et al.[11], 61 cases from marble mines in Southern Spain by González et al.[19], 47 cases from open stope mines in Canada by Potvin [49], 9 cases from Zinkgruvana Mine in Sweden by Sjoberg [50], 49 case from McArthur River Lead-Zinc Mine in Canada by Schubert and Villaescusa [51], 1 case from Dawenkou Gypsum Mine in China by Liu and Zhai[52],1 case from Shizishan Copper Mine in China by Zheng [53], 12 cases from Lo Tacón Zinc and Lead Mine in Spain by Alvarez-Garcia et al.[54], 2 cases from stone mines in USA[55].In this paper,a dataset was generated by combining these investigations including 150 stable cases, 59 unstable cases and 97 failed cases.The stable pillar (S) is defined as a safe structure with minor spalling and no joint opening.The failed pillar (F) has blocks falling out, severe spalling and pronounced opening of joints [23,26].Especially, the unstable pillar(U) is recognized in the underground stope which shows one or more of the signs below (cracking pillars, major displacements within the pillar, deformed drill hole) [50].

In this paper, five parameters including UCS, w, h, w/h and Pswere selected to predict the pillar stability.w and h were measured in the field by engineers with handheld instruments or tools.As demonstrated in Table 1, the maximum value of h is as high as 60 m, while the minimum value of w is only 1.9 m.Low w/h is the main characteristic of the transition from stable to the unstable pillar.The UCS was determined by conducting strength tests of the rock around the pillar.Besides, several scholars considered depth(H)to predict the hard rock pillar stability[11,23,27].Nevertheless,the Psrepresents the bearing pressure of the pillar,which is related to the H, its size and the distance between adjacent pillars [1].Therefore, the H can be replaced by the Ps, which is considered as an input parameter together with the other four parameters(w, h, w/h, UCS) to predict the pillar stability in this paper.Additional statistical information about the input parameters is shown in Table 1.

The label of the pillar stability (stable: S, unstable: U and failed: F) was considered as the output parameter to test and evaluate the model performance.Fig.5 illustrates the correlation between input and output parameters.As shown in this picture,the areas on the diagonal describe the distribution of each input parameter corresponding to three pillar labels.The larger areas indicate that the parameters are closely related to the pillar stability, thus the parameter cannot be removed to reduce operation time.In this paper, the correlation coefficients between the five input parameters are not high, and can be used for subsequent stability prediction.The formation of an excellent classifier requires training and testing.Reasonable allocation of the training set and test set ratio is very important to improve the performance of the classifier.Qualified training sets can correctly learn the relationship between training data and original data, verify the generalization ability, and refuse overfitting.At present, there are two common (70% and 80%) dividing proportions [27].According to the research problem similarity and data situation,214 cases (70%) are used as the training set, and the remaining 92 cases (30%) are taken as the test set in this paper.It should be noted that although the division of data is random, necessary data verification is still needed to make the data distribution of the three pillar labels reasonable to improve the model performance.Furthermore, all data is normalized to [-1, 1] to minimize the performance impact of parameter differentiation before developing the prediction model.

5.Evaluation indices of model performance

Table 1 Statistical description of all input parameters.

After developing three hybrid models,it is necessary to evaluate the predictive performance of these three models.Whether regression or classification problem, the perfect prediction means that the predicted values by the model are equal to the actual values.For predicting the hard rock pillar stability, three stability labels were set as the output parameters in this paper.In other words,this is a multiclass classification problem.Furthermore, the comparison of predicted and actual values is also essentially a statistical analysis.Therefore, some statistical indices were regarded as suitable global evaluation indices to accurately evaluate the performance of each developed hybrid model,including accuracy,Kappa coefficient, the receiver operating characteristic (ROC) curve and the area under the ROC curve(AUC)were utilized as the global performance indices[56].The larger values of the above indices represent the corresponding model has a better predictive performance than other models.If the value is equal to 1, the model has achieved the highest prediction accuracy without zero error.For example,Kappa=0.4 is the standard line to distinguish the strength of agreement.It is generally believed that when Kappa is greater than 0.4, the strength of agreement is acceptable, and vice versa[57,58].Table 2 shows six ranges of the Kappa coefficient representing different consistency levels.Table 3 lists the evaluation standard of the AUC value.Gortmaker et al.[59]demonstrated that the AUC value of an outstanding classifier is above 0.9, being between 0.8 and 0.9 is excellent and being between 0.7 and 0.8 is acceptable.However, good global evaluation indices are not all captured by the best model for forecasting pillar stability [27].Therefore,other local performance indices such as precision,recall and F-measure have been successfully used to evaluate the model performance [4].In this paper, two global and three local evaluation indices [23,65,66] are defined using Eqs.(20)–(24).

Table 2 The basic scale of agreement with the Kappa coefficient.

where cii(i=1,2,...,h)represents the number of samples successfully predicted in each label;h the number of hard rock pillar labels;D the number of total samples; and c*iand ci*are the number of samples originally belonging to label i and the number of samples that were predicted to be in other labels, respectively.

6.Results and discussion

To predict the hard rock pillar stability, three hybrid models combining the metaheuristic algorithms and SVM have been considered in this investigation, i.e., GWO-SVM, WOA-SVM and SSASVM models.The same training set was used to build hybrid models and the test set was utilized to verify the performance of all models for predicting the hard rock pillar stability.Fig.6 briefly displays a flowchart of predicting the pillar stability with the developed models.The development and performance comparison results of all models are introduced and discussed thoroughly in the following sections.

Fig.6.A flowchart of predicting the hard rock pillar stability based on three hybrid SVM models.

6.1.Models development

6.1.1.Support vector machine

The prediction performance of the SVM model is directly related to the selection of structural parameters.First,there are three kernel functions named the polynomial kernel function, sigmoid kernel function and radial basis function (RBF) that can be used to improve the SVM performance.Compared with the other two functions, the RBF kernel function is less complex and more widely applicable [60].Therefore, a series of SVM models based on the RBF kernel function are established in this paper.In the RBF-SVM model, P and γ play a key role in the classification performance of the model.The value range of P is usually set inShariati et al.[47]set the maximum value of P as 210.Consequently, the range value of P is set to[2-2,210]in this paper.Meanwhile,the γ value is usually not greater than 10, and several related studies showed that the optimal γ values of most SVM models are no more than 1,but there are some more than 1 [23,25].Thus, the γ is set as 0.1,1.0 and 10 to simplify the modeling and get a good SVM model.According to this criterion,20 SVM models with different hyperparameter combinations were established to predict pillar stability,and the predictive performance was evaluated using accuracy.As illustrated in Table 4, the SVM model with P of 25and γ of 1.0 has the highest value of accuracy (84.7%) in the testing phase.However, the model classification performance does not simply increase with increasing hyperparameters,it is necessary to search for better hyperparameter combinations.

6.1.2.Development of hybrid SVM models using metaheuristic optimization

The population has a great influence on the running time of the hybrid model and searching for the global optimal solution, while the selection of this value is random.That means that a large population will directly cause the computation time to skyrocket.On the contrary, the hybrid model with too a small population cannot obtain the optimal solution after the iteration.To obtain the appropriate population,ten populations(20,30,40,50,60,70,80,90,100,200)were set to train each hybrid SVM model for predicting the hard rock pillar stability.Besides,the fitness function was represented by a global performance index named accuracy.Fig.7 illustrates the iteration curves of the proposed models with different populations during 100 iterations.As can be seen in these diagrams,the prediction accuracy of all models does not change after the number of iterations reaches 50,regardless of the number of populations.

Table 4 The performance results of SVM model in the testing phase.

Fig.8 displays the accuracy and computing time of three hybrid models during the iteration process.The results showed that the iteration time increases with the number of populations.As can be realized in the Fig.8a, the highest accuracy was achieved at the populations of 60 and 70 for two GWO-SVM models,but obviously, the iteration time of a small population number is shorter,which is more conducive to the overall analysis.Therefore,a model with a short computation time should be preferred when the prediction accuracy is equal to another model with a long computation time.These observations caused 60 wolves to be considered in the GWO-SVM model, 50 whales to be considered in the WOA-SVM model (Fig.8b) and 50 sparrows to be considered in the SSA-SVM model(Fig.8c).The results of accuracy and the optimized hyperparameters of SVM corresponding to each hybrid model are also listed in Table 5.

Fig.8.Iteration time and the accuracy in the development of three hybrid models.

6.2.Comparison of the proposed models for stability prediction

After determining hyperparameters(i.e.,population,P and γ)of the proposed hybrid models, each combination model starts to be continuously run and optimized,and the classification performance of the final model for hard rock pillar stability is evaluated by using global and local performance indices.To that end, the confusion matrix of each hybrid model was preferentially established in the training phase as shown in Fig.9,which is a reliable and important visualization tool to show the prediction results of the classification problems.In this diagram, the relationship between the true label and the predicted label can be represented by matrix values, that is, the larger matrix value corresponding to more labels with the true values equal to the predicted values,and the prediction model has better performance than others.As can be seen in this picture,the GWO-SVM model has a better performance in the stable pillar(label value:S)prediction by means of the larger matrix values(98)than other models.For unstable and failed pillars, the SSA-SVM model is believed to have a better predictive performance by resulting in larger matrix values of 34 and 69,respectively.

Table 5 Development results of three hybrid models.

Fig.9.Confusion matrix of each hybrid model in the training phase.

Furthermore, the results of two global indices (Accuracy and Kappa) and three local indices (Precision, Recall and F-measure)are shown in Table 6.As can be seen in this table, all models showed good predictive ability in the training phase.Among all the models, the SVM model without optimization has the worst performance indices, i.e., Accuracy of 78.037%, Kappa coefficient of 0.638, Precision of 79.328%, Recall of 71.015%, and F-measure of 71.315%, respectively.Other models have achieved better performance by resulting in higher values of the performance indices.In general, the SSA-SVM model was the best model with both the highest values of global and local indices (Accuracy: 93.457%,Kappa coefficient: 0.895, Precision: 93.011%, Recall: 91.020%, and F-measure: 91.667%) for predicting the hard rock pillars stability in the training phase.It can be clearly observed that GWO-SVM and WOA-SVM models have similar values of evaluation indices,which means that their predictive performance is close.After that,the ranking scores of all proposed models are demonstrated in Fig.10.This result indicated that the GWO-SVM model has a better performance than the WOA-SVM model by means of a higher score.As a result, the SSA-SVM has achieved the most satisfactory prediction performance by resulting in the highest score of 20 in the training phase.

Fig.10.The ranking scores of the proposed models using the training set.

Table 6 The performance evaluation results of all hybrid SVM models in the training phase.

To better evaluate the prediction performance of each model,the receiver operating characteristic (ROC) curve and the area under the ROC curve (AUC) value of each model in the training phase was drawn in Fig.11.The ROC curve farther from the diagonal has a larger value of AUC, which means a better performance is obtained by the model.As can be observed in this picture, the maximum AUC values of all models are above 0.9.Particularly,the SSA-SVM model has the largest values of AUC for the stability of pillars prediction, i.e., stable (AUC=0.996), unstable (AUC=0.978)and failed(AUC=0.995).After that,GWO-SVM and WOA-SVM have satisfactory values of AUC.However, only the SVM model has the minimum value of AUC (0.823) for the unstable pillar and its ROC curve is closer to the diagonal than other models, which further confirms the fact that the predictive performance of unoptimized SVM model is lower than other hybrid models in the training phase, especially the SSA-SVM model.

Fig.11.The ROC curves and AUC values of models in the training phase.

However, these trained models need to be verified for their excellent performance in the testing phase before applying in the pillar stability prediction.Fig.12 illustrates the confusion matrix of predictive performance for each hybrid model using the test set.As can be observed in this picture, the proposed models have achieved very satisfactory performance in the prediction of stable and failed pillars based on similar large matrix values (SVM: 49 and 26; GWO-SVM: 50 and 27; WOA-SVM: 50 and 27; SSA-SVM:50 and 27).Nevertheless, each model has a different performance in the stability of unstable pillar prediction.Obviously, the SSASVM model has a better performance by resulting in a larger matrix value of 7 among all models.

Fig.12.Confusion matrix of each hybrid model in the testing phase.

Table 7 shows the performance indices of each model using the test set.As demonstrated in this table, the SSA-SVM model still shows the most excellent indices (Accuracy of 91.304%, Kappa coefficient of 0.845, Precision of 93.667%, Recall of 82.666%, and F-measure of 85.333%) corresponding to the most promising predictive performance.After the SSA-SVM model, GWO-SVM, WOASVM and SVM models also have better performance in terms of the high values of the performance indices.The results showed that the optimized model still shows strong performance in the testing phase without obvious adverse phenomena such as overfitting.Fig.13 visually illustrates the ranking scores of the different models in each performance index.It can be clearly observed that all models have the same performance ordering in the testing phase as they did in the training phase.

Fig.13.The ranking scores of the proposed models using the test set.

Fig.14 further shows ROC curve and AUC value of each model in the testing phase.As can be seen in this picture, the SVM model still has obtained the worst performance by resulting in the lowest value of AUC (Label S: 0.941; Label U: 0.750; Label F: 0.978).By contrast, three hybrid models have better predictive performance and higher AUC values for predicting the hard rock pillar stability,especially for the unstable pillar (GWO-SVM: 0.847; WOA-SVM:0.839; SSA-SVM: 0.899).

Fig.14.The ROC curves and AUC values of models in the testing phase.

7.Validation

To verify the usability of the proposed models in underground mines, the stability of several practical pillar cases is predicted in this paper.It has to be admitted that failed and unstable pillars are not allowed to be approached and observed at will in regular mines.Thus, four failed pillars and one unstable pillar were obtained from the published studies [61,62].And the other four stable pillar cases and one unstable pillar case (Fig.15) were obtained from the Baishixi metal mine, Guizhou Province, China.Therefore, ten cases consisting of four stable pillars, two unstable pillars and four failed pillars were adopted to predict their stability using the proposed models.The validation results are shown in Table 8.It can be observed that the SSA-SVM model successfully predicted nine cases,including four stable pillars,one unstable pillar and four failed pillars.The GWO-SVM model and WOA-SVM model have obtained the same performance in stable pillar prediction, but the latter also incorrectly predicted one failed pillar as a stable pillar.Nevertheless,the unoptimized SVM model incorrectly predicted six pillars,especially for two unstable pillars.It should be noted that the three optimized SVM models also accurately predicted only one unstable pillar.On the whole,three hybrid models proposed in this paper have shown good adaptability and effectiveness in practical application, especially the SSA-SVM model.

Table 7 The performance evaluation results of all hybrid SVM models in the testing phase.

Fig.15.Description of a stable pillar case and an unstable pillar case.

In addition, the results of published studies on pillar stability prediction were compared with the proposed models in this paper,as shown in Table 9.Although the MLPNN model proposed by Tawadrous and Katsabanis [63] and the SVM model proposed by Zhou et al.[25]have presented high accuracy in the pillar stability prediction,the considered pillar labels are only stable and failed.In fact, unstable pillars are common in many underground mines with a long history and play an important role in actual underground stopes,it cannot be ignored in hard rock pillar stability prediction.Therefore,the proposed hybrid metaheuristic optimization based on the SVM models provided more accurate results than all previous works when unstable pillars were considered.It also proved that the proposed models are more acceptable and more reliable than previous AI models.

8.Summary and conclusions

Hard rock pillars always play an important role in protecting underground space stability in metal mines.The pillar stability prediction using AI techniques has been proven to be a reliable method.However,the performance of a single classifier for unbalanced data sets is not ideal, and finding a better model is of greathelp to predict pillar stability.This paper proposed three novel hybrid models by combining metaheuristic optimization algorithms (GWO,WOA and SSA)and SVM model to predict hard rock pillar stability.The main conclusions of this paper are as follows:

Table 8 Prediction results of 10 pillar cases.

(1) The performance comparison results showed that the SSASVM is the best model with the best performance indices in the training and testing phases, i.e., Accuracy of 93.457%and 91.304%,Kappa coefficient of 0.895 and 0.845,Precision of 93.011%and 93.667%,Recall of 91.020%and 82.666%,and F-measure of 91.667% and 85.333%).

(2) Compared with the accuracy of predicting stable and failed pillars, none of the three optimized models could well explain the relationship between input parameters and unstable pillars, although the SSA-SVM model was the best model with the highest AUC of 0.899 among them.

(3) Ten true pillar cases were utilized to verify the validity and adaptability of the proposed models.The validation results indicated that the SSA-SVM model is a reliable and highprecision prediction model.Besides, the SSA-SVM model provided a better performance than previous works for predicting the unstable pillar.

However, the problem of the accuracy of predicting the unstable pillar stability is not ideal compared with failed and stable pillars and should be taken into account in practical engineering and follow-up work.We believe that this phenomenon is caused by the data investigation of the unstable pillar.Since there is a certain error between the actual stable condition of the unstable pillar and the existing input parameters, it is necessary to add parameters that can better reflect its characteristics to improve the prediction performance.At present, we consider a couple of factors describing the quality of hard rock pillar mass such as rock mass rating(RMR)or the geological strength index(GSI)could be added to predict the hard rock pillar stability.

Acknowledgements

This research is partially supported by the National Natural Science Foundation Project of China (Nos.72088101 and 42177164), and the Distinguished Youth Science Foundation of Hunan Province of China (No.2022JJ10073).The first author was funded by China Scholarship Council (No.202106370038).