Jian-Wei Wang(王建伟), Rong Wang(王蓉), and Feng-Yuan Yu(于逢源)

School of Business Administration,Northeastern University,Shenyang 110819,China

Keywords: public goods game,evaluation effect,fitness,network reciprocity

1. Introduction

“The tragedy of the commons”theory was put forward by Harding shows us well that human beings are overusing public resources,leading the resources to be exhausted and collapsed eventually due to the conflict between self-interest and collective interest.[1]However,fortunately,we can see that humans work together to solve the collective problem such as environmental governance,public resource protection,global climate change and others. This seems contrary to the statement that rational individuals choose to defect rather than cooperate in order to maximize their own interests. These universal cooperative phenomena indicate that there are altruistic behaviors among self-interested individuals. At the same time, China also regards cooperation as the basic guarantee to promote sustainable development,understanding cooperation behavior has become one of the important scientific issues concerned in many fields. Many scholars have carried out a large number of behavioral experiments[2–5]and theoretical studies[6–9]in order to explore the causes of cooperation and the mechanisms that can promote cooperation.

Nowak and May initially explored the spatial patterns of individuals on two-dimensional crystal lattice.[10]This pioneering work attracted many scholars to pay attention to spatial topology,then they found that characteristics such as heterogeneity and network structure can promote the cooperation to occur.[11–14]Meanwhile, Nowak came up with five rules for the evolution of cooperation,i.e., kin selection, direct reciprocity, indirect reciprocity, network reciprocity and group selection.[15]Furthermore, researchers have proposed a large number of mechanisms that promote cooperative behavior in recent years, such as reputation,[16–18]rewards and punishments,[19–22]popularity,[23–25]conformity effect,[26–28]to name but a few. Evolutionary game theory based on bounded rationality as a powerful mathematical framework is used to study why individuals have altruistic behavior and how such a behavior sustains.[29–37]The public goods game is considered as a typical evolutionary game model to study the problems of public resources and public facilities with group interaction. In the traditional public goods game(PGG),every individual has two strategies: cooperation (C) and defection(D).In each round,individual receives a fixed incomec,cooperators who are also known as contributors donate all income to the public pool, but defectors contribute nothing, who are also called free-riders or non-contributors. Then,the cumulative contributions made by all participants to the public pool are multiplied by an enhancement factor,and evenly distribute them among all individuals no matter what the initial strategy is. For 1

In the process of strategy learning, realizing the maximum of profit which is a classic mechanism has always been regarded as an individual’s learning purpose. In this process,each individual focuses only on his own payoffs and aims at maximizing benefits. In real life, however, this is not always the case. Individuals generally evaluate the behavior of others,and people also pay more and more attention to the evaluation of their own behaviors by others. Comparing with the traditional game, when making strategic choices, people’s attention is not only limited to the aspect of payoffs, but more likely to focus on others’trust in themselves and their fitness in the society, in the sense of justice, responsibility, conformity,stubbornness and other psychological emotional aspects.Szolnokiet al.considered the“wisdom of crowds”(group interaction)factor when adopting different strategies in the process of studying the evolution of social dilemma game cooperation. They found that “wisdom of crowds” can strongly promote cooperation, and the wider the scope of consideration,the more impaired the evolution of defectors is.[38]Liet al. studied how individual’s popularity affects the evolution of cooperation, and integrated the individual popularity into the calculation of fitness by using a single parameter. Research shows that the personal popularity is a key factor in promoting cooperation.[25]Javaroneet al. investigated the spatial public goods game driven by fitness and conformity. They believed that the evolution of a population may also be influenced by social behavior, and that the probability of an individual to adopt strategy depends on both payoff-driven rule and conformity-driven rule which means that the individuals imitate the strategy adopted by most neighbors. Through numerical simulation,they found that the conformism generally fosters the ordered cooperative phases and may also lead to bistable behaviors,[39]which was also studied at the same time by Yanget al. from the perspective of individual’s conformity psychology through taking payoff-driven strategy updating rule and conformity-driven strategy updating rule as two attributes of individuals. They studied the evolutions of the two attributes by introducing persistence parameter and found that the frequent alternation of these two different updating rules can enhance network reciprocity and promote cooperation.[40]Quanet al. thought that the reputation of an individual will affect the decisions of the neighbors in the process of strategy updating. They proposed an improved strategy learning rule where both reputation and payoff information are considered, and then used the evidence theory to fuse these two aspects of information. By constructing a weighting coefficient to quantify the importance and reliability of reputation it is revealed that the reputation effect can greatly attract neighbors to form larger clusters, thereby promoting the cooperation to occur.[41]Yanget al. investigated the propagation of cooperation in the conformity-driven dynamic social network

where a player adjusts a social tie sometimes in accordance with a rival’s strategy popularity. Their research shows that conformity-driven linking dynamics can not only directly cut off the channel by which defection always takes a free ride,but also dramatically foster the formation and stabilization of the pure strategy group whose members are carrying the same strategy.[42]

Therefore,we regard payoff-driven and fitness-driven updating rules as two attributes of individuals, and believe that individuals’motivation is not only to pursue high payoffs but also to pay attention to the effect of their own behaviors on themselves in this era of “individuals generally evaluate behavior of others” when making strategy choices. Under the stimulation of these facts, we study a public goods game model by considering the effect of others’ evaluation, individual’s payoffs and others’ evaluation on his behavior constituting the fitness of the individual. At the same time, the evaluation of individual behavior is related to the environment in which the individual is located. When individual is in a group with more cooperators, there will less positive evaluation on him if he cooperates with them, on the contrary, the negative evaluation on him will be relatively more if he defects from them. We think that this kind of calculation rule is more in line with the actual situation,because in real life it is often the minority that get more attention. Here we introduce a valuation coefficient 1/αto further study the role of considering others’ evaluation effect in cooperating and how the proportion of fitness-driven individualspinfluences the evolution of cooperation. A large number of simulation results show that the fitness-driven strategy updating rule with considering the evaluation effect of others can enhance network reciprocity and significantly promote cooperation.

The rest of the article is organized as follows. In Section 2, we describe the traditional PGG model and the public goods game model with considering the evaluation effect of others. In Section 3 the corresponding simulation results are presented and the influence of others’evaluation effects on cooperation is further analyzed. Finally, some conclusions are drawn in Section 4.

2. Model

2.1. Payoff-based learning

We study the public goods game onL×Lregular lattice with periodic boundary conditions in which nodes represent players and the edges between nodes refer to the interactions of players, players have two strategies to choose: cooperate(Si=1)or defect(Si=0). Cooperators invest costc(c=1)to public goods and obtain investment income.Defectors as freerider,contribute nothing but gain the same benefits as cooperators. In such a structured population, each individual plays games with his four neighbors,so each playeriparticipates inki+1(kiis the degree of playeri,ki=4)group games at the same time,the player is centered on himself and hiskineighbors respectively. The payoff of playeriparticipating in the game group centered on playerjis

whereΩjis the set of neighbors around playerj,i=jmeans that individualiparticipates in the game of a self-centered team.

A player’s payoffs are the cumulative payoffs generated by all PGGs centered on his four neighbors and himself.Therefore, the total payoffs of individualican be expressed as

When playeriupdates strategy, he will randomly select a neighborjfrom his four neighbors to learn strategy and adjust his own strategy according to the difference among their payoffs,

whereKrepresents environmental noise. As done in most of previous studies,Kis set to be 0.1 here in this work.[43–47]

2.2. Fitness-based learning

With the constant development of the society,people pursue their own sense of value in more diversified ways. Besides payoffs they get, they also focus on others’ evaluation on themselves, their reputation, etc., because in the long run,a good personal character will gain more payoffs in the future,so player who holds fitness-based learning strategy will pay attention to others’ evaluation on himself in the game. Beyond all doubt,cooperative individuals can get positive evaluation,otherwise defective individuals get negative evaluation.In our model setting, others’ evaluation on oneself is related to the environment he is in, and the degree of evaluation is marginally decreasing according to the environment, which,we think,is in line with the reality. For example,if individualiis a cooperator, the positive evaluation will decrease as the number of cooperators in the team increases,that is to say,the positive evaluation obtained from groups with more cooperators will be lower than from the groups with more defectors;andvice versa. Because from a psychological point of view,people tend to be more interested in evaluating the behavior of the minority. So the fitness of player who holds fitness-based learning strategy is given as follows:

whereNiis the number of individuals in the team who adopt the same strategy as playeri,NCrepresents the number of cooperators in the team, andNDrepresents the number of defectors in the team. As shown in Fig.1, in the team centered on playerm, for playeri,Ni=3, for playerj,Nj=2; in the team centered on playern, for playeri,Ni=2, for playerj,Nj=3. Theα>1.3 is required in order to satisfy the condition that defectors’ payoffs are more than cooperators’ in the public goods game. WhenNi=5, all players in the team are cooperators or defectors, the model is transformed into traditional PGG.

Fig.1.Schematic diagram of space public goods game,with red representing cooperative individual and blue referring to defective individual. For interpretation of the references to color in this figure legend,the reader is referred to the web version of this article.

The fitness of a player is the cumulative fitness of all PGGs centered on his neighbors (including himself). Therefore,the total fitness of playerican be expressed as

whereKrepresents environmental noise.As done in most previous studies,Kis set to be 0.1 here in this paper.[43–47]

In the population,we regard the payoff-driven and fitnessdriven updating rules as two attributes of players,i.e.individuals with a proportion ofpfor fitness-driven strategy learning and individuals with a proportion of 1−pfor payoff-driven strategy learning.

Because our simulation is conducted on a lattice network,all participants have the same number of neighbors. Therefore,we introduce the enhancement coefficientηandηis expressed as

3. Results and analysis

Leveraging the computer programs coded by Python,we simulate the cooperative behavior in the evolutionary process of the population on a 100×100 square lattice. Players with equal probability cooperation or defection strategy are randomly distributed in the population, and all players interact with their nearest four neighbors. The level of cooperation is characterized by the fraction of cooperatorsfcwhich is the proportion of the number of cooperators in the population in the steady state. In order to make the results stable and effective,fcis calculated by using the average result of the last 2000 time steps in 104Monte Carlo steps(MCSs)and all the following results are obtained with an average value of 20 independent simulation processes.

Fig. 2. Variations of fraction of cooperators fc and the level of cooperation with enhancement coefficient η for different values of α, with red, blue, green, and purple curves representing the cases of α =1.5, 2, 2.5, 3, respectively, black curve referring to the case of traditional public goods game,proportions of fitness-driven players p being set to be 0.3,0.5,0.7,and 1,respectively,and other parameters being set to be MCS=104,K=0.1. For the interpretation of the colors in the legend of this figure,readers may refer to the web version of this article.

Obviously, our model is controlled by three parameters,so we next study how the enhancement coefficientη,the evaluation coefficient 1/α(for convenience, we take the value ofα)and the proportion of fitness-driven playerpaffect the evolution of cooperation. To begin with, we present the fraction of cooperatorsfcas a function of the enhancement coefficientηat different values ofαfor different values ofpin Fig.2,we select four representative values ofp=0.3,0.5,0.7,1,corresponding to Figs.2(a)–2(d),respectively,and the values ofαare 1.5,2,2.5,and 3,respectively.We can find that the fraction of cooperatorsfcincreases with the enhancement coefficientηincreasing. Compared with the traditional PGG(black curve),the presence of the evaluation coefficient 1/αcan effectively improve the cooperation level of the population. Moreover,for the case of more and more fitness-driven players, the degree of cooperation is higher and the effect of solving social dilemma is more significant. In Fig.2,for the traditional PGG curve, there are three conversions in the cooperative change rate, namely, fast-slow-fast-slow. In Figs. 2(a) and 2(b), the cooperation rates of most of curves are similar, but the level of cooperation is improved and can generate cooperation at a smallerr. Whenp ≤0.5 andr>3.7,due to the fact that the number of fitness-driven individuals is small and the enhancement coefficient is large,the differences in fitness obtained by individuals among different evaluation coefficients are small,there is no significant difference in the level of cooperation for different values ofα,and the evaluation coefficient has little influence on cooperation. It can be seen from Figs. 2(c)and 2(d)that whenp>0.5,the difference in cooperation level among different values ofαgradually increases with the value ofpincreasing;whenα=1.5,cooperation can occur even ifηis extremely small. Figure 2(d)shows the proportions of cooperators when all individuals in the population adopt fitnessdriven strategy updating rule. Whenη=0.28,the number of cooperators in the group is bigger than that of defectors; the rate of change of cooperation in Fig. 2(d) is also the fastest and the state of full cooperation can be reached faster than in other situations.

Fig. 3. Under different values of α, Variations of fraction of cooperators fc and cooperation level with fitness-driven player proportion p,with red, blue, green, and purple curves representing the cases where α =1.5, 2, 2.5, and 3, respectively, with values of η from left to right being 0.6,0.7,and 0.8,respectively, p=0 being the classical case without evaluation effect,and other parameters being set to be MCS=104,K=0.1. For interpretation of the color in legend of this figure,readers may refer to the web version of this article.

Fig.4. Variations of proportion of cooperators with time for different evaluation coefficients,with parameters being set to be[(a),(b)] p=0.5,η =0.64, 0.8, and[(c), (d)] p=0.7, η =0.64, 0.8, and panels(b)and(d)also show the detailed evolution process of the last 2×104 steps when α is small. For the interpretation of the color in legend of this figure,readers may refer to the web version of this article.

Figure 3 shows the levels of cooperation corresponding to different values ofηunder combination of parameterspandα. The fraction of cooperatorsfcincreases as the proportion of fitness-driven individualspincreases, which indicates that the more the individuals in the group who consider evaluation of others, the better the cooperation effect is. Through comparing different values ofη, we can see more intuitively that for weaker dilemmas(Fig.3(c)),different values ofαhave little effect on the level of cooperation,and the effect of others’evaluation on the promotion of cooperation is limited. So the level of cooperation is determined mainly by the number of fitness-driven players. However,for stronger social dilemmas,the larger the evaluation coefficient, the greater the influence on cooperation is and the more significant the effect of cooperation. The results in Fig.3 are consistent with those in Fig.2,and the two figures clearly show how the three parametersη,α,andpaffect the evolution of cooperation.

From the detailed temporal dimension, figure 4 shows the fluctuation of cooperators in the overall process of evolution from the initial state to the final stable state. We can find that the fraction of cooperatorsfcfirst drops sharply to a fairly low level in a very short period of time, and then rises slowly from the bottom to varying levels, until it finally reaches different degrees of stability according to the evaluation coefficient. Through the horizontal comparison,the curve with high enhancement coefficient is always better than that with lowη, and the influence ofαon cooperation is effective within a certain range ofη, which is consistent with our conclusion obtained above. Through the longitudinal comparison, curve with high proportion of fitness-driven individuals is always better than that with lowp. From the perspective of time evolution,the four curves have significant differences(e.g. Fig. 4(c)), except for the scenario ofα=3 where thefcdecreases to zero slowly. We notice that the stronger the evaluation effect, the higher the lowest level of cooperator density is and the steeper the curve in the ascending stage.Whenα=2.5,the density of cooperators drops rapidly to an extremely low level and then it takes a long relaxation time to reach the stability stage. Therefore, in the case of strong dilemma,the level of cooperation under the strong evaluation effect can be improved dramatically in a nonlinear form. In the weak dilemma where the enhancement coefficientη=0.8(as shown in Figs. 4(b) and 4(d)), the weak influence of the evaluation effect is further verified from the dimension of the temporal evolution.

Fig. 5. Snapshots of spatial temporal distribution of cooperators and defectors at t =0, 101, 102, 104, where yellow represents the payoffdriven cooperators, red refers to the fitness-driven cooperators, gray denotes the payoff-driven defectors, blue is the fitness-driven defectors.The selected representative parameter values are[(a), (b)]α =2 and 1.5, p=0.5, and η =0.6; [(c), (d)]α =2, p=0.8, and η =0.8. For interpretation of the color in legend of this figure,readers may refer to the web version of this article.

Next,we explore the evolution process of the spatial distribution of cooperators and defectors from microscopic level.Figure 5 shows the snapshots at timet=0,101,102,104. We select representative parameter values for comparison, which can highlight the influence of different parameters on cooperation, it also covers all typical situations, including low-level cooperation and even the eventual disappearance of cooperators,the persistence of cooperators and defectors indefinitely,develop into a prosperous cooperation phenomenon. An equal number of cooperators and defectors are randomly distributed on the lattice initially and relatively scattered, in the first few dozens of MCS time steps, due to the dispersion of cooperators, a large number of defectors quickly infiltrate into the group of cooperators: the cooperators are weakened and reduced. As the evolution continues, the surviving cooperators form a cluster,and help each other to resist the invasion of defectors. Since the cooperators at the boundary ofC-cluster can obtain more benefits than the defectors at the boundary ofDcluster, this is conducive to the diffusion of cooperators. For Figs.5(a)and 5(b), we select a serious cooperation dilemma,and we can see that with the evolution going on, cooperators in Fig. 5(a) are quickly diluted into small clusters until cooperators disappear, but with a stronger evaluation effect(see Fig. 5(b)), cooperators can resist the invasion of defectors, forming small clusters and spreading slowly, this means that the evaluation effect is conducive to the generation of cooperation. Compared with larger enhancement coefficient(Figs.5(a)and 5(c)),the cooperators in Fig.5(c)not only survive but quickly dominate the population, which means that although the evaluation effect is favorable for cooperation,the cooperation will not occur unless the enhancement factor is large enough. The scenarios with different values ofPare shown in Figs.5(c)and 5(d). Obviously,the largerPis more conducive to the evolution of cooperation, andC-clusters in Fig. 5(d) become larger and more compact at timet= 10.Moreover, the diffusion ofC-clusters is accelerated, and finally the state of complete cooperation is reached.

Fig. 6. Variations of fraction of cooperators fc with enhancement coefficient η at different values of α for different vaues of p on WS network, ER network,and BA network: [(a),(b)] p=0.3,0.7 on WS network,respectively,[(c),(d)] p=0.3,0.7 on ER network,respectively,and[(e),(f)] p=0.3,0.7 on BA network, respectively. The parameters of WS, ER, and BA networks are set to be node size N =104, average degree of nodes k=4, and probability of broken edge reconnection of WS small-world network 0.01. Other parameters are MCS=104 and K=0.1. For interpretation of the color in legend of this figure,readers may refer to the web version of this article.

We have explored the influence of evaluation effect on cooperation in regular lattice networks. As is well known,the evolution of cooperation is also affected by the network topology,such as ER random network,BA scalefree network,WS smallworld network, and interdependent network. Therefore,we finally test the robustness of the above results to the networks with different structures. In order to avoid influencing the node sizes and node degrees of different networks,we construct ER random network, BA scale-free network, and WS small-world network with the same parameters as regular lattice network, that is, the numbers of individual nodes of the three networks are all 104,and their average degree of network nodes are all 4, and the probability of broken edge reconnection of the WS small-world network are all 0.01. Figure 6 shows the results of the WS smallworld network, the Erd¨os–R´enyi(ER)random network and BA scale-free network.It can be observed that full cooperation can be achieved in a smallerηin the WS and ER networks than in the regular lattice networks. As the heterogeneity of network is conducive to cooperation, for BA network which has strong heterogeneity, can have an excellent effect on promoting cooperation. The cooperation effect on the BA network is the strongest among the three kinds of network structures. No matter what kind of structure the network possesses, the effects ofpandαon cooperation are similar to those on the square lattice in Fig.2,when the number of fitnessdriven individuals is small,the evaluation effect can only have a small influence;with the increase of fitness-driven individuals in the population,the role of evaluation effect increases: the higher the evaluation coefficient,the better the cooperation effect is. This is consistent with the main conclusion drawn from Fig.2.

4. Conclusions

In this work,we study the influence of others’evaluation effects on cooperation by introducing fitness-driven individual attribute as one of the strategy updating rules. In the model,in addition to the payoff-driven strategy updating rule,we introduce fitness-driven individual attribute,and find that the cooperation can be improved when there are fitness-driven individuals. In the calculation of individual’s fitness we take the influence of others’ evaluation effects on the individual into account due to the fact that the public goods game is the interaction between groups. Considering that the actual situation that others’evaluation on individual’s behavior is related to the environment around the individual,we combine the number of individuals in the team who adopt the same strategy as playeriwith the evaluation coefficient to calculate the fitness of playeri. The simulation results of the regular grid network show that whenris small,that is,in a strong dilemma,the higher evaluation coefficient can reduce the fitness difference between cooperators and defectors, so that the cooperation dilemma can be weakened,in particular the growing number of individuals who adopt fitness-driven updating rule can bring more flourish cooperations. However,with the increase ofr,the influence of different evaluation coefficients on cooperation is not significant in the case of smallP.In addition,the increase of both the enhancement factor r and the fitness-driven individual proportionpcan effectively improve the level of cooperation. When all individuals in the population adopt the fitness-driven updating rule to learn and update the strategies, the cooperation is greatly promoted by the evaluation effect of others and the cooperators can occupy the entire population quickly. We also analyze on a microscopic scale that cooperators can form close clusters in the process of evolution to avoid being invaded by defectors. Besides the square lattice, we also investigate the model on WS,ER,and BA networks,and find that the results remain unchanged qualitatively and thus verify the robustness of the model. Finally, we hope that our research can provide an insight into the emergence of cooperation from the perspectives of individual’s own emotion, environment and updating rule.