A. M. E. Ramírez-Mendoza

Abstract: A spike response model (SRM) based on the spikes generator circuit (SGC) of adaptive fuzzy spiking neurons (AFSNs) is developed. The SRM is simulated in MatlabTM environment. The proposed model is applied to a configuration of a fuzzy exclusive or(fuzzy XOR) operator, as an illustrative example. A description of the comparison of AFSNs with other similar methods is given. The novel method of the AFSNs is used to determine the value of the weights or parameters of the fuzzy XOR, first with dynamic weights or self-tuning parameters that adapt continuously, then with fixed weights obtained after training, finally with fixed weights and a dynamic gain or self-tuning gain for a fine adjustment of amplitude.

Keywords: Spike response model, spikes generator circuit, fuzzy XOR, adaptive fuzzy spiking neuron, learning algorithm, fuzzy neuron, self-tuning.

1 Introduction

The artificial neurons are mathematical models inspired by the functioning of biological neurons. These had been developed for several decades [McCulloch and Pitts (1943);Widrow, Lehr and Michael (1990); Gupta and Qi (1992); Gupta (1993); Gupta and Rao(1994); Pérez and Ramírez (2001); Ramírez and Pérez (2002); Ghosh-Dastidar and Adeli(2009); Pérez, Garcés, Cabiedes et al. (2009); Ramírez-Mendoza, Pérez-Silva and Lara-Rosano (2011); Ramírez-Mendoza (2014); Ramírez-Mendoza (unpublished); Ramírez-Mendoza (unpublished)]. Among the operations artificial neurons can perform with the purpose of imitating the neuronal characteristics are memory, learning, axonic delay time,the refractory period and the generation of spikes.

The models of fuzzy neurons are based on the human reasoning, the approximate reasoning and fuzzy logic [Zadeh (1977); Lara-Rosano (2017)]. Also, different models of neural networks and learning methods have been developed [Widrow, Lehr and Michael (1990);Gupta and Qi (1992); Gupta (1993); Gupta and Rao (1994); Pérez and Ramírez (2001);Ramírez and Pérez (2002); Pérez, Garcés, Cabiedes et al. (2009); Ramírez-Mendoza,Pérez-Silva and Lara-Rosano (2011); Ramírez-Mendoza (2014); Ramírez-Mendoza(unpublished)], as spiking neural networks [Ghosh-Dastidar and Adeli (2009); Chaturvedi(2011); Ramírez-Mendoza (unpublished); Ramírez-Mendoza (unpublished)], with applications in various areas.

The application areas are control [Yu and Rosen (2013)], instrumentation and measurement[Jiang, Guo and Zhang (2015)], electrophysiological recordings [Rossant, Goodman,Platkiewicz et al. (2010)], pattern recognition [Espinosa-Ramos, Cruz-Cortes, and Vazquez (2013)], and Drones and Induction Motors [Ramírez-Mendoza, Covarrubias-Fabela, Amezquita-Brooks et al. (2018)].

Also, neural networks (NNs) had been applied for stochastic analysis of composite materials for aeronautical and aerospace structures of aircrafts, UAVs, helicopters,improving the efficiency of the simulations of the effects of mechanical vibrations, instead of traditional methods such as the algorithm of the finite element [Tawfik, Bishay and Sadek (2018)].

Other application of artificial neural networks (ANNs) with an optimization algorithm is the prediction of the modulus of rupture values of glass fiber reinforced concrete panels as a preliminary criterion for quality check of fabricated products, the results presented in[Yildizel and Öztürk (2016)], exhibit ANNs with back propagation algorithm models have more fitting performance compared with a classic multiple linear regression (MLR) model.The spiking neuron models are popular for studies of memory, neural coding, neural network dynamics and reproduce spiking behavior of cortical neurons or fast spiking cortical interneurons [Gerstner and Kistler (2002); Izhikevich (2003)]. The simple SRM proposed here is like that of the nonlinear integrate and fire model, the parameters depend on the time since the last output spike, however, the formulation of the equations is different. The main contribution of this paper is a SRM based on a SGC [Ramírez-Mendoza,Pérez-Silva and Lara-Rosano (2011)], of Adaptive Fuzzy Spiking Neurons (AFSNs) with a learning algorithm developed in Ramírez-Mendoza et al. [Ramírez-Mendoza (2014);Ramírez-Mendoza (unpublished); Ramírez-Mendoza (unpublished)].

Spiking neural networks (SNNs) had been developed for imitating the behavior of biological neurons, the ANNs, are based on the training of the weights, but the SNNs also include a delay parameter, the signal delay from one neuron to another is different, giving more flexibility and an additional parameter for the training of the SNN, with applications in pattern recognition and image processing [Rosado-Muñoz, Bataller-Mompeán and Guerrero-Martínez (2012)].

The SRM of the AFSNs proposed in this work, presents also the neuronal characteristics such as the refractory time, axonal delay and the generation of spike trains. An important point is that neuronal characteristics such as refractory time and axonal delay are performed before the generation of the spike trains and therefore, the shape of the spikes remains along the neural network for neuronal coding [Ramírez-Mendoza, Pérez-Silva and Lara-Rosano (2011)].

Learning systems with a hardware spike timing dependent plasticity (STDP) learning rule applied to memristor-based NN, had been implemented in discrete-time where spikes are expressed as pulses, the neurons are synchronous with a global clock [Zheng and Mazumder (2018)], the results demonstrate the efficacy of the learning algorithms.

where X, Y are fuzzy sets of the universe of discourse U.

Many critics state that the witch and the stepmother actually represent the same character or threat to the children. The witch is an extension of the horrible stepmother. The death of the witch also means the stepmother is dead. Their deaths are simultaneous in the story. The stepmother wanted to kill the children so that they would not eat the food she needed to survive. The witch s purpose was even more malignant134. She wanted to kill the children so she could eat them as a delicacy, not even for survival since she had plenty food to sustain her.

The operator XOR is very important in the computational programming, it performs a primitive operation used in different encryption algorithms [Bedregal, Reiser and Dimuro(2009)]. Therefore, a fuzzy XOR is proposed as a primitive fuzzy operator to perform different configurations with AFSNs and thus model systems and processes.

As an illustrative example of the good functioning of the spike model, a configuration of a fuzzy XOR is proposed. The fuzzy XOR is very important for the comprehension and future applications of AFSNs in different areas of knowledge such as systems identification,experimental aerodynamics, industrial applications [Ramírez-Mendoza, Covarrubias-Fabela,Amezquita-Brooks et al. (2018)], process control in the design of control rules for proportional integral derivative controllers (PIDs) [Ramírez-Mendoza (unpublished); Ramírez-Mendoza(unpublished)], and navigation systems and trajectory tracking of low-scale unmanned aerial vehicles (UAVs) [Álvarez, Olascoaga, Rivera et al. (2017); Amezquita-Brooks, Liceaga-Castro, González-Sánchez et al. (2017)], among others. The following section presents a description of the AFSNs and the main contribution of this paper the modeling of the spike response of the AFSNs. The complete development of the AFSNs is presented in Ramírez-Mendoza et al. [Ramírez-Mendoza, Pérez-Silva and Lara-Rosano (2011); Ramírez-Mendoza(2014); Ramírez-Mendoza (unpublished); Ramírez-Mendoza (unpublished); Ramírez-Mendoza, Covarrubias-Fabela, Amezquita-Brooks et al. (2018)].

2 Adaptive fuzzy spiking neurons description

The AFSNs are fuzzy neurons with synaptic and somatic operations. Fig. 1 shows the block diagram of the AFSN with crisp or non-fuzzy inputs. The AFSN model has dendritic input signals zinj(k) ∈ [0,1] for unipolar signals, and zinj(k) ∈ [−1,1] for bipolar signals.

This was their daily routine after Grandpa’s latest stroke. Although once a very active man, his severely1 damaged left arm, difficulty walking and slurred2 speech now kept him housebound. For nearly a year he hadn’t even been to church or to visit family.

Figure 1: Block diagram of the AFSN model

2.1 Somatic aggregation operation

The somatic aggregation operation is performed according to Gupta [Gupta and Qi (1992);Gupta (1993); Gupta and Rao (1994)], as a Gupta fuzzy integrator (GFI) or generalized fuzzy OR of N inputs formed by N-1 fuzzy OR operators (MAX operators), Eq. (1).

Where:

k is the time variable.

• a=1

winj(k) are the synaptic weights.

• γAFSN4(k) = γAFSN5(k)= γAFSN6(k)=1,additional and independent single neurons

2.2 Non-linear somatic operation

For the non-linear somatic operation of the AFSN model, a sigmoid activation function with threshold and a learning algorithm were developed in Ramírez-Mendoza et al. [Ramírez-Mendoza, Pérez-Silva and Lara-Rosano (2011); Ramírez-Mendoza (2014); Ramírez-Mendoza (unpublished); Ramírez-Mendoza (unpublished)], the activation function is defined by Eq. (3),

• For bipolar signals,

Where

λ1(k) is a threshold signal.

−1≤γ≤1, is the somatic gain that determines the slope of the activation function, and the learning factor that regulates the speed of learning.

a is a real number, usually, a＞0.

• For AFSNs without input spikes, only no fuzzy inputs, and no learning algorithm,(k)is calculated with Eq. (4) andtrain(k) with Eq. (6).

Then she set to work, cutting out and sewing as best she could, and by the end of six days she had turned it into an elegant robe with a long train and a mantle21

Where:

λ2(k) is a threshold signal, for example, triangular shape.

Vclock(k) is the clock signal.

• For AFSNs with spikes as inputs, that is, pulses as inputs after a step function with threshold, and learning algorithm,(k) is calculated with Eq. (7).

8. Spinning-wheel: A spinning wheel is a small domestic spinning machine with a single spindle that is driven by hand or foot (WordNet).Return to place in story.

Where

Vref(k) is a reference voltage

Because of the spikes are bipolar signals fortrainbi(k),

It happened that where the Prince s father lay buried, there grew a rosetree--a most beautiful rose tree, which blossomed only once in every fiveyears, and even then bore only one flower, but that was a rose! It smelt2 sosweet that all cares and sorrows were forgotten by him who inhaled3 itsfragrance.

Where,

+V is a positive saturation voltage

A spike model is developed based on the SGC Fig. 1 and Eq. (11). The positive part of the spike model [Desoer and Kuh (1969); Esfandiari and Lu (2014)], is defined by the equations:

Vref(k) is a reference voltage

The error for the adaptive learning algorithm for AFSN model with a Gupta integrator, is defined in Eq. (10).

Where,

The learning algorithm for bipolar signals requires a reference input to each AFSN, the input reference value is the maximum value in the interval [-1, 1] for bipolar signals. The ideal values of the weights are unknown, even though, are calculated by the adaptive learning algorithm, based on proposed initial values.

2.3 Spike model based on the spikes generator circuit

Based on Eq. (9), the spikes are obtained according to Eq. (11), the mathematical model of the SGC [Ramírez-Mendoza, Pérez-Silva and Lara-Rosano (2011)],

-V is a negative saturation voltage

The negative part of the spike model [Esfandiari and Lu (2014)], is defined by the equations:

• Based on Eqs. (13) and (15), if only the spikes are available,rainbi(k) could be obtained with Eq. (16).

Where,

I believe I could graft5 on a pair of outer ears, if they could be procured, the doctor decided6. Whereupon the search began for a person who would make such a sacrifice for a young man.

The flakes8 of snow covered her long fair hair, which fell in beautiful curls around her neck; but of that, of course, she never once now thought. From all the windows the candles were gleaming, and it smelt9 so deliciously of roast goose, for you know it was New Year s Eve; yes, of that she thought.

k is the time variable [seconds].

I had bitten the youngest boy in the leg, because he took away the bone I was gnawing27; a bone for a bone, I thought! But they were very angry, and from that time I have been chained here, and I have lost my voice

R1,R2,R3are resistances [Ω].

C is a capacitance [F].

IDis the diode current [A].

The comparison between the traditional models of adaptive neurons, fuzzy adaptive neurons and fuzzy adaptive spiking neurons with the AFSN model is shown in Tab. 1.

I spent the rest of the afternoon at the park watching a very brave little girl overcome a fear, and congratulating myself for being a self-sufficient single parent.

Table 1: Comparison of Neuron Models

3 Results of the simulation of the Fuzzy XOR based on the AFSNs

In this section, a configuration for realizing a fuzzy XOR based on the AFSNs is proposed.To obtain the fuzzy XOR, it is proposed that the weights adapt continuously, that is,dynamically, all the parameters are adapted online. Also, with values of fixed weights obtained after training. Finally, with fixed weights and a dynamic gain for a fine adjustment of amplitude and a greater definition in the results. The simulation results of the spike model are presented in MatlabTMenvironment.

3.1 Fuzzy XOR configuration

As illustrative example, a configuration of a fuzzy XOR is proposed with base on the fuzzy logic, the equation that defines a non-fuzzy XOR is Eq. (17),

Expressing the Eq. (17), with fuzzy operations, Eq. (18),

A SNN third generation neural network model presented in Lashkare et al. [Lashkare,Chouhan, Chavan et al. (2018)], with hardware realization for high integration, the spike frequency depends on the input current. The spike trains frequency of the SGC presented in Ramírez-Mendoza et al. [Ramírez-Mendoza, Pérez-Silva and Lara-Rosano (2011)], is proportional to the amplitude of the response of the activation function of the AFSN.

The proposed configuration of a fuzzy XOR is based on the AFSNs, the block diagram is shown in Fig. 2.

Figure 2: Block diagram of the fuzzy XOR configuration

According to the proposed model of the AFSN [Ramírez-Mendoza, Pérez-Silva and Lara-Rosano (2011); Ramírez-Mendoza (2014); Ramírez-Mendoza (unpublished); Ramírez-Mendoza (unpublished)], the equations that determine the fuzzy XOR are:

Table 2: Configuration of the Fuzzy XOR operator

3.2 Simulation results

The results of the simulations for the fuzzy XOR configuration is presented, first with dynamic parameters, second with static parameters and in third place with the static parameters and a dynamic gain at the output of each neuron before generating the trains of spikes. The dynamic parameters of the fuzzy XOR, are determined by the learning algorithm of AFSNs in line. The static parameters of the fuzzy XOR, are previously determined by training the learning algorithm of AFSNs. The result of the fuzzy XOR with the configuration here proposed, then is applied a step activation function, for obtaining the spike trains for illustrating the SRM.

Then they carried Kay and Gerda first to the Finland woman, where they warmed themselves thoroughly14 in the hot room, and she gave them directions about their journey home

The input signals Vx(k) and Vy(k) for the example are non-fuzzy and are shown in Fig. 3.For the simulations, the sampling period is Tsampling=0.1 μs.

Figure 3: Input signals Vx(k) and Vy(k)

3.2.1 Fuzzy XOR configuration with dynamic parameters

The initial conditions for k=0 are,

The result of the simulationx⊕y(k) after 7 iterations, is shown in Fig. 4.

Figure 4: Simulation results of the fuzzy XOR configurationx⊕y(k) with dynamic parameters

3.2.2 Fuzzy XOR configuration with static parameters

I was one of the lucky ones; I could choose whether to work or not. By the time it was your turn, the glittering prizes of career advancement3 and a double income had lost their luster4. A splash in the puddles5 with you in your bright red boots or just one more rereading of your favorite book, Frog and Toad6 Are Friends, meant more.

The result of the simulation ofx⊕y(k) for the input signals Vx(k) and Vy(k) of the fuzzy XOR configuration with static parameters, obtained by training with three additional and independent neurons AFSN4, AFSN5, AFSN6, to determine the input weights winAFSNi(k),after 3 iterations, is shown in Fig. 5.The initial conditions for k=0 are,

zinj(k) are the dendrite inputs.

• GAFSN3=4.4047;XOR(k)=x⊕y(k)=sigmoid AFSN 3(k) ∗GAFSN3

• wrefAFSNi(k) = −1,k=0…n,fuzzy XOR configuration neurons

• winAFSNi(k) = 0,fuzzy XOR configuration neurons

• wrefAFSNi(k) =1,k=0…n,additional and independent neurons

• winAFSNi(k) = 0,additional and independent neurons

Gently, gently, said he, not so fast, my friend ; and standing still he waited till the beast was quite near, when he sprang lightly behind a tree; the unicorn ran with all its force against the tree, and rammed40 its horn so firmly into the trunk that it had no strength left to pull it out again, and was thus successfully captured

• VrefAFSNi(k)=1,k=0…n

• λ1AFSNi(k)= −1,k=0…n

• γAFSN1(k) = γAFSN2(k)= 1,k=0…n,fuzzy XOR configuration neurons

• γAFSN3(k)= γ3(k)=sigmoidAFSN2(k),fuzzy XOR configuration neuron

Now listen towhat I tell you! You can snore; you are snoring the whole night, and Ihardly a quarter of an hour! And the blood rose to the head of theexcited criminal; he threw himself upon his comrade, and beat him with his clenced fist in the face

Figure 5: Simulation results of the fuzzy XOR configuration x⊕y(k) with static parameters

The weights obtained for the fuzzy XOR configuration, after training are:

3.2.3 Fuzzy XOR configuration with static parameters and a dynamic gain in each AFSN

The result of the simulation ofx⊕y(k) for the input signals Vx(k) and Vy(k) of the fuzzy XOR configuration with static parameters previously obtained in Section 3.2.2, and a dynamic gain in each AFSN determined with an additional and independent neuron AFSN4,after 1 iteration, is shown in Fig. 6.

The initial conditions for k=0 are,

• a=1

• Tspike=Tclock=2 μs

• GAFSNi(k)=|winAFSN4(k)∗2.3|=0;sigmoidAFSNi(k)∗GAFSNi,i=1…3

• wrefAFSNi(k) = −1,k=0…n,fuzzy XOR configuration neurons

• winAFSNi(k) = 1,k=0…n,i=1…3,fuzzy XOR configuration neurons

• wrefAFSN4(k) = 1,k=0…n,additional and independent neuron

• winAFSN4(k) = 0,additional and independent neuron

• VrefAFSNi(k)=1,k=0…n,i=1…4

• λ1AFSNi(k)= −1,k=0…n,i=1…4

• λ2AFSN3(k)= λ32(k) is a triangular wave of 100 KHz of frequency, amplitude of 1.• γAFSN1(k) = γAFSN2(k)= 1,k=1…n

• γAFSN3(k)= γ3(k)=sigmoidAFSN2(k)

• γAFSN4(k)= 1,additional and independent neuron

Figure 6: Simulation results of the fuzzy XOR configuration x⊕y(k) with static parameters and a dynamic gain in each AFSN

3.2.4 Results of the simulation of the Spike trains with the Spike model

Based on the results obtained from the fuzzy XOR configurationx⊕y(k) with weights or static parameters and a dynamic gain in each AFSN, the spike trains will be generated agree with the spike model proposed in equations Eqs. 35 and 36.

Spike model with constant IDdiode current,

• The values of the capacitors and the resistors are:

R1= 100 KΩ

R2= 100 KΩ

R3= 110 Ω

C1=2.64 nF

• The values of the parameters of the diode are:

VD=0.7 V

K=11600

TK=298 K

ID=10 mA

• It is considered an ultra-fast high conductance diode with a reverse recovery time of 4 ns, also a high-speed precision operational amplifier with a slew rate of 50 V/μs.The simulation results are shown in Figs. 7 and 8 for the input Vx(k) and Vy(k) of Fig. 3.Fig. 9 shows the results for the fuzzy XOR of the input Vx(k) and Vy(k) of Fig. 3, but Vy(k)= Vx(k).

Figure 7: Simulation results of the spike model for the fuzzy XOR configuration x⊕y(k)with static parameters and a dynamic gain in each AFSN

Figure 8: Simulation results of the of the spike model

Figure 9: Simulation results of the spike model for the fuzzy XOR configuration x⊕y(k)with static parameters and a dynamic gain in each AFSN, for the input signals Vx(k)=Vy(k)

Spike model with dynamic IDdiode current,

• The values of the capacitors and the resistors are:

R1= 1 KΩ

R2= 1 KΩ

R3= 100 Ω

C1=2.2 nF

• The values of the parameters of the diode are:

VD=0.7 V

K=11600

TK=298 K

Is=0.015 pA

• It is considered an ultra-fast high conductance diode with a reverse recovery time of 4 ns, also a high-speed precision operational amplifier with a slew rate of 50 V/μs.The simulation results are shown in Figs. 10 and 11 for the inputs Vx(k) and Vy(k) of Fig.3. Fig. 12 shows the results for the fuzzy XOR of the inputs Vx(k) and Vy(k) of Fig. 3, but Vy(k) = Vx(k).

Figure 10: Simulation results of the spike model for the fuzzy XOR configuration x⊕y(k) with static parameters and a dynamic gain in each AFSN

Figure 11: Simulation results of the of the spike model

Figure 12: Simulation results of the spike model for the fuzzy XOR configuration x⊕y(k) with static parameters and a dynamic gain in each AFSN, for the input signals Vx(k)=Vy(k)

4 Conclusions and comments

A SRM was obtained based on the spike generator circuit [Ramírez-Mendoza, Pérez-Silva and Lara-Rosano (2011)], this model describes the response behavior of the spikes inspired by biological spiking neurons.

One of the great advantages of the spike response model is that the process of information of the time firing pattern or pattern of spike firing with respect to time, is performed based on the trains of pulses obtained from the square pulses of the step activation function,proportional in pulse width to the MAX Operator response of the inputs to the neuron,weighted with the MIN operator. That is to say, the neuronal characteristics such as the refractory time, the axonic delay and the generation of spikes, are performed based on bipolar pulses, so that the information of the pattern of time spike firing is preserved despite the aforementioned neuronal characteristics, also the shape of the response spike model for each pulse. It should also be noted that the frequency of the spike trains are directly proportional to the width of the pulses obtained from the step-type activation function and therefore to the Gupta-type fuzzy integral (MAX operator generalized), of the neuron inputs weighted by means of the synaptic MIN operator.

However, a disadvantage of the AFSNs method could be that many systems or industrial processes have inputs or parameters that do not fit the fuzzy values within the interval [0, 1]for unipolar AFSNs, or [-1, 1] for bipolar AFSNs. The solution to these challenges has been to scale the values of the inputs to carry out the information process in a fuzzy way and then,the output response is rescaled to the interval of values of the system or industrial process[Ramírez-Mendoza, Covarrubias-Fabela, Amezquita-Brooks et al. (2018)].

A potential issue for fuzzy XOR connectives is in quantum computing [Ávila, Schmalfuss,Reiser et al. (2015)], however, systems modeled exclusively with AFSNs should be able to be performed based on basic fuzzy logic operators such as MIN, MAX, INV and the XOR fuzzy operator, similarly as the systems based on classic or non-fuzzy logical operators, INV, AND, OR, XOR; which represents a challenge because many linear and non-linear systems models require other types of mathematical operations such as trigonometric, exponential, derivative and integral.

The SRM and the AFSNs method will allow more precise simulations with applications in several areas such as process control, instrumentation, experimental aerodynamics,navigation and trajectory tracking of UAVs, among others. The proposed example, the fuzzy XOR, demonstrates that the basic fuzzy logic operators can be implemented with the AFSNs, and the results of the simulation show that the dynamic gain or self-tuning gain,of the curve fitting can be used for systems modeling optimization applications and computational modeling [Dutta, Murthy, Kim et al. (2017)].

Acknowledgement:This work was supported by UNAM and CONACYT Research Fellows-UANL, FIME, CIIIA, program. I also appreciate the valuable comments of Eduardo Liceaga-Castro for the writing of this article.