Informacije MIDEM 24(1994)2, Ljubljana UDK 621.3:(53+54+621 +66), ISSN0352-9045 TIME PROPERTIES OF THE NEURAL NETWORK CELLS Igor Belič VŠNZ, Ljubljana Keywords: neuronics, neural networks, neural cells, time properties, neural cell models, neural systems, system responses, time constants, input signals, time functions Abstract: The article describes the basic neural cell model. The time properties of the single neural cells are introduced and the influence of the various time constants of the neural cells on the neural system response is studied. The different time constants of the neural cells lead to the completely different time responses of the system. It is clear that the neglecting of the cell time constants can lead to the completely unpredictable results when the analogue neural cell models are used. Časovne lastnosti nevronskih celic Ključne besede: neuronika, omrežja nevronska, celice nevronske, lastnosti časovne, modeli celic nevronskih, sistemi nevronski, odzivi sistema, konstante časovne, signali vhodni, funkcije časovne Povzetek: Prispevek opisuje osnovni model nevronske celice. Študiran je vpliv različnih časovnih konstant nevronskih celic, s katerimi se le te odzivajo na vhodne signale, če upoštevamo različne časovne konstante celic v nevronskih sistemih, potem se sistem pri spreminjanju časovnih konstant, povsem drugače odziva na enake vhodne signale. Zanemaritev različnosti časovnih konstant nevronskih celic lahko vodi pri analognih nevronskih sistemih do nepredvidljivih posledic. Pri digitalnih simulacijah ta zanemaritev ni tako pomembna. NEURAL CELL MODEL So far it is not completely understood how biological neurones functionate. Only the complete knowledge of neural cells functionality makes it possible to develop the adequate artificial model of the neural cell. Si o S3 o-lW2 XnO—lW. Fig. 1: Model of the neural cell It is also very important that the neural cells are understood as the elements that functions only in the interaction among the vast number of neurones. One neurone alone has no sense. The artificial neurone is the element with following properties: from the external or internal (within the neural system) source. 2. Neurone summates the input signals. Prior to summation, input signals are multiplied by the adequate weight constants Wi. Therefore neurone performs the cumulative weighted sum of the input signals. 3. Neurone has only one output. The output value is calculated with the so called activation function f(u). It performs the mapping of the cumulative sum into the output cell value. /(») 1 1 0 u —► -1 1. It receives the set of input signals denoted by xi; (i = 1,2, ..,n). It is an n input element. Input signals can come FiQ- 2: Cell activation function U = X Wi Xi - h (1) i=i Xi - signal on the i - th cell input Wi - weight on the i - th cell input h - cell activation threshold The function f(u) is typically some continuous non-linear function. In most cases the following function is used: f(u) = tanh(Xu) (2) lambda - the activation function slope parameter Time Properties of The Cell Cumulative Sum Formation The input signals cumulative sum development can be described with the differential equation x^=-u(t)+irwixi-h^ i = 1 ^ (3) Fig. 3: Transition of the cell output state from the value AtoB The cell output state transition is represented by the function plotted on the fig. 3. Changing the cell time constants together with the cell input weights adaption is the process of learning. The effect of the cell time constants becomes far more important in a very large neural networks with feedback loops. T - Cell time constant. The solution of this differential equation is the group of parametric functions of the type Examples u(t) = Ce' + K where n K=xrwiXi-h i=i ^ (4) (5) The solution of the differential equation tells that the neural cell responds to the input signals with a time delay proportional to the cell time constant x. In the literature authors usually make some simplifications of the neural cell models in order to make the computations simpler and faster. One of the commonly accepted simplification sets the value of x to 0. The first consequence is that artificial neural cells summate the input signals without the time delay. This simplification neglects the fact that the time properties of the neural cells can be different. The difference in cell time constants introduces the time dependant response in the neural networks and the adaptation of the cell time constants can be denoted as the learning of the time response. Following is the example of the time response for the small neural network system where the cells have the different time constants. y y Fig. 4: Test neural network system The strict proof of the stated effect can only be done with the mathematical tools. Let us make only a short example where the small neural network system is observed. The test neural network system consists of three input cells, two hidden cells and one output cell. The output time function was observed forthe various input signals, for the different weight and time constant sets. Case 1: Inputs signals z and y are sinusoidal functions. 1 111 \ ^ \ \ 1 i 1 i ? ili 'i I«U 3 JJ H:Yi Fig. 5: Input signals Fig. 7: Output time function for randomly chosen time constants with weight set 2. (1) - t J] T«u 4 J-« !J1 Tl * fi-F' "x / Fig. 6: Output time function for randomly chosen time Fig. 8: constants with weight set 1. Output time function for randomly chosen time constants with weight set 3. Case 2: Input signals z and y are impulses. \ v. \ t' 1» / / \ // \ ! // \ J I \ Fig. 9: Input signals Fig. 11: Output time function for randomly chosen time constants with weight set 5. ■ A " / N. 3 "--t--r- H;: Fig. 10: Output time function for randomly chosen time constants with weight set 4. Fig. 12: Output time function for randomly chosen time constants with weight set 6. From the examples it is perfectly clear that the neural cell time constants can contribute to the different system time responses. At this point the architecture of the neural networks can be proposed in the way described on the following figure. Pi, Pa,." Pn - Time independent neural network subsystems Fig. 13: The structure of the neural network system with the time dependant and time independant subsystems. Of the neural network is strong when the large, analogue systems with feedback connections are used. In such systems the neglecting of the fact that cells have different time constants can lead to unpredictable system time responses. Furthermore when different cell time constants are used, the learning process consists of two steps. The first step is the adaption of the cell weights and the second is the adaption of cell time constants. The examples showed that the influence of the various time constants can not be neglected. BIBLIOGRAPHY /1/ D.O.Hebb, Organisation of Behaviour, New York., John Wiley & Sons, 1948. /2/ T.Kohonen, Self Organisation and Associative Memory, Berlin, Springer Verlag, 1985. /3/ G.Matsumoto, Neurocomputing - Neurones as Microconnputers, Future Generations Computer Systems, Elsevier Sei.Publ. vol.4,(39 - 51), 1988., /4/ B.Kosko, Neural Networks and Fuzzy Systems, Englewood Cliffs, Prentice Hall International Inc., 1992. /5/1.Belič, Seit Controlling Neural Network PROCEEDINGS MIPRO 93-MIS, RISC, Rijeka, 1993. CONCLUSION Authors usually neglect the influence of the various neural cell time constants. The impact on the behaviour Prispelo (Arrived): 28.03.94 mag. Igor Belič, dipl. ing. VŠNZ, Kotnil