Detection of jargon words in a text using semi supervised learning

Sundarapandian et al. (Eds) : ACITY, AIAA, CNSA, DPPR, NeCoM, WeST, DMS, P2PTM, VLSI - 2013
pp. 109–121, 2013. © CS & IT-CSCP 2013 DOI : 10.5121/csit.2013.3412
ANALYSIS OF A CHAOTIC SPIKING
NEURAL MODEL : THE NDS NEURON
Mohammad Alhawarat1
, Waleed Nazih2
and Mohammad Eldesouki3
1,2
Department of Computer Science, College of Computer Engineering and
Sciences, Salman Bin Abdulaziz University, Kingdom of Saudi Arabia
m.alhawarat@sau.edu.sa w.nazeeh@sau.edu.sa
2
Department of Information Systems, College of Computer Engineering and
Sciences, Salman Bin Abdulaziz University, Kingdom of Saudi Arabia
m.eldesouki@sau.edu.sa
ABSTRACT
Further analysis and experimentation is carried out in this paper for a chaotic dynamic model,
viz. the Nonlinear Dynamic State neuron (NDS). The analysis and experimentations are
performed to further understand the underlying dynamics of the model and enhance it as well.
Chaos provides many interesting properties that can be exploited to achieve computational
tasks. Such properties are sensitivity to initial conditions, space filling, control and
synchronization. Chaos might play an important role in information processing tasks in human
brain as suggested by biologists. If artificial neural networks (ANNs) is equipped with chaos
then it will enrich the dynamic behaviours of such networks. The NDS model has some
limitations and can be overcome in different ways. In this paper different approaches are
followed to push the boundaries of the NDS model in order to enhance it. One way is to study
the effects of scaling the parameters of the chaotic equations of the NDS model and study the
resulted dynamics. Another way is to study the method that is used in discretization of the
original R¨ossler that the NDS model is based on. These approaches have revealed some facts
about the NDS attractor and suggest why such a model can be stabilized to large number of
unstable periodic orbits (UPOs) which might correspond to memories in phase space.
1. INTRODUCTION
Chaos might play an important role in information processing tasks in human brain as shown in
[Babloyantz and Lourenc¸o(1996), Destexhe(1994), Freeman(1999), Freeman(2000), Freeman
and Barrie(1994), Rapp(1993), Theiler(1995),Wu et al(2009)Wu, Shyu, Chen, and Guo]. Some
properties that might be useful for information processing tasks are: sensitivity to initial
conditions, space filling, control, synchronization and a rich dynamics that can be accessed using
different control methods. In theory, if Artificial Neural Networks (ANN) are equipped with
chaos they will enable a large number of rich dynamic behaviors. After applying control, these
dynamics can be accessed using one of the control mechanisms such as feedback control [Ott et
al(1990)Ott, Grebogi, and Yorke, Pasemann and Stollenwerk(1998), Pyragas(1992)]. Applying
such control mechanisms to discrete chaotic neural models showed that the model would stabilize
into one of many UPOs that are embedded in the chaotic attractor.

110 Computer Science & Information Technology (CS & IT)
Different chaotic neural models have been devised in recent years to explore the possibilities of
exploiting the rich dynamics that such models might provide for information processing tasks.
One of these model is the NDS model [Crook et al(2005)Crook, Goh, and Hawarat].This model is
based on Rössler system [Rössler(1976)].
Rössler is a simple chaotic system. It has been studied many times in terms of control
investigation and biological studies [Ding et al(2010)Ding, Jiang, and Wang, Bershadskii and
Ikegaya(2011)] to name a few.
The origins of this model dates back to 2003 where the authors in [Crook et al(2003)Crook, olde
Scheper, and Pathirana] have proposed a chaotic neuron that is based on Rössler system. The
idea was to exploit the rich dynamics of the chaotic attractor to represent internal states and
therefore the chaotic attractor can represent an infinite state machine. Many experiments have
been carried out to show that using periodic input signals would cause the chaotic attractor to
stabilize to an UPO. The control mechanism used was a modified version of Pyragas
[Pyragas(1992)] where the period length is considered a system variable. Small networks of 2−3
neurons have been studied and the network has stabilized to one UPO according to a periodic
length that is implicitly appears in the input pattern.
The model is very interesting due to the fact that it theoretically allows an access to a large
number of UPOs, which correspond to memories in phase space, using only single NDS neuron.
In contrast, the Hopfield neural network can give only 0.15n memory size (where n is the number
of neurons in the network).
The NDS model is studied in a series of works [Crook et al(2005)Crook, Goh, and Hawarat, Goh
and Crook(2007), Alhawarat(2007), Crook and Goh(2008), Aoun(2010)]. The authors in[Goh and
Crook(2007)] have used Lorenz attractor instead of Rössler. They have used transient
computation machine to detect human motion from sequences of video frames. In another
paper[Crook and Goh(2008)] the authors argued that chaos may equip mammalian brain with the
equivalent of kernel trick for solving hard nonlinear problems.
In [Aoun(2010)] networks of NDS neurons have been investigated in the context of Spike Time
Dependent Plasticity (STDP) which is a property of cortical neurons. The author has suggested
that NDS neurons may own the realism of biological neural networks; this has been supported by
experiments conducted by the author.
The NDS model has been investigated thoroughly in[Alhawarat(2007)]. In his investigation, the
author has studied the chaotic behavior of the model from both experimental and analytical
perspectives. Explanation of the behavior of the model has become clear after the
experimentations and the mathematical analysis and the study has shown interesting results.
In this paper some of the limitations that exist in the NDS model will be investigated. This
includes tuning the model parameters for the sake of enhancing the model capacity in terms of the
number of successfully stabilized UPOs. Moreover, the discretization method that used to to
convert the continuous Rössler system into the discrete NDS model will be discussed and
compared to other well-known methods of discretization.
The paper is organized as follows: in section 2 the original Rössler model is introduced, in
section 3 the NDS model is described, in section 4 a mathematical analysis of the NDS model and

Computer Science & Information Technology (CS & IT) 111
related discretization methods are discussed, section 5 is devoted to describe the experimentation
setups that are carried out to tune the parameters of the NDS model, section 6 includes
discussions and finally section 7 concludes the paper.
2. RÖSSLER CHAOTIC ATTRACTOR
The Rössler system [Rössler(1976)] is a simple dynamical system that exhibits chaos and has
only one nonlinear term in its equations. Rössler built the system in 1976; it describes chemical
fluctuation and is represented by the following differential equations:
Where a and b are usually fixed and c is varied and is called the control parameter. The familiar
parameter settings for the Rössler attractor are a = 0:2, b = 0:2, and c = 5:7, and the
corresponding attractor is shown in figure 1. Note that the Rössler attractor most of the time lies
in the x−y plane and comprises a disk that has a dense number of orbits. Note also that these
orbits are stretching as a result of divergence and sensitivity to initial conditions. From time to
time the Rössler attractor rises in the z direction and then folds back to the disk which forms a
fin-like shape. The folding and stretching keep the Rössler attractor bounded in phase space.
Fig. 1 The Rössler chaotic attractor with parameters a = 0:2, b = 0:2, and c = 5:7.

If a trajectory of a chaotic system evolved starting from an initial point within the attractor for a
long period of time, then that trajectory will fill a bounded part of the phase space and the
attractor of the system will have a fractional dimension. This bounded space is one of the
properties of chaos, and is due to the attracting and the repelling of the trajectory by the fixed
points that govern and organize the system behaviour. The type of these points determines the
shape of the resulting attractor. This fractal dimension can be recognized in figure 1 where the
attractor is not filling the whole space, instead it is filling part of the space.
Unstable periodic orbits
An UPO is one of the dynamic behaviours that a nonlinear system exhibit in phase space. An
UPO is a repeating orbit and is unstable as a result of being attracted and repelled by fixed points
of an attractor. Chaotic attractors usually have a dense number of UPOs which can be accessed
using controlling methods.
For example figure 2 shows two UPOs of period 1 and period 2 in the Rössler attractor when the
c parameter is tuned to 2:5 and 3:5 respectively. Period 2 here means that the UPO repeat twice in
the attractor.
Fig. 2 Two UPOs, one of period 1 and the other of period 2 in the Rössler chaotic attractor
occurred when parameters a;b are fixed to 0:2 and parameter c is set to 2:5 and 3:5.
3. DESCRIBING THE NDS MODEL
In[Crook et al(2005)Crook, Goh, and Hawarat], Crook et al. have proposed a chaotic spiking
neuron model that is called the NDS neuron. The NDS neuron is a conceptual discretized model
that is based on Rössler’s chaotic system [Rössler(1976)]. The NDS model is a modified
version of Rössler’s equations as described by equations 1- 3 in section 2.
By varying the system parameters such as period length , connection time delays and initial
conditions, large number of distinct orbits with varying periodicity may be stabilised.

The NDS model simulates a novel chaotic spiking neuron and is represented by:
Fig. 3 The chaotic behaviour of a NDS neuron without input (a) the time series of u(t) and
and (b) the phase space of x(t) versus u(t)
where x(t), y(t) and u(t) describe the internal state of the neuron, is the neuron’s binary
output, F(t) represent the feedback signals, In(t) is the external input binary spike train, and the
constants and parameters of the model are: a = v = 0:002, b = c = 0:03, d = 0:8, k = −0:057,
θ= −0:01, = −0:7 and is the period length of the feedback signals.
The NDS model is a discrete version of the Rössler system. The main reason to have a discrete
version of the Rössler system is because spikes should occur in discrete time. The discretization
has been carried out by scaling the system variables x(t), y(t) and u(t) using different scaling
constants: b, c, d. The values of these constants have been tuned experimentally until the
dynamics of the Rössler system are preserved. If the values of these constants are large, then a

system trajectory will miss many dynamic evolutions while moving from one discrete iteration to
the next. Therefore, the time steps of the discrete system need to be very small so that a system
trajectory will involve most of the dynamic evolutions of the R¨ossler system. This will become
clear in section 4.
The dynamics of a single NDS neuron without input is shown in Figure 3. However, when the
NDS neuron is equipped with a time delayed feedback connection then the firing pattern of the
neuron can become periodic. This is shown in figure 4 where period-4 orbit is stabilised due to
the feedback control mechanism F
Fig. 4 The stabilizing of period-4 orbit of a NDS neuron with feedback connection.
4. DISCRETIZATION METHOD
There are two kinds of discretization methods when converting a continuous system into a
discrete one: standard and nonstandard methods. One of the nonstandard methods is Eulers
Forward differentiation method that is usually used in developing simple simulation systems. To
convert a continuous system into the corresponding discrete one using this method, then a time
step TS is used to approximate the next value of a continuous system variable that it will evolve
to. For example, equation1 when converted into a discrete equation using Eulers Forward
differentiation it will become:
if this equation is compared with equation 4, it is obvious that the TS is chosen to be parameter b
= 0.03.
In order for the discretized function to behave similarly to the continuous one then TS should be
chosen to be small. For simulation purposes it is preferable to choose TS according to the
following formula :

Where is the largest absolute eigenvalues for the Rössler system. According to the
mathematical analysis that is carried out in [Alhawarat(2007)], = 5:68698. When
substituting this value in equation 14 it becomes:
TS ≤ 0:0176 (14)
If this compared with b it is obvious that the time step that has been chosen doesn’t follow the
simulation preferable setup.
Moreover, if we look at equation 5 not only the TS is chosen to be c = 0:03 but also a scaling
factor is used to reduce the y down by the factor a = 0:002.
For the third variable in Rössler system, viz. z variable, there were many changes because the
authors of the model according to [Crook et al(2005)Crook, Goh, and Hawarat] wanted to invent
a spiking model that is based on a threshold variable. That variable was u which corresponds to z
in the original Rössler system. In addition, the time step TS value that is used in the
discretization process was different from those appear in equations 4-5. While the TS = b = c =
0:03 used in the aforementioned equations, TS is chosen to be TS =d =0:8 in discretization of the
variable u. Moreover, the authors have scaled the constant c from 5:7 down to 0:057 and changed
its sign to negative. A final change was made to the sign of x variable from positive to negative.
All these changes made the new system to behave differently in phase space. To summarize, the
new attractor of the system has different fixed points types. According to the mathematical
analysis carried out by [Alhawarat(2007)], the original Rössler system fixed points, which are
two spiral saddle points, have become two spiral repellors due to the varying scaling factors used
and the change of the sign for both k and x.
These results assure that the NDS model, although has a promising results as a spiking chaotic
neuron model, it doesn’t have a strong connection to the properties of the original Rössler
attractor. This is made obvious in [Alhawarat(2007)] when they concluded that the existence of
the UPOs of the NDS attractor is due to the acting forces of the two spiral repellors and the reset
mechanism. Because without the reset mechanism, the two spiral repellors will enforce any
trajectory that starts near by to evolve away from both of them and approaches infinity.
If the Eulers Forward differentiation discretization method is used to convert the continuous
Rössler system into a discrete model, where the time step is chosen to be TS = 0:0055, then
equations 1- 3 will become:

Where TS = 0:0055, a = 0:2, b = 0:2, and c = 5:7.
Fig. 5 The Discrete version of R¨ossler attractor based on equations 15- 17
To verify if such settings will preserve the shape and properties of the original R¨ossler system,
then an experiment is carried out to depict the discretized system’s attractor as shown in figure 5.
If these equations are converted into the equations of the NDS model then different steps need to
be followed:
1. Specify the value of TS, here TS is chosen to be 0:0055
2. Change the parameter b in equation 17 to v.
3. Change the parameter c in equation 17 to k.
4. Change the constant TS in equation 15 to b and give it the value of TS.
5. Change the constant TS in equation 16 to c and give it the value of TS.
6. Change the constant TS in equation 17 to d and give it the value of TS.
7. Change the value of a and v and give it the value of 0:2.
8. Change the value of k to 5:7
9. Change the variable z to u.
10. Change the sign of the term (x(t)−k) in equation 17 to become (−x(t)+k)
After applying the previous changes the equations 15- 17 become:
x(t +1) = x(t)+b(−y(t)−u(t)) (18)
y(t +1) = y(t)+c(x(t)+ay(t)) (19)
u(t +1) = u(t)+d(v+u(t)(−x(t)+k)) (20)
Where a = 0:2, b = c = d = 0:0055, and k = 5:7.

Now this new model need to be verified, i.e. an experiment need to be carried out to depict the
attractor such equations will produce. Figure 6 depicts the results of evolution of equations 18-
20.
Fig. 6 The attractor of the model that is based on equations 18- 20
Note that the original R¨ossler attractor has disappeared and this is due to the change of sign that
is made to the term (x(t)−k) in equation 20.
5. TUNING THE PARAMETERS OF THE NDS MODEL
In this research, many experiments have been carried out to tune the parameters of the NDS
model. These experiments are carried out by varying one parameter and fix all other parameters
to their default values that are used in the NDS model definition as stated in section 3. The
following parameters have been considered in these experiments: a,v,b,c,d and k because the
other parameters: have already been studied thoroughly in [Alhawarat(2007)]. One
NDS neuron has been used in the experiments setup and random initial conditions are chosen for
the values of the variables: x, y and u. After 1000 iterations the feedback control is applied and
the experiment runs for another 9000 iterations.
To decide whether a specific setting is valid; the values of the variables of the model are recorded
and then depicted in phase space. If an attractor exist, then this setting is considered in the valid
ranges for the values of the system parameters.
Table 1 Parameter value’s ranges
To summarize the results; the valid ranges for the model parameters are shown in table 1.

Table 2 Parameter settings with different selected values from the ranges appear in table1
To judge weather such ranges could enhance the capacity of the attractor in terms of the number
of UPOs that it might encompass, then another experiment setup is used. Here the average
number of successfully stabilized UPOs is computed over 1000 run for different parameter
settings according to table 2 and the results are depicted in figure 6.
Note that the original parameters values of the NDS model is used in setup 7. This is done to
compare the capacity of the attractor with different parameter settings compared to the original
parameter settings.
The results that are shown in figure 6 suggest that there exist better parameter settings such as
Settings 14 and Settings 05, but these will not make a significant enhancement in the capacity of
the attractor in terms of successfully stabilized UPOs when compared to the NDS original settings
(Setting 07).
Based on the results that have been attained in section 4, if a parameter setting need to be chosen
to represent the Rössler system then first a TS need to be set according to equation 14, then a
mapping between the original Rössler equations and the NDS equations need to be performed. If
TS is chosen to be 0:015, then after carrying out the mapping between the equations of both
systems; the parameter settings become as listed in table 3
Table 3 Parameter settings for the original Rössler as discussed in section 2
This setting is considered in another experiment setup, and the result is that there is no single
UPO that can be stabilized. One possible reason is that the fixed points of the system is reserved
in this case and not changed as the case in the NDS model settings. The change in sign in the term

(x−c) and the different nested scaling factors that are applied to the original Rössler equation
have led to change in the types and properties of the fixed points as shown in [Alhawarat(2007)].
This can be proved easily because when multiplying an equation with a constant and then trying
to find its roots, then it should be set to zero. Therefore, the constant will have no effect on the
resulted roots as dividing both sides by constant will eliminate the constant from the left hand
side and will not affect zero in the right hand side.
6. DISCUSSION
The results of both experimentations and mathematical analysis of the discretization method
suggest that the NDS model has weak connections to the original Rössler system. This is due to
many factors. Firstly, the discretization process that is used in devising the NDS equations does
not follow any known discretization method where different discretization time steps and scaling
factors are used. Secondly, Changing the sign of the term (x(t)−k) that appears in equation 20
made the system attractor to disappear and approach infinity instead. These changes affected both
the shape and properties of the original Rössler attractor.
It is important here to stress that even before adding the feedback signal, the input signals and the
reset mechanism to the system equations, the attractor has become completely different from the
original Rössler attractor. This assure that the NDS model has weak connections to the Rössler
model and has different fixed points, eigenvalues and eigenvectors as demonstrated in
[Alhawarat(2007)].
The results also suggest that both the reset mechanism and the feedback signal are the major
ingredients for the NDS model to work and to be stabilised to one of its available UPOs.
The results of experimental setups made to tune the NDS parameters suggest that there exist
better parameter settings but will not enhance the capacity of the attractor in terms of successfully
stabilized UPOs significantly when compared to the NDS original settings.
Also, the settings of the Rössler model when used in the NDS model resulted in no stabilized
UPOs because the discretization method that is used to build the NDS model has led to changes
in the shape and properties of the Rössler attractor.
It is important to mention that the main factors that affect the shape and properties of the original
Rössler attractor are both scaling the parameters of the model with different values and the
change in sign that is made to the term (x(t)−k) that appears in equation 20.
7. CONCLUSION
One chaotic model, viz., the NDS model has been studied in this paper. NDS is one of different
chaotic models that are devised in recent years to explore the possibilities of exploiting the rich
dynamics that such models might provide for carrying out information processing tasks.
The NDS model might be stabilized to a large number of UPOs. These UPOs can be stabilised
using a feedback control mechanism. The NDS model is a modified version of Rössler chaotic

system. The rich dynamics of the Rössler system is supposed to be inherited by the NDS model.
This is suggested by the large number of UPOs that can be stabilised as shown in figure 3.
However, when the discretization methods are discussed in this paper, it is shown that the method
that is used to discretize the original Rössler equations in devising the NDS equations is not a
known discretization method. This along with the change in sign in the term u(t)x(t)−ku(t)) has
affected the shape and properties of the NDS attractor when compared with its origin: the
Rössler attractor.
Different experimental setups have been prepared and performed to tune the NDS model
parameters. The results of these experimentations have revealed the valid ranges of the
parameters of the model. Also, other experimentations have shown different capacities for the
NDS attractor in terms of the number of stabilized UPOs with different parameter settings.
The results attained in this paper suggest that there are weak relationships between the NDS and
the Rössler models. However, the NDS attractor encompasses large number of UPOs as shown
in figure 7. These and the wide range of dynamic behaviours may be exploited to carry out
information processing tasks.
ACKNOWLEDGEMENTS
This work was supported by Salman Bin Abdulaziz University in KSA under grant 40/H/1432.
I’m grateful to this support and would like to thank the deanship of scientific research at the
university, represented by Dr. Abdullah Alqahtani at that time.
REFERENCES
[1] [Alhawarat(2007)] Alhawarat MOI (2007) Learning and memory in chaotic spiking neural
models.PhD thesis, Oxford Brookes University
[2] [Aoun(2010)] Aoun M (2010) Stdp within nds neurons. In: Advances in Neural Networks - ISNN
2010, Lecture Notes in Computer Science, vol 6063, Springer Berlin Heidelberg, pp 33–43
[3] [Babloyantz and Lourenço(1996)] Babloyantz A, Lourenço C (1996) Brain chaos and
computation.International Journal of Neural Systems 7:461–471
[4] [Bershadskii and Ikegaya(2011)] Bershadskii A, Ikegaya Y (2011) Chaotic neuron clock.Chaos,
Solitons & Fractals 44(45):342 – 347, DOI 10.1016/j.chaos.2011.03.001, URL
http://www.sciencedirect.com/science/article/pii/S096007791100035X
[5] [Crook and Goh(2008)] Crook N, Goh WJ (2008) Nonlinear transient computation as a potential
”kernel trick” in cortical processing. Biosystems 94(1-2):55–59
[6] [Crook et al(2003)Crook, olde Scheper, and Pathirana] Crook N, olde Scheper T, Pathirana V (2003)
Self organised dynamic recognition states for chaotic neural networks. Information Sciences 150(1-
2):59–75
[7] [Crook et al(2005)Crook, Goh, and Hawarat] Crook N, Goh W, Hawarat M (2005) The nonlinear
dynamic state neuron. In: Verleysen M (ed) ESANN, 13th European Symposium on Artificial Neural
Networks (ESANN’2005), Bruges, Belgium, pp 37–42
[8] [Destexhe(1994)] Destexhe A (1994) Oscillations, complex spatiotemporal behavior, and information
transport in networks of excitatory and inhibitory neurons. Physical Review E 50(2):1594–1606
[9] [Ding et al(2010)Ding, Jiang, and Wang] Ding Y, JiangW,Wang H (2010) Delayed feedback control
and bifurcation analysis of rossler chaotic system. Nonlinear Dynamics 61:707–715, DOI
10.1007/s11071-010-9681-y, URL http://dx.doi.org/10.1007/s11071-010-9681-y
[10] [Freeman(1999)] FreemanW(1999) How Brains Make up their Minds.Weidenfeld and Nicolson,
London

[11] [Freeman(2000)] Freeman W (2000) A proposed name for aperiodic brain activity: stochastic chaos.
Neural Networks 13(1):11–13
[12] [Freeman and Barrie(1994)] Freeman W, Barrie J (1994) Chaotic oscillations and the genesis of
meaning in cerebral cortex. Temporal Coding in the Brain pp 13–37
[13] [Goh and Crook(2007)] Goh WJ, Crook N (2007) Pattern recognition using chaotic transients. In: In
Submitted to 15th European Symposium on Artificial Neural Networks (ESANN2007
[14] [Ott et al(1990)Ott, Grebogi, and Yorke] Ott E, Grebogi C, Yorke JA (1990) Controlling chaos.
Physical Review Letters 64:1196–1199
[15] [Pasemann and Stollenwerk(1998)] Pasemann F, Stollenwerk N (1998) Attractor switching by neural
control of chaotic neurodynamics. Network: Computational Neural Systems 9(4):549–561
[16] .[Pyragas(1992)] Pyragas K (1992) Continuous control of chaos, by self-controlling feedback. Physics
Letters A 170:421–428
[17] [Rapp(1993)] Rapp PE (1993) Chaos in the neurosciences: Cautionary tales from the frontier.
Biologist 40(2):89–94
[18] [R¨ossler(1976)] R¨ossler OE (1976) An equation for continuous chaos. Physics Letters A 57(5):397–
398
[19] [Theiler(1995)] Theiler J (1995) On the evidence for low-dimensional chaos in an epileptic
electroencephalogram. Physics Letters A 196:335–341
[20] [Wu et al(2009)Wu, Shyu, Chen, and Guo] Wu YT, Shyu KK, Chen TR, Guo WY (2009) Usingthree-
dimensional fractal dimension to analyze the complexity of fetal cortical surface from magnetic
resonance images. Nonlinear Dynamics 58:745–752, DOI 10.1007/s11071-009-9515-y, URL
http://dx.doi.org/10.1007/s11071-009-9515-y

Detection of jargon words in a text using semi supervised learning

More Related Content

What's hot (9)

Viewers also liked (7)

Similar to Detection of jargon words in a text using semi supervised learning (20)

Recently uploaded (20)

Detection of jargon words in a text using semi supervised learning